After 2 PRs merged. I started writing the Analyser.py but was not able to get the idea of writing the test cases and also visualize my approached codes. Adding to my concerns, the ssh webserver I had lent off started having high IOPS and hence breaking in. Adding to it my tor too kept stucking at Starting with guard context "default". Thanks to my mentor who advised me to complete the analyser.py and then think about adding it into the Project.

image. I tried contacting the members but seemed my internet wasn’t allowing Tor which proved not to be True as I checked with the Emma and OONI probe and saw my connections were fine if not good, as my IPV6 wasn’t working. Next I tried tinkering and thought of purging and installing tor again. To my wonder it did work!

Back from the ssh (Not totally) but for the time-being as my home network is quite slow. Sadly I keep ranting about all of these :( because I’m a bit frustrated about the technical errors.

Anyway let’s deep dive what I did this week:

Progress:

I updated my flow of work to the below flowchart and still have few points to add.

  • Use of dashed lines and boxes show the things that haven’t been implemented as of now.
  • As of now K has been set to 150 (Experimental Analysis)
  • The Captcha Checking Module has been proposed recently, which enables the use of “captcha” in the requests path from the responses we get while we load a website.

For more details one you could look into the Experimental code and it’s output to gain an even more insight.

Insights:

At present I’m facing with the reliability of the modules, like for example:

Cloudflare blocks requests library and hence request library isn't suited in here. I read it's because the headers(User-Agent) of the request is sent by the name of python which get's marked as a bot. I changed the User-Agent but it was still the same so it isn't much of use in this case. Also for specific cases like mastercard where there is status 3xx (reload) it returns results easily.

image

So I plan on also making a checker method that would check the following:

def check():
# Non Tor:
  if request_module is blocked:
    # Just to be cautious
    check HAR
    if HAR.first() returns 4xx or 5xx:
      go with request
    elif HAR.first() returns 0:
      "No case found till now"
    else:
      go with HAR.first()
# Tor:
  if request_module is blocked:
    check HAR 
    if HAR.first() returns 3xx or 4xx or 5xx:
      go with request
    if HAR.first() returns 0:
      "check for captcha and warnings"
      pass
    else:
      go with HAR.first()

I hope this would tend to make the code a bit better in terms of reliabilty. Discussions needed in here because it’s my thought as of now. Also HAR.first() mean the first request status code sent to server. Generally the index page.