Ethical Hacker Tip of the Day: User-Agent Manipulation

Listen to this Post

The more I learn doing bug bounties, the more I notice that when I fuzz the User-Agent: field on many (mostly big sites, Google, Tesla, Booking, Priceline etc) the different the response (or varied response) I get.

This holds true with Google most of all. If you view source and look at nearly any of their JS, you will see sections that specifically mention “Opera”, “Edge”, “Safari”, “Mozilla” and although its seriously obfuscated, that is enough for me to know, try those different values and see what happens.

When I do this on google sites I always get a slightly different amount of bytes back, and at times I get a completely different page.

But my tip of the day is for you Burp users, hardcore your User-Agent to the following:

[ Remove the space after the . in code. and google. ]

User-Agent:

Python-httplib2/0.22.0 (gzip) AppEngine-Google; (+http://code.google.com/appengine; appid: s~oauth2playground)

Unfortunately due to LinkedIn’s annoying link shortener, you must make sure you remove those 2 spaces

You can force this in burp by going to Settings -> Proxy -> Scroll Down to HTTP Match and Replace Rules. Edit one of the User-Agent Rules:

Modify the existing one ( I believe it is set to IE ) to the statement above. Click Test, and verify it worked.

Now all your requests will have that User-Agent, and the reason I suggest this is, Ive been getting some interesting responses while pentesting (in scope of course) sites on Google 😉

You Should Know:

1. Burp Suite User-Agent Manipulation:

  • Open Burp Suite.
  • Navigate to `Settings` -> `Proxy` -> HTTP Match and Replace Rules.
  • Edit the existing User-Agent rule or create a new one.
  • Set the User-Agent to:
    Python-httplib2/0.22.0 (gzip) AppEngine-Google; (+http://code.google.com/appengine; appid: s~oauth2playground)
    
  • Click `Test` to verify the rule works.
  • Apply the changes and ensure all requests now use the new User-Agent.

2. Using cURL for User-Agent Manipulation:

  • You can also manipulate the User-Agent using cURL:
    curl -A "Python-httplib2/0.22.0 (gzip) AppEngine-Google; (+http://code.google.com/appengine; appid: s~oauth2playground)" https://example.com
    

3. Python Requests Library: