Unlock Hidden API Endpoints with This Powerful cURL + grep Trick

Introduction

API endpoints are often the gateway to critical vulnerabilities in web applications. Discovering hidden or undocumented API paths can significantly expand your attack surface during penetration testing or bug bounty hunting. In this guide, we’ll break down a powerful cURL + grep command that automates API endpoint discovery, helping security researchers uncover hidden gems efficiently.

Learning Objectives

• Learn how to extract API endpoints from web applications using cURL and grep.
• Understand the regex pattern used to filter and sort discovered endpoints.
• Apply this technique in real-world bug bounty hunting and penetration testing scenarios.

1. Extracting API Endpoints with cURL and grep

Command:

curl -s <URL> | grep -Po "(\/)((?:[a-zA-Z-_:.0-9{}]+)) (\/)((?:[a-zA-Z-_:.0-9{}]+))(\/)((?:[a-zA-Z-_\/:.0-9\ {}]+))" | sort -u

Step-by-Step Explanation:

1. `curl -s ` – Silently fetches the webpage content from the target URL.
2. `grep -Po` – Uses Perl-compatible regex to extract matching patterns.

3. Regex Breakdown:

– `(\/)` – Matches a forward slash.
– `((?:[a-zA-Z\-_\:\.0-9\{\}]+))` – Captures alphanumeric strings, hyphens, underscores, colons, dots, and curly braces.
– `(\/)` – Matches optional additional slashes.
– The full pattern captures common API path structures (e.g., /api/v1/users).
4. `sort -u` – Removes duplicates and sorts the results.

Example Usage:

curl -s https://example.com/api | grep -Po "(\/)((?:[a-zA-Z-_:.0-9{}]+)) (\/)((?:[a-zA-Z-_:.0-9{}]+))(\/)((?:[a-zA-Z-_\/:.0-9\ {}]+))" | sort -u

Output:

[/bash]

/api/v1/users

/api/v1/admin

/api/v2/config

<ol>
<li>Automating Endpoint Discovery with Bash Scripting </li>
</ol>

Command: 
[bash]
!/bin/bash 
url="$1" 
curl -s "$url" | grep -Po 'https?://[^"]+' | grep -E '\/api\/|\/v[0-9]+\/' | sort -u > endpoints.txt 

Step-by-Step Explanation:

1. `curl -s “$url”` – Fetches the target page silently.
2. `grep -Po ‘https?://[^”]+’` – Extracts all HTTP/HTTPS links.
3. `grep -E ‘\/api\/|\/v[0-9]+\/’` – Filters for API-related paths.
4. `sort -u > endpoints.txt` – Saves unique endpoints to a file.

Example Usage:

./discover_endpoints.sh https://example.com 

3. Enhancing Recon with Parallel cURL Requests

Command:

cat urls.txt | xargs -P 10 -I {} curl -s {} | grep -Eo '\/[a-z0-9_-]+\/[a-z0-9_-.]+' | sort -u 

Step-by-Step Explanation:

1. `xargs -P 10` – Runs 10 parallel cURL requests for faster scanning.
2. `grep -Eo` – Extracts paths matching the pattern /

/[bash]</code>.

<h2 style="color: yellow;">3. `sort -u` – Removes duplicates.</h2>

<h2 style="color: yellow;"> Example Usage:</h2>

[bash]
cat targets.txt | xargs -P 10 -I {} curl -s {} | grep -Eo '\/api\/[^"\ ]+' | sort -u 

for endpoint in $(cat endpoints.txt); do curl -s -o /dev/null -w "%{http_code} %{url_effective}\n" "https://example.com$endpoint"; done 

ffuf -u "https://example.com/api/FUZZ" -w /path/to/wordlist.txt -mc 200 -o results.json