Listen to this Post
Introduction:
In the perpetual cat-and-mouse game of cybersecurity, threat actors consistently hide malicious infrastructure behind shared Autonomous System Numbers (ASNs) and subnets, often leveraging legitimate cloud and hosting providers. For defenders, discovering these hidden endpoints is critical for proactive threat hunting and intelligence gathering. UrlScan.io, a free service for scanning and analysing websites, becomes a powerful weapon when wielded with precise search operators, known as “dorks,” to expose these concealed threats.
Learning Objectives:
- Understand how to construct advanced UrlScan.io search queries to filter results by ASN, IP subnet, and other key technical parameters.
- Learn to extract a comprehensive list of domains and IPs associated with a specific network block used by a threat actor.
- Develop the skills to automate and integrate these reconnaissance techniques into a continuous monitoring workflow.
You Should Know:
1. The Foundation: Querying by IP and ASN
UrlScan’s search syntax allows for granular filtering. The most fundamental operators for network-based discovery are `ip:` and asn:.
Verified Command/Query:
asn:AS14061 ip:192.0.2.0/24 task.ip:192.0.2.100 page.domain:example.com
Step-by-step guide:
asn:AS14061: This query filters scans to show only those where the server is located within the autonomous system numbered AS14061 (which belongs to DigitalOcean). Replace AS14061 with the target ASN.
ip:192.0.2.0/24: This narrows results to a specific IP range using CIDR notation (e.g., 192.0.2.0 to 192.2.2.255). This is useful for investigating a specific subnet within a larger ASN.
task.ip:192.0.2.100: This finds scans initiated against a specific IP address, revealing what domains might be resolving to that IP.
How to Use: Go to UrlScan.io, enter these queries into the search bar. Combine them with boolean operators like AND, OR, and `NOT` (e.g., asn:AS14061 AND page.domain:example.).
- Expanding the Net: Using Wildcards and Domain Filters
Threat actors often register domains with similar patterns. Using wildcards and domain filters helps cast a wider net to catch these related assets.
Verified Command/Query:
page.domain:.suspicious-domain.com page.domain:suspicious-domain.com AND page.domain:.net
Step-by-step guide:
page.domain:.suspicious-domain.com: The asterisk (“) acts as a wildcard. This query will find all scans for any subdomain of suspicious-domain.com, such as `api.suspicious-domain.com` or c2.suspicious-domain.com.
Combining Filters: The query `page.domain:suspicious-domain.com AND page.domain:.net` uses the `AND` operator to find results related to the primary domain and any subdomains of `.net` domains associated with it in the scan data. This can uncover related infrastructure across different TLDs.
How to Use: Use wildcards strategically when you have a root domain name but want to discover all associated subdomains that have been scanned by UrlScan.
3. Temporal Analysis: Finding Recently Active Infrastructure
Newly deployed malicious infrastructure is a high-priority indicator. Filtering by date is crucial for identifying recent activity.
Verified Command/Query:
date:>=2024-09-15 asn:AS12345
Step-by-step guide:
date:>=2024-09-15: This operator filters scans to those performed on or after September 15, 2024. You can also use `date:2024-09-15` for a specific day or `date:<=2024-09-15` for scans on or before a date.
Combining with Network Filters: By combining `date:>=2024-09-15` with asn:AS12345, you can quickly identify any new endpoints that have appeared on a threat actor’s preferred hosting network in the last few days. This is invaluable for rapid response.
How to Use: Always pair time-based queries with other technical filters to reduce noise and focus on the most relevant, recent data.
4. Digging Deeper: Analyzing Server Headers and Technologies
The technologies and servers announced by a web endpoint can fingerprint it and link it to other campaigns.
Verified Command/Query:
page.server:nginx/1.18.0 asn:AS14061 page.software:OpenResty meta:php AND asn:AS40676
Step-by-step guide:
page.server:nginx/1.18.0: This searches for a specific web server and version. A threat actor might standardize their infrastructure on a particular, slightly outdated version of nginx or Apache.
page.software:OpenResty: This filters for a specific software component detected on the page. OpenResty (a fork of Nginx) is a common finding.
meta:php: This searches within the page’s metadata for mentions of PHP, which can indicate the backend technology.
How to Use: If you identify a unique technology stack from a known malicious endpoint, use these filters across a large ASN to find other servers with the same configuration.
- Extracting Data for Automation: Using the UrlScan API
For large-scale hunting, the command line and API are more efficient than the web interface. The UrlScan API allows you to programmatically execute these searches.
Verified Command/Query (using curl in Linux/Windows WSL):
Search API (returns JSON list of results)
curl -s "https://urlscan.io/api/v1/search/?q=asn:AS14061%20date:>=2024-09-10&size=100" | jq '.results[] | .task.url'
Get detailed results for a specific scan
curl -s "https://urlscan.io/api/v1/result/{uuid-from-search}" | jq
Step-by-step guide:
Search Query: The first `curl` command performs a search for `asn:AS14061` and date:>=2024-09-10. The `size=100` parameter requests up to 100 results. The `jq` utility is used to parse the JSON and extract only the URLs.
Result Retrieval: The second command fetches the full, detailed result for a specific scan by using its UUID (which you get from the search results).
How to Use: This can be integrated into a Python or Bash script to automatically query UrlScan daily for specific ASNs or IP ranges, exporting the list of discovered domains for further analysis in a SIEM or threat intelligence platform.
- Hunting for Specific Threats: File Hashes and SSL Certs
If you have an indicator from a malware sample, you can hunt for related infrastructure using file hashes or SSL certificate information.
Verified Command/Query:
hash:sha256:a1b2c3... file.name:payload.dll cert.issuer:"Let's Encrypt" AND asn:AS23456
Step-by-step guide:
hash:sha256:a1b2c3...: If a malicious JavaScript or PDF file has been uploaded to UrlScan, you can search for its SHA256 hash to find other scans where the same file appeared.
file.name:payload.dll: Search for scans that involved a file with a specific name.
cert.issuer:"Let's Encrypt": Many threat actors use free “Let’s Encrypt” certificates. While common, searching for this issuer within a specific ASN can still reveal patterns.
How to Use: Apply these filters when you have strong initial indicators (IOCs) to find connected infrastructure that shares the same payloads or SSL certificate properties.
7. Combining All Elements: A Composite Hunt
A professional threat hunt combines multiple techniques to create a highly precise query.
Verified Command/Query:
asn:AS14061 AND date:>=2024-09-20 AND page.domain:.tk AND page.server:cloudflare
Step-by-step guide:
Composite Query: This example hunts for endpoints within DigitalOcean’s ASN (AS14061) that were active after September 20th, use a `.tk` domain (a free TLD popular with attackers), and are behind Cloudflare (a common tactic to hide origin IPs).
Iterative Refinement: Start broad (e.g., asn:AS14061) and then gradually add filters (date, page.domain, page.server) based on the results you see. This iterative process helps you build a profile of the threat actor’s infrastructure choices.
How to Use: Construct these multi-part queries in the UrlScan search bar or via the API. The goal is to maximize signal-to-noise ratio, quickly isolating potentially malicious endpoints from the vast number of benign sites on the same network.
What Undercode Say:
- The democratization of advanced reconnaissance through tools like UrlScan.io fundamentally shifts the advantage towards vigilant defenders, enabling proactive network discovery at scale.
- Mastery of search syntax is not a niche skill but a core competency for modern threat intelligence and security operations center (SOC) analysts.
The techniques outlined represent a systematic approach to cutting through the noise of the modern internet. By moving beyond simple domain lookups and leveraging the rich metadata collected by UrlScan, security teams can pivot from reactive blocking to proactive hunting. This methodology allows for the discovery of attacker infrastructure in its early stages, often before it is weaponized in widespread campaigns. The key is understanding that attackers are lazy and consistent; they reuse ASNs, IP blocks, domain patterns, and software stacks. By codifying these patterns into repeatable searches, defenders can build an early-warning system that significantly reduces the attacker’s dwell time and operational window.
Prediction:
The increasing sophistication of these free OSINT techniques will force threat actors to adopt more expensive and complex obfuscation methods, such as leveraging bulletproof hosting with stricter abuse policies, using residential proxies to blend in with legitimate user traffic, and frequently rotating infrastructure. This will raise the operational cost for attackers, potentially limiting large-scale campaigns to better-funded groups. In response, the next evolution for defender tools will be the integration of AI to automatically identify subtle correlations across ASNs, certificates, and code similarities that are invisible to manual querying, leading to an AI-driven arms race in cyber reconnaissance.
🎯Let’s Practice For Free:
IT/Security Reporter URL:
Reported By: Abhirup Konwar – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
