Listen to this Post
Introduction:
Open‑Source Intelligence (OSINT) transforms scattered, publicly available information into actionable security insights. Unlike passive browsing, OSINT involves systematic collection, correlation, and analysis of data from domains, emails, social media, images, and breach dumps. This article dives into practical OSINT workflows, tool configurations, and command‑line techniques that blue teams, threat hunters, and forensic investigators use daily to uncover hidden relationships and exposure paths.
Learning Objectives:
- Execute DNS, WHOIS, and subdomain enumeration using native Linux/Windows commands and specialised tools like Amass and theHarvester.
- Perform email, username, and metadata forensics to map digital footprints and credential leaks.
- Leverage Shodan, SpiderFoot, and API‑based threat intelligence for continuous exposure monitoring and risk assessment.
You Should Know:
- Mastering Domain & DNS Investigation with CLI Tools
Start with passive DNS reconnaissance to map a target’s infrastructure without direct interaction. Use `dig` (Linux/macOS) or `nslookup` (Windows) to retrieve A, MX, TXT, and NS records. For subdomain brute‑forcing, combine `dnsrecon` or amass.
Step‑by‑step guide – Linux:
Basic DNS enumeration
dig example.com ANY +noall +answer
dig -t MX example.com
Subdomain enumeration using Amass (passive mode)
amass enum -passive -d example.com -o subdomains.txt
Reverse DNS lookup for an IP range
for ip in {1..254}; do dig -x 192.168.1.$ip +short; done
Windows (PowerShell):
Resolve-DnsName example.com -Type A
Resolve-DnsName -Type MX example.com
Subdomain bruteforce with a wordlist
Get-Content subdomains.txt | ForEach-Object { Resolve-DnsName "$_.example.com" -ErrorAction SilentlyContinue }
WHOIS intelligence reveals registrant details, name servers, and creation/expiry dates. Use `whois example.com` on Linux or `Get-Whois` (third‑party module) on Windows. Correlate this data with historical DNS records via SecurityTrails or dnsrecon -d example.com -t brt.
- Email & Username Research – TheHarvester and Manual Validation
Email addresses and usernames act as digital breadcrumbs. Use `theHarvester` to scrape search engines, PGP key servers, and LinkedIn. Validate discovered emails using SMTP checks or breach APIs.
Step‑by‑step guide:
Install theHarvester (Kali/Ubuntu)
sudo apt install theharvester -y
Gather emails from Google, Bing, and LinkedIn
theHarvester -d example.com -b google,bing,linkedin -l 500 -f results.html
Extract unique usernames from emails
cat results.html | grep -oP '\b[\w.%+-]+@[A-Za-z0-9.-]+.[A-Z|a-z]{2,}\b' | cut -d@ -f1 | sort -u > usernames.txt
For username correlation across platforms, use `sherlock` (Python) or whatsmyname.app. Example with Sherlock:
git clone https://github.com/sherlock-project/sherlock.git cd sherlock python3 sherlock.py --username johndoe --output results.txt
To check breach exposure, query the Have I Been Pwned API (rate‑limited, requires API key):
curl -H "hibp-api-key: YOUR_KEY" https://haveibeenpwned.com/api/v3/breachedaccount/[email protected]
3. Social Media & Image Geolocation Intelligence
Extract geolocation from photos and social media profiles. Use `exiftool` for metadata, and Google Maps/Street View for manual verification.
Step‑by‑step guide – Metadata extraction:
Install exiftool sudo apt install libimage-exiftool-perl -y Extract all metadata from an image exiftool -a -u -g1 suspicious.jpg Look for GPS coordinates, camera model, timestamps exiftool -GPSPosition -CreateDate -Make -Model image.jpg
Convert GPS coordinates to a location using `gps2geo` or manual input. For social media, use `tinfoleak` (Twitter) or `Twint` (deprecated, now use snscrape):
pip install snscrape snscrape twitter-user username --jsonl > tweets.json Extract location fields cat tweets.json | jq '.user.location, .geo, .coordinates'
Reverse image search via `google_images_download` or `tineye` API. For automation, use `python3` with requests:
import requests
files = {'image': open('face.jpg', 'rb')}
response = requests.post('https://api.tineye.com/rest/search/', files=files, auth=('api_key', 'secret'))
print(response.json())
4. Metadata Examination in Documents & PDFs
Office documents and PDFs often leak internal paths, author names, and software versions. Use exiftool, pdfid, and `mat2` for extraction and cleaning.
Step‑by‑step guide:
PDF metadata pdfinfo document.pdf Extract hidden text from PDFs pdftotext document.pdf - | head -20 Office documents (docx, xlsx) – unzip and read XML unzip document.docx -d docx_extracted cat docx_extracted/docProps/core.xml | grep -E "<dc:creator|<cp:lastModifiedBy"
For batch analysis on Windows, use PowerShell:
Get Office document properties
$shell = New-Object -ComObject Shell.Application
$folder = $shell.NameSpace('C:\docs')
$file = $folder.Items().Item('report.docx')
$file.ExtendedProperty('{F29F85E0-4FF9-1068-AB91-08002B27B3D9} 4') Author
Remove metadata before sharing: Linux exiftool -all= document.pdf, Windows built‑in “Remove Properties and Personal Information”.
5. Threat Intelligence Collection via Shodan & SpiderFoot
Shodan indexes internet‑connected devices. Use its CLI to discover exposed databases, webcams, and industrial control systems. SpiderFoot automates OSINT correlation across 100+ data sources.
Step‑by‑step guide – Shodan:
Install Shodan CLI pip install shodan shodan init YOUR_API_KEY Search for exposed RDP (port 3389) in a country shodan search --limit 100 'port:3389 country:US' --fields ip_str,port,org Download full scan results shodan download rdp_scan --limit 1000 'port:3389' Convert to CSV shodan parse --fields ip_str,port,org --separator , rdp_scan.json.gz > rdp.csv
SpiderFoot automation:
Docker install docker pull spiderfoot/spiderfoot docker run -p 5001:5001 spiderfoot/spiderfoot Use the web UI or CLI python3 sf.py -s example.com -m sfp_dnsresolve,sfp_shodan,sfp_whois -o results.csv
For API security, query Shodan for open Elasticsearch, MongoDB, or exposed API keys:
shodan search 'product:"MongoDB" "authenticated:false"' --fields ip_str,port shodan search 'ssl:"Let\'s Encrypt" http.title:"login"'
- Breach & Exposure Monitoring with Recon-1g and Leak‑Parse
Recon‑ng is a full‑featured OSINT framework with modules for breach data, pastebin dumps, and credentials. Use it to correlate emails with plaintext passwords from known breaches (legally, only on your own assets).
Step‑by‑step guide:
Launch Recon-1g recon-1g Install marketplace modules marketplace install recon/domains-hosts/brute_hosts marketplace install recon/credentials-credentials/pwnedlist Load a module and set source workspace create example_audit use recon/domains-hosts/brute_hosts set source example.com run For breach data (requires API key) use recon/credentials-credentials/pwnedlist set source [email protected] run
Parse breach dumps (e.g., from Have I Been Pwned) using `leakdb` or h8mail. Example with h8mail:
pip install h8mail h8mail -t [email protected] -j haveibeenpwned -k YOUR_HIBP_KEY --loose
Always anonymise traffic using Tor or a VPN when conducting breach monitoring to avoid attribution.
7. Cloud Hardening & OSINT for Misconfigured Storage
Publicly exposed cloud storage (AWS S3, Azure Blob, GCP buckets) is a goldmine for OSINT. Enumerate bucket names via permutations of a domain, then test for public listing.
Step‑by‑step guide – S3 bucket enumeration:
Install AWS CLI and configure dummy credentials aws configure use fake keys List bucket contents (if public) aws s3 ls s3://example-backup-bucket/ --1o-sign-request Download all files recursively aws s3 sync s3://open-bucket/ ./downloaded/ --1o-sign-request
Use `bucket_finder` or `slurp` for large‑scale enumeration:
ruby bucket_finder.rb --wordlist permutations.txt --download
For Azure Blob misconfigurations:
Check anonymous access az storage container list --account-1ame accountname --auth-mode login If public, list blobs az storage blob list --container-1ame public-container --account-1ame accountname --1um-results 100
Mitigation: Enforce bucket policies that deny `GetObject` for anonymous principals, enable logging, and use tools like `ScoutSuite` to audit cloud configurations.
What Undercode Say:
- Key Takeaway 1: OSINT is not about hacking or finding secrets; it’s about connecting disjointed public data points to reveal relationships, patterns, and exposures that are otherwise invisible. The most dangerous “leak” is often the combination of seemingly harmless information from different sources.
- Key Takeaway 2: Effective OSINT requires structured workflows: start with passive DNS and WHOIS, then branch into email/username correlation, metadata forensics, and finally active (but legal) enumeration via Shodan or breach APIs. Automation with tools like SpiderFoot and Recon‑ng drastically reduces manual effort.
Analysis: The post by Tolga YILDIZ correctly emphasises that raw data collection is worthless without analysis and pattern recognition. Many junior analysts fall into the “data hoarding” trap – collecting thousands of subdomains and emails but failing to contextualise them. Real intelligence emerges when you map an email alias to a leaked password hash, and that hash appears in a pastebin dump alongside a server hostname. Modern OSINT also intersects with cloud security: a misconfigured S3 bucket containing employee timesheets can reveal internal network naming conventions, which then help brute‑force VPN logins. From a defensive perspective, blue teams must constantly scan their own external exposure using these exact techniques. Proactive OSINT of your organisation’s digital footprint – forgotten subdomains, expired certificates, developer emails on GitHub – reduces attack surface before adversaries find them. Ethical boundaries are non‑negotiable: never pivot into private data or attempt login without explicit permission. Finally, OSINT is becoming a mandatory skill for SOC analysts, threat hunters, and even incident responders, as initial footholds often originate from publicly accessible credentials or misconfigurations.
Prediction:
+1 OSINT automation powered by large language models will enable real‑time correlation of unstructured data (images, social media narratives, dark web chatter) reducing investigation times from days to minutes.
-1 Attackers will increasingly use the same OSINT techniques against defenders, mapping security researchers’ identities and infrastructure, leading to targeted doxxing and spear‑phishing campaigns against blue teams.
+1 Cloud service providers will embed OSINT‑style monitoring directly into their native security hubs (e.g., AWS Security Hub, Microsoft Defender for Cloud) as a default feature for external attack surface management.
-1 Regulatory bodies will impose stricter limits on bulk OSINT collection under privacy laws like GDPR and CPRA, forcing analysts to adopt anonymisation and purpose‑limitation frameworks.
▶️ Related Video (86% Match):
🎯Let’s Practice For Free:
🎓 Live Courses & Certifications:
Join Undercode Academy for Verified Certifications
🚀 Request a Custom Project:
Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands
IT/Security Reporter URL:
Reported By: Iamtolgayildiz Osint – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
