Building Leak Radar: Real-Time API Key Exposure Detection at Scale + Video

By /

Listen to this Post

Featured Image

Introduction:

The software development lifecycle has an uncomfortable secret: developers leak credentials at an astonishing rate. In 2024 alone, GitHub detected over 39 million leaked secrets across public repositories—API keys, tokens, passwords, and credentials that expose organizations to data breaches, unauthorized cloud access, and financial theft. The problem isn’t just about developers making mistakes; it’s about the delay between when a secret is committed and when the security team finds out. Most teams discover a leak after it’s been exploited, not before. Leak Radar addresses this critical gap by providing real-time monitoring of GitHub public repositories and JavaScript bundles, alerting security teams within minutes of exposure.

Learning Objectives:

  • Objective 1: Understand the architecture and implementation of a real-time secret detection system that monitors GitHub public repositories and JavaScript bundles for leaked credentials.
  • Objective 2: Master the technical implementation of regex-based pattern matching combined with Shannon entropy analysis for high-accuracy secret detection with minimal false positives.
  • Objective 3: Build and deploy automated alerting pipelines that notify security teams via Slack and email when exposed credentials are detected, with actionable file and commit information.

You Should Know:

1. The Scale of the Secret Leak Problem

Secret leaks remain one of the most common—and preventable—causes of security incidents. Despite GitHub’s Push Protection being enabled by default on all public repositories since February 2024, secrets continue to leak. The primary reasons? Developers prioritize convenience during commits, and accidental repository exposure through git history persists. A leaked API key can lead to account takeover, data exfiltration, and unexpected cloud bills running into thousands of dollars.

GitHub’s native secret scanning, while effective, has limitations—it primarily scans for known partner patterns and may miss secrets in non-standard files like .env.vercel, .env.backup, or custom configuration files. This is precisely why tools like Leak Radar are necessary: they provide supplemental monitoring that catches what falls through the cracks.

Step-by-Step Guide: Building a GitHub Public Repository Scanner

To implement the GitHub scanning component of Leak Radar, follow this architecture:

Step 1: Authenticate with GitHub API

 Generate a GitHub Personal Access Token (Classic)
 Settings → Developer settings → Personal access tokens → Tokens (classic)
 Select 'repo' scope for private repos or no scopes for public repos only
export GITHUB_TOKEN="ghp_xxxxxxxxxxxxxxxxxxxx"

Step 2: Query GitHub Code Search API for Potential Secrets

import requests
import time

def search_github_secrets(org_name, token):
headers = {"Authorization": f"token {token}", "Accept": "application/vnd.github.v3+json"}
queries = [
f'org:{org_name} filename:.env',
f'org:{org_name} "API_KEY"',
f'org:{org_name} "SECRET"',
f'org:{org_name} "sk_live_"',  Stripe live keys
f'org:{org_name} "AIza"',  Google API keys
]

for query in queries:
url = f"https://api.github.com/search/code?q={query}"
response = requests.get(url, headers=headers)
if response.status_code == 200:
results = response.json()
for item in results.get('items', []):
process_finding(item, query)
time.sleep(1)  Respect rate limits

Step 3: Fetch and Scan File Contents

def fetch_and_scan_file(repo_url, file_path, token):
raw_url = f"https://raw.githubusercontent.com/{repo_url}/main/{file_path}"
headers = {"Authorization": f"token {token}"}
response = requests.get(raw_url, headers=headers)

if response.status_code == 200:
content = response.text
detections = detect_secrets(content)
if detections:
send_alert(repo_url, file_path, detections)

Step 4: Implement Entropy-Based Detection

High-entropy strings—random-looking sequences—often indicate real secrets rather than placeholder values like "your_key_here".

import math
import re

def shannon_entropy(data):
"""Calculate Shannon entropy of a string"""
if not data:
return 0
entropy = 0
for x in range(256):
p_x = float(data.count(chr(x))) / len(data)
if p_x > 0:
entropy += - p_x  math.log(p_x, 2)
return entropy

def detect_secrets(content):
patterns = {
'AWS': r'AKIA[0-9A-Z]{16}',
'OpenAI': r'sk-[A-Za-z0-9]{48}',
'Google': r'AIza[0-9A-Za-z\_\-]{35}',
'GitHub': r'ghp_[A-Za-z0-9]{36}',
'Stripe': r'sk_live_[A-Za-z0-9]{24}',
}

findings = []
for service, pattern in patterns.items():
matches = re.findall(pattern, content)
for match in matches:
entropy = shannon_entropy(match)
if entropy > 3.5:  Threshold for real secrets
findings.append({
'service': service,
'match': match[:8] + '...' + match[-4:],  Mask for safety
'entropy': entropy
})
return findings

Step 5: Send Real-Time Alerts

import requests
import smtplib

def send_alert(repo, file_path, detections):
 Slack alert
webhook_url = "https://hooks.slack.com/services/YOUR/WEBHOOK"
message = {
"text": f"🚨 SECRET LEAK DETECTED\n"
f"Repository: {repo}\n"
f"File: {file_path}\n"
f"Findings: {detections}"
}
requests.post(webhook_url, json=message)

Email alert (using SMTP)
 ... email configuration

2. JavaScript Bundle Analysis: The Overlooked Attack Surface

Modern web applications ship massive JavaScript bundles to the browser. These bundles often contain hardcoded secrets—environment variables inlined during build, API keys, and service configurations. Tools like webpack, Vite, and `esbuild` inline environment variables directly into the JavaScript that users download. A scan of approximately 5 million applications found 100MB of plain text containing over 42,000 tokens across 334 types of secrets.

Step-by-Step Guide: Scanning JavaScript Bundles for Secrets

Step 1: Crawl Target Website for JavaScript Files

 Using a tool like surisc (Go-based JS bundle scanner)
go install github.com/marcuwynu23/surisc@latest
surisc --url https://example.com --output results.json

Surisc scrapes every `