Listen to this Post
Introduction:
The bug bounty community has a growing problem: AI-generated vulnerability reports that sound convincing but lack verifiable evidence. Security researchers using cloud-based LLMs to draft reports risk leaking embargoed proof-of-concept code and submitting unsubstantiated claims that waste triagers’ time. ProofForge AI is a local-first, open-source assistant that flips this paradigm—it builds a claim→evidence graph for every report and flags unsupported assertions before they ever reach a triager. Running entirely on an 8GB laptop with no cloud telemetry, this tool ensures your bug reports are honest, compliant, and defensible.
Learning Objectives:
- Understand how claim→evidence graph architecture prevents hallucination in security reporting
- Deploy ProofForge AI locally with Ollama and configure program-specific YAML rule packs
- Master the six-stage pipeline from evidence ingestion to CVSS-scored, compliant report generation
- Implement tamper-evident audit logging for submission integrity and chain-of-custody
- Extend the framework with custom vulnerability classes using YAML templates
You Should Know:
- Claim-Evidence Graph: The Integrity Layer That Kills Hallucination
ProofForge AI’s core innovation is its claim→evidence directed acyclic graph (DAG). Every assertion in your report becomes a node; every piece of evidence—screenshots, HTTP transactions, reproduction steps—becomes another. The system performs three integrity checks before you can export:
- Orphan claims: Assertions with no supporting evidence get flagged immediately
- Over-reach: Claims that exceed what the evidence actually proves are detected
- Inconsistencies: Contradictory statements across the report are surfaced
What this means in practice: When running against OWASP Juice Shop, ProofForge correctly identified that a JSON response alone didn’t prove DOM-XSS execution—only the screenshot did. The LLM (qwen2.5:3b via Ollama) inferred the endpoint and payload, but the integrity layer enforced rigorous evidence mapping before report generation.
Step-by-Step: Deploying ProofForge AI with Local LLM
bash
Clone the repository
git clone https://github.com/Uzair4264/ProofForge-AI.git
cd ProofForge-AI
Install Ollama (Linux/macOS)
curl -fsSL https://ollama.com/install.sh | sh
Pull the recommended model
ollama pull qwen2.5:3b-instruct-q4_K_M
Install Python dependencies (requires Python 3.10+)
pip install -r requirements.txt
Verify installation
python -m bbreport –help
Run a test against sample evidence
python -m bbreport ingest –evidence ./sample_evidence/ –program owasp-juice-shop
[/bash]
For Windows users, install Ollama via the Windows installer from ollama.com, then use the same commands in PowerShell or WSL2 environment.
- Program Rule Packs: Enforcing Scope and Compliance in YAML
Each bug bounty program is encoded as a YAML rule pack. This deterministic layer enforces:
- Scope definitions: Which domains, endpoints, and asset types are in-scope
- Required fields: Mandatory sections like “Steps to Reproduce,” “Impact,” “Remediation”
- Forbidden content: What cannot appear in reports (e.g., PII, internal IPs)
- Severity scoring: Deterministic CVSS v3.1 calculation based on actual impact metrics
Sample YAML Rule Pack Structure:
bash
program:
name: “example-bounty-program”
scope:
domains:
– “.example.com”
excluded:
– “admin.example.com”
required_fields:
– “vulnerability_description”
– “steps_to_reproduce”
– “impact_assessment”
– “remediation”
severity:
cvss_version: “3.1”
override_rules:
– condition: “proof_of_concept_executes_remotely”
severity: “Critical”
forbidden_content:
– patterns:
– “internal\.ip”
– “ssn|social security”
[/bash]
- The Six-Stage Pipeline: From Evidence to Compliant Report
ProofForge AI processes evidence through six deterministic stages:
- Ingestion & Classification: HTTP transactions, screenshots, and reproduction steps are parsed and categorized by vulnerability class (XSS, SSRF, IDOR, CSRF, SQLi—each with YAML templates)
-
Inference: The local LLM (Ollama) infers endpoint details and payload characteristics from the evidence
-
Adaptive Elicitation: The system prompts you to confirm or clarify inferred values—you remain in control
-
Integrity Verification (Gap 1): The claim→evidence graph is constructed; orphans and over-reach are flagged
-
Compliance Enforcement (Gap 3): Program rules are applied deterministically; severity is calculated
-
Rendering: Outputs a submission-ready `report.md` and a tamper-evident `audit.jsonl` log
Verifying the Audit Log:
bash
View the hash chain
cat audit.jsonl | jq ‘.hash_chain’
Verify integrity of a specific report
python -m bbreport verify –report report.md –audit audit.jsonl
[/bash]
4. Tamper-Evident Audit Logging: Chain-of-Custody for Bug Reports
Every interaction with ProofForge AI produces a hash-chained audit log. This creates a cryptographic chain of custody that proves:
- What evidence was available at report generation time
- Which inferences the LLM made and which you confirmed
- When the report was generated and by whom
Audit Log Entry Example (JSONL):
bash
{
“timestamp”: “2026-07-16T10:23:45Z”,
“event”: “claim_added”,
“claim”: “DOM-XSS executed on /search endpoint”,
“evidence_refs”: [“screenshot_001.png”, “burp_request_004.json”],
“hash_prev”: “a3f5c2…”,
“hash_current”: “b8e9d1…”
}
[/bash]
This level of auditability is critical for both researcher credibility and program compliance—especially when dealing with high-severity findings that require rigorous proof.
5. Extending Vulnerability Classes with YAML Templates
One of ProofForge’s most powerful features is that adding new vulnerability classes requires zero Python code. Each class is defined as a YAML template:
Custom Vulnerability Class Template (`custom_rce.yaml`):
bash
vulnerability_class:
name: “RCE”
detection_patterns:
– “system\(|exec\(|eval\(”
– “cmd\.exe|/bin/sh”
evidence_requirements:
– type: “http_request”
required_fields: [“method”, “path”, “payload”]
– type: “screenshot”
required: true
cvss_metrics:
attack_vector: “NETWORK”
attack_complexity: “LOW”
privileges_required: “NONE”
user_interaction: “NONE”
scope: “CHANGED”
confidentiality: “HIGH”
integrity: “HIGH”
availability: “HIGH”
[/bash]
6. API Security and Cloud Hardening Considerations
While ProofForge AI runs 100% locally by default, it includes an opt-in cloud backend behind the same interface. For organizations considering this option:
- API key management: Store keys in environment variables, never in code
- TLS configuration: Ensure all cloud communications use TLS 1.3
- Audit logging: Even with cloud backend, the hash chain remains locally verifiable
Linux Command for Secure Environment Variable Storage:
bash
Store API keys securely
echo “export PROOFFORGE_CLOUD_KEY=’your-key-here'” >> ~/.bashrc
source ~/.bashrc
Verify no keys are exposed in process lists
ps aux | grep -i proofforge
[/bash]
Windows PowerShell Equivalent:
bash
Set environment variable for current session
$env:PROOFFORGE_CLOUD_KEY = “your-key-here”
Persist across sessions
[/bash]
7. Cross-Platform Compatibility and Testing
ProofForge AI has 33 passing tests across Linux, macOS, and Windows. The CLI tool `bbreport` works consistently across platforms, with the same evidence ingestion, integrity verification, and report generation pipeline.
Running the Test Suite:
bash
Run all tests
pytest tests/
Run specific test category
pytest tests/test_integrity.py -v
Check cross-platform compatibility
python -m bbreport platform-check
[/bash]
What Undercode Say:
- Evidence integrity is non-1egotiable: The claim→evidence graph isn’t just a feature—it’s a fundamental shift in how security researchers should approach reporting. The fact that ProofForge flags “orphan” claims before submission means researchers can’t accidentally (or intentionally) submit unsubstantiated findings. This directly addresses the credibility crisis facing the bug bounty ecosystem.
-
Local-first is the only responsible approach: With embargoed PoCs and sensitive vulnerability details, sending data to cloud chatbots is a genuine security risk. ProofForge’s 100% local operation (running on an 8GB laptop) makes it viable for researchers working with confidential disclosures. The opt-in cloud backend is transparent and clearly delineated, not the default.
-
The YAML rule pack system is brilliant for scalability: Adding new vulnerability classes without touching Python code lowers the barrier to contribution significantly. This means the community can rapidly expand coverage—new OWASP Top 10 categories, IoT vulnerabilities, or even blockchain-specific findings can be templated and shared.
-
Deterministic CVSS scoring removes ambiguity: By calculating CVSS v3.1 deterministically based on actual impact metrics, ProofForge eliminates the “severity inflation” problem where researchers overstate findings to increase bounty payouts. Programs will appreciate the consistency.
-
The hash-chained audit log is underappreciated: Beyond just report generation, this creates a verifiable chain of custody that protects both researchers and programs. If a dispute arises about what was submitted, the audit log provides cryptographic proof of the evidence available at the time of reporting.
Prediction:
-
+1 ProofForge AI’s approach will become the de facto standard for professional bug bounty reporting within 18-24 months. As programs increasingly reject AI-generated junk, tools that enforce evidence integrity will be essential for serious researchers.
-
+1 The YAML template system will spawn a community-driven repository of vulnerability class definitions, similar to how Metasploit’s module system evolved. This will dramatically accelerate coverage for emerging vulnerability types.
-
-1 The reliance on local LLMs (even optimized 3B models) may produce inconsistent inference quality compared to cloud-based models. While the integrity layer catches hallucinations, researchers may find themselves manually correcting more inferences than with larger models.
-
+1 Enterprise security teams will adopt ProofForge-like integrity verification for internal pentest reporting, not just bug bounties. The claim→evidence graph provides the same value for compliance audits and regulatory submissions.
-
-1 Adoption may be slow among less-technical researchers who prefer the convenience of GPT wrappers. The tool’s CLI-first nature and local setup requirements create a barrier that will initially limit its user base to more experienced practitioners.
▶️ Related Video (76% Match):
https://www.youtube.com/watch?v=01VIPe-Iuf4
🎯Let’s Practice For Free:
🎓 Live Courses & Certifications:
Join Undercode Academy for Verified Certifications
🚀 Request a Custom Project:
Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands
IT/Security Reporter URL:
Reported By: Syed Uzair – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
