Listen to this Post
Introduction
The cybersecurity industry has long operated under the assumption that frontier AI models—those developed by well-funded labs with billions of parameters—pose the greatest offensive security risk. However, recent benchmarking data from RamiGPT, an autonomous privilege escalation agent, challenges this narrative. The findings reveal a more nuanced reality: smaller, locally-runnable models like DeepSeek-R1-14B are outperforming same-size competitors and achieving pass rates that rival much larger models. More critically, the data suggests that the true inflection point may not be when frontier models become 20% more capable, but when capable offensive AI becomes cheap, local, and accessible to anyone with a consumer GPU—transforming millions of mediocre attackers into substantially more effective threats.
Learning Objectives
- Understand the comparative performance landscape of LLMs in automated privilege escalation scenarios, including pass rates, token efficiency, and policy restriction impacts
- Deploy and configure RamiGPT for autonomous security testing across multiple AI providers (Ollama, OpenRouter, OpenAI, Open WebUI)
- Execute benchmark suites against misconfigured Linux targets and interpret results using the built-in analytics dashboard
- Implement session context controls (Facts, Hints, Avoid) to guide AI behavior and improve penetration testing outcomes
- Evaluate the security implications of democratized offensive AI and develop defensive strategies accordingly
1. RamiGPT Architecture and Core Components
RamiGPT is an AI-powered offensive security agent designed to automate privilege escalation on Linux systems. Built on PwnTools and LLM capabilities, it navigates privilege escalation scenarios across VulnHub machines and custom test environments. The application follows a modular architecture:
ramigpt/web/ – Flask/Socket.IO UI with shell layer and Full AI hooks
ramigpt/ai/ – Provider interface supporting Ollama, Open WebUI, OpenAI, OpenRouter, and Cursor API
ramigpt/domain/ – Privilege escalation prompt engineering and root detection logic
ramigpt/benchmark/ – Orchestrator for remote Ansible deployment and Full AI runs
docker/benchmark/ – One-image LPE labs with 285 misconfiguration targets across ports 2170–2454
The Full AI autonomous loop operates as follows: build prompt → query LLM → execute one shell command → append to history → check for root → repeat. Session v2 (enabled in App Settings) improves command extraction and handles interactive edge cases—sudo password prompts, stuck editors, and nested root shells.
Deployment Options
Docker deployment:
git clone https://github.com/M507/RamiGPT.git cd RamiGPT cp .env.example .env edit with your provider + keys docker compose -f docker/docker-compose.yml up -d
Access at: https://127.0.0.1:8443
Local deployment:
git clone https://github.com/M507/RamiGPT.git cd RamiGPT python3 -m venv venv source venv/bin/activate Windows: venv\Scripts\activate chmod +x ./scripts/generate_certs.sh ./scripts/generate_certs.sh pip install -r requirements.txt cp .env.example .env edit provider + API keys python app.py
The app listens on `127.0.0.1:8443` by default.
Prerequisites
- Docker and Docker Compose (for containerized deployment)
- Python 3 and pip (for local deployment)
– `ansible-core` 2.18–2.19 (via requirements.txt; supports Python 3.8 on Ubuntu 20.04) - Ubuntu/Debian host packages:
openssh-client,sshpass,openssl, `ca-certificates`
– AI backend: OpenAI key, Ollama host, Open WebUI, OpenRouter, or Cursor API key
Run the Ubuntu requirements script once:
python3 scripts/ensure_ubuntu_requirements.py
2. Benchmarking Methodology and Key Findings
The RamiGPT benchmark suite comprises 285 misconfigured Linux targets across 34 preset profiles. Targets include themed mixes: Cloud & DevOps credentials, Obscure GTFOBins, Interpreter escapes, Hard non-sudo, and family buckets like Classic sudo, SUID, and Credentials.
Critical Benchmark Results (322 runs, updated 2026-08-12)
| Model | Pass Rate | Tokens→Root | n |
|-|–|-||
| Claude Opus 4.8 | 100% | 3,834 | 11 |
| Claude Opus 4.1 | 100% | 1,801 | 3 |
| Claude Sonnet 4.5 | 76.5% | 3,835 | 17 |
| Gemini 3.5 Flash-Lite | 61.1% | 5,002 | 18 |
| Qwen 3.6-35B | 61.4% | 33,990 | 381 |
| Claude Sonnet 4.6 | 50.0% | 1,990 | 24 |
| Claude Opus 4 | 50.0% | 2,398 | 18 |
| Gemma-4-31B-IT | 47.8% | 2,671 | 69 |
| GPT-3.5 Turbo | 47.8% | 0 | 90 |
| DeepSeek-R1-14B | 25.5% | 4,653 | 740 |
| Qwen3-14B | 2.3% | 5,655 | 219 |
| Llama-4 Maverick | 12.5% | 9,842 | 80 |
Token data not recorded for OpenAI GPT-3.5 Turbo runs
Key Observations
GPT-3.5 Turbo remains surprisingly solid, achieving a 47.8% pass rate—matching the much newer Gemma-4-31B-IT and outperforming GPT-5.5 (15.4%), GPT-5.2 (19.2%), and several newer mid-tier models.
Small DeepSeek models outperform same-size Qwen models, with deepseek-r1-14b achieving 25.5% pass rate versus Qwen3-14B at just 2.3%—making it the strongest local ≤14B “hacking” model in current results.
Gemma-4-31B-IT is a surprise mid-tier performer with 47.8% pass rate at relatively low token cost (2,671 tokens→root).
Anthropic models dominate the leaderboard, with Opus 4.8 and Opus 4.1 achieving 100% pass rates. However, policy restrictions block testing of the latest models. Claude Sonnet 4.5 and 4.6 show strong performance at 76.5% and 50% respectively.
Uncensored does not mean capable—stripping refusals did not help, and uncensored fine-tunes sometimes made the agent act in worse ways.
Benchmark Execution
From the UI:
1. Configure AI (top bar → AI Settings)
2. Click Benchmark
- Set the remote lab host (SSH for Ansible)
- Configure model plan and role plan (multiple models/roles and runs per target)
- Pick a target profile (default: Regression sample, ~19 labs) or Select all
- Optionally deploy selected targets then test targets (get root)
7. Set per-target timeout (default 180s)
8. Start Benchmark—sessions appear under the Benchmark group
Ansible deployment:
ansible-playbook -i ansible/benchmark/inventory.example.ini ansible/benchmark/playbook.yml
Verify targets:
./scripts/benchmark/verify-misconfigs.sh or python3 -m ramigpt.benchmark.verify
3. AI Provider Configuration and Model Selection
RamiGPT supports five AI backends: Ollama, Open WebUI, OpenAI, OpenRouter, and Cursor API. The selection persists in data/ai_settings.json; `.env` remains the source for API keys and initial defaults.
OpenRouter Configuration
OpenRouter provides access to the widest range of models tested in the benchmark:
– Create an API key at openrouter.ai/keys
– Model IDs use the `provider/model` form (e.g., openai/gpt-4o-mini, anthropic/claude-sonnet-4)
– Uses the official OpenRouter Python SDK
– Leave `OPENROUTER_BASE_URL` empty for `https://openrouter.ai/api/v1`
– Use the refresh icon in AI Settings to pull the live model catalog
.env Configuration
Provider selection AI_PROVIDER=openrouter ollama, openwebui, openai, openrouter, cursor API Keys OPENAI_API_KEY=sk-... OPENROUTER_API_KEY=sk-or-... OLLAMA_HOST=http://localhost:11434 Optional: override base URLs OPENROUTER_BASE_URL=
Recommended Model Selection Strategy
Based on benchmark data:
- Maximum capability: Anthropic Claude Opus 4.8 (100% pass, 3,834 tokens→root)
- Token efficiency: Qwen3-Coder (1,541 tokens→root, 22.2% pass)
- Best local ≤14B: DeepSeek-R1-14B (25.5% pass, 4,653 tokens→root)
- Best value mid-tier: Gemma-4-31B-IT (47.8% pass, 2,671 tokens→root)
- Legacy but capable: GPT-3.5 Turbo (47.8% pass, token data not recorded)
4. Session Context Controls: Facts, Hints, and Avoid
RamiGPT provides per-session queues in the Terminal AI panel to steer Full AI without editing .env:
- Facts — ground truth the model should treat as established (e.g., kernel version, discovered SUID binaries)
- Hints — suggested directions without guaranteeing success
- Avoid — commands or approaches that already failed
Use Import / Export to move this context between sessions or capture it for write-ups and flags.
Practical Example
When targeting a system with known kernel vulnerability:
Facts: "Linux kernel 5.4.0-42-generic, DirtyCow vulnerability exists" Hints: "Try exploiting CVE-2016-5195 via /etc/passwd race condition" Avoid: "sudo -l already attempted, no sudo privileges found"
The AI will incorporate this context into its prompt, reducing trial-and-error and focusing on viable attack paths.
5. Bundled Enumeration Tools and Integration
RamiGPT integrates several industry-standard privilege escalation enumeration tools:
- BeRoot — Identifies common privilege escalation vectors on Linux (sudo, SUID, capabilities, writable paths, and more)
- LinPEAS — Audits Linux environments for misconfigurations and vulnerabilities
- LinEnum — Lightweight enumeration script
Run them from the Terminal tool dropdown. With the AI checkbox enabled, RamiGPT uploads the tool, captures output, and chains into Full AI using the findings.
Manual Usage
From within the RamiGPT terminal ./tools/linpeas.sh ./tools/beRoot.py ./tools/linenum.sh
AI-Assisted Enumeration
The AI can autonomously:
1. Upload the appropriate enumeration tool
2. Parse the output for privilege escalation vectors
3. Generate and execute exploitation commands
4. Verify root access and document the chain
6. Defensive Implications and Hardening Strategies
The democratization of offensive AI presents new challenges for blue teams. Organizations must adapt their defensive posture accordingly.
Key Defensive Recommendations
Assume compromise is inevitable — With local models achieving 25%+ pass rates on misconfigured systems, the barrier to entry for effective privilege escalation has dropped significantly.
Implement least privilege rigorously — Many successful attacks in the benchmark exploited excessive sudo privileges, SUID binaries, and writable paths.
Monitor for unusual AI patterns — AI agents exhibit different command patterns than human attackers. Look for:
– Rapid sequential command execution
– Consistent command structure across attempts
– Unusual enumeration tool usage
Deploy Canary tokens — AI agents may not recognize honeypot files or canary tokens embedded in the filesystem.
Hardening Checklist
Audit sudo privileges sudo -l Find SUID binaries find / -perm -4000 -type f 2>/dev/null Check writable directories in PATH find / -writable -type d 2>/dev/null | grep -v proc Audit capabilities getcap -r / 2>/dev/null
What Undercode Say
- The biggest risk isn’t frontier labs getting stronger; it’s when capable offensive AI becomes cheap, local, and accessible to everyone — The benchmark confirms that models like DeepSeek-R1-14B and GPT-3.5 Turbo, while not matching frontier performance, are sufficiently capable to automate privilege escalation on real-world misconfigurations. The barrier to entry has dropped from “nation-state resource” to “consumer GPU.”
-
Policy restrictions are the primary defense against frontier model abuse — Anthropic’s latest models are blocked from offensive security testing. This artificial constraint may be the only thing preventing 100% pass rates from becoming publicly available. However, open-weight models like DeepSeek and Gemma are not subject to such restrictions.
-
The data suggests a future where offensive AI capability is commoditized — With 322 runs already completed and collaborative benchmarking enabled, the RamiGPT project is building the largest public dataset of LLM-driven privilege escalation. This transparency benefits both attackers and defenders.
-
Defenders must adapt to AI-speed attacks — AI agents operate at machine speed, executing commands in seconds that would take human attackers minutes or hours. Traditional incident response timelines are no longer sufficient.
-
Uncensored models are not the answer — The data shows that stripping refusals doesn’t improve capability; it often degrades performance. This suggests that capability and safety are not necessarily in tension—well-aligned models may actually be more effective.
Prediction
-1 The commoditization of offensive AI will lead to a surge in automated attacks targeting misconfigured Linux systems. Organizations that have not implemented least privilege, proper patch management, and continuous monitoring will face increased breach risk within 12–18 months.
-1 Small, open-weight models (<14B parameters) will continue to improve, potentially achieving 40-50% pass rates within 2-3 years—enough to automate the majority of common privilege escalation vectors without requiring cloud API access or significant compute resources.
+1 The RamiGPT benchmark framework provides a standardized, transparent methodology for evaluating LLM offensive capability. This will accelerate defensive research, enabling blue teams to test their environments against AI-driven attacks using the same tools attackers will employ.
+1 Collaborative benchmarking and open-source tooling democratize security research, allowing smaller organizations and independent researchers to contribute to the understanding of AI-driven offensive capabilities.
-1 The gap between frontier and open-weight models will narrow faster than policy restrictions can adapt, creating a window where capable offensive AI is both publicly available and difficult to regulate.
+1 Organizations that embrace AI-assisted penetration testing now—using tools like RamiGPT in authorized environments—will develop defensive muscle memory before adversarial use becomes widespread.
-1 The day when “a high-schooler with some free time and a consumer GPU can run a genuinely capable hacking model locally” is closer than the industry anticipates. This will fundamentally change the threat landscape, requiring a rethinking of cybersecurity education, certification, and defensive strategy.
▶️ Related Video (78% Match):
🎯Let’s Practice For Free:
🎓 Live Courses & Certifications:
Join Undercode Academy for Verified Certifications
🚀 Request a Custom Project:
Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands
IT/Security Reporter URL:
Reported By: https://lnkd.in/p/eWWMBgq3 – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
