Listen to this Post
Introduction:
The arrest of a 23-year-old Portuguese hacker by Portugal’s Criminal Investigation Police (PJ), following an FBI alert, has shed new light on the inner workings of WormGPT—a malicious ChatGPT clone sold on dark web forums. Unlike mainstream AI models fortified with safety guardrails, WormGPT was deliberately stripped of ethical boundaries and trained on malware samples, jailbreak prompts, and social engineering scripts. This case underscores a critical inflection point: generative AI is no longer just a defensive tool—it is an offensive weapon that can be wielded by anyone with a subscription and malicious intent.
Learning Objectives & Secrets:
- Objective 1: Understand WormGPT’s architecture and how it differs from legitimate LLMs. WormGPT is not a single monolithic model but a client-side wrapper that interfaces with open-source large language models like GPT-J (6 billion parameters) or Mixtral via APIs. Its “evil twin” capability comes from fine-tuning on cybercriminal datasets and removing content filters.
-
Objective 2 secret tip: Identify WormGPT-generated phishing and malware indicators. WormGPT-generated phishing emails often lack the grammatical errors typical of traditional scams but may exhibit unnatural urgency, generic greetings, or slight contextual inconsistencies. Cross-referencing sender domains and inspecting email headers for spoofing remains essential.
-
Objective 3 secret tip: Implement detection and response workflows for AI-generated threats. Deploy secure email gateways with AI-powered detection, enforce zero-trust architecture with hardware-backed identity verification, and conduct regular red-team exercises simulating WormGPT-style attacks.
You Should Know:
1. WormGPT’s Technical Architecture and Deployment
WormGPT operates as a typical API client application, primarily written in Python. Its core workflow involves initializing the environment, reading configuration files (including API keys), and sending prompts to an external LLM via REST APIs. The backend often leverages frameworks like FastAPI or Flask to build lightweight web services.
The model itself is built on transformer architecture—the same foundation as ChatGPT—but trained on a custom dataset composed of malware samples, exploit code, red teaming manuals, and social engineering scripts. This specialized training enables WormGPT to generate functional malicious outputs that legitimate models would refuse.
Key command-line interaction (legitimate security research context):
Clone a legitimate WormGPT-style CLI tool for research (educational use only) git clone https://github.com/hexsecteam/worm-gpt cd worm-gpt Install dependencies pip install -r requirements.txt Configure API key (for OpenRouter or similar services) export OPENROUTER_API_KEY="your_api_key_here" Run the CLI tool python wormgpt.py --model "gpt-j-6b" --prompt "Analyze this email for phishing indicators"
Windows equivalent (PowerShell):
Set environment variable $env:OPENROUTER_API_KEY="your_api_key_here" Run Python script python wormgpt.py --model "gpt-j-6b" --prompt "Generate a security awareness training email"
⚠️ Important: These commands are provided for defensive security research and educational purposes only. Unauthorized use of AI tools to generate malicious content is illegal and unethical.
2. WormGPT’s Capabilities in the Wild
According to investigation sources, approximately 70 hackers from Australia, the US, South Africa, Singapore, Israel, and other countries used WormGPT in August 2023 alone. The platform was marketed on underground hacking forums with subscription-based access.
WormGPT’s advertised features include:
- Phishing campaign automation: Generating convincing Business Email Compromise (BEC) messages and spear-phishing lures
- Malware code generation: Creating functional PowerShell scripts with AES-256 encryption and Tor-based data exfiltration
- Character support and chat memory retention: Maintaining context across conversations for complex attack planning
- Uncensored responses: No refusal to generate harmful content, unlike mainstream models
The creator reportedly earned €24,290 from the operation and had a letter on his computer stating he would sell the program for €55,000.
3. The Arrest and Legal Implications
The hacker, identified only as João (not his real name), began hacking at age 10 and was motivated primarily by learning. However, when other hackers began requesting tools for more serious crimes—like credit card cloning—he attempted to shut down the program, inadvertently alerting the FBI.
Arrested in early 2025 and released on bail, João was initially barred from internet access—a significant obstacle for his computer engineering studies. The privilege was eventually restored, and he has been fully cooperative with the investigation. His cooperation led to the number of alleged crimes being reduced to three charges of computer sabotage. Authorities hope he will help identify the users of his platform.
The judge urged João to “use his brain for good”—a poignant reminder that technical talent, when misdirected, can cause significant harm.
4. Defensive Measures Against AI-Powered Threats
Organizations must adapt their security posture to counter AI-generated attacks:
Step-by-step guide to implementing AI threat defense:
- Deploy AI-powered email security: Implement secure email gateways that use machine learning to detect AI-generated phishing, which often lacks traditional red flags.
-
Enforce zero-trust architecture: Verify every access request with cryptographic identity binding and continuous device posture checks.
-
Conduct regular security awareness training: Train employees to recognize sophisticated AI-generated social engineering attempts.
-
Implement LLM-generated attack detection: Use solutions like Palo Alto Networks’ LLM-Generated Attacks Detection to identify threats created using models like WormGPT.
-
Monitor for jailbreak attempts: Track prompts attempting to bypass content filters on legitimate AI services.
Example SIEM query to detect potential WormGPT-related activity:
-- Splunk query to detect suspicious API calls to LLM services index=firewall OR index=proxy (uri="openrouter" OR uri="api.openai" OR uri="groq") AND (useragent="python" OR useragent="curl") AND dest_ip IN (list_of_known_threat_ips) | stats count by src_ip, user, uri | where count > 100
5. The Future of Malicious AI
WormGPT was reportedly shut down in 2023 after intense public attention, but imitations and rebranded variants—including WormGPT 4 and KawaiiGPT—continue to appear on criminal markets. These newer versions are built on models like Grok and Mixtral, with enhanced capabilities for generating ransomware, encryption tools, and automated attack scripts.
The commoditization of malicious AI tools lowers the barrier to entry for cybercrime, enabling even inexperienced hackers to launch sophisticated attacks. This trend represents a fundamental shift in the threat landscape.
What Undercode Say:
- Key Takeaway 1: The WormGPT case demonstrates that AI security is not just about protecting models from being hacked—it’s about preventing models from being weaponized. Organizations must treat AI as both a defensive tool and an attack surface.
-
Key Takeaway 2: The creator’s journey from curious teenager to criminal defendant highlights the importance of ethical education in computer science. Technical talent without ethical grounding can cause damage on a global scale. The judge’s plea to “use his brain for good” should be a mantra for the entire cybersecurity community.
The arrest sends a strong message: law enforcement is actively tracking and prosecuting developers of malicious AI tools. However, the cat is already out of the bag. WormGPT’s source code and variants will continue to circulate, and new tools will emerge. The real battle is not just against individual creators but against the entire ecosystem that enables AI-powered cybercrime. Organizations must invest in AI-specific defenses, and the industry must develop better mechanisms to detect and disrupt the development of malicious AI models before they reach the dark web.
Prediction:
- +1 The arrest and ongoing cooperation of the WormGPT creator may lead to the identification and prosecution of dozens of cybercriminals who used the platform, potentially disrupting multiple ongoing criminal operations.
-
-1 The open-source nature of many LLMs means that removing one malicious tool does not eliminate the threat. New variants—built on different base models with even more sophisticated capabilities—will continue to emerge, potentially faster than law enforcement can respond.
-
-1 As AI-generated content becomes increasingly indistinguishable from human-written text, traditional security awareness training may become less effective, forcing organizations to rely more heavily on technical controls like AI-powered email filtering and zero-trust authentication.
-
+1 The WormGPT case may serve as a deterrent for other young developers considering creating malicious AI tools, particularly as law enforcement agencies around the world develop specialized units to combat AI-enabled cybercrime.
-
-1 The revenue model for malicious AI tools—subscription-based access starting as low as $50 per month—makes them accessible to a wide range of criminals, ensuring continued demand regardless of individual prosecutions.
▶️ Related Video (82% Match):
https://www.youtube.com/watch?v=1UGJayDzCQo
🎯Let’s Practice For Free:
🎓 Live Courses & Certifications:
Join Undercode Academy for Verified Certifications
🚀 Request a Custom Project:
Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands
IT/Security Reporter URL:
Reported By: https://lnkd.in/p/ej8fEQ2c – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
