How I Exposed My LLM Server to the Internet and Got Hacked in Under 2 Minutes — A Lesson in Internet Background Noise + Video

By /

Listen to this Post

Featured Image

Introduction:

The internet is not a passive space. Every publicly exposed asset exists within a globally scanned address space where probing never stops. Whether you deliberately expose a service via Tailscale Funnel or inadvertently leave a port open, the clock starts ticking immediately — vulnerability scanners, botnets, and reconnaissance tools will find you within minutes. This article explores the reality of “internet background noise,” why your freshly exposed LLM inference server becomes an instant target, and how to defend against the relentless tide of automated attacks using honeypot platforms like T-Pot.

Learning Objectives:

  • Understand the concept of internet background noise and why exposed services are scanned within seconds.
  • Learn how to deploy and configure Tailscale Funnel securely for remote LLM inference.
  • Master the installation and analysis of T-Pot, the all-in-one multi-honeypot platform from Telekom Security.
  • Identify common attack vectors, from SSH brute-force to historical exploits, and implement mitigation strategies.
  • Develop a threat intelligence mindset to distinguish between targeted attacks and background radiation.

You Should Know:

  1. The Reality of Internet Background Noise — Why Your Service Gets Scanned in Under 2 Minutes

Every organization connected to the internet is constantly exposed to unsolicited traffic. Port scans, service probes, crawler activity, misconfigured systems, and bot-driven requests arrive continuously, originating from both benign actors and malicious ones. This is not targeted in the traditional sense — it is systemic and unavoidable.

When Daniel Kloyber set up a Tailscale Funnel to expose his Framework Desktop for remote LLM inference, the logs filled with vulnerability scans in under two minutes — before he had even announced the service or generated any of his own traffic. This is the baseline condition of the internet.

Step‑by‑step guide to understanding and measuring your exposure:

  1. Deploy a test service — Use a simple HTTP server (Python’s `http.server` or Node.js `http` module) and expose it via Tailscale Funnel or a cloud provider.
  2. Monitor logs in real-time — `tail -f /var/log/syslog` or your application logs. Watch for unexpected GET requests, path traversals, and malformed HTTP methods.
  3. Use GreyNoise to classify traffic — Query the GreyNoise API to determine if scanning IPs are mass-scanning the internet or part of targeted campaigns: 
    curl -s "https://api.greynoise.io/v3/community/<IP_ADDRESS>" | jq .
  4. Analyze the first 60 seconds — Count unique source IPs, requested paths, and HTTP methods. You will typically see probes for /wp-admin, /cgi-bin/, .env, and common CVE endpoints.
  5. Document the findings — Create a threat intelligence report categorizing the noise (botnets, researchers, misconfigured crawlers) versus potential threats.

Linux/Windows commands for real-time monitoring:

  • Linux: `sudo tcpdump -i any -1 ‘port 443’ | grep -i “GET\|POST”`
    – Windows (PowerShell): `Get-1etTCPConnection -State Listen | Where-Object {$_.LocalPort -eq 443}`
  1. Tailscale Funnel — A Double-Edged Sword for Remote LLM Inference

Tailscale Funnel provides a permanent public HTTPS URL with automatic TLS, making it an attractive solution for exposing local services like LLM inference endpoints without complex port forwarding. However, this convenience comes with significant security risks.

Known vulnerabilities (TS-2026-008): A single malformed HTTP request to a node running Tailscale Funnel could pin a CPU core indefinitely, causing denial of service. For Funnel, the request could originate from any unauthenticated host on the internet. This vulnerability is fixed in Tailscale version 1.98.9 or newer.

Critical misconfiguration example: Setting `AUTH_DISABLED=true` while Tailscale Funnel is actively serving a public URL means every API endpoint is accessible to anyone without credentials.

Step‑by‑step guide to securing Tailscale Funnel for LLM inference:

  1. Upgrade Tailscale — Ensure you are running version 1.98.9 or newer: 
    sudo tailscale update
tailscale version
  2. Implement authentication — Never disable authentication. Use Tailscale ACLs to restrict access to specific users or groups: 
    {
"acls": [
{
"action": "accept",
"src": ["group:engineering"],
"dst": ["tag:llm-server:"]
}
]
}
  3. Use guardrails for LLM requests — Tailscale Guardrails are pre-request hooks that let you inspect, modify, or block LLM requests before data leaves your network. Configure guardrails to enforce security policies, scrub PII, or restrict tool usage at the gateway level.
  4. Limit exposure duration — Use `tailscale funnel –bg` to run in the background but set up automated scripts to disable the funnel after a defined period.
  5. Monitor with Tailsnitch — Deploy Tailsnitch, a security auditor for Tailscale configurations that scans your tailnet for 50+ misconfigurations, overly permissive access controls, and security best practice violations. 
    Example Tailsnitch scan
tailsnitch scan --tailnet your-tailnet

Windows command to check Tailscale version:

tailscale version

3. T-Pot — The All-In-One Multi-Honeypot Platform

T-Pot, developed by the Deutsche Telekom Security Team, is an open-source honeypot platform that integrates multiple honeypot technologies into a single, user-friendly system. It operates a set of containerized honeypots and monitoring tools under a single web interface, supporting 20+ honeypots and visualization options using the Elastic Stack.

As Koorosh Ghorbani noted, within 24 hours of deploying T-Pot, he logged 1.5 million attacks, from SSH brute-force attempts to typical HTTP scanners — including attack vectors dating back to 2001.

System requirements: At least 8-16 GB RAM, 128 GB free disk space, and a working (outgoing non-filtered) internet connection.

Step‑by‑step guide to deploying T-Pot on Ubuntu:

  1. Provision a VM or cloud instance — Recommended: Ubuntu 22.04 LTS with at least 16 GB RAM and 128 GB storage.

2. Update the system and install dependencies:

sudo apt update && sudo apt upgrade -y
sudo apt install curl docker.io docker-compose -y
sudo systemctl enable docker && sudo systemctl start docker

3. Run the T-Pot installer as a non-root user:

env bash -c "$(curl -sL https://github.com/telekom-security/tpotce/raw/master/install.sh)"

Follow the on-screen instructions, read messages carefully, and check for possible port conflicts.
4. Configure firewall rules — T-Pot requires specific ports to be open for honeypot services. Review the required ports in the documentation and adjust your Network Security Group (NSG) or iptables accordingly.
5. Access the dashboard — After installation and reboot, access the T-Pot web interface at https://<YOUR_VM_IP>:64297. Default credentials are displayed during installation.
6. Analyze attack data — Use Kibana (integrated with T-Pot) to visualize attack patterns, geolocate attackers, and identify emerging threats.

Linux command to monitor T-Pot logs in real-time:

sudo docker logs -f tpot_cowrie_1  Replace with specific honeypot container

Windows (Docker Desktop) installation: T-Pot now supports macOS and Windows via Docker Desktop, although limited to the feature set of Docker Desktop.

4. Common Attack Vectors Observed in Honeypot Deployments

Within hours of exposing any service, you will encounter a predictable set of attack patterns. Understanding these vectors is the first step toward effective mitigation.

Top attack vectors:

  • SSH brute-force attacks — Automated attempts using default credentials and common username/password combinations.
  • HTTP vulnerability scanners — Probes for /wp-admin, /phpMyAdmin, /cgi-bin/, and known CVE endpoints.
  • Path traversal attempts — Requests like `../../../../etc/passwd` to access sensitive files.
  • Historical exploits — Attack vectors dating back to 2001, including old Apache and IIS vulnerabilities.
  • DDoS reflection attacks — Honeypots can be used as amplifiers, filling storage with attack traffic.

Step‑by‑step guide to analyzing and mitigating attacks:

1. Identify the most frequent source IPs:

sudo grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -1r | head -20

2. Block malicious IPs at the firewall level:

sudo iptables -A INPUT -s <MALICIOUS_IP> -j DROP

Or use `fail2ban` for automated blocking:

sudo apt install fail2ban -y
sudo systemctl enable fail2ban && sudo systemctl start fail2ban

3. Implement rate limiting — Use `iptables` to limit connection rates:

sudo iptables -A INPUT -p tcp --dport 22 -m connlimit --connlimit-above 4 -j REJECT

4. Analyze HTTP attack patterns — Use T-Pot’s Kibana dashboard to filter HTTP requests by URI, user-agent, and status code.
5. Create threat intelligence feeds — Export attacker IPs and share them with threat intelligence platforms.

Windows PowerShell command to detect failed logins:

Get-EventLog -LogName Security -InstanceId 4625 | Select-Object -First 20
  1. Distinguishing Signal from Noise — Turning Background Radiation into Threat Intelligence

The fundamental challenge with internet background noise is not the volume of traffic, but its ambiguity. A legitimate internet-wide scanner can resemble a reconnaissance bot; a researcher probing systems may trigger the same alerts as an attacker mapping an environment. Without additional context, security systems must rely on heuristics, rules, and correlation logic to infer intent.

Step‑by‑step guide to converting noise into actionable intelligence:

  1. Enrich IP data — Use GreyNoise to classify IPs as “noise” (mass scanning) or “targeted”: 
    curl -s "https://api.greynoise.io/v3/community/<IP>" | jq '.classification'
  2. Correlate with known threat feeds — Integrate AlienVault OTX, AbuseIPDB, or MISP to cross-reference attacker IPs.
  3. Set up automated alerting — Configure T-Pot’s Elastic Stack to trigger alerts for specific attack patterns (e.g., repeated path traversal attempts, SQL injection probes).
  4. Perform periodic threat hunting — Review logs weekly to identify emerging patterns or new attack vectors.
  5. Share intelligence — Contribute to open-source threat intelligence platforms by reporting observed attack patterns.

Linux command to enrich IPs with GreyNoise in bulk:

while read ip; do curl -s "https://api.greynoise.io/v3/community/$ip" | jq '.classification'; done < suspicious_ips.txt

6. Hardening Your Cloud and API Security Posture

Exposing any service — whether for LLM inference, web development, or internal tools — requires a layered security approach. The following practices are essential for minimizing risk.

Best practices:

  • Zero-trust network access — Use Tailscale’s identity-aware, end-to-end encrypted security, with a firewall around every device.
  • API authentication — Never expose APIs without authentication. Use API keys, OAuth2, or mutual TLS.
  • Network segmentation — Place exposed services in isolated DMZ networks or separate VPCs.
  • Regular vulnerability scanning — Use tools like nmap, OpenVAS, or cloud-1ative scanners to identify weaknesses before attackers do.
  • Log aggregation and monitoring — Centralize logs using SIEM platforms (e.g., Elastic Stack, Splunk) and set up real-time alerting.

Step‑by‑step guide to API security hardening:

  1. Implement API gateway — Use a reverse proxy (NGINX, Traefik) to add authentication, rate limiting, and request validation.
  2. Enforce HTTPS — Ensure all traffic is encrypted. Tailscale Funnel provides automatic TLS, but verify certificate validity.
  3. Set up WAF rules — Use ModSecurity or cloud WAF services to block common attack patterns (SQL injection, XSS, path traversal).
  4. Conduct regular security audits — Use Tailsnitch for Tailscale configurations and cloud-1ative tools for infrastructure-as-code scanning.
  5. Implement incident response plan — Define clear procedures for detecting, containing, and recovering from security incidents.

Linux command to scan open ports with nmap:

nmap -sV -p- <YOUR_SERVER_IP>

Windows command to test TLS certificate validity:

Test-1etConnection -ComputerName <YOUR_DOMAIN> -Port 443

What Undercode Say:

  • Key Takeaway 1: The internet is inherently hostile. Exposing any service publicly triggers an immediate and automated response from scanners and botnets. This is not paranoia — it is the baseline condition of the internet.
  • Key Takeaway 2: Tailscale Funnel is a powerful tool for remote access but requires careful configuration. Always upgrade to the latest version, enforce authentication, and monitor logs continuously.
  • Key Takeaway 3: T-Pot provides an invaluable window into the threat landscape. Within hours, you can observe real-world attack patterns, from SSH brute-force to historical exploits, and use that intelligence to harden your defenses.
  • Key Takeaway 4: Distinguishing between internet background noise and targeted attacks is a critical security skill. Tools like GreyNoise and Tailsnitch help separate signal from noise, reducing alert fatigue and improving incident response.
  • Key Takeaway 5: Security is not a one-time setup — it requires continuous monitoring, regular updates, and proactive threat hunting. The attackers are always evolving; your defenses must evolve with them.

Prediction:

  • -1 As LLM inference becomes more accessible and remote, the number of exposed AI endpoints will grow exponentially, attracting more sophisticated attacks. Expect a rise in AI-specific exploits, including prompt injection, model poisoning, and side-channel attacks targeting speculative decoding.
  • -1 The cost of internet background noise will continue to burden security operations, with SIEM ingestion costs and analyst fatigue becoming critical business risks. Organizations will need to invest in noise-filtering technologies and threat intelligence to remain efficient.
  • +1 The democratization of threat intelligence through open-source tools like T-Pot and GreyNoise will empower smaller organizations to defend against large-scale attacks, leveling the playing field in cybersecurity.
  • +1 Tailscale’s continued investment in security features — guardrails, aperture, and Tailsnitch — signals a positive trend toward identity-aware, zero-trust networking that simplifies secure remote access for developers and enterprises alike.
  • -1 The prevalence of historical attack vectors (dating back to 2001) indicates that many organizations still fail to patch known vulnerabilities. Expect legacy exploits to remain a persistent threat for years to come.

▶️ Related Video (62% Match):

🎯Let’s Practice For Free:

🎓 Live Courses & Certifications:

Join Undercode Academy for Verified Certifications

🚀 Request a Custom Project:

Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands

IT/Security Reporter URL:

Reported By: Daniel Kloyber – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅

🔐JOIN OUR CYBER WORLD [ CVE News • HackMonitor • UndercodeNews ]

💬 Whatsapp | 💬 Telegram

📢 Follow UndercodeTesting & Stay Tuned:

𝕏 formerly Twitter 🐦 | @ Threads | 🔗 Linkedin | 🦋BlueSky

Related Posts: