Why Your OT SIEM Is Failing Before the First Alert (And How to Fix It) + Video

Listen to this Post

Featured Image

Introduction:

Operational Technology (OT) environments generate massive volumes of data from PLCs, HMIs, firewalls, and network traffic – but throwing everything into a SIEM creates an unusable noise factory. The difference between a functional OT SIEM and a broken one lies not in the tool, but in disciplined source selection and noise filtering before ingestion.

Learning Objectives:

  • Identify high-value, low-noise OT data sources (PLCs, RTUs, safety controllers) versus high-noise sources (workstations, HMIs)
  • Implement pre‑filtering and normalization techniques to reduce false positives in OT SIEM
  • Apply Linux and Windows commands to extract, filter, and forward only actionable OT telemetry

You Should Know:

  1. Start Small – Why “Collect Everything” Destroys OT SIEM

Most OT SIEM implementations fail because engineers ingest every log and packet without a strategy. The post’s core insight: raw network traffic from OT assets that lack native logs is useless unless filtered. Begin with structure‑rich sources – firewalls, VPN concentrators, jump hosts – because they produce clear security events with few devices.

Step‑by‑step guide to source triage:

  1. Inventory your OT network using `nmap -sS -p 502,44818,1911,2222 192.168.1.0/24` (common OT ports: Modbus 502, EtherNet/IP 44818, Siemens S7 102, DNP3 20000).
  2. Classify each source by noise level, value, and collection effort (PLC = high value/low noise; HMI = medium value/high noise).
  3. Forward only sources with a signal‑to‑noise ratio > 0.3 for the first phase.

Linux command to sample logs before full ingestion:

tail -n 1000 /var/log/ot/syslog | awk '$5 ~ /PLC|RTU/ {print $0}' | tee filtered_samples.log

This isolates lines containing PLC or RTU keywords – a simple noise filter before SIEM forwarding.

Windows PowerShell (for HMI/Historian logs):

Get-WinEvent -LogName "OT/HMI" -MaxEvents 500 | Where-Object {$_.Message -notmatch "heartbeat|diagnostic"} | Export-Csv -Path filtered_hmis.csv
  1. Normalize Before You Ingest – The “Bad Input, Bad Output” Rule

OT protocols (Modbus, DNP3, OPC UA) produce data in proprietary formats. Sending raw, unnormalized events to a SIEM or AI model guarantees garbage analytics. Normalization means converting timestamps to UTC, mapping vendor‑specific event IDs to a standard taxonomy (e.g., MITRE ATT&CK for ICS), and dropping known benign traffic.

Step‑by‑step using Logstash (open‑source):

  1. Install Logstash on a jump host: `sudo apt install logstash` (Debian/Ubuntu) or `winget install elastic.logstash` (Windows).

2. Create a configuration file `/etc/logstash/conf.d/ot_normalizer.conf`:

input {
syslog { port => 5514 }
}
filter {
grok {
match => { "message" => "%{SYSLOGTIMESTAMP:timestamp} %{GREEDYDATA:ot_source}: %{WORD:protocol} %{NUMBER:value}" }
}
if [bash] == "Modbus" and [bash] == "0" {
drop { }  Drop null Modbus reads (noise)
}
date { match => [ "timestamp", "MMM dd HH:mm:ss" ] }
}
output {
elasticsearch { hosts => ["http://your-siembox:9200"] }
}

3. Restart Logstash: `sudo systemctl restart logstash`

  1. Verify only normalized, filtered events reach the SIEM by tailing the output: `tail -f /var/log/logstash/logstash-plain.log`

    Windows alternative with nxlog: Configure `im_msvistalog` for HMI events, then apply `Exec if $EventID == 1000 drop;` to remove noise.

  2. PLCs and RTUs – Hard but Critical – Extracting Low‑Noise Logs

PLCs rarely generate syslog; you must poll them. The safest method is read‑only Modbus polling from a dedicated collector. This provides boolean and register changes – high value, extremely low noise.

Linux command using `mbpoll` (install via sudo apt install mbpoll):

mbpoll -m tcp -a 1 -t 3 -r 100 -c 10 192.168.1.100 > plc_registers.txt
 -a slave address, -t 3 = holding registers, -r start register, -c count

Pipe output to a parser that forwards only state changes:

while true; do 
current=$(mbpoll -m tcp -a 1 -t 3 -r 100 -c 1 192.168.1.100 -1 2>/dev/null | grep -oP '\d+')
if [ "$current" != "$last" ]; then 
echo "$(date) PLC register 100 changed from $last to $current" >> plc_changes.log
last=$current
fi
sleep 1
done

Send `plc_changes.log` to your SIEM – 1/1000th the volume of raw HMI logs, with 10× the detection value.

  1. Firewalls & Jump Hosts – The Ideal Starting Point

Firewalls (e.g., Palo Alto, Fortinet, open‑source pfSense) already structure security events: denies, allows, anomalies. Jump hosts consolidate access. These are your first‑week win.

Configuration for syslog forwarding (Cisco ASA / pfSense):

  1. On the firewall, enable syslog to your collector IP:

– Cisco: logging host 192.168.10.50, `logging trap informational`
– pfSense: Services > Syslog > Remote Syslog Servers > add collector
2. On the collector (Linux), run a minimal filter that drops “normal” traffic but keeps anomalies:

nc -l -u -p 514 | while read line; do
if echo "$line" | grep -qiE "deny|failure|malformed|unexpected"; then
echo "$line" >> /var/log/ot/firewall_anomalies.log
fi
done

3. Configure your SIEM to monitor only firewall_anomalies.log. Result: 99% noise reduction, instant detection of lateral movement attempts.

  1. Network Traffic – Don’t Send Raw PCAPs to SIEM

The post warns: “raw traffic without filtering is not useful.” Sending full packet captures to a SIEM kills storage and creates log blindness. Instead, use Zeek (formerly Bro) to extract session summaries and protocol anomalies.

Step‑by‑step for lightweight OT traffic analysis:

  1. Install Zeek on a span port or TAP: `sudo apt install zeek` (or `choco install zeek` on Windows with WSL).

2. Create a custom Zeek script `ot_filter.zeek`:

event modbus_read_holding_registers(c: connection, headers: ModbusHeaders, starting_address: count, quantity: count) {
if (quantity > 100) {  Abnormal read range – potential reconnaissance
local msg = fmt("Large Modbus read: addr=%d, qty=%d from %s", starting_address, quantity, c$id$orig_h);
Log::write(Notice::LOG, msg);
}
}
  1. Run Zeek on the OT interface: `zeek -i eth1 ot_filter.zeek`
    4. Zeek outputs `notice.log` – forward this one file (a few KB/hour) to your SIEM instead of gigabytes of PCAP.

  2. Training the AI – Why You Need a “Break It, Fix It, Repeat” Loop

AI/ML models for OT anomaly detection require curated, labeled data. The post’s Labshock approach – break the source, fix it, collect it – is the only way to build a robust classifier.

Tutorial: Simulate a PLC fault to train your SIEM AI

  1. Use `python` with `pymodbus` to write an anomalous value to a test PLC register:
from pymodbus.client import ModbusTcpClient
client = ModbusTcpClient('192.168.1.100')
client.write_register(400, 65535)  Invalid max value
  1. Capture the resulting log from your collector (the change event from Section 3).
  2. Label the event in your SIEM as “malicious write” (or “false alarm” if it’s normal).
  3. Retrain your model weekly using the “repeat” cycle – export labeled events:
curl -X POST "http://siem-api/labeled_events" -H "Content-Type: application/json" -d @labeled_batch.json

This closed loop turns an untuned SIEM into an OT‑specific threat hunter.

What Undercode Say:

  • Key Takeaway 1: OT SIEM success is 80% source selection and pre‑filtering, 20% tool capability. Start with firewalls and jump hosts, then add PLCs – never start with full network PCAP.
  • Key Takeaway 2: “Break it, fix it, collect it” is not a slogan; it’s a required workflow. Without simulating faults and anomalies, your AI model will only learn “normal” and fail on real attacks.

Analysis: The post dismantles the common myth that “more data equals better security.” In OT, false positives from HMIs and workstations overwhelm analysts, while silent PLC anomalies go unnoticed. The practical recommendation to normalize and filter before the SIEM – using lightweight tools like mbpoll, Zeek, and Logstash – aligns with SANS ICS concepts but adds a DevOps‑style iteration loop. Most vendors oversell AI as a magic filter; the reality is that garbage input produces dangerous confidence in false negatives. Undercode’s emphasis on “start small” and “repeat” is the only path to an OT SIEM that survives past month one.

Prediction:

    • OT security teams will shift from “collect all logs” to “pre‑filter at the edge” within 18 months, driven by SIEM cost overruns and alert fatigue.
    • Open‑source normalizers (e.g., Zeek + Logstash pipelines) will become standard in OT reference architectures, replacing proprietary connectors.
    • Vendors that continue to promote AI‑on‑everything without built‑in source triage will lose credibility as breaches caused by misconfigured SIEMs become public.
    • Training programs (like Labshock’s “break‑fix‑collect” method) will emerge as a new certification requirement for OT security engineers.

▶️ Related Video (78% Match):

🎯Let’s Practice For Free:

IT/Security Reporter URL:

Reported By: Zakharb Otsiem – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅

🎓 Live Courses & Certifications:

Join Undercode Academy for Verified Certifications

🚀 Request a Custom Project:

Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]

🔐JOIN OUR CYBER WORLD [ CVE News • HackMonitor • UndercodeNews ]

💬 Whatsapp | 💬 Telegram

📢 Follow UndercodeTesting & Stay Tuned:

𝕏 formerly Twitter 🐦 | @ Threads | 🔗 Linkedin | 🦋BlueSky