Listen to this Post
Introduction:
Security logs are often seen as a chaotic stream of timestamps, IP addresses, and error codes. But experienced SOC analysts know that logs are not the final answer—they are pieces of evidence that must be assembled into a coherent story of system behavior. This article transforms the beginner’s approach from “reading every line” to “investigating patterns,” providing hands-on techniques to identify anomalies, reduce noise, and think like a Level 1 analyst.
Learning Objectives:
- Apply the “5W1H” framework (What, When, Where, Who, How) to any security log entry for rapid context extraction.
- Use native Linux and Windows command-line tools to filter, correlate, and timeline security events.
- Distinguish benign activity from malicious behavior by recognizing five common attack patterns across authentication, network, and process logs.
You Should Know:
1. Understanding Log Types and Their Sources
Before you can investigate, you must know where evidence lives. Security logs come from endpoints (Windows Event Logs, syslog), network devices (firewalls, proxies), and applications (web servers, databases).
Step‑by‑step guide to locate and view logs:
Linux – System and Authentication Logs
View authentication log (failed/successful logins) sudo tail -f /var/log/auth.log Debian/Ubuntu sudo tail -f /var/log/secure RHEL/CentOS View system messages and kernel ring buffer journalctl -f Real-time systemd journal dmesg | grep -i error Kernel messages with errors
Windows – Event Logs via Command Line
List all available event logs
wevtutil el
Export security log to CSV for analysis
wevtutil qe Security /f:text /c:50
PowerShell: Get last 20 failed logon events (Event ID 4625)
Get-WinEvent -LogName Security | Where-Object { $_.Id -eq 4625 } | Select-Object -First 20
Understanding the fields: Every log entry has a timestamp, source (host/IP), event ID, level (info/warning/error), user/process, and message. Your first task is to normalize these fields across different sources—SIEM tools do this automatically, but manual analysis requires grep, cut, and awk.
- The 5W1H Framework – From Noise to Narrative
Instead of reading line‑by‑line, ask these six questions. Each answer narrows your investigation.
Step‑by‑step guide with filtering commands:
What happened? Identify the event ID or message.
- Linux: `grep “Failed password” /var/log/auth.log`
– Windows: `Get-WinEvent -FilterHashtable @{LogName=’Security’; ID=4625}`When did it happen? Establish a time window (attackers often strike off‑hours).
Linux: Show auth logs between 2 AM and 4 AM yesterday journalctl --since "yesterday 02:00:00" --until "yesterday 04:00:00" -u sshd
Windows: Events from last 2 hours $start = (Get-Date).AddHours(-2); Get-WinEvent -LogName Security | Where-Object { $_.TimeCreated -ge $start }
Where did it come from? Extract source IP addresses and geolocate.
Extract unique source IPs from failed SSH attempts
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -nr
Who triggered it? Username, service account, or process ID.
List all usernames used in failed attempts
grep "Failed password" /var/log/auth.log | awk '{print $9}' | sort | uniq -c
Is this normal? Compare against baseline. Use standard deviation or simple thresholds.
Quick Python baseline: if failed attempts > mean + 2std, flag as anomaly
import statistics
failures_per_hour = [5,6,4,45,7,5,50] example hourly counts
mean = statistics.mean(failures_per_hour)
stdev = statistics.stdev(failures_per_hour)
print(f"Anomaly threshold: {mean + 2stdev:.1f}")
3. Recognizing Attack Patterns – Five Red Flags
SOC analysts don’t memorize every event ID; they spot clusters of behavior.
Pattern 1: Repeated failed logins (Brute force)
Count failures per IP, show those with >10 attempts
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | awk '$1 > 10'
Pattern 2: Unusual geographic locations – Use `whois` or `geoiplookup` (Linux)
for ip in $(grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort -u); do
geoiplookup $ip | grep -v "IP not found"
done
Pattern 3: Spikes in outbound traffic (Data exfiltration) – Monitor netstat connections
Linux: Show established connections per destination port
netstat -tun | awk '{print $4}' | cut -d: -f2 | sort | uniq -c
Windows: Show outbound connections to suspicious ports (e.g., 4444 for C2)
Get-NetTCPConnection -State Established | Where-Object { $_.RemotePort -eq 4444 }
Pattern 4: Unknown processes – Compare running processes against known good list.
List processes with no digital signature or uncommon names ps aux --sort=-%cpu | head -20 Check hash against VirusTotal (requires API key) sha256sum /path/to/suspicious_binary
Pattern 5: Suspicious outbound DNS queries – Log DNS requests and filter for known TLDs.
Using tcpdump on DNS port (53) sudo tcpdump -i eth0 -n port 53 | grep -E ".top|.xyz|.tk"
4. Building Timelines – The Analyst’s Superpower
A timeline connects isolated events into an attack chain. Use `awk` and `sort` to reorganize logs.
Step‑by‑step timeline construction:
- Collect all relevant logs (auth, syslog, firewall, app) into a single directory.
2. Normalize timestamps to ISO 8601 (`date -d`).
3. Sort by timestamp and merge.
Merge all logs, extract timestamp and message, sort chronologically
cat /var/log/auth.log /var/log/syslog /var/log/nginx/access.log | \
awk '{print $1, $2, $3, $0}' | sort -k1,1M -k2,2n -k3,3n
Using timeline tools: Zeek (formerly Bro), `lnav` (log navigator), or Splunk’s timechart.
Windows PowerShell timeline example:
Get logon, logoff, and process creation events from last 24h
$events = Get-WinEvent -FilterHashtable @{LogName='Security'; StartTime=(Get-Date).AddDays(-1); ID=4624,4634,4688}
$events | Sort-Object TimeCreated | Format-Table TimeCreated, Id, Message -AutoSize
5. Reducing Noise – Filtering with Confidence
Noise is the enemy of detection. Use negative filters and baseline whitelists.
Linux – Ignore normal cron, systemd, and health checks
grep -v -E "CRON|systemd|session opened" /var/log/syslog | more
Windows – Exclude known benign processes (svchost, lsass)
Get-WinEvent -LogName "Microsoft-Windows-Sysmon/Operational" | Where-Object {
$<em>.Id -eq 1 -and $</em>.Message -notmatch "svchost.exe|lsass.exe"
}
Using sysmon configuration – Deploy a sysmon config that suppresses known low‑noise events (Event ID 7 for module loads, etc.). Example filter:
<ProcessCreate onmatch="exclude"> <Image condition="is">C:\Windows\System32\svchost.exe</Image> </ProcessCreate>
Advanced noise reduction with Elasticsearch (Lucene syntax):
NOT (event.type:"INFO" AND process.name:"cron") AND NOT (winlog.event_id:4688 AND user.name:"SYSTEM")
6. Practical Lab: Investigating a Brute-Force Attack
Step‑by‑step guide you can run in a lab environment:
Scenario: Multiple failed SSH logins from a single IP, then a successful login, followed by outbound `curl` to a known bad domain.
Step 1 – Identify brute force source
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -nr
Output: 117 203.0.113.45
Step 2 – Check if that IP ever succeeded
grep "Accepted password" /var/log/auth.log | grep "203.0.113.45" If found, attacker gained access.
Step 3 – Correlate with process history (using auditd or osquery)
Find all commands run by the compromised user after that login time ausearch -ts 11:30:00 -te 12:00:00 -m execve | grep -i "curl|wget|nc"
Step 4 – Block and remediate
sudo iptables -A INPUT -s 203.0.113.45 -j DROP Force password reset for compromised user sudo passwd -e victimuser
Windows equivalent (PowerShell):
Block IP via Windows Firewall
New-NetFirewallRule -DisplayName "Block BruteForce IP" -Direction Inbound -RemoteAddress 203.0.113.45 -Action Block
Query security log for successful logins after failures
Get-WinEvent -FilterHashtable @{LogName='Security'; ID=4624} | Where-Object {$_.Message -like "203.0.113.45"}
7. AI-Assisted Log Analysis – The Next Frontier
Modern SOCs use machine learning to baseline behavior. Open‑source options include:
- Elasticsearch with ML: Detects rate anomalies (e.g., unusual login frequency per user).
- Splunk’s Built‑in ML Toolkit: `| outlier detection` on time‑series data.
- Sigma rules (converted to AI models): Detect attacker TTPs without hard‑coded signatures.
Example ML logic (pseudo‑command):
Using ELK’s ML job from CLI
curl -X PUT "localhost:9200/_ml/anomaly_detectors/ssh_bruteforce" -H 'Content-Type: application/json' -d'
{
"analysis_config": { "bucket_span": "10m", "detectors": [{"detector_description": "high_failed_logins", "function": "high_non_zero_count", "field_name": "source.ip"}] }
}'
What Undercode Say:
- Logs are evidence, not instructions. Always start with a hypothesis (e.g., “Is this IP part of a campaign?”) before touching a single line.
- Automate pattern recognition. Human analysts should spend 80% of their time on investigation, not manual grepping. Build reusable scripts for the five red flags listed above.
- Context is king. A failed login at 3 AM from a foreign IP is noise if it’s your CEO using a VPN; it’s an incident if it’s a service account with no travel history. Enrich every log with asset criticality and user behavior baselines.
Prediction:
Within two years, entry‑level SOC roles will require proficiency in log analysis assisted by generative AI—not to replace analysts, but to auto‑correlate timelines and suggest attack chains. The analysts who thrive will be those who master the “why” behind the log, leaving the “what” to machines. As SIEM vendors embed LLMs directly into dashboards, expect natural language queries like “Show me all failed logins from last night that came from non‑corporate IPs” to generate complete investigation reports in seconds. However, adversarial attackers will adapt by poisoning log sources with misleading entries, making the human ability to spot “what doesn’t belong” more valuable than ever.
▶️ Related Video (78% Match):
🎯Let’s Practice For Free:
IT/Security Reporter URL:
Reported By: Firdevs Balaban – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
