Listen to this Post
Introduction
In cybersecurity, detection is only as good as the data feeding it. Endpoint Detection and Response (EDR) solutions promise comprehensive threat visibility, but the reality varies dramatically across vendors — and until recently, there was no objective, vendor-1eutral way to compare what telemetry each product actually exposes. The EDR Telemetry project changes that by aggregating thousands of change events across dozens of commits and pull requests, tracking which vendors improve their telemetry coverage over time and which stagnate. With 27 vendors now in the summary set and a CI-driven pipeline feeding statistics forward, this initiative provides security practitioners with data-driven insights into which EDR vendors genuinely invest in telemetry visibility.
Learning Objectives
- Understand the EDR Telemetry project’s scoring methodology and how telemetry visibility is quantified
- Learn to interpret vendor ranking leaderboards, category deltas, and contribution breakdowns
- Gain practical skills for validating EDR telemetry claims using open-source tools and manual verification techniques
You Should Know
1. How EDR Telemetry Statistics Are Calculated
The EDR Telemetry project doesn’t rely on vendor self-reported capabilities or marketing brochures. Instead, it measures actual changes to the vendor matrix — a comprehensive mapping of what telemetry each EDR product exposes across dozens of categories. The methodology is precise and transparent:
- Data Source: All statistics derive from accepted changes in the EDR-Telemetry data/generated dataset on GitHub. The site does not recompute history; it visualizes published summaries from the repository.
-
Scoring Mechanism: Each accepted update to a vendor’s telemetry status is compared against the previous repository state, scored using the project’s published status model, and weighted by the relevant telemetry feature weight. Features with higher investigative value — such as process creation, file modification, and network connections — receive greater weight in the scoring system.
-
Ranking Logic: Leaderboards rank vendors by net weighted score delta, not by pull request count or activity volume. This distinction is critical: a vendor submitting dozens of minor corrections won’t outrank one that delivers meaningful telemetry expansions. Baselines and zero-delta references are excluded from improvement charts, while corrections and downgrades appear as negative deltas where applicable.
-
What It Does Not Measure: The statistics describe changes in user-visible telemetry coverage only. They do not measure product quality, prevention capability, detection quality, alert quality, or managed service performance. A higher score or positive delta means broader exposed telemetry, not a “better” product. New vendor baselines, new categories, renames, documentation-only changes, and corrections are labeled separately so they do not inflate improvement metrics.
Hands-On: Exploring the EDR Telemetry Repository
To understand the data firsthand, clone the repository and explore the telemetry matrix:
Clone the EDR Telemetry repository git clone https://github.com/tsale/EDR-Telemetry.git cd EDR-Telemetry Explore the data structure ls -la data/generated/ View the main telemetry JSON file (if present) cat data/generated/EDR_telem.json | jq '.' | head -100 Check commit history for telemetry changes git log --oneline --since="2024-01-01" -- data/generated/
On Windows (PowerShell):
Clone the repository git clone https://github.com/tsale/EDR-Telemetry.git cd EDR-Telemetry View directory structure Get-ChildItem -Recurse data/generated/ Check commit history git log --oneline --since="2024-01-01" -- data/generated/
2. Interpreting the Vendor Leaderboard and Category Deltas
The statistics page presents several visualizations:
- Vendor Improvement Leaderboard: Ranks vendors by net weighted score delta over the tracking period. This is the primary metric for identifying which vendors are actively expanding their telemetry footprint.
-
Score Trends: Shows how individual vendors’ scores have evolved over time, revealing patterns of consistent investment versus sporadic updates.
-
Category Deltas: Breaks down changes by telemetry category (e.g., process creation, file system, registry, network, memory, privilege escalation). This granularity helps security teams understand where each vendor excels or lags.
-
Contribution Breakdown: Attributes changes to specific commits and pull requests, providing transparency into who contributed what and when.
The CI pipeline on the GitHub repository automatically processes accepted matrix changes, so the statistics stay current without manual intervention.
Practical Exercise: Tracking a Vendor’s Telemetry Evolution
To track a specific vendor’s changes over time:
Find all commits affecting a specific vendor (e.g., CrowdStrike) git log --grep="CrowdStrike" --oneline -- data/generated/ View detailed changes for a specific commit git show <commit-hash> Compare two versions of the telemetry data git diff <old-commit> <new-commit> -- data/generated/EDR_telem.json
3. Validating EDR Telemetry Claims: The Sysmon Baseline
The EDR Telemetry project uses Sysmon as a baseline reference for comprehensive endpoint telemetry. Sysmon (System Monitor) is a Windows Sysinternals tool that logs detailed system activity to the Windows event log, providing visibility into process creation, network connections, file creation, and more.
Installing and Configuring Sysmon for Telemetry Baseline
Download Sysmon from Microsoft Sysinternals Invoke-WebRequest -Uri "https://live.sysinternals.com/Sysmon64.exe" -OutFile "C:\Tools\Sysmon64.exe" Install Sysmon with a basic configuration C:\Tools\Sysmon64.exe -accepteula -i Use a comprehensive configuration (SwiftOnSecurity's sysmon-config) git clone https://github.com/SwiftOnSecurity/sysmon-config.git C:\Tools\Sysmon64.exe -accepteula -c .\sysmon-config\sysmonconfig.xml
Verifying Sysmon Telemetry Collection
Check if Sysmon is running Get-Process -1ame Sysmon View Sysmon events in Event Viewer Open Event Viewer → Applications and Services Logs → Microsoft → Windows → Sysmon → Operational Query Sysmon events using PowerShell Get-WinEvent -LogName "Microsoft-Windows-Sysmon/Operational" -MaxEvents 50 | Format-Table TimeCreated, Id, Message -AutoSize
On Linux, auditd provides similar telemetry capabilities:
Install auditd sudo apt-get install auditd audispd-plugins -y Monitor critical system calls sudo auditctl -a always,exit -S execve -S openat -S connect -k process_monitoring View audit logs sudo ausearch -k process_monitoring --format raw | tail -50
4. Contributing to the EDR Telemetry Project
The project is community-driven and welcomes contributions. Accepted changes feed into the statistics automatically via the CI pipeline.
How to Contribute
1. Fork the repository on GitHub.
- Update the telemetry matrix for one or more vendors based on verified evidence.
- Submit a pull request with clear documentation of the changes.
Evidence Standards: Changes must be backed by verifiable evidence — vendor documentation, product testing, or official communications. The project maintains a methodology page detailing the evidence standard for status values.
Workflow for Submitting a Change
Fork and clone your fork git clone https://github.com/your-username/EDR-Telemetry.git cd EDR-Telemetry Create a feature branch git checkout -b update-vendor-telemetry Edit the telemetry data (JSON format) Update the relevant vendor entry in data/generated/EDR_telem.json Validate JSON structure cat data/generated/EDR_telem.json | jq '.' > /dev/null && echo "Valid JSON" Commit and push git add data/generated/EDR_telem.json git commit -m "Update telemetry for VendorX: added process injection coverage" git push origin update-vendor-telemetry Then open a pull request on GitHub
- Using EDR Telemetry Data for Vendor Selection and Security Architecture
For security architects and SOC leads, the EDR Telemetry statistics provide objective data for vendor selection and capability assessment. When evaluating EDR solutions, consider:
- Telemetry Breadth: Does the vendor expose the specific telemetry categories your threat hunting and detection engineering teams rely on? Process creation, file modification, and network connections are typically highest-value.
-
Telemetry Depth: Beyond “yes/no” availability, assess the granularity of the data exposed. Does the vendor provide command-line arguments, parent-child process relationships, and full file paths?
-
Trend Over Time: A vendor with a positive delta over three years demonstrates commitment to improving visibility. Stagnant scores may indicate a product reaching maturity — or neglect.
-
Correlation with MITRE ATT&CK: The project maps telemetry categories to MITRE ATT&CK techniques, enabling you to assess coverage for specific adversary behaviors.
Practical: Mapping Telemetry to Detection Use Cases
Example Python script to map telemetry categories to MITRE ATT&CK techniques
import json
Load EDR telemetry data
with open('data/generated/EDR_telem.json', 'r') as f:
telemetry_data = json.load(f)
Map categories to detection use cases
detection_map = {
'process_creation': ['T1059', 'T1204', 'T1055'],
'file_modification': ['T1070', 'T1565'],
'network_connection': ['T1071', 'T1572'],
'registry_modification': ['T1112', 'T1547']
}
Evaluate a vendor's coverage for specific techniques
def evaluate_technique_coverage(vendor_data, technique_ids):
coverage = {}
for technique in technique_ids:
coverage[bash] = any(
category in vendor_data.get('telemetry', {})
for category, techs in detection_map.items()
if technique in techs
)
return coverage
- Linux and Windows Commands for EDR Telemetry Validation
Security practitioners can validate EDR telemetry claims using open-source tools and manual testing.
Linux: Generating Telemetry for Validation
Simulate process creation (should be logged by EDR) /bin/echo "Test process" Simulate file modification echo "test" > /tmp/testfile.txt Simulate network connection curl -s https://www.google.com > /dev/null Simulate privilege escalation attempt (requires sudo) sudo -l Check audit logs for these events sudo ausearch -ts recent -m execve -m file -m connect
Windows: Generating Telemetry for Validation
Simulate process creation
Start-Process -FilePath "cmd.exe" -ArgumentList "/c echo Test"
Simulate file creation and modification
New-Item -Path "C:\Temp\testfile.txt" -ItemType File
Set-Content -Path "C:\Temp\testfile.txt" -Value "Test content"
Simulate registry modification
New-Item -Path "HKLM:\SOFTWARE\TestKey" -Force
Simulate network connection
Invoke-WebRequest -Uri "https://www.google.com" -UseBasicParsing
Check Sysmon events for these activities
Get-WinEvent -LogName "Microsoft-Windows-Sysmon/Operational" -MaxEvents 100 |
Where-Object { $_.Message -match "process|file|registry|network" }
Using osquery for Cross-Platform Telemetry Validation
Install osquery (Linux) sudo apt-get install osquery -y Run osquery to query system state osqueryi --json "SELECT FROM processes WHERE name = 'bash' LIMIT 5" osqueryi --json "SELECT FROM file_events WHERE target_path LIKE '%test%'" osqueryi --json "SELECT FROM socket_events WHERE remote_address != '' LIMIT 5"
- Security Implications: What Telemetry Visibility Means for Defense
The EDR Telemetry project reveals a crucial truth: not all EDRs are created equal when it comes to visibility. A vendor with broader telemetry exposure enables:
- Earlier Threat Detection: More data points mean more opportunities to identify malicious activity before it escalates.
-
Richer Incident Investigations: Comprehensive telemetry allows analysts to reconstruct attack timelines with greater fidelity.
-
Better Threat Hunting: Proactive hunting relies on the availability of high-fidelity data across multiple dimensions.
-
Improved Detection Engineering: Security teams can write more precise detection rules when the underlying telemetry is comprehensive and consistent.
However, broader telemetry also introduces tradeoffs: increased data volume, storage costs, and potential privacy considerations. The project’s methodology explicitly distinguishes between “broader exposed telemetry” and “better product” precisely because these tradeoffs exist.
What Undercode Say
- Objective benchmarking matters: The EDR Telemetry project demonstrates that vendor-1eutral, data-driven comparisons are not only possible but essential for informed security decisions. By tracking actual changes over time, it cuts through marketing hype and reveals which vendors genuinely invest in telemetry visibility.
-
Transparency drives improvement: When vendors know their telemetry coverage is being measured and compared publicly, they have a tangible incentive to expand capabilities. The project’s CI-driven approach ensures that improvements are captured and reflected in real-time.
The project’s methodology — scoring weighted by investigative value, ranking by net delta rather than activity volume, and excluding baselines and corrections from improvement metrics — provides a rigorous framework that other security benchmarking initiatives could emulate. For security practitioners, this data is invaluable for vendor selection, capability assessment, and holding vendors accountable to their telemetry claims. The integration with MITRE ATT&CK further enhances its utility for detection engineering and threat hunting teams. As the project grows and more vendors are added, it will become an increasingly authoritative reference for the industry.
Prediction
- +1 The EDR Telemetry project will become an industry-standard reference for EDR capability assessment, similar to how MITRE ATT&CK evaluations influence vendor positioning. Security teams will increasingly require vendors to demonstrate telemetry coverage against the project’s matrix as part of procurement processes.
-
+1 Vendors with consistently positive deltas will gain market share as organizations prioritize visibility over other features. The project’s data will drive competitive pressure, accelerating telemetry improvements across the industry.
-
-1 Vendors with stagnant or negative deltas may face increased scrutiny from customers and analysts, potentially leading to reputational damage and lost deals. Some vendors may attempt to game the system by submitting superficial changes that don’t meaningfully improve visibility.
-
+1 The project’s open-source, community-driven model will expand to include additional telemetry categories, cloud workloads, and perhaps even XDR platforms, creating a comprehensive visibility benchmark for the entire security ecosystem.
▶️ Related Video (80% Match):
https://www.youtube.com/watch?v=55GaIolVVqI
🎯Let’s Practice For Free:
🎓 Live Courses & Certifications:
Join Undercode Academy for Verified Certifications
🚀 Request a Custom Project:
Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands
IT/Security Reporter URL:
Reported By: Kostastsale Edr – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
