Listen to this Post
Introduction:
As large language models become mission-critical infrastructure for modern applications, observability is no longer optional—it’s a security and operational necessity. OpenRouter’s Broadcast feature and Input & Output Logging provide unprecedented visibility into every API request, enabling security teams to detect prompt injection attempts, track anomalous usage patterns, and audit AI interactions across hundreds of models. This article explores how to implement comprehensive prompt tracing, configure external observability platforms, and harden your LLM security posture.
Learning Objectives:
- Master OpenRouter’s Broadcast and Input & Output Logging configurations for complete request/response visibility
- Implement OpenTelemetry-based tracing with Grafana Cloud, SigNoz, LangSmith, and other observability platforms
- Apply security best practices including PII redaction, privacy mode, and audit logging for compliance
You Should Know:
1. Understanding OpenRouter’s Observability Architecture
OpenRouter provides a unified API gateway to hundreds of models from OpenAI, Anthropic, Google, and Meta. Every request passing through this gateway can be automatically traced without any SDK installation or code changes. The platform offers two complementary observability mechanisms:
Broadcast automatically generates OpenTelemetry traces for every API request and sends them to configured external destinations. Each trace includes comprehensive data: request/response messages, token usage (prompt, completion, total), cost information, timing/latency metrics, model slug and provider name, and tool usage details.
Input & Output Logging privately stores prompt and completion content within OpenRouter’s platform for debugging, model comparison, and prompt optimization. Data is encrypted at rest using AES-256 and stored in an isolated Google Cloud Storage project with separate access controls. Retention is a minimum of 3 months.
Privacy Mode can be enabled per destination to exclude prompt and completion content from traces while still sending token usage, costs, timing, model information, and custom metadata.
2. Step-by-Step: Enabling Broadcast and Configuring Destinations
Step 1: Enable Broadcast
Navigate to Settings > Observability in your OpenRouter dashboard and toggle the “Enable Broadcast” switch. For organization accounts, you must be an organization admin to edit broadcast settings.
Step 2: Add Destinations
Click “Add Destination” and select from supported platforms including Grafana Cloud, LangSmith, Langfuse, SigNoz, PostHog, Braintrust, Ramp, and Arize.
Step 3: Configure Destination-Specific Settings
For Grafana Cloud:
- Traces are sent via standard OTLP HTTP/JSON endpoint
- No additional instrumentation required in application code
For LangSmith:
- API Key (starts with
lsv2_pt_...) and Project name required - Traces appear with full details: input/output messages, token usage, cost, model/provider, timing
- Uses OTEL endpoint at `/otel/v1/traces`
For SigNoz:
- Configure OpenTelemetry Collector with endpoint `https://ingest.us.signoz.cloud:443/v1/traces`
- Headers: `{“signoz-ingestion-key”: “
“}`
– Test connection before saving
Step 4: Send Test Trace
Make an API request through OpenRouter and verify traces appear in your destination.
3. Enriching Traces with Custom Metadata and Hierarchies
OpenRouter’s `trace` field accepts arbitrary JSON objects passed through to all configured broadcast destinations. Known keys receive special handling:
{
"trace_id": "unique-trace-identifier",
"trace_name": "user-friendly-trace-1ame",
"span_name": "operation-1ame",
"generation_name": "llm-call-1ame",
"parent_span_id": "parent-trace-id"
}
Linking to External Traces:
Use `parent_span_id` to nest OpenRouter calls under existing spans:
– Track end-to-end workflows spanning multiple LLM calls
– Organize traces by business logic rather than individual API calls
– Build rich observability dashboards with meaningful trace names
– Integrate OpenRouter traces with existing application traces
User and Session Identification:
– `user` field: Associate traces with specific end-users (up to 128 characters)
– `session_id` field: Group related requests for conversation or agent workflow tracking (up to 128 characters)
4. Implementing Prompt Security and Compliance Controls
Private Input & Output Logging Configuration:
Navigate to Observability settings and toggle Input & Output Logging. Once enabled, full prompt and completion content becomes accessible from the Logs page.
Security Considerations:
- OpenRouter does not store prompts or responses unless explicitly opted in
- OpenRouter does not access or use logged data for model training or analytics
- Only organization admins can view stored prompt and response content
- EU routing limitation: Input & Output Logging does not apply to requests routed through `eu.openrouter.ai`
PII Redaction Implementation:
For compliance with GDPR, HIPAA, or other regulations, implement PII redaction before logging:
import re
import sqlite3
from datetime import datetime
def init_audit_db(db_path: str = "openrouter_audit.db"):
"""Create append-only audit table for compliance logging"""
conn = sqlite3.connect(db_path)
conn.execute("""
CREATE TABLE IF NOT EXISTS audit_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
timestamp TEXT,
trace_id TEXT,
user_id TEXT,
model TEXT,
prompt_tokens INTEGER,
completion_tokens INTEGER,
cost REAL,
prompt_hash TEXT,
status TEXT
)
""")
conn.commit()
return conn
def redact_pii(text: str) -> str:
"""Redact email addresses and phone numbers from prompts"""
text = re.sub(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+.[A-Z|a-z]{2,}\b', '[bash]', text)
text = re.sub(r'\b\d{3}[-.]?\d{3}[-.]?\d{4}\b', '[bash]', text)
return text
5. Cost Tracking and Multi-Model Optimization
OpenRouter’s Activity Explorer dashboard enables real-time tracking and analysis of multi-model API spending and cache efficiency. Critical metrics to monitor:
- Total cost across different models – Identify which models drive the highest expenses
- Average cost per user – Understand per-user economics
- Average API response time – Monitor performance degradation
- Model performance comparisons – Optimize model selection based on cost/quality tradeoffs
Cost Extraction from API Responses:
OpenRouter returns cost information in the `openrouter_metadata` field when enabled with the `X-OpenRouter-Metadata` header.
import requests
response = requests.post(
url="https://openrouter.ai/api/v1/chat/completions",
headers={
"Authorization": "Bearer <OPENROUTER_API_KEY>",
"HTTP-Referer": "<YOUR_SITE_URL>",
"X-": "<YOUR_SITE_NAME>",
"X-OpenRouter-Metadata": "true" Enable metadata in response
},
json={
"model": "openai/gpt-4o",
"messages": [{"role": "user", "content": "Analyze this prompt"}]
}
)
data = response.json()
Extract cost from openrouter_metadata
if 'openrouter_metadata' in data:
cost = data['openrouter_metadata'].get('total_cost', 0)
print(f"Request cost: ${cost:.6f}")
6. Hardening LLM Security with Observability
Prompt Injection Detection:
Configure alerts in your observability platform for anomalous patterns:
– Unusually long prompts (potential injection attempts)
– Rapid succession of requests from single user
– Requests targeting sensitive system prompts
– Unexpected model switching patterns
Audit Trail for Compliance:
Maintain append-only audit logs for all AI interactions:
Linux: Monitor OpenRouter API activity with jq
curl -s -X GET "https://openrouter.ai/api/v1/auth/key" \
-H "Authorization: Bearer $OPENROUTER_API_KEY" | jq '.'
Windows PowerShell: Track API usage
$headers = @{ "Authorization" = "Bearer $env:OPENROUTER_API_KEY" }
Invoke-RestMethod -Uri "https://openrouter.ai/api/v1/auth/key" -Headers $headers | ConvertTo-Json
Rate Limiting and Abuse Prevention:
- Implement per-user rate limits using the `user` field for identification
- Monitor for abuse reports which can affect entire applications if individual users are not identified
- Use session_id for sticky routing to maximize prompt cache hits and reduce costs
What Undercode Say:
- OpenRouter’s Broadcast feature eliminates the traditional trade-off between observability and performance—traces are generated automatically with zero additional latency
- The ability to nest OpenRouter calls within existing application traces via `parent_span_id` enables true end-to-end visibility across complex AI workflows
- Privacy-first design (opt-in logging, encryption at rest, separate access controls) makes OpenRouter suitable for regulated industries handling sensitive data
- The platform’s unified API with built-in observability reduces the operational burden of managing multiple provider integrations while maintaining security visibility
- Organizations should implement both Broadcast (external) and Input & Output Logging (internal) for comprehensive observability coverage
- Prompt injection detection requires correlating observability data with security rules—not just logging but active alerting
- Cost tracking across models enables both financial optimization and security anomaly detection (unexpected cost spikes indicate potential abuse)
- The OpenTelemetry-based approach ensures compatibility with existing observability infrastructure
- Session and user identification are critical for both debugging and security incident response
- Privacy Mode provides a middle ground for organizations that need operational visibility without exposing sensitive prompt content
Prediction:
- +1 LLM observability will become a mandatory compliance requirement similar to application logging for GDPR and SOC2 within 18-24 months
- +1 OpenRouter’s Broadcast model will be adopted as the industry standard for AI gateway observability, reducing the need for per-provider instrumentation
- -1 Organizations failing to implement prompt tracing will face increased security incidents from prompt injection and data exfiltration attacks
- +1 Integration of observability with automated security response (e.g., automatically blocking suspicious prompts) will emerge as the next frontier
- +1 Cost optimization through observability data will drive significant operational savings as organizations gain visibility into model-specific spending patterns
▶️ Related Video (80% Match):
🎯Let’s Practice For Free:
🎓 Live Courses & Certifications:
Join Undercode Academy for Verified Certifications
🚀 Request a Custom Project:
Secure, high-velocity infrastructure and disruptive technological engineering. Contact our engineering team for high-tier development and proprietary systems:
[email protected]
💎 Smart Architecture | 🛡️ Secure by Design | ⭐ Trusted by Thousands
IT/Security Reporter URL:
Reported By: Osintech Tracing – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
