Listen to this Post
Introduction
The introduction of Agent 365 marks a new era for Microsoft 365 Copilot, but with great power comes great risk. As AI agents become integral to daily operations, they also become prime targets for sophisticated threats like prompt injection and jailbreak attempts, which can lead to data extraction or tool manipulation. To combat these emerging dangers, security professionals must leverage Kusto Query Language (KQL) within Microsoft Defender XDR to proactively hunt for malicious activities targeting these AI systems.
Learning Objectives
- Master the use of KQL to query Microsoft Defender XDR for AI-specific threat detections and jailbreak attempts.
- Understand the MITRE ATT&CK for ML (AML) framework as it applies to threats against AI agents.
- Learn to implement and customize advanced hunting queries to monitor Agent 365 and Copilot interactions for suspicious behavior.
You Should Know
- Advanced Threat Hunting for AI Agents with KQL
Kusto Query Language (KQL) is the backbone of threat hunting in Microsoft’s security ecosystem, enabling analysts to sift through massive datasets to find anomalies. Advanced hunting, a query-based threat-hunting tool in Microsoft 365 Defender, allows you to proactively inspect raw data from endpoints, identities, email, and cloud apps. For AI threats, you will specifically query the `CopilotActivity` table.
Step‑by‑Step Guide to Using KQL for AI Threat Detection
1. Navigate to Advanced Hunting: Log into the Microsoft 365 Defender portal and go to Hunting > Advanced hunting.
2. Start with a Basic Query: In the query editor, type a simple KQL statement to explore the `CopilotActivity` table. For instance:
CopilotActivity | take 10
This command returns 10 random rows from the table, giving you a snapshot of the data schema.
3. Analyze AI Threat Detection Events: Use the following KQL query to retrieve events blocked by Microsoft Defender for AI. This query, developed by Alex Verboon, parses JSON data to extract detection names for blocked actions like prompt injection or secret leakage.
CopilotActivity | extend Parsed = parse_json(LLMEventData) | mv-expand Resource = Parsed.AccessedResources | extend Action = tostring(Resource.Action) | extend Id = tostring(Resource.id) | extend Name = tostring(Resource.Name) | extend Type = tostring(Resource.Type) | where Name == "Block" | extend DetectionName = extract(@"blocked by <a href="[^'""]+">'""</a>['""] detection", 1, Action) | project TimeGenerated, DetectionName, Action, Id, Name, Type, SrcIpAddr, Workload, AppHost, AppIdentity, LLMEventData | sort by TimeGenerated
4. Run and Analyze: Execute the query. The results will show you the `DetectionName` (e.g., “blocked by prompt injection detection”), the TimeGenerated, and other contextual details about the blocked interaction.
2. Hunting for Copilot Jailbreak Attempts
Jailbreak attempts are a common attack vector against LLMs, where malicious prompts try to bypass the model’s safety guardrails. Microsoft Defender for AI can detect and block these attempts, and you can find them by querying the `CopilotActivity` table for specific `RecordType` and `JailbreakDetected` indicators.
Step‑by‑Step Guide to Detecting Jailbreak Attempts
- Initiate a Query for Jailbreaks: In the Advanced hunting query editor, use the following KQL to find any Copilot interaction where a jailbreak was detected.
CopilotActivity | where RecordType == "CopilotInteraction" | extend LLMData = parse_json(LLMEventData) | mv-expand Message = LLMData.Messages | extend JailbreakDetected = tobool(Message.JailbreakDetected) | where JailbreakDetected == true | project TimeGenerated, ActorName, AppHost, AIModelName, MessageId = tostring(Message.Id), IsPrompt = tobool(Message.isPrompt) | order by TimeGenerated desc
- Review the Output: This query filters for successful jailbreak detections and projects key fields such as the user (
ActorName), the application host (AppHost), and the specific model (AIModelName) that was targeted. - Use an Alternative Detection Method: Another method from Alex Verboon identifies jailbreaks by looking for a resource named “JailBreak”.
CopilotActivity | extend Parsed = parse_json(LLMEventData) | mv-expand Resource = Parsed.AccessedResources | extend Action = tostring(parse_json(Resource.Action)) | extend Id = tostring(parse_json(Resource.id)) | extend Name = tostring(parse_json(Resource.Name)) | extend Type = tostring(parse_json(Resource.Type)) | project TimeGenerated, Action, Id, Name, Type, SrcIpAddr, Workload, AppHost, AppIdentity | where Name == @"JailBreak"
This approach can catch different variations of jailbreak events logged by Defender.
3. Configuring Agent 365 for Security Hardening
Agent 365 is Microsoft’s centralized control plane for managing and securing AI agents across a tenant. It provides IT administrators with the tools to define governance boundaries, much like they do for users and devices. Hardening Agent 365 is crucial to prevent the misuse of AI agents and to protect sensitive data.
Step‑by‑Step Guide to Configure Agent 365 Policies
- Access Agent Settings: Go to the Microsoft 365 admin center and navigate to the Agent settings page. This is the primary interface for agent governance.
- Define Agent Types: Under the “Allowed agent types” setting, specify which kinds of agents can be created or used in your tenant. Restrict this to only approved, secure agent types to minimize the attack surface.
- Set Sharing Permissions: Use the “Sharing permissions” control to determine how agents can be shared between users. Limit sharing to within specific security groups to prevent unauthorized access.
- Configure User Access: Define which users or groups are allowed to interact with AI agents. This is the outermost governance layer, ensuring that only authorized personnel can use agentic AI capabilities.
- Enforce DLP Policies: Integrate Agent 365 with Microsoft Purview Data Loss Prevention (DLP). Configure DLP policies for the “Microsoft 365 Copilot” location to prevent agents from processing or surfacing sensitive content, such as credit card numbers or classified project names.
-
Integrating KQL with Linux and Windows for Cross-Platform Threat Hunting
While KQL is used in the cloud, local threat hunting on endpoints often requires a combination of PowerShell (Windows) and Bash (Linux) commands. KQL can, however, be used to query syslog data from Linux machines that forward logs to a central repository like Microsoft Sentinel. Furthermore, tools like Microsoft’s KqlTools allow you to process real-time event streams on both Windows and Linux systems using KQL queries.
Step‑by‑Step: Hunting for a Service Stop on a Linux VM using KQL
1. Identify the Data Source: If your Linux VMs forward syslog to a Log Analytics workspace, you can query this data in Microsoft Sentinel.
2. Write the KQL Query: Use the following KQL to find when the `sshd` service was stopped. This query looks for a syslog message indicating the service entered a stopped state.
Syslog | where Facility == "auth" or Facility == "daemon" | where ProcessName == "systemd" | where SyslogMessage contains "Stopped OpenSSH server daemon" | project TimeGenerated, Computer, ProcessName, SyslogMessage
This query filters for `systemd` messages that mention the `sshd` service being stopped, providing valuable context for a potential threat.
5. Understanding the MITRE ATT&CK for AI Framework
To effectively hunt for AI threats, it’s essential to understand the tactics and techniques used by adversaries. The MITRE ATT&CK for ML (AML) framework provides a comprehensive taxonomy. Alex Verboon’s queries map directly to techniques such as AML.T0051 (Prompt Injection) and AML.T0054 (Tool Manipulation).
Step‑by‑Step: Creating a Custom Alert for AML.T0054 (Tool Manipulation)
1. Create a Custom Detection Rule: In the Microsoft 365 Defender portal, go to Hunting > Custom detection rules and create a new rule.
2. Write a KQL Query: Input a KQL query that looks for tool manipulation attempts.
CopilotActivity
| where RecordType == "CopilotInteraction"
| extend LLMData = parse_json(LLMEventData)
| mv-expand ToolCall = LLMData.ToolCalls
| extend ToolName = tostring(ToolCall.Name)
| where ToolName in ("ExecuteScript", "DeleteFile", "ModifyRegistry")
| project TimeGenerated, ActorName, ToolName, ToolCall.InputParameters
This query looks for attempts to use high-risk tools like `ExecuteScript` or DeleteFile.
3. Configure Alert Settings: Set the rule to run on a schedule (e.g., every hour). Configure it to generate an alert with medium severity if a match is found, and assign it to a specific response team.
4. Automate Response: Integrate the alert with Logic Apps to automatically trigger a response, such as isolating the user’s device or revoking their session.
What Undercode Say:
- Key Takeaway 1: Microsoft Defender for AI provides robust, built-in protection against emerging threats like prompt injection and jailbreak attempts, but this security must be actively monitored and operationalized through KQL hunting and custom detections.
- Key Takeaway 2: Effective security for AI systems like Agent 365 is not just about using native protections; it requires a layered approach that includes centralized governance (via Agent 365 policies), proactive threat hunting (with KQL), and a deep understanding of threat frameworks like MITRE ATT&CK for ML.
Analysis: The shift to agentic AI introduces a new class of runtime vulnerabilities that traditional security tools may miss. By integrating Alex Verboon’s KQL queries into their daily threat-hunting routines, security teams can bridge the gap between the theoretical risks of AI and the practical, actionable data needed to investigate and respond to incidents. The ability to visualize blocked actions and jailbreak attempts in near real-time transforms AI security from a “black box” into a transparent, auditable system. Furthermore, hardening Agent 365 with strict access and DLP policies provides the essential preventative controls to complement the detective power of KQL. The synthesis of these elements is crucial for organizations to confidently adopt Agent 365 and Microsoft 365 Copilot.
Prediction
As AI agents become ubiquitous in enterprise environments, we will see a surge in automated, AI-driven threats designed specifically to exploit agentic privileges and tool access. Consequently, KQL will evolve from a hunting language to a central policy-enforcement language for AI agents, allowing security teams to define “behavioral firewalls” for LLMs. Microsoft will likely integrate AI-specific threat intelligence directly into Sentinel and Defender, enabling predictive KQL query suggestions that can preemptively block novel jailbreak techniques before they are widely documented.
▶️ Related Video (76% Match):
🎯Let’s Practice For Free:
IT/Security Reporter URL:
Reported By: Verboonalex Kql – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅
