Enhancing Log Analytics: Key Updates for Auxiliary Tables in Microsoft Sentinel

By /

Listen to this Post

Featured Image

Introduction

Microsoft has recently announced the general availability (GA) of Auxiliary tables in Log Analytics, removing previous limitations on data retention and transformations. This update simplifies third-party data ingestion and optimizes cost efficiency for security teams using Microsoft Sentinel.

Learning Objectives

  • Understand the implications of Auxiliary tables going GA in Log Analytics.
  • Learn how to leverage transformations in Data Collection Rules (DCRs) for Auxiliary tables.
  • Explore best practices for cost-effective third-party data ingestion in Microsoft Sentinel.

You Should Know

1. Auxiliary Tables Now Support Flexible Retention Policies

Previously, Auxiliary tables enforced a 365-day retention policy, which could lead to unnecessary storage costs. With this update, administrators can now customize retention periods.

Verified Command (Azure CLI):

az monitor log-analytics workspace table update --name AuxiliaryTableName --resource-group MyResourceGroup --workspace-name MyWorkspace --retention-time 30

Step-by-Step Guide:

  1. Use the Azure CLI command above to modify retention for an Auxiliary table.
  2. Replace AuxiliaryTableName, MyResourceGroup, and `MyWorkspace` with your specific values.
  3. Set `–retention-time` in days (e.g., `30` for one month).

2. Transformations in Data Collection Rules (DCRs)

Auxiliary tables now support transformations via DCRs, enabling preprocessing of ingested data.

Verified KQL Transformation Example:

source 
| extend parsedData = parse_json(rawData) 
| project parsedData.field1, parsedData.field2

Step-by-Step Guide:

  1. Navigate to Microsoft Sentinel > Data Collection Rules.
  2. Create or edit a DCR for your Auxiliary table.
  3. Apply the KQL transformation to filter or enrich data before storage.

3. Optimizing Third-Party Data Ingestion

With Auxiliary tables, low-cost ingestion of external data (e.g., firewall logs, threat feeds) is now more flexible.

Verified PowerShell Command:

New-AzOperationalInsightsDataSource -WorkspaceName MyWorkspace -ResourceGroupName MyResourceGroup -Name "ThirdPartyFeed" -Kind "CustomLog" -DefinitionFile "C:\path\to\definition.json"

Step-by-Step Guide:

  1. Prepare a JSON file defining the custom log schema.
  2. Use the PowerShell cmdlet to register the data source.
  3. Map the data to an Auxiliary table via DCRs.

4. Monitoring Auxiliary Table Costs

Track storage costs using Azure Workbooks or built-in Log Analytics queries.

Verified KQL Query:

AzureDiagnostics 
| where Category == "Storage" 
| summarize TotalGB = sum(Quantity) / 1024 by TableName

Step-by-Step Guide:

  1. Run this query in Log Analytics to monitor table sizes.
  2. Adjust retention policies or data filters based on cost insights.

5. Future-Proofing with Dynamic Types (Pending Update)

As noted by Thomas V., Auxiliary tables currently lack dynamic type support. Monitor Microsoft’s updates for potential enhancements.

What Undercode Say

  • Key Takeaway 1: The removal of retention and transformation restrictions empowers teams to tailor Log Analytics to their budget and operational needs.
  • Key Takeaway 2: While dynamic types are still unsupported, this update marks significant progress in Microsoft Sentinel’s flexibility.

Analysis:

This update reflects Microsoft’s commitment to improving cost efficiency and usability in security operations. By decoupling Auxiliary tables from rigid retention policies, organizations can now optimize storage costs without sacrificing data accessibility. The addition of DCR transformations further streamlines data preprocessing, reducing the need for post-ingest KQL manipulations. However, the lack of dynamic types may limit schema adaptability for complex datasets. Security teams should evaluate these changes against their specific use cases, particularly when integrating heterogeneous data sources.

Prediction

Future updates may introduce dynamic type support, unlocking even broader use cases for Auxiliary tables. As Microsoft Sentinel evolves, expect deeper integration with Azure Purview and enhanced AI-driven analytics for threat detection. Organizations adopting these features early will gain a competitive edge in scalable, cost-effective security monitoring.

IT/Security Reporter URL:

Reported By: Robbe Van – Hackers Feeds
Extra Hub: Undercode MoN
Basic Verification: Pass ✅

🔐JOIN OUR CYBER WORLD [ CVE News • HackMonitor • UndercodeNews ]

💬 Whatsapp | 💬 Telegram

📢 Follow UndercodeTesting & Stay Tuned:

𝕏 formerly Twitter 🐦 | @ Threads | 🔗 Linkedin

Related Posts: