We are announcing a series of major advancements and significant price reduction to Auxiliary Logs, the Azure Monitor plan that is designed for high volume logs.
Azure Monitor logs are trusted by hundreds of thousands of organizations to monitor mission-critical workloads. But with such a diverse customer base, there’s no one-size-fits-all solution. That’s why we’re excited to announce a series of major advancements to Auxiliary Logs, the Azure Monitor plan that is designed for high volume logs.
Auxiliary Logs works in tandem with all other Azure Monitor tools including the more powerful Basic and Analytics Logs plans. Together, they are the one-stop-shop for all the logging needs of an organization.
Auxiliary Logs were introduced last year and have gained a lot of traction since. There are many customers that ingest data into Auxiliary Logs, with several teams ingesting more than a petabyte of logs per day.
Over the last few months, we have moved Auxiliary Logs to General Availability status, made them available in all regions, and made numerous enhancements to the service.
Auxiliary Logs were first introduced with support for Custom Logs only; security data was added shortly afterwards and is now also supported. Additional table support will be available soon. Learn more about table plans here.
We’re also announcing a significant price reduction for Auxiliary Logs, making them even more cost-effective and accessible for high-volume scenarios. For detailed pricing information and charges, visit the Azure Monitor pricing page. This is part of a broader strategy as we align and evolve our data lake assets and pricing models, with the goal of enabling customers to benefit from modern data lake technology including batch computing and federated access without duplication for multiple use-cases spanning security and observability on a common technology stack. The Sentinel data lake announced recently is a key part of this evolution. Data ingested into the Sentinel data lake can be accessed in Auxiliary logs without copy and vice-versa. Stay tuned for more information later this year on how we’re evolving our data strategy for operational scenarios.
Enhanced Query Capabilities
We have worked to make queries on Auxiliary Logs faster and more powerful. That includes:
- Expanded KQL Support: All KQL operators on a single table are now supported, including the lookup operator to Analytics tables.
- Performance Boosts: Built on Delta Parquet, Auxiliary Logs now benefit from improved encoding and partitioning to make queries much more efficient, though indexed technologies like Basic Logs and Analytics Logs will perform better.
- Extended Time Range: Queries are no longer limited to the last 30 days - you can now query across any time period.
- Cost Estimation Preview: Get a cost estimate before running your query.
General Availability of Summary Rules
We are also announcing the General Availability of summary rules. Summary rules have quickly become a key resource for optimizing data ingestion and analysis, having been adopted by a significant number of customers during the preview period. Summary rules enable users to efficiently summarize high-ingestion-rate streams across Analytics, Basic, or Auxiliary plans, supporting robust analysis, dashboarding, and long-term reporting via summarized Analytics tables. Unlike conventional ETL processes, raw data remains in its original tables, allowing for detailed investigations as needed.
Key enhancements include:
- Increased rule limits per workspace
- Enabling users to retry bins affected by incidents
- Expanded regional availability
Customers can now utilize summary rules on a greater scale with increased confidence. Learn more about summary rules here.
Search Jobs: More Power, More Flexibility
Search jobs allow users to scan vast amounts of data asynchronously and ingest the results into Analytics table for further investigation Based on customer feedback, we’ve made the following improvements:
- Enabling more results to be loaded, up to 100 million records (coming soon)
- Improved user interface that streamlines the search job execution
- Providing a cost prediction before running a search job
- Increasing concurrently and removing additional limits.
- Added support for all KQL operators on a single table with the lookup operator to Analytics tables (coming soon).
Learn more about search jobs here.
Public Preview of KQL Transformations for Auxiliary Logs
Last, but not least, we’re excited to announce the public preview of KQL-based transformations for Auxiliary Logs. This milestone brings Auxiliary Logs to feature parity in terms of ingestion-time transformations with other Azure Monitor log tiers, eliminating the previous limitation of ingesting only raw custom logs into Auxiliary storage.
With this new capability, you can now apply filtering and transformation logic at ingestion time, enabling a more strategic and cost-effective approach to managing high-volume, low-fidelity logs. By using Data Collection Rules (DCRs) with Kusto Query Language (KQL) expressions, you can:
- Filter out noise to reduce data volume.
- Parse and shape fields to prepare logs for efficient downstream consumption.
- Split data across multiple tables or tiers, for cost-performance optimization.
What makes this especially powerful is that transformations apply to both custom and standard log streams directed to custom tables in the Auxiliary tier. For example, you can now route a portion, or the entirety, of specific platform logs to a custom table in Auxiliary storage, applying transformations as needed.
Applying custom transformations and filtering data ingested into the Auxiliary tier will incur a log processing charge. For detailed pricing information and charges, please refer to the Azure Monitor pricing page. Learn more about ingestion-time transformations here.
Microsoft
