High-volume logs are essential during investigations, but ingesting them into the Analytics plan can be costly. With Azure Copilot Observability Agent, you can use eligible Basic and Auxiliary logs as investigation evidence while helping reduce costs without losing the context needed to troubleshoot complex issues.
Cloud environments can generate large volumes of application, infrastructure, audit, and platform logs. Moving eligible tables to the Basic or Auxiliary table plan can help control costs while keeping the data available for search. But during an incident, teams may still need those logs to understand what happened. The Azure Copilot Observability Agent brings lower-cost log evidence into the same investigation as alerts, metrics, traces, topology, recent changes, and Azure resource context.
This capability is generally available as part of the Azure Copilot Observability Agent.
Why low-cost tables matter for investigation
During investigation, high-volume logs that are stored in Basic and Auxiliary table plans can help answer questions such as:
- What changed before the issue began?
- Which resource or component produced the error?
- How did a request move through the environment?
- Did operating-system or platform events contribute to the failure?
High-scale Azure Kubernetes Service (AKS) environments provide a useful example because they generate several high-volume data sources:
- ContainerLogV2 (Basic) - application output where errors, stack traces, and request-path details appear.
- AKSAudit and AKSControlPlane (Basic or Auxiliary) - records of who or what changed the cluster and how the control plane responded.
- Syslog (Auxiliary) - node-level operating-system events from components such as kubelet, the container runtime, disk, memory, and networking.
In AKS investigations, these sources provide application, control-plane, and node-level evidence that helps reconstruct what happened—without requiring every verbose table to remain on the Analytics plan.
The agent reads Basic and Auxiliary logs as first-class evidence
Basic and Auxiliary table plans are optimized for cost-efficient, high-volume data. Queries generally start from a single table and support KQL scalar and aggregation functions for filtering, parsing, and summarizing. Basic table queries support up to the previous 30 days. Auxiliary table queries can span the table's total retention period, but they're unoptimized and might take longer. This model matches a common investigation pattern: focus on a resource and time window, query the relevant table, and evaluate the evidence.
The Observability Agent builds log queries in that same shape. It scopes the investigation to the relevant resource and time window, then queries the appropriate table on its configured plan. The agent correlates the results with the fired alert, related resources, metrics, and recent changes. No agent-specific configuration is required for tables already using a low-cost plan, although the underlying data must be collected and accessible. The agent also explains what it examined and why, helping you review its reasoning.
The Azure Copilot Observability Agent queries eligible Basic and Auxiliary tables and correlates their evidence with alerts, metrics, topology, and Azure resource context
Example scenario: tracing authentication with Basic plan container logs
In this scenario, an engineering team uses the agent to trace a request to a document-search API across an AKS cluster. The agent follows the request from the ingress controller through the service-mesh sidecar to the application pod.
At each step, the agent queries ContainerLogV2 on the Basic plan and tracks how identity headers propagate. A series of targeted, single-table queries reconstructs the request path and pinpoints where the identity context is dropped. The team keeps detailed container output on a lower-cost plan without losing the evidence needed for investigation and root cause analysis.
Example scenario: grounding a cluster-health alert in Auxiliary plan Syslog
In this scenario, an Azure Monitor alert fires for a production AKS cluster. An operator starts a deep investigation, and the Observability Agent expands the scope from the alert target to related node pools and infrastructure. It evaluates several hypotheses: an acute change from baseline, an unhealthy node, CPU or memory pressure, and pod-scheduling failures.
The agent uses node-level Syslog on the Auxiliary plan alongside Kubernetes inventory and platform metrics to narrow the issue to a single node. The investigation concludes with a recommendation for human review. Where node logging isn't fully enabled, the agent also recommends enabling it so future investigations have more complete evidence.
What to keep in mind
- Coverage depends on what you collect. The agent investigates using the telemetry you have. If a table isn't being ingested, the agent says so and might recommend enabling it rather than guessing.
- No additional agent configuration. If an eligible table already uses the Basic or Auxiliary plan, no additional Observability Agent configuration is required. The data must still be collected, accessible, and supported by the investigation.
- Table plan query behavior still applies. Basic table queries support up to the previous 30 days. Auxiliary table queries can span total retention, but they're unoptimized and might take longer. The agent works within these characteristics.
- Table plan query billing still applies. Basic and Auxiliary queries done by the agent are subject to the standard query billing charges applicable for those table plans.
- You stay in control. The Observability Agent investigates, correlates, and explains. Humans stay in control of decisions and changes.
The takeaway
By combining cost-efficient retention with AI-assisted investigation, teams can preserve the evidence they need without keeping every high-volume table on the Analytics plan. The result is a more sustainable observability strategy: lower ingestion costs, broader investigation context, and faster progress from alert to root cause - while keeping engineers in control of the final decisions and actions.
Next steps
- Review the Azure Copilot Observability Agent overview for supported investigation experiences and current capabilities.
- Confirm table eligibility and retention options in Log Analytics table plans.
- Review the time-range, query-language, performance, and billing considerations for querying Basic and Auxiliary tables.
Stay connected
- Follow this blog for ongoing deep dives, updates on current capabilities, and a preview of what's coming next.
- Live webinar
A walkthrough of real Observability Agent scenarios, best practices, and what's available today - along with a look at what's coming next and live Q&A with the product team. Register for the Observability Agent webinar.
We'd love your feedback
The Observability Agent continues to evolve based on real-world usage and operator feedback. Share your thoughts directly through the Give Feedback option in the experience, or reach us at azureobsagent@microsoft.com.