Forum Discussion
AKS Log Analytics Workspace records the log for only 3 hours problem
Dear AKS service provider,
My AKS Log Analytics Workspace only records from 13:30 UTC to 16:30 UTC container logs each day in the "ContainerLogV2". I wonder what config causes it. How to diagnose and revive to whole day logs? How to troubleshoot?
I am sure the container is working properly and has a full log for the day.
2 Replies
If ContainerLogV2 only shows data for the same 3-hour window every day, I would first prove whether logs are not being collected, or whether they are being collected but not landing in that table.
A few quick checks:
ContainerLogV2 | summarize count(), min(_TimeReceived), max(_TimeReceived) by bin(TimeGenerated, 1h)
Then check the AMA logs daemonset:
kubectl get ds ama-logs -n kube-system kubectl get pods -n kube-system -l rsName=ama-logs -o wide
If the agent pods restart or go unhealthy around the time logging stops, start there. If the agent is healthy and other tables like KubePodInventory keep updating all day, then I would look at the Container Insights config/ConfigMap, namespace filters, table retention, or ingestion throttling.
The fixed daily window is the clue. That usually points to either an agent/config schedule issue or a query/table setting, not random pod logging behavior.
Docs:
https://learn.microsoft.com/azure/azure-monitor/containers/container-insights-troubleshoot
https://learn.microsoft.com/azure/azure-monitor/containers/container-insights-logs-schema
https://learn.microsoft.com/azure/azure-monitor/logs/data-retention-configureI am not the provider, please allow me to share some ideas:
1. Inspect Data Collection Configuration
- Go to your AKS cluster in the Azure Portal.
- Navigate to Monitoring > Insights > Data Collection Settings.
- Ensure ContainerLogV2 is enabled and not scoped to specific hours or namespaces.
2. Check Agent Health
kubectl get pods -n kube-system | grep omsagentkubectl logs <omsagent-pod-name> -n kube-systemLook for errors or restarts that might explain the gap.
3. Review Workspace Query Scope
In Log Analytics, run:
ContainerLogV2 | where TimeGenerated > ago(24h) | summarize count() by bin(TimeGenerated, 1h)This will show you hourly log volumes and help pinpoint gaps.
4. Validate Cluster Resource Usage
If your nodes are under memory or CPU pressure, logging agents may fail intermittently:
kubectl top nodesand
kubectl top pods --all-namespaces