Forum Discussion

deep-learner's avatar
deep-learner
Copper Contributor
Sep 04, 2025

AKS Log Analytics Workspace records the log for only 3 hours problem

Dear AKS service provider,

My AKS Log Analytics Workspace only records from 13:30 UTC to 16:30 UTC container logs each day in the "ContainerLogV2". I wonder what config causes it. How to diagnose and revive to whole day logs? How to troubleshoot?

I am sure the container is working properly and has a full log for the day. 

 

2 Replies

  • If ContainerLogV2 only shows data for the same 3-hour window every day, I would first prove whether logs are not being collected, or whether they are being collected but not landing in that table.

    A few quick checks:

    ContainerLogV2 | summarize count(), min(_TimeReceived), max(_TimeReceived) by bin(TimeGenerated, 1h)

    Then check the AMA logs daemonset:

    kubectl get ds ama-logs -n kube-system kubectl get pods -n kube-system -l rsName=ama-logs -o wide

    If the agent pods restart or go unhealthy around the time logging stops, start there. If the agent is healthy and other tables like KubePodInventory keep updating all day, then I would look at the Container Insights config/ConfigMap, namespace filters, table retention, or ingestion throttling.

    The fixed daily window is the clue. That usually points to either an agent/config schedule issue or a query/table setting, not random pod logging behavior.

    Docs:
    https://learn.microsoft.com/azure/azure-monitor/containers/container-insights-troubleshoot
    https://learn.microsoft.com/azure/azure-monitor/containers/container-insights-logs-schema
    https://learn.microsoft.com/azure/azure-monitor/logs/data-retention-configure

  • I am not the provider, please allow me to share some ideas:

     

    1. Inspect Data Collection Configuration

    • Go to your AKS cluster in the Azure Portal.
    • Navigate to Monitoring > Insights > Data Collection Settings.
    • Ensure ContainerLogV2 is enabled and not scoped to specific hours or namespaces.

    2. Check Agent Health

    kubectl get pods -n kube-system | grep omsagent

     

    kubectl logs <omsagent-pod-name> -n kube-system

    Look for errors or restarts that might explain the gap.

    3. Review Workspace Query Scope

    In Log Analytics, run:

    ContainerLogV2
    | where TimeGenerated > ago(24h)
    | summarize count() by bin(TimeGenerated, 1h)

    This will show you hourly log volumes and help pinpoint gaps.

    4. Validate Cluster Resource Usage

    If your nodes are under memory or CPU pressure, logging agents may fail intermittently:

    kubectl top nodes

    and

    kubectl top pods --all-namespaces