- Home
- Azure
- Azure Monitor Status Archive
- Experiencing Data Gaps Issue in Azure portal for Log Analytics - 03/01 - Resolved
- Subscribe to RSS Feed
- Mark as New
- Mark as Read
- Bookmark
- Subscribe
- Printer Friendly Page
- Report Inappropriate Content
We've confirmed that all systems are back to normal with no customer impact as of 03/12, 15:00 UTC. Our logs show the incident started on 03/01, 10:30 UTC and that during the 11 days 4 hours 30 mins that it took to resolve the issue some customers using Log Analytics in East US region may have noticed missing or delayed data for newly created monitoring solutions. Additionally, you may have also noticed missing or delayed data after making configuration changes to your existing monitoring solutions.This issue would only impact monitoring solutions configured for Linux resources, Windows monitoring solutions are not impacted.
- Root Cause: As part of the on boarding process for Log Analytics, customer resources are configured with a monitoring agent. In this case, a recent update introduced a configuration change that led to the OMS Linux agent (required to monitor Linux resources) calling an incorrect region endpoint when trying to collect data to send to the upstream ingestion service.
- Incident Timeline: 11 days 4 Hours & 30 minutes - 03/01, 10:30 UTC through 03/12, 15:00 UTC
-Syed
We are aware of issues within Log Analytics and are actively investigating. Some customers continue to experience missing or delayed data for newly created monitoring solutions. Additionally, customers may also notice missing or delayed data after making configuration changes to their existing monitoring solutions. This issue would only impact monitoring solutions configured for Linux resources, Windows monitoring solutions are not impacted.
- Work Around: There are 2 workarounds to resolve this issue:
1) Via Azure Portal, this is for Azure Machines:
- Login to Azure Portal. Navigate to Effected Workspace.
- Click on "Virtual Machines" in "Workspace Data Sources" on the blade left side.
- Select the impacted Virtual Machine. Disconnect the machine and reconnect it back. With this OMSAgent will get re-registered and will resolve the issue
2) Via terminal, this is for On-Prem Azure Machines:
- Connect to the impacted Linux machine.
- Purge the installation of OMSAgent using below command. This command should be executed with root privileges.
sudo "/opt/microsoft/omsagent/bin/uninstall" P
- Re-Onboard the Linux machine to the Log Analytics using steps mentioned in this url (https://github.com/microsoft/OMS-Agent-for-Linux#quick-install-guide
- Next Update: Before 03/13 11:30 UTC
We are aware of issues within Log Analytics and are actively investigating. Some customers continue to experience missing or delayed data for newly created monitoring solutions. Additionally, customers may also notice missing or delayed data after making configuration changes to their existing monitoring solutions. This issue would only impact monitoring solutions configured for Linux resources, Windows monitoring solutions are not impacted.
- Work Around: There are 2 workarounds to resolve this issue:
1) Via Azure Portal, this is for Azure Machines:
- Login to Azure Portal. Navigate to Effected Workspace.
- Click on "Virtual Machines" in "Workspace Data Sources" on the blade left side.
- Select the impacted Virtual Machine. Disconnect the machine and reconnect it back. With this OMSAgent will get re-registered and will resolve the issue
2) Via terminal, this is for On-Prem Azure Machines:
- Connect to the impacted Linux machine.
- Purge the installation of OMSAgent using below command. This command should be executed with root privileges.
sudo "/opt/microsoft/omsagent/bin/uninstall" P
- Re-Onboard the Linux machine to the Log Analytics using steps mentioned in this url (https://github.com/microsoft/OMS-Agent-for-Linux#quick-install-guide
- Next Update: Before 03/13 13:00 UTC
- Work Around: There are 2 workarounds to resolve this issue:
1) Via Azure Portal, this is for Azure Machines:
- Login to Azure Portal. Navigate to Effected Workspace.
- Click on "Virtual Machines" in "Workspace Data Sources" on the blade left side.
- Select the impacted Virtual Machine. Disconnect the machine and reconnect it back. With this OMSAgent will get re-registered and will resolve the issue
2) Via terminal, this is for On-Prem Azure Machines:
- Connect to the impacted Linux machine.
- Purge the installation of OMSAgent using below command. This command should be executed with root privileges.
sudo "/opt/microsoft/omsagent/bin/uninstall" P
- Re-Onboard the Linux machine to the Log Analytics using steps mentioned in this url (https://github.com/microsoft/OMS-Agent-for-Linux#quick-install-guide
- Next Update: Before 03/12 11:30 UTC
- Work Around: There are 2 workarounds to resolve this issue:
1) Via Azure Portal, this is for Azure Machines:
- Login to Azure Portal. Navigate to Effected Workspace.
- Click on "Virtual Machines" in "Workspace Data Sources" on the blade left side.
- Select the impacted Virtual Machine. Disconnect the machine and reconnect it back. With this OMSAgent will get re-registered and will resolve the issue
2) Via terminal, this is for On-Prem Azure Machines:
- Connect to the impacted Linux machine.
- Purge the installation of OMSAgent using below command. This command should be executed with root privileges.
sudo "/opt/microsoft/omsagent/bin/uninstall" P
- Re-Onboard the Linux machine to the Log Analytics using steps mentioned in this url (https://github.com/microsoft/OMS-Agent-for-Linux#quick-install-guide
- Next Update: Before 03/11 11:30 UTC
We are aware of issues within Log Analytics and are actively investigating. Some customers continue to experience missing or delayed data for newly created monitoring solutions. Additionally, customers may also notice missing or delayed data after making configuration changes to their existing monitoring solutions. This issue would only impact monitoring solutions configured for Linux resources, Windows monitoring solutions are not impacted.
- Work Around: There are 2 workarounds to resolve this issue:
1) Via Azure Portal, this is for Azure Machines:
- Login to Azure Portal. Navigate to Effected Workspace.
- Click on "Virtual Machines" in "Workspace Data Sources" on the blade left side.
- Select the impacted Virtual Machine. Disconnect the machine and reconnect it back. With this OMSAgent will get re-registered and will resolve the issue
2) Via terminal, this is for On-Prem Azure Machines:
- Connect to the impacted Linux machine.
- Purge the installation of OMSAgent using below command. This command should be executed with root privileges.
sudo "/opt/microsoft/omsagent/bin/uninstall" P
- Re-Onboard the Linux machine to the Log Analytics using steps mentioned in this url (https://github.com/microsoft/OMS-Agent-for-Linux#quick-install-guide
- Next Update: Before 03/10 11:30 UTC
We are aware of issues within Log Analytics and are actively investigating. Some customers continue to experience missing or delayed data for newly created monitoring solutions. Additionally, customers may also notice missing or delayed data after making configuration changes to their existing monitoring solutions. This issue would only impact monitoring solutions configured for Linux resources, Windows monitoring solutions are not impacted.
- Work Around: There are 2 workarounds to resolve this issue:
1) Via Azure Portal, this is for Azure Machines:
- Login to Azure Portal. Navigate to Effected Workspace.
- Click on "Virtual Machines" in "Workspace Data Sources" on the blade left side.
- Select the impacted Virtual Machine. Disconnect the machine and reconnect it back. With this OMSAgent will get re-registered and will resolve the issue
2) Via terminal, this is for On-Prem Azure Machines:
- Connect to the impacted Linux machine.
- Purge the installation of OMSAgent using below command. This command should be executed with root privileges.
sudo "/opt/microsoft/omsagent/bin/uninstall" P
- Re-Onboard the Linux machine to the Log Analytics using steps mentioned in this url (https://github.com/microsoft/OMS-Agent-for-Linux#quick-install-guide
- Next Update: Before 03/03 10:30 UTC
We are aware of issues within Log Analytics and are actively investigating. Some customers may experience missing or delayed data for newly created monitoring solutions. Additionally, customers may also notice missing or delayed data after making configuration changes to their existing monitoring solutions. This issue would only impact monitoring solutions configured for Linux resources, Windows monitoring solutions are not impacted.
- Work Around: None
- Next Update: Before 03/02 02:00 UTC
-Mohini
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.