How to leverage Azure Monitor to meet functional and non-functional requirements - No.1 overview
Published Aug 22 2022 05:38 PM 2,091 Views
Microsoft

Azure Monitor can be used for centralized monitoring and analysis of log data by using Kusto query, thus Azure Monitor allows you to effectively monitor and visualize Azure resources. Azure Arc also empowers Azure Monitor to expand its capability to on-premise and other public clouds. You can monitor every resources across environments, Azure, AWS, GCP, OCI, on-premise and others, with Azure Monitor and Azure Arc, then Azure Monitor minimize your effort to manage all the resources regardless locations or environments. Refer to Azure Monitor overview article for the detail.

daisami_0-1661214302099.png daisami_1-1661214321174.png

 

Azure Monitor is a very powerful solution, but customers and partners sometimes have a challenge to map Azure Monitor features to their functional and non-functional requirements. These series articles describe how to use various Azure Monitor features in terms of functional and non-functional requirements.

Note: ISVs and partners offer 3rd party monitoring solutions on Azure Marketplace. You should consider to use the solutions if Azure Monitor might not work well for the requirements.

 

Here are popular monitoring objectives based on functional and non-functional requirements for monitoring. 

Article No

Monitoring objective

Monitoring target

Note

2

Compute 

Reboot

Monitor reboot frequency for availability check

 

 

CPU

Monitor CPU usage

 

 

Memory

Monitor memory usage

3

Compute/Inside OS

log file

Monitor Event Log and syslog

 

 

Process

Monitor available process

4

Storage/Disk

Disk

Monitor disk usage

 

 

folder/file

Monitor folder usage and file size

5

Endpoint/IPv4 address

response/service

Monitor specific address and port

 

Web site

Scenario

Monitor web scenario

6

Network

Connectivity

Monitor vNiC and VNET peering

 

 

Firewall

Monitor Azure Firewall rule usage

7

Backup

Backup

Monitor backup status

 

Azure Resources

Resource health

Monitor resource availability

 

Then, these series articles dives deeply into each requirement. Here are check items for the objectives. 

Type

Category

Goal and outcome

Result

1

monitoring

Azure Monitor can satisfy functional requirements

 

2

 

Azure Monitor can setup short granularity for detections

 

3

 

Azure Monitor can setup thresholds for detections

 

4

 

Azure Monitor can setup retry for detections

 

5

 

Azure Monitor can suspend and resume for alert actions

 

6

 

Azure Monitor can send a mail based on detection results

 

7

statistics

Azure Monitor can retrieve workspace logs with specified duration

 

8

 

Azure Monitor can visualize statistic data including old one

 

9

automation

Azure Monitor can have primary action based on alert rules

 

10

 

Azure Monitor can send check results

 

Note: You might need to consider another way if you have to meet scenarios as follow.

  1. Monitor resources with less than 1 minute interval
    Azure Monitor offers 1 min aggregation granularity as minimum. You might need to look for 3rd party solutions if you require to check resources with less than 1 minute for example Zabbix, nagios, or others. https://docs.microsoft.com/en-us/azure/azure-monitor/essentials/metrics-aggregation-explained
  2. Monitor network with SNMP
    Azure network status can be configured from "diagnostic setting". Azure Monitor offers great monitoring features to meet most cases, but Azure Monitor doesn't offer snmp agent or snmpwalk agent.

 

1. Prerequisites

You have to complete three configurations below before following these series articles for Azure Monitor. 

 

1.1. Setup Log Analytics workspace

Azure Monitor uses "metrics" and "logs" for its data. Metric refers to system info of Azure resources for example CPU usage and memory usage. Log refers to Event Log and syslog. Each Azure resource has settings named as "diagnostic setting" and "activity log", which store logs of the resources on Azure services for example Log Analytics, Azure Storage, and others. Especially, Log Analytics workspace can be used for analysis and troubleshooting with Kusto query.

 

1.2. Setup Azure Blob Storage

You can store logs on Azure Blob Storage  "diagnostic setting" log and "activity log". Azure Blob Storage can be used to store logs with long term duration in terms of security rules for example regional compliances.

Note:
While Log Analytics can store logs up between 4 and 730 days and can set retention time of up to 2,555 days, Azure Blog Storage can store logs up to 500TB regardless durations. We should choose data stores depending on use cases.
https://docs.microsoft.com/en-us/azure/azure-monitor/logs/data-retention-archive

1.3. Setup Azure VM

Azure VM is used to take logs. Create Azure VM by following an article here. Open Azure Portal with browser and choose your Log Analytics workspace. Select "Virtual Machine" on left menus. Your fresh VM is not connected to any Log Analytics workspace, thus you need to connect your VM to a Log Analytics workspace. Please note that the VM is required to be running when you enable the VM to connect to a Log Analytics workspace. This operation will install and lunch Log Analytics agent on the VM.

2. Key tips to leverage Azure Monitor for enterprise use cases

Here should be useful tips to configure Azure Monitor for enterprise hybrid network requirements.

 

2.1. Access Log Analytics workspace with private network

AMPLS(Azure Monitor Private Link Scope) can allow you to have closed network access. Log Analytics workspace is required to setup "Network Isolation - Public Access" as "No", thus users privately access to the workspace. Log Analytics or Azure Monitor agent prefers to Azure DNS records than others, so the agent sends logs with HTTPS to Log Analytics workspace via private IP addresses of the workspace FQDN.
https://docs.microsoft.com/en-us/azure/azure-monitor/logs/private-link-configure

 

2.2. Check connectivity with Log Analytics workspace via proxy servers

You can check the status with CUI commands.
https://docs.microsoft.com/en-us/azure/azure-monitor/agents/agent-linux-troubleshoot

 

2.3. Allow omsagent to access some FQDN

omsagent uses https protocol as 443/tcp and access to some FQDN. You need to configure Azure Firewall and NVA to allow omsagent to access the FQDN if your VM needs to use Azure Firewall or NVA for outbound access. Refer to this article for details.

 

2.4 Retrieve all Log Analytics table

We can check Log Analytics tables on Azure Portal, but Azure Portal would show with only metadata and no logging data. We can retrieve all tables having logging data by using search operator with a wildcard. The result contains a $table column that represents the table in which record was found, thus the distinct operator is used to create a list of unique tables.

 

search *
| distinct $table

 

daisami_0-1661778313775.png

 

Now, we can start these series articles for Azure Monitor. In next post, we will dive deep to "Compute" monitoring objective.

 

 

Update date note
2022/9/2 add link for Storage/Disk monitoring
2022/8/29 add "2.4 Retrieve all Log Analytics table" section
2022/8/22 first publish

 

Version history
Last update:
‎Sep 02 2022 10:59 AM
Updated by: