How to leverage Azure Monitor to meet functional and non-functional requirements - No.4 Storage/Disk
Published Sep 02 2022 10:57 AM 1,257 Views
Microsoft

This article is a part of series articles for Azure Monitor. Please refer to How to leverage Azure Monitor to meet functional and non-functional requirements - No.1 overview first before reading this post. This post dives deeply for Disk category among monitoring categories as highlighted blue. 

Article No

monitoring category

monitoring target

Note

2

Compute

Reboot

monitor reboot frequency

 

 

CPU

monitor CPU usage

 

 

Memory

monitor memory usage

3

Compute/Inside OS

Log file

monitor event log and syslog

 

 

Process

monitor available process

4

Storage/Disk

Disk

monitor disk usage

 

 

folder/file

monitor folder usage and file size

5

Endpoint/IPv4 address

response/service

monitor specific address and port

 

Web site

Scenario

monitor web scenario

6

Network

Connectivity

monitor vNiC and VNET peering

 

 

Firewall

monitor Azure Firewall rule usage

7

Backup

Backup

monitor backup status

 

Azure Resources

Resource health

monitor resource availability

 

To monitor disk capacity is mandatory topics for monitoring requirements because disk space depletion can cause serious errors. It's also essential to monitor size of specific file or folder, where is frequently used for example file upload or  logging. We can check Disk and Folder/File several ways with Azure Monitor, thus this post describes how to leverage Azure Monitor for the purpose.

 

4.1 Disk

There are some several ways to monitor Disk as follows. We will deeply dive there methods.

  • Disk free space usage with VM Insights
  • Disk free space usage with Perf 

4.1.1 Disk free space usage with VM Insights

Here is an example query to check disk usage with InsightsMetrics. This feature allows us to check the disk usage per drive and mountpoint.

 

InsightsMetrics
| where Computer == "gcp-instance-01"
| where Namespace == "LogicalDisk"
| where Name == "FreeSpacePercentage"
| extend d=parse_json(Tags)
| project TimeGenerated, Computer, Namespace, Name, Val, d["vm.azm.ms/mountId"]

 

Right screenshot visualizes free space percentage of the mount id within 24 hours. vm.azm.ms/mountId indicates mountpoints on operation systems, thus we can monitor all devices across OS disk, Data disk, and temporary disk. Note that the mountpoint name is different from Disk resource name on Azure Portal. We can use Val for alert threshold, thus create a query to trigger notification when value of Val is less than 10% or others.

We can create a simple graph with project function setting up x-axis as time series and y-axis as expected value for example free space percentage here. We can also disable the alert rule and pause it in this case. 

daisami_0-1662067040358.png daisami_0-1662067163417.png

 

4.1.2 Disk free space usage with Perf

Please note the Log Analytics agents won’t be supported as of August 31, 2024. Plan to migrate to Azure Monitor Agent prior to this date.

Open Legacy agent management menu on your Log Analytics workspace. A screenshot below shows Linux performance counters configuration and added "% Free Space" and "Free Megabytes". It initially takes a couple of hours to reflect this configuration on Log Analytics workspace.

daisami_0-1662073674408.png

As next, run a Kusto query as follows to check free disk space. Update each parameter for your use. 

 

Perf
| where Computer == "gcp-instance-01"
| where ObjectName == "Logical Disk"
| where CounterName  == "% Free Space"
| where InstanceName == "/" or InstanceName == "/mountdisk"
| order by TimeGenerated
| project TimeGenerated, Computer, ObjectName, CounterName, InstanceName, CounterValue, CounterPath

 

We can check all mountpoints across OS disk, Data disk, and temporary disk. You can create a simple graph and disable an alert as same with VM Insights.

daisami_0-1662075066034.png daisami_1-1662075084745.png

 

Finally, here is check result of Disk monitoring. 

Type

category

Goal and outcome

Result

1

monitoring

Azure Monitor can satisfy functional requirements

OK

2

 

Azure Monitor can setup short granularity for detections

1 min

3

 

Azure Monitor can setup thresholds detections

OK

4

 

Azure Monitor can setup retry detections

OK

5

 

Azure Monitor can suspend and resume for checking threshold

OK

6

 

Azure Monitor can send a mail for detection results

OK 

7

statistics

Azure Monitor can retrieve workspace logs with specific duration

OK

8

 

Azure Monitor can visualize statistic data

OK

9

automation

Azure Monitor can have primary action based on alert rules

OK

10

 

Azure Monitor can send validation results

OK

 

4.2 Folder/File

There are some several ways to monitor Folder/File as follows. We will deeply dive there methods.

  • Leverage EventLog and Syslog
  • File Integrity Monitoring of Microsoft Defender for Cloud 

4.2.1 Leverage EventLog and Syslog

Please note the Log Analytics agents won’t be supported as of August 31, 2024. Plan to migrate to Azure Monitor Agent prior to this date.

This method leverages Syslog and a script file. Configure Linux Syslog on Legacy agent management menu on your Log Analytics workspace. A screenshot below shows Linux Syslog configuration and added local0 facility. You can use an other facility depending on your script file. 

daisami_0-1662105930022.png

Then, create a script to check folder or file usage and configure crontab to periodically run the script. This example script put its logs to /var/log/message and an agent sends the data to Log Analytics workspace, thus we can retrieve the log with query and enable an alert.

#!/bin/sh

# folder size check
ETC_HTTPD_DIR=`du -sk /etc/httpd | awk '{ print $1 }'`
if [ ${ETC_HTTPD_DIR} -gt 600 ]
then
        logger -sp local0.err "/etc/httpd dir size over (<700k)"
fi

# file size check
HTTPD_CONF_FILE=`ls -l /etc/httpd/conf/httpd.conf | awk '{ print $5 }'`
if [ ${HTTPD_CONF_FILE} -gt 6224 ]
then
        logger -sp local0.err "/etc/httpd/conf/httpd.conf size over (<7224byte)"
fi

This script does not use any Azure specific commands, thus you can tailor the script for your use cases. This is not built-in feature but more flexible than following method.

 

4.2.2 File Integrity Monitoring of Microsoft Defender for Cloud

File Integrity Monitoring(FIM) can monitor changes of file or directory. FIM can track OS configuration, registry, setting file and setting folder across all VMs associated to Log Analytics workspace. The setting takes a time to be reflected on Azure Portal at first time, thus I recommend to wait a day. Open FIM from Microsoft Defender for Cloud and setup configuration.

daisami_2-1662139889283.png daisami_3-1662140100620.png

 

Here is an example screenshot for FIM. This screenshot shows that FIM can monitor OS configuration, registry, setting file and setting folder across environments for example Azure, AWS, and GCP - you can definitely extend this to on-premise.

daisami_4-1662140276575.png

We can retrieve this information with Kusto query. Here is an example.

ConfigurationChange
| where Computer == "gcp-instance-01"
| where ConfigChangeType in("Files", "Registry")
| order by TimeGenerated
| render table
daisami_5-1662140754005.png

This feature is perfect to track changes, but you should consider to use 4.2.1 Leverage EventLog and Syslog if you have to check size of a file or a folder, which does not have mountpoint itself.

 

Finally, here is check result of File/Folder monitoring. 

Type

category

Goal and outcome

Result

1

monitoring

Azure Monitor can satisfy functional requirements

OK

2

 

Azure Monitor can setup short granularity for detections

1 min

3

 

Azure Monitor can setup thresholds detections

OK

4

 

Azure Monitor can setup retry detections

OK

5

 

Azure Monitor can suspend and resume for checking threshold

OK

6

 

Azure Monitor can send a mail for detection results

OK 

7

statistics

Azure Monitor can retrieve workspace logs with specific duration

OK

8

 

Azure Monitor can visualize statistic data

OK

9

automation

Azure Monitor can have primary action based on alert rules

OK

10

 

Azure Monitor can send validation results

OK

 

Version history
Last update:
‎Sep 02 2022 01:34 PM
Updated by: