Collecting and displaying niping network latency measurements with Azure Monitor

I was recently involved in a project where we needed iostat metrics to be surfaced within the Azure portal, and also having the ability to create alerts based on this data. Here are my instructions on doing this:

First, I created a shell script that runs iostat and collects the data in the proper format. Here is my script which I put in /usr/local/bin/iostat_sh.sh:

#!/bin/bash

while read -r device read_queued_sec write_queued_sec read_completed_sec write_completed_sec read_kbytes_sec write_kbytes_sec avg_sector_size avg_queue_len await_ms read_await_ms write_await_ms svctime_ms util_pct; do
    [ ! -z "$device" ] && echo "iostat,device=$device read_queued_sec=$read_queued_sec,write_queued_sec=$write_queued_sec,read_completed_sec=$read_completed_sec,write_completed_sec=$write_completed_sec,read_kbytes_sec=$read_kbytes_sec,write_kbytes_sec=$write_kbytes_sec,avg_sector_size=$avg_sector_size,avg_queue_len=$avg_queue_len,await_ms=$await_ms,read_await_ms=$read_await_ms,write_await_ms=$write_await_ms,svctime_ms=$svctime_ms,util_pct=$util_pct"
done < <(iostat -dxyk 5 1 |  tail -n +4)
exit 0

Remember that the two number parameters for iostat are the interval and the count. In my case I used 5 seconds for the interval and 1 for the count – you may want to adjust the interval that you are measuring over. Then I did chmod a+x /usr/local/bin/iostat_sh.sh to make the shell script executable.

Next I modified my /etc/telegraf/telegraf.conf to have this input section:

[[inputs.exec]]
  commands = [ "/usr/local/bin/iostat_sh.sh" ]
  timeout = "10s"
  data_format = "influx"

To make the new telegraf.conf take effect, you have to stop and restart the telegraf service:

systemctl stop telegraf
systemctl start telegraf

After a few minutes, the metrics should come under the virtual machine metrics, and in my example below I split by the device: