SOLVED
Home

Heartbeat Azure Monitor OMS VMs

%3CLINGO-SUB%20id%3D%22lingo-sub-174128%22%20slang%3D%22en-US%22%3EHeartbeat%20Azure%20Monitor%20OMS%20VMs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-174128%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20all%2C%20i%20am%20trying%20to%20create%20an%20Alert%20for%20heartbeats%20if%20VM%20is%20didn't%20heartbeat%20in%20the%20last%2015%20min...%20here%20what%20i%20did%20and%20unfortunately%20it%20didnt%20fire%20alert.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3Eso%20i%20created%20a%20new%20alert%20rule%20in%20Azure%20Monitor%20(Alerts)%20the%20new%20one.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3Ei%20used%20the%20following%20query%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHeartbeat%3CBR%20%2F%3E%7C%20where%20TimeGenerated%20%26gt%3B%20ago(1d)%3CBR%20%2F%3E%7C%20summarize%20max(TimeGenerated)%20by%20Computer%3CBR%20%2F%3E%7C%20where%20max_TimeGenerated%20%26lt%3B%20ago(15m)%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3Ethis%20result%20if%20the%20VM%20didnt%20send%20any%20heartbeat%20for%20the%20last%2015%20mins.%26nbsp%3B%3C%2FP%3E%0A%3CP%3Ei%20set%20the%20alert%20logic%20to%20be%20on%20(Number%20of%20results%20Greater%20than%200)%26nbsp%3B%3C%2FP%3E%0A%3CP%3EEvaluated%20based%20on%20the%20Period%2015%20mins%20and%20Frequency%205%20mins.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3Ei%20stopped%20the%20agent%20and%20the%20results%20of%20the%20query%20indeed%20was%20more%20than%200%20but%20the%20monitor%20didnt%20trigger%20%2C%20does%20any%20one%20have%20a%20working%20example%20for%20Alerting%20on%20heartbeats%2C%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EThanks.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-174128%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EOMS%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EQuery%20Language%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-252330%22%20slang%3D%22en-US%22%3ERe%3A%20Heartbeat%20Azure%20Monitor%20OMS%20VMs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-252330%22%20slang%3D%22en-US%22%3E%3CP%3EKeep%20in%20mind%20that%20for%20example%20if%20server%20goes%20down%20and%20it%20is%20not%20available%20for%20an%20hour%20.%20You%20will%20receive%20within%20an%20hour%20roughly%204-5%20alerts%20for%20the%20same%20server.%20This%20is%20because%20you%20period%20is%2024%20hours.%20My%20recommendation%20is%20Period%20and%20frequency%20to%20be%20the%20same.%20For%20example%2015%20mins.%20That%20way%20you%20will%20not%20receive%20so%20many%20alerts%20for%20the%20same%20thing.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-252274%22%20slang%3D%22en-US%22%3ERe%3A%20Heartbeat%20Azure%20Monitor%20OMS%20VMs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-252274%22%20slang%3D%22en-US%22%3E%3CP%3EQuestion...%20is%20the%20following%20configuration%20fine%20for%20alerting%3F%20Or%20should%20I%20increase%20the%20frequency%3F%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%3CDIV%3E%3CDIV%3E%3CSTRONG%3EQuery%3A%3C%2FSTRONG%3E%3C%2FDIV%3E%3CDIV%3E%3CSPAN%3EHeartbeat%3C%2FSPAN%3E%3C%2FDIV%3E%3CDIV%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%3E%3CSPAN%3E%7C%20%3C%2FSPAN%3E%3CSPAN%3Esummarize%3C%2FSPAN%3E%3CSPAN%3E%20%5B%3C%2FSPAN%3E%3CSPAN%3E%22Last%20Heartbeat%22%3C%2FSPAN%3E%3CSPAN%3E%5D%3Dmax(TimeGenerated)%20%3C%2FSPAN%3E%3CSPAN%3Eby%3C%2FSPAN%3E%3CSPAN%3E%20Computer%3C%2FSPAN%3E%3C%2FDIV%3E%3CDIV%3E%3CSPAN%3E%7C%20%3C%2FSPAN%3E%3CSPAN%3Ewhere%3C%2FSPAN%3E%3CSPAN%3E%20%5B%3C%2FSPAN%3E%3CSPAN%3E%22Last%20Heartbeat%22%3C%2FSPAN%3E%3CSPAN%3E%5D%20%26lt%3B%20ago(%3C%2FSPAN%3E%3CSPAN%3E15%3C%2FSPAN%3E%3CSPAN%3Em)%3C%2FSPAN%3E%3C%2FDIV%3E%3CDIV%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%3E%3CSTRONG%3EBased%20on%3A%3C%2FSTRONG%3E%3C%2FDIV%3E%3CDIV%3ENumber%20of%20results%3C%2FDIV%3E%3CDIV%3E%3CSTRONG%3ECondition%3A%3C%2FSTRONG%3E%3C%2FDIV%3E%3CDIV%3EGreater%20than%3C%2FDIV%3E%3CDIV%3E%3CSTRONG%3EThreshold%3A%3C%2FSTRONG%3E%3C%2FDIV%3E%3CDIV%3E0%3C%2FDIV%3E%3CDIV%3E%26nbsp%3B%3C%2FDIV%3E%3CDIV%3E%3CSTRONG%3EEvaluated%20based%20on%3A%3C%2FSTRONG%3E%3C%2FDIV%3E%3CDIV%3EPeriod%3A%201440%20minutes%3C%2FDIV%3E%3CDIV%3EFrequency%3A%2015%20minutes%3C%2FDIV%3E%3CDIV%3E%26nbsp%3B%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-199494%22%20slang%3D%22en-US%22%3ERe%3A%20Heartbeat%20Azure%20Monitor%20OMS%20VMs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-199494%22%20slang%3D%22en-US%22%3E%3CP%3Eafter%20removing%20the%20day%20query%20line%2C%20you%20can%20write%20the%26nbsp%3Bline%20like.%3C%2FP%3E%3CP%3Elet%20start_time%3Dstartofday(datetime(%222018-05-30%22))%3B%3C%2FP%3E%3CP%3Elet%20end_time%3Dstartofday(datetime(%222018-05-31%22))%3B%3C%2FP%3E%3CP%3E%7C%20where%20TimeGenerated%20%26gt%3B%20start_time%20and%20TimeGenerated%20%26lt%3B%20end_time%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-174390%22%20slang%3D%22en-US%22%3ERe%3A%20Heartbeat%20Azure%20Monitor%20OMS%20VMs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-174390%22%20slang%3D%22en-US%22%3E%3CP%3EHi%3C%2FP%3E%0A%3CP%3EYour%20query%20is%20correct.%20Probably%20you%20should%20remove%3C%2FP%3E%0A%3CPRE%3E%7C%20where%20TimeGenerated%20%26gt%3B%20ago(1d)%3C%2FPRE%3E%0A%3CP%3Ebecause%20when%20the%20query%20is%20used%20in%20alert%20the%20timespan%2Ftimeframe%20is%20defined%20in%20the%20alert%20itself.%20In%20the%20heartbeat%20alert%20you%20would%20want%20your%20the%20evaluation%20time%20to%20be%20longer%20than%2015%20minutes.%20Make%20it%20at%20least%20one%20hour%20but%20probably%2024%20hours%20would%20be%20better%20as%20that%20was%20the%20time%20span%20in%20your%20query.%20With%20that%20setting%20you%20should%20get%20alert%20in%2015%20minutes%20after%20the%20vm%20goes%20down.%20Keep%20in%20mind%20that%20you%20want%20the%20VM%20to%20be%20down%20for%2015%20minutes%20at%20least.%20If%20it%20goes%20down%20only%20for%205%20minutes%20you%20will%20probably%20not%20be%20alerted%20because%20heartbeat%20events%20will%20start%20to%20be%20send%20again%20so%20the%20alert%20will%20never%20trigger%20that%20the%20last%20heartbeat%20event%20was%2015%20minutes%20ago.%3C%2FP%3E%0A%3CP%3ELet%20me%20know%20if%20you%20have%20further%20questions.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-440012%22%20slang%3D%22en-US%22%3ERe%3A%20Heartbeat%20Azure%20Monitor%20OMS%20VMs%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-440012%22%20slang%3D%22en-US%22%3E%3CP%3ESurely%20there%20is%20a%20better%20solution%20for%20this%3F%20My%20use%20case%20doesn't%20work%3A%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E1.%20Create%20a%20computer%20group%3C%2FP%3E%3CP%3E2.%20Alert%20when%20an%20agent%20in%20computer%20group%20has%20not%20%22heartbeated%22%20for%20over%2024%20hours.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EBy%20the%20logic%20in%20Alerts%2C%20even%20if%20I%20set%20the%20query%20as%20I%20do%20below%2C%20the%20time%20span%20that%26nbsp%3B%3CSTRONG%3EI%20define%3C%2FSTRONG%3E%20is%26nbsp%3Bignored%20because%20of%20the%20%22Period%22%20in%20Alerts%3A%3C%2FP%3E%3CP%3E%3CBR%20%2F%3EHeartbeat%3CBR%20%2F%3E%7C%20project%20TimeGenerated%2C%20Computer%3CBR%20%2F%3E%7C%20where%20TimeGenerated%20%26lt%3B%20now()%3CBR%20%2F%3E%7C%20where%20Computer%20in%20(COMPUTERGROUP)%3CBR%20%2F%3E%7C%20summarize%20%5B%22Last%20Heartbeat%22%5D%3Dmax(TimeGenerated)%20by%20Computer%3CBR%20%2F%3E%7C%20where%20%5B%22Last%20Heartbeat%22%5D%20%26lt%3B%20ago(24h)%3CBR%20%2F%3E%3CBR%20%2F%3EIs%20there%20any%20way%20to%20get%20around%20this%20extremely%20limiting%20design%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E
Ahmed Atef
Contributor

Hi all, i am trying to create an Alert for heartbeats if VM is didn't heartbeat in the last 15 min... here what i did and unfortunately it didnt fire alert. 

 

so i created a new alert rule in Azure Monitor (Alerts) the new one. 

 

i used the following query:

 

Heartbeat
| where TimeGenerated > ago(1d)
| summarize max(TimeGenerated) by Computer
| where max_TimeGenerated < ago(15m)

 

this result if the VM didnt send any heartbeat for the last 15 mins. 

i set the alert logic to be on (Number of results Greater than 0) 

Evaluated based on the Period 15 mins and Frequency 5 mins. 

 

i stopped the agent and the results of the query indeed was more than 0 but the monitor didnt trigger , does any one have a working example for Alerting on heartbeats,

 

Thanks.

5 Replies
Solution

Hi

Your query is correct. Probably you should remove

| where TimeGenerated > ago(1d)

because when the query is used in alert the timespan/timeframe is defined in the alert itself. In the heartbeat alert you would want your the evaluation time to be longer than 15 minutes. Make it at least one hour but probably 24 hours would be better as that was the time span in your query. With that setting you should get alert in 15 minutes after the vm goes down. Keep in mind that you want the VM to be down for 15 minutes at least. If it goes down only for 5 minutes you will probably not be alerted because heartbeat events will start to be send again so the alert will never trigger that the last heartbeat event was 15 minutes ago.

Let me know if you have further questions.

after removing the day query line, you can write the line like.

let start_time=startofday(datetime("2018-05-30"));

let end_time=startofday(datetime("2018-05-31"));

| where TimeGenerated > start_time and TimeGenerated < end_time

Question... is the following configuration fine for alerting? Or should I increase the frequency? 

Query:
Heartbeat
 
| summarize ["Last Heartbeat"]=max(TimeGenerated) by Computer
| where ["Last Heartbeat"] < ago(15m)
 
Based on:
Number of results
Condition:
Greater than
Threshold:
0
 
Evaluated based on:
Period: 1440 minutes
Frequency: 15 minutes
 

Keep in mind that for example if server goes down and it is not available for an hour . You will receive within an hour roughly 4-5 alerts for the same server. This is because you period is 24 hours. My recommendation is Period and frequency to be the same. For example 15 mins. That way you will not receive so many alerts for the same thing.

Surely there is a better solution for this? My use case doesn't work:

 

1. Create a computer group

2. Alert when an agent in computer group has not "heartbeated" for over 24 hours. 

 

By the logic in Alerts, even if I set the query as I do below, the time span that I define is ignored because of the "Period" in Alerts:


Heartbeat
| project TimeGenerated, Computer
| where TimeGenerated < now()
| where Computer in (COMPUTERGROUP)
| summarize ["Last Heartbeat"]=max(TimeGenerated) by Computer
| where ["Last Heartbeat"] < ago(24h)

Is there any way to get around this extremely limiting design?