Home

get on-prem connector usage

%3CLINGO-SUB%20id%3D%22lingo-sub-906906%22%20slang%3D%22en-US%22%3Eget%20on-prem%20connector%20usage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-906906%22%20slang%3D%22en-US%22%3E%3CP%3Ei%20would%20like%20to%20get%20on-prem%20connector%20(Linux%20and%20MS)%20utilization%20and%20check%20whether%20it%20is%20on%20high%20load%2C%20how%20do%20i%20create%20an%20alert%20for%20that%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-906906%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3Ekql%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESentinel%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-908019%22%20slang%3D%22en-US%22%3ERe%3A%20get%20on-prem%20connector%20usage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-908019%22%20slang%3D%22en-US%22%3E%3CP%3EHello%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F423940%22%20target%3D%22_blank%22%3E%40omrip%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWhen%20you%20say%20high%20load%2C%20do%20you%20mean%20CPU%2FMemory%20on%20the%20Windows%20or%20Linux%20server%3F%26nbsp%3B%20If%20so%20you%20need%20to%20capture%20info%20into%20the%20%3CSTRONG%3EPerf%3C%2FSTRONG%3E%20Table%20using%20Log%20Analytics%2C%20i.e%20the%20%22process%22%20counters%20for%20the%20agent%20or%20the%20server%20a%20whole.%26nbsp%3B%20You%20may%20also%20want%20to%20consider%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-platform-metrics%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20Monitor%20Metric%20alerts%3C%2FA%3E%20-%20as%20they%20are%20near%20real-time.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHowever%20if%20its%20delays%20you%20are%20looking%20for%20then%20the%20Heartbeat%20(and%20all%20tables)%20provide%20the%20ingestion%20and%20latency%20info%2C%26nbsp%3BSome%20examples%20from%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%3C%2FA%3E%26nbsp%3B%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-markup%22%3E%3CCODE%3E%2F%2F%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%3C%2FA%3E%0AHeartbeat%0A%7C%20where%20TimeGenerated%20%26gt%3B%20ago(8h)%20%0A%7C%20extend%20E2EIngestionLatency%20%3D%20ingestion_time()%20-%20TimeGenerated%20%0A%7C%20extend%20AgentLatency%20%3D%20_TimeReceived%20-%20TimeGenerated%20%0A%7C%20summarize%20percentiles(E2EIngestionLatency%2C50%2C95)%2C%20percentiles(AgentLatency%2C50%2C95)%20by%20Computer%20%0A%7C%20top%2020%20by%20percentile_E2EIngestionLatency_95%20desc%0A%0A%0AHeartbeat%20%0A%7C%20where%20TimeGenerated%20%26gt%3B%20ago(24h)%20%0A%7C%20extend%20E2EIngestionLatencyMin%20%3D%20todouble(datetime_diff(%22Second%22%2Cingestion_time()%2CTimeGenerated))%2F60%20%0A%7C%20extend%20AgentLatencyMin%20%3D%20todouble(datetime_diff(%22Second%22%2C_TimeReceived%2CTimeGenerated))%2F60%20%0A%7C%20summarize%20percentiles(E2EIngestionLatencyMin%2C50%2C95)%2C%20percentiles(AgentLatencyMin%2C50%2C95)%20by%20bin(TimeGenerated%2C30m)%20%2C%20Computer%0A%7C%20limit%203%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3Enote%3A%20latency%20can%20be%20caused%20by%20many%20factors%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-910458%22%20slang%3D%22en-US%22%3ERe%3A%20get%20on-prem%20connector%20usage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-910458%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F239477%22%20target%3D%22_blank%22%3E%40Clive%20Watson%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ethanks%20for%20the%20reply%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E1.%20how%20to%20i%20create%20an%20alert%20for%20a%20sourcetypes%20%2F%20connector%20that%20stopped%20sending%20the%20logs.%3C%2FP%3E%3CP%3E2.%20how%20do%20i%20get%20the%20alert%20done%20for%20the%20the%20latency%3F%20anything%20that%20is%20above%20the%20average%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-911273%22%20slang%3D%22en-US%22%3ERe%3A%20get%20on-prem%20connector%20usage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-911273%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F423940%22%20target%3D%22_blank%22%3E%40omrip%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E1.%20To%20Alert%20you%20typically%20need%20an%20AggregatedValue%2C%20this%20allows%20an%20Azure%20Monitor%20Alert%20to%20display%20a%20value%2C%20that%20the%20alert%20can%20threshold%20from.%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-markup%22%3E%3CCODE%3Eunion%20withsource%20%3D%20tt%20*%0A%7C%20where%20TimeGenerated%20%26lt%3B%20now()%0A%7C%20where%20isnotempty(Type)%0A%7C%20summarize%20maxTimeGenerated%3Dmax(TimeGenerated)%20by%20Type%0A%7C%20where%20maxTimeGenerated%20%20%26lt%3B%20ago(24h)%0A%7C%20extend%20SolutionName%20%3D%20strcat(Type%2C%20'%3A%20LatestData%3A%20'%2C%20maxTimeGenerated)%0A%7C%20summarize%20AggregatedValue%20%3D%20count()%20by%20SolutionName%2C%20maxTimeGenerated%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3ESo%20if%20I%20run%20the%20above%2C%20I%20would%20make%20an%20Alert%20by%20pressing%20the%20%22Add%20New%20Alert%20Rule%22%3CBR%20%2F%3EPlease%20see%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Flearn%2Ftutorial-response%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Flearn%2Ftutorial-response%3C%2FA%3E%3C%2FP%3E%0A%3CP%3Eand%20also%20this%20series%20of%20posts%20(this%20is%20Post%207%2C%20but%20start%20at%20%231)%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fcloudadministrator.net%2F2019%2F10%2F07%2Fazure-monitor-alert-series-part-7%2F%3Ffbclid%3DIwAR0pBvGLhqmZFI5eYkrbTwmWdeWlgQpd6zBSA7NPoIJ_KkouuxX9SCdXCpY%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fcloudadministrator.net%2F2019%2F10%2F07%2Fazure-monitor-alert-series-part-7%2F%3Ffbclid%3DIwAR0pBvGLhqmZFI5eYkrbTwmWdeWlgQpd6zBSA7NPoIJ_KkouuxX9SCdXCpY%3C%2FA%3E%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3E2.%20Maybe%20something%20like%20this%2C%20please%20modify%20to%20suit.%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-markup%22%3E%3CCODE%3EHeartbeat%0A%7C%20where%20TimeGenerated%20%26gt%3B%20ago(1h)%20%0A%7C%20extend%20E2EIngestionLatency%20%3D%20ingestion_time()%20-%20TimeGenerated%20%0A%7C%20extend%20AgentLatency%20%3D%20_TimeReceived%20-%20TimeGenerated%20%0A%7C%20summarize%20avgLatency%20%3D%20avg(AgentLatency)%20by%20Computer%20%2C%20E2EIngestionLatency%0A%7C%20where%20avgLatency%20%26gt%3B%20E2EIngestionLatency%0A%7C%20extend%20avgLatencyBreachedfor%20%3D%20strcat(Computer%2C%20'%20%3A%20'%2C%20avgLatency)%0A%7C%20summarize%20AggregatedValue%20%3D%20count()%20by%20avgLatencyBreachedfor%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Highlighted
Occasional Contributor

i would like to get on-prem connector (Linux and MS) utilization and check whether it is on high load, how do i create an alert for that?

3 Replies
Highlighted

Hello @omrip 

 

When you say high load, do you mean CPU/Memory on the Windows or Linux server?  If so you need to capture info into the Perf Table using Log Analytics, i.e the "process" counters for the agent or the server a whole.  You may also want to consider Azure Monitor Metric alerts - as they are near real-time. 

 

However if its delays you are looking for then the Heartbeat (and all tables) provide the ingestion and latency info, Some examples from: https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-ingestion-time  

 

// https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-ingestion-time
Heartbeat
| where TimeGenerated > ago(8h) 
| extend E2EIngestionLatency = ingestion_time() - TimeGenerated 
| extend AgentLatency = _TimeReceived - TimeGenerated 
| summarize percentiles(E2EIngestionLatency,50,95), percentiles(AgentLatency,50,95) by Computer 
| top 20 by percentile_E2EIngestionLatency_95 desc


Heartbeat 
| where TimeGenerated > ago(24h) 
| extend E2EIngestionLatencyMin = todouble(datetime_diff("Second",ingestion_time(),TimeGenerated))/60 
| extend AgentLatencyMin = todouble(datetime_diff("Second",_TimeReceived,TimeGenerated))/60 
| summarize percentiles(E2EIngestionLatencyMin,50,95), percentiles(AgentLatencyMin,50,95) by bin(TimeGenerated,30m) , Computer
| limit 3

note: latency can be caused by many factors 

 

Highlighted

@Clive Watson 

thanks for the reply

 

1. how to i create an alert for a sourcetypes / connector that stopped sending the logs.

2. how do i get the alert done for the the latency? anything that is above the average?

Highlighted

@omrip 

 

1. To Alert you typically need an AggregatedValue, this allows an Azure Monitor Alert to display a value, that the alert can threshold from. 

union withsource = tt *
| where TimeGenerated < now()
| where isnotempty(Type)
| summarize maxTimeGenerated=max(TimeGenerated) by Type
| where maxTimeGenerated  < ago(24h)
| extend SolutionName = strcat(Type, ': LatestData: ', maxTimeGenerated)
| summarize AggregatedValue = count() by SolutionName, maxTimeGenerated

So if I run the above, I would make an Alert by pressing the "Add New Alert Rule"
Please see https://docs.microsoft.com/en-us/azure/azure-monitor/learn/tutorial-response

and also this series of posts (this is Post 7, but start at #1): https://cloudadministrator.net/2019/10/07/azure-monitor-alert-series-part-7/?fbclid=IwAR0pBvGLhqmZFI... 


2. Maybe something like this, please modify to suit. 

Heartbeat
| where TimeGenerated > ago(1h) 
| extend E2EIngestionLatency = ingestion_time() - TimeGenerated 
| extend AgentLatency = _TimeReceived - TimeGenerated 
| summarize avgLatency = avg(AgentLatency) by Computer , E2EIngestionLatency
| where avgLatency > E2EIngestionLatency
| extend avgLatencyBreachedfor = strcat(Computer, ' : ', avgLatency)
| summarize AggregatedValue = count() by avgLatencyBreachedfor