Home

get on-prem connector usage

%3CLINGO-SUB%20id%3D%22lingo-sub-906906%22%20slang%3D%22en-US%22%3Eget%20on-prem%20connector%20usage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-906906%22%20slang%3D%22en-US%22%3E%3CP%3Ei%20would%20like%20to%20get%20on-prem%20connector%20(Linux%20and%20MS)%20utilization%20and%20check%20whether%20it%20is%20on%20high%20load%2C%20how%20do%20i%20create%20an%20alert%20for%20that%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-906906%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3Ekql%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3ESentinel%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-908019%22%20slang%3D%22en-US%22%3ERe%3A%20get%20on-prem%20connector%20usage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-908019%22%20slang%3D%22en-US%22%3E%3CP%3EHello%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F423940%22%20target%3D%22_blank%22%3E%40omrip%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWhen%20you%20say%20high%20load%2C%20do%20you%20mean%20CPU%2FMemory%20on%20the%20Windows%20or%20Linux%20server%3F%26nbsp%3B%20If%20so%20you%20need%20to%20capture%20info%20into%20the%20%3CSTRONG%3EPerf%3C%2FSTRONG%3E%20Table%20using%20Log%20Analytics%2C%20i.e%20the%20%22process%22%20counters%20for%20the%20agent%20or%20the%20server%20a%20whole.%26nbsp%3B%20You%20may%20also%20want%20to%20consider%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-platform-metrics%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20Monitor%20Metric%20alerts%3C%2FA%3E%20-%20as%20they%20are%20near%20real-time.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EHowever%20if%20its%20delays%20you%20are%20looking%20for%20then%20the%20Heartbeat%20(and%20all%20tables)%20provide%20the%20ingestion%20and%20latency%20info%2C%26nbsp%3BSome%20examples%20from%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%3C%2FA%3E%26nbsp%3B%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-markup%22%3E%3CCODE%3E%2F%2F%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fdata-ingestion-time%3C%2FA%3E%0AHeartbeat%0A%7C%20where%20TimeGenerated%20%26gt%3B%20ago(8h)%20%0A%7C%20extend%20E2EIngestionLatency%20%3D%20ingestion_time()%20-%20TimeGenerated%20%0A%7C%20extend%20AgentLatency%20%3D%20_TimeReceived%20-%20TimeGenerated%20%0A%7C%20summarize%20percentiles(E2EIngestionLatency%2C50%2C95)%2C%20percentiles(AgentLatency%2C50%2C95)%20by%20Computer%20%0A%7C%20top%2020%20by%20percentile_E2EIngestionLatency_95%20desc%0A%0A%0AHeartbeat%20%0A%7C%20where%20TimeGenerated%20%26gt%3B%20ago(24h)%20%0A%7C%20extend%20E2EIngestionLatencyMin%20%3D%20todouble(datetime_diff(%22Second%22%2Cingestion_time()%2CTimeGenerated))%2F60%20%0A%7C%20extend%20AgentLatencyMin%20%3D%20todouble(datetime_diff(%22Second%22%2C_TimeReceived%2CTimeGenerated))%2F60%20%0A%7C%20summarize%20percentiles(E2EIngestionLatencyMin%2C50%2C95)%2C%20percentiles(AgentLatencyMin%2C50%2C95)%20by%20bin(TimeGenerated%2C30m)%20%2C%20Computer%0A%7C%20limit%203%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3Enote%3A%20latency%20can%20be%20caused%20by%20many%20factors%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-910458%22%20slang%3D%22en-US%22%3ERe%3A%20get%20on-prem%20connector%20usage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-910458%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F239477%22%20target%3D%22_blank%22%3E%40Clive%20Watson%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ethanks%20for%20the%20reply%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E1.%20how%20to%20i%20create%20an%20alert%20for%20a%20sourcetypes%20%2F%20connector%20that%20stopped%20sending%20the%20logs.%3C%2FP%3E%3CP%3E2.%20how%20do%20i%20get%20the%20alert%20done%20for%20the%20the%20latency%3F%20anything%20that%20is%20above%20the%20average%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-911273%22%20slang%3D%22en-US%22%3ERe%3A%20get%20on-prem%20connector%20usage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-911273%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F423940%22%20target%3D%22_blank%22%3E%40omrip%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E1.%20To%20Alert%20you%20typically%20need%20an%20AggregatedValue%2C%20this%20allows%20an%20Azure%20Monitor%20Alert%20to%20display%20a%20value%2C%20that%20the%20alert%20can%20threshold%20from.%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-markup%22%3E%3CCODE%3Eunion%20withsource%20%3D%20tt%20*%0A%7C%20where%20TimeGenerated%20%26lt%3B%20now()%0A%7C%20where%20isnotempty(Type)%0A%7C%20summarize%20maxTimeGenerated%3Dmax(TimeGenerated)%20by%20Type%0A%7C%20where%20maxTimeGenerated%20%20%26lt%3B%20ago(24h)%0A%7C%20extend%20SolutionName%20%3D%20strcat(Type%2C%20'%3A%20LatestData%3A%20'%2C%20maxTimeGenerated)%0A%7C%20summarize%20AggregatedValue%20%3D%20count()%20by%20SolutionName%2C%20maxTimeGenerated%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3ESo%20if%20I%20run%20the%20above%2C%20I%20would%20make%20an%20Alert%20by%20pressing%20the%20%22Add%20New%20Alert%20Rule%22%3CBR%20%2F%3EPlease%20see%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Flearn%2Ftutorial-response%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Flearn%2Ftutorial-response%3C%2FA%3E%3C%2FP%3E%0A%3CP%3Eand%20also%20this%20series%20of%20posts%20(this%20is%20Post%207%2C%20but%20start%20at%20%231)%3A%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fcloudadministrator.net%2F2019%2F10%2F07%2Fazure-monitor-alert-series-part-7%2F%3Ffbclid%3DIwAR0pBvGLhqmZFI5eYkrbTwmWdeWlgQpd6zBSA7NPoIJ_KkouuxX9SCdXCpY%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehttps%3A%2F%2Fcloudadministrator.net%2F2019%2F10%2F07%2Fazure-monitor-alert-series-part-7%2F%3Ffbclid%3DIwAR0pBvGLhqmZFI5eYkrbTwmWdeWlgQpd6zBSA7NPoIJ_KkouuxX9SCdXCpY%3C%2FA%3E%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%3E2.%20Maybe%20something%20like%20this%2C%20please%20modify%20to%20suit.%26nbsp%3B%3C%2FP%3E%0A%3CPRE%20class%3D%22lia-code-sample%20language-markup%22%3E%3CCODE%3EHeartbeat%0A%7C%20where%20TimeGenerated%20%26gt%3B%20ago(1h)%20%0A%7C%20extend%20E2EIngestionLatency%20%3D%20ingestion_time()%20-%20TimeGenerated%20%0A%7C%20extend%20AgentLatency%20%3D%20_TimeReceived%20-%20TimeGenerated%20%0A%7C%20summarize%20avgLatency%20%3D%20avg(AgentLatency)%20by%20Computer%20%2C%20E2EIngestionLatency%0A%7C%20where%20avgLatency%20%26gt%3B%20E2EIngestionLatency%0A%7C%20extend%20avgLatencyBreachedfor%20%3D%20strcat(Computer%2C%20'%20%3A%20'%2C%20avgLatency)%0A%7C%20summarize%20AggregatedValue%20%3D%20count()%20by%20avgLatencyBreachedfor%3C%2FCODE%3E%3C%2FPRE%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
omrip
Occasional Contributor

i would like to get on-prem connector (Linux and MS) utilization and check whether it is on high load, how do i create an alert for that?

3 Replies

Hello @omrip 

 

When you say high load, do you mean CPU/Memory on the Windows or Linux server?  If so you need to capture info into the Perf Table using Log Analytics, i.e the "process" counters for the agent or the server a whole.  You may also want to consider Azure Monitor Metric alerts - as they are near real-time. 

 

However if its delays you are looking for then the Heartbeat (and all tables) provide the ingestion and latency info, Some examples from: https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-ingestion-time  

 

// https://docs.microsoft.com/en-us/azure/azure-monitor/platform/data-ingestion-time
Heartbeat
| where TimeGenerated > ago(8h) 
| extend E2EIngestionLatency = ingestion_time() - TimeGenerated 
| extend AgentLatency = _TimeReceived - TimeGenerated 
| summarize percentiles(E2EIngestionLatency,50,95), percentiles(AgentLatency,50,95) by Computer 
| top 20 by percentile_E2EIngestionLatency_95 desc


Heartbeat 
| where TimeGenerated > ago(24h) 
| extend E2EIngestionLatencyMin = todouble(datetime_diff("Second",ingestion_time(),TimeGenerated))/60 
| extend AgentLatencyMin = todouble(datetime_diff("Second",_TimeReceived,TimeGenerated))/60 
| summarize percentiles(E2EIngestionLatencyMin,50,95), percentiles(AgentLatencyMin,50,95) by bin(TimeGenerated,30m) , Computer
| limit 3

note: latency can be caused by many factors 

 

@Clive Watson 

thanks for the reply

 

1. how to i create an alert for a sourcetypes / connector that stopped sending the logs.

2. how do i get the alert done for the the latency? anything that is above the average?

@omrip 

 

1. To Alert you typically need an AggregatedValue, this allows an Azure Monitor Alert to display a value, that the alert can threshold from. 

union withsource = tt *
| where TimeGenerated < now()
| where isnotempty(Type)
| summarize maxTimeGenerated=max(TimeGenerated) by Type
| where maxTimeGenerated  < ago(24h)
| extend SolutionName = strcat(Type, ': LatestData: ', maxTimeGenerated)
| summarize AggregatedValue = count() by SolutionName, maxTimeGenerated

So if I run the above, I would make an Alert by pressing the "Add New Alert Rule"
Please see https://docs.microsoft.com/en-us/azure/azure-monitor/learn/tutorial-response

and also this series of posts (this is Post 7, but start at #1): https://cloudadministrator.net/2019/10/07/azure-monitor-alert-series-part-7/?fbclid=IwAR0pBvGLhqmZFI... 


2. Maybe something like this, please modify to suit. 

Heartbeat
| where TimeGenerated > ago(1h) 
| extend E2EIngestionLatency = ingestion_time() - TimeGenerated 
| extend AgentLatency = _TimeReceived - TimeGenerated 
| summarize avgLatency = avg(AgentLatency) by Computer , E2EIngestionLatency
| where avgLatency > E2EIngestionLatency
| extend avgLatencyBreachedfor = strcat(Computer, ' : ', avgLatency)
| summarize AggregatedValue = count() by avgLatencyBreachedfor

 

Related Conversations
Tabs and Dark Mode
cjc2112 in Discussions on
46 Replies
Extentions Synchronization
Deleted in Discussions on
3 Replies
Stable version of Edge insider browser
HotCakeX in Discussions on
35 Replies
flashing a white screen while open new tab
Deleted in Discussions on
14 Replies
How to Prevent Teams from Auto-Launch
chenrylee in Microsoft Teams on
29 Replies
Security Community Webinars
Valon_Kolica in Security, Privacy & Compliance on
13 Replies