SOLVED
Home

Calculating performance counters memory and network usage as percentage

%3CLINGO-SUB%20id%3D%22lingo-sub-212499%22%20slang%3D%22en-US%22%3ECalculating%20performance%20counters%20memory%20and%20network%20usage%20as%20percentage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-212499%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20everyone%2C%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ECalculating%20memory%20and%20network%20usage%20as%20a%20percentage%20of%20the%20resource%20is%20a%20classic%20problem%20for%20Windows%20performance%20counters.%20The%20issue%20is%20that%20the%20available%20counters%20are%20absolute%20numbers%2C%20but%20you%20don't%20necessarily%20know%20the%20total%20for%20these%20resources%20on%20a%20given%20machine.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EHas%20anyone%20come%20up%20with%20a%20clever%20way%20to%20do%20this%20through%20in%20OMS%20and%20log%20analytics%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EFor%20reference%2C%20here%20is%20a%20query%20I'm%20using%20to%20get%20the%20average%20for%20Bytes%20Total%2Fsec%3A%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CDIV%3E%3CDIV%3E%3CSPAN%3EPerf%3C%2FSPAN%3E%3C%2FDIV%3E%3CDIV%3E%3CSPAN%3E%7C%20%3C%2FSPAN%3E%3CSPAN%3Ewhere%3C%2FSPAN%3E%3CSPAN%3E%20ObjectName%20%3D%3D%20%3C%2FSPAN%3E%3CSPAN%3E%22Network%20Interface%22%3C%2FSPAN%3E%20%3CSPAN%3Eand%3C%2FSPAN%3E%3CSPAN%3E%20CounterName%20%3D%3D%20%3C%2FSPAN%3E%3CSPAN%3E%22Bytes%20Total%2Fsec%22%3C%2FSPAN%3E%3C%2FDIV%3E%3CDIV%3E%3CSPAN%3E%7C%20%3C%2FSPAN%3E%3CSPAN%3Ewhere%3C%2FSPAN%3E%3CSPAN%3E%20TimeGenerated%20%26gt%3B%20startofday%3C%2FSPAN%3E%3CSPAN%3E(%3C%2FSPAN%3E%3CSPAN%3Enow%3C%2FSPAN%3E%3CSPAN%3E()%3C%2FSPAN%3E%3CSPAN%3E-%3C%2FSPAN%3E%3CSPAN%3E22%3C%2FSPAN%3E%3CSPAN%3Ed%3C%2FSPAN%3E%3CSPAN%3E)%3C%2FSPAN%3E%20%3CSPAN%3Eand%3C%2FSPAN%3E%3CSPAN%3E%20TimeGenerated%20%26lt%3B%20endofday%3C%2FSPAN%3E%3CSPAN%3E(%3C%2FSPAN%3E%3CSPAN%3Enow%3C%2FSPAN%3E%3CSPAN%3E()%3C%2FSPAN%3E%3CSPAN%3E-%3C%2FSPAN%3E%3CSPAN%3E1%3C%2FSPAN%3E%3CSPAN%3Ed%3C%2FSPAN%3E%3CSPAN%3E)%3C%2FSPAN%3E%3C%2FDIV%3E%3CDIV%3E%3CSPAN%3E%7C%20%3C%2FSPAN%3E%3CSPAN%3Esummarize%3C%2FSPAN%3E%3CSPAN%3E%20AggregatedValue%20%3D%20avg%3C%2FSPAN%3E%3CSPAN%3E(%3C%2FSPAN%3E%3CSPAN%3ECounterValue%3C%2FSPAN%3E%3CSPAN%3E)%3C%2FSPAN%3E%20%3CSPAN%3Eby%3C%2FSPAN%3E%3CSPAN%3E%20Computer%3C%2FSPAN%3E%3CSPAN%3E%2C%3C%2FSPAN%3E%3CSPAN%3E%20InstanceName%20%3C%2FSPAN%3E%3CSPAN%3E%2C%3C%2FSPAN%3E%3CSPAN%3E%20bin%3C%2FSPAN%3E%3CSPAN%3E(%3C%2FSPAN%3E%3CSPAN%3ETimeGenerated%3C%2FSPAN%3E%3CSPAN%3E%2C%3C%2FSPAN%3E%20%3CSPAN%3E1%3C%2FSPAN%3E%3CSPAN%3Ehour%3C%2FSPAN%3E%3CSPAN%3E)%3C%2FSPAN%3E%3C%2FDIV%3E%3C%2FDIV%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-LABS%20id%3D%22lingo-labs-212499%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Log%20Analytics%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EPerformance%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-214603%22%20slang%3D%22en-US%22%3ERe%3A%20Calculating%20performance%20counters%20memory%20and%20network%20usage%20as%20percentage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-214603%22%20slang%3D%22en-US%22%3E%3CP%3EThanks%20Billy%20and%20Noa%2C%20this%20was%20very%20helpful.%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-213138%22%20slang%3D%22en-US%22%3ERe%3A%20Calculating%20performance%20counters%20memory%20and%20network%20usage%20as%20percentage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-213138%22%20slang%3D%22en-US%22%3E%3CP%3EI%20think%20Billy's%20reply%20is%20the%20best%20response.%20Tried%20to%20%22prettify%22%20it%3A%3C%2FP%3E%0A%3CPRE%3Elet%20free_memory_percent%20%3D%20Perf%0A%7C%20where%20CounterName%20%3D%3D%20%22%25%20Committed%20Bytes%20In%20Use%22%0A%7C%20summarize%20arg_max(TimeGenerated%2C%20CounterValue)%20by%20Computer%0A%7C%20project%20TimeGenerated%2C%20Computer%2C%20free_memory_percent%3D100-CounterValue%3B%0A%0Alet%20free_memory_gigs%20%3D%20Perf%0A%7C%20where%20CounterName%20%3D%3D%20%22Available%20MBytes%22%0A%7C%20summarize%20arg_max(TimeGenerated%2C%20CounterValue)%20by%20Computer%0A%7C%20project%20TimeGenerated%2C%20Computer%2C%20free_memory_GB%3DCounterValue%2F1024%3B%0A%0Afree_memory_gigs%20%0A%7C%20join%20kind%3D%20innerunique(free_memory_percent)%20on%20Computer%0A%7C%20where%20time_diff%3Dabs((TimeGenerated-TimeGenerated1)%2F1m)%26lt%3B1%0A%7C%20project%20TimeGenerated%2C%20Computer%2C%20free_memory_percent%2C%20free_memory_GB%2C%20Total_GB%3D(free_memory_GB*100)%2Ffree_memory_percent%3C%2FPRE%3E%0A%3CP%3EA%20link%20to%20the%20example%20on%20the%20demo%20env%20%3CA%20href%3D%22https%3A%2F%2Fportal.loganalytics.io%2Fsubscriptions%2Fe4272367-5645-4c4e-9c67-3b74b59a6982%2Fresourcegroups%2FContosoAzureHQ%2Fworkspaces%2Fcontosoretail-IT%3Fq%3DH4sIAAAAAAAAA72RPU%252FDMBRF90j5D0%252BVkGKUKgliKx4oQ8UAYiiskdO8BBd%252FBH8AQfx4nFZITZsBMTD62ffoHj%252BBDhqDWEqU2vRlh2aDygGFOHpA08TRF7w%252Fo0G40V45NPdMIlAKs7MwkZI7hzUse4cWbhU8WpwNEeulZIZ%252FIjDTlpJ9JGsucYUKDQuB9If2xIRHAlU%252FwDofJkO6M3qLGwcnmf2TdKoxLfJ8fkhdxFEciSO9lrf2F27Xb4wLVgmEu53afzutlvQQlhX5xeXO58RlQG81V%252FDCVU2Bq0D2ir96TCY%252BiYBWo1Z7fRc6lTVvGsoqm4y15qNTQbJCkqvir1s61kxhrR0Tg3AyvjkP%252ByTZBOIb18MphrICAAA%253D%22%20target%3D%22_blank%22%20rel%3D%22nofollow%20noopener%20noreferrer%20noopener%20noreferrer%22%3Ehere%3C%2FA%3E.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-212815%22%20slang%3D%22en-US%22%3ERe%3A%20Calculating%20performance%20counters%20memory%20and%20network%20usage%20as%20percentage%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-212815%22%20slang%3D%22en-US%22%3E%3CP%3ESo%20this%20will%20get%20you%20really%20close%20on%20the%20memory%2C%20however%20it%20would%20be%20much%20easier%20if%20the%20log%20analytics%20just%20collected%20this%20type%20of%20hardware%20inventory%20data.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Eits%20not%20quite%20100%25%20for%20instance%20on%20my%20hyperv%20server%20its%20~4gb%20off%2C%20but%20on%20another%20physical%20server%20it%20was%20less%20than%20.5%20gb%20off.%20Perhaps%20it%20will%20get%20you%20started%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Elet%20Convert%20%3D%20Perf%3CBR%20%2F%3E%7C%20where%20CounterName%20%3D%3D%20%22%25%20Committed%20Bytes%20In%20Use%22%3CBR%20%2F%3E%7C%20extend%20Perc%20%3D%20100%20-%20CounterValue%3B%3C%2FP%3E%3CP%3Elet%20Gigs%20%3D%20Perf%3CBR%20%2F%3E%7C%20where%20CounterName%20%3D%3D%20%22Available%20MBytes%22%3CBR%20%2F%3E%7C%20extend%20available_GB%20%3D%20CounterValue%20%2F%201024%3B%3C%2FP%3E%3CP%3EGigs%20%7C%20join%20(Convert)%20on%20Computer%3CBR%20%2F%3E%7C%20project%20Computer%2C%20available_GB%2C%20Perc%2C%20CounterName1%2C%20CounterValue1%3CBR%20%2F%3E%7C%20extend%20Total_GB%20%3D%20(available_GB%20*%20100)%20%2F%20Perc%3C%2FP%3E%3C%2FLINGO-BODY%3E
Samuel White
Occasional Contributor

Hi everyone,

 

Calculating memory and network usage as a percentage of the resource is a classic problem for Windows performance counters. The issue is that the available counters are absolute numbers, but you don't necessarily know the total for these resources on a given machine.

 

Has anyone come up with a clever way to do this through in OMS and log analytics?

 

For reference, here is a query I'm using to get the average for Bytes Total/sec:

 

Perf
| where ObjectName == "Network Interface" and CounterName == "Bytes Total/sec"
| where TimeGenerated > startofday(now()-22d) and TimeGenerated < endofday(now()-1d)
| summarize AggregatedValue = avg(CounterValue) by Computer, InstanceName , bin(TimeGenerated, 1hour)

 

 

3 Replies
Solution

So this will get you really close on the memory, however it would be much easier if the log analytics just collected this type of hardware inventory data. 

 

its not quite 100% for instance on my hyperv server its ~4gb off, but on another physical server it was less than .5 gb off. Perhaps it will get you started?

 

let Convert = Perf
| where CounterName == "% Committed Bytes In Use"
| extend Perc = 100 - CounterValue;

let Gigs = Perf
| where CounterName == "Available MBytes"
| extend available_GB = CounterValue / 1024;

Gigs | join (Convert) on Computer
| project Computer, available_GB, Perc, CounterName1, CounterValue1
| extend Total_GB = (available_GB * 100) / Perc

I think Billy's reply is the best response. Tried to "prettify" it:

let free_memory_percent = Perf
| where CounterName == "% Committed Bytes In Use"
| summarize arg_max(TimeGenerated, CounterValue) by Computer
| project TimeGenerated, Computer, free_memory_percent=100-CounterValue;

let free_memory_gigs = Perf
| where CounterName == "Available MBytes"
| summarize arg_max(TimeGenerated, CounterValue) by Computer
| project TimeGenerated, Computer, free_memory_GB=CounterValue/1024;

free_memory_gigs 
| join kind= innerunique(free_memory_percent) on Computer
| where time_diff=abs((TimeGenerated-TimeGenerated1)/1m)<1
| project TimeGenerated, Computer, free_memory_percent, free_memory_GB, Total_GB=(free_memory_GB*100)/free_memory_percent

A link to the example on the demo env here.

Thanks Billy and Noa, this was very helpful.