Azure Monitor for WVD generates a lot of data in Log Analytic Workspace

%3CLINGO-SUB%20id%3D%22lingo-sub-2015090%22%20slang%3D%22en-US%22%3EAzure%20Monitor%20for%20WVD%20generates%20a%20lot%20of%20data%20in%20Log%20Analytic%20Workspace%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2015090%22%20slang%3D%22en-US%22%3E%3CP%3EHi%3C%2FP%3E%3CP%3EI%20enabled%20last%20weekend%20the%20Azure%20monitor%20for%20our%20WVD%20cluster%20with%20two%20session%20hosts.%20the%20reporting%20is%20really%20awesome%20and%20very%20helpful%20to%20become%20an%20overview%20about%20the%20system.%3C%2FP%3E%3CP%3EBut%20in%20just%20one%20day%20it%20generated%205GB%20of%20logs%20for%20just%20two%20session%20hosts.%20In%20the%20pay-as-you%20go%20price%20model%20we%20needed%20to%20pay%2016%24%20for%20this%20single%20day.%20In%20the%20end%20of%20a%20month%20there%20will%20be%20costs%20around%20500%24.%20That%20is%20almost%20as%20much%20as%20we%20pay%20for%20both%20session%20hosts.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EIs%20there%20a%20way%20to%20reduce%20the%20amount%20of%20collected%20data%3F%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EThanks%3C%2FP%3E%3CP%3EJ%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2015894%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Monitor%20for%20WVD%20generates%20a%20lot%20of%20data%20in%20Log%20Analytic%20Workspace%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2015894%22%20slang%3D%22en-US%22%3E%3CP%3EHi%20J%C3%BCrg%2C%20please%20have%20a%20look%20at%20this%20article%3A%3C%2FP%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fmanage-cost-storage%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fmanage-cost-storage%3C%2FA%3E%3C%2FP%3E%3CP%3EThere%20is%20some%20info%20on%20how%20to%20adjust%20the%20volume.%20Regards%2C%20Tom%20Wechsler%3C%2FP%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F663696%22%20target%3D%22_blank%22%3E%40breakplease%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2016011%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Monitor%20for%20WVD%20generates%20a%20lot%20of%20data%20in%20Log%20Analytic%20Workspace%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2016011%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F663696%22%20target%3D%22_blank%22%3E%40breakplease%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3ETo%20be%20honest%20mine%20cost%20nothing%20(Beside%20alerts%20costs)%2C%20but%20I%20log%20only%20the%20critical%20stuff%20that%20I%20want.%3C%2FP%3E%3CP%3ENote%20that%2C%20It%20might%20lead%20to%20empty%20charts%20from%20the%20Microsoft%20exemple.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EIt%20really%20come%20down%20to%20what%2C%20how%20often%20and%20the%20data%20retention%20that%20will%20grow%20your%20analytic%20size.%3C%2FP%3E%3CP%3ELike%20Tom%20said%2C%20I%20strongly%20suggest%20that%20you%20put%20a%20datacap%20per%20day%20of%20what%20you%20want%20to%20pay.%3C%2FP%3E%3CP%3EAfter%20that%2C%20if%20you%20miss%20some%20data%2C%20now%20it%20is%20the%20time%20to%20thinking%20about%20stop%20logging%20stuff%20that%20you%20don't%20need.%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20made%20a%20query%20for%20you%20where%20you%20can%20pin%20point%2C%20which%20table%20is%20pumping%20so%20much%20data.%3C%2FP%3E%3CP%3E(It%20will%20probably%20be%20a%20VM%20monitoring%20that%20you%20will%20have%20to%20tune%20down%20or%20remove%20some%20counters.)%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CDIV%3E%3CDIV%3E%3CSPAN%3EUsage%3C%2FSPAN%3E%3C%2FDIV%3E%3CDIV%3E%3CSPAN%3E%7C%26nbsp%3B%3C%2FSPAN%3E%3CSPAN%3Ewhere%3C%2FSPAN%3E%3CSPAN%3E%26nbsp%3BIsBillable%26nbsp%3B%3D%3D%26nbsp%3B%3C%2FSPAN%3E%3CSPAN%3Etrue%3C%2FSPAN%3E%3C%2FDIV%3E%3CDIV%3E%3CSPAN%3E%7C%26nbsp%3B%3C%2FSPAN%3E%3CSPAN%3Esummarize%3C%2FSPAN%3E%3CSPAN%3E%26nbsp%3BTotalVolumeGB%26nbsp%3B%3D%26nbsp%3Bsum%3C%2FSPAN%3E%3CSPAN%3E(%3C%2FSPAN%3E%3CSPAN%3EQuantity%3C%2FSPAN%3E%3CSPAN%3E)%3C%2FSPAN%3E%3CSPAN%3E%26nbsp%3B%2F%26nbsp%3B%3C%2FSPAN%3E%3CSPAN%3E1000%3C%2FSPAN%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CSPAN%3Eby%3C%2FSPAN%3E%3CSPAN%3E%26nbsp%3Bbin%3C%2FSPAN%3E%3CSPAN%3E(%3C%2FSPAN%3E%3CSPAN%3ETimeGenerated%3C%2FSPAN%3E%3CSPAN%3E%2C%3C%2FSPAN%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CSPAN%3E1%3C%2FSPAN%3E%3CSPAN%3Ed%3C%2FSPAN%3E%3CSPAN%3E)%2C%3C%2FSPAN%3E%3CSPAN%3E%26nbsp%3BDataType%3C%2FSPAN%3E%3C%2FDIV%3E%3CDIV%3E%3CSPAN%3E%7C%26nbsp%3B%3C%2FSPAN%3E%3CSPAN%3Erender%3C%2FSPAN%3E%3CSPAN%3E%26nbsp%3Bbarchart%26nbsp%3B%3C%2FSPAN%3E%3C%2FDIV%3E%3C%2FDIV%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2024343%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Monitor%20for%20WVD%20generates%20a%20lot%20of%20data%20in%20Log%20Analytic%20Workspace%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2024343%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F583526%22%20target%3D%22_blank%22%3E%40fmartel%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ehi%20fmartel%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3Ethanks%20for%20your%20post.%20I%20really%20appreciate!%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EI%20used%20your%20query%20and%20saw%20that%20almost%20100%25%20of%20all%20data%20came%20from%20the%20perfromace%20counters.%20I%20changed%20the%20intervall%20and%20deleted%20some.%20Also%20I%20added%20a%20daily%20cap%20for%200.5GB.%26nbsp%3B%3C%2FP%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%3CP%3EHave%20a%20great%20new%20year%E2%80%99s%20eve.%3C%2FP%3E%3CP%3EJ%C3%BCrg%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2024344%22%20slang%3D%22en-US%22%3ERe%3A%20Azure%20Monitor%20for%20WVD%20generates%20a%20lot%20of%20data%20in%20Log%20Analytic%20Workspace%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2024344%22%20slang%3D%22en-US%22%3E%3CP%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fuser%2Fviewprofilepage%2Fuser-id%2F593067%22%20target%3D%22_blank%22%3E%40TomWechsler%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3CP%3ESalut%20Tom.%20Thanks%20for%20your%20answer.%20Looking%20forward%20to%20end%20of%20January%20%3A)%3C%2Fimg%3E%20Happy%20new%20year%20and%20cheers%3C%2FP%3E%3CP%3EJ%C3%BCrg%3C%2FP%3E%3C%2FLINGO-BODY%3E
Occasional Contributor

Hi

I enabled last weekend the Azure monitor for our WVD cluster with two session hosts. the reporting is really awesome and very helpful to become an overview about the system.

But in just one day it generated 5GB of logs for just two session hosts. In the pay-as-you go price model we needed to pay 16$ for this single day. In the end of a month there will be costs around 500$. That is almost as much as we pay for both session hosts.

 

Is there a way to reduce the amount of collected data?

 

Thanks

J

 

5 Replies

Hi Jürg, please have a look at this article:

https://docs.microsoft.com/en-us/azure/azure-monitor/platform/manage-cost-storage

There is some info on how to adjust the volume. Regards, Tom Wechsler

@breakplease 

@breakplease 

 

To be honest mine cost nothing (Beside alerts costs), but I log only the critical stuff that I want.

Note that, It might lead to empty charts from the Microsoft exemple.

 

It really come down to what, how often and the data retention that will grow your analytic size.

Like Tom said, I strongly suggest that you put a datacap per day of what you want to pay.

After that, if you miss some data, now it is the time to thinking about stop logging stuff that you don't need.

 

I made a query for you where you can pin point, which table is pumping so much data.

(It will probably be a VM monitoring that you will have to tune down or remove some counters.)

 

Usage
where IsBillable == true
summarize TotalVolumeGB = sum(Quantity) / 1000 by bin(TimeGenerated, 1d), DataType
render barchart 

@fmartel 

hi fmartel

 

thanks for your post. I really appreciate!

 

I used your query and saw that almost 100% of all data came from the perfromace counters. I changed the intervall and deleted some. Also I added a daily cap for 0.5GB. 

 

Have a great new year’s eve.

Jürg

@TomWechsler 

Salut Tom. Thanks for your answer. Looking forward to end of January :) Happy new year and cheers

Jürg

@breakplease Almost all of our 8GB a day usage comes from the perf counters to. I have added the ones mentioned in the MS setup article i followed. Do you have some suggestions to which can be deleted, and maybe a good and usable intervall for the counters? Thanx :)