Microsoft Secure Tech Accelerator
Apr 03 2024, 07:00 AM - 11:00 AM (PDT)
Microsoft Tech Community

Limit what data in Log Analytics to be passed on to Sentinel?

Brass Contributor

We have been using Sentinel in conjunction with Azure Log Analytics for quite some time to ingest selected security logs (AD, DNS, Windows Security etc.) from VM-agents in our server environment. Last week we upgraded the workspace to enable the newly released "Azure Monitor for VMs" and also installed the Service Map agents on our VMS. This resulted in a huge increase of data ingested into Sentinel (5x, see attached image) due to VM performance metrics and network traffic logs from the the Service Map now being ingested into Sentinel, that I have no interest in having there.

How can I choose/filter what logs from a Log Analytics Workspace that is forwarded to Azure Sentinel?

8 Replies

@Magnus Tjerneld You cannot state which information stored in a Log Analytics workspace is available to Azure Sentinel.  It is an all or nothing proposition.

 

You can control what data gets ingested from the VMs by clicking on the "Windows, Linux, and other sources" link on the Overview page and from there go to the "Data" section.  This will show you what logs you are ingesting as well as severity for each log.  You can go through the list and see if you can trim down some of the information being imported.

 

 

Ok; that's too bad. I dug a little deeper into the increased log volumes and realized that the bulk of the records/sources are not visible under Data Connections. 

 

Previously; before we upgraded the Workspace to onboard "Azure Monitor for VM:s" I was manually pulling a number of selected perfmon counters (CPU, RAM and so on). I did this at a 5 min interval to give me an overview without causing too much data in OMS. But after enabling Azure Monitor for VM:s, a huge number of new counters has been enabled and the resolution is much higher. Over the last 24h I've recieved ~300k datapoints for the 4 VMs I'm monitoring:

 Computer11,499 
 LogicalDisk205,315 
 Memory5,744 
 Processor5,744 
 Network14,360


None of these namespaces are visible under Data Sources in Log Analytics.

It would be great to be able to lower the resolution/interval, but I can't seem to find anywhere to limit the interval on these logs?

I realize that this is now more of an "Azure Monitor for VM:s" question than a Sentinel question.

@Magnus Tjerneld Did you check under the "Windows Performance Counters" in the "Data" section?

@Gary Bushey Yes; only the manual counters that I had set up before I enabled "Azure monitor for VMs" are visible there. AMFVM seems to set up it's own data collection that you don't seem to be able to edit.

Thanks @CliveWatson. I read this before and have now read it again; and I realize that I can delete my old perfmon counters. However, I do not find any information regarding:

- Can I limit the "resolution" of data performance data sent to Log Analytics after upgrading to Azure Monitor for VMs? In the old solution, I could set intervals in seconds.

- Can I choose not to collect data for a specific namespace? For us, Disk-metrics make up 90% of logs ingested and causes a lot of extra costs in Sentinel. If possible, I'd opt out.

 

And my wish would be to be able to exclude all performance counters from ingestion into Sentinel. This only results in added cost and no added value. 

@Magnus Tjerneld Were you able to find a solution to this? Filtering out which data that is ingested by sentinel?

@williamboyd We resorted to using separate workspaces for VM/Performance logs (not piped to Sentinel) and Security/Signin-logs (piped to Sentinel).