Event banner
Microsoft Sentinel Data Tiering Best Practices - Tech Community Live!
Event Ended
Tuesday, Oct 29, 2024, 09:00 AM PDTEvent details
Discover the power of the new Auxiliary logs tier (Public Preview) and learn how to use Summary rules (Public Preview) to summarize data from any log tier in Microsoft Sentinel and Log Analytics. We’...
TrevorRusher
Updated Dec 27, 2024
UnipartnerAssociate
Oct 29, 2024Copper Contributor
Hi everyone, this auxiliary logs is really useful for many clients, but I still have some questions:
1. - Can we use the CommonSecurityLog table and transition it to auxiliary logs? Since this is the table used to ingest logs from firewall, network devices and so on on the CEF format (It was said this isn't supported now, will it be?) - This is the main way I see my clients ingestion firewall logs to Sentinel
2.- How does the ingestion work? Can we still use the same AMA agent and just have a different DCR? (Seems like CEF is still not available), can you provide an example on how to ingest the logs then?
3. - How exactly does the summary rules work? Once a day, hourly?, if the same summary rule has equal data on the CustomTable does it overwrite it and only save the latest timestamp?
- CHARBELNEMNOMOct 29, 2024Copper Contributor
For the question 1 and 2) - You can ingest CEF (Firewall) logs directly to a new custom table (i.e., CommonSecurityLog_CL) using DCR ingestion-time transformation once transformation becomes supported. The custom table (i.e., CommonSecurityLog_CL) has the Auxiliary Tier plan enabled. Then, you create a Summary Rule, which you can schedule to run every couple of hours or day on the (i.e., CommonSecurityLog_CL), then send the summary logs to another custom table with the Analytic tier plan. Check Summary Rule documentation: https://learn.microsoft.com/en-us/azure/sentinel/summary-rules#create-a-summary-rule
- Matt_LoweOct 29, 2024
Microsoft
At the moment CommonSecurityLog would have to directly go to the aux logs table. This will be addressed in the future to allow ingestion time transformation to split the data between the analytics tier table and the aux table. The recommendation would be to filter out less valuable data from the analytics table to be sent to the aux table via DCR based table splitting. Regarding summary rules, you can determine how often that it runs but the minimum limit is every 20 minutes. We have gotten feedback regarding lowering the time but no further details at this time. For limits, please see: - https://learn.microsoft.com/en-us/azure/azure-monitor/logs/create-custom-table-auxiliary#public-preview-limitations - https://learn.microsoft.com/en-us/azure/azure-monitor/logs/summary-rules?tabs=api#create-or-update-a-summary-rule