Azure Log Analytics Log Management using Azure Data Explorer

[Tzvia]

One common questions that we get from customers and partners is how to ingest data from Azure Log Analytics / Azure Sentinel to Azure Data Explorer from different reasons: joining data between different ADX clusters, longer retention period, heavy queries that aren't alight with LA limitations and etc.

There are many good articles in the web, here is a step by step guide and code from Sentinel team at GitHub for automation script to integrate Azure Data Explorer for Long term storage option for Azure Sentinel Log Analytics Workspace

https://github.com/Azure/Azure-Sentinel/tree/master/Tools/AzureDataExplorer

1. Asking input from the user
1. Do you want all the tables from the LA? Yes or No
2. If Yes, will get all tables
3. If No, User will enter table names (,) separated
2. Before creating TableRAW and TableRAWMapping, checking against fully supported tables 
   https://docs.microsoft.com/en-us/azure/azure-monitor/logs/logs-data-export?tabs=rest#supported-table...
3. Dividing the tables into Size 10 and then creating Standard EventHub Namespaces programmatically for each 10 tables
4. Creating “Data Export” rule programmatically using REST API
5. Creating “Data Connection” rule in Azure Data Explorer Database programmatically using REST API
6. Creating Log file to verify what went successfully vs wrong

Azure Log Analytics Log Management using Azure Data Explorer by Sreedhar Ande

Additional resources:

• Blog: https://techcommunity.microsoft.com/t5/azure-sentinel/using-azure-data-explorer-for-long-term-retent...
• Official documentation: https://docs.microsoft.com/en-us/azure/sentinel/store-logs-in-azure-data-explorer?tabs=adx-event-hub
• MSSP Architecture Reference (page 23-24): MSSP Playbook
• Script to provision ADX for Sentinel long-retention: https://github.com/sreedharande/AzureDataExplorer