This blog describes Azure Monitor's built-in machine learning capabilities and explains how you can create and run customized machine learning models on data in Azure Monitor Logs.
Built-in Azure Monitor machine learning capabilities
Azure Monitor's built-in ML capabilities provide insights and automate data-driven tasks, such as predicting capacity usage and autoscaling, identifying and analyzing application performance issues, and detecting anomalous behaviors in virtual machines, containers, and other resources. These features boost your IT monitoring and operations, without requiring machine learning knowledge and further investment.
Use the Kusto Query Language's built-in time series analysis and machine learning functions, operators, and plug-ins to gain insights about service health, usage, capacity and other trends, and to generate forecasts and detect anomalies in Azure Monitor Logs.
Create your own machine learning pipeline on data in Azure Monitor Logs
While Azure Monitor’s built-in ML capabilities are very powerful, you can also build your own machine learning pipeline on data in Azure Monitor Logs to broaden your ability to analyze data and support advanced scenarios, such as:
- Hunting for security attacks or troubleshooting errors in a web application with more sophisticated models than those natively supported by KQL.
- Automating the analysis of Azure Monitor Log data and providing insights into multiple areas.
- Correlating data in Azure Monitor Logs with data from other sources.
There are two approaches to making data in Azure Monitor Logs available to your machine learning pipeline:
- Query data in Azure Monitor Logs - Integrate a notebook with Azure Monitor Logs or run a script on log data using libraries like Azure Monitor Query client library.
- Export data out of Azure Monitor Logs - Export data, usually to a blob storage account. For example, see Analyze data exported from Log Analytics using Synapse.
Once data is retrieved, you can implement your machine learning pipeline using a machine learning library, tool, or service of your choice.
Summary
A machine learning pipeline typically involves various steps, like data exploration, training the model on historical data, deployment, and scoring the model on new data. To benefit from the best of both approaches to making data in Azure Monitor Logs available to your machine learning pipeline, you can use a different approach for each step.
Microsoft