Supply Chain Anomaly Detection and Root Cause Analysis with Azure Metric Advisor
Authors: Tony Xing and Neta Haiby
The COVID-19 pandemic has disrupted supply chains globally. Since 2020 there has been a significant increase in online shopping, home improvements, home entertainment and more but supply chains are struggling to keep up with the demand and we are seeing out of stock items, shipping delays, lost items and more. According to economist John Rutledge, the Covid-19 pandemic will continue to inflict pain on the global supply chain across all sectors, well into 2023 (source: CNBC link).
At the same time, supply chain managers are under constant pressure to improve the process and reduce the decision cycle times. The supply chain generates a huge amount of data, making it difficult to reduce the decision time and to improve the process. A typical supply chain is responsible for thousands of transactions on a global scale every day and must be able to adapt to the constant pressures of change, both internal and external. The overload of data can lead to a high signal-to-noise ratio, so supply chain managers struggle to discern useful signals for meaningful decisions. Anomaly detection can assist these managers in analyzing the data quickly to accurately spot unexpected behaviors such as identifying issues in orders, fulfillment, inventory, shipping, and more.
To maintain a resilient supply chain and avoid dissatisfied customers, companies must constantly monitor the supply chain and detect anomalies. An anomaly can be defined as a significant change in the expected value or variability. Anomaly detection is an instance where AI can help by detecting the anomaly, notifying both the business process and planning team, providing the root cause and either automatically fixing the issue or enabling the supply managers to make better informed decisions. The following are a few examples of types of supply chain anomalies:
Azure Metrics Advisor is an Applied AI Service that monitors metrics and diagnoses issues and is designed to protect your organization’s growth. Embed AI-powered monitoring features to stay one step ahead of incidents—no machine learning expertise required. Monitor the performance of your organization's growth engines, including sales revenue and manufacturing operations, with Azure Metrics Advisor, built on Anomaly Detector that is part of Azure Cognitive Services.
It is very important to map the problem you want to solve to the dataset that Azure Metrics Advisor (AMA) can use to achieve the goal of mentioned AD (anomaly detection) and RCA (root cause analysis). And you would need to get yourself familiar with the data schema which enables the multi-dimensional AD and RCA.
Let’s take the order quantity anomaly detection for example. Suppose you are the supply chain manager of an e-commerce multinational company that sells many products to all major cities across the world. The problem you want to address is the pain of firefighting in the scenario that demand for some unique items with relatively small quantities sold is on the rise. These long tail items suddenly pick up sales and quickly stocks out. To stay one step ahead, you would like to get near real-time anomaly detection to identify which unique SKUs to ship to which city from which fulfillment center is on the rise, and then get your supply chain team to respond immediately to ensure optimization of the inventory and great customer satisfaction.
This is an incredibly hard to problem to solve manually, but with Azure Metrics Advisor you can get insights automatically if you structure your dataset this way:
Now you have dataset with aggregation measurement of “order quantity” every hour segmented by product SKU, Ship to, Fulfillment center. As you can see, this multi-dimensional dataset can be split into millions of unique time series, each representing a unique combination of (SKU, Ship To, Fulfillment Center). Now this dataset is ready for AMA to consume, users will receive automated insights on each time series through email, Teams, or web hooks.
You might wonder, “How I can get such a dataset from my raw transaction record?” You can refer to the “High level solution architecture” section of the blog to use Azure Databricks or any other streaming processing framework you choose to cook the raw data into the dataset above.
Refer to Azure Metrics Advisor documentation Onboard your data feed to Metrics Advisor - Azure Cognitive Services | Microsoft Docs to onboard data you have prepared. Onboarding is basically a simple step to register parameters on what is the data source, how to connect, how to filter and aggregate, what is the ingestion granularity (frequency). This can be done in a few minutes if you have the parameters handy.
There are two ways to customize the anomaly detection result on multi-dimensional dataset so that the detection result can be more accurate as it adapts to your business logics and datasets:
For more details, refer to Configure your Metrics Advisor instance using the web portal - Azure Cognitive Services | Microsoft ...
Stakeholders can get detection results via communication channels set up by the user, along with the anomaly detected and root cause analysis.
The incident tree will tell users which specific dimensions or combination of dimensions contributes to the sudden spike detected. For more information on how to read this, refer to Diagnose an incident using Metrics Advisor - Azure Cognitive Services | Microsoft Docs
For more details, refer to Diagnose an incident using Metrics Advisor - Azure Cognitive Services | Microsoft Docs
Azure Metrics Advisor can be used through either web based workspace or SDK, refer to Metrics Advisor client libraries REST API - Azure Cognitive Services | Microsoft Docs.
You can refer to the solution architecture to achieve the E2E scenario described above. The first step is data preparation we talked about in the earlier section ”data preparation,” to do that, you will need streaming processing framework like Azure Data Bricks to transform the raw data from the supply chain system into the required schema. The cooked dataset then is stored in data stores like Azure Data Lake or Cosmos DB. Then Azure Metrics Advisor can connect directly to those data stores and leverage the dataset for model building and parameter tuning for anomaly detection and root cause analysis. Last but not least, those results can be available for visualization through Power BI or notification to users via emails or other channels.
Go to Azure portal to create your new Metrics Advisor resource here. You can also read the Metrics Advisor document to learn more about the service capabilities.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.