Artificial Intelligence is delivering unprecedented efficiencies in manufacturing. From optimizing operations to streamlining the supply chain to improving how employees work. Techniques like Machine Learning (ML) are a major component in most manufacturing transformation projects today.
Applying Machine Learning to almost any problem is getting easier every day, but it requires a disciplined approach to evaluate a use case and build repeatable processes to scale these techniques beyond a single use case. In this article we will discuss an approach on how to evaluate use cases faster and build this repeatable process. Operationalizing machine learning projects require continuous experimentation and multiple components, for this article we will focus on aspects to accelerate ML lifecycle for Industrial IoT projects.
A typical machine learning journey consists of three high level phases, that are executed in loop for continuous experimentation. Each of these phases answers a certain set of questions that are critical to understanding whether machine learning is the right approach to solve specific business problems and if it is, how to go about building and managing machine learning models. Here are some example questions for each phase:
- What is the business case?
- What is the business impact?
- Is the data confidential?
- Is historical data available?
- Is the data relevant?
- Is the data enough?
- Is there a versioned dataset to start exploration?
Exploratory Data Analysis
- Is the data normalized?
- Is the data distribution normal?
- Is there a strong positive or negative correlation between features and target?
- Is there a need to engineer new features?
- Is the data collected at the right frequency?
- Is there a versioned and labeled dataset ready for modeling?
- Is the model objective aligned with business objective?
Model & Data Management
- Is the labeled dataset split evenly between train, test, and validation datasets?
- Is the baseline modeling done with different algorithm(s)?
- Are the model experiments tracked?
- Are the model(s) registered?
- Is the model training automated?
- Is the model training parameterized for multi-model retraining?
Business Actions Mapping
- Does the business action require real-time prediction?
- Does business action require prediction on the edge?
Deployment & Monitoring
- Is the model’s performance monitored?
- Are the predicted results logged, to compare against actuals?
- Is the model(s) deployment automated?
Now let’s see some techniques to answer these questions and start building a machine learning solution with an example business case. All the sample code is available in our industrial-iot-patterns GitHub repo, along with many other samples to help implement IIoT and AI scenarios in Azure.
High Level Potential Solution
To generate the data, you can either follow the given prerequisites or add the SimulatedSensors module in your IoT Edge device. Here is the high-level design:
For IIoT projects we often need to merge the near real-time telemetry data with data from other systems like MES, ERP Quality Management, etc. to build the initial dataset for modeling. Typically, this step may require heavy compute and scalable post processing storage. Hence, we are using Azure ML managed compute to process both the telemetry data directly from Data Explorer and merge that with the Production Quality Data from Data Lake.
Once you have the ML workspace and datastore created, run the 1_create_raw_dataset.ipynb notebook to see the merged dataset in ML Studio. The notebook uses kqlmagic extension to fetch the sensor data directly data explorer.
Exploratory Data Analysis (EDA) is the first step before we build any custom models using machine learning. This is a critical and often complex step wherein we normalize & clean the data, understand data distribution, outliers, correlations and assess the data for various hypothesis and experiments. For our scenario we will do:
Below correlation heat map shows the positive or negative correlation among the sensor values. Ideally, we are looking to understand correlations between our sensors and the output that we are trying to predict, Quality.
Below table shows the list of all the algorithms that we used for our baseline modeling, along with the model evaluation metrics. For our use case we are optimizing for f1-score and recall. See the ML objectives section for more details on how to determine which metric is right for your use case
Continuous experimentation is key to building machine learning models with high accuracy of its predictions. This often requires periodic retraining of your model with new data. It is also important to log each of these retraining experiments to compare model improvement over time and track model versions. So, to perform these experiments we will build an Azure Machine Learning Pipeline. Azure Machine Learning pipeline is an independently executable workflow of a complete machine learning task. An Azure Machine Learning pipeline helps to standardize the best practices of producing a machine learning model, enables the team to execute at scale, and improves the model building efficiency. Follow steps from here to build the pipeline.
The pipeline has two steps:
This endpoint can now be triggered manually or automatically from tools like Synapse Pipelines or Logic Apps.
Since the endpoint uses a parameter to select the sensors for modeling, we can now build multiple sensor models using a single pipeline endpoint.
Each pipeline run job logs all the outputs so it’s easier to track and debug.
Once the model is trained and registered it’s time to deploy and test our models against live data, also known as inferencing. For our scenario we want quality predictions every few minutes as the production cycle takes some time.
So, we built another pipeline to generate predictions. Follow steps from here
It has only one step but multiple parameters to dynamically fetch live data from data explorer and generate the prediction output.
We can then integrate the model prediction pipeline with Synapse Pipelines to automatically trigger on a schedule and store the output predictions in a Database. Follow steps from here.
We can use the stored prediction outputs to compare against the actuals and see how our model is improving over time or if it needs retraining.
Machine Learning is a major component in most manufacturing transformation projects today. A key skill for a machine learning project is the ability to understand the data coming out of both the IT and OT systems. Most customers do have experts who understand their domains very well, but they lack the skills to apply these new machine learning techniques to better reason about the data. If these domain experts are enabled with the right set of tools and techniques, they can amplify their skills. I hope the above approach helps to sharpen these skills.
To learn more see our Industrial IoT patterns doc and deployment samples. As always, feel free to share your thoughts and ask your questions in the comments!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.