Customizing and deploying pre-existent Machine Learning (ML) models is one of the major challenges for ML customers. Our own manufacturing facility for Surface devices needed different models to detect display defects and ensure employee safety. While working with them, we learned that it is not possible for us to build models for all our customers’ current and future needs.
We collaborated with Custom Vision, Live Video Analytics (LVA), ONNX Runtime, and Azure IoT Edge teams to empower our manufacturing team to build these models themselves with no code or ML experience. We have named this solution template VisionOnEdge(VoE).
VoE empowers experts in different verticals (manufacturing, retail, etc.) to build and scale their models to existing camera feeds and locations. When the model at different locations does not meet performance benchmarks, experts are empowered to collect more data, retrain, and redeploy better models.
VoE can be easily deployed to as wide range of edge devices, from Azure Stack Hub/HCI, to Azure Stack Edge, Azure Stack Mini R, or even your own device powered by Azure IoT Edge like the Intel AI devkit or the NVIDIA Jetson Nano.
Before deployment, you need to ensure the following:
Here is a snapshot of a typical end-to-end Video Analytics solution leveraging VisionOnEdge
After you send the deployment, the IoTEdge agent running on your edge device gets new containers from the container registry and starts them on your edge device.
Out of the box, you will find six scenarios that can help you get started and explore the possibilities of what you can build.
As you move from demo to pilot stage, you may need to run ML on your own cameras, train models for your own objects, and deploy at scale on enterprise-grade devices. Here you can use the same VoE application to build zero to hero pilots on Azure Stack devices.
VoE makes it super easy to add your own existing cameras, collect images, label your object, and deploy. When you deploy a new task, VoE detects newly tagged images that are not part of the current ML model and automatically starts a new training job, exports the model to onnx format, and then runs it accelerated on your edge device using onnxruntime.
Taking images from a camera and manually tagging them could be a tedious task. To ease user pain, VoE captures and tags images automatically. Under advanced task setting, users can set VoE to collect a specified number of images for retraining. VoE will capture and store these images for users to validate auto-labeled images on the images tab. On the next deployment, VoE automatically triggers a training job with newly added images. This way VoE provides an easy way to improve model performance without the pain of manually capturing and tagging images.
Users may want to deploy open-source models, Microsoft cognitive service models, or their own models trained in Azure Machine Learning (AML). VoE allows users to bring their own models in three simple steps. Users first deploy their models as web applications (here is sample deployment using AML). Second, on the VoE model tab users can update the endpoint information and upload its labels.txt that contains information on what object the model can identify. Finally, on the deployment tab, they can see their own model option. Deploy an open-source Yolo model on VoE using the tutorial here.
Once users have a model that is working, they like to deploy this model on more than one camera. On VoE, users can do just that by simply choosing more than one camera during their deployment camera option. VoE uses LVA to parse frames from multiple cameras and auto-adjust frames rate per camera to ensure that each camera frame gets processed. Users can see results on each camera stream by toggling the deployment view to the camera of their choice. Users can also define their area of interest in the camera feed in case they only want a subsection of camera point of view to be used for processing.
In an advanced setting, users can enable sending messages to the IoT hub. They can set how many messages they want to send and when. Once the messages are received at the Azure IoT hub, users can use the IoT hub with logic apps to get notifications on Teams (more details here) and see analytics in time series insights (more details here).
VoE also enables users to store videos using LVA in the cloud, communicate between containers using shared memory by using gRPC, and allows you to disable the video output with results overlayed.
With VoE, going from pilot to production is simple and smooth. VoE is already tested on all Azure Stack production scale devices like Azure Stack Edge and Azure Stack HCI. For bringing your pilot work to production you can choose any of the three paths below:
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.