Model deployment is one of the most critical components in machine learning systems. Model deployment in Azure Machine Learning (AzureML) is evolving. So far, AzureML supports Azure Container Instances (ACI) and Azure Kubernetes Service (AKS) as traditional seamless deployment targets for models.
Recently, at Build 2022, we released managed online endpointsto provide a unified interface to invoke and manage model deployments on Microsoft-managed compute in a turnkey manner. You can take advantage of scalable and reliable endpoints without being concerned about infrastructure management. Already, our several customers and partners are utilizing the inference capability to automate model deployments toward production use.
The developer experience in AzureML is also evolving. The AzureML CLI v2 and Python SDK v2 (preview) no longer support legacy ACI web services.Upgrade to v2 is highly recommended to take full advantage of the consistency and new features to accelerate the machine learning lifecycle in v2 production environments.
In this blog, we summarize the benefits of leveraging the managed online endpoints, cost comparison, and introduce how you can transition from your existing ACI workloads to managed online endpoints.
*As of September 2022, ACI web services are in maintenance mode and will not be invested for new features.
*AzureML CLI v1 is getting retired on 30 Sep 2025, see CLI & SDK v2 for details.
What managed online endpoints bring you
Managed online endpoints handle serving, scaling, securing, and monitoring of your machine learning models without being concerned about the underlying infrastructure. In particular, the recommended deployment purpose of ACI web services was for dev/test environments, while managed online endpoints is designed for use in production environments.
Here are some benefits of using managed online endpoints:
We provide documents and scripts to support upgrade. This tool will automatically create new online endpoint, your original services won't be affected. You can safely route the traffic to the new endpoint and then delete the old one.
There are a few things to note when upgrading from ACI web service: