Following on Azure Arc enabled ML training preview announcement in June, the Azure Machine Learning team is excited to announce the public preview of Azure Arc enabled Machine Learning for inference. Built on top of Azure Arc enabled Kubernetes which provides a single pane of glass to manage Kubernetes anywhere, Azure Arc enabled ML inference extends Azure ML model deployment capabilities seamlessly to Kubernetes, and enables customers to deploy and serve models on Kubernetes anywhere at scale.
It is easy to enable Kubernetes cluster for Azure Arc enabled ML inference by following the 4 steps below:
Connect. Follow the instruction here to connect an existing Kubernetes cluster to Azure Arc. For Azure Kubernetes Service (AKS) cluster, connecting to Azure Arc is not needed and this step can be skipped.
Deploy. The following CLI command will deploy AzureML extension to an Arc-connected Kubernetes cluster and enable model endpoints with private IP. For AzureML extension deployment on AKS cluster without Azure Arc connection, please change --cluster-type parameter value to be managedClusters:
az k8s-extension create --name tailwind-ml-extension --extension-type Microsoft.AzureML.Kubernetes --cluster-type connectedClusters --cluster-name <your-connected-cluster-name> --config enableInference=True privateEndpointILB=True --config-protected sslCertPemFile=<path-to-the-SSL-cert-PEM-ile> sslKeyPemFile=<path-to-the-SSL-key-PEM-file> --resource-group <resource-group> --scope cluster
For additional scenarios to deploy AzureML extension for inference, please see Azure Arc enabled ML documentation.
Attach. Once the AzureML extension is deployed on Kubernetes cluster, run the following CLI command or go to Azure ML Studio UI to attach the Kubernetes cluster to Azure ML workspace:
az ml compute attach --type kubernetes --name tailwind-compute --workspace-name tailwind-ws --resource-group tailwind-rg --resource-id tailwind-arc-k8s --namespace tailwind-ml
Use. After above steps, the Kubernetes compute target now is available in the Azure ML workspace. Users can look up Kubernetes compute target names with CLI command or in Studio UI “Compute->Attached computes” page, and use the Kubernetes compute target to train or deploy models without getting into any Kubernetes details.
For more information about Azure Arc enable ML inference preview announcement, please visit this Azure AI blog post.
Supported Kubernetes distributions
Azure Arc enabled ML is tested and verified on the following Kubernetes distributions:
- Azure Kubernetes Services (AKS)
- AKS Engine
- AKS on Azure Stack HCI (AKS-HCI): AKS-HCI is the on-premises equivalent of popular AKS
- Azure RedHat OpenShift Service (ARO)
- OpenShift Container Platform (OCP)
- Google GKE
- Canonical Kubernetes Distribution
- Amazon EKS
- Kind
- K3s-Lightweight Kubernetes
Get started today
To get started with Azure Arc enabled ML for inference, please visit Azure Arc enabled ML documentation and GitHub repo, where you can find detailed instructions for IT Operator to deploy AzureML extension on Kubernetes, as well as examples for data science professionals to use Kubernetes compute target in Azure ML workspaces.