Azure Machine Learning (AML) natively supports deploying a model as a web service on Azure Kubernetes Service (AKS). Based on the official AML documentation, deploying models to AKS offers the following benefits: Fast response time, Auto-scaling of the deployed service, Logging, Model data collection, Authentication, TLS termination, Hardware acceleration options such as GPU and field-programmable gate arrays (FPGA). Please refer to the official documentation for directions on using AML Python SDK, Azure CLI, or even Visual Studio Code to deploy models to AKS.
This blog article, as well as the accompanying GitHub repo, demonstrates an alternative option, which offers significant flexibility in model deployment. In particular, this solution template helps enable the following use cases:
Before you proceed, please complete the following prerequisites:
Here is the architecture diagram for this solution template:
For easy and quick reference, readers can refer to the following online resources as needed.
Use https://github.com/HZ-MS-CSA/aml_aks_generic_model_deployment as clone URL
As a demonstration, we will be using an onnx model from a Microsoft Cloud Workshop activity.
1if the claim is an auto insurance claim or
0if it is a home insurance claim. The model will be built using a type of Deep Neural Network (DNN) called the Long Short-Term Memory (LSTM) recurrent neural network using TensorFlow via the Keras library." Source here.
There are two files that need to be modified to accommodate the onnx model
Docker-Build an Image: Configure the activity based on the notes and screenshot below
Publish Build Artifact: Leave everything as default
Save and queue the build pipeline.
Start with an empty job
Change Stage name to be AKS-Cluster-Release
Add build artifact
Set up continuous deployment trigger--the release pipeline will be automatically kicked off every time a build pipeline is modified
helm upgrade: Package and deploy helm charts activity.
--create-namespace --set image.repository=IMAGE_REPOSITORY_PATH --set image.tag=$(Build.BuildId) --set amlargs.azureTenantId=$(TenantId) --set amlargs.azureSubscriptionId=$(SubscriptionId) --set amlargs.azureResourceGroupName=$(ResourceGroup) --set amlargs.azureMlWorkspaceName=$(WorkspaceName) --set amlargs.azureMlServicePrincipalClientId=$(ClientId) --set amlargs.azureMlServicePrincipalPassword=$(ClientSecret)
In Variables/Pipeline Variables, create and enter the following required values
Save, create, and deploy release
az account set --subscription SUBSCRIPTION_ID
az aks get-credentials --resource-group RESOURCE_GROUP_NAME --name AKS_CLUSTER_NAME
kubectl get deployments --all-namespaces=true
aml-aks-onnxnamespace, make sure it's ready
kubectl get svc --namespace aml-aks-onnx. External IP will be listed there
http://EXTERNAL_IP:80/score. You can optionally set it to be
http://EXTERNAL_IP:80/healthcheckand then use the get method to do a quick health check
Copyright (c) 2021 HZ-MS-CSA
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.