Model deployment is often the last step in a Machine learning Product Lifecycle. This single step drives model adoption in multitude of ways. Needless to say, if done without precise and thoughtful planning, it can lead to significant problems across multiple aspects of the customer facing service, including but not limited to, security, availability and eventually consumption, which can ultimately result in the wastage of efforts spent in the creation of the model in the first place.
After realizing the necessity of applying careful considerations on the deployment of ML models, I started exploring sophistic ways of deployment.
Exploration was centered around 3 aspects:
Exploration resulted in fruition. I found a holistic and comprehensive offering within Azure with absolute focus on Machine learning Product Lifecycle, replete with the goodness of Azure, which goes by the name of Azure Machine Learning.
The rest of the article focuses on the deployment of ML model as a secured webservice along with its consumption as a RESTful API with Azure Machine Learning, all using Python!
Prerequisites
Before moving ahead, we need to have 5 things:
Workflow
The workflow involves 10 simple steps.
1. Connect to your Azure account:
from azureml.core.authentication import InteractiveLoginAuthentication
interactive_authentication = InteractiveLoginAuthentication(tenant_id="TENANT ID", force=True)
This will complete the authentication and establish a remote connection with the Azure account.
Available controls
2. Create Azure ML workspace:
Workspace is the top-level resource for Azure ML, which provides a centralized place to work with all the artifacts created using Azure ML.
from azureml.core import Workspace
ml_workspace = Workspace.create(name='HousePricePrediction_ws',
subscription_id='SUBSCRIPTION ID',
resource_group='HousePricePrediction_rg',
location='GEOLOCATION'
)
ml_workspace.write_config()
This will create and return a workspace referencing instance after deploying dependent resources i.e., app insights, storage account and key vault.
Available controls
3. Register ML model:
The process of uploading a model separately from webservice code is known as registering of model. Azure ML separates client facing webservice code from the ML model to allow their individual updation.
from azureml.core.model import Model
ml_model = Model.register(model_path = "BostonHousePricePrediction.pickle",
model_name = "HousePricePrediction_Model",
description = "To predict the price of houses",
workspace = ml_workspace)
This will upload the model to the cloud within referenced workspace storage account.
Available controls -
4. Define an entry script:
Entry script acts as an intermediary between model and deployed webservice. It receives data submitted to the webservice and passes it to the model. It then takes the response and return it to the client.
%%writefile score.py
import json
import numpy as np
import os
import joblib
import pickle
def init():
global model
model_path = os.path.join(os.getenv('AZUREML_MODEL_DIR'), 'BostonHousePricePrediction.pickle')
model = joblib.load(model_path)
def run(request_data):
request_data = json.loads(request_data)['data']
data = np.array(request_data)
predicted_price = model.predict(data)
return predicted_price.tolist()
This will create a python script (score.py) locally consisting of 2 functions:
5. Define dependency file:
from azureml.core.conda_dependencies import CondaDependencies
dependency = CondaDependencies()
dependency.add_conda_package("scikit-learn")
dependency.add_pip_package("azureml-model-management-sdk")
with open("HousePricePredictionEnv.yml","w") as file:
file.write(dependency.serialize_to_string())
This will create a dependency file (HousePricePredictionEnv.yml) locally specifying python libraries required by the environment to run the model.
6. Create an inference configuration:
Inference configuration specifies the entry script, runtime and dependency file required to set up the environment needed by both model and the webservice.
from azureml.core.model import InferenceConfig
inference_config = InferenceConfig(entry_script='score.py',
runtime="python",
conda_file="HousePricePredictionEnv.yml")
Available controls
7. Select deployment/compute target:
Compute target is the computing platform used to host webservice.
Below diagram can guide you in choosing and determining deployment target.
This selection primarily depends on the cost, availability, and processing requirements of the service.
Some examples:
In this article, we will use AKS as the deployment target.
An Azure Kubernetes Service (AKS) creation requires the attachment of AKS cluster to the Azure workspace.
from azureml.core.compute import ComputeTarget,AksCompute
provision_config = AksCompute.provisioning_configuration()
deployment_target = ComputeTarget.create(workspace=ml_workspace,
name='Predictionaks-1',
provisioning_configuration=provision_config)
if deployment_target.get_status() != "Succeeded":
deployment_target.wait_for_completion(show_output=True)
Available controls
8. Create a deployment configuration:
Deployment configuration defines the characteristic of the compute/deployment target on granular levels e.g., number of CPU cores, amount of memory etc.
from azureml.core.webservice import AksWebservice
deployment_config = AksWebservice.deploy_configuration(cpu_cores=1,
memory_gb=1,
auth_enabled = True,
enable_app_insights = True,
description='Service to predict house prices')
Available controls
9. Deploy ML model as Azure Kubernetes Service/API:
service = Model.deploy(workspace=ml_workspace,
name="housepriceprediction-service",
models=[ml_model],
inference_config=inference_config,
deployment_config=deployment_config,
deployment_target=deployment_target)
service.wait_for_deployment(show_output = True)
print(service.state)
This will deploy the model and create a consumable RESTful API/webservice.
Available controls
10. Consume the API:
It involves 3 steps:
service_uri = service.scoring_uri
primary, secondary = service.get_keys()
import requests
import json
scoring_uri = service_uri
key = primary
headers = {'Content-Type': 'application/json'}
headers['Authorization'] = f'Bearer {key}'
input_data = [[0.06724, 0.0, 3.24, 0.0, 0.46, 6.333, 17.2, 5.2146, 4.0, 430.0, 16.9, 375.21, 7.34],
[9.2323, 0.0, 18.1, 0.0, 0.631, 6.216, 100.0, 1.1691, 24.0, 666.0, 20.2, 366.15, 9.53]]
input_data = json.dumps({'data': input_data})
response = requests.post(scoring_uri, data=input_data, headers=headers)
print(response.text)
Parting thoughts
Azure Machine Learning provides plethora of options for model deployment from Azure Kubernetes to Azure IOT and many more, the good thing is that the workflow remains reasonably consistent across all, with small obvious changes required in defining deployment configuration.
The aim of this article is to get you started with the creation of a simple and secured webservice for ML models by using cardinal controls that are needed for each operation. Many more controls are available, which can be utilized based on the requirements.
I hope this article will ease your model deployment journey and help you in creating resilient webservices.
Additional Resources
Azure Machine Learning - ML as a Service | Microsoft Azure
How to deploy machine learning models - Azure Machine Learning | Microsoft Docs
Azure Machine Learning SDK for Python - Azure Machine Learning Python | Microsoft Docs
Deploy ML models to Kubernetes Service - Azure Machine Learning | Microsoft Docs
Pricing - Machine Learning | Microsoft Azure
Azure Products by Region | Microsoft Azure
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.