How to build an environment when your Azure ML workspace is behind a VNet

hristinajilova · ‎Dec 19 2022

Build environment for your experiment when your Azure ML workspace is behind a Vnet

We see a common request from our customers to create a ML pipeline using an Azure ML workspace behind a VNet.

Creating and building environment in a workspace behind a VNet has some specifics and causes questions. The reason is that you cannot build the Docker Image directly on ACR, when the ACR is behind a virtual network

In this article you will find the steps that will help you to build your custom environment when you Azure ML workspace is behind a VNet.

Example files: AzureML/CreateEnvBehindVnet at main · HristinaJilova/AzureML (github.com)

1. Prepare the Docker Image Context

Create a folder for you docker context called 'DockerContext' (or choose another name)
In 'DockerContext' folder create a 'requirements.txt' file that contains your packages:

        openpyxl==3.0.10   
        pyod==1.0.4    
        kaleido==0.2.1  
        plotly==5.10.0    
        pandas==1.1.5    
        numpy==1.21.6    
        seaborn==0.11.2    
        pathlib2==2.3.7.post1    
        matplotlib==3.2.1    
        argparse

Note: It’s a good practice to fix the versions for your packages. You can find the version with command: pip freeze | grep 'the package name'

Create a Dockerfile:

        #FROM mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04
        FROM python:3.8

        # python installs
        COPY requirements.txt .
        RUN pip install -r requirements.txt

        # set command
        CMD ["bash"]

Option 1:

Configure an AzureML compute cluster to build environments

Create an Azure Machine Learning compute cluster (Only a CPU SKU is supported). This cluster will be used to build the docker images when ACR is behind a VNet. For more information, see Create a compute cluster.
Use the az ml workspace update command to set a build compute:

az ml workspace update --name myworkspace --resource-group myresourcegroup --image-build-compute mycomputecluster

For reference: Enable Azure Container Registry (ACR)