Connecting to any Azure services (i.e., Azure Key Vault) is a multi-step process:
- Acquire an access token from the Azure Active Directory that allows you to use the target Azure Services (i.e., Azure Blob Storage, Azure Key Vault).
- Use the access token to create the client object for the target service.
- Perform the operation of your choice, available through the client object.
Various Software Development Kits (SDKs), like the Python SDK, hide this complexity and offer a simple usage experience where you initiate a Credentials class and then pass that class to the resource you want to use. Set up authentication discusses different authentication methods that we can use when working with other Azure Services from within Azure ML Studio.
Use Key Vault when training walks us through connecting to Azure Key Vault and getting (or setting) secrets, on a local computer. If you are running a python notebook in Aure ML studio, the Compute Instance is like your local compute option. Please refer to the notebook sample MachineLearningNotebooks/authentication-in-azureml.ipynb at master · Azure/MachineLearningNotebooks ... for details on how to work with different authentication types. The GitHub sample also discusses recommendations on when to leverage what authentication option.
We can leverage the following authentication techniques, when connecting to other Azure Services:
- Interactive Login Authentication
- Azure CLI Authentication
- Managed Service Identity (MSI) Authentication
- Service Principal Authentication
- Token Authentication
Unfortunately, not every option would work from within the Compute Instance, used during development within the ML Studio.
For example, Interactive Login Authentication will not work in Azure ML Studio, since it cannot display the interactive dialog box. Also Managed Service option will not work either, we will need a Virtual Machine to use the Managed Identity. However, if we are working with data from within Compute Cluster (for an example walkthrough, please check out Adreas's GitHub Sample FunWithAzureML/step01.process_data.r at master · rndazurescript/FunWithAzureML (github.com)), you would be able to leverage Managed Identity. The authentication technique would thus depend on the ML training target you are using.
For our example scenario, we used Azure CLI Authentication to obtain credentials (access token) object. This will help us getting authenticated to Azure Key Vault, when we are not using Azure Key Vault part of Azure ML workspace.
Using the credentials we obtained, we can now create client object to work with Azure Key Vault Service. Now, let us say we want to read the secrets we created in the Azure Key Vault. To avoid showing actual secret key, we can store the key as an environment variable (so it is not visible within the notebook). We can set the environment variables in the /etc/environment file, opening a terminal from ML studio (you would use sudo command). For example, to store the key vault name as an environment variable, you can add the following line to the /etc/environment file:
export KEY_VAULT_NAME=<KeyVaultName>
You may have to run the command below or restart the compute instance to take the changes into effect:
source /etc/environment
Now that you have the environment variable created, you can access it from within the notebook as below:
keyVaultName = os.environ["KEY_VAULT_NAME"]
Now that we know the steps how to work with the services like Azure Key Vault, let us walk through in the cell of our Notebook:
# Import required libraries
import os
from azure.keyvault.secrets import SecretClient
from azure.identity import AzureCliCredential
# Prepare your Azure Key Vault URL
keyVaultName = os.environ["KEY_VAULT_NAME"]
KVUri = f"https://{keyVaultName}.vault.azure.net"
print ('Key Vault Name: ' + keyVaultName)
# Create the client object for Azure Key Vault Service
credential = AzureCliCredential()
client = SecretClient(vault_url=KVUri, credential=credential)
# Get the secret name you would like to retrieve the value for
secretName = os.environ['SECRET_NAME']
# Retrieve the secret value from your key value
print(f"Retrieving your secret from {keyVaultName}.")
retrieved_secret = client.get_secret(secretName)
print(f"Your secret is '{retrieved_secret.value}'.")
# Now you are ready to leverage the secret for other purposes,
# like using the secrets to connect to Azure SQL DB,
# if you stored the connecting string in your secret
Please note the following error may encounter, when using Azure CLI Authentication option. You would have to run the ‘az login’ command from the Compute Instance Terminal
Key Vault Name: <key vault name>
Retrieving your secret from <key vault name>.
AzureCliCredential.get_token failed: Please run 'az login' to set up an account
---------------------------------------------------------------------------
In the above sample, if we were storing the data access secrets within the Key Vault, part of our Azure ML Workspace, we could use the code snippet below to access the Key Vault secret, instead of using the SecretClient object:
ws = Workspace.from_config()
keyvault = ws.get_default_keyvault()
keyvault.get_secret(name="mysecret")
-----------------------------------------
In the part-two, we will walkthrough how we can connect to Azure Blob Service to work with blobs from Azure ML Studio, using a notebook leveraging the Azure Key Vault part of Azure ML Workspace. Please stay tuned.
