Thanks to the advancements in the area of natural language processing using machine learning techniques & highly successful new-age LLMs from OpenAI, Google & Microsoft - NLP based modeling & inferencing are one of the top areas of customer interest. These are being widely used in almost all Microsoft products & there is also a huge demand from our customers to utilize these techniques in their business apps. Similarly, there is a demand for privacy preserving infrastructure to run such apps.
In this blog I am going to show how to run basic NLP text summarization using T5 & BART in a python streamlit.io based app & run it as a container on Confidential ACI container group. Before we go there, here are some details about Azure Confidential Computing for the uninitiated. Azure already encrypts data at rest and in transit, and confidential computing helps protect data in use, including the cryptographic keys. Azure confidential computing helps customers prevent unauthorized access to data in use, including from the cloud operator, by processing data in a hardware-based and attested Trusted Execution Environment (TEE). Aside from encrypting the data in-use Confidential computing also allows remote guest attestations of the compute/cluster being used.
A multi party computing clean room is a rather common business use case where multiple parties to safely share code, models & data to compute a shared (shareable to multiple/all parties) or 1-party output/result-set while they do not have to trust each other with the raw data and model/code being used.Example 1 - A bank mashing up its customer data with the retail consumer purchase profiles acquired from participating retail parties to generate offers for its customers.Example 2 - A healthcare pharmaceutical company bringing in health research data from multiple other participating pharmaceutical companies, containing PHI/PII into a clean room and creating highly effective models to for example create novel drugs for advanced diseases.Example 3 - A financial institution mashing up data from multiple banks to create an centralized ML model to prevent fraud by analyzing fraud patterns instead of only using its limited datasets.
The code may be audited by a 3rd party or all the parties involved and can be measured as a part of code deployment. The goal of measuring is to ensure that no individual party is unilaterally able to modify it and inadvertently or intentionally make it less secure or cause any kind of data/code or IP compromise.
Typically in an AI/ML scenario you are trying to protect the following from all other parties involved including the cloud provider.
a) Model IP
b) Training Data
c) Model weights
d) Inferencing data
e) Inference outcome
f) Other miscellaneous items.
In this article we will create a python streamlit.io app and deploy it to Confidential ACI. The use case is loosely based on a 2 party multi party computing scenario where party II provides text data in encrypted format because they cannot allow party I to take a look at the text. However, their contract with party I allows party I to summarize the encrypted text and consume the summary. This entire process runs in the Azure tenant that belongs to party I but due to Confidential ACI, party I is at no point able to take a look at the original text data from party II. That, in a nutshell, is multi party confidential computing. (needless to say there can be N parties building this exact solution)
To start with, we need to create and deploy this app on Confidential ACI. Please follow the instructions given below.
Pre-requisites
Before starting please create the following pre-requisites in your portal or using CLI:
a) Create your app and push it to Azure Container Registry (please see the appendix section to see how I created my app and ACR entry)
b) Create a Managed HSM instance Quickstart - Provision and activate an Azure Managed HSM | Microsoft Learn
c) Create an Azure Storage Account & create a container inside it Create a storage account - Azure Storage | Microsoft Learn
d) Create a SAS token for the container you created in Step c. https://learn.microsoft.com/en-us/azure/cognitive-services/translator/document-translation/how-to-gu...
e) Create a user-assigned managed identity Manage user-assigned managed identities - Microsoft Entra | Microsoft Learn
& give it Storage Blob Data Reader & Reader on the storage account you created. (if you want to write to the blob, instead add Storage Blob Data Contributor)
f) Give the following permissions to your AAD user and the newly created user assigned managed identity on the MHSM
- Managed HSM Crypto User
- Managed HSM Crypto Officer
Demo Steps (windows or linux environment will work; I am using Windows 11 here with WSL2 for Ubuntu Shell):
IMPORTANT: Use `sudo` wherever necessary
a) Open WSL2 on Windows and run the following installs:
sudo apt update
sudo apt install golang-go
sudo apt install gccgo-go
Now, switch to /mnt
cd /mnt
mkdir CONFIDENTIAL_ACI_DEMO
cd CONFIDENTIAL_ACI_DEMO
b) Clone our main github repo as follows: (you may need to install git if not installed)
git clone https://github.com/microsoft/confidential-sidecar-containers.git
c) Switch to the folder containing the next shell script as follows:
cd confidential-sidecar-containers/examples/encfs/generatefs/
d) Install python to run this script if not already done
sudo apt install python3
sudo apt install python-is-python3
python -V
sudo apt install cryptsetup
e) Run the shell script ./generatefs.sh
sudo ./generatefs.sh
This should generate the image (.img file & .bin file). We will use these in later steps.
ls -l
od -x encfs.img
This step creates an encrypted file system that will be uploaded in the next step to the azure storage account as a block blob.
f) Now install the azure cli if not already installed
curl -sL https://aka.ms/InstallAzureCLIDeb | sudo bash
Upload the encfs.img to the blob storage. (Before this steps please create an Azure Storage Account & create a container inside it, then create a SAS token)
Please make sure you use page blob & not block blob
cd ..
az storage blob upload --file generatefs/encfs.img --container-name <your container name> --name <specify a name you would like for the page blob of the image we generated in the previous step> --account-name <your storage account name> --sas-token '<SAS token starting with sp within single quotations>'
g) Post the step above your encfs should show in the portal.
h) Now run the following to figure out the key that was used to encrypt the file system:
cd generatefs
python hexstring.py keyfile.bin
cd ..
(Preserve the output of the above to be used later)
The following is a sample template of the file called encfs-sidecar-args.json which is located in the encfs folder (/confidential-sidecar-containers/examples/encfs/)
{
"azure_filesystems": [
{
"mount_point": "/mnt/remote/share",
"azure_url": "<URL of the blob we uploaded above; grab it from Azure portal>",
"azure_url_private": true,
"key": {
"kid": "<name the key that will be uploaded to M-HSM/AKV later>",
"authority": {
"endpoint": "<MAA Endpoint>"
},
"akv": {
"endpoint": "<Your Managed HSM Endpoint>"
}
}
}
]
}
Note: A default URL for MAA is sharedneu.neu.attest.azure.net/
Please fill in the rest of the details.
i) Install base64 if not already installed:
sudo apt-get install cl-base64
Use the following to convert the above json file into base64 (since this is going to be sent as arguments in another json tempalate)
base64 -w0 encfs-sidecar-args.json
Take the base64 value and update the following section (value field) in the file at aci-arm-template.json at (/confidential-sidecar-containers/examples/encfs)
"environmentVariables": [
{
"name": "EncfsSideCarArgs",
"value": "<Use the hash you generated in the previous steps>"
}
]
Then, please follow instructions here azure-cli-extensions/src/confcom/azext_confcom at main · Azure/azure-cli-extensions (github.com) to install the confcom extension.
az extension add --name confcom
j) Next let's modify the aci-arm-template.json file.
"identity": {
"type": "UserAssigned",
"userAssignedIdentities": {
"<full property value of the user assigned managed identity you created in the pre-requisites>": {}
}
}
At this point, we need to generate the CCE policy using confcom we installed in the previous step. But before that, we need to ensure that we add information about our container and replace the already provided container information in the aci-arm-template.json file.
{
"name": "<name that you pick for the new container in Confidential ACI>",
"properties": {
"command": [],
"image": "<your_ACR_instance_url/your_image:tag>",
"resources": {
"requests": {
"cpu": <2 or, anything you like, please be aware of limits>,
"memoryInGb": <6 or anything you like, please be aware of limits; if you use the app linked in this blog, then use 8>
}
},
"volumeMounts": [
{
"name": "remotemounts",
"mountPath": "/mnt/remote"
}
],
"ports": [
{
"port": <your app port>
}
]
}
}
I would also add the following value but it is optional.
"imageRegistryCredentials": [
{
"server": "<your_ACR_server_URL>",
"username": "<your acr user name under Access Keys>",
"password": "<your ACR password under Access Keys>"
}
]
Please ensure that the CCE policy value is an empty string for now. This will be generated when we run the next command.
"confidentialComputeProperties": {
"ccePolicy": ""
}
k) Now, let's run the command to fill in the CCE policy using policygen.
Note: If docker is not running and you get an error run the following:
sudo service docker start
service docker status
Now, run this to generate ccepolicy
az confcom acipolicygen -a aci-arm-template.json --debug-mod -->> You may remove debug-mod for production releases & ensure that no one can access the container (NOT EVEN GUEST USERS on your side)
Note: Ensuring no debug-mode and auditing are currently something that needs to be performed by the parties together to ensure there is no violation of confidentiality in the infrastructure code.
After this is run, we will see that the same template file aci-arm-template.json has a large hash value in the ccepolicy field.
The output of this command above will also have the value to be filled into in the next step. (so please preserve it)
"claim": "x-ms-sevsnpvm-hostdata"
This needs to be filled into the file called importkeyconfig.json at the same location (/confidential-sidecar-containers/examples/encfs)
l) Now, the next step is to upload the key which will be used to decrypt the file system to MHSM and this key will be released to the container side car using the Secure Key Release process.
In order to upload the key we need to fill up another json template called importkeyconfig.json located in(/confidential-sidecar-containers/examples/encfs). Here is a sample.
Note: A default URL for MAA is sharedneu.neu.attest.azure.net/
{
"key": {
"kid": "<your key name that will be uploaded to MHSM; also used in encfs-sidecar-args.json>",
"authority": {
"endpoint": "<Microsoft Azure Attestation URL; see note above>"
},
"akv": {
"endpoint": "<Your managed HSM endpoint>",
"api_version": "api-version=7.3-preview",
"bearer_token": "<Generate bearer token to authenticate to the MHSM as instructed & place it here>"
}
},
"claims": [
[
{
"claim": "x-ms-sevsnpvm-hostdata",
"equals": "<Use the value generated by previous step k that we preserved>"
},
{
"claim": "x-ms-compliance-status",
"equals": "azure-compliant-uvm"
},
{
"claim": "x-ms-sevsnpvm-is-debuggable",
"equals": "false"
}
]
]
}
Once the values are filled in, now we need to upload the Key into MHSM.
Before we run the actual step, we will need to generate a bearer token (short lived) and use it in the template above as shown.
To generate the bearer token, we can use the following:
az account get-access-token --resource https://managedhsm.azure.net
Now, the template should have all placeholders filled in with real value. Let's run it as follows:
go run ../../tools/importkey/main.go -c importkeyconfig.json -kh <key hash generated in step h above>
Check and validate that the key has been uploaded to MHSM.
m) The time has come to finally create the container group using the template aci-arm-template.json. This can be done from the CLI or the portal. I am going to use a template deployment approach from the portal.
Step 1: Copy the contents of the template from VSCODE or whatever editor you used so far.
Step 2: Go to Azure Portal and search "Deploy a custom template"
Step 3: Click on Build Your Own Template
Step 4: Now replace the contents in the editor with the contents of your aci-arm-template.json & Save
Step 5: It will show a page like this
Step 5a: Deploy!! by clicking Review + Create
Once this is done, you should be able to run your app from a portal.
APPENDIX
Here is how to create & deploy the app - the files can be also found on our official github page. If you decide to use this app, please ensure your container has enough configured memory e.g. 8 GB.
More information on Confidential Containers in Azure is here.
