In this post, I’ll show how you can deploy an AI Agent on Azure Kubernetes Service (AKS) using a multi-tenant approach that maximizes both security and cost efficiency. By isolating each tenant’s agent instance within the cluster and ensuring that every agent has access only to its designated Azure Blob Storage container, cross-tenant data leakage risks are eliminated. This deployment model allows you to allocate compute and storage resources per tenant, optimizing usage and spending while maintaining strong data segregation and operational flexibility—key requirements for scalable, enterprise-grade AI applications.

Multi-tenancy has its roots in the 1960s mainframe era with the emergence of time-sharing systems. Back then, companies couldn't afford their own mainframes, so they rented processing time and resources. Each customer (tenant) accessed the same physical system but operated in a logically distinct environment. This allowed mainframe vendors to bill per customer and efficiently share resources.

Today, multi-tenancy underpins the scalability and cost-efficiency of cloud services. Most SaaS products, like Office 365, Salesforce, or Slack, leverage multi-tenanted architectures to serve thousands of customers with robust data isolation, cost efficiency and operational simplicity.

Multitenancy is crucial for both cost efficiency and development velocity, especially in today’s cloud and SaaS environments. By allowing multiple tenants to share the same application infrastructure—such as servers, databases, and runtime environments—organizations dramatically reduce overhead costs. Instead of each customer requiring dedicated resources and maintenance, infrastructure costs are spread across all users, enabling lower per-tenant expenses and optimal resource utilization. This not only minimizes waste but also simplifies scaling: resources can be dynamically allocated based on demand, avoiding unnecessary overprovisioning. From a velocity perspective, multitenant architectures streamline operations by centralizing software updates, maintenance, and deployment processes. New features, security patches, and upgrades can be rolled out just once to benefit every tenant, rather than managing multiple separate environments. This accelerates time to market for new capabilities, makes onboarding new clients faster and simpler, and frees development teams to focus on innovation rather than repetitive maintenance. As a result, organizations gain a competitive edge by delivering services more quickly and cost-effectively, while maintaining the scalability and flexibility required for modern digital business.

Why use AKS for multi-tenancy?

Azure Kubernetes Service (AKS) stands out for multitenancy compared to services like Azure Container Apps or App Service because it offers the highest degree of control, flexibility, and scalability, which are essential for complex, multi-tenant architectures. With AKS, you get full access to the Kubernetes API, enabling custom resource isolation, advanced network policies, and granular security controls—key for securely segregating tenants and optimizing costs at scale. AKS supports advanced scenarios such as hybrid or multi-cloud deployments, persistent storage for stateful workloads, and sophisticated scaling strategies (including horizontal pod autoscaling and the use of virtual nodes). While Azure Container Apps and App Service provide simplified deployment and management, they lack the deep orchestration, workload isolation, and customization capabilities required for enterprise-grade, highly scalable multi-tenant solutions. For organizations needing robust tenancy separation, fine-tuned resource control, and integrated DevOps workflows, AKS is the platform of choice to accelerate innovation without sacrificing operational excellence.

Choosing your tenancy model

Within multi-tenancy, there exists a spectrum of different forms of deployment, where multi-tenancy can exist on the application layer, the database layer or both. For more details you can look at this Microsoft documentation. For the purposes of this blog, I will be showing you a hybrid model, separate compute and shared storage. This helps to prevent the noisy neighbour effect, while maintaining strict data isolation and cost efficiency.

Challenges in Credential Scoping for Agentic AI with Semantic Kernel

Agentic AI built using the Semantic Kernel framework enables the creation of autonomous agents that can plan, reason, and execute function calls across different plugins and services. These agents leverage the kernel’s orchestration abilities to chain together skills and dynamically select the right functions to accomplish complex goals. However, while execution of function calls is straightforward, assigning fine-grained credentials or restricting permission scopes for those calls remains a challenge. Current implementations often require agents to operate with broad access tokens or API keys, meaning once embedded, the agent has full authority within that integration. This creates a security gap, since there is no built-in mechanism in Semantic Kernel to delegate temporary, least-privilege credentials to agents at runtime. As a result, developers must carefully design trust boundaries and external safeguards when deploying agentic AI systems to ensure function usage does not exceed its intended scope.

Solution overview

Figure 1 shows the architecture of the solution. This post uses Kubernetes constructs to provide compute, data and identity isolation allowing organizations to manage tenants in a single AKS cluster in a scalable, secure and cost-effective manner. While it is common to share the underlying compute, there are benefits for providing dedicated compute per tenant. First, it eliminates the noisy neighbour effect since no other tenants are running within the same compute. If you decide to use the same compute to host multiple tenants, you can overcome the noisy neighbour effect by using resource quotas. Another added benefit of dedicated compute is that you can choose the underlying virtual machine (VM) type based on the profile of your tenants. For example, a tenant that requires memory optimized VM over a general purpose VM. You can also choose a hybrid option where some customers have dedicated compute while the rest utilize shared compute.

Figure 1: Solution architecture diagram

The workflow in Figure 1 is as follows:

1. Dedicated node pools per tenant providing compute isolation.
2. Each tenant is segregated into its own namespace providing control plane isolation.
3. A single storage account containing two Blob storage containers, tenant-a and tenant-b. Role-based access control (RBAC) and attribute-based access control (ABAC) used to govern access control.
4. Managed identity used by each pod to gain access to their respective blob storage containers.
5. An AI Agent orchestrated via semantic kernel and streamlit deployed within each tenant’s pod. AI agent calls gpt models in AI foundry and uploads documents to storage account.

Implement the solution

This section describes how to deploy the solution architecture.

In this post, you’ll perform the following tasks:

• Create an AKS cluster with workload identity enabled.
• Create storage account with two containers named tenant-a and tenant-b
• Create two node pools, tenant-a nodepool and tenant-b nodepool.
• Create taints on nodes and tolerations on pods so that tenant-a pods get deployed on tenant-a node and tenant-b pods get deployed on tenant-b node.
• Create managed identities which the pods will use via WorkloadIdentityCredential to connect to blob storage.
• Create RBAC and ABAC and assign them to respective managed identities.

Prerequisites:

• An Azure Subscription.
• AZ Cli with contributor permission.
• A linux terminal in a venv
• AI foundry project. You can set it up here.
• Azure OpenAI endpoint (required for semantic kernel). You can set it up here.
• Gpt-4.1 deployed (or any model you want). You can set it up following these instructions. Make sure gpt-4.1 is deployed in both AI foundry project and Azure OpenAI. If you use any other model, you will need to substitute the model name in app.py and tools.py
• Azure container registry. You can set it up here.
• Kubectl installed in your environment. You can download it here.
• Docker installed in your environment. You can download it here.

 

Create a virtual environment

python3 -m venv multitenant-aks
cd multitenant-aks
source bin/activate

Export AKS cluster and Storage account variables

In your linux terminal, export these variables with your own values. They will be used in the later commands.

export cluster_name=<CLUSTER_NAME>
export resource_group=<RESOURCE_GROUP>
export storage_account_name=<STORAGE_ACCOUNT_NAME>
export tenant_a_container=tenanta
export tenant_b_container=tenantb
export subscription_id=<SUBSCRIPTION_ID>
export location=eastus2

Create resource group

If you are using an existing resource group, skip this step.

az group create --name $resource_group --location $location

Create AKS cluster

az aks create \
  --resource-group $resource_group \
  --name $cluster_name \
  --location $location \
  --enable-oidc-issuer \
  --enable-workload-identity

Connect to your cluster

az aks get-credentials --name $cluster_name -g $resource_group

Run a sample command to ensure you are connected to your cluster. You should see 3 nodes in your cluster.

kubectl get nodes

Figure 2: AKS nodes

Create storage account and containers

az storage account create \
  --name $storage_account_name \
  --resource-group $resource_group \
  --location $location \
  --sku Standard_LRS

az storage container create \
  --account-name $storage_account_name \
  --name $tenant_a_container \
  --auth-mode login

az storage container create \
  --account-name $storage_account_name \
  --name $tenant_b_container \
  --auth-mode login

You should have two containers created under your storage account.

Figure 3: tenant containers in storage account

Create namespaces for each tenant

kubectl create namespace tenant-a
kubectl create namespace tenant-b
kubectl get ns

This will create control plane isolation between the tenants. It enforces logical separation of workloads and resources within a single cluster. By organizing resources such as pods, services, and roles into different namespaces, administrators can group workloads based on teams, environments, or functions.

Figure 4: Creation of tenant-a and tenant-b namespace

Create individual node pools per tenant

az aks nodepool add \
  --name tenantpoola \
  --cluster-name $cluster_name \
  --resource-group $resource_group \
  --node-taints tenantid=tenanta:NoSchedule \
  --labels tenantid=tenanta \
  --node-vm-size Standard_D4s_v3\
  --node-count 1

az aks nodepool add \
  --name tenantpoolb \
  --cluster-name $cluster_name \
  --resource-group $resource_group \
  --node-taints tenantid=tenantb:NoSchedule \
  --labels tenantid=tenantb \
  --node-vm-size Standard_D4s_v3 \
  --node-count 1

Compute isolation in Azure Kubernetes Service (AKS) can be achieved using node pools, which are collections of virtual machines within an AKS cluster that share the same configuration and underlying resources. By assigning different workloads or tenant applications to separate node pools, administrators can ensure that distinct resource requirements, security needs, or operational policies are enforced at the node level. Each node pool contains a taint “tenantid=tenantb:NoSchedule”. Only a pod with that toleration can be scheduled onto this node, this enforces compute isolation between tenants.

 

Figure 5: Taints and labels on tenant A node pool

Create managed identities per tenant

az identity create --resource-group $resource_group --name tenant-a-mi
az identity create --resource-group $resource_group --name tenant-b-mi

Assign RBAC and ABAC policies on managed identities, restricting access to storage container

az role assignment create \
  --assignee $(az identity show --name tenant-a-mi --resource-group ${resource_group} --query principalId -o tsv) \
  --role "Storage Blob Data Contributor" \
  --scope "/subscriptions/${subscription_id}/resourceGroups/${resource_group}/providers/Microsoft.Storage/storageAccounts/${storage_account_name}/blobServices/default/containers/${tenant_a_container}" \
  --condition "((!(ActionMatches{'Microsoft.Storage/storageAccounts/blobServices/containers/blobs/*'})) OR (@Resource[Microsoft.Storage/storageAccounts/blobServices/containers:name] StringEquals '${tenant_a_container}'))" \
  --condition-version "2.0"

az role assignment create \
  --assignee $(az identity show --name tenant-b-mi --resource-group ${resource_group} --query principalId -o tsv) \
  --role "Storage Blob Data Contributor" \
  --scope "/subscriptions/${subscription_id}/resourceGroups/${resource_group}/providers/Microsoft.Storage/storageAccounts/${storage_account_name}/blobServices/default/containers/${tenant_b_container}" \
  --condition "((!(ActionMatches{'Microsoft.Storage/storageAccounts/blobServices/containers/blobs/*'})) OR (@Resource[Microsoft.Storage/storageAccounts/blobServices/containers:name] StringEquals '${tenant_b_container}'))" \
  --condition-version "2.0"

Create service account and federated credential.

Pods will use a service account to obtain temporary credentials via OIDC. By leveraging service accounts, pods gain a well-defined, managed identity within the cluster, enabling precise access control and improved auditability. When combined with federated credentials—such as those acquired through OIDC tokens and integrated with cloud identity providers like Azure AD—pods can securely obtain short-lived, scoped access tokens for external resources without embedding long-lived secrets or credentials in the application code. This approach enhances security by minimizing the attack surface for credential leaks, supports automated credential rotation, and enables fine-grained least privilege access to cloud services. Moreover, federated identity enables seamless workload authentication across organizational and cloud service boundaries, simplifying the operational overhead associated with credential management and boosting compliance with modern security best practices.

export tenant_a_identity_id=$(az identity show --name tenant-a-mi --resource-group $resource_group --query clientId -o tsv)
export tenant_b_identity_id=$(az identity show --name tenant-b-mi --resource-group $resource_group --query clientId -o tsv)

kubectl apply -n tenant-a -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: storage-access-sa-tenant-a
  namespace: tenant-a
  annotations:
    azure.workload.identity/client-id: $tenant_a_identity_id
  labels:
    azure.workload.identity/use: "true"
EOF

kubectl apply -n tenant-b -f - <<EOF
apiVersion: v1
kind: ServiceAccount
metadata:
  name: storage-access-sa-tenant-b
  namespace: tenant-b
  annotations:
    azure.workload.identity/client-id: $tenant_b_identity_id
  labels:
    azure.workload.identity/use: "true"
EOF

export SERVICE_ACCOUNT_ISSUER=$(az aks show \
  --resource-group $resource_group \
  --name $cluster_name \
  --query oidcIssuerProfile.issuerUrl -o tsv)

echo $SERVICE_ACCOUNT_ISSUER

az identity federated-credential create \
  --name "aks-federated-credential" \
  --identity-name tenant-a-mi \
  --resource-group $resource_group \
  --issuer $SERVICE_ACCOUNT_ISSUER \
  --subject "system:serviceaccount:tenant-a:storage-access-sa-tenant-a"

az identity federated-credential create \
  --name "aks-federated-credential" \
  --identity-name tenant-b-mi \
  --resource-group $resource_group \
  --issuer $SERVICE_ACCOUNT_ISSUER \
  --subject "system:serviceaccount:tenant-b:storage-access-sa-tenant-b"

Check if pods are scheduled on the correct node

kubectl apply -n tenant-a -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tenant-a-app
  labels:
    azure.workload.identity/use: "true"  # Required for workload identity
spec:
  selector:
    matchLabels:
      tenantid: tenanta
  template:
    metadata:
      labels:
        tenantid: tenanta
        azure.workload.identity/use: "true"  # Must be on pod template
    spec:
      serviceAccountName: storage-access-sa-tenant-a  # Reference your service account
      nodeSelector:
        tenantid: tenanta
      tolerations:
      - key: "tenantid"
        operator: "Equal"
        value: "tenanta"
        effect: "NoSchedule"
      containers:
      - name: nginx
        image: nginx
        env:
          - name: TENANT_ID
            value: "tenanta"  # Environment variable to identify tenant
EOF

kubectl apply -n tenant-b -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tenant-b-app
  labels:
    azure.workload.identity/use: "true"  # Required for workload identity
spec:
  selector:
    matchLabels:
      tenantid: tenantb
  template:
    metadata:
      labels:
        tenantid: tenantb
        azure.workload.identity/use: "true"  # Must be on pod template
    spec:
      serviceAccountName: storage-access-sa-tenant-b  # Reference your service account
      nodeSelector:
        tenantid: tenantb
      tolerations:
      - key: "tenantid"
        operator: "Equal"
        value: "tenantb"
        effect: "NoSchedule"
      containers:
      - name: nginx
        image: nginx
        env:
          - name: TENANT_ID
            value: "tenantb"
EOF

Node taints and pod tolerations work together in Kubernetes to control pod scheduling onto nodes by defining rules that repel or allow pods on certain nodes. A node taint is a property applied to a node that marks it as unsuitable for pods unless those pods explicitly tolerate the taint. In the above pod deployment, there is a toleration added which forces it to be deployed on a node with the respective tenant taint.

Figure 6: Enforcement of taints and tolerations

kubectl get pods -n tenant-a -o wide
kubectl get pods -n tenant-b -o wide

If you see your pods stuck in pending state, it is most likely that you don’t have the correct labels on the nodes. Check the portal if the taint and label are set correctly on the node pool.

Figure 7: Pods deployed to its correct tenant node pool

Clean up the pods in tenant-a and tenant-b namespace.

kubectl delete deployment.apps/tenant-a-app -n tenant-a
kubectl delete deployment.apps/tenant-b-app -n tenant-b

 

Deploy AI Agent to tenants in AKS

In this next section, you will be dockerizing your AI Agent application, uploading it to an azure container registry (ACR) and deploying the manifests into AKS.

Export ACR, AI Foundry and Blob account url variables

In the same linux terminal, export these variables with your own values.

export acr=<AZURE_CONTAINER_REGISTRY_NAME>
export ai_foundry_name=<AI_FOUNDRY_NAME>
export ai_foundry_project_name=<AI_FOUNDRY_PROJECT_NAME>
export aoai_name=<AZURE_OPEN_AI_NAME>
export ai_foundry_project_url=https://${ai_foundry_name}.services.ai.azure.com/api/projects/${ai_foundry_project_name}
export blob_account_url=$(az storage account show --name ${storage_account_name} -g ${resource_group} --query "primaryEndpoints.blob" --output tsv)
export aoai_endpoint=$(az cognitiveservices account show --name ${aoai_name} -g ${resource_group} | jq -r .properties.endpoint)

Download Dockerfile and dependencies

Download and unzip the application files named app_files.zip attached in this blog post. It should contain 6 files – app.py, tools.py, Dockerfile, requirements.txt, tenant-a-deployment.yaml and tenant-b-deployment.yaml. You will be using this to build the docker image and deploy it within your AKS cluster.

curl -LO https://github.com/ng-jianshn/multitenant-aks-agentic-ai/raw/refs/heads/main/app_files.zip
sudo apt install -y unzip && unzip app_files.zip

Grant your AKS cluster access to ACR

az aks update \
  --name $cluster_name \
  --resource-group $resource_group \
  --attach-acr $acr

Grant your managed identities access to AI foundry project

az role assignment create \
  --assignee $(az identity show --name tenant-a-mi --resource-group ${resource_group} --query principalId -o tsv) \
  --role "Cognitive Services OpenAI User" \
  --scope /subscriptions/${subscription_id}/resourceGroups/${resource_group}/providers/Microsoft.CognitiveServices/accounts/${ai_foundry_name}

az role assignment create \
  --assignee $(az identity show --name tenant-a-mi --resource-group ${resource_group} --query principalId -o tsv) \
  --role "Cognitive Services OpenAI User" \
  --scope /subscriptions/${subscription_id}/resourceGroups/$resource_group/providers/Microsoft.CognitiveServices/accounts/${aoai_name}

az role assignment create \
  --assignee $(az identity show --name tenant-b-mi --resource-group ${resource_group} --query principalId -o tsv) \
  --role "Cognitive Services OpenAI User" \
  --scope /subscriptions/${subscription_id}/resourceGroups/${resource_group}/providers/Microsoft.CognitiveServices/accounts/${ai_foundry_name}

az role assignment create \
  --assignee $(az identity show --name tenant-b-mi --resource-group ${resource_group} --query principalId -o tsv) \
  --role "Cognitive Services OpenAI User" \
  --scope /subscriptions/${subscription_id}/resourceGroups/${resource_group}/providers/Microsoft.CognitiveServices/accounts/${aoai_name}

Create the docker image

az acr login --name $acr
docker build . -t $acr.azurecr.io/multitenant-ai:latest
docker push $acr.azurecr.io/multitenant-ai:latest

Deploy application manifest file

kubectl apply -f tenant-a-deployment.yaml
kubectl apply -f tenant-b-deployment.yaml

Access the streamlit UI for tenant-a and tenant-b

Access your streamlit service for tenant-a on port 8501 and tenant-b on port 8502. Open your internet browser and type in http://localhost:8501 and http://locahost:8502.

kubectl port-forward svc/tenant-a-streamlit-service 8501:8501 -n tenant-a
kubectl port-forward svc/tenant-b-streamlit-service 8502:8501 -n tenant-b

Here is a sample command for you to type into the chat box. You can see the function invocation logs section to see which agent was called, which function was triggered and what the arguments specified are. Follow the prompt from the model to provide sample personal details (Do not provide your actual personal details) and see how the model selects which storage container to upload the finalized itinerary to. Notice how the tenant id is passed into the tool invocation call by the agent. All of this is transparent to the end user, they do not need to manually set any environment variables or select any tenant. You can look through the code app.py and tools.py for more clarity.

plan me an itinerary to Singapore for 3 days

Figure 8:Passing of Tenant ID into tool invocation call.

Browse to your storage container in the azure portal to verify that the file booking_details.txt is uploaded into each tenant’s container.

Clean up

az group delete --name $resource_group --yes

Conclusion

This post discussed how you can implement Agentic AI frameworks in a multitenancy model in Azure Kubernetes Service (AKS) by leveraging powerful native cloud and Kubernetes features:

• Control plane isolation through namespaces: Namespaces provide logical isolation within a single cluster, separating workloads, configurations, and policies for different tenants without spinning up multiple clusters.
• Compute isolation through node pools and ability to select different VM types: By assigning workloads to dedicated node pools with distinct VM sizes, you ensure resource, security, and performance boundaries tailored to each tenant’s requirements.
• Identity isolation through managed identities: Each tenant’s workloads can use unique Azure managed identities, ensuring strong identity separation and secure, tenant-scoped access to Azure resources without shared secrets.
• Data isolation through RBAC and ABAC on storage accounts: Fine-grained access policies using Azure RBAC and attribute-based access control (ABAC) enable precise, tenant-aware permissions on storage resources, restricting access and operations strictly to authorized identities and contexts.

Together, these strategies deliver robust multitenancy patterns that balance security, flexibility, and operational efficiency within AKS that modern Agentic frameworks can use to delegate temporary, least-privilege credentials to agents at runtime.