AKS Tenant Migration: Considerations and Approach

Introduction:

When an Azure subscription is shifted between directories (Azure AD tenants), the Azure Container Registry (ACR) cannot be directly transferred in its current state. This blog outlines the necessary steps to transition the ACR from one tenant to another tenant.

Prerequisites and Considerations:

1. Placeholder Subscription in Target Tenant
   A newly established subscription in the target tenant to temporarily host services being reconstructed.
2. User Managed Identity in Source Tenant
   Must have ‘Contributor’ IAM permission on the source ACR resource.
3. Client Workstation
   Azure Cloud Shell or PowerShell with the Az module installed.
4. Docker Content Trust (DCT)
   Must be disabled in the source ACR (especially in source environments).

High-Level Approach

1. Freeze Changes
   Suspend any modifications to the source ACR until the migration is complete.
2. Documentation
   Record configuration settings and permissions of the source ACR.
3. New ACR Instance
   Create a new ACR in the placeholder subscription of the target tenant.
4. Configuration Matching
   Replicate the source ACR configuration in the target ACR.
5. Content Transfer
   Migrate repository images and content to the new ACR.
6. Testing
   Validate application integration with the new ACR.
7. Deletion
   Remove the original ACR on the day of subscription migration.
8. Post-Migration Setup
   Reconfigure roles and permissions in the target tenant.
9. Application Configuration
   Set up RBAC for ACR, AKS, and other services as per application needs.
10. Application Testing
    Conduct thorough testing to ensure everything functions as expected.

High-Level Steps

1. Create Subscription B in the target tenant.
2. Redeploy core non-migratable services in Subscription B.
3. Ensure no direct Entra-based authentication or cross-tenant managed identities.
4. Perform VNET peering between Subscription A (source) and Subscription B (target).
5. Redirect the application to the new AKS cluster (including gateway, DB, storage).
6. Test the application in a cross-tenant setup. Admins will need dual logins.
7. Migrate Subscription A to the target tenant.
8. Re-establish VNET peering between Subscription A and B (both now in target tenant).
9. Final application testing.

Detailed steps to be performed in source and target tenant

Source Tenant:

1. Login to azure portal, and Export ARM Template of ACR in source tenant.
2. Download the exported template on the local Machine, this will serve as reference for configuration details in the source /target tenant for redeployment
3. Now, create a User Managed Identity in source tenant. This Identity will be used to remotely access this repository from target tenant, and import ‘registry’ images. 
4. Assign the User Managed Identity contributor permissions on source ACR in source tenant
5. Create windows Virtual machine in source tenant.
6. Assign the User Managed Identity created in step #3 above in source tenant, contributor permissions on Virtual Machines created in source tenant.
7. Login To Windows VM using Bastion or RDP and open Azue powershell in Administrator mode and install Azure PowerShell modules and get access token using below commands.
   1. Install-Module -Name Az -Repository PSGallery -Force
   2. Connect-AzAccount -Identity -AccountId <identity_ID>
   3. Get-AzAccessToken #Get access token using managed identity

Note down the value of “accessToken”. Keep it safe as it is a credential that gives contributor access to source ACR. We must revoke contributor access on ACR after the steps mentioned in this guide are complete.

8. Copy following script content to windows vm and save the file as export_source_images.ps1. Replace the highlighted variable with your ACR name.

9. Now, Upload the output file “outputfile_for_acr_import_script.csv” to any of the storage account in source tenant. Run Below Command to Set Storage Context and upload outputfile_for_acr_import_script.csv file to Storage Account container.

 Target Tenant: 

1. In the placeholder subscription, proceed to set up a new Azure Container Registry, ensuring it mirrors the configuration of the original registry in the source directory. It’s important to note that the registry name must be distinct to maintain global uniqueness.
2. Open cloud shell in target tenant.
3. Import the “outputfile_for_acr_import_script.csv” in target tenant.
4. Open the Editor in PowerShell and copy following PowerShell script, after replacing the highlighted items. Save it as import_images_from_source.ps1.

5. Now, run above script using this command: .\ import_images_from_source.ps1
6. Verify from Azure Portal that all images in all repositories with tags have been migrated