Azure Architecture Blog

9 MIN READ

Gov Topics: AVD Cross-Cloud / Cross-Tenant Deployment

Microsoft

Jun 23, 2022

I’ve run into a great many customers with complicated AAD / Azure tenant setups – some the result of mergers and acquisitions, others sometimes to do regulatory reasons.

Thousands of Microsoft U.S. government and regulated industry customers use the Office 365 Government Community Cloud (GCC) for end-user identity and productivity services. Azure Active Directory (AAD) for the GCC lives in a U.S.-only partition of AAD in the Azure Commercial service.

Many governmental agencies leverage Azure Government for non-Office 365 services, which cannot directly leverage the AAD identities of GCC users.

This post is intended to demonstrate how Azure Virtual Desktop (AVD) might be deployed using Azure Commercial as the control plane, and Azure Government for compute. Such a deployment enables seamless access for GCC user identities, while maintaining data sovereignty inside the Azure Government environment, with additional administrative burden. This same process / deployment pattern can be used to enable cross-tenant access to VMs within the same Azure cloud.

Typical AVD Deployment

A typical Azure Virtual Desktop deployment would be configured within a single Azure subscription, and leverage Azure Active Directory (AAD) synchronized to Active Directory – with AVD host pool VMs joined to the Active Directory Domain Services (AA DS – or simply AD), as shown here:

Typical AVD Deployment: Subscription tied to AAD Tenant

AVD (and other services) relies on user identities in AAD being connected to ADDS to provide access to resources.

Challenge: Separate Clouds Complicates Deployment

While Microsoft recommends customers adopt a one cloud approach for the full seamless experience across all Microsoft cloud services, often business requirements force consumption of multiple clouds – including Sovereign Clouds available in Azure (US Government, China, and Germany). Separate cloud deployments may complicate identity access across clouds, where regulatory or technical requirements have mandated a separation of identity.

The need for U.S. governmental agencies to meet Criminal Justice Information Services (CJIS), IRS 1075, or other compliance bars (including Department of Defense Impact Level 4 and higher) has pushed law enforcement related and revenue responsible agencies toward Azure Government for the hosting of data and services. Many of these agencies have procured and deployed Office 365 using Microsoft Government Community Cloud (GCC) offering, which ensures data will be stored within the continental United States.

Office 365 GCC utilizes Azure Active Directory (AAD) within the Azure Commercial infrastructure and identities within Azure Commercial cannot be used to directly access service in Azure Government. Azure Government uses separate and distinct infrastructure (including Azure Active Directory) to provide services, as detailed here. While O365 GCC agencies have AAD tenants in Azure Government, they are commonly only populated with administrator credentials for “fabric” management without, access to O365 license benefits in the O365 GCC tenant.

Problem: Need Access to Different Cloud / Subscription not Tied to Tenant

For AVD in particular, Azure Active Directory (AAD) serves to manage access to AVD published resources for end users by providing authentication and access to applications and desktops enabled for their AAD identities.

Users and groups are granted access to AVD services through the assignment of their identities to Application Pools (via the role assignment “Desktop Virtualization User”) and later authenticated in the VM hosting the resource using their connected Active Directory credential.

Typical deployments of AVD require:

Azure Active Directory synchronized with Active Directory Domain Services (AD) or AAD DS
Active Directory Domain Services available in Azure (for machine join and profile management)
Microsoft 365 licensing entitlements to support the use of Windows 10 Enterprise as well as key desired security features (example: Conditional Access)

The dependency upon integrated Azure AD / AD for AVD deployments can be easily met by many customers, however Microsoft customers who have deployed in Microsoft national (sovereign) clouds face significant complications and may appear to be blocked from deploying AVD and other Azure services (including AD integrated Azure Files, for example).

Solutions to synchronize identities and federate cross cloud may be an option, but complicate integration and may require additional licensing for additional (duplicate) identities – those options are not detailed here.

Instead, the approach could be to use the existing identity and AVD core services in Azure Commercial, connected to provisioned compute in Azure Government:

Cross-Cloud / Cross-Subscription Registration

This approach can provide seamless access to end-users (same login experience they would have for Office 365 resources) with desktop and application services inside Azure Government, near existing data assets.

Cross Cloud / Cross Subscription Deployment

An AVD deployment which crosses cloud / subscription boundaries has the same high-level requirements of a typical deployment:

Azure Active Directory synchronized with Active Directory Domain Services (AD) or AADDS
Active Directory Domain Services available in Azure (for machine join and profile management)
Microsoft 365 licensing entitlements to support the use of Windows 10 Enterprise as well as key desired security features (example: Conditional Access)

The proposed process to create a cross cloud / cross subscription deployment takes advantage of the fact that VMs in Host Pools do not need to live within the subscription where the host pool is defined – the VMs have the ability to be manually registered the host pool to provide session access even if they are in another cloud. Detail on how to manually attach a VM to an AVD host pool can be found here: https://docs.microsoft.com/en-us/azure/virtual-desktop/create-host-pools-powershell#prepare-the-virtual-machines-for-windows-virtual-desktop-agent-installations .

The Fine Print (AKA the Bad News)

Deploying and using AVD when the VMs are separated from the AVD pool / Azure active directory does introduce some limitations that need to be pointed out and planned around.

Using Azure Files for Storage

Azure Files is often used to store virtual disk files (VHDs) for end user profiles managed by FSLogix. The shares/volumes use NTFS (AD) permissions for the file system, but Azure Files share access is typically granted via Azure Active Directory (which is not exposed to this environment!).

Azure Files now has “default” share permissions that may be set, eliminating this challenge – you can read more about that here: Control access to Azure file shares – on-premises AD DS authentication | Microsoft Docs

You can also deploy FSLogix / MSIX App Attach using Azure NetApp Files (ANF), Nasuni, or traditional file server VMs, since they do not include a dependance on Azure Active Director.

Monitoring / Management / Scaling

Having the AVD control plane in a separate cloud from the VMs means that monitoring is way more complicated (where’s your Log Analytics?), and you’ll have to work that out. The AVD “Insights” you might provision in the portal has no support for VMs in another cloud / tenant!

MORE IMPORTANT may be your strategy to manage the VMs! How will you (for example) manage the power state for the VMs (auto-scaling) since your Azure Portal (connected to a GCC tenant) has no ability to shutdown / start up VMs in Azure Government (or in whatever tenant the VMs live in).

Never Fear… Nerdio Manager is Here!

Microsoft hasn’t (yet) added support for cross-cloud / cross-subscription AVD deployments, but that doesn’t mean others haven’t innovated!

A great example is the support built into Nerdio Manager for Enterprise for these types of complex AVD deployments. Nerdio does all sorts of great things with AVD – optimizing costs, simplifying image management/deployment….and it’s in the Azure Marketplace (including in Azure Gov!):

Nerdio Manager in Azure Gov Marketplace

Now that all that is out of the way, let me show you how to deploy cross-cloud.

AVD Cross-Cloud Walk-Through - Deployment Process

Since AVD is available in Azure Commercial and Azure Government, testing the process is simplified using the Azure Portal in both environments to prepare the infrastructure, as well as leveraging a small bit of PowerShell automation to finish the process (but the same process can be use cross-tenant in the same cloud).

The High-level steps are:

Create empty (target) AVD Host Pool in Azure Commercial (subscription with AAD access)
Create (source) AVD host pool with Session Hosts in Azure Gov (subscription without AAD access, but with AD Domain Controller line of sight)
Connect Session to with “target” Host Pool
- Get AVD host pool token from target Host Pool
- Unregister VM from Source host pool
- Register VMs with target host pool

Details of the process are outlined below.
(Please ignore the old screen shots from BEFORE WVD was renamed to AVD!).

Step 1: Create Empty Target Pool in Azure Commercial (~2 minutes)

Create an empty host pool in the cloud->tenant->subscription which has access to the end user Azure Active Directory:

Create Empty Host Pool in CommercialCreate Empty Host Pool in Commercial

DO NOT CREATE ANY VMs in this pool.

NO VMs!!!!!Empty Host Pool

Step 2: Create Full Source Pool in Azure Government (~15 minutes w/VM provisioning)

Log into the Azure environment that will host your session host (the other cloud->tenant->subscription which does not have access to AAD, but which does have line of sight to an Active Directory Domain Controller connected to that AAD).

Create this host with VMs:

Create Host Pool in Gov - With VMsCreate Host Pool in Gov - With VMs

This time, select the option to create VMs:

Deploy VMs in Azure GovFull Host Pool (for now) in GovVMs Running in Gov

Step3: Register VMs with Target Pool with PowerShell (~5 minutes)

Once the target (empty) and source (full) pools are created, connecting the session hosts created in the “source” (Gov) Host Pool with the “target” (Commercial) Host Pool is a straightforward process of unregistering and reregistering the WVD Agent.

To simply the process, PowerShell is detailed below which can be used to automate the process. Filling in a hand full of variables (host Pool names, subscription IDs, and resource group names) and then executing the code / signing into each Azure environment is all that is needed.

Required Information

Other than properly permissioned credentials for each cloud->tenant->subscription, only the following information is required to run the code (need to update with your deployment specific information):

#################################
# Azure Settings & other parms
#################################
$GovResourceGroup 	= "GBBComm"
$GovSubscriptionID	= "11111111-1111-1111-1111-111111111111”
$GovHostPool		= "Commercial"	
$CommResourceGroup 	= "kelbleyWVD"
$CommSubscriptionID	= "22222222-2222-2222-2222-222222222222”
$CommHostPool		= "Gov"

Install WVD Azure PowerShell Module

If you do not have it already, you will need the Azure WVD PowerShell Module (as well as other Azure-related modules to connect to Azure and manipulate the VMs):

#################################
#Step 0 - Install WVD Module... in case you don't have it
#################################
# Install-Module -Name Az.DesktopVirtualization

Retrieve Target Host Pool Token

Log into the cloud->tenant->subscription with AAD access, and retrieve the token required to join VMs to the deployment:

#Step 1 - Connect to Azure Commercial and retrieve Commercial Host Pool Token
#################################
Connect-AzAccount
Select-AzSubscription -SubscriptionId $CommSubscriptionID
$CommPool = Get-AzWvdRegistrationInfo -ResourceGroupName $CommResourceGroup -HostPoolName $CommHostPool
$Token = $CommPool.Token

Create Script to Inject into VMs to Unregister / Reregister

Save a custom .PS1 file on your local host which includes the necessary host pool token, with all necessary commands to unregister and reregister the VM to the target host pool. The file will be executed later in the code in each VM:

#################################
#Step 2 - Build Command to run in VMs
#################################
$remoteCommand =
@"
#### Run Unregister from Gov Pool / Reregister with Commercial Pool
Stop-Service RDAgentBootLoader
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\RDInfraAgent' IsRegistered -Value 0
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\RDInfraAgent' RegistrationToken -Value $Token
Start-Service RDAgentBootLoader
Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\RDInfraAgent'
"@
### Save the command to a local file
Set-Content -Path .\RegNewPool.PS1 -Value $remoteCommand

Retrieve Source Host Pool VMs and Reconfigure

Log into Azure Government (the cloud->tenant->subscription with VMs in a Host Pool), get the list of VMs, disconnect them from that pool, and reconnect them to the other pool.

#################################
#Step 3 - Remove VMs from Gov Pool & register with Commercial Pool
#################################
Connect-AzAccount -EnvironmentName AzureUSGovernment 
Select-AzSubscription -SubscriptionId $GovSubscriptionID
$VMs = Get-AZWVDSessionHost -ResourceGroupName $GovResourceGroup -HostPoolName $GovHostPool 
Foreach ($VM in $VMs) { 
	$DNSname = $VM.name.split("/")[1]
	$VMname = $DNSname.split(".")[0]
	#################################
	#remove host from Gov Pool
	#################################
	Remove-AZWVDSessionHost -ResourceGroupName $GovResourceGroup -HostPoolName $GovHostPool -SessionHostName $DNSname
	#################################
	# call PowerShell inside VM to register with Commercial Pool
	#################################
	Invoke-AzVMRunCommand -Name $VMname -ResourceGroupName $GovResourceGroup -CommandId 'RunPowerShellScript' -ScriptPath .\RegNewPool.PS1
}
#################################
### Clean-up the local file
#################################
Remove-Item .\RegNewPool.PS1

Setup Process Results

After (hopefully) successful execution of the PowerShell, VMs from the source (Gov) Host Pool should be registered and available in the target (Commercial) Host Pool:

VMs in Gov REGISTERED with Commercial!

You will need to configure a Workspace / App Groups /add Access, but one that is completed published desktop and application resources running on the VMs will be accessible:

AVD Remote Apps and Destktop published to AAD Commercial from VMs in Azure Gov

Here’s all the code together, should you need it:

###############################################################################################################
#
#  Configures WVD in Azure Commercial to use VMs in Azure Government!
#
# Assumes: 
#	pool created in Azure Commercial (with no VMs in it!),
#	pool created in Azure Gov WITH VMs, that will be registered in the Commercial pool
#	(VMs in Gov pool will be poached and registered in Commercial...just using Gov Pool to provision!)
# 
#  questions - johnkel at Microsoft.com
#
#  Version History:
#	Update 9/29/2020 - (bugs) fixed a few variables that had been adjusted 
#
###############################################################################################################
#
#################################
# Azure Settings & other parms
#################################
$GovResourceGroup 	= "GBBComm"
$GovSubscriptionID	= "11111111-1111-1111-1111-111111111111"
$GovHostPool		= "VMsforCommercial"	

$CommResourceGroup 	= "CommercialWVD"
$CommSubscriptionID	= "22222222-2222-2222-2222-222222222222"
$CommHostPool		= "GovVMs"	

#################################
#Step 0 - Install WVD Module... in case you don't have it
#################################
# Install-Module -Name Az.DesktopVirtualization

#################################
#Step 1 - Connect to Azure Commercial and retrieve Commercial Host Pool Token
#################################
Connect-AzAccount 
Select-AzSubscription -SubscriptionId $CommSubscriptionID
$CommPool = Get-AzWvdRegistrationInfo -ResourceGroupName $CommResourceGroup -HostPoolName $CommHostPool
$Token = $CommPool.Token

#################################
#Step 2 - Build Command to run in VMs
#################################
$remoteCommand =
@"
#### Run Unregister from Gov Pool / Reregister with Commercial Pool
Stop-Service RDAgentBootLoader
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\RDInfraAgent' IsRegistered -Value 0
Set-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\RDInfraAgent' RegistrationToken -Value $Token
Start-Service RDAgentBootLoader
Get-ItemProperty -Path 'HKLM:\SOFTWARE\Microsoft\RDInfraAgent'
"@
### Save the command to a local file
Set-Content -Path .\RegNewPool.PS1 -Value $remoteCommand

#################################
#Step 3 - Remove VMs from Gov Pool & register with Commercial Pool
#################################
Connect-AzAccount -EnvironmentName AzureUSGovernment 
Select-AzSubscription -SubscriptionId $GovSubscriptionID
$VMs = Get-AZWVDSessionHost -ResourceGroupName $GovResourceGroup -HostPoolName $GovHostPool 
Foreach ($VM in $VMs) { 

	$DNSname = $VM.name.split("/")[1]
	$VMname = $DNSname.split(".")[0]
	#################################
	#remove host from Gov Pool
	#################################
	Remove-AZWVDSessionHost -ResourceGroupName $GovResourceGroup -HostPoolName $GovHostPool -SessionHostName $DNSname

	#################################
	# call PowerShell inside VM to register with Commercial Pool
	#################################
	Invoke-AzVMRunCommand -Name $VMname -ResourceGroupName $GovResourceGroup -CommandId 'RunPowerShellScript' -ScriptPath .\RegNewPool.PS1
}
#################################
### Clean-up the local file
#################################
Remove-Item .\RegNewPool.PS1

Updated Jun 23, 2022

Version 1.0

infrastructure