Secure Cross-Tenant Network Connectivity for Azure IoT Solutions

The Azure IoT Hub is the link between an IoT application and its attached devices.  Deploying Azure IoT Hub-based solutions across separate Azure tenants is sometimes implemented by organizations that require the device interaction functionality to be deployed in complete separation from the applications and analytical capabilities that utilise the ingested device data.  A typical architecture in this scenario would involve an IoT hub in the “OT Tenant” that interacts with the IoT devices, ingests the data and routes the data to an event hub in the “IT Tenant” to feed the analytics and data stores (Note that other patterns, e.g., routing data from an IoT Hub to Azure Storage, also follow the same methodology presented here).

Routing from the IoT Hub to the Event Hub involves creating a custom routing endpoint on the IoT Hub using the connection string of the Event Hub. For the scenario in Figure 1, as these two services are within separate tenants, the creation of the custom endpoint on the IoT hub cannot be done using the Azure portal but has to be completed using a method such as Azure CLI.  In addition to this, the Event hub has to be accessible over the public internet for the data to flow between the two tenants (Readers not familiar with Azure Active Directory and the concept of tenants can learn more here).

However, security best practice dictates that both the IoT Hub and the Event Hub should be bound to an Azure Virtual Network (VNet) and that their public internet access should be blocked.  Azure networking services provide a number of capabilities targeting PaaS services security requirements. For the IoT Hub this is typically achieved by a combination of Azure Private Endpoint (Azure Private Link) for ingress connectivity, and using trusted Microsoft services exception (with a Managed Identity) for egress connectivity to other Azure resources and disabling public internet access.  A similar approach can be used for an Event Hub with a combination of Azure Private Endpoints and IP firewall rules.

A private endpoint is a component of Azure Private Link that allows private and secure connectivity without the requirement for Public IPs. The private endpoint (PE) uses a private IP address from a VNet, exposed as an additional network interface (NIC), effectively bringing the service within the network perimeter of the VNet itself.

In the arrangement shown in Figure 2, both services are using Private Endpoints located within VNets belonging to the two separate tenants and with public internet blocked to both hubs. This arrangement means that the IoT Hub cannot route data to the Event Hub because the Managed Identity approach mentioned above cannot be used across the boundary of two different Azure Active Directory tenants.

The architecture in Figure 2 needs to be modified to allow access to the Event Hub inside the OT VNet using the power of Private Link as described below and depicted in Figure 3:

1. An Azure Function is introduced in the OT tenant with VNet integration enabled.  Virtual Network integration allows the Function App to access resources inside a VNet, including Private Endpoints, thus it can poll the IoT Hub via Private Link and get triggered by new events. Notice that the Function VNet integration needs a dedicated subnet. The Azure Function is also provided with a Private Endpoint for inbound communication, which will allow resources inside the VNet (e.g. virtual machines) to reach the Function, without needing to allow inbound access to the Function public front end.
2. Leveraging Azure Private Links' built-in ability to work cross Azure-AD tenant boundaries, a Private Endpoint is deployed within the OT VNet for access to the IT Tenant's Event Hub.  This provides the resources in the OT VNet with access to the Event Hub thus allowing the Azure Function to use output binding to send data to the Event Hub, via its VNet integration.  The end result is a secure path without use of Public IPs, between the IoT Hub and the Event Hub.  As the Event Hub is in a different tenant to its Private Endpoint then manual approval of the Endpoint and DNS set up are needed to for this arrangement to work.

Now that the data flow is established to the IT tenant in accordance with the security requirements mentioned earlier, other services can be used to pick the data from the Event Hub for storage and processing.  Figure 4 shows a typical combination of services added to the IT tenant:

1. Azure Stream Analytics can access the Event Hub using a Managed Identity. A Managed Identity is a managed application registered in Azure Active Directory that, in this case, represents the Stream Analytics job. The managed application is used to authenticate to a targeted resource, including Event Hubs that are behind a firewall or a VNet.
2. A Managed Identity can also be used to allow the Stream Analytics Job to access a Blob Storage which is behind a firewall.  It is important to note that the Azure Stream Analytics Job can only use its own Private Endpoint if it is deployed as a dedicated cluster.  The use of Managed Identities is required with the more common Stream Analytics Job deployment option, in order for the Job to access resources that do not have public internet access.
3. Both the Event Hub and the Blob Storage in the IT tenant must allow trusted Azure services to bypass their firewalls.  The Managed Identity of Stream Analytics provides the “Trust” while bypassing the firewall is achieved on the network configuration side of the Event Hub and the Blob Storage.
4. An Azure Function is integrated with the IT Vnet in a similar manner to the configuration on the OT Vnet. This allows the Function to access resources in the Vnet, e.g. to be triggered by events from the Event Hub.

Conclusions

Microsoft Azure PaaS services provide IT architects with configuration options allowing them to meet to most demanding performance and security requirements.  The scenario presented in this document demonstrates how a collection of Azure services that are typically used in IoT scenarios, can be combined with a high degree of network security, even under conditions where separate Azure Active Directory tenants are mandated.  By using Azure Private Link the components of the IoT solution can be accessed over private endpoints in VNets, with traffic between the different VNets travelling only in the Microsoft backbone network. Exposing services to the public internet is not required as part of the solution proposed herein.

The example discussed also shows how service providers can render services in their own virtual network and consumers can access those services in their respective local virtual network. The Private Link platform will handle the connectivity between the consumer and services over the Azure backbone network. Connection requests to resources (e.g. accessing the IT tenant Event Hub from the OT tenant) are managed using a consumer/provider approval flow that is completely under the control of the service provider.

By examining the approaches used in this article, and following the included links for more details, IoT architects and developers can start using Azure networking features to secure the end to end solutions thus meeting the demands of their customers and organizations.