Introduction

Integrating Azure Machine Learning with Azure SQL Managed Instance supports use cases such as feature engineering, model training, and the development of machine learning models using data from Azure SQL MI. This integration enables comprehensive insights and data-driven decision-making. However, deploying Azure SQL MI in reserved IP spaces like 172.17.x.x/16, which is a commonly used IP address range, can cause conflicts in customer environments, disrupting integration. This blog post provides solutions to overcome these challenges and seamlessly integrate Azure ML with Azure SQL MI.

Understanding the Constraint

Azure ML internally utilizes 172.17.0.0/16 IP address range for the Docker bridge network. This commonly used IP address range can cause errors and integration issues. Refer to the official documentation for additional information.

Strategies for Resolving the Integration Challenge

1. IP Address Range Modification

One straightforward solution, if possible, is to modify the IP address range of your Azure SQL MI deployment. By choosing a different address space that does not conflict with the commonly used 172.17.x.x/16 IP address range, can help avoid potential conflicts.

2. Using Private Endpoints

An alternative approach is using Private Endpoints with Azure SQL Managed Instance (MI), configured with an address space other than 172.17.x.x range. This blog post explores how private endpoints enable seamless integration.

Steps to resolve integration challenge using Private Endpoint

To address the challenge of integrating Azure ML with Azure SQL MI, which is already deployed in the 172.17.x.x/16 address range, follow these steps:

• Add additional address space to the VNET used in Azure SQL MI
• Establish a Private Endpoint Connection
• Configure the Private DNS Zone

The reference target architecture for the solution is as below.

The key characteristics of the reference target architecture are,

1. Adopts the recommended Hub-Spoke architecture, enhancing scalability, security, centralized management, and overall network performance.
2. The Network Virtual Appliance (NVA) used in this example is Azure Firewall and is deployed in the Hub.
3. Route table is used to configure the traffic flow between Azure ML & Azure SQL MI.

Let us examine the steps required to address the integration challenge in further detail.

Add additional address space to the VNET used in Azure SQL MI

In the VNET used by Azure SQL MI, add an additional address space apart from 172.17.x.x/16.

For example, you can add 173.17.0.0/24 address space to the VNET.

The next step is to add a subnet to this address space. This subnet can be used when private endpoint is created.

Note, there is no change to the ManagedInstance subnet which is used by Azure SQL MI. In this example, it is deployed in 172.17.2.0/27.

Establish a Private Endpoint Connection

We will utilize the private endpoint connection created in Azure SQL MI and use the same to connect from Azure ML. Thanks to Jose Manuel Jurado’s blog, which provides the steps for creating Private Endpoint connection for Azure SQL MI.

Create the private endpoint using the newly created address space & subnet.

Resource for the private endpoint should be managedInstance.

The virtual network and subnet should be the vnet where Azure SQL MI is deployed and subnet should be the subnet created in the previous step.

The DNS zone will be configured later.

Configure the Private DNS Zone

The next step is to create private DNS zone for Azure SQL MI. The naming convention for the private DNS zone is privatelink.<<DNS Zone>>.database.windows.net.

The DNS Zone in the above naming convention is derived from the hostname of Azure SQL MI. For example, if the host name is sqlmiinstance.2292572b.database.windows.net then the DNS Zone is 2292572b. In this example, the Private DNS Zone to be created will be privatelink.2292572b.database.windows.net.

After creating the private DNS zone, update the DNS zone configuration in the previously created private endpoint and associate it with the new private DNS zone.

In the Private DNS zone, add the spoke VNETs in this case, the vnets used by Azure SQL MI and Azure ML in the Virtual Network Links. This will enable DNS of Azure SQL MI from these Virtual Networks.

Testing

To test the configuration, create a compute instance in Azure ML and attempt to connect to the Azure SQL MI from the same.

The first test is to verify if the compute instance is able to reach Azure SQL MI. As shown below, Private DNS successfully resolves the FQDN of Azure SQL MI to the correct IP address. The resolved IP address is 173.17.x.x, which corresponds to the additional address space and subnet created in the earlier step. This ensures there is no conflict with the 172.17.x.x range internally used by Azure ML.

Second test is to connect to Azure SQL MI and run a query. With this configuration, the connection to Azure SQL ML is successfully established and the specific database can be accessed.

Conclusion

Integrating Azure ML with Azure SQL MI deployed in the 172.17.x.x/16 address space can pose challenges, but with thoughtful planning and the right strategies, these constraints can be effectively resolved. Using Private Endpoints can be an effective solution to resolve the issue when modifying IP address range is not feasible.

By addressing the integration challenge, you can leverage the full potential of Azure ML & Azure SQL MI and create a robust & scalable environment for your applications.