How to set up Private Link with high throughput data ingestion
Published Aug 18 2020 07:49 AM 4,590 Views
Microsoft

 

Azure Synapse Analytics supports Private Link enabling you to securely connect to SQL pools via a private endpoint. This quick how-to guide provides a high-level overview and walks you through how to set up Private Link when you’re using the COPY statement for high-throughput data ingestion. Using the COPY statement is a best practice when data loading where the experience is simple, flexible, and fast.

                                                                                     

The following diagram illustrates a simple set-up and the interactions happening across various components when Private Link is enabled for a SQL pool with a single VM within a VNet accessing the SQL endpoint (front-end control node):

 

Loading using Private Link.png

 

 

 

 

 

The following settings are required on your SQL Server when securing your SQL pool:

  1. Deny public network access: Yes
  2. Allow Azure services and resources to access this server: No
  3. Create a Private endpoint

These steps can all be easily done in the Azure portal. After configuring your SQL Server, access to the SQL pool is secured which can only be done via the private endpoint in your VNet.

 

The following settings are required on your storage account that you are loading from:

  1. Allow access from Selected Networks: On
  2. Create a Private endpoint
  3. Create a system-assigned MI in your AAD tenant for your SQL Server via PowerShell
  4. Give the required Storage Azure role (Storage Blob Data Reader or higher) to your system-assigned MI
  5. Allow trusted Microsoft services to access storage: Yes
    1. This configuration allows the SQL pool backend compute nodes to bypass the storage network configurations using the system-assigned MI for your specific SQL Server resource. This allows the COPY statement to directly access the storage account for high through data ingestion over the Azure backbone.

For more details on setting up your storage account for COPY access, you can visit the following documentation. You can visit the following links to learn how Azure Synapse provides secure network access for your analytics platform:

 

6 Comments
Version history
Last update:
‎Aug 18 2020 12:39 PM
Updated by: