Secure Delta Sharing Between Databricks Workspaces Using NCC and Private Endpoints

This guide walks you through the steps to share Delta tables between two Databricks workspaces (NorthCentral and SouthCentral) and configure Network Connectivity Configuration (NCC) for a Serverless Warehouse. These steps ensure secure data sharing and connectivity for your workloads. 

Part 1: Delta Sharing Between Workspaces

 

1. Access Delta Shares

1. • From your NorthCentral Workspace, go to Catalog.  

• • Hover over Delta Shares Received. 

• • When the icon appears, click it. → This will redirect you to the Delta Sharing page. 

2. Create a New Recipient

• • On the Delta Sharing page, click Shared by me. 

• • Click New Recipient. 

• • Fill in the details: 

• • • Recipient Name: (Enter your recipient name) 

• • • Recipient Type: Select Databricks 

• • • Sharing Identifier: azure:southcentralus:3035j6je88e8-91-434a-9aca-e6da87c1e882  

• • • • To get the sharing identifier using a notebook or Databricks SQL query: (SQL) SELECT CURRENT_METASTORE(); 

• • Click Create. 

3. Share Data

1. • Click "Share Data". 

• • Enter a Share Name. 

• • Select the data assets you want to share. 

Note: Please disable History for the selected data assets, as the current data snapshot. Disabling the History option on the Delta Share will simplify the share and prevent unnecessary access to historical versions. Additionally, review whether you can further simplify your share by partitioning the data where appropriate. 

• • Add the recipient's name you created earlier. 

• • Click Share Data. 

4. Add Recipient 

1. •  From the newly created share, click Add Recipient. Select your South-Central Workspace Metastore ID.  

5. South-CentralWorkspace  

• • In your South-Central Workspace, navigate to the Delta Sharing page.  

• • Under Shared with me tab, locate your newly created share and click on it.  

• • Add the share to a catalog in Unity Catalog. 

Part 2: Enable NCC for Serverless Warehouse

 

6. Add Network Connectivity Configuration (NCC) 

• • Go to the Databricks Account Console: https://accounts.azuredatabricks.net/ 

• • Navigate to Cloud resources, click Add Network Connectivity Configuration. 

• • Fill in the required fields and create a new NCC for SouthCentral. 

7. Associate NCC with Workspace 

• • In the Account Console, go to Workspaces. 

1. • Select your SouthCentral workspace, click Update Workspace. 

• • From the Network Connectivity Configuration dropdown, select the NCC you just created. 

8. Add Private Endpoint Rule 

• • In Cloud resources, select your NCC, select Private Endpoint Rules and click Add Private Endpoint Rule. 

 

• • Provide:  

• • Resource ID: Enter your Storage Account Resource ID in NorthCentral. 

• • Note: This can be found in your storage account (NorthCentral). Click on “JSON View” top right.  

• • Azure Subresource type: dfs & blob. 

9. Approve Pending Connection 

• • Go to your NorthCentral Storage Account, Networking, Private Endpoints. 

1. • You will see a Pending connection from Databricks. 

1. • Approve the connection and you will see the Connection status in your Account Console as ESTABLISHED. 

You will now see your share listed under “Delta Shares Received” 

Note: If you cannot view your share, run the following SQL command: GRANT USE_PROVIDER ON METASTORE TO `username@xxxx.com`.