Data Security: Azure key Vault in Data bricks

Why this article?

To remove the vulnerability of exposing the data base connection string in Databricks notebook directly, by using Azure key vault.

Database connection strings are extremely confidential/vulnerable data, that we should not be exposed in the DataBricks notebook explicitly. Azure key vault is a secure option to read the secrets and establish connection.

What do we need?

• Tenant Id of the app from the app registration with access to the azure key vault secrets
• Client Id of the of the app from the app registration with access to the azure key vault secrets
• Client secret of the app from the app registration with access to the azure key vault

Where to find this information?

• Under the App registration, you can find the (application) Client Id, Directory (tenant) Id.

Azure App registrations

• Client secret value is found in the app registration of the service, under Manage -> Certificate & secrets. You can use an existing secret or create a new one and use it to access the key Vault secrets.

Azure App registrations

• Make sure the application is added with get access to read the secrets. Verify the key vault you are checking and using in Databricks is the same one with read access.

You can verify this by going to the Azure key vault -> Access Policies and search for the application name.

It should show up on search as below, this will confirm that the access of the application.

Verify in Azure Key Vault

What do we need to setup in Databricks notebook?

• Open your cluster and install azure.keyvault and azure-identity (installing version should be compatible with you cluster configuration, refer: https://docs.databricks.com/aws/en/libraries/package-repositories)
• Import required libraries in the cluster

   

• In a new notebook, let’s start by importing the necessary modules.

         Your notebook would start with the modules, followed by tentatId, clientId, client secret, azure key vault URL , secretName of the connection string           in the azure key vault and secretVersion.

Import the modules and setup access to azure key vault

• Lastly, we need to fetch the secret using the below code

   

  Vola, we have the DB connection string to perform the CRUD operations.

  Conclusion:

  By securely retrieving your database connection string from Azure Key Vault, you eliminate credential exposure and strengthen the overall security posture of your Databricks workflows. This simple shift ensures your notebooks remain clean, compliant, and production‑ready.