Blog Post

Microsoft Developer Community Blog
2 MIN READ

Data Security: Azure key Vault in Data bricks

bhramesh's avatar
bhramesh
Icon for Microsoft rankMicrosoft
Dec 24, 2025

Securely read database connection strings in Azure Databricks

 

Why this article?

To remove the vulnerability of exposing the data base connection string in Databricks notebook directly, by using Azure key vault.

Database connection strings are extremely confidential/vulnerable data, that we should not be exposed in the DataBricks notebook explicitly. Azure key vault is a secure option to read the secrets and establish connection.

What do we need?

  • Tenant Id of the app from the app registration with access to the azure key vault secrets
  • Client Id of the of the app from the app registration with access to the azure key vault secrets
  • Client secret of the app from the app registration with access to the azure key vault

Where to find this information?

  • Under the App registration, you can find the (application) Client Id, Directory (tenant) Id.

       

Azure App registrations

     

 

  • Client secret value is found in the app registration of the service, under Manage -> Certificate & secrets. You can use an existing secret or create a new one and use it to access the key Vault secrets.

 

 

Azure App registrations

 

 

  • Make sure the application is added with get access to read the secrets. Verify the key vault you are checking and using in Databricks is the same one with read access.

You can verify this by going to the Azure key vault -> Access Policies and search for the application name.

It should show up on search as below, this will confirm that the access of the application.

 

Verify in Azure Key Vault

What do we need to setup in Databricks notebook?

  • Open your cluster and install azure.keyvault and azure-identity (installing version should be compatible with you cluster configuration, refer: https://docs.databricks.com/aws/en/libraries/package-repositories)
  • Import required libraries in the cluster

     

  • In a new notebook, let’s start by importing the necessary modules.

         Your notebook would start with the modules, followed by tentatId, clientId, client secret, azure key vault URL , secretName of the connection string           in the azure key vault and secretVersion.

          
Import the modules and setup access to azure key vault
  • Lastly, we need to fetch the secret using the below code

     

    Vola, we have the DB connection string to perform the CRUD operations.

    Conclusion:

    By securely retrieving your database connection string from Azure Key Vault, you eliminate credential exposure and strengthen the overall security posture of your Databricks workflows. This simple shift ensures your notebooks remain clean, compliant, and production‑ready.

Updated Dec 20, 2025
Version 1.0
azure
best practices
developer
python

1 Comment

  • dfengler's avatar
    dfengler
    Occasional Reader
    Jan 02, 2026

    Great article on securing secrets in Databricks! However, I'd like to suggest an even more secure approach: Azure Key Vault-backed secret scopes (https://learn.microsoft.com/en-us/azure/databricks/security/secrets/)

    The current approach still requires storing service principal credentials (tenant ID, client ID, and client secret) somewhere in your code or environment variables, which introduces security risks:

    • Client secrets in code = security vulnerability
    • Environment variables = still exposed and manageable

    A better solution: Azure Key Vault-backed secret scopes

    With Azure Key Vault-backed secret scopes, you can:

    1. Create a secret scope directly linked to your Azure Key Vault
    2. Use Databricks' built-in integration with Azure Key Vault
    3. Eliminate the need to store any credentials in your notebooks or environment

    Usage:

    # No credentials needed in code!
# Just reference the secret scope and key name
connection_string = dbutils.secrets.get(scope="my-keyvault-scope", key="db-connection-string")

    Advanced Use secrets in Spark config and environment variables (https://learn.microsoft.com/en-us/azure/databricks/security/secrets/secrets-spark-conf-env-var):

    You can also reference secrets directly in cluster configuration:

    # Spark configuration property
spark.password {{secrets/my-keyvault-scope/db-password}}

# Environment variable
SPARKPASSWORD={{secrets/my-keyvault-scope/db-password}}

    Then access them in your code:

    # From Spark config
password = spark.conf.get("spark.password")

# From environment variable (in init scripts)
password = os.environ.get("SPARKPASSWORD")

    Benefits:

    • No service principal credentials in code or environment
    • Databricks authenticates to Key Vault using managed identity
    • Secrets are never exposed in notebook output
    • Fine-grained access control via Databricks secret scope ACLs
    • Secrets automatically redacted from logs
    • Works seamlessly with cluster-scoped init scripts