Azure Databricks supports Azure Active Directory (AAD) tokens (GA) to authenticate to REST API 2.0. The AAD tokens support enables us to provide a more secure authentication mechanism leveraging Azure Data Factory's System-assigned Managed Identity while integrating with Azure Databricks.
Benefits of using Managed identity authentication:
Earlier, you could access the Databricks Personal Access Token through Key-Vault using Manage Identity. Now, you can directly use Managed Identity in Databricks Linked Service, hence completely removing the usage of Personal Access Tokens.
High-level steps on getting started:
Note: Please toggle between the cluster types if you do not see any dropdowns being populated under 'workspace id', even after you have successfully granted the permissions (Step 1).
Sample Linked Service payload:
{
"name": "AzureDatabricks_ls",
"type": "Microsoft.DataFactory/factories/linkedservices",
"properties": {
"annotations": [],
"type": "AzureDatabricks",
"typeProperties": {
"domain": "https://adb-***.*.azuredatabricks.net",
"authentication": "MSI",
"workspaceResourceId": "/subscriptions/******-3ab0-48f2-b171-0f50ec******/resourceGroups/work-rg/providers/Microsoft.Databricks/workspaces/databricks-****",
"existingClusterId": "****-030259-dent495"
}
}
}
Note: There are no secrets or personal access tokens in the linked service definitions!
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.