Azure Databricks & Azure Confidential Computing are excited to announce the Public Preview of Azure Databricks support for Azure Confidential Computing options for data plane.
Azure Databricks now supports using Confidential computing VM options while creating clusters. This new feature emphasizes our commitment to cloud security. With this new feature, now you can run your Azure Databricks workload securely & confidentially on Azure. Azure confidential computing generally attracts customers in highly regulated industries & regions but is equally applicable to any business that is looking to run their analytics, machine learning and other workloads in a privacy preserving way.
Azure already encrypts data at rest and in transit, and confidential computing helps protect data in use, including the cryptographic keys. Azure confidential computing helps customers prevent unauthorized access to data in use, including from the cloud operator, by processing data in a hardware-based and attested Trusted Execution Environment (TEE).
ACC Technology & Offerings
Please start here to understand the technology behind Azure Confidential Computing in depth and also learn about the GA and in-preview offerings.
The feature release allows adding Confidential VM options while creating clusters in the customer Data plane only. More confidential features will be coming soon.
Configuring Confidential VMs on Azure Databricks
When creating a data plane cluster, select one of the confidential VM options as discussed here. By selecting a confidential VM as the compute option, you would be able to run your workloads in a privacy preserving way & confidential manner. Confidential VMs are available currently in East US, West US, North EU, West EU, with additional regions to be added later.