Preview of multiparty analytics with Azure Confidential Clean Rooms
Today, we are excited to announce the preview of multiparty analytics feature of Azure Confidential Clean Rooms, a fully managed service that allows customers and their partners to securely analyze privacy-sensitive datasets from multiple parties. It uses confidential compute enabled Apache Spark-based big-data analytics (Spark SQL) which helps protect their raw data from other collaborators and from the Azure operator by performing computations in a Trusted Execution Environment (TEE). Privacy-sensitive datasets include personally identifiable information (PII), protected health information (PHI) and cryptographic secrets. Organizations across industries are increasingly looking to supplement their data with data from business partners, to build a complete view of their business. For example, brands, publishers, and their partners need to collaborate using datasets containing Intellectual Property (IP) to improve the relevance of their campaigns. Confidential data clean rooms help solve this challenge by enabling organizations to share and analyze granular datasets in a secure environment that helps prevent raw data exfiltration—protecting intellectual property, preserving customer privacy, and addressing concerns around regulatory compliance. You can sign up for the preview here Key Features Fully Managed: Azure takes care of the infrastructure provisioning and scaling with no user intervention. This significantly reduces your onboarding effort allowing you to focus on the queries and insights, not on infra management. Confidential Spark SQL: Spark SQL allows you to query large datasets and run complex queries in a distributed computing environment. In the confidential computing enabled version, the Spark driver and executors are fully attested policy-governed enclaves running as virtual nodes on confidential Azure Container Instances (ACI) which helps prevent exfiltration of collaborators’ data during query execution. Governance: Helps manage membership to cleanrooms, enables and verifies approval for queries from relevant collaborators before executing them and verifies consent to access sensitive collaborator data. It also helps generate tamper-resistant audit trails containing salient clean room events. This is made possible with the help of an implementation of the Confidential Consortium Framework (CCF). Telemetry: Throughout every clean-room run, detailed logs are streamed out in real time to monitor performance, troubleshoot issues, and keep the analytics healthy — all without ever exposing the collaborators’ data at any time. Verifiable trust: Cryptographic remote attestation viz. full attestation based on confidential hardware reports allows independent verification of the TEE along with along with all components that are part of it, without just trusting the cloud provider, before sensitive data and decryption keys are made available to the TEE Open-source containers: All Microsoft provided cleanroom containers and sidecars are open-sourced here and can be verified for provenance and integrity guarantees using GitHub artifact attestation Use Cases Multi-party confidential big-data analytics unlocks value in scenarios where data sensitivity, regulatory pressure, or competitive concerns previously blocked collaboration. These are some early scenarios that can benefit from this. Media & Advertising Collaboration of advertiser CRM data with publisher data for audience targeting and segment activation. Collaboration of audience data with measurement partners for measurement and attribution. Banking & Finance Collaboration between banks and insurance firms to upsell relevant products to existing bank customers without sharing raw data from either side Collaboration with retailers to generate customized offers for bank customers, without exposing either party’s underlying data. Government & Public Sector Secure collaboration of data across government departments to deliver better citizen welfare outcomes. Secure collaboration between government and private enterprises on shared-interest workloads such as traffic monitoring and weather systems. Healthcare Enable healthcare firms — including biopharma organizations — to combine their data with third-party institutions to accelerate clinical development, like identifying eligible participants for a clinical trial, without exposing underlying patient data. Combine patient datasets across hospitals to study disease patterns or outcomes without exposing sensitive protected health information. "A higher standard for protecting user privacy and trust, the phase-out of third-party cookies, and global regulations demand more sophisticated data collaboration tools to support advertising marketplaces. Azure Confidential Cleanrooms (ACCR) provides a secure, feature-rich, and flexible foundation to implement privacy-preserving functions and enable insights without sharing privacy-sensitive data outside of organization boundaries. Built on the Azure Confidential Compute (ACC) platform and offering cohesion with Azure's diverse set of services, ACCR offers the attestation, audit, fine-grained access control, and verifiable trust tools required for secure and privacy-safe data collaboration in today's world." — Andrei Mackenzie, Engineering Manager, Microsoft AI "Azure Confidential Clean Rooms enabled our team to evaluate how clean room capabilities can support secure, governed data collaboration at scale. Through the Proof-of-Concept (PoC), we explored how privacy-preserving workflows, trusted access controls, and scalable compute can create a stronger foundation for responsibly leveraging first-party data. This helps reduce operational friction while supporting business growth, improving customer engagement, and enabling more relevant customer experiences." — Nic Dregne, Director, Microsoft AdTech Engineering Beyond Spark SQL Realizing other multi-party scenarios like custom analytics, ML training and inferencing on Azure Confidential Clean Rooms is in our roadmap. If you have such a scenario to be realized, you can fill in and submit the preview signup form with the details of your scenario and we’ll get back to you. Learn More · Signup for the preview of Azure Confidential Clean Rooms for Analytics · Confidential Consortium Framework (CCF) · Virtual Nodes on Azure Container InstancesConfidential agentic AI on Azure helps ServiceNow respond to sales commission inquiries in seconds
Introduction AI is transforming how businesses operate and innovate, unlocking opportunities across industries to pioneer new business models, solve previously intractable challenges, and create breakthrough experiences. ServiceNow is at the forefront, deploying powerful, confidential AI agents leveraging confidential computing. Their sales commission help desk faced mounting challenges, supporting their sales force. With thousands of commission inquiries annually requiring access to sales compensation plan information, the help desk needed a solution to accelerate response times while maintaining strict data privacy and security standards. Applications ServiceNow Digital Technology leverages cutting-edge AI to streamline business operations, increase operational efficacy, and enhance employee experiences. The sales commission help desk handles inquiries ranging from policy questions to payout explanations, requiring aggregation and analysis of sensitive data from multiple systems. The manual process of responding to these inquiries—which involves gathering, anonymizing, and analyzing sensitive employee data, sales quotas, and commission structures—created bottlenecks, with resolution times stretching to days for the most complex cases. Use Cases To address ongoing commission management challenges, ServiceNow’s Digital Technology team partnered with Opaque Systems and Azure confidential computing team to design, build, and implement Confidential Agents that enable secure, autonomous AI systems with cryptographic privacy guarantees and auditability. The solution provides their help desk team with instantaneous access to encrypted personal commission data across multiple systems, while AI agents automatically analyze requests and generate custom responses. By integrating securely with various data sources, the system maintains strict privacy controls and compliance while delivering rapid, trusted insights. Every action taken by the AI agents is cryptographically verified, creating an immutable record of data access and usage. This generates detailed audit trails that meet compliance and strengthen governance protocols. Through hardware-based encryption on Azure NCCads H100 v5 confidential virtual machines augmented by NVIDIA H100 Tensor Core GPUs for accelerated computing running on their Microsoft Azure subscription service, the services built by ServiceNow can now harness the full power of AI technology without compromising on capabilities. Opaque’s Confidential AI Platform unlocks new performance potential of AI models that demand high-performing computational resources for all the commission requests, while maintaining robust protection of compensation data, setting a new standard for secure, efficient commission management. Accelerate, Reclaim, and Save Opaque's Confidential AI Agents architecture was uniquely built with NVIDIA H100s to help ServiceNow’s transformation. Once AI agents are connected to sensitive data, every aspect of agent operation maintains verifiable privacy and security, including real-time attestation that verifies agent authenticity and integrity, comprehensive audit trails of all agent actions and data interactions, cryptographic enforcement of data access and usage policies, and protection of valuable agent models and intellectual property. This combination of autonomous capability and verifiable privacy and security makes it ideal for ServiceNow to leverage for sensitive sales commission data while maintaining the highest standards of privacy and trust. The implementation of Opaque's Confidential AI Agents delivered strong results across ServiceNow's sales operations. Average response times decreased from 4 days to just 8 seconds, dramatically improving service delivery. Sellers can find quick summaries of Sales Success Center material and links to learn more. They also reported a 74% accuracy rate of agent responses, demonstrating high relevancy. Beyond operational improvements, ServiceNow improved operating costs while simultaneously strengthening their security posture through confidential computing. Most importantly, this has freed up the help desk team to focus on more strategic, high-value work while delivering faster, more accurate support to the sales force, creating a virtuous cycle of improved efficiency and satisfaction. Learn more Azure confidential VMs with NVIDIA H100 Tensor Core GPUs Azure confidential GPU Options Opaque’s Confidential AI Platform NVIDIA H100 Tensor Core GPUsPreview of Azure Confidential Clean Rooms for secure multiparty data collaboration
Today, we are excited to announce the preview of Azure Confidential Clean Rooms, a cutting-edge solution designed for organizations that require secure multi-party data collaboration. With Confidential Clean Rooms, you can share privacy sensitive data such as personally identifiable information (PII), protected health information (PHI) and cryptographic secrets confidently, thanks to robust trust guarantees that help ensure that your data remains protected throughout its lifecycle from other collaborators and from Azure operators. This secure data sharing is powered by confidential computing, which helps protect data in-use by performing computations in hardware-based, attested Trusted Execution Environments (TEEs). These TEEs help prevent unauthorized access or modification of application code and data during use. Organizations across industries need to perform multi-party data collaboration with business partners, outside organizations, and even within company silos to improve business outcomes and bolster innovation. Confidential Clean Rooms help derive true value from such collaborations by enabling granular and private data to be shared while providing safeguards on data exfiltration hence protecting the intellectual property of the organization and the privacy of its customers and addressing concerns around regulatory compliance. Whether you’re a data scientist looking to securely fine-tune your ML model with sensitive data from other organizations, or a data analyst wanting to perform secure analytics on joint data with your partner organizations, Confidential Clean Rooms will help you achieve the desired results. You can sign up for the preview here Key Features Secure Collaboration and Governance: Allows collaborators to create tamper-resistant contracts that contain the constraints which will be enforced by the clean room. Governance verifies validity of those constraints before allowing data to be released into clean rooms and helps generate tamper-resistant audit trails. This is made possible with the help of an implementation of the Confidential Consortium Framework CCF). Enhanced Data Privacy: Provides a sandboxed execution environment which allows only authorized workloads to execute and prevents any unauthorized network or IO operations from within the clean room. This helps keep your data secure throughout the workload execution. This is possible with the help of deploying clean rooms in confidential containers on Azure Container Instances (ACI) which provides container group level integrity with runtime enforcement of the same. Verifiable trust at each step with the help of cryptographic remote attestation forms the cornerstone of Confidential Clean Rooms. Salient Use Cases Azure Confidential Clean Rooms caters to use cases spanning multiple industries. Healthcare: For fine-tuning and inferencing with predictive healthcare machine-learning (ML) models and for joint data analysis for advancing pharmaceutical research. This can help protect the privacy of patients and intellectual property of organizations while demonstrating regulatory compliance. Finance: For financial fraud detection through analysis of combined data across banks and other financial institutions and for providing personalized offers to customers through secure analysis of transaction data and purchase data in retail outlets Media and Advertising: For improving marketing campaign effectiveness by combining data across advertisers, ad-techs, publishers and measurement firms for audience targeting and attribution and measurement Retail: For enhanced personalized marketing and improved inventory and supply chain management Government and Public Sector Organizations: For analysis of high security data across multiple government and public sector organizations to streamline benefits for citizens Customer Testimonials We are already partnering with several organizations to accelerate their secure multi-party collaboration journey with confidential clean rooms. Confidential computing in healthcare allows secure data processing within isolated environments, called 'clean rooms', protecting sensitive patient data during AI model development, validation and deployment. Apollo Hospitals uses Azure Confidential Clean Rooms to enhance data privacy, encrypt data, and securely train AI models. The benefits include secure collaboration, anonymized patient privacy, intellectual property protection, and enhanced cybersecurity. Apollo’s pilot with Confidential Clean Rooms showed promising results, and future efforts aim to scale secure AI solutions, ensuring patient safety, privacy, and compliance as the healthcare industry advances technologically. - Dr. Sujoy Kar, Chief Medical Information Officer and Vice President, Apollo Hospitals Azure Confidential Clean Rooms is a game changer to make collaborations on sensitive data both seamless and secure. When combined with Sarus, any data processing job is automatically analyzed using the most advanced privacy technology. Once validated, they are processed securely in Confidential Clean Rooms protecting both the privacy of data and the confidentiality of the analysis itself. This eliminates administrative overheads and makes it very easy to build advanced data processing pipelines. With our partner EY, we're already leveraging it to help international banks improve AML practices without compromising privacy. - Maxime Agostini, CEO & Cofounder of Sarus Read here to learn more about how Sarus is using Confidential Clean Rooms. As co-leaders on this Data Consortium Pilot, we are thrilled to be working with industry partners, Sarus and Microsoft, to drive this initiative forward. By combining Sarus’ privacy preserving technologies and Microsoft’s Azure Confidential Clean Rooms, not only does this project push the edge of technology innovation, but it strives to address a pivotal issue that affects us as Canadians. Through this work, we aim to help financial services organizations and regulators navigate the complexities of private and personal data sharing, without compromising the integrity of the data, and adhering to all relevant privacy regulations. For the purposes of this pilot, we are focusing our efforts on how this technology can play a pivotal role in helping better detect cases of human trafficking, however, we recognize that it can be used to help organizations for multiple other use cases, and cross industries, including health care and government & public sector. - Jessica Hansen, Privacy Partner EY Canada, and Dana Ohab, AI & Data Partner EY Canada Retrieval-Augmented Generation (RAG) applications accessing Large Language Models (LLMs) are common in private AI workflows, but managing secure access to sensitive data can be complex. SafeLiShare’s integration of its LLM Secure Data Proxy (SDP) with Azure Confidential Clean Rooms (ACCR) simplifies access control and token management. The joint solution helps ensure runtime security through advanced Public Key Infrastructure (PKI) and centralized policy management in Trusted Execution Environments (TEEs), enforcing strict access policies and admission controls to guarantee authorized access to sensitive data. This integration establishes trust bindings between the Identity Provider (IDP), applications, and data, safeguarding each layer without compromise. It also enables secure creation, sharing, and management of applications and data assets, ensuring compliance in high-performance AI environments. - Cynthia Hsieh, VP of Marketing, SafeLiShare Read here to learn more about how SafeLiShare is using Confidential Clean Rooms. Learn More Signup for the preview of Azure Confidential Clean Rooms Confidential Consortium Framework (CCF) Confidential containers on Azure Container Instances (ACI)
Confidential Data Clean Rooms – The evolution of sensitive data collaboration
Secure data collaboration between multiple parties has the potential to revolutionize societies, businesses and industries for the better. Collaborating on sensitive data assets facilitates innovation to unlock new value for organizations.
BigDL Privacy Preserving Machine Learning with Occlum OSS on Azure Confidential Computing
This blog introduces you to a confidential computing solution for Privacy-Preserving Machine Learning (PPML) made available by Open-Source Software Occlum Library OS for Intel SGX and BigDL on the Azure cloud. This blog demonstrates the solution using a sample analytics application built for the NYTaxi dataset. This sample application leverages Azure Confidential Computing (ACC) components such as SGX Nodes for Azure Kubernetes Service (AKS), Microsoft Azure Attestation, Azure Key Vault (AKV), etc, as well as Occlum LibOS and BigDL PPML.Frictionless Collaborative Analytics and AI/ML on Confidential Data
Secure enclaves protect data from attack and unauthorized access, but confidential computing presents significant challenges and obstacles to performing analytics and machine learning at scale across teams and organizational boundaries. In this article, we'll explore the Opaque platform and describe how it can enable multiple parties to easily collaborate and analyze shared data while keeping it fully confidential.
