Pinned Posts
Forum Widgets
Latest Discussions
Azure Databricks Unity Catalog connector - Metadata
For the Azure Databricks Unity Catalog connector, I believe Microsoft Purview scans and collects the following technical metadata: Table name View name Column names Catalog name Schema name Metastore information Table and column descriptions from Databricks comments Unity Catalog tags, which are stored within Purview properties Table, view and column lineage where supported Storage location information may also appear within the technical properties of the asset, depending on the Databricks object. However, Microsoft does not appear to clearly document storage location as guaranteed metadata from every scan. Can anybody explain this further or confirm whether they have seen different behaviour? Where the metadata is stored or managed in Azure Databricks Metadata collected by Microsoft Purview Where it is stored or managed in Azure Databricks Metastore Unity Catalog Metastore Catalog name Unity Catalog Schema name Unity Catalog Schema Table name Unity Catalog Table metadata View name Unity Catalog View metadata Column names Table schema in Unity Catalog Table description Unity Catalog Table Comment Column description Unity Catalog Column Comment Unity Catalog tags Tags applied to Unity Catalog objects such as tables, views and columns Table and column lineage Unity Catalog lineage metadata and system lineage tables Storage location Table metadata where applicable, particularly for external tables I believe table and column descriptions are stored in Databricks as comments against the Unity Catalog object. These comments can then be scanned into Microsoft Purview and displayed as descriptions. The Microsoft documentation appears clearer for column comments than it does for table comments, so I am planning to test this myself. Does anybody have practical experience of this behaviour? Unity Catalog tags Tags are stored as metadata against Unity Catalog securable objects. Tag information can also be queried through Unity Catalog INFORMATION_SCHEMA, including TABLE_TAGS and the corresponding column tag metadata. The core Unity Catalog structure is: Unity Catalog Metastore Catalog Schema Table or View Columns Additional metadata can include: Comments Tags Lineage Storage location where applicable What Microsoft currently documents as collected by the Microsoft Purview Azure Databricks Unity Catalog connector Databricks Unity Catalog metadata Collected into Purview? Notes Metastore Yes Represented in the Purview hierarchy Catalog name Yes Unity Catalog catalog Schema name Yes Unity Catalog schema Table name Yes Creates/discovers the table data asset View name Yes Views are supported Column names Yes Schema/column metadata is collected Table comment / description Yes Your observation is correct: Databricks table comments/descriptions can appear as the Purview Data Asset description Column comments / descriptions Yes Microsoft explicitly states that Databricks column comments are displayed as column descriptions in Purview Unity Catalog tags Yes Microsoft explicitly states that Unity Catalog tags are scanned into Purview Properties Table/view lineage Yes, subject to prerequisites Extracted from Unity Catalog system lineage tables Column lineage Yes, subject to prerequisites Supported, although there are lineage limitations External tables Metadata yes External table metadata is supported; lineage for external tables is not supported Microsoft's connector documentation currently lists the following core hierarchy and metadata as being extracted by the scan: Metastore Catalogs Schemas Tables and Columns Views and Columns Unity Catalog Tags Data Asset descriptions This behaviour appears to be genuine. Microsoft specifically documents that a Databricks column comment can be used as the description displayed in Microsoft Purview. Microsoft also notes that setting a Databricks table column comment to an empty string prevents the column description from being displayed in Purview. For practical purposes, the relationship is: Databricks comment or description Purview scan Microsoft Purview description This is explicitly documented for column comments. Table comments also appear to be surfaced as the Data Asset description through the connector, based on observed behaviour, although Microsoft's documentation is less explicit about this. Has anybody seen this working consistently for table descriptions? Unity Catalog tags Unity Catalog tags are supported by the current connector. Microsoft documentation states that tags from Unity Catalog are extracted during scanning and displayed within Microsoft Purview Properties. There is a documented exception where tags are not supported when the scan uses the Kubernetes Self-hosted Integration Runtime option. That exception should not apply where Kubernetes SHIR is not being used. Storage location The official connector documentation clearly documents the Unity Catalog hierarchy and the metadata listed above. However, it does not appear to explicitly list the table storage location or storage path as a guaranteed metadata element within the supported metadata capabilities. Storage location information may therefore appear for some Databricks objects, particularly external tables, but I would not currently treat it as guaranteed metadata from every Purview Unity Catalog scan without further testing. My current understanding is that a Microsoft Purview scan of Azure Databricks Unity Catalog can collect: Metastore Catalog Schema Table View Columns Table descriptions or comments Column descriptions or comments Unity Catalog tags Table and view lineage where supported Column lineage where supported External table metadata Storage location may also be visible where applicable, but this appears less clearly documented as a guaranteed field? Does this sound correct based on other people's experience of Microsoft Purview scans of Azure Databricks Unity Catalog data sources?SolvedsashakorniakUKJul 21, 2026Brass Contributor47Views0likes1Comment
How can we identify the size of a Microsoft Purview DLP policy and individual DLP rules?
Hello Microsoft Community, According to the Microsoft Purview DLP policy reference, the following platform limits apply: Maximum size of a DLP policy: 100 KB Maximum size of an individual DLP rule: 100 KB or 102,400 characters Maximum number of rules within a policy: Limited by the overall policy size I would like to understand how these limits are calculated and how administrators can monitor the current size of an existing DLP policy or rule. Could someone please clarify the following? What exactly is included when calculating the 100 KB DLP policy size? Does the policy size include only policy-level configuration, such as locations, users, groups, administrative units, inclusions and exclusions, or does it also include the combined definitions of all rules associated with the policy? What exactly is included in the 100 KB individual rule size? Sensitive information types and their GUIDs Condition groups Instance-count and confidence-level configurations Exceptions Endpoint DLP restrictions User notifications and policy-tip messages Override options Alert and incident-report settings Recipient lists Advanced rule configuration For example, does it include: The documentation describes the rule limit as “100 KB (102,400 characters).” Is the limit based on: The number of characters The UTF-8 or Unicode byte size The serialized JSON/XML representation An internally generated policy payload Is there a supported method in the Microsoft Purview portal, Security & Compliance PowerShell, Microsoft Graph, or another API to display the current size of: A DLP policy Each DLP rule The remaining available policy capacity Can the size be estimated by exporting the output from Get-DlpCompliancePolicy and Get-DlpComplianceRule? If so, which properties should be included in the calculation, and what encoding or serialization format should be used? Does the 100 KB policy limit represent the combined size of the policy and all its rules, or are the policy and rule limits evaluated independently? What error or warning is generated when a policy or rule approaches or exceeds the limit? Is there any notification before the limit is reached? We are designing global Microsoft Purview DLP policies containing multiple Sensitive Information Types, condition groups and workload-specific rules. We need a reliable method to measure policy and rule sizes during design and ongoing policy governance. Any official guidance, supported script, API property, or calculation method would be appreciated.Solved75Views0likes2CommentsPurview HR connector
Hi, I am trying to upload a csv file with resigning users to Purview through the HR connector, following the steps from this documentation (https://learn.microsoft.com/en-us/purview/import-hr-data). Have set up a power automate flow to create a bearer token and upload the csv file using a POST API to "https://webhook.ingestion.office.com/api/signals" as shown in the Github sample solution and script. However, I run into the following error: "No HTTP resource was found that matches the request URI 'https://40.75.149.147/api/signals'. The IP seems to change each time I make the API call. Is the URI correct or should I be using another specific URI? Any help would be greatly appreaciated. Thanks in advance!SolvedrkkfJul 03, 2026Copper Contributor91Views0likes1Comment
Unlabelled Files
I have a requirement to produce a report which contains the number of files in M365 SharePoint & OneDrive which do not have a sensitivity label applied. I am struggling to find a sensible approach to this and I am fairly certain this is not possible in Purview unless I have missed something. If anyone can help it would be appreciated. ThanksSolved145Views0likes3Comments
Data System Wide Lineage via API Request
I'm struggling with finding a solution. My goal is to identify all existing lineage relationships for any data objects within a specific data system they belong to. I've been using the Purview REST API (Datamap Dataplane) but I haven't found an endpoint returning data system side lineage/relationships. For my scenario I have a Databricks metastore and need to know the existing lineage relationships of those data objects within Purview so I can purge them out when we are doing our scheduled lineage refresh.SolvedsouthpawmurphMay 15, 2026Copper Contributor150Views1like3Comments
Auto Labeling Policy Delay for Old Files (Exsisting Files)
Hi Everyone, We are observing a difference in auto labelling policy behaviour in Purview for Sharepoint. An auto labelling policy has been enabled and scoped to sharepoint with metadata based rule(document creation date or document modification date). The scoped sharepoint only contain 7 unlabeled files that were uploaded before the policy turned on. The policy is working because if i placed any new file after enabling the policy got labelled within about 5 minutes, but the exsisting files are not labeled and remains unlabelled. It seems the new files are evalauated via the near time while exsisting file rely on asychronous mode. Can anyone help explain why exsisting files take longer to be proceesed even when there there are only a few files or share if you faced similar behaviour. This is the test scenario, as we plan to enable the same policy across more than 50 plus sites containing millions of unlabeled files and we want to understand and predict that even though its takes time all exsisting unlabeled files will eventually will be labelled. This is very crucial, so please helo us understand this behaviour. Regards, BanuMuraliSolved331Views0likes3Comments
Unified Catalog Self-serve analytics integration
I'm hoping someone has gone through the process of setting up the Self-serve analytics in the Unified Catalog settings to push the Unified Catalog information down to a Fabric Lakehouse. I created a Workspace, and then created a lakehouse in this workspace, and created a folder under the files section in the lakehouse. I used the MSI that is shown in Purview when you configure the storage for the connection and granted it contriubutor access to the Workspace. I then went into Purview, settings for Unified Catalog, and in the solution integrations, set up Fabric storage and provided the URL to the File folder I set up on the lakehouse. I tested the connection and it tested successfully. When I set up the scheduler to run, I received the following: The blacked out is the Workspace ID. I'm trying to understand what I'm missing, I'm assuming write permissions are missing somewhere, but I'm not sure. Any assistance is appreciated.SolvedJBNFMApr 21, 2026Tin Contributor127Views0likes4Comments
How to identify users handling SITs before purchasing Microsoft Purview licenses?
Posting this on behalf of a customer we are currently advising as a Microsoft Partner. The customer is in the evaluation stage of Microsoft Purview and has raised a licensing concern that we would like the community's guidance on. CUSTOMER'S CONCERN Purview licenses are user-based, meaning every user who directly or indirectly benefits from the service needs to be licensed. However, to determine which users actually handle sensitive data (and therefore require a license), tools like Content Explorer and Activity Explorer are needed — both of which require an E5 or equivalent license to access in the first place. This creates a chicken-and-egg problem for the customer: They need Purview to identify who handles sensitive data, but they need to know who handles sensitive data to decide how many Purview licenses to buy. QUESTIONS ON BEHALF OF THE CUSTOMER 1. Is there an official Microsoft-supported mechanism or tool that allows customers to assess their SIT exposure and identify affected users before committing to a full Purview license purchase? 2. Is it viable for the customer to purchase a single license (1 qty) assigned to an admin account to perform a tenant-wide scoping and discovery exercise — and would that single license provide sufficient access to identify all users handling SITs across the tenant? 3. If the 90-day Purview E5 trial is the recommended path, does Content Explorer automatically scan and surface SIT matches across all users in the tenant without requiring any pre-configured DLP policies or sensitivity labels to be set up first? As a partner, we want to ensure we are guiding our customer toward the correct pre-purchase assessment approach before recommending a licensing SKU and quantity. Any guidance from the community or Microsoft would be greatly appreciated.SolvedelangambanApr 19, 2026Tin Contributor94Views0likes2CommentsPurview Graph API
Hello. I'm trying to find information on the Purview Graph API and it's endpoints. It looks like the endpoints aren't posted publicly and are listed within an admin console. Can someone help me with how to view the endpoints? Also, are the graph API endpoints capable of reading and creating assets into Purview?SolvedsouthpawmurphApr 08, 2026Copper Contributor143Views0likes1Comment
Tags
- purview157 Topics
- microsoft purview106 Topics
- Information Protection35 Topics
- Sensitivity Labels31 Topics
- data loss prevention20 Topics
- ediscovery18 Topics
- api18 Topics
- Azure Purview17 Topics
- endpoint dlp15 Topics
- Retention Policy14 Topics