Pinned Posts
Forum Widgets
Latest Discussions
Struggling with running DQ Scans (Long queuing and Retry Count Error Issues)
Hi everyone, I have been exploring Microsoft Purview Data Quality quite extensively. At this point, I have configured more than 4,000 data quality rules across more than 10 Microsoft Fabric capacities, each with a minimum capacity of F16. Fabric is the source for all assets registered in Purview. I have identified several issues with the product, but the two that are currently impacting me the most are the following: DQ scans failing with a generic error“Max Retry Count Reached. Ending Workflow. Current Task HandleError”The challenge is that the error message does not identify which rule is causing the failure. As a result, I have to troubleshoot manually by disabling groups of rules, rerunning the scans, and repeating the process until I find the problematic rule. This trial-and-error approach is very time-consuming, especially at this scale. This seems to be caused by issues in some of the DQ rules, even though all rules are marked as “Good to go” in Purview. When running Data Quality scans, I often receive the following error: DQ scans remain queued for a long timeI am not sure why this happens or what resource, orchestration, or scheduling constraint is causing the delay. Whenever I run these DQ scans, they remain in a Queued state for at least 10 minutes, even when there is nothing running on the Fabric capacities. Has anyone experienced similar behavior with Purview Data Quality at this scale? Specifically, I would appreciate any guidance on: How to identify which DQ rule is causing a scan failure Why scans remain queued even when Fabric capacity appears to be idle Whether there are known limitations or best practices for running thousands of DQ rules in Purview Thank you.
Glossary Terms Governance Model
A practical implementation pattern that scales from a Proof of Concept to enterprise deployment. If you're involved in implementing Microsoft Purview or designing an enterprise data governance model, I'd be interested to hear how you're approaching glossary management within your organisation. Do you manage a single enterprise glossary, separate domain glossaries, or a combination of both? I'd love to hear your experiences and any lessons you've learned. https://www.linkedin.com/search/results/all/?keywords=%23microsoftpurview&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23microsoft&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23datagovernance&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23unifiedcatalog&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23enterprisedatacatalogue&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23metadata&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23metadatamanagement&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23dataproducts&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23datamesh&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23federatedgovernance&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23enterprisearchitecture&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23dataarchitecture&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23informationarchitecture&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23datamanagement&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23azure&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23microsoft365&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23purview&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23glossary&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23businessglossary&origin=HASH_TAG_FROM_FEED https://www.linkedin.com/search/results/all/?keywords=%23dataskylabstudio&origin=HASH_TAG_FROM_FEED 😀 Understanding the Enterprise and Local Glossary Terms Governance Model This diagram demonstrates a scalable approach to managing Glossary Terms in Microsoft Purview Unified Catalog. The objective is simple: Create enterprise business terminology once, reuse it across the organisation, and keep specialist business terminology within the Governance Domain that owns it. This approach creates a Single Source of Truth for shared business language, reduces duplication, improves consistency and allows the governance model to scale as the organisation grows. Step 1 – Create an Enterprise Glossary Terms Governance Domain The first step is to create a dedicated Governance Domain specifically for Enterprise Glossary Terms. For example: Enterprise Glossary Terms This Governance Domain becomes the central location where enterprise-wide business terminology is created, maintained and governed. Examples include: Party Person Organisation Customer Employee Product Account Contract Invoice Order Transaction These are business concepts that are commonly used across multiple business functions and Governance Domains. Each Enterprise Glossary Term has: One agreed business definition. One accountable owner. One authoritative source. One location where it is maintained. Unlimited reuse across Data Products. Think of this Governance Domain as the organisation's enterprise business dictionary. Step 2 – Create Business Governance Domains Next, create Governance Domains that reflect your organisational structure. For example: Finance Sales Human Resources Meat, Dairy & Approved Establishments Each Governance Domain is responsible for: Its own Data Products. Its own local Glossary Terms. Its own stewardship and governance. Step 3 – Create Local Glossary Terms Not every business term belongs in the Enterprise Glossary. Many terms are only relevant to a single business function and should remain owned within that local Governance Domain. For example: Finance General Ledger Cost Centre Budget Code Payment Run Sales Sales Channel Campaign Opportunity Discount Code Human Resources Job Grade Leave Type Shift Pattern Pay Band Meat, Dairy & Approved Establishments Establishment Type Hygiene Rating Carcass Classification Milk Cooler Capacity These Local Glossary Terms remain within their own local Governance Domain because they are not intended for enterprise-wide reuse. They can be seen in the Enterprise Glossary. Step 4 – Create Data Products Each Governance Domain creates the Data Products it owns. Examples include: Finance Financial Reporting Sales Sales Orders Human Resources Employee Directory Meat, Dairy & Approved Establishments Establishment Inspections The Data Product becomes the business representation of a collection of related data. Step 5 – Reuse Enterprise Glossary Terms When creating a Data Product, associate the Enterprise Glossary Terms that describe the business concepts used by that Data Product. For example: Financial Reporting Uses: Customer Account Transaction Invoice Sales Orders Uses: Customer Product Contract Order Employee Directory Uses: Person Employee Organisation Establishment Inspections Uses: Organisation Party Product Notice that these Enterprise Glossary Terms are not recreated inside each Governance Domain. Instead, they are referenced from the Enterprise Glossary Terms Governance Domain. This is the key principle behind the model: Create once. Reuse many times. Step 6 – Add Local Glossary Terms Each Data Product can also reference Local Glossary Terms from its own Governance Domain. For example: Financial Reporting Also uses: General Ledger Cost Centre Sales Orders Also uses: Sales Channel Employee Directory Also uses: Job Grade Establishment Inspections Also uses: Inspection Risk Category Each Data Product therefore combines: Enterprise Glossary Terms Local Glossary Terms This provides consistent enterprise language while still allowing each business area to manage its own specialist terminology. Why this model works This implementation separates enterprise business language from specialist business language. Enterprise Glossary Terms provide: A Single Source of Truth. One agreed definition. One accountable owner. Reuse across every Governance Domain. Local Glossary Terms provide: Business-specific terminology. Local ownership. Specialist business knowledge. Flexibility without affecting enterprise standards. Together they provide: Reduced duplication. Consistent business language. Easier governance. Better Discovery Search. Simpler maintenance. Reusable metadata. A scalable governance model. Relationship with the Conceptual Data Model The Enterprise Glossary answers the question: What business concepts does the organisation use? The Conceptual Data Model answers: How are those business concepts related? For example: Customer │ places│ ▼ Order │ contains ▼ Product The Enterprise Glossary defines each business concept. The Conceptual Data Model describes how those concepts relate to one another. Together they establish a common business language that can be reused consistently across Governance Domains, Data Products and the wider Enterprise Data Catalogue. Microsoft Purview Implementation Steps Create a Governance Domain called Enterprise Glossary Terms. Create enterprise-wide Glossary Terms within this Governance Domain. Assign an accountable owner for each Enterprise Glossary Term. Create business Governance Domains that reflect the organisation. Create Local Glossary Terms only where the terminology is unique to that Governance Domain. Create Data Products within each Governance Domain. Associate Enterprise Glossary Terms with Data Products wherever the same business meaning applies. Associate Local Glossary Terms with Data Products where specialist business terminology is required. Continue expanding the Enterprise Glossary as new shared business concepts emerge. Key Principle Create once. Reuse everywhere. Keep local terms local. Enterprise Glossary Terms establish the organisation's shared business language, while Local Glossary Terms allow individual Governance Domains to manage specialist terminology. Together they provide a scalable governance model for Microsoft Purview Unified Catalog that supports Federated Data Governance, Data Mesh principles, and long-term enterprise growth without unnecessary duplication of business definitions.Microsoft Purview Unified Catalog; Governance Domains and Business Concepts
I've been using the attached artefacts for some time to help explain the knowledge exchange aspects of Microsoft Purview Unified Catalog, particularly how Governance Domains and Business Concepts work together to provide business context, ownership, stewardship and operational insights. They have been useful in workshops with data architects, governance professionals, product owners and business stakeholders to demonstrate how concepts fit together within a governance domain and contribute towards trusted information and better business outcomes. I'm interested in hearing from the wider Purview community: Do these artefacts accurately represent the intent and capabilities of Governance Domains within Microsoft Purview? Are there any concepts that you feel are missing, over-emphasised, or could be represented more clearly? How are others explaining Governance Domains and Business Concepts to non-technical stakeholders? Any feedback, suggestions, or alternative approaches would be greatly appreciated. I'm always looking to refine these materials and make them more useful for organisations adopting Purview Unified Catalog. #MicrosoftPurview #DataGovernance #DataManagement #Metadata #DataProducts #MicrosoftData #Purview #DataArchitecture #UnifiedCatalogMicrosoft Purview Unified Catalog; Governance Domains and Business Concepts
I've been using the attached artefacts for some time to help explain the knowledge exchange aspects of Microsoft Purview Unified Catalog, particularly how Governance Domains and Business Concepts work together to provide business context, ownership, stewardship and operational insights. They have been useful in workshops with data architects, governance professionals, product owners and business stakeholders to demonstrate how concepts fit together within a governance domain and contribute towards trusted information and better business outcomes. I'm interested in hearing from the wider Purview community: Do these artefacts accurately represent the intent and capabilities of Governance Domains within Microsoft Purview? Are there any concepts that you feel are missing, over-emphasised, or could be represented more clearly? How are others explaining Governance Domains and Business Concepts to non-technical stakeholders? Any feedback, suggestions, or alternative approaches would be greatly appreciated. I'm always looking to refine these materials and make them more useful for organisations adopting Purview Unified Catalog. #MicrosoftPurview #DataGovernance #DataManagement #Metadata #DataProducts #MicrosoftData #Purview #DataArchitecture #UnifiedCatalog
Onboarding Devices to Purview
I am not clear on how can I onboard devices to MDE so that I can enforce EDLP policies. We have CrowdStrike as Primary AV and other policies. Devices are managed through Intune for Bitlocker encryption and all the other settings except they don't have Defender. These devices are not showing up in Purview nor under "Endpoint detection and response" location under Endpoint Security. If we create an EDR onboarding policy and deploy to devices, then it shows the devices and says that AMRUnningMode is Passive, but Antivirus is true. Which I feel like Defender is taking over CrowdStrike? or am I wrong. My goal is to make sure CrowdStrike still primary AV and devices should be onboarded to MDE and then to Purview so that we can scope EDLP policies properly. Can anyone help me to understand or provide right steps?
Confusion around Purview Definitions and Risk Scoring
In the early days of implementation and we've done our 'Quick setup' of Insider Risk Management which created our Adaptive Protection Policy for IRM, two IRM DLP policies (Endpoint & Teams/Exchange) and the Conditional Access policy. My question is around 'Triggering events', Indicators and Insider Risk Levels. To my understanding, a triggering event is the event that decides when the policy will start assigning risk scores to user activity which will then allow us to then give users risk levels. We have the option to either set this triggering event to either the DLP policies, or when a user performs an exfiltration activity/ sequence. The DLP policies only match activity when a user has a defined risk level and attempts to perform a specific activity i.e. sharing M365 with people outside the organisation. I'm not sure if I'm thinking about this backwards, but if I set my Adaptive protection policy to only start assigning risk scores to user activity when they match a DLP policy, how can they trigger a DLP policy if they wont be assigned a risk level until that scoring begins to happen? Should I be setting my triggering events to be "User performs an Exfiltration Activity" instead of "User Matches a DLP policy"?# Seeking Feedback – Microsoft Purview Governance Domain Metamodel
I've been working on a proposed metamodel for some time to help organisations decide how to structure Governance Domains within Microsoft Purview and would appreciate feedback from others who have implemented Purview at scale. The intention is not to prescribe a single approach, but to describe several governance patterns that seem to emerge in practice. Some additional assumptions I've made: * Numeric prefixes such as `01.01.01` help maintain sort order and readability. * Standardising on three levels appears easier to manage, although Purview supports five levels. * Microsoft guidance suggests keeping Governance Domains to approximately 200. * Governance Domains themselves are relatively flexible and can be renamed or repositioned within the hierarchy. * Data Products currently appear to be bound to the Governance Domain in which they are created and cannot presently be reassigned to another Governance Domain, making early design decisions more important. I'm interested in hearing from organisations already using Governance Domains in production. A few questions for discussion: Have you adopted one of these patterns, or a hybrid approach? Are there Governance Domain types missing from this metamodel? Is the recommendation of standardising on three hierarchy levels sensible, or have you found deeper structures manageable? Are there any Microsoft best practices, roadmap items or implementation experiences that would suggest a different approach? I've attached an infographic illustrating the proposed metamodel and would welcome any thoughts, criticism or lessons learned from real-world implementations.
Managed VNET Integration Runtime failing with 502 error.
Good afternoon everyone. I'm a DevOps Engineer who is new to Purview. I used Terraform to deploy a Purview account for a POC for a client, however, I'm having a real issue creating a Managed VNET IR. The private endpoints are all visible and approved and if I check in the shell I can see the IR and the Managed VNET both exist (names sanitized). { "name": "SAMPLENAME", "properties": { "managedVirtualNetwork": { "referenceName": "ManagedVnet-name" }, "typeProperties": { "computeProperties": { "location": "WestEurope" } } } } But in the Purview portal the status shows as failed and if I try update it, I get a popup notification stating that the process timed out due to a 502 error. The URL in the error is " https://api.purview-service.microsoft.com/scan/integrationRuntimes/{NAME}?api-version=2022-02-01-preview" I thought this might be an issue with permissions or that I'm not in the admin role group in my client environment so I did the same process in my local purview account (where I'm global admin and in the Purview Administrators role group) and I'm having exactly the same problem. The managed vnet and IR exist when queried in the cloud shell but the state in the portal shows as failed. I am a "Data source Admin" in both purview accounts but I'm wondering if there's some other role assignment or role group assignment that I'm missing? Thanks in advance. Devon Britton.
Terribly lost - what are the basic controlls here?
Hello all. I'm an MSP, looking at methods of securing data in the wake of AI adoption. Obviously, I'm getting pointed to Purview for this. And I've managed to make sense of SOME of it - sensitivity labels, labeling policies, and sensitive info types. The problem I have is that these 'solutions' are spread out amongst 3-4 different 'solutions' - Information Protection, DLP, DSPM (DSPM,, DSPM classic, DSPM for AI 'classic') and it's genuinely just really badly designed. It's done the classic Microsoft move of having the Marketing team build the interface, and caring more about market capture/buzzwords than usability. As is the norm, the documentation quality varies a ton. And between Intune, SharePoint, Entra, Defender, Azure, certifications - I don't actually have time to learn another market-capture tool, which I will use 2% of. We don't license Purview. And I'm not going to license Purview until some effort is put into usability, and the interface is redesigned by native, technical english speakers (no hate, but I've seen first-hand how MBAEnglish-as-a-second-language translates into this sort of opacity). But obviously, we HAVE to use it because a bunch of stuff was pushed into it. Without adding another set of half-automated Microsoft recommendations to my list, and avoiding premium 'solutions' - what are the basic 'solutions' that are required for Data controls, in the face of AI? What exactly was merged into Purview, that existed elsewhere previously? Here is what I've gotten familiar with so far: 1. DLP policies. These are pretty opaque to me, and seem to heavily rely on OTHER 365 products, like Defender for Endpoint, Edge for Business. So again, designed by the marketing team. 2. Sensitivity labels, labeling publishing policies, auto-labeling policies. What am I missing?Best approach for contractor block policy
Hello there I need some assistance with your best approach for vendor block policy. I am thinking to create one policy with three rules Block all vendors with the block AD group Vendors to allow emails to approved domains only vendors to send email to external to organisation with ability to send to approve domains Do you think this is a good approach by breaking down into three different rules ? Also I am bit confused with the conditions on the rule 2 and rule 3. what would you your approach with complete breakdown ?
