Pinned Posts
Forum Widgets
Latest Discussions
Microsoft Fabric metadata in Microsoft Purview
I’ve been mapping how Microsoft Fabric metadata is surfaced in Microsoft Purview through Data Map scanning, and I’ve created this visual to make the relationship easier to understand. The diagram separates: Documented mappings – such as Fabric items, Lakehouse tables, schema and item-level lineage. Metadata known to be scanned, but where the exact Purview UI location needs confirmation. ? Areas still needing validation – particularly Lakehouse table/column descriptions and tags. The principle I’m exploring is: Microsoft Fabric → Purview Data Map Scan → Purview Data Asset → Purview governance enrichment Importantly, a Fabric asset does not automatically become a Purview Data Product. It is first represented as a Data Asset, which can then be governed, enriched and associated with a Data Product. I’d really appreciate feedback from anyone working hands-on with Microsoft Fabric and Microsoft Purview: Does this mapping match what you are seeing in your environment? I’m particularly interested in confirming: Lakehouse table descriptions → Purview Asset Description? Lakehouse column descriptions → Purview Schema → Column Description? Lakehouse table/column tags → where exactly are these surfaced in Purview? Corrections, screenshots or practical experience would be very welcome.Purview SDK
I've been spending quite a bit of time working with Purview APIs, The APIs themselves are fine, but after a while I realized I was writing the same authentication, pagination and relationship handling code over and over again. So instead of construction the same code from project to project, I turned it into a python package, and now it's available on PyPI pip install purview-unified-sdk Right now, the SDK supports most of the common operations, such as creating, retrieving, updating and deleting business domains, data products, glossary terms, objectives, key results and etc., It also make it much easier to work with relationships, add group id as a owner, navigate resources and retrieve metadata across the unified catalog. https://niki9001.github.io/purview-unified-sdk/ https://github.com/purview-unified-sdk Feel free to fork the project, submit a pull request or open an issue if you have ideas or suggestionsUnified Catalog - Why I Think About Governance Domains Vertically and “Domains of Data” Horizontally
One of the more useful ways I have found to think about Microsoft Purview is to separate ownership from meaning. For me, that creates two different but connected structures: Vertical = Ownership Governance Domains tell us WHO owns and governs the data. They provide the organisational structure for accountability. A Governance Domain can contain: Data Products → Data Assets → Critical Data Elements → Columns and Attributes It also gives us the governance context around those objects: Ownership Stewardship Accountability Governance responsibilities Data quality Access Controls So when I look at a Governance Domain, I am really asking: Who is responsible for this data? That is the vertical view. Horizontal = Meaning Enterprise Glossary Terms give us a different perspective. Rather than focusing on who owns the data, they can represent what the data means across the organisation. This is what I think of as a “Domain of Data”. I use the phrase “Domain of Data” as a conceptual way of describing an enterprise business concept that can span multiple Governance Domains. Take Personally Identifiable Information (PII) as a simple example. PII is unlikely to fit neatly within a single Governance Domain. It may exist across: Customer information Complaints and incidents HR and work force data Case management Regulatory data Operational systems Contact information Financial and administrative processes Each Governance Domain may own and govern the PII within its own area. But the concept of PII itself spans all of them. That is where an Enterprise Glossary Term becomes particularly useful. It provides a common enterprise definition that cuts horizontally across multiple ownership boundaries. Why the two structures should not be the same It can be tempting to make the Enterprise Glossary hierarchy simply mirror the Governance Domain hierarchy. I think that misses an important opportunity. They answer different questions. Governance Domains WHO owns and governs the data? They represent: Ownership Accountability Organisational responsibility Enterprise Glossary Terms WHAT does the data represent? They represent: Business meaning Shared concepts Enterprise vocabulary The two structures should connect, but they should not simply duplicate each other. One Governance Domain can contain many Domains of Data A Governance Domain may contain many different business concepts. For example, one Governance Domain might contain: PII Customer Data Location Data Financial Data Operational Data The Governance Domain is therefore not necessarily a “type of data”. It is primarily an ownership and governance boundary. One Domain of Data can cross many Governance Domains The reverse is equally important. A single Domain of Data can exist across many Governance Domains. For example, PII might appear in: Governance Domain 01 – Customer and contact data Governance Domain 02 – Complaints and incident data Governance Domain 03 – Employee and work force data Governance Domain 04 – Regulatory and operational data This gives us a many-to-many relationship: One Governance Domain can contain many Domains of Data. One Domain of Data can exist across many Governance Domains. That is the part I think is especially powerful. Where the two structures intersect This is where the model becomes much more valuable. Think about it as: Governance Domain WHO owns it? Enterprise Glossary Term WHAT does it mean? Governed Business Context For example: HR Governance Domain PII Enterprise Glossary Term The result is: The HR-owned instance of an enterprise-wide PII concept. That intersection gives us both ownership and meaning. Why this matters for data consumers Most data consumers do not necessarily know: which Governance Domain owns the data; which platform contains it; which Data Product it belongs to; what the database table is called; or what the individual column names are. They may simply know the business question they are trying to answer. For example: Where do we hold PII? Which Data Products contain Customer information? Where is Location information used? Which Data Assets contain Financial information? This is where the Enterprise Glossary becomes much more than a list of definitions. It becomes a business discovery layer. From business concept to technical data If the relationships are created properly, a user can begin with a business concept and navigate towards the underlying data. For example: Enterprise Glossary Term → Data Products → Data Assets → Critical Data Elements → Columns and Attributes This creates a bridge between: Business meaning ↔ Technical implementation That is a much more useful experience than expecting users to understand the technical structure of the data estate before they can discover anything. Enterprise terms, local terms and CDEs There is also an important distinction between the different types of business metadata. Enterprise Glossary Terms These should represent concepts that have meaning across multiple Governance Domains. Examples might include: PII Customer Organisation Location Financial Information These provide the horizontal enterprise view. Local Glossary Terms These are better suited to terminology that is specific to: a Governance Domain; a business area; a Data Product; or a specialised process. They provide local business context without forcing every term into the enterprise vocabulary. Critical Data Elements CDEs are different again. A Domain of Data may represent a broad business concept such as PII, while CDEs represent individual important data elements such as: Email Address Date of Birth Customer Identifier Postcode That gives us another useful relationship: Enterprise Glossary Term: PII → Critical Data Element: Email Address → Physical Column: customer_email Now the business concept is connected directly to the technical implementation. The principle I keep coming back to The value is not in creating as many glossary terms as possible. It is in applying: The right term → at the right level → connected to the right data That means asking: Is this genuinely an enterprise-wide concept? Should this be local to one Governance Domain? Is this actually a Critical Data Element? What Data Products and Data Assets should it connect to? The quality of those relationships matters far more than the volume of metadata. The bigger picture This is ultimately why I think the horizontal and vertical model is useful. Vertical = Ownership Governance Domains tell us WHO owns and governs the data. Horizontal = Meaning Enterprise Glossary Terms tell us WHAT the data represents across the organisation. And where they intersect: Ownership + Meaning = Governed Business Context That is what allows Microsoft Purview to move beyond simply listing technical assets. Instead, it starts to create a connected view of: Ownership → Business Meaning → Discovery → Governance across the enterprise data estate. For me, that is where the real value of the catalogue starts to appear.Does this Unity Catalog → Purview guidance make sense?
I’ve been working through how Azure Databricks Unity Catalog metadata surfaces in Microsoft Purview after a scan, and I’ve created the attached visual to make the relationship easier to understand. The principle I’m trying to communicate is simple: Maintain Databricks-native technical metadata in Unity Catalog → scan supported metadata into Microsoft Purview → use Purview for the wider enterprise governance, discovery and business context. The short guidance accompanying the visual would be: In Azure Databricks: Navigate to Catalog Explorer → Catalogue → Schema → Table/View. From here, maintain metadata such as table comments, column comments and Unity Catalog tags. Some comments can also be AI-generated as a starting point and reviewed before saving. After the Microsoft Purview scan: Find the corresponding Data Asset in Purview and review the surfaced metadata: Table Comment → Data Asset → Description Column Comment → Data Asset → Schema → Column Description Table Tag → Data Asset → Properties → Tags Column Tag → Data Asset → Schema → Tags Column Name / Data Type → Data Asset → Schema Table/View and Column Lineage → Data Asset → Lineage, subject to the relevant prerequisites. The distinction I’m trying to reinforce is that this is not Purview vs Unity Catalog, and it is not about manually duplicating technical metadata. Unity Catalog remains the Databricks-native governance and technical metadata layer, while Purview can consume supported metadata through scanning and place it into the wider enterprise governance context. I’d be interested in feedback from people implementing Purview + Azure Databricks Unity Catalog: Does the visual and navigation guidance make the relationship clear? In particular, are the field mappings and terminology intuitive enough for Data Engineers, Data Stewards and Data Owners, or is there anything you would simplify, rename or clarify?Microsoft Purview, Databricks Unity Catalog and Medallion Architecture
Microsoft Purview, Databricks Unity Catalog and Medallion Architecture three different responsibilities, one governed data ecosystem. I created this visual because these concepts are often mixed together. 🔵 Microsoft Purview = Enterprise Data Governance & Discovery Purview provides the enterprise-wide governance layer across Databricks and other platforms — Governance Domains, Data Products, glossary, Critical Data Elements, ownership, stewardship, metadata, data quality and discovery. 🔴 Databricks Unity Catalog = Databricks Data & AI Governance + Access Control Unity Catalog is not simply an access-management tool. It governs Databricks data and AI assets, including catalogs, schemas, tables, permissions, technical metadata, lineage, auditing and discovery. 🥉🥈🥇 Medallion = Data Engineering Bronze → Silver → Gold describes how data is progressively refined. It is not a governance hierarchy. That distinction is important: Governance Domains answer: 👉 Who owns and governs the data? Bronze / Silver / Gold answer: 👉 What stage of engineering and refinement is the data in? And Gold does not automatically equal Data Product. My preferred enterprise model is therefore: Purview → Enterprise Governance & Discovery Unity Catalog → Databricks Governance & Access Medallion → Data Engineering & Refinement The goal isn't Purview vs Unity Catalog. It is clearly defining which platform is authoritative for which responsibility and connecting the metadata and lineage into a coherent governance model. The distinction between governance domains (who owns the data) and Medallion layers (what stage of refinement) is one that trips up many implementation teams. In practice, when Microsoft Fabric is the analytics platform, Purview becomes the authoritative governance layer across both Fabric OneLake and Databricks, with Unity Catalog handling Databricks-internal access. Connecting lineage metadata across both is what creates a truly coherent governance model.
Data Product Lineage not automatic
Hi all i am new in Purview and I am trying to create a lineage with a Report--> Dataset--> Tables in Data Products however after adding report only dataset is shown, how can I view all tables of this dataset. Only when I press open in fabric the Dataset the table list appears, how can I give permissionsMicrosoft Purview Unified Catalog; Governance Domains and Business Concepts
I've been using the attached artefacts for some time to help explain the knowledge exchange aspects of Microsoft Purview Unified Catalog, particularly how Governance Domains and Business Concepts work together to provide business context, ownership, stewardship and operational insights. They have been useful in workshops with data architects, governance professionals, product owners and business stakeholders to demonstrate how concepts fit together within a governance domain and contribute towards trusted information and better business outcomes. I'm interested in hearing from the wider Purview community: Do these artefacts accurately represent the intent and capabilities of Governance Domains within Microsoft Purview? Are there any concepts that you feel are missing, over-emphasised, or could be represented more clearly? How are others explaining Governance Domains and Business Concepts to non-technical stakeholders? Any feedback, suggestions, or alternative approaches would be greatly appreciated. I'm always looking to refine these materials and make them more useful for organisations adopting Purview Unified Catalog. #MicrosoftPurview #DataGovernance #DataManagement #Metadata #DataProducts #MicrosoftData #Purview #DataArchitecture #UnifiedCatalog
Acessos pastas SharePoint / SharePoint folder access
É possível identificar somente acessos a pastas do SharePoint? Algumas informações internas da companhia foram vazadas e precisamos saber quem acessou a pastas do Sharepoint onde estavam essas informações. Outra situação, é que conseguimos identificar que nessa mesma pasta alguns arquivos foram baixados, mas no relatório o usuário aparece "app@sharepoint". Alguém saberia explicar? It is possible to identify access to SharePoint folders? Some of the company's international information was leaked, and we need to know who accessed the SharePoint folders containing that data. Another issue is that we identified files being downloaded from that same folder, but the user appears as "app@sharepoint". Could anyone explain this?
