From Purview Data Governance to the Fabric OneLake Catalog
What is changing, What is available today, and How to transition without losing GOVERNANCE outcomes! Microsoft's data governance story is moving closer to the data. In Microsoft Fabric, the OneLake catalog is now the primary experience for discovering and governing Fabric data, while Microsoft Purview continues as the security and compliance layer that protects, audits, and monitors it. For teams that built their governance practice around Purview Data Governance, this is more than a change of screens. It is a change in architecture and operating model. This article explains the shift, what the OneLake catalog offers today, how responsibilities split between Fabric and Purview, and a practical, outcome-first way to transition. The strategic shift: from a separate control plane to governance in the platform The earlier pattern, often described as a "catalog of catalogs," centered governance on scanning metadata from many systems into a separate control plane. The Fabric approach starts from a lake-first foundation instead: make data available through Fabric and OneLake, whether natively or through supported integration patterns such as shortcuts, mirroring, and sharing. Keep governance context next to that data, and surface governance where owners and consumers already work. Microsoft's own guidance states the division of labor plainly. The OneLake catalog remains the primary experience for governing Fabric data, and using Purview does not change where Fabric governance lives. Purview adds protection, audit, risk, and compliance coverage where needs extend beyond Fabric. A simple rule of thumb: Use the OneLake catalog to discover, understand, and govern Fabric data. Use OneLake security and Fabric controls to enforce access on the actual data path. Use Microsoft Purview to classify, protect, monitor, investigate, and demonstrate compliance across supported Fabric experiences. Data reaches OneLake four ways, the catalog governs it through Explore, Govern, and Secure, and Purview protects and audits the whole path. What the OneLake catalog is The OneLake catalog is organized into three experiences: Explore, Govern, and Secure. It is reached from the Fabric navigation pane, is embedded in Microsoft Teams, Excel, and Copilot Studio, and can be queried programmatically through the Catalog Search REST API. Explore: find and understand Explore is the default tab. It provides an items list with an in-context details view, selectors and filters (including by domain), and the ability to work across workspaces side by side. It is permission-trimmed and focused on Fabric items, so it is not automatically an inventory of every external source. Govern: understand health and act Govern is the centralized place to manage and govern the Fabric estate. It combines: Governance Insights, which explain where your data stands, and Recommended Actions, which give concrete next steps. Two views: All data in Fabric for tenant-wide visibility and My items for the data you own. Both can be scoped to a domain or subdomain. Governance capabilities: domains, tags, endorsement, metadata scanning, cross-tenant sharing discovery, and policies. Administrative experiences that are moving in from the Admin portal: capacities, workspaces, tenant configuration, Azure connections, workloads, and more. The estate-level report, reached through View more, is organized into three tabs: manage your data estate, protect/secure/comply (sensitivity label coverage and DLP evaluation), and discover/trust/reuse (freshness, curation, descriptions, endorsement, sharing). Secure: see and manage access Secure gives a unified view of workspace roles and OneLake security roles across items, so admins can audit permissions, see who has access to what, and create, edit, or delete security roles from one place. Fabric Policies: a growing foundation Fabric Policies, managed from the Policies center in Govern, are the newest piece. Administrators define condition-based rules (by workspace, security group, sensitivity label, item type, and similar attributes), activate them for a Tenant or Capacity admin scope, and Fabric records allow/deny decisions in the audit logs. Policies are stored as Fabric items, so they support REST APIs, Git integration, and CI/CD, and they are included with the Fabric license at no extra charge. Today the capability is deliberately narrow, and it is worth being precise about that: The only effect available is Allow. A policy with rules behaves as an allow list: anything that does not match a rule is blocked. Three policies are supported: external data sharing (tenant scope), item creation (capacity scope), and workspace settings editing (tenant scope). Only one policy can be active per scope, rules are capped (100 at tenant scope, 50 at capacity scope), and changes take effect within about 15 minutes of activation. Policies are not currently supported in the West Europe, North Europe, and West US capacity regions. Treat Fabric Policies as a promising foundation for centralized control, not as evidence of universal policy coverage. Where Purview still matters Purview is not removed from the picture. Microsoft documents the following integrations as available today: Information Protection: sensitivity labels can be set on all Fabric items, and data stays protected when exported through supported paths. This requires additional Purview licensing. Data Loss Prevention: policies evaluate labels and sensitive info types for supported structured data (lakehouses, warehouses, databases, semantic models), with policy tips and alerts. This also requires Purview licensing. Audit: Fabric user activities are logged in the Purview audit log. Insider Risk Management: ready-to-use risk indicators for Fabric activities such as Power BI and lakehouse operations. AI governance: risk discovery in prompts and responses, audit coverage for AI interactions, and retention and eDiscovery applicability for AI-generated content in supported Fabric Copilots and agents. There is one more point that matters for existing customers. Purview's organization-wide governance for Fabric, delivered through its live view, is described in Microsoft's documentation as a preview capability that lets consumers see the Fabric workspaces they can access and lets you run manual scans to make item-level metadata available in Purview. If you rely on Purview as the place where Fabric metadata lands, validate this path against your needs. Mapping Purview Data Governance outcomes to Fabric The most useful way to plan is by the job to be done, not by the old product menu. The table below gives a defensible current-state mapping. "Partial" means part of the outcome is covered and part is not; it is not a polite way of saying "equivalent." Purview outcome Target today Transition guidance Catalog search and browse OneLake catalog Explore, item details, filters, Catalog Search API Move Fabric discovery now. Keep separate discovery for external sources not represented in Fabric. Collections, domains, ownership Fabric domains and subdomains, workspace assignment, domain admins, owners Map organizational boundaries and accountabilities. Keep a responsibility matrix for stewardship duties without a dedicated role. Glossary, critical data elements, custom context Tags, descriptions, endorsement Export and preserve terms and ownership. Do not claim full glossary or CDE parity. Data products and publishing Domains, endorsement, sharing, discoverable semantic models Define the product contract, owner, health, lifecycle, and access process outside the catalog where needed. Lineage and impact analysis Fabric lineage and impact analysis for supported items Move supported paths. Retain another method for cross-system, unsupported-workload, or granular lineage. Governance health Governance Insights and Recommended Actions Use as the primary Fabric posture view. Keep business KPIs and issue closure in the systems that own them. Data quality Freshness and curation insights; workload and source tools Governance Health is not a rules, profiling, and scoring engine. Validate a documented capability before presenting it as a replacement. Access requests and workflows Item sharing, semantic-model request access, source controls Keep the request, approval, expiration, revocation, and evidence process where Fabric does not cover it end to end. Classification and sensitive-data protection Purview sensitivity labels and DLP on supported Fabric items Keep Purview here. Validate item types, inheritance, export behavior, and licensing. Access policy and enforcement Fabric Policies (preview); workspace, item, OneLake, and source permissions Enforce on the specific data path now. Do not assume universal policy coverage. External metadata scanning Shortcuts, mirroring, external sharing, scanner/admin APIs Data connectivity is not enterprise metadata scanning. Define a bridge or partner approach for uncovered sources. APIs and extensibility Catalog Search REST API, Fabric metadata/scanner APIs Validate fields, identities, and scale. Do not assume compatibility with existing Purview API integrations. Audit, risk, compliance Purview Audit, DLP, Insider Risk plus Fabric evidence Correlate evidence across identity, data, action, and retention. One Fabric report is not universal proof. AI and agent governance OneLake controls, Purview AI controls, Fabric semantic capabilities Pilot one end-to-end AI path and verify grounding, permissions, actions, and retained evidence. Governing data, access, and AI as layers No single catalog entry, label, or role governs every path. Microsoft's Cloud Adoption Framework guidance offers a helpful way to think about the layers that need to line up: Workspace access. Use workspace roles when someone needs broad capability across a workspace, assign them to Microsoft Entra security groups rather than individuals, and restrict the Admin and Member roles because they can manage and share content. Item access. Use item permissions when someone needs one data product without the rest of the workspace. Sharing an item does not necessarily grant permission to query the underlying OneLake data. Data access. Use OneLake security or an experience-specific control when different audiences need to see different tables or records from the same data product. Workspace isolation. Create separate workspaces when administration, deployment, or network requirements truly differ, not merely to reproduce boundaries that item permissions or OneLake security can enforce. The practical consequence is that control-plane and data-plane permissions are separate. A role that lets someone manage an item is not the same as one that lets them read its data, so test the actual compute and query path. The highlighted layer is the one that governs the data itself; the others govern who can manage, share, or isolate content. For Copilot and agent scenarios, document and test six things for each use case: identity (whose permissions are used), grounding (what items it can retrieve), context (which definitions, labels, and lineage shape answers), action (answer only, or write, share, and trigger), protection (labels, DLP, tenant and network controls), and evidence (which prompts, responses, and actions are audited and retained). A practical transition method The goal is not to copy every Purview object across. It is to preserve each governance outcome, its owner, its control, and its evidence, while taking advantage of Fabric-native capabilities. Move in waves. Step 1: Inventory the current footprint. Capture registered sources, scans and network runtimes, collections and domains, owners and stewards, glossary terms and CDEs, custom attributes, data products, lineage, quality rules, workflows, access policies, APIs, reports, licenses, and evidence-retention requirements. Step 2: Classify every requirement. Place each job in one of four buckets: move now to a documented Fabric capability; coexist temporarily; bridge with an operating process or partner; or redesign because a one-to-one replacement is neither available nor desirable. Step 3: Design the target operating model. Decide on Fabric domains and subdomains, workspace and capacity topology, delegated administrators, item owners, who may certify, the tag taxonomy, OneLake roles, any Fabric Policies you will use, and the handoff to Purview for security and compliance. The Cloud Adoption Framework advises that domain design is led by business and enterprise architects, with Fabric admins implementing it and a center of excellence involved, because carefully defined domains and workspace-to-domain mapping are what make the catalog useful. Step 4: Pilot one end-to-end scenario. Choose one business domain, one critical asset, one producer, one consumer, one denied identity, and one analytics or AI use case. Test discovery, context, lineage, access, labels, DLP, audit evidence, and remediation. The denied identity matters: proving that access is blocked is as important as proving it works. Step 5: Migrate in waves and verify. Move supported metadata and ownership, train users, and run old and new processes in parallel only as long as necessary. Retire a Purview dependency only after documented acceptance criteria pass. Questions to settle early Which governance outcomes are business- or regulatory-critical, and which are convenience features? Which data is already in Fabric, and which sources can be brought in through shortcuts, mirroring, or supported sharing? Which Purview objects or APIs are embedded in production processes? Where is stewardship a formal role, and where can item ownership and domain administration cover the responsibility? Which use cases need a business glossary, CDEs, custom metadata, broad workflows, or cross-estate lineage? Which labels, DLP policies, audit records, retention rules, and investigations must continue unbroken? Which analytics and AI experiences will consume the data, and what actions can they take? Conclusion The move from Purview Data Governance to the OneLake catalog is best understood as governance becoming a property of the platform rather than a layer scanned on top of it. For Fabric data, the catalog already delivers real value in discovery, domains, endorsement, lineage, governance health, and centralized access views. Purview remains essential for labels, DLP, audit, insider risk, and AI protection. The gaps are equally real: external metadata scanning, glossary and CDE depth, data quality workflows, and broad policy coverage are not one-to-one replacements today. The organizations that will transition best are the ones that start from outcomes, state honestly what is available versus in preview, pilot a single end-to-end path, and retire old tooling only when the evidence says it is safe. Sources OneLake catalog overview Govern and manage your Fabric data estate with the OneLake catalog What are Microsoft Fabric policies (preview)? Governance overview and guidance Use Microsoft Purview to protect Microsoft Fabric Fabric governance in a unified data platform (Cloud Adoption Framework)
Amplify Healthcare Intelligence: Data, AI, and Agent-Powered Transformation
Join Microsoft for Amplify Healthcare Intelligence, a webinar and in-person workshop series designed for healthcare and life sciences organizations building the foundation for trusted AI. Every session starts from the same premise: you cannot deliver trusted AI without a trusted data foundation. Across the series you will see how leading organizations unify their data estate, ground AI agents in real business context, and turn that foundation into results they can measure. What You Will Learn How to build a unified, AI-ready data foundation across a fragmented healthcare data estate How to ground AI agents in trusted healthcare data and real business context How to modernize analytics while reducing complexity and cost How to accelerate innovation with Microsoft Fabric, Azure AI Foundry, Microsoft IQ, and Copilot technologies How to deliver measurable impact across clinical, operational, research, and business scenarios Whether you are defining your AI strategy, modernizing your analytics platform, or scaling AI across your organization, these sessions offer practical guidance, real-world customer examples, and hands-on learning to help you move from AI ambition to business impact. Webinars & In-Person Workshops 🎥 Webinars One-hour virtual sessions with actionable guidance, live demonstrations, customer stories, and best practices from Microsoft experts. Each webinar shows how leading healthcare organizations are turning data, AI, and enterprise intelligence into measurable business outcomes. All sessions are from 3-4 ET (12-1 PT) and are free to attend. Date Topic Register Nov 18 3-4 ET (12-1 PT) From Question to Visuals: Data Insights with Fabric, Rayfin & GitHub Copilot Register Dec 2 3-4 ET (12-1 PT) The Intelligent Healthcare Enterprise: Microsoft IQ in Action Register Dec 9 3-4 ET (12-1 PT) From Trusted Data to Action: Microsoft Fabric + Copilot Cowork for Healthcare & Life Sciences Register Dec 16 3-4 ET (12-1 PT) Beyond Insights: Transforming Pharma & Life Sciences with Data, AI and Intelligent Agents Register Jan 13 3-4 ET (12-1 PT) From Data Silos to AI-Ready: Building a Unified Data Foundation Register Jan 20 3-4 ET (12-1 PT) Ready for AI: Building Trust into Your Data and AI Strategy Register Jan 27 3-4 ET (12-1 PT) From Healthcare Data to AI-Ready Intelligence Register Feb 3 3-4 ET (12-1 PT) Beyond Data: Building the Context AI Needs Register Feb 10 3-4 ET (12-1 PT) AI Agents in Healthcare: Connecting Data, Decisions, and Workflows Register Feb 17 3-4 ET (12-1 PT) Code Smarter, Build Faster: AI-Powered Development with Fabric Register Cannot join live? Register anyway. We will send you the recording and session materials after the event. Additional sessions will be added through the end of the year, so check back or register for one session to be notified as new dates are announced. 🏢 In-Person Workshops Our two-day workshops combine executive strategy, healthcare-specific use cases, architecture guidance, and hands-on labs designed to help teams identify and accelerate high-value AI opportunities. Attendance is free. Participants are responsible for their own travel and accommodation, and space at each location is limited. Workshops run 9am to 4pm local time on both days. Day 1: From Healthcare Data to Healthcare Intelligence Day 1 focuses on healthcare transformation strategy, customer examples, and the architectural patterns that make trusted AI possible at scale. The day closes with a networking reception and peer exchange. The Frontier Transformation imperative: from AI ambition to measurable impact Microsoft IQ: turning data into enterprise intelligence Building the unified data foundation Building trusted AI: security, governance, privacy, and compliance Healthcare transformation in action: clinical, operational, research, and finance scenarios Activating data with AI data agents and Copilot experiences Day 2: Hands-On Healthcare AI and Analytics Lab Day 2 is a guided, end-to-end lab. Participants build a working healthcare intelligence solution from raw data through to a grounded AI agent, using Microsoft Fabric, Azure AI Foundry, Copilot technologies, and modern data architectures. Build the foundation: data ingestion, lakehouse architecture, and data engineering Create actionable insights: semantic models and dashboards Prepare data for AI: AI-ready data assets and data governance Build and ground AI agents in trusted enterprise data From insight to intelligent action: planning your organization's next steps What to bring: a laptop with a current browser. Lab environments and credentials are provided on site, and no prior Fabric or Foundry experience is assumed. Date City Venue Register December 8-9, 2026 9am - 4pm New York Microsoft Garage 300 Lafayette Street New York, NY 10012 Register February 9-10, 2027 9am - 4pm Detroit Microsoft Detroit 1 Campus Martius, Floor 5 Detroit, MI 48226 Register April 13-14, 2027 9am - 4pm Philadelphia Microsoft Malvern 45 Liberty Blvd, Suite 210 Malvern, PA 19355 Register November 10-11, 2027 9am - 4pm Chicago Microsoft Chicago (AON Center) 200 East Randolph Drive, Suite 200 Chicago, IL 60601 Register Who Should Attend This series is built for the people who own the data estate and the people who depend on it. Sessions are technical enough for practitioners and strategic enough for the leaders who prioritize the work. Data & analytics leaders Data platform, data engineering, and business intelligence teams Data architects, engineers, and data scientists AI and innovation leaders Healthcare and life sciences executives Clinical, operational, research, and finance transformation leaders No prior Microsoft Fabric or AI experience is required for any session in this series. Questions Contact Camille Whicker if you have any questions about our upcoming events.From Python Agents to Copilot Harness : Putting Microsoft Foundry to the Test
You have built the agents. Written the Python. Connected the data. Worked out how the pieces talk to each other. Then the conversation shifts to harnesses. The prospect of convergence around the GitHub Copilot harness across GitHub, Microsoft Foundry and Copilot Studio (CPS) raises a very practical question: What happens to the pro-code agents I already built? Do I rewrite them, wrap them, or keep them? Rather than debate it in the abstract, I wanted to give it a shot. I already had a Microsoft Agent Framework (MAF) demo with Python agents, Fabric sales data and product knowledge. Could I keep the useful business workflows, remove the custom agent wrappers, and let a managed harness do more of the work? That experiment became the demo behind this blog. Not another empty chat box, but an app to ask business questions, inspect actual tool activity, follow the evidence and create a brief you can revise and download. This is a practical Foundry experiment and the question I wanted to answer was: how much of this particular agent application can move into the harness, and what still belongs in my code? TL;DR: I moved a Python/MAF research demo to a Foundry-managed Copilot harness to test whether it could handle the same business questions. Native research and source connections reduced the agent-side orchestration I had to write, while the app still owns authentication, streaming and secure access. Downloadable briefs are an extra experiment, not part of the original migration. Focused tests were promising, but do not establish complete feature parity or production readiness. The Starting Point: Pro-Code Agents The starting point is olivierb123/microsoftiq-lead-agent-demo. It used four hosted Agent Framework wrappers and application-controlled fan-out. Python agent code and custom source adapters handled the integrations; the application determined which agents to call and assembled their results. That approach makes the orchestration explicit. It also puts the responsibility for hosting the wrappers, maintaining source adapters and coordinating the workflow in the application. For this experiment with harness, I kept the business scenario but changed the execution model. Instead of recreating those wrappers inside another service, I configured one Foundry-managed Microsoft Copilot harness agent with instructions, native capabilities and connected sources. Responsibility Pro-Code Demo Harness Experiment Workflow orchestration Application-controlled agent calls Managed agent selects tools and can delegate Research Hosted wrappers and custom adapters Native web search and page fetching Business data Application source integration Native Fabric connection Product knowledge Application retrieval integration Foundry IQ knowledge-base retrieval through native MCP Working output Application-managed result assembly Conversation workspace and published artifacts Application code UI plus agent orchestration UI, authentication, streaming and session/download authorization This repository contains the migrated implementation. The original pro-code demo remains separate and was not overwritten. A Practical Question, Not a Tool Demonstration The scenario is a fictional sales team working with FieldForge product documentation. A useful assistant should help answer questions such as: How many opportunities are open in the expanded synthetic demo, what is their total and probability-weighted value, and how do our internal forecasting definitions explain those numbers? That request needs two different kinds of evidence: structured opportunity data and the actual internal document defining the forecast. It should not require the user to name a connector or prescribe a tool sequence. A more advanced request asks for a South territory brief combining sales performance, priority opportunities, product capabilities and official construction heat-exposure guidance. The user can then request a revision or a downloadable Markdown artifact in the same conversation. These examples exercise the harness as a working environment, not just a text-generation endpoint. However, they are requests to explore, not promises that every tool or subagent will run on every turn. Why a Local Web App? I deliberately kept the test experience in a local web app. I wanted to make it easy to ask a question, try a follow-up and inspect what happened, without turning the experiment into a web-hosting project first. The React workbench and Node.js backend run locally; the agent and connected sources run in the cloud. After signing in, I can choose a sample question, watch the streamed response, inspect Activity and Evidence, and request a revision or download in the same conversation. I do not need to reconstruct a raw SDK response to see whether a source was actually called. That separation also makes the migration easier to understand. The web app is the test surface, not the agent runtime. There is no local Python agent server hiding behind it: the backend sends the request to Foundry and handles the application's authentication, streaming and session-owned downloads. For this first experiment, the backend is intentionally loopback-only. Hosting the application publicly is a separate engineering task, not a prerequisite for testing the harness against a useful scenario. The Architecture The configured agent uses microsoft_copilot_preview and gpt-5-mini. The agent definition lives in harness/azure.yaml, rather than a Python agent server. The application, backend and data scripts use JavaScript/Node.js. There is no Python application runtime, pip setup or virtual environment required for this version. That is not a zero-code migration: the application still owns sign-in, token validation, streaming, conversation ownership and secure downloads. The backend does not execute the native source calls a second time. Foundry executes them; the application observes their returned results and presents them in Activity and Evidence. What It Looks Like in the Foundry Portal The local workbench is only one view of the experiment. Foundry also supports hosted pro-code agents: I can package custom agent code, including a Python/MAF implementation, and its dependencies in a container and let Foundry manage the hosting. I still own the agent implementation and its orchestration; using Python does not mean I must run the server myself. This experiment instead uses a managed-harness prompt agent. I configure its instructions, capabilities and source connections, while Foundry supplies the harness execution environment and native tool orchestration. Both approaches can run in Foundry. The distinction is not simply local versus cloud, or Python versus no Python: it is how much agent runtime and orchestration I implement versus delegate to the managed harness. The application's UI, authentication and session/download authorization remain my responsibility in this demo. My microsoftiq-copilot agent and its configured tools/connections make the change tangible. The configuration ties together the native harness capabilities, Fabric access and product-knowledge retrieval shown in the architecture diagram. No Web IQ Wrapper in This Version The Web IQ integration from the earlier demo is not carried forward. External research uses the harness's native web search to discover sources and its Web capability to read relevant pages. This supports the research portion of the scenario without maintaining a separate Web IQ agent wrapper. It is not a claim that native web research reproduces every Web IQ behavior or data contract. Fabric and product knowledge are optional grounding sources, not stages that every question must pass through. A public-research question can stay a public-research question. Private business data must not be sent to public web tools. More Direct Integration with Fabric and Foundry IQ For me, the integration improvement is the ownership boundary: the harness uses configured native connections instead of calling application adapters that obtain credentials and query sources themselves. Fabric supplies structured business facts. The native Fabric connection uses the signed-in user's context to access a dedicated data agent over imported semantic models. There is no app-only Fabric credential fallback. The models contain demonstration data, not live CRM records. Foundry IQ supplies document evidence. The product-knowledge MCP connection reaches a knowledge base backed by Azure AI Search. Query planning and semantic retrieval return extractive evidence, including stored document metadata, for the harness to use in its answer. This connection uses the Foundry project's managed identity; it does not establish per-user document ACL enforcement. The distinction matters. A supportingDocId in an opportunity row is a relationship reference, not proof that the corresponding document has been retrieved. Fabric cannot substitute for document content, titles or update dates from the knowledge base. Calling this integration "better" means more direct native integration with clearer source responsibilities. I have not established a quality, latency or cost improvement over the MAF implementation, nor certified genuinely different-user authorization. What the Experiment Actually Showed Focused tests showed that the harness could retrieve sales figures, find internal documents, research the web and continue a conversation. That supported the core research workflows we wanted to carry over from the MAF demo. The most revealing test asked for sales figures together with the internal document explaining how the forecast should be interpreted. Some early runs got the numbers right but did not retrieve the requested document. The answer sounded convincing, but part of its explanation lacked verified support. After clarifying the agent's instructions and which source should answer each part of the question, one test run successfully retrieved evidence from both sources. This used synthetic demo data, not live customer information. The lesson: connecting the tools is only the start. You still need to test whether the agent uses the right sources and whether those sources actually support its answer. Beyond the Migration: Trying Workspace Files and Briefs Once the core question-answering workflows were working in focused tests, I wanted to try something we had not implemented in the MAF demo: turning an answer into a working document. Why save a brief? Not every answer needs one. But a territory review is a useful example: instead of leaving the analysis in a chat response, I can request a Markdown document to keep, review or hand off, then ask for a revision in the same conversation. The file is an optional deliverable, not a prerequisite for retrieving sales data or product evidence. This is where the harness's conversation workspace and native file capabilities become interesting. I can explore creating, reading back and revising an artifact in the managed environment without rebuilding those agent-side tools in a Python wrapper. MAF could support the same kind of workflow with appropriate tools and code; this is an additional capability in this demo, not something exclusive to a harness. Publication adds a separate step when the user actually wants a download. A workspace file is not automatically downloadable: the workbench registers it only after a successful native upload returns a real file ID, and only the owning unexpired session can download it. The application still implements those access checks. The migration question is therefore "can I keep answering the useful business questions?" The extension question is "what else can I build on the harness's native capabilities?" Briefs help explore the second question. Actual tool results, not assistant narration, establish whether creation, revision or publication occurred. There are still operational boundaries. The app is loopback-only, the harness is a public preview, inference and retrieval incur costs, and workspace state is not durable application storage. Refreshing the browser or restarting the backend loses local session mappings. Production hosting, retention and broader authorization need separate work. Shell and external business-system writes remain disabled. The Takeaway So, what happened to my Python agents? For this scenario, their hosted wrappers were no longer needed. The business questions stayed; their execution moved into a managed harness, and the remaining application code focused on the experience and its security boundaries. For this practical research scenario, that shifts the work from maintaining Python wrappers and fixed fan-out toward configuring the agent, defining source responsibilities, building the user experience and verifying evidence. Fabric and Foundry IQ fit into that environment directly, while native web research replaces the earlier Web IQ path for the tasks explored here. When I Would Still Use MAF Deterministic Routing and Local Hybrid Execution: MAF allows you to deploy hybrid topologies where small, local models (like MAI-Decision / Jev) handle rapid, deterministic triage and tool-gating, reserving expensive cloud-hosted frontier models for complex reasoning. A managed harness forces all routing into its proprietary cloud runtime, stripping away your ability to optimize latency and compute token economics. Absolute Control over the Orchestration Loop: The Copilot Harness uses an abstracted reasoning engine to decide when to activate a skill or invoke a tool. Native Toolchain Continuity and Hosting: MAF agents operate as standard C# or Python applications. Granular State and Semantic Memory: Managed harnesses provide a generalized shared memory layer. MAF allows you to build custom memory providers (like custom Azure SQL or Cosmos DB schemas) Unrestricted Interoperability Mechanics: While the Copilot Harness can consume Agent-to-Agent (A2A) protocols and Model Context Protocol (MCP) servers, building the actual server logic in MAF gives you the power to manipulate the raw protocol mechanics. If you need to manage custom message headers, inject highly specific tokens for platforms like SAP or ServiceNow, or alter payload structures mid-flight, MAF provides the unabstracted access required to do so. For this demo, the harness was a useful way to move beyond fixed fan-out and put research, connected evidence and artifacts into one managed environment. For another application, keeping its MAF agents may be the better decision. My takeaway is simple: start with a real task, move the responsibilities that fit the harness, and test the boundary instead of assuming the migration works. What's Next This experiment answered one question—now I'm curious about others: How does the harness perform under sustained load? What happens when Fabric semantic models grow to production scale? Can the same approach work for agents that need to write back to business systems? If you're exploring similar migrations, I'd be interested to hear what worked and what didn't. Try the Project The code is available at olivierb123/microsoftiq-copilot-harness-demo. Choose your path: Run the demo locally → Start with the README and setup checklist Explore the agent configuration → See harness/azure.yaml for the Foundry agent definition Understand the data model → Review the synthetic data documentation Compare with the original MAF approach → Check out the original pro-code demo Questions or feedback? Open an issue on the repo or share your own migration experience in the comments below.From Files to Insights: Building a Medallion Architecture with Microsoft Fabric
Organizations often receive critical business data through files, spreadsheets, and SharePoint, but turning that information into trusted, analytics-ready insights can quickly become complex. Microsoft Fabric simplifies this journey by bringing ingestion, transformation, governance, and analytics together in a unified data platform. In this article, we’ll explore how a Bronze, Silver, and Gold medallion architecture can transform raw files into clean, trusted, and business-ready data for reporting and insights. Create a dedicated Fabric workspace The SharePoint Documents/fabric-lab/customer-drop folder is created, and the initial baseline Excel workbook is uploaded. This simulates how vendor files can be dropped into a controlled landing area before Fabric processes them. For the initial run, the document specifically starts with: customers_2026-11-01_baseline.xlsx. Flow so far: Vendor Excel file → SharePoint landing folder → Fabric Create the Fabric Lakehouse A Lakehouse called lh_sharepoint_bronze_lab is created inside the workspace. The lab design states that the Dataflow creates the staging table while notebooks are intended to create the Bronze tables. What is happening: We now have a OneLake-backed location where the SharePoint data can land and later progress through the medallion architecture. Connect Fabric Dataflow Gen2 to SharePoint A Dataflow Gen2 is created and connected to the SharePoint folder source. The configuration uses the SharePoint site URL, rather than pointing directly at an individual Excel file. This is important because the design is intended to process files arriving in the folder, rather than building a one-off connection to a single workbook. Apply ingestion logic in Power Query The Dataflow's Advanced Editor contains Power Query logic that: Connects to SharePoint. Filters files to the designated folder. Selects .xlsx files. Ignores temporary Excel files beginning with ~$. Opens each workbook. Finds the tblCustomers table. Extracts its rows. Adds the originating filename as source_file. Adds the SharePoint modified timestamp as source_modified. Combines the rows from the accepted workbooks. This source metadata is very useful later because Vendor Intelligence can retain where a record came from and when its source was modified. Validate the first SharePoint file The Dataflow successfully discovers the November 1 baseline workbook in SharePoint. The binary content shown represents the Excel file that Fabric can open and process. What this proves: Fabric can dynamically discover the staged Excel file from the SharePoint landing location. Transform the Excel data into rows The binary Excel workbook has now been interpreted as structured rows. The screenshot shows fields such as CustomerID, CustomerName, Email, Country, Region, SignupDate, Segment, AnnualRevenueUSD, and IsActive. For the actual Vendor Intelligence scenario, the same pattern can be applied to datasets such as: Vendor master records Contracts Spend Risk/compliance information Assessments and attestations Map the data into the Lakehouse The Dataflow destination configuration maps the incoming columns to the destination Lakehouse table and confirms their target data types. What is happening: The ingestion process is transitioning from simply reading a file to writing structured data into Fabric. Execute the ingestion The Dataflow is saved and executed. The screenshot shows the run completed successfully, including the stg_customers_WriteToDataDestination activity. In simple terms: SharePoint → Dataflow Gen2 → Fabric Lakehouse is now functioning end to end. Confirm the records in the Lakehouse The resulting records are now visible in the Lakehouse staging table. The screenshot shows 10 rows from the first baseline file. This confirms that the data has moved successfully from an external SharePoint-hosted Excel file into Fabric. Simulate the arrival of new vendor data A second workbook, customers_2026-11-02_increment.xlsx, is added alongside the original baseline file. This is an important part of the Vendor Intelligence story because it demonstrates that new source data can continue arriving over time, rather than requiring the original file to be manually replaced. Fabric discovers and processes the additional file When the Dataflow runs again, the ingestion query reads the qualifying workbooks and combines their rows. The resulting view includes the original customer records plus the newly introduced records such as CUST-1011 through CUST-1015. This is where the architecture becomes important: staging can represent the current extraction from SharePoint, while Bronze should preserve the accepted source history rather than treating every incoming file as a completely independent business truth. Move from ingestion into the Medallion architecture The diagram below shows how the ingestion pattern evolves into the production Vendor Intelligence data architecture: Staging Temporary output from the SharePoint read. This layer can be replaced whenever the ingestion process executes. Bronze The raw historical layer. It preserves accepted source data and metadata. Vendor Intelligence examples identified in the design include bronze_vendors_raw, bronze_contracts_raw, bronze_spend_raw, and bronze_risk_raw. Silver The trusted/conformed layer. Here Fabric validates types and keys, resolves vendor aliases, standardizes information, removes duplicates, and retains exceptions. Example tables include silver_vendor, silver_vendor_alias, silver_contract, silver_invoice, and silver_risk. Gold The business-ready layer serving Vendor Intelligence applications and analytics, including the Vendor profile, spend summary, contract expiration information, and risk summary. Creating the Staging-to-Bronze notebook A Fabric notebook named 01_Staging_to_Bronze is created to begin the transformation process. The notebook approach allows Spark-based processing to be applied after Dataflow Gen2 has completed the SharePoint ingestion. The separation is intentional: Dataflow Gen2 handles source ingestion → Notebook handles engineering and medallion transformations. Reading the staging data from the Lakehouse The notebook reads the stg_customers table that was populated by Dataflow Gen2. The resulting dataset contains 15 records and 11 columns, including customer attributes such as CustomerID, CustomerName, Email, geography, segment, annual revenue, and activity status. Source metadata is also retained through source_file and source_modified. This is the handoff between ingestion and engineering: SharePoint → Dataflow Gen2 → stg_customers → Fabric Notebook. At this point, staging represents the current set of records discovered from the SharePoint landing location. Preparing records for Bronze Before writing records to Bronze, the notebook enriches each record with metadata required for historical tracking. A SHA-256 _record_hash is generated from the business attributes. Additional fields identify SharePoint as the source system and capture the Fabric ingestion timestamp. The resulting Bronze-ready record therefore contains: Original business attributes source_file source_modified _record_hash _source_system _ingested_at The record hash becomes particularly useful when determining whether the same version of a source record has already been processed. Creating the Bronze historical table The transformed records are written into the Delta table bronze_customers_raw. The notebook confirms that the initial Bronze table was successfully created with 15 records. Bronze serves a fundamentally different purpose from staging. Staging answers: What does the current source extraction contain? Bronze answers: What source records has the platform historically accepted and observed? For the eventual Vendor Intelligence implementation, equivalent patterns can support raw vendor, contract, spend, and risk entities such as bronze_vendors_raw, bronze_contracts_raw, bronze_spend_raw, and bronze_risk_raw. Validating Bronze history and lineage The Bronze table clearly demonstrates the additional lineage attributes captured during ingestion. Each business record is associated with the source file, source-modified timestamp, unique record hash, source system, and Fabric ingestion timestamp. This metadata provides the foundation for traceability and future change detection. For example, if an existing entity arrives later with different business attributes, the newly calculated hash will differ from the historical hash. Fabric can recognize that observation as a new version rather than blindly duplicating records on every ingestion cycle. Detecting new record versions The next notebook logic compares the hashes calculated from incoming staging records against the hashes already stored in Bronze. A left_anti join identifies only records whose hashes have not previously appeared in Bronze. In the screenshot, the notebook reports zero new record versions detected. This is the expected result because the current staging records were already loaded into Bronze. This is an important behavior: Reprocessing the same SharePoint files does not automatically create duplicate historical observations. When source attributes eventually change, the changed business values generate a different hash and the new version can be appended to Bronze rather than replacing its previous history. Separating Silver and Gold into dedicated Lakehouses The architecture is then separated further by introducing dedicated Silver and Gold Lakehouses. The existing lh_sharepoint_bronze_lab continues to contain the staging and Bronze portions of the lab, while lh_vendor_silver and lh_vendor_gold provide isolated locations for trusted and business-ready data. The implementation now follows: SharePoint Staging and Bronze: lh_sharepoint_bronze_lab Silver: lh_vendor_silver Gold: lh_vendor_gold Analytics and downstream consumption This physical separation also makes the medallion progression much easier to understand visually. Reading Bronze for Silver processing The Silver notebook begins by reading bronze_customers_raw from the Bronze Lakehouse. Before processing continues, the notebook validates that the required business, lineage, and hash columns exist. This validation provides an important quality gate between layers. Silver processing should not silently continue when required fields are missing from the upstream dataset. Standardizing and selecting trusted customer records Silver begins transforming raw historical observations into a cleaner representation of the business entity. The notebook normalizes values such as customer identifiers, customer names, email addresses, countries, regions, and segments. Records without valid customer identifiers are removed from the trusted output. A window operation then groups records by CustomerID and orders observations based on source and ingestion metadata. The notebook retains the latest applicable record for each customer. In this lab, the transformation results in 15 current Silver customers. The distinction is now clear: Bronze = historical observations Silver = standardized current business entities Persisting the Silver customer table The trusted output is persisted as silver_customer in the dedicated Silver Lakehouse. The screenshot shows the customer data stored alongside useful lineage attributes inherited from the Bronze layer. In the eventual Vendor Intelligence implementation, the Silver layer becomes particularly important because this is where vendor identities from different systems can be conformed. The target Vendor Intelligence architecture anticipates Silver entities such as silver_vendor, silver_vendor_alias, silver_contract, silver_invoice, and silver_risk. Silver is also the logical layer for standardization, duplicate resolution, alias handling, and exception management Completing the Silver transformation The completed Silver notebook demonstrates the transition from raw historical ingestion to a current, standardized dataset that downstream workloads can consume. Instead of requiring analytics workloads to understand every SharePoint file, ingestion timestamp, or historical source observation, downstream processing can work against a cleaner representation of the entity. This is an important architectural boundary: Bronze preserves what arrived. Silver establishes what can be trusted and consumed. Moving from Silver to Gold The next stage is Gold, the business-ready layer of the medallion architecture. The target Vendor Intelligence design identifies business-facing outputs such as vendor profiles, spend summaries, contract-expiration information, and risk summaries. For the current lab, which uses customer sample data, Gold should be described as a customer analytics demonstration rather than actual vendor-spend analytics. The Gold notebook reads the trusted Silver customer dataset and aggregates the records into analytics-ready measures such as: Customer count Active customer count Annual customer revenue Customer distribution by region Customer distribution by segment The resulting Gold table can be stored as gold_customer_segment_summary in lh_vendor_gold. Gold therefore answers a different question from the previous layers: Bronze: What records arrived and when? Silver: What is the trusted current representation? Gold: What information should the business consume? The complete lab architecture At this point, the solution demonstrates the complete medallion journey: SharePoint landing Files arrive in a controlled SharePoint folder. Dataflow Gen2 ingestion Fabric dynamically discovers qualifying Excel workbooks and transforms the file content into structured rows. Staging stg_customers represents the current SharePoint extraction. Bronze bronze_customers_raw preserves accepted source observations, lineage metadata, hashes, and ingestion timestamps. Silver silver_customer creates a standardized, current view of customer entities in a dedicated Silver Lakehouse. Gold The Gold layer transforms trusted entities into business-ready aggregations designed for analytics. Create the pipeline Save and Run your pipeline Schedule the pipeline Consuming the Gold Layer Through a Business Dashboard The dashboard surfaces several key metrics: Total Customers provides a consolidated count of customers represented in the Gold layer. Active Customers highlights the portion of those customers currently identified as active. Annual Customer Revenue summarizes the annual revenue values associated with the current customer population. Annual Revenue by Region provides a geographic view of where customer revenue is concentrated. Customers by Segment shows the distribution of customers across the available customer segments. Region × Segment Detail provides a more granular view of customer count, active customers, and annual revenue across the two business dimensions. The dashboard completes the end-to-end data flow demonstrated throughout the lab: SharePoint → Dataflow Gen2 → Staging → Bronze → Silver → Gold → DashboardCan Fabric Copilot Run on a Separate Capacity?
Microsoft Cloud Architects: Yassine El Ouardi and Rafia Aqil Customer question: Can we bill our team’s Fabric Copilot usage to a separate capacity while keeping data processing and reporting on our existing capacity? Answer: Yes. An organization can designate a Fabric capacity for Copilot usage and assign specific users or groups to it. Eligible Copilot usage from those users is then billed to the designated capacity rather than the capacity hosting their content. The workspace can remain on its existing capacity for its underlying workloads. The minimum Fabric SKU for a designated Copilot capacity is F2. That makes F2 a starting point for a small team, not a guarantee that it will be large enough for every usage pattern. Monitor consumption as adoption grows and scale the capacity if needed. LAB: In this lab, I created an F2 capacity for Copilot and kept the Adventure Works workspace on a separate F16 capacity. The setup illustrates the key architectural choice: assign users to the Copilot capacity, rather than moving their workspace to it. F2 capacity created: The Azure overview shows the new, active F2 Fabric capacity intended for the Copilot test. Both capacities visible: The admin portal lists the existing F16 capacity alongside the new F2 capacity. Copilot assignment setting under the capacity: The F2 capacity settings show where an administrator can designate the capacity for Copilot and select users or groups. Original workspace stays on F16: Workspace settings show that the Adventure Works workspace remains assigned to the F16 capacity. Semantic model opened: The Adventure Works model provides a practical workload for testing Copilot’s modeling assistance. Question submitted: The test asks Copilot to help fix the data model; the screenshot captures it processing the request. Recommendations returned: Copilot proposes specific model changes and says it has not applied them without confirmation. Capacity health checked: Fabric Capacity Metrics displays both capacities for comparison; this view shows 8 successful operations completed successfully for the Copilot Capacity. Takeaway: A separate Copilot capacity gives a small team a way to consolidate Copilot billing without relocating its workspace. Start with F2 where the feature’s requirements are met, then size from observed usage.Tableau to Power BI Migration: Semantic Layer-First Approach for Cloud Architects
Author's: Mahjabin Ahmed, Yassine El Ouardi, Lavanya Sreedhar LavanyaSreedhar, Peter Lo PeterLo, Aryan Anmol aryananmol, Shreya Harvu shreyaharvu and Rafia Aqil Rafia_Aqil In this guide, we provide practical guidance for migrating from Tableau to Power BI, with a focus on technical best practices and architecture. Unifying business intelligence on the Microsoft Fabric platform, enterprises gain closer integration with Microsoft 365 (Teams, Copilot, Excel). For cloud solution architects and BI developers, a successful migration is not just about rebuilding dashboards in a new tool. It requires thoughtful architectural planning and a shift to a more model-centric approach to BI. Why Semantic Layer-First Architecture Matters The Traditional Migration Challenge Most Tableau to Power BI migrations follow a dashboard-centric approach: teams attempt to replicate existing Tableau workbooks, calculated fields, and LOD (Level of Detail) expressions directly into Power BI reports. While this may seem efficient initially, it creates significant downstream challenges: Duplicated logic: Each report embeds its own calculations and business rules, leading to conflicting KPIs across the organization Maintenance overhead: Changes to business logic require updating dozens or hundreds of individual reports Governance gaps: Without centralized definitions, semantic drift occurs—different teams calculate "Revenue" or "Active Customer" differently Scalability issues: As data volumes grow, report-level transformations become performance bottlenecks The Semantic Layer-First Alternative Microsoft's recommended approach centers on semantic models (formerly called datasets)—centralized, governed data models that separate business logic from visualization. In this architecture: The payoff is substantial: when data evolves or business rules change, you update the semantic model once, and all dependent reports automatically reflect the changes—no manual redesign required. Understanding Migration Complexity: Simple to Very Complex Dashboards Not all Tableau dashboards are created equal. The migration strategy should align with dashboard complexity, and the semantic layer approach becomes increasingly valuable as complexity grows. Follow a Step-by-Step Migration Strategy Migrating from Tableau to Power BI is not a one-click effort – it requires a mix of automated and manual refactoring, plus a sound change management plan. Below are key strategies and best practices for a successful migration: Audit your Tableau estate: Start by taking inventory of all existing Tableau workbooks, data sources, and dashboards. Determine what needs to be migrated (focus on high-value, widely used reports first) and identify any redundant or obsolete content that can be retired rather than converted. Conduct a proof-of-concept (PoC): Before migrating everything, pick a representative complex dashboard (or a subset of your data) and perform a pilot migration. This will help you validate that Power BI can connect to your data (e.g. setting up the Power BI gateways for on-premises sources), test performance (Import vs DirectQuery modes), and experiment with replicating key visuals or calculations. Use the PoC to uncover any surprises early – for example, test that any Level of Detail expressions or table calculations in Tableau can be re-created in DAX. The lessons learned here should inform your overall project plan. Use a phased migration approach: Plan to run Tableau and Power BI in parallel for some period, rather than switching everything at once. Migrate in waves – for example, by business unit or subject area – and incorporate user feedback as you go. This phased approach reduces risk and allows your team to improve the process with each iteration. It also gives end users time to adjust gradually. Migrate high-impact dashboards first: Prioritize the migration of key reports and dashboards that are critical to the business or have the most usage. Delivering these early wins will not only surface any technical challenges to solve but will also help demonstrate the value of Power BI’s capabilities to stakeholders. Early success builds buy-in and momentum for the rest of the migration. Reimagine (don’t just replicate) the experience: It’s rarely possible – or desirable – to exactly re-create every Tableau visualization pixel-for-pixel in Power BI. Embrace the opportunity to focus on business questions and improve user experience with Power BI’s features. For example, rather than replicating a complex Tableau workaround, you might implement a cleaner solution in Power BI using native features (like bookmarks, drilldowns, or simpler navigation between pages). Engage business users and subject matter experts during this redesign to ensure the new reports meet their needs. Enable dataset reusability: One major benefit of the Power BI approach is the ability to create shared datasets and dataflows. As you migrate, look for opportunities to create central semantic models (datasets) that can serve multiple reports. For instance, if several Tableau workbooks are all using similar data about sales, you can create one central Sales dataset in Power BI. Report creators across the organization can then build different Power BI reports on that single dataset without duplicating data or logic. This reduces maintenance and promotes a “build once, reuse often” strategy. Provide training and support: Expect a learning curve for teams moving to Power BI – especially those who are very fluent in Tableau. Plan for user upskilling and training programs. Establish a support community or office hours where new users can ask questions and get help. If possible, identify Power BI champions or recruit a Power BI Center of Excellence (COE) team who can guide others. During the transition, ensure there are subject matter experts (SMEs) available to address questions and validate that the new reports are correct. Manage change and expectations: It’s important to communicate why the organization is moving to Power BI (e.g. benefits like deeper integration, lower TCO, better governance) to get buy-in from end users. Some users may be resistant to change, especially if they’ve invested a lot of time in mastering Tableau. Prepare to handle varying responses – emphasize the personal benefits (like improved performance, new capabilities, or career growth with popular skills) to encourage adoption. Also, involve influential business users early and gather their feedback, so they feel ownership in the new solution. Establish governance from Day 1: Don’t wait until after migration to think about governance. Use this chance to set up Power BI governance aligned to best practices. Decide on important aspects such as workspace naming conventions, who can create or publish content, how you’ll monitor usage and costs, and how to manage data access and security (for example, designing a strategy for RLS/OLS/CLS, and deciding when to use per-user datasets vs. organizational semantic models). Good governance will ensure your shiny new Power BI environment doesn’t sprawl into chaos over time. Allow time for adjustment and iteration: Finally, be patient and iterative. Depending on the scale of your organization and the number of Tableau assets, a full migration can take months or even a year or more. Plan realistic transition periods where both systems might coexist. Continuously refine your approach with each wave of migration. Power BI’s frequent update cadence (monthly releases) means new features may emerge even during your project – stay updated, as new capabilities could simplify your migration (for example, the introduction of field parameters or Copilot might let you modernize certain Tableau features more easily). Reimagine (don’t just replicate) the experience (Step 5): Phase 1: Assessment and Planning 1. Audit Your Tableau Estate Inventory all workbooks, data sources, and calculated fields Identify high-traffic dashboards (prioritize for early migration) Categorize by complexity (Simple/Medium/Complex/Very Complex) 2. Design Your Semantic Architecture Map Tableau data sources to Power BI data sources (DirectQuery, Import, or Direct Lake) Plan star schema for fact/dimension tables Identify shared calculations that should live in semantic models vs. report-specific logic 3. Choose Storage Modes Source Type Recommended Mode Rationale Databricks Delta Lake Direct Lake Real-time analytics, no refresh lag Azure SQL Database DirectQuery or Import Based on data volume and refresh SLAs On-Premises SQL Server Import (via Gateway) Network latency considerations Excel/CSV files Import Small reference data Phase 2: Build the Semantic Layer 1. Create Star Schema Data Models Tableau often relies on flat, denormalized datasets. Power BI performs best with star schemas: Fact tables: Transactional data (sales, orders, events) with foreign keys to dimensions Dimension tables: Descriptive attributes (customers, products, dates) with primary keys Relationships: One-to-many from dimension to fact, leveraging bidirectional filtering sparingly 2. Migrate Calculations to DAX Measures Convert Tableau calculated fields to DAX measures in the semantic model: --Example of DAX: -- Define as measure: Total Revenue = SUMX( 'Sales', 'Sales'[Quantity] * 'Sales'[Unit Price] ) 2.1 Use Copilot to Accelerate DAX Development Leverage Copilot in Power BI Desktop to generate and validate DAX: Describe the calculation in natural language Copilot suggests DAX syntax Review, test, and refine 2.2 Document your Semantic Model Invest in creating an AI-ready foundation for your semantic model. AI systems need to understand unique business contexts in order to prioritize correct information to provide consistent and reliable responses to your end users. Name Tables and Columns Clearly: Avoid ambiguity in your semantic model. Use human-readable, business-friendly names. Avoid abbreviations, acronyms, or technical terms. This improves Copilot’s ability to interpret user intent. Create Meaningful Measures: Define reusable DAX measures for key business metrics (e.g., Revenue, Profit Margin). AI features rely on these to generate insights and summaries. Document Semantic Model objects: Add descriptions and synonyms to your Tables, Columns and measures. This enhances natural language querying and improves Copilot’s contextual understanding. Build an AI Data Schema: prepare your semantic model for AI by utilizing tooling features such as Prep data for AI. Phase 3: Understanding Migration Complexity: Simple to Very Complex Dashboards Not all Tableau dashboards are created equal. The migration strategy should align with dashboard complexity, and the semantic layer approach becomes increasingly valuable as complexity grows. 1. Dashboard Conversion Best Practices Think in "pages" not "sheets": Power BI reports combine multiple visuals per page; group related visuals logically Use slicers for interactivity: Replace Tableau filters with Power BI slicers and filter pane Leverage bookmarks for navigation: Create dynamic report experiences with show/hide containers Simple Complexity Level Category Tableau Feature Power BI Equivalent Microsoft Fabric Enhancements Best Practice Notes Data Model Single custom SQL Power Query for data shaping and ETL. OneLake Shortcuts for unified data access. Use star schema for optimized performance; push logic into the semantic layer rather than visuals. Calculations Basic IF/ELSE, SUM Data Analysis Expressions (DAX) for measures and calculated columns. Copilot for Power BI to assist with DAX creation. Fabric IQ for natural language queries. Centralize calculations in semantic models for consistency and governance. Medium Complexity Level Category Tableau Feature Power BI Equivalent Fabric Enhancements Best Practice Notes Data Model Multiple custom SQL (up to 3) Connect live to databases (Azure Databricks): DirectQuery in Power BI Connect with cloud data sources: Power BI data sources OneLake Shortcuts for unified access without databricks compute cost. Semantic Models can combine multiple sources. Optimize with star schema; Prefer OneLake Shortcuts for performance; avoid heavy transformations in visuals. Calculations Nested IFs, CASE Data Analysis Expressions (DAX) for measures and calculated columns. Copilot for Power BI to assist with DAX creation. Fabric Data Agent for conversational BI. Fabric IQ for natural language queries: Fabric IQ Centralize logic in semantic models; use Copilot for automation and validation; keep calculations reusable. Reporting Tooltip format in Bar and Map visuals Select All/Clear option for Single Select dropdown Standard tooltips offer help tooltips, text, and background formatting. Dynamic tooltip will be able to create the Tooltip page and reuse it in multiple visuals The customization is so much better than the OOB tooltips Create report tooltip pages in Power BI - Power BI | Microsoft Learn Use Clear All Slicers Button. Disable Single Select, Add Clear All Slicers button, Customize the Button and Use the Button Complex Complexity Level Category Tableau Feature Power BI Equivalent Fabric Enhancements Best Practice Notes Data Model Multiple sources Create relationship using more than one column Composite Models in Power BI (DirectQuery + Import) for combining multiple sources, also connect to various cloud services. Dataflows for pre-processing. Power BI allows a relationship between 2 tables based on only one active column. OneLake Shortcuts for unified access without Azure Databricks compute cost; Microsoft Fabric Dataflows Gen2 offers multiple ways to ingest, transform, and load data efficiently. Consolidate sources into semantic models; use Direct Lake for performance; Plan and design data model to comply with star schema supported by Power BI Relationship DAX USERELATIONSHIP DAX for activating relationships in Power BI for a specific calculation Calculations LOD, window functions Data Analysis Expressions (DAX) for measures and calculated columns. Copilot to assist with complex DAX. Fabric IQ Ontology for semantic alignment. Change how visuals interact in a Power BI report. Centralize calculations in semantic layer; use variables in DAX for readability and performance. Fabric Data Agent for a conversational BI. Very Complex Complexity Level Category Tableau Feature Power BI Equivalent Fabric Enhancements Best Practice Notes Data Model Multi-source, Excel, SQL Composite Models in Power BI (DirectQuery + Import) for combining multiple sources, also connect to various cloud services. Dataflows for pre-processing. OneLake Shortcuts for unified access; Connector overview build-in support. Mirroring for real-time sync. Combine multiple sources into well-structured semantic models for consistency and optimized performance. Calculations Predictive logic Data Analysis Expressions (DAX) for measures and calculated columns. Fabric AutoML, ML models, AI Insights, Python/R, Notebook‑based ML (Spark/Scikit‑Learn), Fabric AI Functions, Fabric IQ Ontology Fabric Data Agent for a conversational BI. Centralize logic in semantic models; leverage Copilot for automation and parameter-driven workflows. Prepare for Copilot. 2. Tableau Feature Equivalents Tableau Feature Power BI Equivalent Microsoft Learn Link Calculated Fields DAX Measures DAX Documentation Parameters Field Parameters / Bookmarks Use report readers to change visuals Actions Drillthrough / Bookmarks Drillthrough Tableau Prep Power Query / Dataflows Differences between Dataflow Gen1 and Dataflow Gen2 Tableau Server Power BI Service What is Power BI? Overview of Components and Benefits Phase 4: Governance and Deployment Workspace Planning (Dev / Test / Prod Separation) A proper workspace strategy is essential for governed deployments in Fabric and Power BI. Fabric supports separate Development, Test, and Production stages using Deployment Pipelines, enabling controlled promotions of semantic models, reports, dataflows, notebooks, lakehouses, and other items. You can assign each workspace to a pipeline stage (Dev → Test → Prod) to ensure safe lifecycle management. Sensitivity Labeling (Microsoft Purview Information Protection) Sensitivity labels allow governed classification and protection of data across Fabric items. Sensitivity labels can be applied directly to Fabric items (semantic models, reports, dataflows, etc.) through the item's header flyout or the item settings. Labels from Microsoft Purview Information Protection enforce data access rules and help organizations meet compliance requirements. Endorsement & Certification (Promoted, Certified, Master Data) Endorsement improves discoverability and trust in shared organizational content. Promoted: Item creators mark content as recommended for broader use. Certified: Administrators or authorized reviewers validate content meets organizational quality standards. Master Data: Indicates authoritative single‑source‑of‑truth items such as semantic models or lakehouses. All Fabric items except dashboards can be promoted or certified; data‑containing items can be designated as Master Data. Monitoring & Capacity Planning Determine the appropriate size for fabric capacity when migrating from Tableau to PowerBI. The Fabric SKU Estimator can generate a SKU recommendation (estimate) for your capacity requirements. Ensuring performance and cost efficiency requires ongoing monitoring of your Fabric capacity. Microsoft recommends evaluating workloads using Fabric Capacity Metrics and planning SKU sizes based on real usage. Fabric uses bursting and smoothing to handle spikes while enforcing capacity limits. Monitoring helps identify high compute usage, background refreshes, and interactive workloads to optimize performance. Fabric Data Source Connections (OneLake+ Manage Connections) Microsoft Fabric is designed as an end‑to‑end analytics platform that integrates data from many different source systems into a unified environment powered by OneLake, Data Factory, Real‑Time Analytics, Dataflows , Lakehouses, Warehouses, and Mirrored Databases. The Strategic Advantage: Semantic Layer + Fabric IQ The semantic layer-first approach sets the foundation for the next evolution in enterprise analytics. Fabric IQ (announced at Ignite 2025) is Microsoft's semantic intelligence platform that auto-elevates semantic models into ontologies—structured knowledge graphs that power AI agents, Copilot experiences, and cross-domain data reasoning. What this means for your migration: Semantic models you build today become the foundation for AI-driven analytics tomorrow Data Agents can reason across multiple semantic models, answering questions that span domains Business users transition from "report consumers" to "data explorers" via natural language interfaces Conclusion: Build for the Future, Not Just for Today Migrating from Tableau to Power BI is more than a technology swap—it's an opportunity to re-architect your analytics strategy for the cloud-native, AI-powered era. The semantic layer-first approach requires upfront investment in data modeling, DAX expertise, and Fabric platform adoption. But the payoff is transformative: Consistency: Single source of truth for all business metrics Scalability: Semantic models that serve hundreds of reports and thousands of users Agility: Changes to business logic propagate instantly across the enterprise Future-readiness: Foundation for Fabric IQ, Data Agents, and AI-driven insights Start your migration with the end in mind: not just convert dashboards, but a modern, governed, AI-ready analytics platform that scales with your business. Addressing Key Migration Concerns (1) Why a semantic‑layered model approach is better than recreating Tableau dashboards A semantic‑layered modeling approach is the optimal strategy for migration and is significantly more effective than attempting to recreate Tableau dashboards exactly as they exist. By contrast, Power BI and Fabric encourage a semantic model–first architecture, where all business rules, relationships, calculations, and transformations are centralized in a governed model that serves many dashboards. The approach not only provides consistency and reuse across the enterprise but also ensures that report authors build on a single certified version of the truth. (2) How semantic-layered model approach reduces the constant redesign caused by changing data needs. A semantic‑layered modeling approach directly addresses concern about constant changes and frequent redesigns of dashboards when data evolves. With a semantic layer, changes are absorbed in the model layer—so the logic is updated once and flows automatically into all dependent reports. Combined with Fabric features like OneLake shortcuts, Direct Lake mode, and centralized governance, the semantic layer drastically reduces breakage, minimizes rework, and ensures scalability as data continues to grow and shift. Additional Resources Direct Lake in Microsoft Fabric Create Fabric Data Agents OneLake Shortcuts Write DAX queries with Copilot - DAX Prepare Your Data for AI - Power BI | Microsoft LearnMGDC for SharePoint FAQ: How to List SharePoint Webs or Subsites?
1. Introduction SharePoint uses a known object hierarchy to store your files, including, in order: Site Collection (site, SPSite) Web (site, subsite, SPWeb) List (SPList, Document Library, SPDocumentLibrary) Folder (SPFolder) List Item (documents, files, SPListItem, SPFile) If you are not familiar with all those, you can read more about them at What is in the Permissions dataset? 2.What we have in MGDC for SharePoint The SharePoint Sites dataset in Microsoft Graph Data Connect offers a list of all sites (site collections) in your tenant. The SharePoint Files will give you details about each file (document). However, there are no MGDC datasets that will give you one object per SharePoint Web (subsite) or SharePoint List (library). There is a workaround, though. If you pay attention to the schema for the SharePoint Sites and SharePoint Files datasets in MGDC, you will find that Sites includes a few interesting details about the Root Web (the main web in the site collection) and the total number of webs (subsites) in each site. Also, the Files dataset includes details about the web (subsite) and the list (library) where the file lives. You could group on these columns to expose more information about webs and lists. 3. One Web These days, if you create a modern SharePoint site, you will get one web (called the Root Web) and no obvious mechanism to create subsites or additional webs. For any sites created in the last few years, you likely have no additional webs outside the Root Web. Your SharePoint might have existed for many years and your company could have sites using old templates with multiple subsites. If you’re trying to find out if that’s the case for your tenant, the information is available in MGDC, if you are willing to dig a little deeper. 4. Finding Old Subsites The main task here is to create a custom query to join the Sites and Files datasets. You would then group that in a particular way to enumerate each subsites (Webs). You might also tag the Webs that are a Root Web, so that you can identify subsites that are not at the top of the Site hierarchy. If you pulled the Sites and Files datasets into a SQL Server or a Lakehouse in Microsoft Fabric, this is what that query could look like: SELECT Webs.SiteId, Sites.RootWeb.Title AS SiteTitle, Sites.Url AS SiteUrl, Sites.WebCount, Webs.WebId, CASE WHEN Root.RootWeb.Id IS NOT NULL THEN 1 ELSE 0 END AS IsRootWeb, Webs.FileCount, Webs.TotalSize, Webs.TotalSizeWithVersions FROM ( SELECT SiteId, WebId, COUNT(*) AS FileCount, SUM(SizeInBytes) AS TotalSize, SUM(SizeInBytesWithVersions) AS TotalSizeWithVersions FROM Files GROUP BY SiteId, WebId ) AS Webs LEFT JOIN Sites AS Sites ON Webs.SiteId = Sites.Id LEFT JOIN Sites AS Root ON Webs.SiteId = Root.Id AND Webs.WebId = Root.RootWeb.Id 4. Empty Webs, Web properties This technique will get you a nice list of Webs, but there are a few problems. First, if a specific web has no files, we won’t see that web in the Files dataset and therefore the query will fail to list that particular web. You would still see that Web counted in the WebCount property of the Sites dataset, but there would be no listing of that web in the query proposed above, since it has no files. Another problem is that there is no way to find in MGDC the details properties of the Web, like the Title. You get some of those for the Root Web in the Sites dataset, but not for the other Webs. For that, you would have to query the Web object using another source like the Graph API or PowerShell. 5. Conclusion I hope this article helped you find details about enumerating SharePoint Webs using the Microsoft Graph Data Connect. For recent deployments of SharePoint Online, you should have only one web per site and the Sites dataset (with Root Web details) should give you plenty of details. However, this might come in handy if you want to manage some of the old subsites of a slightly older SharePoint deployment. Finally, if you’re looking for a challenge, I would suggest adapting the SQL code above to enumerate all the document libraries using the Files dataset. It can be done following a similar pattern…
