This week on the Fabric Engineering Connection
How do you turn live operational signals into actionable business insights? Join us for this week's Fabric Engineering Connection: "From Live Signals to Business Meaning: Unifying Operations with RTI and Fabric IQ" with Ajeta Singhal and Chafia Aouissi. We'll explore the latest across Real-Time Intelligence and Fabric IQ, discuss Real-Time Hub and Agents in IQ, and leave plenty of time for partner questions and discussion. 🗓️ Americas & EMEA: Wednesday, September 16 | 8:00-9:00 AM PT 🗓️ APAC: Thursday, September 17 | 12:00-1:00 AM UTC Bring your questions, learn what's new, and hear directly from the team building these capabilities. Join us in the Fabric Partner Community: https://aka.ms/JoinFabricPartnerCommunity🚀 Join the FY27 Fabric Engineering Connection Kickoff Call
We're excited to kick off FY27 with the first Fabric Engineering Connection call of the year! Join Tamer Farag, Global Partner Ecosystem Lead for Microsoft Fabric & SQL, as he shares the latest strategy, priorities, and opportunities for partners in the year ahead. What we'll cover ✅ Fabric Partner Community updates and upcoming changes ✅ Analytics on Azure Specialization updates ✅ Frontier Accelerate program highlights ✅ Partner enablement plans and resources ✅ Fabric Feature Partner program updates ✅ FabCon and SQLCon insights and opportunities ✅ Open Q&A with the community 🎁 Bonus: Attend live for a chance to win Fabric swag during the call! Event Details Americas & EMEA 📅 Wednesday, September 9, 2026 🕗 8:00 AM – 9:00 AM PT APAC 📅 Thursday, September 10, 2026 🕐 1:00 PM – 2:00 PM UTC (Wednesday, September 9, 5:00 PM – 6:00 PM PT) How to Join To participate, join the Fabric Partner Community Teams channel: 🔗 https://aka.ms/JoinFabricPartnerCommunity Whether you're a long-time community member or just beginning your Microsoft Fabric journey, this is a great opportunity to hear directly from the team, get aligned on FY27 priorities, and connect with fellow partners around the world. We look forward to seeing you there!✨ FabCon + SQLCon Europe 2026: Partner Know Before You Go Guide is Now Live!
Barcelona is just around the corner, and we're excited to welcome our partner community to the first-ever co-located FabCon + SQLCon Europe experience! To help you make the most of your time onsite, the Partner Know Before You Go Guide is now available. This guide brings together everything partners need to know before arriving, including exclusive partner programming, networking opportunities, important deadlines, and event resources. What You'll Find Inside ✅ Partner Day agenda and key sessions ✅ Partner Happy Hour details ✅ Partner Elevator Pitch Challenge information ✅ Executive Meetings with Fabric & SQL Leadership Team ✅ 1:1 Meetings with the Partner Success Team ✅ AMA Session with the Fabric Partner Success Team ✅ Fabric Certification Recognition opportunities ✅ Testimonial Video nominations ✅ Fabric in the Wild Partner Photo Scavenger Hunt ✅ Cvent Event App guidance and Partner Community information Important Partner Deadlines 📅 September 13, 2026 (11:59 PM PT) Submit your Partner Elevator Pitch entry Request an Executive Meeting with Fabric Leadership Request a 1:1 Meeting with the Partner Success Team Nominate yourself or your customer for a testimonial video opportunity Event Details 📍 Centre de Convencions Internacional de Barcelona (CCIB) 📅 September 28 - October 1, 2026 🌍 Barcelona, Spain Whether you're attending Partner Day, meeting with Microsoft leaders, showcasing your expertise, or connecting with peers from around the world, we've built partner-exclusive experiences to help you learn, connect, and accelerate your Microsoft Fabric business. 📘 Download the Partner Know Before You Go Guide below. ⬇️ We can't wait to see you in Barcelona! #FabConEurope #SQLConEurope #MicrosoftFabric #MicrosoftPartners #DataAndAI #FabCon2026 #PartnerCommunity2026 FabCon + SQLCon Europe Fabric in the Wild: Photo Scavenger Hunt
📸 Fabric in the Wild: Photo Scavenger Hunt at FabCon + SQLCon Europe Heading to FabCon + SQLCon Europe in Barcelona? Get ready to explore, connect, and have some fun with the Fabric community! We're excited to launch the Fabric in the Wild: Photo Scavenger Hunt, an exclusive experience for Microsoft partners attending the event. Whether you're networking at Partner Day, taking in the keynotes, meeting fellow partners, or discovering Barcelona, you'll have the opportunity to capture your adventure and win some exclusive Fabric swag. 🏆 Win Fabric Kicks Complete the challenge by sharing photos from any five scavenger hunt moments and you'll be entered for a chance to win 1 of 3 pairs of Fabric Kicks. 📷 How It Works ✅ Complete any 5 photo challenges from the hunt ✅ Share all 5 photos in a single LinkedIn post or carousel ✅ Include: #FabConEurope #MicrosoftPartner #FabricInTheWildSweepstakes ✅ Tag @Stephanie Chimeziri That's it. You're in! The scavenger hunt is all about celebrating the people, experiences, and energy that make the Microsoft Fabric partner community special. From Partner Day and keynotes to new connections and iconic Barcelona landmarks, we can't wait to see Fabric in the wild through your lens. Drop your photos, showcase your creativity, and share your FabCon story with the community! 🌍 See you in Barcelona. For more information about FabCon + SQLCon Europe, visit aka.ms/fabconeu #MicrosoftFabric #FabConEurope #SQLConEurope #MicrosoftPartner #FabricCommunity #DataAnalytics #FabricInTheWildSweepstakes #PartnerDay #MicrosoftPartnersAzure SQL Data Sync Retirement: Migration Insights and Modern Alternatives
What started as a routine customer discussion quickly evolved into a strategic modernization conversation. A service that had quietly synchronized business-critical data for years was approaching retirement, prompting an important question: What should organizations do next? Every service retirement is an opportunity to reassess architecture, reduce technical debt, and build for the future. When a Retirement Notification Becomes a Business Conversation Recently, while working with a customer, we reviewed their Azure SQL Database architecture and discovered a critical dependency on SQL Data Sync For years, the service had reliably synchronized data across multiple databases, enabling applications, reporting workloads, and distributed business processes. Like many organizations, the customer viewed Data Sync as infrastructure that simply worked in the background. However, the discussion took a different turn when we reviewed Microsoft's retirement announcement: Azure SQL Data Sync will be retired on September 30, 2027. What initially appeared to be a migration challenge quickly became an opportunity to modernize the customer's data movement architecture and align with Microsoft's future investments in data integration, analytics, and cloud-native services. Understanding SQL Data Sync Azure SQL Data Sync was designed to synchronize selected data between Azure SQL Databases and, in some cases, between Azure and on-premises databases. Organizations have commonly used Data Sync for: Hybrid data synchronization Distributed application architectures Globally distributed applications Bi-directional data synchronization While the service has served customers well, organizations should begin evaluating alternative solutions now to ensure sufficient planning, testing, and adoption time before retirement. Because both databases were already in Azure, the discussion quickly moved toward identifying strategic alternatives. Customer's Setup This particular customer had a simple, familiar layout: one Azure SQL Database feeding another, both fully in Azure, connected by SQL Data Sync. No on-premises leg, no complicated topology — just two cloud databases that needed to stay aligned. Their requirements were equally straightforward, and honestly, the kind every team asks for: Reliable, dependable synchronization Low operational overhead — nobody wanted a new system to babysit Something with a real future, not another service on a retirement countdown Room to scale as data volumes grow An Azure-native fit, not a bolt-on third-party tool Because both databases already lived in Azure, the conversation moved quickly toward the platform's own native tooling — starting with the option that ended up being the strongest fit. Option 1: Azure Data Factory (Recommended Strategy) For customers running Azure SQL Database to Azure SQL Database synchronization, Azure Data Factory (ADF) emerged as the strongest strategic recommendation. Why ADF? Azure Data Factory provides: Fully managed Azure-native data movement Enterprise-grade monitoring Flexible orchestration Scalability from development through production environments Long-term Microsoft investment and support The migration pattern we typically recommend looks like: Phase1 :Initial Full Load Before anything can stay in sync, both sides need to start from the same place. Phase 1 is a one-time bulk copy: Azure Data Factory reads everything from the source database and writes it into the target, establishing a clean baseline. Phase2 :Incremental Synchronization Once both databases match, you don't need to keep copying everything — just what's changed. This is where Change Tracking (CT) or Change Data Capture (CDC) comes in: SQL Server-native features that flag which rows were inserted, updated, or deleted since the last run. ADF's incremental pipeline reads only those deltas and applies them downstream, on whatever schedule the business needs — minutes, hours, or daily. Phase 3: Scheduling and Monitoring Once both phases are live, ADF takes over the operational side: scheduling pipeline runs, monitoring their health, retrying failures automatically, and alerting your team when something needs attention. That's a level of visibility SQL Data Sync's built-in sync groups never really offered ADF handles: Scheduling Pipeline execution Monitoring Retry mechanisms Alerting This model often delivers greater visibility and operational control than traditional SQL Data Sync implementations. When You Don't Need Synchronization — You Need a Copy Partway through the engagement, one question reframed the whole discussion: do we actually need two-way synchronization, or do we just need a readable copy of the database somewhere else? That distinction matters more than it sounds, and it points to three other options worth knowing. Option 1: Active Geo-Replication If the goal is disaster recovery, serving read traffic closer to users, or keeping the business running through a regional outage, Active Geo-Replication is usually a better fit than rebuilding sync logic from scratch. It gives you a continuously updated, readable secondary — not a bi-directional sync target. Best fit Disaster recovery scenarios Read-intensive applications Global user distribution Secondary readable databases Less ideal for Complex data transformations Bi-directional updates Option 2: Database Copies and Read Replicas Some organizations do not require continuous synchronization at all. In those cases: Read Replicas Database Copy Good Use Cases Reporting databases Analytics environments Refreshable staging systems Read-only workloads This approach significantly reduces architectural complexity while still meeting many business requirements. Option 3: Microsoft Fabric Mirrored Databases As Microsoft Fabric adoption grows, another interesting alternative is Fabric Mirrored Databases. This option is particularly attractive for organizations already investing in: Microsoft Fabric OneLake Real-time analytics AI and data platform modernization Benefits include: Near real-time data availability Simplified analytics architecture Integration with Fabric workloads Reduced data silos For customers modernizing both operational and analytical platforms, this can be an excellent opportunity to rethink data architecture beyond simple synchronization. Option 4: Azure Functions for Event-Driven Synchronization Not every customer requires a large orchestration platform. For lightweight or application-specific synchronization logic, Azure Functions may offer a more agile approach. Example Use Cases Event-driven updates Custom business rules Microservices architectures Low-volume synchronization requirements The tradeoff is that customers assume additional development and operational responsibilities Lessons Learned from the Customer Engagement This engagement reinforced several important lessons: Don't Wait Until 2027 Although retirement is over a year away, large organizations often require significant planning, testing, governance approvals, and deployment cycles. Starting early reduces risk. There Is No Universal Replacement The right solution depends on: Latency requirements Read versus write workloads Synchronization direction Operational complexity DR requirements Budget limitation Different use cases require different migration paths. 3. Migration Is an Opportunity Rather than simply replacing SQL Data Sync, organizations should evaluate: Data architecture modernization Observability improvements Operational simplification Fabric adoption opportunities Long-term cloud strategy Final Recommendations For most customers currently using Azure SQL Database → Azure SQL Database synchronization: Requirement Recommended Solution Ongoing synchronization Azure Data Factory + CDC/Change Tracking Read-only replica Active Geo-Replication Simple duplication Database Copy or Read Replica Analytics modernization Fabric Mirrored Databases Event-driven custom logic Azure Functions In my customer's use case, Azure Data Factory with incremental changes (CDC) emerged as the preferred strategic path because it was Azure-native, scalable, supported long-term, and aligned with Microsoft's future direction for data movement and integration. Closing Thoughts Technology retirements often create urgency, but they also create opportunity. retirement of SQL Data Sync is not merely a migration project. It is an opportunity to reassess data movement architecture, improve resiliency, reduce technical debt, and embrace modern Azure-native services. If your organization is currently using SQL Data Sync, now is the right time to inventory your sync groups, identify dependencies, and begin evaluating alternative architectures before September 30, 2027. References SQL Data Sync Retirement Migration Guide What is SQL Data Sync for Azure SQL Database? SQL Data Sync retirement: Migrate to alternative solutionsFabric Data Agents can choose Query Language based on Context
What if you could combine decades of historical analysis with live, real-time data in one seamless AI chat experience? Traditionally, analyzing large volumes of past data (Analytics / Business Intelligence) has been a separate architecture, or has at least required separate toolsets, from monitoring what’s happening right now (real-time, IoT, etc). With Fabric Data Agents, analytics for large volumes of historical data can be accessible with real-time data via a single AI query endpoint. Analytics can be used to gain understanding from historical data, and findings can be put into action for real-time scenarios, all through a single interface for the end users. Figure 1.0 – Fabric Data Agent can use different query languages for optimal performance with a lambda-style RTI architecture on Fabric Many of the demos I’ve seen for Fabric Data Agents will highlight the capability to connect to different types of queries and sources via a single endpoint such as Lakehouses (SQL), Warehouses (SQL), Semantic Models (DAX), Eventhouses (Kusto), and Ontologies. What I have not seen frequently discussed is adding different query engines on the same data for the purpose of optimizing query performance based upon the context of the query and the latency of the data, as per Figure 1.0 above. To be clear, I am not advocating duplication of data for the purpose of query performance. Rather, this architecture would enable Fabric Data Agents to choose the best query language for the context of the query. Hence the acronym I created for this blog article “NoDAX” which stands for “Not Only DAX.” NoSQL “Not Only SQL” is a real term referring to non-relational database systems designed for flexible schemas, horizontal scaling, and high-performance access to large, distributed datasets. NoDAX exists only within this blog post, and represents the availability of multiple query languages within a single Fabric Data Agent. The best query language can be used based on the context of the question and query. Not only DAX but also SQL, Kusto, and Ontologies can be queried to generate the most efficient contextual query. Figure 1.1 – Explaining the NoDAX acronym created for this article Why does this use case matter? The ability to query both the past and the present in one interface isn’t just a novel technical capability. Many analytic solutions can benefit from a pattern of [analysis + action]. Analytics by itself is great at understanding the past. We can look at years of historical data to figure out patterns, drivers of performance, and what went right or wrong. Historical context is incredibly valuable, but understanding the past doesn’t improve outcomes in the future. The real value comes when we connect findings from the past to decisions we make right now and in the future. If you discover historical data patterns that lead to an inventory shortage, a fraud event, or a patient risk score increasing, you can apply that insight to the latest operational data and act on ongoing workflows. Analytics and action have to reinforce each other. Analytics without action doesn’t create value. But action without understanding can easily make things worse. I once had a veteran analytics manager say to me “Without carefully considered and vetted KPIs you are flying blind. But be careful, because a KPI without proper context and understanding will quickly become a blunt object that an empty suit uses to whack somebody over the head.” Figure 1.2 – The purpose of this architecture is to learn from the past to improve the present and future Different Query Languages without data duplication A Fabric real-time architecture can be part of a design pattern that is similar to if not a version of a Lambda architecture. With a Lambda architecture, hot path data is available for real-time alerting and analytics while cold path data is stored for deep and complete historical analytics and data science. Figure 1.3 – NoDAX architecture can query a lambda-style architecture via multiple query endpoints Per the diagram above, real-time data is available in Fabric ASAP and cycled through an Eventhouse. Historical data can either be batched into a Fabric Lakehouse / Warehouse or copied over from the Eventhouse. A Fabric Data Agent can then generate Kusto queries against the Eventhouse, SQL queries against the Lakehouse / Warehouse, or DAX queries against the Warehouse / Lakehouse via the Direct Lake Semantic Model. DAX is often the best query language for data having deep history with complext analytic logic. SQL can be the best query language for retrieving historical row-level information from a robust relational database. Kusto can be used to query what’s happening right now via a real-time Eventhouse. Lambda architectures have been around for years, so why is this architecture a new option? Past lambda architectures would have hot path data available in a streaming toolset such as Azure Eventhub, and then store historical cold path data in a tool such as Azure Data Lake. Hot path alerting and reporting was usually disparate from historical cold path analytics. Per the diagram 3.4 below, with Microsoft Fabric, you can now: implement both the hot and cold path in a single Fabric environment (Eventstream, Eventhouse, Lakehouse / Warehouse). Ontologies are also an option. query the hot and cold path data via a single agentic endpoint using a Fabric Data agent Query either the hot or cold path using the optimal query language for the context of the question (DAX, SQL, Kusto, Ontologies) Figure 1.4 – Fabric not only unifies components of lambda-style architecture, but Data Agent also unites the query endpoints for AI unification Example use case for Healthcare Here’s an example of a Healthcare use case: A user might ask a question “Show me the percentage of patients who had their pain scores checked every hour for gall bladder removals on floor 5 over the last 3 years.” This will ideally filter three years worth of data for patients with specific procedure codes, filtered for specific rooms, and calculate the pain score check compliance for those visits. This query is ideal for the DAX language with a Semantic Model. The user might then want to see details for a specific time period, and ask “Show me the pain score results for patients who had their gall bladder removed on July 3 2024 on floor 5.” A SQL query might be the best option here against the Fabric Warehouse or Lakehouse, since SQL is better than DAX at retrieving row-level information. Then the user might want to know what is happening today. “Show me the pain score checks for inpatients right now who had their gall bladder removed on floor 5.” The Kusto language can retrieve the information that streams into a Fabric Eventhouse via an Eventstream. Based on the findings, the user may take an action. With the example above, a user was able to query deep history with analytic logic, retrieve historical row-level information from a robust relational database, and then view what’s happening right now for those patients. Action can then be taken in the here and now. Here are some additional use cases for Finance, Supply Chain, and Manufacturing in addition to Healthcare: Figure 1.5 – Industry use cases for Fabric Data Agent with multiple endpoints Video Summary Below is my video summary and demo of the Fabric Data Agent NoDAX architecture: Configure Fabric Data Agent for NoDAX query patterns When more than one source is added to a Fabric Data Agent, by default the source used for a specific query will be chosen based on interpreting available metadata. The Data Agents have a field called "Agent instructions" which can be used to provide detailed instructions about choosing the right source for the right question. Here’s a screenshot of the Agent instructions: Figure 1.6 – Agent instructions will guide the Fabric Data Agent to the best query endpoint I would recommend extensive unit testing and iterative improvements to the Agent instructions based upon your own data and use cases. Here’s a few examples that worked for my initial testing. I would recommend much more robust and carefully designed prompts for a production solution, but this is a baseline of an approach I found to work based on the demo in the video above: The KQL database named SeattleFireEventHouse is a live stream of 911 calls to the fire department in the city of Seattle. Whenever someone asks for “most recent” or “newest” or “latest” use SeattleFireEventHouse The lakehouse SeattleFireLakehouse should be queried with a SQL statement when someone asks for a list of incidents before the year 2026. Use SQL to retrieve row level requests for historical data. The semantic model SeattleFireSemantic Model should be queried when questions ask about historical analytic trends such as call volume averages, Year over year changes, and queries that aggregate data for analytic queries.
Identifying removed Fabric Assets
Is there any way for a data steward to identify a deleted physical asset ingested from Fabric into Purview. The original asset has been deleted in Fabric and another has been created with the exact same name, both assets now show in the catalogue and there doesn't seem to be a way to distinguish clearly which is the deleted and which is the current.
Struggling with running DQ Scans (Long queuing and Retry Count Error Issues)
Hi everyone, I have been exploring Microsoft Purview Data Quality quite extensively. At this point, I have configured more than 4,000 data quality rules across more than 10 Microsoft Fabric capacities, each with a minimum capacity of F16. Fabric is the source for all assets registered in Purview. I have identified several issues with the product, but the two that are currently impacting me the most are the following: DQ scans failing with a generic error“Max Retry Count Reached. Ending Workflow. Current Task HandleError”The challenge is that the error message does not identify which rule is causing the failure. As a result, I have to troubleshoot manually by disabling groups of rules, rerunning the scans, and repeating the process until I find the problematic rule. This trial-and-error approach is very time-consuming, especially at this scale. This seems to be caused by issues in some of the DQ rules, even though all rules are marked as “Good to go” in Purview. When running Data Quality scans, I often receive the following error: DQ scans remain queued for a long timeI am not sure why this happens or what resource, orchestration, or scheduling constraint is causing the delay. Whenever I run these DQ scans, they remain in a Queued state for at least 10 minutes, even when there is nothing running on the Fabric capacities. Has anyone experienced similar behavior with Purview Data Quality at this scale? Specifically, I would appreciate any guidance on: How to identify which DQ rule is causing a scan failure Why scans remain queued even when Fabric capacity appears to be idle Whether there are known limitations or best practices for running thousands of DQ rules in Purview Thank you.🎉APAC FY26 Fabric Partner Community Year‑End Celebration
As we prepare to wrap up FY26, we’re closing the year the same way we built it — together. This year‑end celebration will be held as part of the final Fabric Engineering Connection calls of FY26, giving us space to pause, look back on what we built together, and celebrate the partners who make this community what it is. 🌎Americas & EMEA Wednesday, June 24 | 8:00–9:00 AM PT 🌍APAC Thursday, June 25 | 1:00–2:00 AM UTC / Wednesday, June 24 | 5:00–6:00 PM PT) ✨ What to expect: A look back at the moments that defined FY26 along with partner updates to take you into FY27 Fun & games — including a Mad Libs–style community story built live by partners A community toast and a few surprises along the way 👉 Important: This call is open to members of the Fabric Partner Community on Microsoft Teams. If you’re not already a member, you can join here: https://aka.ms/JoinFabricPartnerCommunity This isn’t just a year‑end recap. It’s a thank‑you to the partners who showed up, shared openly, asked great questions, and helped each other grow real Microsoft Fabric practices. Mark your calendars. We can't wait to celebrate with you! 🥳 🥂
