🚀 Join the FY27 Fabric Engineering Connection Kickoff Call
We're excited to kick off FY27 with the first Fabric Engineering Connection call of the year! Join Tamer Farag, Global Partner Ecosystem Lead for Microsoft Fabric & SQL, as he shares the latest strategy, priorities, and opportunities for partners in the year ahead. What we'll cover ✅ Fabric Partner Community updates and upcoming changes ✅ Analytics on Azure Specialization updates ✅ Frontier Accelerate program highlights ✅ Partner enablement plans and resources ✅ Fabric Feature Partner program updates ✅ FabCon and SQLCon insights and opportunities ✅ Open Q&A with the community 🎁 Bonus: Attend live for a chance to win Fabric swag during the call! Event Details Americas & EMEA 📅 Wednesday, September 9, 2026 🕗 8:00 AM – 9:00 AM PT APAC 📅 Thursday, September 10, 2026 🕐 1:00 PM – 2:00 PM UTC (Wednesday, September 9, 5:00 PM – 6:00 PM PT) How to Join To participate, join the Fabric Partner Community Teams channel: 🔗 https://aka.ms/JoinFabricPartnerCommunity Whether you're a long-time community member or just beginning your Microsoft Fabric journey, this is a great opportunity to hear directly from the team, get aligned on FY27 priorities, and connect with fellow partners around the world. We look forward to seeing you there!Azure SQL Data Sync Retirement: Migration Insights and Modern Alternatives
What started as a routine customer discussion quickly evolved into a strategic modernization conversation. A service that had quietly synchronized business-critical data for years was approaching retirement, prompting an important question: What should organizations do next? Every service retirement is an opportunity to reassess architecture, reduce technical debt, and build for the future. When a Retirement Notification Becomes a Business Conversation Recently, while working with a customer, we reviewed their Azure SQL Database architecture and discovered a critical dependency on SQL Data Sync For years, the service had reliably synchronized data across multiple databases, enabling applications, reporting workloads, and distributed business processes. Like many organizations, the customer viewed Data Sync as infrastructure that simply worked in the background. However, the discussion took a different turn when we reviewed Microsoft's retirement announcement: Azure SQL Data Sync will be retired on September 30, 2027. What initially appeared to be a migration challenge quickly became an opportunity to modernize the customer's data movement architecture and align with Microsoft's future investments in data integration, analytics, and cloud-native services. Understanding SQL Data Sync Azure SQL Data Sync was designed to synchronize selected data between Azure SQL Databases and, in some cases, between Azure and on-premises databases. Organizations have commonly used Data Sync for: Hybrid data synchronization Distributed application architectures Globally distributed applications Bi-directional data synchronization While the service has served customers well, organizations should begin evaluating alternative solutions now to ensure sufficient planning, testing, and adoption time before retirement. Because both databases were already in Azure, the discussion quickly moved toward identifying strategic alternatives. Customer's Setup This particular customer had a simple, familiar layout: one Azure SQL Database feeding another, both fully in Azure, connected by SQL Data Sync. No on-premises leg, no complicated topology — just two cloud databases that needed to stay aligned. Their requirements were equally straightforward, and honestly, the kind every team asks for: Reliable, dependable synchronization Low operational overhead — nobody wanted a new system to babysit Something with a real future, not another service on a retirement countdown Room to scale as data volumes grow An Azure-native fit, not a bolt-on third-party tool Because both databases already lived in Azure, the conversation moved quickly toward the platform's own native tooling — starting with the option that ended up being the strongest fit. Option 1: Azure Data Factory (Recommended Strategy) For customers running Azure SQL Database to Azure SQL Database synchronization, Azure Data Factory (ADF) emerged as the strongest strategic recommendation. Why ADF? Azure Data Factory provides: Fully managed Azure-native data movement Enterprise-grade monitoring Flexible orchestration Scalability from development through production environments Long-term Microsoft investment and support The migration pattern we typically recommend looks like: Phase1 :Initial Full Load Before anything can stay in sync, both sides need to start from the same place. Phase 1 is a one-time bulk copy: Azure Data Factory reads everything from the source database and writes it into the target, establishing a clean baseline. Phase2 :Incremental Synchronization Once both databases match, you don't need to keep copying everything — just what's changed. This is where Change Tracking (CT) or Change Data Capture (CDC) comes in: SQL Server-native features that flag which rows were inserted, updated, or deleted since the last run. ADF's incremental pipeline reads only those deltas and applies them downstream, on whatever schedule the business needs — minutes, hours, or daily. Phase 3: Scheduling and Monitoring Once both phases are live, ADF takes over the operational side: scheduling pipeline runs, monitoring their health, retrying failures automatically, and alerting your team when something needs attention. That's a level of visibility SQL Data Sync's built-in sync groups never really offered ADF handles: Scheduling Pipeline execution Monitoring Retry mechanisms Alerting This model often delivers greater visibility and operational control than traditional SQL Data Sync implementations. When You Don't Need Synchronization — You Need a Copy Partway through the engagement, one question reframed the whole discussion: do we actually need two-way synchronization, or do we just need a readable copy of the database somewhere else? That distinction matters more than it sounds, and it points to three other options worth knowing. Option 1: Active Geo-Replication If the goal is disaster recovery, serving read traffic closer to users, or keeping the business running through a regional outage, Active Geo-Replication is usually a better fit than rebuilding sync logic from scratch. It gives you a continuously updated, readable secondary — not a bi-directional sync target. Best fit Disaster recovery scenarios Read-intensive applications Global user distribution Secondary readable databases Less ideal for Complex data transformations Bi-directional updates Option 2: Database Copies and Read Replicas Some organizations do not require continuous synchronization at all. In those cases: Read Replicas Database Copy Good Use Cases Reporting databases Analytics environments Refreshable staging systems Read-only workloads This approach significantly reduces architectural complexity while still meeting many business requirements. Option 3: Microsoft Fabric Mirrored Databases As Microsoft Fabric adoption grows, another interesting alternative is Fabric Mirrored Databases. This option is particularly attractive for organizations already investing in: Microsoft Fabric OneLake Real-time analytics AI and data platform modernization Benefits include: Near real-time data availability Simplified analytics architecture Integration with Fabric workloads Reduced data silos For customers modernizing both operational and analytical platforms, this can be an excellent opportunity to rethink data architecture beyond simple synchronization. Option 4: Azure Functions for Event-Driven Synchronization Not every customer requires a large orchestration platform. For lightweight or application-specific synchronization logic, Azure Functions may offer a more agile approach. Example Use Cases Event-driven updates Custom business rules Microservices architectures Low-volume synchronization requirements The tradeoff is that customers assume additional development and operational responsibilities Lessons Learned from the Customer Engagement This engagement reinforced several important lessons: Don't Wait Until 2027 Although retirement is over a year away, large organizations often require significant planning, testing, governance approvals, and deployment cycles. Starting early reduces risk. There Is No Universal Replacement The right solution depends on: Latency requirements Read versus write workloads Synchronization direction Operational complexity DR requirements Budget limitation Different use cases require different migration paths. 3. Migration Is an Opportunity Rather than simply replacing SQL Data Sync, organizations should evaluate: Data architecture modernization Observability improvements Operational simplification Fabric adoption opportunities Long-term cloud strategy Final Recommendations For most customers currently using Azure SQL Database → Azure SQL Database synchronization: Requirement Recommended Solution Ongoing synchronization Azure Data Factory + CDC/Change Tracking Read-only replica Active Geo-Replication Simple duplication Database Copy or Read Replica Analytics modernization Fabric Mirrored Databases Event-driven custom logic Azure Functions In my customer's use case, Azure Data Factory with incremental changes (CDC) emerged as the preferred strategic path because it was Azure-native, scalable, supported long-term, and aligned with Microsoft's future direction for data movement and integration. Closing Thoughts Technology retirements often create urgency, but they also create opportunity. retirement of SQL Data Sync is not merely a migration project. It is an opportunity to reassess data movement architecture, improve resiliency, reduce technical debt, and embrace modern Azure-native services. If your organization is currently using SQL Data Sync, now is the right time to inventory your sync groups, identify dependencies, and begin evaluating alternative architectures before September 30, 2027. References SQL Data Sync Retirement Migration Guide What is SQL Data Sync for Azure SQL Database? SQL Data Sync retirement: Migrate to alternative solutions
In a Day (xIAD) Partner Events Program - Train the Trainer Events (DIAD/FAIAD/RTIAD/CDIAD/SQLAIAD)
We invite you to attend an upcoming Train the Trainer session for Microsoft Partners to learn more about the Microsoft In a Day (XIAD) Partner Events Programand how to lead workshops that empower customers to use and adopt Microsoft products. Our Train the Trainer events are designed to provide you with the knowledge and tools necessary to deliver successful Microsoft In a Day (XIAD) sessions. *Please note, participation is restricted to individuals representing a Microsoft Partner organization. You must register using your corporate email address that is associated with your Partner ID. Personal emails will not be approved. ✨ Why Attend? Expert Guidance: Learn from experienced trainers and get your questions answered. Comprehensive Resources: Access all the content and support you need to succeed. Learn more at https://aka.ms/attendTTT 📅 Upcoming Events: Dashboard in a Day This is a one-day, hands-on workshop for business analysts, covering the breadth of Power BI capabilities. September 10 - Americas - Central Time September 18 - APAC - SGT September 25 - EMEA - CEST Fabric Analyst in a Day This is an intermediate-level training designed for Power BI Data Analysts who have at least one year of experience on Power BI but are new to Microsoft Fabric. September 10 - Americas - Central Time September 18 - APAC - SGT September 25 - EMEA - CEST Real-Time Intelligence in a Day This is an intermediate-level training designed for Power BI developers looking to extract insights and visualize streaming and time sensitive data. September 10 - Americas - Central Time September 18 - APAC - SGT September 25 - EMEA - CEST Chat with your Data in a Day This is an intermediate-level training designed for Power BI data analysts and developers to help get their models chat-ready and unlock instant insights using natural language. September 10 - Americas - Central Time September 18 - APAC - SGT September 25 - EMA - CEST New Offer Type: SQL AI App in a Day This is an intermediate-level, one‑day, hands‑on workshop showing SQL Developers and DBAs how to build the database backbone for modern AI apps using Azure SQL. Coming soon! Partner with us! Are you a Microsoft Partner interested in the opportunity to join the program and deliver Microsoft In a Day (XIAD) events? 🔍 Learn more about the program and review partner eligibility criteria: https://aka.ms/xiadpartneropportunity. 📧 Contact the XIAD Program team: xiadevents@microsoft.com 📤 Submit requests to deliver events: https://aka.ms/xIAD/PartnerEvents✨ FabCon + SQLCon Europe 2026: Partner Know Before You Go Guide is Now Live!
Barcelona is just around the corner, and we're excited to welcome our partner community to the first-ever co-located FabCon + SQLCon Europe experience! To help you make the most of your time onsite, the Partner Know Before You Go Guide is now available. This guide brings together everything partners need to know before arriving, including exclusive partner programming, networking opportunities, important deadlines, and event resources. What You'll Find Inside ✅ Partner Day agenda and key sessions ✅ Partner Happy Hour details ✅ Partner Elevator Pitch Challenge information ✅ Executive Meetings with Fabric & SQL Leadership Team ✅ 1:1 Meetings with the Partner Success Team ✅ AMA Session with the Fabric Partner Success Team ✅ Fabric Certification Recognition opportunities ✅ Testimonial Video nominations ✅ Fabric in the Wild Partner Photo Scavenger Hunt ✅ Cvent Event App guidance and Partner Community information Important Partner Deadlines 📅 September 13, 2026 (11:59 PM PT) Submit your Partner Elevator Pitch entry Request an Executive Meeting with Fabric Leadership Request a 1:1 Meeting with the Partner Success Team Nominate yourself or your customer for a testimonial video opportunity Event Details 📍 Centre de Convencions Internacional de Barcelona (CCIB) 📅 September 28 - October 1, 2026 🌍 Barcelona, Spain Whether you're attending Partner Day, meeting with Microsoft leaders, showcasing your expertise, or connecting with peers from around the world, we've built partner-exclusive experiences to help you learn, connect, and accelerate your Microsoft Fabric business. 📘 Download the Partner Know Before You Go Guide below. ⬇️ We can't wait to see you in Barcelona! #FabConEurope #SQLConEurope #MicrosoftFabric #MicrosoftPartners #DataAndAI #FabCon2026 #PartnerCommunity2026 FabCon + SQLCon Europe Fabric in the Wild: Photo Scavenger Hunt
📸 Fabric in the Wild: Photo Scavenger Hunt at FabCon + SQLCon Europe Heading to FabCon + SQLCon Europe in Barcelona? Get ready to explore, connect, and have some fun with the Fabric community! We're excited to launch the Fabric in the Wild: Photo Scavenger Hunt, an exclusive experience for Microsoft partners attending the event. Whether you're networking at Partner Day, taking in the keynotes, meeting fellow partners, or discovering Barcelona, you'll have the opportunity to capture your adventure and win some exclusive Fabric swag. 🏆 Win Fabric Kicks Complete the challenge by sharing photos from any five scavenger hunt moments and you'll be entered for a chance to win 1 of 3 pairs of Fabric Kicks. 📷 How It Works ✅ Complete any 5 photo challenges from the hunt ✅ Share all 5 photos in a single LinkedIn post or carousel ✅ Include: #FabConEurope #MicrosoftPartner #FabricInTheWildSweepstakes ✅ Tag @Stephanie Chimeziri That's it. You're in! The scavenger hunt is all about celebrating the people, experiences, and energy that make the Microsoft Fabric partner community special. From Partner Day and keynotes to new connections and iconic Barcelona landmarks, we can't wait to see Fabric in the wild through your lens. Drop your photos, showcase your creativity, and share your FabCon story with the community! 🌍 See you in Barcelona. For more information about FabCon + SQLCon Europe, visit aka.ms/fabconeu #MicrosoftFabric #FabConEurope #SQLConEurope #MicrosoftPartner #FabricCommunity #DataAnalytics #FabricInTheWildSweepstakes #PartnerDay #MicrosoftPartnersLimited-Time Free DP-600 and DP-700 Partner Certification Offer | Terms and Conditions
Offer Terms and Conditions The DP-600 and DP-700 partner exam voucher offer is available to eligible participants as explained in the Rules and Restrictions below. Eligible participants can receive one free Microsoft Certification exam voucher that they can apply to the following exams only: DP-600 and DP-700. Rules and Restrictions: This offer expires June 30, 2025. This exam offer may be redeemed to take one (1) qualifying Microsoft Certification exam (as outlined below), delivered at an authorized Pearson Vue testing center or through a Pearson Vue online proctoring site by the expiration date on the voucher. This exam offer is exam-specific and only redeemable for the following exams: DP-600 - Fabric Analytics Engineer Associate DP-700 - Fabric Data Engineer Associate To be eligible to receive this exam offer you must: Be an employee of a Microsoft partner organization. Provide your partner org email address. Must be a member of the Fabric Partner Community Team channel. If you are not a member, you can join by completing the form at https://aka.ms/JoinFabricPartnerCommunity. You must submit your request via this Form. This offer is subject to availability and can be ended at any time. The Form submissions will be reviewed monthly and exam vouchers will be distributed the first Monday of each month, pending eligibility and availability. Exam vouchers, redemption details, and other communications related to this Offer will be distributed via email. You must use the exam voucher to book the exam by the last day of the month in which you receive the voucher. The exam must be taken before the expiration date on the voucher. This exam offer may only be redeemed once. This exam offer may not be redeemed or exchanged for cash, credit or refund. This exam offer is nontransferable and is void if you alter, revise or transfer it in any way. Any sale or transfer of this voucher code is expressly prohibited and constitutes fraud. Microsoft, Pearson VUE, and the VUE test centers are not responsible for lost or stolen voucher codes. Individuals that (a) fail to show up for their scheduled exam appointment, or (ii) cancel or reschedule their appointment to take an exam seventy-two hours or less from the scheduled time may forfeit their voucher code (voucher code cannot be used again). The fair market value of this exam voucher is $165 (USD). Taxes, if any, are the sole responsibility of the recipient. This offer is subject to availability and can be ended at any time. Prior to redeeming your free certification exam, government employees must check with their employers to ensure their participation is permitted, and in accordance with applicable with ethics and compliance policies and laws. This exam offer is not valid and void to applicants residing in Belarus, Cuba, Iran, North Korea, Russia, Syria, Region of Crimea, and where prohibited. For additional issues and questions, please contact fabricpartnersteam@microsoft.com.
Identifying removed Fabric Assets
Is there any way for a data steward to identify a deleted physical asset ingested from Fabric into Purview. The original asset has been deleted in Fabric and another has been created with the exact same name, both assets now show in the catalogue and there doesn't seem to be a way to distinguish clearly which is the deleted and which is the current.
Struggling with running DQ Scans (Long queuing and Retry Count Error Issues)
Hi everyone, I have been exploring Microsoft Purview Data Quality quite extensively. At this point, I have configured more than 4,000 data quality rules across more than 10 Microsoft Fabric capacities, each with a minimum capacity of F16. Fabric is the source for all assets registered in Purview. I have identified several issues with the product, but the two that are currently impacting me the most are the following: DQ scans failing with a generic error“Max Retry Count Reached. Ending Workflow. Current Task HandleError”The challenge is that the error message does not identify which rule is causing the failure. As a result, I have to troubleshoot manually by disabling groups of rules, rerunning the scans, and repeating the process until I find the problematic rule. This trial-and-error approach is very time-consuming, especially at this scale. This seems to be caused by issues in some of the DQ rules, even though all rules are marked as “Good to go” in Purview. When running Data Quality scans, I often receive the following error: DQ scans remain queued for a long timeI am not sure why this happens or what resource, orchestration, or scheduling constraint is causing the delay. Whenever I run these DQ scans, they remain in a Queued state for at least 10 minutes, even when there is nothing running on the Fabric capacities. Has anyone experienced similar behavior with Purview Data Quality at this scale? Specifically, I would appreciate any guidance on: How to identify which DQ rule is causing a scan failure Why scans remain queued even when Fabric capacity appears to be idle Whether there are known limitations or best practices for running thousands of DQ rules in Purview Thank you.Fabric Data Agents can choose Query Language based on Context
What if you could combine decades of historical analysis with live, real-time data in one seamless AI chat experience? Traditionally, analyzing large volumes of past data (Analytics / Business Intelligence) has been a separate architecture, or has at least required separate toolsets, from monitoring what’s happening right now (real-time, IoT, etc). With Fabric Data Agents, analytics for large volumes of historical data can be accessible with real-time data via a single AI query endpoint. Analytics can be used to gain understanding from historical data, and findings can be put into action for real-time scenarios, all through a single interface for the end users. Figure 1.0 – Fabric Data Agent can use different query languages for optimal performance with a lambda-style RTI architecture on Fabric Many of the demos I’ve seen for Fabric Data Agents will highlight the capability to connect to different types of queries and sources via a single endpoint such as Lakehouses (SQL), Warehouses (SQL), Semantic Models (DAX), Eventhouses (Kusto), and Ontologies. What I have not seen frequently discussed is adding different query engines on the same data for the purpose of optimizing query performance based upon the context of the query and the latency of the data, as per Figure 1.0 above. To be clear, I am not advocating duplication of data for the purpose of query performance. Rather, this architecture would enable Fabric Data Agents to choose the best query language for the context of the query. Hence the acronym I created for this blog article “NoDAX” which stands for “Not Only DAX.” NoSQL “Not Only SQL” is a real term referring to non-relational database systems designed for flexible schemas, horizontal scaling, and high-performance access to large, distributed datasets. NoDAX exists only within this blog post, and represents the availability of multiple query languages within a single Fabric Data Agent. The best query language can be used based on the context of the question and query. Not only DAX but also SQL, Kusto, and Ontologies can be queried to generate the most efficient contextual query. Figure 1.1 – Explaining the NoDAX acronym created for this article Why does this use case matter? The ability to query both the past and the present in one interface isn’t just a novel technical capability. Many analytic solutions can benefit from a pattern of [analysis + action]. Analytics by itself is great at understanding the past. We can look at years of historical data to figure out patterns, drivers of performance, and what went right or wrong. Historical context is incredibly valuable, but understanding the past doesn’t improve outcomes in the future. The real value comes when we connect findings from the past to decisions we make right now and in the future. If you discover historical data patterns that lead to an inventory shortage, a fraud event, or a patient risk score increasing, you can apply that insight to the latest operational data and act on ongoing workflows. Analytics and action have to reinforce each other. Analytics without action doesn’t create value. But action without understanding can easily make things worse. I once had a veteran analytics manager say to me “Without carefully considered and vetted KPIs you are flying blind. But be careful, because a KPI without proper context and understanding will quickly become a blunt object that an empty suit uses to whack somebody over the head.” Figure 1.2 – The purpose of this architecture is to learn from the past to improve the present and future Different Query Languages without data duplication A Fabric real-time architecture can be part of a design pattern that is similar to if not a version of a Lambda architecture. With a Lambda architecture, hot path data is available for real-time alerting and analytics while cold path data is stored for deep and complete historical analytics and data science. Figure 1.3 – NoDAX architecture can query a lambda-style architecture via multiple query endpoints Per the diagram above, real-time data is available in Fabric ASAP and cycled through an Eventhouse. Historical data can either be batched into a Fabric Lakehouse / Warehouse or copied over from the Eventhouse. A Fabric Data Agent can then generate Kusto queries against the Eventhouse, SQL queries against the Lakehouse / Warehouse, or DAX queries against the Warehouse / Lakehouse via the Direct Lake Semantic Model. DAX is often the best query language for data having deep history with complext analytic logic. SQL can be the best query language for retrieving historical row-level information from a robust relational database. Kusto can be used to query what’s happening right now via a real-time Eventhouse. Lambda architectures have been around for years, so why is this architecture a new option? Past lambda architectures would have hot path data available in a streaming toolset such as Azure Eventhub, and then store historical cold path data in a tool such as Azure Data Lake. Hot path alerting and reporting was usually disparate from historical cold path analytics. Per the diagram 3.4 below, with Microsoft Fabric, you can now: implement both the hot and cold path in a single Fabric environment (Eventstream, Eventhouse, Lakehouse / Warehouse). Ontologies are also an option. query the hot and cold path data via a single agentic endpoint using a Fabric Data agent Query either the hot or cold path using the optimal query language for the context of the question (DAX, SQL, Kusto, Ontologies) Figure 1.4 – Fabric not only unifies components of lambda-style architecture, but Data Agent also unites the query endpoints for AI unification Example use case for Healthcare Here’s an example of a Healthcare use case: A user might ask a question “Show me the percentage of patients who had their pain scores checked every hour for gall bladder removals on floor 5 over the last 3 years.” This will ideally filter three years worth of data for patients with specific procedure codes, filtered for specific rooms, and calculate the pain score check compliance for those visits. This query is ideal for the DAX language with a Semantic Model. The user might then want to see details for a specific time period, and ask “Show me the pain score results for patients who had their gall bladder removed on July 3 2024 on floor 5.” A SQL query might be the best option here against the Fabric Warehouse or Lakehouse, since SQL is better than DAX at retrieving row-level information. Then the user might want to know what is happening today. “Show me the pain score checks for inpatients right now who had their gall bladder removed on floor 5.” The Kusto language can retrieve the information that streams into a Fabric Eventhouse via an Eventstream. Based on the findings, the user may take an action. With the example above, a user was able to query deep history with analytic logic, retrieve historical row-level information from a robust relational database, and then view what’s happening right now for those patients. Action can then be taken in the here and now. Here are some additional use cases for Finance, Supply Chain, and Manufacturing in addition to Healthcare: Figure 1.5 – Industry use cases for Fabric Data Agent with multiple endpoints Video Summary Below is my video summary and demo of the Fabric Data Agent NoDAX architecture: Configure Fabric Data Agent for NoDAX query patterns When more than one source is added to a Fabric Data Agent, by default the source used for a specific query will be chosen based on interpreting available metadata. The Data Agents have a field called "Agent instructions" which can be used to provide detailed instructions about choosing the right source for the right question. Here’s a screenshot of the Agent instructions: Figure 1.6 – Agent instructions will guide the Fabric Data Agent to the best query endpoint I would recommend extensive unit testing and iterative improvements to the Agent instructions based upon your own data and use cases. Here’s a few examples that worked for my initial testing. I would recommend much more robust and carefully designed prompts for a production solution, but this is a baseline of an approach I found to work based on the demo in the video above: The KQL database named SeattleFireEventHouse is a live stream of 911 calls to the fire department in the city of Seattle. Whenever someone asks for “most recent” or “newest” or “latest” use SeattleFireEventHouse The lakehouse SeattleFireLakehouse should be queried with a SQL statement when someone asks for a list of incidents before the year 2026. Use SQL to retrieve row level requests for historical data. The semantic model SeattleFireSemantic Model should be queried when questions ask about historical analytic trends such as call volume averages, Year over year changes, and queries that aggregate data for analytic queries.
