Announcing Mirroring for Azure Database for PostgreSQL in Microsoft Fabric for Public Preview
Back at the first European Microsoft Fabric Community Conference in September 2024 we announced our Private Preview program for Mirroring for Azure Database for PostgreSQL in Microsoft Fabric. Today, in conjunction with 2025 edition of Microsoft Fabric Community Conference in Las Vegas, we're thrilled to announce our Public Preview milestone, giving customers the ability to leverage friction-free near-real time replication from Azure Database for PostgreSQL flexible server to Fabric OneLake in Delta tables, providing a solid foundation for reporting, advanced analytics, AI, and data science on operational data with minimal effort and impact on transactional workloads. Mirroring is setup from Fabric Data Warehousing experience by providing the Azure Database for PostgreSQL flexible server and database connection details, provide selections on what needs to be mirrored into Fabric, either all data or user selected eligible mirrored tables. And, just like that, mirroring is ready to go. Mirroring Azure Database for PostgreSQL flexible server creates an initial snapshot in Fabric OneLake, after which data is kept in sync in near-real time with every transaction. How mirroring to Fabric works in Azure Database for PostgreSQL flexible server Fabric mirroring in Azure Database for PostgreSQL flexible server is based on principles such as logical replication and the Change Data Capture (CDC) design pattern. Once Fabric mirroring is established for a database in Azure Database for PostgreSQL flexible server, an initial snapshot is created by a background process for selected tables to be mirrored. That snapshot is shipped to a Fabric OneLake's landing zone in Parquet format. A process running in Fabric, known as replicator, takes these initial snapshot files and creates tables in Delta format in the Mirrored database artifact. Subsequent changes applied to selected tables are also captured in the source database and shipped to the OneLake landing zone in batches. Those batches of changes are finally applied to the respective Delta tables in the Mirrored database artifact. For Fabric mirroring, the CDC pattern is implemented in a proprietary PostgreSQL extension called azure_cdc, which is installed and registered in source databases during Fabric mirroring enablement workflow. This guided process has a new dedicated page in Azure Portal and is setting up all required pre-requisites and is offering a simplified experience where you just need to select which databases you want to replicate to Fabric OneLake (default is up to 3). You can read additional details regarding the server enablement process and other critical configuration and monitoring options on a dedicated page in Azure Database for PostgreSQL flexible server product documentation. Explore advanced analytics and data engineering for PostgreSQL in Microsoft Fabric Once data is on OneLake, mirrored data in the delta format is ready for immediate consumption across all Fabric experiences and features, such as Power BI with new Direct Lake mode, Data Warehouse, Data Engineering, Lakehouse, KQL Database, Notebooks and Copilot, which work instantly. Direct Lake mode is a fast path to load the data from the lake with groundbreaking semantic model capability for analyzing very large data volumes in Power BI. As Direct Lake mode also supports reading Delta tables right from OneLake, the Mirrored PostgreSQL database is Power BI ready along with Copilot capabilities. Data across any mirrored database (either Azure Database for PostgreSQL, Azure SQL DB, Azure Cosmos DB or Snowflake) can be cross-joined as well, enabling querying across any database, warehouse or Lakehouse (either as a shortcut to AWS S3 or ADLS Gen 2 etc.). With the same approach, you can also have multiple PosgreSQL databases from multiple servers mirrored to OneLake like in a typical SaaS provider scenario, where each database belongs to a different tenant, and execute cross-database queries to aggregate and analyze critical business metrics. Data scientists and data engineers can work with the mirrored Azure Database for PostgreSQL data joined with other sources (see this example with CosmosDB data) that are created as shortcuts in Lakehouse. Read about endless possibilities when loading operational databases in OneLake and Microsoft Fabric in related section of our product documentation here. Getting started with Mirroring for Azure Database for PostgreSQL in Fabric To summarize, Mirroring Azure Database for PostgreSQL in Microsoft Fabric plays a crucial role in enabling analytics and driving insights from operational data by ensuring that the most recent data is available for analysis. This allows businesses to make decisions based on the most current situation, rather than relying on outdated information. Improving accuracy also reduces the risk of discrepancies between the source and the replicated data, leading to more accurate analytics and reliable insights. In addition, is essential for predictive analytics and AI models provide the most recent data to make accurate predictions and decisions. To get started and learn more about Mirroring Azure Database for PostgreSQL flexible server in Microsoft Fabric, its pre-requisites, setup, FAQ’s, current limitations, and tutorial, please click here to read all about it and stay tuned for more updates and new features coming soon. To get more updates also on overall Mirroring capabilities in Fabric, please read this other blog post where you will get the latest news.Upgrade performance, availability and security with new features in Azure Database for PostgreSQL
At Microsoft Build 2025 the Postgres on Azure team is announcing an exciting set of improvements and features for Azure Database for PostgreSQL. One area we are always focused on is the enterprise. This week we are delighted to announce improvements across the enterprise pillars of Performance, Availability and Security. In addition, we're improving Integration of Postgres workloads with services like ADF and Fabric. Here's a quick tour of the enterprise enhancements to Azure Database for PostgreSQL being announced this week. Performance and scale SSD v2 with HA support - Public Preview The public preview of zone-redundant high availability (HA) support for the Premium SSD v2 storage tier with Azure Database for PostgreSQL flexible server is now available. You can now enable High Availability with zone redundancy using Azure Premium SSD v2 when deploying flexible server, helping you achieve a Recovery Point Objective (RPO) of zero for mission-critical workloads. Premium SSD v2 offers sub-millisecond latency and outstanding performance at a low cost, making it ideal for IO-intensive, enterprise-grade workloads. With this update, you can significantly boost the price-performance of your PostgreSQL deployments on Azure and improve availability with reduced downtime during HA failover. The key benefits of SSD v2 include: Flexible disk sizing from 1 GiB to 64 TiB, with 1-GiB increment support Independent performance configuration: scale up to 80,000 IOPS and 1,200 MBps throughput without needing to provision larger disks To learn more about how to upgrade and best practices, visit: Premium SSDv2 PostgreSQL 17 Major Version Upgrade – Public Preview PostgreSQL version 17 brings a host of performance improvements, including a more efficient VACUUM process, faster sequential scans via streaming IO, and optimized query execution. Now, with the public preview of in-place major version upgrades to PostgreSQL 17 there is an easier path to v17 for your existing flexible server workloads. With this release, you can upgrade from earlier versions (14, 15, or 16) to PostgreSQL 17 without the need to migrate data or change server endpoints, simplifying the upgrade process and minimizing downtime. Azure’s in-place upgrade capability offers a native, low-disruption upgrade path directly from the Azure Portal or CLI. For upgrade steps and best practices, check out our detailed blog post. Availability Long-Term Backup (LTR) for Azure Database for PostgreSQL flexible server - Generally Available Long-term backups are essential for organizations with regulatory, compliance, and audit-driven requirements, especially in industries like finance and healthcare. Certifications such as HIPAA often mandate data retention periods up to 10 years, far exceeding the default 35-day retention limit provided by point-in-time restore (PITR) capabilities. Long-term backup for Azure Database for PostgreSQL flexible server, powered by Azure Backup is now generally available. With this release, you can now benefit from: Policy-driven, one-click enablement of long-term backups Resilient data retention across Azure Storage tiers Consumption-based pricing with no egress charges Support for restoring backups well beyond community-supported PostgreSQL versions This LTR capability uses a logical backup approach based on pg_dump and pg_restore, offering a flexible, open-source format that enhances portability and ensures your data can be restored across a variety of environments including Azure VMs, on-premises, or even other cloud providers. Learn more about long term retention: Backup and restore - Azure Database for PostgreSQL flexible server Azure Databases for PostgreSQL flexible server Resiliency Solution accelerator When it comes to ensuring business continuity, your database infrastructure is the most critical component. In addition to product documentation, it is important to have access to opinionated solution architecture, industry-proven recommended practices, and deployable infra-as-code that you can learn and customize to ensure an automated production-ready resilient infrastructure for your data. The Azure Database for PostgreSQL Resiliency Solution Accelerator is now available, providing a set of deployable architectures to ensure business continuity, minimize downtime, and protect data integrity during planned and unplanned events. In additional to architecture and recommended practices, a customizable Terraform deployment workflow is provided. Learn more: Azure Database for PostgreSQL Resiliency Solution Accelerator Security Automatic Customer Managed Key (CMK) version updates - Generally Available Azure Database for PostgreSQL flexible server data is fully encrypted, supporting both Service Managed and Customer Managed encryption keys (CMK). Automatic version updates for CMK (also known as “versionless keys”) is now generally available. This change simplifies the key lifecycle management by allowing PostgreSQL to automatically adopt new keys without needing manual updates. Combined with Azure Key Vault's auto-rotation feature this significantly reduces the management overhead of encryption key maintenance. Learn more about automatic CMK version updates. Azure confidential computing SKUs for flexible server - Public Preview Azure confidential computing enables secure sensitive and regulated data, preventing unwanted access of data in-use, by cloud providers, administrators, or external users. With the public preview of Azure confidential SKUs for Azure Database for PostgreSQL flexible server you can now select from a range of Confidential Computing VM sizes to run your PostgreSQL workloads in a hardware-based trusted execution environment (TEE). Azure confidential computing encrypts data in TEE, processing data in a verified environment, enabling you to securely process workloads while meeting compliance and regulatory demands. Learn more about confidential computing with the Azure Database for flexible server. Integration Entra Authentication for Azure Data Factory & Azure Synapse - Generally Available In an era of bring-your-own-device and cloud-enabled apps it is increasingly important for enterprises to maintain central control an identity-based security perimeter. With integrated Entra ID support, Azure Database for PostgreSQL flexible server allows you to bring your database workloads within this perimeter. But how do you securely connect to other services? Entra ID authentication is now supported in the Azure Data Factory and Azure Synapse connectors for Azure Database for PostgreSQL. This feature enables seamless, secure connectivity using Service Principal (key or certificate) and both User-Assigned and System-Assigned Managed Identities, streamlining access to your data pipelines and analytics workloads. Learn more about How to Connect from Azure Data Factory and Synapse Analytics to Azure Database for PostgreSQL. Fabric Data Factory – Upsert Method & Script Activity - Generally Available The Microsoft Fabric has become to go-to data analytics platform with services and tools for every data lifecycle state. To improve customization and fine-grained control over processing of PostgreSQL data, the Upsert Method and custom Script Activity are now generally available in Fabric Data Factory when using Azure Database for PostgreSQL as a source or sink. Upsert Method enables intelligent insert-or-update logic for PostgreSQL, making it easier to handle incremental data loads and change data capture (CDC) scenarios without complex workarounds. Script Activity allows you to embed and execute your own SQL scripts directly within pipelines—ideal for advanced transformations, procedural logic, and fine-grained control over data operations. These capabilities offer enhanced flexibility for building robust, enterprise-grade data workflows, simplifying your ETL processes. Connect to VS Code from the Azure Portal - Public Preview With the exciting announcement of a revamped VS Code PostgreSQL extension preview this week, we're adding a new connection option to the Azure Portal to connect to your flexible server with VS Code, creating a more unified and efficient developer experience. Here's why it matters: One Click Connectivity: No manual connection strings or configuration needed. Faster Onboarding: Go from provisioning a database in Azure to exploring and managing it in VS Code within seconds. Integrated Workflow: Manage infrastructure and development from a single, cohesive environment. Productivity: Connect directly from the Portal to leverage VS Code extension features like query editing, result views, and schema browsing. Where to learn more The Build 2025 announcements this week are just the latest in a compelling set of features delivered by the Azure Database for PostgreSQL team and build on our latest set of monthly feature updates (see: April 2025 Recap: Azure Database for PostgreSQL Flexible Server). Follow the Azure Database for PostgreSQL Blog where you'll see many of the latest updates from Build, including What's New with PostgreSQL @Build, and New Generative AI Features in Azure Database for PostgreSQL.Microsoft Fabric & AI Learning Hackathon Informational AMA: What will you build?
The Microsoft Fabric & AI Learning Hackathon is underway and we're excited to learn about your project ideas and questions! Join us for an interactive session where we'll look at recent announcements from the European Microsoft Fabric Community Conference, share tips and best practices for competing in the Hackathon, provide technical and informational support, and connect you with expert resources to support ongoing development of your Hackathon project! An AMA is a live text-based online event similar to an “Ask Me Anything” on Reddit. This AMA gives you the opportunity to connect with Microsoft product experts who will be on hand to answer your questions and listen to feedback. The AMA takes place entirely in the comments below. There is no additional video or audio link as this is text-based. Check out this blog post for the official announcement about the Hackathon and check out the Fabric Hackathon website for all the details about the Hackathon. Feel free to post your questions anytime in the comments below beforehand, if it fits your schedule or time zone better, though questions will not be answered until the live hour.Microsoft Fabric & AI Learning Hackathon Kickoff AMA: Ready, Set, Hack!
Get ready for the Microsoft Fabric & AI Learning Hackathon! We're calling all Data/AI Enthusiasts and Data/AI practitioners to join us for the kick-off event where we will give background on the Hackathon challenges, provide links to get started, and give you a preview of what's to come in our upcoming Microsoft Fabric & AI Learning Hackathon Workshops! Plus, you'll get to meet other hackers, mentors, and experts who will help you along the way. Come with ideas or come for inspiration, we'd love to hear what you're planning to build! An AMA is a live text-based online event similar to an “Ask Me Anything” on Reddit. This AMA gives you the opportunity to connect with Microsoft product experts who will be on hand to answer your questions and listen to feedback. The AMA takes place entirely in the comments below. There is no additional video or audio link as this is text-based. Check out this blog post for the official announcement about the Hackathon and check out the Fabric Hackathon website for all the details about the Hackathon. Feel free to post your questions anytime in the comments below beforehand, if it fits your schedule or time zone better, though questions will not be answered until the live hour.
[New post] Microsoft Fabric real-time analytics exploration: Home Assistant integration
Now, your Home Assistant can send telemetry to Microsoft Fabric using the EventStream Custom App source acting as an Azure Event Hub. Then, use KQL to render historical charts. Christmas time seems to be the most wonderful time of the year for IoT Developers. The combination of both having some spare time and getting geeky gifts (or buying them for yourself) makes it the perfect moment to do some home automation. So, let's have some fun! Check out the full story of https://sandervandevelde.wordpress.com/2023/12/20/microsoft-fabric-real-time-analytics-exploration-home-assistant-integration-christmas-edition/ about Real-Time Analytics in 10+ parts. https://www.linkedin.com/feed/hashtag/?keywords=mvpbuzz&highlightedUpdateUrns=urn%3Ali%3Aactivity%3A7143383503651446785[New post] Microsoft Fabric real-time analytics exploration: Managed Grafana integration
The goal for today is to render real-time ingested Eventstream IoT data available in a Managed Grafana instance. https://sandervandevelde.wordpress.com/2023/12/05/microsoft-fabric-real-time-analytics-exploration-managed-grafana-integration/[New post] Microsoft Fabric real-time analytics exploration: Eventstream Custom apps integration
Learn how customs apps can be integrated in Microsoft Fabric using EventHub-compatible public endpoints living in Fabric Eventstream, both for ingress and egress. These are part eight and nine in a series of posts about Fabric for IoT Engineers. Check out Ingress https://sandervandevelde.wordpress.com/2023/12/02/microsoft-fabric-real-time-analytics-exploration-eventstream-source-for-custom-app-ingress/%20 and Egress https://sandervandevelde.wordpress.com/2023/12/02/microsoft-fabric-real-time-analytics-exploration-eventstream-destination-for-custom-app-egress/#mvpbuzz[new blog post] Microsoft Fabric real-time analytics exploration: Streaming to a Lakehouse
The goal for this post is to ingest Eventstream IoT data coming from an actual device and to pass that data on to a Lakehouse. We will learn how to process the IoT data using the SQL language and will touch on the usage of Notebooks. This is part six about Microsoft Fabric for IoT Developers. Read the full story https://sandervandevelde.wordpress.com/2023/11/20/microsoft-fabric-real-time-analytics-exploration-streaming-to-a-lakehouse/.[blog series] Introduction to Microsoft Fabric Real-Time Analytics for IoT developers
Are you interested in combining Microsoft Fabric Real-Time Analytics with Azure IoT? In my series of five blog posts, learn how to combine Eventstreams, KQL Databases, Reflect, and Data Activator for powerful queries, insightful reports, and condition-based alerts. Learn about working with (streaming) sample data and actual IoT data ingested from the industrial router via an Azure IoT Hub. For those who are just exploring Microsoft Fabric, learn how to pause your Capacity automatically using an Azure Function and save some money. Just start here https://sandervandevelde.wordpress.com/2023/11/10/microsoft-fabric-real-time-analytics-exploration-for-iot-developers/ https://sandervandevelde.wordpress.com/2023/11/12/microsoft-fabric-real-time-analytics-exploration-storing-and-reporting-time-series-data/ https://sandervandevelde.wordpress.com/2023/11/13/microsoft-fabric-real-time-analytics-exploration-ingesting-streams-of-sample-events/ https://sandervandevelde.wordpress.com/2023/11/14/microsoft-fabric-real-time-analytics-exploration-ingesting-iot-device-events/ https://sandervandevelde.wordpress.com/2023/11/15/microsoft-fabric-real-time-analytics-exploration-activating-iot-device-reflex-data/
