Reliable, in-depth, and at-scale monitoring of database performance has been a long-standing top priority for SQL customers. Today, we are pleased to announce the public preview of database watcher for Azure SQL, a managed database monitoring solution to help our customers use Azure SQL reliably and efficiently.
Managed monitoring for Azure SQL
To enable database watcher, you do not need to deploy any monitoring infrastructure or install and maintain any monitoring agents. You can create a new watcher and start monitoring your Azure SQL estate in minutes.
Once enabled, database watcher collects detailed monitoring data from your databases, elastic pools, and managed instances into a central data store in your Azure subscription. Data is collected with minimal latency – when you open a monitoring dashboard, you see database state as of just a few seconds ago.
Figure 1: A conceptual diagram of a database watcher deployment, showing the flow of monitoring data from Azure SQL resources to a database watcher, its data store, dashboards, and an analytic query console.
Dashboards in the Azure portal show you a single-pane-of-glass view across your Azure SQL estate, and a detailed view of each database, elastic pool, and managed instance.
Figure 2: An estate dashboard in database watcher, shown in dark mode. In this example, each hexagon represents a primary, secondary, or geo-replica of an Azure SQL database.
Figure 3: A partial view of the detailed resource dashboard, showing performance metrics for an Azure SQL database.
Database watcher is powered by Azure Data Explorer, a fully managed, highly scalable data service, purpose-built for fast ingestion and analytics on time-series monitoring data. A single Azure Data Explorer cluster can scale to support monitoring data from thousands of Azure SQL resources. You have an option to use the same core engine in Real-Time Analytics in Microsoft Fabric, an all-in-one SaaS analytics solution for enterprises. Real-Time Analytics seamlessly integrates with other experiences and items in Microsoft Fabric.
Once the SQL monitoring data is in Azure Data Explorer or Real-Time Analytics, it becomes available not just on dashboards in the Azure portal, but also for a wide variety of other uses: you can analyze data on-demand using KQL or T-SQL, build custom visualizations in Power BI and Grafana or use native Azure Data Explorer dashboards, analyze monitoring data in Excel, or export it for integration with downstream systems.
Database watcher supports all service tiers and SKUs of Azure SQL Database and Azure SQL Managed Instance. This includes vCore and DTU purchasing models, provisioned and serverless compute tiers, single databases and elastic pools, and Hyperscale. It can monitor all types of secondary readable replicas, including high availability replicas, geo-replicas, and Hyperscale named secondary replicas.
To get started and to learn more, review database watcher documentation:
A retrospective of database monitoring in Azure SQL
Over the last decade, our customers have consistently told us that reliable, low-latency, and detailed monitoring is a must-have for a cloud database service. In the early days of Azure SQL, Azure Monitor metrics and diagnostic telemetry met database monitoring requirements for many of our customers. But when large and complex mission-critical database applications became common in Azure, it was clear that we needed to do more.
In the following years, we released two monitoring solutions that were intended to bring advanced monitoring capabilities to Azure SQL: Azure SQL Analytics and SQL Insights. However, customer feedback for Azure SQL Analytics highlighted gaps in the monitoring data coverage, an excessively high data latency, and high cost at scale. SQL Insights improved data coverage to a degree, but the data latency and cost concerns remained. An even stronger customer feedback for SQL Insights was that customers didn’t want to set up and maintain an IaaS VM to monitor PaaS database services. Reliability problems in Azure SQL Analytics and SQL Insights have affected customers as well. Today, these earlier solutions remain in preview.
We learned from this experience and feedback when designing and building database watcher. This new managed database monitoring solution brings truly in-depth data coverage, collecting data from over 70 SQL system views and visualizing it directly in the Azure portal. Data latency is typically in single-digit seconds or less, not minutes. Azure Data Explorer and Real-Time Analytics bring their advanced analytics capabilities, enabling us, our customers, and partners to develop actionable advice and insights from the rich data that database watcher collects.
Roadmap
The preview we are launching today makes a managed monitoring solution for Azure SQL broadly available to our customers. But our journey is not yet complete. As we make progress toward general availability and beyond, customers can expect the following new features and improvements:
- Alerts
- Monitoring for SQL Server on Azure VM
- Support for an expanded set of Azure regions
- An increased limit on the number of monitoring targets per watcher
- Configuration and management with PowerShell and Azure CLI
- Support for user-assigned managed identity
- Database watcher specific RBAC roles and actions
- Configurable data collection
- Extended events collection
- Manageability improvements for monitoring larger estates
This is not a static list. Our work on database watcher is largely driven by your feedback. We encourage you to tell us what is most important for you in the SQL monitoring space, both in the short and in the long term.
Your feedback
During the private preview, dozens of Azure SQL customers have evaluated database watcher for their monitoring scenarios and provided detailed feedback to help us deliver a functional and reliable database monitoring solution. We are very grateful to every private preview participant – you all have made database watcher better!
Now that database watcher is available in public preview, we invite and encourage all Azure SQL customers worldwide to do the same: set up and use database watcher in your environments, tell us what works well, what needs improvement, and what might be missing. Your feedback is critical to help us deliver a generally available solution that you will like. We are looking forward to hearing from you!