Enabling Full-stack Observability with Azure Monitor and Grafana
Along with the announcement of Azure Managed Grafana, we are excited to introduce new Grafana integrations with Azure Monitor including the ability to pin Azure Monitor visualizations from Azure Portal to Grafana dashboards and new out-of-the-box Azure Monitor dashboards.Visualizing data in realtime with Azure Managed Grafana
We’re delighted to announce that Azure Managed Grafana, which has been in public preview since April, is now generally available (GA). It brings the Grafana visualization platform to Azure as a fully managed and highly available service. Azure Managed Grafana enables near realtime data monitoring with rich graphs and alerts specifically designed for Azure data sources. It compliments existing visualization solutions such as Azure Dashboards and Workbooks. Get started with Azure Managed Grafana Azure Managed Grafana is operated and supported by Microsoft. Like any 1 st -party Azure service, you can deploy a new Grafana instance (a.k.a., workspace) using the Azure portal, command-line tools, Bicep/ARM templates or control plane APIs. Create your first Managed Grafana workspace The easiest way to stand up a Grafana workspace is through the Azure portal. Each instance is regional and, for the current Standard tier, hosted on two dedicated Virtual Machines (VM) to provide better resiliency. These VMs are located in the same datacenter by default; they can be optionally placed in different availability zones using the Zone redundancy feature, for which there is an additional charge. Connect seamlessly to Azure data sources Azure Monitor and Azure Data Explorer are two of the most commonly used data stores for operational telemetry and analytics. Azure Managed Grafana supports these data sources out-of-the-box. When it is created, a Grafana workspace is automatically assigned with an Azure Active Directory managed identity, which then is added to the Monitor Reader role on the subscription. This gives you immediate access to Azure Monitor from the new Grafana workspace without needing to set permissions manually. View your Azure resources with built-in dashboards Grafana provides a library of pre-built dashboards that you can use as-is or customize further. We included Azure Monitor integrations and several dashboards for popular Azure services in the Managed Grafana public preview: Application Insights VM Insights Container Insights Azure Monitor alerts As a part of the GA release, we’re adding more curated dashboards for Azure services: Container Apps Spring Apps API Management Application Insights availability geo map Network Insights These dashboards either are pre-installed in your Managed Grafana workspace or can be downloaded from Azure Monitor Team dashboards. Easily add log queries and charts from Azure portal to Managed Grafana The new “pin to Grafana” feature for Azure Monitor Logs allows you to seamlessly add charts and queries from Azure Monitor Logs in the Azure Portal to new or existing Grafana dashboards. To get started, navigate to Logs for your resource or the relevant Log Analytics workspace in the Azure Portal. Next, create your own query or choose to edit queries from the list of out of the box sample queries. Once your query works as expected, click ‘Pin to’, followed by Grafana dashboard. Then, choose the Grafana workspace and dashboard and click ‘Pin to’ complete the operation. The query from Azure Monitor Logs is then used to automatically create a new panel in the selected Grafana dashboard. Test it out today Give Azure Managed Grafana a try. Get going with this Quickstart and create your first Grafana workspace.Introducing a unified Azure Monitor cloud native offering for Kubernetes monitoring
Azure Monitor now offers Managed Prometheus service alongside Azure Managed Grafana and Azure Monitor Container insights, fostering an ecosystem of open-source and vendor neutrality. Now you can get the power of cloud native monitoring tools with enterprise grade security and reliability of Azure Monitor. What is Azure Monitor’s unified cloud native offering for Kubernetes monitoring? We are excited to announce the preview for Azure Monitor managed service for Prometheus to monitor your Kubernetes cluster. But it is more than “just another service” to monitor your Kubernetes cluster. Managed service for Prometheus is deeply integrated with Azure Monitor tools. You can now set up end-to-end monitoring for your Kubernetes cluster without the headache of patching various tools and manual integrations. With Azure Monitor’s unified cloud native offering, you get: A fully managed service that handles ingestion, storage for long-term data retention and querying of your Prometheus metrics using Prometheus Query Language (PromQL) with Azure Monitor managed service for Prometheus. Advanced troubleshooting capabilities with Azure Monitor Container Insights that offers visualizations and troubleshooting logs of nodes, controllers, and containers. A full stack observability with Azure Managed Grafana by a single-click configuration to seamlessly link various data sources and get popular out-of-the-box Grafana dashboards Get started with Azure Monitor’s cloud native offering Getting started with Azure Monitor’s cloud native offering for Kubernetes monitoring is a breeze. Once you create your Kubernetes cluster, you can enable monitoring using Azure portal, Azure resource manager templates and CLI. Using Azure portal, you can easily enable Prometheus metrics by selecting ‘Insights’ in your cluster view. You can store the Prometheus metrics in an Azure monitor workspace. Additionally, you can pick a log analytics workspace and enable logs collection for advanced troubleshooting. Finally, link your Grafana workspace (or create a new one) to visualize your monitoring data. Once you configure, unleash the power of the cloud native capabilities for monitoring the Kubernetes cluster with Azure Monitor. Open-source compatibility and portability for Prometheus metrics Azure Monitor managed service for Prometheus is a fully managed service that handles ingestion for the native Prometheus metrics types. It is available to use on its own or as an integrated component of Azure Monitor container insights and Azure Managed Grafana. Additional benefits include: 18-month data retention for storage and PromQL based query service Highly available, scalable, and enterprise-grade secure service Easily enable managed Prometheus using our collector or use it as a drop-in replacement for self-managed Prometheus Retain your existing Prometheus configurations, recording rules and alert rules. Monitor health and performance of your Kubernetes cluster with Azure Monitor container insights Azure Monitor container insights collects critical logs, enables alerts to identify issues, and provides visualization to monitor health and performance of your Kubernetes cluster. It complements CNCF backed open-source tools for end-to-end Kubernetes monitoring including logs collection for advanced troubleshooting. Enabling container insights with managed Prometheus will open possibilities such as: Correlating spikes in Prometheus metrics with troubleshooting logs for Kubernetes cluster Identifying capacity needs and determining the maximum load that the cluster can sustain by understanding the behavior of the cluster under average and heaviest loads Advanced diagnostics with collection of container logs (stdout/stderror), events, and pod metrics We understand that your monitoring needs may vary depending on scale, topology, organizational roles, and multi-cluster tenancy. Learn more about the best practices for monitoring each layer starting from infrastructure up through applications with Prometheus metrics and logs collected with container insights. Get full stack observability with Azure Managed Grafana Getting the right monitoring data is halfway through to get started with monitoring. It is imperative to have a single pane of glass to determine the performance issues and quickly mitigate them. Azure Managed Grafana is integrated with Azure Monitor and allows you to get full stack observability from multiple data sources on a single screen. With Azure Managed Grafana, you can: Get popular Grafana dashboards out-of-the-box for the Prometheus metrics Easily add your existing Grafana dashboards or Azure Monitor visualizations in a single view Combine application metrics and infrastructure metrics from various data sources into a single dashboard for full stack visibility Add Grafana dashboards from the open-source community That is all you need to get started with Kubernetes monitoring on Azure Monitor. We’d love to hear what you like and don’t like about this feature, and where you’d like us to take it. Please provide feedback on Azure Monitor Community under Managed Prometheus category. If you wish to learn more, you can always find a great ton of learning content in our documentation. Learn more Azure Monitor managed service for Prometheus (preview) | Microsoft Learn Overview of Container insights - Azure Monitor | Microsoft Learn Quickstart: create an Azure Managed Grafana instance using the Azure portal | Microsoft Learn Monitor Azure services and applications using Grafana - Azure Monitor | Microsoft LearnAzure Managed Grafana Adds New SKU and Features
At the first anniversary of Azure Managed Grafana general availability, we’re excited to announce the preview of the Essential SKU as well as numerous new functionalities and enhancements. These additions bring more choices and value to our users. Essential SKU We’ve observed that many current users of Managed Grafana exclusively or primarily view data from Azure Monitor services, including the managed service for Prometheus. These users don’t require the full extensibility Grafana provides. We’re introducing a simplified Managed Grafana SKU called Essential that focuses on Azure Monitor dashboarding. Because it consumes less resources, the Essential SKU is more cost-effective than the Standard SKU for development and testing uses. You pay only for active usage based on the number of actual users you have in each calendar month. You can have one Essential SKU workspace per Azure subscription. To create an Essential SKU workspace, choose “Essential” in the Pricing Plans during a new deployment. You may also upgrade an Essential SKU workspace to Standard at a later point, as your needs for Grafana functionality change over time. Grafana Version 10 Users creating new deployments of Azure Managed Grafana Standard SKU can now choose Grafana version 10 for access to a wide range of features and enhancements including time region support and annotation filtering improvements. With this opt in model, you have the flexibility to use a newer Grafana version much earlier than when it becomes mandatory for all Managed Grafana instances. Existing Managed Grafana workspaces running on Grafana version 9 can’t be upgraded in-place to version 10 yet. We’ll announce when it’s supported, once we’ve resolved all the migration issues. Team Sync with Entra Groups Grafana team sync is a Grafana Pro feature that lets you connect teams in Grafana to an external identity provider. You typically set up Grafana teams to manage permissions for accessing Grafana dashboards at a better scale. With regular Grafana teams however, you’ll have to configure and maintain their member lists as a separate task. Managed Grafana utilizes team sync to allow you to define Grafana teams as Entra (formerly, Azure Active Directory) groups. With such functionality, you can manage your groups and memberships centrally and with greater consistency. Plugin Management Until very recently, Managed Grafana packaged and executed all plugins as a part of the Grafana software image that’s deployed to all workspace instances. That approach impacted both the image size and startup time as the number of plugins increases. With plugin management, Managed Grafana pre-installs a smaller set of plugins, and you can add optional plugins to your Grafana workspace after it’s created. Initially only plugins that have been reviewed by the Managed Grafana team are available for installation. Application Insights Traces in Grafana Starting with Grafana 10, Azure Monitor users can now accelerate their application performance triage and troubleshooting workflows by searching, filtering and viewing Application Insights traces natively in Grafana’s trace visualizations and Tracing in Explore. Any application being monitored by Azure Monitor Application Insights stores its logs and traces in an Application Insights resource. Prior to Grafana 10, these application traces could only be queried and searched for as logs, and viewed in Grafana’s standard timeseries and table visualization panels. Search for traces in Azure Monitor query wizard In Grafana 10, users can now filter and isolate traces of interest with the new trace query wizard in Explore or dashboard panels. View Traces After identifying an anomalous trace, users can choose to view trace details directly in Grafana using the split view in Explore or by opening a new tab. Click on the itemId field to view all the spans within the end-to-end trace. View Trace details The selected trace or span opens in Grafana’s dedicated trace viewer where you can use the Gantt chart view to understand the contribution of delays by individual microservices and dependencies. This is a fast and efficient approach to performing root cause of analysis to identify why a specific transaction was slow. Start with new out-of-the-box dashboards To help you get started working with App Insights traces in Grafana, there are now 5 new dashboards that are shipped with the Azure Monitor data source plugin and available for import from Dashboards | Grafana Labs. Each of these dashboards provides a curated experience to triage performance problems or failures and allows users to click on a single operationID to view the trace details. Azure / Insights / Applications - Performance - 1. Operations | Grafana Labs Azure / Insights / Applications - Performance - 2. Dependencies | Grafana Labs Azure / Insights / Applications - Failures - 1. Operations | Grafana Labs Azure / Insights / Applications - Failures - 2. Dependencies | Grafana Labs Azure / Insights / Applications - Failures - 3. Exceptions | Grafana Labs User-Based Authentication for Azure Data Explorer (ADX) Grafana normally uses a special identity (e.g., managed identity or app registration) as the access credential for a data source. The ADX plugin is the first data source plugin to support user-based authentication and to enable end-to-end permission control for the currently signed in user. Most ADX customers have already configured their clusters for user access. The user-based authentication for ADX allows those same settings to be applied to Grafana access to ADX. It not only simplifies the data source access configuration but also makes possible to apply policies (e.g., audit logging) based on who the user is. It’s worth noting that user-based authentication doesn’t work with Grafana automation features, such as alerts or reporting. If you need these features, you’ll have to set up a separate ADX data source with a managed identity or app registration and use that with the features. User Deduplication in Billing One customer pain-point we’ve heard is that Managed Grafana charges for each active user in each workspace. If a user interacts with two workspaces, that user is billed twice. The Managed Grafana service are region based. There is no easy way for it to enumerate all workspaces belonging to a customer. It, however, can do this reliably at the Azure subscription level. Starting with this December’s billing cycle, Managed Grafana will charge each user once based on the first workspace in a subscription they sign into. A user can access any number of Standard SKU workspaces within the same subscription, and they will be counted as one active user for billing purposes. Try Them Out Today With these updates, along with private link and managed private endpoint that we released earlier this year, we hope that Azure Managed Grafana will better meet your visualization needs. Give them a try and leave us a comment to let us know what you think.
