Azure Event Hubs update – Q4 2022
Azure Event Hubs enables you to stream millions of events per second from any source using Kafka, AMQP or HTTPS protocols and build end-to-end event streaming pipelines. We are excited to announce the availability of several new features and major enhancements in Azure Event Hubs, which will be available in Q4 of calendar year 2022.Announcing self-serve experience for Azure Event Hubs Clusters
For businesses today, data is indispensable. Innovative ideas in manufacturing, health care, transportation, and financial industries are often the result of capturing and correlating data from multiple sources. Now more than ever, the ability to reliably ingest and respond to large volumes of data in real time is the key to gaining competitive advantage for consumer and commercial businesses alike. To meet these big data challenges, Azure Event Hubs offers a fully managed and massively scalable distributed streaming platform designed for a plethora of use cases from telemetry processing to fraud detection. Event Hubs has been immensely popular with Azure’s largest customers and now even more so with the recent release of Event Hubs for Apache Kafka. With this powerful new capability, customers can stream events from Kafka applications seamlessly into Event Hubs without having to run Zookeeper or manage Kafka clusters, all while benefitting from a fully managed platform-as-a-service (PaaS) with features like auto-inflate and geo-disaster recover. As the front door to Azure’s data pipeline, customers can also automatically Capture streaming events into Azure Storage or Azure Data Lake, or natively perform real-time analysis on data streams using Azure Stream Analytics. For customers with the most demanding streaming needs, Event Hubs clusters in our Dedicated tier provide a single-tenant offering that guarantees the capacity to ingest millions of events per second while boasting a 99.99% SLA. Clusters are used by the Xbox One Halo team, as well as powers both Microsoft Teams and Microsoft Office client application telemetry pipelines. Azure portal experience for Event Hubs clusters Today, we are excited to announce that Azure Event Hubs clusters can be easily created through the Azure portal or through Azure Resource Manager as a self-serve experience (preview), and is instantly available with no further setup. Within your cluster, you can subsequently create and manage namespaces and event hubs per usual and ingest events with no throttling. Creating a cluster to contain your event hubs offers the following benefits: Single tenant hosting for better performance with guaranteed capacity at full scale, enabling ingress of gigabytes of streaming data at millions of events per second while maintaining fully durable storage and sub-second latency. Capture feature included at no additional cost, which allows you to effortlessly batch and deliver your events to Azure Storage or Azure Data Lake. Significant savings on your Event Hubs cloud costs with fixed hourly billing while scaling your infrastructure with Dedicated Event Hubs. No maintenance since we take care of load balancing, security patching, and OS updates. You can spend less time on infrastructure maintenance and more time building client-side features. Exclusive access to upcoming features like bring your own key (BYOK). In the self-serve experience (preview), you can create 1 CU clusters in the following strategic regions through the Azure portal: North Europe West Europe US Central East US East US 2 West US West US 2 North US South Central US South East Asia UK South Larger clusters of up to 20 CUs or clusters in regions not listed above will also be available upon direct request to the Event Hubs team. Data is key to staying competitive in this fast moving world and Azure Event Hubs can help your organization gain the competitive edge. With so many possibilities, it’s time to get started. Learn more about Event Hubs clusters in our Dedicated offering. Get started with an Event Hubs cluster on the Azure portal. Quickstart: Data streaming with Event Hubs using the Kafka protocolAzure Event Hubs Dedicated Self-Serve Scalable Clusters GA for Mission-Critical Streaming Workloads
Today, we are announcing the general availability of Azure Event Dedicated Self-Serve clusters which are designed for mission-critical Kafka and AMQP workloads that require low-latency and high-volume data streaming with dynamic scaling.
Announcing the Event Hubs Data Explorer: a handy tool for getting started and debugging
Transform your event-driven architectures with the new Event Hubs Data Explorer! Whether you're debugging, optimizing, or just getting started, this tool offers a unified interface for producing and consuming event data, providing invaluable insights. Explore the endless possibilities with Event Hubs Data Explorer!
Steps to upgrade control plane API references for Azure Service Bus, Event Hubs and Relay
On 30 September 2026, Azure Resource Manager control plane APIs 2014-09-01, 2015-08-01, and 2016-07-01 will be retired. Migrate to the latest control plane API version by that date to avoid potential service outages in Azure Service Bus, Event Hubs, and Relay. The latest API for control plane operations, version 2021-11-01, offers feature updates and performance improvements to make your applications more resilient.Geo-Replication is Here! Now generally available for Event Hubs Premium & Dedicated
Today, we are thrilled to announce the General Availability of the Geo-replication feature for Azure Event Hubs, now available in both Premium and Dedicated tiers. This milestone marks a significant enhancement in our service, providing our customers with robust business continuity and disaster recovery capabilities – ensuring high availability for their mission-critical applications. The Geo-replication feature allows you to replicate your Event Hubs data across multiple regions either synchronously or asynchronously, ensuring that your data remains accessible in the event of maintenance activities, regional degradation, or a regional outage. With Geo-replication, you can seamlessly promote a secondary region to a primary, minimizing downtime and ensuring business continuity. Before failover (promotion of secondary to primary) After failover (promotion of secondary to primary) With general availability, we are excited to announce that the Geo-replication feature now supports all the features that are generally available in the service today. This includes private networking, customer-managed key encryption, Event Hubs Capture, and many more. These enhancements ensure that you can leverage the full capabilities of Event Hubs while benefiting from the added reliability of Geo-replication. We have also increased visibility into the health and metrics of your replicas. This means you can now monitor the status of your replicas more effectively and know exactly when it is appropriate to promote your secondary to primary. This added visibility ensures that you can make informed decisions and maintain the high availability of your applications. Since the announcement of public preview, we’ve had several customers try out the Geo-replication feature and appreciate the enhanced reliability and peace of mind that comes with having a robust disaster recovery solution in place. Learn more Learn more about geo-replication concepts and the pricing model and try out this quickstart to learn how to setup geo-replication for your premium and dedicated tier namespaces. We encourage our customers to try out the Geo-replication feature and experience the benefits of turnkey business continuity and disaster recovery features firsthand. Your feedback is invaluable to us, and we look forward to hearing about your experiences.
Introducing Azure Event Hubs Dedicated Self-Serve Scaling experience –Public Preview
Since GA of Azure Event Hubs dedicated clusters 3 years ago, the service has been catering to high demand, critical workloads of customers. Dedicated Event Hubs clusters offer single tenant deployments with massive throughput and sub-second latencies. We wanted to empower our customers with the ability to scale the clusters at will with the announcement and public preview of Self-Serve Event Hubs cluster scaling experience Why Scaling? It is inevitable to run out of resource capacity during final hour needs. With dynamic loads across the systems, flexibility to scale resources at will becomes more important than ever. Generally, you can use CPU consumption metric under your dedicated cluster to understand when you’re required to scale. You can find more details here Here are a few use cases to consider: Scale out when you need it the most. Having the control on the capacity units that you own, provides confidence in ensuring that business goals are met even when the traffic becomes unpredictable. Scaling in when you don't use that much capacity so that it regulates the cost associated with the offering you use. You only pay for the higher capacity units for the time you scale out. Linear scaling ensures that you have as granular control as possible to ensure that you can choose the right capacity unit to fit your needs. As an example. If you are using 2 CU cluster and need one more unit, you don't need to move to 4 CU cluster and can safely scale to 3 CUs. Architectural changes for stability and scalability Scalable clusters stem out of different design and architecture paradigm, which allow them to have the ability to support scalability operations Since existing dedicated clusters were built on a different platform having its own architectural limits, they cannot cater to self-serve scaling functionalities. NOTE - Migrating from the existing (already provisioned) dedicated clusters to self-serve clusters is not possible due to the difference in underlying architectural paradigms How to scale dedicated clusters? To use the scalability feature of Event Hubs, you would need to check "Support Scaling" checkbox during Event Hubs cluster creation. Go to “Scale” tab located in option tree on Event Hubs Cluster UI: Clicking on the “Scale” tab would lead you to a slider, which can be used to scale out/ scale-in to desired number of CUs as per your needs Another way to scale would be to click on “Capacity” property on essentials pane on event hub cluster UI. This would lead you to slider as shown above which you can use to scale out/ scale in. Supported regions Since self-serve scaling experience is in public preview stage, it is currently supported to create clusters in limited regions for now (Asia East, South Central US, US West). Please check create cluster experience on Azure Portal to find out the newly added supported regions in future. Know more To know more about Azure Event hubs Dedicated clusters: Overview of Azure Event Hubs dedicated tier - Azure Event Hubs | Microsoft Docs How to create a dedicated cluster on your own: Create an Event Hubs dedicated cluster using the Azure portal - Azure Event Hubs | Microsoft Docs
