Azure Event Grid - Serverless event-based applications

Iron Contributor

Mar 19, 2020

This post provides an overview of Azure Event Grid. The Azure Event Grid provides the capability for building event-based applications. For the purpose of this post, let's define event-based application as applications designed to respond to changes in state. The change in state, referred to as the event, is really anything that we can make the application aware of ranging from the creation, update and delete of entities as well as timer events or failures within the application.

Event-based architecture fits well in cloud based solutions especially when combined with serverless resources, for example Azure Logic Apps and Azure Functions. Azure Event Grid provides a flexible and reliable platform for building event-based architecture.

Events vs Messages

When designing an event-based application it is important to understand the difference between event-based applications and message-based applications. Let's spend some time describing the differences between an event and a message.

Event

An event is a lightweight notification of a condition or state change. The producer of an event does not have any expectation on how the event is handled. Some examples:

Blob has been created
Sales order 107 has been updated to a confirmed status
Exception occurred during the load of CRM data

Message

A message is data that is produced to be consumed or stored. The producer of a message does have an expectation on how the event will be handled. Some examples:

Create this blob
Confirm sales order 107
Notify the support team about a failure during the load of CRM data

Contrasting event-based and message-based applications

The difference might be subtle but the design paradigm between the two is not. Event-based applications lend themselves well to reactive programming where the full set of requirements is not known at design time or when addressing new or shifting requirements. Because of this event-based applications tend to handle change better than message-based applications. Message-based applications fit systems where the producers and consumers are more well known and defined. The requirements and design are more easily conveyed when reviewing the implementation. For example, a set of APIs named CreateBob, ConfirmSalesOrder, and NotifyTeam provide some meaning simply by their names.

Having said that, these two design paradigms are not mutually exclusive, and often are applied to the same application. In some situations, reacting to an event will make more sense, and in other situations sending a message will do.

With Event Grid building event-based alongside message-based functionality can be easily incorporated into an application. Let's expand upon the Blob example given earlier.

Blob created

Let's say the requirements around this example is to allow for images to be stored in Azure Blob Storage. The stored image should be optimized for displaying on the web without degrading the original quality of the image. There are two approaches here that I will suggest (I know there are more including Azure Content Delivery Network Image Processing but let's keep things simple for now😞

Create a new a Azure Function that receives the image and applies the required optimization and stores both the original and the optimized version to storage.
Use the existing Azure Storage API for uploading a blob and use Event Grid to subscribe to the blob created event in order to trigger an Azure Function to apply and store an optimized version.

Both approaches will result in achieving the same requirement: two blobs stored. The advantage of the first is there are fewer moving parts: an Azure Function and an Azure Storage Account. While the second approach includes both the Azure Function and the Azure Storage Account, as well as an Event Grid subscription. Another distinction is the first approach, being message-based, would include the blob as part of the data supplied to the message. The second approach, being event-based, would be more lightweight so would not include the blob as part of the event. The blob itself would need to be retrieved from storage. Technically they are about the same level of complexity but an important advantage is how they will react to change.

Let's say requirements are added, for example sending an email when the file is not in an known format. The temptation would be for the first approach, to expand the function to provide this additional functionality. As more requirements are added though, this would contribute to the function becoming more difficult to maintain. An advantage of the second approach, would be instead of extending the existing function, a new subscription could be added with a filter to detect when the created blob is in an unknown format.

Both approaches would work, but the event-driven approach lends itself more to environments with shifting or evolving requirements.

Event Grid Concepts

The following are the main concepts when working with Azure Event Grid:

Event source - where the event took place
Event - what happened
Topic – endpoint where the event was published
Event subscription – route to send the event
Event handler – a service reacting to the event

Going back to the blob created example, the event source would be blob storage. The event would be information about the created blob, and as this is lightweight, it would not include the itself but metadata such as the url, etag, content type and content size of the created blob. Think of the topic as the means an event is sent to Event Grid while a subscription is used to inform Event Grid that the handler should be sent the event.

Please see documentation for additional information on concepts.

Event Grid Features

Let's take a moment to highlight some features of the Event Grid:

Guaranteed Delivery - Event Grid will deliver an event at least once for each subscription. This makes building fan-out integration
Batch Delivery - By default Event Grid will not batch events. For high-throughput scenarios, it is possible to batch events by events per batch and with a preferred batch size in kilobytes.
Retry - Event Grid will retry the delivery of events to a subscription in the case of failure. There is a customizable back-off retry policy with the following defaults: 10s, 30s, 1m, 5m, 10m, 30m, 1h for up to 24 hours
Dead-letter events - By default events that fail to be sent are lost but dead-letter storage can be enabled.
Security - Event Grid is secured with either a SAS token or SAS key

Please see documentation for additional information on features.

Event Grid Monitoring

The primary means of monitoring the Event Grid is using the metrics supplied in the portal. These metrics are related to topics or subscriptions:

Topics

Publish Succeeded – Sent to the topic and processed with a 2xx response
Publish Failed – Sent to the topic but rejected with an error code
Unmatched – No matching event subscription

The following illustrates an example of a custom topic. This shows all 71 events have successfully been published to at least one subscription.

Subscriptions

Delivery Succeeded – Delivered successfully with a 2xx response
Delivery Failed – Delivered but received a 4xx or 5xx response
Expired Events – Not delivered with all retry attempts and the event was dropped
Matched Events – Event in the topic was matched by the subscription

The following illustrates the view of a subscription. In this example, all 71 events we delivered to the subscription but 31 of them failed to be processed successfully.

Please see documentation for additional information on monitoring.

Event Grid Pricing

Event Grid is priced as a pay-per-operation model with a free monthly usage of 100,000. Keep in mind this is per operation and not per event. Operations include ingress of events to Domains or Topics, advanced matches, delivery attempts, and management calls.

The pay-per-operation billing will depend on your subscription but as an example, the pricing calculator is .60 USD per million operations.

Please see documentation for additional information on pricing.