In part 1 we reviewed what are the benefits of an event-driven architecture and when you would use it. Now we will dive into the considerations necessary when designing your event driven application. When do you use events or messages, queues or streams, order, or unordered processing?
Events and messages
Although events and messages have some similarities, both are designed for certain scenarios and have unique properties that distinguish them from one another.
An event is a lightweight notification of a condition or a state change that occurs in your environment. Events are sent by a producer which doesn’t know and doesn’t care about the consumers of the event. The consumer of the event decides what to do with the notification. Events can be discrete units or part of a series.
Discrete events are actionable and for next steps to occur, the consumer only needs to know that something happened. The event would have data about what happened but doesn’t have the data that initiated the event. For example, an event notifies consumers that a file was created. It may have general information about the file, but it doesn't have the file itself.
Events that are part of a stream (series) are time-ordered and interrelated. The consumer needs the sequenced series of events to analyze what happened. Typical use cases are telemetry data or events from IoT devices.
A message is raw data produced by a service to be consumed or stored elsewhere. The message contains the data that triggered the message pipeline.
When using messages, a contract exists between producers and consumers, where both have an expectation of the data format. The producer must receive a confirmation that the information has been delivered to a consumer. The producer will know the intended recipient and will want response or action to occur.
For example, the producer sends a message with the raw data, and expects the consumer to create a file from that data and send a response when the work is done.
However, the use of events and messages is not mutually exclusive. There may be scenarios where you may need to use both messages and events in a solution.
Message queues and event streams
An event or message broker must be present for an event-driven application to transmit events or messages between producers and consumers. When using messages in an event driven system, reliable message delivery is a broker requirement. To ensure guaranteed delivery, brokers rely on a message queue to store and order the messages until the consuming applications can process them. If there is a network connectivity issue, a queue will hold the message until the connection is reestablished and the message is not lost.
If there is a constant flow of data, with each event containing information about the change of state you would look to an event steam to manage the communication. Event streams can support the rebuilding of the state through replaying events in the order they were received.
If you deploy your own event broker, you are required to manage the infrastructure that runs the broker software. Commonly used open-source brokers like RabbitMQ or Apache Kafka can be self-hosted. A major drawback to this approach is the
However if you can use a serverless service like Azure Event Hubs or Azure Service Bus as your event broker. The key benefit of a serverless broker service is the ability to dynamically scale based upon the number of events that come in.
Ordered or unordered processing
By implementing first-in-first-out (FIFO) queue, consumers will receive messages in order by which they were sent. For example, an order system in an e-commerce system sends some messages to a sales system. Let’s call them "GetPayment" and "Make a Shipment". If these messages are not queued, the sales System could malfunction by notifying to "Make a Shipment" before "Getting a Payment".
There may be some instances in which messages have no relation to each other, so they don’t need to be processed in any order as long as they are processed in a timely manner.
What should you use and when?
The type of event broker you will use for your event-driven architecture is dependent on the application requirements, purpose of your application and the type of event messages it will transmit.
When to use
Event distribution (discrete)
React to status changes
Event Hubs/Apache Kafka
Big data pipeline
Event streaming (series)
Telemetry and distributed data streaming
Service Bus/Rabbit MQ
High-value enterprise messaging
Order processing and financial transactions
There isn’t a one size fits all approach to choosing a message broker, however it is important to evaluate what your business needs are in order to make the right decision of what makes sense for your solution.
To learn more about messaging services on Azure, check out MS Learn.