Announcing MQTT Last Will and Testament Public Preview in Azure Event Grid

[george-guirguis]

Overview

Azure Event Grid is a pub-sub message broker that enables you to integrate your solutions at scale using HTTP pull delivery, HTTP push delivery, and MQTT broker capability. The MQTT broker capability leverages standard features from MQTT protocol to enable your clients to communicate in a reliable, secure, and scalable manner. This capability is vital for IoT solutions where efficient communication is essential for seamless operations, driving the digital transformation of organizations across various industries.

Today, I am thrilled to announce the public preview release of the Last Will and Testament (LWT) support in Azure Event Grid's MQTT broker capability, in compliance with the MQTTv3.1.1 and MQTTv5 specifications. LWT enables your MQTT clients to get notified with the abrupt disconnections of other MQTT clients. This powerful feature ensures predictable and reliable flow of communication among MQTT clients during unexpected disconnections, which is valuable for scenarios where real-time communication and coordinated actions are critical.

Need for client disconnection notifications

IoT devices usually operate in environments characterized by unreliable network connectivity, where connections might be sporadic or prone to disruptions due to signal loss, limited power, or other factors. Consequently, MQTT clients might disconnect from the MQTT broker without clear indication whether the disruption was intentional or unexpected. The absence of notifications about the unexpected client disconnections poses a significant challenge across industries, potentially causing service unavailability, increased downtime, and communication failures. In scenarios where multiple MQTT clients collaborate to execute intricate tasks or distribute workload, real-time communication and reliability are critical. These disconnections disrupt the coordination of tasks within the system, leading to inefficiency and performance degradation.

LWT support in Event Grid’s MQTT broker

Event Grid’s MQTT broker capability now supports MQTT last will and testament for MQTTv3.1.1 and MQTTv5 clients to address this challenge. When a client connects to the MQTT broker, it can specify a will message, will topic, and the rest of the will properties in the CONNECT packet. If the client disconnects gracefully through the MQTT DISCONNECT packet with reason code 0x00, the will message is discarded. However, if the client ungracefully disconnects, the MQTT broker publishes the will message to all the clients that subscribed to the will topic. Accordingly, the subscribing clients can adapt swiftly to this disconnection by redistributing tasks, reallocating responsibilities, or adjusting their behavior to ensure continued system performance and stability.

How to use LWT with Event Grid’s MQTT broker?  

In this example, there are actuators that operate based on the data generated from sensors. If the sensors disconnect abruptly, the actuators need to change their behavior or stop since they don’t have real time data from the sensors to act upon. The sensors use MQTT LWT to reliably notify these actuators whenever it disconnects unexpectedly. The clients use the following topics to achieve this capability.

Client	Role	Topic/Topic Filter
sensor1	Publisher	clients/sensor1/status
sensor2	Publisher	clients/sensor2/status
actuator1	Subscriber	clients/+/status
actuator2	Subscriber	clients/+/status

Configuration

The following steps show the configuration of the namespace and the sample clients to model this example.

Namespace configuration

Let’s start by configuring the Event Grid namespace as the MQTT broker that will route these messages among the sensors and actuators. The following steps show the configuration using command-line and Azure CLI. For more information about the Azure portal configuration of the namespace, go to this quickstart.

Generate certificates

You can use step CLI to create sample certificates for your clients to authenticate with the namespace.

1. To create root and intermediate certificates, run the following command. Remember the password, which you need to use in the next step.
   • step ca init --deployment-type standalone --name MqttAppSamplesCA --dns localhost --address 127.0.0.1:443 --provisioner MqttAppSamplesCAProvisioner
2. Use the certificate authority (CA) files generated to create a certificate for each one of your clients. Make sure to use the correct path for the cert and secrets files in the command.
   • step certificate create sensor1 sensor1.pem sensor1.key --ca .step/certs/intermediate_ca.crt --ca-key .step/secrets/intermediate_ca_key --no-password --insecure --not-after 2400h
   • step certificate create sensor2 sensor2.pem sensor2.key --ca .step/certs/intermediate_ca.crt --ca-key .step/secrets/intermediate_ca_key --no-password --insecure --not-after 2400h
   • step certificate create actuator1 actuator1.pem actuator1.key --ca .step/certs/intermediate_ca.crt --ca-key .step/secrets/intermediate_ca_key --no-password --insecure --not-after 2400h
   • step certificate create actuator2 actuator2.pem actuator2.key --ca .step/certs/intermediate_ca.crt --ca-key .step/secrets/intermediate_ca_key --no-password --insecure --not-after 2400h
3. To view the thumbprints for the client certificates, run the following step command.
   • step certificate fingerprint sensor1.pem
   • step certificate fingerprint sensor2.pem
   • step certificate fingerprint actuator1.pem
   • step certificate fingerprint actuator2.pem

Create a namespace

Use the command to create a namespace

• az eventgrid namespace create -g rg1 -n ns1 --location westus2 --topic-spaces-configuration "{state:Enabled}"

Create clients

Use the following command to create the client resources. Replace the [Client Thumbprint] in the commands below with the thumbprints that are retrieved from step 3 from the Generate certificates instructions.

• az eventgrid namespace client create -g rg1 --namespace-name ns1 -n sensor1 --authentication-name sensor1 --attributes "{'type':'sensor'}" --client-certificate-authentication "{validationScheme:ThumbprintMatch,allowed-thumbprints:[Client Thumbprint]}"
• az eventgrid namespace client create -g rg1 --namespace-name ns1 -n sensor2 --authentication-name sensor2 --attributes "{'type':'sensor'}" --client-certificate-authentication "{validationScheme:ThumbprintMatch,allowed-thumbprints:[Client Thumbprint]}"
• az eventgrid namespace client create -g rg1 --namespace-name ns1 -n actuator1 --authentication-name actuator1 --attributes "{'type':'actuator'}" --client-certificate-authentication "{validationScheme:ThumbprintMatch,allowed-thumbprints:[Client Thumbprint]}"
• az eventgrid namespace client create -g rg1 --namespace-name ns1 -n actuator2 --authentication-name actuator2 --attributes "{'type':'actuator'}" --client-certificate-authentication "{validationScheme:ThumbprintMatch,allowed-thumbprints:[Client Thumbprint]}"

Create client groups

Use the following command to create client groups: one for the sensors and one for the actuators.

• az eventgrid namespace client-group create -g rg1 --namespace-name ns1 -n sensors --group-query "attributes.type='sensor'"
• az eventgrid namespace client-group create -g rg1 --namespace-name ns1 -n actuators --group-query "attributes.type='actuator'"

Create topic spaces

Create a topic space to configure the MQTT will topics that the sensors use to publish and another one to configure the topic filter to which the actuators subscribe.

• az eventgrid namespace topic-space create -g rg1 --namespace-name ns1 -n lwtPub --topic-templates 'clients/${client.authenticationName}/status'
• az eventgrid namespace topic-space create -g rg1 --namespace-name ns1 -n lwtSub --topic-templates 'clients/+/status'

Create permission bindings

Create a permission binding to grant the sensors access to publish on their topic space, and another one to grant the actuators access to subscribe to their topic space.

• az eventgrid namespace permission-binding create -g rg1 --namespace-name ns1 -n sensorsPub --client-group-name sensors --permission publisher --topic-space-name lwtPub
• az eventgrid namespace permission-binding create -g rg1 --namespace-name ns1 -n actuatorsSub --client-group-name actuators --permission subscriber --topic-space-name lwtSub

Clients configuration through MQTTX

You can use MQTTX app to model your client samples and the data flow.

1. In MQTTX, Select New Connection to create new connections for each of the clients and configure LWT settings for the sensors.
   • For all clients, use the following general settings.
     • Name: <client name>
     • Client Id: <client name>
     • Username: <client name>
     • Hostname: <namespace name>.<namespace region>.ts.eventgrid.azure.net
     • Port: 8883
     • Toggle SSL/TLS to on.
     • Toggle Auto Reconnect to off.
     • Select Certificate as CA or Self signed certificates.
     • Provide the path for client certificate file.
     • Provide the path for the client key file.
   • For sensor1 and sensor2, use the following Last Will and Testament settings.
     • Last-Will Topic: clients/<client name>/status
     • Last-Will QoS: 1
     • Last-Will Payload: offline
   • Select Connect
2. For actuator1 and actuator2, subscribe to the will topic by clicking on New Subscription in the Subscriber pane. Enter clients/+/status in the Topic textbox and select 1 as the QoS. Select the Confirm button.
3. Disconnect sensor1 and sensor2, and you will notice that the will message was not sent since the clients disconnected gracefully through the MQTT DISCONNECT packet.
4. Select sensor1 and sensor2 in the Connections pane. Right-click and select New Window. In the new window, select the Connect buttons.
5. Simulate an abrupt disconnection by closing the new windows that you just opened.
6. You should be able to see the will message received by the actuator1 and actuator2 as shown below.

Summary

Event Grid's MQTT Broker capability now supports Last Will and Testament (LWT) for enhanced reliability among MQTT clients. This feature ensures that MQTT clients are notified in case of unexpected disconnections of other MQTT clients, allowing them to adapt and maintain system stability. Event Grid is committed to ongoing investment in MQTT features to align with MQTT specifications and better support evolving IoT scenarios, ensuring continuous enhancement to meet industry demands.

References

You can learn more about Azure Event Grid by visiting the links below. If you have questions or feedback, you can contact us at askmqtt@microsoft.com.

• Event Grid Overview
• Overview of the MQTT broker feature in Azure Event Grid
• MQTT features support in Event Grid
• Quickstart: Publish and subscribe to MQTT messages on Event Grid Namespace