CloudEvents APIs and Azure Event Grid

[As of October 31, 2020]

Original publication is in medium.

https://logico-jp.medium.com/use-cloudevents-apis-to-interact-with-azure-event-grid-32dc63518af3

Japanese edition is listed below.

https://logico-jp.io/2020/09/06/use-cloudevents-schema-in-azure-event-grid/
https://logico-jp.io/2020/10/23/tips-for-using-event-grid-sdk-to-handle-cloudevents/
https://logico-jp.io/2020/10/30/using-cloudevents-apis-to-post-events-to-azure-event-grid/
https://logico-jp.io/2020/10/31/using-cloudevents-apis-to-create-an-application-which-subscribe-an-azure-event-grid-topic/

Introduction

Azure Event Grid supports CloudEvents 1.0. And Azure Event Grid client library also supports sending/receiving events in the form of CloudEvents.

Use CloudEvents v1.0 schema with Event Grid

Introducing the new Azure Event Grid Client Libraries with CloudEvents v1.0 Support

https://devblogs.microsoft.com/azure-sdk/event-grid-client-libraries/

If Azure Event Grid is the only system which consumes and posts cloud events in your environment, Azure Event Grid SDK should be chosen. However, if several systems which consume and post cloud events have already existed in your environment and you plan to introduce Azure Event Grid, you would look for ways to interact with Azure Event Grid using industry standard APIs. In this article, I describe how to interact with Azure Event Grid using CloudEvents APIs.

Prerequisites and basic information

What is CloudEvents?

If you are not familiar with CloudEvents, please check the following URL.

CloudEvents

https://cloudevents.io/

What format does Azure Event Grid support?

As of now, Azure Event Grid supports Structured Content mode only (Binary Content mode is not supported). We have to follow JSON Event Format specification in case of creating events.

JSON Event Format for CloudEvents - Version 1.0

https://github.com/cloudevents/spec/blob/v1.0/json-format.md

What language and SDK is available?

CloudEvents SDKs are provided in several languages. In this article, sample applications are created with Java APIs for CloudEvents. Json EventFormat implementation with Jackson and HTTP Protocol Binding APIs for Jakarta RESTful Web Services allows us to create applications easier than using core APIs.

Java SDK for CloudEvents API
https://github.com/cloudevents/sdk-java

How do we post events to Azure Event Grid via CloudEvents APIs?

When posting events to Azure Event Grid through CloudEvents APIs, the following URL is helpful.

Quickstart: Route custom events to web endpoint with the Azure portal and Event Grid
https://docs.microsoft.com/azure/event-grid/custom-event-quickstart-portal

According to this document, we can post events to Azure Event Grid topic with the following URL (Access key or Shared Access Signature is required). We can get the access key in Azure Portal and via Azure CLI.

https://{topic-endpoint}?api-version=2018-01-01

Shared Access Signature is similar to access access key, but it can be configured with an expiration time. It might be suitable if access restriction to a topic or domain is required. The following URL describes how to create Shared Access Signature.

Creating a shared access signature

https://github.com/Azure/azure-sdk-for-java/blob/master/sdk/eventgrid/azure-messaging-eventgrid/README.md#creating-a-shared-access-signature

Send CloudEvents to Azure Event Grid

 

In this part, a REST client application which uses CloudEvents APIs is created in order to post events to Azure Event Grid topic. Azure Event Grid Viewer application verifies and shows these events. This viewer application is described in the following URL.

Quickstart: Route custom events to web endpoint with the Azure portal and Event Grid
https://docs.microsoft.com/azure/event-grid/custom-event-quickstart-portal

With following the document above, you can configure Event Grid topic. No special configuration is required.

Dependencies

As this client application requires JAX-RS related modules, the following dependencies should be added to pom.xml.

<!-- for CloudEvents API -->
<dependency>
  <groupId>io.cloudevents</groupId>
  <artifactId>cloudevents-http-restful-ws</artifactId>
  <version>2.0.0-milestone3</version>
</dependency>
<dependency>
  <groupId>io.cloudevents</groupId>
  <artifactId>cloudevents-json-jackson</artifactId>
  <version>2.0.0-milestone3</version>
</dependency>

<!-- for JAX-RS -->
<dependency>
  <groupId>org.glassfish.jersey.core</groupId>
  <artifactId>jersey-client</artifactId>
  <version>3.0.0-M6</version>
</dependency>
<dependency>
  <groupId>org.glassfish.jersey.inject</groupId>
  <artifactId>jersey-hk2</artifactId>
  <version>3.0.0-M6</version>
</dependency>
<dependency>
  <groupId>org.glassfish.jersey.media</groupId>
  <artifactId>jersey-media-json-jackson</artifactId>
  <version>3.0.0-M6</version>
</dependency>
<dependency>
  <groupId>org.glassfish</groupId>
  <artifactId>jakarta.json</artifactId>
  <version>2.0.0-RC3</version>
</dependency>
<dependency>
  <groupId>jakarta.ws.rs</groupId>
  <artifactId>jakarta.ws.rs-api</artifactId>
  <version>3.0.0-M1</version>
</dependency>
<dependency>
  <groupId>jakarta.json</groupId>
  <artifactId>jakarta.json-api</artifactId>
  <version>2.0.0-RC3</version>
</dependency>

Create events using CloudEvents APIs

CloudEvents::v1() method allows us to create events. JSON is used as a format of custom data, and we use the withDataContentType() method to specify application/json as Content-Type.

JsonObject jsonObject = Json.createObjectBuilder()
                            .add("message", "Using CloudEvents.io API to send CloudEvents!!")
                            .build();
 
CloudEvent ce = CloudEventBuilder.v1()
        .withId("A234-1234-1234")
        .withType("io.logico-jp.ExampleEventType")
        .withSource(URI.create("io/logico-jp/source"))
        .withTime(OffsetDateTime.now(ZoneId.ofOffset("UTC", ZoneOffset.UTC)))
        .withDataContentType(MediaType.APPLICATION_JSON)
        .withData(jsonObject.toString().getBytes(StandardCharsets.UTF_8))
        .build();

Serialization

Serialization of created JSON formatted events is required. To do so, we use “Json EventFormat implementation with Jackson” APIs. Steps for creating a client application are completed.

EventFormat format =EventFormatProvider
        .getInstance()
        .resolveFormat(JsonFormat.CONTENT_TYPE);
 
byte[] serialized = format.serialize(ce);

Create REST Client

We can follow typical ways to creating REST client. No special configuration is required. Access key of Event Grid should be set to HTTP Header. Note that not application/json but application/cloudevents+json should be set as Content-Type.

MultivaluedMap<String, Object> headers = new MultivaluedHashMap<>();
headers.add("aeg-sas-key", AEG_KEY);
Response response = ClientBuilder.newClient().target(AEG_ENDPOINT)
        .path("/api/events")
        .queryParam("api-version", "2018-01-01")
        .request("application/cloudevents+json")
        .headers(headers)
        .post(Entity.entity(serialized, "application/cloudevents+json"));

Receive CloudEvents through Azure Event Grid

 

In this part, a JAX-RS application is created to subscribe the Event Grid topic. As Azure Event Grid send events using webhook, the JAX-RS application requires a POST endpoint to listen events.

Event Grid topic which we use is already configured in the previous section (precisely, the Event Grid topic should have been already configured).

Dependencies

In this case, Helidon MP is chosen to create JAX-RS application. Needless to say, you can choose any development framework freely.

Helidon Project

https://helidon.io/

This application depends on the following modules.

<!-- for Cloud Event -->
<dependency>
  <groupId>io.cloudevents</groupId>
  <artifactId>cloudevents-http-restful-ws</artifactId>
  <version>2.0.0-milestone3</version>
</dependency>
<dependency>
  <groupId>io.cloudevents</groupId>
  <artifactId>cloudevents-json-jackson</artifactId>
  <version>2.0.0-milestone3</version>
</dependency>

Create an endpoint

We can create a JAX-RS applications without special configuration. As Azure Event Grid supports Structured Content mode only, event format is JSON. So, the sample application waits for events using JsonObject. AndEventFormat::deserialize() method is used for deserialization of events.

@Path("/updates")
@POST
public Response receiveEvent(Optional<JsonObject> obj) {
        if(obj.isEmpty()) return Response.noContent().status(Response.Status.OK).build();

    EventFormat format = EventFormatProvider
            .getInstance()
            .resolveFormat(JsonFormat.CONTENT_TYPE);

    CloudEvent ce = format.deserialize(obj.get().toString().getBytes(StandardCharsets.UTF_8));
    JsonObject customData = JsonUtil.toJson(new String(ce.getData())).asJsonObject();
    // output to console
    System.out.println("Received JSON String -- " + obj.get().toString());
    System.out.println("Converted to CloudEvent -- " + ce.toString());
    System.out.println("Data in CloudEvent -- " + customData.toString());
    return Response.noContent().status(Response.Status.ACCEPTED).build();
}

Configure OPTIONS method for enabling webhook

When configuring Azure Event Grid integration through webhook, subscriber (i.e. this JAX-RS application) has to respond to Azure Event Grid using OPTIONS method.

@Path("/updates")
@OPTIONS
public Response isWebhookEnabled() {
    return Response.ok()
            .allow("GET", "POST", "OPTIONS")
            .header("Webhook-Allowed-Origin","eventgrid.azure.net")
            .build();
}

Create Docker container and deploy to Azure App Service

After these steps are completed, we build the JAX-RS application, containerize it, and deploy it on Azure App Service.

Test

The following events are posted to Azure Event via the client application.

[{
  "specversion": "1.0",
  "id": "A234-1234-1234",
  "source": "io/logico-jp/source",
  "type": "io.logico-jp.ExampleEventType",
  "datacontenttype": "application/json",
  "time": "2020-10-31T13:54:34.308619Z",
  "data": {
    "message": "Using CloudEvents.io API to send CloudEvents!!"
  }
},
{
  "specversion": "1.0",
  "id": "A234-1234-1234",
  "source": "io/logico-jp/source",
  "type": "io.logico-jp.ExampleEventType",
  "datacontenttype": "application/json",
  "time": "2020-10-31T13:54:26.082221Z",
  "data": {
    "message": "Using CloudEvents.io API to send CloudEvents!!"
  }
}]

 

We can see these delivered events also appear in Azure Event Grid Viewer.

And from JAX-RS application side, we can observe each delivered event in App Service console log. Three logs appears per each event.

Conclusion

CloudEvents APIs allow us to post structured events to Azure Event Grid, and to handle structured events delivered from Azure Event Grid. CloudEvents APIs support various languages, and especially Java, if you are familiar with JAX-RS and Jackson, you would easily create applications with these APIs.

If Azure Event Grid were the only system which consumes and posts cloud events in your environment, Azure Event Grid SDK would be the best APIs. However, if Azure Event Grid is one of services which consume and post cloud events, this industry standard API is often more suitable than Azure Event Grid SDK.

I hope this article would be helpful for you.