Today, we're excited to announce that compression for Kafka clients is generally available in Azure Event Hubs.
Azure Event Hubs is a cloud native streaming service enabling you to build scalable, durable and low-latency workflows with massive volumes of event data with ease. As you onboard more workloads to Azure and build them around Azure Event Hubs, your bandwidth and storage requirements may scale exponentially.
Kafka compression can help with this by reducing the data payloads that are stored and transmitted across your architecture, thus reducing network bandwidth and storage requirements and costs, while still keeping the programming model simple.
To utilize Kafka compression, you must set the below setting in the Kafka producer configuration properties -
compression.type = gzip
No changes are required on the Kafka consumer side, since the compression information is made available in the message header and the consumer can automagically uncompress these and make it available for processing.
Learn more
To learn more about Kafka compression in Azure Event Hubs, please refer to the documentation.
Microsoft
