Monitoring Azure Storage operations is crucial for ensuring performance, compliance, and security. Azure provides various options to collect and route diagnostic logs. One powerful option is sending logs to Azure Event Hub, which allows real-time streaming and integration with external tools and analytics platforms.
In this blog, we’ll walk through setting up diagnostic logging for an Azure Storage account with Event Hub as the destination, and then demonstrate how to analyse incoming logs using Service Bus Explorer.
Prerequisites
Before we begin, make sure you have the following set up:
1. Azure Event Hub Configuration
- An Event Hub namespace and instance set up in your Azure subscription.
2. Service Bus Explorer Tool
We'll use Service Bus Explorer to connect to Event Hub and analyse log data.
Download and Setup:
- Go to the official GitHub page:
https://github.com/paolosalvatori/ServiceBusExplorer - Download the latest release .zip file from the Releases section.
- Extract the contents and launch ServiceBusExplorer.exe (no installation needed).
Note:- This is a Windows-only tool. Make sure .NET runtime is installed on your system.
3. Event Hub Connection String
- You’ll need a connection string with appropriate permissions to connect via Service Bus Explorer:
- Azure Portal → Navigate to your Event Hub Namespace → Shared Access Policies
- Select RootManageSharedAccessKey with Managed rights
- Copy the Connection string–primary key
Note:- Ensure the connection string includes the Entity Path if you're targeting a specific Event Hub.
4. Ensure Diagnostic Logging is Enabled for Azure Storage
To stream logs into Event Hub, make sure that diagnostic logging is configured properly on your Azure Storage account.
Steps:
- Navigate to your Storage Account in the Azure Portal.
- Go to Monitoring > Diagnostic settings.
- Click Add diagnostic setting or edit an existing one.
- Select the required log categories: Blob, Table, Queue, File (as needed).
- Set Event Hub as the destination.
Let's take a look at the steps below for configuring Service Bus Explorer and Reviewing the logs ahead
Step 1: Connect to Event Hub Using Service Bus Explorer
To analyze the streamed logs, we will use Service Bus Explorer, a powerful tool for inspecting messages within Azure Event Hub.
Open Service Bus Explorer.
Click on File > Connect.
Step 2: Provide Event Hub Connection String
- Paste your Event Hub-compatible connection string at the Namespace level from the Portal which has Manage permissions and paste it in the right-hand text field under Connection Settings and click on Save.
Step 3: View Available Event Hubs
- Once connected, you'll see a list of Event Hubs under your namespace.
- Expand the Event Hub you configured for diagnostic logs.
Step 4: Start Listening to the Consumer Group
- Right-click on your Consumer Group (usually $Default).
- Select "Create Consumer Group Listener" to begin listening for incoming data.
Step 5: Enable Verbose Logging
Check the Verbose option to view more detailed information about the incoming messages, including metadata.
Step 6: View and Analyse Events
- Navigate to the Events tab at the top.
- Click on Start to begin streaming live log events.
You’ll now see:
- Metadata: Sequence Number, Offset, Enqueued Time (UTC), Partition Info.
- Payload: The actual log data coming from Azure Storage (in JSON format).
🧠 Tip: You can copy and analyse this payload further using tools like Power BI, Stream Analytics, or even a custom parser.
Conclusion
Streaming diagnostic logs to Event Hub gives you flexibility in how you handle monitoring data. Whether you're integrating with external SIEM solutions, triggering alerts, or performing deep analytics, Event Hub provides a real-time backbone for diagnostics.
Using Service Bus Explorer, you can quickly validate and analyse the incoming logs to ensure your setup is working and to inspect what's being captured.
Reference Link :-
https://learn.microsoft.com/en-us/azure/azure-monitor/essentials/diagnostic-settings
Microsoft