Reliable blob processing using Azure Logic Apps: Recommended architecture

Understanding the Limitations of In-App Blob Triggers

The Logic App Blob - In-App trigger https://learn.microsoft.com/en-us/azure/logic-apps/connectors/built-in/reference/azureblob/ is built upon the same architecture as the Azure Function App Blob trigger and therefore inherits similar features and limitations. Notably, the in-app blob trigger uses polling as its core mechanism to detect changes in blob containers Polling and latency.

Key behaviors and limitations include:

• Polling-Based Detection: The trigger polls the associated container rather than subscribing to events. This hybrid mechanism combines periodic container scans with Azure Storage analytics log inspection.
• Batch Scanning: Blobs are scanned in batches of up to 10,000 per interval, using a continuation token to track progress.
• Best-Effort Logging: The system relies on Azure Storage logs, which are generated on a best-effort basis. These logs are not guaranteed to capture all blob events.
• Event Loss Risk: Under specific conditions — such as high blob throughput, delayed log generation, or log retention settings — some blob creation events may be missed.
• Latency and Missed Triggers: Due to the asynchronous and non-deterministic nature of polling and logging, there may be latency in triggering workflows, and in some cases, triggers may not fire at all.

Given these limitations, relying solely on the in-app blob trigger may not be suitable for mission-critical scenarios that require guaranteed blob processing.

Recommended Alternative Approach: Queue-Based Reliable Triggering

To ensure reliable and scalable blob processing without missing any events, we recommend an alternative architecture that introduces explicit event signaling via Azure Storage Queues. This approach provides greater reliability and control, especially in high-throughput or mission-critical scenarios. For more info on this pattern please refer Claim-Check pattern - Azure Architecture Center | Microsoft Learn

In this approach, the Logic App uses a Storage Queue trigger, and the source system must send a queue message whenever a blob is uploaded.

1. The source system should implement the following steps:
   • File path or blob name
   • Filename
   • Additional context (for example, customer ID or upload timestamp)
1. Blob Upload
   Upload the file to an Azure Blob Storage container as usual.
2. Queue Message Creation (Source responsibility)
   Immediately after the blob upload completes, the source system constructs a queue message containing relevant blob metadata, such as:
3. Send Message to Azure Storage Queue
   The source system sends the constructed metadata message to a pre‑configured Azure Storage Queue.
   This queue message acts as the explicit trigger for downstream processing.
2. Trigger Logic App via Queue Message
   The Logic App is configured with a Storage Queue trigger and is triggered only when a new message arrives in the queue.
3. Process Blob via Metadata
   Upon receiving the queue message, the Logic App:
• Parses the blob metadata (for example, file path and filename)
• Uses the “Get blob content” action (or equivalent) to retrieve and process the actual blob content from Azure Blob Storage

Benefits of This Approach

• Guaranteed Triggering: The Logic App is reliably triggered by a queue message, ensuring no blob is missed.
• Decoupled Workflow: This architecture decouples blob creation from processing logic, enabling better scalability and fault isolation.
• Resilience and Retry Support: Azure Storage Queues provide built-in retry capabilities, message visibility timeouts, and dead-lettering to handle transient failures gracefully.
• Better Monitoring and Control: You can monitor the queue depth, message age, and processing status to track workflow health and throughput.

Alternative Approach: Event Grid Integration

Another reliable method for handling blob events is using Event Grid with Single Tenant Logic App workflow endpoints.

Using Single Tenant Logic App Workflow Endpoint as Event Grid Subscription method allows you to handle events from either system or custom Event Grid topics. However, Event Grid topics won't be able to deliver events on private endpoints. Hence, if you have Logic App enabled with Private Endpoint, you wouldn't be able to use the workflow endpoints as Event Grid subscriptions

In this approach, blob processing is triggered by events. Here, Azure Blob Storage acts as the event publisher, and Event Grid delivers blob lifecycle events to the Logic App endpoint, which then processes the blob.

Steps to Implement Event Grid Integration:

1. Create Event Grid Subscription
   Configure an Event Grid subscription on the Azure Storage account or specific blob container to listen for blob events
2. Configure Logic App Endpoint
   Configure the Logic App Standard workflow endpoint (HTTP or Event Grid trigger) as the event subscriber.
   The endpoint must be publicly reachable, or an intermediate component must be used if private networking is required.
3. Handle Subscription Handshake (Logic App responsibility)
   For Single‑Tenant Logic Apps, implement explicit Event Grid subscription validation (handshake) to confirm that the Logic App endpoint accepts events from Event Grid.
   • When an Event Grid subscription is created, Event Grid sends a SubscriptionValidationEvent to the subscriber endpoint.
   • The Logic App must:
     1. Detect the Microsoft.EventGrid.SubscriptionValidationEvent
     2. Extract the validation Code from the request payload
     3. Return an HTTP 200 OK response

        For detailed implementation guidance and example, see: Using Single‑Tenant Logic App Workflow Endpoint as Event Grid Subscription

4. Trigger Logic App via Events
   Once the subscription is active, Event Grid automatically pushes blob events to the Logic App endpoint whenever a blob is created, updated.
5. Process Blob Based on Event Data
   Upon receiving the event payload, the Logic App:
   1. Parses event data (container name, blob name, blob URL, event type)
   2. Retrieves the blob using actions such as “Read blob content”
   3. Processes the blob according to business logic

Benefits of Event Grid Integration

• Event-Driven Architecture: Provides real-time event handling, reducing latency compared to polling mechanisms.
• Scalability: Event Grid can handle many events efficiently.
• Flexibility: Supports various event sources and custom topics.

Note: If you have Logic App enabled with Private Endpoint, refer to how to trigger Azure Function from Event Grid over virtual network for alternative configurations