Forum Discussion

mikejminto's avatar
mikejminto
Copper Contributor
Apr 05, 2024

How to rigger an azure synapse pipeline after a file is dropped into an azure file-share.

I am currently looking for ideas to trigger an azure synapse pipeline after a file is dropped into an azure file-share. I feel that azure functions might be a suitable candidate to implement this. Azure synapse pipelines don't natively support this at the moment. Microsoft do offer a custom events trigger extension capability. However, so far I have found very little evidence demonstrating how to leverage this capability to trigger my synapse pipeline. Any assistance with approaches to solving this would be greatly appreciated. Thanks.

1 Reply

  • MoritzG's avatar
    MoritzG
    Iron Contributor
    Jul 07, 2025

    To my knowledge, you are right: Azure File Shares do not support Event Grid or native triggers like Azure Blob Storage, and Synapse pipelines can not natively watch Azure Files. However, Azure Functions can fill that gap:

    If you are trying to trigger an Azure Synapse pipeline when a file is dropped into an Azure File Share, one (perfectly fine) option is to use an Azure Function that monitors the file share and directly calls the Synapse pipeline using its REST API.

    Here is how to approach this:

    1. Create an Azure Function to monitor the file share:

    • Use a timer trigger (e.g., run every minute).
    • Use the Azure.Storage.Files.Shares SDK to connect to the file share.
    • List the contents of the target directory.
    • Compare current contents to a cache (in memory, blob, or durable entity) to detect new files.
    • When a new file is detected, proceed to call the Synapse pipeline.

    2. Call the Synapse pipeline using the REST API:
    Make an HTTP POST request like this:
    https://<your-synapse-workspace-name>.dev.azuresynapse.net/pipelines/<pipeline-name>/createRun?api-version=xxx

    Authentication:

    • You can authenticate using a managed identity or a service principal.
    • If using managed identity, assign the Function App a role like "Synapse Contributor" or "Synapse Pipeline Run Contributor".
    • Use Azure AD to obtain an access token and attach it to the Authorization header.


    If your setup allows it, you could also switch to Azure Blob Storage - that would be the easiest and cleanest Option. It works with Event Grid out of the box, so you can trigger Synapse pipelines automatically when a file is uploaded (no polling or extra code needed).
    If that is not possible, I would stick to the solution above (using Azure Functions).

    Best regards! 

Resources