Forum Discussion

mikhailf's avatar
mikhailf
Iron Contributor
Jul 25, 2024

Export data from Log Analytics Workspace to Storage Account

Hello community, 

Could you please recommend a solution to migrate data from Log Analytics Workspace (1 table) to Storage Account?

There are about 70 million rows that should be exported.

The continuous export is not the solution here.

We were thinking about a Logic App but there is too much data.

1 Reply

  • Matthias-Braun's avatar
    Matthias-Braun
    Brass Contributor
    Jul 26, 2024

    Hi mikhailf,

    Have you ever thought about exporting with Azure Data Factory (ADF)?

     

     

    Azure Data Factory is designed for large-scale data movement and transformation. It's well-suited for extracting data from Log Analytics and loading it into a Storage Account.

    Step-by-Step Example with Azure Data Factory

    1. Create Azure Data Factory:

      • Go to the Azure portal and create a new Data Factory instance.

    2. Create Linked Services:

      • For Log Analytics Workspace:
        • In the Data Factory, go to Manage -> Linked Services -> New.
        • Choose Azure Log Analytics and provide the necessary credentials.
      • For Storage Account:
        • Again, create a new linked service for the Azure Storage Account.

    3. Create a Pipeline:

      • Go to Author -> Pipelines -> New pipeline.
      • Add a Copy Data activity.
      • In the Source, select the Log Analytics linked service and configure the query to select the desired table.
      • In the Sink, select the Storage Account linked service and configure the target dataset (e.g., CSV or Parquet).

    4. Run the Pipeline:

      • Trigger the pipeline manually or schedule it to run at a convenient time.
      • Monitor the pipeline execution to ensure all data is transferred.

     

    Please give me feedback or if you need help.

     

    Regards
    Matthias

     

     

     

     

Resources