Azure synapse analytics: understand how to find cost per pipeline

hi kavitha_eswarappa​ adding to Kidd_Ip​ That's a great question - understanding cost per pipeline in Azure Synapse Analytics can be tricky because costs are not directly itemized by pipeline in Azure billing. However, you can achieve a good approximation using a combination of Azure Monitor logs, Azure Cost Management, and custom tagging.

Here's how you can approach it:

Enable diagnostic logging for Synapse

Enable diagnostic settings for your Synapse workspace and send logs (especially PipelineRuns and ActivityRuns) to Log Analytics.

Go to your Synapse workspace ‚Üí Diagnostic settings

Choose to send logs to a Log Analytics workspace

Enable these categories:

o   PipelineRuns

o   ActivityRuns

o   IntegrationRuntimeMetrics

This gives you detailed telemetry per pipeline and activity, including duration and compute utilization.

Use tagging for cost attribution

Add tags to your Synapse resources or even dynamically to pipeline runs (for example, customer name or project ID).

Example tag scheme:

Customer: ContosoProject: DataMigrationPipeline: SalesDataLoad

Tags propagate into Azure Cost Management, so you can later group or filter costs per customer, project, or pipeline.

Correlate logs with cost data

In Cost Management + Billing, you'll see cost data by resource (for example, your Integration Runtime).

Use Log Analytics queries to calculate execution time and run frequency for each pipeline, then multiply by the Integration Runtime cost rate.

Sample Log Analytics query:

AzureDiagnostics| where Category == "ActivityRuns"| summarize    TotalRuns = count(),    TotalDuration = sum(todouble(DurationInMs))/1000/60 by PipelineName| order by TotalDuration desc

Then cross-reference with Integration Runtime cost:

Azure-SSIS IR: billed by vCore-hour

Synapse pipeline activities (Data Movement, Lookup, Copy): billed per run + data volume

Use Azure Cost Management APIs

You can pull detailed cost data via Cost Management Query API and join it with pipeline metadata to generate a per-pipeline cost dashboard (Power BI or Log Analytics Workbook).

Alternative: Azure Purview + Tags

If your pipelines are registered in Microsoft Purview, you can leverage metadata to track which datasets or pipelines belong to which customer/project - and align those to cost tags.

Ref links:

https://learn.microsoft.com/en-us/azure/synapse-analytics/monitor-synapse-analytics