Oct 26 2022 04:16 AM
Synapse analytics vs ADF pipeline using SQL server vs ADF pipeline using CSV ? Cost-effective?
Hi folks,
My task here is like, i wanted to delete some stale data in my dataverse table and display those deleted records into a power BI report and main thing is this should be cost-effective.
for this I found 3 different ways to achieve this,
1. Data verse table to Power BI using Azure synapse analytics(storing stale data into azure synapse workspace)
2. Data verse table to Power BI using ADF pipeline with azure SQL server(to store deleted records):
In detail, I have stored dataverse table data into azure data lake. once data is stored into a csv file within the data lake
we have created ADF pipeline and dataflow to read that csv file and applied where clause using data flow filter and stored the filtered records into a Azure SQL DB and shown those records into power BI.
3. Data verse table to Power BI using ADF pipeline with CSV file(to store deleted records):
In detail, I have stored dataverse table data into azure data lake. once data is stored into a csv file within the data lake
we have created ADF pipeline and dataflow to read that csv file and applied where clause using data flow filter and stored the filtered records into another CSV file and shown those records into power BI.
All the ways will work
But for a 1GB of table stale data what way is much cost effective? if you can calculate what will be the approx. cost for each way?
or how to calculate the cost for each ways?
Thanks in advance
Nov 06 2022 11:07 PM
Dec 21 2022 02:25 PM
@charlesmicheloo7, Pricing estimates are subject to many other variables other than just the amount of data collected from data sources like Dataverse tables, but here are some estimates based on a series of assumptions I've made. If you have a Microsoft Power Platform or Microsoft 365 license where Azure Synapse Link for Dataverse is included, this option will be cheaper than adding additional components such as Data Factory and Azure SQL Database for data processing and storage.