Forum Discussion
ADF unable to ingest partitioned Delta data from Azure Synapse Link (Dataverse/FnO)
We are ingesting Dynamics 365 Finance & Operations (FnO) data into ADLS Gen2 using Azure Synapse Link for Dataverse, and then attempting to load that data into Azure SQL Database using Azure Data Factory (ADF).
This is part of a migration effort as Export to Data Lake is being deprecated.
Source Details
Source: ADLS Gen2
Data generated by: Azure Synapse Link for Dataverse (FnO)
Format on lake:
Delta / Parquet
Partitioned folder structure (e.g. PartitionId=xxxx)
Destination:
Azure SQL Database
Issue Observed in ADF
When configuring ADF pipelines:
Using ADLS Gen2 dataset with:
Delta / Parquet
Recursive folder traversal
Wildcard paths
We encounter:
No data returned in Data Preview
Or runtime error such as:
“No partitions information found in metadata file”
Despite this:
The data is present in ADLS
The same data can be successfully queried using Synapse serverless SQL
Key Question for ADF / Synapse Engineers
What is the recommended and supported ADF ingestion pattern for:
Partitioned Delta/Parquet data produced by Azure Synapse Link for Dataverse
Specifically:
Should ADF:
Read Delta tables directly, or
Use Synapse serverless SQL external tables/views as an intermediate layer?
Is there a reference architecture for:
Synapse Link → ADLS → ADF → Azure SQL
Are there ADF limitations when consuming Synapse Link–generated Delta tables?
Many customers are now forced to migrate due to Export to Data Lake deprecation, but current ADF documentation does not clearly explain how to replace existing ingestion pipelines when using Synapse Link for FnO.
Any guidance, patterns, or official documentation would be greatly appreciated.