Data Ingestion in Azure Data Factory

Occasional Visitor

I need guidance to ingest data from multiple tables in different RDBMS tables/.csv files (salesforce,SFTP,...)


Create a configuration table with column (schema-name, table-name, optional column list, filter condition)

·         In ADF pipeline

o   Read data from configuration-table

o   create [Copy-each-table] activity

§  create [copy-table] activity

·         Create ADLS sink dataset to copy data into HD-Insight ADLS.



·         By this, the ADF pipeline will execute and ingest multiple-tables data in parallel

·         Add a record in the configuration-table when a new table to be ingested.


Kindly provide guidance...Thanks !!



0 Replies