I need guidance to ingest data from multiple tables in different RDBMS tables/.csv files (salesforce,SFTP,...)
Create a configuration table with column (schema-name, table-name, optional column list, filter condition)
· In ADF pipeline
o Read data from configuration-table
o create [Copy-each-table] activity
§ create [copy-table] activity
· Create ADLS sink dataset to copy data into HD-Insight ADLS.
Benefits
· By this, the ADF pipeline will execute and ingest multiple-tables data in parallel
· Add a record in the configuration-table when a new table to be ingested.
Kindly provide guidance...Thanks !!