Forum Discussion

frederic MICHALAK's avatar
frederic MICHALAK
Copper Contributor
Feb 28, 2022

Push data from SQL to Hadoop with SSIS

Hello, 

We have a SQL 2019, a SSIS 2019 (configured in scaleout) and a Cloudera HDFS Datalake 2.6.

We want to push data from SQL DWH to HDFS. We did not have implemented Polybase because the datalake is setup with a knox gateway.

- First we have tried to push to Hive thru ODBC connector is SSIS, but performance are ugly (100 rows = 8 minutes)

- Then we have tried to push directly to HDFS but all test connection are failing despite browsing https://myurl.com/gateway/default/webhdfs/v1/dev/raw/ssis_poc.db?op=LISTSTATUS is working fine (asking a credential obviously).

Any idea of what are we missing ?  

No RepliesBe the first to reply

Resources