Forum Discussion
Azure Data Factory SAP HANA Connector poor performance
Hi Experts,
with Azure Data Factory I am carrying out tests on reading data from a SAP HANA database and storing it in Synapse tables but I am noticing disappointing performance. An example: Using Polybase, I have 3,492,246 rows x 184 columns transferred end-to-end in 16 minutes, 12 of which are just pulling from SAP HANA and writing to the staging repository. Consider that the "Physical partitions of table" flag is enabled and the SHIR is on a fully dedicated Standard D4s v5 (4 vcpus, 16 GiB memory), with a limit of 16 concurrent jobs.
I tried many parameter combinations:
- Increase Packet size (KB) up to 20960
- Increase the maximum data integration units
- Increase the degree of copy parallelism
- Increase the SHIR concurrent jobs limit
- Disable performance metrics analytics
but the final result is always almost the same, in fact sometimes it gets worse.
I also noticed that the maximum number of open connections to SAP HANA is always 4.
Just as the maximum number of used DIUs in the "Blob Storage -> Synapse Analytics" transfer is always equal to 2 and the number of used parallel copies is always 1.
Do you have any idea what can cause such poor performance?
May it depend on the SHIR VM (although I have never seen it in difficulty in terms of CPUs and RAM during flows). What can I try to investigate further?
Am I the one who has too high expectations?
Thank you very much in advance for your feedback
Luca
1 Reply
- abhisheksharma637Copper Contributor
Luca_Campeti We have similar issue , did u find the solution