Azure Data Factory Updated Pipeline Designer

Hi ,

I think the changes are good and I don't have any issues with them.  It's always good to see ongoing improvements to the environment.  I'm wondering when the activity limit of 40 will be removed (https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits#data-factory-limits) .  If ADF is every going to be an enterprise tool to replace SSIS then this limit will have to go.  For me it's important to log all activity before and after each activity.  I normally have a success log and error log.  This really means that every activity has 3 components.  This reduces the total activates down to around 14.  When you through in say a condition or a few set variables the number of true activities is very small.  My pattern to get around this limitation is that almost every activity is its own pipeline.

I hope the team can understand the frustration when trying to develop large ADF's with these limitations.  My current project is working with Microsoft consulting to develop a migration into Dynamics 365 and while I am using an ADF Microsoft is using SSIS because of the limits in ADF.

cheers

Greg 

_MartinB , Synapse will automatically get these changes when we release them. 

Hi Mark Kromer ,

Are there plans to bring those changes to Synapse as well?

How can we limit the amount if IP Addresses we come from when using "Data Flows" in Azure Data Factory?
When just using scripts in a pipeline we can for example use the "small" amount of public IP's we have in Azure Data Factory West Europe segment BUT! when using Data Flow this is not member of the same IP Segment, we are then being forced to use Azure West Europe which is a enormous IP Segment in comparison with thousands of IP's.

We are using services like Snowflake etc. and one would like to whitelist our Azure Services on just a few Public IP's.

Possible?

Big thanks for your work!

Regards, Magnus

RayisImayev thanks for your feedback! 

#2 goes hand in hand with #1. The gesture for adding port icon is now used for showing/adding activities. 

#3: There is the option (top right gesture) to collapse the container box so you get back to the original size for exactly the reason you mentioned. Please try it out from the experimental preview to get a feel. Many customers love the ability to see what's inside the container, we also added the collapse/expand option based on preview customer feedback. 

Thank you for all the updates to the ADF product, I'm being positive with my feedback towards them:

1. The activity connectors: good update, I liked it!
2. The location of the current output icon: I don’t see a big value in this change. A decision to add new activity is always a precalculated step. I spend more time adjusting and configuring new activities after adding them to the pipeline. I was ok to drag new activities from the list knowing where to place them and dedicate all my efforts to configuring it correctly.
3. Updated container objects: It’s a big NO from me and if this becomes a default behaviour for any containers in ADF pipelines, it would be a discouragement. You always “fight” with a limited browser window space to visually locate all the pipeline activities. When all of a sudden the containers will be visually “exploded”, this will eat up this limited window space even more. I don’t need to see the content of the container every time I open my pipelines, I solve this issue by naming and describing this container meaningfully. Also, it would help to have an option to collapse and expand a container part of the code, so that ADF will remember what container I kept collapsed or expanded. Currently behaviour: I collapse a container, but when I open my pipeline, it gets expanded again: not the best user experience.