Blog Post

Azure Data Factory Blog
2 MIN READ

You can now customize the return value from your pipeline!

Mark Kromer's avatar
Mark Kromer
Icon for Microsoft rankMicrosoft
Feb 16, 2023

When building complex workflows in the cloud with Azure Data Factory and Azure Synapse Pipelines, a very common pattern is to separate different workflow branches into child pipelines. The ADF team is super-excited to announce that we've made creating workflows even simpler with this new preview feature! Now you can customize the output of your pipeline and consume that return value from the calling pipeline.

 

In this simple example of a parent pipeline below, I have an execute pipeline activity that is calling a child pipeline. The executed pipeline will return a custom value after it has been executed. I am then consuming that return value in the subsequent activity in my parent pipeline. The 2nd activity below is a Script activity which I am using for logging and I use the expression below to grab the return value from the previous activity.

 

 

You can set the return value to either an expression or a static value. The expression to use in the parent pipeline is available as part of the pipeline expression builder:

 

 

The ADF pipeline expression builder provides quick access to the return value (preview) that I set in the child pipeline using Set Variable. Notice that I called my return value variable "myval" so that I can then access it from the parent pipeline after the previous activity expression like this: 

 

 

@activity('Execute Pipeline1').output.pipelineReturnValue.myval

 

 

The mechanism used to set the pipeline return value is from the Set Variable activity which has been enhanced to now allow an option for either a user-defined variable (previously the only option) or now you can choose "Pipeline return value (preview)". Choose a name for your return value variable and then set it either to a static value or an expression.

 

 

Resources

 

We are always open to feedback so please let us know your thoughts in the comments below or add to our Ideas forum.

 

Updated Feb 21, 2023
Version 3.0
azure data factory

18 Comments

Show More
  • MartinA2185's avatar
    MartinA2185
    Copper Contributor
    Feb 21, 2023

    Mark Kromer Thanks for your reply!
    Though, I cannot find "Expression" variable type as you mention - all I see is as the screenshot shows above.
    Or did you mean I can write the actual expression directly into the Value cell as per the screenshot?
    If it works that's great, but it would be really good to have the help of the "dynamic content expression editor" as well, just as for the "normal" variable type.

  • ade21435's avatar
    ade21435
    Copper Contributor
    Feb 21, 2023

    wow, we implemented this a year ago by setting the output of a set variable activity as a json string and looking up the result through the rest api. This is great news!

    edit: the only drawback of the above is that you cannot dynamically build the name of the pipeline from which you want to fetch the output.

  • MartinA2185's avatar
    MartinA2185
    Copper Contributor
    Feb 20, 2023

    Mark Kromer, thanks, this is great news indeed!

    Still, it seems to me that it is not yet possible to get the Dynamic Content expression GUI popup for this new variable type?

    Please correct me if I am wrong.

    For normal variables, in the Value section you get a link "Use Dynamic Content" which pops up a windows in which you can construct the expression. But I don't get that when clicking in the Value section for a "Pipeline return value" variable. What am I missing here?