Azure Data Factory orchestration allows conditional logic and enables user to take different based upon outcomes of a previous activity. In total we allows four conditional paths: Upon Success (default pass), Upon Failure, Upon Completion, and Upon Skip. Using different paths allow users to build robust pipelines and incorporates error handling in their ETL/ELT logic.
Here are two common error handling pattern we see customers use:
Both are valid ways to incorporate error handling into the pipeline. However, upon pipeline execution, they may show different outcomes. Approach #1, TRY-CATCH, shows pipeline succeeds if Upon Failure path clears, where as approach #2, DO-IF-ELSE show pipeline failed if Upon Failure path is enacted.
Technical reasons for the difference is that, Azure Data Factory defines pipeline success and failures as follows:
Applying the logic to previous examples.
Here is a table summarizing the difference
|Approach||Error Handling Defines||When Activity Succeeds||When Activity Fails|
|TRY-CATCH||Only Upon Failure path||Pipeline shows Success||Pipeline shows Success|
|DO-IF-ELSE||Upon Failure and Upon Success paths||Pipeline shows Success||Pipeline shows Failure|
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.