Introducing the Flowlets preview for ADF and Synapse

Published Dec 07 2021 11:33 PM 4,838 Views
Microsoft

Next week, we will roll out the public preview for "Flowlets", a new feature of Mapping Data Flows found in Azure Data Factory and Azure Synapse Analytics. Flowlets will enable data engineers to design portions of new data flow logic, or to extract portions of an existing data flow, and save them as separate "Flowlet" artifacts inside your factory or workspace. Flowlets can then be re-used inside other data flows. 

 

 

The design experience to add new sources to your data flows will now allow for optional Flowlets as a source. Clicking on the Add Source directly inside the new source tile will provide the same 1-click action as today for adding a dataset source.

 

flowlet7.png

Flowlets can also be added inline in the data flow as a custom transformation.

 

flowlet3.png

 

Your saved Flowlets will appear in the Data Flows category in the ADF and Synapse object explorers. You will see an option for both Data Flow and Flowlet now from the new Data Flow menu.

 

flowlet1.png

MarkKromer_0-1638948410134.png

To create a new Flowlet from an existing data flow, you can shift-click or turn on the selector button then highlight the area of your graph which you choose to turn into a flowlet.

 

flowlets-create.png

 

The design surface for Flowlets is the same as the Data Flow canvas with a few exceptions. There are 2 new transformations: Input and Output. This is how you will define the contract or the argument for your Flowlet as inputs and return values. While Output replaces Sink from Data Flows, the Input also include the option to point to an inline dataset which you can use as a Source Flowlet in your data flows.

 

flowlet2.png

 

flowlet6.png

 

We'll post instructional videos with samples next week, so stay tuned for more!

6 Comments
Co-Authors
Version history
Last update:
‎Dec 09 2021 08:26 PM
Updated by: