Blog Post

Azure Data Factory Blog
1 MIN READ

Synapse Data Flows Enables Direct Workspace DB Connector

Mark Kromer's avatar
Mark Kromer
Icon for Microsoft rankMicrosoft
Aug 14, 2021

Azure Synapse Analytics Data Flows has enabled Direct Workspace DB Connector as a public preview. This new connector type in data flows enables data engineers to quickly and easily build ETL processes using Spark-based lake databases in Synapse without the need to first create linked services or datasets.

 

https://docs.microsoft.com/en-us/azure/data-factory/data-flow-source#workspace-db-synapse-workspaces-only

https://docs.microsoft.com/en-us/azure/data-factory/data-flow-sink#workspace-db-synapse-workspaces-only

 

By using Synapse Analytics for your end-to-end big data analytics projects, you can now define lake database tables using Spark Notebooks, then open the visual data flow designer graph environment and immediately access those tables and data for ETL pipeline building. This new Workspace DB process eliminates the need to create ADF-based linked services and datasets inside of your Synapse workspace studio UI because Synapse is providing the complete integrated experience for data engineers in a single pane of glass.

 

Updated Aug 13, 2021
Version 1.0

2 Comments

  • Hello Mark Kromer , 

     

    This is a great feature in Azure Synapse.

    Is this feature going to be available in Azure Data Factory also ? 

     

    Thanks.

     

     

     

  • _MartinB's avatar
    _MartinB
    Iron Contributor

    Hi Mark Kromer,
    I do have some managed and unmanaged Spark Tables registered which also show up in the Data section of Synapse Studio as Workspace DBs.

    Nevertheless, the new "Workspace DB" tile does not show up in my Data Flow sink.
    Is the preview feature already live? Do I need to activate it somehow?

     

    Update 2021-08-18: now it is also available in West EU