Blog Post

Azure Data Factory Blog
1 MIN READ

Parquet format support added to Wrangling Data Flow in Azure Data Factory

Gaurav Malhotra's avatar
Gaurav Malhotra
Icon for Microsoft rankMicrosoft
Feb 05, 2020

Wrangling Data Flow (WDF) in ADF now supports Parquet format. You can have your data stored in ADLS Gen2 or Azure Blob in parquet format and use that to do agile data preparation using Wrangling Data Flow in ADF

 

Create a parquet format dataset in ADF and use that as an input in your wrangling data flow

 

 

You can then use the parquet format dataset as an input to your Wrangling Data Flow to do agile data preparation at cloud scale via spark execution

 

 

 

 

Learn more about using Wrangling Data Flow to do data preparation at cloud scale here.

 

Published Feb 05, 2020
Version 1.0
azure data factory
Big Data Analytics
Wrangling Data Flows
  • mcole360's avatar
    mcole360
    Copper Contributor
    Feb 07, 2020

    Is managed identify auth planned for the near term? My understanding is that if your ADLS instance is in a VNet, service principal auth can't be used with an Azure Auto resolve IR, so until MI auth is supported this feature can't be used.

  • sonnychilds's avatar
    sonnychilds
    Copper Contributor
    Feb 07, 2020

    Gaurav Malhotra, it is exciting to see Parquet in Power Query. Do you know if this same capability is coming to Power Query in the Power Platform, such as Power BI dataflows? Or is this feature dependent on ADF-specific parquet interpretation tech?