Blog Post

Azure Data Factory Blog
1 MIN READ

XML added to Parse transformation in ADF and Synapse Data Flows

Mark Kromer's avatar
Mark Kromer
Icon for Microsoft rankMicrosoft
May 10, 2021

The Parse transformation in Azure Data Factory and Synapse Analytics data flows allows data engineers to write ETL data transformations that take embedded documents inside of string fields and parse them as their native types. For example, you can set parsing rules in the Parse transformation to handle JSON and delimited text strings and transform those fields into complex types. Now, we've updated Parse to also understand XML as a source type in your incoming string data.

 

In this example, I have a text delimited CSV data source. Since this is a simple text file, the embedded XML document in the column labeled "xml" is read as a string, so I am unable to treat it like a hierarchical structure. However, by adding the Parse transformation, I can select XML as my incoming embedded type and define the structure "customers" as the new column that is now hierarchical:

 

 

  • Source XML data: <Customers><Customer>122</Customer><CompanyName>Great Lakes Food Market</CompanyName></Customers>

    • Expression: (Customers as (Customer as integer, CompanyName as string))
Updated May 10, 2021
Version 1.0

5 Comments

  • Aart-Hoevenberg's avatar
    Aart-Hoevenberg
    Copper Contributor

    Hi,

     

    It looks as if you are the only one working with xml parsing in synapse :smile:. Therefore this question. Is it possible to extract repeating groups from an XML column into separate columns (the columns being 'Where do you live?' -> City, 'What's your age?' -> Age in the example)? 

     

    <questionnaire>

           <question>Where do you live?</question>

           <answer>New York</answer>

           <question>What's your age?</question>

           <answer>57<answer>

    <questionnaire>

     

    | City          | Age |

    | New York | 57 |

     

     

     

    Kind regards

     

  • Henk_Standhardt's avatar
    Henk_Standhardt
    Copper Contributor

    It is helpful if there was more information available about the expressions to use for the different XML elements and structures...

    It needs an updated video on the youtube channel about the xml specifics of the parse transformation (this one only shows json and csv: https://www.youtube.com/watch?v=r7O7AJcuqoY)

  • dimeji98's avatar
    dimeji98
    Copper Contributor

    This NOT HELPFUL AT ALL when trying to grab the element's attribute value or working with complex XML structures @

    • ckrig's avatar
      ckrig
      Copper Contributor

      Agreed, this was less than helpful - after reading this post the reader is left less able to perform any useful XML data transformations. This blog post should come with a warning.