Blog Post

Azure Data Factory Blog
1 MIN READ

Azure Data Factory adds support for XML format

Linda_Wang's avatar
Linda_Wang
Icon for Microsoft rankMicrosoft
Jul 17, 2020
We are glad to announce that now in Azure Data Factory, you can extract data from XML files by using copy activity and mapping data flow. With such capability, you can either directly load XML data t...
Updated Jul 17, 2020
Version 1.0
Azure Data Integration
Copy Activity
Mapping Data Flows
Mysticode's avatar
Mysticode
Copper Contributor
Jul 29, 2020

WillT985 I must be missing something. This is what I have so far.

 

1. Linked Service: HTTP type, base URL is my external API. Basic auth, username, password

2. A dataset, type HTTP/XML type, connected to my Linked Service above. I have entered a base URL. It connects without issue

3. Pipeline activity of "Copy data"

  • Source dataset: the HTTP/XML dataset from above
  • Request method: POST
  • Additional headers: Content-Type: application/soap+xml;charset=UTF-8;action="http://outsidesoft.com/webservices/API/Authenticate"
  • Request body: <soap:abc123....><soap:/Header/>....<soap:Body><api:Authenticate>...etc etc

What are the next steps? When I try to set up a Sink dataset, I choose my data lake, but then when to choose a format is when I run into a problem. I want to save the file in a format I can parse. If I choose CSV, then run a debug, I get the following error:

ErrorCode=SchemaMappingFailedInHierarchicalToTabularStage,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Failed to process hierarchical to tabular stage, error message: One or more errors occurred.,Source=Microsoft.DataTransfer.ClientLibrary,'",

 

What are the proper steps to take to get a sink working?

My future plans are to do another "Copy data", and set up the variable to get the token as you demo'd above, and then make subsequent API calls with the token and save that newly retrieved data to the data lake in their own tables.

 

Thank you all