REST source and sink now available for data flows

Published Jul 30 2021 01:31 PM 6,795 Views
Microsoft

Data flows in Azure Data Factory and Azure Synapse Analytics now support REST endpoints as both a source and sink with full support for both JSON and XML payloads.

 

You can now connect to REST endpoints natively in ADF & Synapse data flows as a way to transform and process data inline with the code-free design experience. 

 

rest1.png

 

You can use the built-in ETL feature of ADF, Synapse pipelines, and data flows to perform CRUD operations on your REST sink via the Alter Row transformation for inserts, updates, upserts, and deletes.

 

rest2.png

 

https://docs.microsoft.com/en-us/azure/data-factory/connector-rest 

1 Comment
Occasional Visitor

I'm trying to use the REST connector as a Source in some Data Flows with no success. All attempts to import a schema never complete within our Synapse Workspace - we just get an indefinite spinner.  The endpoints we're using are the same endpoints we've had success with when using the HTTP connector from within the same workspace.  Is this a known issue? Any suggestions?

%3CLINGO-SUB%20id%3D%22lingo-sub-2596484%22%20slang%3D%22en-US%22%3EREST%20source%20and%20sink%20now%20available%20for%20data%20flows%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2596484%22%20slang%3D%22en-US%22%3E%3CP%3EData%20flows%20in%20Azure%20Data%20Factory%20and%20Azure%20Synapse%20Analytics%20now%20support%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fconnector-rest%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EREST%3C%2FA%3E%20endpoints%20as%20both%20a%20source%20and%20sink%20with%20full%20support%20for%20both%20JSON%20and%20XML%20payloads.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EYou%20can%20now%20connect%20to%20REST%20endpoints%20natively%20in%20ADF%20%26amp%3B%20Synapse%20data%20flows%20as%20a%20way%20to%20transform%20and%20process%20data%20inline%20with%20the%20code-free%20design%20experience.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22rest1.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F299581i8E941F80DEEDC6D7%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22rest1.png%22%20alt%3D%22rest1.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EYou%20can%20use%20the%20built-in%20ETL%20feature%20of%20ADF%2C%20Synapse%20pipelines%2C%20and%20data%20flows%20to%20perform%20CRUD%20operations%20on%20your%20REST%20sink%20via%20the%20Alter%20Row%20transformation%20for%20inserts%2C%20updates%2C%20upserts%2C%20and%20deletes.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22rest2.png%22%20style%3D%22width%3A%20341px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F299582i415E16014856FE1D%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22rest2.png%22%20alt%3D%22rest2.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fconnector-rest%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehttps%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fconnector-rest%3C%2FA%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-2596484%22%20slang%3D%22en-US%22%3E%3CP%3EData%20flows%20in%20Azure%20Data%20Factory%20and%20Azure%20Synapse%20Analytics%20now%20support%20REST%20endpoints%20as%20source%20and%20sink%20with%20full%20support%20for%20both%20JSON%20and%20XML%20payloads.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22rest1.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F299580i608BA8BE720EAA75%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22rest1.png%22%20alt%3D%22rest1.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2596484%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Data%20Factory%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20ETL%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EMapping%20Data%20Flows%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2737345%22%20slang%3D%22en-US%22%3ERe%3A%20REST%20source%20and%20sink%20now%20available%20for%20data%20flows%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2737345%22%20slang%3D%22en-US%22%3E%3CP%3EI'm%20trying%20to%20use%20the%20REST%20connector%20as%20a%20Source%20in%20some%20Data%20Flows%20with%20no%20success.%20All%20attempts%20to%20import%20a%20schema%20never%20complete%20within%20our%20Synapse%20Workspace%20-%20we%20just%20get%20an%20indefinite%20spinner.%26nbsp%3B%20The%20endpoints%20we're%20using%20are%20the%20same%20endpoints%20we've%20had%20success%20with%20when%20using%20the%20HTTP%20connector%20from%20within%20the%20same%20workspace.%26nbsp%3B%20Is%20this%20a%20known%20issue%3F%20Any%20suggestions%3F%3C%2FP%3E%3C%2FLINGO-BODY%3E
Co-Authors
Version history
Last update:
‎Jul 30 2021 01:31 PM
Updated by: