Copy data to/from Azure Data Explorer using Azure Data Factory or Synapse Analytics

Published Oct 21 2021 02:22 AM 1,703 Views

The Microsoft Data Integration team has just released a new connector for Mapping Data Flows, in both ADF & Synapse. You can now natively connect to your Azure Data Explorer clusters in mapping data flows.

Mapping Data Flows provides scale-out data transformation in the cloud in Azure Data Factory and Azure Synapse Analytics. With these additional connectors, you can build ETL patterns at Spark scale in a code-free design environment without ever touching the Spark compute. Azure Integration Runtimes allow you to define the Spark environment and provide a serverless Spark compute for your data transformation pipelines.

 

Read Mapping Data Flow gets new native connectors to see an example of how to use the ADX connector as a source in a data flow to read NYC taxi data and to perform any of the hundreds of data transformation patterns to reshape and transform the data. 

 

TzviaGitlinTroyna_1-1634807306604.png

 

read more on Copy data to or from Azure Data Explorer using Azure Data Factory or Synapse Analytics

 

%3CLINGO-SUB%20id%3D%22lingo-sub-2868596%22%20slang%3D%22en-US%22%3ECopy%20data%20to%2Ffrom%20Azure%20Data%20Explorer%20using%20Azure%20Data%20Factory%20or%20Synapse%20Analytics%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2868596%22%20slang%3D%22en-US%22%3E%3CP%3EThe%20Microsoft%20Data%20Integration%20team%20has%20just%20released%20a%20new%20connector%20for%20Mapping%20Data%20Flows%2C%20in%20both%20ADF%20%26amp%3B%20Synapse.%20You%20can%20now%20natively%20connect%20to%20your%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fconnector-azure-data-explorer%3Ftabs%3Ddata-factory%23mapping-data-flow-properties%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EAzure%20Data%20Explorer%3C%2FA%3E%3CSPAN%3E%26nbsp%3B%3C%2FSPAN%3Eclusters%20in%20mapping%20data%20flows.%3C%2FP%3E%0A%3CP%3EMapping%20Data%20Flows%20provides%20scale-out%20data%20transformation%20in%20the%20cloud%20in%20Azure%20Data%20Factory%20and%20Azure%20Synapse%20Analytics.%20With%20these%20additional%20connectors%2C%20you%20can%20build%20ETL%20patterns%20at%20Spark%20scale%20in%20a%20code-free%20design%20environment%20without%20ever%20touching%20the%20Spark%20compute.%20Azure%20Integration%20Runtimes%20allow%20you%20to%20define%20the%20Spark%20environment%20and%20provide%20a%20serverless%20Spark%20compute%20for%20your%20data%20transformation%20pipelines.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3ERead%20%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fazure-data-factory%2Fmapping-data-flow-gets-new-native-connectors%2Fba-p%2F2866754%22%20target%3D%22_self%22%3EMapping%20Data%20Flow%20gets%20new%20native%20connectors%3C%2FA%3E%20to%20see%20an%20example%20of%20how%20to%20use%26nbsp%3Bthe%20ADX%20connector%20as%20a%20source%20in%20a%20data%20flow%20to%20read%20NYC%20taxi%20data%20and%20to%20perform%20any%20of%20the%20hundreds%20of%20data%20transformation%20patterns%20to%20reshape%20and%20transform%20the%20data.%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22TzviaGitlinTroyna_1-1634807306604.png%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F318911iA1A9F5089C4D2A32%2Fimage-size%2Fmedium%3Fv%3Dv2%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22TzviaGitlinTroyna_1-1634807306604.png%22%20alt%3D%22TzviaGitlinTroyna_1-1634807306604.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3Eread%20more%20on%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Fconnector-azure-data-explorer%3Ftabs%3Ddata-factory%23mapping-data-flow-properties%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3ECopy%20data%20to%20or%20from%20Azure%20Data%20Explorer%20using%20Azure%20Data%20Factory%20or%20Synapse%20Analytics%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-2868596%22%20slang%3D%22en-US%22%3E%3CP%3E%3CSPAN%3EMapping%20Data%20Flows%20provides%20scale-out%20data%20transformation%20in%20the%20cloud%20in%20Azure%20Data%20Factory%20and%20Azure%20Synapse%20Analytics.%20With%20these%20additional%20connectors%2C%20you%20can%20build%20ETL%20patterns%20at%20Spark%20scale%20in%20a%20code-free%20design%20environment%20without%20ever%20touching%20the%20Spark%20compute.%20Azure%20Integration%20Runtimes%20allow%20you%20to%20define%20the%20Spark%20environment%20and%20provide%20a%20serverless%20Spark%20compute%20for%20your%20data%20transformation%20pipelines.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Capture.PNG%22%20style%3D%22width%3A%20485px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F318914i2238BBCE9E2E52D3%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Capture.PNG%22%20alt%3D%22Capture.PNG%22%20%2F%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2868596%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EAzure%20Data%20Explorer%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Data%20Factory%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EKusto%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Version history
Last update:
‎Oct 26 2021 02:04 AM
Updated by: