Home
%3CLINGO-SUB%20id%3D%22lingo-sub-317016%22%20slang%3D%22en-US%22%3EAzure%20Data%20Architecture%20Guide%20%E2%80%93%20Blog%20%239%3A%20Extract%2C%20transform%2C%20load%20(ETL)%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-317016%22%20slang%3D%22en-US%22%3E%3CP%3E%3CSPAN%3EThis%20is%20our%20ninth%20and%20final%20blog%20entry%20exploring%20the%20%3CA%20href%3D%22http%3A%2F%2Faka.ms%2FADAG%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20Data%20Architecture%20Guide%3C%2FA%3E.%26nbsp%3BThe%20previous%20entries%20for%20this%20blog%20series%26nbsp%3Bare%3A%3C%2FSPAN%3E%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzureCAT%2FAzure-Data-Architecture-Guide-Blog-1-Introduction%2Fba-p%2F306477%22%20target%3D%22_blank%22%3EAzure%20Data%20Architecture%20Guide%20%E2%80%93%20Blog%20%231%3A%20Introduction%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzureCAT%2FAzure-Data-Architecture-Guide-Blog-2-On-demand-big-data%2Fba-p%2F306505%22%20target%3D%22_blank%22%3EAzure%20Data%20Architecture%20Guide%20%E2%80%93%20Blog%20%232%3A%20On-demand%20big%20data%20analytics%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzureCAT%2FAzure-Data-Architecture-Guide-8211-Blog-3-Advanced-analytics-and%2Fba-p%2F306556%22%20target%3D%22_blank%22%3EAzure%20Data%20Architecture%20Guide%20%E2%80%93%20Blog%20%233%3A%20Advanced%20analytics%20and%20deep%20learning%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzureCAT%2FAzure-Data-Architecture-Guide-8211-Blog-4-Hybrid-data%2Fba-p%2F306579%22%20target%3D%22_blank%22%3EAzure%20Data%20Architecture%20Guide%20%E2%80%93%20Blog%20%234%3A%20Hybrid%20data%20architecture%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzureCAT%2FAzure-Data-Architecture-Guide-8211-Blog-5-Clickstream-analysis%2Fba-p%2F306630%22%20target%3D%22_blank%22%3EAzure%20Data%20Architecture%20Guide%20%E2%80%93%20Blog%20%235%3A%20Clickstream%20analysis%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzureCAT%2FAzure-Data-Architecture-Guide-8211-Blog-6-Business-intelligence%2Fba-p%2F306646%22%20target%3D%22_blank%22%3EAzure%20Data%20Architecture%20Guide%20%E2%80%93%20Blog%20%236%3A%20Business%20intelligence%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzureCAT%2FAzure-Data-Architecture-Guide-8211-Blog-7-Intelligent%2Fba-p%2F306656%22%20target%3D%22_blank%22%3EAzure%20Data%20Architecture%20Guide%20%E2%80%93%20Blog%20%237%3A%20Intelligent%20applications%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2FAzureCAT%2FAzure-Data-Architecture-Guide-Blog-8-Data-warehousing%2Fba-p%2F352865%22%20target%3D%22_blank%22%3EAzure%20Data%20Architecture%20Guide%20%E2%80%93%20Blog%20%238%3A%20Data%20warehousing%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%3ELike%20all%20the%20previous%20posts%20in%20this%20series%2C%20we'll%20work%20from%20a%20technology%20implementation%20seen%20directly%20in%20our%20customer%20engagements.%20The%20example%20can%20help%20lead%20you%20to%20the%20ADAG%20content%20to%20make%20the%20right%20technology%20choices%20for%20your%20business.%20%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%3E%26nbsp%3B%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%0A%3CH1%20id%3D%22toc-hId-1898415547%22%20id%3D%22toc-hId-1927973031%22%3EExtract%2C%20transform%2C%20load%20(ETL)%3C%2FH1%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EIn%20this%20example%2C%20the%20web%20application%20logs%20and%20custom%20telemetry%20are%20captured%20with%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fapplication-insights%2Fapp-insights-overview%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EApplication%20Insights%3C%2FA%3E%3C%2FSPAN%3E%2C%20sent%20to%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fstorage%2Fblobs%2Fstorage-blobs-introduction%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20Storage%20blobs%3C%2FA%3E%3C%2FSPAN%3E%2C%20and%20then%20the%20ETL%20pipeline%20is%20created%2C%20scheduled%2C%20and%20managed%20using%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20Data%20Factory%3C%2FA%3E%3C%2FSPAN%3E.%20The%20SSIS%20packages%20are%20deployed%20to%20Azure--with%20the%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-factory%2Ftutorial-deploy-ssis-packages-azure%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure-SSIS%20integration%20runtime%3C%2FA%3E%3C%2FSPAN%3E%20(IR)%20in%20Azure%20Data%20Factory--to%20apply%20data%20transformation%20as%20a%20step%20in%20the%20ETL%20pipeline%2C%20before%20loading%20the%20transformed%20data%20into%20%3CSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fsql-database%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20SQL%20Database%3C%2FA%3E%3C%2FSPAN%3E.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%20%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20683px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F67045i0F17BA50E59E43D8%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20alt%3D%22ADAG_ETL.png%22%20title%3D%22ADAG_ETL.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId--850254919%22%20id%3D%22toc-hId--820697435%22%3EHighlighted%20services%3C%2FH2%3E%0A%3CUL%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Fapplication-insights%2Fapp-insights-overview%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EApplication%20Insights%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Fstorage%2Fblobs%2Fstorage-blobs-introduction%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20Storage%20blobs%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Fdata-factory%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20Data%20Factory%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Fdata-factory%2Ftutorial-deploy-ssis-packages-azure%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure-SSIS%20integration%20runtime%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Fsql-database%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EAzure%20SQL%20Database%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%26nbsp%3B%3C%2FP%3E%0A%3CH2%20id%3D%22toc-hId-892555416%22%20id%3D%22toc-hId-922112900%22%3ERelated%20ADAG%20articles%3C%2FH2%3E%0A%3CUL%3E%0A%3CLI%3ETraditional%20RDBMS%20workloads%3CUL%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Farchitecture%2Fdata-guide%2Frelational-data%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3ETraditional%20relational%20database%20solutions%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Farchitecture%2Fdata-guide%2Frelational-data%2Fetl%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EExtract%2C%20transform%2C%20and%20load%20(ETL)%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Farchitecture%2Fdata-guide%2Frelational-data%2Fonline-transaction-processing%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EOnline%20transaction%20processing%20(OLTP)%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3C%2FLI%3E%0A%3CLI%3ETechnology%20choices%3CUL%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Farchitecture%2Fdata-guide%2Ftechnology-choices%2Fpipeline-orchestration-data-movement%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EChoosing%20a%20data%20pipeline%20orchestration%20technology%20in%20Azure%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Farchitecture%2Fdata-guide%2Ftechnology-choices%2Freal-time-ingestion%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EChoosing%20a%20real-time%20message%20ingestion%20technology%20in%20Azure%3C%2FA%3E%3C%2FLI%3E%0A%3CLI%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Farchitecture%2Fdata-guide%2Ftechnology-choices%2Fdata-storage%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3EChoosing%20a%20big%20data%20storage%20technology%20in%20Azure%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%26nbsp%3B%20%26nbsp%3B%26nbsp%3B%3C%2FP%3E%0A%3CP%3EPlease%20peruse%20ADAG%20to%20find%20a%20clear%20path%20for%20you%20to%20architect%20your%20data%20solution%20on%20Azure%3A%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3E%3CA%20href%3D%22http%3A%2F%2Faka.ms%2FADAG%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noopener%20noreferrer%20noopener%20noreferrer%22%3ERead%20the%20Azure%20Data%20Architecture%20Guide%3C%2FA%3E%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3E%3CSPAN%3E%26nbsp%3B%20%26nbsp%3B%20%26nbsp%3B%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3E%3CA%20href%3D%22http%3A%2F%2Faka.ms%2FCAT%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%20noopener%20noreferrer%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%3E%3CSPAN%20class%3D%22lia-message-image-wrapper%20lia-message-image-actions-narrow%20lia-message-image-actions-below%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20style%3D%22width%3A%20136px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F64635i735A53524DEAB702%2Fimage-dimensions%2F136x136%3Fv%3D1.0%22%20width%3D%22136%22%20height%3D%22136%22%20alt%3D%22AzureCAT_Icon.jpg%22%20title%3D%22AzureCAT_Icon.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FA%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSTRONG%3EAzure%20CAT%20Guidance%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%3E%3CEM%3E%22Hands-on%20solutions%2C%20with%20our%20heads%20in%20the%20Cloud!%22%3C%2FEM%3E%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-317016%22%20slang%3D%22en-US%22%3E%3CP%3E%3CSPAN%3EThis%20is%20our%20ninth%20and%20final%20blog%20entry%20exploring%20the%20%3C%2FSPAN%3E%3CA%20href%3D%22http%3A%2F%2Faka.ms%2FADAG%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%20target%3D%22_blank%22%3EAzure%20Data%20Architecture%20Guide%3C%2FA%3E%3CSPAN%3E.%26nbsp%3BIn%20this%20example%2C%20the%20web%20application%20logs%20and%20custom%20telemetry%20are%20captured%20with%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fapplication-insights%2Fapp-insights-overview%22%20rel%3D%22noopener%20noreferrer%20noopener%20noreferrer%22%20target%3D%22_blank%22%3EApplication%20Insights%3C%2FA%3E.%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-317016%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EData%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Microsoft

This is our ninth and final blog entry exploring the Azure Data Architecture Guide. The previous entries for this blog series are:

Like all the previous posts in this series, we'll work from a technology implementation seen directly in our customer engagements. The example can help lead you to the ADAG content to make the right technology choices for your business.

  

Extract, transform, load (ETL)

 

In this example, the web application logs and custom telemetry are captured with Application Insights, sent to Azure Storage blobs, and then the ETL pipeline is created, scheduled, and managed using Azure Data Factory. The SSIS packages are deployed to Azure--with the Azure-SSIS integration runtime (IR) in Azure Data Factory--to apply data transformation as a step in the ETL pipeline, before loading the transformed data into Azure SQL Database.

   

ADAG_ETL.png

 

Highlighted services

  

Related ADAG articles

    

Please peruse ADAG to find a clear path for you to architect your data solution on Azure:

     

 

AzureCAT_Icon.jpg

Azure CAT Guidance

"Hands-on solutions, with our heads in the Cloud!"