CI CD in Azure Synapse Analytics Part 1

Published Dec 07 2020 01:18 PM 6,963 Views

I was helping a friend earlier today with their Azure Synapse Studio CI / CD integration.  They had followed our Docs page Source control in Azure Synapse Studio and then they shared errors they were seeing in their release pipeline during deployment. 

We took a step back to discuss what they wanted to do, and it looked like they were too far in the weeds for ADO.  So I walked through creating an Azure DevOps Project, connecting Git to my Azure Synapse Studio, and then creating a branch and pushing some changes.  We'll push changes in a follow up blog post.  Today we cover the basics.

First let's navigate to Projects - Home (azure.com).  We will create a New Project and title it Azure Synapse Studio CI CD.  I'm going to mark this repo private because it's just for us.

Create New ADO Project.png

 

Now I will click on the Repos menu.

 

Repos.png

 

Next I will go to the bottom of the page.  I want to select Initialize main branch with a README or gitignore.  I will click Initialize.

 

Initialize.png

 

At this point I have a Repo that is initialized.

initalized repo.png

 

Now we can connect this to our Azure Synapse Studio.  Let us travel over to https://web.azuresynapse.net/ and log into our Azure Synapse Studio.  After we login we need to navigate to the Manage screen.  If you are not on the Git configuration page, navigate there.  

 

Connect Azure Synapse Studio.png

 

Next we want to click on Set up code repository.  You can select Azure DevOps Git or GitHub.  For this blog we will be selecting Azure DevOps Git.  Then select your organizations Azure Active Directory tenant. *a quick side note, make sure the AAD account you are using to connect to Azure DevOps is the same account that has permissions to your Azure Synapse Studio workspace.

Then click Continue.
Connect Azure Synapse Studio2.png

Select the Azure DevOps Account that our organization is using. The Project and Git repository name are the same, and are the Project Name we created earlier.

My collaboration branch is main, my Publish branch is workspace_publish, my Root folder is the default, I have checked import existing resources to repository.

 

As this is my initial commit I want to commit this to my main branch.

 

Then I click Apply.

 

*Another note: your company will have a DevOps environment, and specific rules on how you want things to connect.  If I'm doing anything that makes you scream from a developer philosophy, please find me on Twitter under BuckWoody_MSFT .... also don't tell Buck I did this .... I'm not Buck.


Connect Azure Synapse Studio3.png
At this point in time your Azure DevOps Git Rep should be connected.

 

Connect Azure Synapse Studio4.png

If we go to our Azure DevOps Repo we should see that it is populated with objects from our Azure Synapse Studio.

ADO Populated.png

 

Back in Azure Synapse Studio, we can navigate to the develop pane, create a new branch to ensure any changes I make will not be automatically deployed against my main version of Azure Synapse Studio.
final.png

This is what we will tackle next time.

1 Comment
Microsoft

After configure the repository I got this error on Synapse: 

 

Please set the working branch. Previously used working branch 'developer' is not available.

 

Set working branchSet working branch

 

Just to update the article to avoid doubts. I created an developer branch.

%3CLINGO-SUB%20id%3D%22lingo-sub-1964172%22%20slang%3D%22en-US%22%3ECI%20CD%20in%20Azure%20Synapse%20Analytics%20Part%201%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1964172%22%20slang%3D%22en-US%22%3E%3CP%20data-unlink%3D%22true%22%3EI%20was%20helping%20a%20friend%20earlier%20today%20with%20their%20Azure%20Synapse%20Studio%20CI%20%2F%20CD%20integration.%26nbsp%3B%20They%20had%20followed%20our%20Docs%20page%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fsynapse-analytics%2Fcicd%2Fsource-control%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3ESource%20control%20in%20Azure%20Synapse%20Studio%3C%2FA%3E%20and%20then%20they%20shared%20errors%20they%20were%20seeing%20in%20their%20release%20pipeline%20during%20deployment.%26nbsp%3B%3CBR%20%2F%3E%3CBR%20%2F%3EWe%20took%20a%20step%20back%20to%20discuss%20what%20they%20wanted%20to%20do%2C%20and%20it%20looked%20like%20they%20were%20too%20far%20in%20the%20weeds%20for%20ADO.%26nbsp%3B%20So%20I%20walked%20through%20creating%20an%20Azure%20DevOps%20Project%2C%20connecting%20Git%20to%20my%20Azure%20Synapse%20Studio%2C%20and%20then%20creating%20a%20branch%20and%20pushing%20some%20changes.%26nbsp%3B%20We'll%20push%20changes%20in%20a%20follow%20up%20blog%20post.%26nbsp%3B%20Today%20we%20cover%20the%20basics.%3CBR%20%2F%3E%3CBR%20%2F%3EFirst%20let's%20navigate%20to%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fdev.azure.com%2F%22%20target%3D%22_self%22%20rel%3D%22nofollow%20noopener%20noreferrer%22%3EProjects%20-%20Home%20(azure.com)%3C%2FA%3E.%26nbsp%3B%20We%20will%20create%20a%20New%20Project%20and%20title%20it%20Azure%20Synapse%20Studio%20CI%20CD.%26nbsp%3B%20I'm%20going%20to%20mark%20this%20repo%20private%20because%20it's%20just%20for%20us.%3CBR%20%2F%3E%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Create%20New%20ADO%20Project.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238422iCFA0A048775A170B%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Create%20New%20ADO%20Project.png%22%20alt%3D%22Create%20New%20ADO%20Project.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3ENow%20I%20will%20click%20on%20the%20Repos%20menu.%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Repos.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238441iD0CE0B4510900155%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Repos.png%22%20alt%3D%22Repos.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3ENext%20I%20will%20go%20to%20the%20bottom%20of%20the%20page.%26nbsp%3B%20I%20want%20to%20select%20Initialize%20main%20branch%20with%20a%20README%20or%20gitignore.%26nbsp%3B%20I%20will%20click%20Initialize.%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Initialize.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238442iE664F77C3098A04A%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Initialize.png%22%20alt%3D%22Initialize.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3EAt%20this%20point%20I%20have%20a%20Repo%20that%20is%20initialized.%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22initalized%20repo.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238427i45958D102511E4C0%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22initalized%20repo.png%22%20alt%3D%22initalized%20repo.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3ENow%20we%20can%20connect%20this%20to%20our%20Azure%20Synapse%20Studio.%26nbsp%3B%20Let%20us%20travel%20over%20to%26nbsp%3B%3CA%20href%3D%22https%3A%2F%2Fweb.azuresynapse.net%2F%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehttps%3A%2F%2Fweb.azuresynapse.net%2F%3C%2FA%3E%26nbsp%3Band%20log%20into%20our%20Azure%20Synapse%20Studio.%26nbsp%3B%20After%20we%20login%20we%20need%20to%20navigate%20to%20the%20Manage%20screen.%26nbsp%3B%20If%20you%20are%20not%20on%20the%20Git%20configuration%20page%2C%20navigate%20there.%26nbsp%3B%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Connect%20Azure%20Synapse%20Studio.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238445iBA8552C4943C572F%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Connect%20Azure%20Synapse%20Studio.png%22%20alt%3D%22Connect%20Azure%20Synapse%20Studio.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%E2%80%83%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3ENext%20we%20want%20to%20click%20on%20Set%20up%20code%20repository.%26nbsp%3B%20You%20can%20select%20Azure%20DevOps%20Git%20or%20GitHub.%26nbsp%3B%20For%20this%20blog%20we%20will%20be%20selecting%20Azure%20DevOps%20Git.%26nbsp%3B%20Then%20select%20your%20organizations%20Azure%20Active%20Directory%20tenant.%20*a%20quick%20side%20note%2C%20make%20sure%20the%20AAD%20account%20you%20are%20using%20to%20connect%20to%20Azure%20DevOps%20is%20the%20same%20account%20that%20has%20permissions%20to%20your%20Azure%20Synapse%20Studio%20workspace.%3CBR%20%2F%3E%3CBR%20%2F%3EThen%20click%20Continue.%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Connect%20Azure%20Synapse%20Studio2.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238446iF6EE63C734ACA3F4%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Connect%20Azure%20Synapse%20Studio2.png%22%20alt%3D%22Connect%20Azure%20Synapse%20Studio2.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3ESelect%20the%20Azure%20DevOps%20Account%20that%20our%20organization%20is%20using.%20The%20Project%20and%20Git%20repository%20name%20are%20the%20same%2C%20and%20are%20the%20Project%20Name%20we%20created%20earlier.%3CBR%20%2F%3E%3CBR%20%2F%3EMy%20collaboration%20branch%20is%20main%2C%20my%20Publish%20branch%20is%20workspace_publish%2C%20my%20Root%20folder%20is%20the%20default%2C%20I%20have%20checked%20import%20existing%20resources%20to%20repository.%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3EAs%20this%20is%20my%20initial%20commit%20I%20want%20to%20commit%20this%20to%20my%20main%20branch.%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3EThen%20I%20click%20Apply.%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E*Another%20note%3A%20your%20company%20will%20have%20a%20DevOps%20environment%2C%20and%20specific%20rules%20on%20how%20you%20want%20things%20to%20connect.%26nbsp%3B%20If%20I'm%20doing%20anything%20that%20makes%20you%20scream%20from%20a%20developer%20philosophy%2C%20please%20find%20me%20on%20Twitter%20under%20BuckWoody_MSFT%20....%20also%20don't%20tell%20Buck%20I%20did%20this%20....%20I'm%20not%20Buck.%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Connect%20Azure%20Synapse%20Studio3.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238448i42570C64E9BC5E1D%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Connect%20Azure%20Synapse%20Studio3.png%22%20alt%3D%22Connect%20Azure%20Synapse%20Studio3.png%22%20%2F%3E%3C%2FSPAN%3E%3CBR%20%2F%3EAt%20this%20point%20in%20time%20your%20Azure%20DevOps%20Git%20Rep%20should%20be%20connected.%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22Connect%20Azure%20Synapse%20Studio4.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238449i85834F4AB9709268%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Connect%20Azure%20Synapse%20Studio4.png%22%20alt%3D%22Connect%20Azure%20Synapse%20Studio4.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3EIf%20we%20go%20to%20our%20Azure%20DevOps%20Repo%20we%20should%20see%20that%20it%20is%20populated%20with%20objects%20from%20our%20Azure%20Synapse%20Studio.%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22ADO%20Populated.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238452i09B8D8FE16933657%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22ADO%20Populated.png%22%20alt%3D%22ADO%20Populated.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3EBack%20in%20Azure%20Synapse%20Studio%2C%20we%20can%20navigate%20to%20the%20develop%20pane%2C%20create%20a%20new%20branch%20to%20ensure%20any%20changes%20I%20make%20will%20not%20be%20automatically%20deployed%20against%20my%20main%20version%20of%20Azure%20Synapse%20Studio.%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22final.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238454i2AEAB306E6B81C77%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22final.png%22%20alt%3D%22final.png%22%20%2F%3E%3C%2FSPAN%3E%3CBR%20%2F%3E%3CBR%20%2F%3E%3C%2FP%3E%0A%3CP%20data-unlink%3D%22true%22%3EThis%20is%20what%20we%20will%20tackle%20next%20time.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-1964172%22%20slang%3D%22en-US%22%3E%3CP%3EIn%20this%20blog%20we%20create%20an%20Azure%20DevOps%20Project%2C%20Configure%20Git%2C%20and%20Create%20a%20Branch%20for%20future%20changes%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22final.png%22%20style%3D%22width%3A%20952px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F238453i134B5618D29D1728%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22final.png%22%20alt%3D%22final.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-1964172%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EData%20Architecture%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EData%20Pipelines%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2385524%22%20slang%3D%22en-US%22%3ERe%3A%20CI%20CD%20in%20Azure%20Synapse%20Analytics%20Part%201%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2385524%22%20slang%3D%22en-US%22%3E%3CP%3EAfter%20configure%20the%20repository%20I%20got%20this%20error%20on%20Synapse%3A%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3EPlease%20set%20the%20working%20branch.%20Previously%20used%20working%20branch%20'developer'%20is%20not%20available.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22Screen%20Shot%202021-05-25%20at%2015.55.09.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F283717iD9D426024C80BBB7%2Fimage-size%2Flarge%3Fv%3Dv2%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22Screen%20Shot%202021-05-25%20at%2015.55.09.png%22%20alt%3D%22Set%20working%20branch%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3ESet%20working%20branch%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%3EJust%20to%20update%20the%20article%20to%20avoid%20doubts.%20I%20created%20an%20developer%20branch.%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-BODY%3E
Version history
Last update:
‎Dec 07 2020 01:18 PM