Enhancing Team Collaboration in Azure Synapse Analytics using a Git Branching Strategy – Part 1 of 3
Introduction Over the past few years of working with numerous Synapse Studio users, many have asked how to make the most of collaborative work in Synapse Studio —especially in complex development scenarios where developers work on different projects in parallel within a single Synapse workspace. Based on our experience and internal feedback from other Synapse experts, our general recommendation is that each development team or project should have its own Synapse workspace. This approach is particularly effective when the maturity level of the teams—both in Synapse and Git, is still developing. In such cases, having separate workspaces simplifies the CI/CD journey. However, in scenarios where teams demonstrate greater maturity (especially in Git) and the number or complexity of Synapse projects is relatively low, it is possible for multiple teams and projects to coexist within a single Synapse development workspace. In these cases, evaluating your team’s maturity in both Synapse and Git is crucial. Teams must honestly assess their comfort level with these technologies. For example, expecting success from teams that are just beginning their Synapse journey and have limited Git experience—or planning to develop more than five projects in parallel within a single workspace—would likely lead to challenges. Managing even a single project in Synapse can be complex; doing so for multiple projects without sufficient expertise in both Synapse and Git could be a recipe for disaster. That said, the main objective of this article is to demonstrate how a simple Git branching strategy can enhance collaborative work in Synapse Studio, enabling different projects to be developed in parallel within a single Synapse workspace. This guide can help teams at the beginning of their Synapse journey assess their current maturity level (in both Synapse and Git) and understand what level they should aim for to adopt this approach confidently. For teams with a reasonable level of maturity, this article can help validate whether this strategy can further improve their collaborative efforts in Synapse. This is the first of three articles, where we’ll show how to implement a simple branching strategy that allows two development teams working on separate projects to share a single Synapse workspace. The strategy supports isolated code promotion through various environments without interfering with each team’s work. While we use Azure DevOps as our Git provider throghout these articles, the approach is also applicable to GitHub GitHub. Start elevating your collaborative work in Synapse Studio, by implementing a simple and effictive Git branching strategy Let’s begin by outlining our scenario: two development teams—Data Engineering and Data Science—are about to start their projects in Synapse. Both teams have substantial experience with Synapse and Git. Together, they’ve agreed on a simple Git branching strategy that will enable them to collaborate effectively in Synapse Studio while supporting a CI/CD flow designed to automate the promotion of their code from the development environment to higher environments. The Git branching strategy involves creating feature branches and environment branches, organized by team, as illustrated in the following diagram. Figure 1: A Simple Git Branching Strategy Important note on governance of the branching strategy: The first branches that should be created are the environment branches. Once these are in place, any time a developer needs to create a feature branch, it must always be based on the production environment branch of their respective team. In this strategy, the production branch serves as the team’s collaboration branch, ensuring consistency and alignment across development efforts. Figure 2: Creating a Feature Branch Based on the Production Environment Branch In the initial phase of implementing this strategy, environment branches can be created using the "Branches" feature in Azure DevOps, or locally in a developer’s repository and then pushed to the remote repository. Alternatively, teams can use the branch selector functionality within Synapse Studio. The team should choose the method they are most comfortable with. Below is an example of the branch structure that will be developed throughout this article: Figure 3: Example of Branching Structure Visualization from DevOps Start at the feature branch level... With the branching strategy defined, we can now demonstrate how the two teams will carry out their respective developments within a single Synapse development workspace. Let’s begin with Mary from the Data Engineering team, who will develop a new pipeline. She creates this pipeline in her feature branch: features/data_eng/mary/mktetl. Figure 4: Creating a Pipeline in a Feature Branch of the Data Engineering Team Meanwhile, Anna, a developer from the Data Science team, also begins working on a new feature for the Data Science project. Figure 5: Creating a Notebook in a Feature Branch of the Data Science Team Both teams are ready to start their unit testing independently, at different times, and with distinct code executions. This is where the Environment Branches come into play. …and end at the Environment branch level! After completing the development of her feature, Anna promotes her changes to the development environment. It’s important to note that the code has only been committed to Git—it has not been published to Live Mode yet. You might wonder why Anna didn’t simply use the Publish button in Synapse Studio to push her changes live. That would be a valid question—if both teams were sharing a single collaboration branch (as described here). In such a setup, the collaboration branch would contain code from both the Data Engineering and Data Science teams. However, that’s not the goal of our branching strategy. Our strategy is designed to ensure segregation at both the source control and CI/CD levels for all teams working within a shared Synapse development workspace. Instead of using a single collaboration branch for everyone, each team uses its own production environment branch as its collaboration branch. In this context, using the Publish button in Synapse Studio is not appropriate. Instead, we leverage a feature of the Synapse public extension—specifically, the the Synapse Workspace Deployment Task in Azure DevOps (or the GitHub Action for Synapse Workspace Artifacts Deployment, if using GitHub). This extension allows us to publish Synapse artifacts to any environment from any user branch—in this case, from the environment branches. Therefore, when configuring Git for your Synapse development workspace under this strategy, you can set the collaboration branch to any placeholder (e.g., main, master, or develop), as it will be ignored. This approach ensures that each team maintains code isolation throughout the development and deployment lifecycle. It’s important to understand that the decision not to use the Publish functionality in Synapse Studio is intentional and directly tied to our strategy of supporting multiple teams and multiple projects within a single Synapse workspace. Figure 6: Data Science Team: Creating a Pull Request from the Feature Branch to an Environment Branch in Synapse Studio Figure 7: Data Science Team: Configuring the Pull Request in DevOps, Indicating the Source (Feature Branch) and Destination (DEV Environment Branch) Meanwhile, Mary, our Data Engineer, has also completed the development of her feature and is now ready to publish her pipeline to the development environment. Figure 8: Data Engineering Team: Creating a Pull Request from the Feature Branch to an Environment Branch in Synapse Studio Figure 9: Data Engineering Team: Configuring the Pull Request in DevOps, Indicating the Source (Feature Branch) and Destination (DEV Environment Branch) Conclusion In conclusion, this article has demonstrated how different development teams can effectively leverage a Git branching strategy to develop their code within a single Synapse development workspace. By creating both feature branches and environment branches, the teams are able to work in parallel without interfering with each other’s development processes. This approach ensures proper isolation and enables smooth code promotion across environments. As we move forward, the next article in this series will explore how this strategy helps both teams accelerate their development lifecycle and streamline the CI/CD flow in Synapse.Enhancing Team Collaboration in Azure Synapse Analytics using a Git Branching Strategy – Part 2 of 3
Introduction In the first part of this blog series, we introduced a Git branching strategy designed to enhance collaboration within Azure Synapse Studio. By enabling multiple teams to work in parallel within a shared Synapse workspace, this approach can accelerate not only the development cycle of Synapse code but also the entire Synapse CI/CD flow. In this second part of this blog series, we take a practical step forward by demonstrating how to implement a CI/CD flow that supports this Git branching strategy. This flow will help streamline the Synapse code development cycle for our Data Engineering and Data Science teams, accelerating code releases across different environments without interfering with their respective work. Although this article series demonstrates a scenario where different teams working on separate projects share the same Synapse development workspace, you can adapt this CI/CD flow to fit your own Git branching strategy. Whether you're managing a single team or coordinating across multiple projects, this guide will help you build a scalable and efficient deployment workflow tailored for Azure Synapse Analytics. Prerequisites: - An Azure DevOps project. - An ability to run pipelines on Microsoft-hosted agents. You can either purchase a parallel job or you can request a free tier. - Basic knowledge of YAML and Azure Pipelines. For more information, see Create your first pipeline. - Permissions: To add environments, the Creator role for environments in your project. By default, members of the Build Administrators, Release Administrators, and Project Administrators groups can also create environments. - The appropriate assigned user roles to create, view, use, or manage a service connection. For more information, see Service connection permissions. To learn more about setting up Azure DevOps Environments for pipelines and setting up Service Connections, please refer to these documents: Create and target Azure DevOps environments for pipelines - Azure Pipelines | Microsoft Learn Service connections - Azure Pipelines | Microsoft Learn Defining Azure DevOps Environments An environment represents a logical target where your pipeline deploys software. Common environment names include Dev, Test, QA, Staging, and Production. You can learn more about environments here. Since our Git branching strategy is based on environment-specific branches, we’ll leverage this Azure DevOps environments feature to monitor and track Synapse code deployments by environment/team. From a security perspective, this also ensures that pipeline execution can be authorized and approved by specific users per environment. ⚠️ Note: Azure DevOps environments are not available in Classic pipelines. For Classic pipelines, Release Stages offer similar functionality. Let’s begin by creating the necessary Azure DevOps environments for our Synapse CI/CD flow. In this first step, we’ll create four environments: DEV, UAT, PRD, and EMPTY. Each environment will be associated with its corresponding environment branch. The purpose of the EMPTY environment is to ensure that the deployment job only runs when the branch is recognized as valid (e.g., environments/<team>/dev, environments/<team>/uat, or environments/<team>/prd). Even if someone modifies the trigger or manually runs the pipeline from another branch, the job will be automatically skipped. To create these environments, follow these steps: Sign in to your Azure DevOps organization at https://dev.azure.com/{yourorganization} and open your project. Go to Pipelines > Environments > Create environment. Figure 1: How to create your pipeline Environments Once you’ve created all four environments, your environment list should resemble the one shown in the figure below. Figure 2: All environments for this tutorial created To add an extra layer of security to each of these environments, we can configure an approval step and specify the user(s) authorized to approve pipeline execution in each environment. After selecting your environment, go to the Approvals and checks tab, then click the + icon to add a new check. Figure 3: Adding approvers to your pipeline environments Select Approvals, and then select Next. Add users or groups as your designated Approvers, and, if desired, provide instructions for the approvers. Specify if you want to permit or restrict approvers from approving their own runs, and specify your desired Timeout. If approvals aren't completed within the specified Timeout, the stage is marked as skipped. Figure 4: Adding an approver to your pipeline environment Creating the Pipeline for Synapse Code Deployment With the Azure DevOps environments defined, we can now create the pipeline that will drive the CI/CD flow. From the left Navigation menu, Go to "Pipelines" and select "New pipeline" Note: The following images correspond to the native Azure DevOps pipeline configuration experience. Since we are using Azure DevOps, we will select the first option presented. Figure 5: Selecting your git provider Select your repository Figure 6: Selecting your repository and then select the “Starter pipeline” option Figure 7: Configuring your pipeline Now it’s time to define the code that our pipeline will use to deploy Synapse code to the corresponding environments. Configuring your Pipeline Figure 8: Reviewing your YAML pipeline Replace the existing sample code with this code below. trigger: - environments/data_eng/dev - environments/data_eng/uat - environments/data_eng/prd - environments/data_sci/dev - environments/data_sci/uat - environments/data_sci/prd variables: - name: workspaceEnv ${{ if endsWith(variables['Build.SourceBranch'], '/uat') }}: value: 'UAT' ${{ elseif endsWith(variables['Build.SourceBranch'], '/dev') }}: value: 'DEV' ${{ elseif endsWith(variables['Build.SourceBranch'], '/prd') }}: value: 'PRD' ${{ else }}: value: 'EMPTY' jobs: - deployment: deploy_workspace displayName: Deploying to ${{ variables.workspaceEnv }} environment: $(workspaceEnv) condition: and(succeeded(), not(eq(variables['workspaceEnv'], 'EMPTY'))) strategy: runOnce: deploy: steps: - checkout: self - template: /adopipeline/deploy_template.yml parameters: serviceConnection: 'Service Connection name goes here' resourceGroup: 'Target workspace resource group name goes here' ${{ if endsWith(variables['Build.SourceBranch'], '/dev') }}: workspace: 'Development workspace name goes here' ${{ elseif endsWith(variables['Build.SourceBranch'], '/uat') }}: workspace: ' UAT workspace name goes here ' ${{ elseif endsWith(variables['Build.SourceBranch'], '/prd') }}: workspace: ' Production workspace name goes here ' ${{ else }}: workspace: '' ⚠️Important notes: In case you don’t have a service connection created yet, you can refer to this document ARM service connection and create one. Because the Synapse Workspace Deployment task does not support the “Workload Identity Federation” credential type, you must select the “Secret” credential type. Figure 9: Setting the credential type for your Azure Service Connection In the YAML pipeline provided above, you should replace the highlighted placeholder, with your service connection name. Figure 10: Configuring the serviceConnection parameter The service connection is the resource used to provide the credentials on the task execution allowing it to connect to the workspace for deployment. In our example, the same service connection is allowing access to all of our workspaces. You may need to provide a different service connection depending on the workspace and the pipeline will need to be adjusted for this use case. Same logic should apply to the resourceGroup parameter. If your workspaces belong to different resource groups, you can adapt the if condition in the parameters section, including the resource group parameter on each if clause to assign a different value to the resourceGroup parameter depending on the environment branch that is triggering the YAML pipeline. Creating a service connection in Azure DevOps, using automatic App registration, will trigger the provisioning of a new service principal in your Microsoft Entra ID. Before starting the CI/CD flow to promote Synapse code across different workspaces, this service principal must be granted the appropriate Synapse RBAC role — either Synapse Administrator or Synapse Artifact Publisher, depending on whether your Synapse deployment task is configured to deploy Managed Private Endpoints. How can you identify the service principal associated with the service connection? In your DevOps project settings, go to Service Connections and select your service connection. On the Overview tab, click the "Manage App registration" link. This will take you to the Azure Portal, specifically to Microsoft Entra ID, where you can copy details such as the display name of the service principal. Figure 11: Service connection details - selecting the Manage App registration Then, in the destination Synapse Studio environment, you can assign the appropriate Synapse RBAC role to this service principal. If you skip this step, the Synapse code deployment will fail with an authorization error (HTTP 403 – Forbidden). Figure 12: Granting Synapse RBAC to the SPN associated to your DevOps service connection Once you're done, don’t forget to rename your pipeline and save it in your preferred branch location. In this example, I’m saving the pipeline.yaml file inside the “adopipeline” folder. After renaming the file, save your pipeline — but do not run it yet. Figure 13: Saving your YAML pipeline Configuring the Synapse Deployment Task You may have noticed that this pipeline uses another file as a template, named deploy_template.yml. Templates allow us to create steps, jobs, stages and other resources that we can re-use across multiple pipelines for easier management of shared pipeline components. Let’s go ahead and create that file. Figure 14: Saving your template files in your branch We’ll start by adding the following content to our new file: parameters: - name: workspace type: string - name: resourceGroup type: string - name: serviceConnection type: string steps: - task: AzureSynapseWorkspace.synapsecicd-deploy.synapse-deploy.Synapse workspace displayName: 'Synpase deployment task for workspace: ${{ parameters.workspace }}' inputs: operation: validateDeploy ArtifactsFolder: '$(System.DefaultWorkingDirectory)/workspace' azureSubscription: '${{ parameters.serviceConnection }}' ResourceGroupName: '${{ parameters.resourceGroup }}' TargetWorkspaceName: '${{ parameters.workspace }}' condition: and(succeeded(), not(eq(length('${{ parameters.workspace }}'), 0))) This template is responsible for adding the Synapse Workspace Deployment Task, which handles deploying Synapse code to the target environment. We configure this task using the “Validate and Deploy” operation — a key enabler of our Git branching strategy. It allows Synapse code to be deployed from any user branch, not just the publish branch. Previously, Synapse users could only deploy code that existed in the publish branch. This meant they had to manually publish their changes in Synapse Studio to ensure those changes were reflected in the ARM templates generated in that branch. With the new “Validate and Deploy” operation, users can now automate this publishing process — as described in [this article]. ⚠️ Important note about the ArtifactsFolder input: The specified path must match the Root Folder defined in the Git repository information associated with your Synapse Workspace. Figure 15: The Git configuration in your Development Synapse workspace Once this file is saved, your Azure DevOps setup is complete and ready to support the development and promotion of Synapse code across multiple environments leveraging our Git branching strategy! In the next and final blog post of this series, we’ll walk through an end-to-end demonstration of the Synapse CI/CD flow using our Git branching strategy. Conclusion In this second part of our blog series, we demonstrated how to implement a CI/CD flow for Azure Synapse Analytics that fully leverages our Git branching strategy. With this CI/CD flow in place, teams are now equipped to develop, test, and promote Synapse artifacts across environments in a streamlined, secure, and automated manner. In the final post of this series, we’ll walk through a complete end-to-end demonstration of this CI/CD flow in action — showcasing how our Git branching strategy empowers collaborative work in Synapse Studio and turbo-charges your code release cycles.Innovating with PostgreSQL @Build
At this year's Microsoft Build, we're excited to share the latest updates and innovations in Microsoft Azure Database for PostgreSQL. Whether you're building AI powered apps and agents or just looking to uplevel your PostgreSQL experience, we've got sessions packed with insights and tools tailored for developers and technical leaders. As a fully managed, AI-ready open source relational database that offers 58% cost savings over an on-premises PostgreSQL database – Azure Database for PostgreSQL enhances your security, scalability, and management of enterprise workloads. Check out what’s happening with Postgres at Build — in Seattle and online: 🔍 Breakout Sessions BRK211: Building Advanced Agentic Apps with PostgreSQL on Azure What benefits do agentic architectures bring compared to traditional RAG patterns? Find out how we answer this question by exploring advanced agentic capabilities offered by popular GenAI frameworks (LangChain, LlamaIndex, Semantic Kernel) and how they transform RAG applications built on Azure Database for PostgreSQL. Learn how to further improve agentic apps by integrating advanced RAG techniques, making vector search faster with the DiskANN vector search algorithm, and more accurate with Semantic Ranking and GraphRAG. BRK204: What’s New in Microsoft Databases: Empowering AI-Driven App Dev Explore advanced applications powered by Microsoft databases on-premises, on Azure and in Microsoft Fabric. Uncover innovative approaches to scalability and learn about intelligent data processing with AI-driven insights and agentic integrations. See new features with engaging demos across all databases including Azure Database for PostgreSQL. 💻 Demo Session DEM564: Boost Your Development Workflows with PostgreSQL Discover how to transform your development workflow on PostgreSQL. Whether you're building AI-powered apps, managing complex datasets, or just looking to streamline your PostgreSQL experience, this demo will show you how to level up your productivity with PostgreSQL on Azure. 🧪 Hands-On Labs LAB360: Build an Agentic App with PostgreSQL, GraphRAG, and Semantic Kernel Sign up to get hands-on experience building an agent-driven, RAG-based application with Azure Database for PostgreSQL and VS Code. Explore coding and architectural concepts while using DiskANN Index for Vector Search, and integrating Apache AGE for PostgreSQL to extend into a GraphRAG pattern leveraging the Semantic Kernel Agent Framework. 💬 Meet the Experts Have questions? Looking to talk open source, AI agentic apps, or migration? Visit us in the Expert Meetup Zone to connect with the Postgres product teams, engineers, and architects. 🔎 How to find it: Log into the Microsoft Build 2025 website or use the official event mobile app to view the venue map and session schedule. 📍 To find the Expert Meetup zone, check out the official MS Build Event Guide for a venue maps and other logistical information. 🐘Get Started with Azure Database for PostgreSQL Want to try it out firsthand? 🚀 Start building 📘 Explore the documentation Let's connect, code, and grow together at Build 2025!Just published: What's new with Postgres at Microsoft, 2025 edition
If you’re using Postgres on Azure—or just curious about what the Postgres team at Microsoft has been up to during the past 12 months—this annual update might be worth a look. The blog post covers: New features in Azure Database for PostgreSQL – Flexible Server Open source code contributions to Postgres 18 (including async I/O) Work on the Citus extension to Postgres Community efforts like POSETTE, helping with PGConf.dev, our monthly Talking Postgres podcast, and more There’s also a hand-made infographic that maps out the different Postgres workstreams at Microsoft over the past year. It's a lot to take in, but the infographic captures so much of the work across the team—I think it's kind of a work of art. 📝 Read the full post here: https://techcommunity.microsoft.com/blog/adforpostgresql/whats-new-with-postgres-at-microsoft-2025-edition/4410710 And, I'd love to hear your thoughts or questions.
