Azure Synapse Analytics is designed for modern data analytics — and when it comes to secure networking, Managed Private Endpoints (MPEs) are key. They ensure your Synapse workspace communicates with Azure resources (like Azure SQL DB, Storage, Key Vault, etc.) over private IP addresses, completely isolated from the public internet.
Challenge: Deploying MPEs automatically across environments (Dev, Test, Prod) as part of a CI/CD pipeline? That’s where things get tricky — and this blog has you covered.
How Custom Parameters Work in Synapse
Custom parameters in Synapse allow you to override default settings and configurations during the deployment process. This is particularly useful when you need to deploy the same pipeline across multiple environments (e.g., development, testing, production) with different configurations.
Creating Custom Parameter Templates
To create custom parameter templates, follow these steps:
- Define Parameters: Identify the settings and features you want to parameterize. This can include connection strings, trigger values, and other pipeline configurations.
- Create JSON Template: Create a JSON file named template-parameters-definition.json in the root folder of your Git repository's collaboration branch. This file will contain the parameter definitions for each feature you want to override.
- Use Dynamic Expressions: Utilize dynamic expressions to define pipeline parameters that support different values for different environments.
- Below is the template-parameters-definition.json which should be present in you root Folder of Azure Synapse
{
"Microsoft.Synapse/workspaces/notebooks": {
"properties": {
"bigDataPool": {
"referenceName": "="
}
}
},
"Microsoft.Synapse/workspaces/sqlscripts": {
"properties": {
"content": {
"currentConnection": {
"*": "-"
}
}
}
},
"Microsoft.Synapse/workspaces/pipelines": {
"properties": {
"activities": [{
"typeProperties": {
"waitTimeInSeconds": "-::int",
"headers": "=::object",
"activities": [
{
"typeProperties": {
"url": "-:-webUrl:string"
}
}
]
}
}
]
}
},
"Microsoft.Synapse/workspaces/sparkConfigurations":{
"properties":{
"configs": {
"*":"="
}
}
},
"Microsoft.Synapse/workspaces/integrationRuntimes": {
"properties": {
"typeProperties": {
"*": "="
}
}
},
"Microsoft.Synapse/workspaces/triggers": {
"properties": {
"typeProperties": {
"recurrence": {
"*": "=",
"interval": "=:triggerSuffix:int",
"frequency": "=:-freq"
},
"maxConcurrency": "="
}
}
},
"Microsoft.Synapse/workspaces/linkedServices": {
"*": {
"properties": {
"typeProperties": {
"accountName": "=",
"username": "=",
"connectionString": "|:-connectionString:secureString",
"secretAccessKey": "|"
}
}
},
"AzureDataLakeStore": {
"properties": {
"typeProperties": {
"dataLakeStoreUri": "="
}
}
},
"AzureKeyVault": {
"properties": {
"typeProperties": {
"baseUrl": "|:baseUrl:secureString"
},
"parameters": {
"KeyVaultURL": {
"type": "=",
"defaultValue": "|:defaultValue:secureString"
}
}
}
},
"AzureBlobFS": {
"properties": {
"typeProperties": {
"url": "=",
"servicePrincipalId": "="
}
}
},
"AzureSqlDatabase": {
"properties": {
"typeProperties": {
"servicePrincipalId": "="
}
}
},
"AzureSqlDW": {
"properties": {
"typeProperties": {
"servicePrincipalId": "="
}
}
}
},
"Microsoft.Synapse/workspaces/datasets": {
"*": {
"properties": {
"typeProperties": {
"folderPath": "=",
"fileName": "="
}
}
}
},
"Microsoft.Synapse/workspaces/credentials" : {
"properties": {
"typeProperties": {
"resourceId": "="
}
}
},
"Microsoft.Synapse/workspaces/managedVirtualNetworks/managedPrivateEndpoints": {
"properties": {
"privateLinkResourceId": "-:-privateLinkResourceId",
"groupId": "-:-groupId",
"fqdns": "-:-fqdns:array"
}
}
}
What Are Managed Private Endpoints in Synapse?
Managed Private Endpoints are network interfaces created in Synapse-managed virtual networks. They enable secure, private connections from Synapse to other Azure services — no public internet exposure.
Use cases:
- Access Azure SQL DB securely
- Read from ADLS Gen2 with network isolation
- Connect to Azure Key Vault without public endpoint
Network Security Best Practices for Synapse
To enforce a secure and compliant Synapse deployment, follow these steps:
- Create Synapse with Data Exfiltration Protection Enabled
When provisioning your Synapse workspace:
- Enable managed VNET
- Set Data Exfiltration Protection = Yes (mandatory for secure enterprise environments)
- Disable Public Network Access to ensure all traffic is routed privately
- Disable Public Access to Linked Services
- For each resource (Storage Account, Key Vault, SQL DB), ensure that:
- Public Network Access is disabled
- Firewall is restricted to Synapse's private endpoint subnet only
Creating Managed Private Endpoints in Synapse
Once the Synapse workspace is created:
- Navigate to Manage > Managed Private Endpoints.
- Click + New > Choose the Azure service (e.g., Storage Account, Azure SQL).
- Select the appropriate resource and sub-resource.
- Once the Private Endpoint is created, navigate to the target resource's Networking settings.
- Manually approve the connection request (Pending PE requests will appear in the target resource’s Private Endpoint Connections blade).
CI/CD Approach with custom parameter for Synapse Managed PE
Adding the Microsoft.Synapse/workspaces/managedVirtualNetworks/managedPrivateEndpoints section to your template-parameters-definition.json is essential for automating or customizing the deployment of Managed Private Endpoints (MPEs) in CI/CD.
CI/CD Sample template for Deploying the Synapse Managed Private Endpoint.
parameters:
- name: environment
type: string
- name: variablegroup
type: string
- name: serviceconnection
type: string
- name: envsuffix
type: string
- name: RGName
type: string
- name: SynapseWorkspace
type: string
stages:
- stage: Deploy_${{ parameters.environment }}_Synapse_application
jobs:
- deployment:
environment: ${{ parameters.environment }}
displayName: Deploy_Synapse_application
variables:
- group: ${{ parameters.variablegroup }}
strategy:
runOnce:
deploy:
steps:
- task: DownloadBuildArtifacts@1
displayName: 'Download Synapse Build Artifacts (ARM Templates)'
inputs:
buildType: 'specific'
buildVersionToDownload: 'latestFromBranch'
branchName: '${{ parameters.branchName }}'
tags: '${{ parameters.buildtags }}'
downloadType: 'single'
artifactName: 'synapse-artifacts'
downloadPath: '$(System.DefaultWorkingDirectory)'
cleanDestinationFolder: true
- task: AzureSynapseWorkspace.synapsecicd-deploy.toggle-trigger.toggle-triggers-dev@2
displayName: 'Turn Off Synapse Triggers'
inputs:
azureSubscription: '${{ parameters.serviceconnection }}'
ResourceGroupName: '${{ parameters.RGName }}'
WorkspaceName: '${{ parameters.SynapseWorkspace }}'
ToggleOn: false
- task: Synapse workspace deployment@2
inputs:
operation: 'deploy'
TemplateFile: '$(System.DefaultWorkingDirectory)/export/TemplateForWorkspace.json'
ParametersFile: '$(System.DefaultWorkingDirectory)/export/TemplateParametersForWorkspace.json'
azureSubscription: '${{ parameters.serviceconnection }}'
ResourceGroupName: '${{ parameters.RGName }}'
TargetWorkspaceName: '${{ parameters.SynapseWorkspace }}'
DeleteArtifactsNotInTemplate: true
OverrideArmParameters: >
-workspaceName ${{ parameters.SynapseWorkspace }}
-StorageAccount_connectionString $(SynapseConnectionString)
-StorageAccount_url https://$(StorageAccountName).dfs.core.windows.net
-ManagedPE_Storage_privateLinkResourceId $(PrivateLinkResourceId_Storage)
-ManagedPE_Storage_groupId $(GroupId_Storage)
DeployManagedPrivateEndpoints: true
Environment: '$(environment)'
"Note: When deploying the managed private endpoint across different environments (Dev/Stage/Prod), always try maintain consistent naming conventions across all environments."
Microsoft