Blog Post

Azure Data Factory Blog
1 MIN READ

CI/CD improvement using Global Parameters in Azure Data Factory

Abhishek Narain's avatar
Jun 28, 2022

We are introducing a new mechanism to include Global parameters in the ARM templates (from 'Manage hub' ->' ARM template' -> 'Include global parameters in ARM template').

 

It solves the earlier known issue, which overrode some Data Factory level configurations during deployments if the global parameters were included in the ARM template (from 'Manage hub' -> 'Global parameters' -> 'Include in ARM template'). Hence, earlier in such scenarios, we recommended using PowerShell to deploy global parameters and not include them in the ARM template. It resulted in a complex release process.  

 

Note:

  • The change breaks any existing release pipelines you may have for deploying the global parameters (using PowerShell). It is transparent to existing CICD pipelines.
  • If you deployed the global parameters through the ARM template using the older mechanism (from 'Manage hub' -> 'Global parameters' -> 'Include in ARM template').
  • All new setups should include global parameters in the ARM template using the more recent mechanism (from 'Manage hub' ->' ARM template' -> 'Include global parameters in ARM template').
  • The Pernille-Eskebo/azure-data-factory-utilities npm package for automated deployments is also compatible with the new mechanism for deploying global parameters across environments. 

 

Reference:

Global parameters - Azure Data Factory | Microsoft Docs

 

 

Published Jun 28, 2022
Version 1.0

9 Comments

  • adfss990's avatar
    adfss990
    Copper Contributor

    Hi,

     

    Did anyone get this solved? I'm also facing similar problem: npm package (1.0.0) build artifact doesn't include global parameters. I tried the "updated https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-resource-manager-custom-parameters#default-parameterization-template" as suggested above without success.

     

    This results to validation error in deployment:

    "Deployment template validation failed: 'The template parameters "default_properties_xxx_value" in the parameters file are not valid; they are not present in the original template and can therefore not be provided at deployment time."

  • joanteixi's avatar
    joanteixi
    Copper Contributor

    Hi 

     

    I'm using the @microsoft/azure-data-factory-utilities npm package to deploy my datafactory and I always get an error related to a global parameter that didn't exists... some of you have tried with a successfull result? 

     

    Thanks!

  • IvanRamirez - The generated arm template has issues if you have 'Include in ARM template (deprecated)' enabled in Global parameters/Customer managed key pages along with 'Include global parameters in ARM template' in Arm template page.

    Since you switched to the new way to include global parameters, please disable deprecated option of 'Include in ARM template' in Global parameters/Customer managed key pages and publish the factory to fix the generated arm template.

  • IvanRamirez's avatar
    IvanRamirez
    Copper Contributor

    Hi, I was already using parameters inside the ARM Template and with this change my CI/CD pipeline "ARM Template deployment: Resource Group scope" activity stopped working. 

     

    I did the switch to the new parameters inside the ARM Template configuration already. How can I get assistance with this?

     

     

  • JChrisaero's avatar
    JChrisaero
    Copper Contributor

    As a heads up, if you want to add Global Parameters to the ARMTemplateParametersForFactory.json file, this is not documented anywhere, and for my company clicking the Include global parameters in ARM template button did not add the global parameters to the factory, though there were globalParameter JSON keys in the parameter configuration file. I had to add these lines to the parameter configuration to get it to show in the Parameters file:

    "Microsoft.DataFactory/factories/globalparameters": {
            "*": {
                "*": {
                    "value": "="
                }
            }
        }
  • oyvind1500 This is a transparent to you existing ADO release processes.
    If you were earlier using separate steps to deploy the 'ARM template without global params' and 'Global params using PSH' in DevOps due to a known issue of factory-level params being overwritten during deployment, then this would help in getting rid of the PSH, since you could include the Global params in ARM template without breaking ADF level configurations during deployment.
    If you were already deploying Global params through the ARM template, the new capability does not add any other value and does not break your existing releases!

  • oyvind1500's avatar
    oyvind1500
    Copper Contributor

    hi. how does this affect the release process if we are using Devops to deploy the ARM template? Will it break?

     

    currently, we have some global parameters that change dependent on environment (i.e. for email address).