ARM Deployment considerations for Azure Data Factory
Published Sep 21 2022 10:25 AM 2,955 Views
Microsoft

Some of the main goals for adopting DevOps culture in our organization are the reduction of failures in new deployments, be able to update our solutions frequently, improving deployments time, among others.

Implementing DevOps processes into your Team requires trust and responsibility, because as Uncle Ben said, “With great power comes great responsibility.” It’s very common to have elevated access to perform almost or sometimes all actions in an environment. With an Elevated Account or Service Principal, there are some important aspects that we need to consider in order to prevent a disaster.

In this case, I focus on Azure Data Factory (ADF) because it has a special treatment when integrating automatization deployments in Azure DevOps.

Here is the official documentation: Automate continuous integration using Azure Pipelines releases

 

As described, to deploy changes that were built into our ADF we have to use “ARM Template Deployment” task which is used to deploy all kind of ARM templates into our environment, but this task has an important and very powerful option, which is “Deployment mode”.

Deployment mode can be “Incremental”, “Complete” and “Validation only”. To see information about what these modes do, you can click the little “i” symbol. Incremental mode will deploy, and update resources described in the ARM template. Validation only will make sure there is access and that the template and parameters are well formed. The option most people don’t need, that is dangerous is “Complete mode”. Complete mode says to make an environment (Management Group, Subscription, or Resource Group) look EXACTLY like the provided ARM template. That means that any resource not defined will be deleted. In ADF deployments, if you have other resources in the same resource group that aren’t in the ADF ARM template, they will be deleted.

There are ways to help mitigate this in case that happened.

 

  • Lock or add a policy in the Resource Group to avoid deletion

Lock your resources to protect your infrastructure

Tutorial: Create and manage policies to enforce compliance

 

  • Integrate Infrastructure as Code

What is infrastructure as code (IaC)?

 

Other General considerations

  • Fully define your environments and components in Infrastructure as Code so that you can quickly recreate environments either for testing or for Disaster Recovery
  • Test things in multiple environments first

Repeatable Infrastructure

 

Security is a priority. In all aspects of a solution. Have a plan for (BC/DR) Business Continuity / Disaster Recovery from the beginning. That includes testing deployments in environments and having ways to recreate your environment. Make sure that you understand how ARM templates are deployed if using them for deployments. Thank you and please consider these recommendations.

 

1 Comment
Version history
Last update:
‎Sep 20 2022 03:06 PM
Updated by: