Blog Post

FastTrack for Azure
2 MIN READ

ARM Deployment considerations for Azure Data Factory

Vanessa_Segovia's avatar
Sep 21, 2022

Some of the main goals for adopting DevOps culture in our organization are the reduction of failures in new deployments, be able to update our solutions frequently, improving deployments time, among others.

Implementing DevOps processes into your Team requires trust and responsibility, because as Uncle Ben said, “With great power comes great responsibility.” It’s very common to have elevated access to perform almost or sometimes all actions in an environment. With an Elevated Account or Service Principal, there are some important aspects that we need to consider in order to prevent a disaster.

In this case, I focus on Azure Data Factory (ADF) because it has a special treatment when integrating automatization deployments in Azure DevOps.

Here is the official documentation: Automate continuous integration using Azure Pipelines releases

 

As described, to deploy changes that were built into our ADF we have to use “ARM Template Deployment” task which is used to deploy all kind of ARM templates into our environment, but this task has an important and very powerful option, which is “Deployment mode”.

Deployment mode can be “Incremental”, “Complete” and “Validation only”. To see information about what these modes do, you can click the little “i” symbol. Incremental mode will deploy, and update resources described in the ARM template. Validation only will make sure there is access and that the template and parameters are well formed. The option most people don’t need, that is dangerous is “Complete mode”. Complete mode says to make an environment (Management Group, Subscription, or Resource Group) look EXACTLY like the provided ARM template. That means that any resource not defined will be deleted. In ADF deployments, if you have other resources in the same resource group that aren’t in the ADF ARM template, they will be deleted.

There are ways to help mitigate this in case that happened.

 

  • Lock or add a policy in the Resource Group to avoid deletion

Lock your resources to protect your infrastructure

Tutorial: Create and manage policies to enforce compliance

 

  • Integrate Infrastructure as Code

What is infrastructure as code (IaC)?

 

Other General considerations

  • Fully define your environments and components in Infrastructure as Code so that you can quickly recreate environments either for testing or for Disaster Recovery
  • Test things in multiple environments first

Repeatable Infrastructure

 

Security is a priority. In all aspects of a solution. Have a plan for (BC/DR) Business Continuity / Disaster Recovery from the beginning. That includes testing deployments in environments and having ways to recreate your environment. Make sure that you understand how ARM templates are deployed if using them for deployments. Thank you and please consider these recommendations.

 

Updated Sep 20, 2022
Version 1.0

1 Comment

  • Cameron_Mertz30's avatar
    Cameron_Mertz30
    Copper Contributor

    I believe that adopting a DevOps culture in an organization is crucial for reducing failures in new deployments, updating solutions frequently, and improving deployment time. However, implementing DevOps processes also requires trust and responsibility, especially when it comes to elevated access to perform actions in an environment.

     

    In the context of Azure Data Factory (ADF), it is essential to be aware of the potential dangers of using the "Complete mode" option when deploying changes through the ARM Template Deployment task. This option can cause resources not defined in the ARM template to be deleted, which can be disastrous.

     

    To prevent this from happening, I recommend either locking or adding a policy in the resource group to avoid deletion, or locking your resources to protect your infrastructure. Additionally, integrating Infrastructure as Code (IaC) can help mitigate this risk by fully defining environments and components in IaC so that they can be quickly recreated for testing or Disaster Recovery purposes.

     

    Other general considerations include testing things in multiple environments before deployment, as well as having a plan for recover deleted files from the beginning, including testing deployments in environments and having ways to recreate your environment.

     

    In summary, while adopting a DevOps culture can bring significant benefits, it's important to be aware of the potential risks and take steps to mitigate them. By following these recommendations, you can help ensure that your deployments are secure, reliable, and efficient.