Automated provisioning in Azure Data Explorer
Published Feb 07 2022 06:00 AM 1,563 Views
Microsoft

We recently published an article in the online documentation about deployment automation with Azure Data Explorer.  The goal was to centralize the information about automated deployment.

 

Automating deployment on a project is an up-front cost that pays dividends as the project evolves, generating agility.  When a process is automated, the cost of running it is low:  it becomes easy to destroy and rebuild an environment, to synchronize different environments, to rollback to a previous configuration, etc.  .

 

We structured the approach in three steps:

 

general-flow[1].png

 

For each step we provided a list of tools & approaches one can use.  For instance, deploying infrastructure (ADX cluster, its databases, data connections, VNET configuration, etc.), can be done with ARM templates, Terraform or imperative approach (e.g. Azure CLI).

 

Some projects will use only some of those steps.  Some will use all of them.  An IT team is only responsible for the infrastructure.  A project team might be responsible for the schema entities (tables, functions, policies, etc.).  A more integrated DevOps team might be responsible for both and will also load data in their test environment to run automated tests and / or prepare a QA environment.

 

DevOps Job ResultDevOps Job ResultAt the end of the article we gave a concrete example with an Azure DevOps Pipeline orchestrating those three steps.  This complete pipeline deploys a cluster with a database, 2 tables and loads data into one of the tables.

 

We hope this guidance helps you accelerate the automation of your projects.  We encourage you to go and start automating parts of your projects to gain agility.  Start with low-hanging fruits, measure the benefit and continue when it makes sense.

Co-Authors
Version history
Last update:
‎Feb 04 2022 12:35 PM
Updated by: