ARM Deploy Azure Data Explorer DB with Event Hub connection

Microsoft

Apr 05, 2022

We recently talked about the new capability to deploy Azure Data Explorer (ADX) solutions end-to-end using ARM template.

We have published an Azure ARM Quick Template doing exactly that.

Here is the solution it deploys:

The deployment was written using Bicep ARM Template that were transpiled into JSON ARM Template.

The Bicep version has the advantage of decoupling the KQL file using the Bicep function loadTextContent:

resource kustoScript 'scripts' = {
  name: 'db-script'
  properties: {
    scriptContent: loadTextContent('script.kql')
    continueOnErrors: false
  }
}

Now that you use Bicep or JSON template, you can include the script and have dependencies make sure the script is deployed when the database is ready and the data connection is deployed when the script has been executed, making sure the landing table used by the data connection exists in the database.

At the end, we have all the pieces of our pipeline deployed from one ARM template and our database can ingest data from our Event Hub!

The script deployment is the key component enabling this scenario. Today we can deploy up to 50 scripts per cluster (typically one script per database) ; that limitation will be lifted in a few months.

This capability will soon be available in Terraform as well!

Try that Quick ARM Template here and see our video demonstrating its use here: