Azure Data Explorer (Kusto) Bindings for Azure Functions - Public Preview
Published May 23 2023 11:50 PM 6,062 Views
Microsoft

Screenshot 2023-05-23 171414.png

 

We're excited to announce the public preview of Azure Data Explorer (Kusto) Bindings for Azure Functions! With the bindings, read and write operations can be performed against Kusto clusters in declarative fashion leveraging the Azure functions framework. 

 

This new capability makes it simpler to build data processing pipelines that involve Kusto and enables you to easily integrate Kusto with other Azure services using Azure Functions. 

 

How does the Azure Data Explorer (Kusto) bindings work

Azure Data Explorer (Kusto) bindings provides input and output bindings for Azure Functions, which allow you to read and write data from and to Kusto clusters respectively. With these bindings, you can use Kusto as a data source or sink in your Azure Functions, enabling you to build end-to-end data processing pipelines. Here's an example of how it works:

 

  • Define an input binding that retrieves data from a Kusto cluster based on a specified query.
  • Processing and enrichment of the data in your Azure Function as needed.
  • Write the transformed data back to Kusto using the output binding.

Integrating this simple workflow with various Triggers offered by the Functions framework enables easy data movement between from/to Azure Data Explorer (Kusto) and other Azure services, making it simpler to build powerful data processing pipelines.

 

Getting Started

 

To get started with Azure Kusto Bindings for Azure Functions, you'll need:

  • An Azure subscription
  • A Kusto cluster
  • An Azure Function app

Here's how to get started:

    Log in to the Azure portal (https://portal.azure.com/).

    In the left-hand menu, click on "Create a resource".

    Search for "Function App" and click "Create".

  • Fill out the required fields, including the resource group, name, and hosting plan.
  • Click "Create".

Once you've created your Azure Function app, you'll need to install the Microsoft.Azure.WebJobs.Extensions.Kusto package. This can be installed on the solution using the NuGet Package Manager or by running the following command in the Package Manager Console:

 

Install-Package Microsoft.Azure.WebJobs.Extensions.Kusto -Version {version}

 

Creating a simple functions project

The following illustrates a simple function project in Visual studio.

 

Ramacg_0-1684908669012.png

 

Ramacg_1-1684908669023.png

 

Ramacg_2-1684908669047.png

 

Ramacg_3-1684908669075.png

 

Ramacg_4-1684908669111.png

 

Sample Input Bindings

An extremely simplistic example of input bindings is as follows.

 

[Kusto(Database:Samples ,KqlCommand = declare query_parameters (productId:long);Products | where ProductID == productId ,KqlParameters = "@productId={productId} ", // productId is parameter passed from a trigger
Connection = "KustoConnectionString")] IAsyncEnumerable<Product> productList

 

The parameters of the bindings are as explained in the Webjobs.Extensions.Kusto/README.md at main · Azure/Webjobs.Extensions.Kusto · GitHub

 

This query sends a query against the Products sample table allowing a filter by productId. The trigger for example can pass in the product id as the query parameter. The output of this then collected to the Enumerable variable productList.

On deployment the function on a sample Azure function App, the following binding details should show up corresponding to the declaration of the binding.

 

Ramacg_4-1684839977740.png

 

The complete set of input binding samples are published on Github at : Webjobs.Extensions.Kusto/samples/samples-csharp/InputBindingSamples at main · Azure/Webjobs.Extensio...

 

Some sample scenarios where Input bindings can be used are: 

Sample Output Bindings

An extremely simplistic example of output bindings is as below.

 

[Kusto(Database:Samples ,TableName = "Products”,Connection = "KustoConnectionString")] out Product[] collector

 

The parameters of the bindings are as explained in the Webjobs.Extensions.Kusto/samples/samples-csharp at main · Azure/Webjobs.Extensions.Kusto · GitHub

This binding ingests data to a Products table from the value set on the variable collector. This variable can be populated from values processed, for example a product message posted on a HTTPTrigger or a set of products ingested on a blob.

 

On deployment the function on a sample Azure function App, the following binding details should show up corresponding to the declaration of the binding.

 

Ramacg_5-1684839977817.png


The complete set of output binding samples are published on Github at:

Webjobs.Extensions.Kusto/samples/samples-csharp/OutputBindingSamples at main · Azure/Webjobs.Extensi...

 

The bindings can also be invoked imperatively (leveraging the Azure functions framework) and can be adapted to the use cases at hand. A sample of this is provided at : Webjobs.Extensions.Kusto/TimeBasedExport.cs at main · Azure/Webjobs.Extensions.Kusto · GitHub

 

Some sample scenarios where Output bindings can be used are: 

Deployment to Azure

Deployment to Function apps on Azure can be performed directly from the Visual studio tooling and can be monitored from the portal. In this example the samples provided are published to Azure function app

Function tools can also be used to deploy function apps to Azure.

 

func azure functionapp publish <name-of-the-function-app>

 

Ramacg_6-1684839977839.png

 

Conclusion

With Azure Data Explorer (Kusto) Bindings for Azure Functions, it's now easier than ever to read and write data from and to Kusto clusters in your Azure Functions (docs). This new capability enables you to easily integrate Kusto with other Azure services and build powerful data processing pipelines. Try it out today and let us know what you think!

 

3 Comments
Co-Authors
Version history
Last update:
‎May 23 2023 11:48 PM
Updated by: