We're excited to announce the public preview of Azure Data Explorer (Kusto) Bindings for Azure Functions! With the bindings, read and write operations can be performed against Kusto clusters in declarative fashion leveraging the Azure functions framework.
This new capability makes it simpler to build data processing pipelines that involve Kusto and enables you to easily integrate Kusto with other Azure services using Azure Functions.
How does the Azure Data Explorer (Kusto) bindings work
Azure Data Explorer (Kusto) bindings provides input and output bindings for Azure Functions, which allow you to read and write data from and to Kusto clusters respectively. With these bindings, you can use Kusto as a data source or sink in your Azure Functions, enabling you to build end-to-end data processing pipelines. Here's an example of how it works:
- Define an input binding that retrieves data from a Kusto cluster based on a specified query.
- Processing and enrichment of the data in your Azure Function as needed.
- Write the transformed data back to Kusto using the output binding.
Integrating this simple workflow with various Triggers offered by the Functions framework enables easy data movement between from/to Azure Data Explorer (Kusto) and other Azure services, making it simpler to build powerful data processing pipelines.
Getting Started
To get started with Azure Kusto Bindings for Azure Functions, you'll need:
- An Azure subscription
- A Kusto cluster
- An Azure Function app
Here's how to get started:
Log in to the Azure portal (https://portal.azure.com/).
In the left-hand menu, click on "Create a resource".
Search for "Function App" and click "Create".
- Fill out the required fields, including the resource group, name, and hosting plan.
- Click "Create".
Once you've created your Azure Function app, you'll need to install the Microsoft.Azure.WebJobs.Extensions.Kusto package. This can be installed on the solution using the NuGet Package Manager or by running the following command in the Package Manager Console:
Install-Package Microsoft.Azure.WebJobs.Extensions.Kusto -Version {version}
Creating a simple functions project
The following illustrates a simple function project in Visual studio.
Sample Input Bindings
An extremely simplistic example of input bindings is as follows.
[Kusto(Database:Samples ,KqlCommand = declare query_parameters (productId:long);Products | where ProductID == productId ,KqlParameters = "@productId={productId} ", // productId is parameter passed from a trigger
Connection = "KustoConnectionString")] IAsyncEnumerable<Product> productList
The parameters of the bindings are as explained in the Webjobs.Extensions.Kusto/README.md at main · Azure/Webjobs.Extensions.Kusto · GitHub
This query sends a query against the Products sample table allowing a filter by productId. The trigger for example can pass in the product id as the query parameter. The output of this then collected to the Enumerable variable productList.
On deployment the function on a sample Azure function App, the following binding details should show up corresponding to the declaration of the binding.
The complete set of input binding samples are published on Github at : Webjobs.Extensions.Kusto/samples/samples-csharp/InputBindingSamples at main · Azure/Webjobs.Extensio...
Some sample scenarios where Input bindings can be used are:
Sample Output Bindings
An extremely simplistic example of output bindings is as below.
[Kusto(Database:Samples ,TableName = "Products”,Connection = "KustoConnectionString")] out Product[] collector
The parameters of the bindings are as explained in the Webjobs.Extensions.Kusto/samples/samples-csharp at main · Azure/Webjobs.Extensions.Kusto · GitHub
This binding ingests data to a Products table from the value set on the variable collector. This variable can be populated from values processed, for example a product message posted on a HTTPTrigger or a set of products ingested on a blob.
On deployment the function on a sample Azure function App, the following binding details should show up corresponding to the declaration of the binding.
The complete set of output binding samples are published on Github at:
The bindings can also be invoked imperatively (leveraging the Azure functions framework) and can be adapted to the use cases at hand. A sample of this is provided at : Webjobs.Extensions.Kusto/TimeBasedExport.cs at main · Azure/Webjobs.Extensions.Kusto · GitHub
Some sample scenarios where Output bindings can be used are:
Deployment to Azure
Deployment to Function apps on Azure can be performed directly from the Visual studio tooling and can be monitored from the portal. In this example the samples provided are published to Azure function app
Function tools can also be used to deploy function apps to Azure.
func azure functionapp publish <name-of-the-function-app>
Conclusion
With Azure Data Explorer (Kusto) Bindings for Azure Functions, it's now easier than ever to read and write data from and to Kusto clusters in your Azure Functions (docs). This new capability enables you to easily integrate Kusto with other Azure services and build powerful data processing pipelines. Try it out today and let us know what you think!