Over the last few months, we have been hearing our partners and customers tell us that they want to take advantage of the power and performance of Azure Data Explorer (Kusto) for interactive analytics over real time telemetry data. They were wondering how to smoothly and seamlessly migrate from their current databases and query tools to Azure Data Explorer. One such common scenario is that end-users use Kibana to query and explore log and telemetry data with Elasticsearch as the data store. Leveraging Azure Data Explorer for these scenarios increases the analytical power available for users, and reduces the cost, operational complexity and query latency. However, until now it has not been possible for users to continue using Kibana as they were used to, with Azure Data Explorer as the new data store and analytics service.
Delivering on our mission to empower you to achieve more, we are excited to announce K2Bridge (Kibana-Kusto Bridge), an open-source project that offers connectivity between the familiar Kibana interface to Azure Data Explorer. You can now easily ingest data to Azure Data Explorer or migrate data from Elasticsearch to Azure Data Explorer and then continue to use Kibana alongside the other experiences for querying data supported by Azure Data Explorer. K2Bridge is a containerized application that acts as a proxy between Kibana and an Azure Data Explorer cluster. K2Bridge translates Kibana queries to Kusto Query Language (KQL) and sends the results returned from Azure Data Explorer back to Kibana.
Currently, K2Bridge supports Kibana’s ”Discover" tab, where you can quickly search and explore the data, filter results, add or remove fields from the results grid, view record content, save and share searches. The image below shows Kibana connected to Azure Data Explorer by the K2Bridge. The search experience in Kibana, and the overall look and feel is as usual.
Query tabular function data
K2Bridge can be configured to access any Azure Data Explorer table in any database under your cluster. K2Bridge can also access Azure Data Explorer tabular functions. Such functions can be used to join or union multiple tables into a single tabular output table, or to redefine a table’s schemas on dynamic (JSON-like) columns, effectively creating a stored view. You can then query this stored view from Kibana as if it was a "regular" table.
Get started today. Install K2Bridge
You are invited to explore our GitHub repository and install the K2Bridge connector.
To find out more about K2Bridge see the documentation.
You can also view a step-by-step installation video as part of a Pluralsight free course.
Credits
This project was made thanks to a collaboration between Microsoft CSE (Commercial Software Engineering) group and the Azure Data Explorer team. Together, we worked directly with an engaged customer to solve real-world problems that came from him, and lay the groundwork for other customers who are interested in such a solution.
What's Next
K2Bridge is a Microsoft open-source project, more Kibana functionality will be added over time by our team and by the rest of the community. We welcome you to contribute and collaborate! All contributions and suggestions are welcome. More details can be found on the contribution section of the GitHub repository
Feel free to leave any feedback or requests on the GitHub issues page, on the Feedback Form, or just leave a comment below.
Guy Reginiano
The Azure Data Explorer team
Microsoft