Video series by boB Taylor, intro by Edwin Hernandez
Hello everyone! This time we bring you a series of video tutorials by boB Taylor, who is a member of the Microsoft's Performance & Quality Services Team. He recently completed this video series about Kusto Query Language, and this is a great opportunity for us to compile it into a single place for you along with some introductory information.
What is Kusto Query Language (KQL)?
You may be asking yourself: What is Kusto? What is KQL? And why would this matter for a Quality and Performance Test Blog? Well, Kusto itself was the internal code name for Azure Data Explorer, and Kusto Query Language (KQL) is the primary means of interaction with it. KQL allows you to send data queries, process data, and return the results of this processing without modifying the data or metadata. Now, why would this matter for Test Engineers? Let's first define what Azure Data Explorer can do for you:
Azure Data Explorer
Azure Data Explorer is a service that allows you to store log and telemetry data. It can handle many data streams emitted by several entities concurrently (websites, LOB apps, CRMs, Social data sources, IoT devices, Cloud components, etc.). This data is collected and stored. Data Explorer then provides a way for you to analyze this large volume of data, perform complex queries and drill down into specific events.
Therefore, Azure Data Explorer provides a great way for you to perform diagnostics, monitoring, and reporting of all components of your application and environment. This kind of analysis is key for performance testing and application optimization.
Kusto Query Language(KQL)
KQL is a read-only query language. The syntax is similar to SQL, but it was created specifically to work with large datasets in Azure. Since it's read-only there are no update or delete clauses. It is based on relational management systems, which use schema entities, and is organized into a hierarchy like SQL's databases tables and columns.
Other Azure Services
KQL is the primary mean to query data from Azure Data Explorer; however, it is also used to interact with the following inter-related Azure services:
- Application Insights
- Log Analytics
- Azure Monitor, and again:
- Azure Data Explorer
Microsoft documentation
If you want to learn more from the official Microsoft documentation, make sure to check out the following:
- https://docs.microsoft.com/en-us/azure/data-explorer/data-explorer-overview Check this one to learn about the possible data sources and workflows.
- https://docs.microsoft.com/en-us/azure/azure-monitor/app/app-insights-overview Fundamentals, how to implement the telemetry, do analysis, etc. We hope to bring you a detailed guide on this blog in the future.
- https://docs.microsoft.com/en-us/azure/azure-monitor/overview. Solution for collecting, analyzing, and acting on telemetry from your cloud and on-premises environments.
- https://docs.microsoft.com/en-us/azure/azure-monitor/logs/log-analytics-overview Tool in the Azure portal used to edit and run log queries with data in Azure Monitor Logs.
- https://docs.microsoft.com/en-us/azure/data-explorer/kusto/concepts/. Check this one to learn the basics about KQL, where there is a tutorial and samples.
- For a more hands-on kind of learning, check out this lab in Microsoft Learn: https://docs.microsoft.com/en-us/learn/modules/monitor-performance-using-azure-monitor-for-vms/
boB Taylor's Video Series
If you are done leafing through the official documentation above and want a much more detailed training. boB created a series of video tutorials where he goes into detail on KQL, from the fundamentals to complex functions, passing through the most common operators:
|
1 |
What is KQL and why should I care? |
http://www.sqlbobt.com/first-up-on-the-menu-an-appetizer-what-is-kql-and-why-should-i-care/ |
|
2 |
The Editors or how do I write and execute KQL queries |
http://www.sqlbobt.com/next-on-the-menu-the-editors-or-how-do-i-write-and-execute-kql-queries/ |
|
3 |
Our first KQL operators |
http://www.sqlbobt.com/next-on-the-menu-our-first-kql-operators/ |
|
4 |
Wait where can I consume Kusto data? |
http://www.sqlbobt.com/wait-where-can-i-consume-kusto-data/ |
|
5 |
Summarize |
http://www.sqlbobt.com/next-on-the-menu-summarize/ |
|
6 |
Project, extend, and explain |
http://www.sqlbobt.com/next-on-the-menu-project-extend-and-explain/ |
|
7 |
Distinct, sample-distinct, top, and top-nested |
http://www.sqlbobt.com/next-on-the-menu-distinct-sample-distinct-top-and-top-nested/ |
|
8 |
Scalar functions part 1 |
http://www.sqlbobt.com/next-on-the-menu-scalar-functions-part-1/ |
|
9 |
More scalar operators! |
http://www.sqlbobt.com/next-on-the-menu-more-scalar-operators/ |
|
10 |
Fun with datetime, timespan and date_part |
http://www.sqlbobt.com/next-on-the-menu-fun-with-datetime-timespan-and-date_part/ |
|
11 |
Conditional logic and strings |
http://www.sqlbobt.com/next-on-the-menu-conditional-logic-and-strings/ |
|
12 |
Advanced aggregation operators |
http://www.sqlbobt.com/next-on-the-menu-advanced-aggregation-operators/ |
|
13 |
Let, join, and union |
http://www.sqlbobt.com/next-on-the-menu-let-join-and-union/ |
|
14 |
Datatable, prev, next, row_cumsum, and materialize |
http://www.sqlbobt.com/next-on-the-menu-datatable-prev-next-row_cumsum-and-materialize/ |
|
15 |
Range, make-series |
http://www.sqlbobt.com/sql-snacks-are-back-range-make-series/ |
|
16 |
Series_decompose family of functions |
http://www.sqlbobt.com/next-on-the-menu-series_decompose-family-of-functions/ |
|
17 |
The remaining time series functions |
http://www.sqlbobt.com/next-on-the-menu-the-remaining-time-series-functions/ |
|
18 |
Machine Learning Plugins |
http://www.sqlbobt.com/next-on-the-menu-machine-learning-plugins/ |
|
19 |
User Analytics Plug-ins |
http://www.sqlbobt.com/1388-2/ |
|
20 |
Last on KQL - charting |
http://www.sqlbobt.com/last-on-the-kusto-query-language-menu-charting/ |
In Conclusion…
If you want to monitor and diagnose the performance of your application in Azure, KQL is a key resource. It is a good skill to have in your portfolio, especially if you are involved in load testing. Please make sure to check our other article about a Collection of Useful Tool for Performance Test Engineers, and please leave any questions in the comments section.
Thanks!
