We are very excited to introduce the new Azure Data Explorer samples gallery feature. This provides an easy way to upskill and demo Azure Data Explorer with an end-to-end experience on a free, publicly available cluster known by the name of a ‘help’ cluster.
There is no need to ingest data or create a cluster, it's all done for you with the pre-loaded sample datasets. To learn, you just need to follow the tutorial, analyze data by executing provided sample queries, follow KQL commands to alter policies and explore parameterized dashboards.
To access the ‘help’ cluster, you need either an Azure active directory (AAD) user identity or a Microsoft account (MSA).
The samples gallery comprises of sample datasets with guided tutorials in the form of commonly used queries along with the detailed description and sample dashboards. You can navigate to the samples gallery from home page -
Following is a high-level view of the components in the samples gallery -
To get started with Azure Data Explorer, you can either read through the documentation and learn by trial and error, or you can use these samples to guide you through, as shown in the following 3 steps.
Step 1 - Pick a sample dataset - Logs, Metrics, IoT, or Basic from the home page
Step 2 - Follow the tutorial on each of these datasets to understand what you can achieve with this type of data. For example, the ‘SampleMetrics’ tutorial covers the following scenarios:
- 'SampleMetrics' comprises of SQL database monitoring metrics collected from multiple Azure SQL databases along with the reference data of server's location.
- Raw data is ingested into the staging table - ‘RawServerMetrics’.
- The ingestion batching policy on ‘RawServerMetrics’ table is adjusted to reduce the default ingestion latency from 5 minutes to 20 seconds.
- The data retention policy of ‘RawServerMetrics’ table can be adjusted to few or 0 days to avoid the duplication of data across raw and transformed tables. If needed, you can keep raw data for longer. For example, if there are any issues, rather than going back to the source data, you can refer to the data in the raw table.
- The update policy is applied to the destination table – ‘TransformedServerMetrics’ – to transform and parse complex raw json data with nested arrays.
- Commonly used KQL queries are provided for interactive analytics. For example, learn how to use the parse operator, bin function, mv-expand operator to parse complex json payloads with nested arrays, join fact and dimension tables.
- T-SQL is also supported along with KQL so you may use T-SQL to query and use 'Explain' operator in KQL to transform SQL to KQL. For more details on SQL to KQL conversions, refer to this cheatsheet.
- Lookup server's location reference data from another Azure Data Explorer table. The reference data can also be joined from SQL DB or Cosmos DB using these sql and cosmos plugins.
- Create materialized views( MVs) for deduplication of data, down sampling, and getting the last known/latest value.
A summary of above scenario's data flow is shown in the following diagram -
Step 3 - Explore a sample dashboard. There are 4 sample dashboards created on top of the sample datasets that will give you a fair idea on how easy it is to build dashboards with natively supported visuals and parameters to slice and dice data. When you select any of the sample dashboard, a copy of the dashboard will be generated for you to try -
Just to give you a flavor of another dataset, lets try creating 'Sample Logs Dashboard' -
- It has live ingestions so you can try auto refresh and see the number of records increasing
- Try filtering by time range filter - custom range or select from values in the drop down
- Try filtering logs based on text given in the 'Message' filter as shown below -
This is just a beginning, we will continue adding more datasets based on your feedback so please share your ideas here.
Enjoy learning!