Forum Discussion
Azure Data Explorer - Pivot Table
Hi,
In an IoT project we are gathering sensor data in Azure Data Explorer. All sensor data is stored in a "signals" table. To uniqely identify a timeseries for a given sensor, we query like this:
signals
| where TestId == "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3" and SignalName == "Signal1"
We want to be able to Pivot all timeseries from a given TestId, from the "signals" Table Rows into Columns.
I have been unable to write a Kusto Query that Achieves this, and I am hoping for some help on this forum.
The current signals table schema looks like this:
Timestamp | TestId | SignalName | Value |
2021-01-01 12:00:30 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | Signal1 | 23400 |
2021-01-01 12:00:30 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | Signal2 | 0.113 |
2021-01-01 12:00:30 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | Signal3 | 77.5 |
2021-01-01 12:00:31 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | Signal1 | 23450 |
2021-01-01 12:00:31 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | Signal2 | 0.114 |
2021-01-01 12:00:31 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | Signal3 | 75.4 |
2021-01-01 12:00:32 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | Signal1 | 22450 |
2021-01-01 12:00:32 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | Signal2 | 0.113 |
2021-01-01 12:00:32 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | Signal3 | 80.05 |
I want to be able to Pivot the Table, to the following Schema
Timestamp | TestId | Signal1 | Signal2 | Signal3 |
2021-01-01 12:00:30 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 23400 | 0.113 | 77.5 |
2021-01-01 12:00:31 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 23450 | 0.114 | 75.4 |
2021-01-01 12:00:32 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 22450 | 0.113 | 80.05 |
I have tried the following query:
let testId = "cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3";
signals
| where TestId == testId
| where SignalName == "Signal1" or SignalName == "Signal2" or SignalName == "Signal3"
| order by Timestamp desc
| evaluate pivot(SignalName)
But the resulting table, seems to repeat the timestamp - the timestamp is represented multiple times and a default value "0" is inserted in other signal columns:
Timestamp | TestId | Signal1 | Signal2 | Signal3 |
2021-01-01 12:00:30 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 23400 | 0 | 0 |
2021-01-01 12:00:30 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 0 | 0.113 | 0 |
2021-01-01 12:00:30 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 0 | 0 | 77.5 |
2021-01-01 12:00:31 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 23450 | 0 | 0 |
2021-01-01 12:00:31 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 0 | 0.114 | 0 |
2021-01-01 12:00:31 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 0 | 0 | 75.4 |
2021-01-01 12:00:32 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 22450 | 0 | 0 |
2021-01-01 12:00:32 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 0 | 0.113 | 0 |
2021-01-01 12:00:32 | cbb8bff1-ee9d-4ead-bbd6-c9c246d84fd3 | 0 | 0 | 80.05 |
I do not need to do any aggregation using the Pivot operator, since all Signals should have a value on the exact same timestamp.
- Can anyone help me writing a KQL query for this?
- Do I need to create a Materialized View in Azure Data Explorer to Achieve this? An update Policy or Function?
Thanks