Forum Discussion

ic2092's avatar
ic2092
Copper Contributor
Sep 17, 2024

Extending by a function output

 

datatable(ids: dynamic)
[
    dynamic(["value1", "value2"])
]
| function(ids)

 

This above snippet works fine and returns a table

 

datatable(date: datetime, ids: dynamic)
[
    datetime(2022-01-01), dynamic(["value1", "value2"]),
    datetime(2022-01-02), dynamic(["value3", "value4"])
]
| extend outputs = function(ids)

 

This one however complains that extend expects a scalar and not table that the function returns

 

datatable(date: datetime, ids: dynamic)
[
    datetime(2022-01-01), dynamic(["value1", "value2"]),
    datetime(2022-01-02), dynamic(["value3", "value4"])
]
| extend outputs = toscalar(function(ids))

 

When using toscalar, ids cannot be referenced. Is there a workaround?

The function take in dynamic and returns a tubular expression of one row and two columns

3 Replies

  • To work around this issue in Kusto (Azure Data Explorer), you can restructure your approach by using a join instead of extend, as extend only works with scalar values. Since your function returns a table, join can be used to align your datatable with the function output for each row.

    Here’s a workaround you can try:

    1. Use mv-expand to expand the dynamic array of IDs.
    2. Apply the function on each expanded value.
    3. Re-aggregate if needed to get the original structure with function results.

    Here’s an example of how this can be done:

    // Sample function that takes dynamic input and returns a table with one row and two columns.
    let function = (ids: dynamic)
    {
        datatable(col1: string, col2: string)
        [
            tostring(ids[0]), tostring(ids[1])
        ]
    };
    
    // Original table with date and dynamic ids
    let original_table = datatable(date: datetime, ids: dynamic)
    [
        datetime(2022-01-01), dynamic(["value1", "value2"]),
        datetime(2022-01-02), dynamic(["value3", "value4"])
    ];
    
    // Expand ids, apply function, and reassemble
    original_table
    | mv-expand id = ids // Expand to create a row for each id
    | extend result = function(dynamic([id])) // Apply the function to each id
    | project-away id // Remove intermediate column
    | join kind=inner (
        original_table
    ) on date // Join back with the original table by date
    

    This approach:

    • Expands the ids dynamic array so each row has a single value from ids.
    • Applies the function on each expanded value.
    • Joins back with the original table on a common key (date in this case).

    Alternatively, if you have control over the function, you could modify it to return a scalar result based on aggregated data. But if it’s essential to use the function as is, then this join method allows you to align it with your original data.

  • Para obtener un resultado escalado, puedes agregar un paso adicional al final para transformar los resultados tabulares en escalares si es necesario.

    Por ejemplo, podrías sumar los valores obtenidos y así obtener un único valor escalar. Aquí te dejo un ejemplo:

    ```kusto
    let function = (ids: dynamic) {
    datatable(output1: string, output2: string)
    [
    tostring(ids[0]), tostring(ids[1])
    ]
    };
    datatable(date: datetime, ids: dynamic)
    [
    datetime(2022-01-01), dynamic(["value1", "value2"]),
    datetime(2022-01-02), dynamic(["value3", "value4"])
    ]
    | mv-apply ids on (
    extend (output1, output2) = function(ids)
    )
    | summarize total_outputs = strcat_array(make_list(output1), ", ") by date
    ```

    Este ejemplo utiliza `summarize` para concatenar los valores obtenidos en un solo valor escalar por fecha.
  • Cuando usas `extend` espera un escalar pero la función devuelve una tabla, puedes utilizar `mv-apply` para trabajar con datos tabulares dentro de la columna dinámica. Aquí hay un enfoque alternativo:

    ```kusto
    let function = (ids: dynamic) {
    datatable(output1: string, output2: string)
    [
    tostring(ids[0]), tostring(ids[1])
    ]
    };
    datatable(date: datetime, ids: dynamic)
    [
    datetime(2022-01-01), dynamic(["value1", "value2"]),
    datetime(2022-01-02), dynamic(["value3", "value4"])
    ]
    | mv-apply ids on (
    extend (output1, output2) = function(ids)
    )
    ```

    Al usar `mv-apply`, puedes aplicar la función a cada elemento de la columna dinámica y extender los resultados a nuevas columnas. Prueba este enfoque y dime si te funciona.

Resources