Cross Service Query - Azure Monitor and Azure Data Explorer

[OREN_SALZBERG]

Azure Monitor<->Azure Data Explorer cross-service querying

This experience enables you to query Azure Data Explorer in Azure Log Analytics/Application Insights tools (See more info here),

and the ability to query Log Analytics/Application Insights from Azure Data Explorer tools to make cross resource queries. (See more info here.),

For example (querying Azure Data Explorer from Log Analytics):

Where the outer query is querying a table in the workspace, and then joining with another table in an Azure Data Explorer cluster (in this case, clustername=help, databasename=samples) by using a new "adx()" function, like how you can do the same to query another workspace from inside query text.

Both experiences are in Private Preview.

The ability to query Azure Monitor from Azure Data Explorer is open for everyone to use – no need to be allowlisted,

The ability to query Azure Data Explorer from Log Analytics/Application Insights requires to be allowlisted - We need the following to get you enrolled (you can send the info to me):

1. Tenant ID
2. List of the Azure Data Explorer clusters (the list is required to enable the team to modify the callout policy of that cluster, that will allow them to communicate with the proxy)
3. Email address

We started a private preview program, and we are happy to add early adopters to experience the new functionality.

Please note that the product is new with limited SLA, and we estimate that we will be able to move to pubic preview with production level SLA within ~2-4 months.

[maayanh]

Can you explain the considerations that should be taken when planning to use cross-service querying vs. other solutions (such as copying the data from the other service into ADX)? I'm especially interested in performance considerations. 

[OREN_SALZBERG]

Sure @maayanh . There are customers who are having an Azure Data Explorer cluster along with their Log Analytics workspaces, there are times on which the customer want to join the two data sources either for a query result or a dashboard, in these cases he can copy his data out side of the cluster or workspace (which requires him to have a copy of the data, hence probably pay more, or he can use the cross service ability to connect the two services.

[maayanh]

@OREN_SALZBERG Thank you. I'm trying to understand if there is a performance penalty to the cross-service query, similar to the penalties of cross-database queries done over linked databases. 

For example, if I have ADX + Log Analytics, and I need a query which combines data from both sources.

If I understand correctly, I can either:

1. Periodically copy the data from Log Analytics into ADX, and perform an ADX-only query

2. Perform a cross service ADX+Log Analytics query

So my question is: is there a significant performance penalty for the 2nd option over the 1st, in terms of the query alone.

[OREN_SALZBERG]

You are correct on your options. regarding performance issues, I have tested the feature with several environments and many early onboarders to our private preview, and couldn't find any performance penalties.

Please note that this service is still in public preview, with support, and we still work on improving any issues encountered using the service.