Cross Service Query - Azure Monitor and Azure Data Explorer

Published Nov 24 2020 12:32 AM 2,887 Views
Microsoft
Azure Monitor and Azure Data Explorer cross-service querying
This experience enables you to query Azure Data Explorer in Azure Log Analytics/Application Insights tools (See more info here),
and the ability to query Log Analytics/Application Insights from Azure Data Explorer tools to make cross resource queries. (See more info here.),

For example (querying Azure Data Explorer from Log Analytics):

Where the outer query is querying a table in the workspace, and then joining with another table in an Azure Data Explorer cluster (in this case, clustername=help, databasename=samples) by using a new "adx()" function, like how you can do the same to query another workspace from inside query text.
Both experiences are in Public Preview.
4 Comments
Regular Visitor

Can you explain the considerations that should be taken when planning to use cross-service querying vs. other solutions (such as copying the data from the other service into ADX)? I'm especially interested in performance considerations. 

Microsoft

Sure @maayanh . There are customers who are having an Azure Data Explorer cluster along with their Log Analytics workspaces, there are times on which the customer want to join the two data sources either for a query result or a dashboard, in these cases he can copy his data out side of the cluster or workspace (which requires him to have a copy of the data, hence probably pay more, or he can use the cross service ability to connect the two services.

Regular Visitor

@OREN_SALZBERG Thank you. I'm trying to understand if there is a performance penalty to the cross-service query, similar to the penalties of cross-database queries done over linked databases. 

For example, if I have ADX + Log Analytics, and I need a query which combines data from both sources.

If I understand correctly, I can either:

1. Periodically copy the data from Log Analytics into ADX, and perform an ADX-only query

2. Perform a cross service ADX+Log Analytics query

So my question is: is there a significant performance penalty for the 2nd option over the 1st, in terms of the query alone.

 

Microsoft

You are correct on your options. regarding performance issues, I have tested the feature with several environments and many early onboarders to our private preview, and couldn't find any performance penalties.

 

Please note that this service is still in public preview, with support, and we still work on improving any issues encountered using the service.

%3CLINGO-SUB%20id%3D%22lingo-sub-1926080%22%20slang%3D%22en-US%22%3ECross%20Service%20Query%20-%20Azure%20Monitor%20and%20Azure%20Data%20Explorer%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-1926080%22%20slang%3D%22en-US%22%3E%3CP%3E%3CSTRONG%3E%3CU%3EAzure%20Monitor%26lt%3B-%26gt%3BAzure%20Data%20Explorer%20cross-service%20querying%20%3C%2FU%3E%3C%2FSTRONG%3E%3C%2FP%3E%0A%3CP%3EThis%20experience%20enables%20you%20to%20query%20Azure%20Data%20Explorer%20in%20Azure%20Log%20Analytics%2FApplication%20Insights%20tools%20(See%20more%20info%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fazure-monitor%2Fplatform%2Fazure-monitor-data-explorer-proxy%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3Ehere%3C%2FA%3E)%2C%3C%2FP%3E%0A%3CP%3Eand%20the%20ability%20to%20query%20Log%20Analytics%2FApplication%20Insights%20from%20Azure%20Data%20Explorer%20tools%20to%20make%20cross%20resource%20queries.%20(See%20more%20info%20%3CA%20href%3D%22https%3A%2F%2Fnam06.safelinks.protection.outlook.com%2F%3Furl%3Dhttp%253A%252F%252Faka.ms%252Fadxproxy%26amp%3Bdata%3D04%257C01%257CWWCOperationsManagement%2540service.microsoft.com%257C673832ca1a774ee9064f08d88f9f0c81%257C72f988bf86f141af91ab2d7cd011db47%257C1%257C0%257C637417261983627273%257CUnknown%257CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%253D%257C1000%26amp%3Bsdata%3DhNubK%252FY2LTAVa%252FiUjOLI2k0wCBzssZOa3KMsppjAvOQ%253D%26amp%3Breserved%3D0%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Ehere%3C%2FA%3E.)%2C%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22adx-proxy-workflow.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F235736iCD7409084B8846FB%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22adx-proxy-workflow.png%22%20alt%3D%22adx-proxy-workflow.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EFor%20example%20(querying%20Azure%20Data%20Explorer%20from%20Log%20Analytics)%3A%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-left%22%20image-alt%3D%222020-11-24_10-24-28.png%22%20style%3D%22width%3A%20613px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F235735i514C72098765D852%2Fimage-dimensions%2F613x49%3Fv%3D1.0%22%20width%3D%22613%22%20height%3D%2249%22%20role%3D%22button%22%20title%3D%222020-11-24_10-24-28.png%22%20alt%3D%222020-11-24_10-24-28.png%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3E%3CSPAN%20style%3D%22font-family%3A%20inherit%3B%22%3E%3CBR%20%2F%3EWhere%20the%20outer%20query%20is%20querying%20a%20table%20in%20the%20workspace%2C%20and%20then%20joining%20with%20another%20table%20in%20an%20Azure%20Data%20Explorer%20cluster%20(in%20this%20case%2C%20clustername%3Dhelp%2C%20databasename%3Dsamples)%20by%20using%20a%20new%20%22adx()%22%20function%2C%20like%20how%20you%20can%20do%20the%20same%20to%20query%20another%20workspace%20from%20inside%20query%20text.%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EBoth%20experiences%20are%20in%20Private%20Preview.%3C%2FP%3E%0A%3CP%3EThe%20ability%20to%20query%20Azure%20Monitor%20from%20Azure%20Data%20Explorer%20is%20open%20for%20everyone%20to%20use%20%E2%80%93%20%3CSTRONG%3Eno%20need%20to%20be%20allowlisted%3C%2FSTRONG%3E%2C%3C%2FP%3E%0A%3CP%3EThe%20ability%20to%20query%20Azure%20Data%20Explorer%20from%20Log%20Analytics%2FApplication%20Insights%20%3CSTRONG%3Erequires%20to%20be%20allowlisted%3C%2FSTRONG%3E%20-%20%3CU%3EWe%20need%20the%20following%20to%20get%20you%20enrolled%20(%3CA%20href%3D%22mailto%3Aorens%40microsoft.com%22%20target%3D%22_blank%22%20rel%3D%22noopener%20nofollow%20noreferrer%22%3Eyou%20can%20send%20the%20info%20to%20me%3C%2FA%3E)%3C%2FU%3E%3A%3C%2FP%3E%0A%3COL%3E%0A%3CLI%3ETenant%20ID%3C%2FLI%3E%0A%3CLI%3EList%20of%20the%20Azure%20Data%20Explorer%20clusters%20(the%20list%20is%20required%20to%20enable%20the%20team%20to%20modify%20the%20callout%20policy%20of%20that%20cluster%2C%20that%20will%20allow%20them%20to%20communicate%20with%20the%20proxy)%3C%2FLI%3E%0A%3CLI%3EEmail%20address%3C%2FLI%3E%0A%3C%2FOL%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CP%3EWe%20started%20a%20%3CSTRONG%3Eprivate%20preview%20program%3C%2FSTRONG%3E%2C%20and%20we%20are%20happy%20to%20add%20early%20adopters%20to%20experience%20the%20new%20functionality.%3C%2FP%3E%0A%3CP%3EPlease%20note%20that%20the%20product%20is%20new%20with%20limited%20SLA%2C%20and%20we%20estimate%20that%20we%20will%20be%20able%20to%20move%20to%20pubic%20preview%20with%20production%20level%20SLA%20within%20~2-4%20months.%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-1926080%22%20slang%3D%22en-US%22%3E%3CP%3EQuery%20Azure%20Data%20Explorer%20data%20through%20Azure%20Log%20Analytics%20tools%20(or%20the%20other%20way%20around)%20to%20join%20and%20analyze%20all%20your%20data%20in%20one%20place.%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-SUB%20id%3D%22lingo-sub-2116799%22%20slang%3D%22en-US%22%3ERe%3A%20Cross%20Service%20Query%20-%20Azure%20Monitor%20and%20Azure%20Data%20Explorer%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2116799%22%20slang%3D%22en-US%22%3E%3CP%3ECan%20you%20explain%20the%20considerations%20that%20should%20be%20taken%20when%20planning%20to%20use%20cross-service%20querying%20vs.%20other%20solutions%20(such%20as%20copying%20the%20data%20from%20the%20other%20service%20into%20ADX)%3F%20I'm%20especially%20interested%20in%20performance%20considerations.%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E
Co-Authors
Version history
Last update:
‎Jul 30 2021 05:13 AM
Updated by: