ADX Insights - Azure Monitor for Azure Data Explorer (preview)

Published Jan 18 2021 11:37 PM 2,352 Views

We are excited to announce the public preview of Azure Data Explorer Insights!
ADX Insights (Azure Monitor for Azure Data Explorer) provides comprehensive monitoring of your clusters by delivering a unified view of your cluster performance, operations, and usage.

 

How ADX Insights can help - a success story to inspire

One of our users recently used ADX Insights. By using its "Cache" tab, they identified tables with redundant cache policy periods and unused tables with unnecessary cache.
After identifying such tables and adjusting the cache policy to the minimum required period, the hot cache size decreased dramatically.

Then, the optimized autoscale removed the unrequired VM instances (scale-in). The cluster scaled-in from 142 instances to 32 instances!
This demonstrates how cache policy optimization can result in a significant cost reduction (fewer VM instances are required to store the hot data).

Hot cache reduction (click to enlarge, then use the 'full size' button at the top-right corner)Hot cache reduction (click to enlarge, then use the 'full size' button at the top-right corner)

Instance count decreased following the hot cache reduction (click to enlarge, then use the 'full size' button at the top-right corner)Instance count decreased following the hot cache reduction (click to enlarge, then use the 'full size' button at the top-right corner)

User experience overview of ADX Insights

Built on the Azure Monitor Workbooks platform, ADX insights offers:

 

  • At scale perspective: showing a snapshot view of the clusters' primary metrics to easily track performance, ingestion, and export operations. Unsustainable values are highlighted in orange. You can drill down to the “at-resource” view by clicking on the cluster name.
    The 'At scale' perspective (click to enlarge, then use the 'full size' button at the top-right corner)The 'At scale' perspective (click to enlarge, then use the 'full size' button at the top-right corner)

     

  • At-resource perspective: drill-down analysis of a particular Azure Data Explorer cluster based on metrics and platform usage logs.
    With this view, you can:
    • Identify query lookback patterns per table and compare them to the table's cache policy.
      The Cache tab (click to enlarge, then use the 'full size' button at the top-right corner)The Cache tab (click to enlarge, then use the 'full size' button at the top-right corner)

       

    • Identify tables that are used by the most queries. 
    • Identify unused tables or redundant cache policy periods.
    • Find which tables are consuming the most space in the cluster
    • Track data growth history by table size, hot data, and the number of rows over time.
      The Tables tab (click to enlarge, then use the 'full size' button at the top-right corner)The Tables tab (click to enlarge, then use the 'full size' button at the top-right corner)

       

    • Identify which users and applications are sending the most queries or consuming the most CPU and memory.
    • Find changes in the number of queries by user and track the number of unique users over time.
    • See top users by command and query count and identify top users by the number of failed queries.
    • See the query count, CPU, and memory consumption over time.
      The Usage tab (click to enlarge, then use the 'full size' button at the top-right corner)The Usage tab (click to enlarge, then use the 'full size' button at the top-right corner)

       

    • Get a summary of active Advisor recommendations and resource health status.
      The Overview tab (click to enlarge, then use the 'full size' button at the top-right corner)The Overview tab (click to enlarge, then use the 'full size' button at the top-right corner)
    • Explore multiple key metrics on one page and discover correlations between them.
      The Key Metrics tab (click to enlarge, then use the 'full size' button at the top-right corner)The Key Metrics tab (click to enlarge, then use the 'full size' button at the top-right corner)

       

Azure Monitor for Azure Data Explorer is now available in the Azure Monitor and Azure Data Explorer blades in the Azure portal. We look forward to hearing your feedback on the new experience.
To learn more, see the ADX Insights documentation.


 

%3CLINGO-SUB%20id%3D%22lingo-sub-2070257%22%20slang%3D%22en-US%22%3EADX%20Insights%20-%20Azure%20Monitor%20for%20Azure%20Data%20Explorer%20(preview)%3C%2FLINGO-SUB%3E%3CLINGO-BODY%20id%3D%22lingo-body-2070257%22%20slang%3D%22en-US%22%3E%3CP%3EWe%20are%20excited%20to%20announce%20the%20public%20preview%20of%20Azure%20Data%20Explorer%20Insights!%3CBR%20%2F%3EADX%20Insights%20(Azure%20Monitor%20for%20Azure%20Data%20Explorer)%20provides%20comprehensive%20monitoring%20of%20your%20clusters%20by%20delivering%20a%20unified%20view%20of%20your%20cluster%20performance%2C%20operations%2C%20and%20usage.%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CH4%20id%3D%22toc-hId-149939480%22%20id%3D%22toc-hId-149939455%22%3E%3CSTRONG%3EHow%20ADX%20Insights%20can%20help%20-%20a%20success%20story%20to%20inspire%3C%2FSTRONG%3E%3C%2FH4%3E%0A%3CP%3EOne%20of%20our%20users%20recently%20used%20ADX%20Insights.%20By%20using%20its%20%22Cache%22%20tab%2C%20they%20identified%20tables%20with%20redundant%20cache%20policy%20periods%20and%20unused%20tables%20with%20unnecessary%20cache.%20%3CBR%20%2F%3EAfter%20identifying%20such%20tables%20and%20adjusting%20the%20cache%20policy%20to%20the%20minimum%20required%20period%2C%20the%20hot%20cache%20size%20decreased%20dramatically.%3C%2FP%3E%0A%3CP%3EThen%2C%20the%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fen-us%2Fazure%2Fdata-explorer%2Fmanage-cluster-horizontal-scaling%23optimized-autoscale%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3Eoptimized%20autoscale%3C%2FA%3E%20removed%20the%20unrequired%20VM%20instances%20(scale-in).%20%3CSTRONG%3EThe%20cluster%20scaled-in%20from%20142%20instances%20to%2032%20instances!%3C%2FSTRONG%3E%20%3CBR%20%2F%3EThis%20demonstrates%20how%20cache%20policy%20optimization%20can%20result%20in%20a%20significant%20cost%20reduction%20(fewer%20VM%20instances%20are%20required%20to%20store%20the%20hot%20data).%3CBR%20%2F%3E%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22cache-reduction.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F247422i45248E20A7CAE9F2%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22cache-reduction.png%22%20alt%3D%22Hot%20cache%20reduction%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3EHot%20cache%20reduction%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22cache-reduction-2.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F247425iC000F2E203EF68D5%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22cache-reduction-2.png%22%20alt%3D%22Instance%20count%20decreased%20following%20the%20hot%20cache%20reduction%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3EInstance%20count%20decreased%20following%20the%20hot%20cache%20reduction%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FP%3E%0A%3CH4%20id%3D%22toc-hId--1657514983%22%20id%3D%22toc-hId--1657515008%22%3E%3CSTRONG%3EUser%20experience%20overview%20of%20ADX%20Insights%3C%2FSTRONG%3E%3C%2FH4%3E%0A%3CP%3EBuilt%20on%20the%20%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Fazure-monitor%2Fplatform%2Fworkbooks-overview%22%20target%3D%22_blank%22%20rel%3D%22noopener%20noreferrer%22%3EAzure%20Monitor%20Workbooks%3C%2FA%3E%20platform%2C%20ADX%20insights%20offers%3A%3C%2FP%3E%0A%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3CUL%3E%0A%3CLI%3EAt%20scale%20perspective%3A%20showing%20a%20snapshot%20view%20of%20the%20clusters'%20primary%20metrics%20to%20easily%20track%20performance%2C%20ingestion%2C%20and%20export%20operations.%20Unsustainable%20values%20are%20highlighted%20in%20orange.%26nbsp%3B%3CSPAN%3E%3CSPAN%3E%3CSPAN%3E%3CSPAN%3E%3CSPAN%3E%3CSPAN%3E%3CSPAN%3EYou%20can%20drill%20down%20to%20the%20%E2%80%9Cat-resource%E2%80%9D%20view%20by%20clicking%20on%20the%20cluster%20name.%3CBR%20%2F%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FSPAN%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%221-ingestion-performance.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F247386i92827DC6A06B1BF3%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%221-ingestion-performance.png%22%20alt%3D%22The%20'At%20scale'%20perspective%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3EThe%20'At%20scale'%20perspective%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%3C%2FSPAN%3E%3C%2FSPAN%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%3E%3CSPAN%3EAt-resource%20perspective%3A%20drill-down%20analysis%20of%20a%20particular%20Azure%20Data%20Explorer%20cluster%20based%20on%20metrics%20and%20platform%20usage%20logs.%3CBR%20%2F%3E%3C%2FSPAN%3EWith%20this%20view%2C%20you%20can%3A%3CUL%3E%0A%3CLI%3EIdentify%20query%20lookback%20patterns%20per%20table%20and%20compare%20them%20to%20the%20table's%20cache%20policy.%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22cache-tab-blog-02.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F247235i97EED8CD38CCA06C%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22cache-tab-blog-02.png%22%20alt%3D%22The%20Cache%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3EThe%20Cache%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%3C%2FSPAN%3E%3C%2FSPAN%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%3EIdentify%20tables%20that%20are%20used%20by%20the%20most%20queries.%26nbsp%3B%3C%2FLI%3E%0A%3CLI%3EIdentify%20unused%20tables%20or%20redundant%20cache%20policy%20periods.%3C%2FLI%3E%0A%3CLI%3EFind%20which%20tables%20are%20consuming%20the%20most%20space%20in%20the%20cluster%3C%2FLI%3E%0A%3CLI%3ETrack%20data%20growth%20history%20by%20table%20size%2C%20hot%20data%2C%20and%20the%20number%20of%20rows%20over%20time.%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22tables_blog_2.png%22%20style%3D%22width%3A%20962px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F247239iA9D188AAD07D2AA8%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22tables_blog_2.png%22%20alt%3D%22The%20Tables%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3EThe%20Tables%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%3C%2FSPAN%3E%3C%2FSPAN%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%3EIdentify%20which%20users%20and%20applications%20are%20sending%20the%20most%20queries%20or%20consuming%20the%20most%20CPU%20and%20memory.%3C%2FLI%3E%0A%3CLI%3EFind%20changes%20in%20the%20number%20of%20queries%20by%20user%20and%20track%20the%20number%20of%20unique%20users%20over%20time.%3C%2FLI%3E%0A%3CLI%3ESee%20top%20users%20by%20command%20and%20query%20count%20and%20identify%20top%20users%20by%20the%20number%20of%20failed%20queries.%3C%2FLI%3E%0A%3CLI%3ESee%20the%20query%20count%2C%20CPU%2C%20and%20memory%20consumption%20over%20time.%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22picture_compress.bmp%22%20style%3D%22width%3A%20980px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F247397iA1D546F17AE2F01D%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22picture_compress.bmp%22%20alt%3D%22The%20Usage%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3EThe%20Usage%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%3C%2FSPAN%3E%3C%2FSPAN%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3C%2FLI%3E%0A%3CLI%3EGet%20a%20summary%20of%20active%20Advisor%20recommendations%20and%20resource%20health%20status.%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22overview-blog.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F247268iB464044B33AAD407%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22overview-blog.png%22%20alt%3D%22The%20Overview%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3EThe%20Overview%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%3C%2FSPAN%3E%3C%2FSPAN%3E%3C%2FLI%3E%0A%3CLI%3EExplore%20multiple%20key%20metrics%20on%20one%20page%20and%20discover%20correlations%20between%20them.%3CBR%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-center%22%20image-alt%3D%22metrics_blog.png%22%20style%3D%22width%3A%20999px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F247377iCAF33E11B627BEE5%2Fimage-size%2Flarge%3Fv%3D1.0%26amp%3Bpx%3D999%22%20role%3D%22button%22%20title%3D%22metrics_blog.png%22%20alt%3D%22The%20Key%20Metrics%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%22%20%2F%3E%3CSPAN%20class%3D%22lia-inline-image-caption%22%20onclick%3D%22event.preventDefault()%3B%22%3EThe%20Key%20Metrics%20tab%20(click%20to%20enlarge%2C%20then%20use%20the%20'full%20size'%20button%20at%20the%20top-right%20corner)%3C%2FSPAN%3E%3C%2FSPAN%3E%3CP%3E%26nbsp%3B%3C%2FP%3E%0A%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3C%2FLI%3E%0A%3C%2FUL%3E%0A%3CP%3EAzure%20Monitor%20for%20Azure%20Data%20Explorer%20is%20now%20available%20in%20the%20Azure%20Monitor%20and%20Azure%20Data%20Explorer%20blades%20in%20the%20Azure%20portal.%20W%3CSPAN%3Ee%20look%20forward%20to%20hearing%20your%20feedback%20on%20the%20new%20experience%3C%2FSPAN%3E.%3CBR%20%2F%3E%3CSPAN%3ETo%20learn%20more%2C%20see%20the%26nbsp%3B%3C%2FSPAN%3E%3CA%20href%3D%22https%3A%2F%2Fdocs.microsoft.com%2Fazure%2Fazure-monitor%2Finsights%2Fdata-explorer%3Ftoc%3D%2Fazure%2Fdata-explorer%2Ftoc.json%26amp%3Bbc%3D%2Fazure%2Fdata-explorer%2Fbreadcrumb%2Ftoc.json%22%20target%3D%22_self%22%20rel%3D%22noopener%20noreferrer%22%3EADX%20Insights%20documentation.%3C%2FA%3E%3C%2FP%3E%0A%3CP%3E%3CBR%20%2F%3E%26nbsp%3B%3C%2FP%3E%3C%2FLINGO-BODY%3E%3CLINGO-TEASER%20id%3D%22lingo-teaser-2070257%22%20slang%3D%22en-US%22%3E%3CP%3EWe%20are%20excited%20to%20announce%20public%20preview%20of%20Azure%20Data%20Explorer%20Insights!%3CBR%20%2F%3E%26nbsp%3BADX%20Insights%20(Azure%20Monitor%20for%20Azure%20Data%20Explorer)%20provides%20comprehensive%20monitoring%20of%20your%20clusters%20by%20delivering%20a%20unified%20view%20of%20your%20cluster%20performance%2C%20operations%20and%20usage.%3C%2FP%3E%0A%3CP%3E%3CSPAN%20class%3D%22lia-inline-image-display-wrapper%20lia-image-align-inline%22%20image-alt%3D%22all-blog.jpg%22%20style%3D%22width%3A%20400px%3B%22%3E%3CIMG%20src%3D%22https%3A%2F%2Ftechcommunity.microsoft.com%2Ft5%2Fimage%2Fserverpage%2Fimage-id%2F247387iD4E9D4D1917EF5DE%2Fimage-size%2Fmedium%3Fv%3D1.0%26amp%3Bpx%3D400%22%20role%3D%22button%22%20title%3D%22all-blog.jpg%22%20alt%3D%22all-blog.jpg%22%20%2F%3E%3C%2FSPAN%3E%3C%2FP%3E%3C%2FLINGO-TEASER%3E%3CLINGO-LABS%20id%3D%22lingo-labs-2070257%22%20slang%3D%22en-US%22%3E%3CLINGO-LABEL%3EADX%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EADX%20Insights%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Data%20Explorer%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EAzure%20Monitor%3C%2FLINGO-LABEL%3E%3CLINGO-LABEL%3EInsights%3C%2FLINGO-LABEL%3E%3C%2FLINGO-LABS%3E
Version history
Last update:
‎Jan 19 2021 11:11 AM