A new generation of cache recommendations for Azure Data Explorer is now available in the Azure portal!
This update introduces significant improvements, including enhanced logic, additional statistics for end users, an improved user interface, and a streamlined process for reviewing and applying recommendations. In this blog post, we will explore the new features and benefits offered by this latest update.
New user interface, more data
One of the notable improvements in the user interface is the aggregation of recommendations per database, replacing the previous table-level granularity in the recommendations grid.
Additionally, a new side pane has been introduced for each database-level recommendation, providing users with a comprehensive view of the following details:
- Expandable list of tables that have recommendations to optimize the cache. The new user interface allows for easy identification of these tables, and it is expandable to provide more detailed information.
- Users can easily apply recommendations to either all suggested tables or specific tables, providing a convenient way to implement changes. Additionally, users have the flexibility to modify the suggested cache policy on a per-table basis.
- The recommended cache policy: The specific cache policy suggested for each table is displayed, enabling users to make informed decisions based on their use-case.
- Potential cache hit percentage: Based on historical usage statistics, users can determine the percentage of queries that are likely to find data in the hot cache. This valuable insight helps optimize cache efficiency.
- Potential cache data reduction (applicable to cost and service excellence recommendations, as explained below): Users can assess the amount of data that can be removed from the hot cache, leading to improved resource utilization. Conversely, for performance recommendations, we provide information on how much data will be stored in the cache
- Current cache usage statistics: Users can view the percentage of queries that have found the requested data in the hot cache, along with the total number of analyzed queries. In cases of low usage (less than 50 queries), a note is provided to highlight the limited sample size.
- Predicted usage statistics: Users are presented with information on the projected number of queries that will hit or miss the cache, aiding in capacity planning and performance optimization.
Types of recommendations:
Cost optimization:
The "reduce Azure Data Explorer table cache period for cluster cost optimization" recommendation is designed to help clusters reduce the cache policy for their tables while maintaining high performance. The goal is to ensure that over 95% of queries read data from the hot cache. This recommendation analyzes the query look-back period over the last 30 days and provides insights into potential cache savings for the five most relevant tables per database. By enabling autoscale, clusters can scale in and reduce costs. This recommendation is available for clusters that are "bounded by data," meaning the number of instances in the cluster is determined by the data held in the hot cache.
Service excellence:
This recommendation focuses on updating the cache policy based on actual usage during the last month to reduce the hot cache for a table. Unlike the previous cost recommendation, this particular recommendation is applicable to clusters where the number of instances is determined by CPU and ingestion load rather than the amount of data stored in the hot cache. In such cases, changing the cache policy alone is insufficient to reduce the number of instances, further optimizations such as changing the SKU, reducing CPU load, and enabling autoscale are recommended to efficiently scale in.
Performance:
To enhance performance based on actual usage, this recommendation suggests updating the cache policy to increase the hot cache for a table. By increasing the cache policy, users can improve query response times and optimize overall system performance.
Conclusion:
The latest enhancements to the cache recommendations for Azure Data Explorer bring significant improvements to Advisor. With an updated user interface, aggregated recommendations per database, and detailed insights into cache usage and future predictions, users can make informed decisions to optimize cost and performance. These new features provide a seamless experience for reviewing and applying recommendations, helping users achieve better performance and cost efficiency in their Azure Data Explorer clusters.
To learn more, visit: Use Azure Advisor recommendations to optimize your Azure Data Explorer cluster
Microsoft
