We are committed to continually advancing the capabilities of Azure Synapse Analytics Spark, and are pleased to announce substantial improvements that could increase Spark performance by as much as 77%.
Performance Metrics
Our internal testing, utilizing the 1TB TPC-H industry standard benchmark, indicates performance gains of up to 77%. It's important to note that individual workloads may vary, but the enhancements are designed to benefit all Azure Synapse Analytics Spark users.
Technological Foundations
This performance uptick is attributable to our transition to the latest Azure v5 Virtual Machines. These VMs bring improved CPU performance, increased SSD throughput, and elevated remote storage IOPS.
Regional Availability
We have implemented these performance improvements in the following regions, bold indicates a new region:
- Australia East
- Australia Southeast
- Canada Central
- Canada East
- Central India
- Germany West Central
- Japan West
- Korea Central
- Poland Central
- South Africa North
- South India
- Sweden Central
- Switzerland North
- Switzerland West
- UAE North
- UK South
- UK West
- West Central US
Additionally, all Microsoft Fabric regions, with the exception of Qatar Central, are already operating with these enhanced performance capabilities.
Future Rollout
The global rollout of these improvements is an ongoing process and expected to take several quarters to complete. We will provide updates as additional regions are upgraded. Customers in updated regions will automatically benefit from the performance enhancements at no additional cost.
Next Steps for Users
No action is required on your part to benefit from these improvements. Once your region receives the upgrade, you may notice reduced job completion times. If cost-efficiency is a priority, you may opt to decrease node size or the number of nodes while maintaining improved performance levels.
Learn more about Optimizing Spark performance, Apache Spark pool configurations, Spark compute for Data Engineering and Data Science - Microsoft Fabric
Microsoft