Update: On 19 May 2025 we announced the General Availability of enhanced log throughput for Azure SQL Database Hyperscale tier to upto 150Mib/s. For more details, please read the GA announcement.
We have recently announced the limited public preview of enhanced log throughput in Azure SQL Database Hyperscale tier to up to 150 MiB/s and are currently accepting requests from customers who wish to participate.
Log throughput refers to the rate at which transaction log records are generated and written to log service. The rate is directly correlated with the number of transactions processed by the database, as each transaction generates log records that are subsequently written to the log service. With the premium hardware series (premium-series and memory-optimized premium-series), users can experience an increased log rate of up to 150 MiB/s, a significant boost from the previous 100 MiB/s.
What are the uses cases of high log throughput?
- Large ETL - Bulk inserts: For recurring ETL scenarios, daily or more frequent intervals. This is not only about the volume of data being loaded but also the speed at which it is processed. Business demands necessitate more frequent data loads in the shortest time possible.
- Data loads have higher ranges, whether from data lake, Synapse or other sources.
- Data loads have SLAs that cannot be met with the current 100MB/sec log rate.
- ELT & Data Manipulation: For workloads requiring extensive data manipulation that exhaust the log generation rate, consider the following scenarios:
- Load data into staging tables first, then use merge or other reconciliation/upsert procedures to move data into final tables.
- Large Delete operations particularly challenging for non-partitioned tables.
- Index rebuilds and creations
- Frequent data modifications and faster transaction commits.
How can I monitor the increased log throughput?
Once a higher log rate is enabled for a database, the log rate governance limits will be increased. The log rate will then be calculated based on the new rate of 150 MiB/s, and this change will automatically be reflected in the Database Watcher, Azure Portal and DMVs SYS.DM_USER_DB_RESOURCE_GOVERNANCE, SYS.DM_RESOURCE_GOVERNOR_WORKLOAD_GROUPS_HISTORY_EX, SYS.DM_DB_RESOURCE_STATS and DMF SYS.DM_HS_DATABASE_LOG_RATE()
- Database watcher showing log rate of up to 150MiB/s.
- DMV screenshot showing log rate of up to 150MiB/s.
What are the potential challenges of increasing the log generation rate limit to 150 MiB/s?
By increasing the log generation rate limits, the log governance limit for a Hyperscale premium series database will be set to 150 MiB/s.
While we do not anticipate major impacts, there are components that consume log records, such as readable secondary replicas (HA and named replicas) and geo-secondary replicas. All the log records generated on primary replica are consumed by these replicas. Consequently, their catch-up rate may be slower when these replicas are used for read-scale purposes or synchronization between geo-regions, or with a lower named replica compute size than primary replica. This may delay the synchronization process, resulting in a wait “RBIO_RG_REPLICA“on the primary replica, indicating that the secondary replica is lagging in applying log records. To prevent the secondary replica from falling behind, primary log generation rate may reduce to below 150MiB/s during high log generation rate periods (you can monitor the details of current or reduced log rate using the DMF SYS.DM_HS_DATABASE_LOG_RATE()). Additionally, databases with Change Data Capture (CDC) enabled may face similar challenges.
Limitation:
Can be enabled only on Hyperscale premium-series hardware (premium-series and memory-optimized premium-series). Refer Regional availability to check availability for premium-series hardware for single databases and elastic pools.
How do I sign-up for the 150MiB/s log rate?
We are currently accepting requests from customers interested in participating in the limited public preview to test the new log throughput for Hyperscale databases. This feature can be enabled on existing Hyperscale database in the premium series, or by scaling existing databases from the standard to the premium series. You can also create a new Hyperscale database in the premium series to test your workloads.
Please fill out this form to enroll for this preview: 150MiB/s sign up
Microsoft