In today's rapidly evolving digital landscape, businesses face continuously changing data storage requirements. It has become crucial to have flexible and scalable solutions in place to adapt to these evolving needs. Many customers initially start with smaller databases, but as their data grows, it eventually consumes all available storage. This often leads to the database becoming read-only, as there is no space left for new transactions. To address this challenge, two important capabilities are needed, Online disk Scaling and Storage auto-grow. Online disk scaling ensures that server restart is not needed when expanding a disk, enabling seamless scalability without interruptions. Storage auto-grow automatically scales the disk when storage capacity is running low, ensuring that your server always has sufficient storage without any downtime or disruptions.
Note: The feature was originally previewed in June-2023, and is generally available starting Sept-2023.
We are excited to announce that storage auto-grow and online disk scaling for Azure Database for PostgreSQL - Flexible Server - are now generally available in all Azure regions. In this blog, we will delve into the benefits and functionalities of storage auto-grow and online disk scaling. We will explore how these features empower you to achieve seamless scalability for your Azure PostgreSQL databases, allowing you to effectively manage your growing data storage requirements.
Online Disk Scaling
One of the primary obstacles that Azure Database for PostgreSQL Flexible Server customers faced in the past was the need to restart servers during disk scaling operations. However, starting today, most disk scaling operations no longer need service restarts. This enhancement enables customers to expand disks online without any impact on their business operations. This feature ensures that scaling operations for your PostgreSQL Flexible Server, in most cases, do not cause any downtime, with the exception mentioned below.
Online Disk Scaling is a valuable feature that eliminates the need for server restarts and is particularly beneficial for storage auto-grow. It relies on this feature to perform online scaling operations, ensuring uninterrupted availability of the PostgreSQL Flexible Server. This enhancement brings substantial advantages to users of Azure Database for PostgreSQL Flexible Server. By removing the requirement for server restarts during disk scaling operations, customers can seamlessly expand their disks online, ensuring smooth scaling without any disruptions. This improvement further strengthens the value of storage auto-grow, enabling uninterrupted availability and efficient management of storage capacity.
It is important to note that you do not need to take any action to use this feature. Azure Database for PostgreSQL Flexible service has enabled this capability for all regions, allowing customers to benefit from online disk scaling without any server restarts in most cases.
The below image depicts the disk scaling experience before and after the recent improvement, highlighting the elimination of the need for a server restart during disk scaling operations. In the earlier experience, when you scaled the disk from 128 GiB to 1 TiB, you would receive a server restart message. However, with the new and enhanced experience, you no longer receive that message because the server restart now happens seamlessly online, thanks to this feature.
Exception: Disk scaling operations are always online except in specific scenarios involving the 4096 GiB boundary. These scenarios include reaching, starting at, or crossing the 4096 GiB limit, such as when scaling from 2048 GiB to 8192 GiB etc. This limitation is due to an underlying Azure Managed disk V1 which needs a manual disk scaling operation.
Storage Auto-grow
In Azure Database for PostgreSQL Flexible Server, enabling the storage auto-grow feature offers a proactive solution for managing storage capacity. With storage auto-grow enabled, your storage expands automatically without affecting your workload or incurring downtime. By configuring this feature, you mitigate the risk of your server becoming read-only due to insufficient storage. Storage auto-grow event triggers depending upon the provisioned storage for your flexible server.
For servers with more than 1 TiB of provisioned storage, the storage autogrow mechanism activates when the available space falls to less than 10% of the total capacity or 64 GiB of free space, whichever of the two values is smaller. Conversely, for servers with storage under 1 TB, this threshold is adjusted to 20% of the available free space or 64 GiB, depending on which of these values is smaller.
As an illustration, take a server with a storage capacity of 2 TiB ( greater than 1 TIB). In this case, the autogrow limit is set at 64 GiB. This choice is made because 64 GiB is the smaller value when compared to 10% of 2 TiB, which is roughly 204.8 GiB. In contrast, for a server with a storage size of 128 GiB (less than 1 TiB), the autogrow feature activates when there's only 25.8 GiB of space left. This activation is based on the 20% threshold of the total allocated storage (128 GiB), which is smaller than 64 GiB. This automated scaling mechanism ensures that your storage capacity aligns with the growth of your database, eliminating the need for manual intervention.
Benefits:
Managing unpredictable workloads, particularly in terms of storage capacity planning, can be challenging. Enabling storage auto-grow proves exceptionally valuable in such scenarios. The feature intelligently detects low free space conditions and scales up the storage accordingly. With auto-grow in place, you can focus on your business operations, knowing that your storage will seamlessly adapt to accommodate increasing data volumes. Whether you are dealing with predictable or unpredictable workloads, storage auto-grow offers a proactive approach to storage management.
Storage auto-grow can be enabled during server provisioning or later by accessing the Compute + Storage blade from your Azure Database for PostgreSQL Flexible Server.
Limitations:
- Host Caching (ReadOnly and Read/Write) is supported on disk sizes less than 4 TiB. This means any disk that is provisioned up to 4095 GiB can take advantage of Host Caching. Host caching is not supported for disk sizes more than or equal to 4096 GiB. For example, a P50 premium disk provisioned at 4095 GiB can take advantage of Host caching and a P50 disk provisioned at 4096 GiB cannot take advantage of Host Caching. Customers moving from lower disk size to 4096 Gib or higher will lose disk caching ability.
- Storage auto-grow currently does not work for Read replica-enabled servers and we are going to support this very soon.
- Storage auto-grow isn't triggered when you have high WAL usage.
Storage auto-grow only supports scaling up, not down. Once your storage has expanded, it cannot be reduced in size. Therefore, it is crucial to allocate storage capacity based on your projected needs and growth expectations. If you have scenarios that cause uncontrolled WAL growth, then the only option is to perform a PITR as PostgreSQL does not support shrink operation on the database.
Summary
In this post, I explained storage auto-grow and online disk scaling for Azure Database for PostgreSQL flexible server and its benefits. To learn more about this feature please go through storage auto-grow documentation.
We are always eager to get your feedback, please reach out via email to Ask Azure DB for PostgreSQL.
Updated Oct 13, 2023
Version 3.0kabharati
Microsoft
Joined November 03, 2021
Azure Database for PostgreSQL Blog
Follow this blog board to get notified when there's new activity