Migrating from AWS to Azure, particularly when transitioning databases like PostgreSQL, demands a meticulous strategy to ensure a seamless and efficient process. Crafting a well-thought migration strategy is paramount, as it not only mitigates potential risks but also optimizes the utilization of Azure's features. This strategic shift encompasses careful consideration of cost, downtime, networking, security, data integrity, performance optimization, and resource utilization.
The equivalent of Amazon RDS for PostgreSQL in Microsoft Azure is Azure Database for PostgreSQL - Flexible Server. Azure Database for PostgreSQL is a fully managed service that offers built-in high availability, data protection, automated maintenance, predictable performance, elastic scaling, enterprise grade security, monitoring and automation, and industry-leading support.
In this blog, we'll delve into the critical aspects of why a thoughtful migration plan is indispensable, providing a comprehensive understanding of the technical nuances involved in the Amazon RDS for PostgreSQL to Azure Database for PostgreSQL - Flexible Server migration journey.
Migration Considerations
The key considerations to take note of before starting the migration activities are as follows:
Cost and Sizing
Cost
- Consider the cost of migrating your database and applications to Azure and make sure you have a plan in place to optimize your costs over time. How to optimize costs in Azure Database for Postgres Flexible Server | Microsoft Learn.
- Best practice(s): Estimate the Azure Database for PostgreSQL flexible server cost with the help of pricing calculator and review with the stakeholders.
Sizing
- Choosing the right tier: Consider the following factors when choosing an Azure Database for PostgreSQL tier: Instance Type, vCores, Memory, Storage, Backup Storage. Read more about various tier and offerings here: Compute and Storage options in Azure Database for PostgreSQL – Flexible Server. Please note that Azure Database for PostgreSQL - Single Server is scheduled for retirement by March 28, 2025. We strongly recommend you upgrade to Azure Database for PostgreSQL - Flexible Server.
- Best practice(s): Choose a higher compute and memory SKU to complete the migration quickly. You may scale back easily to the desired configuration post migration. Additionally, leverage IOPS scaling to increase the number of IOPS for the duration of migration without increasing the storage size.
Migration Downtime
- Regardless of the migration approach employed for workloads, there could be a potential risk of downtime. This is often a result of the necessity to quiesce or temporarily pause an application or dataset to guarantee the continuity of its copy, validation, resetting the server parameters and other critical processes.
- Best practice(s): It is important to connect with business and application teams to inform, plan, and analyze the impact of downtime during the cutover.
Version compatibility and dependency
- Ensure that your database is compatible with Azure Database for PostgreSQL, or you have the necessary tools and processes to convert your database to be able to run it on Azure Database for PostgreSQL. Please note that we don't support PostgreSQL version 10 and older for Azure Database for PostgreSQL - Flexible Server. For more details refer to Supported versions - Azure Database for PostgreSQL.
- Best practice(s): Application compatibility issues can arise when migrating to a different version of PostgreSQL. These issues are primarily due to changes in the database system's behavior, features, or syntax between versions, deprecated features, changes in default setting, extension compatibility, character set, etc. Comprehensive testing is crucial to identify and address compatibility issues before migrating to a higher PostgreSQL version.
Resiliency
- Azure Database for PostgreSQL provides features that protect data and mitigates downtime for your mission-critical databases during planned and unplanned downtime events. Built on top of the Azure infrastructure that offers robust resiliency and availability, the flexible server has business continuity features that provide another fault-protection, address recovery time requirements, and reduce data loss exposure.
- As you architect your applications, you should consider the downtime tolerance - the recovery time objective (RTO), and data loss exposure - the recovery point objective (RPO). For example, your business-critical database requires stricter uptime than a test database. Overview of business continuity with Azure Database for PostgreSQL - Flexible Server.
- Best practice(s): You should consider the trade-offs between availability, performance, and cost when choosing HA architecture for your database. Zone-redundant HA provides the highest level of availability and isolation across multiple availability zones, but it may incur higher network latency. Same-zone HA provides high availability within a single availability zone with lower network latency, but it does not protect against zone failures. You should also implement retry logic in your application to handle the transient errors or timeouts during failover process and monitor the failover status using Azure Monitor.
Networking and Security
- Azure Database for PostgreSQL - Flexible Server instance support choosing one of the following networking options: Private access (VNet integration) or Public access (allowed IP addresses). For private and secure network communication, go for Private access. For more information, refer to Networking overview - Azure Database for PostgreSQL - Flexible Server.
- Multiple layers of security are available in Azure Database for PostgreSQL - Flexible Server. For more details refer to Security in Azure Database for PostgreSQL - Flexible Server.
- Establish a connection between Azure and AWS by connecting AWS Virtual Private Gateway with the Azure VPN Gateway. For more information refer to How to create a VPN between Azure and AWS using only managed solutions. Please refer to the following reference architecture:
- Alternatively, review if there is AWS direct connect and Azure express route already setup between AWS and Azure. However, consider the cost factor.
Performance
- Evaluate the performance of your database and applications on Amazon RDS for PostgreSQL and Azure Database for PostgreSQL, and make sure you have the plan to optimize performance as needed. Query Performance Insight provides intelligent query analysis for Azure Postgres Flexible Server databases. It helps identify the top resource consuming and long-running queries in your workload.
- Query Store feature in Azure Database for PostgreSQL provides a way to track query performance over time. For more details refer to: Query Store - Azure Database for PostgreSQL - Flexible Server.
- Azure Database for PostgreSQL - Flexible Server has an intelligent tuning feature that's designed to enhance performance automatically and help prevent problems. For more details refer to: Intelligent tuning - Azure Database for PostgreSQL - Flexible Server.
Other considerations
- Support: Make sure you have a plan in place to ensure that you receive the necessary support and guidance as you migrate. Contact Microsoft support team.
- Operational Excellence: Consider CI/CD, logging, diagnostics, and monitoring requirements.
- Upskilling needs: There are ample resources and programmes from Microsoft to get your team up to speed with Azure Database for PostgreSQL. Reach out to your account contact for training needs. You may also refer to self-study training modules: Browse all learning paths and modules - Training | Microsoft Learn.
Migration Options and Approaches
Although there are different options for migrating from Amazon RDS for PostgreSQL to Azure Database for PostgreSQL, broadly migration approached can be categorized as either offline or online, as shown below:
Offline migration
With an offline migration, the source server is taken offline, and a dump and restore of the databases are performed on the target server. Offline migrations are always the preferred choice if your application can afford some downtime, as they are simple and easy to execute.
|
Offline Approaches |
Details |
|
Dump and Restore |
You can use pg_dump to extract a PostgreSQL database into a dump file. Then use pg_restore or psql (depending on the dump format) to restore the PostgreSQL database from an archive file created by pg_dump. For more details refer to: Dump and restore - Azure Database for PostgreSQL. |
|
pgcopydb |
pgcopydb tool automates running pg_dump | pg_restore between two running PostgreSQL servers. The pgcopydb tool provides performance gains by allowing below two aspects: · Bypass intermediate files for the TABLE DATA · For each table, build all indexes concurrently Please refer to the GitHub link for more information: GitHub - dimitri/pgcopydb: Copy a Postgres database to a target Postgres server. |
Online migration
With online migration, the application can continue to access the database during the migration process, except for a short cutover window when the final switch is made from the source to the target server. Online migration is the best choice if your application can only afford minimal downtime. Migrations of most OLTP systems, such as payment processing and e-commerce, fall into this category. However, online migration is more complex and requires more planning and testing than offline migration. You need to ensure that the source and target servers are compatible, replication lag is minimal, and the cutover window is short and well-timed.
|
Online Approaches |
Details |
|
Azure DMS |
Fully managed Azure service that helps you migrate your databases to Azure with minimal downtime. There are no costs incurred while running the migration using DMS. For more details refer to Tutorial: Migrate RDS PostgreSQL online to Azure Database for PostgreSQL. |
|
Open Source (Logical Replication, Debezium, etc.) |
Logical replication is a method of replicating data objects and their changes, based upon their replication identity. Please refer to: Using logical replication to replicate managed Amazon RDS for PostgreSQL and Amazon Aurora to self-managed PostgreSQL | AWS Database Blog
Debezium is an open source distributed platform for change data capture. For more details refer to: Debezium |
|
Commercial (Striim, HVR, etc.) |
The Striim platform is an enterprise-grade data integration and streaming solution for moving data in real-time to the cloud. Microsoft and Striim providing exclusive Striim licenses at zero-cost for online database migrations including CDC to Azure Cosmos DB, Azure SQL and ORCAS. These Striim licenses provide: · Online migrations for homogeneous and heterogeneous migrations · Change data capture (CDC) for continuous, real-time data movement · In-flight data processing · Near-zero downtime migrations
HVR is a powerful software product that enables real-time homogeneous and heterogeneous data replication. HVR uses various CDC (Change Data Capture) methods to replicate changes between databases, directories (file locations), as well as between databases and directories. For more details read: Introduction - HVR 5 Documentation (fivetran.com). |
Depending on the downtime window and business impact, you may decide to use offline (pg_dump and pg_restore/pgcopydb) or online mode of migration using DMS. The final choice of migration approach depends on factors such as, permissible downtime, cost, ease of use and criticality of applications.
Migration Phases
Below is the guidance for each of the phases of migration, as shown in the image above:
Discovery and Assessment
- Conduct customer discovery to understand the business objectives and motivations.
- Assess the existing Amazon RDS for PostgreSQL architecture and evaluate the target Azure Database for PostgreSQL Architecture.
- Confirm the downtime expectations with the stakeholders.
- Estimate cost savings: Part of the business's plan to migrate to Azure could be to reduce costs. Use the Azure pricing calculator to estimate monthly running costs. You can save on your Azure Database for PostgreSQL costs when you pre-purchase compute resources for one or three years. Refer Reserved compute pricing - Azure Database for PostgreSQL for more details.
- Gather AWS inventory information to understand the AWS SKU’s and pricing tiers used.
- Finalize the list of the databases that needs to be migrated. Clarify and confirm from the customers if there are any transformations and modernization requirements.
- Finalize target SKU based on the requirements of Azure Database for PostgreSQL. Validate service, AZ, and DR availability in the region.
- If there are any special requirements that cannot be met due to restrictions on Azure Database for PostgreSQL, also consider PostgreSQL on Azure Virtual Machine as a migration destination.
- Identify upskilling needs.
Plan and Design
- Ensure to evaluate the key considerations and the assessment in the discovery step impacting the migration plan.
- Define and establish target state architecture. Finalize the migration method/approach.
- Validate and finalize the network connectivity strategy between single to flexible server.
- Review migration readiness.
- Plan and prepare detailed migration plan, involving key stakeholders, based on the responses gathered in Amazon RDS for PostgreSQL to Azure Database for PostgreSQL technical evaluation.
- Ensure to review the key considerations impacting the migration plan.
Migrate
- Perform smoke test/pilot migrations before actual migrating for production:
- If you are migrating to a higher PostgreSQL version, test for application compatibility.
- All stakeholders who need to be involved should be available and ready. The production migration requires close monitoring.
- For an online migration, the replication must be completed before you perform the cutover, to prevent data loss.
- The key steps for migration are:
- Prepare the source and target servers. This includes creating the flexible server with the desired configuration, setting the server parameters, configuring the networking and firewall rules, and creating the databases and users on the target server.
- Start the migration and monitor the progress and status.
- Perform the cutover when the replication lag is minimal, and the application traffic is low.
- Refer to the migration documentation for the selected method and tool.
- Post migration activities:
- Perform validation and data integrity checks.
- Redirect all dependent applications to access the new primary database and make the source server read-only. Then, open the applications for production usage.
- After the application starts running on the target flexible server, monitor the database performance closely to see if further performance tuning is required.
Monitor and Govern
- Azure Database for PostgreSQL uses Azure Monitor to provide several options for monitoring your cache instances. Use these tools to monitor the health of your Azure Database for PostgreSQL instances. Use insights to understand performance and utilization. For more details refer to: Monitoring and metrics - Azure Database for PostgreSQL - Flexible Server.
Optimize and Modernize
- Identify opportunities to further optimize cost, performance, security, scalability, and operational excellence. Conduct well-architected (WAF) review to ensure that the workloads are optimized as per the best practices and standards.
Known Issues and Limitations
It is also important to consider the known issues and limitations of the migration tools you choose to use. Refer to the links below for more information:
- Limits - Azure Database for PostgreSQL - Flexible Server
- Known issues: Online migrations from PostgreSQL to Azure Database for PostgreSQL
- Limitations with online migration from AWS RDS PostgreSQL
Conclusion
By meticulously considering various factors, employing the right tools, adhering to best practices, and addressing limitations, you can seamlessly transition your database from Amazon RDS for PostgreSQL to Azure Database for PostgreSQL. Success lies in thorough preparation, comprehensive testing, ongoing performance monitoring, and optimization.
If you have any feedback or questions about the information provided above, please leave a comment below or email us at AskAzureDBforPostgreSQL@service.microsoft.com. Thank you!
Microsoft
