Migrate your database from PostgreSQL Single Server to PostgreSQL Flexible Server using Azure DMS

[Ahmed_S_Mahmoud]

Overview:

Azure Database for PostgreSQL recently announced Flexible Server,  Flexible Server is a new deployment option for Azure Database for PostgreSQL that gives you the control you need with multiple configuration parameters for fine-grained database tuning along with a simpler developer experience to accelerate end-to-end deployment. With Flexible Server, you will also have a new way to optimize cost with stop/start capabilities. 

In this article, we will provide a guideline on how to leverage Azure DMS for seamless and simplified migrations of your database from Azure PostgreSQL Single Server to Azure PostgreSQL Flexible Server with minimal downtime.

Important - Azure Database for PostgreSQL - Flexible Server is now on general availability.

Note: Azure recently launched a (public preview) of Flexible Server migration feature providing zero cost, automated and friction-free migrations.

Migration to Azure made easy: Upgrade Your PostgreSQL Experience (microsoft.com)

For older method, See this documentation GitHub - shriram-muthukrishnan/single-to-flex on using the feature and the associated pre-requisites and limitations.

Migration tool - Azure Database for PostgreSQL Single Server to Flexible Server - Concepts - Azure D...

1 – Prepare the Source – Azure PostgreSQL - Single Server

You will need to set some Postgresql Parameters on the source as follows:

-- parameter wal_level should be logical, on Azure PgSQL this can be done from Azure Portal as shown blow:

-- max_replication_slots = [number of slots], recommend setting to 5 slots
-- max_wal_senders =[number of concurrent tasks] - The max_wal_senders parameter sets the number of concurrent tasks that can run, recommend setting to 10 tasks

-- the user must have 'userepl' privilege, if replication not granted, you can grant it as follows:

select * from pg_roles;

ALTER USER <user-name> WITH REPLICATION;

2- Prepare the target for the migration - Flexible Server

You will need to migrate database schema from source to target using commands:

-- on the source, create a schema dump file for a database

pg_dump -o -h hostname -U db_username -d db_name -s > your_schema.sql

-- Import the schema into the target database

psql -h hostname -U db_username -d db_name < your_schema.sql

-- Remove foreign keys in schema at target Azure Database for PostgreSQL

SELECT Q.table_name
    ,CONCAT('ALTER TABLE ', table_schema, '.', table_name, STRING_AGG(DISTINCT CONCAT(' DROP CONSTRAINT ', foreignkey), ','), ';') as DropQuery
        ,CONCAT('ALTER TABLE ', table_schema, '.', table_name, STRING_AGG(DISTINCT CONCAT(' ADD CONSTRAINT ', foreignkey, ' FOREIGN KEY (', column_name, ')', ' REFERENCES ', foreign_table_schema, '.', foreign_table_name, '(', foreign_column_name, ')' ), ','), ';') as AddQuery
FROM
    (SELECT
    S.table_schema,
    S.foreignkey,
    S.table_name,
    STRING_AGG(DISTINCT S.column_name, ',') AS column_name,
    S.foreign_table_schema,
    S.foreign_table_name,
    STRING_AGG(DISTINCT S.foreign_column_name, ',') AS foreign_column_name
FROM
    (SELECT DISTINCT
    tc.table_schema,
    tc.constraint_name AS foreignkey,
    tc.table_name,
    kcu.column_name,
    ccu.table_schema AS foreign_table_schema,
    ccu.table_name AS foreign_table_name,
    ccu.column_name AS foreign_column_name
    FROM information_schema.table_constraints AS tc
    JOIN information_schema.key_column_usage AS kcu ON tc.constraint_name = kcu.constraint_name AND tc.table_schema = kcu.table_schema
    JOIN information_schema.constraint_column_usage AS ccu ON ccu.constraint_name = tc.constraint_name AND ccu.table_schema = tc.table_schema
WHERE constraint_type = 'FOREIGN KEY'
    ) S
    GROUP BY S.table_schema, S.foreignkey, S.table_name, S.foreign_table_schema, S.foreign_table_name
    ) Q
    GROUP BY Q.table_schema, Q.table_name;

-- Disable triggers at target Azure Database for PostgreSQL

The migration service internally handles the enable/disable of foreign keys and triggers to ensure a reliable and robust data migration. As a result, you do not have to worry about making any modifications to the target database schema.

So you don't need to manually disable it.

SELECT DISTINCT CONCAT('ALTER TABLE ', event_object_schema, '.', event_object_table, ' DISABLE TRIGGER ', trigger_name, ';')
FROM information_schema.triggers

-- Provision Database Migration Service and create a migration task

More details in Tutorial: Migrate PostgreSQL to Azure Database for PostgreSQL online via the Azure CLI - Azure Datab...

3 – Create Migration Project

You will need to use PostgreSQL as source server type and provide the single server endpoint and credentials.

Note:- You will need to make sure the connectivity between the source single server and DMS service on one side and the DMS service and the target flexible server on the other side.

Details steps: Tutorial: Migrate PostgreSQL to Azure DB for PostgreSQL online via the Azure portal - Azure Database...

4- Monitoring the migration project progress and you can perform the migration once it’s ready for cutover

Note:- After migration is complete, you might need to re-enable the triggers and create foreign keys. 

I hope you find this article helpful. If you have any feedback please do not hesitate to provide it in the comment section below.

Ahmed S. Mazrouh