PostgreSQL: Migrating Large Objects (LOBs) with Parallelism and PIPES

Hi Francisco
Thank-you so much for posting this script
I'd like to point out that it doesn't copy postgres largeobjects that have a been created logically (have an entry in pg_largeobject_metadata) but are empty and have no entries in pg_largeobject.
We modified the script a bit to get those too

echo > nohup.out
echo 'ParallelDegree: '$1 'DateTime: '`date +"%Y%m%d%H%M%S"`
# Check if no large objects to migrate
count=$(echo "select count(1) from pg_largeobject_metadata;"|./pgsource.sh)
count=$(echo "$count" | xargs)
if [ "$count" -eq 0 ]; then
  echo "There are no large objects to migrate. Stopping the script."
  exit 0
fi
par=$1
for i in $(seq 1 $1); do
nohup /bin/bash <<EOF &
  echo "\copy (select data from (select 0 as rowsort, 'begin;' as data union select 1, concat('SELECT pg_catalog.lo_create(',   lo.loid,    ');SELECT pg_catalog.lo_open(',      lo.loid, ', 131072);SELECT pg_catalog.lowrite(0,''',  string_agg(lo.data, '' ORDER BY pageno), ''');SELECT pg_catalog.lo_close(0);') from  pg_largeobject lo  where mod(lo.loid::BIGINT,$par)=$i-1  group by lo.loid union select 1, 'SELECT pg_catalog.lo_create(' || lom.oid || ');' from pg_largeobject_metadata lom where not exists (select 1 from pg_largeobject lo where lo.loid = lom.oid) and mod(lom.oid::BIGINT,$par)=$i-1 union select 2, 'commit;') AS subq order by rowsort) to STDOUT;"|./pgsource.sh|sed 's/\\\\\\\/\\\\/g'|./pgdestination.sh; echo "Execution thread $i finished. DateTime: ";date +"%Y%m%d%H%M%S";
EOF
done
tail -f nohup.out|grep Execution

Disclaimer

The script is provided as it is. Please review carefully when running/testing, the script is just a starting point to show how to migrate large objects in a parallelized way without intermediate storage, but it can be also implemented/improved using other methods as mentioned.