I am "lucky" in that I have an on-premise Exchange server with retention policies, so I was just able to import all the archive PSTs to the user's main mailbox even if it grows to 400GB and then just "wait" until space becomes available in the archive, and then the retention policies with Managed Folder Agent archives it for me.
If you just have a bunch of PSTs and only the archive, then yeah, you would have to use some kind of tool that allows you to import the PSTs some chunks at a time. Assuming you have a retention policy in Exchange Online that will archive old items you could import them straight to the main mailbox using Outlook or another tool, and then just wait until the retention policy kicks in. Or directly into the archive with AzCopy. Whatever works basically! It will involve some babysitting until it's all done and it is really tough if you have a lot of users with a ton of archive data. If it's really that important maybe there is a 3rd party tool that can help.