Excel Pivot issue with switching data source

Hello Excel community,

I have a bunch of Pivot tables getting data from a Power BI Dataset in an Excel sheet. I am trying to switch the datasource for these pivot tables by using the switch data source button in PivotTable Analyze and selecting a key file I retrieved from the correct data source.

The issue arises when I switch out some of the larger tables. I can easily construct the same setup in Power BI or in a fresh Excel sheet but when I try to switch the source, I hit the memory capacity of ~2GB. So I've verified that the query in itself is not too heavy.

What can cause this issue? There's a lot of tables with very specific filter selections so recreating the full file would be a nightmare.