RDS shadow over site-to-site VPN problem - not working

I have a problem with RDS shadow on Windows Server 2022 Standard. If the client connects to the server via RDP from the local network where the server is, RDS shadow works without a problem. If the client is outside this local network on a site-to-site VPN connection, RDS shadow does not work properly, the screen is black and only the moving mouse cursor is visible. If the client connects to the server via the VPN client (receives an address from the server's network), RDS shadow also works properly. I have already tried various settings. I suspected blocking some ports, but disabling Windows defender did not change anything. How to solve this problem?