Remote desktop session frequently crashes when "Optimize for Retina displays" is selected

I use the MacOS Remote Desktop client to connect to my Windows PC on Corpnet. I have connected two external monitors to my MacBook, both with 4K resolution. When I'm connected to the remote desktop, frequently the remote session crashes with an error saying, "The display components failed to start", and then I have to reconnect. This repeats several times over the course of the day and it's very annoying.

The issue goes away if I uncheck the "Optimize for Retina displays" option when I'm connecting to the remote desktop, so I think the root cause has something to do with this optimization. The display is super blurry when it's unchecked though, so it would be really great to fix this issue.