As I'm sure you know, fundamentally, the VHD container is a simple abstraction of a real hard drive in a container. Since the underlying operating system will write to that disk in a way to minimize fragmentation, the overall size of a dynamic VHD is going to increase over time with the more writes you have. Additionally, the default block size for VHDX was increased, which makes dynamic VHDXs use even more space than before. You can however, build/convert VHDXs to as low as 1MB block size, which will keep them slightly smaller and closer to the real size, at the expensive of performance.
I've been using Hyper-V for a solid decade now, and I've never had a need to use dynamic VHDs on production servers. But in my homelab I do, and after messing about with sdelete, powershell scripts, etc. for years, I realized I was wasting my time, and hitting the Compact option in the Hyper-V Manager on occasion was good enough for my typical use case.
In this case of 500GB VHDs where there's only a few GB of data, I think you have a few options. Perhaps all you need is a regularly scheduled nightly Powershell task that will shut down the VM (if turned on), run Optimize-VHD and then turn it back on, or something similar. Or if you're doing a lot of writes to the VHD, you might want to consider mapping a drive letter or mount in a directory a SMB path to the host, and doing the writes there instead to avoid inflating the size.
