sxc7886 Just use the Azure cost calculator to price up appropriately specced guest VM's (plus appropriate storage) based on your requirements. Add in an amount of "out of Azure" network bandwidth if you want, though I reckon it wont be more than about ~5% percent on top of your VM costs under most normal circumstances. You can also power the machines down/up on a timed schedule when not in use (or user reserved instances if they are utilised 24/7) to reduce cost.
The management tooling is free. No need to pay for gateways etc. You will need access to a domain controller somewhere though (might be worth spinning a B2 one up on Azure IAAS in the same region as your pool for about £30 a month?)
If you are using W10 as the OS for your VM's you aren't paying any extra over the cost of the base VM hardware (as these WVD W10 licenses are covered by your M365 e3/e5 entitlement). If you are using Windows Server for your pools you'll still pay a little bit for that licence.
Remember you'll need RDS CAL's per user if you are using Windows Server. M365 e3/e5 don't include this. So consider using W10 multi-session if you are doing session based desktops,