Protocol 50 problem for VPN

So this is with windows firewall block outbound and allow by rules in a nutshell:

L2TP/IPsec you have to allow UDP port 500 and not really 4500 because no NAT so you would think you allow Protocol 50 but no you allow UDP port 1701 and all works fine.

Now IKEv2 allow UDP port 500 but the firewall blocks Protocol 50 even if allowed in firewall and the only way to allow it is by IP to the VPN on any protocol.