Insider Preview: Single sign-on and passwordless authentication for Azure Virtual Desktop

I am excited about this feature but the consent prompt for each new server is certainly not ideal. Hopefully, this is something that is being addressed when it reaches Public Preview! It wouldn't be so bad if it was simply a matter of telling users to click "Yes" when it comes up but we are seeing that users also have to authenticate using their password or some other sign-in method. Is there a reason why Seamless SSO does not work for this?

In the Azure AD sign-in logs we see a sign-in failure saying "The user or administrator has not consented connecting to the target-device: '{identifier}'. Send an interactive authorization request for this user and target-machine." and the user is prompted with a message saying "Because you're accessing sensitive info, you need to verify your password." I have not found any way of getting Seamless SSO to work with this and I am wondering if others are seeing the same issue or there is something I am missing.