Which client are you using? I can redirect USB smartcard readers from the full Windows 10 Remote Desktop client. You can even add them plug-and-play. It doesn't seem to work when accessing WVD from the webclient though. The big problem with smartcard redirection is the middleware used by many smartcard readers. Sometimes they require special installations before they work. But that is the same behaviour as in classic RDS deployments.
Obviously you can also block/enable smartcard redirection through GPO's.