aad login linux broke ssh with key

have anyone face an issue where if we enable aad login extension for linux VMs, key based ssh is asking for password.

I am not sure which password it is asking. Can some one please help on that?