Emails being encrypted for sender in Microsoft Outlook for Microsoft 365 MSO (16.0.13628.20234)

I have setup email encryption functionality, however it has not been operating correctly.

Around 1/2 of the time, the emails will be encrypted for the sender, not the recipient. This can be checked in the Encryption Layer description of these emails.

I can confirm that all setup has been done correctly. Has anyone else faced this issue and know how to fix it?