I recently ran into issues with publishing Exchange 2010 OWA behind Azure AD Proxy. I had tested Exchange 2013 OWA with Azure AD Proxy successfully and then decided to deploy it for a customer for Exchange 2010. Deployed it successfully and user were able to logon successfully through Azure AD Proxy and MFA. However users experienced annoying Notification Icon on top of the page and when they click it it will show error message, "Notifications could not be retrieved", users were not able to see new emails unless they refresh it.
I worked with Microsoft Premium Support over 40 hours and we were not able to fix it. I eventually added Exchange 2016 to Exchange Org and Published that through Azure AD Proxy.
Again users with mailboxes on Exchange 2016 had no issues, but users on Exchange 2010 continued to have same bad experience.
As of now there is no plans by Microsoft to fix it, therefore realistically Exchange 2010 OWA with Azure AD Proxy is not supported.