Word and Excel get "RPC server is unavailable" exception when RDC is established to Win Server 2012

I have a c# program that implements some automation tasks with Office PIA (Primary Interop Assembly) in Windows Server 2012.  It works fine until a RDC (Remote Desktop Connection) is established by another user.  A COM exception of "RPC server is unavailable" is thrown and the application errors from event viewer are

May I know is it an installation/deployment issue of Office 365 in a RDS server? How can I fix it?

Thank you.