Ok, I have found the problem: It's a bad idea to use a (un-trusted) self signed certificate. After I have assign a certificate from my certificate authority it works how excepted. The next trap was the using of the "websocket" protocol in the communication to events, powershell and remote desktop. But this issue is described in the FAQ and after I have enabled the feature on the IIS I can access (theoretical) from anywhere to the servers. Now all works great and I'm very happy.
I know it's been a while, but I am having no end of trouble trying to get this working. Initially it was the SSL certificate, which is now fixed, but now I just get repeated password prompts, and constant 401 responses from server > client.
Did you follow a guide to get everything (except websockets obviously) working? Could you share?