ActiveX Sprayer: A Real World Adventure with Enterprise Mode for Internet Explorer

[Chris Jackson]

First posted to MSDN on Oct, 26 2014

Before we released Enterprise Mode for Internet Explorer 11, we ran a fairly large pilot program – we targeted 30 customers, and ended up with more than 90. But there’s still always the question – will we have, by chance alone, gotten unlucky and chosen a group of very non-representative customers? So far, however, what I’m seeing is that the experience of our pilot users is playing out, again and again, in the real world – terrific news! But, I’m still listening for those customers who aren’t having a good experience. We expect Enterprise Mode to help with the vast majority of issues – and when customers find more than a handful that it doesn’t catch, I’m curious why. (I’m also curious when it’s only 1, to see if we can somehow make Enterprise Mode even more compatible and drive that number to 0.) So, I had one customer who believed they were the exception – convinced that they had at least 30 applications that Enterprise Mode wasn’t fixing, and perhaps more. That’s quite a bit more than we expected! So, I tried to find out why, and was able to schedule a visit to investigate. The first thing I came across was a series of issues with Java and certificates. In particular, starting with Java 1.7 Update 51, Java happens to enforce more rules around certificate quality, and the unsigned and self-signed applets driving these applications were presenting a problem. The customer had already figured out a workaround, and we just finished verifying it and then troubleshooting any test cases where the workaround appeared not to work, got it to work and we were on the way. So, in the end, these issues turned out to have nothing to do with Internet Explorer 11 at all – they were issues brought about by the Java upgrade they were pairing it with. And that’s an important learning – while Microsoft (and every other software vendor) clearly sees the separation between their products and the other guys’ (e.g. IE and Java), customers and users don’t find it nearly as cut and dry . The solution to seek, then, is to find ways to behave better as an ecosystem so the customer doesn’t get caught up in turf wars – I’ll project that to be a longer term solution. Up next, we had a collection of issues with URLs being malformed, and therefore generating 404 errors. But the URLs were being generated by another application and fed to IE already mangled, so again, IE wasn’t implicated. Another reminder that customers see the ecosystem, not the individual products. And again, we verified the workaround and marched on. Finally, we came down to bona fide IE issues that Enterprise Mode actually wasn’t fixing! And this one was fascinating. The application I was troubleshooting, to make things more exciting, was operating in a popup window, so I didn’t have the benefit of developer tools at my beck and call, but I’m hardly one to back away from such a challenge. I had a script error, so I figured I’d just sort out with bit of HTML was the problem, and go troubleshoot that one directly, outside of the content of the script window. Interestingly, though, the path to the source file shown in the error message pointed to the %temp% directly. Temp? Really? Really. I headed there on the file system, and sure enough, there it was. Did we navigate to temp? Well, no – actually it was a web page hosting an ActiveX control hosting a web browser control hosting this page. So, you know it has to be good! OK, so I have the file in temp, ready to debug – and I could make quick work of it. The script error was that the function wasn’t found. The function wasn’t found because it was in VBScript, and it was being forced to Edge mode which, in IE11, no longer supports VBScript. Now, if you’re following along and are an IE nerd, you’ll probably be asking yourself – wait, if it’s pulling from the %temp% directory, that counts as a local file, and should automatically get compatibility view, right? And you would be correct! However, the page needed to leverage features that IE7 didn’t have – so, rather than add in X-UA-Compatible, the developers chose to add Mark of the Web, which forces us to Internet Zone, which nudges us to Edge mode. Whew. OK, so, mental note that we’re not communicating on X-UA-Compatible well enough, and onward – we should be able to fix this up with the simple addition of an X-UA-Compatible tag, so I do. Once pushed downwards, the VBScript is supported again, and the page lights up. Problem solved! But, before I assign it a pass, let’s see it in the context of the full app, and not a local file. Fire up the full app again and … fail. So, I took a peek at the local file again, assuming I missed something, and it had reverted back. What happened? Well, when in doubt, use ProcMon – sure enough, every time you navigate to that page, the ActiveX control was replacing the %temp% file with a new copy of the old (broken) version of the page. Oy. But, I’m not ready to give up yet – let’s just write protect that file. I changed the permissions so that the user couldn’t update any longer, and then tried again. And ProcMon confirmed my fears – the control caught the ACCESS_DENIED, retried 3 times, and then gave up – not going any further. It wouldn’t work unless it got to lay down a new copy of the (broken) old HTML file. I just need to go and find where it’s pulling from then. ProcMon shows no trace of a direct file copy, and a network trace shows it’s not coming from the network at all. We even scanned the database to see if we could find it. Where was it then? Well, just where you’d expect – the file was being pulled from the resources section of a compiled DLL. Seriously. Rather than pulling the file from the web server that, by definition, they already have, they compiled it into a DLL and placed it on every client system. Then, they spray the HTML file into %temp% every time they want to use it. How do you fix that up? With a resource hacker of course. Hack up the HTML, and then close up the binary. Sure, enough, it started spraying my fixed up HTML, and it worked – repackage the CAB with the new DLL, and we’re on our way. Enterprise Mode wasn’t fixing it because Enterprise Mode is an IE feature. Technically this web page wasn’t being hosted inside of IE, it was being hosted inside of an ActiveX control, and that ActiveX control didn’t implement an Enterprise Mode feature. It was a consequence of a really … erm … unusual? unique? completely mental? … architecture.

The takeaway here is that Enterprise Mode ended up fixing nearly, but not quite, everything. And, when it doesn’t fix it up, chances are it’s a really interesting situation!