ISSCAN is a utility I created several years ago, and it is unique in my Microsoft experience in that it was designed, written, tested and shipped in a span of about 24 hours. Now, I’ve worked at small software firms in the past where releases of “Thursday at 2:00” were not uncommon, but such a thing is pretty unusual at MS (and I think this incident is a pretty good indication of why that is a good thing). The reason for the speed in this case was because it was in response to the first big email virus to hit the world, i.e. Melissa. Melissa really took us by surprise. We were not prepared for such an eventuality at all.
So what we had was Exchange systems all over the world inundated with these emails. We needed some way to strip out the bad messages. The closest thing we had was something called ISINTEG, which is the store integrity checker. It scanned through an Exchange store at a low level and looked for problems and inconsistencies. I decided to take this tool and morph it into ISSCAN. I disabled all the existing tests and created two new ones which were copied from two of the existing ones. One would scan all the messages in each folder, and the attachment list for each message, and delete the attachment. Another would scan the attachments table and delete all attachments that matched a given file name. Since Melissa wasn’t self-morphing like some of the later viruses, this was sufficient to do the job. Now, both of these tests ended up doing pretty much the same thing, but the attachments one was quicker because there are typically fewer attachments than messages. Speed was pretty important because to run ISSCAN, like ISINTEG, the mdb being processed needed to be dismounted.
Well, as I said this was turned around completely in 24 hours and by Sunday morning (the virus hit late Friday afternoon) Exchange administrators around the world were running it. Some time later I discovered to my horror that the code was actually causing a database corruption that in certain cases would case some 5.5 systems to not properly clean up attachments (because they would choke on the missing ones). Fortunately, the nature of the corruption wasn’t terrible but I did try stop it from being used after I discovered that (unsuccessfully – people were still using it when “I Love You” hit later on).
The next week we began designing VSAPI, the interface used by virus scanners today for scanning Exchange mail. It took a little longer than 24 hours to release, however.
The one thing this experience did for me was to drive home the critical nature of the work we do here – had I made a bigger mistake I could have caused major problems for the entire planet’s email. A very sobering thought. So next time you wonder why we take so long to turn fixes around and release new functionality, consider this case.
- Jon Avner
