Registry cleaners have always been popular, but I never paid much attention to them. I originally thought that there might be valid reasons for their existence, but over time changed my mind, only to recently recognize that even today they can help maintain Registry hygiene.
It used to be common for developers to write their own application installation routines in order to avoid paying hundreds of dollars for commercial setup toolkits. Their focus in coding installers was of course the install part of setup, because coding uninstallers is in some sense an admission that the software you’ve developed might not be useful or robust enough to become a permanent fixture on end-user systems. As a result, software uninstalls were often incomplete, leaving behind Registry and file system detritus. A few hundred kilobytes of unused keys and values causes no noticeable performance impact on system operation, but I figured it was natural for a Registry cleaner to be an essential part of running a tight ship for the anal retentive systems administrator.
Installer technology has come a long way and today there are literally dozens of reliable freeware and low-cost installation toolkits available both for old-style and Windows Installer Package (.MSI)-based setup. I previously believed that meant the end of Registry scrap and any reason for the existence of Registry cleaners. However, one of the Regmon troubleshooting examples Dave and I present in our Windows internals seminars made me realize that it’s not only possible, but common, for even best-of-breed uninstallers that have earned the Windows logo from Microsoft to leave our Registries littered with traces of applications deleted long ago.
The troubleshooting example that triggered my realization is one where a user discovers that Internet Explorer (IE) hangs upon startup unless they’ve used their ISP’s dialer to establish an Internet connection before opening IE. The user captured a Regmon trace of the hang and found references to their previous ISP’s dialer in the “RAS Phonebook” key under HKEY_CURRENT_USER (HKCU, the personal settings area of the Registry). After deleting the stale values their new ISP’s dialer began launching automatically when IE opened like it was supposed to.
Some research on my part suggests that the user had manually configured a RAS phonebook entry and when they changed ISPs forgotten to update the entry, but the fact that the problematic key in the example was in the per-user part of the Registry got me thinking. If the user’s old dialer had created the phonebook entry automatically and then been uninstalled from a different account the user would have been left with a broken IE configuration. That’s because uninstallers typically delete their application’s system-wide settings from the HKEY_LOCAL_MACHINE part of the Registry and any per-user settings of the user running the uninstaller from HKEY_CURRENT_USER. But what happens to the per-user settings of the other users that used the application? You guessed it, Registry junk gets created - and possibly file system junk in the application's Application Data folder in the Documents and Settings directories of other users. An uninstall is only thorough if the user performing it is the only one that used the software.
You can verify my assertion by first installing and using any application that creates personal settings in the Registry (try Winzip if you want one to experiment with). Then log into a different account and use the software. Uninstall from either account then log into the other and you’ll find the application’s per-user settings left behind (Winzip stores its settings in HKCUSoftwareNico Mak ComputingWinZip).
Uninstallers that want to be as meticulous as possible could use the LoadUserProfile Windows API to load each profile stored on the system, as listed in HKLMSOFTWAREMicrosoftWindows NTCurrentVersionProfileList, and look for their application settings in other profiles. The problem with that approach is that Registry permissions would likely prevent an uninstalling user from deleting keys out of other user profiles. And even if permissions weren’t an issue domain users with roaming profiles carry application settings with them to other computers, so profiles that need to be updated might not even be locally accessible.
So it seems that Registry junk is a Windows fact of life and that Registry cleaners will continue to have a place in the anal-sysadmin’s tool chest, at least until we’re all running .NET applications that store their per-user settings in XML files – and then of course we’ll need XML cleaners.
Let's follow through that train of thought. If applications need per-user configuration but can't store it in the Registry, where can they store it? The choices are pretty much some kind of configuration under the user's profile (either Application Data or Local SettingsApplication Data depending on size and roaming requirements), or under the application's directory.
If you place the configuration data in the application's directory, it becomes difficult to back up - one of the main reasons for the Program Files/My Documents split way back in the Windows 9x days. It's also impossible to roam. It's pretty terrible for security too as you'd have to grant write permissions for the application directory to all potential users. This is already a major issue for auto-updating apps.
Placing the configuration files in the user's Application Data or Local SettingsApplication Data folder (CSIDL_APPDATA or CSIDL_LOCAL_APPDATA) has exactly the same problem when uninstalling - that only the current user's profile will be available. The default security descriptor in Windows XP for a user's profile is that only the user and the SYSTEM account can read or write these folders, so in fact the problem is worse than the registry.
In any case, any situation where you as a programmer allow third parties to alter your configuration allows them to make this kind of configuration issue. IE perhaps needed to be more robust in the face of this error. Perhaps Windows in general needs a way to allow the programmer to hook up a cleanup routine to run the next time a user with a profile on the system logs in - but at what point can the system be declared 'clean' and the cleanup routine finally removed?
10/2/2005 7:06:00 PM by Mike Dimmick
"Perhaps Windows in general needs a way to allow the programmer to hook up a cleanup routine to run the next time a user with a profile on the system logs in - but at what point can the system be declared 'clean' and the cleanup routine finally removed?"
More importantly, this way trojans and security flaws lie. This would require write access to someone elses registry (for example) in order to run software when they log in, to clear up their settings.
And if you can do that while uninstalling software, what's to stop malicious software from using the same loophole?
10/3/2005 11:16:00 AM by icabod
If the user is using roaming profiles, it is possible they still have the application installed on another PC so removing keys from HKCU might still be wrong. For example, you might have an application installed on your normal desktop, roam to another PC, install the application, uninstall it again when you have finished and expect your settings not to be lost on your main PC.
10/3/2005 12:44:00 PM by Anonymous
No news here. Anybody who had written more or less complicated installer should know about this issue.
What is amazing is that having forced everybody to use this humongous pile of crap which is Windows Installer, Microsoft didn't think this issue through.
This is a simple resource usage problem. A single resource (registry key, XML config file etc.) is being used by multpile entities and needs to be cleaned up when no longer in use. The answer is also known a long time and it is some form of garbage collection. For example in this case one could imagine built-in reference counting for registry keys. An installer will increment reference count while uninstaller decrement it. A system process will periodically scan the registry and clean up unreferenced keys. This would also take care of multiple installations on a roaming profile.
Of course if an application forgot to manage reference count the junk will still be there but at least good developers would have a way to write well behaved applications.
But of course such issues are not a priority these days. It is much more exciting to push some piece of .CRAP bloatware on developers than to take care of robustness and stability.
10/3/2005 1:43:00 PM by Anonymous
XML Config files..
Sort of like a Windows 3.1 INI file.
We have come full circle.
10/3/2005 11:04:00 PM by Anonymous
A very unsophisticated comment: If the user used a more thorough third-party uninstaller, such as Total Uninstall, would that help?
10/4/2005 4:36:00 AM by Anonymous
Obviously, we have another no-thought anti-MS poster!
Imagine we're talking about linux installers for a moment, the problems are exactly the same.
Root installs some software, and every user can use it. No doubt it'd store its per-user configuration in a config file in the user's home directory. So far, no problems - this is what happens already.
Now, 2 users log in and use the software. Again, no problem.
After a while, the admin decides to remove the software. Log in as root, uninstall the package... ok. What happens with the 2 user's config files? can the uninstaller go and delete them? will you trust the uninstaller to delete what it likes in every user's home dir? no? Then how do those files get deleted?
Its not a '.crap' 'bloatware' issue, but just the way the world is. No installer in the world can fix this issue without being a security issue.
10/4/2005 5:40:00 AM by andyB
Is another potential solution to go back to .INI files (which I always prefered anyway!) but to create them one per user in the directory that the application is installed in.
The name of each .INI file would be based on the name of the user running the app at that time...
Then uninstalling the app just removes all of the .INI files...
Would that work? It seems an obvious answer to me...
The registry has been nothing but bad news from a design viewpoint.
10/4/2005 6:27:00 AM by Anonymous
personally I also prefered the .INI-files before the registry. It allowed all files belonging to an application to be in one directory and so was uncomplicated to delete and above all to move to another location :)
10/4/2005 9:03:00 AM by Anders Karlsson
1) Who said anything about Linux? I don't use it and don't like it much.
2) Re "but just the way the world is" and "no-thought ... poster". If you strain your brain cells and actually read what I have written you will discover a suggestion on how to solve this issue on the system level. So this is not "the way the world is" unless Linux and Windows are your whole world.
10/4/2005 3:34:00 PM by Anonymous
XML vs .INI
Is XML necessary for a config file ?
I think that the simple VAR=VALUE form of .INI is sufficient for this kind of datas.
AND... you can edit them with vi (just kidding, use notepad++).
Other topic : the problem with LINUX is not to UNINSTALL ; it is just to INSTALL with the junk of dynamic libraries/packages dependency (somone say DLL Hell?)
10/5/2005 4:58:00 AM by Anonymous
IE shouldn't break if there's a simple user reference to a global entity which is not there anymore. This is a clear IE bug IMO.
This whole issue is OS and storage-flavour independent. Same issue is there in OS X, or whether you use .xml, .ini, registry, whatever. There's no real uninstall-time solution to remove per user leftovers. In OS X there's no install/uninstall for the majority of apps, it's just copy/delete.
Removing the leftovers is not always preferable anyway, since user can reinstall app later on and expect the settings to be there.
So, what apps can do is to handle user-settings/data with great care, and what the OS can do is to not slowdown/crash/misbehave if there are a few settings left there. Plus both of them should make it as easy as possible to identify, delete/move/save/restore these data. Fully file based setting/data storage has an advantage here, especially if there's a clear folder layout helped/enforced by the OS.
10/5/2005 7:11:00 AM by gar
I agree that the troubleshooting example highlights a bug in IE. I also agree that the problem is universal to all operating systems and I haven't thought of a clean solution. My intent was to bring the problem to the attention of people that might not have ever thought of it.
10/5/2005 9:19:00 AM by Mark Russinovich
I'm not sure that I agree that the original symptom is an IE bug. I believe that the problem you are seeing (no dialer popping up) is actually the fault of the underlying TCP/IP stack code and would have shown up if you ran Firefox, TFTP, Netscape, or any other program which attempted to reference data on the external internet.
Yes, the problem is still Microsoft's, but it is not an IE problem.
10/5/2005 10:49:00 AM by Eric Horner
The configuration data is just one part of application data and one part of data in general. I'd like to think about it in terms of "storages" having different scopes. When we are using some application we have (per-user or per-machine or per-installation ...) * (configuration or preferences or environment or application ...) data. The work document you are editing is also data. What to do with it when we uninstalling Word? There are shared documents and configurations as well. It is so complex; probably, I would rather like easy possibility to cleanup my house myself, instead of allowing everyone to come and take things from it. And all I need is a clear picture what is where. Does someone uses currently the blah-blah-Local Settings-Application Data-blah-blah-crap.xml?
It would be nice to have an intelegent garbage collector, though, which obviously must be a part of OS/file system.
An application must not even worry about how the data are stored, ini-file or registry or xml. It must be just an API for that. Unfortunately Registry API was not universal enough, it just added complexify instead of solve the problem completely.
10/5/2005 3:17:00 PM by Valery Tolkov
Even if your (un)installer is configured to gently remove all te settings from all the available users; (even if there's a way to do that, reference counting, or something), there still can happen a registry junking - you mentioned it yourself a couple of records ago, about paint shop pro. If you have an app, run without (un)installer, it may choose to create its own missing settings, and you will have no way of knowing about that.
And, in this aspect, echh, fallback to ini-files (or xml) seems to be so very preferrable (if they're in writeable app-dir - you see them, and can delete them; but if they are somewhere in user-dir - most likely you won't even notice them - BAD), which brings a wild idea that an application could have write permissions to the directory it's resided in. I can't quickly figure out drawbacks (except for system applications, and keeping user settings after uninstall when needed), but this could make uninstall just by deleting the app folder.
Just a wild idea.
10/5/2005 5:06:00 PM by Anonymous
I was expecting Mark to end his article with something like "...and therefore, I've decided to take the matter into my hands and here is my new registry cleaner!" and come up with yet another excellent tool :)
10/5/2005 6:51:00 PM by sc0ri0n
Hi Mark, do you really think that Registry junk left by uninstalled programs could severely slow down the computer? I would like to 'hear' your opinion.
Great Blog, man.
10/6/2005 3:25:00 PM by Anonymous
No, even if the registry was massively bloated there would be little impact on the performance of anything other than exhaustive searches.
On Win2K Terminal Server systems, however, there is a limit on the total amount of Registry data that can be loaded and so large profile hives can limit the number of users that can be logged on simultaneously.
I haven't and never will implement a Registry cleaner since it's of little practical use on anything other than Win2K terminal servers and developing one that's both safe and effective requires a huge amount of application-specific knowledge.
10/7/2005 9:41:00 AM by Mark Russinovich
No, not exactly. If per-user configs are stored in a user-specific directory (~username on Unix/Linux or Documents and SettingsUsername on Windows), then any user configs left over after an uninstall clutter up the user's personal space
-- they don't affect anyone else. If they're stored in a shared location like the Registry, they clutter things up for everyone who stores their settings there.
10/7/2005 12:37:00 PM by Ernie
The solution: Make everything "per-user". When a user starts an application for the first time, copy it to the user's application folder (after asking the user if this is ok). The user is then responsible for uninstalling his/her own copy of the software, and is not affected when an administrator removes the application from the "Program Files" folder. Let the file system handle duplicate files efficiently.
10/8/2005 5:24:00 AM by Anonymous
I think this is something that Windows Installer could take care of -- we have Application Data and Local SettingsApplication Data in the per-user area, and we already have the ability to specify what registry keys should be removed when the application is uninstalled. Why don't we have a way to run the installer in a 'cleanup' role, during the time that userinit is being run?
Things like NSI should be able to tell MSI information about its' installed apps' own registry keys and such -- but I dunno how well that would work, from a security standpoint. [the installer database should already contain information on what files go where and what else is using them -- being able to tell MSI that your own custom installer is using something even if it's not in MSI format would be a great boon.]
10/8/2005 7:55:00 PM by Kyle H
Remember that some applications require an uninstall of the old version
they will allow install of a new one (VMware is a good example of one of these).
If the uninstall were to clean out all of the user settings, then the new version of the product would have no user settings to "upgrade". Another example is in moving from Office XP to Office 2003. The cleanest way to deal with it in a large organization is to uninstall Office XP completely (including MUI's, etc.), and then install Office 2003 (and replace whatever MUI's they had). When you do that, the Office XP HKCU stuff is left. That's a good thing in this case because it allows Office to "upgrade" the settings that still make sense in the new version. If the HKCU settings were completely removed the user would have to start from scratch to make their customizations.
It would be nice if there was a way to tell the uninstall "leave my settings" or "remove my settings" - but I don't think a standard answer of "always remove user settings" would be a good thing.
10/9/2005 11:29:00 AM by Jerry Ham
"the anal retentive systems administrator"
Isn't the correct term "anally retentive"? I mean, duh!
10/10/2005 6:44:00 AM by CowHat
andyB: this is not a security issue. Since the uninstall is done under the security context of an administrator (regardless of whether it's done manually, via group policy, sms, or whaterver), then by definition you DO "trust the uninstaller to delete what it likes in every user's home dir". One must, of course, assume that the vendor has implemented their uninstall routine properly and it only removes original program keys and files.
It's really not that difficult of a technical issue, although the software package vendors (e.g., wise) would have to make changes to their products to allow for a proper cleanup.
10/10/2005 5:32:00 PM by Manny
Desktop/server application installs will always have the potential to be complicated and messy. There should have been a set of install standards created long ago that addressed these issues so platform/OS creators could offer APIs, libraries and utilities to guide app developers.
I use the past tense because it's probably too late to make a difference with installs. Mainstream users will move overwhelmingly to web/thin client apps over the next 10 years. The only people who will have to worry about installs will be server admins, and nobody cares if a server admin has to spend weekends with the mess left over from a nasty uninstall.
10/11/2005 12:18:00 AM by Henry Tero
I too prefer user settings stored in HKCU to be kept.
One annoying observation: Some recent games refused to run on my system, complaining about "CD emulation software detected". The culprit was some CloneCD settings left in HKCU after a short evaluation of that software a couple of years ago. (thank you Mark for regmon!)
I deleted those settings and ran the game (BF2), but what a truly moronic way to detect the presence of third party software... I dual boot and my x64 Windows share a user profile with my 32-bit Windows. So even a freshly installed Windows version failed to launch BF2. Until I started suspecting HKCU... (no thanks to EA Support btw)
Finally: I prefer one big registry hive as opposed to thousands of .ini (or .xml) files! Much easier to copy!
10/12/2005 3:02:00 AM by Rune
On the comment above about "per-user" space rather than "shared" storage space, this shows a fundamental misunderstanding of how the registry works (which isn't that uncommon for someone unfamiliar with Windows internals unfortunately).
The Registry isn't a monolithic entity that exists in some shared data space. It's broken up into shared sections (HKLM and friends) and per-user sections (HKCU etc). Just as on a unix system, the per-user configuration is stored in your home directory (~/NTUSER.DAT to be specific). So, there is really no fundamental difference between an arbitrary ~/.XXX file on Unix and the HKCU tree in the Registry on Windows in terms of "shared" vs. "per-user" space. The only difference is the Registry on Windows provides a centralized and common storage format which all applications can use and share while Unix apps tend to have to write their own custom dotfile parsers.
10/13/2005 11:41:00 AM by johnw
. about that but I think that Windows Installer Service (MSI) has a way to solve the left behind keys that are left in the HKLU.
When you uninstall a program that other users have used then the for each user the MSI will run again uninstalling the only the current user parts.
10/19/2005 5:16:00 AM by Shay Erlichmen
How could an uninstall engine load a user profile if the user is not logged on?
It seems that an access token is needed for LoadUserProfile. So the most an uninstaller can do is clean up all profiles already loaded through HKEY_USERS.
10/19/2005 9:17:00 AM by Super Letop
1) An application can't store settings in its own installation folder. Because users belonging to the User group have no write access to "Program files" folders. That's the reason of "Application data" being in the user profile folder.
2) INI files are good for simple settings, their lack of any structured storage capability IMHO makes them not so useful sometimes. I already use XML config files for Win32 applications.
10/26/2005 5:01:00 AM by LDS
I use linux with "apt-get" and I don't have this problem.
Maybe spending so much time thinking about the design of Windows is stupid. After all, the creators certainly didn't make that "mistake."
11/1/2005 1:46:00 AM by RareCactus
"Is another potential solution to go back to .INI files (which I always prefered anyway!) but to create them one per user in the directory that the application is installed in.
The name of each .INI file would be based on the name of the user running the app at that time..."
This is not a good idea:
a) INI files are slow compared to a indexed database (such like the registry).
b) More importantly the user would need write access to the app directory. Definitely a no-go! Settings and Programs need to go in different places.
c) And even worse, roaming profiles would not work.
The Registry is a great thing for storing settings.
Any leaving user settings unintact is no bad thing as a previous poster said: the (roaming) user might use the software on a different computer, people always seem to forget that.
11/1/2005 7:16:00 AM by Lofote
"I already use XML config files for Win32 applications"
Uh another one :(... You won't believe what a big pain these files are for automatic deployment of some(!) settings for all users for an admin, uh? And how slow these files are?
XML can store quite complex structures. They were made to exchange information between programs or platforms and are superp for this purpose.
But XML are defintely a no-go for saving settings out of the reasons above.
Please, as a developer, stay with the standards, don't be "smarter" in this case, because you always ruin things, that work perfectly when using the standards.
The standard for Windows programs are Registry, HKCU for users-based settings, HKLM for computer-based configuration.
The standard for Linux are config files. Plain text files.
Don't go overboard with "smarter" things, because they have lots of drawbacks - stuff that is handled perfectly when using standard stuff, that the system is optimized for.
Performance for example is *by far* bigger when using the registry. Anybody who learned the advantage of indexed(!) databases will know that.
11/1/2005 7:23:00 AM by Lofote
When I was writing an installer for one of my programs recently I was looking for a way to make sure the installer would also allow to uninstall the program completely. My conclusion was:
Install a program 1) either for all users of a computer or 2) only for the currently active user. If the user has admin rights, ask them which option they want. Otherwise, give them only the second option.
1) If the installation is for all users of a computer, the installing user must be an admin. Install the program under "Program Files" and the registry keys under HKLM. Per-User settings are not possible in this scenario.
2) If the installation is for the current user, install the program under "%userprofile%Program Files..." (you must create the directory) and store any configuration data in the registry under HKCU.
If more than one user installs and uses the software on the same computer, you end up with several copies of the program files under the users %userprofile% directories, but this is the only sure way for a complete uninstallation with the given conditions under Windows XP.
11/2/2005 8:54:00 AM by Caspar
There should be a wrapper, which intercepts registry access and re-routes it to INI files. So, if an app reads an entry from the registry, the wrapper should look whether the value exists in the ini file. If yes, get it from there, if no: read the registry value.
If the app tries to write to the registry, the wrapper should write the entry to the ini file and leave the registry untouched.
It's something like a registry-sandbox. And uninstalling would be no problem.
11/4/2005 1:41:00 PM by Markus Birth
I was wondering along the lines of an alternative heirarchy for the per-user config directories.
Right now we have a user profile directory and the software creates it's own directory inside that. Access rights are easy to implement (we just need to secure the user's parent directory) but we end up with config junk (here i'm assuming the registry has not been used).
Alternatively we could have a common config directory for all software with a seperate sub dir for each sofware and per-user settings would appear inside individual (secure) user directories. I think in terms of fragmentation and disk utilization it would be a step
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.