For scoring purposes, you’re absolutely right that system changes cleaned up by uninstall (or not) have to be counted with NO other programs added or removed during the time that the program being tested is on the system, or there won’t be either fair or consistent results.

I’ll also agree that one high-CPU-use startup entry is far worse than 4 entries in the ‘extra button’ category. (And so on. Mileage will vary.) But some kind of scoring is needed, and CPU utilization can’t be easily measured for an installation package; the process of monitoring throws off the accuracy of the measurement, and the effects of any startupware on CPU usage will vary with the speed of the test box. (The Heisenberg principle comes to computers…)

Something simpler is needed, and that does mean something arbitrary; can’t be helped. Suggestions?

I don’t see the need to count any settings changes if they are uninstalled correctly. If the application chooses to use many of what you have called “settings” (but haven’t defined, mind you), how is that bad for the speed of the machine? The registry might be larger, so everything using the registry could be affected slightly by the bloat, but that’s not an ongoing effect once things have initialized.

Note that there are likely to be some settings that cannot be undone if other applications overwrite them since the application being uninstalled changed them. You can’t count those as not being uninstalled. A prime example is extension mappings. You should clarify that an application is only responsible to undo what it has done to settings.

The scoring should account for memory and CPU utilization. An application running several small auto-run programs that are efficient consumers of memory and CPU can have a higher score than an application with just one auto-run program that’s a resource pig. The former would be more appropriate to run, if you really needed to choose one or the other, so the scores should reflect that.