Most of the major security suite vendors release their new editions in the fall, but some run ahead or behind the rest of the pack. With the release of Kaspersky Internet Security 2009 the fall flood of security suites has officially begun. I'm working on two other suites right now, and by the time I finish with them the flood will be in full spate. This year I'm adding a new kind of testing to see just how much performance you're giving up in exchange for security.
I hear tons of complaints about how this-or-that suite slowed the computer to a crawl, added minutes to boot time, or made web surfing unbearably slow. But I can't rely on anecdotal evidence for reviews - I had to develop some repeatable and quantifiable tests to see just how much a given suite affects performance.
I created and collected a set of batch files and scripts that check the time required to perform certain real-world actions, things that a security suite might affect. One script visits each of a lengthy list of URLs and measures how long it takes to completely load the page (an ActiveX control reports when loading is complete). Parental control, antiphishing, and ad blocking could easily slow this process. Another test copies and moves files of various types. Depending on whether the suite scans files more times than needed and depending on what types of files it scans this might be slowed. To this same purpose another test times how long it takes to zip and unzip a large collection of files.
The process of installing a Windows Installer package involves tons of file and Registry events that must be monitored by the suite. For testing purposes I use commands that run a silent install and silent uninstall of several large packages. Another script records the time required to make some lengthy audio format conversions - if the suite is sucking up CPU resources this time could be affected.
Every time you boot the computer the suite must load a ton of different security monitoring modules, some before you even see the desktop. I set to work on a script to measure the time from the start of the boot process until the system was totally ready, but ran into some snags. I got some help with the scripting from a Symantec engineer. Windows itself will report when the most recent session started to boot, and I defined "ready" as the point where ten full seconds have passed with under 5% CPU activity.
With everything in place I ran the main set of real world tests over and over and averaged the results. I ran the boot-time test over fifty times and again averaged the results. Now I'm prepared to run the same tests after installing each security suite and compare the results. It's going to be a little rough to start - I won't know exactly how to interpret the results until I have a few data points from different suites. And I plan to add a few more tests specifically targeting CPU and resource consumption. Going forward I'll be able to say "Suite X slows browsing by half" or "Suite Y adds two minutes to boot time". That will be fantastic information for those trying to make a choice.
Source: Yahoo
