Testing companies make money in various ways. AV-Test.org is often commissioned by technology magazines such as PC World (a magazine owned by IDG). Virus Bulletin licenses its logo to companies for use in promotional material and publishes a monthly online magazine.

Earlier this month, Virus Bulletin announced that its latest round of testing produced some "big-name failures," including products from Kaspersky Lab and Grisoft SRO.

The company's VB100 tests antivirus engines against malware samples collected by the Wildlist Organization International, a group of security researchers who collect and study malware. To pass the VB100, products must detect all samples.

Kaspersky briefly removed a signature for a worm out of its product for "optimization" purposes on the day of the test, wrote Roel Schouwenberg, senior research engineer for Kaspersky, in an e-mail. The signature has since been put back in, he said.

"Obviously, we would have rather passed than failed," Schouwenberg wrote. "Had the test been conducted a day earlier or a day later, we would have passed."

Similarly, F-Secure initially failed its test also because of a technicality, but the failed rating was later reversed. All vendors are told after testing which samples they failed to detect, thus most end up adding signatures to their products.

So what should a user do? John Hawes, a technical consultant for Virus Bulletin, cautioned that the signature-based tests are "not enormously representative of the way things are in the real world."

But Hawes also noted that signature-based tests can indicate the reliability and consistency of a vendor's software. Virus Bulletin also writes reviews of AV suites, which take into account aspects such as usability, which may be just as important as detection for consumers. The company is developing more advanced tests that will test new security technologies.

At a bare minimum, through, users should install some security software, as computers without it can face high risks, Marx said. Several free suites are available that may be fine for light Internet use, he said.

Ironically, Marx doesn't use any antivirus software. That's because AV-Test.org collects malware for its testing, most of which comes through e-mail from other researchers. "I'm getting about 1,000 viruses a day," he said. "It [antivirus software] would be counterproductive."

Correction: Due to a reporting error, this story as originally posted incorrectly stated the number of malware samples. The article has been amended.