Shall we all write viruses to find the best antivirus?
Wednesday August 16, 2006 at 5:38 am CST
Posted by Igor Muttik
A Website called ConsumerReports.org today published an article (strangely it was dated “September 2006) about a test they conducted involving 5,500 samples of artificially created virus samples.
There are several things here that do not seem right:
- It is claimed that created viruses were “the kind you’d most likely encounter in real life” which is, of course, something the testers cannot know.
- Creating new viruses for the purpose of testing and education is generally not considered a good idea - viruses can leak and cause real trouble (you can read an open letter on the AVIEN site about that).
- There is a more scientific way of measuring real proactive detection of AV products on future malware - it is called “proactive testing” or “retrospective testing”. The idea is to measure, say, 3-month old AV product against real field viruses that appeared within these last 3 months. The discussion of the methodology of such tests can be found here and some real test results with common AV products are on the AV-comparatives.org site.
- Objection #1, that ConsumerReports.org cannot know what viruses we are going to face in future could be moot as their testing team apparently invented a time machine and shifted themselves forward to September
.
January 20th, 2008 at 6:22 am
Dear Sir,
isn’t the tragic truth rather that you are upset because your “security software” was widely unable to detect simple modifications of existing malware, although it obviously should?
You are trying to rule out something that your software did not detect because your software did not detect it. This is the same approach that the many “secure by obscurity” security systems take. It also strongly reminds me of the propaganda and ideology that were common in the USSR and East Germany, and which is still encountered in China today (”we have no gays”, “there are no HIV infections in our country”, “there is nothing to criticise”).
Denying an obvious condition does not make it disappear, and reacting to criticism by declaring criticism invalid or illegal is simply evidence of incapacity.
You make your living by selling a solution that presumably makes your customers’ computers safe for a lot of money. What does it tell us if a few people who aren’t even professional virus writers can easily bypass your security, and instead of assessing the problem, you leash out propaganda?
April 26th, 2008 at 9:42 pm
So perhaps the methodology wasn’t as professional as you would like. So what. The simple fact is that Consumer Reports created a variety of viruses and anti-virus protection software such as McAfee didn’t do a very good job in protecting against those viruses.
Listen, as you well know there are thousands of new viruses for Windows created every month. How is Consumer Reports creating viruses for the purpose of testing different then the real-world reality of hackers creating viruses every day?
Good article: http://securityabsurdity.com/failure.php