Rather an unusual meeting was held just a few days ago in Iceland. Major AV developers and AV testers met for two days and discussed many important aspects of how contemporary products should and should not be tested.

The workshop was held on 15-16 May and there were 66 attendees representing many computer security companies (AhnLabs, Aladdin, Alwil, Authentium, Avira, Bit9, Datsec, Earthlink, EMSI, Eset, Frisk, F-Secure, Gdata, Grisoft, K7, Kaspersky, McAfee, Message Labs, Microsoft, Norman, Panda, Softwin, Sophos, Symantec, Trend, UMU) and major testing bodies (K.Brunnstein of Virus Test Center in Hamburg University, A.Clementi of av-comparatives.org, J.Hawes of Virus Bulletin, A.Marx of av-test.org and M.Parsons of Westcoast Labs).

The attendees, of course, knew each other very well even before the meeting. After all we do regularly get together at major conferences devoted to malware research. But never before have we had a chance to focus our discussions of security products’ testing and do it in such a depth.

Let me highlight the following points in our discussions:

- how to test and compare security products that provide different and multiple lines of defense (for example – pure scanner versus behavioural/heuristic system or a system with protection rules; with or without firewall, etc.)

- focus on prevention and positive user experience

- total live virus testing (real sample running on a real computer – not only an on-demand scan of a file collection)

- detecting programs that are packaged using “bad” packers/protectors (they are frequently used to obfuscate programs and many of them are almost exclusively used by the bad guys to hide and mutate viruses, trojans and adware)

- discussions about the Wildlist and how the “telemetry” data collected by AV companies can improve the information about threats in the field

- discussions about government-approved backdoors and surrounding laws should they appear (also known as “Magic Lantern” and “Bundestrojan” problems)

- test strategies and common mistakes

- testing heuristic detections

I found the meeting very useful and I am very glad to use this opportunity to thank all the employees of Frisk Software who organized the workshop!