Danger And Benefits of Obfuscation
Most of the malicious code we see on a daily hourly basis is obfuscated in one way or another. PE (portable executable format) files are packed (compressed and/or encrypted), scripts are encoded and/or encrypted, etc. Obfuscation is one of the biggest challenges for content scanners today, both on the host and on the wire. Emulation has been instrumental in getting past layers of obfuscation without the need for custom decryption code with each passing threat, by allowing malicious code to decrypt itself in a “sandbox”. However, over time anti-emulation, anti-debugger, and more generally anti-deobfuscation techniques have made this more challenging; and along with emulation, comes a performance impact. But, there are more rudimentary ways to tackle this problem.

Obfuscation is a double edged sword. At some point, the methods used go to such an extreme to evade detection, that the method itself is enough to base detection. In October last year, an obfuscation module was introduced for a popular penetration testing toolkit. One of the methods used involves generating random white space inside of HTML exploits. This tactic can evade detection in some cases, but this “noise” itself can be enough to trigger on. Valid files do not typically contain such noise. This paradox is present in other areas of threat tactics as well. Take social engineering for example. Email spam, and even viruses, that are so overwhelmingly written to trick users into taking some action often stick out like a soar thumb. Another example is the plethora of threats that do not function in typical forensic environments used by researchers. Today it seems that more bots are built to NOT run under virtual machine environments than those that are. The result—many infections can be avoided by simply by running in a virtual environment. As more and more users run virtual machines, the anti-researcher technique becomes a hindrance to the malware.

Internet Explorer “Feature” Exploited
In June 2006, an issue was reported in the way Internet Explorer interprets ASCII characters. IE only takes into account 7-bits while interpreting ASCII encoded 8-bit streams, ignoring the most significant bit (8th bit). For example, both values shown below are interpreted as character ‘A’ if we consider only 7-bits and ignore the 8th, but the representation is different if all 8-bits are accounted for. Other browsers however do not show this behavior.

This issue has been discussed before and it seems due to the ambiguity in specifications, it cannot exactly be considered as a bug in IE. Whether Microsoft got it right in IE and most everyone else got it wrong (including Mozilla and Opera), or the other way around, it is a challenge for most traditional anti-malware scanners when looking at 8-bit character representations of web pages. This technique can be used for malicious purposes and otherwise-known threats can suddenly “appear unknown” to scanners and yet render fine with IE. We ran a small test by “encoding” some of the well known and detected threats using this technique, and none of the AV scanners tested passed (including Microsoft’s).

Obviously, this problem can be solved. Either by fixing the bug/feature in IE or by updating most content scanners to function the way that IE behaves.

There may be another option. Like the aforementioned obfuscation techniques, this encoding poses a hindrance in detection and at the same time opens a window for some proactive detection where existence of 8-bit characters with values greater that 0×7F (maximum possible with 7-bit) can be considered suspicious in the context of ASCII encoded web pages.

Real-World Attack
McAfee Avert Labs has been monitoring this technique being employed in the wild for malicious purposes. In one recent case, the payload exploits MS06-055, a patched Microsoft VML vulnerability to download the W32/Fujacks.ab virus.

Fujacks.ab is a variant of Fujack.aa (the first known worm to leverage the recent ANI file handling vulnerability). The nefarious group behind these Fujack variants was one of the frontrunners in hosting ANI exploits (patched in MS07-017). So not only were they early adopters of ANI file exploitation, they were also early adopters of 8-bit ASCII malware obfuscation.

The MS06-055 exploit connected with Fujacks.ab utilizes obfuscation techniques discussed in the first section of this blog, and is proactively detected as Exploit-ObscuredHtml as a result. Also VirusScan’s ScriptScan is able to see past the 8-bit ASCII encoding and detects as JS/Exploit-BO.gen.

It is interesting to note that none of the other AV scanners tested detect this obfuscated sample even though many do detect once decoded.

McAfee ScriptScan to the Rescue
Emulation can be an effective way to get underneath obfuscation, but anti-emulation techniques may circumvent this approach. McAfee VirusScan products contain a feature known as ScriptScan. ScriptScan is a technology capable of scanning beyond the obfuscated layer in client-side web script files. Most obfuscated scripts contain simple but redundant arithmetic algorithms and variable randomization that bypasses most file scanners. They are a challenge to products that only scan these files at the top layer because legitimate scripts can contain similar algorithms. ScriptScan monitors script execution in Microsoft Internet Explorer (IE) and scans the underlying scripts exactly as decoded by IE. More critically, scripts must be decoded to run and ScriptScan is initiated before they can execute; effectively blocking malicious scripts from execution.

Why Many Comparative Tests Are Flawed
While our tests show a lack of file detection, they are admittedly flawed. They are flawed for the same reason that VirusTotal and a number of other comparative tests are flawed; they don’t test threats in their real-world environment. 8-bit ASCII obfuscated threats may not be detected by command-line, on-demand, or even certain on-access scanners. However, if those threats are scanned in the course of being rendered by Internet Explorer, the obfuscation is removed (which is what allows VirusScan’s ScriptScan to detect). It is unclear how many AV products contain this feature. While such an approach is not possible at the gateway, emulation may be a partial solution. Clearly a challenge with emulation is that one must code the emulator to mirror the behavior of the interpreter, in this case Internet Explorer. And by mirror, that includes coding in the same bugs and features, such as IE’s 8-bit ASCII decoding. It is believed that the majority of web content scanners do not handle such decoding the way IE does. It would be prudent for Microsoft to resolve this, and remove the capability from the hands of attackers.