Technical Director
Norman Data Defense Systems Development B.V.

EMULATORS for Generic Decryptors

My name is Frans Veldman. I'm the author of Thunderbyte Anti Virus, also known as TBAV. I have been working in the anti-virus industry since 1989, and I witnessed the evolution of the computer viruses.

A binary DOS virus is a small program that attaches itself to other programs. The usual approach to detect these viruses was to scan a file to see of some piece of the virus, a signature, could be found. Of course, such a signature has to be composed carefully, to avoid false alarms.

Since the beginning, virus authors have tried to make their viruses undetectable. Several approaches have been tried, but none of them turned out to be successful. One approach was to encrypt the virus with a variable key, which was an attempt to make it impossible to find a signature. However, it is impossible to execute an encrypted piece of code, so the virus has to decrypt itself prior to execution. Usually, these encrypted viruses have a small routine in the beginning, which decrypts the rest of the virus. The solution for the anti-virus industry was to create a signature from the decryptor on the beginning of the virus. A problem here was that the decryptor was usually very small, so only a very small signature could be made. This would cause false positives, but the solution here was to look only for this signature on exactly the place in the file where the decryptor could be expected.

Mark Washburn was the first one who created a semi-polymorphic virus. To make it impossible for the anti-virus industry to create a signature from the decryptor, he designed the virus in such a way that it generated random instructions within the decryptor. Now even the decryptor was not fixed anymore. However, the set of instructions used to generate the random sequences was very limited. In addition, in order to avoid interference from the random instructions to the proper working of the decryptor, the instructions were of a do-nothing type.

Dark Avenger was the first virus author who created a real polymorphic virus: MTE. This virus does not sprinkle random instructions between the real decryptor instructions, but it dynamically generates the decryptor itself. So, also the instructions which play an effective role in the decryption of the virus are now randomized!

Algorithmic approaches

This was quite a problem for the anti-virus industry. Most solutions to detect MTE at that time relied on an algorithmic approach. An algorithmic approach consists of a set of rules and an interpreter. The anti-virus program carefully looked at every instruction in the beginning of the program to see if it could have been generated by MTE. If this was the case, and the piece of code looped back with a conditional jump, followed by undecipherable code, the scanner decided that this program contained an MTE virus.

The success of Dark Avenger's approach was not quite ignored by the virus authors. Several followed his ideas, and at some time, the anti-virus industry was almost daily confronted with yet-another polymorphic virus. These polymorphic viruses were very time consuming for the anti-virus industry, because it was impossible to create a signature and every time one has to find an algorithmic way to detect the virus, which got harder every time.

Generic decryptors

Around 1993, it was clear to everyone that something needed to be done. As usual, several anti-virus companies were independently and secretly working on a solution called a Generic Decryptor. The idea behind it was simple: if we find a way to decrypt an encrypted virus, we can yet again start looking for conventional signatures. But how to decrypt something which is unknown?

The emulator

The answer to this was the development of an emulator. An emulator is a routine that understands every instruction, and it is therefor able to keep track of what would happen if a piece of code would be really executed. If all instructions are applied to a memory buffer, the result will be a copy of the decrypted virus, just like it would appear when it has decrypted itself in a non-emulated environment.

You can compare it with designing the interior of a house by using a computer. If you have the building plan of the house, and the right software tools, you can see if the furniture fits into the house, without the risk of buying furniture that will not fit. If it fits in the emulated house, it will also fit into the real house. Emulation is a process that imitates a real situation, but without any risks.

Basically, a generic decryptor consists of four modules:

• A processor emulator
• A memory emulator
• A system emulator
• A decision mechanism

The processor emulator

The processor emulator is an imitation of one of the Intel processors. It has an Instruction Pointer, and feeding the emulator with clock cycles, it will read the instruction pointed to, update the virtual registers and flags, and it advances the instruction pointer to the next instruction. The better emulators even imitate Intel's pre-fetch queue, and yes, even the undocumented instructions and bugs of the Intel processors!

The memory emulator

The memory emulator imitates the memory of the PC. So, if a emulated piece of code reads or writes something to memory, the emulated memory is used instead of the real memory. Better emulators also maintain a map, so that it is possible afterwards to see which memory cells are executed, read, and modified.

The system emulator

The system emulator imitates the operating system and hardware of a PC. If a piece of code performs a system call to get the DOS version number, the system emulator will reply with a meaningful value. If the program being emulated asks for keyboard input, the system emulator will act as if a key is pressed. Better system emulators even include a virtual drive which can be read, formatted, or whatever the emulated code wants to do.

A decision mechanism

The decision mechanism is the most important part of the whole generic decryption unit. There is no point in emulating a program from the beginning until the bare end. This would take hours to complete, and the goal is to decrypt the code if it is encrypted, not to run an entire program! Therefore, the decision unit applies some rules to see if it is worth continuing emulation of a program. One of the rules could be to continue until a program writes something to the screen. Viruses usually do not write to the screen, at least not before the virus has decrypted itself. There are many rules, and sometimes these rules need to be fine-tuned if a new virus pops up. In addition, another important task of the decision mechanism is to return the result of the generic decryption process back to the scan engine. The result could be that there is nothing to be decrypted, or, if indeed something has been decrypted, it tells the scan engine where the now decrypted code can be found. Heuristics and signature scanning can now fully access the virus.

The advantages of an emulator are that we can now again use signatures to detect the virus, and very interesting, that we can apply heuristics to the decrypted virus! Usually, heuristics does not work on an encrypted virus, since it is impossible to see-through. The only thing visible from the outside is that the code looks like it is encrypted, but unfortunately, there are many legitimate programs around which are encrypted, but have nothing to do with viruses. So, an encryption shell alone is not sufficient to trigger the heuristics system of the scanner. But, now we are using emulators, we have gained full access to the viral code and a well designed heuristic scanner can detect up to 80% of the new viruses, which is a major achievement considering the fact that there are popping up new viruses every day, and that usually there has to be a victim before the virus finds its way to the anti-virus industry and a signature can be made.

Time to party!

For a long time, anti-virus producers have kept it a secret that they were using an emulator. One of the main reasons for this was the fear that the virus authors would find out and take counter measures. It has indeed been a long time before the virus authors started to understand how we suddenly succeeded in detecting a polymorphic virus just the same day it was released. It was a funny time: development of a polymorphic virus is something which costs many man-hours, and it took us just one minute to create a signature. I simply loved to waste some virus authors nifty creation he has been working on for weeks, in just a minute. We were on the winning side again, and the virus authors kept on writing viruses with the idea to make it difficult to create a signature to detect their decryptor. They were simply on the wrong track. They kept on assuming that we were trying to create signatures to detect the decryptor-part, or were writing labor intensive algorithmic approaches.

The battle starts again!

Now, five years later, things have changed. Many virus authors know that we are using emulators, and they are trying to take counter measures. All four modules of the generic decryptor can be attacked, and all of them have been attacked indeed.

Attacking the processor emulator

• using undocumented instructions
• making use of processor bugs
• making use of differences between various processors

The first attempts to attack the processor emulator were by coincidence. The first two Uruguay viruses were written on a XT machine equipped with a NEC V20 processor. The NEC V20 processor was at that time a faster alternative for the 8088 processor, and many hobbyists had their Intel processor replaced by this NEC alternative. So did the author of the Uruguay viruses. The NEC V20 processor is not completely equal to the 8088 processor, and the author of Uruguay was apparently not completely aware of this, so it happened that his first two creations only worked on a NEC V20 processor. As you might guess, these Uruguay viruses also fail inside a real 8088 processor. Fortunately, it is possible to create a signature for these viruses, so it did not bother much at that time, but it was a sign for some things that could go wrong in the future.

Things have gone wrong indeed! Virus authors have been trying to attack the processor emulator for a long time. The usual approach was to make use of an undocumented instruction, or to exploit a particular bug of the real processor. The answer for the anti-virus industry was easy: we just included these undocumented instructions and bugs as well, and that was it for the virus authors. The possibilities were soon exhausted.

However, at the end of last year, we have seen something new and extraordinary. A virus called "Cryptor" caused quite some confusion in the anti-virus industry. The whole confusion started when Virus Bulletin conducted their usual comparative review, and found out that some scanners did not detect all Cryptor samples. However, the anti-virus companies were convinced they did, generated thousands of samples which were all detected. So, Virus Bulletin forwarded some of the missing samples to the anti-virus producers, and some of them found out that the virus didn't work and shouldn't be detected at all, and some others found out that the virus was working properly. Funny enough, both groups were right.

The key to the correct answer depends on the processor on which the Cryptor samples were generated, and which processor was being emulated. All samples generated on a 80386 processor worked properly on machines with a 80386 processor. And all samples generated on a Pentium machine worked properly on a Pentium machine. However, crossing the samples between these two processors caused many samples to crash. The same applied to the scanners, if they were emulating a Pentium processor, many samples generated on a 80386 machine crashed inside the emulator, and vice versa. Which samples were detected simply relied on the method used to detect the virus, and if one was using an emulator, which processor was imitated.

There is still a debate going on whether this incompatibility in the virus is a coincidence or intentional. Personally, I think that it has been designed this way intentionally. Anyway, it was a difficult problem to solve. The virus made use of a processor flag that was documented by Intel as "undefined" after a multiply instruction. However, the term "undefined" doesn't mean "random". It simply means that the result is unpredictable and may not be used for any purpose. The virus however was using this flag in its decryptor, so it had to be emulated as well. But how to emulate something which is unpredictable? Research showed that the flag was dependant on the two multipliers, and that it was the result of some internal optimization process of the multiply instruction inside the processor. And these optimization processes were different on different processor families, and so the outcome of the undefined flag.

Multiple passes?

To detect Cryptor, one can of course emulate every program multiple times, each run with a different processor. But emulation is extremely time consuming. To give you an example, the SMEG virus has a decryptor which can have a size up to 2000 instructions. In every loop through this decryptor, just one byte is being decrypted. So, if one wants to decrypt the whole virus, about 2000 bytes, one has to emulate 2000*2000 instructions. This means that an instruction emulator has to be called 4 million times, and that the decision unit has to be called 4 million times. Going through this whole process for every processor once is too time consuming. Reliably results can be obtained if the emulator itself is able to detect if the outcome is processor-dependant. That's how we did it.

Attacking the memory emulator

We have found some attempts to attack the memory emulator, by simply exhausting it. Of course, the memory which can be controlled by the memory emulator is far less than the amount of memory a "real PC" has. However, since many scanners are using Extended memory or are running in

32bit mode, these kinds of attacks lost their meaning.

Attacking the system emulator

FinishPoly was one of the first viruses that were trying to confuse the emulator by using many system calls. Most emulators do not support the whole set of system calls available on a real PC. Usually, if a program asks for the Ctrl-Break setting, or asks for the number of drives, this means that at least some non-encrypted program is running, and this usually means the end of the emulation process. So there is no need to support the whole set of system calls. However, FinishPoly is not using the answers from the system at all, it just randomly adds these system calls to confuse the generic decryptor.

Another attack I have recently seen is performed by a virus that asks twice for the time. Inside the emulator, you usually respond with a fixed value to any request. The virus however compares the milliseconds to see if there is any difference. In the real PC, there would definitely be a difference, and if the value is the same, the virus concludes that it is running inside an emulator and it simply quits without decrypting itself. The solution to this was to change the system emulator, to make it returning a different value for each time the system time is requested.

Attacking the decision unit

The decision unit was one of the first targets of the virus authors. In September 1994, a new virus popped up called One-Half. This virus didn't have a decryptor at the beginning, but the decryptor was sprinkled all over the file. Our decision unit, like many others, had a rule that said that if there was a jump to a complete other part of the file that IF there was something encrypted at all, it should have been decrypted by now. It assumed that the virus jumped to the decrypted part, which wasn't true. It was time to revise some rules. After this, we have seen some more attempts, like multiple encryption passes. It seems that some of the virus authors, but a very limited minority, have quite a good understanding how a generic decryptor works and what kind of rules we have for the decision unit.

Who's going to be the winner?

Depite all attacks, so far it is still possible to decrypt every virus. The only problem is that scanning becomes more and more time consuming, because multiple emulation passes are sometimes necessary, and some early-out mechanisms had to be abandoned. On the long term, it may even be necessary to emulate a program until the bare end, when it exits to the system. Viruses get more complex, and so do the scanners. There are no winners here, but only losers, but these are not the people playing the game. The losses are paid by the users, who are confronted with more complex viruses, slower scanners, and expensive protection schemes.

Conclusion

So far, generic decryptors equipped with emulators are still on the winning hand. Advantages are a fast response time on new viruses, reduction of development costs because fewer human resources needed and the ability to apply heuristics. Disadvantages are slower scan speeds, and like all anti-virus measures, it stimulates the virus writers to produce viruses that are even more complex. However, the advantages are much stronger than the disadvantages so there is a big future for Emulating Generic Decryptors. Also, a new challenge is raised by the newer 32-bit protected mode viruses. This makes it a requirement to develop a protected mode emulator. Maybe I will tell you more about this issue next year.

Frans Veldman,

Technical Director
Norman Data Defense Systems Development B.V.