Dean Carlson and Beth Anne Byrd
CpSc 420
What is reverse engineering?
Brief History
Usefulness
The process
Bagle Virus example
“[T]he process of analyzing a subject system
to create representations of the system at a
higher level of abstraction” (Chikofsky, 1990).
Going through the software development
cycle backwards
Started as analyzing hardware in an attempt
to gain an advantage.
The first time this was applied to a piece of
malware was in 1987.
Bernt Fix disassembled and neutralized the
Charlie virus.
Analysis of a product
Recreating lost or nonexistent
documentation
Academic use
Curiosity
With Malware
Contain it
Remove it
Prevent it
Diffuse “time bombs”
Conficker virus
▪ The Y2K of today
Set up a controlled, isolated laboratory
Perform behavioral analysis to examine the
specimen’s interactions with its environment.
3. Perform static code analysis to further
understand the specimen’s inner-workings.
4. Perform dynamic code analysis to understand
the more difficult aspects of the code.
5. If necessary, unpack the specimen.
6. Repeat steps 2, 3, and 4 (order may vary) until
sufficient analysis objectives are met.
7. Document findings and clean-up the laboratory
for future analysis.
1.
2.
HOST:
Windows XP in Virtual Machine
DataRescue IDA Pro
Microsoft Visual C++
▪ Dumpbin
UltraEdit
SERVER
Solaris 9 (SPARC)
Snoop
BIND (DNS)
GCC
GDB
The Email and DNS programs on the server
were setup to log all of their activity and
network traffic in order to see the virus
interact with the server.
Open in IDA Pro
Breaks it down into assembly and hex
Open in dumpbin to determine type
PE (Portable Executable)
Walk through the virus step by step with a
debugger and look at register values.
Especially EAX, EIP, ZF bit of EFLAGS
EAX = return values from functions
ZF = flag used for comparisons and decisions
EIP = useful for thread usage
Use IDA to chart subroutines
Use IDA to identify function parameters and
variables
arg_8 can be accessed by adding “10h” to the EBP
Register
Multiple Thread
Extended Instruction Pointer (EIP) doesn’t follow
new threads unless specified
The Bagle virus was not packed
Compressed or encrypted
It also was not polymorphic
Changing the assembly, usually by inserting
“noop” thus changing the virus signature but not
changing the effectiveness
The Bagle virus has many removal tools
Reverse engineering malware started in 1987
It is good to contain, remove, and prevent
malware
7 steps
1. Set up lab
2. Behavioral analysis
3. Static code analysis
4. Dynamic code analysis
5. Unpack
6. Repeat steps 2, 3, and 4
7. Document and clean-up
0
