Malgram Malware Analysis:
Malware Unpacking
Static Analysis
Code Deobfuscation
Decompilation
Phillip Porras and Hassen Saidi
Computer Science Lab
SRI International

• Now that we have various ways of knowing what the malware does when running on an infected system, we aim at answering two fundamental questions:
1. How does it do it?
1. What are the full capability of the malware: both observed behavior and yet to be triggered behavior

Dynamic vs Static Malware Analysis
• Dynamic Analysis
– Techniques that profile actions of binary at runtime
– Only provides partial `` effects-oriented profile ’’ of malware potential
• Static Analysis
– Techniques that apply program analysis to the binary code
– Can provide complementary insights
– Potential for more comprehensive assessment

• …go interactive
Malgram Report

Raw Binary
From Binary To Semantically Rich C Code
Disassembly

From Binary To Semantically Rich C Code
Complete Disassembly

From Binary To Semantically Rich C Code
Decompiled C code

Raw Binary
Challenges in Static Analysis
Complete Disassembly
Disassembly
Decompiled C code

Malware Obfuscation
• Most malware is obfuscated
• Packing is the most used obfuscation technique
• Packing is often combined with other advanced forms of obfuscation:
• Binary Rewrite to create semantically equivalent code with vastly different structure
• Call obfuscation in general and API obfuscation in particular
• Chuncking or “code spaghettisation”
• Dead code (or functionally irrelevant code)
Page 9

Raw Binary
Challenges in Static Analysis
Challenge : Does the binary represents the full malware binary logic.
Disassembly

Page 11
Unpacking Result
Unpacking

Packed vs Unpacked
• go interactive…

Coarse-grained Execution Monitoring
• Generalized unpacking principle
– Execute binary till it has sufficiently revealed itself
– Dump the process execution image for static analysis
• Monitoring execution progress
– Eureka employs a Windows driver that hooks to
SSDT (System Service Dispatch Table)
– Callback invoked on each NTDLL system call
– Filtering based on malware process pid

Statistics-based Unpacking
• Observations
– Statistical properties of packed executable differ from unpacked executable
– As malware executes code-to-data ratio increases
• Complications
– Code and data sections are interleaved in PE executables
– Data directories (import tables) look similar to data but are often found in code sections
– Properties of data sections vary with packers

Statistics-based Unpacking (3)
Bigram Calc
117 KB
FF 15 call
246
Explorer
1010 KB
3045
Ipconfig
59 KB
184 lpr
11
KB
Mshearts
131 KB
24 192
Notepad
72 KB
Ping
21 KB
Shutdown
23 KB
Taskman
19 KB
415 58 132 126
2494 272 33 274 254 41 63 85 FF 75 push
235
E8 _ _
_ 0xff call
E8 _ _
_ 0x00 call
1583
746
2201
1091
181
152
19
62
369
641
180
108
87
57
49
66
41
50

Evaluation (ASPack)

Evaluation (MoleBox)

API Resolution
• User-level malware programs require system calls to perform malicious actions
• Use Win32 API to access user level libraries
• Obfuscations impede malware analysis using disassemblers and decompilers
– Packers use non-standard linking and loading of dlls
– Obfuscated API resolution

Imports in IAT identified by IDA by looking at Import Table

• Identify register based indirect calls
GetEnvironmentStringW use def

Evaluation Metrics
• Measuring analyzability
– Code-to-data ratio
• Use disassembler to separate code and data.
• Most successfully unpacked malware have code-to-data ratio over 50%
– API resolution success
• Percentage of API calls that have been resolved from the set of all call sites.
• Higher percentage implies more the malware is amenable to static anlaysis.

Challenges in Static Analysis
Complete Disassembly
Challenge : Can we isolate subroutines?
Disassembly

Binary Rewrites
• go interactive …

Raw Binary
From Raw Binary To Decompiled C Code
Complete Disassembly
Disassembly
Decompiled C code

Renaissance: Improving C Code Readability void *sub_9AB966(int a1, void *source, unsigned int a3)
{ int v3, v4, v5, v6, v8;
} v3 = a1; v4 = *(_DWORD *)(a1 + 20) + 8 * a3; v5 = (*(_DWORD *)(a1 + 20) >> 3) & 0x3F;
*(_DWORD *)(a1 + 20) = v4; if ( v4 < 8 * a3 )
++*(_DWORD *)(a1 + 24);
*(_DWORD *)(a1 + 24) += a3 >> 29; if ( v5 + a3 <= 0x3F )
{ v6 = 0;
} else
{ v6 = 64 - v5; memcpy((void *)(v5 + a1 + 28), source, 64 - v5); sub_9A9F13(a1, (void *)(a1 + 28)); if ( v6 + 63 < a3 )
{ v8 = v6 + 63; do
{ sub_9A9F13(v3, (char *)source + v8 - 63); v8 += 64; v6 += 64;
} while ( v8 < a3 );
} v5 = 0;
} return memcpy((void *)(v5 + v3 + 28),(char *)source + v6, a3 - v6);
Hex Rays void *sub_9AB966(unsigned int *destination1, unsigned int *source, size_t num1)
{ unsigned int *destination2; size_t num3, num2, num4, num5;
} destination2 = destination1; num3 = destination1[20] + 8 * num1; num2 = (destination1[20] >> 3) & 0x3F; destination1[20] = num3; if ( num3 < 8 * num1 )
++destination1[24]; destination1[24] += num1 >> 29; if ( num2 + num1 <= 0x3F )
{ num4 = 0;
} else
{ num4 = 64 - num2; memcpy( &destination1[num2 + 28], source,
64 - num2); sub_9A9F13( destination1, &destination1[28] ); if ( num4 + 63 < num1 )
{ num5 = num4 + 63; do
{ sub_9A9F13( destination2, &source[num5 - 63] ); num5 += 64; num4 += 64;
} while ( num5 < num1 );
} num2 = 0;
} return memcpy( &destination2[num2 + 28], &source[num4], num1 - num4 );
Hex Rays + Renaissance

void *sub_9AB966( int a1, void * source, unsigned int a3)
{ int v3, v4, v5, v6, v8;
} v3 = a1; v4 = *(_DWORD *)(a1 + 20) + 8 * a3; v5 = (*(_DWORD *)(a1 + 20) >> 3) & 0x3F;
*(_DWORD *)(a1 + 20) = v4; if ( v4 < 8 * a3 )
++*(_DWORD *)(a1 + 24);
*(_DWORD *)(a1 + 24) += a3 >> 29; if ( v5 + a3 <= 0x3F )
{ v6 = 0;
} else
{ v6 = 64 - v5; memcpy((void *)(v5 + a1 + 28), source, 64 - v5); sub_9A9F13(a1, (void *)(a1 + 28)); if ( v6 + 63 < a3 )
{ v8 = v6 + 63; do
{ sub_9A9F13(v3, (char *)source + v8 - 63); v8 += 64; v6 += 64;
} while ( v8 < a3 );
} v5 = 0;
} return memcpy((void *)(v5 + v3 + 28),(char *)source + v6, a3 - v6);
1. Typing and naming variables
Hex Rays void *sub_9AB966( unsigned int * destination1, unsigned int * source, size_t num1)
{ unsigned int * destination2; size_t num3, num2, num4, num5;
} destination2 = destination1; num3 = destination1[20] + 8 * num1; num2 = (destination1[20] >> 3) & 0x3F; destination1[20] = num3; if ( num3 < 8 * num1 )
++destination1[24]; destination1[24] += num1 >> 29; if ( num2 + num1 <= 0x3F )
{ num4 = 0;
} else
{ num4 = 64 - num2; memcpy( &destination1[num2 + 28], source,
64 - num2); sub_9A9F13( destination1, &destination1[28] ); if ( num4 + 63 < num1 )
{ num5 = num4 + 63; do
{ sub_9A9F13( destination2, &source[num5 - 63] ); num5 += 64; num4 += 64;
} while ( num5 < num1 );
} num2 = 0;
} return memcpy( &destination2[num2 + 28], &source[num4], num1 - num4 );
Hex Rays + Renaissance

void *sub_9AB966(int a1 , void * source , unsigned int a3 )
{ int v3, v4, v5, v6, v8;
} v3 = a1; v4 = *(_DWORD *)(a1 + 20) + 8 * a3; v5 = (*(_DWORD *)(a1 + 20) >> 3) & 0x3F;
*(_DWORD *)(a1 + 20) = v4; if ( v4 < 8 * a3 )
++*(_DWORD *)(a1 + 24);
*(_DWORD *)(a1 + 24) += a3 >> 29; if ( v5 + a3 <= 0x3F )
{ v6 = 0;
} else
{ v6 = 64 - v5; memcpy((void *)(v5 + a1 + 28), source, 64 - v5); sub_9A9F13(a1, (void *)(a1 + 28)); if ( v6 + 63 < a3 )
{ v8 = v6 + 63; do
{ sub_9A9F13(v3, (char *)source + v8 - 63); v8 += 64; v6 += 64;
} while ( v8 < a3 );
} v5 = 0;
} return memcpy((void *)(v5 + v3 + 28),(char *)source + v6, a3 - v6);
2. Highlighting important vars
Hex Rays void *sub_9AB966(unsigned int * destination1 , unsigned int * source , size_t num1 )
{ unsigned int *destination2; size_t num3, num2, num4, num5;
} destination2 = destination1; num3 = destination1[20] + 8 * num1; num2 = (destination1[20] >> 3) & 0x3F; destination1[20] = num3; if ( num3 < 8 * num1 )
++destination1[24]; destination1[24] += num1 >> 29; if ( num2 + num1 <= 0x3F )
{ num4 = 0;
} else
{ num4 = 64 - num2; memcpy( &destination1[num2 + 28], source,
64 - num2); sub_9A9F13( destination1, &destination1[28] ); if ( num4 + 63 < num1 )
{ num5 = num4 + 63; do
{ sub_9A9F13( destination2, &source[num5 - 63] ); num5 += 64; num4 += 64;
} while ( num5 < num1 );
} num2 = 0;
} return memcpy( &destination2[num2 + 28], &source[num4], num1 - num4 );
Hex Rays + Renaissance

3. Improvements to decompilation void *sub_9AB966(int a1 , void *source, unsigned int a3)
{ int v3 , v4, v5, v6, v8;
} v3 = a1; v4 = *(_DWORD *)(a1 + 20) + 8 * a3; v5 = (*(_DWORD *)(a1 + 20) >> 3) & 0x3F;
*(_DWORD *)(a1 + 20) = v4; if ( v4 < 8 * a3 )
++*(_DWORD *)(a1 + 24);
*(_DWORD *)(a1 + 24) += a3 >> 29; if ( v5 + a3 <= 0x3F )
{ v6 = 0;
} else
{ v6 = 64 - v5; memcpy((void *)(v5 + a1 + 28), source, 64 - v5); sub_9A9F13(a1, (void *)(a1 + 28)); if ( v6 + 63 < a3 )
{ v8 = v6 + 63; do
{ sub_9A9F13(v3, (char *)source + v8 - 63); v8 += 64; v6 += 64;
} while ( v8 < a3 );
} v5 = 0;
} return memcpy((void *)(v5 + v3 + 28),(char *)source + v6, a3 - v6);
Hex Rays void *sub_9AB966(unsigned int * destination1 , unsigned int *source, size_t num1)
{ unsigned int * destination2 ; size_t num3, num2, num4, num5;
} destination2 = destination1; num3 = destination1[20] + 8 * num1; num2 = (destination1[20] >> 3) & 0x3F; destination1[20] = num3; if ( num3 < 8 * num1 )
++destination1[24]; destination1[24] += num1 >> 29; if ( num2 + num1 <= 0x3F )
{ num4 = 0;
} else
{ num4 = 64 - num2; memcpy( &destination1[num2 + 28], source,
64 - num2); sub_9A9F13( destination1, &destination1[28] ); if ( num4 + 63 < num1 )
{ num5 = num4 + 63; do
{ sub_9A9F13( destination2, &source[num5 - 63] ); num5 += 64; num4 += 64;
} while ( num5 < num1 );
} num2 = 0;
} return memcpy( &destination2[num2 + 28], &source[num4], num1 - num4 );
Hex Rays + Renaissance

void *sub_9AB966(int a1, void *source, unsigned int a3)
{ int v3, v4, v5, v6, v8;
} v3 = a1; v4 = *(_DWORD *)(a1 + 20) + 8 * a3; v5 = (*(_DWORD *)(a1 + 20) >> 3) & 0x3F;
*(_DWORD *)(a1 + 20) = v4; if ( v4 < 8 * a3 )
++*(_DWORD *)(a1 + 24);
*(_DWORD *)(a1 + 24) += a3 >> 29; if ( v5 + a3 <= 0x3F )
{ v6 = 0;
} else
{ v6 = 64 - v5; memcpy((void *)(v5 + a1 + 28), source, 64 - v5); sub_9A9F13(a1, (void *)(a1 + 28)); if ( v6 + 63 < a3 )
{ v8 = v6 + 63; do
{ sub_9A9F13(v3, (char *)source + v8 - 63); v8 += 64; v6 += 64;
} while ( v8 < a3 );
} v5 = 0;
} return memcpy((void *)(v5 + v3 + 28),(char *)source + v6, a3 - v6);
4. Caller → Callee type info
Hex Rays void *sub_9AB966(unsigned int *destination1, unsigned int *source, size_t num1)
{ unsigned int *destination2; size_t num3, num2, num4, num5;
} destination2 = destination1; num3 = destination1[20] + 8 * num1; num2 = (destination1[20] >> 3) & 0x3F; destination1[20] = num3; if ( num3 < 8 * num1 )
++destination1[24]; destination1[24] += num1 >> 29; if ( num2 + num1 <= 0x3F )
{ num4 = 0;
} else
{ num4 = 64 - num2; memcpy( &destination1[num2 + 28], source,
64 - num2); sub_9A9F13( destination1, &destination1[28] ); if ( num4 + 63 < num1 )
{ num5 = num4 + 63; do
{ sub_9A9F13( destination2, &source[num5 - 63] ); num5 += 64; num4 += 64;
} while ( num5 < num1 );
} num2 = 0;
} return memcpy( &destination2[num2 + 28], &source[num4], num1 - num4 );
Hex Rays + Renaissance

Evaluation
Adialer
Adpclient
Adultbrowser
Agent.DZ (packed)
Browsermodifier
Casino_c12a
Conficker-A
Conficker-B
Cycbot
Duqu
Lexotan32-A
Lexotan32-B
Lolyda.AA
Magiccasino
Mydoom_aa32
Podnhua_f0a6
Qakbot-A
Stuxnet
Torpig
Notepad
Quake
Total
IDA Pro
153/606 (25%)
12/300 (4%)
296/762 (39%)
1/61 (2%)
161/469 (34%)
794/7207 (11%)
243/781 (31%)
296/1516 (20%)
267/2842 (9%)
66/300 (22%)
2/40 (5%)
2/50 (4%)
10/134 (7%)
42/1064 (4%)
153/543 (28%)
99/372 (27%)
179 (29%)
64/320 (20%)
629 (44%)
145/273 (53%)
260/4054 (6%)
3697/23721 (16%)
Renaissance
276/606 (46%)
93/300 (31%)
339/762 (44%)
14/61 (23%)
252/469 (54%)
2614/7207 (36%)
318/781 (41%)
735/1516 (49%)
881/2842 (31%)
117/300 (37%)
9/40 (23%)
8/50 (16%)
40/134 (30%)
351/1064 (33%)
189/543 (35%)
139/372 (37%)
183 (30%)
134/320 (42%)
717 (51%)
108/273 (40%)
1271/4054 (31%)
9503/23721 (40%)

Raw Binary
Challenges in Static Analysis
Complete Disassembly
Disassembly
Decompiled C code

The Need for Rapid Crypto-Algorithm Isolation
RC4
Rustock
Zeus
Conficker
AES
Truecrypt
Waledac
SSL
Agobot (IRC over SSL)
Custom Crypto / Encoding
Pushdo
Kraken mebroot
Mega-D
Serpent
Truecrypt
Twofish
Truecrypt
XOR-Custom
Lethic
Virut
Hydraq
Torpig
Cascades
Truecrypt
HASH Whirlpool
Truecrypt
RSA variants
Nugashe
Conficker
Waledac
HASH MD6 conficker BC
Blowfish - 448 bit
Clampi
HASH SHA1 conficker A
Truecrypt

IntraModule isCrypto()
Intra-module Analyzer
Constant
Detector
Constant
Data
Loading
Padding
Analysis
Microsoft
CryptoAPI
CAPICON
Unknown
Computation cryptoFnDetection () isCrypto Score = isConst + isPadded + Crypt API fn (LargeVar, Loop Detection, Opcodes, BigMath)
Large
Local
Variables cryptoFnDetection () – At least 2 matches
Loop
Detection
Big Number
Math
Opcode
Analysis

SHA1
SHA256
SHA512
SHARK
SKIPJACK
Square
Tiger
Twofish
WAKE
Whirlpool zlib
AES
MD6
Blowfish
Camelia
CAST
CAST256
CRC32
DES
GOST
HAVAL
MARS
MD2
PKCS_MD2
PKCS_MD5
PKCS_RIPEMD160
PKCS_SHA256
PKCS_SHA384
PKCS_SHA512
PKCS_Tiger
RawDES
RC2
Rijndael
SAFER
Constant detection
Direct Reference

Rijndael
SAFER
SHA1
SHA256
SHA512
SHARK
SKIPJACK
Square
Tiger
Twofish
WAKE
Whirlpool zlib
AES
MD6
Blowfish
Camelia
CAST
CAST256
CRC32
DES
GOST
HAVAL
MARS
MD2
PKCS_MD2
PKCS_MD5
PKCS_RIPEMD160
PKCS_SHA256
PKCS_SHA384
PKCS_SHA512
PKCS_Tiger
RawDES
RC2
Data array contains
Known crypto content
Load
Array
Indirect Load
Unknown
Computation
This could be Encryption
Or Decryption

Inter-module Analyzer
}
{ func ColorNode (Subgraph) if (exists uncolored subgraph)
ColorNode (subgraph) foreach leaf in subgraph { isCrypto(Leaf) }
If (exist green leaf) then color root green if (exist orange leaf) then color root orange if (exist > 2 red leaves) then color root red func cryptoString (per subroutine) if node contains known crypto implementation substring, label node with corresponding crypto library.
MD6
Vowpal wabbit
AES

IDA Pro Call Graph w/ Crypto-routine detection

Example
Running SRI Crypt Finder
(c) SRI International
Finding crypto constants and subroutines in binary files automatic discovery of crypto functions as unknown computations
4BABF1: found sparse constants for SHA-1
50C254: found const array sbox_AES (used in AES)
50E354: found const array rsbox_AES (used in AES)
50F574: found const array Twofish_q (used in Twofish)
50F7A4: found const array MARS_Sbox (used in MARS)
510EA4: found const array zinflate_lengthExtraBits (used in zlib)
510F18: found const array zinflate_distanceExtraBits (used in zlib)
511918: found const array CRC32_m_tab (used in CRC32)
514F98: found const array CRC32_m_tab (used in CRC32)
Found 9 known constant arrays in total.
Scanning code for crypto subroutines found crypto in Function @ 407334 found crypto in Function @ 40E5B4 found crypto in Function @ 47D954 found crypto in Function @ 47ED34 found crypto in Function @ 4816F4 found crypto in Function @ 4B6624 found crypto in Function @ 4B9980 found crypto in Function @ 4CCBD4 found crypto in Function @ 4CCD4C found crypto in Function @ 4CE208 found crypto in Function @ 4CE7CC found crypto in Function @ 4CEBE8 found crypto in Function @ 4D9B00 found crypto in Function @ 4D9EE4
Done labelling crypto subroutines
Found 14 subroutine(s) with possible crypto

Running SRI Crypt Finder

Report Generation
• go interactive