SMT Solvers for Software Security
George Nosenko,
Security researcher at Digital Security
SMT Solvers for Software Security
#whoami
• Member of DSecRG.
• System Developer
• Reverse Engineer
• Security Researcher
© 2002—2013, Digital Security
SMT Solvers in very simple terms
What is a SMT Solver?
Just like the first time using a SMT
constraint solver
© 2002—2013, Digital Security
SMT Solvers in very simple terms
What is a SMT Solver?
 Solver is a program
 You ask a question
“What is the meaning of life?”
 Solver tries to answer
“42”
© 2002—2013, Digital Security
SMT Solvers in very simple terms
How can I ask a question?
 Question is a logical formula
b+2 = c, f(read(write(a,b,3), c-2)) ≠ f(c-b+1)
 SMT-LIB: Language for expressing formulas
http://smtlib.org/
 All solvers understand this language
© 2002—2013, Digital Security
SMT Solvers in very simple terms
What solver should I choose?
There are many SMT-solvers (over 20)
CVC3, CVC4, STP, Alt-Ergo, Yices, Z3, etc
Z3 is my choice
 Efficient SMT solver
 Open Source Project: http://z3.codeplex.com
 Python, C/C++, .NET binding
 Available online
 Support Windows & Linux
© 2002—2013, Digital Security
SMT Solvers in very simple terms
SAT or not SAT? Ask a question.
 structure of formula
 declaration
 precondition
 postcondition
 answer
 sat, unsat, unknown
 satisfiability
 validity
 model
Taint Nobody Got Time for Crash
© 2002—2013, Digital Security
(declare-const work Int)
(declare-const sleep Int)
(declare-const fun Int)
(assert (>= work 40))
(assert (>= sleep 42))
(assert (>= fun work))
(assert (= (+ work (+ sleep fun)) 168))
(check-sat)
(get-model)
sat (model (sleep: 42, fun: 63, work 63)
http://rise4fun.com/Z3/pLpMc
SMT Solvers in very simple terms
Properties of SMT solvers
 Mathematical precision
 Expressive power
 Data model
 Efficient implementation
Support Bit-vector & Array
© 2002—2013, Digital Security
SMT Solvers for Software Security
SMT Solvers for Software Security
© 2002—2013, Digital Security
SMT Solvers for Software Security
Applications
 Bug Hunting
 Fuzzing (whitebox or blackbox)
 Program Verification & Analysis
 Exploit Generation
 PoC, AEG, APEG
 Automate generate payload
 Protection Analysis
 Obfuscation
 Crypto Analysis
 Malware Analysis
© 2002—2013, Digital Security
SMT Solvers for Software Security
What’s the point?
Idea: convert portions of code into logical formulas,
and use SMT solver to prove properties about them
add eax, ebx
xor ebx, ebx
sub ecx, 0x123
setz bl
add eax, ebx
Is this snippet equivalent to “add eax, ebx”?
sub
bl, bl
What value must EAX have at the beginning
movzx ebx, bl
add
ebx, 0xbbbbbbbb of this snippet in order for EAX to be
0x12345678 after the snippet executes?
add
eax, ebx
http://recon.cx/2012/schedule/attachments/52_semantics-based-methods.pdf
Taint Nobody Got Time for Crash
© 2002—2013, Digital Security
SMT Solvers for Software Security
BV Operations in SMT-LIB 2.0
=/bvcomp
distinct
ite
bvand
bvor
bvxor
bvnot
bvnand
bvnor
bvxnor
bvneg
bvadd
bvmul
bvudiv
bvurem
bvsub
bvsdiv
bvsrem
bvsmod
bvshl
bvlshr
bvashr
bvult
bvule
bvugt
bvuge
bvslt
bvsle
bvsgt
bvsge
concat
extract
bvshl
bvlshr
bvashr
repeat
zero_extend
sign_extend
rotate_left
rotate_right
Slides - SMT Workshop 2013
© 2002—2013, Digital Security
SMT Solvers for Software Security
Array Operations in SMT-LIB 2.0: select-store axioms
1. Expression (select a i) returns the value stored at position i of the array a;
2. And (store a i v) returns a new array identical to a, but on position i it
contains the value v.
(declare-const x Int)
(declare-const y Int)
(declare-const a1 (Array Int Int))
(assert (= (select a1 x) x))
(assert (= (store a1 x y) a1))
(check-sat)
© 2002—2013, Digital Security
SMT Solvers for Software Security
Binary Analysis Platform: http://bap.ece.cmu.edu/
BIL code for add %rax, %rbx
addr 0x0 @asm "add %rax,%rbx"
label pc_0x0
T_t1:u64 = R_RBX:u64
T_t2:u64 = R_RAX:u64
R_RBX:u64 = R_RBX:u64 + T_t2:u64
R_CF:bool = R_RBX:u64 < T_t1:u64
R_OF:bool = high:bool((T_t1:u64 ^ ~T_t2:u64) & (T_t1:u64 ^ R_RBX:u64))
R_AF:bool = 0x10:u64 == (0x10:u64 & (R_RBX:u64 ^ T_t1:u64^T_t2:u64))
R_PF:bool =
~low:bool(let T_acc:u64 := R_RBX:u64 >> 4:u64 ^ R_RBX:u64 in
let T_acc:u64 := T_acc:u64 >> 2:u64 ^ T_acc:u64 in
T_acc:u64 >> 1:u64 ^ T_acc:u64)
R_SF:bool = high:bool(R_RBX:u64) R_ZF:bool = 0:u64 == R_RBX:u64
© 2002—2013, Digital Security
SMT Solvers for Software Security
Bug Hunting
© 2002—2013, Digital Security
Bug Hunting
Vulnerability related with Integer
 CWE-190,191,192,194,196
 May cause:
 Bypass sanity check
 Buffer Overflow
 Dangling Pointer
 Use after free
 Application specific
© 2002—2013, Digital Security
Bug Hunting
Integer Overflow in Linux Kernel. CVE-2013-2596
© 2002—2013, Digital Security
Bug Hunting
Integer Overflow in Linux Kernel. CVE-2013-2596
static int fb_mmap(struct file *file, struct vm_area_struct * vma){
if (!info) return -ENODEV;
...
off = vma->vm_pgoff << PAGE_SHIFT;
fb = info->fbops;
if (!fb)
return -ENODEV;
...
/* frame buffer memory */
start = info->fix.smem_start;
len = PAGE_ALIGN((start & ~PAGE_MASK) + info->fix.smem_len);
if (off >= len) {
/* memory mapped io */
off -= len;
...
start = info->fix.mmio_start;
len = PAGE_ALIGN((start & ~PAGE_MASK) + info>fix.mmio_len);
}
mutex_unlock(&info->mm_lock);
start &= PAGE_MASK;
if ((vma->vm_end - vma->vm_start + off) > len)
return -EINVAL;
...
fb_pgprotect(file, vma, off);
if (io_remap_pfn_range(vma, vma->vm_start, off >> PAGE_SHIFT,
vma->vm_end - vma->vm_start, vma>vm_page_prot))
return -EAGAIN;
return 0;
}
© 2002—2013, Digital Security
Bug Hunting
How does Motochopper work?
1728 open("/dev/graphics/fb0", O_RDWR) = 6
...
1728 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0) = 0x400f2000
...
1728 munmap(0x4015b000, 9433088)
= 0
1728 mmap2(NULL, 9437184, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0) = 0x4015b000
1728 munmap(0x4015b000, 9437184)
= 0
1728 mmap2(NULL, 9441280, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0) = -1 EINVAL (Invalid
argument)
1728 mmap2(NULL,
1728 mmap2(NULL,
(Out of memory)
1728 mmap2(NULL,
(Out of memory)
...
(Out of memory)
1728 mmap2(NULL,
0x4015b000
NAME
2415919104, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0x70900) = -1 ENOMEM
2231369728, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0x7b900) = -1 ENOMEM
2214592512, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0x7c900) = -1 ENOMEM
2113929216, PROT_READ|PROT_WRITE, MAP_SHARED, 6, 0x82900) =
mmap2 - map files or devices into memory
#include <sys/mman.h>
void *mmap2(void *addr, size_t length, int prot,
int flags, int fd, off_t pgoffset);
© 2002—2013, Digital Security
Bug Hunting
Integer Overflow in Linux Kernel. CVE-2013-2596
© 2002—2013, Digital Security
Bug Hunting
Integer Overflow in OpenSSH. CVE-2002-0639
© 2002—2013, Digital Security
Bug Hunting
Integer Overflow in OpenSSH. CVE-2002-0639
input_userauth_info_response(){
...
u_int nresp;
...
nresp = packet_get_int();
if (nresp > 0) {
response = xmalloc(nresp * sizeof(char*));
for (i = 0; i < nresp; i++)
response[i] = packet_get_string(NULL);
}
packet_check_eom();
}
© 2002—2013, Digital Security
Bug Hunting
Integer Overflow in OpenSSH. CVE-2002-0639
(declare-const
sizeof (_ BitVec 32))
(declare-const
nresp
(_ BitVec 32))
(declare-const
mult
(_ BitVec 32))
(assert ( =
4
sizeof
(assert ( = mult
nresp*sizeof
(_ bv4 32)))
(bvmul nresp sizeof)))
(assert ( bvugt
nresp
(assert ( bvult
nresp
mult nresp))
(get-model)
© 2002—2013, Digital Security
;
(_ bv0 32) )) ; nresp > 0
(assert ( = mult (_ bv256 32)))
nresp*sizeof = 256
(check-sat)
; sizeof (char*) =
; nresp*sizeof <
;
Bug Hunting
Verification & Static analyze with SMT
 Single collaborative framework
 It’s not heuristic bug-finding
 It allows user to manipulate
 Functional specification
 Prove that source code satisfies
specification
 Expands with plug-ins
 ACSL is a behavioral
specification language
© 2002—2013, Digital Security
Bug Hunting
Jessie: verification tools for C programs
 Jessie is a plug-in for the Frama-C
 Functional Checking
 Safety Checking
 Memory Safety
 Integer Overflow
 Checking Termination
© 2002—2013, Digital Security
Bug Hunting
Jessie: Integer Overflow Safety
#pragma JessieTerminationPolicy(user)
//@ requires n >= 0 && \valid_range(t,0,n−1);
int binary_search(long t[], int n, long v) {
int l = 0, u = n-1;
//@ loop invariant 0 <= l && u <= n−1;
while (l <= u) {
int m = l + (u - l) / 2; //int m = (l + u) / 2;
if (t[m] < v)
l = m + 1;
else if (t[m] > v)
u = m - 1;
else return m;
}
return -1;
}
> frama-c -jessie binary-search.c
© 2002—2013, Digital Security
Bug Hunting
Immunity Debugger & SMT: Infrastructure
SequenceAnalyzer – Models x86 as operations over a set of SMT primitives.
Solver – Ctypes interface to the CVC3 SMT solver API. Supports a variety of
theories including quantifier free, bit-vector arithmetic, linear arithmetic etc.
CodeGraph/PathGenerator – Purely static CFG building and path generation.
PathWalker – SMT based path traversal. Each conditional jump is checked for
feasibility and the path discarded if not SAT.
BugChecker – Subclasses provide the check_ins method which will be passed the
SMT context representing the current path.
© 2002—2013, Digital Security
Bug Hunting
Immunity Debugger & SMT: !find_int_overwlow.py
© 2002—2013, Digital Security
SMT in protection analysis
PROTECTION ANALYSIS
© 2002—2013, Digital Security
SMT in protection analysis
Using SMT to defeat simple hashing algorithms
def round_hash(a, b, c, d):
out = [ ]
for i, n in enumerate((a, b, c, d)):
nn = 0
for j in range(32):
nn |= (rotl(n, SCRAMBLE_TABLE[(i << 2)+j]) & 1) << j
nn ^= XOR_TABLE[i]
out.append(nn)
out[0] = rotl(out[0], ROT_TABLE[0])
out[1] = rotl(out[1], ROT_TABLE[1])
out[2] = rotl(out[2], ROT_TABLE[2])
out[3] = rotl(out[2], ROT_TABLE[3])
return out
a ^= c
b ^= d
for i in range(128):
a, b, c, d = round_hash(a, b, c, d)
© 2002—2013, Digital Security
SMT in protection analysis
Automated KeyGen Generation. Kao’s Toy Project
© 2002—2013, Digital Security
SMT in protection analysis
Automated KeyGen Generation. Kao’s Toy Project
 Lift the checking algorithm to BIL
./toil -binrange ~/toyproject.exe 0x401105 0x401111 -o checkUnlockCode.il
 Convert BIL to single static assignment form (SSA), unroll loop
./iltrans -il checkUnlockCode.il -to-ssa -simp-ssa -to-cfg -unroll 31 -rm-cycles \
-rm-indirect-ast -to-ast -normalize-mem -flatten-mem -pp-ast checkUnlockUnroll.il
egrep -v '^cjmp.*$' checkUnlockUnroll.il > checkUnlockUnrollOpt.il
 Convert BIL to SMT-formula
./topredicate -il checkUnlockUnrollOpt.il -noopt -solver z3 -stp-out checkLoop.smt
line 18: assert --> define-fun alg () (Array (_ BitVec 32) (_ BitVec 8))
line 921: false --> ?mem_array_83_670
© 2002—2013, Digital Security
SMT in protection analysis
Create precondition and postcondition
© 2002—2013, Digital Security
AEG
Automatic Exploit Generation
 Automatically craft an input that redirects control flow
Loosely defined as “Given a program and a vulnerability,
automatically craft an input that redirects control flow to
malicious code”
 Automated Payload Creation
© 2002—2013, Digital Security
AEG
Automatically craft an input that hijacks control flow
 Get the trace to vulnerable code
 Convert the trace into set of constraints
Freach
 Generate the set of conditions that make code exploitable
Fexploit = Cval U Caddr
 Solve (Freach U Fexploit)
 SMT-solver defines required input
© 2002—2013, Digital Security
AEG
Automatically craft an input that hijacks control flow
Freach =
{ t0= eax + ebx, zf ==1 }
Cval =
{ eax = 0xdeadbeef }
Caddr =
{ t1= ebp + 4, t1 = ebp +ecx }
Fexploit = Cval U Caddr
© 2002—2013, Digital Security
AEG
Automatically craft an input that hijacks control flow
Freach =
{t0 = eax + ebx, zf == 1}
Cval =
{ eax = 0xdeadbeef }
Caddr =
{t1 = ebp + 4, t1 = ebp + ecx}
Fexploit = Cval U Caddr
Input ={eax = 0xdeadbeef,
ebx = 0x21524111, ecx = 4}
(declare-const t0 (_ BitVec 32))
(declare-const t1 (_ BitVec 32))
(declare-const eax (_ BitVec 32))
(declare-const ebx (_ BitVec 32))
(declare-const ecx (_ BitVec 32))
(declare-const ebp (_ BitVec 32))
; Freach = {zf = 1, t0 = eax + ebx}
(assert (= t0 (bvadd eax ebx)))
; t0 = eax + ebx
(assert (= t0 #x00000000))
; zf = 1
; Cval = { eax = 0xDEADBEEF }
(assert (= eax #xdeadbeef))
; eax = 0xDEADBEEF
; Caddr = { t1 = ebp + 4, t1 = ebp + ecx}
(assert
(and
(= t1 (bvadd ebp #x00000004)) ; t1 = ebp + 4
(= t1 (bvadd ebp ecx)))
; t2 = ebp + ecx
)
sat (model
(define-fun ecx () (_ BitVec 32) #x00000004)
(define-fun eax () (_ BitVec 32) #xdeadbeef)
(define-fun ebx () (_ BitVec 32) #x21524111)
)
http://rise4fun.com/Z3/j2Y
© 2002—2013, Digital Security
Automate Generation Payload
Automated Payload Creation
 Data Execution Prevention (DEP)
 Windows 8 ROP mitigation enforces policies on who/where can call VirtualAlloc() or
VirtualProtect() to enable memory executable at run-time
 IOS already totally forbid code injection: Writable pages have NX permission & only signed
pages are executable
 Return Oriented Programming
 fun at first time, then hurt
 hundreds and thousands of ROP-gadgets
 “bad characters"
 find a suitable gadget can be difficult
 research efforts aimed at solving the problem of automatic generation ROP-chains
© 2002—2013, Digital Security
Automate Generation Payload
An interesting example from 0verckl0ck
Given:
 we can write into eax, but only ASCII printable char
 we have ROP-gadgets like these:
add eax, 0xc9f4458b;
add eax, 0x0fcf;
add eax, 0x1337;
add eax, 0xdeadbeef;
add eax, 0x13b2;
add eax, 0x42;
Goal:
 make eax = 0xb00bdead
 determine the initial value eax
 find the minimum sequence of calls gadgets
© 2002—2013, Digital Security
Automate Generation Payload
An interesting example from 0verckl0ck
source: http://rise4fun.com/Z3Py/OrzP
assert( init_eax + g1*0xc9f4458b +
g2*0xdeadbeef +
g3*0x0fcf +
g4*0x13b2 +
g5*0x1337 +
g6*0x42 = 0xb00bdead)
assert(ascii_printable( init_eax ) )
sum (g1,g2,g3,g4,g5,g6) --> min
answer:
0x522e707c + 3*0xc9f4458b + 8*0x13b2 = 0xb00bdead
© 2002—2013, Digital Security
Automate Generation Payload
Finding gadgets with specific samntics
http://www.immunitysec.com/downloads/sean_ruxcon2010.pdf
© 2002—2013, Digital Security
Automate Generation Payload
Immunity Debugger: !find_gadget
This script looks for a sequence that satisfies
the constraints we specify
© 2002—2013, Digital Security
Automate Generation Payload
OptiROP
© 2002—2013, Digital Security
Automate Generation Payload
OptiROP
© 2002—2013, Digital Security
Automate Generation Payload
ROPC: https://github.com/pakt/ropc
© 2002—2013, Digital Security
Automate Generation Payload
ROPC : Type of gadgets that ROPC find &use
Name
Input
NopG
_
_
nop
LoadConstG
OutReg, Value
_
OutReg  Value
MoveRegG
InReg, OutReg
_
OutReg  InReg
ArithmeticG
InReg1, InReg2, OutReg
op
OutReg <- InReg1 op InReg2
StoreMemG
AddrReg, InReg
# Bytes, Offset
M[AddrReg+Offset]<-InReg
LoadMemReg
AddrReg, OutReg
# Bytes, Offset
OutRegM[AddrReg+Offset]
ArithmeticStoreG
InReg, AddrReg
# Bytes, Offset, op
M[AddrReg+Offset] op  InReg
ArithmeticLoadG
OutReg, AddrReg
# Bytes, Offset, op
OutReg op  M[AddrReg+Offset
© 2002—2013, Digital Security
Parameters
Semantic Definition
Automate Generation Payload
ROPC-LLVM: https://github.com/programa-stic/ropc-llvm
© 2002—2013, Digital Security
SMT Solvers for Software Security
Questions ?
© 2002—2013, Digital Security