GÖDEL'S GOURD
Fuzzing for logic and state issues
Introductions

Michael Eddington
CTO and Principal Consultant @ Déjà vu Security
12+ years in security consulting
Senior developer/architect in prior life
Author of Peach, an open source fuzzer

Device, Kernel, User, Web, Network




DARPA Cyber Fast track
Thanks Mudge!
Defining the Problem

Fuzzers are good at findings implementation issues
 …that
crash the target
 …that are generically detectable (sqli, xss)

Not good at finding design, logic, and state issues
 …that
do not crash the target
 …that are not generically detectable
Examples
Easy




Buffer Overflows
Memory Corruption
Resource Usage
SQL Injection
Hard



Missing authentication
State corruption
Incorrect logic
Authentication Examples

Out of 100 admin pages, 5 are missing
authentication

Microsoft SSPI skip a step auth bypass

OpenBSD IPSEC incorrect if/then logic
Authentication – Detect


Web – Missing Auth
Trigger
 Request
page w/o
logging in


MS SSPI/OBSD IPSEC
Trigger
 Skip
a step




Status Code
(200/403)
What pages require
auth
Result (Pass)
Did we complete all
steps
Logic Example






Windows 95 SMB Flaw
Logic error in password checking code
Length of loop determined by client input
Modified SMB client, ~32 attemps always wins
We never throw an exception or crash
Typical generic fuzzer will never find this
Logic – Win95 SMB
bool CheckPw(
int userdata_len, char* userdata,
int sysdata_len, char* sysdata )
{
for(int i=0; i<userdata_len; i++)
if(userdata[i] != sysdata[i])
return false;
return true;
}
Logic – Detect


Win 95 SMB
Trigger
 Try
all chars
 Remove NULL


Result
Does password match
State Example




Device (phone/tablet/laptop) with theft system
Agent “heartbeats” to server
Server can trigger “stolen” mode in laptop
Laptop will trigger if unable to “heartbeat”
 Timer/counter
runs down
State – Detect


System Server
Trigger
 Cause
exception
 Flow locked
 Unable to heartbeat


Can we perform state
flow?
Check result of each
step
How to detect?

Goal – Modify existing fuzzer to detect these issues
 We
already produce triggers
 How do we add detection?
How to detect?

What do we need to detect these issues?

Provide system constraints
 If
not authenticated result is 402
 If steps 1, 2, and 3 not performed step 4 is error
 Result is never 500

Verify we are still working
 Perform
state flow w/o mutations
Proposed Solution

Gödel's Gourd

Re-use Peach fuzzing engine
 Mutation
engine
 Fault detection/reporting



Constraint language
Control iterations (non mutation iterations)
Mutate state model (skip, order, etc.)
Control Iterations




Goal: Verify target is working correctly
No mutations
Constraints pass
State model is followed
 Matches
recorded control iteration
How it works







R – Record iteration
1 – Fuzzing iteration
C – Control iteration
2 – Fuzzing Iteration
C – Control iteration
3 – Fuzzing iteration
…



Remember all
states/actions from
record iteration
Verify on control
iterations
Control iterations
every N fuzzing
iterations
Outcome

If control does not match record – throw fault

Identify conditions that stop normal operation
Constraints


Verify logic via simple constraint expressions
Apply constraints to state model
 State
 Action

Does not modify fuzzer state
Language Options

Existing Traditional
Languages

Pro
 Well
known
 Available via .NET
scripting interface
 JavaScript
 Python
 Ruby
 etc.

Cons
 Allows
modification of
fuzzer state.
Other Options


Domain Specific
Language (DSL)
Use existing

Pros
 Meet

Cons
 Must

Create our own
all requirements
implement
 Not well known
DSL Selection

Object Constraint Language (OCL)
 Specification
language, no side effects
 Developed as part of new UML standards
 Familiar syntax

Relatively easy to implement
Object Constraint Language (OCL)

Expression types

Invariant (inv)
 Always

true
Pre (pre)
 Evaluated

before [ something ]
Post (post)
 Evaluated
after [ something ]
 Can access state from Pre. (@pre)
OCL Examples
“Car owner must be at least 18 years old”
context Car
inv: self.owner.age >= 18
“If passwords match result is true”
context Login
post: result = true implies pass1 = pass2
OCL Context



Groups sets of constraints
Constraints for a context are run together
Association based on context
Normal Fuzzing Iteration


Enter State Model
State 1
 Action
 Send
 Action
1.1
Data
1.2
 Receive


State N
…
Data
Fuzzing Iteration With Constraints


Enter State Model
State 1
 Action
 Send
 Action

Inv(pre)
Pre

EVENT

1.1
Data
1.2
 Receive
Data



State N
…

Inv(post)
Post
Applying (Authentication)

Web Authentication
# Verify authentication occurred
post:
(reply = 200 &&
url.indexOf(‘/admin’) > -1)
implies
auth.reply = 200
Applying (Authentication)

Windows SSPI
# Verify all steps completed
post: reply = true implies (
auth.step1.reply = true &&
auth.step2.reply = true &&
auth.step3.reply = true)
Applying (Logic)

Windows 95 Bug
post: reply = true implies userpw = ‘password’
Applying (State)

Antitheft System

Perform control iteration
Implementation
Technologies Used

Microsoft .NET Framework – C#
Peach Fuzzer 3

Cross platform using Mono

 OS
X
 Linux
Implementation Diagram
OCL Implementation

Irony .NET Language Toolkit
 Many
differences from traditional
 Grammar is code
 Easy AST hookups

LINQ Expressions
 From
IronPython work
 Last mile is already done
LINQ Expressions


Exposes language constructs for use in AST classes.
Does all the heavy lifting.
return Expression.Condition(
(Expression)ifNode.Evaluate(thread),
(Expression)thenNode.Evaluate(thread),
(Expression)elseNode.Evaluate(thread));
Gödel Usage
All the things that do the stuff
Peach Pit vs. Gödel Gourd

Data Model



State Model

Data Model
OCL Definitions
State Model
 OCL


Agents
Test


Agents
Test
Associations
Gödel: Define Constraints
<Ocl>
<![CDATA[
context StatusCodeOk
post: context.test.publishers[self.publisher].Result =
'OK'
]]>
</Ocl>
Gödel: Associate Constraints
<Action type="call" method="Logout">
<Ocl context="StatusCodeOk" />
</Action>
Constraints will now run with this Action.
Gödel: Control Iterations
<Test name=“Default” controlIteration=“1”>
<Agent … />
<StateModel … />
<Publisher … />
<Logger … />
</Test>
Define how often control iterations occur.
Usage Feasibility
Time and Cost
Adding Gödel

Process:
 Existing
Peach PIT
 Add OCL Constraints
 Test and Verify Definition

Not recreating full application logic
 Just
our “view of the world”
Time per Protocol



Based on current experience of limited protocol set
Decent in 1 – 2 days
Complete in 1 week or less
Performance

What performance impact does Gödel incur?
 Constraint
evaluation
 Control iterations

No performance optimizations…yet
Performance of Constraints
Time for 10,000 (Seconds)
60
50
40
30
20
10
0
1
5
10
15
Constraint Count
20
25
Performance Control Iterations

Depends on how often, worst case half speed

Never longer than mutation iterations
Performance Conclusions




Performance impact dependent on speed of fuzzing
Ability to scale fuzzing lowers impact
For fast fuzzers, acceptable impact
For slower fuzzers, adjust control iterations to occur
less often
Conclusions

Pentesting/Quick fuzzing
 Reasonable
for “basics” (verify state’s work, critical
logic flows)

General definition building
 Reasonable

to implement decent coverage
1-2 days “good enough”
Wrapping it up…
Lessons Learned





Constraints applied only to control iterations
Writing good constraints that apply to all mutation
cases is challenging
A few constraints can go along ways
Performance overhead needs to be lowered when
many constraints used.
Optimize access to most used variables/objects
Looking towards next rev…



Can we “learn” basic constraints?
Performance optimizations
Shorted “name” of common objects
 context.test.publishers[self.publisher].Result
 self.Result
Thanks for all the fish!




Michael Eddington
mike@dejavusecurity.com
http://dejavusecurity.com
http://peachfuzzer.com