Automated Security Testing with
Formal Threat Models
Frank Xu Ph.D.
Overview





Introduction
Objectives
Approach
Experiments
Contribution & Conclusions
Introduction

Application security


Application vulnerabilities exceed Networking and OS vulnerabilities


Bypass authentication attack, SQL injection attack
Weak authentication mechanism, unsanitized inputs
Preventing malicious security attacks by detecting vulnerabilities
SANS' 2009 Top Cyber Security Risks (http://www. sans.org/top-cyber-security-risks/),
Introduction

How to detect software vulnerabilities?



Similar to detect software bugs
Security testing
Tradition testing vs. security testing


Traditional testing : test if a program does what it is supposed to do
Testing for security: test a program against possible vulnerabilities for
checking if it contains unintended behaviors


Sql injection to log into the system
Problem?

Security testing is very labor-intensive

Sql injection string: ' or '1'='1

databases, inputs, paths
Objectives
Presents an approach to automatically test
software security
Approach

Create formal threat models


Automatically generates all attack paths,


represented as Predicate/Transition nets
i.e., security tests
Converts attach path into executable test code

according to the given MIM (Model-Implementation Mapping)
specification
PrT net
http://www.informatik.uni-hamburg.de/TGI/PetriNets/introductions/aalst/elevator1.swf
Prt Net for dictionary attack
Notations

Variable Binding: ø = ?x/V


?x is bound to value V.
Variable Substituting: l/ø :


the tuple (or token) obtained by substituting each variable in l for its
bound value in ø.
If l= <?u,?p> and ø={?u/ID1,?p/PSWD1}, then l/ø=<ID1, PSWD1>.
l= (?u,?p)
P(ID1,PSWD1)
Enabled by ø={?u/ID1,?p/PSWD1},
Transition Enabled
Threat Model
SQL injection attacks
t11:do shopping,
t12: login
t13: check out”
t21: go to login page
t22: retrieve password
t23: forgot your password
t31: login,
t32: do shopping,
t33: check out using coupon code
sqlstr: or 1=1--, ‘) or ‘1’=’1--, and 1’ or ‘1=’1.
Generating Attack Paths
Generating Test Code
http://seleniumhq.org/movies/intro.mov
Model-Implementation Mapping
CASE STUDIES



Case Study I: Magento
Case Study II: FileZilla Server
Mutation (S.T.R.I.D.E. )



Spoofing, Tampering, Repudiation, Information disclosure, Denial
of Service, Elevation of privilege
Kill the mutations
Both studies show that security testing with formal threat
models is very effective.

They have killed 93.2% (41/44) and 96.7% (29/30) of the
mutants, respectively
Contributions & Conclusion


First, automated generation of executable security tests
from formal threat models is a novel contribution to
software security testing.
Injection of security vulnerabilities for evaluating the
effectiveness of security tests is a novel contribution to
mutation testing.