Threat analysis as methodology
for deriving risk-based security
tests of web application
software
Marco Morana
OWASP
marco.m.morana@gmail.com
OWASP
IMI 2009 Security Summit
Copyright 2009 © The OWASP Foundation
Permission is granted to copy, distribute and/or modify this document
under the terms of the OWASP License.
The OWASP Foundation
http://www.owasp.org
Presentation Agenda
 Security Testing 101 (In 10 steps)
 Security
 Security
 Security
 Security
 Security
requirements derivation
testing in the SDLC
testing workflows, techniques and tools
testing guides
test reporting and metrics
 State of The Art Of Security Testing
 Effective testing, compliance driven testing, tool centric testing
 Risk Based Security Testing Methodology
 Deriving test cases from cyber-intelligence, attack tree analysis,
attack vectors analysis, ATM/secure architecture analysis
 Security Risk Testing Strategy
 Q&A
OWASP
2
Security Testing 101 (In 10 Steps)
OWASP
3
Why Security Testing?
How Security Testing Is Different From
Traditional Quality Testing?
Test that security controls function as designed with
“positive” tests
Test that security controls cannot be abused by
exercising un-intended/abused functionality with “negative”
tests
Test to find vulnerabilities in applications with an
application scan/pen test
Test to find vulnerabilities in the source code with a
source code analysis/secure code review
Test to find security flaws in design with application
threat modeling/secure architecture review
OWASP
4
Testing For Software Security: Quality
Assurance vs. Software Assurance
Quality faults:
test as
Designed for
what we
want to do
http://www.ddj.com/184405196
Security
faults: test
as built for
anything
unexpected
OWASP
5
The authentication control should lock a user
Step 1: Derive Security Requirements
after 3 failed attempts
Functional requirements
Validate
security
controls
work as designed
to function
The application
need
to implement
input filtering
and
(e.g. authentication,
authorization,
encryption etc)
output
encoding to mitigate
XSS attacks
Non functional requirements
Validate there are no vulnerabilities and security flaws
 Security compliance requirements
Validate
compliance
with FFIEC,
PCI-DSS,
SB in
Do not store
CC authentication
dataGLBA,
protect
CCH data
1386, and
InfoSec,
storage
maskAppSec
CCN onstandards
display
Threat mitigation requirements
Validate mitigation against threats : Spoofing user
identity,authenticate
Tampering with
theand
data,
Repudiation,
Mutually
client
server
to prevent
Informationattacks
Disclosure,
Elevation
of
repudiation
such Denial
as ManofInservice,
the Middle
(MiTM)
privileges
attacks
OWASP
6
Derivation Of Security Requirements To
Validate Compliance With Security Standards
 [PCI-DSS] 6 Develop and Maintain Secure
Systems and Applications
 All vulnerabilities must be corrected.
 The application must be re-evaluated after the corrections.
 The application firewall must detect and prevent web based
attacks such as cross site scripting and SQL injection.
 [PCI-DSS] 11 Regularly Test Security Systems
and Processes
 [PCI-DSS] 11.3.2 External application layer
penetration test.
 For web applications, the tests should include, at a minimum,
the following vulnerabilities: OWASP T10
OWASP
7
Step 2: Use Security Requirement Engineering
Techniques such as Use And Misuse Cases
Enter Username and
password
Includes
User
User Authentication
Threatens
Brure Force
Authentication
Includes
Includes
Show Generic Error
Message
Includes
Mitigates
Harverst (e.g. guess)
Valid User Accounts
Mitigates
Includes
Application/Server
Validate Password
Minimum Length and
Complexity
Mitigates
Hacker/Malicious User
Dictionary Attack
Includes
Mitigates
Lock Account After N.
Failed Login Attempts
OWASP
8
Step 3: Identify Security Testing Tollgates
Iterative approach
Secure
Design
Review
Security
requirements
Threat and
risk
Modeling
Requirements
and use cases
Design
Risk-based
security tests
Test plans
Code
Review
Static
analysis
(tools)
Code
Penetration
testing
Test
results
Field
feedback
OWASP
9
Step 4: Integrate Security Testing In Developer’s
Workflows
Developers Security Tests
Test security of functions, methods and classes
Security unit/component tests
Secure code reviews/security bugs testing
Dynamic analysis (e.g. application scans)
Main developer’s workflow test check-points
Secure code reviews :validate compliance with
secure coding standards during coding interactions
Defect management: clean security bugs before
check-in code in application builds
OWASP
10
Step 5: Integrate Security Testing in Tester’s
Workflows
Testers Security Tests
Test security in integrated application builds
Validate application security requirements
Assess and exploit vulnerabilities
Test for secure configuration in operational environment
Fuzz testing, reverse engineering
Tester Workflow Check Points
Validate application security requirements before
release to QA or production: security assurance
Remediate all HIGH RISK vulnerabilities before
release to production according to compliance
requirements
OWASP
11
Step 6: Adopt Manual and Automated
Security Testing Techniques
Find Vulnerabilities Using Combining All Four Find Vulnerabilities Using
the Running Application
Techniques is Most Effective the Source Code
Manual
Penetration
Testing
Automated
Vulnerability
Scanning
Manual
Code
Review
Automated
Static Code
Analysis
OWASP
12
Step 7: Decide Which Security Testing Tools
To Use
Vulnerability Scanning
ISS, Foundscan, Nessus, Nikto
Penetration Testing (Black Box Testing)
Webinspect, Appscan, Hailstorm, Paros, Peach
Binary Analysis/Reverse Engineering
IDA Pro, @stake SmartRisk
Source Code Analysis
Fortify, Klockworks, Parasoft, Free Tools (e.g. FindBugs)
Threat Modeling
MS TAM, TRIKE, PTA Technologies
Rootkit BackDoor Analysis
ootkits.org and rootkit.nl
OWASP
13
Step 8: Document Security Testing Cases
Using Testing Guides
The OWASP Testing Guide
Testing Principles
Testing Process
Custom Web Applications
Black Box Testing
Grey Box Testing
Risk and Reporting
Appendix: Testing Tools
Appendix: Fuzz Vectors
Information Gathering
Business Logic Testing
Authentication Testing
Session Management Testing
Data Validation Testing
Denial of Service Testing
Web Services Testing
Ajax Testing
OWASP
14
Step 9: Report The Findings
http://www.owasp.org/index.php/How_to_write_the_report_of_the_testing_AoC
OWASP
15
What I Should Put in The Testing Report?
The security threat that can potentially exploit
the vulnerability/defect
The root cause of security issues (e.g., security
bugs, security flaw)
The testing technique being used (e.g.
source code analysis, pen test, threat modeling,
security test case)
The remediation of the vulnerability/defect
(requirement, design, code configuration change)
The risk rating of the vulnerability (Critical,
High, Medium, Low)
OWASP
16
Step 10: Collect Security Testing Metrics
Measurement Goals:
Measure vulnerabilities found with pen-tests to
validate that are closed before production release
Measure vulnerabilities found with source code
analysis and correlate with the ones found in pen tests
Measure time it takes to fix vulnerabilities during
coding and during validation tests
Security Metrics:
No. of High, Mediums for each application
No. of vulnerabilities found in source code vs. the
application
Time (man/hours) required to close security issues
OWASP
17
State of the Art Of Security Testing
OWASP
18
Risk Based Security Testing
The main objective is to assert threat mitigation:
Mitigate attacks targeting different exposure
factors : internet, extranet, intranet
Mitigate attacks targeting different threats:
fraud, confidential PII, credit card data, denial of
service
Mitigate attacks that might exploit known
vulnerabilities (e.g. OWASP T10)
Mitigate attacks that might abuse functionality
to cause damage/business impact (e.g. design
flaw that allow to reset password insecurely)
OWASP
19
Security Testing From Compliance Perspective
 Good reasons for compliance security testing:
Avoid fines : $500,000- 800,000 x breach
Doing business with third party (e.g. Walmart)
Minimal security assurance: CYA
According to a recent survey (*) “71 percent of
Not
convincedDO
checklists
will lead
security
companies
NOT treat
PCItoasbetter
a strategic
 Butinitiative
not just rely
security
tests since
and on
79 compliance
percent have
experienced
a
data
breaches
occur even to PCI compliant
DATA
BREACH”
(*) http://www.imperva.com/docs/AR_Ponemon_2009_PCI_DSS_Compliance_Survey.pdf
companies
:
Heartland Payment Systems, 130 million credit cards
Hannaford Bros, 4.2 million credit card
 How aware are you about software security
assurance ? Beware of bad examples…
OWASP
20
Security Testing From Security Tools
Perspective
 MITRE found that all application
Beware of the silver bullet
security tool vendors’ claims put
security
and of
false
togethermentality
cover only 45%
the known
sense
of security
given600
byin CWE)
vulnerability
types (over
tools !
 They found very little overlap between
tools, so to get 45% you need them all
(assuming their claims are true)
OWASP
21
Security Testing From The Application
Business Logic Perspective
“76% of financial sites still have at least one
critical design flaw that can be exploited to
cause financial fraud, identify theft, reputationbrand damage, denial of service to customers”
(University Of Michigan study)
“Information leakage, insufficient authentication
and authorization, and abuse of the website’s
functionality are prime money-makers”
(WhiteHat Security Inc.)
OWASP
22
Risk Based Security Testing Methodology
OWASP
23
Starting With Risk Based Security Testing
 Basic Security Tests Are Prerequisite:
Functional and non functional security
requirements are tested in compliance with standards
and policies
Known vulnerabilities are assessed and mitigated
 Expand Security Testing to:
Real attack scenarios (e.g. cybercrime)
Same methods, tools attack vectors and
vulnerability exploits used by the attackers
Potential abuse/misuse of business logic
All entry points and access levels
Probing secure design/architecture
OWASP
24
Risk Based Security Testing Methodology
1. Threat Intelligence: gather information about real
attacks to simulate the same attack scenarios
2. Threat tree analysis: learn attacker goals and
methods to derive test cases
3. Use and misuse cases to validate countermeasures for
potential abuse of functionality
4. Attack vector analysis for testing attacks against
defense evasion techniques (e.g. data encoding,
automation)
5. Secure architecture analysis for proving
countermeasures at different layers: external
entity, trust boundary, data flow, process and asset.
OWASP
25
Cyber-Attacks Intelligence Driven Security
Tests
Gather information from cybercrime intelligence:
Learn from public cyber-crime reports: IC3,
Symantec, McAfee AVERT Labs, Finjan software
Analyze the attacks: industry, geographic, local
market, overall business, branch
Become your enemy: derive the attack scenarios,
analyze the attack vectors and vulnerabilities
exploited by cybercriminals.
 Probe your defenses:
Try to exploit vulnerabilities using attack vectors
Use same botnet Trojans and PERL script tools to test
application resilience against automation attacks
OWASP
26
Cybercrime Threat Intelligence and Analysis:
ATTACK:
Attack “xp_cmdshell on MSQL server to upload sniffers to
capture CC transactions and ATM PINs from DB, HSM
TEST CASES AGAINST THIS THREAT:
1. Test xp_cmdshell is disabled
2. Test extended URLs are denied
3. Test escape for special characters “”, comma
4. Test store procedures run with minimum privileges
5. Test SQL Server and IIS run under non-privilege
6. Test appserver not use “sa” hardcoded
7. Test account locking on mainframes against brute force
8. Test access to DB server is internally restricted by IP
9. Test a proxy server is used for internet access
10. Test firewall rules are updated
OWASP
11. Test HSM not use commands with PIN in the clear
27
Threat Tree Analysis of Credit Card
Compromise Attacks
Card
#1 TestCredit
web
Data
Compromise
application
assuming browser
compromise
and/or automation
attacks
Attack User/
Browser
Clickjacking
Man In The
Middle/Browser
Attack
Phishing Email/
Social
Engineering
Serve Invisible
Frame that runs
malware
Serve malicious
IFRAME to
victim visiting
the web site
Take
Credentials and
CC data from
user
SQL Injection
Exploit
Upload
Sniffer To Get
CC data
Alter Query
To Get CC
data
#2 Test for SQL injection and
code injection (Frames)
vulnerabilities
Automated SQL
Injection Attack
To upload
malware
Attack Web
Application
Insecure
Cryptographic
Storage/
Transit
Capture NonEncrypted CC
Data
#3 Test
encryption of
sensitive CC
data in storage
and transit
Exploit Weak
Session
Management
Impersonate
user to get
access to CC
data
Session
Fixation to
get access to
CC data
#4 Test for session
fixation and hijacking
OWASP
28
Security Testing With Attack Libraries
Derive a list of attack vectors that can be used
for testing countermeasures such as filtering of
encoded cross-site scripting and SQL injection
commands.
Start with code injection attacks library:
SQL injection attacks
HTML (IFRAME) injection attacks
Script injection (e.g. cross-site scripting) attacks
Command shell injection attacks
File injection attacks
Server pages injection attacks (e.g. ASP, PHP)
Cookie poisoning attacks
XML poisoning attacks
OWASP
29
Attack Vectors For Testing Web Applications
OWASP
30
Security Testing Using Application Threat
Modeling Analysis
DFD-secure architecture analysis provide
testers information about the application
scope for risk based testing such as:
The location of the application entry points that
need to be tested
The access levels that are required to access the
entry points
The vulnerabilities that can be exploited at
each layer of the architecture that need to be tested
The countermeasures that need to be tested for
mitigation of identified threats.
OWASP
31
Application Threat Modeling
Application
Calls
Request
Users
Web Server
DMZ (User/Web Server Boundary)
Access
Level
External
Internal (Web Server/ App & DB Server Boundary)
I. Spoofing
II. Repudiation
Message
Response
Application
Server
Message
Call
Encryption
+
Authentication
Encryption +
Authentication
Database
Server
Restricted Network
(App & DB Server/Financial Server Boundary)
Application
Responses
Responses
AuthN,
Encryption
Digital,
signatures,
HMAC, TS
Access
Level
Internal
Auth Data SQL Query Call
Authentication
Data
Access
Level
Restricted
Financial
Server
Customer
Financial
Data
I.
AuthN,
Encryption
II. Digital
signatures,
HMAC,
TS,AuthZ
Audit
III. Encryption,
AuthZ
IV. Filtering,
AuthN
Account/
Transaction
Query Calls
Financial
Data
I. Tampering
II. Repudiation
III. Info
Disclosure
IV. Denial OF
service
OWASP
32
Threat Modeling Driven Security Test Cases
 Spoofing Identity
 Test attempts to impersonate a user by sniffing user
credentials on the wire or in persistent storage
 Test that security tokens (e.g. cookie) issued before
authentication cannot be replayed to bypass authentication
 Tampering with the data
 Test with a web proxy that is not possible to tamper and
replay the data
 Test strength of hashes against replay attacks (e.g. use of
salted hashes)
 Repudiation
 Test mutual authentication and/or digital signatures to
trust connection between client and server
 Test all security events are audited and logged
OWASP
33
Threat Driven Security Test Cases (Con’t)
 Information Disclosure
 Test that the access of non-public data requires authentication
 Test that error messages and exceptions do not disclose useful
information to an attacker
 Test that processes do not cache or log passwords, session
information or PII
 Denial of Service (Dos)
 Test you cannot flood a process with so much data it stops
responding to valid requests.
 Test that malformed data cannot crash the process
 Elevation of Privilege
 Test user cannot tamper client parameters or forceful
browsing to his elevate privileges
 Test that a process cannot be forced to load a command shell,
which in turn will execute with elevated privileges
OWASP
34
Threat Driven Security Testing Activities
Throughout the SDLC
Definition
Definition
Threat Modeling in the SDLC
Use and Abuse
Development
Development
Design
Design
Identify data flowsCases
to
be tested
Secure
Architecture
Modeling
Deployment
Deployment
Testing
Testing
Secure Coding
Requirements drive
testing during coding
Secure
Requirements
Engineering
Data Flow
Diagrams
Secure Coding
Standards
Derive security test
cases for integrated
system tests
Identify security
requirements to be
tested
Security
Requirements
Threat Modeling
Secure Code
Reviews
Security Testing
Guidelines
(System Tests)
Attack Trees
Security Unit
Testing
Security Integrated
Testing
Assessments
Test secure
(Production)
configuration before
production release
Vulnerability
Identify attack
scenarios, goals and
methods to test
Security Testing
Guidelines (Unit
Tests)
Vulnerability
Assessments
(User Acceptance
Test)
Secure
Configuration &
Installation
Identify security flaws
and countermeasures
to be tested
Identify security bugs
with secure code
analysis
Identify vulnerabilities
with pen tests during
UAT cycles
OWASP
35
Security Risk Driven Strategy
OWASP
36
The Security Risk Testing Strategy
Identify which attacks can be most likely used
against your web application and your
customers
Use attack methods where difficulty of attack is
the least and the impact is the highest
Use misuse cases to test abuse of
functionality/business logic
Use the same same attack vectors to try to evade
countermeasures/defense mechanisms
Test defense in depth against threats
identified in the application threat model
OWASP
37
QUESTIONS
ANSWERS
OWASP
38
References
 OWASP Testing Guide
 http://www.owasp.org/index.php/OWASP_Testing_Guide_v3_Table_o
f_Contents
 Testing for Software Security Rethinking security bugs
 http://www.ddj.com/184405196
 Education Module Embed within SDLC
 http://www.owasp.org/index.php?title=Education_Module_Embed_wi
thin_SDLC&setlang=es
 OWASP CAL9000 Project
 http://www.owasp.org/index.php/Category:OWASP_CAL9000_Project
 Security Flaws Identification and Technical Risk Analysis Through Threat
Modeling
 http://www.net-security.org/dl/insecure/INSECURE-Mag-17.pdf
 OWASP Application Threat Modeling
 http://www.owasp.org/index.php/Application_Threat_Modeling
OWASP
39