COTS Based System Security Economics
- A Stakeholder/Value Centric Approach
Related tool demo session:
COTS Based System Security Test-bed (Tiramisu)
Tuesday at Davidson Conference Center
Yue Chen
PhD Candidate in Computer Science
Advisor: Dr. Barry Boehm
941 W. 37th Place, SAL Room 330
University of Southern California
Los Angeles, CA, 90089, USA
Phone: (213)740-6470
Email: yuec@usc.edu
©All rights are reserved by the authors
Agenda








Background
Goal of Research
Nature of the Problem
T-MAP Framework
Tiramisu tool Demo
Model Applications
Initial Validation Results
Conclusions and Future Work
2
Background
 Trends
– Increasing usage of COTS software in IT systems
– Increasing concerns on COTS software vulnerabilities
 Challenges
– Evaluating CBS security in business context
– Benefit of security investment is difficult to measure
– “Twenty percent of vulnerabilities caused eighty
percent of the security risk”, but, what are they?
3
Goals of T-MAP
 T-MAP: Threat Modeling based on Attack Path analysis
– A Stakeholder Value Centric Approach
 Help making decisions on how much security investment
would be optimal
– Max security strategy
– Max cost-effectiveness strategy
 Help system designers understand the security of COTS
combinations in early project life-cycle
 Help network administrators determine vulnerability
priorities
4
Nature of The Problem
Attacking Paths
Permitted Ports
Unblocked vulnerabilities
Vulnerabilities impacting
confidentiality, availability,
integrity
Blocked vulnerabilities
Firewall Wrapper
e.g. Windows Server
2003
e.g. SQL Server
2000
Software Applications, COTS
e.g. Web Server
e.g. IIS 6.0
e.g. CRM Server
IT Infrastructure
e.g. Regulatory
Productivity
Org. Values
Reputation
5
T-MAP Framework
 Three key steps:
– Step 1: Interview with key stakeholders to determine
how organizational value rely upon IT security
– Step 2: Enumerate what are the scenarios that COTS
system vulnerability can compromise organizational
values
– Step 3: Evaluate the severity of each scenario by
weights, and model COTS system security threat with
total weights of all scenarios
Step 2 and 3 are tool automated (Tiramisu)
6
USC-ITS Server X Case Study – Background




Security protection of Server X, a sensitive database
Determine best practice under limited budget
Key stakeholders: students, faculties, staff
Organizational goals
– Productivity of the teaching and research community
– Regulation compliance
– Privacy of students, faculties, and staff
 COTS software installed on Server X:
7
Step 1 – Determine stakeholder/value dependencies on IT
security
 Evaluate the severity of security hazard scenarios by
stakeholder/value impacts
 Involves both qualitative and quantitative criteria
 Technical approach: Figure of merits and Analytical
Hierarchy Process (AHP)
 Example output (from USC Server X Case Study)
8
Determine the Weights - AHP Pair-wise Comparison
Example – Stakeholder value priority weights:
Reading: regulation is “very
strongly” more important than
productivity
9
Step 2 – Attack Scenario Analysis
 Enumerate the scenarios how an attacker can
compromise stakeholder values through COTS system
vulnerabilities
 Attack Graph is established based on a comprehensive
COTS vulnerability database involves 18,800 known
vulnerabilities reside in 31,713 COTS software
10
Step 2 (Continued) – Example Output and Observations
 Example out put of Step 2 (Tiramisu screenshot below)
(Example output – from USC Server X Case Study)
11
Step 3 – Security Scenario Severity Evaluation
Severity Drivers
 Stakeholder value impacts
 Vulnerability technical
attributes
– Impact on confidentiality,
integrity and/or availability
– Remotely exploitable
– Require valid user
account on victim host
– Needs user activities
 Attackers
– Group size
– Skill level
– Motivation to attack
12
Step 3 (continued) T-MAP Severity Rating System
 Severity Weight of Attack Path P:
 Overall Security Threat Score of COTS System G:
 ThreatKey of elements in Attack Graph:
 Effectiveness of Security Practice:
13
Tiramisu Tool Demo
 Tiramisu is the software implementation of T-MAP
14
T-MAP Applications (1)
Security Investment Effectiveness Estimation
 How much security threats can be avoided by implementing
Firewall, Software hardening (patching), user account control, or
file system encryption?
 Results as well depends on the total value of the protected system
* Case study results estimated by professional security manager at USC-ITS
15
T-MAP Applications (2) Security Patching Economics
 Prioritize COTS Based System vulnerabilities under
business context
– “20% percent of vulnerabilities causes 80% of the
security risks”, T-MAP tells what are the 20%
 Rational: Prioritize vulnerabilities with its ThreatKey;
 Example screenshot:
16
T-MAP Applications(3) COTS Security Economics
 Economic curve of security
patching
(from USC Server X case study)
 Sweet spot to invest in security
 Also driven by the total value of
system
(from USC Server X case study)
Sweet spots to invest
17
Initial Validation Results
 Vulnerability priority comparison:
Security Manager’s manual results vs. Tiramisu results
 Tow case studies conducted at USC Information Technology
Services Division
 Two more case studies in progress with:
– Manual Art Senior High School
– African Millennium Foundation
18
Limitations
 Only sensitive to known COTS vulnerabilities
– Empirical study by Arora shows that the average attacks per host per
day jumped from 0.31 to 5.45 after vulnerability get published
 Only cover “one-step-attacks” that exploiting COTS
vulnerabilities
 Depends on comprehensive vulnerability database
– Our database: 188,000 vulnerability published from 1999-2006 that
resides in 31,313 COTS software
 Cannot effectively address passive attacks such as
Phishing
19
Conclusions
 A COTS security evaluation framework that captures
stakeholder value propositions
 Distill the potential impacts of thousands of vulnerabilities
into management friendly numbers at a high-level
 Results are organizational IT infrastructure specific
20
Future work
 Explore applying game theory in T-MAP
 We are looking for real-life projects/system to further
validate and mature the framework
 Close integration with risk driven win-win spiral process
to engineer more secure COTS Based System (CBS)
– Proactively evaluate CBS security in early life-cycle
– Making convincing security business case for CBS
– Help make better security protection plan
Contact: Yue Chen, yuec@usc.edu
21