Risk Analysis
Class Four
University at Albany, SUNY
Spring 2004
Controls
VAM Approach
•
•
•
VAM reflects a systems approach developed for the military defense
environment.
VAM matrix compares security vulnerabilities to controls.
VAM characterizes in terms of four high-level aspects:
–
–
–
–
•
Resilience and robustness
Intelligence, Surveillance, Reconnaissance (ISR) and self-awareness
Counter intelligence, denial of ISR, and target acquisition
Deterrence and punishment
Many of the controls embrace the entire system architecture, e.g.,
– heterogeneity is possible if several different operating systems and hardware
are incorporated in the architecture.
•
VAM provides a matrix that links attributes leading to vulnerabilities to
controls.
INF 766 Risk Analysis
2
Controls
VAM Matrix
• VAM recognizes that controls can have both positive
and negative effects simultaneously, e.g.,
– Encryption enhances confidentiality, but creates key
management issues
– Heterogeneity improves robustness of system, but adds
complexity making system harder to maintain.
INF 766 Risk Analysis
3
Controls
VAM Matrix
• Rating Scheme
• Each cell relating a security technique to a vulnerability
contains a number from -2 to 2, according to this scheme
– 2 means that the control mitigates the vulnerability significantly
• A primary candidate for addressing it.
– 1 means that the control mitigates the vulnerability somewhat, but not
as well as one labeled 2,
• A secondary candidate for addressing it.
– 0 means that the vulnerability may have beneficial side effects that
enhance some aspect of security.
– -1 means that the control worsens the vulnerability somewhat or incurs
new vulnerabilities.
– -2 means that the control worsens the vulnerability significantly or
incurs new vulnerabilities.
INF 766 Risk Analysis
4
Risk Analysis
Security technique may:
2: mitigate vulnerability(primary)
1: mitigate vulnerability(secondary)
0: be facilitated by vulnerability
-1:incur vulnerability(secondary)
-2:incur vulnerability(primary)
VAM Example
Trust, Authentication, and Access
Resilience/Robustness
2
2
1 1 2
2
2
1
1
1
1
0 -2 2
2
1
-1 1
-1 2
-2
Singularity
Uniqueness
Centrality
Homogeneity
Separability
Logic/
Implementation
Errors; Fallability
2
Design
Sensitivity/Fragility
/Limits/Finiteness 2 -1 2
Unrecoverability
2
2
2
2
2
2
2
1
1 -1 2
1
1
2 -1 2
2
2
2
1
1
2
2 -1 1
1
1
1 -2 1 -1
2 1
2 -1
1 -1
Valuation of Security Techniques
Controls
Which Controls are Best?
• The VAM process requires refinement of controls based on the
role:
– Operational: System management perspective
– Design: Developers’ perspective
– Policy: Organizational perspective
• VAM Refinement matrix simplifies control selection
– Rows represent security controls
– Columns serve two functions
• The first three columns represent the perspectives for the relevance of the
control
• The next five columns represent the stage of attack when the control is
useful
INF 766 Risk Analysis
6
Controls
Relevance Matrix
Offense and
Retribution
1
1
1
2
2
1
1
1
Counter-intelligence
2
2
2
2
2
Unperdictability
2
2
2
2
2
Deception to ISR
2
2
2
2
2
Denial of ISR
2
2
2
2
2
Deception to ISR
2
Attack Detection & Forensics
Deterrence
2
Preventative and Retribution
1
2
2
2
2
Criminal and Legal
2
2
2
Civil Proceedings
2
2
2
Assess
2
Self-Awareness
Access
1
2
Knowledge
1
Intelligence Operations
Policy
1
Developer
1
Operational
Non-Retribution
•
1
Apply to Physical, Cyber,
Human/Social, and Infrastructure
Components
Intelligence,
CounterSurveillance,
Intelligence,
Reconnsaissn
Denial of ISR
ace (ISR) and
& Target
SelfAcquisition
Awareness
Helps Protect
These Attack
Stages
Target
Useful to
Users
•
The content of the relevance
matrix, including the columns
and rows, should be
customized to individual
organizations
Rating Scheme
– 1: that control is weakly
relevant to the perspective or
attack stage
– 2: that it is strongly relevant
•
INF 766 Risk Analysis
The VAM approach is
sophisticated and deals with
elaborate systems
– Simpler approach that lists the
controls and weaknesses may
be appropriate.
7
Risk Analysis
Project Savings
• We now determine whether the costs outweigh the benefits of
preventing or mitigating the risks.
• The effective cost of a given control is the actual cost of the
control minus expected loss from using the control
– Cost of control examples: purchase price, installation costs, and training
costs minus.
– Loss from using control examples: administrative or maintenance costs).
– Thus the true cost of a control may be positive if the control is
expensive to administer or introduces new risk in another area of the
system.
INF 766 Risk Analysis
8
Risk Analysis
Example
• Simple Case:
A company uses a common carrier to link to a network for certain
computing applications.
The company has identified the risks of unauthorized access to data and
computing facilities through the network.
These risks can be eliminated by replacement of remote network access
with the requirement to access the system only from a machine operated
on the company premises.
The machine is not owned; a new one would have to be acquired.
• The economics of this example, which are not very
promising, are as shown in the following slide.
INF 766 Risk Analysis
9
Risk Analysis
Cost/Benefit Analysis
Cost/Benefit Analysis for Replacing Network Access
Item
Amount
Risk: unauthorized access and use
Access to unauthorized data and programs
$100,000 @ 2% likelihood per year
$2,000
Unauthorized use of computing facilities
$10,000 @40% likelihood per year
$4,000
Expected annual loss (2,000 + 4,000)
$6,000
Effectiveness of network control: 100%
-$6,000
Risk Analysis
Cost Control
Control cost:
Hardware (50,000 amortized over 5 years)
+$10,000
Software (20,000 amortized over 5 years)
+$4,000
Support personnel (each year)
+$40,000
Annual cost
$54,000
Expected annual loss (6,000 – 6,000 +54,000)
$54,000
Savings (6,000 – 54,000)
-$48,000
INF 766 Risk Analysis
11
Risk Analysis
Cost/Benefit – Graphical Technique
•
Case:
We are considering making use of regression testing after making an upgrade to
fix a security flaw.
– Regression Testing means applying tests to verify that all remaining functions
are unaffected by the change.
•
•
•
Using the diagram on the following slide, compare the risk impact of
doing regression testing with not doing it.
The upper part of the diagram shows the risk of conducting regression
testing, and the lower part shows the risks of not doing regression testing.
In each of the two cases, one of three things can happen:
– We find a critical fault
– We miss finding the critical fault
– There are no critical faults to be found.
INF 766 Risk Analysis
12
Risk Analysis
Cost Savings
•
For each possibility
– Calculate the probability of an unwanted outcome (UO), P(UO).
– Associate a loss with that unwanted outcome, L(UO).
•
Thus, in our example,
– If we do regression testing and miss a critical fault in the system (a
probability of 0.05), the loss could be $30 million.
– Multiplying the two, we find the risk exposure for that strategy to be $1.5
million.
– As the following calculations in the figure show, it is much safer to do the
regression testing than to skip it.
INF 766 Risk Analysis
13
Risk Analysis
Calculation for Regression Testing
Risk Exposure
P(UO) = 0.75
Find critical fault
yes
Do
regression
testing?
no
P(UO) = 0.05
Don’t find critical fault
$1.500M
$1.975M
L(UO) = $30M
L(UO) = $0.5M
P(UO) = 0.20
$0.100M
No critical fault
L(UO) = $0.5M
P(UO) = 0.25
Find critical fault
$0.125M
P(UO) = 0.55
Don’t find critical fault
P(UO) = 0.20
$0.375M
Combined
Risk
Exposure
L(UO) = $0.5M
L(UO) = $30M
$16.500M
$16.725M
L(UO) = $0.5M
$0.100M
No critical fault
INF 766 Risk Analysis
14
Risk Analysis
Pros & Cons
• Pros
–
–
–
–
–
Improve awareness.
Relate the security mission to management objectives.
Identify assets, vulnerabilities, and controls.
Improve basis for decisions.
Justify expenditures for security.
• Cons
–
–
–
–
Decreased sense of precision and confidence
Hard to perform.
Immutability.
Lack of accuracy.
INF 766 Risk Analysis
15
Matrix Based Approach
Risk Analysis
Aggregation
• Individual risks aggregated to get the total risk posture
– True comparison of relative risks of different organizations
• A mathematical approach to correlate risks with assets of the
organization is provided
– Methodology is standardized.
– Data needs to be customized for the organization.
• Controls can reduce the cost of the exposure
– Need to determine the optimum controls for the organization.
– Methodology for determining controls (shown later).
• Analysis should be undertaken to see the impact of new
projects on security.
INF 766 Risk Analysis
17
Risk Analysis
Assets and Vulnerabilities
Client Secrets
Lost
Sales/Revenue
Reputation (Trust)
Cleanup Costs
Info/ Integrity
Hardware
Software
Services
•
Relative Impact
Trade Secrets (IP)
Web Servers
Compute Servers
Firewalls
Routers
Client Nodes
Databases
Vulnerabilities
Crit. Infrastructure
Not Relevant - 0
Low – 1
Medium – 3
High – 9
Assets & Costs
Value Scale:
Customize the matrix to assets & vulnerabilities applicable to your case:
– Compute the cost of each asset and put them in the value row
– Determine the correlation with the vulnerability and asset (L(ow)/M(edium)/H(igh))
– Compute the sum of product of vulnerability & asset values; add to impact column
INF 766 Risk Analysis
18
Risk Analysis
Vulnerabilities and Threats
Web Servers
Compute Servers
Firewalls
Routers
Client Nodes
Databases
…
…
…
Relative Threat
Importance
…
Denial of Service
Spoofing, Masquerading
Malicious Code
Human Errors
Insider Attacks
Intrusion …
Not Relevant - 0
Low– 1
Medium – 3
High – 9
Threats
•
Vulnerabilities
Scale
Value
Complete the matrix based on the specific case
– Add the values from the Impact column of the previous matrix
– Determine the association between the threat and vulnerability
– Compute aggregate exposure values by multiplying the impact and the associations
INF 766 Risk Analysis
19
Risk Analysis
Threats and Controls
Human Errors
Insider Attacks
Intrusion
Physical
Damage
Spam
…
…
Value of Control
Malicious Code
•
Spoofing
Value
Controls
Firewalls
IDS
Single Sign-On
DMZ
Training
Security Policy
Network Configuration
Hardening Environment
DOS
Not Relevant - 0
Low – 1
Medium – 3
High – 9
Threats
Scale
Customize the matrix based on the specific case:
– Add the values from the relative exposure column of the previous matrix.
– Determine the impact of different controls on different threats.
– Compute the aggregate value of benefit of each control.
INF 766 Risk Analysis
20
Risk Analysis
Total Exposure
•
Determine the frequency of each attack using statistical data
– With controls,
– Without controls.
•
•
•
Compute the exposure of each asset given the frequency of attacks
Compare the exposure with controls, with cost of controls, with the
exposure without controls, to determine the optimum control
The hardest part of the exercise is getting the frequency of attacks using
statistical data since,
– data is difficult to obtain,
– data is often inaccurate.
•
Use your best judgment for obtaining frequency data.
INF 766 Risk Analysis
21
Quantitative Analysis
Risk Aggregation
INF 766 Risk Analysis
23
Risk Aggregation
Model Cont’d.
INF 766 Risk Analysis
24
Risk Aggregation
Modeling Uncertainties
• Some uncertainty exists regarding the value that should be
assumed by one or more independent variables in the Risk
model.
• This lack of knowledge about particular values or the
knowledge that some values might always vary contributes
to the model’s uncertainty.
• If it cannot be determined with certainty what value one or
more input variables in a model will assume, this
uncertainty is naturally reflected on the outcome of the
dependent variable(s).
• The risk metric is not determined by the value of its
independent variables (asset values and vulnerabilities,
frequency and impact of threats); it is also a function of the
probability distribution of each of these random variables
• A good approach to dealing with uncertainty is simulation
INF 766 Risk Analysis
25
Monte Carlo Simulation
Approach
• The simulation approach uses the following steps:
– Develop the risk model.
– Define the shape and parameters of the probability
distributions of each input variable.
– Run the Monte Carlo simulation.
– Build a histogram for each of the dependent variables
in the model (risk and updated risk).
– Compute summary statistics for each dependent
variable in the model.
– Perform sensitivity analysis to detect variability
sources.
– Analyze potential dependency relationships among
variables in the model.
INF 766 Risk Analysis
26
Monte Carlo Simulation
Value of Assets
INF 766 Risk Analysis
27
Monte Carlo Simulation
Annualized Frequency of Threats
Truncated Normal (mean, std. dev, min=0)
INF 766 Risk Analysis
28
Monte Carlo Simulation
Impact of Threats
INF 766 Risk Analysis
29
Monte Carlo Simulation
Controls
INF 766 Risk Analysis
30
Monte Carlo Simulation
Distribution of Risk Exposure
Histogram of Exposure Risk
(1000 runs)
30
25
Frequency
20
15
10
5
0
5610
10627
15643
20660
25677
Cumulative Distribution
Risk (in $)
Histogram of Exposure Risk
Cumulative Distribution of Exposure Risk
(1000 runs)
1000
900
800
Frequency
700
600
500
400
300
200
100
0
5610
10627
15643
20660
25677
Risk (in $)
INF 766 Risk Analysis
31
Monte Carlo Simulation
Distribution of Reduced Risk Exposure
Histogram of Reduced Exposure Risk
(1000 runs)
45
40
35
Frequency
30
25
20
15
10
5
0
47
271
496
720
Cumulative Distribution
945
Risk (in $)
Histogram of Reduced Exposure Risk
Cumulative Distribution of Reduced Exposure Risk
(1000 runs)
1000
900
800
Frequency
700
600
500
400
300
200
100
0
47
271
496
720
945
Risk (in $)
INF 766 Risk Analysis
32
Monte Carlo Simulation
Sensitivity Analysis of Exposure Risk
A
n
n
u
a
l
i
z
e
d
F
r
e
q
u
e
n
c
y
Sensitivity Analysis
Exposure Risk
-100.0% -80.0%
-60.0%
-40.0%
-20.0%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Worms
Password Based Attacks
Viruses
Intrusion
Overflow Attacks
INF 766 Risk Analysis
33
Risk Management in
Other Fields
Operational Risk Management
Objectives?
• Internal Control to Manage Financial Exposure,
Profitability.
• Enhancing Quality as a Competitive Business Strategy.
• Maintaining the Firm’s Reputation for Integrity.
• Managing Strategic/Business Risk.
INF 766 Risk Analysis
35
Operational Risk Management
Key Aspects
• A Deep Understanding of the Business
• Its key risks and its major exposures
• Careful Thought About “What If”
• Awareness that Most Problems Involve Weak
Management Supervision and Multiple Breakdowns in
Controls
• Realization that the basis of any measure is inherently
subjective.
INF 766 Risk Analysis
36
Security for Business Process
Internal Revenue Service
• The example process is a taxpayer requesting information on
status of refund via the Internet.
• The components of this process are: the taxpayer, the Internet,
the central entry point on the Internet for IRS (the portal), the
Internet Refund Fact of Filing application that processes the
request (IRFOF), and the databases containing information
on the taxpayer and his tax information.
Refund Data
T
PORTAL
IRFOF
INF 766 Risk Analysis
T
37
Security for Business Process
IRS
• Security (confidentiality, integrity, and availability) must be
considered in multiple dimensions and layers
• The databases and applications are evaluated in terms of who
or what other applications should have access (confidentiality)
to the data and what can be done to the data (integrity).
• The actual computer the application runs upon, along with its
operating system, is evaluated against who or what
application needs access (confidentiality), the priority of that
need (availability), and what operations can occur on that
computer by that person or application (integrity).
• The network is evaluated for availability and confidentiality.
In this example, the network is not only the internal IRS
network but also includes the Internet.
• And lastly, one needs to look at the physical protections
surrounding the employees and the information technology
(equipment and communication lines).
INF 766 Risk Analysis
38
Security for Business Process
IRS
• To begin discussing security considerations, we use the
following diagram.
• The taxpayer provides name and tax information which
authenticates the taxpayer for this application.
Access Control
Authentication Credentials
A
T
T: Refund Info
3: Taxpayer
E: IRFOF
A
PORTAL
A
IRFOF
Refund Data
T
Data Sensitivity
User Type
Authentication Decision
INF 766 Risk Analysis
39
Security for Business Process
IRS
• In the preceding example, confidentiality was addressed
through the various authentication and identification checks
• Data integrity was addressed during the identification checks.
These checks would determine the operations in which an
authorized user could engage, as in being able to read the tax
information
• Availability was not addressed in the example, but could be
addressed by adding in redundant access points, as in adding a
second portal for users to access
Refund Data
T
PORTAL
IRFOF
INF 766 Risk Analysis
T
40
Security for Business Process
Conclusions
• Not all business processes will have multiple layers of defense.
Some may be missing security features, such as access
controls or audit trails, at the application or database level.
• The type and number of security features will be driven by the
level of risk an organization is willing to accept.
• Because of the interconnectivity of the Internet and internal
networks, no one process owner can accept a higher level of
risk than the organization has stated it is willing to accept.
• As new security defenses are added to an infrastructure,
existing vulnerabilities in business processes need to be
reviewed to determine whether the level of risk is still at an
acceptable level.
• When a business process changes, the security features should
always be reviewed to determine any impacts and to ensure
no new risks have been introduced into the process.
INF 766 Risk Analysis
41
Risk Management in Manufacturing
Strong Financial Pressures
• We need to redefine roles and ownership structures to manage
contracting & ownership risks.
• Risk management has to be used as strategic lever to create
customer and shareholder value.
• A customized delivery system has to be developed to
minimize working capital risks.
INF 766 Risk Analysis
42
Manufacturing Sector
Many sources of risk, some endangering the enterprise
• Financial Valuation & Economic Factors
• Volumetric (Forecasting)
• Pricing and Upstream Cost (Economic)
• Strategic Behavior (Supply Chain Partners, Competitors,
Customers)
• Political and Regulatory Factors
• Disruptions (Accidents, Sabotage, Strikes)
INF 766 Risk Analysis
43
Manufacturing Sector
Strategies
• Discovery and Valuation
–
–
–
Scenario and Vulnerability Analysis
Exchanges and Benchmarks
Enterprise Risks and Product Line Risks
• Risk Transfer Tools, including Insurance
–
–
New tools from Weather derivatives to securitization
New assessment methodologies
• Disruption Risks: The New Kid on the Block
–
–
–
Supply Chain Changes Resulting from Unbundling
Security and Major Accidents/Incidents
Operational Risks, from Fraud to Governance
INF 766 Risk Analysis
44
Health Care Industry
Unique Attributes
•
•
•
•
•
•
Most hazardous industry.
Light regulation.
Paucity of data.
Primitive use of IT.
Primitive standards of performance.
Safety is not a major priority.
INF 766 Risk Analysis
45
Health Care Industry
Annual Statistics
• 1.3 million injuries
• 180,000 deaths
• $ 50 billion total
costs (1989 $)
• $ 14 billion
uncompensated
costs
• Most dangerous
industry
100000
90000
80000
70000
60000
50000
40000
30000
20000
10000
0
Deaths
Medical
INF 766 Risk Analysis
Auto
Workplace
Air
46
Health Care Industry
Risk Management
• Approach:
– Traditional “Damage control” – Liability management
– Recently: Error prevention
• Barriers
– Accountability - A blaming culture
– Lack of leadership
– Limited physician engagement
INF 766 Risk Analysis
47
Summary
1. Risk assessment is a technique supporting
security planning.
2. Inherent in several sectors of business (nuclear,
financial, chemical industry)
3. Lack of data is key issue in information security
risk analysis
INF 766 Risk Analysis
48
Decision Making Framework
Supply Chain Variables Impact on ROA
Material
and MOH
Freight
and
Duties
Inventory-driven
Costs
ROA = (Revenues - Costs) * (1 - TaxRate)
Current Assets + Physical Assets
Site
Location(s)
Inventory
Levels
INF 766 Risk Analysis
49
Vulnerability/risk
Assessment
Corporate
Business Continuity
Functions
Crisis Management
Business Area
Analysis
Business area
impact analysis
Risk Management
and Loss control
Crisis
Communications
CRISIS
EVENT
Restoration
Incident Management
Contingency
planning
From J.
Harrald,
GWU
Exercises/
Drills
Incident Response
Business
Resumption
INF 766 Risk Analysis
Disaster
Recovery
Business
Recovery
50
Decision Making Framework
Try to operate in the green area
Unacceptable Consequences
CRITICALITY
(Eliminate Unacceptable Risk Factors)
Risk Reduction
Risk Acceptance
(Baseline Protection)
Source: ManTech, Inc.
VULNERABILITY
INF 766 Risk Analysis
51