Establishing a Security
Metrics Program
Tiger Team Final Report
Chris Cain & Erik Couture
October 2011
SANS Technology Institute - Candidate for Master of Science Degree
Introduction
•
•
•
•
Team Members
Mandate
Overall project aim
Methodology
SANS Technology Institute - Candidate for Master of Science Degree
Security Metrics Overview
• “How secure are we?”
• “Are our security investments making a
difference?”
• “Where can we have the most impact on
our security posture?"
SANS Technology Institute - Candidate for Master of Science Degree
Why Metrics?
• Metrics vs Measurement
• The importance of context and knowledge,
not just data
• The challenge of what to measure
SANS Technology Institute - Candidate for Master of Science Degree
Goal/Scope
• Paint a clear picture of our security posture
• Identify areas of greatest risk
• Help educate resource allocation towards
areas of greatest security gain
• Educate senior management on possible
business impacts of our security posture
• Provide a method to monitor the
effectiveness of our policy and technological
changes over time
SANS Technology Institute - Candidate for Master of Science Degree
Example 1
Secure Firewalls, Routers, and Switches
Aim
Firewalls
• Visibility of the
Routers
Switches
‘ground truth’
Workstations
• Ensure minimal
Laptops
ports/services exposed Servers
Input Data
0
5
10
15
20
25
Nov
•Network Device Threat Level
•Average days to fix configuration issues
•Total insecure configurations found
Visualization
• Horizontal bar charts – give a good sense of progress over
several reporting periods and between each device type
SANS Technology Institute - Candidate for Master of Science Degree
30
Oct
Sept
Example 2
Boundary Defense
Aim
• Reduce by 80% the number of internet entry points
• Achieve 100% of hosts pointed at secure DNS servers
• Achieve 100% physical network verification.
Input Data
• Total quantity of defenses scored Score from 1 to 5
• Boundary Defense Threat Level (subjectively assigned)
Visualization
• Line graph comparing boundary
device types against their scores
SANS Technology Institute - Candidate for Master of Science Degree
Example 3
Incident Response Capability
Aim
• Assess ability to detect and respond
• Fuse/visualize end-to-end IH timelines
16
14
Input Data
12
• Mean time to incident recovery 108
6
• Number of Lessons Learned
4
as a result of the incident.
2
• Mean time to incident eradication 0
Sept
Oct
Nov
Avg Time to Detect (hrs)
Avg Time to Eradicate (hrs)
• Mean time to incident detection/identification
Avg Time to Recover (hrs)
Visualization
• Stacked Bar Chart – allows reader to quickly compare the
relative time involved in each phase of incident handling
SANS Technology Institute - Candidate for Master of Science Degree
Visualization / Dashboard (1)
SANS Technology Institute - Candidate for Master of Science Degree
Visualization / Dashboard (2)
SANS Technology Institute - Candidate for Master of Science Degree
Recommendations
• The establishment of an enterprise-wide security
metrics program.
• The adoption of the SANS Top 20 Security
Controls framework as a basis for the ongoing
gathering and reporting of security metrics.
• The institution of a security metrics board which will
regularly assess the effectiveness and adjust the
security metrics program.
SANS Technology Institute - Candidate for Master of Science Degree
References
•
•
•
•
•
•
•
•
•
•
Twenty Critical Security Controls for Cyber Defense: SANS/CAG
NIST Special Publication 800-61
Beautiful Security Metrics by Elizabeth Nichols
Twenty Most Important Controls and Metrics for Effective Cyber Defense
and Continuous FISMA Compliance by John Gilligan
Seven Myths about Information Security Metrics by Dr. Gary Hinson
Security Metrics, Replacing Fear, Uncertainty and Doubt, Gary McGraw
FISMA FY2011 - CIO Reporting Metrics by US DHS
IT Security Metrics, A Practical Framework for Measuring Security &
Protecting Data, Lance Hayden, Ph.D.
A Guide to Security Metrics (SANS Reading Room), Shirley C. Payne
CSO Security and Risk by Scott Berinato
SANS Technology Institute - Candidate for Master of Science Degree