INDULGENCE
There is no need for oversight or management direction. All staff members
are superstars and act in the best interest of the company.
How to Audit
Vulnerability Scans
Doug Landoll
CEO, Assero Security LLC
dlandoll@asserosecurity.com
(512) 633-8405
http://twitter.com/douglandoll
www.douglandoll.com
ISACA Phoenix Chapter
Monthly Meeting - January
Agenda

Background – Security Risk Management &
Assessments
– Assessments as a process
– Security risk management
– Types of assessments

Anatomy of a Vulnerability Scan
–
Vulnerability Scan




Objective, Scope, and Execution
Vulnerability Scan phases
How to Audit Vulnerability Scan (by phase)
Checklist
Security Assessment as Process
Risk
High
Security Improvements Lower Risk
•Security awareness training
•Security policy development
•Operating system hardening
•Security patches
•Anti-virus updates
•Incident handling
Changing Threats and Environment
Increase Risk Over Time
•New regulations
•New exploits
•New system functions •Staff turnover
Low
Time
Security Risk Management
Risk Assessment
• threats / likelihood
• vulnerabilities / exploitation
• assets / impact
• risk / countermeasures
Test & Review
• scanning
• audit of controls
Operational Security
• patches
• incident handling
• training
Risk Mitigation
• safeguard implementation
• additional controls
Types of Assessments
Term
Definition
Purpose
Gap Assessment
A review of security
To provide a list of controls
controls against a standard. required to become compliant.
Compliance
Audit
Verification that all
required security controls
are in place.
To attest to an organization’s
compliance with a standard.
Security Audit
A verification that
specified security controls
are in place.
To attest to an organization’s
adherence to industry standards.
Penetration
Testing
A methodical and planned
attack on a system’s
security controls.
To test the adequacy of security
controls in place.
Vulnerability
Scanning
An element of penetration
testing that searches for
obvious vulnerabilities.
To test for the existence of
obvious vulnerabilities in the
system’s security controls.
Types of Assessments Illustrated
Assessments
Gap Assessment
Standard,
Regulation
Action List
Compliance Audit
Attestation
Security Audit
Security Risk
Assessment
Effectiveness
Risk &
Recommendations
Required
Covered
Selected
Scoped
Controls
Anatomy of a Vulnerability Scan
Pre-Inspection
Enumeration
Footprint
Vulnerability Assessment
• Define Scope
• Define Objective
• Define Project
• Define Team
• Document IP ownership
• Public Information Search
• DNS Retrieval
Discovery
• Open ports
• OS fingerprint
• General exploits
•open access, password guessing
• Specific exploits
•Sendmail, DNS, SQL
False positive removal
Severity rating
Remediation advice
Report Generation
• Introduction
• Findings & Recommendations
• Appendices
Pre-Inspection: Scope
Control Areas:
What controls were covered
by the assessment?

–
–
–
–
–
IP addresses (complete,
internal/external)
Web applications
Remote access
VOIP, Telephones
Wireless

Boundaries
What were the boundaries
of the assessment?
–
–
–
–
–

Physical boundary
Logical boundary
Outsourced functions
External interfaces
Relevant systems
Rigor
To what level of rigor was
the assessment performed?
–
–
Defined
Adequate
Scope: Physical Boundaries
Scope: Logical Boundaries
External
Interfaces
Scope: Level of Rigor

Low
–

Moderate
–

Limited review, inspections, and tests.
Substantial examination, inspections, and extended tests.
High
–
Comprehensive analysis, inspections, and extended depth
and scope of test
Document and communicate level of rigor through the
adoption of a standard approach (e.g., NIST SP 800-53A,
RIIOT, etc.)
Scope: Implications

Meeting scan objective
Objective analysis of the effectiveness
of current security controls that protect
an organization’s assets.

Scan caveats
If assessor believes the scope of the assessment
is limited and may not meet the stated objective,
the report should clearly indicate this.
Scoping: Limitations

Reasonable limitations
–
Common controls assessed elsewhere

–
Control limitations – sponsor does not control
other area


Obtain report to ensure
Clearly indicate scope of assessment
Unreasonable limitations
–
Sever restrictions on rigor, methods, interfaces,
time, budget.


Clearly state limitations in report
Is it an adequate vulnerability scan?
Pre-Inspection: Objective
Objective Statement
Is the objective of the
assessment clearly stated?

–
–
–
Defined
Frequency
Driver

Restrictions
What restrictions were
placed on the assessment?
–
–

Reasonableness
Acceptance
Permissions
Were appropriate
permissions granted?
–
–
–
Granted
DOS inclusion
Data modification
inclusion
Pre-Inspection: Team

Independence
–
–

Claimed?
Adequate?
Expertise
–
Security expertise

–
Credentials (CISSP)
Audit expertise

–
Was the team performing
the assessment independent
and qualified?
Credentials (CISA)
Regulation / Business expertise (knowledge)
Team: Objectivity

Who should perform the Vulnerability Scan?
–
–
Objectivity vs. independence
Budget and other factors affecting the decision
Footprint Audit Points
Pre-Inspection
• Define Scope
• Define Objective
• Define Team
Footprint
• Document IP ownership
• Public Information Search
• DNS Retrieval
Discovery
• Open ports
• OS fingerprint
Enumeration
• General exploits
•open access, password guessing
• Specific exploits
•Sendmail, DNS, SQL
Vulnerability Assessment
False positive removal
Severity rating
Remediation advice
Report Generation
• Introduction
• Findings & Recommendations
• Appendices
Footprint: IP Ownership




Did the assessment cover all the IP addressed
identified by the system owner?
Did the assessment team independently verify the
ownership of the IP addresses?
Were any of the identified IP addresses owned by a
third party (i.e., hosting company), if so did the
assessment team obtain permission?
Did the report clearly identify IP addresses not
covered by the assessment (for example email
server not covered for continuity reasons)?
Discovery Audit Points
Pre-Inspection
• Define Scope
• Define Objective
• Define Team
Footprint
• Document IP ownership
• Public Information Search
• DNS Retrieval
Discovery
• Open ports
• OS fingerprint
Enumeration
• General exploits
•open access, password guessing
• Specific exploits
•Sendmail, DNS, SQL
Vulnerability Assessment
False positive removal
Severity rating
Remediation advice
Report Generation
• Introduction
• Findings & Recommendations
• Appendices
Discovery: Discover Interfaces

Were interfaces within the boundary and
scope completely discovered?
–
–
–
Did the assessor discover any additional
interfaces?
Did the assessment cover multiple protocols to
the same IP address? (ports?)
Did the assessment include:



VPN, IPS
Web servers, application servers, custom apps
DNS, mail servers
Discovery: Discover Information

Did the assessment team perform adequate
analysis to discover information?
–
–
–
Public information (e.g. google hack)
Internal information (FTP, file shares)
Operating systems fingerprinted
Discovery: Complete Discover

Did the assessment team ensure complete
discovery?
–
–
–
Load balancers
Virtual host (recent scan)
Wireless access points
Enumeration Audit Points
Pre-Inspection
• Define Scope
• Define Objective
• Define Team
Footprint
• Document IP ownership
• Public Information Search
• DNS Retrieval
Discovery
• Open ports
• OS fingerprint
Enumeration
• General exploits
•open access, password guessing
• Specific exploits
•Sendmail, DNS, SQL
Vulnerability Assessment
False positive removal
Severity rating
Remediation advice
Report Generation
• Introduction
• Findings & Recommendations
• Appendices
Enumeration: Determine Exploits
Did the assessment team
adequately determine exploits?

General exploits
–
–

Open access – no passwords
Password guessing and cracking
Specific exploits
–
Sendmail, DNS, SQL
Vulnerability Assessment Audit Points
Pre-Inspection
• Define Scope
• Define Objective
• Define Team
Footprint
• Document IP ownership
• Public Information Search
• DNS Retrieval
Discovery
• Open ports
• OS fingerprint
Enumeration
• General exploits
•open access, password guessing
• Specific exploits
•Sendmail, DNS, SQL
Vulnerability Assessment
False positive removal
Severity rating
Remediation advice
Report Generation
• Introduction
• Findings & Recommendations
• Appendices
Vulnerability Assessment: Determine Impact




Did the team have a process for identifying
and removing false positives?
Did the report utilize a ranking process for
found vulnerabilities?
Was the security service (confidentiality,
integrity, availability) affected indicated for
each vulnerability?
Was there a re-test? Was the final scan free
of “high” level vulnerabilities?
Report Audit Points
Pre-Inspection
• Define Scope
• Define Objective
• Define Team
Enumeration
• General exploits
•open access, password guessing
• Specific exploits
•Sendmail, DNS, SQL
Footprint
Vulnerability Assessment
Discovery
Report Generation
• Document IP ownership
• Public Information Search
• DNS Retrieval
• Open ports
• OS fingerprint
• False positive removal
• Severity rating
• Remediation advice
• Introduction
• Findings & Recommendations
• Appendices
Report: Introduction

Dates
Is the assessment
recent and relevant?
–
–

Report date. Recent?
Assessment date.
Consistent?
Method
Was the method used
appropriate?
–
–
–
Described adequately?
Meets rigor objective?
Meets compliance needs?

Findings & Remediation
Were the findings detailed,
useful, and accurate?
–
Each vulnerability





–
–
Described
Patch guidance
Rated (impact)
Ranked (order)
Organized
Rigorous enough to
meet goals?
Persistent findings?
Report: Appendices

Start and Stop Times
Do the start and stop
times match the report?
–
–

Findings
Are the findings consistent?
Match assessment date?
Adequate length?
–

Match main report and
summaries?
Remediation
Is there a remediation
for each finding?
–
Match findings?
Checklist

See Handout