Educating System Testers in Vulnerability Analysis:
Laboratory Development and Deployment
Leonardo A. Martucci, Hans Hedbom, Stefan Lindskog, and
Simone Fischer-Hübner
Department of Computer Science
Karlstad University
SWEDEN
Outline
Introduction and background
Course overview
Course content
Hands-on assignments
Evaluation and lessons learned
Conclusion
Errata
2
WECS'7
Introduction
The constantly growing number of security
vulnerabilities
threats
incidents
has led to increased investments in the development of
more secure systems
The lack of security functionality and assurance may
result in high costs
Vulnerability analysis (VA) is an important means for
improving security assurance of IT systems during test
and integration phases
3
WECS'7
Background
A large telecom company decided to increase their
efforts in VA by educating their software testers
They decided to outsource the education and
training of its testers
A compact (3 days) VA course was developed at
our department
The course has been held 3 times during 2005 for
a total of 45 participants
4
WECS'7
Course Overview
The emphasis of the course is on practical hands-on
assignments
The course is aimed for software testers with
little or no security experience
extensive knowledge in software testing
The topics included in the course is based on a
preliminary list of topics specified by the contractor
A set of laboratory assignments were derived from this list
Approximately 30-40% covers theoretical aspects and the
rest is used for practical assignments
5
WECS'7
Course Content
The course content is divided into 4 blocks:
Introduction to computer and network security
Motivation, evaluation criteria, security standards, risk analysis, and
ethics
Computer and network security protocols and tools
Cryptography, IPSec, SSH, SSL/TLS, PKI, VPNs, IDSs, firewalls, and a
set of laboratory assignments
Vulnerability analysis
The four steps of VA: (1) reconnaissance, (2) research and planning, (3)
attack mounting, and (4) assessment
Known vulnerabilities, reconnaissance tools and information
gathering
Common host attacks, malicious code, node hardening, and several
practical laboratory assignments
6
WECS'7
Hands-on Assignments
The following laboratory assignments are included:
password cracking
testing for randomness
firewall
black box testing
network analyzing (and ARP spoofing)
port scanning
node hardening
security scanning
Final project
Putting it all together (i.e., “from grain to bread”)
7
WECS'7
Ethical Rules
The participants were requested to follow the following
ethical rules:
Do not experiment with VA-tools without explicit permission of
an authorized party
Do not pass on/publish material, tools, and vulnerabilities to
unauthorized parties
Do not use your technical skills in criminal or ethically
questionable activities
Always report flaws to vendors/developers first
Software tools provided in this course must only be used in a
laboratory environment and on laboratory computers
8
WECS'7
The Laboratory Environment
The laboratory was prepared for 20 students working in
pairs
Each pair have their own workstation
Each workstation
Was dual boot – Windows XP and Feodora Core 3 Linux
Equipped with an Ethernet NIC
The laboratory was also configured with two servers
One running Windows 2000 Server
The other running Feodora Core 3 Linux
The servers were in some assignments the target
9
WECS'7
Password Cracking
Goal
To show that weak passwords could be a serious threat
Running the assignment
The password cracking tool “John the Ripper” was used to
detect weak passwords on their own workstation running Linux
Some easy to break passwords were introduced in the password
file
Knowledge obtained
The participants have tested a password cracking tool to identify
weak passwords
10
WECS'7
Testing for Randomness
Goal
To educate the participants in how to identify non-random
properties in sequences produced by a pseudo random number
generator (PRNG)
Running the assignment
The NIST statistical test suite was used to evaluate outputs from
different PRNGs
A short introduction on hypothesis testing was needed in order
for the participants to evaluate the output from the tool
Knowledge obtained
The participants have learned that:
good PRNGs are a crucial cryptographic primitive
automatic tools exist to validate PRNGs
11
WECS'7
Firewall
Goal
To provide hands-on experience on how firewall rules in Linux
using ipTables can be used
Running the assignment
The participants were
asked to write firewall rules
for the setup in the figure in
order to implement a given
policy
Knowledge obtained
The participants have the knowledge to write, read, understand,
verify and evaluate firewall rules
12
WECS'7
Black Box Testing
Goal
To learn how a protocol implementation can be evaluated using
a black box testing method
Running the assignment
The PROTOS tool was used
to evaluate the SNMP protocol
in a CISCO 1005 router
A ready-made test suite to perform
a DoS attack was used
Knowledge obtained
The participants have learned that black box testing using
automatic tools can be used to evaluate implementations of
communication protocols
13
WECS'7
Network Analyzing (and ARP Spoofing)
Goal
To show how easy it is to capture network traffic in a
LAN using Ethereal
Running the assignment
Ethereal was used to capture
a password sent over the
network using TELNET
Knowledge obtained
The participants have learned how to manage a
network analyzer to capture network traffic
14
WECS'7
Port Scanning
Goal
To demonstrate how port scanners can be used to find open
ports in a networked computer
Running the assignment
The participants were asked to gather information about open
ports on the two servers using the Network MAPper (NMAP) in
Linux
Knowledge obtained
The participants have learned how to use a port scanner to find
unexpected open ports in a product before deployment
15
WECS'7
Node Hardening
Goal
To educate the participants on how to increase the security of nodes by
turning off unnecessary services
restricting the rights of necessary services
verifying that used software uses the latest patches
Running the assignment
The Bastille tool was used
When running Bastille, a large set of questions are asked on how the user
would like the node to be configured and after that automatically configure
the system according to the answers
Knowledge obtained
The participants have learned the importance of correct configurations and
to handle a node hardening tool
16
WECS'7
Security Scanning
Goal
To show how to use security scanners in order to automatically
scan the system for known vulnerabilities
Running the assignment
Two unpatched servers running Windows 2000 Server and
Fedora Core 3 Linux were acting as targets
Both the Internet Scanner (IS) and Nessus were used as
scanners
Neither the configuration nor the IP addresses of the servers
were known to the students
Knowledge obtained
The participants have learned that security scanners are tools
that can assist the testers in the verification process
17
WECS'7
Putting it all Together
Goal
To let the participants conduct a full VA of a target with limited resources and
time (<8 hours).
Running the assignment
The assignment was conducted in groups of 4 students
Each group had two workstations and one server that was the target of
evaluation
The group was given a requirement specification describing the role of the
server and its security requirements
The exercise was to find out what has to be done to fulfill the requirements,
perform the necessary changes and verify the result
Knowledge obtained
The participants have gained a better understanding on how to perform a fullscale VA
18
WECS'7
Evaluation and Lessons Learned
After each course instance, the participants have been asked to fill
in a questionnaire used to evaluate the course
Based on the answers, the following conclusion can be drawn
The most popular assignments have been:
Security scanning, port scanning, and node hardening
The least interesting assignments have been:
Testing for randomness and firewall
Each participant has either been satisfied or very satisfied with the course
We have also noticed that having a system administrator available
during the course would greatly reduce the burden on the teachers
19
WECS'7
Concluding Remarks
A vulnerability analysis (VA) course aimed for software
testers is described in the paper
The focus is on the various laboratory assignments provided
within the course
All participants have either been satisfied or very satisfied
with the course and we are convinced that the course has
significantly raised their awareness concerning security and
VA
An investigation of how the participants use their knowledge
in VA will be performed during spring 2006
Three new instances of the course are scheduled in 2006
20
WECS'7
Errata
Page 2, third sentence in second paragraph, i.e.:
“Students from an applied computer security course
were engaged and trained to attack a target system
and evaluate its security [2].”
Delete “and trained” in the sentence.
21
WECS'7