ASSESSMENT
EVALUATION AND
STANDARDIZATION (AES)
Operator Skills Test (OST) Mission Guide
November 2022
U.S. Department of Homeland Security
Cybersecurity and Infrastructure Security Agency
Copyright 2022 Carnegie Mellon University.
This material is based upon work funded and supported by the Department of Homeland Security under
Contract No. FA8702-15-D-0002 with Carnegie Mellon University for the operation of the Software
Engineering Institute, a federally funded research and development center sponsored by the United States
Department of Defense.
The view, opinions, and/or findings contained in this material are those of the author(s) and should not be
construed as an official Government position, policy, or decision, unless designated by other documentation.
References herein to any specific commercial product, process, or service by trade name, trademark,
manufacturer, or otherwise, does not necessarily constitute or imply its endorsement, recommendation, or
favoring by Carnegie Mellon University or its Software Engineering Institute.
NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING
INSTITUTE MATERIAL IS FURNISHED ON AN "AS-IS" BASIS. CARNEGIE MELLON UNIVERSITY
MAKES NO WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY
MATTER INCLUDING, BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR
MERCHANTABILITY, EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL.
CARNEGIE MELLON UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH
RESPECT TO FREEDOM FROM PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT.
[DISTRIBUTION STATEMENT C] Distribution authorized to U.S. Government Agencies and their
contractors (materials intended for administrative or operational use) (determination date: 2022-05-23). Other
requests for this document shall be referred to DHS CISA.
Notice to DoD Subcontractors: This document may contain Covered Defense Information (CDI). Handling of
this information is subject to the controls identified in DFARS 252.204-7012 – SAFEGUARDING COVERED
DEFENSE INFORMATION AND CYBER INCIDENT REPORTING
Carnegie Mellon® and CERT® are registered in the U.S. Patent and Trademark Office by Carnegie Mellon
University.
DM22-0475
Operator Skills Test Mission Guide
Table of Contents
Table of Contents
1
Introduction
1
2
Assignment
2
3
Scoring
3
3.1
OST Lab
3
3.2
OST Quiz Questions
3
3.3
Candidate Evaluation Exam
3
3.4
Passing the OST Lab, OST Quiz Questions and CE Exam
3
4
Authorized Scope
4
5
Excluded Systems and Activities
5
6
Failed Exploits or Crashed Services
6
7
System Information
7
8
Tools
8
Assessment Evaluation and Standardization (AES)
[DISTRIBUTION STATEMENT C] U.S. Government Agencies and their contractors only.
ii
Operator Skills Test Mission Guide
Introduction
1 Introduction
The Operator Skills Test (OST) is a pre-training exercise for the CISA Assessment Evaluation and
Standardization (AES) program. The purpose of the test is to ensure that individual members of a
team under evaluation have the required skills to both understand the content of the training and
provide a quality assessment. Operator candidates perform this exercise individually. The exercise
tests basic penetration testing technical skills, such as enumeration, exploitation, escalation, and
pivoting within a network.
The OST includes a lab with hands-on tasks, corresponding quiz with lab questions, and a Candidate
Evaluation (CE) quiz. Candidates have three attempts, each lasting 24 hours, to complete the
exercise during a designated 30-day period. This allows candidates to identify areas for improvement
before re-attempting the OST. The AES OST Resource Guide, in Moodle, outlines tools, processes,
and resources that are relevant to the OST. Use the resource guide prior to, during, and between OST
attempts.
Assessment Evaluation and Standardization (AES)
[DISTRIBUTION STATEMENT C] U.S. Government Agencies and their contractors only.
1
Operator Skills Test Mission Guide
Assignment
2 Assignment
Your assignment is to evaluate an unknown network. The tasks include, but are not limited to, the
following:
•
mapping the network
•
performing network and web application vulnerability scans
•
exploiting vulnerabilities to gain access
•
escalating privileges
•
pivoting to systems/services unavailable from the attack network.
Evaluation is based on a quiz and the following hands-on tasks:
•
•
Quiz, which is part of the Operator Candidate Evaluation (CE) exam and indicated as such therein.
−
enumeration, specifically with Nmap and Nessus
−
web application exploitation (select the correct hash obtained from Dia)
Hands-On Tasks (task names correlate with host names)
−
Dia: web application exploitation to obtain a hash (NOT a shell)
−
Chaldene: exploitation
−
Rogers: exploitation and escalation
−
Callisto: exploitation and escalation
−
Thyone: pivoting, exploitation, and escalation
Assessment Evaluation and Standardization (AES)
[DISTRIBUTION STATEMENT C] U.S. Government Agencies and their contractors only.
2
Operator Skills Test Mission Guide
Scoring
3 Scoring
We recommend you complete the prerequisites in the following order:
1. OST Lab
2. OST Quiz Questions
3. CE Exam
3.1 OST Lab
Each vulnerable system has a local.txt and/or a proof.txt file located on the Desktop. To
receive points, you must note or otherwise capture the information within the local.txt file after
you have gained access to the system it is on and, similarly, with the proof.txt after you have
escalated privileges. You will be required to input this information into the quiz in Moodle. If the
system was compromised as a privileged user, you will find only a proof.txt file. The hands-on
portion will not score automatically. See the TopoMojo Operator Skills Test Exercise Guide for
additional information.
3.2 OST Quiz Questions
In accordance with best practices, perform your scanning first, then answer the lab questions
pertaining to ports and services based on scans performed prior to exploiting and restarting targets.
If not, your scan results will vary from the correct answers, which are based on machines in their
initial state.
IMPORTANT! Submit the Lab after you have answered all questions You must submit Lab answers for
points to be assigned. Note that Excluded Systems/Activities impact your results. See
TopoMojo Operator Skills Test Exercise Guide for additional information.
3.3 Candidate Evaluation Exam
An evaluation covering cybersecurity topics to confirm that you have the pre-requisite knowledge and
skills to succeed in course.
3.4 Passing the OST Lab, OST Quiz Questions and CE Exam
To be permitted to enroll in a requested AES course, you must pass the OST Lab, the OST Lab
Questions, and the CE Exam by meeting all the following criteria:
•
OST Lab – exploit successfully a minimum of 3 of 5 boxes using the instructions in this guide.
•
OST Lab Questions – achieve a minimum score of 70%.
•
AES HVA or RVA CE Exam – achieve a minimum score of 70%.
Assessment Evaluation and Standardization (AES)
[DISTRIBUTION STATEMENT C] U.S. Government Agencies and their contractors only.
3
Operator Skills Test Mission Guide
Authorized Scope
4 Authorized Scope
You will operate from the 10.20.150.0/24 subnet. The 10.20.150.0/24 subnet is not in scope for
testing. Your targets are in the 10.20.160.0/24 subnet as listed in the table below.
Table 1 - Authorized Testing Sites and Services
IP Addresses
In-Scope IPs
Excluded IPs
Services
Location
10.20.160.0/24
10.20.160.2-255
10.20.160.1
Penetration Testing, Web
Application Assessment
Internal
Assessment Evaluation and Standardization (AES)
[DISTRIBUTION STATEMENT C] U.S. Government Agencies and their contractors only.
4
Operator Skills Test Mission Guide
Excluded Systems and Activities
5 Excluded Systems and Activities
Scanning, probing, or otherwise testing the following systems is not authorized:
•
10.20.150.0/24
•
10.20.160.1
Scanning the following system using Nessus is not authorized:
•
10.20.160.41
Brute forcing of passwords for any discovered logins or accounts is not authorized and should not be
performed.
Assessment Evaluation and Standardization (AES)
[DISTRIBUTION STATEMENT C] U.S. Government Agencies and their contractors only.
5
Operator Skills Test Mission Guide
Failed Exploits or Crashed Services
6 Failed Exploits or Crashed Services
If a failed exploit attempt crashes a service, restart the lab. As in a real penetration test, it is
important to understand the consequences of your action before executing potentially harmful code
or commands. If restarting the lab does not correct the problem, send an email to the appropriate
address:
•
If you are registered for an AES HVA course, email aes-hva@cert.org.
•
If you are registered for an AES RVA course, email aes-rva@cert.org.
Note: Emails are answered during regular business hours. Emails must also include the eight-digit
support code within the email. This code is used to help identify the student’s environment.
A proctor is not available to troubleshoot your targets if they become unresponsive or do not behave
as expected. Therefore, take detailed notes to avoid making the same mistakes on your remaining
attempt(s). If you experience unexpected behavior during the exercise unrelated to an action you
have taken, or if you are unsure how you caused the unexpected behavior, send details of your issue
in an email to one of the addresses above.
Assessment Evaluation and Standardization (AES)
[DISTRIBUTION STATEMENT C] U.S. Government Agencies and their contractors only.
6
Operator Skills Test Mission Guide
System Information
7 System Information
You will be operating from a virtual Kali Linux machine through the TopoMojo platform in your
browser, so you do not need to set up anything locally. The Kali machine does not have access to the
internet to assure safe containment of scanning and exploitation activity. However, the Kali machine
is pre-loaded with all tools and resources needed to achieve exploitation. Without internet access,
you may need to exercise your ability to find alternate routes using the resources you have. It is
necessary to give the Kali machine the right Static IP address. Consult the AES OST Resource Guide
in Moodle and online resources outside of the virtual environment while you complete the OST.
Note: Nessus can be accessed at https://127.0.0.1:8834 after the Nessus service is started
(‘service nessusd start’).
Host or Application
Username
Password
Kali
root
toor
Nessus
root
toor
Note: The Kali machine should have an IP address of 10.20.150.101 already configured. If it is not
the case, you will have to statically assign the IP address by editing /etc/network/interfaces.
Step by step process can be found in the AES Topomojo OST Exercise Guide.
Assessment Evaluation and Standardization (AES)
[DISTRIBUTION STATEMENT C] U.S. Government Agencies and their contractors only.
7
Operator Skills Test Mission Guide
Tools
8 Tools
The table below lists the primary tools. Use other tools as needed.
Service
Tool
Port scans
Nmap
Network vulnerability scans
Nessus
Web application vulnerability scans
Vega
Enumeration
PowerSploit
Assessment Evaluation and Standardization (AES)
[DISTRIBUTION STATEMENT C] U.S. Government Agencies and their contractors only.
8