Homeland Security Exercise and
Evaluation Program (HSEEP)
Rebecca L. Siceloff
For Official Use Only
1
An Introduction to Exercises
Assess and validate policies, plans, procedures, training,
equipment, assumptions, and interagency agreements
Clarify roles and responsibilities
Improve interagency coordination and communications
Identify gaps
Measure performance
Identify opportunities for improvement
For Official Use Only
2
HSEEP Cycle
For Official Use Only
3
The Building Block Approach
Key Concept:
Exercises should be planned in a
cycle that increases in complexity.
Each successive exercise should
build on the scale and experience
of the previous exercise.
For Official Use Only
4
Discussion-based Exercises
Provide a forum for discussing
or developing plans,
agreements, training and
procedures
Less complicated than
operations-based exercises
Focus on strategic, policyoriented issues
Include seminars, workshops,
tabletops, and games
Do not involve deployment of
resources
For Official Use Only
5
Seminars
An informal discussion-based exercise led by a presenter or a
facilitator, used to teach or orientate participants.
Goals
Orientate participants to new or existing plans, policies, or procedures
Research or assess interagency capabilities or inter-jurisdictional
operations
Construct a common framework of understanding
Conduct Characteristics
Casual atmosphere
Minimal time constraints
Lecture-based
For Official Use Only
6
Workshops
A formal discussion-based exercise led by a facilitator, used
to build or achieve a product.
Goals
Develop new ideas, policies, or processes
Develop a written product as a group in coordinated activities
Obtain consensus
Collect or share information
Conduct Characteristics
Involves more participant discussion than lecture-based seminar
Uses break-out sessions to explore parts of issue with smaller groups
For Official Use Only
7
Tabletop Exercises
A tabletop exercise (TTX) involves senior staff, elected or appointed
officials, or other key personnel in an informal group discussion centered
on a hypothetical scenario.
Goals
Identify strengths and shortfalls
Enhance understanding of new concepts
Identify strengths and shortfalls
Conduct Characteristics
Requires an experienced facilitator
In-depth discussion
Slow-paced problem solving
For Official Use Only
8
Games
Key Concept: A game is a simulation of operations using rules, data,
and procedures designed to depict an actual or assumed real-life
situation.
Goals
Explore the processes and consequences of decision-making
Conduct “what-if” analysis of existing plans
Test existing and potential strategies
Conduct Characteristics
Does not involve the use of actual resources
Often involves two or more teams
Includes models/ simulations of increasing complexity
For Official Use Only
9
Operations-based Exercises
Involve deployment of resources and personnel
More complex than discussion-based exercises
Require execution of plans, policies, procedures
Clarify roles and responsibilities
Improve individual and team performances
Include drills and both functional and full-scale exercises
For Official Use Only
10
Drills
A drill is a supervised activity that tests a specific operation or function of
a single agency.
Goals
Gain training on new equipment
Test new procedures
Practice and maintain skills
Prepare for more complex exercises
Conduct Characteristics
Immediate feedback
Realistic but isolated environment
For Official Use Only
11
Functional Exercises
A functional exercise (FE) is a single or multi-agency activity designed to
evaluate capabilities and multiple functions using simulated response.
Goals
Evaluate management of Emergency Operations Centers, command
posts, and headquarters
Assess the adequacy of response plans and resources
Conduct Characteristics:
Simulated deployment of resources and personnel
Rapid problem solving
Highly stressful environment
For Official Use Only
12
Full-Scale Exercises
A full-scale exercise (FSE) is a high-stress multi-agency, multijurisdictional activity involving actual deployment of resources in a
coordinated response, as if a real incident had occurred.
Goals
Assess plans and procedures under crisis
conditions
Evaluate coordinated responses under crisis
conditions
Conduct Characteristics:
Mobilization of units, personnel, and equipment
Stressful, realistic environment
Scripted exercise scenario
For Official Use Only
13
HSEEP Cycle: Project Management
For Official Use Only
14
Exercise Project Management
Project managers are responsible for the design,
development, and execution of a specific exercise, followed
by evaluation and improvement planning.
Develop a project management timeline
Establish project milestones
Identify the exercise planning team
Schedule planning conferences
For Official Use Only
15
Exercise Timeline
In project management, the Exercise Timeline identifies key
conferences and tasks for an individual exercise.
Exercise project managers build timelines to include:
A schedule of key conferences and milestones
Planning team task assignments
Generally, timelines for discussion-based exercises are shorter and
have fewer tasks than timelines for operations-based exercises.
For Official Use Only
16
Exercise Planning Team
Every exercise requires an Exercise Planning Team – the
core group responsible for the design, development, conduct,
and evaluation of an exercise.
Determines exercise objectives
Creates the scenario
Develops exercise documentation
Conducts pre-exercise briefing and training sessions
Composed of members of key participating agencies
Is manageable and flexible in size, depending on exercise type and
scope
For Official Use Only
17
Defining Design and Development
Exercise design includes:
Assessing exercise needs
Defining the scope of the exercise
Writing a statement of purpose
Defining exercise objectives
Creating a scenario for the exercise
Exercise development includes:
Creating exercise documentation
Arranging logistics, actors, and safety
Coordinating participants and media
Other supporting planning tasks
For Official Use Only
18
Purpose
An exercise's purpose states the desired goal of the exercise.
Governs the selection of the objectives
Clarifies why the exercise is being conducted
Communicates plans to the media and community leaders
For Official Use Only
19
Scope
Scope establishes realistic limits on the exercise.
Scope includes:
Type of exercise
Location of exercise
Agencies/Emergency Support Functions
Participants
Type of emergency
Capabilities to be exercised
For Official Use Only
20
Objectives
An objective is a description of the performance you expect
from participants. It conveys specifically how the exercise
should achieve its purpose.
Objectives
Define performance/actions expected from participants
Defines the specific conditions under which those actions will
be performed
Provide framework for scenario development
Provide exercise evaluation criteria
.
For Official Use Only
21
Scenario
A scenario is the storyline that drives an exercise.
A scenario includes:
General context or comprehensive story
Technical details of story's conditions and events
Conditions for assessing capabilities
Scenarios should be:
Threat-based and performance-based
Realistic
Challenging
For Official Use Only
22
Exercise Documentation
Exercise documents are the most tangible
elements of design and development.
Different exercise types require different
documentation, ranging from simple sign-in sheets
to media releases and exercise evaluation guides.
Basic exercise documentation includes:
Situation Manual
Exercise Plan
Controller Evaluator Handbook
Master Scenario Events List
Exercise Evaluation Guides
For Official Use Only
23
HSEEP Cycle: Project Management
For Official Use Only
24
Discussion-Based Conduct
Discussion-based exercise conduct involves:
Site setup
Guided presentation
Facilitated/ moderated discussion
Wrap-up activities (including a hot wash)
The majority of discussion-based “action” comes from moderated
participant discussions, either as a whole group or in break-out
sessions. Moderators and facilitators are essential to keeping the
discussions on track to meet exercise objectives.
For Official Use Only
25
Operations-Based Conduct
Conduct of operations-based exercises involves:
Site setup
Exercise briefings
Exercise play
Wrap-up activities
For Official Use Only
26
Evaluation
Key Concept: Exercise evaluation assesses how well the
exercise objectives were achieved. It also identifies
opportunities for improvement.
Evaluators accomplish this by:
Observing the exercise and collecting supporting data.
Gauging performance against expected outcomes.
Determining what changes are needed to ensure desired outcomes.
Evaluation is the yardstick by which an organization measures its
capabilities. Good evaluations result in suggestions for filling and
bridging capability gaps or making needed improvements.
For Official Use Only
27
Exercise Evaluation Guides
Exercise Evaluation guides (EEGs) provide structured evaluation
measures of participant conduct, listing critical activities and tasks to be
completed during an exercise.
Three distinct levels of analysis:
Task-level
Activity-level
Capability-level
For Official Use Only
28
After Action Reports
Prepared by the members of the planning team and evaluation
team, the After Action Report (AAR) provides participant
officials with feedback on the exercise's results and suggests
recommendations for improvement.
After Action Reports:
Prepared after every exercise type.
Summarize what happened during the exercise.
Provide feedback to participants on their performance.
Recommend improvements for better preparedness.
For Official Use Only
29
The After Action Conference
The After Action Conference is a forum for exercise participants to
discuss the draft AAR and identify action items for the Improvement Plan.
The conference should address:
Specific improvement actions that agencies
can take
Feedback and validation of observations and
recommendations
Key lessons learned from the exercise
experience
Distribution of AAR and IP
For Official Use Only
30
Improvement Planning Process
The Improvement Planning Process is the means for
converting recommendations from the After Action Report
(AAR) into measurable steps that, when implemented, lead to
improved response capabilities.
By focusing on performance and how actual outcomes
differ from expectations, public officials and exercise
planners can:
Target their improvement resources more effectively
Modify their exercise programs before having to respond to a real
incident
For Official Use Only
31
The Improvement Plan
Organizations execute the improvement planning process
using a tool called the Improvement Plan (IP).
The IP identifies:
Actions to address each AAR recommendation.
Who will be responsible for taking each action.
A timeline for completion of those actions.
Once recommendations and action items have been identified,
organizations should ensure that each item is tracked to
completion and improvements are implemented.
For Official Use Only
32
The Improvement Plan Matrix
The Improvement Plan Matrix is a spreadsheet template for
completing the IP in a condensed form.
The Improvement Plan Matrix finalizes the AAR/IP
It includes:
Tasks
Recommendations
Improvement Actions
Responsible Party (Parties)
Completion Date
For Official Use Only
33
HSEEP Cycle
For Official Use Only
34
Program Management
Exercise program management involves developing and
executing an exercise program.
An exercise program provides the administration,
supporting resources, and strategic goals for an
organization's exercise efforts
This includes:
Multiyear training and exercise program planning.
Budgeting and grant writing.
Planning and executing individual exercises.
Tracking improvements.
For Official Use Only
35
HSEEP Cycle: Program Management
For Official Use Only
36
Improvement Planning Workshop (IPW)
The Improvement Planning Workshop is a Plan is a forum
to focus on previous year’s After Action Reports (AARs)
and Improvement Plans (IPs) for exercises and real-world
events
Allows for the development of a more cohesive Training and
Exercise Plan
Participants can review priorities with their colleagues to
ensure these are the correct priorities before the Training
and Exercise Planning Workshop (TEPW)
Gain buy-in across department before the TEPW
Better prepare TEPW participants
For Official Use Only
37
HSEEP Cycle: Program Management
For Official Use Only
38
Multi-Year Training & Exercise Plan
The Multiyear Training & Exercise Plan is essential to
managing an exercise program. The plan is a document
that outlines a program's long-term strategy for building
capabilities through exercises and training.
Takes stock of prioritized capabilities
Lays out long-term program goals and objectives
Develops a mix of training and exercises to meet goals and objectives
Determines what training is needed as a prerequisite to planned
exercises
Sets a multi-year schedule of training events and exercises
For Official Use Only
39
Multi-Year Training & Exercise
Schedule
Exercise coordination is done through the Multi-Year
Training and Exercise Schedule. This schedule lays out a
long-term schedule of planned and potential training dates
and exercises.
The Multi-Year Training and Exercise Schedule is used to:
Avoid duplicating their efforts
Combine exercises and ensure the exercises don't conflict
Combine training and ensure training does not conflict
Optimize and combine funding where possible
Prevent "over" training and exercising
For Official Use Only
40
Training & Exercise Planning
Workshop (TEPW)
The Multi-Year Training and Exercise Plan and the MultiYear Training and Exercise Schedule are created at a
Training and Exercise Planning Workshop (TEPW).
Involves various stakeholders from both exercise and training
community
Is conducted annually
Is one to two days long
Allows agency representatives from numerous training and exercise
programs to coordinate activities
For Official Use Only
41
HSEEP Cycle
For Official Use Only
42
For Official Use Only
43
0
