HSEEP:
All You Need to Know in 90 Minutes or
Less….
Region 2 Conference
June 2, 2010
Amelia Muccio, NJPCA
Dave Weidner, HCANJ
Today’s Agenda
• HSEEP Compliance
–
–
–
–
TEPW & Multiyear Training and Exercise Plan
Conferences and Documents
AAR/IP
Track IP
• Practical Applications of HSEEP
HSEEP
• The Homeland Security Exercise and Evaluation Program
(HSEEP) is a capabilities- and performance-based
exercise program that was developed to provide common
exercise policy and program guidance that constitutes a
national standard for exercises.
• HSEEP is designed to be adaptable to any exercise
program, regardless of the nature and composition of its
sponsoring agency or organization, and to the full
spectrum of exercise scopes and scenarios.
• This includes consistent terminology, design process,
evaluation tools, and documentation standards.
• HSEEP reflects community best practices as well as
lessons learned from previous and existing exercise
programs.
HSEEP Compliance (#1)
• Conduct an annual Training and Exercise Planning Workshop
(TEPW), and maintain a Multiyear Training and Exercise Plan
(TEP).
• An annual TEPW provides an opportunity to develop, review, or
update an entity’s Multiyear TEP.
• The TEPW also provides a forum for determining how an entity will
execute its multiyear plan in a given year.
• The purpose of the TEPW and the Multiyear TEP is to translate
strategic goals and priorities into specific training and exercise
activities and to coordinate and deconflict all training and exercise
activities on a schedule.
• While all exercises conducted by an entity are not required to be
included in its Multiyear TEP, the entity should follow the guidance
and priorities established during its TEPW.
TEPW
• Discuss priorities from HSEEP, NJDHSS,
previous year’s AAR/IP
• Translate priorities and their associated
capabilities into training and exercise
activities
• Develop Multi-Year Training and Exercise
Plan
• Coordinate training and exercise schedules
• Improve coordination between training and
exercises
Capabilities-Based Planning
• Target Capabilities List (TCL)
• Derived from the Universal Task List
(UTL)
• In the Multi-Year Training and Exercise
Plan, capabilities will be identified that
support and are necessary to successfully
implement the chosen NJPCA/FQHC
priorities
• HSPD 8NPGNPSUTLTCL
Capabilities Assessment
• Identify capabilities that need improving
• Look at what capabilities are needed to fill
shortfalls and gaps in New Jersey
• Decide what tasks need to be accomplished
to achieve the capabilities
• Design exercises that can improve the
ability to complete the tasks
• WHAT ROLE DOES/CAN FQHCs
SERVE?
National Priorities
• Reflect a limited number of cross-cutting
initiatives and critical capabilities that
should drive near term planning and
resource allocation efforts
• Overarching
• Capability-Specific
Overarching
• NIMS/NRF
– Coordinated and effective response by an
appropriate combination of Fed, State, local,
tribal, private and NGOs
• Expanded Regional Collaboration
– Regional approach to building capabilities
• NIPP
– Identifying Critical Infrastructure/Key
Resource (CI/KR), assessing risk, prioritizing,
implementing protective programs and
measuring effectiveness
Capability-Specific
• Strengthen information sharing and
collaboration
• Strengthen interoperable communications
• Strengthen CBRNE detection, response,
and decon
• Strengthen medical surge and mass
prophylaxis
• Citizen protection
State/Urban Area Priorities
• State and Urban Areas should consider
National priorities and should consider
collaboration across disciplines,
jurisdictions, and agencies when describing
the strategies, goals, and objectives within
the framework of the mission areas
Multi-Year Training and Exercise Plan
•
•
•
•
•
•
•
State/Urban Area Priorities
National Priorities
Improvement Plan
Associated Target Capabilities
Training
Exercises
Schedule
Training
• For each priority, list type of training to
meet the priority and achieve the associated
capabilities
• Determine the training that is needed as a
prerequisite to planned exercises and to
satisfy prior year IP
• Ensure a building-block approach
Exercises
• For each priority, list the type of exercise to
meet the priority and achieve the associated
capabilities
• Determine the type of exercise that will
validate the training received
• Ensure a building-block approach to
exercises
Building-Block Approach
•
•
•
•
•
•
•
Seminar (Discussion)
Workshop (Discussion)
Tabletop Exercise (TTX) (Discussion)
Games (Discussion)
Drills (Operations)
Functional Exercises (FXE) (Operations)
Full Scale Exercises (FSE) (Operations)
Multi-Year Training and Exercise Plan Schedule
• Cycle, mix and range of training and
exercise activities
• Cycle of activity employs increasing
degrees of complexity and ensures
continuous improvement of capabilities
• Combination of training courses and
exercise types that accomplish priorities
• TTXFXEFSE
• WorkshopDrillFSE
• SeminarWorkshopTTX
HSEEP Compliance #2
• Plan and conduct exercises in accordance with the
guidelines set forth in HSEEP policy.
• HSEEP policy guidance includes an overview of exercise
planning and conduct.
• Specific areas for compliance include the use of various
types of planning conferences and exercise
documentation.
• The number of conferences and types of documentation
required are flexible and depend on the full scope of the
exercise being completed.
• HSEEP also provides sample documents for all potential
presentations and manuals for all types of exercises.
Exercise Planning Team
• One of the most important factors for a successful exercise
is skilled planning and conduct by the exercise planning
team.
• The exercise planning team oversees, and is ultimately
responsible for, exercise foundation, design, development,
conduct, and evaluation.
• The team determines exercise objectives, tailors the
scenario to meet the exercising entity’s needs, and
develops documentation used in evaluation, control, and
simulation.
• Planning team members also help with developing and
distributing pre-exercise materials and conducting
exercise planning conferences, briefings, and training
sessions.
Exercise Planning Team
• Because planning team members are highly
involved in the exercise, they are ideal
selections for exercise facilitator,
controller, and/or evaluator positions.
• The exercise planning team should be of
manageable size yet represent the full range
of participating entities, as well as other
relevant stakeholder organizations.
Exercise Planning Team Structure
Planning Conferences: Concept and Objectives
Meeting
• A Concept and Objectives (C&O) Meeting is the
formal beginning of the planning process. It is
held to identify the type, scope, objectives, and
purpose of the exercise.
• Depending on the scope of the exercise, the C&O
Meeting can range from 2 to 4 hours.
C & O Objectives
• Exercise purpose
• Proposed exercise scenario, capabilities, tasks, and
objectives
• Available exercise resources
• Proposed exercise location, date, and duration
• Exercise planning team and exercise participants
• Exercise assumptions and artificialities
• Exercise control and evaluation
• Exercise security organization and structure
• Local issues, concerns, and sensitivities
• Exercise logistics
Initial Planning Conference (IPC)
• The IPC marks the beginning of the exercise development
phase.
• Unless a separate C&O Meeting is conducted, the IPC is
typically the first official step in the planning process.
• Its purpose is to determine exercise scope by gathering:
input from the exercise planning team; design
requirements and conditions (e.g., assumptions and
artificialities); objectives; extent of play; and scenario
variables (e.g., time, location, hazard selection).
• The IPC is also used to develop exercise documentation
by obtaining the planning team’s input on exercise
location, schedule, duration, and other relevant details.
IPC
• During the IPC, exercise planning team members are
assigned responsibility for activities associated with
designing and developing exercise documents—such as
the Master Scenario Events List (MSEL) and the Situation
Manual (SitMan), which are described later in this
volume—and logistics, such as scene management and
personnel.
• In addition to conducting the conference, the exercise
planning team gathers appropriate photographs and audio
recordings to enhance the realism and informational value
of the final document(s) and/or multimedia presentation(s)
presented during the exercise.
• Depending on the scope of the exercise, the IPC can range
from 3 to 6 hours.
IPC Objectives
• Understanding the rationale for exercise development
• Ensuring clearly defined and measurable capabilities, tasks, and
objectives
• Incorporating community emergency operations plans (EOPs),
memoranda of agreement (MOA), participating agency standard
operating procedures (SOPs), and/or other relevant policy into the
exercise design
• Identifying local issues, concerns, or sensitivities
• Determining the extent of play for each participating entity by
establishing what each entity will demonstrate and be evaluated on at
the exercise, allowing for appropriate logistical needs to be arranged
in order to support those activities
• Ensuring that exercise planners consider themselves trusted agents
and understand that, in most cases, they will participate as facilitators,
controllers, or evaluators (rather than as players)
IPC Objectives
• Choosing subjects for photographs and/or audio/visual (A/V)
recordings to incorporate into exercise documents and multimedia
presentations (to enhance realism)
• Deciding whether or not to record exercise proceedings (audio or
video)
• Determining the optimum duration of the exercise
• Ensuring that exercise planners understand that the exercise is
conducted in a no-fault environment intended to validate plans and
procedures and identify problems and potential solutions
• Selecting or customizing the appropriate Exercise Evaluation Guides
(EEGs) to determine whether or not exercise capabilities, tasks, and
objectives were achieved and to allow participants to provide
feedback
• Reaching a consensus regarding the date, time, and location for the
next conference
IPC Outcomes
• A planning schedule
• Clearly defined, obtainable, and measurable
capabilities, tasks, and objectives
• Identified exercise scenario variables (e.g., threat
scenario, scope of hazard, venue, conditions)
• A list of exercise participants
• Identification and availability of SMEs and
presenters, as necessary, for scenario vetting
and/or expert evaluation
• Determination of the best communication method
among exercise planning team members
IPC Outcomes
• A list of which exercise documents and presentations must
be employed, and assignments for drafting each
• Availability of all source documents (e.g., policies, plans,
procedures) needed to draft exercise documents and
presentations
• Clearly identified and assigned responsibility for exercise
logistical issues (e.g., registration, badges, invitations)
• A list of established dates for completion of corrective
actions and responsibilities
• A list of critical activities for the next planning conference
• An agreed upon date, time, and location for the next
conference and the actual exercise
Mid-Term Planning Conferences (MPCs)
• Mid-Term Planning Conferences (MPCs) are typically
used in (FEs) and FSEs.
• MPCs provide additional opportunities to settle logistical
and organizational issues that may arise during planning.
• The MPC is a full day working session to discuss exercise
organization and staffing concepts, scenario and timeline
development, scheduling, logistics, and administrative
requirements.
• Review draft documentation (Exercise Plan [ExPlan],
Controller and Evaluator [C/E] Handbook, MSEL).
• At the conclusion of the MPC, selected planners should
conduct a walkthrough of the proposed exercise site.
• The second half of the MPC should be devoted to
developing the MSEL.
Final Planning Conference (FPC)
• The FPC is the final forum for reviewing exercise
processes and procedures.
• Prior to the FPC, the exercise planning team receives final
drafts of all exercise materials.
• No major changes to the design or scope of the exercise,
or its supporting documentation, should take place at the
FPC.
• The FPC ensures that all logistical requirements have been
met, all outstanding issues have been identified and
resolved, and all exercise products are ready for printing.
• Generally, the FPC is a half-day conference for
discussion-based exercises and a full day for operationsbased exercises.
FPC Outcomes
• Attendees have a clear understanding of—
and give final approval for—exercise
processes and procedures.
• Exercise documents and materials for
production are approved.
• Last-minute issues are identified and
resolved.
• Logistical elements, including A/V
equipment, room configuration and setup,
refreshments, and schedule, are confirmed.
Scenario
• A scenario consists of three basic elements:
(1) the general context or comprehensive
story; (2) the conditions that allow players
to demonstrate proficiency and competency
in meeting the exercise capabilities, tasks,
and objectives; and (3) the technical details
necessary to accurately depict scenario
conditions and events.
Scenario
• A scenario is an outline or a model of the simulated
sequence of events for the exercise.
• It can be written as a narrative or depicted by an event
timeline.
• For a discussion-based exercise, a scenario provides the
backdrop that drives participant discussion, and it is
contained in a Situation Manual (SitMan).
• For an operations-based exercise, a scenario provides
background information on the incident catalyst(s) of the
exercise—the overall scenario is provided in the
Controller and Evaluator (C/E) Handbook, and specific
scenario events are contained in the Master Scenario
Events List (MSEL).
Exercise Documentation
Exercise Evaluation Guidelines (EEGs)
• The HSEEP series of tools includes EEGs to help
evaluators collect and interpret relevant exercise
observations.
• These sets of documents outline and provide guidance on
assessing the tasks and activities to be accomplished for
each capability being validated by an exercise.
• EEGs were developed for use by experienced exercise
evaluators and by practitioners who are subject matter
experts (SMEs).
• Each EEG provides evaluators with information on what
they should expect to see demonstrated or hear discussed,
space to record their observations, and criteria to consider
after the exercise (as the first step in the analysis process).
Situation Manual (SitMan)
• SitMans are usually provided for discussionbased exercises, especially tabletop exercises
(TTXs), as the core documentation that provides
the textual background for a multimedia,
facilitated exercise.
• The SitMan supports the scenario narrative and
allows participants to read along while watching
the multimedia events unfold.
• All participants (i.e., players, facilitators,
evaluators, and observers) should receive SitMans
at the beginning of the exercise.
SitMan
• The SitMan’s introduction provides an overview of the
exercise—including scope, capabilities, tasks and
objectives, structure, rules, and conduct—as well as an
exercise agenda.
• The next section of the SitMan is the scenario itself, which
is divided up into distinct modules.
• Modules provide the basic structure of the exercise and
are chronologically sequenced.
• Each module represents a specific time segment of the
overall scenario—pre-incident warning, notification,
response, or recovery—selected based on exercise
objectives and scenario requirements.
SitMan
•
•
•
•
•
•
•
•
•
•
Introduction
Schedule of events
Exercise purpose, scope, capabilities, tasks, and objectives
Exercise structure (i.e., order of the modules)
Instructions for facilitators, players, and observers
Exercise assumptions and artificialities
Exercise rules
Exercise scenario background
Discussion questions and key issues
Reference appendices with relevant supporting
information
• Relevant documents regarding plans, SOPs, etc.; and -a
list of reference terms
Multimedia Presentation
• Multimedia presentations are often used for discussion-based
exercises to provide the general scenario.
• They are given at the start of the exercise (StartEx) and support (both
visually and with audio) the written documentation, the SitMan.
• Participants read the written material while watching or listening to
the presentation. The presentation itself should concisely summarize
information contained in the written documentation.
• This presentation typically contains, at a minimum, the following
information:
• Introduction
• Background/history on the threat and the scenario
• Exercise capabilities, tasks, and objectives
• Exercise play rules and administrative information
Tabletop Exercises (TTX)
• Key staff, decision makers, and elected and appointed
officials are typical participants in a tabletop exercise
(TTX).
• This type of exercise is generally held in an informal
setting intended to generate discussion of various issues
regarding a hypothetical, simulated emergency incident.
• TTXs can be used to enhance general awareness, validate
plans and procedures, and/or assess the types of systems
needed to guide prevention of, protection from, response
to, and recovery from a defined incident.
• Generally, TTXs are aimed at facilitating concept
understanding, identifying strengths and weaknesses,
and/or achieving changes in attitudes.
TTXs
• TTXs may be conducted using either breakout or plenary formats.
• The breakout format uses several breakout groups, which can vary in
size but may include as many as 10 to 15 players each, seated at
different tables.
• They incorporate group problem solving.
• Senior officials become familiar with critical issues related to their
responsibilities.
• They employ the conditions of a specific scenario.
• Personnel contingencies are examined.
• Group message interpretation is examined.
• Participants share information.
• Interagency/inter-organization coordination is assessed.
• Limited or specific objectives are achieved.
• They prepare participants for more complex exercises.
Setup for TTX
• Prior to exercise conduct, the exercise planning team must deliver the
necessary exercise materials and equipment, which include the
following:
• Adequate number of Situation Manuals (SitMans) or other written
materials for exercise participants
• Multimedia presentation
• Appropriate A/V equipment including televisions, projectors,
projection screens, microphones, and speakers
• Table tents for each table
• Name tents for each participant
• Badges identifying the role of each exercise participant (e.g., player,
observer, VIP, facilitator, evaluator)
• Sign-in sheets
• Participant Feedback Forms
Player Hot Wash
• Immediately following the exercise, a controller in each functional
area leads a hot wash and allows players to provide immediate
feedback.
• This enables controllers and evaluators to capture information about
events while they are still fresh in the players’ minds.
• The hot wash is an opportunity to ascertain the level of satisfaction
with the exercise, identify issues or concerns, and propose areas for
improvement.
• Players complete and submit their Participant Feedback Forms during
the hot wash.
• All evaluators take notes during play and hot washes for later
compilation with other observations from their functional areas.
• Information from Participant Feedback Forms is used to help generate
the AAR/IP. The exercise planning team leader collects and secures
attendance lists.
HSEEP Compliance #3
• Develop and submit a properly formatted After Action
Report/Improvement Plan (AAR/IP).
• An AAR/IP is used to capture events as they occurred
during an exercise, provide analysis of the events relative
to exercise objectives, and suggest development actions to
either further enhance or improve agencies’ planning and
response capabilities.
• It also evaluates achievement of the selected exercise
objectives and demonstration of the overall capabilities
being validated.
• The IP portion of the AAR/IP includes corrective actions
for improvement, along with timelines for their
implementation and assignment to responsible parties.
Exercise Evaluation Methodology
• The HSEEP evaluation methodology is an
analytical process used to assess the
demonstration of capabilities during
exercises. According to this methodology,
exercise evaluation incorporates three
distinct levels of analysis: task-level
analysis, activity-level analysis, and
capability-level analysis.
Evaluation
EEGsAAR/IP
• Consistent EEGs facilitate the creation of
effective AAR/IPs. By relating capabilities
to activities, tasks, and performance
measures, EEGs establish the foundation
for IPs that can strategically target
personnel, planning, organization and
leadership, equipment and systems, training
and exercises, and assessments and
corrective actions pertaining to identified
shortcomings in priority capabilities.
After Action Report/Improvement Plan (AAR/IP)
• An AAR captures observations of an
exercise and makes recommendations for
post-exercise improvements; and an IP
identifies specific corrective actions,
assigns these actions to responsible parties,
and establishes target dates for action
completion.
• The AAR and the IP should be printed and
distributed jointly as a single AAR/IP
following an exercise.
Road to the AAR/IP
Evaluators
•
•
•
•
Lead evaluator
Select evaluators and train evaluators
C/E Briefing
Evaluation requirements are based on initial
exercise scope/objectives
• For discussion-based exercises, consideration of
the exercise’s goals and objectives helps inform
the development of a Situation Manual (SitMan),
which provides the exercise facilitator with
suggestions for how to steer exercise discussion
to the capabilities being evaluated.
Types of Information Recorded
•
•
•
•
•
•
•
•
•
•
What plans, policies, and procedures would players implement to prevent,
protect against, respond to, or recover from the incident described in the
exercise scenario?
Are roles and responsibilities of the various government agencies and private
organizations clearly defined?
How are various decisions made? Who has authority to make decisions?
What information about the scenario, the hazard, the victims, and the risks to
participants and the public is collected? Who collects it, and what do they do
with it?
How is information shared with other agencies and with the public? What
information is shared?
What are the roles of Federal and State agencies? How are Federal and State
resources requested? Who makes the request? How are the resources
distributed and controlled?
What mutual aid agreements (MAAs) exist? How would they be activated?
What recommendations for improvements are made by the group?
Which issues are unresolved or require follow-up?
What actions do players plan to take in order to address outstanding issues?
Identifying Root Cause and Developing
Recommendations
• In order for the exercise evaluation process to produce an AAR/IP that
makes useful recommendations for improving an entity’s
preparedness capabilities, it is critical for evaluators to discover not
only what happened, but why events happened.
• Each task that is not completed as expected offers evaluators the
opportunity to search for a root cause.
• A root cause is the source of or underlying reason behind an
identified issue (as uncovered during careful analysis) toward which
the evaluator can direct an improvement.
• To arrive at a root cause, an evaluator should attempt to trace the
origin of each event back to earlier events and their respective causes.
• Root cause analysis may also require the review and evaluation of an
entity’s emergency plans; training programs; and other plans,
policies, and procedures.
Identifying Root Cause and Developing
Recommendations
• Uncovering root causes enables an evaluator to
work with the rest of the analysis team to develop
actionable solutions to improvement areas
identified in the AAR.
• These recommendations are based on the
evaluation team’s experience and best judgment,
although the responsibility for implementing
recommendations ultimately lies with the leaders
and managers of the participating entities.
Develop the Draft AAR/IP
• All discussion-based and operations-based
exercises result in the development of an
AAR/IP, the final exercise document that
provides a description of what happened,
describes any best practices or strengths,
identifies areas for improvement that need
to be addressed, and provides
recommendations for improvement.
AAR Format
•
•
•
•
•
•
Report Cover
Administrative Handling
Instructions
Contents
Executive Summary
Section 1: Exercise Overview
(includes identifying information,
such as the exercise name, date,
duration)
Section 2: Exercise Design
Summary (includes the
overarching exercise purpose;
objectives, capabilities, activities,
and tasks identified for validation;
a summary of designed initiating
event(s) / key scenario events; and
exercise design issues)
• Section 3: Analysis of
Capabilities
• Section 4: Conclusion
• Appendix A: Improvement
Plan
• Appendix B: Lessons Learned
(optional)
• Appendix C: Participant
Feedback Summary (optional)
• Appendix D: Exercise Events
Summary Table (optional)
• Appendix E: Performance
Ratings (optional)
• Appendix F: Acronyms
Road to Final Four
Conduct After Action Conference
• As soon as possible after completion of a draft AAR, the
lead evaluator, members of the evaluation team, and other
members of the exercise planning team conduct an After
Action Conference to present, discuss, and refine the draft
AAR, and to develop an IP.
• This conference is a chance to present the AAR to
participating entities in order to solicit feedback and make
necessary changes.
• The After Action Conference is a critical component of
the exercise planning process to ensure that exercises are
results-oriented and contribute to preparedness by
translating AAR/IP analyses into concrete improvements
for validation in subsequent exercises.
Identify Corrective Actions to be Implemented
• In addition to refining the draft AAR, much of the
After Action Conference is devoted to discussing
specific corrective actions to address the observed
areas for improvement and associated
recommendations identified in the draft AAR.
• This discussion takes place in a moderated,
disciplined environment and yields the IP; a list of
corrective actions that identify what should be
done to address observations and
recommendations; who (person or entity) is
responsible; and the timeframe for
implementation.
Guide for Developing Corrective Actions
• What changes need to be made to plans and procedures to
improve performance?
• What changes need to be made to organizational structures
to improve performance?
• What changes need to be made to leadership and
management processes to improve performance?
• What training is needed to improve performance?
• What changes to (or additional) equipment is needed to
improve performance?
• What lessons can be learned that will direct how to
approach a similar problem in the future?
Finalize AAR/IP
• Finalizing the AAR/IP involves
incorporating the corrections, clarifications,
and other feedback provided by participants
at the After Action Conference.
• Once these inputs have been incorporated,
the AAR/IP is distributed to members of
the exercise planning team for validation
that it is an accurate document that meets
the exercise objectives.
HSEEP Compliance #4
• Track and implement corrective actions identified in the AAR/IP.
• Once recommendations, corrective actions, responsibilities, and due
dates are identified in the IP, the exercising entity ensures that each
corrective action is tracked to completion.
• Exercising entities review all exercise evaluation feedback and
resulting IPs to assess progress on enhancing preparedness.
• This analysis and information are incorporated into the capabilitiesbased planning process because they may identify needs for additional
equipment, training, exercises, coordination, plans, or procedures that
can be validated through future exercises.
• Continual IP tracking and implementation should be part of a
corrective action program within each participating entity.
Track Implementation
• To track the implementation of corrective
actions identified in the final AAR/IP,
exercise teams must include individuals
responsible for complying with the CAP
process.
• Exercising entities are not expected to have
dedicated staff members for these positions.
Agency’s POC to Track IP
• The progress reports issued by an entity’s
POC and exercise program manager should
illustrate a consistent trend of progress
toward implementation of the corrective
actions listed in an AAR/IP.
• Because the AAR/IP ties these corrective
actions to specific capabilities, these reports
ultimately demonstrate the concrete ways
in which exercises enhance capabilities.
Continual Improvement Process
Questions??
• Amelia Muccio
NJPCA
amuccio@njpca.org
• Dave Weidner
HCANJ
dave@hcanj.org
• .