Homeland Security Exercise and Evaluation Program (HSEEP) Rebecca L. Siceloff For Official Use Only 1 An Introduction to Exercises Assess and validate policies, plans, procedures, training, equipment, assumptions, and interagency agreements Clarify roles and responsibilities Improve interagency coordination and communications Identify gaps Measure performance Identify opportunities for improvement For Official Use Only 2 HSEEP Cycle For Official Use Only 3 The Building Block Approach Key Concept: Exercises should be planned in a cycle that increases in complexity. Each successive exercise should build on the scale and experience of the previous exercise. For Official Use Only 4 Discussion-based Exercises Provide a forum for discussing or developing plans, agreements, training and procedures Less complicated than operations-based exercises Focus on strategic, policyoriented issues Include seminars, workshops, tabletops, and games Do not involve deployment of resources For Official Use Only 5 Seminars An informal discussion-based exercise led by a presenter or a facilitator, used to teach or orientate participants. Goals Orientate participants to new or existing plans, policies, or procedures Research or assess interagency capabilities or inter-jurisdictional operations Construct a common framework of understanding Conduct Characteristics Casual atmosphere Minimal time constraints Lecture-based For Official Use Only 6 Workshops A formal discussion-based exercise led by a facilitator, used to build or achieve a product. Goals Develop new ideas, policies, or processes Develop a written product as a group in coordinated activities Obtain consensus Collect or share information Conduct Characteristics Involves more participant discussion than lecture-based seminar Uses break-out sessions to explore parts of issue with smaller groups For Official Use Only 7 Tabletop Exercises A tabletop exercise (TTX) involves senior staff, elected or appointed officials, or other key personnel in an informal group discussion centered on a hypothetical scenario. Goals Identify strengths and shortfalls Enhance understanding of new concepts Identify strengths and shortfalls Conduct Characteristics Requires an experienced facilitator In-depth discussion Slow-paced problem solving For Official Use Only 8 Games Key Concept: A game is a simulation of operations using rules, data, and procedures designed to depict an actual or assumed real-life situation. Goals Explore the processes and consequences of decision-making Conduct “what-if” analysis of existing plans Test existing and potential strategies Conduct Characteristics Does not involve the use of actual resources Often involves two or more teams Includes models/ simulations of increasing complexity For Official Use Only 9 Operations-based Exercises Involve deployment of resources and personnel More complex than discussion-based exercises Require execution of plans, policies, procedures Clarify roles and responsibilities Improve individual and team performances Include drills and both functional and full-scale exercises For Official Use Only 10 Drills A drill is a supervised activity that tests a specific operation or function of a single agency. Goals Gain training on new equipment Test new procedures Practice and maintain skills Prepare for more complex exercises Conduct Characteristics Immediate feedback Realistic but isolated environment For Official Use Only 11 Functional Exercises A functional exercise (FE) is a single or multi-agency activity designed to evaluate capabilities and multiple functions using simulated response. Goals Evaluate management of Emergency Operations Centers, command posts, and headquarters Assess the adequacy of response plans and resources Conduct Characteristics: Simulated deployment of resources and personnel Rapid problem solving Highly stressful environment For Official Use Only 12 Full-Scale Exercises A full-scale exercise (FSE) is a high-stress multi-agency, multijurisdictional activity involving actual deployment of resources in a coordinated response, as if a real incident had occurred. Goals Assess plans and procedures under crisis conditions Evaluate coordinated responses under crisis conditions Conduct Characteristics: Mobilization of units, personnel, and equipment Stressful, realistic environment Scripted exercise scenario For Official Use Only 13 HSEEP Cycle: Project Management For Official Use Only 14 Exercise Project Management Project managers are responsible for the design, development, and execution of a specific exercise, followed by evaluation and improvement planning. Develop a project management timeline Establish project milestones Identify the exercise planning team Schedule planning conferences For Official Use Only 15 Exercise Timeline In project management, the Exercise Timeline identifies key conferences and tasks for an individual exercise. Exercise project managers build timelines to include: A schedule of key conferences and milestones Planning team task assignments Generally, timelines for discussion-based exercises are shorter and have fewer tasks than timelines for operations-based exercises. For Official Use Only 16 Exercise Planning Team Every exercise requires an Exercise Planning Team – the core group responsible for the design, development, conduct, and evaluation of an exercise. Determines exercise objectives Creates the scenario Develops exercise documentation Conducts pre-exercise briefing and training sessions Composed of members of key participating agencies Is manageable and flexible in size, depending on exercise type and scope For Official Use Only 17 Defining Design and Development Exercise design includes: Assessing exercise needs Defining the scope of the exercise Writing a statement of purpose Defining exercise objectives Creating a scenario for the exercise Exercise development includes: Creating exercise documentation Arranging logistics, actors, and safety Coordinating participants and media Other supporting planning tasks For Official Use Only 18 Purpose An exercise's purpose states the desired goal of the exercise. Governs the selection of the objectives Clarifies why the exercise is being conducted Communicates plans to the media and community leaders For Official Use Only 19 Scope Scope establishes realistic limits on the exercise. Scope includes: Type of exercise Location of exercise Agencies/Emergency Support Functions Participants Type of emergency Capabilities to be exercised For Official Use Only 20 Objectives An objective is a description of the performance you expect from participants. It conveys specifically how the exercise should achieve its purpose. Objectives Define performance/actions expected from participants Defines the specific conditions under which those actions will be performed Provide framework for scenario development Provide exercise evaluation criteria . For Official Use Only 21 Scenario A scenario is the storyline that drives an exercise. A scenario includes: General context or comprehensive story Technical details of story's conditions and events Conditions for assessing capabilities Scenarios should be: Threat-based and performance-based Realistic Challenging For Official Use Only 22 Exercise Documentation Exercise documents are the most tangible elements of design and development. Different exercise types require different documentation, ranging from simple sign-in sheets to media releases and exercise evaluation guides. Basic exercise documentation includes: Situation Manual Exercise Plan Controller Evaluator Handbook Master Scenario Events List Exercise Evaluation Guides For Official Use Only 23 HSEEP Cycle: Project Management For Official Use Only 24 Discussion-Based Conduct Discussion-based exercise conduct involves: Site setup Guided presentation Facilitated/ moderated discussion Wrap-up activities (including a hot wash) The majority of discussion-based “action” comes from moderated participant discussions, either as a whole group or in break-out sessions. Moderators and facilitators are essential to keeping the discussions on track to meet exercise objectives. For Official Use Only 25 Operations-Based Conduct Conduct of operations-based exercises involves: Site setup Exercise briefings Exercise play Wrap-up activities For Official Use Only 26 Evaluation Key Concept: Exercise evaluation assesses how well the exercise objectives were achieved. It also identifies opportunities for improvement. Evaluators accomplish this by: Observing the exercise and collecting supporting data. Gauging performance against expected outcomes. Determining what changes are needed to ensure desired outcomes. Evaluation is the yardstick by which an organization measures its capabilities. Good evaluations result in suggestions for filling and bridging capability gaps or making needed improvements. For Official Use Only 27 Exercise Evaluation Guides Exercise Evaluation guides (EEGs) provide structured evaluation measures of participant conduct, listing critical activities and tasks to be completed during an exercise. Three distinct levels of analysis: Task-level Activity-level Capability-level For Official Use Only 28 After Action Reports Prepared by the members of the planning team and evaluation team, the After Action Report (AAR) provides participant officials with feedback on the exercise's results and suggests recommendations for improvement. After Action Reports: Prepared after every exercise type. Summarize what happened during the exercise. Provide feedback to participants on their performance. Recommend improvements for better preparedness. For Official Use Only 29 The After Action Conference The After Action Conference is a forum for exercise participants to discuss the draft AAR and identify action items for the Improvement Plan. The conference should address: Specific improvement actions that agencies can take Feedback and validation of observations and recommendations Key lessons learned from the exercise experience Distribution of AAR and IP For Official Use Only 30 Improvement Planning Process The Improvement Planning Process is the means for converting recommendations from the After Action Report (AAR) into measurable steps that, when implemented, lead to improved response capabilities. By focusing on performance and how actual outcomes differ from expectations, public officials and exercise planners can: Target their improvement resources more effectively Modify their exercise programs before having to respond to a real incident For Official Use Only 31 The Improvement Plan Organizations execute the improvement planning process using a tool called the Improvement Plan (IP). The IP identifies: Actions to address each AAR recommendation. Who will be responsible for taking each action. A timeline for completion of those actions. Once recommendations and action items have been identified, organizations should ensure that each item is tracked to completion and improvements are implemented. For Official Use Only 32 The Improvement Plan Matrix The Improvement Plan Matrix is a spreadsheet template for completing the IP in a condensed form. The Improvement Plan Matrix finalizes the AAR/IP It includes: Tasks Recommendations Improvement Actions Responsible Party (Parties) Completion Date For Official Use Only 33 HSEEP Cycle For Official Use Only 34 Program Management Exercise program management involves developing and executing an exercise program. An exercise program provides the administration, supporting resources, and strategic goals for an organization's exercise efforts This includes: Multiyear training and exercise program planning. Budgeting and grant writing. Planning and executing individual exercises. Tracking improvements. For Official Use Only 35 HSEEP Cycle: Program Management For Official Use Only 36 Improvement Planning Workshop (IPW) The Improvement Planning Workshop is a Plan is a forum to focus on previous year’s After Action Reports (AARs) and Improvement Plans (IPs) for exercises and real-world events Allows for the development of a more cohesive Training and Exercise Plan Participants can review priorities with their colleagues to ensure these are the correct priorities before the Training and Exercise Planning Workshop (TEPW) Gain buy-in across department before the TEPW Better prepare TEPW participants For Official Use Only 37 HSEEP Cycle: Program Management For Official Use Only 38 Multi-Year Training & Exercise Plan The Multiyear Training & Exercise Plan is essential to managing an exercise program. The plan is a document that outlines a program's long-term strategy for building capabilities through exercises and training. Takes stock of prioritized capabilities Lays out long-term program goals and objectives Develops a mix of training and exercises to meet goals and objectives Determines what training is needed as a prerequisite to planned exercises Sets a multi-year schedule of training events and exercises For Official Use Only 39 Multi-Year Training & Exercise Schedule Exercise coordination is done through the Multi-Year Training and Exercise Schedule. This schedule lays out a long-term schedule of planned and potential training dates and exercises. The Multi-Year Training and Exercise Schedule is used to: Avoid duplicating their efforts Combine exercises and ensure the exercises don't conflict Combine training and ensure training does not conflict Optimize and combine funding where possible Prevent "over" training and exercising For Official Use Only 40 Training & Exercise Planning Workshop (TEPW) The Multi-Year Training and Exercise Plan and the MultiYear Training and Exercise Schedule are created at a Training and Exercise Planning Workshop (TEPW). Involves various stakeholders from both exercise and training community Is conducted annually Is one to two days long Allows agency representatives from numerous training and exercise programs to coordinate activities For Official Use Only 41 HSEEP Cycle For Official Use Only 42 For Official Use Only 43