FEMA IS-120.C: An Introduction to Exercises Course Summary

• Develop a baseline knowledge of exercise fundamentals.
• Identify the tasks necessary to complete each phase of the exercise process.
• Define how exercises complete the preparedness process.
• Identify the role of exercises in validating capabilities.
• Identify phases of exercise evaluation and the improvement planning process.

HSEEP Fundamental Principles. You learned the fundamental HSEEP principles of; guidance by elected and appointed officials, the National Preparedness Goal (NPG), the Progressive Planning Approach, the importance of the whole community integration, risk management and a common methodology for exercises.

The reasons for conducting exercises. You learned the reasons for conducting exercises which included- National Preparedness Components, clarifying responsibilities and roles, improving interagency coordination and communication, identifying gaps in resources, developing individual performance, and identifying areas/opportunities for improvement.

Identification of how the participant has shareholder support. You learned how participants have shareholder support by engaging the whole community, and state and local jurisdictions.

Discussion-based exercises. You learned about discussion-based exercises as- a forum for discussing or developing plans and procedures, less complicated than operations-based exercises, focused on strategy and policy.

Operations-based exercises. You learned about operations-based exercises and their characteristics which included: drills, the involvement of deployment of resources and personnel, complexity compared to discussion based exercises, and their advantages including the improvement of individual and team performances.

• Presidential Policy Directive 8
• National Preparedness Goal
• National Preparedness System
• Overview of the National Planning Frameworks
• Threat and Hazard Identification and Risk Assessment Guide: Comprehensive Preparedness Guide(CPG) 201
• FEMA Preparedness Toolkit/HSEEP Policy and Guidance
• Homeland Security Digital Library
• FEMA Home Page

Objectives: After completing this course, you will be able to:

• Develop a baseline knowledge of exercise fundamentals.
• Identify the tasks necessary to complete each phase of the exercise process.
• Define how exercises complete the preparedness process.
• Identify the role of exercises in validating capabilities.
• Identify phases of exercise evaluation and the improvement planning process.

The Homeland Security Exercise and Evaluation Program (HSEEP) provides a set of guiding principles for exercise programs and a common approach to exercise program management, design and development, conduct, evaluation, and improvement planning.

This course introduces the basics of emergency management exercises. It also builds a foundation for subsequent exercise courses, which provide the specifics of HSEEP.

An Introduction to Exercises is designed for personnel from all emergency management sectors to understand the basic fundamentals of exercise design, development, conduct, evaluation, and improvement planning.

The remainder of Lesson 1, Introduction and Program Fundamentals, covers HSEEP fundamentals, reasons to exercise, and the types of exercises.

• Lesson 2, Exercise Management Program, delves deeper into the functionality, goals, and outcomes from the different types of discussion- and operations-based exercises, and exercise program management.
• Lesson 3, Exercise Overview, discusses multi-year exercise planning priorities, the benefits of exercise foundation, and defines the exercise cycle.
• Lesson 4, Establishing Exercise Foundation, steps through building the foundation from the prject management timeline to planning the activities based on the capabilities to be tested, and scheduling the appropriate planning meetings.
• Lesson 5, Design and Development Phase, defines key elements of the design including,the objectives and scope of the exercise and the scenario to be used. This is where required documentation is determined and developed and needed resources are identified.
• Lesson 6, Conduct, covers the actions taken for discussion- and operations-based exercises prior to, during conduct and wrap-up activities at the end of the exercise.
• Lesson 7, Evaluation, addresses evaluator selection and required documentation and methods of observation and data collection.
• Lesson 8, Improvement Planning, provides insight into corrective action and improvements following exercise evaluation analysis.

Objectives: After completing this lesson, you will be able to:

• Identify the benefits of using the HSEEP methodology to plan and carry out effective exercises.
• Explain the National Preparedness System (NPS) components.
• Explain the purpose of Core Capabilities.

Exercise:

• (n) Something performed or practiced in order to develop, improve, or display a specific power or skill.
• (v) To practice in order to train, strengthen, or develop

Source: Merriam Webster’s Dictionary

Exercises improve readiness by:

• Involving the whole community.
• Providing a way to evaluate operations and validate plans and capabilities.
• Reinforcing teamwork.
• Identify both capability gaps and areas for improvement.
• Demonstrating a community’s resolve to prepare for disastrous and catastrophic events.

“Planning, training, exercising, sometimes it gets a little boring, repetitious, until you see what happens in the real world.”

—James Hooley, Chief of Emergency Medical Services, Boston, MA

• Guided by Elected and Appointed officials
• Capability-based, Objective Driven
• Progressive Planning Approach
• Whole Community Integration
• Informed by Risk
• Common Methodology

Routine engagement with elected and appointed officials ensures that exercises have the support necessary for success.

Exercises are built around core capabilities as identified in the National Preparedness Goal (NPG). NPG identifies 32 core capabilities that are associated with the five mission areas: Prevention, Protection, Mitigation, Response, and Recovery. When capabilities are aligned with mission areas the outcome examines current capability levels and identifies gaps, or opportunities for improvement.

Using the HSEEP methodology, organizations can use exercises to focus on assessing performance against capability-based objectives.

Senior leadership positions in state, territory, tribal, and local government, and organizations of all sizes can provide direction on the principal objectives that guide each organization’s approach to planning and preparedness.

Exercise planners are encouraged to engage the whole community, where appropriate, throughout exercise program management, design and development, conduct, evaluation, and improvement planning.

• Representatives from relevant disciplines, first responders or utility companies, for example
• Senior leaders or those responsible for providing support and/or resources
• Community member representation
• Representatives with administrative responsibility
• Representatives from volunteer, nongovernmental or nonprofit sectors, and social support organizations such as advocates for children, seniors, limited access and functional needs, limited language proficiency, and racially and/or ethnically diverse neighborhoods

Planners should first identify pertinent risks, and then assess the potential impacts associated with each risk. For example, looking at a community’s Threat and Hazard Identification and Risk Assessment (THIRA) is a good starting place for exercise program priorities.

The Exercise Cycle is the common planning methodology provided by HSEEP for use with all exercise types. Regardless of scope or scale of an exercise, the exercise cycle will include exercise program management, design and development, conduct, evaluation, and improvement planning.

The National Preparedness Goal (NPG) is the cornerstone for the implementation of PPD-8. The Nation’s core capabilities are identified across five mission areas: Prevention, Protection, Mitigation, Response, and Recovery. The NPG identifies the desired achievements and the goals that have been set.  Mission areas and core capabilities will be covered in depth later in this lesson.

The National Preparedness System (NPS) is the mechanism the Nation uses to build, sustain, and deliver the core capabilities to meet the NPG and meet the goal of a secure and resilient Nation. The NPS helps define, the process communities can use to reach identified preparedness goals. The NPS includes a series of five national planning frameworks, one for each mission area. Each framework provides the following guidance:

• Clarification and assignment of roles and responsibilities
• Core capabilities for the specific mission area
• Critical tasks for each of the core capabilities
• Direction for delivering the core capabilities

HSEEP provides a common approach and consistent methodology for exercise planning. It is flexible, scalable, and available to all organizations regardless of size or scope of the organization or planned activity.

Exercises help to:

• Clarify roles and responsibilities
• Improve interagency coordination
• Find resource gaps
• Develop individual performance
• Identify opportunities for improvement

NPS consists of six components, including identify and assess risk, estimate capability requirements, build and sustain capabilities, plan to deliver capabilities, validate capabilities, and review and update risks, tools, and resources. Together, these components provide communities with a consistent and reliable approach for identifying high-priority capability targets.

Advance to the next screen for descriptions of each component.

1. Prevention
2. Protection
3. Mitigation
4. Response
5. Recovery

Visit the Preparedness Resource Library page to select the National Preparedness Goal

• Prevention: The capabilities necessary to avoid, prevent, or stop a threatened or actual act of terrorism.
• Protection: The capabilities necessary to secure the homeland against acts of terrorism and manmade or natural disasters.
• Mitigation: The capabilities necessary to reduce loss of life and property by lessening the impact of disasters.
• Response: The capabilities necessary to save lives, protect property and the environment, and meet basic human needs after an incident has occurred.
• Recovery: The capabilities necessary to assist communities affected by an incident to recover effectively.

Core capabilities are used to measure, describe, and implement our security and resilience techniques. There are 32 core capabilities identified in the National Preparedness Goal, each classified beneath one of the five mission areas.

The core capabilities:

• Are distinct critical elements necessary to meet the National Preparedness Goal (NPG)
• Are essential for the execution of each of the five mission areas
• Provide a common language for preparedness across the whole community
• Are not exclusive to any single level of government or organization

As we begin to look at the core capabilities that fall under the five mission areas, notice that three capabilities span all mission areas: Planning, Public Information and Warning, and Operational Coordination. These help to unify mission areas and, in many ways, are necessary for the success of the remaining core capabilities. In addition, many core capabilities involve more than one mission area and are displayed in multiple mission areas, as appropriate.

For additional information on Core Capabilities, visit the Preparedness Resource Library and select the FEMA Home Page, then search for Core Capabilities.

Planning: Conduct a systematic process engaging the whole community as appropriate in the development of executable strategic, operational, and/or tactical-level approaches to meet defined objectives.

Public Information and Warning: Deliver coordinated, prompt, reliable, and actionable information to the whole community through the use of clear, consistent, accessible, and culturally and linguistically appropriate methods to effectively relay information regarding any threat or hazard, as well as the actions being taken and the assistance being made available, as appropriate.

Operational Coordination: Establish and maintain a unified and coordinated operational structure and process that appropriately integrates all critical stakeholders and supports the execution of core capabilities.

Objectives: Having completed this lesson, you are able to:

• Identify the benefits of using the HSEEP methodology to plan and carry out effective exercises.
• Explain the National Preparedness System (NPS) components.
• Explain the purpose of Core Capabilities.

Objectives: After completing this lesson, you will be able to:

• Explain exercise management program.
• Define the progressive approach to exercises.
• Explain the purpose and use of discussions-based exercises, including goals, conduct, types and outcomes.
• Explain the purpose and use of operations-based exercises, including goals, conduct, types and outcomes.

An effective exercise program helps organizations maximize efficiency, resources, time, and funding by ensuring that exercises are part of a coordinated, integrated approach to building, sustaining, and delivering core capabilities. This approach—called multi-year planning—begins when elected and appointed officials, working with whole community stakeholders, identify and develop a set of multi-year exercise priorities informed by existing assessments, strategies, and plans. These long-term priorities help exercise planners design and develop a progressive program of individual exercises to build, sustain, and deliver core capabilities.

Effective exercise program management promotes a multi-year approach to:

• Engaging elected and appointed officials.
• Establishing multi-year exercise program priorities
• Developing a multi-year TEP.
• Maintaining a rolling summary of exercise outcomes.
• Managing exercise program resources.

The multiyear approach is driven by these steps:

• Engaging elected and appointed officials
• Establishing multiyear exercise program priorities
• Developing a multi-year Training and Exercise Plan (TEP)
• Maintaining a rolling summary of exercise outcomes
• Managing exercise program resources

Regardless of exercise type, each exercise within the progressive series is linked to a set of common program priorities and designed to evaluate associated capabilities.

Effective planning of exercises and integration of the necessary training will reduce the waste of limited exercise resources and serve to address known shortfalls prior to the conduct of the exercise.

The different types of exercises that may be included in a multi-year plan are described in the following sections.

Discussion-based “action” comes from facilitated discussion with participants, either as a whole group or in break-out sessions. Facilitators are essential to keeping the discussions on track to meet exercise objectives.

These types of exercises help to familiarize participants with current plans, policies, agreements, and procedures and are used to develop new ones.

These types of exercises are characterized by actual implementation in reaction to an exercise scenario.

Operations-based exercises are used to validate plans, policies, agreements, and procedures. These exercises are characterized by actual implementation of response activities in reaction to an exercise scenario.

Objectives: Having completed this lesson, you are able to:

• Explain exercise management program.
• Define the progressive approach to exercises.
• Explain the purpose and use of discussions-based exercises, including goals, conduct, types and outcomes.
• Explain the purpose and use of operations-based exercises, including goals, conduct, types and outcomes.

Objectives: After completing this lesson, you will be able to:

• Identify the components of developing and executing an exercise program.
• Describe the purpose of the Training and Exercise Plan (TEP).
• Explain the purpose of the Training and Exercise Planning Workshop (TEPW).

Project managers are responsible for:

• Design and development (Design and Development)
• Execution of a specific exercise (Conduct)
• Evaluation (Evaluation)
• Planning improvements (Improvement Planning)

Good project management involves:

• Developing a project management timeline
• Establishing project milestones
• Identifying the exercise planning team
• Scheduling planning meetings

• Identified by elected and appointed officials
• Guide program planning and resource allocation
• Determine type and range of training/exercises
• Provide a roadmap for selection and prioritizing individual exercise development

An essential component of the exercise management process, the TEPW is a collaborative environment where whole community stakeholders can engage in a forum to discuss and coordinate training and exercise activities across local organizations to maximize the use of available resources and prevent duplication of effort.

During the workshop, participants:

• Review program accomplishments to date
• Review each jurisdictions progress and accomplishments over the past year, or since the last TEPW
• Identify needs and modifications required—including changes to the midyear training and exercise schedule and planning that might need updating
• Translate needs into priorities and develop specific objectives to address through exercises
• Compare Improvement Plan (IP) actions against current capabilities, training, and exercises
• Identify and coordinate potential funding sources
• Coordinate exercise activities and scheduling—a major part of the workshop should be spent on schedule coordination. The workshop is an excellent opportunity for all jurisdictions to coordinate their exercise schedules to avoid duplication of efforts and collaborate to maximize resources

TEPWs are convened on a periodic basis, usually annually or biannually, based on the program needs.

The outcome of the TEPW is the Multi-year Training and Exercise Plan.

The schedule should include a graphic representation of the proposed activities scheduled in the multi-year exercise plan, it should emphasize coordination through jurisdictions, and allow adequate time for progressive exercises designed to build and validate capabilities.

Program managers use the Multi-year Training and Exercise Schedule to:

• Avoid duplicating their efforts
• Combine exercises and ensure the exercises don’t conflict
• Combine training and ensure training does not conflict
• Optimize and combine funding where possible
• Prevent “over” training and exercising

Objectives: Having completed this lesson, you are able to:

• Identify the components of developing and executing an exercise program.
• Describe the purpose of the Training and Exercise Plan (TEP).
• Explain the purpose of the Training and Exercise Planning Workshop.

Objectives: After completing this lesson, you will be able to:

• Identify the roles and responsibilities of the exercise planning team.
• Identify discussion points to be addressed during a concept and objectives meeting.
• Identify the primary focus, discussion points, tools, and outcomes of the three planning meetings.
• List the major components of a master scenario events list meeting.

• Design
• Development
• Conduct
• Evaluation

The size of the team will vary depending on the type and scale of the exercise. Exercise planners may elect to use the Incident Command System (ICS) structure when establishing the structure and organization of the planning team (see organization chart).

A team consists of a Lead Exercise Planner and planning team members. The team lead has complete management responsibility, assigning tasks to team members and ensuring the successful execution of the exercise.

The team size should:

• Be kept manageable but represent a full range of participating organizations and other relevant stakeholders.
• Include whole community stakeholders such as:
  • First responders/support agencies
  • Advocacy groups
  • Those with limited English language proficiency

In addition, Subject Matter Experts (SMEs) bring functional knowledge from their area of expertise.

• Adding expertise to the exercise planning team
• Providing functional knowledge for player-specific tasks evaluated through objectives
• Helping make the scenario realistic and plausible
• Ensuring appropriate evaluation of capabilities

Trusted agents:

• Are individuals who understand that confidentiality must be preserved to maintain the integrity of the exercise.
• All members of the exercise planning team are trusted agents.
• Team members must not reveal details or share insight in order to protect the integrity and confidentiality of the exercise and evaluation process.
• This is especially important to those team members who may also serve as controllers or evaluators during exercise conduct.

Regardless of the size or scope of an exercise, an exercise planning team performs best with a clear organizational structure which clearly defines roles, responsibilities, and functional requirements for each role or position on the planning team.

Tips for creating a successful planning team:

• Define roles, responsibilities, and functional requirements
• Engage elected and appointed officials and whole community leadership in exercise planning
• Use project management principles
• Follow standardized process
• Organize the team using NIMS Incident Command System (ICS) or other structure that defines support roles for each team member

Planning Team Roles and Responsibilities

Exercise planning team members are responsible for the exercise design and development, conduct of the exercise, and exercise evaluation. Based on elected and appointed officials’ guidance and program priorities, the team will establish exercise objectives and determine the core capabilities to be assessed during exercise play. Next, the team is responsible for creating a realistic scenario to assess the identified capabilities and objectives. As mentioned earlier, SMEs play a big role in creating plausible scenarios. The team develops supporting documentation for the exercise as well as process documentation, control and simulation documentation, and customized evaluation packets for participants. Team members also help with building and distributing pre-exercise materials and conducting exercise planning meetings, briefings, and training sessions. Keep in mind that being part of an exercise planning team is usually a collateral duty. It is very important to ensure candidates can make the commitment to actively participate throughout the process.

Next, the team is responsible for creating a realistic scenario to assess the identified capabilities and objectives. As mentioned earlier, SMEs play a big role in creating plausible scenarios. The team develops supporting documentation for the exercise as well as process documentation, control and simulation documentation, and customized evaluation packets for participants.

Team members also help with building and distributing pre-exercise materials and conducting exercise planning meetings, briefings, and training sessions.

Keep in mind that being part of an exercise planning team is usually a collateral duty. It is very important to ensure candidates can make the commitment to actively participate throughout the process.

Meetings are held to discuss, review, or develop exercise content. Typically, these are face-to-face meetings, which are critical in the initial and final stages of exercise development. This forum allows for coordination and collaboration among participants. Face-to-face meetings help organizations share knowledge and build relationships.

While not all development can be completed during face-to-face meetings, they are a great medium to assess team progress, assign new responsibilities, review completed work, and establish deadlines. Once the team is fully established, online collaborative meetings may be used to facilitate the planning process.

• Concept and Objectives Meeting (C&O)
• Initial Planning Meeting (IPM)
• Midterm Planning Meeting (MPM)
• Master Scenario Events List Meeting (MSEL)
• Final Planning Meeting (FPM)

The remainder of this lesson will discuss each meeting type in further detail.

During the IPM, exercise planning team members are assigned responsibility for activities associated with designing and developing exercise documents, such as the Exercise Plan (ExPlan) and the Situation Manual (SitMan), and coordinating exercise logistics.

If only three planning meetings are held (i.e., IPM, MPM, and Final Planning Meeting [FPM]), a portion of the MPM should be devoted to developing the Master Scenario Exercise List (MSEL).

An FPM should be conducted for all exercises to ensure that all elements of the exercise are ready for conduct.

Objectives: Having completed this lesson, you are able to:

• Identify the roles and responsibilities of the exercise planning team.
• Identify discussion points to be addressed during a concept and objectives meeting.
• Identify the primary focus, discussion points, tools, and outcomes of the three planning meetings.
• List the major components of a master scenario events list meeting.

Objectives: After completing this lesson, you will be able to:

• Identify the key elements used to determine the scope of the exercise design.
• Define SMART objective.
• Identify the components of exercise documentation.
• Identify the documentation needed for an operations-based exercise.
• Identify the logistical concerns when developing an exercise.

Exercise design includes:

• Establish exercise scope
• Set exercise objectives
• Create exercise scenario
• Develop exercise documentation
• Determine media and public relations guidance

Key elements of Scope:

• Type of exercise
• Participation level
• Duration
• Location
• Parameters

Some elements are discussed and/or determined through program management activities or grant requirements. However, the responsibility of the planning team is to finalize the scope based on exercise objectives.

Objectives:

• Driven by Exercise Program priorities
• Establish the cornerstone of scenario design, development, exercise conduct, and evaluation
• Follow SMART guidelines for development

Generally, the number of exercise objectives should be limited to enable timely execution, facilitate design of a reasonable scenario, and promote successful completion of the exercise purpose.

Aligning objectives to capabilities ensures:

• Methodical tracking of progress throughout the exercise programs and/or cycles
• Standardized collection of exercise data used to inform preparedness assessments
• Fulfillment of grant or funding-specific reporting requirements

SMART objectives are specific, measurable, achievable, relevant, and time-bound. Use the following guide to develop good exercise objectives.

Three basic elements:

• Context or comprehensive story of an incident event (narrative)
• Conditions allowing players to exhibit proficiency in demonstrating core capabilities and satisfying objectives
• Technical details needed to depict the conditions and events of the scenario (timelines, etc.)

Based on:

• Realistic events
• Plausible threat
• Challenging, but not too complicated that it overwhelms players

Exercise documentation ensures that an accurate account of the exercise is preserved, which allows organizations to leverage past documentation to support future exercises. Consideration should be given to the accessibility of presentations and documents, such as making information available in alternative formats (e.g., large print, compact disc, Braille, and closed captioning).

This table lists the key exercise design and development documents identified by exercise type and distribution audience.

Facilitator Guide – The purpose of the facilitator guide is to assist facilitators managing discussion-based exercises. It outlines facilitator instructions and key issues for discussion during the exercise. The facilitator guide also includes background information to prepare the facilitator to answer questions.

Multimedia Presentation – A multimedia presentation is frequently used to illustrate the scenario. It helps participants focus and makes the exercise more realistic. The presentation is shown at the start of the exercise (StartEx) and supports the SitMan. Presentation components include an introduction; exercise scope, objectives, and core capabilities; exercise play rules and administrative information and is segmented into modules that correlate with the SitMan.

Controller/Evaluator Handbook (C/E Handbook) – The C/E Handbook contains detailed scenario and logistics information and a communications plan which is meant for designated controllers and evaluators only.

• The controller portion of the C/E handbook, also known as the Control Staff Instructions (COSIN), provides guidelines for control and simulation support and establishes a management structure for the activities.
• The evaluator portion of the C/E handbook, sometimes known as the EvalPlan, provides evaluation staff with guidance and instruction on evaluation or observation methodology, as well as essential materials required to perform their specific functions.

Master Scenario Events List (MSEL) – A chronological listing of events, in spreadsheet format, that drive exercise play. The MSEL is used during operations-based exercises or complex discussion-based exercises. MSEL events include contextual injects, expected action events (milestones), and contingency injects.

Components that build an inject:

• Designated scenario time and actual execution time
• Who delivers the inject (and by what means) and who is designated to receive the inject
• Event description (what will happen)
• Inject content (quoted text as it is to be delivered) plus any specific direction needed
• Expected action (Based on the objective being tested, identify specific tasks, conditions, and standards to accomplish the objective.
• Note section for controllers/simulators to record pertinent information

• Review capabilities
• Identify chronology of key events
• Anticipate player actions
• Identify information resources
• Compile all MSEL events into a single list
• Refine selected MSEL events; create a detailed long version

Once the MSEL is drafted, the exercise planning team should review it, arrange entries in sequential order, and resolve any scheduling conflicts between events for best results during the exercise.

Extent of Play Agreements (XPAs) – XPAs are agreements formed between the exercise sponsor and participating organizations. Specific details of participant involvement are identified, such as, one fire company, with two ambulances for 5 hours. These agreements can be vital to exercise planning, evaluator recruitment, and development of support requirements.

Exercise Evaluation Guides (EEGs) – Exercise Evaluation Guides (EEGs) provide evaluators with a checklist of critical tasks to be completed by participants during an exercise. EEGs contain the information to be discussed by participants, space to record evaluator observations, and questions to consider after the exercise.

Participant feedback form – The participant feedback form should be given to participants during exercise wrap-up. At a minimum, ask participants for feedback on the strengths and opportunities for improvement based on the organization’s plans, policies, SOPs and their impressions on the exercise conduct and logistics.

Waiver forms – All actors should receive a waiver form prior to the exercise. Once signed, these forms waive liability for all exercise planners and participants. Volunteers under the age of 18 must be signed by parental or legal guardian.

Weapons and Safety Policy – All exercises, where applicable, should employ a written weapon and safety policy that is in accordance with all applicable State and/or local laws and regulations. Exercise sponsors should coordinate the application of this policy with the appropriate safety and/or legal department, as necessary.

• Logistics
• Control Structure
• Communications
• Safety and Security

Considerations should include:

• Physical Environment: venue, facilities/rooms, supplies, food, refreshments, IDs.
• Actors (Volunteers) are used to make exercises more realistic as players administer simulated victim care.
  • Prior to StartEx, actors should be provided instruction and logistical details, as well as symptomatology cards containing their signs and symptoms.
• Parking, transportation and designated areas are to be clearly identified and staffed to direct vehicles.

Exercises that span multiple sites and/or venues can improve communication and coordination by establishing a master control cell to oversee and coordinate venue control cells, as identified in the graphic.

A SimCell is used to generate injects, receive player responses, and disseminate information in place of an organization that may not participate in the exercise but would be on the scene of the incident, if it were real.

• Controller communications
• Timing and content of communications
• Communications methodology

Safety is paramount when planning any exercise. For operations-based exercises, consider appointing safety controller(s), outline safety requirements and policies, and plan for dedicated ambulance/EMT coverage for real-world emergencies.

Security of the exercise site should also be considered. Local law enforcement can be employed where appropriate. If classified information is involved, follow appropriate security protocols to ensure information is not compromised.

Objectives: Having completed this lesson, you are able to:

• Identify the key elements used to determine the scope of the exercise design.
• Define SMART objective.
• List the three basic elements of the exercise scenario.
• Identify the components of exercise documentation.
• Identify the documentation needed for an operations-based exercise.
• Identify the logistical concerns when developing an exercise.

Objectives: After completing this lesson, you will be able to:

• Identify the activities that take place prior to exercise conduct.
• Explain the purpose of orientation briefings for the different participant groups.

• Exercise Director
• Evaluator(s)
• Lead Evaluator
• Facilitator(s)
• Controller(s)
• Simulators
• Senior Controller
• Safety Controllers
• Exercise Assembly Area Controller
• Players
• Actors
• Observers

Elected/Appointed Officials should be reengaged before exercise conduct to verify that the exercise, as designed, meets leadership expectations and intent.

Controllers should receive an overview of the exercise, specifics on their role and assigned location, the schedule of events, and direction on MSEL injects they will be delivering during the exercise.

Evaluators should receive an overview of the evaluation plan, methodology, objectives, and evaluation materials. Evaluators should receive instruction on the use of exercise materials, including the Exercise Handbook and Exercise Evaluation Guide (EEG). Evaluators should understand the exercise methodology and objectives, and know the schedule or agenda.

Controllers are assigned participant groups to oversee. These oversight roles include Actor Controller, Exercise Assembly Area Controller, and Observer/Media Controller. Shortly before StartEx, controllers conduct a briefing for their particular groups.

Actor Controllers deliver the actor briefing. The briefing gives actors an overview of the exercise and their roles as “victims”, adding realism to the exercise.

Exercise Assembly Area Controllers deliver the player briefing, which relays information on player roles and responsibilities, safety considerations, exercise parameters, and responses to questions and concerns from players. If security credentials are needed, proper usage should be covered in the briefing.

Observer/Media Controllers deliver the observer briefing, which provides a background on the exercise program, specifics for the exercise about to commence, observer restrictions and/or limitations, and the exercise schedule of events. This briefing is also attended by VIPs and media representatives.

During conduct of operations-based exercises, the Senior Controller or Exercise Director role is usually filled by the exercise planning team leader. Controllers and evaluators report their findings and concerns to the senior controller. The senior controller is responsible for officially beginning (StartEx) and ending (EndEx) exercise play.

Operations-based exercises can be staged in an actual environment or simulated if the environment is too dangerous to stage. For example, responding to an infectious virus spillage would be simulated to protect all involved.

Exercise data collection is recorded in Exercise Evaluation Guides (EEGs) by all evaluators. Positioning evaluators strategically to observe play will accommodate better data collection.

Lastly, an effective contingency plan is crucial to assure the exercise can be stopped, rescheduled, or canceled if a real-world event were to take place. All participants and stakeholders should be briefed on this plan.

Control is guidance for:

• Communication and coordination
• Roles and responsibilities
• Safe and effective play
• Simulation

During functional exercises (FEs), SimCell control is especially important. FEs use a lot of simulated activity, Thus, requiring a robust, detailed MSEL and close communication between site controllers and SimCells.

During full-scale exercises (FSEs) and drills, the exercise assembly area controller plays an integral role. Thus, the exercise assembly area controller must be in close communication with controllers and personnel to ensure safe and realistic deployment of resources.

Debriefings immediately follow exercise conduct. They are short meetings conducted with the exercise planning team to discuss levels of satisfaction with the exercise, air any issues or concerns, and propose improvements. Debriefing notes, attendance lists, and participant feedback forms should be included in the meeting outcome.

The Player Hot Wash provides an opportunity for players to discuss strengths and opportunities for improvement. Hot washes are facilitator-led. They are usually brief and constructive. Comments can be used to develop the After Action Report/Improvement Plan (AAR/IP).

The Controller/Evaluator Debriefing is a forum for controllers and evaluators to review the exercise. The exercise planning team leader facilitates this meeting. Captured discussion comments an lessons learned are captured and incorporated into the AAR/IP.

Objectives: Having completed this lesson, you are able to:

• Identify the activities that take place prior to discussion-based exercise conduct.
• Explain the purpose of orientation briefings for the different participant groups.

Objectives: After completing this lesson, you will be able to:

• Identify the information contained in an Exercise Evaluation Guide (EEG).
• Identify the information contained in an exercise evaluation plan.
• Identify how exercise data is collected and analyzed.
• Identify the information contained in an After-Action Report (AAR).

Effective exercise evaluation involves:

• Planning for exercise evaluation
• Observing the exercise and collecting exercise data during conduct
• Analyzing collected data to identify strengths and areas for improvement
• Reporting exercise outcomes in a draft After-Action Report (AAR)

Together, the planning team and lead evaluator should determine which tools and documentation are needed to support the evaluation team. Data collection methods are determined by the lead evaluator.

• Be familiar with the mission areas, core capabilities, plans, policies, and procedures to be examined during the exercise
• Determine the structure of the evaluation team
• Determine the supporting tools and documentation
• Conduct a pre-exercise C/E Briefing
• Recruit, train, and assign additional evaluators

Evaluation Plan development tasks:

• Define evaluation requirements
• Prepare a plan for evaluating the exercise
• Select or develop evaluation forms
• Finalize the evaluation plan

EEGs accomplish several goals:

• Streamline data collection
• Enable thorough assessments of the participant organizations’ capability targets
• Support development of the After-Action Report (AAR)
• Provide a consistent process for assessing preparedness through exercises
• Help organizations map exercise results to exercise objectives, core capabilities, capability targets, and critical tasks for further analysis an assessment

They document the following evaluation requirements for exercise evaluators:

• Core capabilities: Critical elements needed to achieve a specific mission area
• Capability target(s): Performance thresholds for each core capability. Typically written as quantitative or qualitative statements.
• Critical Tasks: Elements required to perform a core capability and describe how the target will be met. Generally, they include activities, resources, and responsibilities required to fulfill capability targets.
• Performance ratings: Description of performance against target levels. Includes target ratings and core capability ratings.

Evaluation Documentation and Tools include:

• Evaluation Plan
  • SitMan or scenario
  • C/E Handbook
• Evaluator Team Organization
  • Assignments
  • Locations
• Evaluation Instructions
  • Instruction on use of tools, logs, and forms
• Evaluation Tools
  • Jurisdiction-specific or organization-specific SOPs
  • EEGs
  • MSEL

• Exercise-specific details: Exercise scenario, schedule of events, and evaluation schedule
• Evaluator team organization, assignments, and locations: A list of evaluator locations, shift assignments, a map of the exercise site(s), evaluation team organizational chart, and team contact information
• Evaluator instructions: Step-by-step instructions for evaluators for activities before, during, and after the exercise
• Evaluation tools: EEGs, MSEL, or a list of venue-specific injects, electronic or manual evaluation logs or data collection forms, relevant plans and procedures, Participant Feedback Forms, and Hot Wash Templates

Discussion-based exercises focus on issues involving plans, policies, and procedures. Observations may consist of an evaluator recording data from participant discussions on the EEGs.

Operations-based exercises focus on issues affecting the operational execution of critical tasks and capabilities. Observations are recorded from participant actions. These observations form the basis used to determine if critical tasks were successfully demonstrated and capability targets met.

Some additional sources include:

• Event logs
• Video or audio recordings
• Evaluator notes
• Photographs

Additionally, observations should include:

• Actual time required for exercise players to complete the critical task(s)
• How target was or was not met
• Decisions made and information gathered to make decision
• Requests made and how requests were handled
• Resources utilized
• Plans, policies, procedures, or legislative authorities used or implemented
• Any other factors contributed to the outcomes.

Based on their observations, evaluators assign a target rating for each capability target listed on the EEG. Evaluators then consider all target ratings for the core capability and assign an overall core capability rating.

The rating scale includes four ratings:

• Performed without Challenge (P)
• Performed with Some Challenges (S)
• Performed with Major Challenges (M)
• Unable to be performed (U)

Root-cause analysis may also require the review and evaluation of an organization’s plans, policies, and procedures. When completing the analysis, evaluators should consider the following questions:  Root Cause Analysis

• The document that summarizes key information related to the evaluation
• Overview of performance related to each exercise objective and associated core capabilities
• Length, format, and development timeframe of the AAR depend on the exercise type and scope

Elements of the AAR include:

• Exercise overview
• Analysis of core capabilities (The main focus of the AAR)
• Appropriate appendices (Improvement plan and participant list)

AARs also include basic exercise information such as the name and type of the exercise, dates, location, participating organizations, mission areas, sponsor, and POC.

• Review and determine areas for improvement
• Determine organization with responsibility for corrective actions

Objectives: Having completed this lesson, you are able to:

• Identify the information contained in an Exercise Evaluation Guide (EEG).
• Identify the information contained in an exercise evaluation plan.
• Identify how exercise data is collected and analyzed.
• Identify the information contained in an After-Action Report (AAR).

Objectives: After completing this lesson, you will be able to:

• Identify the process of improvement planning, including its inputs and outputs.

Corrective actions may include:

• Changes to plans and procedures, organizational structures, and/or management processes
• Additional training, equipment, or resources

Organizational reviewers determine which issues fall within their purview and take ownership of those issues for action. At this point, a draft improvement plan (IP) is developed.

Meeting Outcomes:

• Final consensus on draft corrective actions
• Develop deadlines for implementation of corrective actions
• Identify corrective action owners and assignees

• Distribute to exercise planners, participants, and other preparedness stakeholders, as appropriate
• Track corrective actions to completion
• Ensure a system is in place to validate previous corrective actions have been successfully implemented (Tracked continuously until implementation)

The principles of continuous improvement are:

• Important to the National Preparedness System
• Consistent approach to strengthening Whole Community preparedness
• Builds capabilities as part of a larger continuous improvement process
• Proven method of issue resolution and information sharing
• Applicable to all operational phases

Objectives: Having completed this lesson, you are able to:

• Identify the process of improvement planning, including its inputs and outputs.

• Develop a baseline knowledge of exercise fundamentals.
• Identify the tasks necessary to complete each phase of the exercise process.
• Define how exercises complete the preparedness process.
• Identify the role of exercises in validating capabilities.
• Identify phases of exercise evaluation and the improvement planning process.

HSEEP Fundamental Principles. You learned the fundamental HSEEP principles of; guidance by elected and appointed officials, the National Preparedness Goal (NPG), the Progressive Planning Approach, the importance of the whole community integration, risk management and a common methodology for exercises.

The reasons for conducting exercises. You learned the reasons for conducting exercises which included- National Preparedness Components, clarifying responsibilities and roles, improving interagency coordination and communication, identifying gaps in resources, developing individual performance, and identifying areas/opportunities for improvement.

Identification of how the participant has shareholder support. You learned how participants have shareholder support by engaging the whole community, and state and local jurisdictions.

Discussion-based exercises. You learned about discussion-based exercises as- a forum for discussing or developing plans and procedures, less complicated than operations-based exercises, focused on strategy and policy.

Operations-based exercises. You learned about operations-based exercises and their characteristics which included: drills, the involvement of deployment of resources and personnel, complexity compared to discussion based exercises, and their advantages including the improvement of individual and team performances.