Benefits of low-fidelity simulations like Emergo Train System (ETS) for healthcare providers emergency preparedness: a scoping review study

Introduction

It is a statutory obligation for UK healthcare providers to deliver high-quality medical care to patients during emergencies (1). Emergency preparedness activities embrace many components, including a complex cycle of planning, preparing resources, training staff, conducting exercises and implementing improvements (2). Emergency preparedness exercises (EPEs) play an essential role in the preparedness cycle. At the individual level, EPEs offer response staff opportunity to learn about emergency plans and procedures experientially through hands-on practice in simulated responses. At the organisational/system level, EPEs help to test the capability of systems to handle major incidents (MIs) and identify gaps in planning, training and resources (3). Indeed, UK healthcare and emergency services have a legal obligation to regularly participate in EPEs to strengthen their preparedness (1).

There are two major types of EPEs—discussion-based exercises and operation-based exercises (2). The purpose of discussion-based exercises is to engage participants in facilitated discussions to familiarise themselves with emergency plans and response roles for different emergency scenarios. Operation-based exercises involve functional elements and are classified into different types depending on the exercise purpose (2). Drills are used to test a specific operation or function under the response plan, typically within a single entity and involving operational staff. Functional exercises (FXs) or command-post exercises (CPXs) are broader in administrative scope than drills and involve both operational and tactical/strategic staff. The most complex type of operation-based exercise is a full-scale exercise (FSX) that offers greatest level of realism by replicating environment, resources, and conditions of a MI as close as possible to the real one. Therefore, FSXs are also referred as high-fidelity exercises. Operation-based simulation exercises typically aim to achieve high environmental and physical fidelity to deliver a ‘realistic’ exercise. Table 1 provides a summary of different types of EPEs.

Exercise fidelity can be defined as the degree of which a simulation accurately represents the physical and mental authenticity by inclusion of stimuli, clues and actions (4). Different EPE can offer different types of fidelity known as environmental, physical, functional or social fidelity depending on the exercise’s objectives and purpose (4,5). Table 2 provides a summary of the types of fidelity in EPE. However, high physical fidelity (having literal representation of physical elements involved in a specific emergency response) or environmental fidelity not necessarily a guarantee of high functional fidelity (6-8), and there is a growing narrative that supports a departure from focusing on replicating physical fidelity in simulation-based training to achieve functional fidelity (5).

This study focuses on one form of operation-based exercise methodology commonly used by healthcare services in the UK and worldwide—Emergo Train System^® (ETS) (9). ETS is known as a low-fidelity functional simulation exercise. While it is acknowledged that environmental and physical fidelity of ETS-type exercise are low, it nonetheless offers conditions of functional and social demands of an emergency incident (10). Whiteboards are used to represent the locations relevant to the incident, such as incident site and various hospital departments and symbols are used to represent casualties, staff involved in the response and available resources (9). One of the main features of ETS is a large bank of casualty profiles (over 800) with a range of injuries and conditions that can be used to create various scenarios across different settings. Casualty symbols (puppets) contain detailed information about casualty’s injuries, physical characteristics and treatment time. Similar style exercises can involve board-game style simulations (11) or the use of computers (12).

For almost a decade the format of the ETS had been used to satisfy UK Cabinet Office requirements for category 1 responders to take part in a live exercise (13). However, since 2016, this arrangement is no longer acceptable. Resultantly, UK hospitals are now expected to take part in a live exercise once every 3 years or find other arrangements to comply with the legal requirements (1). Presumably, live exercises are perceived as a superior method of emergency preparedness training with healthcare staff to ETS due to the higher level of environmental and physical fidelity they offer compared to ETS and ETS-like exercises.

However, the benefits of taking part in live exercises and the benefits of taking part in simulation exercises like ETS for healthcare responders are not conclusively understood. Furthermore, it is not understood how different types and levels of fidelity achieved in live exercises and exercises like ETS affect exercise outcomes as well as the learning retention over time and its translation to daily practices and real responses. The main disadvantage of a FSX is in its cost and resource demand, making them less practical to deliver regularly (14).

The purpose of this review is to collect and analyse evidence related to benefits of high-fidelity live EPEs and benefits of low-fidelity simulations like ETS, to develop better understanding of the impact of simulation exercises’ fidelity on healthcare staff learning from EPEs and on their preparedness to respond in a MI. We present this article in accordance with the PRISMA-ScR reporting checklist (available at https://jphe.amegroups.com/article/view/10.21037/jphe-24-55/rc).

Methods

This study adopted a scoping review framework, which facilitates the comprehensive investigation of an emerging domain (15). Utilising a scoping review method is particularly useful when dealing with a field that lacks consensus of its methodological features and quality of research evidence.

The evidence base for EPEs lacks uniformity in the methodologies used for data collection, evaluation and types of evidence that is reported. Therefore, a scoping review framework is the most suitable review format to review the current state of the evidence-base for EPEs. The study utilised a five-stage framework for scoping review suggested by Arksey and O’Malley (15).

Stage 1: identifying the research question

The study focused on the broad research question: what is known from the existing literature about benefits from FSXs and low-fidelity simulations like ETS on emergency preparedness. The study also attempts to answer the following specific question: what are the impacts of environmental, physical, functional and social fidelity on exercise outcomes for healthcare responders, such as response technical skills (triage, casualty management, decontamination, infection control), non-technical skills (decision making, coordination, communication, situational awareness, teamworking), attitudes, and behaviours.

Stage 2: identifying relevant studies

The search strategy was developed for published studies in major medical databases. A two-step search strategy was used in this review. A preliminary limited search was conducted on PubMed and Embase for studies reporting outcomes for EPEs. Text words in title and abstracts, and keywords and index terms were analysed. A second search was conducted with keywords and index terms identified from the preliminary search across five databases (Embase, Medline, CINAHL, Global Health, and Scopus). The databases were searched using terms related to different types of emergency situations and different descriptive names of types of emergency exercises (see Appendix 1 for search terms). Associated MeSH/subject terms were used when available. A Boolean approach for the search string was adopted; synonyms for different types of emergency situations were combined by the operator “AND” with synonyms for different types of emergency exercises. The search was limited to literature published between 1990 through 2019 in English.

Stage 3: study selection

The literature search was conducted in December 2019 across five databases (Embase, Medline, CINAHL, Global Health, and Scopus). Studies were the first title screened by a single researcher. Qualitative and quantitative studies were both considered. The abstracts were reviewed by two researchers using Covidence, an online systematic review tool. Clearly irrelevant studies were illuminated at this stage. The inclusion and exclusion criteria continued to be refined at this stage (Appendix 2) in order to appropriately capture the full scope of available evidence, as recommended by scoping study methodology (15). Full-text screening was conducted by two researchers to determine inclusion for data extraction. In accordance with scoping review guidelines, the studies were not assessed for quality (15).

Stages 4 and 5: data gathering, charting, organisation and summarising

Data was extracted and tabulated according to: author(s), year of publication, country of publication, exercise classification [ETS-type, FSX, FX, and drill/live drill (LD)], exercise purpose, exercise focus, scope (geographical, organisation, functional, administrative), fidelity (environmental, physical, functional and social), participants, threats, evaluation and outcomes [disaster plan/protocol; fidelity; resources; clinical skills; decision making; communication; teamwork; casualty management; command, control, and coordination (C3)]. Data analysis involved a descriptive numeric summary of publications and exercise characteristics, followed by qualitative analysis of extracted data within categories to provide deeper insights into the nuances of the exercises conduct and their outcomes.

Results

A total of 2,836 studies across five databases were identified (Embase: 611; Medline: 936; CINAHL: 631; Global Health: 230; Scopus: 428), of which 330 studies were duplicates and discarded. A total of 2,506 studies were title screened to assess relevancy. A large number of studies, which were irrelevant based on the title, and were discarded at this stage leaving 282 studies for abstract screening. A total of 182 studies were excluded from abstract screening by mutual agreement between two researchers, leaving 100 studies for full-text screening. Further 64 papers were excluded from full-text screening leaving 36 studies for further detailed analysis (Figure 1). Appendix 3 contains detailed summary of all analysed studies according to the exercise type.

Figure 1 PRISMA flow diagram of the literature selection process.

Year of publication

Figure 2 shows some increase in the number of publications between 2012 and 2016 that has dropped starting from 2017.

Figure 2 Number of publications between 1990 and 2019.

Country of publication

Most papers were published in the USA (n=22, 61%) (16-37). Two publications were identified from the UK (38,39), Sweden (40,41), Israel (42,43), and Australia (44,45). A single publication was identified from the following countries: Malaysia (46), Canada (12), Rwanda (47), Italy (48), Netherlands (49), and Norway (50).

Exercise type

FSXs were discussed in seven studies (16,22,25,28,31,32,44), FX in 13 (17-19,23,26,27,29,30,36,37,45,48,49), LDs in 10 (20,21,24,33-35,38,43,47,50) and ETS in six studies (12,39-42,46). One study reported results on two separate ETS-type exercises (39) and is reported separately therefore, a total of seven ETS-type exercises were reported on from six studies.

Sources of evidence

Most of the reviewed studies (17/36; 47%) were qualitative in nature and reported outcomes either from observations, including structured observation using a performance checklist (12,20,33,40,42-44) or unstructured observation (34,39), or from a post-exercise debrief/after action report (AAR) (22,24,25,28,31,34,39,44). Evaluations of exercises’ effect on participants via pre-post evaluations were reported in 6/36, 17% of studies (24,27,35,38,46,50). Five studies (14%) utilised patient data to assess performance in the exercise (22,23,26,29,50), and three studies (8%) used post-exercise surveys (20,41,42). Additionally, there were five case studies (14%) where exercise outcomes were reported, but information on the evaluation methods was not provided (17,30,32,37,47).

Exercise conduct

Purpose

For FSX, the primary purpose was to improve preparedness (n=4, 57%) (16,28,31,44), and test preparedness or plans (n=2, 29%) (25,32). For FX the major purpose was to test preparedness or plans (n=8, 62%) (17,19,23,26,29,30,36,37) and improve exercise design (methodological; n=3, 23%) (27,48,49). The primary purpose of LDs was to improve preparedness (n=7, 70%) (24,33,34,38,43,47,50), followed by testing preparedness or plans (n=3, 30%) (20,21,35). ETS-type exercises were conducted to evaluate/test plans (12,41), test hospital surge capacity (40) and clinical decision making (42). Most ETS-type exercises also discussed enhancing live exercises design (methodological; n=5, 83%) (12,39,41,42,46). Improvements to exercise design include using puppets (‘Gerbers’) instead of live casualties when transferring patients from the incident site to hospital (39) and the use of additional casualty cards to facilitate a clinical decision making in exercises (42).

Focus

The types of focus were very similar between ETS-type exercises and live exercises, including focus on C3, decision making, communication, casualty transport and management, triage and teamwork. However, none of the ETS exercises focused on decontamination, stockpile distribution and outbreak response.

Scope—geographical

ETS-type exercises displayed a wide range of geographical scope, ranging from being localised in a single hospital (40) to regional coverage in the UK (38). All FSX had a wide geographical scope, inclusive of regional and multi-regional exercises (USA and Australia) and state-wide exercises (USA) (16,25,28,44). FX and LDs had more localised scope, with the majority being localised to a single hospital (FX: n=5, 38%/LD: n=8, 80%) or to multiple hospitals within a restricted geographical region.

Scope—functional

Some exercises across categories had multiple elements under its functional scope. The most prevalent functional scope for ETS-type exercises was hospital casualty management (n=6, 100%) (12,39-42,46), followed by pre-hospital casualty management (n=3, 30%) (39,41,46). A similar tendency was observed with LDs with the focus of functional scope being hospital casualty management (n=8, 80%) (20,21,24,33,34,38,43,50) followed by pre-hospital casualty management (n=5, 50%) (21,35,43,47,50). Hospital casualty management was the dominant functional scope element of FSX too (n=4, 57%), followed by pre-hospital casualty management (22,25,28,32) and outbreak surveillance (16,31,44), (n=3, 43%). The main functional scope of FXs covered hospital casualty management (n=7, 54%) (19,23,26,29,36,48,49), followed by pre-hospital casualty management (n=5, 38%) (3,18,27,30,45).

Scope—organisational

ETS-type exercises included multi-agency exercises (n=3, 43%) (39,41,46), a multi-hospital exercise (n=1, 14%) (39) and single hospital exercises (n=2, 43%) (12,40). FSX only included multi-agency exercises (n=4, 57%) (16,25,31,32) and multi-hospital exercises (n=3, 43%) (22,28,44). Organisational scope of FXs and LDs was similar and primarily involved multi-agencies (n=6, 46%) (FX) (17-19,27,30,45), (n=4, 40%) (LD) (33,43,47,50); and single hospital exercises (n=5, 38%) (FX) (26,29,36,48,49), (n=4, 40%) (LD) (21,24,34,38).

Scope—administrative

Operational staff was involved in all exercises reviewed in this study. In addition, four ETS exercises were operational and strategic-staff focused exercises (n=4, 57%) (12,39-41) and only a single ETS-type exercise involved strategic, tactical and operational staff (14%) (39). All FSX involved operational and strategic staff (n=7, 100%), and two exercises involved operational, tactical and strategic staff (n=2, 29%) (16,44). For FX, the majority of exercises involved operational and strategic staff (n=9, 69%). Only a single FX reported involving operational, tactical and strategic staff (8%) (49). All LDs only involved operational staff.

Fidelity—environmental

Environmental fidelity of ETS-type exercises is low; whiteboards are used to represent hospital departments involved in the exercises (n=4, 67%) (39-41,46). One ETS-type exercise reported conducting the exercise using computers (12). FSX utilised simulated incident sites and simulated incident command centres (n=6, 86%) (16,22,25,28,32,44). Similarly, FX utilised simulated incident command centres and simulated dispensary/decontamination sites. Both FXs and LDs included atmospheric affects such as the use of simulated smoke (27), sound effects to simulate shooting scenario (24) temperature changes and general ambient noises (38) to enhance environmental fidelity. Most of FSXs, FXs, and LDs were conducted in-situ whereas only one ETS-type exercise was conducted in-situ (42).

Fidelity—physical

Live exercises featured multiple different elements to achieve physical fidelity. All categories of live exercise included moulaged human casualties, some used real communication systems/technologies [FSX (n=3, 42%) (22,28,31); FX (n=3, 23%) (19,36,49); real medical and decontamination tools: FSX (n=1) (22), FX (n=7, 53%) (17-19,23,30,37,45), LDs (n=8, 80%) (20,21,24,33,35,38,43,50); and real transport vehicles/facilities: FSX (n=1) (31), FX (n=2, 15%) (18,30), LDs (n=1, 10%) (47)]. In ETS-type exercises casualties were represented by paper casualty puppets and medical resources and tools were represented by magnetic symbols (n=6, 100%) (12,39-42,46) (with one study additionally including moulaged casualty actors (39). One FSX mentioned using ‘casualty information cards’ alongside moulaged actors (25).

Fidelity—functional

Functional fidelity for all considered exercise types was achieved through realistic scenarios, real-time simulations, time and resource pressures and realistic tasks and challenges. While ETS-type exercises functional fidelity was achieved via the use of real-time response elements, FSXs, FXs, and LDs also featured behavioural challenges, such as language barriers between participants and casualties and the need for interpreters (31) and low instructional compliance of casualties in decontamination exercises (45), to enhance functional fidelity. Four FSX studies utilised injects and simulated media- and community-related concerns (16,28,32,44). Similarly, three FX studies also reported simulating media-related pressures such as having to deal with reporters demanding interviews and communicating statements to the public (17,19,48).

Fidelity—social

Social fidelity was similar across all types of exercises considered, with emphasis given to achieving balanced representation of relevant roles and agencies that would be involved in a real MI response.

Participants

Participants in ETS-type exercises were predominantly operational clinical staff, including ambulance and hospital staff; incident emergency command and managerial staff have also been involved. In addition to clinical staff other types of live exercises considered in this study involved fire, military, police, scientific and volunteers as exercise participants.

Type of threats

ETS-type exercises addressed various mass casualty threats, such as building fires (39,41), car collisions (12,42), explosions (39,40,42), and an airport disaster (46). All other live exercises reviewed in this study also addressed mass casualty-related threats (n=16, 44%) but considered other types of threats: including chemical, biological, radiological, or nuclear (CBRN) terrorism events (n=11, 31%) of different scales (city-wide bioterrorism attacks and localised intentional release of the plague), infectious disease outbreak of pathogens such as anthrax and influenza (n=8, 22%), and civilian rescue missions (n=1, 3%) (Table 3).

Evaluation

ETS-type exercises and live exercises all used a range of different types of evaluation methods however, the most popular form of evaluation for ETS-type and live exercises was structured observation: ETS-type exercises (n=4, 67%), FSX (n=4, 57%), FX (n=5, 33%), and LDs (n=5, 50%). Other types of evaluation applied across all exercise types included hot debriefs, post-exercise, and pre-post exercise surveys. Cold debrief was reported only for one FXS as part of the exercise evaluation (28).

Exercise outcomes

In analysis of study outcomes, different themes have emerged that included plans/protocols, fidelity, clinical skills, resources, decision-making, communication, teamwork, casualty management, and C3. Results for each of these themes are reported separately.

Plans and protocols

An opportunity to observe participants adherence to plans and protocols was reported as an outcome across FSX, FX, and ETS-type exercises. Similar to live type exercises, studies of ETS-type exercises reported identification of gaps in plans/protocols (40,46), improved understanding of roles (46) and modification of plans/protocols according to exercise outcomes (12). One FX identified that legal implications for external personnel were not covered in emergency response protocols (49). For FSX the implementation of revised plan in real response was reported (44). LDs reported measuring adherence to plans/protocols (21,47) and identifying gaps or issues (35).

Fidelity

For ETS-type exercises, one publication reported that fidelity offered by the exercise allowed almost identical surgical decisions to be made during the exercise to those surgical decisions made in the original response, from which the casualty database and exercise were modelled (40). One ETS exercise reported that participants felt that realistic pressure was achieved due to the real-time simulation and the rate of casualty flow (12). Similarly, FSX reported that the element of real-time simulation enhanced fidelity and helped to achieve realistic pressure in order to observe communication and decision-making (31). The combination of real-time simulation and the use of human role-play casualty actors were used to achieving full immersion in this FSX. The use of role-play actors to achieve physical fidelity helped participants to identify unforeseen obstacles such as language barriers (26,31). For FXs, the only reference made to fidelity was that the time limitations of the exercise did not realistically replicate response conditions. For in-situ LD, the use of role-play actors made it difficult to distinguish between real casualties and simulated casualties thus compromising the safety of patients attending hospitals during the exercise (21). Benefits from in-situ simulation included an ability to identify unforeseeable obstacles such as the risk of casualties in decontamination exercises entering hospital areas non-designated for decontamination and risking contaminating the hospital (32,45).

Resources

ETS-type exercises allowed identification of medical resource inefficiencies (39,40) and strengths in resource coordination and mobilisation (41). For FSX, a range of observations were made in relation to resources including issues relating to clinical resource availability (31,32), resource delivery (44), and the efficiency of resource anticipation and requesting (25,31). Specific lacking in resources such as hospital security and psychological support were also reported in FSX (32). Issues with the efficiency of available clinical resources were identified in FX, including delays in resource delivery and provision (29,30) and the need to identify task-specific teams (19). For LDs, the primary observations were related to identifying inefficiencies with clinical resources/equipment (20,21,33) and assessment of participant skills and training level when using equipment (33,35).

Clinical skills

Accuracy of triage was reported with respect to clinical skills for ETS-type exercises (40,42). All live exercises (FSXs, FXs, and LDs) also made observations relating to the accuracy of triage and identified obstacles to triage relating to poor team management (18,19,21,25,32,34,48). For FSX additional observations related to the quality of participant medication management were reported (32). FX reported observations about applications of infection control procedures and the accuracy of symptoms’ categorisation (23,26). LDs reported on casualty assessment (20), diagnostic and treatment accuracy (43), and triage accuracy (21,34,47).

Casualty management

One ETS-type exercise reported the adoption of novelty casualty care approaches (12), FSXs identified inefficient casualty information management and tracking in some hospitals (25), issues around the accuracy of decontamination procedures, as well as the efficiency of casualty transport and transfer and improvements in casualty coordination and hospital pathways (28). FXs reported on the efficiency of casualty transport and transfer, highlighted issues faced with hospitals receiving casualties, issues with decontamination resources and procedures (30,45). LDs reported on the efficiency of casualty transport and transfer, improvements in casualty care as well as issues with decontamination procedures (21) and casualty treatment (33).

Decision making

Decision-making was observed as an exercise outcome in two ETS-type exercises (39,41), one FSX (31), five FXs (19,26,29,30,37), and a single LD (38). Improved decision-making was reported for two ETS-type exercises, which reported on the quality of resource allocation related to decision-making (41), including a hospital ambulance liaison control (39). With regards to live exercises, real-time exercising helped identify complex decision-making and means to facilitate them in FSX (31); observations were made about the quality of decision making for FX relating to medical treatment and delegation of responsibility to personnel (26,29,30,37) and LDs observed improvements in decision making (38).

Communication

ETS-type exercises reported the examination of issues around inter-agency communication (39,41). For FSX, observations were made on the efficiency of inter-agency use of communication technology and general issues faced with communication (22,28,31). FXs observed the efficiency with use of communication technology and highlighted issues with inter-agency communication (31) as well as a lack of communication training (19). LDs made observations on the importance of practicing communication, observed improvements in communication and commented on the efficiency of inter-agency communication (35).

Teamwork

ETS-type exercises did not report any observations on teamwork. FSX exercises demonstrated good inter-agency transfer of duties and reported on improvements in teamworking (25). FX exercises reported improved inter-agency relationships (27). LDs reported changes in participants’ perceptions about inter-agency teamwork and improvements in inter-agency teamwork (35).

Command, control, and coordination

ETS-type exercises did not report any outcomes related to C3. Outcomes on C3 were reported in four FSXs (16,22,25,32), in five FXs (17,29,30,37,48) and in two LDs (34,35). FSXs and FX demonstrated existence of the established points of authority, assessed the effectiveness of control centre communication practices and overall coordination with other zones/departments/agencies (including military personnel) (16,17,22,29,32,48). Difficulties with crowd control and the implications of the law enforcement personnel absence were reported in one FX (30), and the importance of the incident command to maintain the response oversight rather take part in the front-line response was demonstrated in another FX exercise (23). LDs reported an improved knowledge of command structure and identified issues relating to the presence of established points of authority (35).

Table 4 provides a summary of exercise design and outcomes for live exercises and ETS-type exercises.

Discussion

FSXs play an important role in emergency preparedness activities by offering to practise responding to a MI in the closest to reality way as it could be possible. Typically, FSX scenario involves responding to a complex emergency situation covering wide geographical area with a variety of response partners from multiple organisations and agencies. For health emergency responders such exercises provide an option to test organisational preparedness and practise responding with response partners. It is believed that the closer learning environment represents the real situation the easier it will be for responders to transfer learning from such exercises into a real response (51). However, the major disadvantage of FSX is in their resource and cost demands (14). Understanding learning benefits achieved from lower-fidelity low-cost simulations, like ETS, will provide further evidence regarding the appropriateness of ETS in emergency preparedness of health care providers and direct emergency preparedness activities.

Systematic analysis of the data conducted in this study allowed to define similarities and differences in the design and outcomes for major live exercises, including FSXs, FXs, LDs, and ETS-type exercises, which can be of importance for exercise planners. ETS exercises are not run to test preparedness, unlike FSX, but interestingly were used to improve the design of FSX, by allowing more detailed information on patients and thus greater focus on clinical management of casualties through detailed patient information provided by the ETS (39). Focus of ETS-type exercises was similar to the focus of live exercises, including casualty transport and management, decision making, communication, triage and teamwork. Even though FSXs tend to cover a wide geographical area in the response, geographical scope of ETS exercises included multi-agency regional response in addition to a single hospital or multi-hospital exercises. Administrative scope of FSX and ETS was similar and included staff in all three major response categories: operational, tactical, and strategic. Participants in ETS exercises were mainly clinical staff while FSX included other agencies such as police, fire and local authorities. The only type of threat practised in ETS exercises was mass casualty incidents; live exercises dealt with wider range of threats, including CBRN terrorism, infectious diseases outbreaks as well as mass casualty threats.

Detailed analysis of outcomes identified areas where ETS-type exercises produced similar results to FSX. Similar to live type exercises, ETS allowed identification of gaps in plans/protocols, improved understanding of roles and stimulated modification of plans/protocols according to exercise outcomes. ETS outcomes reported limitations in resource utilisation and coordination; although it is recognised that due to the low environmental and physical fidelity, only limited observations and impacts on the element of resources can be expected from ETS-type exercises. Triage and medical management of patients were the most common clinical skills reported from ETS exercises, similar to live exercises.

All simulation exercises offer different elements of fidelity (environmental, physical, social and functional) to a real incident, but the way this is achieved in different exercises differs. Environmental and physical fidelity of ETS exercise is low, while emphasis is made to recreate the scene of an incident, treatment facilities, patients and tools in FSX and other types of live exercises. However social fidelity, achieved by ensuring that all relevant agencies and staff involved in the response are taking part in the exercise, is similar in ETS and FSX. Functional fidelity achieved in ETS and FSX is largely similar too and achieved by offering realistic scenarios, information exchange, time and resource pressures to simulate clinical, resource and coordination challenges during the response. However behavioural challenges cannot be simulated in ETS-type exercises. FSXs and LDs use casualty actors to simulate casualties who can be trained or instructed specifically to create behavioural-related challenges for participants to respond to, thus more accurately simulating a real emergency response situation and thus offering higher functional fidelity.

However, observation made across all categories of live exercises clarified that moulaged casualty actors utilised casualty information cards, like ETS, to provide more detailed clinical information for exercise participants to practise their roles (25,43,45). This suggests that in some instances casualty actors are unable to adequately portray clinical features for exercise participants to functionally carry out their roles and utilise additional information, like ETS casualty cards/actors, to augment functional fidelity (39,40). This demonstrates that simply attaining high physical fidelity is not adequate to achieve high functional fidelity, in the context of the objectives of these exercises, in line with previous studies (6-8). Moreover, using casualty actors caused confusion with distinguishing between real casualties and actor-casualties during a LD exercise (21), which could even compromise the safety of real-casualties not involved in the exercise that are attending clinical facilities taking part in the exercise. This supports concern with the use of casualty-actors in live exercises (5,52). ETS-type exercises are conducted with minimal disturbance to the normal organisational function as they do not involve live casualty actors and do not need to be conducted in-situ.

The evidence suggests that learning from ETS is transferrable as experience in a recent ETS made a significant difference in the healthcare staff response to a real mass casualty incident (53).

Study limitations

Only studies that were identified through database searches were included- grey literature and unpublished studies were not included in this review. Scientific quality of included studies was not assessed due to the lack of high-quality scientific evidence in this area and is permissible by the scoping study methodology. Most studies were carried out in the USA; unique local regulatory and cultural issues mean that care must be taken when extrapolating or applying to other contexts. Only a limited number of publications related to ETS-type exercises have been identified and publication bias cannot be excluded.

Conclusions

The study provides further evidence to support the value of low-fidelity simulations, like ETS in preparing healthcare staff to respond to mass casualty incidents. This comprehensive review of features and outcomes achieved from high-fidelity live exercises and low-fidelity ETS-type exercises can be used as a reference in the design and delivery of emergency preparedness training for healthcare staff.

Acknowledgments

Funding: This study was funded by the National Institute for Health and Care Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response, a partnership between the UK Health Security Agency, King’s College London and the University of East Anglia (funding reference NIHR200890).

Footnote

Reporting Checklist: The authors have completed the PRISMA-ScR reporting checklist. Available at https://jphe.amegroups.com/article/view/10.21037/jphe-24-55/rc

Peer Review File: Available at https://jphe.amegroups.com/article/view/10.21037/jphe-24-55/prf

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jphe.amegroups.com/article/view/10.21037/jphe-24-55/coif). All authors reported that this study was funded by the National Institute for Health and Care Research Health Protection Research Unit (NIHR HPRU) in Emergency Preparedness and Response, a partnership between the UK Health Security Agency, King’s College London and the University of East Anglia (funding reference NIHR200890). The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. NHS. NHS Core Standards for EPRR. 2020 [cited 17.09.2020]. Available online: https://www.england.nhs.uk/ourwork/eprr/gf/#core
2. European Centre for Disease Prevention and Control. Handbook on simulation exercises in EU public health settings. 2014 [cited 17.09.2020]. Available online: https://www.ecdc.europa.eu/en/publications-data/handbook-simulation-exercises-eu-public-health-settings
3. Skryabina E, Reedy G, Amlôt R, et al. What is the value of health emergency preparedness exercises? A scoping review study. Int J Disaster Risk Reduct 2017;21:274-83. [Crossref]
4. Tun JK, Alinier G, Tang J, et al. Redefining simulation fidelity for healthcare education. Simulation & Gaming 2015;46:159-74. [Crossref]
5. Hamstra SJ, Brydges R, Hatala R, et al. Reconsidering fidelity in simulation-based training. Acad Med 2014;89:387-92. [Crossref] [PubMed]
6. Havighurst LC, Fields LE, Fields CL. High versus low fidelity simulations: does the type of format affect candidates performance or perceptions. In: Proceedings from the 27th Annual IPMAAC Conference on Personnel Assessment. 2003:22-5.
7. Norman G, Dore K, Grierson L. The minimal relationship between simulation fidelity and transfer of learning. Med Educ 2012;46:636-47. [Crossref] [PubMed]
8. Toups Dugas PO, Kerne A, Hamilton WA. The Team Coordination Game: Zero-fidelity simulation abstracted from fire emergency response practice. ACM Transactions on Computer-Human Interaction 2011;18:1-37. (ToCHI). [Crossref]
9. Emergo Train System. ETS Training Material. 2020 [cited 17.09.2020]. Available online: https://www.emergotrain.com/index.php?option=com_content&view=article&id=107&Itemid=804
10. Waring S, Skryabina E, Goodwin D, et al. What components of emergency preparedness exercises improve healthcare practitioners’ emergency response learning? Int J Disaster Risk Reduct 2021;62:102357. [Crossref]
11. McGrath D, Hill D. UnrealTriage: A game-based simulation for emergency response. In: The Huntsville Simulation Conference. 2004.
12. Franc-Law JM, Bullard M, Della Corte F. Simulation of a hospital disaster plan: a virtual, live exercise. Prehosp Disaster Med 2008;23:346-53. [Crossref] [PubMed]
13. Cabinet Office. Emergency planning and preparedness: exercises and training. 2013 [cited 17.09.2020]. Available online: https://www.gov.uk/guidance/emergency-planning-and-preparedness-exercises-and-training#:~:text=Exercises%20are%20both%20a%20type,to%20test%20well%2Destablished%20procedures
14. Farra SL, Gneuhs M, Hodgson E, et al. Comparative Cost of Virtual Reality Training and Live Exercises for Training Hospital Workers for Evacuation. Comput Inform Nurs 2019;37:446-54. [Crossref] [PubMed]
15. Arksey H, O'Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2005;8:19-32. [Crossref]
16. Crandall M. This is an exercise: Operation Red Clover 1. Vermont Nurse Connection 2004;7:9.
17. Denny J, Aakko E, Holm R, et al. Mass dispensing in a suburban county: lessons in a full-scale bioterrorism exercise. Northwest Public Health 2005;22:8-9.
18. FitzGerald DJ, Sztajnkrycer MD, Crocco TJ. Chemical weapon functional exercise--Cincinnati: observations and lessons learned from a "typical medium-sized" city's response to simulated terrorism utilizing Weapons of Mass Destruction. Public Health Rep 2003;118:205-14. [Crossref] [PubMed]
19. Hoffman RE, Norton JE. Lessons learned from a full-scale bioterrorism exercise. Emerg Infect Dis 2000;6:652-3. [Crossref] [PubMed]
20. Jasper E, Miller M, Sweeney B, et al. Preparedness of hospitals to respond to a radiological terrorism event as assessed by a full-scale exercise. J Public Health Manag Pract 2005;S11-6. [Crossref] [PubMed]
21. Jung D, Carman M, Aga R, et al. Disaster Preparedness in the Emergency Department Using In Situ Simulation. Adv Emerg Nurs J 2016;38:56-68. [Crossref] [PubMed]
22. Klima DA, Seiler SH, Peterson JB, et al. Full-scale regional exercises: closing the gaps in disaster preparedness. J Trauma Acute Care Surg 2012;73:592-7; discussion 597-8. [Crossref] [PubMed]
23. Kohlhoff SA, Crouch B, Roblin PM, et al. Evaluation of hospital mass screening and infection control practices in a pandemic influenza full-scale exercise. Disaster Med Public Health Prep 2012;6:378-84. [Crossref] [PubMed]
24. Kotora JG, Clancy T, Manzon L, et al. Active shooter in the emergency department: a scenario-based training approach for healthcare workers. Am J Disaster Med 2014;9:39-51. [Crossref] [PubMed]
25. McElroy JA, Steinberg S, Keller J, et al. Operation continued care: A large mass-casualty, full-scale exercise as a test of regional preparedness. Surgery 2019;166:587-92. [Crossref] [PubMed]
26. Nathawad R, Roblin PM, Pruitt D, et al. Addressing the gaps in preparation for quarantine. Prehosp Disaster Med 2013;28:132-8. [Crossref] [PubMed]
27. Perry RW. Disaster exercise outcomes for professional emergency personnel and citizen volunteers. Journal of Contingencies and Crisis Management 2004;12:64-75. [Crossref]
28. Petinaux B, Valenta AL, Deatley C, et al. District of Columbia Emergency Healthcare Coalition Burn Mass Casualty Plan: Development to Exercise Date. J Burn Care Res 2017;38:e299-305. [Crossref] [PubMed]
29. Shah VS, Pierce LC, Roblin P, et al. Waterworks, a full-scale chemical exposure exercise: interrogating pediatric critical care surge capacity in an inner-city tertiary care medical center. Prehosp Disaster Med 2014;29:100-6. [Crossref] [PubMed]
30. Siegel D, Younggren BN, Ness B, et al. Operation Castle Cascade: managing multiple casualties from a simulated chemical weapons attack. Mil Med 2003;168:351-4. [Crossref] [PubMed]
31. Stone KW, Morehead BF, Karaye I, et al. Evaluating the effectiveness of a full-scale exercise of epidemiologic capacity for bioterrorism response. J Homel Secur Emerg Manag 2018;15:20170061. [Crossref]
32. Tyre TE. Wake-up call: a bioterrorism exercise. Mil Med 2001;166:90-1. [Crossref] [PubMed]
33. Vinson E. Managing bioterrorism mass casualties in an emergency department: lessons learned from a rural community hospital disaster drill. Disaster Manag Response 2007;5:18-21. [Crossref] [PubMed]
34. Wexler B, Flamm A. Lessons Learned From an Active Shooter Full-Scale Functional Exercise In a Newly Constructed Emergency Department. Disaster Med Public Health Prep 2017;11:522-5. [Crossref] [PubMed]
35. Peterson DM, Perry RW. The impacts of disaster exercises on participants. Disaster Prevention and Management: An International Journal 1999;8:241-55. [Crossref]
36. Weil KM. Lockdown: A bioterrorism drill provides valuable information. Am J Nurs 2003;103:64CC-64GG.
37. Thew J. Local health department shines during emergency drill. Nursing Spectrum-Greater Chicago 2003;16:13.
38. Arora S, Cox C, Davies S, et al. Towards the next frontier for simulation-based training: full-hospital simulation across the entire patient pathway. Ann Surg 2014;260:252-8. [Crossref] [PubMed]
39. Riley PW, Dalby DJ, Turner EA. Making acute hospital exercises more realistic without impacting on healthcare delivery. J Bus Contin Emer Plan 2012;6:143-50. [Crossref] [PubMed]
40. Lennquist Montán K, Riddez L, Lennquist S, et al. Assessment of hospital surge capacity using the MACSIM simulation system: a pilot study. Eur J Trauma Emerg Surg 2017;43:525-39. [Crossref] [PubMed]
41. Nilsson H, Jonson CO, Vikström T, et al. Simulation-assisted burn disaster planning. Burns 2013;39:1122-30. [Crossref] [PubMed]
42. Ashkenazi I, Ohana A, Azaria B, et al. Assessment of hospital disaster plans for conventional mass casualty incidents following terrorist explosions using a live exercise based upon the real data of actual patients. Eur J Trauma Emerg Surg 2012;38:113-7. [Crossref] [PubMed]
43. Leiba A, Goldberg A, Hourvitz A, et al. Who should worry for the "worried well"? Analysis of mild casualties center drills in non-conventional scenarios. Prehosp Disaster Med 2006;21:441-4. [Crossref] [PubMed]
44. Eastwood K, Durrheim D, Merritt T, et al. Field exercises are useful for improving public health emergency responses. Western Pac Surveill Response J 2010;1:12-8. [Crossref] [PubMed]
45. Edwards NA, Caldicott DG, Eliseo T, et al. Truth hurts--hard lessons from Australia's largest mass casualty exercise with contaminated patients. Emerg Med Australas 2006;18:185-95. [Crossref] [PubMed]
46. Idrose AM, Adnan WA, Villa GF, et al. The use of classroom training and simulation in the training of medical responders for airport disaster. Emerg Med J 2007;24:7-11. [Crossref] [PubMed]
47. Mbanjumucyo G, Nahayo E, Polzin-Rosenberg N, et al. Major incident simulation in Rwanda: A report of two exercises. Afr J Emerg Med 2018;8:75-8. [Crossref] [PubMed]
48. Djalali A, Carenzo L, Ragazzoni L, et al. Does Hospital Disaster Preparedness Predict Response Performance During a Full-scale Exercise? A Pilot Study. Prehosp Disaster Med 2014;29:441-7. [Crossref] [PubMed]
49. Haverkort JJ, Biesheuvel TH, Bloemers FW, et al. Hospital evacuation: Exercise versus reality. Injury 2016;47:2012-7. [Crossref] [PubMed]
50. Rehn M, Vigerust T, Andersen JE, et al. Major incident patient evacuation: full-scale field exercise feasibility study. Air Med J 2011;30:153-7. [Crossref] [PubMed]
51. Ford JK, Schmidt AM. Emergency response training: strategies for enhancing real-world performance. J Hazard Mater 2000;75:195-215. [Crossref] [PubMed]
52. Haji FA, Cheung JJ, Woods N, et al. Thrive or overload? The effect of task complexity on novices' simulation-based learning. Med Educ 2016;50:955-68. [Crossref] [PubMed]
53. Skryabina EA, Betts N, Reedy G, et al. The role of emergency preparedness exercises in the response to a mass casualty terrorist incident: A mixed methods study. Int J Disaster Risk Reduct 2020;46:101503. [Crossref] [PubMed]