Construction of an evaluation system for public first aid education and training based on the Kirkpatrick model
Highlight box
Key findings
• A comprehensive evaluation system for public first aid education and training was developed based on the Kirkpatrick model, comprising four domains (reaction, learning, behavior, and outcomes), 12 secondary indicators, and 45 tertiary indicators. High expert authority coefficients (0.85–0.95) indicate strong reliability and consensus.
What is known and what is new?
• The Kirkpatrick model is widely used to assess training effectiveness, but its application in public first aid education lacks standardized and context-specific indicators.
• This study establishes a structured, multi-level evaluation system tailored to public first aid training, integrating qualitative insights from emergency professionals and expert consensus to enhance its scientific rigor and practical relevance.
What is the implication, and what should change now?
• This evaluation system provides a practical tool for systematically assessing and improving public first aid training programs. Its adoption can support evidence-based optimization of training design, enhance learning and behavioral outcomes, and inform public health policy. Standardized evaluation frameworks should be incorporated into future first aid education initiatives to improve training quality and effectiveness.
Introduction
With the accelerating global population aging and the increasing burden of chronic diseases, emergencies and unintentional injuries have become major threats to public health. In particular, in critical situations such as out-of-hospital cardiac arrest (OHCA), timely and effective on-site first aid interventions play an important role in improving patient survival and prognosis (1). Substantial evidence has demonstrated that bystander-initiated cardiopulmonary resuscitation (CPR) can significantly increase the chances of survival (2,3). Therefore, enhancing public awareness of first aid and improving first aid competencies have become essential components of strengthening public health systems and improving societal emergency response capacity.
Public first aid education and training have increasingly attracted attention from governments and health institutions worldwide. In recent years, various forms of first aid training programs have been widely implemented both domestically and internationally, including community-based training, school-based education, and online public education initiatives (4-6). These interventions have improved the public’s knowledge of first aid and basic operational skills. However, most existing studies primarily focus on the short-term evaluation of knowledge acquisition and skill performance following training, typically using questionnaire surveys or skill assessments. There remains a lack of systematic evaluation of post-training behavioral transformation, actual rescue performance, and long-term societal impact. Furthermore, considerable heterogeneity exists across studies in terms of evaluation content, indicator selection, and assessment methods. The absence of a unified and standardized evaluation framework limits the generalizability of findings.
From the perspective of training evaluation theory, the Kirkpatrick four-level assessment model is a widely recognized framework encompassing four dimensions: reaction, learning, behavior, and outcomes, and it systematically captures the entire process from training input to final impact (7,8). This model has been extensively applied in medical education, nursing training, and continuing education, demonstrating strong structural integrity and applicability in evaluating training effectiveness (9-11). However, in the field of public first aid education and training, existing studies have generally adopted only partial aspects of this model, with a primary focus on the reaction and learning levels. The systematic integration and in-depth application of the behavior and outcome levels remain limited, particularly in the evaluation of broader societal impact and cost-effectiveness.
Therefore, establishing a theoretically grounded and well-structured evaluation framework for public first aid education and training is of great significance for providing a structured approach to the assessment of training programs. Based on a systematic literature review, qualitative interviews, and expert consultation, this study developed a public first aid training evaluation framework grounded in the Kirkpatrick four-level evaluation model. The framework encompasses four levels—reaction, learning, behavior, and outcomes—and provides a theoretical foundation for the future evaluation and validation of public first aid education and training programs. We present this article in accordance with the COREQ reporting checklist (available at https://jphe.amegroups.com/article/view/10.21037/jphe-2026-0033/rc).
Methods
Based on literature review and qualitative study, this study summarized the indicators of public first aid education and training and created the first draft of the index system. Subsequently, the expert panel consultation was used to invite experts to analyze and discuss the initially developed public first aid training evaluation system, and to further refine its first draft.
The study was conducted in accordance with the Declaration of Helsinki and its subsequent amendments. This study was approved by the medical ethics committee of Hunan Provincial People’s Hospital (The First Affiliated Hospital of Hunan Normal University; No. [2024]-166). Informed consent was obtained from all individual participants.
Establishment of the research team
The research team consisted of 9 members, including two associate chief physicians from the emergency medicine department, one emergency physician, two chief nurses, and one chief nurse specializing in emergency and critical care, all with more than 5 years of professional experience. In addition, two certified first aid training instructors and two full-time postgraduate nursing students participated in the study. The team’s responsibilities included conducting literature reviews to support the theoretical framework, synthesizing evidence-based findings, designing interview outlines, conducting interviews, collecting and analyzing the data.
Theoretical research
This study adopted the Kirkpatrick evaluation model proposed by Donald L. Kirkpatrick (10). This model is widely recognized as an effective approach for training evaluation, as it systematically captures training outcomes and supports multi-source data analysis. The framework comprises four levels: reaction, which assesses participants’ satisfaction with the training; learning, which evaluates gains in knowledge and skills; behavior, which examines the application of learning in practice; and outcomes, which measure the overall impact on organizational performance. The overall framework development process is illustrated in Figure 1.
Evidence-based integration
The search strategy for literature was conducted using the following subject terms and free-text keywords: “first aid”, “cardiopulmonary resuscitation”, “CPR”, “public”, “residents”, “lay public”, “general public”, “training”, and “popular science training”. China National Knowledge Infrastructure (CNKI), Wanfang, VIP, Sionmed, PubMed, Web of Science, Embase, Cochrane Library, and CINAHL were searched for this review from January 2019 to June 2024. According to the inclusion criteria, two reviewers independently assessed the literature. After deduplication, titles and abstracts were screened, followed by full-text evaluation.
Studies were included if they: (I) focused on first aid or CPR-related education/training; (II) targeted the lay/public population (e.g., community residents, non-healthcare professionals); (III) reported structured educational or training interventions or evaluation outcomes; and (IV) were original research articles or guideline. Only English and Chinese-language studies were included.
Studies were excluded if they: (I) focused exclusively on professional healthcare workers or medical/nursing students; (II) were editorials, commentaries, protocols, or conference abstracts without full data; or (III) lacked sufficient information on training content or outcomes.
Two reviewers independently (Y.T. and H.L.) screened all identified records. After deduplication, titles and abstracts were screened, followed by full-text assessment. Any discrepancies between the two reviewers were resolved through discussion and consensus, with a third senior reviewer (Z.S.) consulted when necessary. Data extraction was then performed for all included studies. The screening process is presented in Figure 2.
Qualitative study method
Selecting interview participants
From May 2024 to October 2024, physicians, nurses, and other healthcare professionals specializing in emergency and critical care, as well as those engaged in emergency and critical care research in a tertiary hospital in Hunan Province, were recruited as interview participants through purposive sampling.
Potential participants were identified through hospital department rosters and professional research networks, and were invited by the research team via direct contact. A total of 16 interview participants were recruited, including 7 intensive care specialist physicians and 9 intensive care unit (ICU) nurses. The inclusion criteria are as follows: physicians were required to have more than 10 years of working experience at or above the associate senior level and at least 5 years of experience in first aid popular science training; nurses were required to have at least 5 consecutive years of experience in emergency and critical care, as well as more than 3 years of experience in first aid popular science training; and researchers in critical care nursing were required to have at least 3 years of relevant research experience in these fields. The sample size of interviewees followed the criterion of data saturation, meaning interviews were terminated once no novel information emerged from the interview content.
Interview process
Data were collected through individual semi-structured interviews conducted in a quiet and comfortable setting. Prior to each interview, the researcher introduced their identity, explained the study objectives and content in detail, and clearly informed participants that the collected data would be used solely for academic purposes, with strict protection of personal privacy. Informed consent was obtained before proceeding. All interviews were audio-recorded with participants’ permission. During the interviews, the researcher maintained a patient and attentive approach, listening carefully and documenting participants’ behavioral and emotional responses in real time. Each interview lasted approximately 20–30 minutes. All audio recordings were transcribed into text within 24 hours after completion of the interview. The detailed structure of the interview is outlined in Table S1.
The coding process of interview data
After completing the expert interviews, researchers need to complete the text transcription of the audio materials within 24 hours. This study adopted Colaizzi’s phenomenological method and systematically analyzed the interview transcripts according to its seven-step analytical procedure (12). Specifically, the analysis involved the following steps. First, the interview transcripts and field notes were read repeatedly to obtain an overall understanding of the data. Second, significant statements closely related to the research topic were identified and extracted. Third, meanings were formulated from these significant statements. Fourth, similar meanings were grouped into thematic categories, and the boundaries of each theme were clarified. Fifth, the relationships between the identified themes and the research phenomenon were integrated to develop a comprehensive description. Sixth, the essential structure and underlying meaning of the phenomenon were identified. Finally, the preliminary findings were compared with the original interview data and returned to the interview participants for member checking to verify the accuracy of the analysis.
Data collection continued until data saturation was reached, defined as the point at which no new themes or conceptual information emerged from additional interviews. Saturation was determined through concurrent data analysis conducted alongside data collection. Any discrepancies in coding between the two analysts (Y.T. and H.L.) were resolved through discussion and consensus. When disagreement persisted, a third senior researcher (Z.S.) was consulted to reach agreement. Member checking was conducted by returning preliminary findings to selected participants to ensure accuracy and credibility of the analysis. Guided by the Kirkpatrick evaluation model, the interview findings were further synthesized to identify key themes and generate statements with both theoretical and practical significance.
Expert panel meeting
Expert selection
A total of 14 experts were invited to participate in the expert panel meeting. The inclusion criteria for experts in this study were as follows: (I) emergency medicine experts were required to hold a bachelor’s degree or above, possess an associate senior professional title or higher, and have more than 20 years of clinical experience in emergency medicine; (II) critical care nursing experts were required to hold a bachelor’s degree or above, possess an intermediate professional title or higher, and have more than 15 years of experience in critical care clinical nursing; (III) experts in critical care research were required to hold a doctoral degree or above, possess an associate senior professional title or higher, and have more than 10 years of experience in critical care research; (IV) experts engaged in first aid science popularization and training were required to hold a bachelor’s degree or above, possess an intermediate professional title or higher, and have more than 5 years of experience in first aid science popularization and training.
Themes for the expert panel meeting
Themes of the expert panel meeting: (I) assessing the appropriateness of the hierarchical structure of the initially developed public first aid science popularization and training evaluation system; (II) evaluating the suitability of the indicators at each level within the system; and (III) discussing potential revisions and improvements to the indicators across all levels.
Preparation of the expert panel meeting
One week prior to the meeting, the initial draft of the evaluation system and the meeting themes were distributed to all experts. The draft evaluation system was printed in advance and compiled into a booklet, which was provided to the experts as reference material for on-site discussion. The meeting was held in the hospital’s conference room and lasted approximately 30–60 minutes. At the beginning of the meeting, the researcher served as the chairperson and introduced the purpose and themes of the meeting, followed by a presentation of the initial draft of the public first aid science popularization and training evaluation system. Subsequently, experts engaged in discussions based on the meeting themes and provided their opinions and suggestions. Finally, the chairperson summarized the key points and confirmed whether consensus had been reached among the experts. If agreement was achieved, the meeting was concluded. Consensus was defined as agreement among at least 75% of the experts on the inclusion, exclusion, or modification of an indicator. When disagreement occurred, items were further discussed until consensus was reached.
Statistical analysis
This study employed the expert authority coefficient (Cr) to assess the reliability of the content of the public first aid science popularization and training evaluation system. The authority coefficient was calculated based on two dimensions: the expert judgment basis (Ci) and content familiarity (Cs), using the formula: Cr = (Ci + Cs)/2. An Cr value greater than 0.70 was considered to indicate acceptable expert authority and reliability of the evaluation results.
Results
Evidence-based integrated indicator system
In the current study, a total of 1,831 studies were identified across 10 databases. CNKI (n=245), Wanfang (n=353), VIP (n=25), Sionmed (n=251), Web of Science (n=621), PubMed (n=34), Embase (n=2), CINAHL (n=1), and Cochrane Library (n=262). After removing 392 duplicated studies, 1321 articles were excluded based on title and abstracts. The remaining 110 studies were reviewed and further excluded for the following reasons: 23 books or letters, and 17 conference abstracts. Finally, 12 articles were included in the review (13-26). Figure 2 presents the PRISMA flow diagram. Based on the Kirkpatrick model, four primary indicators have been identified: reaction, learning, behavior, and outcomes characteristics. Through comprehensive literature synthesis, secondary and tertiary indicators were identified. The evidence-based integration included 4 primary indicators, 12 secondary indicators and 40 tertiary indicators. The specific indicators were shown in Table S2.
Qualitative study results
In the current study, 7 intensive care specialist physicians and 9 ICU nurses were interviewed on-site from May 2024 to October 2024. The basic characteristics of the experts are presented in Table 1. The duration of each interview was 20–30 minutes. After completing the interview, the researchers need to complete the text transcription of the audio materials within 24 hours and then follow the seven-step analysis method of Colaizzi’s phenomenological research using Nvivo software. A total of two secondary indicators and 33 tertiary indicators were ultimately identified to supplement the competency framework for public first aid education and training.
Table 1
| No. | Gender | Age (years) | Professional title | Education | Specialization | Years of practice |
|---|---|---|---|---|---|---|
| D1 | Female | 44 | Chief Physician | Master’s degree | Emergency Physician | 20 |
| D2 | Male | 57 | Chief Physician | Bachelor’s degree | Emergency Physician | 32 |
| D3 | Female | 45 | Associate Chief Physician | Master’s degree | Emergency Physician | 22 |
| D4 | Male | 45 | Associate Chief Physician | Bachelor’s degree | Emergency Physician | 21 |
| D5 | Male | 42 | Associate Chief Physician | Bachelor’s degree | Emergency Physician | 20 |
| D6 | Male | 39 | Associate Chief Physician | Master’s degree | Emergency Physician | 14 |
| D7 | Male | 38 | Associate Chief Physician | Doctoral degree | Emergency Physician | 11 |
| S1 | Female | 38 | Associate Chief Nurse | Bachelor’s degree | Critical Care Nursing Research | 17 |
| S2 | Female | 29 | Nurse-in-charge | Master’s degree | Critical Care Nursing Research | 5 |
| S3 | Female | 36 | Nurse-in-charge | Master’s degree | Critical Care Nursing Research | 13 |
| S4 | Female | 31 | Nurse-in-charge | Master’s degree | Critical Care Nursing Research | 10 |
| N1 | Female | 40 | Nurse-in-charge | Bachelor’s degree | Emergency & Critical Care Nurse | 20 |
| N2 | Female | 31 | Nurse-in-charge | Bachelor’s degree | Emergency & Critical Care Nurse | 8 |
| N3 | Female | 34 | Nurse-in-charge | Bachelor’s degree | Emergency & Critical Care Nurse | 11 |
| N4 | Female | 30 | Nurse-in-charge | Bachelor’s degree | Emergency & Critical Care Nurse | 8 |
| N5 | Female | 33 | Nurse-in-charge | Bachelor’s degree | Emergency & Critical Care Nurse | 11 |
D, doctor; N, nurse; S, definition.
Information of expert panels
A total of 14 experts were included in this study. The experts’ age from 30 to 57 years, with a mean age of 42.0±6.34 years. Their years of professional experience ranged from 10 to 32 years, with an average of 19.4±6.18 years. Among them, 11 experts held senior professional titles and 3 held intermediate titles. In terms of educational background, 2 experts had doctoral degrees, 5 had master’s degrees, and 7 held bachelor’s degrees. The basic characteristics of the experts are presented in Table 2.
Table 2
| No. | Gender | Age (years) | Professional title | Education | Specialization | Years of practice |
|---|---|---|---|---|---|---|
| 1 | Male | 57 | Chief Physician | Bachelor | Emergency Medicine | 32 |
| 2 | Female | 45 | Associate Chief Physician | Master | Emergency Medicine | 22 |
| 3 | Male | 45 | Associate Chief Physician | Bachelor | Emergency Medicine | 21 |
| 4 | Male | 42 | Associate Chief Physician | Bachelor | Emergency Medicine | 20 |
| 5 | Male | 38 | Associate Chief Physician | Doctor | Critical Care Medicine Research | 11 |
| 6 | Female | 50 | Chief Nursing Officer | Doctor | Critical Care Medicine Research | 30 |
| 7 | Female | 41 | Associate Chief Nursing Officer | Master | Critical Care Nursing | 20 |
| 8 | Female | 39 | Associate Chief Nursing Officer | Master | Critical Care Nursing | 17 |
| 9 | Female | 40 | Charge Nursing Officer | Master | Critical Care Nursing | 18 |
| 10 | Female | 38 | Charge Nursing Officer | Bachelor | Critical Care Nursing | 15 |
| 11 | Female | 45 | Chief Nursing Officer | Bachelor | First Aid Science Popularization | 23 |
| 12 | Female | 38 | Associate Chief Nursing Officer | Bachelor | First Aid Science Popularization | 17 |
| 13 | Female | 41 | Associate Chief Nursing Officer | Bachelor | First Aid Science Popularization | 16 |
| 14 | Female | 30 | Charge Nursing Officer | Master | First Aid Science Popularization | 10 |
Degree of expert motivation and authority coefficient
The Cr in this study ranged from 0.85 to 0.95, while the Cs ranged from 0.70 to 0.90, indicating a high level of reliability in the study findings. Th degree of expert motivation and authority coefficient is shown in Table 3.
Table 3
| No. | Judgment basis | Ci | Cs | Cr | |||
|---|---|---|---|---|---|---|---|
| I1 | I2 | I3 | I4 | ||||
| 1 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
| 2 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
| 3 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.7 | 0.85 |
| 4 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
| 5 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
| 6 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
| 7 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
| 8 | 0.4 | 0.3 | 0.1 | 0.1 | 0.9 | 0.9 | 0.90 |
| 9 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.7 | 0.85 |
| 10 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
| 11 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
| 12 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
| 13 | 0.5 | 0.2 | 0.1 | 0.1 | 0.9 | 0.9 | 0.9 |
| 14 | 0.5 | 0.3 | 0.1 | 0.1 | 1.0 | 0.9 | 0.95 |
I1: theoretical analysis; I2: practical experience; I3: reference to domestic and international data; I4: subjective judgment. Ci, the expert judgment basis; Cr, expert authority coefficient; Cs, content familiarity.
Results of expert panel meeting
Based on feedback from the expert panel consultation, the researchers refined and improved the initial draft of the indicator system, resulting in a revised version of the public first aid science popularization and training evaluation system. This revised system comprises four first-level indicators including response, learning, behavior, and outcomes, along with 12 second-level indicators and 45 third-level indicators. The reaction level focuses on training implementation quality, including course design, instructor qualification, training tools, and organizational management. This level reflects the immediate delivery process of first aid education and the structural conditions that support effective training. The learning level evaluates participants’ theoretical knowledge, skill knowledge, attitudes, and abilities related to first aid. It captures the short-term educational effectiveness of training, including both cognitive understanding and practical competencies. The behavioral level assesses the extent to which trainees are willing and able to apply first aid knowledge in real-world emergency situations. It reflects behavioral intention, confidence, and willingness to perform or disseminate first aid. The outcome level represents the long-term impact of first aid training, including clinical outcomes such as survival rate, neurological prognosis, complications, and economic burden. These indicators are intended to reflect broader societal and health system benefits and may require linkage with emergency medical services or hospital databases for measurement. The specific hierarchical structure is shown in Figure 3 and Table 4.
Table 4
| First-level indicator | Second level indicator | Third-level indicators | Source |
|---|---|---|---|
| Reaction level | Course design | 1. Standardization of training methods | Q + L |
| 2. Diversification of science popularization models | Q + L | ||
| 3. Reasonable arrangement of training course duration | Q | ||
| 4. Practical and easy-to-understand course content | Q | ||
| 5. Appropriate proportion of theory and operation in course content design | L | ||
| 6. Reasonable arrangement of course schedule | L | ||
| 7. Moderate difficulty level of training content | L | ||
| 8. Inclusion of real cases and skill teaching videos | L | ||
| 9. On-site scenario simulation operation teaching | Q + L | ||
| 10. Training content developed based on guidelines | L | ||
| 11. Responsible attitude of instructors | Q | ||
| 12. Clear teaching logic and easy-to-understand content | Q | ||
| Instructor qualification | 13. Solid professional knowledge and rich practical experience | Q + L | |
| 14. Effective interaction and communication with trainees | Q | ||
| 15. Instructors engaged in emergency-related professions, qualified to teach, holding professional training certificates (e.g., AHA Instructor/First Responder Instructor/Red Cross Training Instructor/Local Hospital and Emergency Center Instructor) | E | ||
| 16. Promotional materials and training manuals | L | ||
| Training tools | 17. Complete basic training equipment and facilities to meet basic training needs | L | |
| Organization & management | 18. Complete training aids and high-fidelity simulation models | L | |
| 19. Multimedia equipment for teaching | L | ||
| 20. Appropriate scale of training venues | L | ||
| 21. Reasonable arrangement of training time, frequency, and number of participants; appropriate trainee-to-instructor ratio | E | ||
| 22. Strict organization and management during training | Q | ||
| 23. Unified and standardized assessment methods | E | ||
| 24. Correct initial identification and assessment of vital signs | Q + L | ||
| Learning level | Theoretical knowledge | 25. Correct dialing of emergency hotline | Q |
| 26. CPR + AED | Q + L | ||
| 27. Other first aid knowledge: management of common medical emergencies (airway obstruction, four basic trauma skills, etc.) | Q + L | ||
| 28. Laws and regulations | E | ||
| Skill knowledge | 29. Cardiopulmonary resuscitation (adults, children, and infants) + AED | Q + L | |
| 30. Airway foreign body obstruction first aid | Q | ||
| 31. Four basic on-site trauma rescue methods (hemostasis, bandaging, immobilization, and transportation) | L | ||
| Attitude | 32. Responsibility and positive attitudes toward performing first aid | Q | |
| 33. Willingness to perform CPR/first aid in the absence of professional personnel | Q | ||
| Ability | 34. Rapid assessment of different emergencies and provision of emergency treatment and symptomatic care | Q | |
| 35. Confidence to respond to emergencies in different scenarios and take appropriate measures | Q | ||
| 36. Provide first aid to others in various emergency scenarios | Q | ||
| Behavioral level | Willingness | 37. Willingness to disseminate first aid science knowledge and skills to others | Q |
| 38. Performing rescue under the guidance of professionals | Q | ||
| Cost indicators | 39. Hospitalization costs of patients who received first aid from trained individuals | Q | |
| 40. On-site first aid situation | Q | ||
| 41. Length of hospital stay | Q | ||
| Outcome level | Outcome indicators | 42. Survival rate (where outcome linkage is available) | Q + L |
| 43. Neurological prognosis assessed using standardized clinical measures (e.g., CPC score), where available | E | ||
| 44. Complications during hospitalization (where outcome data are available) | E | ||
| Effectiveness indicators | 45. Estimated healthcare cost savings based on health economic evaluation, where feasible | Q |
AED, automated external defibrillator; AHA, American Heart Association; CPC, Cerebral Performance Category; CPR, cardiopulmonary resuscitation; E, expert meeting newly added; L, literature only; Q, qualitative interviews only.
Discussion
The findings from the literature review and qualitative study provided a robust foundation for developing the initial draft of the evaluation index system. The 14 invited experts demonstrated a high level of authority, supported by extensive clinical and nursing experience. Through an expert panel meeting, the initial draft was carefully reviewed and refined, resulting in a set of scientifically grounded and practically applicable recommendations. Ultimately, the final version of the evaluation system was established.
The constructed public first aid science popularization and training evaluation system comprises 4 first-level indicators, 12 second-level indicators, and 45 third-level indicators. It encompasses four key dimensions: the reaction layer, learning layer, behavior layer, and outcome layer, guided by the Kirkpatrick four-level evaluation model. The system begins with the reaction layer, including elements such as training course design, instructor qualifications, training tools, and organizational management. It then progresses to the learning layer, covering theoretical knowledge, practical skills, attitudes, and competencies. At the behavior layer, it evaluates participants’ willingness to provide first aid and their actual performance capabilities. Finally, the outcome layer incorporates cost indicators, effect indicators, and benefit indicators. Together, these components form a comprehensive evaluation chain of training input-learning acquisition-behavioral transformation-social impact. Compared with previous first aid training evaluation studies that primarily focused on knowledge and skills assessment, this system is more systematically structured and logically coherent, enabling a comprehensive assessment of both short-term training outcomes and long-term societal benefits of public first aid science popularization programs. The prominence of CPR and AED across multiple indicators reflects their central role in public first aid education, as consistently identified through the literature review, qualitative interviews, and expert consultation. However, the proposed framework is intended as a general evaluation framework for public first aid education rather than CPR training alone, and it can be adapted to incorporate other first aid topics according to local training priorities and contexts.
Compared with previous studies on first aid training evaluation, the evaluation system constructed in this study demonstrates substantial expansion in both assessment dimensions and structural integrity. Although most existing studies adopt the Kirkpatrick model to evaluate training effectiveness, their practical applications often focus on only selected levels, particularly the learning level (knowledge and skills) and the reaction level (satisfaction). For instance, Do Thi et al. primarily assessed trainees’ knowledge acquisition using questionnaires and skill checklists, while paying limited attention to behavioral transformation and long-term outcomes (27). Similarly, Avau et al. incorporated follow-up assessments to evaluate practical skills; however, their evaluation of outcomes remained confined to first aid knowledge and practical skills, lacking a comprehensive assessment of social benefits or cost-effectiveness (28). Furthermore, A systematic review evidence has highlighted that current first aid training evaluations are generally characterized by an overemphasis on knowledge acquisition rather than behavioral change, and a focus on short-term effects rather than long-term outcomes (29). In addition, the American Heart Association (AHA) guidelines, reported that most training programs lack systematic tracking of actual rescue behaviors and clinical outcomes (30). In contrast, the public first aid training evaluation system constructed in this study, based on the Kirkpatrick model, integrates findings from qualitative interviews to identify key themes such as willingness, ability, and attitude. These elements were systematically incorporated into the behavioral-level indicators through comprehensive literature synthesis and expert consultation. Moreover, at the outcome level, the system was further extended to include cost indicators, outcome indicators, and effectiveness indicators, thereby emphasizing the impact of training on health outcomes, such as survival rate.
This design not only addresses the limitations of previous studies in evaluating behavioral and outcome levels, but also extends the scope of evaluation from training effectiveness to broader public health impact. It aligns with the emerging international trend in first aid education, which is shifting from a skill-oriented approach toward a more outcome-oriented and impact-driven framework. However, there are some limitations in this study. First, this framework was developed based on evidence and expert consensus from a single tertiary hospital in one province of China, it is intended as a foundational template rather than a universally applicable model. Adaptation may be required to accommodate different healthcare systems, resource availability, and local first aid needs, particularly in rural, low-resource, and international settings. Future studies should further validate and refine the framework across diverse contexts. Second, although the expert panel included experienced emergency physicians, nurses, and first aid educators, representation from health economics and public health was limited. Consequently, outcome-level indicators, particularly those related to cost-effectiveness and public health impact, require further refinement and validation through multidisciplinary expert consultation. Third, the behavioral level contains fewer indicators than the reaction and learning levels, this reflects the practical challenges of measuring behavioral outcomes in public first aid education. Unlike reaction and learning outcomes, behavioral change often depends on unpredictable opportunities to encounter real emergencies, making direct observation difficult. Therefore, the behavioral indicators were intentionally restricted to core constructs that can be feasibly assessed across different training settings. Future studies should further expand this level by incorporating indicators such as sustained willingness to provide first aid, participation in refresher training, actual first aid provision during real emergencies, and long-term retention of behavioral change. Fourth, the Delphi method is widely used for achieving structured expert consensus, it was not adopted in this study due to practical considerations, including time constraints, the availability of experts from a single tertiary hospital, and the feasibility of conducting multiple iterative rounds. Instead, a structured expert panel meeting was used to obtain consensus in a more efficient manner. Fifth, the proposed framework provides a structured evaluation system, practical implementation guidance was beyond the scope of this study. Future research should develop and validate standardized implementation strategies, including measurement tools, data collection procedures, and indicator application across different training settings. Sixth, outcome-level indicators represent the ultimate societal impact of public first aid education, their measurement requires access to clinical or administrative data sources and may not be feasible for all training providers. Therefore, these indicators should be regarded as aspirational evaluation targets rather than routine evaluation measures. Future studies should further validate their operational definitions and data collection methods. Finally, this study focused on the development of a theoretically grounded evaluation framework and did not include empirical validation of its applicability. Therefore, the proposed system should be further tested in different settings to assess its robustness and generalizability.
Conclusions
Based on literature reviews, qualitative interviews, and expert consultations, this study developed a public first aid training evaluation system based on the Kirkpatrick four-level assessment model, encompassing four dimensions: reaction, learning, behavior, and outcomes. The system establishes a comprehensive evaluation chain of training input, learning acquisition, behavioral transformation, social impact. It strengthens the inclusion of behavioral and outcome-level indicators, enabling a more objective evaluation of training effectiveness and its broader social value, and providing both a theoretical foundation and practical guidance for optimizing public first aid training and informing policy-making.
Acknowledgments
None.
Footnote
Reporting Checklist: The authors have completed the COREQ reporting checklist. Available at https://jphe.amegroups.com/article/view/10.21037/jphe-2026-0033/rc
Data Sharing Statement: Available at https://jphe.amegroups.com/article/view/10.21037/jphe-2026-0033/dss
Peer Review File: Available at https://jphe.amegroups.com/article/view/10.21037/jphe-2026-0033/prf
Funding: This work was supported by
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at https://jphe.amegroups.com/article/view/10.21037/jphe-2026-0033/coif). The authors have no conflicts of interest to declare.
Ethical Statement:
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Leng Z, Li J, Tang K, et al. Effect of Bystander CPR on Survival After Out-of-Hospital Cardiac Arrest: A Systematic Review and Meta-Analysis. Shock 2026;66:579-90. [Crossref] [PubMed]
- Yu Y, Meng Q, Munot S, et al. Assessment of Community Interventions for Bystander Cardiopulmonary Resuscitation in Out-of-Hospital Cardiac Arrest: A Systematic Review and Meta-analysis. JAMA Netw Open 2020;3:e209256. [Crossref] [PubMed]
- Christensen DM, Rajan S, Kragholm K, et al. Bystander cardiopulmonary resuscitation and survival in patients with out-of-hospital cardiac arrest of non-cardiac origin. Resuscitation 2019;140:98-105. [Crossref] [PubMed]
- Yin G, Chen L, Wu Y, et al. The implementation of a community-centered first aid education program for older adults-community health workers perceived barriers. BMC Health Serv Res 2023;23:128. [Crossref] [PubMed]
- Reveruzzi B, Buckley L, Sheehan M. First aid training in secondary schools: A comparative study and implementation considerations. J Safety Res 2020;75:32-40. [Crossref] [PubMed]
- Chan MM, Barchino R, Medina-Merodio JA, et al. MOOCs, an innovative alternative to teach first aid and emergency treatment: A practical study. Nurse Educ Today 2019;79:92-7. [Crossref] [PubMed]
- Bates R. A critical analysis of evaluation practice: the Kirkpatrick model and the principle of beneficence. Evaluation and Program Planning 2004;27:341-7.
- Reio TG Jr, Rocco TS, Smith DH, et al. A Critique of Kirkpatrick’s Evaluation Model. New Horizons in Adult Education and Human Resource Development 2017;29:35-53.
- Lee H, Song Y. Kirkpatrick Model Evaluation of Accelerated Second-Degree Nursing Programs: A Scoping Review. J Nurs Educ 2021;60:265-71. [Crossref] [PubMed]
- Alhassan AI. Implementing Faculty Development Programs in Medical Education Utilizing Kirkpatrick's Model. Adv Med Educ Pract 2022;13:945-54. [Crossref] [PubMed]
- Cheung VK, Chia NH, So SS, et al. Expanding scope of Kirkpatrick model from training effectiveness review to evidence-informed prioritization management for cricothyroidotomy simulation. Heliyon 2023;9:e18268. [Crossref] [PubMed]
- Edward KL, Welch T. The extension of Colaizzi's method of phenomenological enquiry. Contemp Nurse 2011;39:163-71. [Crossref] [PubMed]
- Cheng A, Magid DJ, Auerbach M, et al. Part 6: Resuscitation Education Science: 2020 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation 2020;142:S551-79. [Crossref] [PubMed]
- Chien CY, Fang SY, Tsai LH, et al. Traditional versus blended CPR training program: A randomized controlled non-inferiority study. Sci Rep 2020;10:10032. [Crossref] [PubMed]
- Munot S, Redfern J, Bray JE, et al. Improving community-based first response to out of hospital cardiac arrest (FirstCPR): protocol for a cluster randomised controlled trial. BMJ Open 2022;12:e057175. [Crossref] [PubMed]
- Ng TP, Eng SW, Ting JXR, et al. Global prevalence of basic life support training: A systematic review and meta-analysis. Resuscitation 2023;186:109771. [Crossref] [PubMed]
- Lin C, Lu L, Tang Y. Application effect of diversified teaching methods in first aid knowledge and skills training among community residents. Lingnan Journal of Emergency Medicine 2023;28:368-9.
- Shao M, Wang J, Hou Y, et al. Effect of deliberate practice teaching based on the SEED curriculum in public cardiopulmonary resuscitation training. Nursing Practice and Research 2022;19:1874-7.
- Zeng R, Ma B, Jiang Y, et al. Application of a flipped classroom teaching method based on WeChat platform in public cardiopulmonary resuscitation training. Chinese Journal of Emergency Resuscitation and Disaster Medicine 2022;17:544-8.
- Zhao Y, Lai W, Zhao W, et al. Evaluation of the effectiveness of cardiopulmonary resuscitation education and training models among residents. Chinese Journal of Health Education 2021;37:1020-3.
- Tan M. Comparison of two training models for community residents in first aid training. General Practice Nursing 2021;19:2440-2.
- Zhang Q, Zhou S, Deng G. Differences in training effectiveness between intelligent feedback report system training and traditional training in public cardiopulmonary resuscitation skills. Chinese Journal of Emergency Resuscitation and Disaster Medicine 2024;19:425-7.
- Huang L, Tian X, Liu Y, et al. Application effect of WeChat-based training in bystander cardiopulmonary resuscitation among community residents. Medical Information 2020;33:96-9.
- Tang G, Yin W, Fu Y, et al. Analysis of problems and countermeasures in the popularization of community cardiopulmonary resuscitation education. Chinese Journal of Emergency Medicine 2022;42:570-4.
- Zhu Y, Shi Z. Expert consensus on first responder actions in on-site first aid. Journal of Practical Shock (Chinese and English) 2019;3:359-72.
- Zhang H, Cheng S, Wang P, et al. Expert consensus on public response measures during the pre-hospital waiting period. Chinese Journal of Emergency Medicine 2022;42:380-6.
- Do Thi N, Thi GH, Lee Y, et al. First-aid training for primary Healthcare providers on a remote Island: a mixed-methods study. BMC Med Educ 2024;24:790. [Crossref] [PubMed]
- Avau B, Vanhove AC, Scheers H, et al. Impact of the Use of Simulated Patients in Basic First Aid Training on Laypeople Knowledge, Skills, and Self-efficacy: A Controlled Experimental Study. Simul Healthc 2022;17:213-9. [Crossref] [PubMed]
- Doherty Z, Bray JE, Finn J, et al. Basic life support training targeted to family members or carers of those at high-risk of out-of-hospital cardiac arrest: a systematic review. Resusc Plus 2025;25:101031. [Crossref] [PubMed]
- Hewett Brumberg EK, Douma MJ, Alibertis K, et al. 2024 American Heart Association and American Red Cross Guidelines for First Aid. Circulation 2024;150:e519-79. [Crossref] [PubMed]
Cite this article as: Tian Y, Liu H, Cheng Y, Zhang C, Hu Q, Xu X, Zhou Y, Shi Z. Construction of an evaluation system for public first aid education and training based on the Kirkpatrick model. J Public Health Emerg 2026;10:18.


