Assessing simulation-based clinical training: comparing the concurrent validity of students' self-reported satisfaction and confidence measures against objective clinical examinations.

Introduction: Simulation-based education (SBE) literature is replete with student satisfaction and confidence measures to infer educational outcomes. This research aims to test how well students' satisfaction and confidence measures correlate with expert assessments of students' improvements in competence following SBE activities. Methods: N=85 paramedic students (mean age 23.7 years, SD=6.5; 48.2% female) undertook a 3-day SBE workshop. Students' baseline competence was assessed via practical scenario simulation assessments (PSSAs) administered by expert paramedics and confidence via a questionnaire. Postworkshop competence and confidence plus self-reported students' satisfaction were remeasured. Results: PSSA scores increased significantly between baseline and post workshop (35.7%→53.4%, p<0.001), as did students' confidence (55.7%→60.5%, p<0.001), and their workshop satisfaction was high (71.0%). Satisfaction and postworkshop confidence measures were moderately correlated (r=0.377, p=0.001). However, competence improvements were not significantly correlated with either satisfaction (r=-0.107 p=0.344) or change in confidence (r=-0.187 p=0.102). Discussion: Students' self-reported satisfaction and confidence measures bore little relation to expert paramedics' judgements of their educational improvements. Satisfaction and confidence measures appear to be dubious indicators of SBE learning outcomes.

Effects of Low- Versus High-Fidelity Simulations on the Cognitive Burden and Performance of Entry-Level Paramedicine Students: A Mixed-Methods Comparison Trial Using Eye-Tracking, Continuous Heart Rate, Difficulty Rating Scales, Video Observation and Interviews.

INTRODUCTION: High-fidelity simulation-based training is often avoided for early-stage students because of the assumption that while practicing newly learned skills, they are ill suited to processing multiple demands, which can lead to "cognitive overload" and poorer learning outcomes. We tested this assumption using a mixed-methods experimental design manipulating psychological immersion. METHODS: Thirty-nine randomly assigned first-year paramedicine students completed low- or high-environmental fidelity simulations [low-environmental fidelity simulations (LF(en)S) vs. high-environmental fidelity simulation (HF(en)S)] involving a manikin with obstructed airway (SimMan3G). Psychological immersion and cognitive burden were determined via continuous heart rate, eye tracking, self-report questionnaire (National Aeronautics and Space Administration Task Load Index), independent observation, and postsimulation interviews. Performance was assessed by successful location of obstruction and time-to-termination. RESULTS: Eye tracking confirmed that students attended to multiple, concurrent stimuli in HF(en)S and interviews consistently suggested that they experienced greater psychological immersion and cognitive burden than their LF(en)S counterparts. This was confirmed by significantly higher mean heart rate (P < 0.001) and National Aeronautics and Space Administration Task Load Index mental demand (P < 0.05). Although group allocation did not influence the proportion of students who ultimately revived the patient (58% vs. 30%, P < 0.10), the HF(en)S students did so significantly more quickly (P < 0.01). The LF(en)S students had low immersion resulting in greater assessment anxiety. CONCLUSIONS: High-environmental fidelity simulation engendered immersion and a sense of urgency in students, whereas LF(en)S created assessment anxiety and slower performance. We conclude that once early-stage students have learned the basics of a clinical skill, throwing them in the "deep end" of high-fidelity simulation creates significant additional cognitive burden but this has considerable educational merit.

Clinical Placement Before or After Simulated Learning Environments?: A Naturalistic Study of Clinical Skills Acquisition Among Early-Stage Paramedicine Students.

BACKGROUND: There is conflicting evidence surrounding the merit of clinical placements (CPs) for early-stage health-profession students. Some contend that early-stage CPs facilitate contextualization of a subsequently learned theory. Others argue that training in simulated-learning experiences (SLEs) should occur before CP to ensure that students possess at least basic competency. We sought to investigate both claims. METHODS: First-year paramedicine students (n = 85) undertook 3 days of CP and SLEs as part of course requirements. Students undertook CP either before or after participation in SLEs creating 2 groups (Clin → Sim/Sim → Clin). Clinical skills acquisition was measured via direct scenario-based clinical assessments with expert observers conducted at 4 intervals during the semester. Perceptions of difficulty of CP and SLE were measured via the National Aeronautics and Space Administration Task Load Index. RESULTS: Students' clinical assessment scores in both groups improved significantly from beginning to end of semester (P < 0.001). However, at semester's end, clinical assessment scores for the Sim → Clin group were statistically significantly greater than those of the Clin → Sim group (P = 0.021). Both groups found SLEs more demanding than CP (P < 0.001). However, compared with the Sim → Clin group, the Clin → Sim group rated SLE as substantially more time-demanding than CP (P = 0.003). CONCLUSIONS: Differences in temporal demand suggest that the Clin → Sim students had fewer opportunities to practice clinical skills during CP than the Sim → Clin students due to a more limited scope of practice. The Sim → Clin students contextualized SLE within subsequent CP resulting in greater improvement in clinical competency by semester's end in comparison with the Clin → Sim students who were forced to contextualize skills retrospectively.

Quantification of opportunities for early-stage paramedicine students to practice clinical skills during clinical placements compared with an equal dose of simulation-based workshops.

Introduction: A reported advantage of simulation-based learning environments (SLE) over clinical placements (CPs) is that the former can provide a greater number and breadth of opportunities to practice level-appropriate clinical skills compared with the random patient presentations provided during the latter. Although logical and widely accepted as fact, we find no published evidence to demonstrate the magnitude, nor indeed veracity, of this assumption. We therefore sought to quantify the clinical skills practiced by entry-level paramedicine students attending a well-selected CP compared with an equal dosage of SLE. Methods: N=37 first-year paramedicine students completed activity diaries during 3 days of CP and 3 days of SLE. Opportunities to practice clinical skills were quantified and coded as either: level-appropriate, beyond-level or of non-discipline relevance. Results: During SLE, the average student was exposed 226 times to 11 level-appropriate clinical procedures. During CP the average student was exposed 48 times to 24 clinical procedures, the majority relevant to paramedicine (63%), but a minority level-appropriate (38%). Students' opportunities for supervised, 'hands on' practice represented only 10% of exposures in either SLE or CP but in terms of raw numbers of level-appropriate opportunities, SLE provided more than CP (n=23 vs 2). Discussion: Our results confirm that SLE provides substantially more opportunities than CP for students to practice level-appropriate skills and is therefore more appropriate for repetitive practice. However, CP is likely to remain useful to students for practicing interpersonal skills and contextualisation of knowledge within the broader health system. Educators should therefore carefully articulate learning objectives before choosing between SLE and CP.