A view from the top: A qualitative exploration of top-level health care leaders' perceptions on the implementation of simulation-based education in postgraduate medical education.

INTRODUCTION: Although there is substantial evidence supporting the benefits of simulation-based education (SBE), its widespread and effective implementation remains challenging. The aim of this study was to explore the perceptions of top-level health care leaders regarding SBE and the barriers and facilitators that influence its wide implementation as part of the postgraduate surgical curricula in Denmark. METHODS: We conducted semi-structured interviews with top-level health care leaders who were chosen based on their roles in ensuring high-quality patient care delivery and developing strategies to achieve the goals of the entire health care system. The interview transcripts were translated into English, and a thematic approach was used to code and inductively analyse the data. We used the Consolidated Framework for Implementation Research to identify and understand the determinants to a successful implementation of SBE. RESULTS: We interviewed 13 participants from different political and administrative levels. We found that the participants had limited knowledge about SBE, which highlighted a disconnection between these leaders and the educational environment. This was further compounded by a lack of effective communication and inadequate information dissemination between simulation centres and higher-level organisations. While participants recognised the benefits of SBE for doctors in training, they expressed concerns about the implementation given the already strained health care system and limited resources. The need for evidence, particularly in the context of patient safety, was emphasised to facilitate SBE implementation. Although participants supported the implementation of SBE, it was unclear who should initiate action. DISCUSSION: This study highlighted the perspectives of top-level health care leaders regarding SBE and identified the determinant factors for a successful implementation. Effective communication channels are crucial to enhance collaborations and reduce the disconnection between the different health care organisational levels. Strategic implementation processes, including the roles and responsibilities, should be defined and established. These will inform decisions regarding the implementation strategies to effectively integrate SBE into the residency training curricula.

The Role of Collaborative Observational Practice and Feedback-Discourse to Promote Remote Acquisition of Technical Surgical Skills.

INTRODUCTION: Acquisition of technical skills remotely in a decentralized model requires an efficacious way of providing feedback. The primary objective was to test the efficacy of various forms of feedback on the acquisition of surgical skills by medical students. METHODS: Forty volunteers were randomized to four experimental groups, differing from the nature of feedback (free text versus structured) and who provided the feedback (expert versus peer learners). They had to perform sutures and upload attempts on a learning management system to receive interactive feedback. The pretest and retention test performances were assessed. RESULTS: All groups significantly improved from pretests to retention tests; however, participants using checklist showed statistically lower improvements than the other groups, which did not differ from each other. CONCLUSIONS: Remote learners can acquire surgical skills, and most importantly, peers who provide feedback, are as effective as experts if they use open-ended comments and not checklists.

Hand Sewn Anastomosis Skill Acquisition and In Vivo Transfer Using 3D-Printed Small Bowel Simulator.

INTRODUCTION: General surgery residents need to master the hand-sewn bowel anastomosis (HSBA) technique. However, practice opportunities outside of the operating room are rare, and commercial simulators are often costly. The objective of this study is to assess the efficacy of a new, affordable silicone small bowel simulator, made with a three-dimensional (3D) printed mold, as a training tool to learn this technique. METHODS: This was a single-blinded pilot randomized controlled trial comparing two groups of eight junior surgical residents. All participants completed a pretest using an inexpensive, custom developed 3D-printed simulator. Next, participants randomized to the experimental group practiced the HSBA skill at home (eight sessions), while those randomized to the control group did not receive any hands-on practice opportunities. A posttest was done using the same simulator as for the pretest and practice sessions, and the retention-transfer test was performed on an anesthetized porcine model. Pretests, posttests and retention-transfer tests were filmed and graded by a blinded evaluator using assessments of technical skills, quality of final product, and tests of procedural knowledge. RESULTS: The experimental group significantly improved after practicing with the model (P = 0.01), while an equivalent improvement was not noted in the control group (P = 0.07). Moreover, the experimental group's performance remained stable between the posttest and the retention-transfer test (P = 0.95). CONCLUSIONS: Our 3D-printed simulator is an affordable and efficacious tool to teach residents the HSBA technique. It allows development of surgical skills that are transferable to an in vivo model.

Using Kane's framework to build an assessment tool for undergraduate medical student's clinical competency with point of care ultrasound.

INTRODUCTION: Point-of-care ultrasonography (POCUS) is a portable imaging technology used in clinical settings. There is a need for valid tools to assess clinical competency in POCUS in medical students. The primary aim of this study was to use Kane's framework to evaluate an interpretation-use argument (IUA) for an undergraduate POCUS assessment tool. METHODS: Participants from Memorial University of Newfoundland, the University of Calgary, and the University of Ottawa were recruited between 2014 and 2018. A total of 86 participants and seven expert raters were recruited. The participants performed abdominal, sub-xiphoid cardiac, and aorta POCUS scans on a volunteer patient after watching an instruction video. The participant-generated POCUS images were assessed by the raters using a checklist and a global rating scale. Kane's framework was used to determine validity evidence for the scoring inference. Fleiss' kappa was used to measure agreement between seven raters on five questions that reflected clinical competence. The descriptive comments collected from the raters were systematically coded and analyzed. RESULTS: The overall agreement between the seven raters on five questions on clinical competency ranged from fair to moderate (κ = 0.32 to 0.55). The themes from the qualitative data were poor image generation and interpretation (22%), items not applicable (20%), poor audio and video quality (20%), poor probe handling (10%), and participant did not verbalize findings (14%). CONCLUSION: The POCUS assessment tool requires further modification and testing prior before it can be used for reliable undergraduate POCUS assessment.

Maternal and co-parental experiences and satisfaction with a co-parenting breastfeeding eHealth intervention in Canada.

BACKGROUND: Suboptimal breastfeeding rates are a public health priority. Interventions that include both breastfeeding women and their co-parents can increase breastfeeding initiation, duration, and exclusivity. eHealth can be an effective means of designing such interventions, as parents increasingly use the internet to access health information. The objective of this study was to determine maternal and co-parent satisfaction with an eHealth intervention. METHODS: The study was part of a larger randomized controlled trial that took place in Canada between March 2018 and April 2020. Data was collected from mothers (n = 56) and co-parents (n = 47). INTERVENTION: The eHealth intervention group received: 1) continued access to an eHealth breastfeeding co-parenting resource from the prenatal period to 52 weeks postpartum; 2) a virtual meeting with a research assistant; and 3) 6 weekly emails reminders. Follow-up data were collected via online questionnaires completed at 2 weeks post enrollment and 4, 12, 26, and 52 weeks postpartum to determine use and satisfaction with the intervention components. FINDINGS: The majority of mothers and co-parents independently reviewed the eHealth resource (95% and 91%, respectively), with higher use in the prenatal period. Participants found the resource to be useful (92%), informative (93%), targeted both parents (90%), and easy to understand (97%). Participants indicated the resource was comprehensive, easily navigated, convenient, and engaging. APPLICATION TO PRACTICE: Providing mothers and their co-parents with breastfeeding co-parenting support via an eHealth intervention delivers accessible, comprehensive information which may assist them in meeting their breastfeeding goals.

Virtual interprofessional education to support medical laboratory technologists' participation in interprofessional collaborative practice within integrated healthcare models.

Interprofessional collaborative practice is a key requirement for the successful implementation of integrated healthcare models. Current interprofessional education opportunities seldom include medical laboratory technologists who oversee the production of data that informs the diagnosis, treatment, and monitoring of patients. Errors in the laboratory process mostly occur in the pre-analytical and post-analytical phases, which both involve the need for collaboration between medical laboratory technologists and other healthcare providers. In this article, we introduce and describe an innovative work-integrated virtual learning experience that provides technologists with the opportunity to fully participate in interprofessional education.

To sim or not to sim-choosing wisely for procedural skills training in paediatrics.

Objectives: Simulation is a commonly used modality to teach paediatric procedural skills, however, it is resource intensive. Which paediatric skills are best taught using simulation is not known. This study aims to examine what skills to simulate, allowing for the best use of resources in ever expanding curricula. Method: We administered a questionnaire to community and hospitalist general paediatricians in Canada asking them to rate the importance of maintaining competency in each paediatric procedural skill and the frequency with which they perform the skill in their practice. Results: Skills that were rated as highly important and also high frequency were: bag-mask ventilation (BMV), lumbar puncture, neonatal cardiopulmonary resuscitation (CPR), specimen procurement for infectious diseases, immunization, and ear curettage. Skills that were rated as highly important but low frequency were: paediatric CPR, intraosseous needle insertion, neonatal intubation, defibrillation, gathering specimens for evidence of child maltreatment, paediatric intubation, cervical spine immobilization, and oral/nasogastric tube placement. Conclusion: Paediatric procedural skills are rated variably in terms of importance and frequency of use in general paediatric practice. Eight skills of high importance are infrequently performed and should be targeted for teaching via simulation.

Simulation and Active Learning Decreases Training Time of an Emergency Triage Assessment and Treatment Course in Pilot Study in Malawi: Implications for Increasing Efficiency and Workforce Capacity in Low-Resource Settings.

OBJECTIVES: The aim of the Emergency Triage Assessment and Treatment (ETAT) plus trauma course is to improve the quality of care provided to infants and children younger than 5 years. The curriculum was revised and shortened from 5 to 2.5 days by enhancing simulation and active learning opportunities. The aim of this study was to examine the feasibility and value of the new short-form ETAT course by assessing postcourse knowledge and satisfaction. METHODS: We delivered the short-form ETAT course to a group of interdisciplinary health workers in Malawi. Precourse and postcourse knowledge was assessed using a standardized 20 questions short answer test used previously in the 5-day courses. A 13-statement survey with 2 open-ended questions was used to examine participant satisfaction. RESULTS: Participants' postcourse knowledge improved significantly (P < 0.001) after the shorter ETAT course. Participants reported high levels of satisfaction with the short-form ETAT. CONCLUSIONS: Simulation and other active learning strategies reduced training time by 50% in the short-form ETAT course. Participants with and without previous ETAT training improved their knowledge after participating in the short-form ETAT course. Reduced training time is beneficial in settings already burdened by scarce human resources, may facilitate better access to in-service training, and build capacity while conserving resources in low-resource settings.

Development, manufacture and initial assessment of validity of a 3-dimensional-printed bowel anastomosis simulation training model.

BACKGROUND: It is critical that junior residents be given opportunities to practise bowel anastomosis before performing the procedure in patients. Three-dimensional (3D) printing is an affordable way to provide realistic, reusable intestinal simulators. The aim of this study was to test the face and content validity of a 3D-printed simulator for bowel anastomosis. METHODS: The bowel anastomosis simulator was designed and assembled with the use of desktop 3D printers and silicone solutions. The production cost ranges from $2.67 to $131, depending on which aspects of the model one prefers to include. We incorporated input from a general surgeon regarding design modifications to improve the realism of the model. Nine experts in general surgery (6 staff surgeons and 3 senior residents) were asked to perform an anastomosis with the model and then complete 2 surveys regarding face and content validity. Items were rated on a 5-point Likert scale ranging from 1 ("strongly disagree") to 5 ("strongly agree"). RESULTS: The overall average score for product quality was 3.58, indicating good face validity. The average score for realism (e.g., flexibility and texture of the model) was 3.77. The simulator was rated as being useful for training, with an overall average score of 3.98. In general, the participants agreed that the simulator would be a valuable addition to current simulation-based medical education (average score 4.11). They commented that the model would be improved by adding extra layers to simulate mucosa. CONCLUSION: Experts found the 3D-printed bowel anastomosis simulator to be an appropriate tool for the education of surgical residents, based on the model's texture, appearance and ability to undergo an anastomosis. This model provides an affordable way for surgical residents to learn bowel anastomosis. Future research will focus on proving educational efficacy, effectiveness and transfer that can be adapted for laparoscopic anastomosis training, hand-sewing and stapling procedures.

Repercussions of the COVID-19 pandemic on the well-being and training of medical clerks: a pan-Canadian survey.

BACKGROUND: The COVID-19 pandemic has been an unprecedented and potentially stressful event that inserted itself into the 2019-2020 Canadian medical curriculum. However, its impact on stress and subsequent professional pathways is not well understood. This study aims to assess the impact of the COVID-19 pandemic on the mental well-being, training, and career choices of Canadian medical clerks within the first three months of the pandemic. It also aims to assess their use of university support systems and their appreciation of potential solutions to common academic stressors. METHODS: An electronic survey composed of four sections: demographics, stressors experienced during the pandemic, World Health Organization (WHO) well-being index, and stress management and resources was distributed to Canadian clerks. RESULTS: Clerks from 10 of the 17 Canadian medical faculties participated in this study (n = 627). Forty-five percent of clerks reported higher levels of stress than usual; 22% reconsidered their residency choice; and 19% reconsidered medicine as a career. The factors that were most stressful among clerks were: the means of return to rotations; decreased opportunities to be productive in view of residency match; and taking the national licensing exam after the beginning of residency. The mean WHO well-being index was 14.8/25 ± 4.5, indicating a poor level of well-being among a considerable proportion of students. Clerks who reconsidered their residency choice or medicine as a career had lower mean WHO well-being indices. Most clerks agreed with the following suggested solutions: training sessions on the clinical management of COVID-19 cases; being allowed to submit fewer reference letters when applying to residency; and having protected time to study for their licensing exam during residency. Overall, clerks were less concerned with being infected during their rotations than with the impact of the pandemic on their future career and residency match. CONCLUSION: The COVID-19 pandemic had a considerable impact on the medical curriculum and well-being of clerks. A number of student-identified solutions were proposed to reduce stress. The implementation of these solutions throughout the Canadian medical training system should be considered.

Evaluation of a Mobile Telesimulation Unit to Train Rural and Remote Practitioners on High-Acuity Low-Occurrence Procedures: Pilot Randomized Controlled Trial.

BACKGROUND: The provision of acute medical care in rural and remote areas presents unique challenges for practitioners. Therefore, a tailored approach to training providers would prove beneficial. Although simulation-based medical education (SBME) has been shown to be effective, access to such training can be difficult and costly in rural and remote areas. OBJECTIVE: The aim of this study was to evaluate the educational efficacy of simulation-based training of an acute care procedure delivered remotely, using a portable, self-contained unit outfitted with off-the-shelf and low-cost telecommunications equipment (mobile telesimulation unit, MTU), versus the traditional face-to-face approach. A conceptual framework based on a combination of Kirkpatrick's Learning Evaluation Model and Miller's Clinical Assessment Framework was used. METHODS: A written procedural skills test was used to assess Miller's learning level- knows-at 3 points in time: preinstruction, immediately postinstruction, and 1 week later. To assess procedural performance (shows how), participants were video recorded performing chest tube insertion before and after hands-on supervised training. A modified Objective Structured Assessment of Technical Skills (OSATS) checklist and a Global Rating Scale (GRS) of operative performance were used by a blinded rater to assess participants' performance. Kirkpatrick's reaction was measured through subject completion of a survey on satisfaction with the learning experiences and an evaluation of training. RESULTS: A total of 69 medical students participated in the study. Students were randomly assigned to 1 of the following 3 groups: comparison (25/69, 36%), intervention (23/69, 33%), or control (21/69, 31%). For knows, as expected, no significant differences were found between the groups on written knowledge (posttest, P=.13). For shows how, no significant differences were found between the comparison and intervention groups on the procedural skills learning outcomes immediately after the training (OSATS checklist and GRS, P=1.00). However, significant differences were found for the control versus comparison groups (OSATS checklist, P<.001; GRS, P=.02) and the control versus intervention groups (OSATS checklist, P<.001; GRS, P=.01) on the pre- and postprocedural performance. For reaction, there were no statistically significant differences between the intervention and comparison groups on the satisfaction with learning items (P=.65 and P=.79) or the evaluation of the training (P=.79, P=.45, and P=.31). CONCLUSIONS: Our results demonstrate that simulation-based training delivered remotely, applying our MTU concept, can be an effective way to teach procedural skills. Participants trained remotely in the MTU had comparable learning outcomes (shows how) to those trained face-to-face. Both groups received statistically significant higher procedural performance scores than those in the control group. Participants in both instruction groups were equally satisfied with their learning and training (reaction). We believe that mobile telesimulation could be an effective way of providing expert mentorship and overcoming a number of barriers to delivering SBME in rural and remote locations.

Remote Mentoring of Point-of-Care Ultrasound Skills to Inexperienced Operators Using Multiple Telemedicine Platforms: Is a Cell Phone Good Enough?

OBJECTIVES: Telemedicine technology contributes to the teaching of point-of-care ultrasound (US); however, expensive equipment can limit its deployment in resource-challenged settings. We assessed 3 low-cost telemedicine solutions capable of supporting remote US training to determine feasibility, acceptability, and effectiveness. We also explored the value of instructional videos immediately before telementoring. METHODS: Thirty-six participants were randomly assigned to receive US mentoring in 1 of 3 telemedicine conditions: multiple fixed cameras, a smartphone, and traditional audio with a live US stream. Participants were then asked to perform a standardized US examination of the right upper quadrant under remote guidance. We measured observer's global ratings of performance along with the mentor's and student's rating of effort and satisfaction to determine which of the 3 approaches was most feasible, acceptable, and effective. During the second phase, students were randomized to watch an instructional video or not before receiving remote coaching on how to complete a subxiphoid cardiac examination. Effort, satisfaction, and performance from the independent observer's and student's perspective were surveyed. RESULTS: There was no significant difference between the different telemedicine setups from the observer's perspective; however, the mentor rated the smartphone significantly worse (P = .028-.04) than other technologies. Platforms were rated equivalent from the student's perspective. No benefit was detected for watching an instructional video before the mentored task. CONCLUSIONS: Remote US skills can be taught equally effectively by using a variety of telemedicine technologies. Smartphones represent a viable option for US training in resource-challenged settings.

Thrive or overload? The effect of task complexity on novices' simulation-based learning.

CONTEXT: Fidelity is widely viewed as an important element of simulation instructional design based on its purported relationship with transfer of learning. However, higher levels of fidelity may increase task complexity to a point at which novices' cognitive resources become overloaded. OBJECTIVES: In this experiment, we investigate the effects of variations in task complexity on novices' cognitive load and learning during simulation-based procedural skills training. METHODS: Thirty-eight medical students were randomly assigned to simulation training on a simple or complex lumbar puncture (LP) task. Participants completed four practice trials on this task (skill acquisition). After 10 days of rest, all participants completed one additional trial on their assigned task (retention) and one trial on a 'very complex' simulation designed to be similar to the complex task (transfer). We assessed LP performance and cognitive load on each trial using multiple measures. RESULTS: In both groups, LP performance improved significantly during skill acquisition (p ≤ 0.047, f = 0.29-0.96) and was maintained at retention. The simple task group demonstrated superior performance compared with the complex task group throughout these phases (p ≤ 0.002, d = 1.13-2.31). Cognitive load declined significantly in the simple task group (p < 0.009, f = 0.48-0.76), but not in the complex task group during skill acquisition, and remained lower at retention (p ≤ 0.024, d = 0.78-1.39). Between retention and transfer, LP performance declined and cognitive load increased in the simple task group, whereas both remained stable in the complex task group. At transfer, no group differences were observed in LP performance and cognitive load, except that the simple task group made significantly fewer breaches of sterility (p = 0.023, d = 0.80). CONCLUSIONS: Reduced task complexity was associated with superior LP performance and lower cognitive load during skill acquisition and retention, but mixed results on transfer to a more complex task. These results indicate that task complexity is an important factor that may mediate (via cognitive overload) the relationship between instructional design elements (e.g. fidelity) and simulation-based learning outcomes.

Validity of a new assessment rubric for a short-answer test of clinical reasoning.

BACKGROUND: The validity of high-stakes decisions derived from assessment results is of primary concern to candidates and certifying institutions in the health professions. In the field of orthopaedic manual physical therapy (OMPT), there is a dearth of documented validity evidence to support the certification process particularly for short-answer tests. To address this need, we examined the internal structure of the Case History Assessment Tool (CHAT); this is a new assessment rubric developed to appraise written responses to a short-answer test of clinical reasoning in post-graduate OMPT certification in Canada. METHODS: Fourteen physical therapy students (novices) and 16 physical therapists (PT) with minimal and substantial OMPT training respectively completed a mock examination. Four pairs of examiners (n = 8) participated in appraising written responses using the CHAT. We conducted separate generalizability studies (G studies) for all participants and also by level of OMPT training. Internal consistency was calculated for test questions with more than 2 assessment items. Decision studies were also conducted to determine optimal application of the CHAT for OMPT certification. RESULTS: The overall reliability of CHAT scores was found to be moderate; however, reliability estimates for the novice group suggest that the scale was incapable of accommodating for scores of novices. Internal consistency estimates indicate item redundancies for several test questions which will require further investigation. CONCLUSION: Future validity studies should consider discriminating the clinical reasoning competence of OMPT trainees strictly at the post-graduate level. Although rater variance was low, the large variance attributed to error sources not incorporated in our G studies warrant further investigations into other threats to validity. Future examination of examiner stringency is also warranted.

Measuring cognitive load: performance, mental effort and simulation task complexity.

CONTEXT: Interest in applying cognitive load theory in health care simulation is growing. This line of inquiry requires measures that are sensitive to changes in cognitive load arising from different instructional designs. Recently, mental effort ratings and secondary task performance have shown promise as measures of cognitive load in health care simulation. OBJECTIVES: We investigate the sensitivity of these measures to predicted differences in intrinsic load arising from variations in task complexity and learner expertise during simulation-based surgical skills training. METHODS: We randomly assigned 28 novice medical students to simulation training on a simple or complex surgical knot-tying task. Participants completed 13 practice trials, interspersed with computer-based video instruction. On trials 1, 5, 9 and 13, knot-tying performance was assessed using time and movement efficiency measures, and cognitive load was assessed using subjective rating of mental effort (SRME) and simple reaction time (SRT) on a vibrotactile stimulus-monitoring secondary task. RESULTS: Significant improvements in knot-tying performance (F(1.04,24.95)  = 41.1, p < 0.001 for movements; F(1.04,25.90)  = 49.9, p < 0.001 for time) and reduced cognitive load (F(2.3,58.5)  = 57.7, p < 0.001 for SRME; F(1.8,47.3)  = 10.5, p < 0.001 for SRT) were observed in both groups during training. The simple-task group demonstrated superior knot tying (F(1,24)  = 5.2, p = 0.031 for movements; F(1,24)  = 6.5, p = 0.017 for time) and a faster decline in SRME over the first five trials (F(1,26)  = 6.45, p = 0.017) compared with their peers. Although SRT followed a similar pattern, group differences were not statistically significant. CONCLUSIONS: Both secondary task performance and mental effort ratings are sensitive to changes in intrinsic load among novices engaged in simulation-based learning. These measures can be used to track cognitive load during skills training. Mental effort ratings are also sensitive to small differences in intrinsic load arising from variations in the physical complexity of a simulation task. The complementary nature of these subjective and objective measures suggests their combined use is advantageous in simulation instructional design research.

Operationalising elaboration theory for simulation instruction design: a Delphi study.

OBJECTIVE: The aim of this study was to assess the feasibility of incorporating the Delphi process within the simplifying conditions method (SCM) described in elaboration theory (ET) to identify conditions impacting the complexity of procedural skills for novice learners. METHODS: We generated an initial list of conditions impacting the complexity of lumbar puncture (LP) from key informant interviews (n = 5) and a literature review. Eighteen clinician-educators from six different medical specialties were subsequently recruited as expert panellists. Over three Delphi rounds, these panellists rated: (i) their agreement with the inclusion of the simple version of the conditions in a representative ('epitome') training scenario, and (ii) how much the inverse (complex) version increases LP complexity for a novice. Cronbach's α-values were used to assess inter-rater agreement. RESULTS: All panellists completed Rounds 1 and 2 of the survey and 17 completed Round 3. In Round 1, Cronbach's α-values were 0.89 and 0.94 for conditions that simplify and increase LP complexity, respectively; both values increased to 0.98 in Rounds 2 and 3. With the exception of 'high CSF (cerebral spinal fluid) pressure', panellists agreed with the inclusion of all conditions in the simplest (epitome) training scenario. Panellists rated patient movement, spinal anatomy, patient cooperativeness, body habitus, and the presence or absence of an experienced assistant as having the greatest impact on the complexity of LP. CONCLUSIONS: This study demonstrated the feasibility of using expert consensus to establish conditions impacting the complexity of procedural skills, and the benefits of incorporating the Delphi method into the SCM. These data can be used to develop and sequence simulation scenarios in a progressively challenging manner. If the theorised learning gains associated with ET are realised, the methods described in this study may be applied to the design of simulation training for other procedural and non-procedural skills, thereby advancing the agenda of theoretically based instruction design in health care simulation.

Measuring cognitive load during simulation-based psychomotor skills training: sensitivity of secondary-task performance and subjective ratings.

As interest in applying cognitive load theory (CLT) to the study and design of pedagogic and technological approaches in healthcare simulation grows, suitable measures of cognitive load (CL) are needed. Here, we report a two-phased study investigating the sensitivity of subjective ratings of mental effort (SRME) and secondary-task performance (signal detection rate, SDR and recognition reaction time, RRT) as measures of CL. In phase 1 of the study, novice learners and expert surgeons attempted a visual-monitoring task under two conditions: single-task (monitoring a virtual patient's heart-rate) and dual-task (tying surgical knots on a bench-top simulator while monitoring the virtual patient's heart-rate). Novices demonstrated higher mental effort and inferior secondary-task performance on the dual-task compared to experts (RRT 1.76 vs. 0.73, p = 0.012; SDR 0.27 vs. 0.97, p < 0.001; SRME 7.75 vs. 2.80, p < 0.001). Similarly, secondary task performance deteriorated from baseline to dual-task among novices (RRT 0.63 vs. 1.76 s, p < 0.006 and SDR 1.00 vs. 0.27, p < 0.001), but not experts (RRT 0.63 vs. 0.73 s, p = 0.124 and SDR 1.00 vs. 0.97, p = 0.178). In phase 2, novices practiced surgical knot-tying on the bench top simulator during consecutive dual-task trials. A significant increase in SDR (F(9,63) = 6.63, p < 0.001, f = 0.97) and decrease in SRME (F(9,63) = 9.39, p < 0.001, f = 1.04) was observed during simulation training, while RRT did not change significantly (F(9,63) = 1.18, p < 0.32, f = 0.41). The results suggest subjective ratings and dual-task performance can be used to track changes in CL among novices, particularly in early phases of simulation-based skills training. The implications for measuring CL in simulation instructional design research are discussed.

Part versus whole: a randomized trial of central venous catheterization education.

Central venous catheterization (CVC) is a complex but commonly performed procedure. How best to teach this complex skill has not been clearly delineated. We conducted a randomized trial of the effects of two types of teaching of CVC on skill acquisition and retention. We randomly assigned novice internal medicine residents to learning CVC in-part or in-whole. The part-group was taught the first part of the procedure, followed by practice, followed by being taught the second and final portion of the procedure, and followed by practice. The whole-group was taught the procedure in its entirety, followed by practice. Teaching and practice time for both groups was otherwise held constant. Performances were assessed at baseline, post-training, and at 1 month. The primary outcome was skill retention at 1-month, rated by using a global rating scale and a 22-item checklist, and defined as the score increase between 1-month and baseline. Skill acquisition is defined as the score increase post-training and baseline. Raters were blinded to the participants' identity, group assignment, and time point. Participants in the part-task group outperformed the whole-task group in skill acquisition (2.2 ± 0.8 vs 1.3 ± 1.0; g = 1.01; p = 0.04) and in skill retention (1.5 ± 0.7 vs 0.5 ± 0.8; g = 1.39; p = 0.006) using the global rating scale. Scores rated by the checklist were not significantly different (52.0 ± 25.3 vs 43.5 ± 23.4; g = 0.33; p = 0.47 for skill acquisition; and 48.5 ± 34.9 vs 41.1 ± 20.4; g = 0.35; p = 0.44 for skill retention). For teaching ultrasound-guided CVC to novice learners, teaching in part is preferable than teaching in whole.

Establishing assessment criteria for clinical reasoning in orthopedic manual physical therapy: a consensus-building study.

OBJECTIVES: Clinical reasoning (CR) represents one of the core components of clinical competence in Orthopaedic Manual Physical Therapy (OMPT). While education standards have been developed to guide curricular design, assessment of CR has not yet been standardized. Without theory-informed and rigorously developed measures, the certification of OMPTs lacks credibility and is less defensible. The purpose of this study was to use a theory-informed approach to generate assessment criteria for developing new assessment tools to evaluate CR in OMPT. METHODS: A list of assessment criteria was generated based on international education standards and multiple theoretical perspectives. A modified Delphi method was used to gain expert consensus on the importance of these assessment criteria for the assessment of CR in OMPT. The OMPTs from 22 countries with experience in assessing CR were invited to participate in three rounds of online questionnaires to rate their level of agreement with these criteria. Responses were tabulated to analyze degree of consensus and internal consistency. RESULTS: Representatives from almost half of the OMPT member organizations (MO) participated in three rounds of the Delphi. High levels of agreement were found among respondents regarding the importance and feasibility of most assessment criteria. There was high internal consistency among items within the proposed item subgroupings. DISCUSSION: A list of assessment criteria has been established that will serve as a framework for developing new assessment tools for CR assessment in OMPT. These criteria will be important for guiding the design of certification processes in OMPT as well as other episodes of CR assessment throughout OMPT training.

Comparing the cost-effectiveness of simulation modalities: a case study of peripheral intravenous catheterization training.

While the ultimate goal of simulation training is to enhance learning, cost-effectiveness is a critical factor. Research that compares simulation training in terms of educational- and cost-effectiveness will lead to better-informed curricular decisions. Using previously published data we conducted a cost-effectiveness analysis of three simulation-based programs. Medical students (n = 15 per group) practiced in one of three 2-h intravenous catheterization skills training programs: low-fidelity (virtual reality), high-fidelity (mannequin), or progressive (consisting of virtual reality, task trainer, and mannequin simulator). One week later, all performed a transfer test on a hybrid simulation (standardized patient with a task trainer). We used a net benefit regression model to identify the most cost-effective training program via paired comparisons. We also created a cost-effectiveness acceptability curve to visually represent the probability that one program is more cost-effective when compared to its comparator at various 'willingness-to-pay' values. We conducted separate analyses for implementation and total costs. The results showed that the progressive program had the highest total cost (p < 0.001) whereas the high-fidelity program had the highest implementation cost (p < 0.001). While the most cost-effective program depended on the decision makers' willingness-to-pay value, the progressive training program was generally most educationally- and cost-effective. Our analyses suggest that a progressive program that strategically combines simulation modalities provides a cost-effective solution. More generally, we have introduced how a cost-effectiveness analysis may be applied to simulation training; a method that medical educators may use to investment decisions (e.g., purchasing cost-effective and educationally sound simulators).