Defining a Framework and Evaluation Metrics for Sustainable Global Surgical Partnerships: A Modified Delphi Study.

OBJECTIVE: The aim of this study was to use expert consensus to build a concrete and realistic framework and checklist to evaluate sustainability in global surgery partnerships (GSPs). BACKGROUND: Partnerships between high-resourced and low-resourced settings are often created to address the burden of unmet surgical need. Reflecting on the negative, unintended consequences of asymmetrical partnerships, global surgery community members have proposed frameworks and best practices to promote sustainable engagement between partners, though these frameworks lack consensus. This project proposes a cohesive, consensus-driven framework with accompanying evaluation metrics to guide sustainability in GSPs. METHODS: A modified Delphi technique with purposive sampling was used to build consensus on the definitions and associated evaluation metrics of previously proposed pillars (Stakeholder Engagement, Multidisciplinary Collaboration, Context-Relevant Education and Training, Bilateral Authorship, Multisource Funding, Outcome Measurement) of sustainable GSPs. RESULTS: Fifty global surgery experts from 34 countries with a median of 9.5 years of experience in the field of global surgery participated in 3 Delphi rounds. Consensus was achieved on the identity, definitions, and a 47-item checklist for the evaluation of the 6 pillars of sustainability in GSPs. In all, 29% of items achieved consensus in the first round, whereas 100% achieved consensus in the second and third rounds. CONCLUSIONS: We present the first framework for building sustainable GSPs using the input of experts from all World Health Organization regions. We hope this tool will help the global surgery community to find noncolonial solutions to addressing the gap in access to quality surgical care in low-resource settings.

Competencies for proficiency in basic point-of-care ultrasound in anesthesiology: national expert recommendations using Delphi methodology.

PURPOSE: Point-of-care ultrasound (POCUS) allows for rapid bedside assessment and guidance of patient care. Recently, POCUS was included as a mandatory component of Canadian anesthesiology training; however, there is no national consensus regarding the competencies to guide curriculum development. We therefore aimed to define national residency competencies for basic perioperative POCUS proficiency. METHODS: We adopted a Delphi process to delineate relevant POCUS competencies whereby we circulated an online survey to academic anesthesiologists identified as POCUS leads/experts (n = 25) at all 17 Canadian anesthesiology residency programs. After reviewing a list of competencies derived from the Royal College of Physicians and Surgeons of Canada's National Curriculum, we asked participants to accept, refine, delete, or add competencies. Three rounds were completed between 2022 and 2023. We discarded items with < 50% agreement, revised those with 50-79% agreement based upon feedback provided, and maintained unrevised those items with ≥ 80% agreement. RESULTS: We initially identified and circulated (Round 1) 74 competencies across 19 clinical domains (e.g., basics of ultrasound [equipment, nomenclature, clinical governance, physics]; cardiac [left ventricle, right ventricle, valve assessment, pericardial effusion, intravascular volume status] and lung ultrasound anatomy, image acquisition, and image interpretation; and clinical applications [monitoring and serial assessments, persistent hypotension, respiratory distress, cardiac arrest]). After three Delphi rounds (and 100% response rate maintained), panellists ultimately agreed upon 75 competencies. CONCLUSION: Through national expert consensus, this study identified POCUS competencies suitable for curriculum development and assessment in perioperative anesthesiology. Next steps include designing and piloting a POCUS curriculum and assessment tool(s) based upon these nationally defined competencies.

RéSUMé: OBJECTIF: L'échographie ciblée (POCUS) permet une évaluation rapide au chevet des patient·es et l'orientation des soins aux patient·es. Récemment, l'échographie ciblée a été incluse en tant que composante obligatoire de la formation en anesthésiologie au Canada; cependant, il n'y a pas de consensus national sur les compétences qui guideront l'élaboration des programmes d'études. Nous avons donc cherché à définir les compétences à inclure dans les programmes de résidence nationaux pour acquérir des compétences de base en échographie ciblée périopératoire. MéTHODE: Nous avons adopté un processus Delphi pour délimiter les compétences pertinentes en échographie ciblée, processus dans le cadre duquel nous avons fait circuler un sondage en ligne auprès d'anesthésiologistes universitaires identifié·es comme des responsables/expert·es en échographie ciblée (n = 25) dans les 17 programmes canadiens de résidence en anesthésiologie. Après avoir examiné une liste de compétences tirées du programme d'études national du Collège royal des médecins et chirurgiens du Canada, nous avons demandé aux participant·es d'accepter, de peaufiner, de supprimer ou d'ajouter des compétences. Trois rondes ont été complétées entre 2022 et 2023. Nous avons écarté les éléments ayant < 50 % d'accord, révisé ceux avec 50 à 79 % d'accord en fonction des commentaires fournis, et maintenu sans révision les éléments obtenant ≥ 80 % d'accord. RéSULTATS: Nous avons d'abord identifié et diffusé (ronde 1) 74 compétences dans 19 domaines cliniques (p. ex., les bases de l'échographie [équipement, nomenclature, gouvernance clinique, physique]; anatomie échographique cardiaque [ventricule gauche, ventricule droit, évaluation valvulaire, épanchement péricardique, état du volume intravasculaire] et pulmonaire [acquisition et interprétation d'images]; et applications cliniques [surveillance et évaluations en série, hypotension persistante, détresse respiratoire, arrêt cardiaque]). Après trois rondes Delphi (et un taux de réponse de 100 % maintenu), les panélistes se sont finalement mis·es d'accord sur 75 compétences. CONCLUSION: Grâce à un consensus d'expert·es au pays, cette étude a permis d'identifier les compétences en échographie ciblée adaptées à l'élaboration et à l'évaluation de programmes d'études en anesthésiologie périopératoire. Les prochaines étapes comprennent la conception et la mise à l'essai d'un programme d'études et d'outils d'évaluation en échographie ciblée basés sur ces compétences définies à l'échelle nationale.

Exploring Patient Advisors' Perceptions of Virtual Care Across Canada: Qualitative Phenomenological Study.

BACKGROUND: While virtual care services existed prior to the emergence of COVID-19, the pandemic catalyzed a rapid transition from in-person to virtual care service delivery across the Canadian health care system. Virtual care includes synchronous or asynchronous delivery of health care services through video visits, telephone visits, or secure messaging. Patient advisors are people with patient and caregiving experiences who collaborate within the health care system to share insights and experiences in order to improve health care. OBJECTIVE: This study aimed to understand patient advisors' perceptions related to virtual care and potential impacts on health care quality. METHODS: We adopted a phenomenological approach, whereby we interviewed 20 participants who were patient advisors across Canada using a semistructured interview protocol. The protocol was developed by content experts and medical education researchers. The interviews were audio-recorded, transcribed verbatim, and analyzed thematically. Data collection stopped once thematic saturation was reached. The study was conducted at Queen's University, Kingston, Ontario. We recruited 20 participants from 5 Canadian provinces (17 female participants and 3 male participants). RESULTS: Six themes were identified: (1) characteristics of effective health care, (2) experiences with virtual care, (3) modality preferences, (4) involvement of others, (5) risks associated with virtual care encounters, and (6) vulnerable populations. Participants reported that high-quality health care included building relationships and treating patients holistically. In general, participants described positive experiences with virtual care during the pandemic, including greater efficiency, increased accessibility, and that virtual care was less stressful and more patient centered. Participants comparing virtual care with in-person care reported that time, scheduling, and content of interactions were similar across modalities. However, participants also shared the perception that certain modalities were more appropriate for specific clinical encounters (eg, prescription renewals and follow-up appointments). Perspectives related to the involvement of family members and medical trainees were positive. Potential risks included miscommunication, privacy concerns, and inaccurate patient assessments. All participants agreed that stakeholders should be proactive in applying strategies to support vulnerable patients. Participants also recommended education for patients and providers to improve virtual care delivery. CONCLUSIONS: Participant-reported experiences of virtual care encounters were relatively positive. Future work could focus on delivering training and resources for providers and patients. While initial experiences are positive, there is a need for ongoing stakeholder engagement and evaluation to improve patient and caregiver experiences with virtual care.

High-fidelity, simulation-based microsurgical training for neurosurgical residents.

OBJECTIVE: Simulation is increasingly recognized as an important supplement to operative training. The live rat femoral artery model is a well-established model for microsurgical skills simulation. In this study, the authors present an 11-year experience incorporating a comprehensive, longitudinal microsurgical training curriculum into a Canadian neurosurgery program. The first goal was to evaluate training effectiveness, using a well-studied rating scale with strong validity. The second goal was to assess the impact of the curriculum on objective measures of subsequent operating room performance during postgraduate year (PGY)-5 and PGY-6 training. METHODS: PGY-2 neurosurgery residents completed a 1-year curriculum spanning 17 training sessions divided into 5 modules of increasing fidelity. Both perfused duck wing and live rat vessel training models were used. Three modules comprised live microvascular anastomosis. Trainee performance was video recorded and blindly graded using the Objective Structured Assessment of Technical Skills Global Rating Scale. Eleven participants who completed the training curriculum and 3 subjects who had not participated had their subsequent operative performances evaluated when they were at the PGY-5 and PGY-6 levels. RESULTS: Eighteen participants completed 106 microvascular anastomoses during the study. There was significant improvement in 6 measurable skills during the curriculum. The mean overall score was significantly higher on the fifth attempt compared with the first attempt for all 3 live anastomotic modules (p < 0.001). Each module had a different improvement profile across the skills assessed. Those who completed the microvascular skills curriculum demonstrated a greater number of independent evaluations during superficial surgical exposure, deep exposure, and primary maneuvers at the PGY-5 and PGY-6 levels. CONCLUSIONS: High-fidelity microsurgical simulation training leads to significant improvement in microneurosurgical skills. Transfer of acquired skills to the operative environment and durability for at least 3 to 4 years show encouraging preliminary results and are subject to ongoing investigation.

A model for global surgical training and capacity development: the Children's of Alabama-Viet Nam pediatric neurosurgery partnership.

INTRODUCTION: Training capable and competent neurosurgeons to work in underserved regions of the world is an essential component of building global neurosurgical capacity. One strategy for achieving this goal is establishing longitudinal partnerships between institutions in low- and middle-income countries (LMICs) and their counterparts in high-income countries (HICs) utilizing a multi-component model. We describe the initial experience of the Children's of Alabama (COA) Global Surgery Program partnership with multiple Vietnamese neurosurgical centers. METHODS: The training model developed by the COA Global Surgery Program utilizes three complementary and interdependent methods to expand neurosurgical capacity: in-country training, out-of-country training, and ongoing support and mentorship. Multiple Vietnamese hospital systems have participated in the partnership, including three hospitals in Hanoi and one hospital in Ho Chi Minh City. RESULTS: During the 7 years of the partnership, the COA and Viet Nam teams have collaborated on expanding pediatric neurosurgical care in numerous areas of clinical need including five subspecialized areas of pediatric neurosurgery: cerebrovascular, epilepsy, neuroendoscopy for hydrocephalus management, craniofacial, and neuro-oncology. CONCLUSION: Long-term partnerships between academic departments in LMICs and HICs focused on education and training are playing an increasingly important role in scaling up global surgical capacity. We believe that our multi-faceted approach consisting of in-country targeted hands-on training, out-of-country fellowship training at the mentor institution, and ongoing mentorship using telecollaboration and Internet-based tools is a viable and generalizable model for enhancing surgical capacity globally.

Simulation and Active Learning Decreases Training Time of an Emergency Triage Assessment and Treatment Course in Pilot Study in Malawi: Implications for Increasing Efficiency and Workforce Capacity in Low-Resource Settings.

OBJECTIVES: The aim of the Emergency Triage Assessment and Treatment (ETAT) plus trauma course is to improve the quality of care provided to infants and children younger than 5 years. The curriculum was revised and shortened from 5 to 2.5 days by enhancing simulation and active learning opportunities. The aim of this study was to examine the feasibility and value of the new short-form ETAT course by assessing postcourse knowledge and satisfaction. METHODS: We delivered the short-form ETAT course to a group of interdisciplinary health workers in Malawi. Precourse and postcourse knowledge was assessed using a standardized 20 questions short answer test used previously in the 5-day courses. A 13-statement survey with 2 open-ended questions was used to examine participant satisfaction. RESULTS: Participants' postcourse knowledge improved significantly (P < 0.001) after the shorter ETAT course. Participants reported high levels of satisfaction with the short-form ETAT. CONCLUSIONS: Simulation and other active learning strategies reduced training time by 50% in the short-form ETAT course. Participants with and without previous ETAT training improved their knowledge after participating in the short-form ETAT course. Reduced training time is beneficial in settings already burdened by scarce human resources, may facilitate better access to in-service training, and build capacity while conserving resources in low-resource settings.

The initial experience of InterSurgeon: an online platform to facilitate global neurosurgical partnerships.

OBJECTIVE: Despite general enthusiasm for international collaboration within the organized neurosurgical community, establishing international partnerships remains challenging. The current study analyzes the initial experience of the InterSurgeon website in partnering surgeons from across the world to increase surgical collaboration. METHODS: One year after the launch of the InterSurgeon website, data were collected to quantify the number of website visits, average session duration, total numbers of matches, and number of offers and requests added to the website each month. Additionally, a 15-question survey was designed and distributed to all registered members of the website. RESULTS: There are currently 321 surgeon and institutional members of InterSurgeon representing 69 different countries and all global regions. At the time of the survey there were 277 members, of whom 76 responded to the survey, yielding a response rate of 27.4% (76/277). Twenty-five participants (32.9%) confirmed having either received a match email (12/76, 15.8%) or initiated contact with another user via the website (13/76, 17.1%). As expected, the majority of the collaborations were either between a high-income country (HIC) and a low-income country (LIC) (5/18, 27.8%) or between an HIC and a middle-income country (MIC) (9/18, 50%). Interestingly, there were 2 MIC-to-MIC collaborations (2/18, 11.1%) as well as 1 MIC-to-LIC (1/18, 5.6%) and 1 LIC-to-LIC partnership. At the time of response, 6 (33.3%) of the matches had at least resulted in initial contact via email or telephone. One of the partnerships had involved face-to-face interaction via video conference. A total of 4 respondents had traveled internationally to visit their partner's institution. CONCLUSIONS: Within its first year of launch, the InterSurgeon membership has grown significantly. The partnerships that have already been formed involve not only international visits between HICs and low- to middle-income countries (LMICs), but also telecollaboration and inter-LMIC connections that allow for greater exchange of knowledge and expertise. As membership and site features grow to include other surgical and anesthesia specialties, membership growth and utilization is expected to increase rapidly over time according to social network dynamics.

Engagement: what is it good for? The role of learner engagement in healthcare simulation contexts.

Learner engagement matters, particularly in simulation-based education. Indeed, it has been argued that instructional design only matters in the service of engaging learners in a simulation encounter. Yet despite its purported importance, our understanding of what engagement is, how to define it, how to measure it, and how to assess it is limited. The current study presents the results of a critical narrative review of literature outside of health sciences education, with the aim of summarizing existing knowledge in these areas and providing a research agenda to guide future scholarship on learner engagement in healthcare simulation. Building on this existing knowledge base, we provide a working definition for engagement and provide an outline for future research programs that will help us better understand how health professions' learners experience engagement in the simulated setting. With this in hand, additional research questions can be addressed including: how do simulation instructional design features (fidelity, range of task difficulty, feedback, etc.) affect engagement? What is the relationship between engagement and simulation learning outcomes? And how is engagement related to or distinct from related variables like cognitive load, motivation, and self-regulated learning?

Thrive or overload? The effect of task complexity on novices' simulation-based learning.

CONTEXT: Fidelity is widely viewed as an important element of simulation instructional design based on its purported relationship with transfer of learning. However, higher levels of fidelity may increase task complexity to a point at which novices' cognitive resources become overloaded. OBJECTIVES: In this experiment, we investigate the effects of variations in task complexity on novices' cognitive load and learning during simulation-based procedural skills training. METHODS: Thirty-eight medical students were randomly assigned to simulation training on a simple or complex lumbar puncture (LP) task. Participants completed four practice trials on this task (skill acquisition). After 10 days of rest, all participants completed one additional trial on their assigned task (retention) and one trial on a 'very complex' simulation designed to be similar to the complex task (transfer). We assessed LP performance and cognitive load on each trial using multiple measures. RESULTS: In both groups, LP performance improved significantly during skill acquisition (p ≤ 0.047, f = 0.29-0.96) and was maintained at retention. The simple task group demonstrated superior performance compared with the complex task group throughout these phases (p ≤ 0.002, d = 1.13-2.31). Cognitive load declined significantly in the simple task group (p < 0.009, f = 0.48-0.76), but not in the complex task group during skill acquisition, and remained lower at retention (p ≤ 0.024, d = 0.78-1.39). Between retention and transfer, LP performance declined and cognitive load increased in the simple task group, whereas both remained stable in the complex task group. At transfer, no group differences were observed in LP performance and cognitive load, except that the simple task group made significantly fewer breaches of sterility (p = 0.023, d = 0.80). CONCLUSIONS: Reduced task complexity was associated with superior LP performance and lower cognitive load during skill acquisition and retention, but mixed results on transfer to a more complex task. These results indicate that task complexity is an important factor that may mediate (via cognitive overload) the relationship between instructional design elements (e.g. fidelity) and simulation-based learning outcomes.

Needs Assessment for Incoming PGY-1 Residents in Neurosurgical Residency.

BACKGROUND: Residents must develop a diverse range of skills in order to practice neurosurgery safely and effectively. The purpose of this study was to identify the foundational skills required for neurosurgical trainees as they transition from medical school to residency. METHODS: Based on the CanMEDS competency framework, a web-based survey was distributed to all Canadian academic neurosurgical centers, targeting incoming and current PGY-1 neurosurgical residents as well as program directors. Using Likert scale and free-text responses, respondents rated the importance of various cognitive (e.g. management of raised intracranial pressure), technical (e.g. performing a lumbar puncture) and behavioral skills (e.g. obtaining informed consent) required for a PGY-1 neurosurgical resident. RESULTS: Of 52 individuals contacted, 38 responses were received. Of these, 10 were from program directors (71%), 11 from current PGY-1 residents (58%) and 17 from incoming PGY-1 residents (89%). Respondents emphasized operative skills such as proper sterile technique and patient positioning; clinical skills such as lesion localization and interpreting neuro-imaging; management skills for common scenarios such as raised intracranial pressure and status epilepticus; and technical skills such as lumbar puncture and external ventricular drain placement. Free text answers were concordant with the Likert scale results. DISCUSSION: We surveyed Canadian neurosurgical program directors and PGY-1 residents to identify areas perceived as foundational to neurosurgical residency education and training. This information is valuable for evaluating the appropriateness of a training program's goals and objectives, as well as for generating a national educational curriculum for incoming PGY-1 residents.

Teaching for the Transition: the Canadian PGY-1 Neurosurgery 'Rookie Camp'.

BACKGROUND: Transitioning from medical school to residency is difficult and stressful, necessitating innovation in easing this transition. In response, a Canadian neurosurgical Rookie Camp was designed and implemented to foster acquisition of technical, cognitive and behavioral skills among incoming Canadian post graduate year one (PGY-1) neurosurgery residents. METHODS: The inaugural Rookie Camp was held in July 2012 in Halifax. The curriculum was developed based on a national needs-assessment and consisted of a pre-course manual, 7 case-based stations, 4 procedural skills stations and 2 group discussions. The content was clinically focused, used a variety of teaching methods, and addressed multiple CanMEDS competencies. Evaluation included participant and faculty surveys and a pre-course, post-course, and 3-month retention knowledge test. RESULTS: 17 of 23 PGY-1 Canadian neurosurgical residents participated in the Camp. All agreed the course content was relevant for PGY-1 training and the experience prepared them for residency. All participants would recommend the course to future neurosurgical residents. A statistically significant improvement was observed in knowledge related to course content (F(2,32) = 7.572, p<0.002). There were no significant differences between post-test and retention-test scores at three months. CONCLUSION: The inaugural Canadian Neurosurgery Rookie Camp for PGY-1 residents was successfully delivered, with engagement from participants, training programs, the Canadian Neurosurgical Society, and the Royal College. In addition to providing fundamental knowledge, which was shown to be retained, the course eased junior residents' transition to residency by fostering camaraderie and socialization within the specialty.

Measuring cognitive load: performance, mental effort and simulation task complexity.

CONTEXT: Interest in applying cognitive load theory in health care simulation is growing. This line of inquiry requires measures that are sensitive to changes in cognitive load arising from different instructional designs. Recently, mental effort ratings and secondary task performance have shown promise as measures of cognitive load in health care simulation. OBJECTIVES: We investigate the sensitivity of these measures to predicted differences in intrinsic load arising from variations in task complexity and learner expertise during simulation-based surgical skills training. METHODS: We randomly assigned 28 novice medical students to simulation training on a simple or complex surgical knot-tying task. Participants completed 13 practice trials, interspersed with computer-based video instruction. On trials 1, 5, 9 and 13, knot-tying performance was assessed using time and movement efficiency measures, and cognitive load was assessed using subjective rating of mental effort (SRME) and simple reaction time (SRT) on a vibrotactile stimulus-monitoring secondary task. RESULTS: Significant improvements in knot-tying performance (F(1.04,24.95)  = 41.1, p < 0.001 for movements; F(1.04,25.90)  = 49.9, p < 0.001 for time) and reduced cognitive load (F(2.3,58.5)  = 57.7, p < 0.001 for SRME; F(1.8,47.3)  = 10.5, p < 0.001 for SRT) were observed in both groups during training. The simple-task group demonstrated superior knot tying (F(1,24)  = 5.2, p = 0.031 for movements; F(1,24)  = 6.5, p = 0.017 for time) and a faster decline in SRME over the first five trials (F(1,26)  = 6.45, p = 0.017) compared with their peers. Although SRT followed a similar pattern, group differences were not statistically significant. CONCLUSIONS: Both secondary task performance and mental effort ratings are sensitive to changes in intrinsic load among novices engaged in simulation-based learning. These measures can be used to track cognitive load during skills training. Mental effort ratings are also sensitive to small differences in intrinsic load arising from variations in the physical complexity of a simulation task. The complementary nature of these subjective and objective measures suggests their combined use is advantageous in simulation instructional design research.

Operationalising elaboration theory for simulation instruction design: a Delphi study.

OBJECTIVE: The aim of this study was to assess the feasibility of incorporating the Delphi process within the simplifying conditions method (SCM) described in elaboration theory (ET) to identify conditions impacting the complexity of procedural skills for novice learners. METHODS: We generated an initial list of conditions impacting the complexity of lumbar puncture (LP) from key informant interviews (n = 5) and a literature review. Eighteen clinician-educators from six different medical specialties were subsequently recruited as expert panellists. Over three Delphi rounds, these panellists rated: (i) their agreement with the inclusion of the simple version of the conditions in a representative ('epitome') training scenario, and (ii) how much the inverse (complex) version increases LP complexity for a novice. Cronbach's α-values were used to assess inter-rater agreement. RESULTS: All panellists completed Rounds 1 and 2 of the survey and 17 completed Round 3. In Round 1, Cronbach's α-values were 0.89 and 0.94 for conditions that simplify and increase LP complexity, respectively; both values increased to 0.98 in Rounds 2 and 3. With the exception of 'high CSF (cerebral spinal fluid) pressure', panellists agreed with the inclusion of all conditions in the simplest (epitome) training scenario. Panellists rated patient movement, spinal anatomy, patient cooperativeness, body habitus, and the presence or absence of an experienced assistant as having the greatest impact on the complexity of LP. CONCLUSIONS: This study demonstrated the feasibility of using expert consensus to establish conditions impacting the complexity of procedural skills, and the benefits of incorporating the Delphi method into the SCM. These data can be used to develop and sequence simulation scenarios in a progressively challenging manner. If the theorised learning gains associated with ET are realised, the methods described in this study may be applied to the design of simulation training for other procedural and non-procedural skills, thereby advancing the agenda of theoretically based instruction design in health care simulation.

Measuring cognitive load during simulation-based psychomotor skills training: sensitivity of secondary-task performance and subjective ratings.

As interest in applying cognitive load theory (CLT) to the study and design of pedagogic and technological approaches in healthcare simulation grows, suitable measures of cognitive load (CL) are needed. Here, we report a two-phased study investigating the sensitivity of subjective ratings of mental effort (SRME) and secondary-task performance (signal detection rate, SDR and recognition reaction time, RRT) as measures of CL. In phase 1 of the study, novice learners and expert surgeons attempted a visual-monitoring task under two conditions: single-task (monitoring a virtual patient's heart-rate) and dual-task (tying surgical knots on a bench-top simulator while monitoring the virtual patient's heart-rate). Novices demonstrated higher mental effort and inferior secondary-task performance on the dual-task compared to experts (RRT 1.76 vs. 0.73, p = 0.012; SDR 0.27 vs. 0.97, p < 0.001; SRME 7.75 vs. 2.80, p < 0.001). Similarly, secondary task performance deteriorated from baseline to dual-task among novices (RRT 0.63 vs. 1.76 s, p < 0.006 and SDR 1.00 vs. 0.27, p < 0.001), but not experts (RRT 0.63 vs. 0.73 s, p = 0.124 and SDR 1.00 vs. 0.97, p = 0.178). In phase 2, novices practiced surgical knot-tying on the bench top simulator during consecutive dual-task trials. A significant increase in SDR (F(9,63) = 6.63, p < 0.001, f = 0.97) and decrease in SRME (F(9,63) = 9.39, p < 0.001, f = 1.04) was observed during simulation training, while RRT did not change significantly (F(9,63) = 1.18, p < 0.32, f = 0.41). The results suggest subjective ratings and dual-task performance can be used to track changes in CL among novices, particularly in early phases of simulation-based skills training. The implications for measuring CL in simulation instructional design research are discussed.

Development and content validation of performance assessments for endoscopic third ventriculostomy.

PURPOSE: This study aims to develop and establish the content validity of multiple expert rating instruments to assess performance in endoscopic third ventriculostomy (ETV), collectively called the Neuro-Endoscopic Ventriculostomy Assessment Tool (NEVAT). METHODS: The important aspects of ETV were identified through a review of current literature, ETV videos, and discussion with neurosurgeons, fellows, and residents. Three assessment measures were subsequently developed: a procedure-specific checklist (CL), a CL of surgical errors, and a global rating scale (GRS). Neurosurgeons from various countries, all identified as experts in ETV, were then invited to participate in a modified Delphi survey to establish the content validity of these instruments. In each Delphi round, experts rated their agreement including each procedural step, error, and GRS item in the respective instruments on a 5-point Likert scale. RESULTS: Seventeen experts agreed to participate in the study and completed all Delphi rounds. After item generation, a total of 27 procedural CL items, 26 error CL items, and 9 GRS items were posed to Delphi panelists for rating. An additional 17 procedural CL items, 12 error CL items, and 1 GRS item were added by panelists. After three rounds, strong consensus (>80% agreement) was achieved on 35 procedural CL items, 29 error CL items, and 10 GRS items. Moderate consensus (50-80% agreement) was achieved on an additional 7 procedural CL items and 1 error CL item. The final procedural and error checklist contained 42 and 30 items, respectively (divided into setup, exposure, navigation, ventriculostomy, and closure). The final GRS contained 10 items. CONCLUSIONS: We have established the content validity of three ETV assessment measures by iterative consensus of an international expert panel. Each measure provides unique assessment information and thus can be used individually or in combination, depending on the characteristics of the learner and the purpose of the assessment. These instruments must now be evaluated in both the simulated and operative settings, to determine their construct validity and reliability. Ultimately, the measures contained in the NEVAT may prove suitable for formative assessment during ETV training and potentially as summative assessment measures during certification.

Readiness for practice: a survey of neurosurgery graduates and program directors.

BACKGROUND: Postgraduate neurosurgical education is undergoing significant reform, including transition to a competency-based training model. To support these efforts, the purpose of this study was to determine neurosurgical graduates' and program directors' (PDs) opinions about graduates' level of competence in reference to the 2010 Royal College Objectives of Training in Neurosurgery. METHODS: An electronic survey was distributed to Canadian neurosurgery PDs and graduates from 2011. The questionnaire addressed graduates' abilities in nonprocedural knowledge and skills, CanMEDS roles, proficiency with core neurosurgical procedures and knowledge of complex neurosurgical techniques. RESULTS: Thirteen of 22 (59%) graduate and 17/25 (65%) PD surveys were completed. There were no significant differences between PD and graduate responses. Most respondents agreed that these graduates possess the knowledge and skills expected of an independently practicing neurosurgeon across current objectives of training. A small proportion felt some graduates did not achieve this level of proficiency on specific vascular, functional, peripheral nerve and endoscopic procedures. This was partially attributed to limited exposure to these procedures during training and perceptions that some techniques required fellowship-level training. CONCLUSIONS: Graduating neurosurgical residents are perceived to possess a high level of proficiency in the majority of neurosurgical practice domains. Inadequate exposure during training or a perception that subspecialists should perform some procedures may contribute to cases where proficiency is not as high. The trends identified in this study could be monitored on an ongoing basis to provide supplemental data to guide curricular decisions in Canadian neurosurgical training.

What we call what we do affects how we do it: a new nomenclature for simulation research in medical education.

Rapid technological advances and concern for patient safety have increased the focus on simulation as a pedagogical tool for educating health care providers. To date, simulation research scholarship has focused on two areas; evaluating instructional designs of simulation programs, and the integration of simulation into a broader educational context. However, these two categories of research currently exist under a single label-Simulation-Based Medical Education. In this paper we argue that introducing a more refined nomenclature within which to frame simulation research is necessary for researchers, to appropriately design research studies and describe their findings, and for end-point users (such as program directors and educators), to more appropriately understand and utilize this evidence.