Mapping the complexity of task demands during constant observation of patients with complex health behaviors.

Unlicensed patient care assistants (PCAs) are often tasked with providing constant observation (CO) of patients with complex health behaviors. The unique demands of CO are poorly understood, making it challenging to design effective training. Four-component instructional design theory suggests the tasks associated with CO feature non-recurrent challenges, such that effective, simulation-based deliberate practice must reflect real-life variability. This study aimed to elucidate the variety of challenges PCAs encounter during CO of patients with complex health behaviors. We used a qualitative descriptive design and developed a preliminary framework to code interviews. The data collection framework was designed to support creating numerous realistic scenarios to support generalized and transferable learning. From our interviews with 16 participants, we identified 1,066 statements associated with variations in CO tasks. We grouped our findings by two domains, "patient factors" and "environmental contexts." Our results revealed many recurring and non-recurring challenges inherent in CO, requiring a range of skills to maintain patient and staff safety. Instructional design elements may include scenarios incorporating environment and resource assessment, cognitive feedback for non-recurrent tasks, novel methods of self-harm initiated by the patient, and incorporating interdisciplinary staff in which the learner must navigate a complex conversation. This study clarifies the task demands of CO and is useful as a task analysis to guide scenario development for simulation-based experiences. A less systematic approach would risk underrepresenting the difficulties inherent in the task, reinforcing a limited set of behaviors that may not generalize well to the non-recurrent challenges in CO.

The red hat - designating leadership using visual and verbal cues: a mixed-methods study.

BACKGROUND: During a critical event in the labor and delivery operating room, it is crucial for team members responding to the situation to be aware of the designated leaders. Visual and verbal cues have been utilized to designate leadership in various healthcare settings; however, previous research has indicated mixed results using visual cues for role designation. METHODS: The purpose of this study was to explore the use of the red surgical hat as a visual cue of leadership during obstetric emergency simulation training. We used a mixed-methods design to analyze simulation-based education video and debriefing transcripts. RESULTS: There was a statistically significant difference in the proportion of participants who declared leadership vs. those who donned the red hat. Participants were more likely to visually declare leadership utilizing a red surgical bouffant hat than to verbally declare leadership. Most participants indicated that observing the red hat to detect leadership in the operating room was more effective than when leaders used a verbal declaration to inform others who was leading. CONCLUSIONS: Our findings suggest that utilizing a visual cue of leadership with the red surgical bouffant hat improves participant perceptions of communication of the surgical team during an obstetrical critical event in a simulation environment.

Putting the "learning" in "pre-learning": effects of a self-directed study hall on skill acquisition in a simulation-based central line insertion course.

BACKGROUND: Opportunities to practice procedural skills in the clinical learning environment are decreasing, and faculty time to coach skills is limited, even in simulation-based training. Self-directed learning with hands-on practice early in a procedural skill course might help maximize the benefit of later faculty coaching and clinical experience. However, it may also lead to well-learned errors if learners lack critical guidance. The present study sought to investigate the effects of a hands-on, self-directed "study hall" for central line insertion among first-year residents. METHODS: Learner cohorts before vs. after introduction of the study hall (n = 49) were compared on their pre- and post-test performance of key procedural behaviors that were comparable across cohorts, with all learners receiving traditional instructor-led training between tests. RESULTS: Study hall participants spent a median of 116 min in hands-on practice (range 57-175). They scored higher at pre-test (44% vs. 27%, p = .00; Cohen's d = 0.95) and at post-test (80% vs. 72%, p = .02; Cohen's d = 0.69). A dose-response relationship was found, such that 2 h of study hall were roughly equivalent to the performance improvement seen with four clinical observations or supervised insertions of central lines. CONCLUSIONS: Self-directed, hands-on "study hall" supported improved procedural skill learning in the context of limited faculty availability. Potential additional benefits make the approach worth further experimentation and evaluation.

Integrative Literature Review on Cognitive Science to Reconsider Failure to Rescue in Nursing: A Call to Action.

BACKGROUND: When current workload demands exceed the cognitive capacity of nurses, tasks that are critical to early detection of clinical deterioration may be omitted. In many high-risk industries that require vigilance over sustained periods, such as the military, the cognitive resource theory of vigilance heavily guides systematic approaches. Yet, in research and policy on nursing failure to rescue, that same application of a cognitive science framework has been overlooked. METHOD: The nursing literature on failure to rescue was reviewed through the lens of the cognitive resource theory of vigilance. Fifteen articles met the inclusion criteria. RESULTS: Four themes emerged: relationship between clinical judgment and failure to rescue, implicit reference to the cognitive resource theory of vigilance, benefits of simulation-based education, and caregiver fatigue. CONCLUSION: The use of cognitive science to target advancement of patient management strategies has the potential to lead to a decrease in failure to rescue and an increase in nursing cognitive task efficiency. [J Contin Educ Nurs. 2023;54(6):253-260.].

Integrating simulation and interpretive description to explore operating room leadership: critical event continuing education.

In Obstetrics and Gynecologic operating room emergencies, the surgeon cannot both operate and lead a suddenly expanded and redirected team response. However, one of the most often used approaches to interprofessional continuing education designed to improve teams' ability to respond to unanticipated critical events still emphasizes surgeon leadership. We developed Explicit Anesthesia and Nurse Distributed (EXPAND) Leadership to imagine a workflow that might better distribute emergency leadership task responsibilities and practices. The purpose of this exploratory study was to investigate teams' responses to distributing leadership during an interprofessional continuing education simulated obstetrical emergency. We used interpretive descriptive design in a secondary analysis of teams' post-simulation reflective debriefings. One-hundred sixty providers participated, including OB-Gyn surgeons, anesthesiologists, CRNAs, scrub technicians, and nurses. Using reflective thematic analysis, we identified three core themes: 1) The surgeon is focused on the surgical field, 2) Explicit leadership initiates a nurse transition from follower to leader in a hierarchical environment, and 3) Explicit distributed leadership enhances teamwork and taskwork. Continuing education which uses distributed leadership to improve teams' ability to respond to an obstetric emergency is perceived to enhance team members' response to the critical event . The potential for nurses' career growth and professional transformation was an unexpected finding associated with this continuing education which used distributed leadership. Our findings suggest that healthcare educators should consider ways in which distributed leadership may improve teams' response to critical events in the operating room.

Modeling Diagnostic Expertise in Cases of Irreducible Uncertainty: The Decision-Aligned Response Model.

PURPOSE: Assessing expertise using psychometric models usually yields a measure of ability that is difficult to generalize to the complexity of diagnoses in clinical practice. However, using an item response modeling framework, it is possible to create a decision-aligned response model that captures a clinician's decision-making behavior on a continuous scale that fully represents competing diagnostic possibilities. In this proof-of-concept study, the authors demonstrate the necessary statistical conceptualization of this model using a specific electrocardiogram (ECG) example. METHOD: The authors collected a range of ECGs with elevated ST segments due to either ST-elevation myocardial infarction (STEMI) or pericarditis. Based on pilot data, 20 ECGs were chosen to represent a continuum from "definitely STEMI" to "definitely pericarditis," including intermediate cases in which the diagnosis was intentionally unclear. Emergency medicine and cardiology physicians rated these ECGs on a 5-point scale ("definitely STEMI" to "definitely pericarditis"). The authors analyzed these ratings using a graded response model showing the degree to which each participant could separate the ECGs along the diagnostic continuum. The authors compared these metrics with the discharge diagnoses noted on chart review. RESULTS: Thirty-seven participants rated the ECGs. As desired, the ECGs represented a range of phenotypes, including cases where participants were uncertain in their diagnosis. The response model showed that participants varied both in their propensity to diagnose one condition over another and in where they placed the thresholds between the 5 diagnostic categories. The most capable participants were able to meaningfully use all categories, with precise thresholds between categories. CONCLUSIONS: The authors present a decision-aligned response model that demonstrates the confusability of a particular ECG and the skill with which a clinician can distinguish 2 diagnoses along a continuum of confusability. These results have broad implications for testing and for learning to manage uncertainty in diagnosis.

The Purpose, Design, and Promise of Medical Education Research Labs.

Medical education researchers are often subject to challenges that include lack of funding, collaborators, study subjects, and departmental support. The construct of a research lab provides a framework that can be employed to overcome these challenges and effectively support the work of medical education researchers; however, labs are relatively uncommon in the medical education field. Using case examples, the authors describe the organization and mission of medical education research labs contrasted with those of larger research team configurations, such as research centers, collaboratives, and networks. They discuss several key elements of education research labs: the importance of lab identity, the signaling effect of a lab designation, required infrastructure, and the training mission of a lab. The need for medical education researchers to be visionary and strategic when designing their labs is emphasized, start-up considerations and the likelihood of support for medical education labs is considered, and the degree to which department leaders should support such labs is questioned.

Publication Inaccuracies Listed in General Surgery Residency Training Program Applications.

BACKGROUND: Professionalism is a core competency that is difficult to assess. We examined the incidence of publication inaccuracies in Electronic Residency Application Service applications to our training program as potential indicators of unprofessional behavior. STUDY DESIGN: We reviewed all 2019-2020 National Resident Matching Program applicants being considered for interview. Applicant demographic characteristics recorded included standardized examination scores, gender, medical school, and medical school ranking (2019 US News & World Report). Publication verification by a medical librarian was performed for peer-reviewed journal articles/abstracts, peer-reviewed book chapters, and peer-reviewed online publications. Inaccuracies were classified as "nonserious" (eg incorrect author order without author rank promotion) or "serious" (eg miscategorization, non-peer-reviewed journal, incorrect author order with author rank promotion, nonauthorship of cited existing publication, and unverifiable publication). Multivariate logistic regression analysis was performed for demographic characteristics to identify predictors of overall inaccuracy and serious inaccuracy. RESULTS: Of 319 applicants, 48 (15%) had a total of 98 inaccuracies; after removing nonserious inaccuracies, 37 (12%) with serious inaccuracies remained. Seven publications were reported in predatory open access journals. In the regression model, none of the variables (US vs non-US medical school, gender, or medical school ranking) were significantly associated with overall inaccuracy or serious inaccuracy. CONCLUSIONS: One in 8 applicants (12%) interviewing at a general surgery residency program were found to have a serious inaccuracy in publication reporting on their Electronic Residency Application Service application. These inaccuracies might represent inattention to detail or professionalism transgressions.

Multi-level longitudinal learning curve regression models integrated with item difficulty metrics for deliberate practice of visual diagnosis: groundwork for adaptive learning.

Visual diagnosis of radiographs, histology and electrocardiograms lends itself to deliberate practice, facilitated by large online banks of cases. Which cases to supply to which learners in which order is still to be worked out, with there being considerable potential for adapting the learning. Advances in statistical modeling, based on an accumulating learning curve, offer methods for more effectively pairing learners with cases of known calibrations. Using demonstration radiograph and electrocardiogram datasets, the advantages of moving from traditional regression to multilevel methods for modeling growth in ability or performance are demonstrated, with a final step of integrating case-level item-response information based on diagnostic grouping. This produces more precise individual-level estimates that can eventually support learner adaptive case selection. The progressive increase in model sophistication is not simply statistical but rather brings the models into alignment with core learning principles including the importance of taking into account individual differences in baseline skill and learning rate as well as the differential interaction with cases of varying diagnosis and difficulty. The developed approach can thus give researchers and educators a better basis on which to anticipate learners' pathways and individually adapt their future learning.

Medical student perspectives on conducting patient experience debrief interviews with hospitalized children and their families.

PURPOSE: To explore how medical students completing a pediatric clerkship viewed the benefits and barriers of debrief interviews with hospitalized patients and families. METHODS: In this study, focus groups were conducted with pediatric clerkship students after completion of a debrief interview. The constant comparative method was used with Mezirow's transformative learning theory as a lens to explore perceptions of the benefits and challenges of performing the interview. RESULTS: Focus groups revealed five benefits and two challenges. The benefits were that the debrief interviews helped students (1) humanize patients and appreciate social and environmental influences on patient health, (2) assess caregiver/patient understanding about care to correct misunderstandings, (3) actively involve caregivers/patients in treatment plan development, (4) engage patients in active expression of questions/concerns, and (5) recognize the value of their own role on the healthcare team. The challenges were that students felt (1) a lack of knowledge to answer caregivers'/patients' questions about diagnoses and (2) discomfort responding to caregiver/patient frustration, anxiety, or sadness. Student feedback on feasibility and implementation led to guidelines for selecting patients and conducting small group discussions after the debrief interviews. CONCLUSIONS: Debrief interviews offer a unique approach for learners to explore patient perspectives during hospitalization through direct patient engagement and dialogue, contributing to professional development, empathy, and potentially more positive patient care experiences.

The Fundamentals of Endoscopic Surgery (FES) skills test: factors associated with first-attempt scores and pass rate.

BACKGROUND: The Fundamentals of Endoscopic Surgery (FES) program became required for American Board of Surgery certification as part of the Flexible Endoscopy Curriculum (FEC) for residents graduating in 2018. This study expands prior psychometric investigation of the FES skills test. METHODS: We analyzed de-identified first-attempt skills test scores and self-reported demographic characteristics of 2023 general surgery residents who were required to pass FES. RESULTS: The overall pass rate was 83%. "Loop Reduction" was the most difficult sub-task. Subtasks related to one another only modestly (Spearman's ρ ranging from 0.11 to 0.42; coefficient α = .55). Both upper and lower endoscopic procedural experience had modest positive association with scores (ρ = 0.14 and 0.15) and passing. Examinees who tested on the GI Mentor Express simulator had lower total scores and a lower pass rate than those tested on the GI Mentor II (pass rates = 73% vs. 85%). Removing an Express-specific scoring rule that had been applied eliminated these differences. Gender, glove size, and height were closely related. Women scored lower than men (408- vs. 489-point averages) and had a lower first-attempt pass rate (71% vs. 92%). Glove size correlated positively with score (ρ = 0.31) and pass rate. Finally, height correlated positively with score (r = 0.27) and pass rate. Statistically controlling for glove size and height did not eliminate gender differences, with men still having 3.2 times greater odds of passing. CONCLUSIONS: FES skills test scores show both consistencies with the assessment's validity argument and several remarkable findings. Subtasks reflect distinct skills, so passing standards should perhaps be set for each subtask. The Express simulator-specific scoring penalty should be removed. Differences seen by gender are concerning. We argue those differences do not reflect measurement bias, but rather highlight equity concerns in surgical technology, training, and practice.

The Patient Experience Debrief Interview: How Conversations With Hospitalized Families Influence Medical Student Learning and Reflection.

PURPOSE: To determine the effect of patient debrief interviews on pediatric clerkship student depth of reflection and learning. METHOD: The authors conducted a multi-institutional, mixed-methods, cluster randomized trial among pediatric clerkship students from May 2016 to February 2017. Intervention students completed a debrief interview with a patient-caregiver, followed by a written reflection on the experience. Control students completed a written reflection on a memorable patient encounter. Three blinded authors scored written reflections according to the 4-level REFLECT rubric to determine depth of reflection. Interrater reliability was examined using kappa. REFLECT scores were analyzed using a chi-square test; essays were analyzed using content analysis. RESULTS: Eighty percent of eligible students participated. One hundred eighty-nine essays (89 control, 100 intervention) were scored. Thirty-seven percent of the control group attained reflection and critical reflection, the 2 highest levels of reflection, compared with 71% of the intervention group; 2% of the control group attained critical reflection, the highest level, compared with 31% of the intervention group (χ(3, N = 189) = 33.9, P < .001). Seven themes were seen across both groups, 3 focused on physician practice and 4 focused on patients. Patient-centered themes were more common in the intervention group, whereas physician-focused themes were more common in the control group. CONCLUSIONS: Patient debrief interviews offer a unique approach to deepen self-reflection through direct dialogue and exploration of patient-caregiver experiences during hospitalization.

Balancing Deliberate Practice and Reflection: A Randomized Comparison Trial of Instructional Designs for Simulation-Based Training in Cardiopulmonary Resuscitation Skills.

INTRODUCTION: A key question in simulation-based education is how to maximize learning within time and resource limits, including how best to balance hands-on practice versus reflective debriefing. Several instructional design frameworks suggest setting the balance according to the type of learning objective(s); however, broad professional activities such as team-based cardiopulmonary resuscitation include several interrelated component skills. This study experimentally manipulated hands-on practice versus reflective debriefing for cardiopulmonary resuscitation skills, hypothesizing that the former best supports learning taskwork (eg, compression quality), whereas the latter best supports learning teamwork. METHODS: The study was a randomized comparison trial with a pretest and posttest. Twenty-six teams of 5 to 6 first-year residents underwent either "drill" practice of key resuscitation phases, designed to maximize deliberate practice opportunities for individual and team skills, or "scrimmage" practice, designed to maximize full-scenario rehearsals and reflective debriefs. Key taskwork and teamwork behaviors were coded, and compression quality was collected and analyzed from an accelerometer. RESULTS: Most performance parameters improved considerably from a pretest to posttest for both taskwork (eg, percent correct compression depth 62%-81%, P = 0.01) and teamwork (eg, role leadership, 47%-70%, P = 0.00). Only 2 parameters improved differently by condition, favoring "drill" training: checking "Do Not Actively Resuscitate" wristband (odds ratio = 14.75, P = 0.03) and use of compression adjuncts (estimated marginal means = 75% versus 67%, P = 0.03). CONCLUSIONS: Consistent with the notion that component skills in resuscitation do not clearly and exclusively constitute "taskwork" versus "teamwork," both instructional designs led to similar improvements despite differences in the balance between hands-on practice versus reflection.

Coaching for Chaos: A Qualitative Study of Instructional Methods for Multipatient Management in the Emergency Department.

BACKGROUND: Busy environments, like the emergency department (ED), require teachers to develop instructional strategies for coaching trainees to function within these same environments. Few studies have documented the strategies used by emergency physician (EP)-teachers within these busy, chaotic environments, instead emphasizing teaching in more predictable environments such as the outpatient clinic, hospital wards, or operating room. The authors sought to discover what strategies EP-teachers were using and what trainees recalled experiencing when learning to handle these unpredictable, overcrowded, complex, multipatient environments. METHOD: An interpretive description study was conducted at multiple teaching hospitals affiliated with McMaster University from July 2014 to May 2015. Participants (10 EP-teachers and 10 junior residents) were asked to recall teaching strategies related to handling ED patient flow. Participants were asked to describe techniques that they used, observed, or experienced as trainees. Two independent coders read through interview transcripts, analyzing these documents inductively and iteratively. RESULTS: Two main types of strategies to teach ED management were discovered: 1) workplace-based methods, including both observation and in situ instruction; and 2) principle-based advice. The most often described techniques were workplace-based methods, which included a variety of in situ techniques ranging from conversations to managerial coaching (e.g., collaborative problem-solving of real-life administrative dilemmas). CONCLUSIONS: A mix of strategies are used to teach and coach trainees to handle multipatient environments. Further research is required to determine how to optimize the use of these techniques and innovate new strategies to support the learning of these crucial skills.

Experts' responses in script concordance tests: a response process validity investigation.

CONTEXT: The script concordance test (SCT), designed to measure clinical reasoning in complex cases, has recently been the subject of several critical research studies. Amongst other issues, response process validity evidence remains lacking. We explored the response processes of experts on an SCT scoring panel to better understand their seemingly divergent beliefs about how new clinical data alter the suitability of proposed actions within simulated patient cases. METHODS: A total of 10 Argentine gastroenterologists who served as the expert panel on an existing SCT re-answered 15 cases 9 months after their original panel participation. They then answered questions probing their reasoning and reactions to other experts' perspectives. RESULTS: The experts sometimes noted they would not ordinarily consider the actions proposed for the cases at all (30/150 instances [20%]) or would collect additional data first (54/150 instances [36%]). Even when groups of experts agreed about how new clinical data in a case affected the suitability of a proposed action, there was often disagreement (118/133 instances [89%]) about the suitability of the proposed action before the new clinical data had been introduced. Experts reported confidence in their responses, but showed limited consistency with the responses they had given 9 months earlier (linear weighted kappa = 0.33). Qualitative analyses showed nuanced and complex reasons behind experts' responses, revealing, for example, that experts often considered the unique affordances and constraints of their varying local practice environments when responding. Experts generally found other experts' alternative responses moderately compelling (mean ± standard deviation 2.93 ± 0.80 on a 5-point scale, where 3 = moderately compelling). Experts switched their own preferred responses after seeing others' reasoning in 30 of 150 (20%) instances. CONCLUSIONS: Expert response processes were not consistent with the classical interpretation and use of SCT scores. However, several fruitful and justifiable alternatives for the use of SCT-like methods are proposed, such as to guide assessments for learning.

Influencing Mindsets and Motivation in Procedural Skills Learning: Two Randomized Studies.

OBJECTIVES: An incremental (growth) theory of intelligence (mindset), compared with an entity (fixed) mindset, has been associated with improved motivation and performance. Interventions to induce incremental beliefs have improved performance on non-surgical motor tasks. We sought to evaluate the impact of 2 brief interventions to induce incremental beliefs in the context of learning a surgical task. DESIGN: Two randomized experiments. PARTICIPANTS AND SETTING: Secondary school students participating in medical simulation-based training activities at an academic medical center. INTERVENTIONS: We created 4 instructional messages intended to influence mindsets (two 60-second videos in Study 1, 2 fabricated "journal articles" in Study 2). In each study, one message emphasized that ability improves with practice (incremental); the other emphasized that ability is fixed (entity). After reviewing their randomly-assigned message, participants completed a laparoscopic cutting task as many times as they desired. Measurements included performance (product quality, self-reported task, and completion time), task persistence (repetitions), and entity beliefs. RESULTS: Two hundred and three students completed Study 1. Postevent entity beliefs (1 = lowest, 6 = highest) were similar between groups (incremental, 2.0vs entity, 2.0; p = 0.78). Contrary to hypothesis, the incremental video group demonstrated slower time (276vs 191 seconds; p < 0.0001), lower product quality (7.2vs 3.8mm deviation; p < 0.0001), and fewer task repetitions (1.4vs 1.8; p = 0.02). In Study 2, 113 participants provided outcomes related to mindset beliefs, but only 14 provided usable performance outcomes. Postevent entity beliefs were lower in the incremental article group (1.7vs 2.4; p < 0.0001). Task time (507vs 585 seconds; p = 0.40) and quality (7.1vs 7.5mm deviation; p = 0.85) were similar between groups. CONCLUSIONS: Brief motivational interventions can influence procedural performance and motivation. We need to better understand motivation and other affective influences on procedural skills learning. Mindset theory shows promise in this regard.

How well is each learner learning? Validity investigation of a learning curve-based assessment approach for ECG interpretation.

Learning curves can support a competency-based approach to assessment for learning. When interpreting repeated assessment data displayed as learning curves, a key assessment question is: "How well is each learner learning?" We outline the validity argument and investigation relevant to this question, for a computer-based repeated assessment of competence in electrocardiogram (ECG) interpretation. We developed an on-line ECG learning program based on 292 anonymized ECGs collected from an electronic patient database. After diagnosing each ECG, participants received feedback including the computer interpretation, cardiologist's annotation, and correct diagnosis. In 2015, participants from a single institution, across a range of ECG skill levels, diagnosed at least 60 ECGs. We planned, collected and evaluated validity evidence under each inference of Kane's validity framework. For Scoring, three cardiologists' kappa for agreement on correct diagnosis was 0.92. There was a range of ECG difficulty across and within each diagnostic category. For Generalization, appropriate sampling was reflected in the inclusion of a typical clinical base rate of 39% normal ECGs. Applying generalizability theory presented unique challenges. Under the Extrapolation inference, group learning curves demonstrated expert-novice differences, performance increased with practice and the incremental phase of the learning curve reflected ongoing, effortful learning. A minority of learners had atypical learning curves. We did not collect Implications evidence. Our results support a preliminary validity argument for a learning curve assessment approach for repeated ECG interpretation with deliberate and mixed practice. This approach holds promise for providing educators and researchers, in collaboration with their learners, with deeper insights into how well each learner is learning.

Simulation as More Than a Treatment-Planning Tool: A Systematic Review of the Literature on Radiation Oncology Simulation-Based Medical Education.

PURPOSE: Simulation-based medical education (SBME) is gaining prominence as a tool to meet Accreditation Council for Graduate Medical Education-mandated competency-based assessment educational goals. SBME is used in radiation oncology, although the type and extent are not clear. This study reports a systematic literature review designed to clarify the type and extent of radiation oncology SBME. METHODS AND MATERIALS: The systematic review focused on radiation oncology SBME literature. The methods followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The inclusion criteria were identified according to the PICOS (population, intervention, comparison, outcome, and setting) framework. The population included undergraduate, graduate, and continuing medical education learners. Studies were limited to English-language studies published on or after January 1, 1990, in peer-reviewed journals. PubMed, MedEdPORTAL, and in-press articles were searched. The PubMed search was conducted using predefined search terms. References and similar articles were examined. Medical Subject Headings terms in selected articles were reviewed to ensure relevant terms were included. RESULTS: Fifty-four SBME publications met the inclusion criteria. Only 9 of 54 studies (17%) self-identified as SBME. SBME types included screen-based simulators (56%), simulated environments (13%), virtual reality and haptic systems (13%), simulated patients (11%), part-task trainers (6%), and computer-based systems with mannequins (2%). A variety of radiation oncology skill sets were addressed, including contouring (54%), treatment planning (20%), clinical decision making (17%), anatomy and/or radiology (13%), radiation biology and/or physics (13%), communication skills and/or patient education (13%), brachytherapy (13%), and immobilization (11%). A target learning population was defined in 47 studies, including residents (53%), attending physicians (36%), medical students (21%), medical physicists (11%), radiation therapists (9%), nurses (6%), administrative staff (4%), and dosimetrists (4%). Learner feedback was reported in 32 studies. CONCLUSIONS: Overall, this systematic literature review provides context and guidance for future radiation oncology SBME development. Appropriately framing SBME reports in the radiation oncology literature will facilitate development, implementation, and evaluation of SBME interventions. SBME resources should be centralized to facilitate dissemination and share resources.

Development, administration, and validity evidence of a subspecialty preparatory test toward licensure: a pilot study.

BACKGROUND: Trainees in medical subspecialties lack validated assessment scores that can be used to prepare for their licensing examination. This paper presents the development, administration, and validity evidence of a constructed-response preparatory test (CRPT) administered to meet the needs of nephrology trainees. METHODS: Learning objectives from the licensing examination were used to develop a test blueprint for the preparatory test. Messick's unified validity framework was used to gather validity evidence for content, response process, internal structure, relations to other variables, and consequences. Questionnaires were used to gather data on the trainees' perception of examination preparedness, item clarity, and curriculum adequacy. RESULTS: There were 10 trainees and 5 faculty volunteers who took the test. The majority of trainees passed the constructed-response preparatory test. However, many scored poorly on items assessing renal pathology and physiology knowledge. We gathered the following five sources of validity evidence: (1) Content: CRPT mapped to the licensing examination blueprint, with items demonstrating clarity and range of difficulty; (2) Response process: moderate rater agreement (intraclass correlation = .58); (3) Internal structure: sufficient reliability based on generalizability theory (G-coefficient = .76 and Φ-coefficient = .53); (4) Relations to other variables: CRPT scores reflected years of exposure in nephrology and clinical practice; (5) Consequences: post-assessment survey revealed that none of the test takers felt "poorly prepared" for the upcoming summative examination and that their studying would increase in duration and be adapted in terms of content focus. CONCLUSIONS: Preparatory tests using constructed response items mapped to licensure examination blueprint can be developed and used at local program settings to help prepare learners for subspecialty licensure examinations. The CRPT and questionnaire data identified shortcomings of the nephrology training program curriculum. Following the preparatory test, trainees expressed an improved sense of preparedness for their licensing examination.

Simulation-based mastery learning significantly reduces gender differences on the Fundamentals of Endoscopic Surgery performance exam.

BACKGROUND: Analysis of the Fundamentals of Endoscopic Surgery (FES) performance exam showed higher scores for men than women. Gender differences have been reduced with task-specific practice. We assessed the effect of simulation-based mastery learning (SBML) on FES performance exam differences by gender. METHODS: Forty-seven surgical trainees [29 men (m), 18 women (w)] completed a SBML curriculum and were assessed by FES. Fourteen trained on the GI Mentor 2, 18 on the Endoscopy Training System, and 15 using the Surgical Training for Endoscopic Proficiency curriculum. Performance of male and female trainees was compared. RESULTS: On the pre-training assessment, there were large differences between genders in FES pass rates (m 77%, w 15%, p < 0.001), total scores (m 69 ± 11, w 50 ± 12; p < 0.001), and in four of five FES sub-task scores (Navigation, m 73 ± 19, w 55 ± 22, p = 0.02; Loop reduction, m 34 ± 29, w 14 ± 22, p = 0.02; Retroflexion, m 81 ± 17, w 47 ± 27, p < 0.001; Targeting, m 89 ± 10, w 66 ± 23, p = 0.002). No differences were discernible post training (Pass rate, m 100%, w 94%, p = 0.4; Total score, m 77 ± 8, w 72 ± 12, p = 0.2; Navigation, m 91 ± 13, w 80 ± 13, p = 0.009; Loop reduction, m 49 ± 26, w 46 ± 36, p = 0.7; Retroflexion, m 82 ± 18, w 81 ± 15, p = 0.9; Targeting, m 92 ± 15, w 86 ± 12, p = 0.12). Time needed to complete curricula was not discernably different by gender (m 3.8 ± 1.7 h, w 5.0 ± 2.6 h, p = 0.17). CONCLUSIONS: Gender-based differences are nearly eliminated through task-specific SBML training. This lends further evidence to the validity argument for the FES performance exam as a measure of basic endoscopic skills.