Personalisation and embodiment in e-Learning for health professionals: A randomised controlled trial.

PURPOSE: Mayer's theory of multimedia learning proposes that personalisation and embodiment (P/E) can improve outcomes in e-Learning. The authors hypothesised that an e-Learning module enhanced by P/E principles would lead to higher knowledge, perceived P/E and motivation among health care professionals, compared with an unenhanced module. METHODS: The authors conducted a randomised trial comparing two versions of a 30-minute multimedia e-Learning module addressing the antibiotic management of pneumonia. The unenhanced format used slides with voiceover (human voice but no visible speaker), formal language and no specific P/E strategies. The enhanced format additionally implemented P/E strategies including conversational style, polite language, visible author, social congruence, human-like presence and professional presence by subtly changing the script and substituting several short videos of subject matter experts. Participants included pharmacists, physicians and advanced practice providers from three academic and several community hospitals. Outcomes included knowledge, perceived P/E (assessed by the Congruence Personalisation Questionnaire, CPQ), motivation (assessed via the Instructional Materials Motivation Survey [IMMS] and Motivated Strategies for Learning Questionnaire [MSLQ]) and course satisfaction. RESULTS: There were 406 participants including 225 pharmacists, 109 physicians and 72 advanced practice providers. Post-module knowledge was slightly higher for the enhanced versus the unenhanced format, but the difference did not reach statistical significance (adjusted mean difference, 0.04 of 10 possible, [95% CI -0.26, 0.34], p = 0.78; Cohen d 0.02). Participant perceptions of P/E (measured via CPQ) were significantly greater for the enhanced format (difference 0.46 of 5 possible [0.35, 0.56], p < 0.001; Cohen d 0.85), as were motivational features of the e-Learning course (measured via IMMS) (difference 0.14 of 5 possible [0.02, 0.26], p = 0.02; Cohen d 0.24). Participants' overall motivational orientation (measured via MSLQ) and course satisfaction were not significantly different between the two formats (p > 0.05). CONCLUSION: Application of P/E principles to an e-Learning module led to greater perceived P/E and motivational features but did not influence knowledge.

Creating virtual patients using large language models: scalable, global, and low cost.

Virtual patients (VPs) have long been used to teach and assess clinical reasoning. VPs can be programmed to simulate authentic patient-clinician interactions and to reflect a variety of contextual permutations. However, their use has historically been limited by the high cost and logistical challenges of large-scale implementation. We describe a novel globally-accessible approach to develop low-cost VPs at scale using artificial intelligence (AI) large language models (LLMs). We leveraged OpenAI Generative Pretrained Transformer (GPT) to create and implement two interactive VPs, and created permutations that differed in contextual features. We used systematic prompt engineering to refine a prompt instructing ChatGPT to emulate the patient for a given case scenario, and then provide feedback on clinician performance. We implemented the prompts using GPT-3.5-turbo and GPT-4.0, and created a simple text-only interface using the OpenAI API. GPT-4.0 was far superior. We also conducted limited testing using another LLM (Anthropic Claude), with promising results. We provide the final prompt, case scenarios, and Python code. LLM-VPs represent a 'disruptive innovation' - an innovation that is unmistakably inferior to existing products but substantially more accessible (due to low cost, global reach, or ease of implementation) and thereby able to reach a previously underserved market. LLM-VPs will lay the foundation for global democratization via low-cost-low-risk scalable development of educational and clinical simulations. These powerful tools could revolutionize the teaching, assessment, and research of management reasoning, shared decision-making, and AI evaluation (e.g. 'software as a medical device' evaluations).

Validation of the motivated strategies for learning questionnaire and instructional materials motivation survey.

PURPOSE: To validate the Motivated Strategies for Learning Questionnaire (MSLQ), which measures learner motivations; and the Instructional Materials Motivation Survey (IMMS), which measures the motivational properties of educational activities. METHODS: Participants (333 pharmacists, physicians, and advanced practice providers) completed the MSLQ, IMMS, Congruence-Personalization Questionnaire (CPQ), and a knowledge test immediately following an online learning module (April 2021). We randomly divided data for split-sample analysis using confirmatory factor analysis (CFA), exploratory factor analysis (EFA), and the multitrait-multimethod matrix. RESULTS: Cronbach alpha was ≥0.70 for most domains. CFA using sample 1 demonstrated suboptimal fit for both instruments, including 3 negatively-worded IMMS items with particularly low loadings. Revised IMMS (RIMMS) scores (which omit negatively-worded items) demonstrated better fit. Guided by EFA, we identified a novel 3-domain, 11-item 'MSLQ-Short Form-Revised' (MSLQ-SFR, with domains: Interest, Self-efficacy, and Attribution) and the 4-domain, 12-item RIMMS as the best models. CFA using sample 2 confirmed good fit. Correlations among MSLQ-SFR, RIMMS, and CPQ scores aligned with predictions; correlations with knowledge scores were small. CONCLUSIONS: Original MSLQ and IMMS scores show poor model fit, with negatively-worded items notably divergent. Revised, shorter models-the MSLQ-SFR and RIMMS-show satisfactory model fit (internal structure) and relations with other variables.

Assessment of management reasoning: Design considerations drawn from analysis of simulated outpatient encounters.

PURPOSE: Management reasoning is a distinct subset of clinical reasoning. We sought to explore features to be considered when designing assessments of management reasoning. METHODS: This is a hybrid empirical research study, narrative review, and expert perspective. In 2021, we reviewed and discussed 10 videos of simulated (staged) physician-patient encounters, actively seeking actions that offered insights into assessment of management reasoning. We analyzed our own observations in conjunction with literature on clinical reasoning assessment, using a constant comparative qualitative approach. RESULTS: Distinguishing features of management reasoning that will influence its assessment include management scripts, shared decision-making, process knowledge, illness-specific knowledge, and tailoring of the encounter and management plan. Performance domains that merit special consideration include communication, integration of patient preferences, adherence to the management script, and prognostication. Additional facets of encounter variation include the clinical problem, clinical and nonclinical patient characteristics (including preferences, values, and resources), team/system characteristics, and encounter features. We cataloged several relevant assessment approaches including written/computer-based, simulation-based, and workplace-based modalities, and a variety of novel response formats. CONCLUSIONS: Assessment of management reasoning could be improved with attention to the performance domains, facets of variation, and variety of approaches herein identified.

Understanding Reader Variability: A 25-Radiologist Study on Liver Metastasis Detection at CT.

Background Substantial interreader variability exists for common tasks in CT imaging, such as detection of hepatic metastases. This variability can undermine patient care by leading to misdiagnosis. Purpose To determine the impact of interreader variability associated with (a) reader experience, (b) image navigation patterns (eg, eye movements, workstation interactions), and (c) eye gaze time at missed liver metastases on contrast-enhanced abdominal CT images. Materials and Methods In a single-center prospective observational trial at an academic institution between December 2020 and February 2021, readers were recruited to examine 40 contrast-enhanced abdominal CT studies (eight normal, 32 containing 91 liver metastases). Readers circumscribed hepatic metastases and reported confidence. The workstation tracked image navigation and eye movements. Performance was quantified by using the area under the jackknife alternative free-response receiver operator characteristic (JAFROC-1) curve and per-metastasis sensitivity and was associated with reader experience and image navigation variables. Differences in area under JAFROC curve were assessed with the Kruskal-Wallis test followed by the Dunn test, and effects of image navigation were assessed by using the Wilcoxon signed-rank test. Results Twenty-five readers (median age, 38 years; IQR, 31-45 years; 19 men) were recruited and included nine subspecialized abdominal radiologists, five nonabdominal staff radiologists, and 11 senior residents or fellows. Reader experience explained differences in area under the JAFROC curve, with abdominal radiologists demonstrating greater area under the JAFROC curve (mean, 0.77; 95% CI: 0.75, 0.79) than trainees (mean, 0.71; 95% CI: 0.69, 0.73) (P = .02) or nonabdominal subspecialists (mean, 0.69; 95% CI: 0.60, 0.78) (P = .03). Sensitivity was similar within the reader experience groups (P = .96). Image navigation variables that were associated with higher sensitivity included longer interpretation time (P = .003) and greater use of coronal images (P < .001). The eye gaze time was at least 0.5 and 2.0 seconds for 71% (266 of 377) and 40% (149 of 377) of missed metastases, respectively. Conclusion Abdominal radiologists demonstrated better discrimination for the detection of liver metastases on abdominal contrast-enhanced CT images. Missed metastases frequently received at least a brief eye gaze. Higher sensitivity was associated with longer interpretation time and greater use of liver display windows and coronal images. © RSNA, 2022 Online supplemental material is available for this article.

Learner engagement and teaching effectiveness in livestreamed versus in-person CME.

INTRODUCTION: Engaging learners in continuing medical education (CME) is challenging. Recently, CME courses have transitioned to livestreamed CME, with learners viewing live, in-person courses online. The authors aimed to (1) compare learner engagement and teaching effectiveness in livestreamed with in-person CME and (2) determine how livestream engagement and teaching effectiveness is associated with (A) interactivity metrics, (B) presentation characteristics and (C) medical knowledge. METHODS: A 3-year, non-randomised study of in-person and livestream CME was performed. The course was in-person for 2018 but transitioned to livestream for 2020 and 2021. Learners completed the Learner Engagement Inventory and Teaching Effectiveness Instrument after each presentation. Both instruments were supported by content, internal structure and relations to other variables' validity evidence. Interactivity metrics included learner use of audience response, questions asked by learners and presentation views. Presentation characteristics included presentations using audience response, using pre/post-test format, time of day and words per slide. Medical knowledge was assessed by audience response. A repeated measures analysis of variance (anova) was used for comparisons and a mixed model approach for correlations. RESULTS: A total of 159 learners (response rate 27%) completed questionnaires. Engagement did not significantly differ between in-person or livestream CME. (4.56 versus 4.53, p = 0.64, maximum 5 = highly engaged). However, teacher effectiveness scores were higher for in-person compared with livestream (4.77 versus 4.71 p = 0.01, maximum 5 = highly effective). For livestreamed courses, learner engagement was associated with presentation characteristics, including presentation using of audience response (yes = 4.57, no = 4.45, p < .0001), use of a pre/post-test (yes = 4.62, no = 4.54, p < .0001) and time of presentation (morning = 4.58, afternoon = 4.53, p = .0002). Significant associations were not seen for interactivity metrics or medical knowledge. DISCUSSION: Livestreaming may be as engaging as in-person CME. Although teaching effectiveness in livestreaming was lower, this difference was small. CME course planners should consider offering livestream CME while exploring strategies to enhance teaching effectiveness in livestreamed settings.

Management reasoning and patient-clinician interactions: Insights from shared decision-making and simulated outpatient encounters.

PURPOSE: To expand understanding of patient-clinician interactions in management reasoning. METHODS: We reviewed 10 videos of simulated patient-clinician encounters to identify instances of problematic and successful communication, then reviewed the videos again through the lens of two models of shared decision-making (SDM): an 'involvement-focused' model and a 'problem-focused' model. Using constant comparative qualitative analysis we explored the connections between these patient-clinician interactions and management reasoning. RESULTS: Problems in patient-clinician interactions included failures to: encourage patient autonomy; invite the patient's involvement in decision-making; convey the health impact of the problem; explore and address concerns and questions; explore the context of decision-making (including patient preferences); meet the patient where they are; integrate situational preferences and priorities; offer >1 viable option; work with the patient to solve a problem of mutual concern; explicitly agree to a final care plan; and build the patient-clinician relationship. Clinicians' 'management scripts' varied along a continuum of prioritizing clinician vs patient needs. Patients also have their own cognitive scripts that guide their interactions with clinicians. The involvement-focused and problem-focused SDM models illuminated distinct, complementary issues. CONCLUSIONS: Management reasoning is a deliberative interaction occurring in the space between individuals. Juxtaposing management reasoning alongside SDM generated numerous insights.

Evaluating education innovations rapidly with build-measure-learn: Applying lean startup to health professions education.

PURPOSE: The Lean Startup approach allows innovators (including innovative educators) to rapidly identify and refine promising ideas into models that actually work. Our aim is to outline key principles of Lean Startup, apply these to health professions education, and illustrate these using personal experience. METHODS AND RESULTS: All innovations are grounded in numerous assumptions; these assumptions should be explicitly identified, prioritized, and empirically tested ('validated learning'). To identify and test assumptions, innovators need to get out of the office and interact with customers (learners, teachers, administrators, etc). Assumptions are tested using multiple quick cycles of Build (a 'minimal viable product' [MVP]), Measure (using metrics that relate meaningfully to the assumption), and Learn (interpret data and decide to persevere with further refinements, or pivot to a new direction). The MVP is a product version that allows testing of one or more key assumptions with the least effort. We describe a novel 'Lean Education Canvas' that synopsizes an innovation and its business model on one page, to help identify assumptions and monitor progress. We illustrate these principles using three cases from health professions education. CONCLUSIONS: Lean Startup has tremendous potential for rapid, robust innovation and evaluation in education.[Box: see text].

Digital Education for Health Professionals: An Evidence Map, Conceptual Framework, and Research Agenda.

BACKGROUND: Health professions education has undergone major changes with the advent and adoption of digital technologies worldwide. OBJECTIVE: This study aims to map the existing evidence and identify gaps and research priorities to enable robust and relevant research in digital health professions education. METHODS: We searched for systematic reviews on the digital education of practicing and student health care professionals. We searched MEDLINE, Embase, Cochrane Library, Educational Research Information Center, CINAHL, and gray literature sources from January 2014 to July 2020. A total of 2 authors independently screened the studies, extracted the data, and synthesized the findings. We outlined the key characteristics of the included reviews, the quality of the evidence they synthesized, and recommendations for future research. We mapped the empirical findings and research recommendations against the newly developed conceptual framework. RESULTS: We identified 77 eligible systematic reviews. All of them included experimental studies and evaluated the effectiveness of digital education interventions in different health care disciplines or different digital education modalities. Most reviews included studies on various digital education modalities (22/77, 29%), virtual reality (19/77, 25%), and online education (10/77, 13%). Most reviews focused on health professions education in general (36/77, 47%), surgery (13/77, 17%), and nursing (11/77, 14%). The reviews mainly assessed participants' skills (51/77, 66%) and knowledge (49/77, 64%) and included data from high-income countries (53/77, 69%). Our novel conceptual framework of digital health professions education comprises 6 key domains (context, infrastructure, education, learners, research, and quality improvement) and 16 subdomains. Finally, we identified 61 unique questions for future research in these reviews; these mapped to framework domains of education (29/61, 47% recommendations), context (17/61, 28% recommendations), infrastructure (9/61, 15% recommendations), learners (3/61, 5% recommendations), and research (3/61, 5% recommendations). CONCLUSIONS: We identified a large number of research questions regarding digital education, which collectively reflect a diverse and comprehensive research agenda. Our conceptual framework will help educators and researchers plan, develop, and study digital education. More evidence from low- and middle-income countries is needed.

Autonomy and focus of attention in medical motor skills learning: a randomized experiment.

BACKGROUND: The 'OPTIMAL' (Optimizing Performance Through Intrinsic Motivation and Attention for Learning) theory of motor learning suggests that autonomy, external focus of attention, and perceived competence can improve learning of simple motor tasks. The authors hypothesized that enhanced (vs. routine) autonomy and external (vs. internal) focus of attention would improve first-try performance of two medical motor tasks. METHODS: The authors conducted a randomized two-by-two factorial design study with high school students as participants. Task instructions promoted either enhanced or routine autonomy, and either external or internal focus of attention. These conditions were replicated in a crossover design for two common medical tasks (chest compressions on a manikin and a Fundamentals of Laparoscopic Surgery peg transfer task). Primary outcomes were objective measures of task performance (chest compression deviation from target depth; peg transfer time with penalties for errors). Secondary outcomes included subjective perceptions of confidence, autonomy, and workload. RESULTS: One hundred thirty-three high school students participated in this study. The primary outcomes concerning enhanced vs. routine autonomy demonstrated no statistically significant difference in either task (chest compression depth deviation: difference -0.7 mm [score range 0 to 37.5 mm]; 95% confidence interval (CI95) -3.85, 2.41; p = .65; peg transfer penalized time: rate ratio 1.03; CI95 0.91, 1.31; p = .79). The authors likewise found no statistically significant difference for external vs. internal focus of attention (depth deviation: difference 1.1 mm; CI95 -2.04, 4.17; p = .50; penalized time: rate ratio 0.89; CI95 0.75, 1.13; p = .33). The authors found no statistically significant differences for either comparison in confidence, autonomy and workload (p > .09; differences ranged from -0.83 to 0.79 [scale range 0 to 10]). CONCLUSIONS: First-try performance of chest compressions and peg transfer by novice learners is not significantly affected by enhanced (vs. routine) autonomy or external (vs. internal) focus of attention.

Multi-level longitudinal learning curve regression models integrated with item difficulty metrics for deliberate practice of visual diagnosis: groundwork for adaptive learning.

Visual diagnosis of radiographs, histology and electrocardiograms lends itself to deliberate practice, facilitated by large online banks of cases. Which cases to supply to which learners in which order is still to be worked out, with there being considerable potential for adapting the learning. Advances in statistical modeling, based on an accumulating learning curve, offer methods for more effectively pairing learners with cases of known calibrations. Using demonstration radiograph and electrocardiogram datasets, the advantages of moving from traditional regression to multilevel methods for modeling growth in ability or performance are demonstrated, with a final step of integrating case-level item-response information based on diagnostic grouping. This produces more precise individual-level estimates that can eventually support learner adaptive case selection. The progressive increase in model sophistication is not simply statistical but rather brings the models into alignment with core learning principles including the importance of taking into account individual differences in baseline skill and learning rate as well as the differential interaction with cases of varying diagnosis and difficulty. The developed approach can thus give researchers and educators a better basis on which to anticipate learners' pathways and individually adapt their future learning.

How to conduct cost and value analyses in health professions education: AMEE Guide No. 139.

Growing demand for accountability, transparency, and efficiency in health professions education is expected to drive increased demand for, and use of, cost and value analyses. In this AMEE Guide, we introduce key concepts, methods, and literature that will enable novices in economics to conduct simple cost and value analyses, hold informed discussions with economic specialists, and undertake further learning on more advanced economic topics. The practical structure for conducting analyses provided in this guide will enable researchers to produce robust results that are meaningful and useful for improving educational practice. Key steps include defining the economic research question, identifying an appropriate economic study design, carefully identifying cost ingredients, quantifying, and pricing the ingredients consumed, and conducting sensitivity analyses to explore uncertainties in the results.

Barriers to identifying and obtaining CME: a national survey of physicians, nurse practitioners and physician assistants.

BACKGROUND: CPD educators and CME providers would benefit from further insight regarding barriers and supports in obtaining CME, including sources of information about CME. To address this gap, we sought to explore challenges that clinicians encounter as they seek CME, and time and monetary support allotted for CME. METHODS: In August 2018, we surveyed licensed US clinicians (physicians, nurse practitioners, and physician assistants), sampling 100 respondents each of family medicine physicians, internal medicine and hospitalist physicians, medicine specialist physicians, nurse practitioners, and physician assistants (1895 invited, 500 [26.3%] responded). The Internet-based questionnaire addressed barriers to obtaining CME, sources of CME information, and time and monetary support for CME. RESULTS: The most often-selected barriers were expense (338/500 [68%]) and travel time (N = 286 [57%]). The source of information about CME activities most commonly selected was online search (N = 348 [70%]). Direct email, professional associations, direct mail, and journals were also each selected by > 50% of respondents. Most respondents reported receiving 1-6 days (N = 301 [60%]) and $1000-$5000 (n = 263 [53%]) per year to use in CME activities. Most (> 70%) also reported no change in time or monetary support over the past 24 months. We found few significant differences in responses across clinician type or age group. In open-ended responses, respondents suggested eight ways to enhance CME: optimize location, reduce cost, publicize effectively, offer more courses and content, allow flexibility, ensure accessibility, make content clinically relevant, and encourage application. CONCLUSIONS: Clinicians report that expense and travel time are the biggest barriers to CME. Time and money support is limited, and not increasing. Online search and email are the most frequently-used sources of information about CME. Those who organize and market CME should explore options that reduce barriers of time and money, and creatively use online tools to publicize new offerings.

Adaptive instruction and learner interactivity in online learning: a randomized trial.

The purpose of this study was to evaluate two online instructional design features, namely adaptation to learner prior knowledge and use of questions to enhance interactivity in online portrayals of physician-patient encounters, in the context of instructing surgical specialists to deliver perioperative tobacco interventions. An online learning module on perioperative tobacco control was developed, in formats incorporating permutations of adaptive/non-adaptive and high/low interactivity (i.e., 2 × 2 factorial design). Participants (a national sample of US anesthesiology residents) were randomly assigned to module format. Primary outcomes included tobacco knowledge, time to complete the module, and self-efficacy in delivering tobacco interventions. One hundred fourteen residents completed the module, which required a median of 60 min (interquartile range 49, 138). The difference in post-module tobacco knowledge score was similar for adaptive and non-adaptive formats [mean difference 0.3 of 10 possible (95% CI - 0.3, 1.0), p = 0.25] but time was shorter for the adaptive format [- 7 min (95% CI - 14, 0), p = 0.01] and knowledge efficiency (knowledge score divided by time) was higher [0.08 units (95% 0.03, 0.14), p = 0.004]. The level of interactivity had no significant effect on self-efficacy [- 0.1 on a 5-point scale (95% CI - 0.3, 0.1), p = 0.50] in delivering tobacco interventions (both outcomes using 5-point scales). Adapting online instruction to learners' prior knowledge appears to improve the efficiency of learning; adaptation should be implemented when feasible. Adding features that encourage learner interaction in an online course does not necessarily improve learning outcomes.

Speed and quality goals in procedural skills learning: A randomized experiment.

Purpose: Compare time (speed) and product quality goals in a surgical procedural task.Methods: Secondary school students participating in a medical simulation-based training activity participated in a randomized experiment. Each participant completed eight repetitions of a blood vessel ligation. Once, between repetitions four and five, each participant received a randomly-assigned speed goal or quality goal. Outcomes included time and leak-free ligatures.Results: 80 students participated. The speed-goal group performed 18% faster on the final repetition than the quality-goal group, with adjusted fold change (FC) 0.82 (95% confidence interval [CI], 0.71, 0.94; p = 0.01). Conversely, the speed-goal group had fewer high-quality (leak-free) ligatures (odds ratio [OR] 0.36 [95% CI, 0.22, 0.58; p < 0.001]). For the quality-goal group, leaky ligatures took longer post-intervention than leak-free ligatures (FC 1.09 [95% CI, 1.02, 1.17; p = 0.01]), whereas average times for leaky and leak-free ligatures were similar for the speed-goal group (FC 0.97 [95% CI, 0.91, 1.04; p = 0.38]). For a given performance time, the speed-goal group had more leaks post-intervention than the quality-goal group (OR 3.35 [95% CI, 1.58, 7.10; p = 0.002]).Conclusions: Speed and quality goals promote different learning processes and outcomes among novices. Use of both speed and quality goals may facilitate more effective and efficient learning.

Managing the tension: From innovation to application in health professions education.

Innovations in education are essential for solving problems and introducing new ways of thinking. However, implementation of these innovations must take several factors into consideration, including the context, the environment, the stakeholders, the technology needed, the cost, the pace of implementation, appropriateness, and available resources. When these factors are not balanced and considered, tensions arise. This paper describes tensions in five major educational domains, namely curriculum, instruction, assessment, accreditation/regulation, and faculty development. For each of these domains, the typical problems are described, and solutions are presented to manage the tension.

An Inexpensive, Portable Physical Endoscopic Simulator: Description and Initial Evaluation.

BACKGROUND: Surgeons and gastroenterologists in training benefit from simulation-based endoscopy education, yet the price of most training endoscopy simulators is prohibitive. We set out to create and evaluate a low-cost endoscopic simulator and box model trainer for learning fundamental endoscopic skills. METHODS: After adding a wireless network-enabled camera (total cost, $20) to a discarded clinical endoscope, we paired this with an easily constructed box trainer (cost $32) to generate an endoscopic simulator system (YazanoScope) for simulation training. Participants (general surgery interns, research fellows, and medical and college students) used the YazanoScope to train to mastery on 5 FES tasks. Outcomes included skill assessments on a computer simulator and trainees' perceptions comparing the physical model to the computer simulator. RESULTS: Forty trainees participated. The median (range) training time was 110 (60-180) min. Only 10% of trainees were able to reach the cecum at baseline compared to 100% after training. The mean (SD) time was 253 (154) s at baseline (including completers and non-completers) and 249 (89) after training (P = 0.88). On a retention test 2 wk later, 21 of 22 (96%) completed the computer simulator assessment (endoscope tip reached the cecum). Mean time was 214 (67) s (P = 0.32 compared with immediate posttraining). All 40 trainees believed the YazanoScope provided better haptic feedback than the computer simulator. CONCLUSIONS: Training with this inexpensive, portable endoscopic simulator (YazanoScope) was associated with increased procedure completion with no change in procedure time. All participants favored the haptic feedback of the $52 YazanoScope over a computer simulator.

Cost evaluations in health professions education: a systematic review of methods and reporting quality.

CONTEXT: High-quality research into education costs can inform better decision making. Improvements to cost research can be guided by information about the research questions, methods and reporting of studies evaluating costs in health professions education (HPE). Our objective was to appraise the overall state of the field and evaluate temporal trends in the methods and reporting quality of cost evaluations in HPE research. METHODS: We searched the MEDLINE, CINAHL (Cumulative Index to Nursing and Allied Health Literature), EMBASE, Business Source Complete and ERIC (Education Resources Information Centre) databases on 31 July 2017. To evaluate trends over time, we sampled research reports at 5-year intervals (2001, 2006, 2011 and 2016). All original research studies in HPE that reported a cost outcome were included. The Medical Education Research Study Quality Instrument (MERSQI) and the BMJ economic checklist were used to appraise methodological and reporting quality, respectively. Trends in quality over time were analysed. RESULTS: A total of 78 studies were included, of which 16 were published in 2001, 15 in 2006, 20 in 2011 and 27 in 2016. The region most commonly represented was the USA (n = 43). The profession most commonly referred to was that of the physician (n = 46). The mean ± standard deviation (SD) MERSQI score was 10.9 ± 2.6 out of 18, with no significant change over time (p = 0.55). The mean ± SD BMJ score was 13.5 ± 7.1 out of 35, with no significant change over time (p = 0.39). A total of 49 (63%) studies stated a cost-related research question, 23 (29%) stated the type of cost evaluation used, and 31 (40%) described the method of estimating resource quantities and unit costs. A total of 16 studies compared two or more interventions and reported both cost and learning outcomes. CONCLUSIONS: The absolute number of cost evaluations in HPE is increasing. However, there are shortcomings in the quality of methodology and reporting, and these are not improving over time.

Comfort with uncertainty: reframing our conceptions of how clinicians navigate complex clinical situations.

Learning to take safe and effective action in complex settings rife with uncertainty is essential for patient safety and quality care. Doing so is not easy for trainees, as they often consider certainty to be a necessary precursor for action and subsequently struggle in these settings. Understanding how skillful clinicians work comfortably when uncertain, therefore, offers an important opportunity to facilitate trainees' clinical reasoning development. This critical review aims to define and elaborate the concept of 'comfort with uncertainty' in clinical settings by juxtaposing a variety of frameworks and theories in ways that generate more deliberate ways of thinking about, and researching, this phenomenon. We used Google Scholar to identify theoretical concepts and findings relevant to the topics of 'uncertainty,' 'ambiguity,' 'comfort,' and 'confidence,' and then used preliminary findings to pursue parallel searches within the social cognition, cognition, sociology, sociocultural, philosophy of medicine, and medical education literatures. We treat uncertainty as representing the lived experience of individuals, reflecting the lack of confidence one feels that he/she has an incomplete mental representation of a particular problem. Comfort, in contrast, references confidence in one's capabilities to act (or not act) in a safe and effective manner given the situation. Clinicians' 'comfort with uncertainty' is informed by a variety of perceptual, emotional, and situational cues, and is enabled through a combination of self-monitoring and forward planning. Potential implications of using 'comfort with uncertainty' as a framework for educational and research programs are explored.

Supporting self-regulation in simulation-based education: a randomized experiment of practice schedules and goals.

Self-regulated learning is optimized when instructional supports are provided. We evaluated three supports for self-regulated simulation-based training: practice schedules, normative comparisons, and learning goals. Participants practiced 5 endoscopy tasks on a physical simulator, then completed 4 repetitions on a virtual reality simulator. Study A compared two practice schedules: sequential (master each task in assigned order) versus unstructured (trainee-defined). Study B compared normative comparisons framed as success (10% of trainees were successful) versus failure (90% of trainees were unsuccessful). Study C compared a time-only goal (go 1 min faster) versus time + quality goal (go 1 min faster with better visualization and scope manipulation). Participants (18 surgery interns, 17 research fellows, 5 medical/college students) were randomly assigned to groups. In Study A, the sequential group had higher task completion (10/19 vs. 1/21; P < .001), longer persistence attempting an ultimately incomplete task (20.0 vs. 15.9 min; P = .03), and higher efficiency (43% vs. 27%; P = .02), but task time was similar between groups (20.0 vs. 22.6 min; P = .23). In Study B, the success orientation group had higher task completion (10/16 vs. 1/24; P < .001) and longer persistence (21.2 vs. 14.6 min; P = .001), but efficiency was similar (33% vs. 35%; P = .84). In Study C, the time-only group had greater efficiency than time + quality (56% vs. 41%; P = .03), but task time did not differ significantly (172 vs. 208 s; P = .07). In this complex motor task, a sequential (vs. unstructured) schedule, success (vs. failure) orientation, and time-only (vs. time + quality) goal improved some (but not all) performance outcomes.