Looking Across the Drape: A Novel Quality Improvement Approach to Understanding Surgeon and Anesthesiologist Burnout.

OBJECTIVE: To identify well-being threats for surgeons and anesthesiologists and develop interventions using the Quality of Life Improvement (QOLI) approach. BACKGROUND: Developing feasible perioperative well-being interventions requires identifying shared and specialty-specific well-being needs. The QOLI framework integrates human-centered design, implementation science, and quality improvement to address well-being needs. METHODS: Anesthesia and surgery faculty in eight perioperative departments at an academic medical center completed cross-sectional surveys containing validated measures of well-being and workplace satisfaction, and open-ended questions about professional motivations, pain points, strategies for improvement, and well-being priorities. Using template analysis, we analyzed open-ended survey data and presented resulting themes at a joint-specialty town hall for live-voting to identify well-being priorities. RESULTS: 104 perioperative faculty completed the survey. Across specialties, higher MHC-SF scores (representative of individual global well-being) were associated with higher satisfaction with workplace control, values, decision latitude, and social support. Anesthesiologists reported lower satisfaction and control than surgeons across multiple domains. Template analysis yielded five areas for intervention: (1) Work culture, (2) Work environment/resources, (3) Sources of fulfillment, (4) Work/life harmony, (5) Financial compensation. Surgeons and anesthesiologists both prioritized high-quality patient care but differed in their other top priorities. The most frequently cited well-being threats for surgeons were OR inefficiencies/delays and excessive workload, while anesthesiologists cited understaffing and unpredictable work hours. CONCLUSIONS: Surgeons and anesthesiologists share many needs and priorities, with pain points that are often negatively synergistic. Applying the QOLI approach across specialties allows for well-being interventions that honor complexity and promote the development of feasible solutions.

Assessing supervisor versus trainee viewpoints of entrustment through cognitive and affective lenses: an artificial intelligence investigation of bias in feedback.

The entrustment framework redirects assessment from considering only trainees' competence to decision-making about their readiness to perform clinical tasks independently. Since trainees and supervisors both contribute to entrustment decisions, we examined the cognitive and affective factors that underly their negotiation of trust, and whether trainee demographic characteristics may bias them. Using a document analysis approach, we adapted large language models (LLMs) to examine feedback dialogs (N = 24,187, each with an associated entrustment rating) between medical student trainees and their clinical supervisors. We compared how trainees and supervisors differentially documented feedback dialogs about similar tasks by identifying qualitative themes and quantitatively assessing their correlation with entrustment ratings. Supervisors' themes predominantly reflected skills related to patient presentations, while trainees' themes were broader-including clinical performance and personal qualities. To examine affect, we trained an LLM to measure feedback sentiment. On average, trainees used more negative language (5.3% lower probability of positive sentiment, p < 0.05) compared to supervisors, while documenting higher entrustment ratings (+ 0.08 on a 1-4 scale, p < 0.05). We also found biases tied to demographic characteristics: trainees' documentation reflected more positive sentiment in the case of male trainees (+ 1.3%, p < 0.05) and of trainees underrepresented in medicine (UIM) (+ 1.3%, p < 0.05). Entrustment ratings did not appear to reflect these biases, neither when documented by trainee nor supervisor. As such, bias appeared to influence the emotive language trainees used to document entrustment more than the degree of entrustment they experienced. Mitigating these biases is nonetheless important because they may affect trainees' assimilation into their roles and formation of trusting relationships.

Student, Staff and Faculty Experience with a Medical School Racial and Sociopolitical Trauma Protocol: A Mixed Methods Study.

Problem: Medical students experience racial and sociopolitical trauma that disrupts their learning and wellbeing. Intervention: University of California, San Francisco (UCSF) School of Medicine students advocated for a systems approach to responding to traumatic events. Students partnered with educators to introduce an innovative protocol that affords short-term flexibility in curricular expectations (e.g., defer attendance, assignments, assessments) to empower students to rest, gather, or pursue community advocacy work. This study explored students' protocol utilization and student, staff, and faculty experience with its implementation. Context: UCSF is a public medical school with a diverse student body. Students raised the need to acknowledge the effects of trauma on their learning and wellbeing. Consequently, students and educators created the UCSF Racial and Sociopolitical Trauma protocol ('protocol') to allow students time-limited flexibility around academic obligations following events anticipated to inflict trauma on a school community level. The protocol affords students space to process events and engage with affected communities while ensuring all students achieve school competencies and graduation requirements. Impact: We conducted a two-phase mixed methods study: (1) retrospective analysis of quantitative data on students' protocol use and (2) focus groups with students, staff, and faculty. We used descriptive statistics to summarize students' protocol use to adjust attendance, assignment submission, and assessments and thematic analysis of focus group data. Across eight protocol activations June 2020 - November 2021, 357 of 664 (54%) students used it for 501 curricular activities: 56% (n = 198) for attendance, 71% (n = 252) for assignments, and 14% (n = 51) for assessments. When deciding to utilize the protocol, student focus group participants considered sources of restoration; impact on their curricular/patient responsibilities; and their identities. The protocol symbolized an institutional value system that made students feel affirmed and staff and faculty proud. Staff and faculty initially faced implementation challenges with questions around how to apply the protocol to curricular components and how it would affect their roles; however, these questions became clearer with each protocol activation. Questions remain regarding how the protocol can be best adapted for the clerkship setting. Lessons Learned: High protocol usage and focus group data confirmed that students found value in the protocol, and staff and faculty felt invested in the protocol mission. This student-initiated intervention supports a cultural shift beyond diversity toward trauma-informed medical education. Partnership among learners and educators can contribute to transforming learning and healthcare environments by enacting systems and structures that enable all learners to thrive.

Development of obstetric anesthesia core competencies for USA residency programs through a Delphi process.

PURPOSE: The standard for anesthesia residency training in the USA mainly relies on the Accreditation Council for Graduate Medical Education (ACGME) Outcome Project, a framework that lacks specific directives for subspecialties including obstetric anesthesia. We aimed to identify core competencies in obstetric anesthesiology that can be adapted to different residency training programs to help improve the quality of training and accountability of the institutions within the USA. METHODS: We identified a preliminary list of competencies from review of existing competency-based obstetric anesthesia training curricula and practice guidelines. We used a modified Delphi methodology to achieve expert consensus among members of the Society for Obstetric Anesthesia and Perinatology education committee. The panellists were asked to evaluate the importance of each competency using a five-point Likert scale, with consensus after two rounds defined at 80% agreement. The responders were also asked at which level of training each competency should be attained. RESULTS: The Delphi rounds had 75% response rate and derived 94 competencies that were categorized under the six ACGME domains: patient care (38), medical knowledge (45), system-based practice (two), practice-based learning and improvement (five), interpersonal communication skills (two), and professionalism (two). CONCLUSION: We generated a residency training competency list for obstetric anesthesiology through expert consensus. This list can be used by residency training programs to develop a structured competency-based curriculum with tangible milestones, thereby reducing heterogeneity in the standard of training.

RéSUMé: OBJECTIF: La norme pour la formation en résidence en anesthésie aux États-Unis repose principalement sur le Projet de résultats (Outcome Project) de l'Accreditation Council for Graduate Medical Education (ACGME), un cadre qui ne dispose pas de directives spécifiques pour les surspécialités, notamment pour l'anesthésie obstétricale. Notre objectif était d'identifier les compétences de base en anesthésiologie obstétricale qui pourraient être adaptées aux différents programmes de formation en résidence afin d'améliorer la qualité de la formation et la responsabilisation des établissements aux États-Unis. MéTHODE: Nous avons dressé une liste préliminaire de compétences en passant en revue les programmes de formation axés sur les compétences et les lignes directrices de pratique existants en anesthésie obstétricale. Nous avons utilisé une méthodologie Delphi modifiée pour parvenir à un consensus d'expert·es parmi les membres du comité d'éducation de la Society for Obstetric Anesthesia and Perinatology. Les panélistes ont été invité·es à évaluer l'importance de chaque compétence à l'aide d'une échelle de Likert à cinq points, le consensus étant défini à 80 % d'accord après deux tours. On a également demandé aux répondant·es à quel niveau de formation chaque compétence devrait être atteinte. RéSULTATS: Les étapes du processus Delphi ont eu un taux de réponse de 75 % et ont permis de déterminer 94 compétences qui ont été classées dans les six domaines ACGME : soins aux patient·es (38), connaissances médicales (45), pratique systémique (deux), apprentissage et amélioration basés sur la pratique (cinq), compétences en communication interpersonnelle (deux) et professionnalisme (deux). CONCLUSION: Nous avons généré une liste de compétences pour la formation de résidence en anesthésiologie obstétricale grâce à un consensus d'expert·es. Cette liste peut être utilisée par les programmes de formation en résidence pour élaborer un programme structuré axé sur les compétences avec des jalons tangibles, réduisant ainsi l'hétérogénéité dans la norme de formation.

Investigating feedback orientation in medical learners.

BACKGROUND/PURPOSE: Feedback processes in health professions education (HPE) are not always successful. While recommendations to improve feedback provision dominate the literature, studying specific learner attributes that impact feedback uptake may also improve feedback processes. Feedback orientation is a concept from management science involving four dimensions of learner attributes and attitudes that impact their feedback uptake: utility, accountability, social awareness, and feedback self-efficacy. Feedback orientation may represent a valuable concept in HPE. We aimed to understand medical learners' feedback orientation at different stages in their development. METHODS: We used the Feedback Orientation Scale, a 20-item survey instrument, for a cross-sectional analysis of feedback orientation in medical students and Internal Medicine residents at one large academic center. We performed descriptive statistics and analysis of variance for data analysis. RESULTS: We found the same factors (dimensions) to feedback orientation in our population as in management science. Overall feedback orientation scores were high and were largely consistent across trainee levels. Utility was the domain that was highest across learners, whereas feedback self-efficacy was lowest. CONCLUSIONS: Feedback orientation represents a useful concept to explore medical learners' attitudes toward feedback's role in their development. The four domains can help guide further nuanced feedback research and application.[Box: see text].

Students' Perspectives on Basic and Clinical Science Integration When Step 1 is Administered After the Core Clerkships.

Phenomenon: According to adult learning theories, effective cognitive integration of basic and clinical science may promote the transfer of knowledge to patient care. The placement of the U.S. Medical Licensing Examination (USMLE) Step 1 after the core clerkships is one strategy intended to facilitate cognitive integration, though learner experiences with this model are unexplored. The purpose of this study is to understand students' perspectives on basic and clinical science integration in a post-clerkship Step 1 curriculum. Approach: Focus groups were conducted between August and September 2020 with senior medical students from the University of California, San Francisco School of Medicine and University of Michigan Medical School. Data were analyzed using a constructivist approach to thematic analysis. Findings: Thirty-three students participated in six focus groups. Participants described multiple barriers to cognitive integration in the clerkship learning environment, though they also identified examples of teaching and learning that facilitated integration. Early in their clerkships, students struggled to integrate because of their tenuous basic science foundation, cognitive overload, and difficulty perceiving the relevance of basic science to patient care. They felt that educators primarily focused on clinical science, and many basic science teaching sessions during clerkships felt irrelevant to patient care. However, students also described experiences that made the connection between basic and clinical science more explicit, including modeling by educators and clerkship learning activities that more overtly encouraged the application of basic science to clinical care. In addition, the return to basic science studying during the post-clerkship dedicated Step 1 study period offered powerful integration opportunities. These facilitators of cognitive integration helped students recognize the value of integration for enduring learning. Insights: There are myriad barriers to cognitive integration of basic and clinical science during clerkships in a post-clerkship Step 1 curriculum. The relevance of basic science to patient care needs to be made more explicit to students through modeling by clinician educators to augment the potential benefits of curricular change. The post-clerkship Step 1 study period appears to offer a unique opportunity for cognitive integration later in a learner's trajectory that may be related to curricular design. When learners recognize the applicability of basic science to patient care, they may more intentionally transfer basic science knowledge to clinical practice.

The fundamentals of Artificial Intelligence in medical education research: AMEE Guide No. 156.

The use of Artificial Intelligence (AI) in medical education has the potential to facilitate complicated tasks and improve efficiency. For example, AI could help automate assessment of written responses, or provide feedback on medical image interpretations with excellent reliability. While applications of AI in learning, instruction, and assessment are growing, further exploration is still required. There exist few conceptual or methodological guides for medical educators wishing to evaluate or engage in AI research. In this guide, we aim to: 1) describe practical considerations involved in reading and conducting studies in medical education using AI, 2) define basic terminology and 3) identify which medical education problems and data are ideally-suited for using AI.

The efficacy of an anatomy and ultrasonography workshop on improving residents' confidence and knowledge in regional anesthesia.

BACKGROUND: Ultrasound Guided Regional Anesthesia (UGRA) has become the standard for regional anesthesia practice, but there is not a standardized educational approach for training residents. The objective of this study was to evaluate the efficacy of an UGRA workshop utilizing the theoretical framework of embodied cognition for anesthesiology residents. METHODS: A workshop was developed consisting of didactics, scanning training on standardized patients (SPs) and anatomy reviews on prosected cadavers that focused on the most common UGRA procedures for the upper and lower extremity. At the beginning of the workshop and at the end of the workshop residents completed pre-test and pre-confidence surveys, as well as post-test and post-confidence surveys, respectively to assess the impact of the workshop. RESULTS: 39 residents (100% of the possible residents) participated in the workshop in 2019. Residents' confidence in identifying relevant anatomy for the most common UGRA procedures significantly increased in 13 of the 14 measurements. Residents' knowledge gain was also statistically significant from the pre-test to post-test (20.13 ± 3.61 and 26.13 ± 2.34; p < .0001). The residents found the course overall to be very useful (4.90 ± 0.38) and in particular the cadaveric component was highly rated (4.74 ± 0.55). CONCLUSIONS: In this study, we developed a workshop guided by the embodied cognition framework to aid in shortening the overall learning curve of UGRA for anesthesiology residents. Based on our results this workshop should be replicated by institutions that are hoping to decrease the learning curve associated with UGRA and increase residents' confidence in identifying the relevant anatomy in UGRA nerve blocks.

Development and validation of the safe transfusion assessment tool.

BACKGROUND: Given the prevalence and risks of blood transfusion, it is essential that trainees and practicing clinicians have a thorough understanding of relevant transfusion medicine competencies. The aim of this research was to develop and gather validity evidence for an instrument to assess knowledge of core transfusion-related competencies. METHODS: We developed the safe transfusion assessment tool (STAT) using a multistep process. Initially, 20 core competencies in transfusion medicine were identified through a consensus-driven Delphi process. Learning objectives and assessment items pertinent to each competency were created. Next, a 13-item assessment tool was piloted with multidisciplinary experts and trainees. Multiple iterative revisions were made based on feedback. Finally, the 12-item STAT was administered to 100 participants of varying training level and specialty to establish validity, difficulty and item discrimination indices, and perceived utility. RESULTS: Analysis of instrument item difficulty and item discrimination indices demonstrated the ability of the STAT to assess essential knowledge in transfusion medicine relevant to trainees and clinicians in multiple programs and practice settings. Eight of twelve items discriminated between learners with varying degrees of expertise. Hundred percent of students and trainees rated the STAT as Extremely Helpful or Somewhat Helpful and the majority planned to utilize the answer guide as a study aid. CONCLUSION: The STAT is a concise, valid, and reliable knowledge assessment tool that may be used by researchers and educators to augment transfusion medicine curricula (www.safetransfusion.ucsf.edu). Scores can help inform departments on areas in which trainees require additional support and areas of potential educational interventions.

Exploring how feedback reflects entrustment decisions using artificial intelligence.

CONTEXT: Clinical supervisors make judgements about how much to trust learners with critical activities in patient care. Such decisions mediate trainees' opportunities for learning and competency development and thus are a critical component of education. As educators apply entrustment frameworks to assessment, it is important to determine how narrative feedback reflecting entrustment may also address learners' educational needs. METHODS: In this study, we used artificial intelligence (AI) and natural language processing (NLP) to identify characteristics of feedback tied to supervisors' entrustment decisions during direct observation encounters of clerkship medical students (3328 unique observations). Supervisors conducted observations of students and collaborated with them to complete an entrustment-based assessment in which they documented narrative feedback and assigned an entrustment rating. We trained a deep neural network (DNN) to predict entrustment levels from the narrative data and developed an explainable AI protocol to uncover the latent thematic features the DNN used to make its prediction. RESULTS: We found that entrustment levels were associated with level of detail (specific steps for performing clinical tasks), feedback type (constructive versus reinforcing) and task type (procedural versus cognitive). In justifying both high and low levels of entrustment, supervisors detailed concrete steps that trainees performed (or did not yet perform) competently. CONCLUSIONS: Framing our results in the factors previously identified as influencing entrustment, we find a focus on performance details related to trainees' clinical competency as opposed to nonspecific feedback on trainee qualities. The entrustment framework reflected in feedback appeared to guide specific goal-setting, combined with details necessary to reach those goals. Our NLP methodology can also serve as a starting point for future work on entrustment and feedback as similar assessment datasets accumulate.

Profiles for Success: Examining the Relationship between Student Profiles and Clerkship Performance Using Latent Profile Analysis.

PHENOMENON: Learning is a complex phenomenon in which learners can vary in their learning orientation, learning approaches, and perception of the learning environment. Learners may be characterized as mastery oriented or performance oriented, and this learning orientation can influence their learning approaches, such as whether to seek feedback or ask for help. The learning environment includes institutional assessment policies and informal interactions that emphasize outcomes and differentiation among students. When learners perceive the learning environment as performance oriented, they feel competition from other learners and fear negative feedback. This complex interplay of learning orientation, its influence on feedback-seeking behavior, and the environmental influence and reinforcement of both can be captured through learner profiles. APPROACH: In this multi-institutional cross-sectional survey study of students from six U.S. medical schools, we investigated learner profiles that characterize how these domains manifest together across individual learners. We then determined how these profiles are associated with students' clerkship grades. Measures included the Patterns of Adaptive Learning for learning orientation, self-reports for learning approaches, and perceptions of the learning environment. We used latent profile analysis to cluster students who share common characteristics around orientation, behavior, and environment. The relationship between these profiles and the percentage of honors earned was examined using multiple regression analysis. FINDINGS: We found four distinct learner profiles within 666 of 974 (68%) analyzable responses: (1) performance oriented with negative perceptions of environment, (2) mastery oriented with desirable learning approaches, (3) "average" group, and (4) mastery orientation only. Profile 1 (M = 39%) and Profile 4 (M = 38%) students received fewer clerkship honors compared with other profiles. Profile 2 students earned a significantly higher percentage of honors grades (M = 54%) compared with other students. Profile 3 students (average group) earned the second highest percentage of honors (M = 46%). Effect sizes comparing the percentage of honors varied from 0.32 to 0.65, indicating medium to large differences. INSIGHTS: The highest performers, as measured by clerkship honors grades, were mastery oriented, perceived a more positive learning environment, and were comfortable asking questions and seeking feedback for their learning. The lowest performing students were performance oriented and had negative perceptions of their learning environments. Contrary to previous findings, we found that the relationship between mastery orientation and learning approaches is not unidirectional but rather seems to be mediated by perceptions of the learning environment.

Emotion in remediation: A scoping review of the medical education literature.

OBJECTIVES: Remediation can be crucial and high stakes for medical learners, and experts agree it is often not optimally conducted. Research from other fields indicates that explicit incorporation of emotion improves education because of emotion's documented impacts on learning. Because this could present an important opportunity for improving remediation, we aimed to investigate how the literature on remediation interventions in medical education discusses emotion. METHODS: The authors used Arksey and O'Malley's framework to conduct a scoping literature review of records describing remediation interventions in medical education, using PubMed, CINAHL Complete, ERIC, Web of Science and APA PsycInfo databases, including all English-language publications through 1 May 2020 meeting search criteria. They included publications discussing remediation interventions either empirically or theoretically, pertaining to physicians or physician trainees of any level. Two independent reviewers used a standardised data extraction form to report descriptive information; they reviewed included records for the presence of mentions of emotion, described the mentions and analysed results thematically. RESULTS: Of 1644 records, 199 met inclusion criteria and were reviewed in full. Of those, 112 (56%) mentioned emotion in some way; others focused solely on cognitive aspects of remediation. The mentions of emotion fell into three themes based on when the emotion was cited as present: during regular coursework or practice, upon referral for remediation and during remediation. One-quarter of records (50) indicated potential intentional incorporation of emotion into remediation programme design, but they were non-specific as to how emotions related to the learning process itself. CONCLUSION: Even though emotion is omnipresent in remediation, medical educators frequently do not factor emotion into the design of remediation approaches and rarely explicitly utilise emotion to improve the learning process. Applications from other fields may help medical educators leverage emotion to improve learning in remediation, including strategies to frame and design remediation.

Assessment to Optimize Learning Strategies: A Qualitative Study of Student and Faculty Perceptions.

Phenomenon: The format of medical knowledge assessment can promote students' use of effective learning strategies from the learning sciences literature, such as elaboration, interleaving, retrieval practice, and distributed learning. Assessment format can also influence faculty teaching. Accordingly, our institution implemented a new assessment strategy in which pre-clerkship medical students answered weekly formative quizzes with constructed response questions (also referred to as open-ended questions) and multiple-choice questions in preparation for summative open-ended question examinations, to support students' use of recommended learning strategies. Our qualitative study explored medical student and faculty perceptions of this assessment strategy on learning and teaching. Approach: We conducted semi-structured interviews with 16 second-year medical students to explore their preparation for quizzes and summative examinations. We also interviewed 10 faculty responsible for writing and grading these assessments in the pre-clerkship foundational sciences curriculum regarding their approach to writing assessments and rubrics, and their perceptions of how their teaching may have changed with this assessment strategy. We analyzed interview transcripts using thematic analysis with a priori sensitizing concepts from the learning sciences literature. Findings: We identified four major themes characterizing student and faculty perceptions of weekly formative quizzes and summative OEQ examinations. Participants found that this assessment strategy helped (1) prioritize conceptual understanding, (2) simulate clinical problem solving, and (3) engage students and faculty in continuous improvement in their approach to learning or teaching. Faculty and students also recognized challenges and potential tradeoffs associated with these assessment formats. Participants identified (4) facilitators and barriers when implementing this assessment strategy. Insights: Our findings suggested that assessment of medical knowledge through weekly formative quizzes and summative open-ended question examinations can facilitate students' use of effective learning strategies. Faculty also recognized improvements in their teaching and in quality of assessment. This format of assessment also presented some challenges and potential tradeoffs and significant institutional resources were required for implementation.

The role of data science and machine learning in Health Professions Education: practical applications, theoretical contributions, and epistemic beliefs.

Data science is an inter-disciplinary field that uses computer-based algorithms and methods to gain insights from large and often complex datasets. Data science, which includes Artificial Intelligence techniques such as Machine Learning (ML), has been credited with the promise to transform Health Professions Education (HPE) by offering approaches to handle big (and often messy) data. To examine this promise, we conducted a critical review to explore: (1) published applications of data science and ML in HPE literature and (2) the potential role of data science and ML in shifting theoretical and epistemological perspectives in HPE research and practice. Existing data science studies in HPE are often not informed by theory, but rather oriented towards developing applications for specific problems, uses, and contexts. The most common areas currently being studied are procedural (e.g., computer-based tutoring or adaptive systems and assessment of technical skills). We found that epistemic beliefs informing the use of data science and ML in HPE poses a challenge for existing views on what constitutes objective knowledge and the role of human subjectivity for instruction and assessment. As a result, criticisms have emerged that the integration of data science in the field of HPE is in danger of becoming technically driven and narrowly focused in its approach to teaching, learning and assessment. Our findings suggest that researchers tend to formalize around the epistemological stance driven largely by traditions of a research paradigm. Future data science studies in HPE need to involve both education scientists and data scientists to ensure mutual advancements in the development of educational theory and practical applications. This may be one of the most important tasks in the integration of data science and ML in HPE research in the years to come.

A view from the sender side of feedback: anticipated receptivity to clinical feedback when changing prior physicians' clinical decisions-a mixed methods study.

When physicians transition patients, the physician taking over may change the diagnosis. Such a change could serve as an important source of clinical feedback to the prior physician. However, this feedback may not transpire if the current physician doubts the prior physician's receptivity to the information. This study explored facilitators of and barriers to feedback communication in the context of patient care transitions using an exploratory sequential, qualitative to quantitative, mixed methods design. Twenty-two internal medicine residents and hospitalist physicians from two teaching hospitals were interviewed and data were analyzed thematically. A prominent theme was participants' reluctance to communicate diagnostic changes. Participants perceived case complexity and physical proximity to facilitate, and hierarchy, unfamiliarity with the prior physician, and lack of relationship to inhibit communication. In the subsequent quantitative portion of the study, forty-one hospitalists completed surveys resulting in 923 total survey responses. Multivariable analyses and a mixed-effects model were applied to survey data with anticipated receptivity as the outcome variable. In the mixed-effects model, four factors had significant positive associations with receivers' perceived receptivity: (1) feedback senders' time spent on teaching services (ß = 0.52, p = 0.02), (2) receivers' trustworthiness and clinical credibility (ß = 0.49, p < 0.001), (3) preference of both for shared work rooms (ß = 0.15, p = 0.006), and (4) receivers being peers (ß = 0.24, p < 0.001) or junior colleagues (ß = 0.39, p < 0.001). This study suggests that anticipated receptivity to feedback about changed clinical decisions affects clinical communication loops. Without trusting relationships and opportunities for low risk, casual conversations, hospitalists may avoid such conversations.

Twelve tips for assessing medical knowledge with open-ended questions: Designing constructed response examinations in medical education.

Medical knowledge examinations employing open-ended (constructed response) items can be useful to assess medical students' factual and conceptual understanding. Modern day curricula that emphasize active learning in small groups and other interactive formats lend themselves to an assessment format that prompts students to share conceptual understanding, explain, and elaborate. The open-ended question examination format can provide faculty with insights into learners' abilities to apply information to clinical or scientific problems, and reveal learners' misunderstandings about essential content. To implement formative or summative assessments with open-ended questions in a rigorous manner, educators must design systems for exam creation and scoring. This includes systems for constructing exam blueprints, items and scoring rubrics, and procedures for scoring and standard setting. Information gained through review of students' responses can guide future educational sessions and curricular changes in a cycle of continuous improvement.

Trainee perception of cognitive load during observed faculty staff teaching of procedural skills.

OBJECTIVES: Although teachers impact learners' cognitive load, how specific teaching activities affect intrinsic, germane and extraneous load during procedural skills training is unknown. We sought to characterise teaching activities used in the exemplar procedural setting of colonoscopy, and to explore how they were enacted and how learners perceived them as affecting intrinsic, germane and extraneous cognitive load. METHODS: We observed 10 colonoscopies performed by eight different gastroenterology fellows and supervised by 10 different attending physicians at two hospitals, and recorded the teaching activities observed, as well as details of when they were used and how they were enacted. After the colonoscopy, each fellow completed the Cognitive Load Inventory for Colonoscopy to quantify intrinsic, germane and extraneous load. We then interviewed each fellow to determine how he or she perceived teaching as affecting cognitive load. Qualitative data were subjected to content analysis. Instances of germane load-promoting activities were correlated with measured germane load. RESULTS: We observed 515 instances of teaching activities. The intensity of teaching varied substantially, ranging from 0.7 to 3.3 activities per minute, as did the pattern of teaching activities used by different attending physicians. Little teaching occurred immediately before or after a procedure. Fellows usually perceived teaching as affecting cognitive load in ways that promoted learning, particularly by reducing intrinsic load and increasing germane load. Fellows strongly perceived that the provision of autonomy promoted germane load. Conversely, fellows perceived that excessive teaching increased extraneous load. Instances of germane load-promoting teaching activities correlated moderately with measured germane load. CONCLUSIONS: Teaching in the exemplar procedural setting of colonoscopy affected learners' cognitive load in mostly beneficial ways, yet even 'good' teaching activities had detrimental effects when used excessively. Teachers of procedures should consider learner experience, task complexity and environmental factors to modulate the modality, content and intensity of teaching to promote balanced cognitive load and learning. Teaching more reservedly during the procedure and taking advantage of pre- and post-procedure opportunities may help.

Twelve tips to promote successful development of a learner performance dashboard within a medical education program.

Easily accessible and interpretable performance data constitute critical feedback for learners that facilitate informed self-assessment and learning planning. To provide this feedback, there has been a proliferation of educational dashboards in recent years. An educational (learner) dashboard systematically delivers timely and continuous feedback on performance and can provide easily visualized and interpreted performance data. In this paper, we provide practical tips for developing a functional, user-friendly individual learner performance dashboard and literature review of dashboard development, assessment theory, and users' perspectives. Considering key design principles and maximizing current technological advances in data visualization techniques can increase dashboard utility and enhance the user experience. By bridging current technology with assessment strategies that support learning, educators can continue to improve the field of learning analytics and design of information management tools such as dashboards in support of improved learning outcomes.

Motivation for feedback-seeking among pediatric residents: a mixed methods study.

BACKGROUND: For effective self-directed life-long learning physicians need to engage in feedback-seeking, which means fostering such behavior during training. Self-determination theory (SDT) posits that intrinsic motivation is fostered when the environment optimizes the individual's experience of autonomy, relatedness, and competence. Educational settings meeting these psychological needs may foster intrinsic motivation in trainees, enhance their desire for feedback, and promote feedback-seeking. We sought to examine residents' feedback-seeking behaviors through the lens of SDT and explore the association with intrinsic motivation and career choice. METHODS: We used a mixed-methods approach with an explanatory sequential design. Residents participated in simulation training, completed an inventory of intrinsic motivation (IMI) and responded to sequential opportunities for performance feedback requiring different levels of effort. We compared IMI scores and career choice between groups with different effort. We interviewed high-effort feedback-seekers and conducted thematic analysis of interview data. RESULTS: Thirty-four of 35 residents completed the survey (97%). Of those completing the study, 12 engaged in low-effort feedback-seeking only, 10 indicated intent for high-effort feedback-seeking and 10 actually engaged in higher effort to get feedback. Groups did not differ in mean IMI scores. Among high-effort feedback-seekers more residents were interested in critical care-related fields compared to the other groups. We identified 5 themes around autonomy, relatedness, and competence clarifying residents' reasons for feedback-seeking. CONCLUSIONS: Our findings suggest that among residents, the relationship between motivation and feedback-seeking is complex and cannot be predicted by IMI score. Career plans and relationships with feedback providers impact feedback-seeking, which can inform educational interventions.

Can You See Me Now? Video Supplementation for Pediatric Teledermatology Cases.

BACKGROUND/OBJECTIVES: Digital video is widely available and is used sporadically in clinical settings to evaluate patients, but whether it helps improve clinical management has not been determined. The aim of this study was to assess whether recorded video in addition to still images can improve residents' diagnostic and management accuracy and confidence with pediatric teledermatology cases. METHODS: Dermatology residents from three programs were assigned alternately to an online survey with 15 pediatric teledermatology cases presented with still images only (still) or still images plus recorded video (mixed). Participants provided free-text diagnoses and management recommendations and rated their confidence and image quality. Responses were scored using a modified script concordance grading key based on reference panelists' responses. RESULTS: Thirty-one residents participated (response rate 57%). Participants in the mixed group scored significantly higher on management accuracy (87.6 ± 12.9 vs 71.7 ± 14.2; p = 0.003). Both groups performed better on more common conditions than less common conditions. The mixed group outperformed the still group on less common conditions with respect to management recommendations. CONCLUSION: This novel study suggests that supplemental recorded video may improve the management accuracy of pediatric teledermatology consultations, particularly for complex cases. Residents may benefit from training in recording and interpreting video.