Factors and Interactions Influencing Direct Observation: A Literature Review Guided by Activity Theory.

PhenomenonEnsuring that future physicians are competent to practice medicine is necessary for high quality patient care and safety. The shift toward competency-based education has placed renewed emphasis on direct observation via workplace-based assessments in authentic patient care contexts. Despite this interest and multiple studies focused on improving direct observation, challenges regarding the objectivity of this assessment approach remain underexplored and unresolved. Approach: We conducted a literature review of direct observation in authentic patient contexts by systematically searching databases PubMed, Embase, Web of Science, and ERIC. Included studies comprised original research conducted in the patient care context with authentic patients, either as a live encounter or a video recording of an actual encounter, which focused on factors affecting the direct observation of undergraduate medical education (UME) or graduate medical education (GME) trainees. Because the patient care context adds factors that contribute to the cognitive load of the learner and of the clinician-observer we focused our question on such contexts, which are most useful in judgments about advancement to the next level of training or practice. We excluded articles or published abstracts not conducted in the patient care context (e.g., OSCEs) or those involving simulation, allied health professionals, or non-UME/GME trainees. We also excluded studies focused on end-of-rotation evaluations and in-training evaluation reports. We extracted key data from the studies and used Activity Theory as a lens to identify factors affecting these observations and the interactions between them. Activity Theory provides a framework to understand and analyze complex human activities, the systems in which people work, and the interactions or tensions between multiple associated factors. Findings: Nineteen articles were included in the analysis; 13 involved GME learners and 6 UME learners. Of the 19, six studies were set in the operating room and four in the Emergency department. Using Activity Theory, we discovered that while numerous studies focus on rater and tool influences, very few study the impact of social elements. These are the rules that govern how the activity happens, the environment and members of the community involved in the activity and how completion of the activity is divided up among the members of the community. Insights: Viewing direct observation via workplace-based assessment through the lens of Activity Theory may enable educators to implement curricular changes to improve direct observation of assessment. Activity Theory may allow researchers to design studies to focus on the identified underexplored interactions and influences in relation to direct observation.

Gathering Validity Evidence on an Internal Medicine Clerkship Multistep Exam to Assess Medical Student Analytic Ability.

Construct: The definition of clinical reasoning may vary among health profession educators. However, for the purpose of this paper, clinical reasoning is defined as the cognitive processes that are involved in the steps of information gathering, problem representation, generating a differential diagnosis, providing a diagnostic justification to arrive at a leading diagnosis, and formulating diagnostic and management plans. Background: Expert performance in clinical reasoning is essential for success as a physician, and has been difficult for clerkship directors to observe and quantify in a way that fosters the instruction and assessment of clinical reasoning. The purpose of this study was to gather validity evidence for the Multistep exam (MSX) format used by our medicine clerkship to assess analytical clinical reasoning abilities; we did this by examining the relationship between scores on the MSX and other external measures of clinical reasoning abilities. This analysis used dual process theory as the main theoretical framework of clinical reasoning, as well as aspects of Kane's validity framework to guide the selection of validity evidence for the investigation. We hypothesized that there would be an association between the MSX (a three-step clinical reasoning tool developed locally), and the USMLE Step 2 CS, as they share similar concepts in assessing the clinical reasoning of students. We examined the relationship between overall scores on the MSX and the Step 2 CS Integrated Clinical Encounter (ICE) score, in which the student articulates their reasoning for simulated patient cases, while controlling for examinee's internal medicine clerkship performance measures such as the NBME subject exam score and the Medicine clerkship OSCE score. Approach: A total 477 of 487 (97.9%) medical students, representing the graduating classes of 2015, 2016, 2017, who took the MSX at the end of each medicine clerkship (2012-2016), and Step 2 CS (2013-2017) were included in this study. Correlation analysis and multiple linear regression analysis were used to examine the impact of the primary explanatory variables of interest (MSX) onto the outcome variable (ICE score) when controlling for baseline variables (Medicine OSCE and NBME Medicine subject exam). Findings: The overall MSX score had a significant, positive correlation with the Step 2 CS ICE score (r = .26, P < .01). The overall MSX score was a significant predictor of Step 2 CS ICE score (ß = .19, P < .001), explaining an additional 4% of the variance of ICE beyond the NBME Medicine subject score and the Medicine OSCE score (Adjusted R2 = 13%). Conclusion: The stepwise format of the MSX provides a tool to observe clinical reasoning performance, which can be used in an assessment system to provide feedback to students on their analytical clinical reasoning. Future studies should focus on gaining additional validity evidence across different learners and multiple medical schools.

Expectations of and for Clerkship Directors 2.0: A Collaborative Statement from the Alliance for Clinical Education.

This article presents an update of the collaborative statement on clerkship directors (CDs), first published in 2003, from the national undergraduate medical education organizations that comprise the Alliance for Clinical Education (ACE). The clerkship director remains an essential leader in the education of medical students on core clinical rotations, and the role of the CD has and continues to evolve. The selection of a CD should be an explicit contract between the CD, their department, and the medical school, with each party fulfilling their obligations to ensure the success of the students, the clerkship and of the CD. Educational innovations and accreditation requirements have evolved in the last two decades and therefore this article updates the 2003 standards for what is expected of a CD and provides guidelines for the resources and support to be provided.In their roles as CDs, medical student educators engage in several critical activities: administration, education/teaching, coaching, advising, and mentoring, faculty development, compliance with accreditation standards, and scholarly activity. This article describes (a) the work products that are the primary responsibility of the CD; (b) the qualifications for the CD; (c) the support structure, resources, and personnel that are necessary for the CD to accomplish their responsibilities; (d) incentives and career development for the CD; and (e) the dedicated time that should be provided for the clerkship and the CD to succeed. Given all that should rightfully be expected of a CD, a minimum of 50% of a full-time equivalent is recognized as appropriate. The complexity and needs of the clerkship now require that at least one full-time clerkship administrator (CA) be a part of the CD's team.To better reflect the current circumstances, ACE has updated its recommendations for institutions and departments to have clear standards for what is expected of the director of a clinical clerkship and have correspondingly clear guidelines as to what should be expected for CDs in the support they are provided. This work has been endorsed by each of the eight ACE member organizations.

Exploring the Predictors of Post-Clerkship USMLE Step 1 Scores.

Theory: We used two theoretical frameworks for this study: a) experiential learning, whereby learners construct new knowledge based on prior experience, and learning grows out of a continuous process of reconstructing experience, and b) deliberate practice, whereby the use of testing (test-enhanced learning) promotes learning and produces better long-term retention. Hypothesis: We hypothesized that moving the USMLE Step 1 exam to follow the clerkship year would provide students with a context for basic science learning that may enhance exam performance. We also hypothesized that examination performance variables, specifically National Board of Medical Examiners (NBME) Customized Basic Science Examinations and NBME subject examinations in clinical disciplines would account for a moderate to large amount of the variance in Step 1 scores. Thus we examined predictors of USMLE Step 1 scores when taken after the core clerkship year. Method: In 2011, we revised our medical school curriculum and moved the timing of Step 1 to follow the clerkship year. We performed descriptive statistics, an ANCOVA to compare Step 1 mean scores for three graduating classes of medical students before and after the curriculum changes, and stepwise linear regression to investigate the association between independent variables and the primary outcome measure after curriculum changes. Results: 993 students took the Step 1 exam, which included graduating classes before (2012-2014, N = 491) and after (2015-2017, N = 502) the curriculum change. Step 1 scores increased significantly following curricular revision (mean 218, SD 18.2, vs. 228, SD 16.7, p < 0.01) after controlling for MCAT and undergraduate GPA. Overall, 66.4% of the variance in Step 1 scores after the clerkship year was explained by: the mean score on fourteen pre-clerkship customized NBME exams (p < 0.01, 57.0% R2); performance on the surgery NBME subject exam (p < 0.01, 3.0% R2); the pediatrics NBME subject exam (p < 0.01, 2.0% R2); the Comprehensive Basic Science Self-Assessment (p < .01, 2.0% R2) ; the internal medicine NBME subject exam (p < 0.01, 0.03% R2), pre-clerkship Integrated Clinical Skills score (p < 0.01, 0.05% R2), and the pre-matriculation MCAT (p < 0.01, 0.01% R2). Conclusion: In our institution, nearly two-thirds of the variance in performance on Step 1 taken after the clerkship year was explained mainly by pre-clerkship variables, with a smaller contribution emanating from clerkship measures. Further study is needed to uncover the specific aspects of the clerkship experience that might contribute to success on high stakes licensing exam performance.

Post-Carnegie II curricular reform: a north American survey of emerging trends & challenges.

BACKGROUND: In 2010, coincident with the 100th anniversary of Flexner's sentinel report, the Carnegie Foundation published an updated review of North American medical education and challenged medical schools to initiate further educational reforms. Specific recommendations pertained to a) ensuring standardized outcomes while allowing for individualized processes, b) integrating foundational knowledge with clinical experience, c) cultivating habits of inquiry and innovation and d) professional identity formation. As we approach the 10-year anniversary of this latest report, we sought to determine what type of curricular revisions have been emerging within the past decade and what types of challenges have been encountered along the way? METHODS: In 2018, an electronic survey was sent to all 166 Liaison Committee on Medical Education (LCME) accredited North American Medical Schools, using the points of contact (educational deans) that were listed in a publicly available, Association of American Medical Colleges database. Free text comments were grouped into themes using the constant-comparative technique. RESULTS: Sixty unique responses yielding a 36.14% response rate. The distribution of responses was proportionally representative of the distribution of public vs. private, old vs. new vs. established North American medical schools. Self-reported curricular changes aggregated into five main themes: Changes in curricular structure/organization, changes in curricular content, changes in curricular delivery, changes in assessment, and changes involving increased use of technology/informatics. Challenges were predominantly focused on overcoming faculty resistance, faculty development, securing adequate resourcing, change management, and competition for limited amounts of curricular time. CONCLUSIONS: Changes in curricular organization, content, delivery, assessment and the use of technology reflect reforms that are broad and deep. Empowering faculty to "let go" of familiar constructs/processes requires strong leadership, particularly when initiating particularly disruptive curricular changes, such as relocating the Step 1 examination or shifting to a competency-based curriculum. While North American medical schools are responding to the calls for action described in the second (2010) Carnegie Foundation report, the full vision has yet to be achieved.

Regular Formal Evaluation Sessions are Effective as Frame-of-Reference Training for Faculty Evaluators of Clerkship Medical Students.

BACKGROUND: Face-to-face formal evaluation sessions between clerkship directors and faculty can facilitate the collection of trainee performance data and provide frame-of-reference training for faculty. OBJECTIVE: We hypothesized that ambulatory faculty who attended evaluation sessions at least once in an academic year (attendees) would use the Reporter-Interpreter-Manager/Educator (RIME) terminology more appropriately than faculty who did not attend evaluation sessions (non-attendees). DESIGN: Investigators conducted a retrospective cohort study using the narrative assessments of ambulatory internal medicine clerkship students during the 2008-2009 academic year. PARTICIPANTS: The study included assessments of 49 clerkship medical students, which comprised 293 individual teacher narratives. MAIN MEASURES: Single-teacher written and transcribed verbal comments about student performance were masked and reviewed by a panel of experts who, by consensus, (1) determined whether RIME was used, (2) counted the number of RIME utterances, and (3) assigned a grade based on the comments. Analysis included descriptive statistics and Pearson correlation coefficients. KEY RESULTS: The authors reviewed 293 individual teacher narratives regarding the performance of 49 students. Attendees explicitly used RIME more frequently than non-attendees (69.8 vs. 40.4 %; p < 0.0001). Grades recommended by attendees correlated more strongly with grades assigned by experts than grades recommended by non-attendees (r = 0.72; 95 % CI (0.65, 0.78) vs. 0.47; 95 % CI (0.26, 0.64); p = 0.005). Grade recommendations from individual attendees and non-attendees each correlated significantly with overall student clerkship clinical performance [r = 0.63; 95 % CI (0.54, 0.71) vs. 0.52 (0.36, 0.66), respectively], although the difference between the groups was not statistically significant (p = 0.21). CONCLUSIONS: On an ambulatory clerkship, teachers who attended evaluation sessions used RIME terminology more frequently and provided more accurate grade recommendations than teachers who did not attend. Formal evaluation sessions may provide frame-of-reference training for the RIME framework, a method that improves the validity and reliability of workplace assessment.

Physician Leadership and Burnout: The Need for Agency; a Qualitative Study of an Academic Institution.

Purpose: Organizational approaches to physician burnout are limited. Training physician leaders to influence the organizational environment is a growing area of study. This study explored perceived physician leadership behaviors in response to burnout from the viewpoint of faculty physicians not in formal leadership positions. Understanding physician leadership behaviors from the viewpoint of those faculty being led can inform organizational strategy and leadership training to address physician burnout. Subjects and Methods: Interview requests were sent to 70 randomly identified faculty physicians from a roster containing all 1145 physician faculty that excluded the Pediatric Department, at an academic health care institution in Southern California. The first ten respondents were asked to participate in a 30-to-40-minute semi-structured virtual interview via Zoom. The interviewees were asked two questions pertaining to burnout and their perception of how leadership responded. The two questions were "What has leadership done to address burnout?" and "If you had five minutes to advise your leaders on burnout, what would you say?" The recorded interviews were transcribed, redacted, and then sent to two reviewers. Thematic analysis through iterative coding was completed, and categories were constructed that aligned with the two interview questions. Results: Overall, five themes were identified. These themes were organized according to the interview questions and broadly categorized as physician leadership behaviors observed that corresponded to the interview question of what leadership had done to address burnout and physician leadership behaviors desired corresponding to the second interview question of what advice should be given. Leadership behaviors observed in the context of burnout included three themes; referral to individual wellness programs, increased number of meetings and events, and a lack of agency in addressing wellness issues. The two themes of leadership behaviors desired were the obtainment of more resources and the granting of greater appreciation and recognition for work done through enhanced communication. Conclusion: This small study of faculty physician perceptions of leadership behaviors identified several themes that had been identified in previous studies of leadership and burnout; need for relationship building through communication, need for resources to address work issues, and referral to wellness programs. However, the identification of a lack of agency in addressing factors in the wellness environment has not been identified in the previous burnout and physician leadership literature. Further study into the causes of this perceived lack of agency should be explored. Understanding the root causes of physician leaders' lack of agency can further inform physician leadership education as an organizational approach to burnout.

Assessment Tools for Patient Notes in Medical Education: A Scoping Review.

PURPOSE: Physician proficiency in clinical encounter documentation is a universal expectation of medical education. However, deficiencies in note writing are frequently identified, which have implications for patient safety, health care quality, and cost. This study aimed to create a compendium of tools for educators' practical implementation or future research. METHOD: A scoping review was conducted using the Arksey and O'Malley framework. PubMed, Embase, Ovid All EBM Reviews, Web of Science, and MedEdPortal were searched for articles published from database inception to November 16, 2023, using the following search terms: documentation, note-writing, patient note, electronic health record note, entrustable professional activity 5, and other terms. For each note-writing assessment tool, information on setting, section(s) of note that was assessed, tool properties, numbers and roles of note writers and graders, weight given, if used in grading, learner performance, and stakeholder satisfaction and feasibility was extracted and summarized. RESULTS: A total of 5,257 articles were identified; 32 studies were included. Eleven studies (34.4%) were published since 2018. Twenty-two studies (68.8%) outlined creating an original assessment tool, whereas 10 (31.2%) assessed a curriculum intervention using a tool. Tools varied in length and complexity. None provided data on equity or fairness to student or resident note writers or about readability for patients. Note writers often had missing or incomplete documentation (mean [SD] total tool score of 60.3% [19.4%] averaged over 25 studies), often improving after intervention. Selected patient note assessment tool studies have been cited a mean (SD) of 6.3 (9.2) times. Approximately half of the tools (53.1%) or their accompanying articles were open access. CONCLUSIONS: Diverse tools have been published to assess patient notes, often identifying deficiencies. This compendium may assist educators and researchers in improving patient care documentation.

Frameworks for learner assessment in medicine: AMEE Guide No. 78.

In any evaluation system of medical trainees there is an underlying set of assumptions about what is to be evaluated (i.e., which goals reflect the values of the system or institution), what kind of observations or assessments are useful to allow judgments 1 ; and how these are to be analyzed and compared to a standard of what is to be achieved by the learner. These assumptions can be conventionalized into a framework for evaluation. Frameworks encompass, or "frame," a group of ideas or categories to reflect the educational goals against which a trainee's level of competence or progress is gauged. Different frameworks provide different ways of looking at the practice of medicine and have different purposes. In the first place, frameworks should enable educators to determine to what extent trainees are ready for advancement, that is, whether the desired competence has been attained. They should provide both a valid mental model of competence and also terms to describe successful performance, either at the end of training or as milestones during the curriculum. Consequently, such frameworks drive learning by providing learners with a guide for what is expected. Frameworks should also enhance consistency and reliability of ratings across staff and settings. Finally, they determine the content of, and resources needed for, rater training to achieve consistency of use. This is especially important in clinical rotations, in which reliable assessments have been most difficult to achieve. Because the limitations of workplace-based assessment have persisted despite the use of traditional frameworks (such as those based on knowledge, skills, and attitudes), this Guide will explore the assumptions and characteristics of traditional and newer frameworks. In this AMEE Guide, we make a distinction between analytic, synthetic, and developmental frameworks. Analytic frameworks deconstruct competence into individual pieces, to evaluate each separately. Synthetic frameworks attempt to view competence holistically, focusing evaluation on the performance in real-world activities. Developmental frameworks focus on stages of, or milestones, in the progression toward competence. Most frameworks have one predominant perspective; some have a hybrid nature.

Student-Led Medical Education Elective Initiatives.

The four-week Medical Education elective at the Uniformed Services University (USU) provides an opportunity to learn and apply theories of medical education. We will describe the structure of the elective and a final, formative exercise implemented and endorsed by all participants as one of the most impactful activities.

Quality of outpatient clinical notes: a stakeholder definition derived through qualitative research.

BACKGROUND: There are no empirically-grounded criteria or tools to define or benchmark the quality of outpatient clinical documentation. Outpatient clinical notes document care, communicate treatment plans and support patient safety, medical education, medico-legal investigations and reimbursement. Accurately describing and assessing quality of clinical documentation is a necessary improvement in an increasingly team-based healthcare delivery system. In this paper we describe the quality of outpatient clinical notes from the perspective of multiple stakeholders. METHODS: Using purposeful sampling for maximum diversity, we conducted focus groups and individual interviews with clinicians, nursing and ancillary staff, patients, and healthcare administrators at six federal health care facilities between 2009 and 2011. All sessions were audio-recorded, transcribed and qualitatively analyzed using open, axial and selective coding. RESULTS: The 163 participants included 61 clinicians, 52 nurse/ancillary staff, 31 patients and 19 administrative staff. Three organizing themes emerged: 1) characteristics of quality in clinical notes, 2) desired elements within the clinical notes and 3) system supports to improve the quality of clinical notes. We identified 11 codes to describe characteristics of clinical notes, 20 codes to describe desired elements in quality clinical notes and 11 codes to describe clinical system elements that support quality when writing clinical notes. While there was substantial overlap between the aspects of quality described by the four stakeholder groups, only clinicians and administrators identified ease of translation into billing codes as an important characteristic of a quality note. Only patients rated prioritization of their medical problems as an aspect of quality. Nurses included care and education delivered to the patient, information added by the patient, interdisciplinary information, and infection alerts as important content. CONCLUSIONS: Perspectives of these four stakeholder groups provide a comprehensive description of quality in outpatient clinical documentation. The resulting description of characteristics and content necessary for quality notes provides a research-based foundation for assessing the quality of clinical documentation in outpatient health care settings.

Opportunities and challenges in integrating electronic health records into undergraduate medical education: a national survey of clerkship directors.

BACKGROUND: Few studies have reported on the utilization and the effect of electronic health records on the education of medical students. PURPOSE: The purpose of this study was to describe the current use of electronic health records by medical students in the United States and explore the opportunities and challenges of integrating electronic health records into daily teaching of medical students. METHODS: A survey with 24 questions regarding the use of electronic health records by medical students was developed by the Alliance for Clinical Educators and sent to clerkship directors across the United States. Both quantitative and qualitative responses were collected and analyzed to determine current access to and use of electronic health records by medical students. RESULTS: This study found that an estimated 64% of programs currently allow student use of electronic health records, of which only two thirds allowed students to write notes within the electronic record. Overall, clerkship directors' opinions on the effects of electronic health records on medical student education were neutral, and despite acknowledging many advantages to electronic health records, there were many concerns raised regarding their use in education. CONCLUSIONS: Medical students are using electronic health records at higher rates than physicians in practice. Although this is overall reassuring, educators have to be cautious about the limitations being placed on student's documentation in electronic health records as this can potentially have consequences on their training, and they need to explore ways to maximize the benefits of electronic health records in medical education.

Does the authenticity of preclinical teaching format affect subsequent clinical clerkship outcomes? A prospective randomized crossover trial.

BACKGROUND: Educational theories predict conflicting results for the effect of increasing the authenticity of the teaching format of complex information on educational outcomes. We sought to determine the effect of increasingly authentic small-group, preclerkship teaching format on clerkship outcomes to further enlighten this debate. SUMMARY: Students enrolled in a prospective randomized crossover trial that involved three content areas. For each content area, three teaching formats were tested. Participants were randomized to teaching format by content area. Clerkship outcomes were performance on an objective structured clinical exam, a DVD exam, internal medicine clerkship grades, and performance on the subject examination. The data were analyzed using a multivariate analysis of covariance. One hundred and thirty-three (78%) students participated. Teaching format did not have a statistically significant effect on any of the specified clerkship outcomes. However, number of patients seen was significantly associated with higher scores in respective outcomes by topic. CONCLUSIONS: Second-year teaching format did not directly influence subsequent clerkship performance. Our study adds to the literature by demonstrating that the authenticity of preclinical teaching format does not appear to matter for clerkship performance; however, the number of actual patients seen does appear to influence related clerkship outcomes.

Medical student documentation in electronic health records: a collaborative statement from the Alliance for Clinical Education.

PURPOSE: The electronic health record (EHR) is an important advancement in health care. It facilitates improvement of health care delivery and coordination of care, but it creates special challenges for student education. This article represents a collaborative effort of the Alliance for Clinical Education (ACE), a multidisciplinary group formed in 1992. ACE recognizes the importance of medical student participation in patient care including the ability of documentation. This article proposes guidelines that can be used by educators to establish expectations on medical student documentation in EHRs. SUMMARY: To provide the best education for medical students in the electronic era, ACE proposes to use the following as practice guidelines for medical student documentation in the EHR: (a) Students must document in the patient's chart and their notes should be reviewed for content and format, (b) students must have the opportunity to practice order entry in an EHR--in actual or simulated patient cases--prior to graduation, (c) students should be exposed to the utilization of the decision aids that typically accompany EHRs, and (d) schools must develop a set of medical student competencies related to charting in the EHR and state how they would evaluate it. This should include specific competencies to be documented at each stage, and by time of graduation. In addition, ACE recommends that accreditation bodies such as the Liaison Committee for Medical Education utilize stronger language in their educational directives standards to ensure compliance with educational principles. This will guarantee that the necessary training and resources are available to ensure that medical students have the fundamental skills for lifelong clinical practice. CONCLUSIONS: ACE recommends that medical schools develop a clear set of competencies related to student in the EHR which medical students must achieve prior to graduation in order to ensure they are ready for clinical practice.

Instructional authenticity and clinical reasoning in undergraduate medical education: a 2-year, prospective, randomized trial.

OBJECTIVE: The purpose of the study was to investigate the relationship between authenticity (how close to actual practice) of preclerkship instructional formats and preclerkship and clerkship outcome measures. A secondary purpose was to investigate the effect of student's small-group assignment during preclerkship period on instructional formats and preclerkship and clerkship outcome measurements. METHODS: A prospective, randomized, crossover study was carried out with preclerkship students taking a clinical reasoning course. Students were randomized to small groups and exposed to three formats of differing instructional authenticity across three subject areas. Three student cohorts were taught using one instructional format per subject area with each cohort receiving a different instructional format for each of the three areas. Outcome measures at the end of the preclerkship year and the clerkship year were selected to determine the effect of each instructional format. Hierarchical linear modeling was performed to assess impact of format on outcomes and to assess potential group effect on outcomes. RESULTS: Increasingly authentic instructional formats did not significantly improve clinical reasoning performance. Small-group assignment did not significantly contribute to the outcomes providing evidence that teacher training was successful. CONCLUSIONS: Increasing authenticity of instructional formats does not appear to significantly improve clinical reasoning performance. Faculty can be successfully trained to teach consistently in a clinical reasoning course. Medical educators should balance increasing authenticity with factors such as cognitive load and learner experience as well as engaging in frame-of-reference training to minimize group effects with designing new instructional formats.

Using functional neuroimaging combined with a think-aloud protocol to explore clinical reasoning expertise in internal medicine.

BACKGROUND: Clinical reasoning is essential to medical practice, but because it entails internal mental processes, it is difficult to assess. Functional magnetic resonance imaging (fMRI) and think-aloud protocols may improve understanding of clinical reasoning as these methods can more directly assess these processes. The objective of our study was to use a combination of fMRI and think-aloud procedures to examine fMRI correlates of a leading theoretical model in clinical reasoning based on experimental findings to date: analytic (i.e., actively comparing and contrasting diagnostic entities) and nonanalytic (i.e., pattern recognition) reasoning. We hypothesized that there would be functional neuroimaging differences between analytic and nonanalytic reasoning theory. METHODS: 17 board-certified experts in internal medicine answered and reflected on validated U.S. Medical Licensing Exam and American Board of Internal Medicine multiple-choice questions (easy and difficult) during an fMRI scan. This procedure was followed by completion of a formal think-aloud procedure. RESULTS: fMRI findings provide some support for the presence of analytic and nonanalytic reasoning systems. Statistically significant activation of prefrontal cortex distinguished answering incorrectly versus correctly (p < 0.01), whereas activation of precuneus and midtemporal gyrus distinguished not guessing from guessing (p < 0.01). CONCLUSIONS: We found limited fMRI evidence to support analytic and nonanalytic reasoning theory, as our results indicate functional differences with correct vs. incorrect answers and guessing vs. not guessing. However, our findings did not suggest one consistent fMRI activation pattern of internal medicine expertise. This model of employing fMRI correlates offers opportunities to enhance our understanding of theory, as well as improve our teaching and assessment of clinical reasoning, a key outcome of medical education.

Physician Burnout-Evidence That Leadership Behaviors Make a Difference: A Cross-Sectional Survey of an Academic Medical Center.

INTRODUCTION: High rates of physician burnout are well documented in the USA. Identifying beneficial leadership behaviors as an organizational approach to mitigating burnout can lead to improved wellness in the physicians that they lead; however, few studies have examined which leadership behaviors are beneficial and which may be detrimental. MATERIALS AND METHODS: This survey study of academic medical center physicians and their physician leaders assessed the correlation between burnout and leadership behaviors. Data were analyzed for the strength of correlation between scores for leadership behaviors and self-reported physician burnout with analysis of variance by sex, time from training, specialty, and age. RESULTS: Of 1,145 physicians surveyed, 305 returned surveys. Among the respondents, 45% were female, 25% were 56 years or older, and 57% self-identified as practitioners of medicine or medicine subspecialties. Two transformational leadership categories of behaviors (idealized influence behaviors and individualized consideration) and one transactional leadership behavior category (contingent reward) correlated favorably with all domains of burnout (P < .0001). Conversely, two transactional leadership categories of burnout (management by exception passive and laissez-faire) correlated unfavorably with all burnout domains. CONCLUSIONS: Organizational interventions are needed to improve burnout in physicians. Adopting favorable leadership behaviors while avoiding unfavorable leadership behaviors can improve burnout in those physicians being led. These findings could inform the conceptual basis of future physician leadership training programs as transactional leadership behaviors also have an impact on physician wellness.

Context and clinical reasoning: understanding the perspective of the expert's voice.

OBJECTIVES: Prior work has found that a doctor's clinical reasoning performance varies on a case-by-case (situation) basis; this is often referred to as 'context specificity'. To explore the influence of context on diagnostic and therapeutic clinical reasoning, we constructed a series of videotapes to which doctors were asked to respond, modifying different contextual factors (patient, doctor, setting). We explored how these contextual factors, as displayed by videotape encounters, may have influenced the clinical reasoning of board-certified internists (experts). Our purpose was to clarify the influence of context on reasoning, to build upon education theory and to generate implications for education practice. METHODS: Qualitative data about experts were gathered from two sources: think-aloud protocols reflecting concurrent thought processes that occurred while board-certified internists viewed videotape encounters, and free-text responses to queries that explicitly asked these experts to comment on the influence of selected contextual factors on their clinical reasoning processes. These data sources provided both actual performance data (think-aloud responses) and opinions on reflection (free-text answers) regarding the influence of context on reasoning. Results for each data source were analysed for emergent themes and then combined into a unified theoretical model. RESULTS: Several themes emerged from our data and were broadly classified as components influencing the impact of contextual factors, mechanisms for addressing contextual factors, and consequences of contextual factors for patient care. Themes from both data sources had good overlap, indicating that experts are somewhat cognisant of the potential influences of context on their reasoning processes; notable exceptions concerned the themes of missed key findings, balancing of goals and the influence of encounter setting, which emerged in the think-aloud but not the free-text analysis. CONCLUSIONS: Our unified model is consistent with the tenets of cognitive load, situated cognition and ecological psychology theories. A number of potentially modifiable influences on clinical reasoning were identified. Implications for doctor training and practice are discussed.

Authenticity of instruction and student performance: a prospective randomised trial.

OBJECTIVES: This study aimed to investigate the relationship between the authenticity of instructional formats and outcome measures within a pre-clerkship clinical reasoning course. METHODS: We conducted a randomised, prospective, crossover study with Year 2 medical students taking a pre-clerkship clinical reasoning course. Students were randomised to small groups and exposed to three formats of differing instructional authenticity (paper case, DVD presentation, standardised patient [SP] presentation) across three subject areas (abdominal pain, anaemia, polyuria). Three student cohorts were taught using one instructional format per subject area so that each cohort received a different instructional format for each of the three subject areas. Outcome measures (objective structured clinical examination, video quiz, written examination) were selected to determine the effect of each instructional format on the clinical reasoning of students. RESULTS: Increasingly authentic instructional formats did not significantly improve clinical reasoning performance across all outcome measures and subject areas. However, the results of the video quiz showed significant differences in the anaemia subject area between students who had been instructed using the paper case and live SP-based formats (scores of 47.4 and 57.6, respectively; p = 0.01) and in the abdominal pain subject area, in which students instructed using the DVD format scored higher than students instructed using either the paper case or SP-based formats (scores of 41.6, 34.9 and 31.2, respectively; p=0.002). CONCLUSIONS: Increasing the authenticity of instructional formats does not appear to significantly improve clinical reasoning performance in a pre-clerkship course. Medical educators should balance increases in authenticity with factors such as cognitive load, subject area and learner experience when designing new instructional formats.

Research in assessment: consensus statement and recommendations from the Ottawa 2010 Conference.

Medical education research in general is a young scientific discipline which is still finding its own position in the scientific range. It is rooted in both the biomedical sciences and the social sciences, each with their own scientific language. A more unique feature of medical education (and assessment) research is that it has to be both locally and internationally relevant. This is not always easy and sometimes leads to purely ideographic descriptions of an assessment procedure with insufficient general lessons or generalised scientific knowledge being generated or vice versa. For medical educational research, a plethora of methodologies is available to cater to many different research questions. This article contains consensus positions and suggestions on various elements of medical education (assessment) research. Overarching is the position that without a good theoretical underpinning and good knowledge of the existing literature, good research and sound conclusions are impossible to produce, and that there is no inherently superior methodology, but that the best methodology is the one most suited to answer the research question unambiguously. Although the positions should not be perceived as dogmas, they should be taken as very serious recommendations. Topics covered are: types of research, theoretical frameworks, designs and methodologies, instrument properties or psychometrics, costs/acceptability, ethics, infrastructure and support.