"They were very very nice but just not very good": The interplay between resident-supervisor relationships and assessment in the emergency setting.

Purpose: Clinical supervisors hesitate to report learner weaknesses, a widely documented phenomenon referred to as "failure to fail." They also struggle to discuss weaknesses with learners themselves. Their reluctance to report and discuss learner weaknesses threatens the validity of assessment-of-learning decisions and the effectiveness of assessment for learning. Personal and interpersonal factors have been found to act as barriers to reporting learners' difficulties, but the precise role of the resident-supervisor relationship remains underexplored, specifically in the emergency setting. This study aims to better understand if and how factors related to the resident-supervisor relationship are involved in assessment of and for learning in the emergency setting. Methods: We conducted a qualitative study, using semistructured interviews of 15 clinical supervisors in emergency medicine departments affiliated with our institution. Transcripts were independently coded by three members of the team using an iterative mixed deductive-inductive thematic analysis approach. The team then synthesized the coding and discussed analysis following guidelines for thematic analysis. Results: Participating emergency medicine supervisors valued resident-supervisor relationships built on collaboration and trust and believed that such relationships support learning. They described how these relationships influenced assessment of and for learning and how in turn assessment influenced the relationship. Almost all profiles of resident-supervisor relationships in our study could hinder the disclosing of resident weaknesses, through a variety of mechanisms. To protect residents and themselves from the discomfort of disclosing weaknesses and to avoid deteriorating the resident-supervisor relationship, many downplayed or even masked residents' difficulties. Supervisors who described themselves as able to provide negative assessment of and for learning often adopted a more distant or professional stance. Conclusions: This study contributes to a growing literature on failure to fail by confirming the critical impact that the resident-supervisor relationship has on the willingness and ability of emergency medicine supervisors to play their part as assessors.

If we assess, will they learn? Students' perspectives on the complexities of assessment-for-learning.

Introduction: Assessment can positively influence learning, however designing effective assessment-for-learning interventions has proved challenging. We implemented a mandatory assessment-for-learning system comprising a workplace-based assessment of non-medical expert competencies and a progress test in undergraduate medical education and evaluated its impact. Methods: We conducted semi-structured interviews with year-3 and 4 medical students at McGill University to explore how the assessment system had influenced their learning in year 3. We conducted theory-informed thematic analysis of the data. Results: Eleven students participated, revealing that the assessment influenced learning through several mechanisms. Some required little student engagement (i.e., feed-up, test-enhanced learning, looking things up after an exam). Others required substantial engagement (e.g., studying for tests, selecting raters for quality feedback, using feedback). Student engagement was moderated by the perceived credibility of the system and of the costs and benefits of engagement. Credibility was shaped by students' goals-in-context: becoming a good doctor, contributing to the healthcare team, succeeding in assessments. Discussion: Our assessment system failed to engage students enough to leverage its full potential. We discuss the inherent flaws and external factors that hindered student engagement. Assessment designers should leverage easy-to-control mechanisms to support assessment-for-learning and anticipate significant collaborative work to modify learning cultures.

Introduction: L'évaluation peut influencer positivement l'apprentissage mais la conception de dispositifs d'évaluation pour l'apprentissage efficaces s'avère difficile. Nous avons implanté en formation prédoctorale un système obligatoire d'évaluation pour l'apprentissage comprenant une évaluation en milieu clinique des compétences transversales et un test de rendement progressif, puis évalué ses effets. Méthodes: Nous avons mené des entretiens semi-dirigés avec des étudiants en troisième et quatrième années de médecine à l'Université McGill pour explorer la manière dont le système d'évaluation avait influencé leur apprentissage au cours de la troisième année. Nous avons effectué une analyse thématique, informée par la théorie, des données. Résultats: Onze étudiants ont participé. Les résultats indiquent que l'évaluation a influencé leur apprentissage par le biais de plusieurs mécanismes différents. Certains d'entre eux nécessitaient une implication faible de la part de l'étudiant, comme l'identification des objectifs à atteindre (feed-up), l'apprentissage amélioré par les tests, la recherche d'informations après un examen. D'autres exigeaient une implication importante (par exemple, étudier pour les tests, sélectionner les évaluateurs pour obtenir une rétroaction de qualité, mettre à profit la rétroaction). L'implication des étudiants était modulée par leur perception des avantages et des inconvénients de s'impliquer, et de la crédibilité du système. Cette dernière était influencée par les objectifs-en-contexte des étudiants: devenir un bon médecin, contribuer à l'équipe soignante, réussir les épreuves d'évaluation. Discussion: Notre système d'évaluation n'a pas réussi à impliquer suffisamment les étudiants que pour réaliser son potentiel. Nous abordons les défauts inhérents au système ainsi que les facteurs externes qui ont entravé l'implication des apprenants. Pour implanter efficacement un dispositif d'évaluation pour l'apprentissage, les concepteurs d'évaluations devraient optimiser les mécanismes qui sont faciles à contrôler et être prêts à s'investir dans un important travail de collaboration pour changer les cultures d'apprentissage.

Impact of emotional competence on physicians' clinical reasoning: a scoping review protocol.

INTRODUCTION: Clinical reasoning (CR) is a key competence for physicians and a major source of damaging medical errors. Many strategies have been explored to improve CR quality, most of them based on knowledge enhancement, cognitive debiasing and the use of analytical reasoning. If increasing knowledge and fostering analytical reasoning have shown some positive results, the impact of debiasing is however mixed. Debiasing and promoting analytical reasoning have also been criticised for their lack of pragmatism. Alternative means of increasing CR quality are therefore still needed. Because emotions are known to influence the quality of reasoning in general, we hypothesised that emotional competence (EC) could improve physicians' CR. EC refers to the ability to identify, understand, express, regulate and use emotions. The influence of EC on CR remains unclear. This article presents a scoping review protocol, the aim of which will be to describe the current state of knowledge concerning the influence of EC on physicians' CR, the type of available literature and finally the different methods used to examine the link between EC and CR. METHOD AND ANALYSIS: The population of interest is physicians and medical students. EC will be explored according to the model of Mikolajczak et al, describing five major components of EC (identify, understand, express, regulate and use emotions). The concept of CR will include terms related to its processes and outcomes. Context will include real or simulated clinical situations. The search for primary sources and reviews will be conducted in MEDLINE (via Ovid), Scopus and PsycINFO. The grey literature will be searched in the references of included articles and in OpenGrey. Study selection and data extraction will be conducted using the Covidence software. Search and inclusion results will be reported using the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for scoping review model (PRISMA-ScR). ETHICS AND DISSEMINATION: There are no ethical or safety concerns regarding this review. REGISTRATION DETAILS: OSF Registration DOI: https://doi.org/10.17605/OSF.IO/GM7YD.

Does Educational Handover Influence Subsequent Assessment?

PURPOSE: Educational handover (i.e., providing information about learners' past performance) is controversial. Proponents argue handover could help tailor learning opportunities. Opponents fear it could bias subsequent assessments and lead to self-fulfilling prophecies. This study examined whether raters provided with reports describing learners' minor weaknesses would generate different assessment scores or narrative comments than those who did not receive such reports. METHOD: In this 2018 mixed-methods, randomized, controlled, experimental study, clinical supervisors from 5 postgraduate (residency) programs were randomized into 3 groups receiving no educational handover (control), educational handover describing weaknesses in medical expertise, and educational handover describing weaknesses in communication. All participants watched the same videos of 2 simulated resident-patient encounters and assessed performance using a shortened mini-clinical evaluation exercise form. The authors compared mean scores, percentages of negative comments, comments focusing on medical expertise, and comments focusing on communication across experimental groups using analyses of variance. They examined potential moderating effects of supervisor experience, gender, and mindsets (fixed vs growth). RESULTS: Seventy-two supervisors participated. There was no effect of handover report on assessment scores (F(2, 69) = 0.31, P = .74) or percentage of negative comments (F(2, 60) = 0.33, P = .72). Participants who received a report indicating weaknesses in communication generated a higher percentage of comments on communication than the control group (63% vs 50%, P = .03). Participants who received a report indicating weaknesses in medical expertise generated a similar percentage of comments on expertise compared to the controls (46% vs 47%, P = .98). CONCLUSIONS: This study provides initial empirical data about the effects of educational handover and suggests it can-in some circumstances-lead to more targeted feedback without influencing scores. Further studies are required to examine the influence of reports for a variety of performance levels, areas of weakness, and learners.

Learner handover: Perspectives and recommendations from the front-line.

INTRODUCTION: Current medical education models increasingly rely on longitudinal assessments to document learner progress over time. This longitudinal focus has re-kindled discussion regarding learner handover-where assessments are shared across supervisors, rotations, and educational phases, to support learner growth and ease transitions. The authors explored the opinions of, experiences with, and recommendations for successful implementation of learner handover among clinical supervisors. METHODS: Clinical supervisors from five postgraduate medical education programs at one institution completed an online questionnaire exploring their views regarding learner handover, specifically: potential benefits, risks, and suggestions for implementation. Survey items included open-ended and numerical responses. The authors used an inductive content analysis approach to analyze the open-ended questionnaire responses, and descriptive and correlational analyses for numerical data. RESULTS: Seventy-two participants completed the questionnaire. Their perspectives varied widely. Suggested benefits of learner handover included tailored learning, improved assessments, and enhanced patient safety. The main reported risk was the potential for learner handover to bias supervisors' perceptions of learners, thereby affecting the validity of future assessments and influencing the learner's educational opportunities and well-being. Participants' suggestions for implementation focused on who should be involved, when and for whom it should occur, and the content that should be shared. DISCUSSION: The diverse opinions of, and recommendations for, learner handover highlight the necessity for handover to maximize learning potential while minimizing potential harms. Supervisors' suggestions for handover implementation reveal tensions between assessment-of and for-learning.

Mapping clinical reasoning literature across the health professions: a scoping review.

BACKGROUND: Clinical reasoning is at the core of health professionals' practice. A mapping of what constitutes clinical reasoning could support the teaching, development, and assessment of clinical reasoning across the health professions. METHODS: We conducted a scoping study to map the literature on clinical reasoning across health professions literature in the context of a larger Best Evidence Medical Education (BEME) review on clinical reasoning assessment. Seven databases were searched using subheadings and terms relating to clinical reasoning, assessment, and Health Professions. Data analysis focused on a comprehensive analysis of bibliometric characteristics and the use of varied terminology to refer to clinical reasoning. RESULTS: Literature identified: 625 papers spanning 47 years (1968-2014), in 155 journals, from 544 first authors, across eighteen Health Professions. Thirty-seven percent of papers used the term clinical reasoning; and 110 other terms referring to the concept of clinical reasoning were identified. Consensus on the categorization of terms was reached for 65 terms across six different categories: reasoning skills, reasoning performance, reasoning process, outcome of reasoning, context of reasoning, and purpose/goal of reasoning. Categories of terminology used differed across Health Professions and publication types. DISCUSSION: Many diverse terms were present and were used differently across literature contexts. These terms likely reflect different operationalisations, or conceptualizations, of clinical reasoning as well as the complex, multi-dimensional nature of this concept. We advise authors to make the intended meaning of 'clinical reasoning' and associated terms in their work explicit in order to facilitate teaching, assessment, and research communication.

A Scoping Review of Physicians' Clinical Reasoning in Emergency Departments.

STUDY OBJECTIVE: Clinical reasoning is considered a core competency of physicians. Yet there is a paucity of research on clinical reasoning specifically in emergency medicine, as highlighted in the literature. METHODS: We conducted a scoping review to examine the state of research on clinical reasoning in this specialty. Our team, composed of content and methodological experts, identified 3,763 articles in the literature, 95 of which were included. RESULTS: Most studies were published after 2000. Few studies focused on the cognitive processes involved in decisionmaking (ie, clinical reasoning). Of these, many confirmed findings from the general literature on clinical reasoning; specifically, the role of both intuitive and analytic processes. We categorized factors that influence decisionmaking into contextual, patient, and physician factors. Many studies focused on decisions in regard to investigations and admission. Test ordering is influenced by physicians' experience, fear of litigation, and concerns about malpractice. Fear of litigation and malpractice also increases physicians' propensity to admit patients. Context influences reasoning but findings pertaining to specific factors, such as patient flow and workload, were inconsistent. CONCLUSION: Many studies used designs such as descriptive or correlational methods, limiting the strength of findings. Many gray areas persist, in which studies are either scarce or yield conflicting results. The findings of this scoping review should encourage us to intensify research in the field of emergency physicians' clinical reasoning, particularly on the cognitive processes at play and the factors influencing them, using appropriate theoretical frameworks and more robust methods.

Nudging clinical supervisors to provide better in-training assessment reports.

INTRODUCTION: In-training assessment reports (ITARs) summarize assessment during a clinical placement to inform decision-making and provide formal feedback to learners. Faculty development is an effective but resource-intensive means of improving the quality of completed ITARs. We examined whether the quality of completed ITARs could be improved by 'nudges' from the format of ITAR forms. METHODS: Our first intervention consisted of placing the section for narrative comments at the beginning of the form, and using prompts for recommendations (Do more, Keep doing, Do less, Stop doing). In a second intervention, we provided a hyperlink to a detailed assessment rubric and shortened the checklist section. We analyzed a sample of 360 de-identified completed ITARs from six disciplines across the three academic years where the different versions of the ITAR were used. Two raters independently scored the ITARs using the Completed Clinical Evaluation Report Rating (CCERR) scale. We tested for differences between versions of the ITAR forms using a one-way ANOVA for the total CCERR score, and MANOVA for the nine CCERR item scores. RESULTS: Changes to the form structure (nudges) improved the quality of information generated as measured by the CCERR instrument, from a total score of 18.0/45 (SD 2.6) to 18.9/45 (SD 3.1) and 18.8/45 (SD 2.6), p = 0.04. Specifically, comments were more balanced, more detailed, and more actionable compared with the original ITAR. DISCUSSION: Nudge interventions, which are inexpensive and feasible, should be included in multipronged approaches to improve the quality of assessment reports.

The challenges of detecting progress in generic competencies in the clinical setting.

CONTEXT: Competency-based medical education has spurred the implementation of longitudinal workplace-based assessment (WBA) programmes to track learners' development of competencies. These hinge on the appropriate use of assessment instruments by assessors. This study aimed to validate our assessment programme and specifically to explore whether assessors' beliefs and behaviours rendered the detection of progress possible. METHODS: We implemented a longitudinal WBA programme in the third year of a primarily rotation-based clerkship. The programme used the professionalism mini-evaluation exercise (P-MEX) to detect progress in generic competencies. We used mixed methods: a retrospective psychometric examination of student assessment data in one academic year, and a prospective focus group and interview study of assessors' beliefs and reported behaviours related to the assessment. RESULTS: We analysed 1662 assessment forms for 186 students. We conducted interviews and focus groups with 21 assessors from different professions and disciplines. Scores were excellent from the outset (3.5-3.7/4), with no meaningful increase across blocks (average overall scores: 3.6 in block 1 versus 3.7 in blocks 2 and 3; F = 8.310, d.f. 2, p < 0.001). The main source of variance was the forms (47%) and only 1% of variance was attributable to students, which led to low generalisability across forms (Eρ2  = 0.18). Assessors reported using multiple observations to produce their assessments and were reluctant to harm students by consigning anything negative to writing. They justified the use of a consistent benchmark across time by citing the basic nature of the form or a belief that the 'competencies' assessed were in fact fixed attributes that were unlikely to change. CONCLUSIONS: Assessors may purposefully deviate from instructions in order to meet their ethical standards of good assessment. Furthermore, generic competencies may be viewed as intrinsic and fixed rather than as learnable. Implementing a longitudinal WBA programme is complex and requires careful consideration of assessors' beliefs and values.

Examining the effects of gaming and guessing on script concordance test scores.

INTRODUCTION: In a script concordance test (SCT), examinees are asked to judge the effect of a new piece of clinical information on a proposed hypothesis. Answers are collected using a Likert-type scale (ranging from -2 to +2, with '0' indicating no effect), and compared with those of a reference panel of 'experts'. It has been argued, however, that SCT may be susceptible to the influences of gaming and guesswork. This study aims to address some of the mounting concern over the response process validity of SCT scores. METHOD: Using published datasets from three independent SCTs, we investigated examinee response patterns, and computed the score a hypothetical examinee would obtain on each of the tests if he 1) guessed random answers and 2) deliberately answered '0' on all test items. RESULTS: A simulated random guessing strategy led to scores 2 SDs below mean scores of actual respondents (Z-scores -3.6 to -2.1). A simulated 'all-0' strategy led to scores at least 1 SD above those obtained by random guessing (Z-scores -2.2 to -0.7). In one dataset, stepwise exclusion of items with modal panel response '0' to fewer than 10% of the total number of test items yielded hypothetical scores 2 SDs below mean scores of actual respondents. DISCUSSION: Random guessing was not an advantageous response strategy. An 'all-0' response strategy, however, demonstrated evidence of artificial score inflation. Our findings pose a significant threat to the SCT's validity argument. 'Testwiseness' is a potential hazard to all testing formats, and appropriate countermeasures must be established. We propose an approach that might be used to mitigate a potentially real and troubling phenomenon in script concordance testing. The impact of this approach on the content validity of SCTs merits further discussion.

Diagnosis and management of clinical reasoning difficulties: Part II. Clinical reasoning difficulties: Management and remediation strategies.

Part II of this AMEE Guide provides a detailed overview of the main difficulties in clinical reasoning, including the cues to look out for in clinical supervision, the root causes of each difficulty and targeted remediation strategies. Specific challenges and issues related to the management of clinical reasoning difficulties will also be discussed.

Diagnosis and management of clinical reasoning difficulties: Part I. Clinical reasoning supervision and educational diagnosis.

There are many obstacles to the timely identification of clinical reasoning difficulties in health professions education. This guide aims to provide readers with a framework for supervising clinical reasoning and identifying the potential difficulties as they may occur at each step of the reasoning process.

In Brief: Validity of Case Summaries in Written Examinations of Clinical Reasoning.

Construct: The purpose of this study was to provide initial evidence of the validity of written case summaries as assessments of clinical problem representation in a classroom setting. BACKGROUND: To solve clinical problems, clinicians must gain a clear representation of the issues. In the clinical setting, oral case presentations-or summaries-are used to assess learners' ability to gather, synthesize, and "translate" pertinent case information. This ability can be assessed in Objective Structured Clinical Examination and Virtual Patient settings using oral or written case summaries. Evidence of their validity in these settings includes adequate interrater agreement and moderate correlation with other assessments of clinical reasoning. We examined the use of written case summaries in a classroom setting as part of an examination designed to assess clinical reasoning. APPROACH: We developed and implemented written examinations for 2 preclerkship general practice courses in Years 4 and 5 of a 7-year curriculum. Examinations included 8 case summary questions in Year 4 and 5 in Year 5. Seven hundred students participated. Cases were scored using 3 criteria: extraction of pertinent findings, semantic quality, and global ratings. We examined the item parameters (using classical test theory) and generalizability of case summary items. We computed correlations between case summary scores and scores on other questions within the examination. RESULTS: Item parameters were acceptable (average item difficulty = 0.49-0.73 and 0.59-0.68 in Years 4 and 5; average point-biserials = 0.21-0.24 and 0.18-0.21). Scores were moderately generalizable (G coefficients = 0.40-0.50), with case-specificity a substantial source of measurement error (10.2%-19.5% of variance). Scoring and rater had small effects. Correlations with related constructs were low to moderate. CONCLUSIONS: There is good evidence regarding the scoring and generalizability of written case summaries for assessment of clinical problem representation. Further evidence regarding the extrapolation and implications of these assessments is warranted.

Do emergency physicians trust their patients?

The primary focus of research on the physician-patient relationship has been on patients' trust in their physicians. In this study, we explored physicians' trust in their patients. We held semi-structured interviews with expert emergency physicians concerning a patient they had just been managing. The physicians had been equipped with a head-mounted micro camera to film the encounter from an "own point of view perspective". The footage was used to stimulate recall during the interviews. Several participants made judgments on the reliability of their patients' accounts from the very beginning of the encounter. If accounts were not deemed reliable, participants implemented a variety of specific strategies in pursuing their history taking, i.e. checking for consistency by asking the same question at several points in the interview, cross-referencing information, questioning third-parties, examining the patient record, and systematically collecting data held to be objective. Our study raises the question of the influence of labeling patients as "reliable" or "unreliable" on their subsequent treatment in the emergency department. Further work is necessary to examine the accuracy of these judgments, the underlying cognitive processes (i.e. analytic versus intuitive) and their influence on decision-making.

From Context Comes Expertise: How Do Expert Emergency Physicians Use Their Know-Who to Make Decisions?

STUDY OBJECTIVE: Decisionmaking is influenced by the environment in which it takes place. The objective of our study was to explore the influence of the specific features of the emergency department (ED) environment on decisionmaking. In this paper, we specifically report on the way emergency physicians use their knowledge of their collaborators to make their decisions. METHODS: We conducted a qualitative study on emergency physicians recruited in 3 French hospitals. Physicians were equipped with a microcamera to record their clinical activity from their "own-point-of-view perspective." Semistructured interviews, based on viewing the video, were held with each physician after an actual clinical encounter with a patient. They were then analyzed thematically, using constant comparison and matrices, to identify the central themes. RESULTS: Fifteen expert emergency physicians were interviewed. Almost all of them reported using their knowledge of other health care professionals to assess the seriousness of the patient's overall condition (sometimes even before his or her arrival in the ED) to optimize the patient's treatment and to anticipate future care. CONCLUSION: Emergency physicians interact with many other health care workers during the different stages of the patient's management. The many ways in which experts use their knowledge of other health care professionals to make decisions puts traditional conceptions of expert knowledge into perspective and opens avenues for future research.

Social dominance theory and medical specialty choice.

Understanding how medical students select their specialty is a fundamental issue for public health and educational policy makers. One of the factors that students take into account is a specialty's prestige which hinges partly on its focus on technique rather than whole person. We examine the potential of a psychological framework, social dominance theory, to explain why some students, and not others, are drawn to more prestigious, technique-oriented specialties, based on their desire for hierarchy. We conducted a cross-sectional study among medical students at Institution X (N = 359). We examined the link between medical students' characteristics i.e. social dominance orientation (SDO), gender, age, and their career intention. We also examined level of medical students' SDO at different stages of the curriculum. SDO scores were significantly associated with technique-oriented career intentions (OR 1.56; 95 % CI [1.18, 2.06]; p = 0.001). The effect was independent of gender. Medical students' SDO scores were significantly higher in later stages of the medical curriculum (F = 6.79; p = 0. 001). SDO is a significant predictor of medical students' career intention. SDO scores are higher in students during the clinical phase of the curriculum. Medical socialization, involving the internalization of implicit and explicit norms, particularly in hospital settings, is likely to underpin our findings. This theory illuminates consistent findings in the literature on specialty prestige and the influence of medical school on career choice.

Considering "Nonlinearity" Across the Continuum in Medical Education Assessment: Supporting Theory, Practice, and Future Research Directions.

The purpose of this article is to propose new approaches to assessment that are grounded in educational theory and the concept of "nonlinearity." The new approaches take into account related phenomena such as "uncertainty," "ambiguity," and "chaos." To illustrate these approaches, we will use the example of assessment of clinical reasoning, although the principles we outline may apply equally well to assessment of other constructs in medical education. Theoretical perspectives include a discussion of script theory, assimilation theory, self-regulated learning theory, and situated cognition. Assessment examples to include script concordance testing, concept maps, self-regulated learning microanalytic technique, and work-based assessment, which parallel the above-stated theories, respectively, are also highlighted. We conclude with some practical suggestions for approaching nonlinearity.

Insights into emergency physicians' minds in the seconds before and into a patient encounter.

Clinical reasoning is a core competency in medical practice. No study has explored clinical reasoning occurring before a clinical encounter, when physicians obtain preliminary information about the patient, and during the first seconds of the observation phase. This paper aims to understand what happens in emergency physicians' minds when they acquire initial information about a patient, and when they first meet a patient. The authors carried out in-depth interviews based on the video recordings of emergency situations filmed in an "own-point-of-view-perspective". 15 expert emergency physicians were interviewed between 2011 and 2012. Researchers analysed data using an interpretive approach based on thematic analysis and constant comparison. Almost all participants used a few critical pieces of information to generate hypotheses even before they actually met the patient. Pre-encounter hypotheses played a key role in the ensuing encounter by directing initial data gathering. Initial data, collected within the first few seconds of the encounter, included the patient's position on the stretcher, the way they had been prepared, their facial expression, their breathing, and their skin colour. Physicians also rapidly appraised the seriousness of the patient's overall condition, which determined their initial goals, i.e. initiating emergency treatment or pursuing the diagnostic investigation. The study brings new insights on what happens at the very beginning of the encounter between emergency physicians and patients. The results obtained from an innovative methodological approach open avenues for the development of clinical reasoning in learners.