Assessing the acceptability of script concordance testing: a nationwide study in otolaryngology.

Background: Script concordance testing (SCT) is an objective method to evaluate clinical reasoning that assesses the ability to interpret medical information under conditions of uncertainty. Many studies have supported its validity as a tool to assess higher levels of learning, but little is known about its acceptability to major stakeholders. The aim of this study was to determine the acceptability of SCT to residents in otolaryngology ­ head and neck surgery (OTL-HNS) and a reference group of experts. Methods: In 2013 and 2016, a set of SCT questions, as well a post-test exit survey, were included in the National In-Training Examination (NITE) for OTL-HNS. This examination is administered to all OTL-HNS residents across Canada who are in the second to fifth year of residency. The same SCT questions and survey were then sent to a group of OTL-HNS surgeons from 4 Canadian universities. Results: For 64.4% of faculty and residents, the study was their first exposure to SCT. Overall, residents found it difficult to adapt to this form of testing, thought that the clinical scenarios were not clear and believed that SCT was not useful for assessing clinical reasoning. In contrast, the vast majority of experts felt that the test questions reflected real-life clinical situations and would recommend SCT as an evaluation method in OTL-HNS. Conclusion: Views about the acceptability of SCT as an assessment tool for clinical reasoning differed between OTL-HNS residents and experts. Education about SCT and increased exposure to this testing method are necessary to improve residents' perceptions of SCT.

Contexte: Le test de concordance de script (TCS) est une méthode objective d'évaluation du raisonnement clinique qui mesure la capacité d'interpréter les renseignements médicaux en contexte d'incertitude. Beaucoup d'études en appuient la validité en tant qu'outil pour évaluer l'enseignement supérieur, mais on en sait peu sur son acceptabilité auprès des principales parties prenantes. Le but de cette étude était de déterminer l'acceptabilité du TCS chez les résidents en otorhinolaryngologie ­ chirurgie de la tête et du cou (ORL ­ chirurgie tête et cou) et un groupe de référence composé d'experts. Méthodes: En 2013 et 2016, une série de questions de TCS, de même qu'un questionnaire post-test, ont été inclus dans l'examen national en cours de formation NITE (National In-Training Examination) pour l'ORL ­ chirurgie tête et cou. Cet examen est administré à tous les résidents en ORL ­ chirurgie tête et cou au Canada qui sont entre leurs deuxième et cinquième années de résidence. Les mêmes questions de TCS ont été envoyées à un groupe de chirurgiens en ORL ­ chirurgie tête et cou de 4 université canadiennes. Résultats: Pour 64,4 % des membres facultaires et des résidents, l'étude était leur première exposition au TCS. Dans l'ensemble, les résidents ont trouvé difficile de s'adapter à cette forme de test, même si les scénarios cliniques étaient clairs, et ils ont estimé que le TCS était peu utile pour évaluer le raisonnement clinique. En revanche, la grande majorité des experts ont jugé que les questions du test reflétaient la réalité des cas cliniques et recommanderaient le TCS comme méthode d'évaluation en ORL ­ chirurgie tête et cou. Conclusion: Entre les résidents et les experts en ORL ­ chirurgie tête et cou, les points de vue quant à l'acceptabilité du TCS comme outil d'évaluation du raisonnement clinique ont différé et il faudrait y exposer les résidents davantage pour améliorer leur perception du TCS.

Mapping clinical reasoning literature across the health professions: a scoping review.

BACKGROUND: Clinical reasoning is at the core of health professionals' practice. A mapping of what constitutes clinical reasoning could support the teaching, development, and assessment of clinical reasoning across the health professions. METHODS: We conducted a scoping study to map the literature on clinical reasoning across health professions literature in the context of a larger Best Evidence Medical Education (BEME) review on clinical reasoning assessment. Seven databases were searched using subheadings and terms relating to clinical reasoning, assessment, and Health Professions. Data analysis focused on a comprehensive analysis of bibliometric characteristics and the use of varied terminology to refer to clinical reasoning. RESULTS: Literature identified: 625 papers spanning 47 years (1968-2014), in 155 journals, from 544 first authors, across eighteen Health Professions. Thirty-seven percent of papers used the term clinical reasoning; and 110 other terms referring to the concept of clinical reasoning were identified. Consensus on the categorization of terms was reached for 65 terms across six different categories: reasoning skills, reasoning performance, reasoning process, outcome of reasoning, context of reasoning, and purpose/goal of reasoning. Categories of terminology used differed across Health Professions and publication types. DISCUSSION: Many diverse terms were present and were used differently across literature contexts. These terms likely reflect different operationalisations, or conceptualizations, of clinical reasoning as well as the complex, multi-dimensional nature of this concept. We advise authors to make the intended meaning of 'clinical reasoning' and associated terms in their work explicit in order to facilitate teaching, assessment, and research communication.

A Scoping Review of Physicians' Clinical Reasoning in Emergency Departments.

STUDY OBJECTIVE: Clinical reasoning is considered a core competency of physicians. Yet there is a paucity of research on clinical reasoning specifically in emergency medicine, as highlighted in the literature. METHODS: We conducted a scoping review to examine the state of research on clinical reasoning in this specialty. Our team, composed of content and methodological experts, identified 3,763 articles in the literature, 95 of which were included. RESULTS: Most studies were published after 2000. Few studies focused on the cognitive processes involved in decisionmaking (ie, clinical reasoning). Of these, many confirmed findings from the general literature on clinical reasoning; specifically, the role of both intuitive and analytic processes. We categorized factors that influence decisionmaking into contextual, patient, and physician factors. Many studies focused on decisions in regard to investigations and admission. Test ordering is influenced by physicians' experience, fear of litigation, and concerns about malpractice. Fear of litigation and malpractice also increases physicians' propensity to admit patients. Context influences reasoning but findings pertaining to specific factors, such as patient flow and workload, were inconsistent. CONCLUSION: Many studies used designs such as descriptive or correlational methods, limiting the strength of findings. Many gray areas persist, in which studies are either scarce or yield conflicting results. The findings of this scoping review should encourage us to intensify research in the field of emergency physicians' clinical reasoning, particularly on the cognitive processes at play and the factors influencing them, using appropriate theoretical frameworks and more robust methods.

Scoping reviews in health professions education: challenges, considerations and lessons learned about epistemology and methodology.

Scoping reviews are increasingly used in health professions education to synthesize research and scholarship, and to report on the depth and breadth of the literature on a given topic. In this Perspective, we argue that the philosophical stance scholars adopt during the execution of a scoping review, including the meaning they attribute to fundamental concepts such as knowledge and evidence, influences how they gather, analyze, and interpret information obtained from a heterogeneous body of literature. We highlight the principles informing scoping reviews and outline how epistemology-the aspect of philosophy that "deals with questions involving the nature of knowledge, the justification of beliefs, and rationality"-should guide methodological considerations, toward the aim of ensuring the production of a high-quality review with defensible and appropriate conclusions. To contextualize our claims, we illustrate some of the methodological challenges we have personally encountered while executing a scoping review on clinical reasoning and reflect on how these challenges could have been reconciled through a broader understanding of the methodology's philosophical foundation. We conclude with a description of lessons we have learned that might usefully inform other scholars who are considering undertaking a scoping review in their own domains of inquiry.

The terminology of clinical reasoning in health professions education: Implications and considerations.

Introduction: Clinical reasoning is considered to be at the core of health practice. Here, we report on the diversity and inferred meanings of the terms used to refer to clinical reasoning and consider implications for teaching and assessment. Methods: In the context of a Best Evidence Medical Education (BEME) review of 625 papers drawn from 18 health professions, we identified 110 terms for clinical reasoning. We focus on iterative categorization of these terms across three phases of coding and considerations for how terminology influences educational practices. Results: Following iterative coding with 5 team members, consensus was possible for 74, majority coding was possible for 16, and full team disagreement existed for 20 terms. Categories of terms included: purpose/goal of reasoning, outcome of reasoning, reasoning performance, reasoning processes, reasoning skills, and context of reasoning. Discussion: Findings suggest that terms used in reference to clinical reasoning are non-synonymous, not uniformly understood, and the level of agreement differed across terms. If the language we use to describe, to teach, or to assess clinical reasoning is not similarly understood across clinical teachers, program directors, and learners, this could lead to confusion regarding what the educational or assessment targets are for "clinical reasoning."

Rheumatoid Meningitis Presenting With Acute Parkinsonism and Protracted Non-convulsive Seizures: An Unusual Case Presentation and Review of Treatment Strategies.

Rheumatoid meningitis is a rare complication of rheumatoid arthritis (RA). It is associated with substantial morbidity and mortality. The condition may present in a variety of ways and is therefore diagnostically challenging. Uncertainty still exists regarding the optimal treatment strategy. Herein, we describe the case of a 74-year-old man with a history of well-controlled seropositive RA on low-dose prednisone, hydroxychloroquine, and methotrexate. The patient presented with a several-month history of multiple prolonged episodes of expressive aphasia, right hemiparesis, and encephalopathy. Although no epileptiform activity was recorded on repeated electroencephalography, the symptoms fully resolved following treatment with antiepileptic drugs. He subsequently developed acute asymmetrical parkinsonism of the right hemibody. Magnetic resonance imaging revealed subtle enhancement of the leptomeninges over the left frontoparietal convexity. Cerebrospinal fluid analysis revealed a mild lymphocytic pleocytosis and elevated proteins. Histopathologic analysis of a meningeal biopsy revealed nodular rheumatoid meningitis. The patient was treated with corticosteroids and cyclophosphamide, following which he incompletely recovered. This is the first description of rheumatoid meningitis manifesting with acute parkinsonism and protracted non-convulsive seizures. A summary of cases reported since 2005, including data on pathology, therapy and outcomes, along with a discussion on the efficacy of different treatment strategies are provided.

Clinical Reasoning Assessment Methods: A Scoping Review and Practical Guidance.

PURPOSE: An evidence-based approach to assessment is critical for ensuring the development of clinical reasoning (CR) competence. The wide array of CR assessment methods creates challenges for selecting assessments fit for the purpose; thus, a synthesis of the current evidence is needed to guide practice. A scoping review was performed to explore the existing menu of CR assessments. METHOD: Multiple databases were searched from their inception to 2016 following PRISMA guidelines. Articles of all study design types were included if they studied a CR assessment method. The articles were sorted by assessment methods and reviewed by pairs of authors. Extracted data were used to construct descriptive appendixes, summarizing each method, including common stimuli, response formats, scoring, typical uses, validity considerations, feasibility issues, advantages, and disadvantages. RESULTS: A total of 377 articles were included in the final synthesis. The articles broadly fell into three categories: non-workplace-based assessments (e.g., multiple-choice questions, extended matching questions, key feature examinations, script concordance tests); assessments in simulated clinical environments (objective structured clinical examinations and technology-enhanced simulation); and workplace-based assessments (e.g., direct observations, global assessments, oral case presentations, written notes). Validity considerations, feasibility issues, advantages, and disadvantages differed by method. CONCLUSIONS: There are numerous assessment methods that align with different components of the complex construct of CR. Ensuring competency requires the development of programs of assessment that address all components of CR. Such programs are ideally constructed of complementary assessment methods to account for each method's validity and feasibility issues, advantages, and disadvantages.

Lessons learned from a student-driven initiative to design and implement an Organ and Tissue Donation course across Canadian medical schools.

The competencies required of the well-trained physician are constantly evolving, and medical education must adapt accordingly. In response, a growing number of influential medical education licensing and accreditation bodies have proposed frameworks that outline society's expectations of physician competencies. In Canada, undergraduate and graduate curricula have undergone major changes to meet the specifications of the CanMEDS framework, and similar efforts are underway internationally. Nonetheless, ensuring the values enshrined within such frameworks become integral to a physician's identity remains challenging. We believe that student-led curricular initiatives represent a novel way of approaching this shifting medical education landscape.In this article, we reflect on lessons we learned as medical students spearheading an initiative to change how organ and tissue donation is taught in Canadian medical schools. Citing relevant medical education literature where applicable, we include a detailed description of our approach as a roadmap for students contemplating their own curricular innovations. By outlining the factors influencing this project's implementation, as well as the benefits and limitations of student participation in curriculum reform, we offer educators a fresh perspective on optimizing the student role in this important process. Ultimately, the authors argue that not only can student participation render curricular content more accessible to learners, but that the responsibilities students take on in this role naturally lead to the development of CanMEDs-based competencies such as advocacy, scholarship, and inter-professionalism.

Examining the effects of gaming and guessing on script concordance test scores.

INTRODUCTION: In a script concordance test (SCT), examinees are asked to judge the effect of a new piece of clinical information on a proposed hypothesis. Answers are collected using a Likert-type scale (ranging from -2 to +2, with '0' indicating no effect), and compared with those of a reference panel of 'experts'. It has been argued, however, that SCT may be susceptible to the influences of gaming and guesswork. This study aims to address some of the mounting concern over the response process validity of SCT scores. METHOD: Using published datasets from three independent SCTs, we investigated examinee response patterns, and computed the score a hypothetical examinee would obtain on each of the tests if he 1) guessed random answers and 2) deliberately answered '0' on all test items. RESULTS: A simulated random guessing strategy led to scores 2 SDs below mean scores of actual respondents (Z-scores -3.6 to -2.1). A simulated 'all-0' strategy led to scores at least 1 SD above those obtained by random guessing (Z-scores -2.2 to -0.7). In one dataset, stepwise exclusion of items with modal panel response '0' to fewer than 10% of the total number of test items yielded hypothetical scores 2 SDs below mean scores of actual respondents. DISCUSSION: Random guessing was not an advantageous response strategy. An 'all-0' response strategy, however, demonstrated evidence of artificial score inflation. Our findings pose a significant threat to the SCT's validity argument. 'Testwiseness' is a potential hazard to all testing formats, and appropriate countermeasures must be established. We propose an approach that might be used to mitigate a potentially real and troubling phenomenon in script concordance testing. The impact of this approach on the content validity of SCTs merits further discussion.

Drawing Boundaries: The Difficulty in Defining Clinical Reasoning.

Clinical reasoning is an essential component of a health professional's practice. Yet clinical reasoning research has produced a notably fragmented body of literature. In this article, the authors describe the pause-and-reflect exercise they undertook during the execution of a synthesis of the literature on clinical reasoning in the health professions. Confronted with the challenge of establishing a shared understanding of the nature and relevant components of clinical reasoning, members of the review team paused to independently generate their own personal definitions and conceptualizations of the construct. Here, the authors describe the variability of definitions and conceptualizations of clinical reasoning present within their own team. Drawing on an analogy from mathematics, they hypothesize that the presence of differing "boundary conditions" could help explain individuals' differing conceptualizations of clinical reasoning and the fragmentation at play in the wider sphere of research on clinical reasoning. Specifically, boundary conditions refer to the practice of describing the conditions under which a given theory is expected to hold, or expected to have explanatory power. Given multiple theoretical frameworks, research methodologies, and assessment approaches contained within the clinical reasoning literature, different boundary conditions are likely at play. Open acknowledgment of different boundary conditions and explicit description of the conceptualization of clinical reasoning being adopted within a given study would improve research communication, support comprehensive approaches to teaching and assessing clinical reasoning, and perhaps encourage new collaborative partnerships among researchers who adopt different boundary conditions.

Knowledge Syntheses in Medical Education: Demystifying Scoping Reviews.

An unprecedented rise in health professions education (HPE) research has led to increasing attention and interest in knowledge syntheses. There are many different types of knowledge syntheses in common use, including systematic reviews, meta-ethnography, rapid reviews, narrative reviews, and realist reviews. In this Perspective, the authors examine the nature, purpose, value, and appropriate use of one particular method: scoping reviews. Scoping reviews are iterative and flexible and can serve multiple main purposes: to examine the extent, range, and nature of research activity in a given field; to determine the value and appropriateness of undertaking a full systematic review; to summarize and disseminate research findings; and to identify research gaps in the existing literature. Despite the advantages of this methodology, there are concerns that it is a less rigorous and defensible means to synthesize HPE literature. Drawing from published research and from their collective experience with this methodology, the authors present a brief description of scoping reviews, explore the advantages and disadvantages of scoping reviews in the context of HPE, and offer lessons learned and suggestions for colleagues who are considering conducting scoping reviews. Examples of published scoping reviews are provided to illustrate the steps involved in the methodology.

Crossing boundaries in interprofessional education: A call for instructional integration of two script concepts.

Clinical work occurs in a context which is heavily influenced by social interactions. The absence of theoretical frameworks underpinning the design of collaborative learning has become a roadblock for interprofessional education (IPE). This article proposes a script-based framework for the design of IPE. This framework provides suggestions for designing learning environments intended to foster competences we feel are fundamental to successful interprofessional care. The current literature describes two script concepts: "illness scripts" and "internal/external collaboration scripts". Illness scripts are specific knowledge structures that link general disease categories and specific examples of diseases. "Internal collaboration scripts" refer to an individual's knowledge about how to interact with others in a social situation. "External collaboration scripts" are instructional scaffolds designed to help groups collaborate. Instructional research relating to illness scripts and internal collaboration scripts supports (a) putting learners in authentic situations in which they need to engage in clinical reasoning, and (b) scaffolding their interaction with others with "external collaboration scripts". Thus, well-established experiential instructional approaches should be combined with more fine-grained script-based scaffolding approaches. The resulting script-based framework offers instructional designers insights into how students can be supported to develop the necessary skills to master complex interprofessional clinical situations.