An exploration of values in medical school admissions processes: the interplay between contextual factors, admissions practices, and validity.

Medical school admissions is a contentious and high stakes selection activity. Many assessment approaches are available to support selection; but how are decisions about building, monitoring, and adapting admissions systems made? What shapes the processes and practices that underpin selection decisions? We explore how these decisions are made across several Canadian medical schools, and how values shape the creation, monitoring, and adaptation of admissions systems. Using phenomenography (a qualitative method suited to examining variability), the authors analyzed interviews with 10 current or previous heads of admissions from 10 different undergraduate medical education programs in Canada. Interviews were conducted in English and French, and data was collected between 2016 and 2017 (therefore participants no longer hold these roles). Data was coded and analyzed iteratively, focusing on identifying underlying values, and exploring how these values shape admissions practices and considerations for validity. Eight different intersecting values were identified. Of these, four were shared across all participants: critically questioning the process and tools, aiming for equity, striving for better, and embracing the challenges of change. The expression of these values depended on different contextual variables (e.g., geographic location, access to expertise, resource availability), and values shaped how admissions systems were built, enacted, and monitored for quality. Ultimately, values shaped: (1) admissions practices resulting in different candidates being offered admission, and (2) how arguments supporting score interpretation are built (i.e., validity). This study documents various values that influence admissions processes, practices, and quality monitoring. The values that shape what is assessed, how it is assessed, and how fair and defensible practices are conceptualized have significant impact, ultimately determining who is selected. These values-whether implicit or explicit-result in intended and unintended consequences in selection processes. However, these values are rarely explicitly examined and questioned, leaving it uncertain as to which consequences are the intended outcomes of deliberately chosen values, and which are unintended consequences of implicitly held values of admissions systems and their actors.

Digging Deeper, Zooming Out: Reimagining Legacies in Medical Education.

Although the wide-scale disruption precipitated by the COVID-19 pandemic has somewhat subsided, there are many questions about the implications of such disruptions for the road ahead. This year's Research in Medical Education (RIME) supplement may provide a window of insight. Now, more than ever, researchers are poised to question long-held assumptions while reimagining long-established legacies. Themes regarding the boundaries of professional identity, approaches to difficult conversations, challenges of power and hierarchy, intricacies of selection processes, and complexities of learning climates appear to be the most salient and critical to understand. In this commentary, the authors use the relationship between legacies and assumptions as a framework to gain a deeper understanding about the past, present, and future of RIME.

Action-project method: An approach to describing and studying goal-oriented joint actions.

CONTEXT: Practicing health professionals and educators frequently act together in an interdependent or joint capacity to reach goals. Teaching or learning a new skill or engaging with patients in shared decision-making exemplifies this joint and goal-directed nature of Health Professions Education (HPE) and practice. However, building a robust understanding of the complexity of action, and joint action in particular, in HPE or patient care remains a challenge because of a limited number of methodologies available within HPE research. METHODS: In this manuscript, we describe the Action-Project Method (A-PM) as a qualitative research approach that can be used to describe and understand goal-directed joint actions. A-PM is grounded in contextual action theory and is a methodology focussed on action as an object of study, as it is occurring. A-PM uses three distinct perspectives to understand goal-directed joint actions: observable behaviour, internal processes (i.e. reported thoughts and feelings) and the social meaning reflected in goals. Data collection in A-PM involves observations, interviews, recording of actions and a self-confrontation procedure-where participants watch video-recorded segments of action and reflect on their internal processes, describing what they were thinking or feeling as they were completing the action. Together, the rich data generated and the layered approach to analysis provide a means to better understand the joint actions embedded in complex systems and collaborative work. Furthermore, the participants are treated as equal partners within A-PM, ensuring data equity even when the research context includes hierarchical relationships. DISCUSSION: Given increasing recognition to the importance of teamwork, relationships, interdependence, complex environments and centring patient or learner voices, A-PM is a valuable research approach for HPE. A-PM deepens our research arsenal with an approach that focusses on interdependent dyads or teams and provides a deeper understanding for how individuals engage together in goal-oriented actions.

Can I Ask a Question About URiM Awards That I Don't Know the Answer to? Designing an Award for Underrepresented Medical Education Researchers.

Meaningful Equity, Diversity, and Inclusion (EDI) efforts may be stymied by concerns about whether proposed initiatives are performative or tokenistic. The purpose of this project was to analyze discussions by the Research in Medical Education (RIME) Program Planning committee about how best to recognize and support underrepresented in medicine (URiM) researchers in medical education to generate lessons learned that might inform local, national, and international actions to implement meaningful EDI initiatives. Ten RIME Program Planning Committee members and administrative staff participated in a focus group held virtually in August 2021. Focus group questions elicited opinions about "if and how" to establish a URiM research award. The focus group was recorded, transcribed, and thematically analyzed. Recognition of privilege, including who has it and who doesn't, underpinned the focus group discussion, which revolved around 2 themes: (1) tensions between optics and semantics, and (2) potential unintended consequences of trying to level the medical education playing field. The overarching storyline threaded throughout the focus group discussion was intentionality. Focus group participants sought to avoid performativity by creating an award that would be meaningful to recipients and to career gatekeepers such as department chairs and promotion and tenure committees. Ultimately, participants decided to create an award that focused on exemplary Equity, Diversity, and Inclusion (EDI) scholarship, which was eventually named the "RIME URiM Research Award." Difficult but productive conversations about EDI initiatives are necessary to advance underrepresented in medicine (URiM) scholarship. This transparent commentary may trigger further critical conversations.

Mapping clinical reasoning literature across the health professions: a scoping review.

BACKGROUND: Clinical reasoning is at the core of health professionals' practice. A mapping of what constitutes clinical reasoning could support the teaching, development, and assessment of clinical reasoning across the health professions. METHODS: We conducted a scoping study to map the literature on clinical reasoning across health professions literature in the context of a larger Best Evidence Medical Education (BEME) review on clinical reasoning assessment. Seven databases were searched using subheadings and terms relating to clinical reasoning, assessment, and Health Professions. Data analysis focused on a comprehensive analysis of bibliometric characteristics and the use of varied terminology to refer to clinical reasoning. RESULTS: Literature identified: 625 papers spanning 47 years (1968-2014), in 155 journals, from 544 first authors, across eighteen Health Professions. Thirty-seven percent of papers used the term clinical reasoning; and 110 other terms referring to the concept of clinical reasoning were identified. Consensus on the categorization of terms was reached for 65 terms across six different categories: reasoning skills, reasoning performance, reasoning process, outcome of reasoning, context of reasoning, and purpose/goal of reasoning. Categories of terminology used differed across Health Professions and publication types. DISCUSSION: Many diverse terms were present and were used differently across literature contexts. These terms likely reflect different operationalisations, or conceptualizations, of clinical reasoning as well as the complex, multi-dimensional nature of this concept. We advise authors to make the intended meaning of 'clinical reasoning' and associated terms in their work explicit in order to facilitate teaching, assessment, and research communication.

Postpositivism in Health Professions Education Scholarship.

An understanding of the diversity of perspectives within the research paradigms of health professions education (HPE) is essential for rigorous research design and more purposeful engagement with the contributions of others. In this article, the authors explicitly discuss the underlying assumptions, notions of good scholarship, and shortcomings of the postpositivism research paradigm. While postpositivism is likely one of the more familiar paradigms within HPE research, it is rarely formally or explicitly described. Drawing on key literature and contemporary examples, the authors describe the ontology, epistemology, methodologies, axiology, signs of rigor, and common critiques of postpositivism. A case study provides the focus for a practical illustration of how a postpositivist approach to education research could be applied. Suggestions for further reading are provided for those who are keen to delve deeper into the history and key tenants of the postpositivist stance.

Scoping reviews in health professions education: challenges, considerations and lessons learned about epistemology and methodology.

Scoping reviews are increasingly used in health professions education to synthesize research and scholarship, and to report on the depth and breadth of the literature on a given topic. In this Perspective, we argue that the philosophical stance scholars adopt during the execution of a scoping review, including the meaning they attribute to fundamental concepts such as knowledge and evidence, influences how they gather, analyze, and interpret information obtained from a heterogeneous body of literature. We highlight the principles informing scoping reviews and outline how epistemology-the aspect of philosophy that "deals with questions involving the nature of knowledge, the justification of beliefs, and rationality"-should guide methodological considerations, toward the aim of ensuring the production of a high-quality review with defensible and appropriate conclusions. To contextualize our claims, we illustrate some of the methodological challenges we have personally encountered while executing a scoping review on clinical reasoning and reflect on how these challenges could have been reconciled through a broader understanding of the methodology's philosophical foundation. We conclude with a description of lessons we have learned that might usefully inform other scholars who are considering undertaking a scoping review in their own domains of inquiry.

Facilitating admissions of diverse students: A six-point, evidence-informed framework for pipeline and program development.

INTRODUCTION: Several national level calls have encouraged reconsideration of diversity issues in medical education. Particular interest has been placed on admissions, as decisions made here shape the nature of the future physician workforce. Critical analysis of current practices paired with evidence-informed policies may counter some of the barriers impeding access for underrepresented groups. METHODS: We present a framework for diversity-related program development and evaluation grounded within a knowledge translation framework, and supported by the initiation of longitudinal collection of diversity-related data. We provide an illustrative case study for each component of the framework. Descriptive analyses are presented of pre/post intervention diversity metrics if applicable and available. RESULTS: The framework's focal points are: 1) data-driven identification of underrepresented groups, 2) pipeline development and targeted recruitment, 3) ensuring an inclusive process, 4) ensuring inclusive assessment, 5) ensuring inclusive selection, and 6) iterative use of diversity-related data. Case studies ranged from wording changes on admissions websites to the establishment of educational and administrative offices addressing needs of underrepresented populations. CONCLUSIONS: We propose that diversity-related data must be collected on a variety of markers, developed in partnership with stakeholders who are most likely to facilitate implementation of best practices and new policies. These data can facilitate the design, implementation, and evaluation of evidence-informed diversity initiatives and provide a structure for continued investigation into 'interventions' supporting diversity-related initiatives.

Knowledge Syntheses in Medical Education: Demystifying Scoping Reviews.

An unprecedented rise in health professions education (HPE) research has led to increasing attention and interest in knowledge syntheses. There are many different types of knowledge syntheses in common use, including systematic reviews, meta-ethnography, rapid reviews, narrative reviews, and realist reviews. In this Perspective, the authors examine the nature, purpose, value, and appropriate use of one particular method: scoping reviews. Scoping reviews are iterative and flexible and can serve multiple main purposes: to examine the extent, range, and nature of research activity in a given field; to determine the value and appropriateness of undertaking a full systematic review; to summarize and disseminate research findings; and to identify research gaps in the existing literature. Despite the advantages of this methodology, there are concerns that it is a less rigorous and defensible means to synthesize HPE literature. Drawing from published research and from their collective experience with this methodology, the authors present a brief description of scoping reviews, explore the advantages and disadvantages of scoping reviews in the context of HPE, and offer lessons learned and suggestions for colleagues who are considering conducting scoping reviews. Examples of published scoping reviews are provided to illustrate the steps involved in the methodology.

Recruitment manoeuvres for adults with acute respiratory distress syndrome receiving mechanical ventilation.

BACKGROUND: Recruitment manoeuvres involve transient elevations in airway pressure applied during mechanical ventilation to open ('recruit') collapsed lung units and increase the number of alveoli participating in tidal ventilation. Recruitment manoeuvres are often used to treat patients in intensive care who have acute respiratory distress syndrome (ARDS), but the effect of this treatment on clinical outcomes has not been well established. This systematic review is an update of a Cochrane review originally published in 2009. OBJECTIVES: Our primary objective was to determine the effects of recruitment manoeuvres on mortality in adults with acute respiratory distress syndrome.Our secondary objective was to determine, in the same population, the effects of recruitment manoeuvres on oxygenation and adverse events (e.g. rate of barotrauma). SEARCH METHODS: For this updated review, we searched the Cochrane Central Register of Controlled Trials (CENTRAL), MEDLINE (OVID), Embase (OVID), the Cumulative Index to Nursing and Allied Health Literature (CINAHL, EBSCO), Latin American and Caribbean Health Sciences (LILACS) and the International Standard Randomized Controlled Trial Number (ISRCTN) registry from inception to August 2016. SELECTION CRITERIA: We included randomized controlled trials (RCTs) of adults who were mechanically ventilated that compared recruitment manoeuvres versus standard care for patients given a diagnosis of ARDS. DATA COLLECTION AND ANALYSIS: Two review authors independently assessed trial quality and extracted data. We contacted study authors for additional information. MAIN RESULTS: Ten trials met the inclusion criteria for this review (n = 1658 participants). We found five trials to be at low risk of bias and five to be at moderate risk of bias. Six of the trials included recruitment manoeuvres as part of an open lung ventilation strategy that was different from control ventilation in aspects other than the recruitment manoeuvre (such as mode of ventilation, higher positive end-expiratory pressure (PEEP) titration and lower tidal volume or plateau pressure). Six studies reported mortality outcomes. Pooled data from five trials (1370 participants) showed a reduction in intensive care unit (ICU) mortality (risk ratio (RR) 0.83, 95% confidence interval (CI) 0.72 to 0.97, P = 0.02, low-quality evidence), pooled data from five trials (1450 participants) showed no difference in 28-day mortality (RR 0.86, 95% CI 0.74 to 1.01, P = 0.06, low-quality evidence) and pooled data from four trials (1313 participants) showed no difference in in-hospital mortality (RR 0.88, 95% CI 0.77 to 1.01, P = 0.07, low-quality evidence). Data revealed no differences in risk of barotrauma (RR 1.09, 95% CI 0.78 to 1.53, P = 0.60, seven studies, 1508 participants, moderate-quality evidence). AUTHORS' CONCLUSIONS: We identified significant clinical heterogeneity in the 10 included trials. Results are based upon the findings of several (five) trials that included an "open lung ventilation strategy", whereby the intervention group differed from the control group in aspects other than the recruitment manoeuvre (including co-interventions such as higher PEEP, different modes of ventilation and higher plateau pressure), making interpretation of the results difficult. A ventilation strategy that included recruitment manoeuvres in participants with ARDS reduced intensive care unit mortality without increasing the risk of barotrauma but had no effect on 28-day and hospital mortality. We downgraded the quality of the evidence to low, as most of the included trials provided co-interventions as part of an open lung ventilation strategy, and this might have influenced results of the outcome.

Objective assessment of Myringotomy and tympanostomy tube insertion: A prospective single-blinded validation study.

OBJECTIVES/HYPOTHESIS: Despite the transition to competency-based education in surgery, few standardized assessment tools exist in otolaryngology training. In particular, myringotomy and tympanostomy tube insertion (M+T) is a common surgical procedure with few validated assessment tools available. Our objectives were to develop an objective structured assessment of operative skills in M+T and to provide validity evidence for the developed assessment tool within otolaryngology training. STUDY DESIGN: Prospective study involving the evaluation of an assessment tool. METHODS: Through consultation with a panel of experts in otolaryngology and medical education we developed a Task-Specific Checklist and Global Rating Scale for M+T. Postgraduate year 2 junior residents, postgraduate year 3 senior residents, and attending otolaryngologists were video recorded performing M+T at a tertiary care pediatric hospital. The videos were subsequently reviewed and independently evaluated by three blinded raters from an unaffiliated academic institution. RESULTS: The average score of junior residents, senior residents, and attending otolaryngologists using the Task-Specific Checklist was 21.7/30 (±7.1), 26.3/30 (±3.5), and 27.3/30 (±6.2), respectively (P = .04). For the Global Rating Scale, the scores for junior residents, senior residents, and attending surgeons were 27.7/50 (±11.2), 34.5/50 (±9.5), and 45.1/50 (±4.6), respectively (P < .001). The inter-rater and intrarater reliability were both above 0.88. CONCLUSIONS: The Task-Specific Checklist and Global Rating Scale for M+T appear reliable, with validity evidence supporting their use in otolaryngology training. LEVEL OF EVIDENCE: NA Laryngoscope, 126:2140-2146, 2016.

Candidates' and interviewers' perceptions of multiple-mini interviews for admission to an occupational therapy professional program.

Multiple-Mini Interviews (MMIs) were used to assess professional attributes of candidates seeking admission to an occupational therapy professional entry-level master's program. Candidates and interviewers were invited to complete a questionnaire comprised of quantitative and open-ended questions following the MMIs. The MMIs were perceived to be fair, enjoyable, and capable of capturing professional attributes. Descriptive analysis of candidates' data revealed perceptions regarding logistics, interview station content, process, and interviewers. Interviewers commented on the positive and challenging aspects of the scenarios and the MMI process. Admissions committees need to consider several logistical, content, and process issues when designing and implementing MMIs as a selection tool.

Disrupting diagnostic reasoning: do interruptions, instructions, and experience affect the diagnostic accuracy and response time of residents and emergency physicians?

PURPOSE: Others have suggested that increased time pressure, sometimes caused by interruptions, may result in increased diagnostic errors. The authors previously found, however, that increased time pressure alone does not result in increased errors, but they did not test the effect of interruptions. It is unclear whether experience modulates the combined effects of time pressure and interruptions. This study investigated whether increased time pressure, interruptions, and experience level affect diagnostic accuracy and response time. METHOD: In October 2012, 152 residents were recruited at five Medical Council of Canada Qualifying Examination Part II test sites. Forty-six emergency physicians were recruited from one Canadian and one U.S. academic health center. Participants diagnosed 20 written general medicine cases. They were randomly assigned to receive fast (time pressure) or slow condition instructions. Visual and auditory case interruptions were manipulated as a within-subject factor. RESULTS: Diagnostic accuracy was not affected by interruptions or time pressure but was related to experience level: Emergency physicians were more accurate (71%) than residents (43%) (F = 234.0, P < .0001) and responded more quickly (54 seconds) than residents (65 seconds) (F = 9.0, P < .005). Response time was shorter for participants in the fast condition (55 seconds) than in the slow condition (73 seconds) (F = 22.2, P < .0001). Interruptions added about 8 seconds to response time. CONCLUSIONS: Experienced emergency physicians were both faster and more accurate than residents. Instructions to proceed quickly and interruptions had a small effect on response time but no effect on accuracy.

The benefits of testing for learning on later performance.

Testing has been shown to enhance retention of learned information beyond simple studying, a phenomena known as test-enhanced learning (TEL). Research has shown that TEL effects are greater for tests that require the production of responses [e.g., short-answer questions (SAQs)] relative to tests that require the recognition of correct answers [e.g., multiple-choice questions (MCQs)]. High stakes licensure examinations have recently differentiated MCQs that require the application of clinical knowledge (context-rich MCQs) from MCQs that rely on the recognition of "facts" (context-free MCQs). The present study investigated the influence of different types of educational activities (including studying, SAQs, context-rich MCQs and context-free MCQs) on later performance on a mock licensure examination. Fourth-year medical students (n = 224) from four Quebec universities completed four educational activities: one reading-based activity and three quiz-based activities (SAQs, context-rich MCQs, and context-free MCQs). We assessed the influence of the type of educational activity on students' subsequent performance in a mock licensure examination, which consisted of two types of context-rich MCQs: (1) verbatim replications of previous items and (2) items that tested the same learning objective but were new. Mean accuracy scores on the mock licensure exam were higher when intervening educational activities contained either context-rich MCQs (Mean z-score = 0.40) or SAQs (M = 0.39) compared to context-free MCQs (M = -0.38) or study only items (M = -0.42; all p < 0.001). Higher mean scores were only present for verbatim items (p < 0.001). The benefit of testing was observed when intervening educational activities required either the generation of a response (SAQs) or the application of knowledge (context-rich MCQs); however, this effect was only observed for verbatim test items. These data provide evidence that context-rich MCQs and SAQs enhance learning through testing compared to context-free MCQs or studying alone. The extent to which these findings generalize beyond verbatim questions remains to be seen.