Education Scholarship Assessment Reconsidered: Expansion of Glassick's Criteria to Incorporate Health Equity.

ABSTRACT: Recent events have ignited widespread attention to structural racism and implicit bias throughout the U.S. health care system and medical institutions, resulting in a call for antiracism approaches to advance health equity. Medical education leaders are well positioned to advance health equity, not only through their training of fellows, residents, and medical students, but also in their approach to scholarship. Education scholarship drives innovation and critical evaluation of current practices; it impacts and intersects with multiple factors that have the potential to reduce health inequities. Thus, it is critical to prioritize the assessment of education scholarship through a health equity lens. Medical education scholarly dissemination has markedly expanded over the past 2 to 3 decades, yet medical educators have continued to embrace Boyer's and Glassick and colleagues' definitions of scholarship. The authors propose an approach to medical education scholarship assessment that expands each of Glassick's 6 existing criteria to address health inequities and adds health equity as a seventh criterion. With this, medical educators, researchers, reviewers, and others can consider how education scholarship affects diverse populations and settings, direct educational products and scholarship to address health inequities, and raise the importance of advancing health equity in medical education scholarship. By expanding and standardizing the assessment of scholarship to incorporate health equity, the medical education community can foster a cultural shift that brings health equity to the forefront of education scholarship.

Promotion Criteria for Medical Educators: Are We Climbing a Ladder with Invisible Rungs?

OBJECTIVE: In 2006 the Association of American Medical Colleges recommended standardization of documentation of the contributions of medical educators and guidelines for their academic promotion. The authors characterized current United States (US) medical school promotion guidelines for medical educators. METHODS: Authors collected publicly available data from medical school promotion websites from March through July 2022 after determining categories by traditional-set domains as well as peer-reviewed standards. Extracted data were analyzed using descriptive and inferential statistics, and frequencies were calculated for nominal and categorical data. RESULTS: Of 155 medical schools identified, promotion criteria were publicly available for 143 (92%) schools. Ninety-one (64%) schools identified a distinct educator track. Of those with a defined educator track, 44 (48%) schools consider workshops or other media when evaluating candidates for promotion, and only 52 (57%) of schools with a specified educational track require additional documentation of teaching or education as part of their promotion process. Notably, 34 (37%) of the 91 schools with an educator track specifically require an Educational Portfolio, compared to 27 (52%) of the 52 schools that do not have a specific educator track for promotion. CONCLUSION: This study describes the current lack of clarity and consistency of the promotion criteria for medical educators and indicates that the guidelines proposed by the Association of American Medical Colleges over 15 years ago have not been widely adopted. These data amplify previous calls for a more objective set of criteria for evaluating and recognizing the contributions of medical educators.

The ABCs of OTCs: A Video-Based Curriculum Regarding Over-the-Counter Pediatric Products.

Introduction: Over-the-counter (OTC) products are widely used by families with young children. To educate future pediatricians on OTC product counseling and support the health and safety of children under their care, modern, accessible, and engaging curricula are needed. Methods: We developed an OTC product curriculum consisting of seven videos and one facilitated group discussion using a flipped classroom pedagogy to educate students on counseling parents about OTC product use. Fourth-year medical students pursuing pediatric training from four institutions participated in the curriculum during their end-of-year transition-to-residency course. We measured effectiveness via a pre/post comparison using a student self-assessment with multiple-choice questions. A simulated parent call OSCE provided participants with an opportunity to apply their knowledge and receive directed formative feedback. Data were analyzed using descriptive and inferential statistics. Results: A total of 41 students participated in the curriculum and completed all assessments. The majority (93%) watched all the videos. All participants (100%) agreed the videos were useful. Knowledge improved significantly (pretest mean score = 70%, posttest mean score = 87%, p < .001). No significant differences were found when comparing institution, gender, prior experience, or electives. Discussion: We developed a feasible and effective video-based curriculum to teach OTC product guidance. Given the importance of discussing OTC medications with families and the need for convenient educational tools, this curriculum may have widespread application to medical students during clinical rotations as well as pediatric and family medicine trainees.

Fidelity to Best Practices in EPA Implementation: Outcomes Supporting Use of the Core Components Framework From the University of Virginia Entrustable Professional Activity Program.

PROBLEM: The rapid expansion of entrustable professional activity (EPA) assessment programs has led to calls to ensure fidelity in implementation and integrity in meeting the goals of competency-based medical education. Initiated in July 2017, in advance of the articulated core components of EPA implementation, this article describes the structure and outcomes of the University of Virginia (UVA) EPA Program and provides support for the identified essential components. APPROACH: The UVA EPA Program includes workplace assessments by residents/fellows, attending faculty, and master assessors (MAs), experienced clinicians who assess students across disciplines and clinical settings. All assessors participate in formal professional development and provide verbal and written comments to support their supervision ratings. The Entrustment Committee, composed of 12 MAs, uses a shared mental model and aggregates all assessor data to make a high-stakes summative entrustment decision about students' readiness to assume the role of an acting intern. OUTCOMES: Since 2017, over 2,000 assessors have completed 56,969 EPA assessments for 1,479 students. Ninety-four percent of assessments have been done during the clerkship phase. Residents/fellows have completed a mean of 18 assessments, attending faculty a mean of 27, and MAs a mean of 882. Seventy-four percent of observed encounters involved patients with acute concerns with or without a co-morbid condition. Fifty percent of assessments occurred in inpatient and 32% in ambulatory settings. Eighty-seven percent of assessments contained narrative comments with more than 100 characters. NEXT STEPS: Planned next steps will include earlier identification of students who require individualized learning to promote the development of skills related to EPAs, expansion of the remediation program to enable more students to engage in a clinical performance mastery elective, and creation of targeted professional development for assessors to reinforce the tenets of the EPA program.

Medical School to Residency: How Can We Trust the Process?

Background To say that the transition from undergraduate medical education (UME) to graduate medical education (GME) is under scrutiny would be an understatement. Findings from a panel discussion at the 2018 Association of American Medical Colleges Annual meeting entitled, "Pass-Fail in Medical School and the Residency Application Process and Graduate Medical Education Transition" addressed what and when information should be shared with residency programs, and how and when that information should be shared. Materials and Methods Over 250 participants representing UME and GME (e.g. leadership, faculty, medical students) completed worksheets addressing these questions. During report-back times, verbal comments were transcribed in real time, and written comments on worksheets were later transcribed. All comments were anonymous. Thematic analysis was conducted manually by the research team to analyze the worksheet responses and report back comments. Results Themes based on suggestions of what information should be shared included the following: 1) developmental/assessment benchmarks such as demonstrating the ability/competencies to do clinical work; 2) performance on examinations; 3) grades and class ranking; 4) 360 evaluations; 5) narrative evaluations; 6) failures/remediation/gaps in training; 7) professionalism lapses; 8) characteristics of students such as resiliency/reliability; and 9) service/leadership/participation. In terms of how this information should be shared, the participants suggested enhancements to the current process of transmitting documents rather than alternative methods (e.g., video, telephonic, face-to-face discussions) and information sharing at both the time of the match and again near/at graduation to include information about post-match rotations. Discussion Considerations to address concerns with the transition from medical school to residency include further enhancements to the Medical Student Performance Evaluation, viewing departmental letters as ones of evaluation and not recommendation, a more meaningful educational handoff, and limits on the number of residency applications allowed for each student. The current medical education environment is ready for meaningful change in the UME to GME transition.

Teaching pediatric otoscopy skills to the medical student in the clinical setting: preceptor perspectives and practice.

BACKGROUND: Acute otitis media (AOM) is the most frequent indication for antibiotic treatment of children in the United States. Its diagnosis relies on visualization of the tympanic membrane, a clinical skill acquired through a deliberate approach. Instruction in pediatric otoscopy begins in medical school. Medical students receive their primary experience with pediatric otoscopy during the required pediatric clerkship, traditionally relying on an immersion, apprentice-type learning model. A better understanding of their preceptors' clinical and teaching practices could lead to improved skill acquisition. This study investigates how pediatric preceptors (PP) and members of the Council on Medical Student Education in Pediatrics (COMSEP) perceive teaching otoscopy. METHODS: A 30-item online survey was administered to a purposeful sample of PP at six institutions in 2017. A comparable 23-item survey was administered to members through the 2018 COMSEP Annual Survey. Only COMSEP members who identified themselves as teaching otoscopy to medical students were asked to complete the otoscopy-related questions on the survey. RESULTS: Survey respondents included 58% of PP (180/310) and 44% (152/348) of COMSEP members. Forty-one percent (62/152) of COMSEP member respondents identified themselves as teaching otoscopy and completed the otoscopy-related questions. The majority agreed that standardized curricula are needed (PP 78%, COMSEP members 97%) and that all graduating medical students should be able to perform pediatric otoscopy (PP 95%, COMSEP members 79%). Most respondents reported usefulness of the American Academy of Pediatrics (AAP) AOM guidelines (PP 95%, COMSEP members 100%). More COMSEP members than PP adhered to the AAP's diagnostic criteria (pediatric preceptors 42%, COMSEP members 93%). The most common barriers to teaching otoscopy were a lack of assistive technology (PP 77%, COMSEP members 56%), presence of cerumen (PP 58%, COMSEP members 60%), time to teach in direct patient care (PP 46%, COMSEP members 48%), and parent anxiety (PP 62%, COMSEP members 54%). CONCLUSIONS: Our study identified systemic and individual practice patterns and barriers to teaching pediatric otoscopy. These results can inform education leaders in supporting and enabling preceptors in their clinical teaching. This approach can be adapted to ensure graduating medical students obtain intended core clinical skills.

Mitigating "Educational Groundhog Day" - The Role of Learner Handoffs Within Clinical Rotations: A Survey of Pediatric Educational Leaders.

BACKGROUND: Medical students decry frequent changes in faculty supervision, leading to the experience of "educational groundhog day." The discontinuity in supervision, cursory relationships, and uncoordinated feedback impede students' skill acquisition and delay entrustment decisions. Whereas patient handoff bundles are common, little is known about similarly structured approaches to learner handoffs (LHs). OBJECTIVE: To describe current LH procedures and practices within pediatric clerkships and subinternships and to gauge interest in a future LH bundle. METHODS: Nine items included in the 2016 Council on Medical Student Education in Pediatrics annual member survey were analyzed using mixed-methods. RESULTS: The response rates were 66% (101 of 152) and 40% (165 of 411) for institutions and individuals, respectively. After limiting data to complete responses to programs with traditional block rotations, 54% of individual respondents (76 of 141) identified as inpatient faculty and about a quarter endorsed providing LHs. Inpatient faculty most commonly supervise medical students for 5 to 7 days. Most endorsed needing 1 to 3 days to determine a student's baseline performance and 5 days or more to make entrustment decisions. Three-quarters of inpatient faculty endorsed interest in LHs, while fewer than 16% of course directors currently provide LH expectations. Four themes emerged: instrument features, stakeholder buy-in, impact, and utility. CONCLUSIONS: Typical inpatient faculty service days approximate the time required for making entrustment decisions about clinical students. While most inpatient faculty desire a LH bundle for use within a clinical rotation, few institutions and faculty currently use LHs. LHs could accelerate entrustment decisions by allowing coordinated feedback that might hasten learner clinical-skill development.

Moving Toward Summative Competency Assessment to Individualize the Postclerkship Phase.

In the move toward competency-based medical education, leaders have called for standardization of learning outcomes and individualization of the learning process. Significant progress has been made in establishing defined expectations for the knowledge, skills, attitudes, and behaviors required for successful transition to residency training, but individualization of educational processes to assist learners in reaching these competencies has been predominantly conceptual to date. The traditional time-based structure of medical education has posed a challenge to individualization within the curriculum and has led to more attention on innovations that facilitate transition from medical school to residency. However, a shift of focus to the clerkship-to-postclerkship transition point in the undergraduate curriculum provides an opportunity to determine how longitudinal competency-based assessments can be used to facilitate intentional and individualized structuring of the long-debated fourth year.This Perspective demonstrates how 2 institutions-the University of Virginia School of Medicine and the University of Michigan Medical School-are using competency assessments and applying standardized outcomes in decisions about individualization of the postclerkship learning process. One institution assesses Core Entrustable Professional Activities for Entering Residency, whereas the other has incorporated Accreditation Council for Graduate Medical Education core competencies and student career interests to determine degrees of flexibility in the postclerkship phase. Individualization in addition to continued assessment of performance presents an opportunity for intentional use of curriculum time to develop each student to be competently prepared for the transition to residency.

Defining Learning Communities in Undergraduate Medical Education: A National Study.

BACKGROUND: Learning communities (LCs) are intentionally designed groups that are actively engaged in learning with and from each other. While gaining prominence in US medical schools, LCs show significant variability in their characteristics across institutions, creating uncertainty about how best to measure their effects. OBJECTIVE: The aim of this study is to describe the characteristics of medical school LCs by primary purpose, structures, and processes and lay the groundwork for future outcome studies and benchmarking for best practices. METHODS: Medical school LC directors from programs affiliated with the Learning Communities Institute (LCI) were sent an online survey of program demographics and activities, and asked to upload a program description or summary of the LC's purpose, goals, and how it functions. Descriptive statistics were computed for survey responses and a qualitative content analysis was performed on program descriptions by 3 authors to identify and categorize emergent themes. RESULTS: Of 28 medical school LCs surveyed, 96% (27) responded, and 25 (89%) provided program descriptions for qualitative content analysis. All programs reported longitudinal relationships between students and faculty. Most frequently cited objectives were advising or mentoring (100%), professional development (96%), courses (96%), social activities (85%), and wellness (82%). Primary purpose themes were supporting students' professional development, fostering a sense of community, and creating a sense of wholeness. Structures included a community framework, subdivisions into smaller units, and governance by faculty and students. Process themes included longitudinal relationships, integrating faculty roles, and connecting students across class years. CONCLUSIONS: Medical school LCs represent a collection of high-impact educational practices characterized by community and small-group structures, relational continuity, and collaborative learning as a means to guide and holistically support students in their learning and development as physicians. In describing 27 medical school LCs, this study proposes a unifying framework to facilitate future educational outcomes studies across institutions.

Too Many Hats? Conflicts of Interest in Learning Community Faculty Roles.

PURPOSE: Many US medical schools have adopted learning communities to provide a framework for advising and teaching functions. Faculty who participate in learning communities often have additional educator roles. Defining potential conflicts of interest (COIs) among these roles is an important consideration for schools with existing learning communities and those looking to develop them, both for transparency with students and also to comply with regulatory requirements. METHODS: A survey was sent to the institutional contact for each of the 42 Learning Communities Institute (LCI) member medical schools to assess faculty opinions about what roles potentially conflict. The survey asked the role of learning community faculty in summative and formative assessment of students and whether schools had existing policies around COIs in medical education. RESULTS: In all, 35 (85%) LCI representatives responded; 30 (86%) respondents agreed or strongly agreed that learning community faculty should be permitted to evaluate their students for formative purposes, while 19 (54%) strongly agreed or agreed that learning community faculty should be permitted to evaluate their students in a way that contributes to a grade; 31 (89%) reported awareness of the accreditation standard ensuring "that medical students can obtain academic counseling from individuals who have no role in making assessment or promotion decisions about them," but only 10 (29%) had a school policy about COIs in education. There was a wide range of responses about what roles potentially conflict with being a learning community faculty. CONCLUSION: The potential for COIs between learning community faculty and other educator roles concerns faculty at schools with learning communities, but most schools have not formally addressed these concerns.

Developing Professionalism and Professional Identity Through Unproctored, Flexible Testing.

The medical education community has devoted a great deal of attention to the development of professionalism in trainees within the context of clinical training-particularly regarding trainees' handling of ethical dilemmas related to clinical care. The community, however, knows comparatively less about the development of professional behavior in medical students during the preclerkship years. In medical schools with flexible testing, students take quizzes or examinations in an unproctored setting at a time of their choosing-as long as it falls within a specified window of time. Unproctored, flexible testing offers students early opportunities to develop appropriate professional behavior. In this Perspective, the authors outline different flexible testing models from three institutions-University of Virginia School of Medicine, University of Michigan Medical School, and Icahn School of Medicine at Mount Sinai-all of which offer various levels of testing flexibility in relation to time and location. The authors' experiences with these models suggest that preclinical medical students' early development of professional behavior requires scaffolding by faculty and staff. Scaffolding involves setting clear, specific expectations for students (often through the form of an honor code), as well as active engagement and discussion with learners about the expectations and procedures for self-regulation in the academic environment.

Teaching Pediatric Otoscopy Skills to Pediatric and Emergency Medicine Residents: A Cross-Institutional Study.

OBJECTIVE: To evaluate a pediatric otoscopy curriculum with the use of outcome measures that included assessment of skills with real patients. METHODS: Thirty-three residents in an intervention group from 2 institutions received the curriculum. In the previous year, 21 residents in a nonintervention group did not receive the curriculum. Both groups were evaluated at the beginning and end of their internship years with the use of the same outcome assessments: 1) a written test, 2) an objective standardized clinical examination (OSCE), and 3) direct observation of skills in real patients with the use of a checklist with established validity. RESULTS: The intervention group had a significant increase in percentage reaching minimum passing levels between the beginning and end of the internship year for the written test (12% vs 97%; P < .001), OSCE (0% vs 78%; P < .001), and direct observation (0% vs 75%; P < .001); significant mean percentage gains for the written test (21%; P < .001), OSCE (28%; P < .001), and direct observation (52%; P = .008); and significantly higher (P < .001) mean percentage gains than the nonintervention group on the written test, OSCE, and direct observation. The nonintervention group did not have a significant increase (P = .99) in percentage reaching minimum passing levels, no significant mean percentage gains in the written test (2.7%; P = .30) and direct observation (6.7%; P = .61), and significant regression in OSCE (-5.2%; P = .03). CONCLUSIONS: A pediatric otoscopy curriculum with multimodal outcome assessments was successfully implemented across different specialties at multiple institutions and found to yield gains, including in skills with real patients.

Responsible Milestone-Based Educational Handover With Individualized Learning Plan From Undergraduate to Graduate Pediatric Medical Education.

Communication of medical students' pediatric milestone assessments and individual learning plans from medical schools to pediatric residency directors allows for effective educational handovers promoting the continuum of education. Existing undergraduate medical education assessments can provide meaningful data to determine most pediatric milestone levels.

Motivational Advising Workshop: Utilizing Motivational Interviewing Theory to Facilitate and Engage Intrinsic Motivation to Change Learners' Behavior.

Introduction: Motivational interviewing (MI) is a counseling method that utilizes a patient's own motivation to effect personal change. MI has been applied routinely and successfully to managing medical conditions (e.g., substance abuse). Employing MI techniques to engage medical learners (termed motivational advising [MA]) may help them overcome professional and/or personal challenges limiting their career development. Methods: Medical educators from four academic medical centers developed a module focused on teaching fellow educators MI theory and techniques for MA using didactic and interactive components. Participants participated in facilitated role-plays to practice MA delivery techniques and observed videos of a traditional advisor-advisee interaction as well as an MA-focused engagement. A postworkshop survey was used to evaluate the workshop. Results: In a survey of 48 educators attending the workshop at two medical conferences, over 80% of respondents demonstrated an interest in learning more about MA. Additionally, over 60% indicated that they would seek opportunities to practice and/or implement MA with their advisees. Knowledge of the technical components of MA also increased significantly in pre- and posttest analysis. Discussion: This module introducing the concept of MA was well received by medical educators and was viewed as a valuable tool in advising medical learners. The provided components enable replication of this workshop in other settings with or without an expert in MI techniques. Although the workshop has been conducted with physicians involved in medical education, it would be applicable to other health professionals who advise trainees such as nursing, dentistry, pharmacy, or veterinary medicine.

The impact of learning communities on interpersonal relationships among medical students.

BACKGROUND: Medical students at the University of Virginia (UVA) are mentored and learn within the framework of a four college learning community. Uniquely, these learning communities are used to organize the third-year clerkship rotations. METHODS: Students were surveyed after their first pre-clinical year and after their clerkship year to determine what the effect of the learning community was on their social and educational interpersonal relationships. RESULTS: Students knew a higher percentage of their college mates after completing their third-year clerkships within the framework of the college system. Students chose peers from within the college system for social and educational interpersonal scenarios statistically more often than what would be expected at random. Small group learning environments that were not formed within the framework of the college system at UVA did not have the same effect on interpersonal relationships, indicating that learning communities are uniquely able to provide a context for relationship building. Students felt more positively about the social and educational effects of the college system after the clerkship year, with a corresponding increase in the strength of their interpersonal bonds with their college peers. CONCLUSION: This work is the first to investigate the effects of learning communities on interpersonal relationships among medical students and finds that learning communities positively impact both social and educational medical student bonds.

Standardized Checklist for Otoscopy Performance Evaluation: A Validation Study of a Tool to Assess Pediatric Otoscopy Skills.

INTRODUCTION: Acute otitis media (AOM) is the most frequently diagnosed pediatric illness in the United States and the most common indication for antibiotic prescription. Skill in pediatric otoscopy is essential to correctly identify children with AOM. However, studies have found diagnostic inconsistency and significant overdiagnosis among practitioners. Given the potential public and individual health consequences, there has been a call for improved education regarding the diagnostic certainty of AOM. Yet educational efforts continue to be limited, particularly in regard to competency assessment. The lack of a validated tool to assess otoscopy skill attainment objectively diminishes the instructor's ability to provide useful feedback and direction to the learner. METHODS: We have undertaken an educational intervention with the goal of developing a validated Standardized Checklist for Otoscopy Performance Evaluation (SCOPE), building on key principles of the general pediatric ear exam. The SCOPE was developed with the input of process and content experts with attention to specific domains of validity. RESULTS: Our analysis provides important validity evidence for the SCOPE assessment tool. The instrument was piloted and successfully implemented with medical students and varying levels of residents in pediatrics and emergency medicine over a 5-year period in varied settings: urgent care, large and small pediatric clinics, and the emergency departments at two institutions. It has been used for both instruction and assessment purposes. DISCUSSION: Because the SCOPE can be used in teaching demonstration purposes, in formative and summative assessment settings, and across the continuum of learners, this instrument offers the potential for more educational efforts in the field of assessment in direct patient care. We anticipate that the SCOPE will foster an environment of efficient yet high-yield review and discussion of otoscopy and diagnostic competency.

Benefits to faculty involved in medical school learning communities.

PURPOSE: Job satisfaction plays a large role in enhancing retention and minimizing loss of physicians from careers in academic medicine. The authors explored the effect of learning communities (LCs) on the faculty members' job satisfaction. METHODS: Between October 2011 and May 2012, the authors surveyed 150 academic clinical faculty members serving as LC mentors for students at five US medical schools. Factor analysis was used to explore satisfaction themes and relationships between these themes and other characteristics. RESULTS: Factor analysis revealed two major sources of this satisfaction: a Campus Engagement factor (e.g., feeling happier, improved sense of community, better communication skills, and feeling more productive) and a skills factor (e.g., improved clinical skills, being a better doctor). Higher Campus Engagement factor satisfaction was associated with less desire to leave the learning community (p = 0.01) and more FTE support for role in LC (p = 0.01). Higher skills factor satisfaction was associated with the school that provided more structured faculty development (p = 0.0001). CONCLUSION: Academic clinical faculty members reported serving as a mentor in an LC was a strong source of job satisfaction. LC may be a tool for retaining clinical faculty members in academic careers.

The growth of learning communities in undergraduate medical education.

PURPOSE: To determine the presence and characteristics of learning communities (LCs) in undergraduate medical education. METHOD: The authors updated an earlier Web-based survey to assess LCs in medical education. Using a cross-sectional study design, they sent the survey to an LC leader or dean at each Association of American Medical Colleges member medical school (n = 151) between October 2011 and March 2012. The first survey item asked respondents to indicate if their institution had LCs. Those with LCs were asked to provide details regarding the structure, governance, funding, space, curricular components, extracurricular activities, and areas addressed as part of the LCs. Those without LCs were asked only if they were considering developing them. The full survey instrument contained 35 items including yes/no, multiple-choice, and open-ended questions. The authors analyzed data using descriptive statistics and examined open-ended responses for recurrent themes. RESULTS: The response rate was 83.4% (126/151). Sixty-six schools (52.4%) had LCs. Of the 60 remaining schools without LCs, 29 (48.3%) indicated that they were considering creating them. Of the 52 schools that provided the year their LCs were established, 27 (51.9%) indicated they began in 2007 or later. LC characteristics varied widely. CONCLUSIONS: The number of medical schools with LCs is increasing rapidly. LCs provide an opportunity to transform medical education through longitudinal relationships and mentoring. Further study is needed to document outcomes and best practices for LCs in medical education.