An Analysis of Workplace-Based Assessments for Core Entrustable Professional Activities for Entering Residency: Does Type of Clinical Assessor Influence Level of Supervision Ratings?

PURPOSE: The authors describe use of the workplace-based assessment (WBA) coactivity scale according to entrustable professional activities (EPAs) and assessor type to examine how diverse assessors rate medical students using WBAs. METHOD: A WBA data collection system was launched at Oregon Health and Science University to visualize learner competency in various clinical settings to foster EPA assessment. WBA data from January 14 to June 18, 2021, for medical students (all years) were analyzed. The outcome variable was level of supervisor involvement in each EPA, and the independent variable was assessor type. RESULTS: A total of 7,809 WBAs were included. Most fourth-, third-, and second-year students were assessed by residents or fellows (755 [49.5%], 1,686 [48.5%], and 918 [49.9%], respectively) and first-year students by attending physicians (803 [83.0%]; P < .001). Attendings were least likely to use the highest rating of 4 (1 was available just in case; 2,148 [56.7%] vs 2,368 [67.7%] for residents; P < .001). Learners more commonly sought WBAs from attendings for EPA 2 (prioritize differential diagnosis), EPA 5 (document clinical encounter), EPA 6 (provide oral presentation), EPA 7 (form clinical questions and retrieve evidence-based medicine), and EPA 12 (perform general procedures of a physician). Residents and fellows were more likely to assess students on EPA 3 (recommend and interpret diagnostic and screening tests), EPA 4 (enter and discuss orders and prescriptions), EPA 8 (give and receive patient handover for transitions in care), EPA 9 (collaborate as member of interprofessional team), EPA 10 (recognize and manage patient in need of urgent care), and EPA 11 (obtain informed consent). CONCLUSIONS: Learners preferentially sought resident vs attending supervisors for different EPA assessments. Future research should investigate why learners seek different assessors more frequently for various EPAs and if assessor type variability in WBA levels holds true across institutions.

Impact of Students' Scheduling Choice on Clerkship Examination Score Performance in a Time-Varying Competency-Based Curriculum.

PURPOSE: Standardized end-of-clerkship examinations typically occur on the last day of the clerkship. However, recent trends toward time-varying competency-based medical education have offered students more test scheduling flexibility, creating an opportunity to study the impact of student-selected examination timing. METHOD: Starting with the graduating class of 2018, students took the required standardized end-of-core clerkship examinations at any available time they chose during their clinical years. Before this change, these examinations were administered to all students on the last day of the clerkship. Students' examination dates relative to clerkship completion were analyzed between 2017 and 2020 (inclusive of before and after flexible exam timing) to assess the impact that student-selected exam timing had on test performance on National Board of Medical Examiners shelf clinical science examinations for required core clerkships. RESULTS: Data on 146 medical students in 2017 (fixed exam timing) and 466 medical students between 2018 and 2020 (flexible exam timing) were included. Among students offered flexible exam timing, between 2.7% (internal medicine) and 14.6% (psychiatry) took their exam before actually taking clerkship, while between 22.7% (psychiatry) and 40.0% (surgery) took their exam more than 90 days after the clerkship ended. Exam scores were statistically higher for those who took the exam at a time of their choosing compared with those who were required to take it at the end of individual rotations and when the exam scores were combined (fixed exam timing mean = 73.9, standard deviation [SD] = 7.8; flexible exam timing mean = 77.4, SD = 6.0, P < .001). The percent of students with passing scores was statistically higher in internal medicine, pediatrics, and psychiatry. CONCLUSIONS: Self-selection of shelf exam timing appears to increase shelf exam scores. As more medical schools transition to competency-based medical education, providing scheduling flexibility appears not to negatively affect student achievement.

Extension of community healthcare outcomes in Parkinson disease (Parkinson ECHO): A feasibility study.

Background: Parkinson's disease is the second most common neurodegenerative disorder and presents with a heterogeneous group of symptoms. Managing these symptoms requires coordinated care from a neurology specialist and a primary care provider. Access to neurology care is limited for those patients with Parkinson's disease who reside in rural areas given financial and mobility constraints along with the rarity of specialty providers. Methods: To close this gap, we developed and implemented a telehealth-based Project ECHO® (Extension for Community Healthcare Outcomes) program, "Parkinson ECHO," to provide education and support for rural clinicians and allied health members. The sessions focused on a topic within Parkinson's disease diagnosis or management followed by case discussions. We assessed the feasibility of this tele-mentoring educational offering, the favorability of this approach, and the effect it had on clinician confidence in diagnosing and treating Parkinson's disease using Likert-based surveys. Results: Thirty-three unique participants from 13 Oregon counties and one county in the state of Washington, of whom 70 % served rural and/or medically underserved communities, participated in Parkinson ECHO. There was a 52 % dropout rate based on survey response, though session attendance was higher. Participants were overall satisfied with the format and content of Parkinson ECHO. There were improvements in knowledge and confidence in diagnosing and treating Parkinson's disease which persisted 6 months following the conclusion of the program. Unexpectedly, two participants reported convening a multidisciplinary group to discuss improvements to PD care. Conclusion: The COVID-19 pandemic was an unexpected obstacle, but the teleconference nature permitted us to complete the program to positive effect. We found Parkinson ECHO did significantly increase participant confidence levels in diagnosing and managing Parkinson's disease.

Contemporary Neuroscience Core Curriculum for Medical Schools.

Medical students need to understand core neuroscience principles as a foundation for their required clinical experiences in neurology. In fact, they need a solid neuroscience foundation for their clinical experiences in all other medical disciplines also, because the nervous system plays such a critical role in the function of every organ system. Due to the rapid pace of neuroscience discoveries, it is unrealistic to expect students to master the entire field. It is also unnecessary, as students can expect to have ready access to electronic reference sources no matter where they practice. In the pre-clerkship phase of medical school, the focus should be on providing students with the foundational knowledge to use those resources effectively and interpret them correctly. This article describes an organizational framework for teaching the essential neuroscience background needed by all physicians. This is particularly germane at a time when many medical schools are re-assessing traditional practices and instituting curricular changes such as competency-based approaches, earlier clinical immersion, and increased emphasis on active learning. This article reviews factors that should be considered when developing the pre-clerkship neuroscience curriculum, including goals and objectives for the curriculum, the general topics to include, teaching and assessment methodology, who should direct the course, and the areas of expertise of faculty who might be enlisted as teachers or content experts. These guidelines were developed by a work group of experienced educators appointed by the Undergraduate Education Subcommittee (UES) of the American Academy of Neurology (AAN). They were then successively reviewed, edited, and approved by the entire UES, the AAN Education Committee, and the AAN Board of Directors.

Education Research: NeuroBytes: A New Rapid, High-Yield e-Learning Platform for Continuing Professional Development in Neurology.

OBJECTIVE: To determine whether NeuroBytes is a helpful e-Learning tool in neurology through usage, viewer type, estimated time and cost of development, and postcourse survey responses. BACKGROUND: A sustainable Continuing Professional Development (CPD) system is vital in neurology due to the field's expanding therapeutic options and vulnerable patient populations. In an effort to offer concise, evidence-based updates to a wide range of neurology professionals, the American Academy of Neurology (AAN) launched NeuroBytes in 2018. NeuroBytes are brief (<5 minutes) videos that provide high-yield updates to AAN members. METHODS: NeuroBytes was beta tested from August 2018 to December 2018 and launched for pilot circulation from January 2019 to April 2019. Usage was assessed by quantifying course enrollment and completion rates; feasibility by cost and time required to design and release a module; appeal by user satisfaction; and effect by self-reported change in practice. RESULTS: A total of 5,130 NeuroBytes enrollments (1,026 ± 551/mo) occurred from January 11, 2019, to May 28, 2019, with a median of 588 enrollments per module (interquartile range, 194-922) and 37% course completion. The majority of viewers were neurologists (54%), neurologists in training (26%), and students (8%). NeuroBytes took 59 hours to develop at an estimated $77.94/h. Of the 1,895 users who completed the survey, 82% were "extremely" or "very likely" to recommend NeuroBytes to a colleague and 60% agreed that the depth of educational content was "just right." CONCLUSIONS: NeuroBytes is a user-friendly, easily accessible CPD product that delivers concise updates to a broad range of neurology practitioners and trainees. Future efforts will explore models where NeuroBytes combines with other CPD programs to affect quality of training and clinical practice.

Managing expansions in medical students' clinical placements caused by curricular transformation: perspectives from four medical schools.

Many challenges could occur that result in the need to handle an increase in the number of medical student clinical placements, such as curricular transformations or viral pandemics, such as COVID 19. Here, we describe four different institutions' approaches to addressing the impact of curricular transformation on clerkships using an implementation science lens. Specifically, we explore four different approaches to managing the 'bulge' as classes overlap in clerkships Curriculum leaders at four medical schools report on managing the bulge of core clinical placements resulting from reducing the duration of the foundational sciences curriculum and calendar shifts for the respective clerkship curriculum. These changes, which occurred between 2014 and 2018, led to more students being enrolled in core clinical rotations at the same time than occurred previously. Schools provided respective metrics used to evaluate the effectiveness of their bulge management technique. These data typically included number of students affected in each phase of their curricular transformation, performance on standardized examinations, and student and faculty feedback. Not all data were available from all schools, as some schools are still working through their 'bulge' or are affected by COVID-19. There is much to be learned about managing curricular transformations. Working on such endeavors in a learning collaborative such as the AMA Accelerating Change in Medical Education Initiative provided support and insights about how to survive, thrive and identifying lessons learned during curricular transformation.

An observational study of an approach to accommodate a fourth-year to third-year neurology clerkship curricular transition.

Background: Curricular transformation can result in bulges in students' clinical placements.Objective: To report on learner outcomes associated with a competency-based opt-out approach for a required 4th-year neurology clerkship.Methods and Study Design: During Oregon Health & Science University's recent undergraduate medical education curricular transformation, a 4-week required neurology clerkship transitioned from the fourth-year to the third-year in academic year 2016-17. Because this would have resulted in the neurology clerkship accommodating double enrollment for an entire academic year, 4th year medical students from the prior curriculum (graduating class of 2017) were offered the option of opting-out of the required neurology clerkship if they demonstrated competency by passing the USA National Board of Medical Examiners (US-NBME) clinical neurology subject examination and completing a neurology faculty-observed history and complete neurological examination.Results: Fifty-seven of 133 fourth-year students (42.9%) chose to complete the required neurology clerkship with 77 (57.9%) choosing to opt-out. All opt-out students passed the neurological exam assessment and scored similarly to the students who took the clerkship (US-NBME Neurology Subject Exam mean raw score in the opt-out group 76.9 vs, 77.6; p = 0.61). Students grades did not differ. Students who opted-out tended to pursue surgical careers (e.g., general surgery -10.8% opted-out vs 0% clerkship, OB/GYN - 6.8% opted-out vs 0% clerkship, orthopedic surgery 5.4% opted-out vs 0% clerkship) where those who took the clerkship tended to choose medical residency training disciplines (family medicine -16.1% clerkship vs 10.8% opting-out; internal medicine - 32.1% clerkship vs 14.9% opting-out; psychiatry 10.7% clerkship vs 2.7% opting-out (p = 0.042)Conclusion: While undertaking the neurology clerkship would have been the desired approach, students appear not to have been harmed by the opt-out approach regarding performance on the US-NBME clinical neurology subject exam. Choices regarding opting-out versus taking the neurology clerkship appear to be associated with career choice.

Measuring Coaching in Undergraduate Medical Education: the Development and Psychometric Validation of New Instruments.

BACKGROUND: Coaching is emerging as a novel approach to guide medical students toward becoming competent, reflective physicians and master adaptive learners. However, no instruments currently exist to measure academic coaching at the undergraduate medical education level. OBJECTIVE: To describe the development and psychometric assessments of two instruments designed to assess academic coaching of medical students toward creating a robust measurement model of this educational paradigm. DESIGN: Observational psychometric. PARTICIPANTS: All medical students in the 2014 and 2015 cohorts and all their coaches were invited to complete the instruments being tested, which led to 662 medical student responses from 292 medical students and 468 coaching responses from 22 coaches being included in analyses. Medical student response rates were 75.7% from 2014 and 75.5% from 2015. Overall, the coaches' response rate was 71%. MAIN MEASURES: Two 31-item instruments were initially developed, one for medical students to assess their coach and one for faculty coaches to assess their students, both of which evaluated coaching based on definitions we formulated using existing literature. Each was administered to two cohorts of medical students and coaches in 2015 and 2016. An exploratory factor analysis using principal component analysis as the extraction method and Varimax with Kaiser normalization as the rotation method was conducted. KEY RESULTS: Eighteen items reflecting four domains were retained with eigenvalues higher than 1.0 for medical students' assessment of coaching, which measured promoting self-monitoring, relationship building, promoting reflective behavior, and establishing foundational ground rules. Sixteen items reflecting two domains were retained for the faculty assessment of coaching with eigenvalues higher than 1.0: the Practice of Coaching and Relationship Formation. CONCLUSIONS: We successfully developed and psychometrically validated surveys designed to measure key aspects of the coaching relationship, coaching processes, and reflective outcomes. The new validated instruments offer a robust measurement model for academic coaching.

Entrustable professional activities: A useful concept for neurology education.

Medical education is currently undergoing a paradigm shift from process-based to competency-based education, focused on measuring the desired competence of a physician. In an attempt to improve the assessment framework used for medical education, the concept of entrustable professional activities (EPAs) has gained traction. EPAs are defined as professional activities that can be entrusted to an individual in a clinical context. The Association of American Medical Colleges (AAMC) defined a set of 13 such EPAs to define the core of what all students should be able to do on day 1 of residency, regardless of specialty choice. The AAMC is currently piloting these EPAs with 10 medical schools to determine if EPAs can be used as a way to observe, measure, and entrust medical students with core clinical activities by the end of the clinical immersion experiences of the third year. The specialty of pediatrics is piloting the use of specialty-specific EPAs at 5 medical schools to assess readiness for transitions from medical school into pediatric residency training and practice. To date, no neurology-specific EPAs have been published for use in neurology clerkships or neurology residencies. This article introduces the concept of EPAs in the context of competency-based medical education and describes how EPAs might be relevant and applicable in neurologic education across the continuum. The Undergraduate Education Subcommittee of the American Academy of Neurology advocates for a proactive approach to incorporating core EPAs in undergraduate medical education and to considering an EPA-based specialty-specific assessment framework for neurology.

Crafting a positive professional digital profile to augment your practice.

A digital profile is the sum content about a person on the Internet. A digital profile can be composed of personal or professional information shared on public Web sites posted personally or by others. One of the most effective ways to build a positive professional digital profile is through social media. It is increasingly important to maintain a positive digital profile as others mine the Internet to find out about a professional prior to meeting him or her. As the digital environment continues to grow, it will become increasingly difficult to neglect a professional digital profile without potential negative consequences. There are many benefits to creating a digital presence and using the tools available to learn about neurology and interact with other professionals and patients in ways that were not possible in the past. The spread of social media to a large part of the population makes it unlikely to go away.

Status of neurology medical school education: results of 2005 and 2012 clerkship director survey.

OBJECTIVE: To survey all US medical school clerkship directors (CDs) in neurology and to compare results from a similar survey in 2005. METHODS: A survey was developed by a work group of the American Academy of Neurology Undergraduate Education Subcommittee, and sent to all neurology CDs listed in the American Academy of Neurology database. Comparisons were made to a similar 2005 survey. RESULTS: Survey response rate was 73%. Neurology was required in 93% of responding schools. Duration of clerkships was 4 weeks in 74% and 3 weeks in 11%. Clerkships were taken in the third year in 56%, third or fourth year in 19%, and fourth year in 12%. Clerkship duration in 2012 was slightly shorter than in 2005 (fewer clerkships of ≥4 weeks, p = 0.125), but more clerkships have moved into the third year (fewer neurology clerkships during the fourth year, p = 0.051). Simulation training in lumbar punctures was available at 44% of schools, but only 2% of students attempted lumbar punctures on patients. CDs averaged 20% protected time, but reported that they needed at least 32%. Secretarial full-time equivalent was 0.50 or less in 71% of clerkships. Eighty-five percent of CDs were "very satisfied" or "somewhat satisfied," but more than half experienced "burnout" and 35% had considered relinquishing their role. CONCLUSION: Trends in neurology undergraduate education since 2005 include shorter clerkships, migration into the third year, and increasing use of technology. CDs are generally satisfied, but report stressors, including inadequate protected time and departmental support.

A competency-based longitudinal core curriculum in medical neuroscience.

Current medical educational theory encourages the development of competency-based curricula. The Accreditation Council for Graduate Medical Education's 6 core competencies for resident education (medical knowledge, patient care, professionalism, interpersonal and communication skills, practice-based learning, and systems-based practice) have been embraced by medical schools as the building blocks necessary for becoming a competent licensed physician. Many medical schools are therefore changing their educational approach to an integrated model in which students demonstrate incremental acquisition and mastery of all competencies as they progress through medical school. Challenges to medical schools include integration of preclinical and clinical studies as well as development of learning objectives and assessment measures for each competency. The Undergraduate Education Subcommittee (UES) of the American Academy of Neurology (AAN) assembled a group of neuroscience educators to outline a longitudinal competency-based curriculum in medical neuroscience encompassing both preclinical and clinical coursework. In development of this curriculum, the committee reviewed United States Medical Licensing Examination content outlines, Liaison Committee on Medical Education requirements, prior AAN-mandated core curricula for basic neuroscience and clinical neurology, and survey responses from educators in US medical schools. The newly recommended curriculum provides an outline of learning objectives for each of the 6 competencies, listing each learning objective in active terms. Documentation of experiences is emphasized, and assessment measures are suggested to demonstrate adequate achievement in each competency. These guidelines, widely vetted and approved by the UES membership, aspire to be both useful as a stand-alone curriculum and also provide a framework for neuroscience educators who wish to develop a more detailed focus in certain areas of study.

Survey of Residents' Attitudes and Awareness Toward Teaching and Student Feedback.

BACKGROUND: Teaching medical students is an important component of residency; however, little is known about student feedback regarding resident teaching skills. OBJECTIVE: We sought to explore resident awareness of medical student feedback mechanisms and how feedback is obtained, and also identified attitudes about teaching more commonly found in residents who seek feedback. METHODS: We surveyed all resident physicians at a university-affiliated academic health center about awareness of student feedback regarding their teaching abilities, and their attitudes related to teaching that may impact whether residents seek feedback. RESULTS: Of 605 residents, 335 (55%) responded, with 72% (242 of 335) noting they did not formally review student feedback of their teaching with their advisor during regularly scheduled meetings, 42% (140 of 332) reporting they did not know of any formal feedback mechanisms, and 28.4% (95 of 334) reporting they had not received feedback from students in any format. Although only a quarter of residents solicit feedback always or often, more than half would like feedback always or often. Reported barriers to feedback included student apprehension, time constraints, and lack of a formal system. A majority of residents had positive attitudes toward teaching and felt that student feedback would help teaching ability and medical proficiency. CONCLUSIONS: A large percentage of residents at 1 teaching institution reported not receiving feedback from students on their teaching abilities. Residents who did receive feedback were more likely to have actively solicited it. Overall, residents believe that this feedback from students would benefit their clinical and teaching performance.