Advancing anti-oppression and social justice in healthcare through competency-based medical education (CBME).

Competency-based medical education (CBME) focuses on preparing physicians to improve the health of patients and populations. In the context of ongoing health disparities worldwide, medical educators must implement CBME in ways that advance social justice and anti-oppression. In this article, authors describe how CBME can be implemented to promote equity pedagogy, an approach to education in which curricular design, teaching, assessment strategies, and learning environments support learners from diverse groups to be successful. The five core components of CBME programs - outcomes competency framework, progressive sequencing of competencies, learning experiences tailored to learners' needs, teaching focused on competencies, and programmatic assessment - enable individualization of learning experiences and teaching and encourage learners to partner with their teachers in driving their learning. These educational approaches appreciate each learner's background, experiences, and strengths. Using an exemplar case study, the authors illustrate how CBME can afford opportunities to enhance anti-oppression and social justice in medical education and promote each learner's success in meeting the expected outcomes of training. The authors provide recommendations for individuals and institutions implementing CBME to enact equity pedagogy.

Twelve tips for embedding assessment for and as learning practices in a programmatic assessment system.

Programmatic assessment supports the evolution from assessment of learning to fostering assessment for learning and as learning practices. A well-designed programmatic assessment system aligns educational objectives, learning opportunities, and assessments with the goals of supporting student learning, making decisions about student competence and promotion decisions, and supporting curriculum evaluation. We present evidence-based guidance for implementing assessment for and as learning practices in the pre-clinical knowledge assessment system to help students learn, synthesize, master and retain content for the long-term so that they can apply knowledge to patient care. Practical tips are in the domains of culture and motivation of assessment, including how an honour code and competency-based grading system can support an assessment system to develop student self-regulated learning and professional identity, curricular assessment structure, such as how and when to utilize low-stakes and cumulative assessment to drive learning, exam and question structure, including what authentic question and exam types can best facilitate learning, and assessment follow-up and review considerations, such exam retake processes to support learning, and academic success structures. A culture change is likely necessary for administrators, faculty members, and students to embrace assessment as most importantly a learning tool for students and programs.

Guiding principles for undergraduate medical education in the time of the COVID-19 pandemic.

As the early epicenter of the COVID-19 pandemic, New York City's medical schools experienced dramatic disruptions in every aspect of medical education. Remote learning was created, seemingly overnight, clerkships were disrupted, licensing examinations were cancelled, teaching faculty were redeployed, student volunteers rallied, and everyone was required to shelter at home. Seismic changes were required to adapt the authors' educational programs to a constantly evolving, unpredictable, and ever-worsening public health crisis. Entirely new communication strategies were adopted and thousands of decisions had to be made, often with little time to carefully reflect on the consequences of those decisions. What allowed each school to navigate these treacherous waters was a set of guiding principles that were used to ground each conversation, and inform every decision. While the language varied somewhat between schools, the core principles were universal and framed a way forward at a time when information, data, precedent, and best practices did not exist. The authors share these guiding principles in the hope that colleagues at other medical schools will find them to be a useful framework as we all continue to cope with the impact of COVID-19 on the future of medical education.

Machine learning to extract communication and history-taking skills in OSCE transcripts.

OBJECTIVES: Observed Structured Clinical Exams (OSCEs) allow assessment of, and provide feedback to, medical students. Clinical examiners and standardised patients (SP) typically complete itemised checklists and global scoring scales, which have known shortcomings. In this study, we applied machine learning (ML) to label some communication skills and interview content information in OSCE transcripts and to compare several ML methodologies by performance and transferability. METHODS: One-hundred and twenty-one transcripts of two OSCE scenarios were manually annotated per utterance across 19 communication skills and content areas. Utterances were converted to two types of numeric sentence vector representations and were paired with three types of ML algorithms. First, ML models (MLMs) were evaluated using a five K-fold cross-validation technique on all transcripts in one scenario to generate precision and recall, and their harmonic mean, F1 scores. Second, ML models were trained on all 101 transcripts from scenario 1 and tested for transferability on 20 scenario 2 transcripts. RESULTS: Performance testing in the K-fold cross-validation demonstrated relatively high mean F1 scores: median 0.87 and range 0.53-0.98 across all 19 labels. Transferability testing demonstrated success: F1 median 0.76 and range 0.46-0.97. The combination of a bi-directional long short-term memory neural network (biLSTM) algorithm with GenSen numeric sentence vector representations was associated with greater F1 scores across both performance and transferability (P < .005). CONCLUSIONS: We report the first application of ML in the context of student-SP OSCEs. We demonstrated that several MLMs automatically labelled OSCE transcripts for a range of interview content and some clinical communications skills. Some MLMs achieved greater performance and transferability. Optimised MLMs could provide automated and accurate assessment of OSCEs with potential to track student progress and identify areas for further practice.

Development of an online public health curriculum for medical students: the public health commute.

BACKGROUND: As public health becomes increasingly central to the practice of medicine, educational efforts are necessary to prepare medical students to apply public health concepts in their care of patients. There are few accessible and informative tools to prepare students to engage with population health challenges. METHODS: We distributed an online questionnaire to clinical students, querying gaps in their education on public health topics. Based upon the responses, we developed a web-based curriculum for medical students rotating at a public safety-net hospital on pediatrics, medicine, primary care, psychiatry, and surgery services from April-December 2017 (available at www.publichealthcommute.com ). Students received guiding questions and media-based resources (e.g. podcasts, TedTalks, YouTube videos) in weekly modules addressing topics in public health. Each module incorporated 30 min of mobile-optimized content, including specific data relating the topic to the Central Harlem community. Familiarity with public health was assessed with pre- and post-program quizzes, including 10 multiple-choice and 2 open-ended questions. RESULTS: Among the 70 participating students, 59 (84%) completed both the pre- and post-assessments. The five-week curriculum covered health systems, social determinants, race, substance use, violence, and alternative care models. After completing the five-week curriculum, the mean correct score on a multiple-choice quiz rose from 57 to 66% (p = 0.001). In the qualitative section of the test, students were asked what public health topics should be taught in medical school. Frequently suggested topics included social determinants of health (25%), epidemiology (25%), health systems (25%), insurance (21%), policy (17%), economics (17%), racism (15%), and health disparities (8%). When asked how public health will impact their medical career, students frequently responded that it would greatly impact their clinical practice (49%), choice of residency program (17%), and decision to pursue advocacy or additional degrees (15%). CONCLUSIONS: Learners participating in this five-week online public health curriculum demonstrated a significant increase in public health knowledge. The online format allowed for high participation across five different specialty rotations, and community-specific data allowed students to recognize the importance of public health in medical practice.

Writing competitive research conference abstracts: AMEE Guide no. 108.

The ability to write a competitive research conference abstract is an important skill for medical educators. A compelling and concise abstract can convince peer reviewers, conference selection committee members, and conference attendees that the research described therein is worthy for inclusion in the conference program and/or for their attendance in the meeting. This AMEE Guide is designed to help medical educators write research conference abstracts that can achieve these outcomes. To do so, this Guide begins by examining the rhetorical context (i.e. the purpose, audience, and structure) of research conference abstracts and then moves on to describe the abstract selection processes common to many medical education conferences. Next, the Guide provides theory-based information and concrete suggestions on how to write persuasively. Finally, the Guide offers some writing tips and some proofreading techniques that all authors can use. By attending to the aspects of the research conference abstract addressed in this Guide, we hope to help medical educators enhance this important text in their writing repertoire.

Practical, Privacy and Ethical, and Philosophical Considerations for Using Big Data in Medical Education.

ABSTRACT: In this issue of Academic Medicine , Thelen and colleagues present a thoughtful perspective on the emerging opportunity to use longitudinal educational data to improve graduate medical education and optimize the education of individual residents, and call for the accelerated development of large interinstitutional data sets for this purpose. Such applications of big data to medical education hold great promise in terms of informing the teaching of individuals, enhancing transitions between phases of training and between institutions, and permitting better longitudinal education research. At the same time, there is a tension between whose data they are and consequently how they ought to be used. This commentary proposes some practical, privacy and ethical, and philosophical considerations that need to be explored as early efforts to aggregate data across the medical education continuum mature and new efforts are undertaken.

The Promise and Perils of Artificial Intelligence in Health Professions Education Practice and Scholarship.

ABSTRACT: Artificial intelligence (AI) methods, especially machine learning and natural language processing, are increasingly affecting health professions education (HPE), including the medical school application and selection processes, assessment, and scholarship production. The rise of large language models over the past 18 months, such as ChatGPT, has raised questions about how best to incorporate these methods into HPE. The lack of training in AI among most HPE faculty and scholars poses an important challenge in facilitating such discussions. In this commentary, the authors provide a primer on the AI methods most often used in the practice and scholarship of HPE, discuss the most pressing challenges and opportunities these tools afford, and underscore that these methods should be understood as part of the larger set of statistical tools available.Despite their ability to process huge amounts of data and their high performance completing some tasks, AI methods are only as good as the data on which they are trained. Of particular importance is that these models can perpetuate the biases that are present in those training datasets, and they can be applied in a biased manner by human users. A minimum set of expectations for the application of AI methods in HPE practice and scholarship is discussed in this commentary, including the interpretability of the models developed and the transparency needed into the use and characteristics of such methods.The rise of AI methods is affecting multiple aspects of HPE including raising questions about how best to incorporate these models into HPE practice and scholarship. In this commentary, we provide a primer on the AI methods most often used in HPE and discuss the most pressing challenges and opportunities these tools afford.

Statement of Partnership and Humility: A Structural Intervention to Improve Equity and Justice in Medical Education.

PROBLEM: Due to generational exposure to the Black Lives Matter movement, other anti-bias social movements, and diverse peer advocacy groups, health professions students are often more knowledgeable than their teachers about ways in which systemic racism and bias have led to scientific inaccuracies that contribute to health inequities. However, traditional hierarchies and concerns about retaliation may limit educational communities from benefiting maximally from students' contributions. APPROACH: In spring 2021, faculty and students at the Vagelos College of Physicians and Surgeons, Columbia University, designed a structural innovation to engage faculty and students in partnership toward decreasing bias in medical education. This article discusses development and implementation of a Statement of Partnership and Humility (SPH) disclosure slide on which faculty acknowledge consideration of potential teaching biases and invite student feedback. OUTCOMES: The initial primary goal of the SPH slide was to increase faculty awareness and engagement in anti-bias topics; however, the unexpected dividends of decreasing faculty anxiety about receiving student feedback and promoting student engagement have proven equally powerful in promoting a healthy, inclusive learning environment. NEXT STEPS: Next steps include gathering qualitative and quantitative data to elicit both faculty and student perspectives on the use of the SPH slide, particularly with regard to psychological safety and openness to feedback.

Implementation of a weight loss program for Latino outpatients with severe mental illness.

To determine feasibility of implementation of a weight loss program for overweight Latinos with severe mental illness. In this quasi-experimental study, a 14-week behavioral weight loss course (extended) was implemented at one clinic. A one-time nutrition class (brief) was given at a sister clinic. Implementation feasibility was assessed by consent and participation rates. Weight was followed for 6 months. Consent rates were high [77 % (49/64) extended; 68 % (39/57) brief], and 88 % (43/49) of extended subjects participated and 88 % (38/43) completed follow-up. Weight loss did not differ between groups. A behavioral weight loss course is feasible to implement for this population.

Entrustable professional activities as a training and assessment framework in undergraduate medical education: A case study of a multi-institutional pilot.

In 2014, the Association of American Medical Colleges (AAMC) published 13 Core Entrustable Professional Activities (EPAs) that graduating students should be able to perform with indirect supervision when entering residency. A ten-school multi-year pilot was commissioned to test feasibility of implementing training and assessment of the AAMC's 13 Core EPAs. In 2020-21, a case study was employed to describe pilot schools' implementation experiences. Teams from nine of ten schools were interviewed to identify means and contexts of implementing EPAs and lessons learned. Audiotapes were transcribed then coded by investigators using conventional content analysis and a constant comparative method. Coded passages were organized in a database and analyzed for themes. Consensus among school teams regarding facilitators of EPA implementation included team commitment to piloting EPAs; agreement that: proximal EPA adoption with curriculum reform facilitates EPA implementation; EPAs 'naturally fit' in clerkships and provided opportunity for schools to reflect on and adjust curricula and assessments; and inter-school collaboration bolstered individual school progress. Schools did not make high-stakes decisions about student progress (e.g., promotion, graduation), yet EPA assessment results complemented other forms of assessment in providing students with robust formative feedback about their progress. Teams had varied perceptions of school capability to implement an EPA framework, influenced by various levels of dean involvement, willingness, and capability of schools to invest in data systems and provide other resources, strategic deployment of EPAs and assessments, and faculty buy-in. These factors affected varied pace of implementation. Teams agreed on the worthiness of piloting the Core EPAs, but substantial work is still needed to fully employ an EPA framework at the scale of entire classes of students with enough assessments per EPA and with required data validity/reliability. Recommendations stemming from findings may help inform further implementation efforts across other schools adopting or considering an EPA framework.

Core Entrustable Professional Activities for Entering Residency: A National Survey of Graduating Medical Students' Self-Assessed Skills by Specialty.

BACKGROUND: The Association of American Medical Colleges described 13 Core Entrustable Professional Activities (EPAs) that graduating students should be prepared to perform under indirect supervision on day one of residency. Surgery program directors recently recommended entrustability in these Core EPAs for incoming surgery interns. We sought to determine if graduating students intending to enter surgery agreed they had the skills to perform these Core EPAs. STUDY DESIGN: Using de-identified, individual-level data collected from and about 2019 Association of American Medical Colleges Graduation Questionnaire respondents, latent profile analysis was used to group respondents based on their self-assessed Core EPAs skills' response patterns. Associations between intended specialty, among other variables, and latent profile analysis group were assessed using independent sample t -tests and chi-square tests and multivariable logistic regression methods. RESULTS: Among 12,308 Graduation Questionnaire respondents, latent profile analysis identified 2 respondent groups: 7,863 (63.9%) in a high skill acquisition agreement (SAA) group and 4,445 (36.1%) in a moderate SAA group. Specialty was associated with SAA group membership (p < 0.001), with general surgery, orthopaedic surgery, and emergency medicine respondents (among others) overrepresented in the high SAA group. In the multivariable logistic regression models, each of anesthesiology, ophthalmology, pediatrics, psychiatry, and radiology (vs general surgery) specialty intention was associated with a lower odds of high SAA group membership. CONCLUSION: Graduating students' self-assessed Core EPAs skills were higher for those intending general surgery than for those intending some other specialties. Our findings can inform collaborative efforts to ensure graduates' acquisition of the skills expected of them at the start of residency.

Comparing Entrustment Decision-Making Outcomes of the Core Entrustable Professional Activities Pilot, 2019-2020.

Importance: Gaps in readiness for indirect supervision have been identified for essential responsibilities encountered early in residency, presenting risks to patient safety. Core Entrustable Professional Activities (EPAs) for entering residency have been proposed as a framework to address these gaps and strengthen the transition from medical school to residency. Objective: To assess progress in developing an entrustment process in the Core EPAs framework. Design, Setting, and Participants: In this quality improvement study in the Core EPAs for Entering Residency Pilot, trained faculty made theoretical entrustment determinations and recorded the number of workplace-based assessments (WBAs) available for each determination in 2019 and 2020. Four participating schools attempted entrustment decision-making for all graduating students or a randomly selected subset of students. Deidentified, individual-level data were merged into a multischool database. Interventions: Schools implemented EPA-related curriculum, WBAs, and faculty development; developed systems to compile and display data; and convened groups to make theoretical summative entrustment determinations. Main Outcomes and Measures: On an EPA-specific basis, the percentage of students for whom an entrustment determination could be made, the percentage of students ready for indirect supervision, and the volume of WBAs available were recorded. Results: Four participating schools made 4525 EPA-specific readiness determinations (2296 determinations in 2019 and 2229 determinations in 2020) for 732 graduating students (349 students in 2019 and 383 students in 2020). Across all EPAs, the proportion of determinations of "ready for indirect supervision" increased from 2019 to 2020 (997 determinations [43.4%] vs 1340 determinations [60.1%]; 16.7 percentage point increase; 95% CI, 13.8-19.6 percentage points; P < .001), as did the proportion of determinations for which there were 4 or more WBAs (456 of 2295 determinations with WBA data [19.9%] vs 938 [42.1%]; 22.2 percentage point increase; 95% CI, 19.6-24.8 percentage points; P < .001). The proportion of EPA-specific data sets considered for which an entrustment determination could be made increased from 1731 determinations (75.4%) in 2019 to 2010 determinations (90.2%) in 2020 (14.8 percentage point increase; 95% CI, 12.6-16.9 percentage points; P < .001). On an EPA-specific basis, there were 5 EPAs (EPA 4 [orders], EPA 8 [handovers], EPA 10 [urgent care], EPA 11 [informed consent], and EPA 13 [patient safety]) for which few students were deemed ready for indirect supervision and for which there were few WBAs available per student in either year. For example, for EPA 13, 0 of 125 students were deemed ready in 2019 and 0 of 127 students were deemed ready in 2020, while 0 determinations in either year included 4 or more WBAs. Conclusions and Relevance: These findings suggest that there was progress in WBA data collected, the extent to which entrustment determinations could be made, and proportions of entrustment determinations reported as ready for indirect supervision. However, important gaps remained, particularly for a subset of Core EPAs.

Columbia COVID-19 Student Service Corps: Harnessing student skills and galvanizing the power of service learning.

The COVID-19 pandemic in New York City led to the forced rapid transformation of the medical school curriculum as well as increased critical needs to the health system. In response, a group of faculty and student leaders at CUIMC developed the COVID-19 Student Service Corps (Columbia CSSC). The CSSC is an interprofessional service-learning organization that galvanizes the skills and expertise of faculty and students from over 12 schools and programs in the response to the COVID-19 pandemic, and is agile enough to shift and respond to future public health and medical emergencies. Since March 2020, over 30 projects have been developed and implemented supporting needs identified by the health system, providers, faculty, staff, and students as well as the larger community. The development of the CSSC also provided critical virtual educational opportunities in the form of service learning for students who were unable to have any in-person instruction. The CSSC model has been shared nationally and nine additional chapters have started at academic institutions across the country.

Strategies From 11 U.S. Medical Schools for Integrating Basic Science Into Core Clerkships.

Calls for curricular reform in medical schools and enhanced integration of basic and clinical science have resulted in a shift toward preclerkship curricula that enhance the clinical relevance of foundational science instruction and provide students with earlier immersion in the clinical environment. These reforms have resulted in shortened preclerkship curricula, yet the promise of integrated basic science education into clerkships has not been sufficiently realized because of barriers such as the nature of clinical practice, time constraints, and limited faculty knowledge. As personalized medicine requires that physicians have a more nuanced understanding of basic science, this is cause for alarm. To address this problem, several schools have developed instructional and assessment strategies to better integrate basic science into the clinical curriculum. In this article, faculty and deans from 11 U.S. medical schools discuss the strategies they implemented and the lessons they learned to provide guidance to other schools seeking to enhance basic science education during clerkships. The strategies include program-level interventions (e.g., longitudinal sessions dedicated to basic science during clerkships, weeks of lessons dedicated to basic science interspersed in clerkships), clerkship-level interventions (e.g., case-based learning with online modules, multidisciplinary clerkship dedicated to applied science), bedside-level interventions (e.g., basic science teaching scripts, self-directed learning), and changes to formative and summative assessments (e.g., spaced repetition/leveraging test-enhanced learning, developing customized examinations). The authors discovered that: interventions were more successful when buy-in from faculty and students was considered, central oversight by curricular committees collaborating with faculty was key, and some integration efforts may require schools to provide significant resources. All schools administered the United States Medical Licensing Examination Step 1 exam to students after clerkship, with positive outcomes. The authors have demonstrated that it is feasible to incorporate basic science into clinical clerkships, but certain challenges remain.

Revisiting the Core Entrustable Professional Activities for Entering Residency.

The Core EPAs for Entering Residency Pilot project aimed to test the feasibility of implementing 13 entrustable professional activities (EPAs) at 10 U.S. medical schools and to gauge whether the use of the Core EPAs could improve graduates' performance early in residency. In this manuscript, the authors (members of the pilot institutions and Association of American Medical Colleges staff supporting the project evaluation) describe the schools' capacity to collect multimodal evidence about their students' performance in each of the Core EPAs and the ability of faculty committees to use those data to make decisions regarding learners' readiness for entrustment. In reviewing data for each of the Core EPAs, the authors reflected on how each activity performed as an EPA informed by how well it could be assessed and entrusted. For EPAs that did not perform well, the authors examined whether there are underlying practical and/or theoretical issues limiting its utility as a measure of student performance in medical school.

Toward a Bias-Free and Inclusive Medical Curriculum: Development and Implementation of Student-Initiated Guidelines and Monitoring Mechanisms at One Institution.

As research and attention on implicit bias and inclusiveness in medical school is expanding, institutions need mechanisms for recognizing, reporting, and addressing instances of implicit bias and lack of inclusiveness in medical school curricular structures. These instances can come as a result of a lack of both awareness and communication around these sensitive issues. To identify and address cases of implicit bias in the medical school curriculum, a student-led initiative at Columbia University Vagelos College of Physicians and Surgeons (VP&S) developed guidelines and a bias-reporting process for educators and students. The guidelines, co-created by students and faculty, help educators identify and address implicit bias in the curriculum. Furthermore, to allow for continued development of the curriculum and the guidelines themselves, the group adapted an existing learning environment reporting and review process to identify and address instances of implicit bias. In the first year since their implementation, these tools have already had an impact on the learning climate at VP&S. They have led to enhanced identification of implicit bias in the curriculum and changes in instructional materials. The courage and inspiration of the students and the initial investment and commitment from the administration and faculty were crucial to this rapid effect. The authors present an approach and resources from which other institutions can learn, with the goal of reducing implicit bias and improving inclusiveness throughout medical education. In the long run, the authors hope that these interventions will contribute to better preparing future providers to care for all patients equitably.

Mobilization of health professions students during the COVID-19 pandemic.

The COVID-19 pandemic has posed challenges for medical education and patient care, which were felt acutely in obstetrics due to the essential nature of pregnancy care. The mobilization of health professions students to participate in obstetric service-learning projects has allowed for continued learning and professional identify formation while also providing a motivated, available, and skilled volunteer cohort to staff important projects for obstetric patients.

Faculty Development Revisited: A Systems-Based View of Stakeholder Development to Meet the Demands of Entrustable Professional Activity Implementation.

In 2015, the Association of American Medical Colleges implemented an interinstitutional pilot of 13 core entrustable professional activities (EPAs) for entering residency, activities that entering residents should be expected to perform with indirect supervision. The pilot included a concept group on faculty development; this group previously offered a shared mental model focused on the development of faculty who devote their efforts to clinical teaching and assessment for learning and entrustment decision making. In this article, the authors draw from the literature of competency-based education to propose what is needed in overall approaches to faculty development to prepare institutions for undergraduate EPA implementation.Taking a systems-based view that defines the necessary tasks of EPA implementation, the authors move beyond the variably used term "faculty" and enumerate a comprehensive list of institutional stakeholders who can meaningfully support and/or engage in the relationships and organizational processes required for EPA learning and assessment. They consider each group's responsibilities and development needs according to five domains delineated by Steinert: teaching improvement, leadership and management, research-building capacity, academic career building, and organizational change.The authors argue that the EPA framework addresses barriers posed with the use of a competency-based framework. By facilitating the communication required for organizational change, enabling valid assessment with comprehensive yet feasible levels of faculty development, and incorporating all relevant data on student professional behavior into summative assessment decisions, EPAs may offer a clearer path toward the goal of competency-based education.