Should I do a synthesis (i.e. literature review)?

This column is intended to address the kinds of knotty problems and dilemmas with which many scholars grapple in studying health professions education. In this article, the authors address the question of whether one should conduct a literature review or knowledge synthesis, considering the why, when, and how, as well as its potential pitfalls. The goal is to guide supervisors and students who are considering whether to embark on a literature review in education research.

Screening for Food Insecurity: A Curriculum for Medical Students.

Introduction: Food insecurity (FI) is defined as a lack of access to enough food for an active, healthy life. We sought to determine how a longitudinal FI screening curriculum impacts medical students' knowledge, attitudes, and behavior in screening for FI. Methods: This was a prospective, single-institution study. The curriculum consisted of three components completed over 3 years. We administered a survey to the intervention cohort before and after the curriculum and analyzed their written reflections. We also evaluated whether students screened for FI during an objective structured clinical exam (OSCE) and compared their performance to a control cohort, which did not receive the curriculum. Results: Preintervention, students felt screening for FI was important for physicians to do with their patients, but most felt uncomfortable addressing it in clinical settings. Postintervention, there was a statistically significant increase in mean scores for knowledge questions (45.24% vs 74.74%, P<.001, pre- and postintervention, respectively). Students also felt more confident in their abilities to screen and follow up about FI. Additionally, compared to the control cohort, the intervention cohort screened for FI more often during their OSCE (28.21% vs 10.71%, P<.001). Conclusion: A longitudinal curriculum using minimal curricular time can improve students' knowledge, attitudes, and behavior when screening for FI. Students who received the curriculum were more likely to recognize the need for and perform FI screening. Based on these findings, we anticipate that the curriculum will increase the likelihood of students identifying, screening for, and intervening in cases of FI in future clinical encounters.

Advancing anti-oppression and social justice in healthcare through competency-based medical education (CBME).

Competency-based medical education (CBME) focuses on preparing physicians to improve the health of patients and populations. In the context of ongoing health disparities worldwide, medical educators must implement CBME in ways that advance social justice and anti-oppression. In this article, authors describe how CBME can be implemented to promote equity pedagogy, an approach to education in which curricular design, teaching, assessment strategies, and learning environments support learners from diverse groups to be successful. The five core components of CBME programs - outcomes competency framework, progressive sequencing of competencies, learning experiences tailored to learners' needs, teaching focused on competencies, and programmatic assessment - enable individualization of learning experiences and teaching and encourage learners to partner with their teachers in driving their learning. These educational approaches appreciate each learner's background, experiences, and strengths. Using an exemplar case study, the authors illustrate how CBME can afford opportunities to enhance anti-oppression and social justice in medical education and promote each learner's success in meeting the expected outcomes of training. The authors provide recommendations for individuals and institutions implementing CBME to enact equity pedagogy.

Faculty and Student Perceptions of Unauthorized Collaborations in the Preclinical Curriculum: Student or System Failure?

PURPOSE: Unauthorized collaboration among medical students, including the unauthorized provision of assistance and sharing of curricular and assessment materials, is a reported problem. While many faculty view such sharing as academic dishonesty, students do not always perceive these behaviors as problematic. With the trend toward more small-group and team-based learning and the proliferation of resource-sharing and online study aids, collaboration and sharing may have become a student norm. This multi-institutional, qualitative study examined faculty and student perceptions of and student motivations for unauthorized collaboration. METHOD: Using a constructivist approach, the authors conducted scenario-prompted semistructured interviews with faculty and students in the preclinical curriculum. Participants were asked to reflect on scenarios of unauthorized collaboration and discuss their perceptions of student motivation and the influence of personal or environmental factors. The authors performed inductive thematic analysis of the interview transcripts using open and axial coding followed by abstraction and synthesis of themes. RESULTS: Twenty-one faculty and 16 students across 3 institutions were interviewed in 2021. There was variation in perceptions among faculty and among students, but little variation between faculty and students. Both participant groups identified the same 3 areas of tension/themes: faculty/curriculum goals vs student goals, inherent character traits vs modifiable behavioral states, and student relationships with their peer group vs their relationships with the medical education system. Student behaviors were perceived to be influenced by their environment and motivated by the desire to help peers. Participants suggested cultivating trust between students and the education system, environmental interventions, and educating students about acceptable and unacceptable behaviors to prevent unauthorized collaboration. CONCLUSIONS: Given the various tensions and positive motivations behind unauthorized collaborations, institutions should consider explicitly preparing students to make thoughtful decisions when faced with competing priorities in addition to developing mitigation strategies that address the environment and its interactions with students.

Using Self-Regulated Learning Microanalysis to Examine Regulatory Processes in Clerkship Students Engaged in Practice Questions.

Introduction: Self-regulated learning is a cyclical process of forethought, performance, and self-reflection that has been used as an assessment tool in medical education. No prior studies have evaluated SRL processes for answering multiple-choice questions (MCQs) and most evaluated one or two iterations of a non-MCQ task. SRL assessment during MCQs may elucidate reasons why learners are successful or not on these questions that are encountered repeatedly during medical education. Methods: Internal medicine clerkship students at three institutions participated in a SRL microanalytic protocol that targeted strategic planning, metacognitive monitoring, causal attributions, and adaptive inferences across seven MCQs. Responses were transcribed and coded according to previously published methods for microanalytic protocols. Results: Forty-four students participated. In the forethought phase, students commonly endorsed prioritizing relevant features as their diagnostic strategy (n = 20, 45%) but few mentioned higher-order diagnostic reasoning processes such as integrating clinical information (n = 5, 11%) or comparing/contrasting diagnoses (n = 0, 0%). However, in the performance phase, students' metacognitive processes included high frequencies of integration (n = 38, 86%) and comparing/contrasting (n = 24, 55%). In the self-reflection phase, 93% (n = 41) of students faulted their management reasoning and 84% (n = 37) made negative references to their abilities. Less than 10% (n = 4) of students indicated that they would adapt their diagnostic reasoning process for these questions. Discussion: This study describes in detail student self-regulatory processes during MCQs. We found that students engaged in higher-order diagnostic reasoning processes but were not explicit about it and seldom reflected critically on these processes after selecting an incorrect answer. Self-reflections focused almost exclusively on management reasoning and negative references to abilities which may decrease self-efficacy. Encouraging students to identify and evaluate diagnostic reasoning processes and make attributions to controllable factors may improve performance.

The Impact on Medical Student Stress in Relation to a Change in USMLE Step 1 Examination Score Reporting to Pass/Fail.

The United States Medical Licensing Examination (USMLE) Step 1 was designed to be a benchmark measure of knowledge and has been used heavily in the residency application process. Step 1 has moved from 3-digit scoring to a pass/fail scoring system, in part to decrease the stress associated with the exam. Emerging literature suggests that this transition has led to other stresses for students. Our study compared student stress levels, both overall and in relation to Step 1, leading up to the exam between a scored cohort and pass/fail cohort. We administered to each cohort a 14-item survey that included demographics, the PSS-4 stress scale, and 6 other potential stressors. Data was analyzed using two-tailed t test for independent means and analysis of variance. We found that while there was no difference in general overall stress between the students who took Step 1 for a score and students who took Step 1 pass/fail, we did see differences in stress related to the Step 1 exam. Step 1 stress was significantly lower for the pass/fail cohort than the score cohort during the second year of medical education leading up to the exam. However, this difference in Step 1 stress between the cohorts disappeared by the dedicated study period immediately before the exam. The change in scoring appears to have decreased stress specifically related to Step 1, but this reduction was not sustained as students entered their study period to prepare for Step 1.

The Consequences of Step 2 Clinical Skills Examination Discontinuation for Medical Schools and Sustainability Plans for Clinical Skills Assessment.

PURPOSE: Comprehensive clinical skills examinations using standardized patients are widely used to assess multiple physician competencies. However, these exams are resource intensive. With the discontinuation of the Step 2 Clinical Skills (CS) exam in 2021, how medical schools will change their approaches to comprehensive clinical skills exams is unknown. This study explores school responses to this change and future directions of comprehensive clinical skills exams using the program sustainability framework. METHOD: This cross-sectional, descriptive study surveyed medical school curriculum deans at 150 Liaison Committee on Medical Education-accredited U.S. medical schools from September to October 2021. The 30-question survey included questions about medical school and participant role, current comprehensive clinical skills exams, sustainability dimensions, and challenges and future directions. Descriptive statistics were used to characterize responses, and content analysis was used to identify themes in the open-ended responses. RESULTS: Educators at 75 of 150 institutions (50%) responded. Sixty-three respondents (84%) reported conducting a comprehensive clinical skills exam. The comprehensive clinical skills exam assessed readiness for graduation (51 [81%]), provided feedback for students (49 [78%]), evaluated curricula (38 [60%]), provided information for medical student performance evaluation or communication with residency (10 [16%]), and assessed other factors (6 [10%]), including preparation for Step 2 CS in the past and readiness for advancement to fourth year of medical school (multiple responses were allowed). Factors facilitating sustainability included sufficient funding to continue the exam (55 [87%]) and the belief that clinical skills assessment in medical school is now more important after discontinuation of the Step 2 CS exam (55 [87%]). Challenges to sustainability included organizational capacity and limited interinstitutional collaboration. CONCLUSIONS: Educators remain committed to the purpose of comprehensive clinical skills exams. Adapting to changed licensing requirements while sustaining clinical skills exams enables innovation and improvement in assessment of clinical competence.

When students' words hurt: 12 tips for helping faculty receive and respond constructively to student evaluations of teaching.

Student evaluations of curricular experiences and instructors are employed by institutions to obtain feedback and guide improvement. However, to be effective, evaluations must prompt faculty action. Unfortunately, evaluative comments that engender strong reactions may undermine the process by hindering innovation and improvement steps. The literature suggests that faculty interpret evaluation feedback as a judgment not just on their teaching ability but on their personal and professional identity. In this context, critical evaluations, even when constructively worded, can result in disappointment, hurt, and shame. The COVID pandemic has challenged institutions and faculty to repeatedly adapt curricula and educational practices, heightening concerns for faculty burnout. In this context, the risk of 'words that hurt' is higher than ever. This article offers guidance for faculty and institutions to support effective responses to critical feedback and ameliorate counterproductive effects of learner evaluations.

The COVID-19 Pandemic as an Imperative to Advance Medical Student Assessment: Three Areas for Change.

Conditions caused by the COVID-19 pandemic have disrupted clinical practice and all aspects of medical education. Yet the need to continue to train physicians to care for patients and communities is greater than ever. Medical educators are responding by adapting curricula in response to requirements for social distancing, changing availability of clinical learning opportunities, and limitations on testing center availability for examinations. These disruptions require a systems approach to ensure that learners achieve competence in preparation for advancement in training toward unsupervised practice. In this article, the authors assert that medical educators, obligated by current conditions to adapt educational experiences, should seize the opportunity presented by the pandemic to make needed changes in 3 areas aligned with competency-based medical education: focusing on outcomes, broadening the assessment toolbox, and improving the undergraduate medical education-to-graduate medical education (UME-to-GME) transition. Defined outcomes, as exemplified by entrustable professional activities, will enable curricular designers to shorten and adapt learning experiences by focusing on students' achievement of prespecified learning outcomes. Broadening the assessment toolbox entails capturing more and different assessment information about learners to provide a well-rounded view of their strengths and areas for growth in both traditional and novel settings, such as telehealth. Limitations on available data, such as licensing examination scores and clerkship grades, heighten the urgency to revise the system for the UME-to-GME transition by enhancing the quality and usability of information available to residency program directors. Educators should capitalize on the opportunity presented by altered conditions due to the COVID-19 pandemic to make these needed changes to the educational system, to prepare physicians to provide health care and lead the health care system into the future.

Medical Students' Perspectives on Entrustment Decision Making in an Entrustable Professional Activity Assessment Framework: A Secondary Data Analysis.

PURPOSE: Research on how entrustment decisions are made identifies 5 influential factors (supervisor, trainee, supervisor-trainee relationship, context, task). However, this literature primarily represents the perspective of supervisors in graduate medical education and is conducted outside of an assessment framework where entrustment decisions have consequences for trainees and for patients. To complement the literature, the authors explored how medical students in a pilot program that used an entrustable professional activity (EPA) assessment framework perceived factors influencing entrustment decisions. METHOD: The authors conducted a secondary analysis of qualitative data from program evaluation of a pilot project using an EPA assessment framework to advance students through their curriculum and into residency. Data were derived from 8 focus groups conducted over 4 years (2015-2018) with 27 students who shared their experience of learning and supervision in the project. Sensitized by the entrustment literature, the authors coded and analyzed focus group transcripts according to principles of thematic analysis. RESULTS: Components of the trainee and supervisor-trainee relationship factors predominated students' perceptions of entrustment decisions. Students directed their own learning by asking for feedback, seeking opportunities to engage in learning, sharing limitations of their knowledge with supervisors, and pushing supervisors to recalibrate assessments when appropriate. Students' agentic actions were facilitated by longitudinal supervisor-trainee relationships wherein they felt comfortable asking for help and built confidence in patient care. Students mentioned components of other factors that influenced entrustment decisions (supervisor, clinical task, clinical context), but did so less frequently and from a nonagentic vantage point. CONCLUSIONS: Students' perspectives on entrustment decisions can be derived from their views on learning and supervision in an EPA assessment framework. Their perspectives complement the literature by highlighting students' agentic actions to influence entrustment decisions and promotion of agentic action through practices incorporating longitudinal supervisor-trainee relationships.

The ingredients of a rich entrustment decision.

Entrustment decision-making has become a topic of interest in workplace-based assessment in the health professions and is germane to the use of entrustable professional activities. Entrustment decisions stem from judgments of a trainee's competence and include the permission to act with a higher level of responsibility or autonomy and a lower level of supervision. Making entrustment decisions differs from regular assessment of trainees, which usually has no consequences beyond marking trainee progress. Studies show that clinicians generally weigh more factors in making an entrustment decision than when merely assessing trainee competence or performance without direct consequences for patient care. To synthesize the varying factors reported in literature, the authors performed a thematic analysis of key qualitative studies that investigated trainee features clinical supervisors find important when making entrustment decisions. Five themes emerged from the 13 publications: Capability (specific knowledge, skills, experience, situational awareness), Integrity (truthful, benevolent, patient-centered), Reliability (conscientious, predictable, accountable, responsible), Humility (recognizes limits, asks for help, receptive to feedback), Agency (proactive toward work, team, safety, personal development). Thoughtful entrustment decisions, made either by individual clinical supervisors or by clinical competency committees, may be enriched by taking into account these five features.

Assessing Trainees and Making Entrustment Decisions: On the Nature and Use of Entrustment-Supervision Scales.

Clinical teachers are continuously entrusting trainees with care responsibilities in health care settings. Entrustable professional activities employ entrustment decision making as an approach to assessment in the workplace.Various scales have been created to measure "entrustment," all basically expressing the level or type of supervision a trainee requires for safe and high-quality care. However, some of these scales are only weakly related to the purpose of making decisions about the autonomy trainees will be granted. The authors aim to increase understanding about the nature, purpose, and practice of supervision scales aimed at entrustment.After arguing for entrustment as a component of workplace-based assessment, the distinction between ad hoc entrustment decisions (daily decisions in health care settings) and summative entrustment decisions (with a certifying nature) is clarified. Next, the noncontinuous nature of entrustment-supervision (ES) scales, as opposed to most workplace-based assessment scales, is explained. ES scales have ordinal, rather than interval, properties and focus on discrete decisions. Finally, some scales are retrospective ("how much supervision was provided?"), and others are prospective ("how much supervision will be needed in the near future?"). Although retrospective scales reflect observed behavior, prospective scales truly focus on entrustment and ask for more holistic judgment, as they include a broader evaluation and a risk estimation to enable a decision about increase of autonomy.The analysis concludes with a discussion about entrustment for unsupervised practice and supervision of others, as well as the program, context, and specialty specificity of scales.

Developing a Novel 4-C Framework to Enhance Participation in Faculty Development.

Phenomenon: Universities offer a variety of voluntary faculty development to ensure quality education, but face inconsistent faculty participation. Therefore, all Dutch universities require all faculty to obtain a teaching qualification certificate. Yet, like other medical centers, University Medical Center Utrecht continued to struggle with faculty nonparticipation. It has been postulated that clinician teachers may face unique challenges with responsibilities for patient care in addition to teaching and research, challenges that cannot be overcome by merely mandating faculty development or a teaching certificate. This project was conducted to gain insight into factors that hinder faculty participation and better understand what is needed to enhance faculty engagement in their professional development as teachers. Approach: UMC Utrecht has had a teaching certificate requirement for over 20 years. In 2015-2016, we conducted a local needs assessment, gathering faculty perspectives about the teaching certification process. To convey seriousness of purpose and promote commitment to change, we formally engaged key stakeholders from the outset, obtained grant funding for the needs assessment, and had an outside consultant lead the project. Faculty who were stalled or never started were questioned via semi-structured interviews. A focus group with those actively in the process of obtaining their certificate discussed perceived challenges in the process and recommended solutions. Faculty who obtained their teaching certificate completed an anonymous evaluation form. All evaluation comments and transcripts were thematically analyzed using open and axial coding. A literature review was performed to contextualize our findings and identify potential solutions. We compared our initial themes to these findings and found key challenge/solution categories, which we subsequently developed into a novel framework. Findings from the study and literature review were organized using this framework and shared with different stakeholders, all of whom engaged in problem-solving. Ideas and potential solutions were incorporated into a final report with recommendations for improving faculty support and provided to the institutional leadership. Findings: Of 23 faculty teachers approached, 8 (34.8%) agreed to be interviewed; 7 of 25 (28.0%) participated in the focus group; and 83 of 156 (53.2%) completed the evaluation. From the transcripts and evaluation comments, three themes emerged related to context and barriers: (a) skill development versus certification; (b) workplace priorities and culture, and (c) visibility and feasibility of the teacher's role. Triangulation of these themes with the literature revealed four challenge/solution categories - Competence, Context, Community, and Career. This 4-C framework facilitated communication of findings, structured the development of an action plan in response to the findings, and assured implementation of new initiatives for faculty support beyond competence development. Insights: Simply adopting requirements for faculty development may be insufficient and even invoke resistance. Improving faculty participation in faculty development and the quality of education requires institutional attention to not just faculty Competence needs, but also the factors of Context, Community, and Career that together comprise the culture experienced by faculty teachers. With institutional buy-in and commitment to change, the 4-C framework can help focus institutional attention on existing gaps in all four domains and guide the development of comprehensive solutions.

Scoping Review of Entrustable Professional Activities in Undergraduate Medical Education.

PURPOSE: Entrustable professional activities (EPAs) are a hot topic in undergraduate medical education (UME); however, the usefulness of EPAs as an assessment approach remains unclear. The authors sought to better understand the literature on EPAs in UME through the lens of the 2010 Ottawa Conference Criteria for Good Assessment. METHOD: The authors conducted a scoping review of the health professions literature (search updated February 2018), mapping publications to the Ottawa Criteria using a collaboratively designed charting tool. RESULTS: Of the 1,089 publications found, 71 (6.5%) met inclusion criteria. All were published after 2013. Forty-five (63.4%) referenced the 13 Core Entrustable Professional Activities for Entering Residency developed by the Association of American Medical Colleges (AAMC). Forty (56.3%) were perspectives, 5 (7.0%) were reviews, and 26 (36.6%) were prospective empirical studies. The publications mapped to the Ottawa Criteria 158 times. Perspectives mapped more positively (83.7%) than empirical studies (76.7%). Reproducibility did not appear to be a strength of EPAs in UME; however, reproducibility, equivalence, educational effect, and catalytic effect all require further study. Inconsistent use of the term "EPA" and conflation of concepts (activity vs assessment vs advancement decision vs curricular framework) limited interpretation of published results. Overgeneralization of the AAMC's work on EPAs has influenced the literature. CONCLUSIONS: Much has been published on EPAs in UME in a short time. Now is the time to move beyond opinion, clarify terms, and delineate topics so that well-designed empirical studies can demonstrate if and how EPAs should be implemented in UME.

Addressing the hidden curriculum in the clinical workplace: A practical tool for trainees and faculty.

INTRODUCTION: The hidden curriculum, commonly described in negative terms, is considered highly influential in medical education, especially in the clinical workplace. Structured approaches to address it are limited in number and scope. METHODS AND RESULTS: This paper presents a practical, value-neutral method called REVIEW (Reflecting & Evaluating Values Implicit in Education in the Workplace), to facilitate reflection and discussion on the hidden curriculum by faculty members and trainees. REVIEW approaches the hidden curriculum as a reflection of the professional microculture of a clinical team. This microculture results from collective problem solving and mutual negotiation when facing different, often conflicting, demands and interests, and their underlying values in daily clinical practice. Using this nonjudgmental conceptual framework, REVIEW employs a series of 50 culture statements that must be prioritized using Q-sort methodology, reflecting how the culture in a particular clinical context (e.g. ward or department) is perceived by faculty members and trainees. This procedure can be done individually or in groups. Most important is the resulting team discussion after the exercise - a discussion about perceptions of actual team culture and the culture desired by the team. DISCUSSION AND CONCLUSIONS: Our early experiences suggest that REVIEW can be a useful tool for addressing the hidden curriculum.

The Feasibility of Longitudinal Patient Contacts in a Large Medical School.

PROBLEM: Longitudinal patient contacts are being implemented worldwide as a way to enhance a patient-centered orientation among medical students. In large medical schools, longitudinal integrated clerkships may not be feasible, so other ways must be sought to expose students to prolonged contact with patients. INTERVENTION: Medical students were attached to a family practice and assigned a panel of 4 patients to follow over the 3 years of their clinical training. Their role was that of companion on the patient's medical journey. The program consisted of several encounters, joining the patient in the medical setting for significant events, and written assignments. This intervention was piloted with 35 students. We describe our experiences from the 1st pilot year of this program. CONTEXT: The intervention was performed with 3rd-year students-of a 6-year curriculum-at a large medical school in the Netherlands. OUTCOME: Finding enough patients per practice was feasible. On the whole, students fulfilled the program's expectations regarding frequency of patient encounters and assignments. The most frequent problems encountered by the students were uncertainty about their role and setting boundaries in their contact with the patients. They needed more preceptor supervision and coaching than they received. LESSONS LEARNED: For junior students, close and structured supervision led by the faculty is necessary to help them navigate and learn from a panel of patients. Students need guidance about what role they should take on and on how to manage both their own and their patient's expectations. Guided reflection is necessary to help students give meaning to their experiences with patients.

Examining the readiness of best evidence in medical education guides for integration into educational practice: A meta-synthesis.

BACKGROUND: To support evidence-informed education, health professions education (HPE) stakeholders encourage the creation and use of knowledge syntheses or reviews. However, it is unclear if these knowledge syntheses are ready for translation into educational practice. Without understanding the readiness, defined by three criteria-quality, accessibility and relevance-we risk translating weak evidence into practice and/or providing information that is not useful to educators. METHODS: A librarian searched Web of Science for knowledge syntheses, specifically Best Evidence in Medical Education (BEME) Guides. This meta-synthesis focuses on BEME Guides because of their explicit goal to inform educational practice and policy. Two authors extracted data from all Guides, guided by the 25-item STructured apprOach to the Reporting In healthcare education of Evidence Synthesis (STORIES). RESULTS: Forty-two Guides published in Medical Teacher between 1999 and 2017 were analyzed. No Guide met all STORIES criteria, but all included structured summaries and most described their literature search (n = 39) and study inclusion/exclusion (n = 40) procedures. Eleven Guides reported the presence of theory and/or educational principles, and eight consulted with external subject matter experts. Accessibility to each Guide's full-text and supplemental materials was variable. DISCUSSION: For a subset of HPE knowledge syntheses, BEME Guides, this meta-synthesis identifies factors that support readiness and indicates potential areas of improvement, such as consistent access to Guides and inclusion of external subject matter experts on the review team. This analysis is useful for understanding the current readiness of HPE knowledge syntheses and informing future reviews to evolve so they can catalyze translation of evidence into educational practice.

Twelve tips for the implementation of EPAs for assessment and entrustment decisions.

The concept of entrustable professional activities (EPAs) reframes the approach to assessment in competency-based medical education. Key to this concept is the linking of assessment to decision making about entrusting learners with clinical responsibilities. Based on recent literature and the authors' experiences with implementing EPAs, this article provides practical recommendations for how to implement EPAs for assessment and entrustment decisions in the workplace. Tips for supervising clinicians include talking to learners about trust, using EPA descriptions to guide learning and teaching, providing learners with greater ad hoc responsibilities, using EPAs to identify/create opportunities for assessment and feedback, including case-based discussions and acknowledging gut feelings about learner readiness for more autonomy. Tips for curriculum leaders entail enabling the trust development, applying trust decisions at all levels of the supervision scale, employing all available information sources for entrustment, empowering learner ownership of the assessment process and using technology for learner tracking and program evaluation.