Viewing Readiness-for-Residency through Binoculars: Mapping Competency-Based Assessments to the AAMC's 13 Core Entrustable Professional Activities (EPAs).

Construct: The construct being assessed is readiness-for-residency of graduating medical students, as measured through two assessment frameworks. Background: Readiness-for-residency of near-graduate medical students should be but is not consistently assessed. To address this, the Association of American Medical Colleges (AAMC), in 2014, identified and described 13 core Entrustable Professional Activities (EPAs), which are tasks that all residents should be able to perform unsupervised upon entering residency. However, the AAMC did not initially provide measurement guidelines or propose standardized assessments. We designed Night-onCall (NOC), an immersive simulation for our near-graduating medical students to assess and address their readiness-for-residency, framed around tasks suggested by the AAMC's core EPAs. In adopting this EPA assessment framework, we began by building upon an established program of competency-based clinical skills assessments, repurposing competency-based checklists to measure components of the EPAs where possible, and designing new checklists, when necessary. This resulted in a blended suite of 14 checklists, which theoretically provide substantive assessment of all 13 core EPAs. In this paper, we describe the consensus-based mapping process conducted to ensure we understood the relationship between competency and EPA-based assessment lenses and could therefore report meaningful feedback on both to transitioning students in the NOC exercise. Approach: Between January-November 2017, five clinician and two non-clinician health professions educators at NYU Grossman School of Medicine conducted a rigorous consensus-based mapping process, which included each rater mapping each of the 310 NOC competency-based checklist items to lists of entrustable behaviors expected of learners according to the AAMC 13 core EPAs. Findings: All EPAs were captured to varying degrees by the 14 NOC checklists (overall Intraclass Correlation Coefficient (ICC) = 0.77). Consensus meetings resolved discrepancies and improved ICC values for three (EPA-9, EPA-10, EPA-12) of the four EPAs that initially showed poor reliability. Conclusions: Findings suggest that with some limitations (e.g., EPA-7 "form clinical questions/retrieve evidence") established competency-based assessments can be repurposed to measure readiness-for-residency through an EPA lens and both can be reported to learners and faculty.

Professionalism Training For Surgical Residents: Documenting the Advantages of a Professionalism Curriculum.

OBJECTIVES: Professionalism education is a vital component of surgical training. This research attempts to determine whether an annual, year-long professionalism curriculum in a large surgical residency can effectively change professionalism attitudes. SUMMARY OF BACKGROUND DATA: The ACGME mandated 6 competencies in 2003. The competencies of Professionalism and Interpersonal/Professional Communication Skills had never been formally addressed in surgical resident education in the past. METHODS: A professionalism curriculum was developed focusing on specific resident professionalism challenges: admitting mistakes, effective communication with colleagues at all levels, delivering the news of an unexpected death, interdisciplinary challenges of working as a team, the cultural challenge of obtaining informed consent through an interpreter, and the stress of surgical practice on you and your family. These professionalism skills were then evaluated with a 6-station Objective Structured Clinical Examination (OSCE). Identical OSCE scenarios were administered to 2 cohorts of surgical residents: in 2007 (before instituting the professionalism curriculum in 2008) and again in 2014. Surgical residents were rated by trained Standardized Patients according to a behaviorally anchored professionalism criteria checklist. RESULTS: An analysis of variance was conducted of overall OSCE professionalism scores (% well done) as the dependent variable for the 2 resident cohorts (2007 vs 2014). The 2007 residents received a mean score of 38% of professionalism items "well done" (SD 9%) and the 2014 residents received a mean 59% "well done" (SD 8%). This difference is significant (F = 49.01, P < .001). CONCLUSIONS: Professionalism education has improved surgical resident understanding, awareness, and practice of professionalism in a statistically significant manner from 2007 to 2014. This documented improvement in OSCE performance reflects the value of a professionalism curriculum in the care of the patients we seek to serve.

Barriers and facilitators to providing primary care-based weight management services in a patient centered medical home for Veterans: a qualitative study.

BACKGROUND: Obesity is highly prevalent among Veterans. In the United States, the Veterans Health Administration (VHA) offers a comprehensive weight management program called MOVE!. Yet, fewer than 10 % of eligible patients ever attend one MOVE! visit. The VHA has a patient-centered medical home (PCMH) model of primary care (PC) called Patient-Aligned Care Teams (PACT) at all Veterans Affairs (VA) Medical Centers. PACT teamlets conduct obesity screening, weight management counseling, and refer to MOVE!. As part of a needs assessment to improve delivery of weight management services, the purpose of this study was to assess PACT teamlet and MOVE! staff: 1) current attitudes and perceptions regarding obesity care; 2) obesity-related counseling practices 3) experiences with the MOVE! program; and 4) targets for interventions to improve implementation of obesity care in the PC setting. METHODS: We recruited 25 PACT teamlet members from a single VA study site-11 PC physicians, 5 registered nurses, 5 licensed practical nurses, 1 clerical assistant, and 3 MOVE! staff (2 dietitians, 1 psychologist)-for individual interviews using a combination of convenience and snowball sampling. Audio recorded interviews were professionally transcribed and iteratively coded by two independent reviewers. The analytic process was guided by discourse analysis in order to discover how the participants perceived and provided weight management care and what specific attitudes affected their practices, all as bounded within the organization. RESULTS: Emerging themes included: 1) role perceptions, 2) anticipated outcomes of weight management counseling and programs, and 3) communication and information dissemination. Perceived role among PCPs was influenced by training, whereas personal experience with their own weight management impacted role perception among LPNs/RNs. Attitudes about whether or not they could impact patients' weight outcomes via counseling or referral to MOVE! varied. System-level communication about VHA priorities through electronic health records and time allocation influenced teams to prioritize referral to MOVE! over weight management counseling. CONCLUSION: We found a diversity of attitudes, and practices within PACT, and identified factors that can enhance the MOVE! program and inform interventions to improve weight management within primary care. Although findings are site-specific, many are supported in the literature and applicable to other VA and non-VA sites with PCMH models of care.

Twelve tips for improving the effectiveness of web-based multimedia instruction for clinical learners.

Using educational technology does not necessarily make medical education more effective. There are many different kinds of technology available to the contemporary medical teacher and what constitutes effective use may depend on the technology, the learning situation and many other factors. Web-based multimedia instruction (WBMI) provides learners with self-directed independent learning opportunities based on didactic material enhanced with multimedia features such as video and animations. WBMI may be used to replace other didactic events (e.g. lectures) or it may be provided in addition to other learning opportunities. Clinical educators looking to use WBMI need to make sure that it will meet both their learners' needs and the program's needs, and it has to align to the contexts in which it is used. The following 12 tips have been developed to help guide faculty through some of the key features of the effective use of WBMI in clinical teaching programs. These tips are based on more than a decade developing, using and appraising WBMI in support of surgical clerkship education across the USA and beyond and they are intended both to inform individual uses of WBMI in clinical training and to guide the strategic use of WBMI in clinical clerkship curricula.

Context matters: emergent variability in an effectiveness trial of online teaching modules.

CONTEXT: Conducting research in real life settings (effectiveness studies) can introduce many confounding factors. Efficacy studies seek to control for researcher bias and data quality rather than considering how the efficacy of an intervention is changed by the contexts in which it is used. Relatively little is known about the impact of context on educational interventions, in particular on multimedia learning. METHODS: An effectiveness study to understand implementation variance of online educational modules in surgery clerkships was conducted in six US medical schools participating in an efficacy trial of different multimedia designs. Student and teacher experiences were captured through focus groups and one-to-one interviews with trial participants and their teachers. Audio-recordings of these sessions were transcribed and analysed using grounded theory techniques. RESULTS: Differences were identified in student and teacher perceptions of how the educational intervention had been implemented and how its uptake had been influenced by context-dependent factors: (i) the intervention was implemented in different ways to suit different educational contexts and this influenced how students and teachers responded to it; (ii) the ways students and teachers interacted with, and behaved around, the intervention influenced its uptake; (iii) the way the intervention was perceived by students and teachers influenced its uptake; and (iv) the medium and design of the intervention had a directing influence on its uptake. CONCLUSIONS: It was observed that each institutional context formed a complex educational ecology. The intervention became interwoven with different educational ecologies so that it could no longer be considered a stable variable across the study. We suggest that researchers should conduct implementation-profiling studies in advance of any intervention-based research to account for the constructing nature of educational ecologies on their interventions and in doing so to more clearly differentiate between efficacy and effectiveness studies.

The cognitive impact of interactive design features for learning complex materials in medical education.

To identify the most effective way for medical students to interact with a browser-based learning module on the symptoms and neurological underpinnings of stroke syndromes, this study manipulated the way in which subjects interacted with a graphical model of the brain and examined the impact of functional changes on learning outcomes. It was hypothesized that behavioral interactions that were behaviorally more engaging and which required deeper consideration of the model would result in heightened cognitive interaction and better learning than those whose manipulation required less deliberate behavioral and cognitive processing. One hundred forty four students were randomly assigned to four conditions whose model controls incorporated features that required different levels of behavioral and cognitive interaction: Movie (low behavioral/low cognitive, n = 40), Slider (high behavioral/low cognitive, n = 36), Click (low behavioral/high cognitive, n = 30), and Drag (high behavioral/high cognitive, n = 38). Analysis of Covariates (ANCOVA) showed that students who received the treatments associated with lower cognitive interactivity (Movie and Slider) performed better on a transfer task than those receiving the module associated with high cognitive interactivity (Click and Drag, partial eta squared = .03). In addition, the students in the high cognitive interactivity conditions spent significantly more time on the stroke locator activity than other conditions (partial eta squared = .36). The results suggest that interaction with controls that were tightly coupled with the model and whose manipulation required deliberate consideration of the model's features may have overtaxed subjects' cognitive resources. Cognitive effort that facilitated manipulation of content, though directed at the model, may have resulted in extraneous cognitive load, impeding subjects in recognizing the deeper, global relationships in the materials. Instructional designers must, therefore, keep in mind that the way in which functional affordances are integrated with the content can shape both behavioral and cognitive processing, and has significant cognitive load implications.

Racial Implicit Bias and Communication Among Physicians in a Simulated Environment.

Importance: Racial implicit bias can contribute to health disparities through its negative influence on physician communication with Black patients. Interventions for physicians to address racial implicit bias in their clinical encounters are limited by a lack of high-fidelity (realistic) simulations to provide opportunities for skill development and practice. Objective: To describe the development and initial evaluation of a high-fidelity simulation of conditions under which physicians might be influenced by implicit racial bias. Design, Setting, and Participants: This cross-sectional study, performed on an online platform from March 1 to September 30, 2022, recruited a convenience sample of physician volunteers to pilot an educational simulation. Exposures: In the simulation exercise, physicians saw a 52-year-old male standardized patient (SP) (presenting as Black or White) seeking urgent care for epigastric pain, nausea, and vomiting. The case included cognitive stressors common to clinical environments, including clinical ambiguity, stress, time constraints, and interruptions. Physicians explained their diagnosis and treatment plan to the SP, wrote an assessment and management plan, completed surveys, and took the Race Implicit Association Test (IAT) and Race Medical Cooperativeness IAT. The SPs, blinded to the purpose of the study, assessed each physician's communication using skills checklists and global rating scales. Main Outcomes and Measures: Association between physicians' IAT scores and SP race with SP ratings of communication skills. Results: In 60 physicians (23 [38.3%] Asian, 4 [6.7%] Black, 23 [38.3%] White, and 10 [16.7%] other, including Latina/o/x, Middle Eastern, and multiracial; 31 [51.7%] female, 27 [45.0%] male, and 2 [3.3%] other), the interaction of physicians' Race IAT score and SP race was significant for overall communication (mean [SD] ß = -1.29 [0.41]), all subdomains of communication (mean [SD] ß = -1.17 [0.52] to -1.43 [0.59]), and overall global ratings (mean [SD] ß = -1.09 [0.39]). Black SPs rated physicians lower on communication skills for a given pro-White Race IAT score than White SPs; White SP ratings increased as physicians' pro-White bias increased. Conclusions and Relevance: In this cross-sectional study, a high-fidelity simulation calibrated with cognitive stressors common to clinical environments elicited the expected influence of racial implicit bias on physicians' communication skills. The outlined process and preliminary results can inform the development and evaluation of interventions that seek to address racial implicit bias in clinical encounters and improve physician communication with Black patients.

REACH: A Required Curriculum to Foster the Well-Being of Medical Students.

PROBLEM: The extent of medical student unwellness is well documented. Learner distress may impact patient care, workforce adequacy, and learners' performance and personal health. The authors describe the philosophy, structure, and content of the novel REACH (Recognize, Empathize, Allow, Care, Hold each other up) curriculum and provide a preliminary evaluation. APPROACH: The REACH curriculum is a mandatory, longitudinal well-being curriculum for first- and second-year medical students at the Medical College of Wisconsin (MCW) designed to prepare them for the emotional life of being a physician. The curriculum uses a framework, core concepts, and skills from the field of trauma stewardship. It builds on effective medical student well-being interventions (e.g., mindfulness-based training) and the sharing of personal stories by instructors during didactic and small-group sessions that are integrated into the regular MCW curriculum. During the first 2 years of implementation (2018-2019 and 2019-2020), the curriculum was evaluated using mid- and postcurriculum student surveys. OUTCOMES: Over 700 students have completed the REACH curriculum as of March 2022. Overall, most students who responded to the surveys in 2018-2020 reported that they felt the REACH curriculum material was important, that the curriculum met their expectations for a quality medical school course, and that they would recommend other schools incorporate a similar curriculum. Respondents to the 2019-2020 postcurriculum survey indicated the REACH curriculum helped them develop self-care (84% [85/101]), mindfulness (76% [76/101]), and help-seeking (71% [72/101]) skills. NEXT STEPS: The initial outcomes show that integrating a mandatory well-being curriculum is feasible and acceptable to medical students. The authors plan to examine the relationships between student-reported well-being metrics, academic and clinical performance data, and professional identity formation. They are also prototyping electronic dashboards that will allow students to interact with their well-being data to promote timely help-seeking and behavior change.

Assessing medical students' self-regulation as aptitude in computer-based learning.

We developed a Self-Regulation Measure for Computer-based learning (SRMC) tailored toward medical students, by modifying Zimmerman's Self-Regulated Learning Interview Schedule (SRLIS) for K-12 learners. The SRMC's reliability and validity were examined in 2 studies. In Study 1, 109 first-year medical students were asked to complete the SRMC. Bivariate correlation analysis results indicated that the SRMC scores had a moderate degree of correlation with student achievement in a teacher-developed test. In Study 2, 58 third-year clerkship students completed the SRMC. Regression analysis results indicated that the frequency of medical students' usage of self-regulation strategies was associated with their general clinical knowledge measured by a nationally standardized licensing exam. These two studies provided evidence for the reliability and concurrent validity of the SRMC to assess medical students' self-regulation as aptitude. Future work should provide evidence to guide and improve instructional design as well as inform educational policy.

From Stigma to Validation: A Qualitative Assessment of a Novel National Program to Improve Retention of Physician-Scientists with Caregiving Responsibilities.

Background: Research is needed to improve understanding of work-life integration issues in academic medicine and to guide the implementation of the Doris Duke Charitable Foundation's Fund to Retain Clinical Scientists (FRCS), a national initiative offering financial support to physician-scientists facing caregiving challenges. Materials and Methods: In 2018, as part of a prospective program evaluation, the authors conducted a qualitative study to examine FRCS program participants' initial impressions, solicit descriptions of their career and caregiving experiences, and inquire how such factors might influence their professional advancement. The authors invited all 33 awardees who had been granted FRCS funding in the first year of the program to participate in the study, of whom 28 agreed to complete an interview. Analysts evaluated de-identified transcripts and explicated the data using a thematic analysis approach. Results: While participants described aspects of a culture that harbor stigma against caregivers and impede satisfactory work-life integration, they also perceived an optimistic cultural shift taking place as a result of programs like the FRCS. Their comments indicated that the FRCS has the potential to influence culture if institutional leadership simultaneously fosters a community that validates individuals both as caregivers and as scientists. Conclusions: Insights garnered from this qualitative study suggest that there is a pressing need for institutional leaders to implement programs that can foster awareness and normalization of caregiving challenges. In addition to providing funding and other tangible resources, interventions should strive to reinforce a broader culture that affirms the presence of work-life integration challenges and openly embraces solutions.

Defining and Adopting Clinical Performance Measures in Graduate Medical Education: Where Are We Now and Where Are We Going?

Assessment and evaluation of trainees' clinical performance measures is needed to ensure safe, high-quality patient care. These measures also aid in the development of reflective, high-performing clinicians and hold graduate medical education (GME) accountable to the public. Although clinical performance measures hold great potential, challenges of defining, extracting, and measuring clinical performance in this way hinder their use for educational and quality improvement purposes. This article provides a way forward by identifying and articulating how clinical performance measures can be used to enhance GME by linking educational objectives with relevant clinical outcomes. The authors explore four key challenges: defining as well as measuring clinical performance measures, using electronic health record and clinical registry data to capture clinical performance, and bridging silos of medical education and health care quality improvement. The authors also propose solutions to showcase the value of clinical performance measures and conclude with a research and implementation agenda. Developing a common taxonomy of uniform specialty-specific clinical performance measures, linking these measures to large-scale GME databases, and applying both quantitative and qualitative methods to create a rich understanding of how GME affects quality of care and patient outcomes is important, the authors argue. The focus of this article is primarily GME, yet similar challenges and solutions will be applicable to other areas of medical and health professions education as well.

Proposed standards for medical education submissions to the Journal of General Internal Medicine.

To help authors design rigorous studies and prepare clear and informative manuscripts, improve the transparency of editorial decisions, and raise the bar on educational scholarship, the Deputy Editors of the Journal of General Internal Medicine articulate standards for medical education submissions to the Journal. General standards include: (1) quality questions, (2) quality methods to match the questions, (3) insightful interpretation of findings, (4) transparent, unbiased reporting, and (5) attention to human subjects' protection and ethical research conduct. Additional standards for specific study types are described. We hope these proposed standards will generate discussion that will foster their continued evolution.

What do resident physicians know about nutrition? An evaluation of attitudes, self-perceived proficiency and knowledge.

OBJECTIVE: Despite the increased emphasis on obesity and diet-related diseases, nutrition education remains lacking in many internal medicine training programs. We evaluated the attitudes, self-perceived proficiency, and knowledge related to clinical nutrition among a cohort of internal medicine interns. METHODS: Nutrition attitudes and self-perceived proficiency were measured using previously validated questionnaires. Knowledge was assessed with a multiple-choice quiz. Subjects were asked whether they had prior nutrition training. RESULTS: Of the 114 participants, 61 (54%) completed the survey. Although 77% agreed that nutrition assessment should be included in routine primary care visits, and 94% agreed that it was their obligation to discuss nutrition with patients, only 14% felt physicians were adequately trained to provide nutrition counseling. There was no correlation among attitudes, self-perceived proficiency, or knowledge. Interns previously exposed to nutrition education reported more negative attitudes toward physician self-efficacy (p = 0.03). CONCLUSIONS: Internal medicine interns' perceive nutrition counseling as a priority, but lack the confidence and knowledge to effectively provide adequate nutrition education.

Situating Remediation: Accommodating Success and Failure in Medical Education Systems.

There has been a widespread shift to competency-based medical education (CBME) in the United States and Canada. Much of the CBME discourse has focused on the successful learner, with relatively little attention paid to what happens in CBME systems when learners stumble or fail. Emerging issues, such as the well-documented problem of "failure to fail" and concerns about litigious learners, have highlighted a need for well-defined and integrated frameworks to support and guide strategic approaches to the remediation of struggling medical learners.This Perspective sets out a conceptual review of current practices and an argument for a holistic approach to remediation in the context of their parent medical education systems. The authors propose parameters for integrating remediation into CBME and describe a model based on five zones of practice along with the rules of engagement associated with each zone. The zones are "normal" curriculum, corrective action, remediation, probation, and exclusion.The authors argue that, by linking and integrating theory and practice in remediation with CBME, a more integrated systems-level response to differing degrees of learner difficulty and failure can be developed. The proposed model demonstrates how educational practice in different zones is based on different rules, roles, responsibilities, and thresholds for moving between zones. A model such as this can help medical educators and medical education leaders take a more integrated approach to learners' failures as well as their successes by being more explicit about the rules of engagement that apply in different circumstances across the competency continuum.

Medical training in school-based health centers: a collaboration among five medical schools.

School-based health centers (SBHCs) have tremendous untapped potential as models for learning about systems-based care of vulnerable children. SBHCs aim to provide comprehensive, community-based primary health care to primary and secondary schoolchildren who might not otherwise have ready access to that care. The staffing at SBHCs is multidisciplinary, including various combinations of nurse practitioners, physicians, dentists, nutritionists, and mental health providers. Although this unique environment provides obvious advantages to children and their families, medical students and residents receive little or no preparation for this type of practice. To address these deficiencies in medical education, five downstate New York state medical schools, funded by the New York State Department of Health, collaborated to define, develop, implement, and evaluate curricula that expose health professions students and residents to SBHCs. The schools identified core competencies and developed a comprehensive training model for the project, including clinical experiences, didactic sessions, and community service opportunities, and they developed goals, objectives, and learning materials for each competency for all types and levels of learners. Each school has implemented a wide range of learning activities based on the competencies. In this paper, the authors describe the development of the collaboration and illustrate the process undertaken to implement new curricula, including considerations made to address institutional needs, curricula development, and incorporation into existing curricula. In addition, they discuss the lessons learned from conducting this collaborative effort among medical schools, with the goal of providing guidance to establish effective cross-disciplinary curricula that address newly defined competencies.

Promoting professionalism through an online professional development portfolio: successes, joys, and frustrations.

Medical educators strive to promote the development of a sound professional identity in learners, yet it is challenging to design, implement, and sustain fair and meaningful assessments of professionalism to accomplish this goal. The authors developed and implemented a program built around a Web-based Professional Development Portfolio (PDP) to assess and document professional development in medical students at New York University School of Medicine. This program requires students to regularly document their professional development through written reflections on curricular activities spanning preclinical and clinical years. Students post reflections, along with other documents that chronicle their professional growth, to their online PDP. Students meet annually with a faculty mentor to review their portfolios, assess their professional development based on predetermined criteria, and establish goals for the coming year. In this article, the authors describe the development of the PDP and share four years of experience with its implementation. We describe the experiences and attitudes of the first students to participate in this program as reported in an annual student survey. Students' experiences of and satisfaction with the PDP was varied. The PDP has been a catalyst for honest and lively debate concerning the meaning and behavioral manifestations of professionalism. A Web-based PDP promoted self-regulation on an individual level because it facilitated narrative reflection, self-assessment, and goal setting, and it structured mentorship. Therefore, the PDP may prepare students for the self-regulation of the medical profession--a privilege and obligation under the physician's social contract with society.

Defining, navigating, and negotiating success: the experiences of mid-career Robert Wood Johnson Clinical Scholar women.

BACKGROUND: We studied female graduates of the Robert Wood Johnson Clinical Scholars Program (CSP, Class of 1984 to 1989) to explore and describe the complexity of creating balance in the life of mid-career academic woman physicians. METHODS: We conducted and qualitatively analyzed (kappa 0.35 to 1.0 for theme identification among rater pairs) data from a semi-structured survey of 21 women and obtained their curricula vitae to quantify publications and grant support, measures of academic productivity. RESULTS: Sixteen of 21 (76%) women completed the survey. Mean age was 48 (range: 45 to 56). Three were full professors, 10 were associate professors, and 3 had left academic medicine. Eleven women had had children (mean 2.4; range: 1 to 3) and 3 worked part-time. From these data, the conceptual model expands on 3 key themes: (1) defining, navigating, and negotiating success, (2) making life work, and (3) making work work. The women who described themselves as satisfied with their careers (10/16) had clarity of values and goals and a sense of control over their time. Those less satisfied with their careers (6/16) emphasized the personal and professional costs of the struggle to balance their lives and described explicit institutional barriers to fulfillment of their potential. CONCLUSION: For this group of fellowship-prepared academic women physicians satisfaction is achieving professional and personal balance.