Development and Initial Validation of the Shame Frequency Questionnaire in Medical Students.

PURPOSE: This study presents the steps taken to develop and collect initial validity evidence for the Shame Frequency Questionnaire in Medical Students. METHOD: The instrument was based on a 7-step survey design; validity evidence was collected from content, response process, internal structure, and relationship to other variables. A literature review and qualitative interviews led to the design of the initial 16-item scale. Expert review and cognitive interviewing led to minor modifications in the original structure. Initial pilot testing was conducted in August 2019 in Uniformed Services University (USU) medical students; reliability assessment and exploratory factor analysis were performed. The revised 12-item scale was tested in January 2022 in Duke University School of Medicine medical students; reliability assessment, exploratory factor analysis, and correlation analysis with depression, burnout, anxiety, emotional thriving, and emotional well-being were performed. RESULTS: A total of 336 of 678 USU students (50%) and 106 of 522 Duke students (20%) completed the survey. Initial exploratory factor analysis of the USU data revealed 1 factor (shame), and 4 items were dropped from the scale according to predefined rules. Subsequent exploratory factor analysis of the Duke data revealed 1 factor; no further items were removed according to predefined rules. Internal consistent reliability was 0.95, and all interitem correlations were less than 0.85 for USU and Duke samples. As predicted, mean shame scale scores were positively correlated with anxiety (r = 0.54, P < .001), burnout (r = 0.50, P < .001), and depression (r = 0.47, P < .001) and negatively correlated with emotional thriving (r = -0.46, P < .001) and emotional recovery (r = -0.46, P < .001). CONCLUSIONS: The Shame Frequency Questionnaire in Medical Students is a psychometrically sound instrument with strong internal reliability and multisource validity evidence, supporting its use in studying shame in medical students.

Well-being at a Military Medical School and Implications for Military Retention.

INTRODUCTION: Physical and psychological well-being play a critical role in the academic and professional development of medical students and can alter the trajectory of a student's quality of personal and professional life. Military medical students, given their dual role as officer and student, experience unique stressors and issues that may play a role in their future intentions to continue military service, as well as practice medicine. As such, this study explores well-being across the 4 years of medical school at Uniformed Services University (USU) and how well-being relates to a student's likelihood to continue serving in the military and practicing medicine. METHODS: In September 2019, 678 USU medical students were invited to complete a survey consisting of three sections-the Medical Student Well-being Index (MSWBI), a single-item burnout measure, and six questions regarding their likelihood of staying in the military and medical practice. Survey responses were analyzed using descriptive statistics, analysis of variance (ANOVA), and contingency table analysis. Additionally, thematic analysis was conducted on open-ended responses included as part of the likelihood questions. RESULTS: Our MSWBI and burnout scores suggest that the overall state of well-being among medical students at USU is comparable to other studies of the medical student population. ANOVA revealed class differences among the four cohorts, highlighted by improved well-being scores as students transitioned from clerkships to their fourth-year curriculum. Fewer clinical students (MS3s and MS4s), compared to pre-clerkship students, indicated a desire to stay in the military. In contrast, a higher percentage of clinical students seemed to "reconsider" their medical career choice compared to their pre-clerkship student counterparts. "Medicine-oriented" likelihood questions were associated with four unique MSWBI items, whereas "military-oriented" likelihood questions were associated with one unique MSWBI item. CONCLUSION: The present study found that the overall state of well-being in USU medical students is satisfactory, but opportunities for improvement exist. Medical student well-being seemed to have a stronger association with medicine-oriented likelihood items than with military-oriented likelihood items. To obtain and refine best practices for strengthening engagement and commitment, future research should examine if and how military and medical contexts converge and diverge throughout training. This may enhance the medical school and training experience and, ultimately, reinforce, or strengthen, the desire and commitment to practice and serve in military medicine.

Breaking Down Grit: Persistency and Flexibility in the Career Plans of Military Medical Students.

INTRODUCTION: The field of medicine is experiencing a crisis as high levels of physician and trainee burnout threaten the pipeline of future physicians. Grit, or passion and perseverance for long-term goals, has been studied in high-performing and elite military units and found to be predictive of successful completion of training in adverse conditions. The Uniformed Services University of the Health Sciences (USU) graduates military medical leaders who make up a significant portion of the Military Health System physician workforce. Taken together, an improved understanding of the relationships between burnout, well-being, grit, and retention among USU graduates is critical to the success of the Military Health System. MATERIALS AND METHODS: The current study was approved by the Institutional Review Board at USU and explored these relations among 519 medical students across three graduating classes. These students participated in two surveys approximately one year apart from October 2018 until November 2019. Participants completed measures on grit, burnout, and likelihood of leaving the military. These data were then merged with demographic and academic data (e.g., Medical College Admission Test scores) from the USU Long Term Career Outcome Study. These variables were then analyzed simultaneously using structural equation modeling to examine the relationships among variables in a single model. RESULTS: Results reaffirmed the 2-factor model of grit as both passion and perseverance (or interest consistency). No significant relationships emerged between burnout and other study variables. Sustained and focused interest was predictive of less likelihood of staying in the military. CONCLUSION: This study offers important insights into the relationship among well-being factors, grit, and long-term career planning in the military. The limitations of using a single-item measure of burnout and measuring behavioral intentions in a short time frame during undergraduate medical education highlight the importance of future longitudinal studies that can examine actual behaviors across a career lifespan. However, this study offers some key insights into potential impacts on the retention of military physicians. The findings suggest that military physicians who are most likely to stay in the military tend to embrace a more fluid and flexible medical specialty path. This is critical in expectation setting for the military to train and retain military physicians across a wide range of critical wartime specialties.

Pre-clerkship EPA assessments: a thematic analysis of rater cognition.

BACKGROUND: Entrustable Professional Activities (EPAs) assessments measure learners' competence with an entrustment or supervisory scale. Designed for workplace-based assessment EPA assessments have also been proposed for undergraduate medical education (UME), where assessments frequently occur outside the workplace and may be less intuitive, raising validity concerns. This study explored how assessors make entrustment determinations in UME, with additional specific comparison based on familiarity with prior performance in the context of longitudinal student-assessor relationships. METHODS: A qualitative approach using think-alouds was employed. Assessors assessed two students (familiar and unfamiliar) completing a history and physical examination using a supervisory scale and then thought-aloud after each assessment. We conducted a thematic analysis of assessors' response processes and compared them based on their familiarity with a student. RESULTS: Four themes and fifteen subthemes were identified. The most prevalent theme related to "student performance." The other three themes included "frame of reference," "assessor uncertainty," and "the patient." "Previous student performance" and "affective reactions" were subthemes more likely to inform scoring when faculty were familiar with a student, while unfamiliar faculty were more likely to reference "self" and "lack confidence in their ability to assess." CONCLUSIONS: Student performance appears to be assessors' main consideration for all students, providing some validity evidence for the response process in EPA assessments. Several problematic themes could be addressed with faculty development while others appear to be inherent to entrustment and may be more challenging to mitigate. Differences based on assessor familiarity with student merits further research on how trust develops over time.

ACGME Milestones in the Real World: A Qualitative Study Exploring Response Process Evidence.

Background: Since the Accreditation Council for Graduate Medical Education (ACGME) introduced the Milestones in 2013, the body of validity evidence supporting their use has grown, but there is a gap with regard to response process. Objective: The purpose of this study is to qualitatively explore validity evidence pertaining to the response process of individual Clinical Competency Committee (CCC) members when assigning Milestone ratings to a resident. Methods: Using a constructivist paradigm, we conducted a thematic analysis of semi-structured interviews with 8 Transitional Year (TY) CCC members from 4 programs immediately following a CCC meeting between November and December 2020. Participants were queried about their response process in their application of Milestone assessment. Analysis was iterative, including coding, constant comparison, and theming. Results: Participant interviews identified an absence of formal training and a perception that Milestones are a tool for resident assessment without recognizing their role in program evaluation. In describing their thought process, participants reported comparing averaged assessment data to peers and time in training to generate Milestone ratings. Meaningful narrative comments, when available, differentiated resident performance from peers. When assessment data were absent, participants assumed an average performance. Conclusions: Our study found that the response process used by TY CCC members was not always consistent with the dual purpose of the Milestones to improve educational outcomes at the levels of residents and the program.

A Pilot Study of the Generalizability of Preclinical Entrustment Assessments in Undergraduate Medical Education.

PURPOSE: The reproducibility and consistency of assessments of entrustable professional activities (EPAs) in undergraduate medical education (UME) have been identified as potential areas of concern. EPAs were designed to facilitate workplace-based assessments by faculty with a shared mental model of a task who could observe a trainee complete the task multiple times. In UME, trainees are frequently assessed outside the workplace by faculty who only observe a task once. METHOD: In November 2019, the authors conducted a generalizability study (G-study) to examine the impact of student, faculty, case, and faculty familiarity with the student on the reliability of 162 entrustment assessments completed in a preclerkship environment. Three faculty were recruited to evaluate 18 students completing 3 standardized patient (SP) cases. Faculty familiarity with each student was determined. Decision studies were also completed. Secondary analysis of the relationship between student performance and entrustment (scoring inference) compared average SP checklist scores and entrustment scores. RESULTS: G-study analysis revealed that entrustment assessments struggled to achieve moderate reliability. The student accounted for 30.1% of the variance in entrustment scores with minimal influence from faculty and case, while the relationship between student and faculty accounted for 26.1% of the variance. G-study analysis also revealed a difference in generalizability between assessments by unfamiliar (φ = 0.75) and familiar (φ = 0.27) faculty. Subanalyses showed that entrustment assessments by familiar faculty were moderately correlated to average SP checklist scores (r = 0.44, P < .001), while those by unfamiliar faculty were weakly correlated (r = 0.16, P = .13). CONCLUSIONS: While faculty and case had a limited impact on the generalizability of entrustment assessments made outside the workplace in UME, faculty who were familiar with a student's ability had a notable impact on generalizability and potentially on the scoring validity of entrustment assessments, which warrants further study.

Academies in Health Professions Education: A Scoping Review.

PURPOSE: Thirty years ago, academies were conceived as a sociocultural approach to revitalize the teaching mission of medical schools and to promote educators' career advancement. The academy movement has grown rapidly and now reaches a broad range of health professions education organizations. The authors conducted a scoping review to map the literature and describe the evidence that guides the formation of new academies and justifies the continuation of existing ones. METHOD: The authors searched MEDLINE (via Ovid), Embase (via Elsevier and Ovid), CINAHL (via EBSCOhost), and Web of Science (via Clarivate Analytics) from inception through March 6, 2020, for publications regarding academy-like organizations. They mapped the relevant literature using logic modeling as an organizing framework and included the mission, resources, activities, output, outcomes, and impact of the included academies. RESULTS: Of the 513 publications identified, 43 met the inclusion criteria, the oldest of which was published in 2000. Most publications were either case reports or perspective/opinion pieces (26, 57.8%), while studies presenting empirical findings were less common (11, 24.4%). Publications showed that academies were diversifying and increasingly were part of a broad range of organizations, including departments, hospitals, health science campuses, and national organizations. The mission, resources, and activities were similar across academies. Evaluation studies were largely limited to process measures, and rigorous studies examining outcomes (i.e., changes in academy participants) and impact on the organization at large were rare. CONCLUSIONS: The increase in the number of academy-related publications parallels the accelerating speed of the academy movement. To sustain this movement, rigorous studies must provide evidence that academies contribute to the revitalization of organizations' teaching mission and bring about an academic culture where educators thrive and where education is a legitimate career path.

EQual Rubric Evaluation of the Association of American Medical Colleges' Core Entrustable Professional Activities for Entering Residency.

PURPOSE: To have subject matter experts evaluate the Core Entrustable Professional Activities for Entering Residency (Core EPAs) with the EQual rubric to determine if revisions were required and, if applicable, how to focus revision efforts. METHOD: Ten entrustable professional activity (EPA) experts were invited to evaluate the 13 Core EPAs. Experts had a 6-month window (December 2018-May 2019) to complete the evaluation, which contained the complete EQual rubric and 3 additional prompts, one of which-"Do you think this EPA requires revision?"-was limited to a "yes/no" response. Descriptive statistics for overall and domain-specific EQual rubric scores for each of the 13 Core EPAs were calculated. Free-text responses to why and/or how a Core EPA should be revised were summarized for any Core EPA that scored below a cutoff or for which the majority of experts recommended revision. RESULTS: Six experts completed the evaluation. Most Core EPAs' (9/13) overall score was above the cutoff, indicating that they align with the key domains of the EPA construct. The remaining 4 Core EPAs (2, 7, 9, and 13) scored below the overall cutoff, suggesting that they may require revision. A majority of experts felt that Core EPAs 6, 7, 9, and 13 required revision. With regard to domain-specific scores, Core EPAs 2, 3, 7, 9, and 13 were below the discrete units of work cutoff; Core EPAs 7, 9, and 13 were below the entrustable, essential, and important tasks of the profession of medicine cutoff; and Core EPA 9 was below the curricular role cutoff. CONCLUSIONS: The Core EPAs represent a promising initial framework of EPAs for undergraduate medical education. Some Core EPAs might benefit from revision. The process of improving the Core EPAs must continue if they are to standardize outcomes for medical school graduates.

The Psychology of Shame: A Resilience Seminar for Medical Students.

Introduction: Shame is a powerful emotion that can cause emotional distress, impaired empathy, social isolation, and unprofessional behavior in medical learners. However, interventions to help learners constructively engage with shame are rare. This module educated medical students about shame, guided them through an exploration of their shame experiences, and facilitated development of shame resilience. Methods: In this 2-hour workshop, clinical-year medical students were guided through the psychology of shame through didactic slides. Next, a small panel of volunteer students, recruited and coached prior to the workshop, shared reflections on the content, including their shame experiences during medical school. This was followed by didactic slides outlining strategies to promote shame resilience. Participants then broke into faculty-led small groups to discuss session content. The module included a small-group facilitator guide for leading discussions on shame, didactic slides, discussion prompts, an evaluation tool, and a film entitled The Shame Conversation that was created after the initial workshop. Results: A retrospective pre/postsurvey revealed statistically significant increases in: (1) importance ascribed to identifying shame in one's self or colleagues, (2) confidence in one's ability to recover from a shame reaction, and (3) comfort in reaching out to others when shame occurs. Analysis of open-ended questions showed that students felt the seminar would enhance future resilience by helping them identify and normalize shame, distinguish shame from guilt, and reach out to others for help. Discussion: This workshop appears to prepare students to more constructively engage with shame when it occurs in medical training.

The Distinctions Between Theory, Theoretical Framework, and Conceptual Framework.

Health professions education (HPE) researchers are regularly asked to articulate their use of theory, theoretical frameworks, and conceptual frameworks in their research. However, all too often, these words are used interchangeably or without a clear understanding of the differences between these concepts. Further problematizing this situation is the fact that theory, theoretical framework, and conceptual framework are terms that are used in different ways in different research approaches. In this article, the authors set out to clarify the meaning of these terms and to describe how they are used in 2 approaches to research commonly used in HPE: the objectivist deductive approach (from theory to data) and the subjectivist inductive approach (from data to theory). In addition to this, given that within subjectivist inductive research theory, theoretical framework, and conceptual framework can be used in different ways, they describe 3 uses that HPE researchers frequently rely on: fully inductive theory development, fully theory-informed inductive, and theory-informing inductive data analysis.

Scoping Review of Entrustable Professional Activities in Undergraduate Medical Education.

PURPOSE: Entrustable professional activities (EPAs) are a hot topic in undergraduate medical education (UME); however, the usefulness of EPAs as an assessment approach remains unclear. The authors sought to better understand the literature on EPAs in UME through the lens of the 2010 Ottawa Conference Criteria for Good Assessment. METHOD: The authors conducted a scoping review of the health professions literature (search updated February 2018), mapping publications to the Ottawa Criteria using a collaboratively designed charting tool. RESULTS: Of the 1,089 publications found, 71 (6.5%) met inclusion criteria. All were published after 2013. Forty-five (63.4%) referenced the 13 Core Entrustable Professional Activities for Entering Residency developed by the Association of American Medical Colleges (AAMC). Forty (56.3%) were perspectives, 5 (7.0%) were reviews, and 26 (36.6%) were prospective empirical studies. The publications mapped to the Ottawa Criteria 158 times. Perspectives mapped more positively (83.7%) than empirical studies (76.7%). Reproducibility did not appear to be a strength of EPAs in UME; however, reproducibility, equivalence, educational effect, and catalytic effect all require further study. Inconsistent use of the term "EPA" and conflation of concepts (activity vs assessment vs advancement decision vs curricular framework) limited interpretation of published results. Overgeneralization of the AAMC's work on EPAs has influenced the literature. CONCLUSIONS: Much has been published on EPAs in UME in a short time. Now is the time to move beyond opinion, clarify terms, and delineate topics so that well-designed empirical studies can demonstrate if and how EPAs should be implemented in UME.

Addressing the Elephant in the Room: A Shame Resilience Seminar for Medical Students.

PROBLEM: Medical schools face the challenge of developing efficacious resources to promote well-being and foster resilience in students. The authors developed, implemented, and evaluated a shame resilience seminar for second-year clerkship medical students. APPROACH: In February 2018, the authors conducted a 2.5-hour seminar (part of a longitudinal series) about shame, a common and potentially damaging emotion. The seminar consisted of a large-group session to introduce the psychology of shame, during which speakers shared their personal experiences with the emotion. Next, a small-group session allowed students to discuss their reactions to the large-group content in a safe and familiar environment. Before the seminar, faculty development was provided to small-group leaders (upper-level medical students and faculty) to increase their comfort leading discussions about shame. Students completed a pre/post retrospective survey immediately following the seminar. OUTCOMES: The authors found statistically significant increases in students' confidence in identifying shame and differentiating it from guilt; in their attitudes regarding the importance of identifying shame reactions in themselves and others; and in their willingness to reach out to others during a shame reaction. Surveys of group leaders revealed no reports of significant student distress during or after the seminar. NEXT STEPS: This seminar represents a reproducible means of promoting shame resilience in medical students. The speakers' personal shame experiences and the safety of the small groups for discussions about shame were central to the seminar's apparent success. Next steps include developing an empirically derived, longitudinal shame resilience curriculum spanning the medical school years.

Sentinel Emotional Events: The Nature, Triggers, and Effects of Shame Experiences in Medical Residents.

PURPOSE: This study explores an under-investigated topic, how medical residents experience shame within clinical learning environments, by asking residents to reflect on (1) the nature of their shame experiences; (2) the events that triggered, and factors that contributed to, those shame experiences; and (3) the perceived effects of those shame experiences. METHOD: In this hermeneutic phenomenology study, the authors recruited 12 (self-nominated) residents from an internal medicine residency at a large teaching hospital in the United States. Data collection from each participant in 2016-2017 included (1) a written reflection about an experience during medical training in which the participant felt "flawed, deficient, or unworthy," and (2) a semi-structured interview that explored the participant's shame experience(s) in depth. The data were analyzed according to hermeneutic traditions, producing rich descriptions about participants' shame experiences. RESULTS: Participants' shame experiences ranged from debilitating emotional and physical reactions to more insidious, fleeting reactions. Participants reported shame triggers relating to patient care, learning processes, and personal goals; numerous factors contributed to their shame experiences. The effects of shame reactions included social isolation, disengagement from learning, impaired wellness, unprofessional behavior, and impaired empathy. Positive effects of shame reactions included enhanced learning, increased willingness to reach out for help, and improved relationships. CONCLUSIONS: Shame reactions can be sentinel emotional events with significant physical and/or psychological effects in medical learners. This study has implications for learners, educators, and patients, and it may pave the way toward open, honest conversations about the role shame plays in medical education.

Authorship Order in Medical Education Publications: In Search of Practical Guidance for the Community.

Phenomenon: With scholarly collaborations come questions about the order of authorship. Authorship order is an important consideration because it often used as an indicator of seniority, expertise, leadership, and scholarly productivity. As a result, authorship order factors into decisions about hiring, salary, resource allocation, and professional advancement. This study describes principles commonly applied to authorship order decisions within the medical education community and educators' opinions about the significance of authorship order. Approach: A questionnaire was developed to ascertain current practices related to authorship decisions. Sixteen items were rated in terms of frequency of actual use and the desirability of the practice using a 4-point rating scale: 1 (never), 2 (sometimes), 3 (often), and 4 (always). Additional questions addressed the perceived significance of authorship order. The last set of questions provided information about respondents' personal and career characteristics. The survey was delivered via e-mail to a random sample of 391 subscribers from the DR-ED listserv. Findings: Fifty-four e-mail addresses were returned as undeliverable; of the remaining 337 mailed surveys, 109 responses (32.3%) were received. Five of the current practices for determining authorship order were rated as both frequent and desirable; 4 items had low ratings suggesting that these practices were both infrequent and undesirable. For 7 items, there was a significant gap between the ratings of practice frequency and desirability. When asked about preferred authorship order strategies, most respondents (94%) endorsed listing authors by descending order based on contribution but were split in identifying the last author (47%) or second author (46%) as the next best placement after first author. Respondents supported the practice of many journals requiring authors to disclose their contributions, agreeing (69%) that it provides useful information for promotion and tenure committees; however, 43% were uncertain about how disclosed contributions were reflected in authorship order. Insights: Respondents strongly endorsed the importance of listing authors in order of decreasing contribution, although the meaning of second versus last author lacks consensus. This finding, together the other strategies that received strong endorsement and those that were not strongly endorsed, provides a starting point to develop guidance for medical educators about how to determine authorship order. Clear guidance for authors would promote fairness and accountability within the medical education community as well as provide more consistent interpretation for those who consider authorship order for career- and resource-related decisions.

Evaluation of a Curricular Addition to Assist Medical Students in Specialty Selection.

Early hands-on experience with surgical procedures may help medical students make better-informed choices if considering a surgical specialty. Here, we evaluate a curricular addition in surgical anatomy, formally exposing second-year students to different surgical subspecialties. Students met with surgeons for 7 weeks (one afternoon per week) and practiced surgical procedures on human cadavers with supervision. About a quarter of the participants reported a change in their top choice of specialty upon completing the course, and about half of the students reported changes in their second and third choices. At the time of graduation, 85% of those surveyed reported participation in the course impacted their final choice of specialty. These results demonstrate such a course helped medical students select a specialty during early training.