Milestone Implementation's Impact on Narrative Comments and Perception of Feedback for Internal Medicine Residents: a Mixed Methods Study.

BACKGROUND: Feedback is a critical element of graduate medical education. Narrative comments on evaluation forms are a source of feedback for residents. As a shared mental model for performance, milestone-based evaluations may impact narrative comments and resident perception of feedback. OBJECTIVE: To determine if milestone-based evaluations impacted the quality of faculty members' narrative comments on evaluations and, as an extension, residents' perception of feedback. DESIGN: Concurrent mixed methods study, including qualitative analysis of narrative comments and survey of resident perception of feedback. PARTICIPANTS: Seventy internal medicine residents and their faculty evaluators at the University of Utah. APPROACH: Faculty narrative comments from 248 evaluations pre- and post-milestone implementation were analyzed for quality and Accreditation Council for Graduate Medical Education competency by area of strength and area for improvement. Seventy residents were surveyed regarding quality of feedback pre- and post-milestone implementation. KEY RESULTS: Qualitative analysis of narrative comments revealed nearly all evaluations pre- and post-milestone implementation included comments about areas of strength but were frequently vague and not related to competencies. Few evaluations included narrative comments on areas for improvement, but these were of higher quality compared to areas of strength (p < 0.001). Overall resident perception of quality of narrative comments was low and did not change following milestone implementation (p = 0.562) for the 86% of residents (N = 60/70) who completed the pre- and post-surveys. CONCLUSIONS: The quality of narrative comments was poor, and there was no evidence of improved quality following introduction of milestone-based evaluations. Comments on areas for improvement were of higher quality than areas of strength, suggesting an area for targeted intervention. Residents' perception of feedback quality did not change following implementation of milestone-based evaluations, suggesting that in the post-milestone era, internal medicine educators need to utilize additional interventions to improve quality of feedback.

Capturing growth curves of medical students' clinical skills performance.

INTRODUCTION: A benefit of a milestone or Entrustable Professional Activity (EPA) assessment framework is the ability to capture longitudinal performance with growth curves using multi-level modelling (MLM). Growth curves can inform curriculum design and individualised learning. Residency programmes have found growth curves to vary by resident and by milestone. Only one study has analysed medical students' growth curves for EPAs. Analysis of EPA growth curves is critical because no change in performance raises concerns for EPAs as an assessment framework. METHODS: Spencer Fox Eccles School of Medicine-University of Utah students' workplace-based assessment ratings for 7 EPAs were captured at 3 time-points in years 3-4 of AY2017-2018 to AY2020-2021. MLM was used to capture EPA growth curves and determine if variation in growth curves was explained by internal medicine (IM) clerkship order. FINDINGS: A curvilinear slope significantly captured 256 students' average ratings overtime for EPA1a-history-taking, EPA2-clinical reasoning, EPA3-diagnostics, EPA5-documentation and EPA6-presentation, and a linear slope significantly captured EPA9-teamwork ratings, p ≤ 0.001. Growth curves were steepest for EPA2-clinical reasoning and EPA3-diagnostics. Growth curves varied by students, p < 0.05 for all EPA ratings, but IM clerkship rotation order did not significantly explain the variance, p > 0.05. DISCUSSION: The increase in ratings from Year 3 to Year 4 provides validity evidence for use of EPAs in an assessment framework. Students may benefit from more curriculum/skills practice for EPA2-clinical reasoning and EPA3-diagnostics prior to year 3. Variation in student's growth curves is important for coaching and skill development; a one size fits all approach may not suffice.

Making the Personal Statement "Truly Personal": Recommendations From a Qualitative Case Study of Internal Medicine Program and Associate Program Directors.

Background: The personal statement is an integral part of a residency application but little guidance exists for medical students about what content to include. Objective: We use the framework of isomorphism, the process by which institutions model themselves after one another, to understand what internal medicine program directors (PDs) and associate program directors (APDs) recommend be included in the personal statement and how programs use personal statements in their selection of applicants to interview and rank. Methods: Semi-structured phone interviews were conducted between August and October 2020 with 13 academic PDs and APDs, who were selected for participation based on program size and geographic location. Interviews were recorded, transcribed, and coded using content analysis. Results: Effective personal statements should be well-written, present unique information, and demonstrate fit with a residency program. PDs and APDs recommended against expressing lack of interest in a program or highlighting negative personal characteristics. PDs and APDs used personal statements to distinguish between applicants and noted that personal statements help programs form an impression of the applicant. Consensus among PDs and APDs about what personal statements should include and how they are used indicates that isomorphism influences the match process. Conclusions: Our study found that the personal statement is a valued part of the residency application when it includes unique attributes and reveals personal values that align with that of the program. Additionally, PDs and APDs noted that when applicants highlight their unique characteristics, it can help distinguish themselves from others.

Parental Leave Policies in Residency: A National Survey of Internal Medicine Program Directors.

PURPOSE: To characterize the existence, accessibility, and content of parental leave policies, as well as barriers to program-level policy implementation among internal medicine (IM) program directors (PDs) and to assess the willingness of PDs to implement a national standardized policy. METHOD: In 2019, the Association of Program Directors in Internal Medicine conducted a survey of 422 IM PDs. Along with other content, 38 questions addressed 4 primary outcomes: parental leave policy existence, accessibility, content, and barriers. The authors compared programs with and without a program-level policy and applied qualitative content analysis to open-ended questions about barriers to policy implementation and openness to a national standard. RESULTS: The response rate was 69.4% (293/422). Of responding programs, 86% (250/290) reported a written parental leave policy with 43% (97/225) of these originating at the program level. Program-level policies, compared with policies at other levels, were more likely to address scheduling during pregnancy (38%, 36/95 vs 22%, 27/124; P = .018); peer coverage (24%, 21/89 vs 15%, 16/109; P = .037), how the duration of extended training is determined (81%, 72/89 vs 44%, 48/109; P < .001), and associated pay and benefits 61%, 54/89 vs 44%, 48/109; P = .009). PDs without program-level policy reported lacking guidance to develop policy, deferring upward to institutional policies, and wishing to retain flexibility. More than half of PDs (60%, 170/282) expressed agreement that a national standard for a residency program-level parental leave policy should exist. Those not in favor cited organization equity, lack of resources, implementation challenges, loss of flexibility, and potentially disadvantaging recruitment. CONCLUSIONS: While existing program-level policies included important content, most PDs reported not having them. A national standard to guide the development of program-level parental leave policies could be embraced if it provided flexibility for programs with limited resources.

Childbearing Decisions in Residency: A Multicenter Survey of Female Residents.

PURPOSE: To characterize how female residents make decisions about childbearing, factors associated with the decision to delay childbearing, and satisfaction with these decisions. METHOD: In 2017, the authors sent a voluntary, anonymous survey to 1,537 female residents enrolled across 78 graduate medical education programs, consisting of 25 unique specialties, at 6 U.S. academic medical centers. Survey items included personal, partner, and institutional characteristics, whether the respondent was delaying childbearing during residency, and the respondent's satisfaction with this decision. RESULTS: The survey response rate was 52% (n = 804). Among the 447 (56%) respondents who were married or partnered, 274 (61%) were delaying childbearing. Residents delaying childbearing were significantly more likely to be younger (P < .001), not currently a parent (P < .001), in a specialty with an uncontrollable lifestyle (P = .001), or in a large program (P = .004). Among self-reported reasons for delaying childbearing, which were not mutually exclusive, the majority cited a busy work schedule (n = 255; 93%) and desire not to extend residency training (n = 145; 53%). Many cited lack of access to childcare (n = 126; 46%), financial concerns (n = 116; 42%), fear of burdening colleagues (n = 96; 35%), and concern for pregnancy complications (n = 74; 27%). Only 38% (n = 103) of respondents delaying childbearing were satisfied with this decision, with satisfaction decreasing with increasing age. CONCLUSIONS: Decisions to delay childbearing are more common in certain specialties, and many residents who delay childbearing are not satisfied with that decision. These findings suggest that greater attention is needed overall, and particularly in certain specialties, to promote policies and cultures that both anticipate and normalize parenthood in residency, thus minimizing the conflict between biological and professional choices for female residents.

Maternity Leave in Residency: A Multicenter Study of Determinants and Wellness Outcomes.

PURPOSE: To characterize determinants of resident maternity leave and the effect of length of leave on maternal well-being. METHOD: In 2017, the authors sent a voluntary, anonymous survey to female residents at 78 programs, in 25 unique specialties, at 6 institutions. Survey items included personal, partner, and child demographics, and logistics of leave, including whether leave was paid or vacation or sick leave was used. Outcomes were maternity leave length; duration of breastfeeding; burnout and postpartum depression screens; perceptions of support; and satisfaction with length of leave, breastfeeding, and childbearing during residency. RESULTS: Fifty-two percent (804/1,537) of residents responded. Among 16% (126) of respondents who were mothers, 50% (63) had their first child during residency. Seventy-seven maternity leaves were reported (range, 2-40 weeks), with most taking 6 weeks (32% of leaves; 25) and including vacation (81%; 62) or sick leave (64%; 49). Length of leave was associated with institution, use of sick leave or vacation, and amount of paid leave. The most frequently self-reported determinant of leave was the desire not to extend residency training (27%; 59). Training was not extended for 53% (41) of mothers; 9% (7) were unsure. Longer breastfeeding duration and perceptions of logistical support from program administration were associated with longer maternity leaves. Burnout affected approximately 50% (38) of mothers regardless of leave length. CONCLUSIONS: This study illustrates variability in administration of resident maternity leaves. Targets for intervention include policy clarification, improving program support, and consideration of parent wellness upon return to work.

Impact of Laboratory Charge Display Within the Electronic Health Record Across an Entire Academic Medical Center: Results of a Randomized Controlled Trial.

OBJECTIVES: To determine the impact of systemwide charge display on laboratory utilization. METHODS: This was a randomized controlled trial with a baseline period and an intervention period. Tests were randomized to a control arm or an active arm. The maximum allowable Medicare reimbursement rate was displayed for tests in the active arm during the intervention period. Total volume of tests in the active arm was compared with those in the control arm. Residents were surveyed before and after the intervention to assess charge awareness. RESULTS: Charge display had no effect on order behavior. This result held for patient type (inpatient vs outpatient) and for insurance category (commercial, government, self-pay). Residents overestimated the charges of tests both before and after the intervention. Many residents failed to notice the charge display in the computerized order entry system. CONCLUSIONS: The impact of charge display depends on context. Charge display is not always effective.

Effectiveness of an Interprofessional and Multidisciplinary Musculoskeletal Training Program.

BACKGROUND: Musculoskeletal (MSK) problems are common, and a recent US Bone and Joint Initiative calls for new models of education and professional collaboration. Evidence of feasibility and acceptability of innovative methods are needed. OBJECTIVE: We assessed if an experimental immersion interdisciplinary MSK curriculum would be acceptable to residents from different specialties, be feasible within existing rotations, and be effective in strengthening clinical skills. METHODS: Through funding from the Veterans Affairs Office of Academic Affiliations and the Office of Specialty Care, we developed a Center of Excellence in MSK Care and Education. A core element is the monthly MSK Education Week, which teaches skills and provides opportunities to apply these in clinical settings. Participants include internal medicine, physical medicine and rehabilitation, and orthopaedic surgery residents, as well as students and residents from other health professions programs. All were assigned to the MSK week in lieu of other clinical experiences. Faculty encompassed primary care, rheumatology, endocrinology, orthopaedics, and physical medicine and rehabilitation. Assessments include surveys and a 2-station objective structured clinical examination (OSCE). RESULTS: Since 2012, a total of 176 trainees have participated. Percentage of trainees reporting ability to evaluate and manage MSK complaints increased (9% to 87% for shoulder; 18% to 86% for knee), and confidence performing MSK injections increased from 10% to 70%. Competency in evaluation of shoulder and knee pain was confirmed by OSCEs. CONCLUSIONS: The MSK week program was accepted by residents from the 3 specialties, with learners reporting improved ability to perform shoulder and knee examinations, as demonstrated by OSCEs.

Emergency medicine and internal medicine trainees' smartphone use in clinical settings in the United States.

PURPOSE: Smartphone technology offers a multitude of applications (apps) that provide a wide range of functions for healthcare professionals. Medical trainees are early adopters of this technology, but how they use smartphones in clinical care remains unclear. Our objective was to further characterize smartphone use by medical trainees at two United States academic institutions, as well as their prior training in the clinical use of smartphones. METHODS: In 2014, we surveyed 347 internal medicine and emergency medicine resident physicians at the University of Utah and Brigham and Women's Hospital about their smartphone use and prior training experiences. Scores (0%-100%) were calculated to assess the frequency of their use of general features (email, text) and patient-specific apps, and the results were compared according to resident level and program using the Mann-Whitney U-test. RESULTS: A total of 184 residents responded (response rate, 53.0%). The average score for using general features, 14.4/20 (72.2%) was significantly higher than the average score for using patient-specific features and apps, 14.1/44 (33.0%, P<0.001). The average scores for the use of general features, were significantly higher for year 3-4 residents, 15.0/20 (75.1%) than year 1-2 residents, 14.1/20 (70.5%, P=0.035), and for internal medicine residents, 14.9/20 (74.6%) in comparison to emergency medicine residents, 12.9/20 (64.3%, P= 0.001). The average score reflecting the use of patient-specific apps was significantly higher for year 3-4 residents, 16.1/44 (36.5%) than for year 1-2 residents, 13.7/44 (31.1%; P=0.044). Only 21.7% of respondents had received prior training in clinical smartphone use. CONCLUSION: Residents used smartphones for general features more frequently than for patient-specific features, but patient-specific use increased with training. Few residents have received prior training in the clinical use of smartphones.

Evaluation of 2009 pandemic influenza A (H1N1) exposures and illness among physicians in training.

BACKGROUND: A cluster of influenza-like illness (ILI) among physicians in training during the 2009 influenza A (H1N1) pandemic (pH1N1) led to a health hazard evaluation. METHODS: We conducted a cross-sectional study to examine exposures, infection control practices, ILI prevalence, and transmission among physicians in training at 4 affiliated hospitals during the pandemic. We administered an electronic survey and met with physicians in training and hospital personnel. RESULTS: Of the 88 responding physicians, 85% reported exposure to pH1N1. Exposures occurred at work from patients or coworkers and outside of work from coworkers, household members, or the community. Thirteen cases of ILI were reported in May-June 2009; 10 respondents reported working while ill (duration, 1-4 days). Between 13% and 88% of respondents knew which personal protective equipment (PPE) was recommended when caring for influenza patients at the 4 hospitals. The most common reasons for not using PPE were not knowing that a patient had pH1N1 or ILI and not having PPE readily available. CONCLUSIONS: Physicians in training have gaps in their knowledge of and adherence to recommended PPE and compliance with work restrictions. Our findings underscore the importance of installing isolation precaution signage, making PPE readily available near patients with influenza, and facilitating work restrictions for ill health care personnel.

Residents as members of intern selection committees: can they partially replace faculty?

BACKGROUND: Faculty involvement in the residency selection process is important, but increasing pressures on available faculty resources have made their ability to participate more difficult. Residents may be acceptable substitutions for some faculty in the selection process. PURPOSE: To test 2 new interview models and explore whether residents could partially replace faculty by examining what role the interview itself plays in the overall assessment of an applicant, comparing faculty and resident ratings of the interview, and asking whether partially substituting residents for faculty had an impact on the match outcome. METHODS: Applicants to an internal medicine residency program were assigned to 1 of 3 interview models: faculty-faculty, resident-faculty, resident-faculty-faculty. The 12 interview days were randomly assigned to a model and all applicants on each day had the same model. Interviewers used an applicant assessment form to assign ratings to 6 components of the portfolio and a final score. RESULTS: For both residents and faculty, the final score was highly correlated with the formal interview component. Within-model analyses showed residents consistently gave more favorable scores than faculty interviewers. There was no impact of interview model on initial or final rank position. Similarly, there was no difference between models in their match "success" rates. CONCLUSIONS: Residents can be successfully substituted for some faculty in the residency selection process. The use of residents does not impact the match results. Developing a small, committed group of interviewers should benefit both programs and applicants.