Measuring Clinical Preparedness After Residency Training: Development of a New Instrument.

BACKGROUND AND OBJECTIVES: Research on preparedness for independent clinical practice typically uses surveys of residents and program directors near graduation, which can be affected by several biases. We developed a novel approach to assess new graduates more objectively using physician and staff member assessors 3 months after graduates started their first job. METHODS: We conducted a literature review and key informant interviews with physicians from varying practice types and geographic regions in the United States to identify features that indicate a lack of preparedness for independent clinical practice. We then held a Clinical Preparedness Measurement Summit, engaging measurement experts and family medicine education leaders, to build consensus on key indicators of readiness for independent clinical practice and survey development strategies. The 2015 entrustable professional activities for family medicine end-of-residency training provided the framework for assessment of clinical preparedness by physician assessors. Sixteen published variables assessing interpersonal communication skills and processes of care delivery were identified for staff assessors. We assessed frequencies and compared survey findings between physician and staff assessors in 2016 to assist with survey validation. RESULTS: The assessment of frequencies demonstrated a range of responses, supporting the instrument's ability to distinguish readiness for independent practice of recent graduate hires. No statistical differences occurred between the physician and staff assessors for the same physician they were evaluating, indicating internal consistency. CONCLUSIONS: To learn about the possible impact of length of training, we developed a novel approach to assess preparedness for independent clinical practice of family medicine residency graduates.

Resident Involvement in Curricular and Clinical Practice Change and Satisfaction With Training According to Length of Training in Family Medicine.

BACKGROUND AND OBJECTIVES: Most research in residency training has focused on quality improvement within a single program. We explored resident involvement in curricular and clinical practice change, the learning environment, and resident satisfaction in 3-year family medicine residencies compared to matched 4-year residencies. METHODS: We used two surveys to capture data. One was for program directors, which assessed the level of resident involvement in curricular and practice transformation. The second was a resident survey, which asked residents to rate their involvement in curricular change and practice transformation, the learning environment, and satisfaction with training. Both were administered annually between 2013 and 2019. Response rates ranged from 84.6% to 100%. RESULTS: Findings revealed no overall difference in resident involvement in curricular change, but the program director survey findings indicated that a higher proportion of residents in 4-year programs were using a broader diversity of approaches to working on quality improvement (QI) projects compared to those in 3-year programs. We also found statistical differences in the number of QI projects completed per year, with 34.1% completing three or more in 4-year programs compared to 13.3% in 3-year programs (P<.001). We found a positive correlation between resident involvement, learning environment, and satisfaction with training for both 3-year (range 0.489-0.666; P=.001) and 4-year residents (range 0.441-0.529; P=.001). CONCLUSIONS: Four-year residents were involved in a greater number of quality improvement projects and had a more diverse profile of involvement than those in 3-year residency programs. Involvement in practice and curricular change and the learning environment were associated with greater levels of resident satisfaction with training in both 3-year and 4-year programs.

Postresidency Practice Setting and Clinical Care Features According to 3 Versus 4 Years of Training in Family Medicine: A Length of Training Pilot Study.

BACKGROUND AND OBJECTIVES: Factors associated with physician practice choice include residency location, training experiences, and financial incentives. How length of training affects practice setting and clinical care features postgraduation is unknown. METHODS: In this Length of Training Pilot (LoTP) study, we surveyed 366 graduates of 3-year (3YR) and 434 graduates of 4-year (4YR) programs 1 year after completion of training between 2013 and 2021. Variables assessed included reasons for practice setting choice, practice type, location, practice and community size, specialty mix, and clinical care delivery features (eg, integrated behavioral health, risk stratified care management). We compared different length of training models using χ2 or Fisher's exact tests for categorical variables and independent samples, and t test (unequal variances) for continuous variables. RESULTS: Response rates ranged from 50% to 88% for 3YR graduates and 68% to 95% for 4YR graduates. Scope of practice was a predominant reason for graduates choosing their eventual practice, and salary was a less likely reason for those completing 4 years versus 3 years of training (scope, 72% vs 55%, P=.001; salary, 15% vs 22%, P=.028). Community size, practice size, practice type, specialty mix, and practice in a federally designated underserved site did not differ between the two groups. We found no differences in patient-centered medical home features when comparing the practices of 3YR to 4YR graduates. CONCLUSIONS: Training length did not affect practice setting or practice features for graduates of LoTP programs. Future LoTP analyses will examine how length of training affects scope of practice and clinical preparedness, which may elucidate other elements associated with practice choice.

A Longitudinal Assessment of Resident and New Graduate Well-Being According to Length of Training: A Report From the Length of Training Pilot in Family Medicine.

Background & Objectives: No prior studies have examined how length of training may influence wellness. As part of the Length of Training Pilot (LoTP), we explored resident and new graduate well-being according to program year and length of training in 3- and 4-year family medicine residency training programs. METHODS: Two surveys captured data included in these analyses. One was a resident survey that included the Mayo Clinic physician-expanded Well-Being Index (eWBI) administered annually during the In-Training Examination (2014-2019). The second was administered to graduates 1 year after completion of training between 2016 and 2022 and included the same well-being questions. Response rates ranged between 77.7% and 96.8%. RESULTS: The eWBI summary scores for burnout were highest in postgraduate year 1 (PGY1) and did not differ statistically according to length of training (PGY1: 2.02 in 3-year [3YR] programs vs 1.93 in 4-year [4YR] programs, P=.55; postgraduate year 2 [PGY2]: 2.42 in 3YR programs vs 2.38 in 4YR programs, P=.83; postgraduate year 3 [PGY3]: 2.18 in 3YR programs vs 2.28 in 4YR programs, P=.59; and 2.34 in postgraduate year 4 [PGY4] for those in 4YR programs), though some statistical differences were noted for three items. New graduates' eWBI summary scores before the COVID-19 pandemic were 1.77 among 3YR graduates and 1.66 among 4YR graduates (P=.59). These scores were higher during COVID-19 at 1.89 for 3YR graduates and 2.02 for 4YR graduates (P=.62). Length of training was not associated with differences in well-being before or during COVID-19. CONCLUSIONS: We found no associations between length of training and physician well-being during training or among new graduates before or during COVID-19.

Optimizing Survey Response Rates in Graduate Medical Education Research Studies.

BACKGROUND AND OBJECTIVE: Survey response rates of 70% or higher are needed if findings are to be considered generalizable. Unfortunately, survey studies of health professionals have declining response rates. We have conducted survey research with residents and residency directors for over 13 years. Here we describe the strategies we used to obtain optimal response rates in residency training research collaboratives. METHODS: We administered over 6,000 surveys between 2007 and 2019 to evaluate the Preparing the Personal Physician for Practice and Length of Training Pilot studies, both of which involved redesigning residency training. Survey recipients included program directors, clinic managers, residents, graduates, as well as supervising physicians and clinic staff members. We logged and analyzed survey administration efforts and approaches to optimize strategies. RESULTS: Overall, we obtained response rates of 100% for program director surveys, 98% for resident surveys, 97% for continuity clinic surveys, 81% for graduates surveys, and 48% for the supervising physician and 43% for the clinic staff. Response rates were highest when the relationships between the evaluation team and survey recipients were closest. Strategies for optimizing response rates included (1) building relationships with all participants whenever possible, (2) sensitivity to survey timing and fatigue, and (3) using creative and persistent follow-up measures to encourage survey completion. CONCLUSION: High response rates are achievable, though they require an investment in time, resources, and ingenuity in connecting with study populations. Investigators conducting survey research must consider administrative efforts needed to achieve target response rates, including planning for funds accordingly.

The Association Between Length of Training and Family Medicine Residents' Clinical Knowledge: A Report From the Length of Training Pilot Study.

BACKGROUND AND OBJECTIVE: The associations between training length and clinical knowledge are unknown. We compared family medicine in-training examination (ITE) scores among residents who trained in 3- versus 4-year programs and to national averages over time. METHODS: In this prospective case-control study, we compared the ITE scores of 318 consenting residents in 3-year programs to 243 who completed 4 years of training between 2013 through 2019. We obtained scores from the American Board of Family Medicine. The primary analyses involved comparing scores within each academic year according to length of training. We used multivariable linear mixed effects regression models adjusted for covariates. We performed simulation models to predict ITE scores after 4 years of training among residents who underwent only 3 years of training. RESULTS: At baseline postgraduate year-1 (PGY1), the estimated mean ITE scores were 408.5 for 4-year programs and 386.5 for 3-year programs, a 21.9 point difference (95% CI=10.1-33.8). At PGY2 and PGY3, 4-year programs scored 15.0 points higher and 15.6 points higher, respectively. When extrapolating an estimated mean ITE score for 3-year programs, 4-year programs would still score 29.4 points higher (95% CI=15.0-43.8). Our trend analysis revealed those in 4-year programs had a slightly lesser slope increase compared to 3-year programs in the first 2 years. Their drop-off in ITE scores is less steep in later years, though these differences were not statistically significant. CONCLUSIONS: While we found significantly higher absolute ITE scores in 4 versus 3-year programs, these increases in PGY2, PGY3 and PGY4 may be due to initial differences in PGY1 scores. Additional research is needed to support a decision to change the length of family medicine training.

Resident Visit Productivity and Attitudes About Continuity According to 3 Versus 4 Years of Training in Family Medicine: A Length of Training Study.

BACKGROUND AND OBJECTIVES: Training models in the Length of Training Pilot (LOTP) vary. How innovations in training length affect patient visits and resident perceptions of continuity is unknown. METHODS: We analyzed resident in-person patient encounters (2013-2014 through 2018-2019) for each postgraduate year (PGY) and total visits at graduation derived from the Accreditation Council for Graduate Medical Education reports for each LOTP program. We collected data on residents' perceptions of continuity from annual surveys (2015-2019). We analyzed continuous variables using independent samples t tests with unequal variance and categorical variables using χ2 tests in comparing 3-year (3YR) versus 4-year (4YR) programs. RESULTS: PGY-1 and PGY-2 residents in 4YR programs saw statistically more patients than their counterparts in 3YR programs. In PGY3, 3YR program residents had statistically higher visit volume compared to 4YR program residents. Visits conducted in PGY4 ranged from 832 to 884. The additional year of training resulted in approximately 1,000 more total patient visits. Most residents in 3YR and 4YR programs rated their continuity clinic experience as somewhat or very adequate (range 86.3% to 93.7%), which did not statistically differ according to length of training. CONCLUSIONS: Resident visits were significantly different at each PGY level when comparing 3YR and 4YR programs in the LOTP and the additional year of training resulted in about 1,000 more total visits. Resident perspectives on the adequacy of their continuity clinic experience appeared to not be affected by length of training. Future research should explore how the volume of patient visits performed in residency affects scope of practice and clinical preparedness.

Financial Considerations Associated With a Fourth Year of Residency Training in Family Medicine: Findings From the Length of Training Pilot Study.

BACKGROUND AND OBJECTIVES: The feasibility of funding an additional year of residency training is unknown, as are perspectives of residents regarding related financial considerations. We examined these issues in the Family Medicine Length of Training Pilot. METHODS: Between 2013 and 2019, we collected data on matched 3-year and 4-year programs using annual surveys, focus groups, and in-person and telephone interviews. We analyzed survey quantitative data using descriptive statistics, independent samples t test, Fisher's Exact Test and χ2. Qualitative analyses involved identifying emergent themes, defining them and presenting exemplars. RESULTS: Postgraduate year (PGY)-4 residents in 4-year programs were more likely to moonlight to supplement their resident salaries compared to PGY-3 residents in three-year programs (41.6% vs 23.0%; P=.002), though their student debt load was similar. We found no differences in enrollment in loan repayment programs or pretax income. Programs' descriptions of financing a fourth year as reported by the program director were limited and budget numbers could not be obtained. However, programs that required a fourth year typically reported extensive planning to determine how to fund the additional year. Programs with an optional fourth year were budget neutral because few residents chose to undertake an additional year of training. Resources needed for a required fourth year included resident salaries for the fourth year, one additional faculty, and one staff member to assist with more complex scheduling. Residents' concerns about financial issues varied widely. CONCLUSIONS: Adding a fourth year of training was financially feasible but details are local and programs could not be compared directly. For programs that had a required rather than optional fourth year much more financial planning was needed.

A Comparison of Residency Applications and Match Performance in 3-Year vs 4-Year Family Medicine Training Programs.

BACKGROUND AND OBJECTIVES: The optimal length of residency training in family medicine is under debate. This study compared applicant type, number of applicants, match positions filled, matched applicant type, and ranks to fill between 3-year (3YR) and 4-year (4YR) residencies. METHODS: The Length of Training Pilot (LOTP) is a case-control study comparing 3YR (seven residencies) and 4YR (six residencies) training models. We collected applicant and match data from LOTP programs from 2012 to 2018 and compared data between 3YR and 4YR programs. National data provided descriptive comparisons. An annual resident survey captured resident perspectives on training program selection. Summary statistics and corresponding t-tests and χ2 tests of independence were performed to assess differences between groups. We used a linear mixed model to account for repeated measures over time within programs. RESULTS: There were no differences in the mean number of US MD, US DO, and international medical graduate applicants between 3YR and 4YR programs. Both the 3YR and 4YR programs had a substantially higher number of US MD and DO applicants compared to national averages. The percentages of positions filled in the match and positions filled by US MDs, DOs and IMGs were not different between groups. The percentage of residents in 4YR programs who think training in family medicine requires a fourth year varied significantly during the study period, from 35% to 25% (P<.001). The predominant reasons for pursuing training in a 4YR program was a desire for more flexibility in training and a desire to learn additional skills beyond clinical skills. CONCLUSIONS: The applicant pool and match performance of the residencies in the LOTP was not affected by length of training. Questions yet to be addressed include length of training's impact on medical knowledge, scope of practice, and clinical preparedness.

The Importance of and the Complexities Associated With Measuring Continuity of Care During Resident Training: Possible Solutions Do Exist.

BACKGROUND AND OBJECTIVES: Evolutions in care delivery toward the patient-centered medical home have influenced important aspects of care continuity. Primary responsibility for a panel of continuity patients is a foundational requirement in family medicine residencies. In this paper we characterize challenges in measuring continuity of care in residency training in this new era of primary care. METHODS: We synthesized the literature and analyzed information from key informant interviews and group discussions with residency faculty and staff to identify the challenges and possible solutions for measuring continuity of care during family medicine training. We specifically focused on measuring interpersonal continuity at the patient level, resident level, and health care team level. RESULTS: Challenges identified in accurately measuring interpersonal continuity of care during residency training include: (1) variability in empanelment approaches for all patients, (2) scheduling complexity in different types of visits, (3) variability in ability to attain continuity counts at the level of the resident, and (4) shifting make-up of health care teams, especially in residency training. Possible solutions for each challenge are presented. Philosophical issues related to continuity are discussed, including whether true continuity can be achieved during residency training and whether qualitative rather than quantitative measures of continuity are better suited to residencies. CONCLUSIONS: Measuring continuity of care in residency training is challenging but possible, though improvements in precision and assessment of the comprehensive nature of the relationships are needed. Definitions of continuity during training and the role continuity measurement plays in residency need further study.

Elements for successful collaboration between K-8 school, community agency, and university partners: the lead peace partnership.

BACKGROUND: Researchers, schools, and community organizations are increasingly interested in forming partnerships to improve health and learning outcomes for adolescents. School-based service learning programs with young adolescents have been shown to improve students' health and educational outcomes. Quality school-based service learning practice requires partnerships that are collaborative, mutually beneficial, and address community needs. This article examines core elements of a community-school-university partnership engaged in implementing and evaluating Lead Peace, a service learning program for urban middle school youth. METHODS: The partnership was assessed through (1) semistructured group interviews with program facilitators at each school at the end of the 2006 to 2007 and 2007 to 2008 school years; (2) key informant interviews with school administrators; and (3) participant observations of partnership meetings. Qualitative analysis was conducted to identify common and emerging themes that contribute to the success of the Lead Peace partnership. RESULTS: Ten themes were identified as keys to the success of the Lead Peace partnership: (1) communication; (2) shared decision making; (3) shared resources; (4) expertise and credibility; (5) sufficient time to develop and maintain relationships; (6) champions and patron saints; (7) being present; (8) flexibility; (9) a shared youth development orientation; and (10) recognition of other partners' priorities. CONCLUSIONS: Partnerships that are essential to quality service learning practice require deliberate planning and ongoing attention. Elements of the successful Lead Peace partnership may be useful for other collaborators to consider.