Postresidency Practice Setting and Clinical Care Features According to 3 Versus 4 Years of Training in Family Medicine: A Length of Training Pilot Study.

BACKGROUND AND OBJECTIVES: Factors associated with physician practice choice include residency location, training experiences, and financial incentives. How length of training affects practice setting and clinical care features postgraduation is unknown. METHODS: In this Length of Training Pilot (LoTP) study, we surveyed 366 graduates of 3-year (3YR) and 434 graduates of 4-year (4YR) programs 1 year after completion of training between 2013 and 2021. Variables assessed included reasons for practice setting choice, practice type, location, practice and community size, specialty mix, and clinical care delivery features (eg, integrated behavioral health, risk stratified care management). We compared different length of training models using χ2 or Fisher's exact tests for categorical variables and independent samples, and t test (unequal variances) for continuous variables. RESULTS: Response rates ranged from 50% to 88% for 3YR graduates and 68% to 95% for 4YR graduates. Scope of practice was a predominant reason for graduates choosing their eventual practice, and salary was a less likely reason for those completing 4 years versus 3 years of training (scope, 72% vs 55%, P=.001; salary, 15% vs 22%, P=.028). Community size, practice size, practice type, specialty mix, and practice in a federally designated underserved site did not differ between the two groups. We found no differences in patient-centered medical home features when comparing the practices of 3YR to 4YR graduates. CONCLUSIONS: Training length did not affect practice setting or practice features for graduates of LoTP programs. Future LoTP analyses will examine how length of training affects scope of practice and clinical preparedness, which may elucidate other elements associated with practice choice.

Measuring Clinical Preparedness After Residency Training: Development of a New Instrument.

BACKGROUND AND OBJECTIVES: Research on preparedness for independent clinical practice typically uses surveys of residents and program directors near graduation, which can be affected by several biases. We developed a novel approach to assess new graduates more objectively using physician and staff member assessors 3 months after graduates started their first job. METHODS: We conducted a literature review and key informant interviews with physicians from varying practice types and geographic regions in the United States to identify features that indicate a lack of preparedness for independent clinical practice. We then held a Clinical Preparedness Measurement Summit, engaging measurement experts and family medicine education leaders, to build consensus on key indicators of readiness for independent clinical practice and survey development strategies. The 2015 entrustable professional activities for family medicine end-of-residency training provided the framework for assessment of clinical preparedness by physician assessors. Sixteen published variables assessing interpersonal communication skills and processes of care delivery were identified for staff assessors. We assessed frequencies and compared survey findings between physician and staff assessors in 2016 to assist with survey validation. RESULTS: The assessment of frequencies demonstrated a range of responses, supporting the instrument's ability to distinguish readiness for independent practice of recent graduate hires. No statistical differences occurred between the physician and staff assessors for the same physician they were evaluating, indicating internal consistency. CONCLUSIONS: To learn about the possible impact of length of training, we developed a novel approach to assess preparedness for independent clinical practice of family medicine residency graduates.

Optimizing Survey Response Rates in Graduate Medical Education Research Studies.

BACKGROUND AND OBJECTIVE: Survey response rates of 70% or higher are needed if findings are to be considered generalizable. Unfortunately, survey studies of health professionals have declining response rates. We have conducted survey research with residents and residency directors for over 13 years. Here we describe the strategies we used to obtain optimal response rates in residency training research collaboratives. METHODS: We administered over 6,000 surveys between 2007 and 2019 to evaluate the Preparing the Personal Physician for Practice and Length of Training Pilot studies, both of which involved redesigning residency training. Survey recipients included program directors, clinic managers, residents, graduates, as well as supervising physicians and clinic staff members. We logged and analyzed survey administration efforts and approaches to optimize strategies. RESULTS: Overall, we obtained response rates of 100% for program director surveys, 98% for resident surveys, 97% for continuity clinic surveys, 81% for graduates surveys, and 48% for the supervising physician and 43% for the clinic staff. Response rates were highest when the relationships between the evaluation team and survey recipients were closest. Strategies for optimizing response rates included (1) building relationships with all participants whenever possible, (2) sensitivity to survey timing and fatigue, and (3) using creative and persistent follow-up measures to encourage survey completion. CONCLUSION: High response rates are achievable, though they require an investment in time, resources, and ingenuity in connecting with study populations. Investigators conducting survey research must consider administrative efforts needed to achieve target response rates, including planning for funds accordingly.

Resident Visit Productivity and Attitudes About Continuity According to 3 Versus 4 Years of Training in Family Medicine: A Length of Training Study.

BACKGROUND AND OBJECTIVES: Training models in the Length of Training Pilot (LOTP) vary. How innovations in training length affect patient visits and resident perceptions of continuity is unknown. METHODS: We analyzed resident in-person patient encounters (2013-2014 through 2018-2019) for each postgraduate year (PGY) and total visits at graduation derived from the Accreditation Council for Graduate Medical Education reports for each LOTP program. We collected data on residents' perceptions of continuity from annual surveys (2015-2019). We analyzed continuous variables using independent samples t tests with unequal variance and categorical variables using χ2 tests in comparing 3-year (3YR) versus 4-year (4YR) programs. RESULTS: PGY-1 and PGY-2 residents in 4YR programs saw statistically more patients than their counterparts in 3YR programs. In PGY3, 3YR program residents had statistically higher visit volume compared to 4YR program residents. Visits conducted in PGY4 ranged from 832 to 884. The additional year of training resulted in approximately 1,000 more total patient visits. Most residents in 3YR and 4YR programs rated their continuity clinic experience as somewhat or very adequate (range 86.3% to 93.7%), which did not statistically differ according to length of training. CONCLUSIONS: Resident visits were significantly different at each PGY level when comparing 3YR and 4YR programs in the LOTP and the additional year of training resulted in about 1,000 more total visits. Resident perspectives on the adequacy of their continuity clinic experience appeared to not be affected by length of training. Future research should explore how the volume of patient visits performed in residency affects scope of practice and clinical preparedness.

The Association Between Length of Training and Family Medicine Residents' Clinical Knowledge: A Report From the Length of Training Pilot Study.

BACKGROUND AND OBJECTIVE: The associations between training length and clinical knowledge are unknown. We compared family medicine in-training examination (ITE) scores among residents who trained in 3- versus 4-year programs and to national averages over time. METHODS: In this prospective case-control study, we compared the ITE scores of 318 consenting residents in 3-year programs to 243 who completed 4 years of training between 2013 through 2019. We obtained scores from the American Board of Family Medicine. The primary analyses involved comparing scores within each academic year according to length of training. We used multivariable linear mixed effects regression models adjusted for covariates. We performed simulation models to predict ITE scores after 4 years of training among residents who underwent only 3 years of training. RESULTS: At baseline postgraduate year-1 (PGY1), the estimated mean ITE scores were 408.5 for 4-year programs and 386.5 for 3-year programs, a 21.9 point difference (95% CI=10.1-33.8). At PGY2 and PGY3, 4-year programs scored 15.0 points higher and 15.6 points higher, respectively. When extrapolating an estimated mean ITE score for 3-year programs, 4-year programs would still score 29.4 points higher (95% CI=15.0-43.8). Our trend analysis revealed those in 4-year programs had a slightly lesser slope increase compared to 3-year programs in the first 2 years. Their drop-off in ITE scores is less steep in later years, though these differences were not statistically significant. CONCLUSIONS: While we found significantly higher absolute ITE scores in 4 versus 3-year programs, these increases in PGY2, PGY3 and PGY4 may be due to initial differences in PGY1 scores. Additional research is needed to support a decision to change the length of family medicine training.

Implementation and Qualitative Evaluation of a Primary Care Redesign Model with Expanded Scope of Work for Medical Assistants.

BACKGROUND: Implementation of primary care models involving expanded scope of work and redesigned workflows for medical assistants (MAs) as primary care team members can be challenging. Implementation strategies and participatory evaluation informed by implementation science frameworks may inform organizational decisions about model scale-up and sustainment. OBJECTIVE: This paper reports implementation strategies and qualitative evaluation of a primary care redesign (PCR) model implementation that included an expanded scope of work for MAs. DESIGN: Qualitative evaluation of implementation strategies and clinician and staff experience with implementation of PCR using semi-structured key informant interviews. The evaluation was guided by the RE-AIM framework and the Consolidated Framework for Implementation Research. PARTICIPANTS: Sixty-nine clinicians, staff, practice leaders, and administrators from 7 primary care practices (4 general internal medicine, 3 family medicine) implementing PCR. INTERVENTIONS: The PCR model included enhanced rooming and documentation support. The health system used multiple strategies to implement PCR, including rapid improvement events, changing clinic space configurations, developing electronic health record templates and performance dashboards, and practice coaching. APPROACH: The Consolidated Framework for Implementation Research and the RE-AIM evaluation and planning framework guided development of semi-structured interview guides. A deductive, structural coding approach was used for analysis. KEY RESULTS: PCR implementation was facilitated by clear communication about the intervention source, mechanisms for feedback about model goals, and physical environments and electronic health record (EHR) systems that supported the added staff and modified clinic workflow. Clinicians and staff benefited from the ability to see the model in action prior to go-live and opportunities for consistent provider-MA pairings. CONCLUSIONS: The PCR model can support achieving the Quadruple Aim when fully implemented with paired MAs and clinicians who are well prepared to follow redesigned workflows and function as a team. Implementation can be effectively supported by a participatory evaluation guided by implementation science frameworks.

Financial Considerations Associated With a Fourth Year of Residency Training in Family Medicine: Findings From the Length of Training Pilot Study.

BACKGROUND AND OBJECTIVES: The feasibility of funding an additional year of residency training is unknown, as are perspectives of residents regarding related financial considerations. We examined these issues in the Family Medicine Length of Training Pilot. METHODS: Between 2013 and 2019, we collected data on matched 3-year and 4-year programs using annual surveys, focus groups, and in-person and telephone interviews. We analyzed survey quantitative data using descriptive statistics, independent samples t test, Fisher's Exact Test and χ2. Qualitative analyses involved identifying emergent themes, defining them and presenting exemplars. RESULTS: Postgraduate year (PGY)-4 residents in 4-year programs were more likely to moonlight to supplement their resident salaries compared to PGY-3 residents in three-year programs (41.6% vs 23.0%; P=.002), though their student debt load was similar. We found no differences in enrollment in loan repayment programs or pretax income. Programs' descriptions of financing a fourth year as reported by the program director were limited and budget numbers could not be obtained. However, programs that required a fourth year typically reported extensive planning to determine how to fund the additional year. Programs with an optional fourth year were budget neutral because few residents chose to undertake an additional year of training. Resources needed for a required fourth year included resident salaries for the fourth year, one additional faculty, and one staff member to assist with more complex scheduling. Residents' concerns about financial issues varied widely. CONCLUSIONS: Adding a fourth year of training was financially feasible but details are local and programs could not be compared directly. For programs that had a required rather than optional fourth year much more financial planning was needed.

Practice Transformation Under the University of Colorado's Primary Care Redesign Model.

PURPOSE: We compared the transformation experience of 2 family medicine practices that implemented the Primary Care Redesign (PCR) team-based model to improve access, quality, and experience without increasing cost. The University of Colorado's A.F. Williams Family Medicine clinic (pilot practice) implemented the model in February 2015, and a smaller, community-based practice (wave 2 practice) did so 2 years later, in February 2017. METHODS: The PCR model increased the ratio of medical assistants to clinicians from about 1:2 to 2.5:1 while expanding the role of the medical assistants, through enhanced rooming procedures, in-room support (eg, scribing), postclinician wrap-up, and in-basket assistance. We assessed access, clinical quality metrics, staffing costs, and clinician and staff experience and burnout for at least 7 months before and 42 months after the intervention. RESULTS: In the pilot practice, compared with preimplementation, there were improvements in total appointments and rates of hypertension control, colorectal cancer screening, and most diabetic quality metrics. In the wave 2 practice, total appointments increased slightly when clinicians were added pre-PCR and then increased substantially after implementation; initially variable hypertension control improved rapidly after implementation. The wave 2 practice's colorectal cancer screening improved gradually, then accelerated postimplementation, while diabetic metrics initially remained stable or declined, then improved postimplementation. New patient appointments began to increase for both practices in late 2015, but grew faster in the pilot practice under PCR. Over time, all experiential domains improved for clinicians; most remained stable for staff. Clinician burnout was reduced by at least one-half in both practices except during low staffing periods, which also adversely affected staff. After a ramp-up period, the number of staff hours per visit remained stable. CONCLUSIONS: The PCR model is associated with simultaneous improvements in quality, access, and clinician experience, as well as reductions in burnout, while maintaining staffing costs.

Residency Training in Family Medicine: A History of Innovation and Program Support.

BACKGROUND: Residency programs have been integral to the development, expansion and progression of family medicine as a discipline. Three reports formed the foundation for graduate medical education in family medicine: Meeting the Challenge of Family Practice, The Graduate Education of Physicians, and Health is a Community Affair. In addition, the original core concepts of comprehensiveness, coordination, continuity, and patient centeredness continue to serve as the foundation for residency training in family medicine. While the Residency Review Committee for Family Medicine of the Accreditation Council for Graduate Medical Education has provided the requirements for training throughout the years, key organizations including the Society of Teachers of Family Medicine, the American Academy of Family Physicians, the Association of Family Medicine Residency Directors, and the American Board of Family Medicine have provided resources for and supported innovation in programs. Residency Program Solutions, National Institute for Program Director Development, and Family Medicine Residency Curriculum Resource are several of the resources developed by these organizations. The future of family medicine residency training should continue the emphasis on innovation and development of resources to enhance the training of residents. Areas for further development include leadership and health care systems training that allows residents to assume leadership of multidisciplinary health care teams and increase focus on the family medicine practice population as the main unit for resident education.

The Importance of and the Complexities Associated With Measuring Continuity of Care During Resident Training: Possible Solutions Do Exist.

BACKGROUND AND OBJECTIVES: Evolutions in care delivery toward the patient-centered medical home have influenced important aspects of care continuity. Primary responsibility for a panel of continuity patients is a foundational requirement in family medicine residencies. In this paper we characterize challenges in measuring continuity of care in residency training in this new era of primary care. METHODS: We synthesized the literature and analyzed information from key informant interviews and group discussions with residency faculty and staff to identify the challenges and possible solutions for measuring continuity of care during family medicine training. We specifically focused on measuring interpersonal continuity at the patient level, resident level, and health care team level. RESULTS: Challenges identified in accurately measuring interpersonal continuity of care during residency training include: (1) variability in empanelment approaches for all patients, (2) scheduling complexity in different types of visits, (3) variability in ability to attain continuity counts at the level of the resident, and (4) shifting make-up of health care teams, especially in residency training. Possible solutions for each challenge are presented. Philosophical issues related to continuity are discussed, including whether true continuity can be achieved during residency training and whether qualitative rather than quantitative measures of continuity are better suited to residencies. CONCLUSIONS: Measuring continuity of care in residency training is challenging but possible, though improvements in precision and assessment of the comprehensive nature of the relationships are needed. Definitions of continuity during training and the role continuity measurement plays in residency need further study.

Program Director Participation in a Leadership and Management Skills Fellowship and Characteristics of Program Quality.

BACKGROUND AND OBJECTIVES: The association between a residency program director completing a leadership and management skills fellowship and characteristics of quality and innovation of his/her residency program has not been studied. Therefore, the aim of this study is to examine the association between a residency program director's completion of a specific fellowship addressing these skills (National Institute for Program Director Development or NIPDD) and characteristics of quality and innovation of the program they direct. METHODS: Using information from the American Academy of Family Physicians (AAFP), National Resident Matching Program (NRMP) and FREIDA® program characteristics were obtained. Descriptive statistics were used to summarize the data. The relationship between programs with a NIPDD graduate as director and program quality measures and indicators of innovation was analyzed using both chi square and logistic regression. RESULTS: Initial analyses showed significant associations between the NIPDD graduate status of a program director and regional location, mean years of program director tenure, and the program's 5-year aggregate ABFM board pass rate from 2007--2011. After grouping the programs into tertiles, the regression model showed significant positive associations with programs offering international experiences and being a NIPDD graduate. CONCLUSIONS: Program director participation in a fellowship addressing leadership and management skills (ie, NIPDD) was found to be associated with higher pass rates of new graduates on a Board certification examination and predictive of programs being in the upper tertile of programs in terms of Board pass rates.

Prioritization of patients for comprehensive medication review by a clinical pharmacist in family medicine.

BACKGROUND: This pilot study describes and evaluates the clinical pharmacy priority (CP2) score. We hypothesize that patients with high CP2 scores are more likely to receive a medication recommendation after comprehensive medication review (CMR) than patients with lower scores. Prioritization of patients for CMR by a clinical pharmacist in family medicine could enhance the provision of interprofessional care within the patient-centered medical home. METHODS: The CP2 score was developed collaboratively by the research team and is derived from 11 patient-specific factors extracted from the electronic health record. To evaluate the utility of the score, CMR was performed prospectively by a clinical pharmacist for patients with appointments between October 1 and December 31, 2012, at 2 University of Colorado family medicine clinics. RESULTS: CMR was performed for 1107 patient appointments. Of these, 101 were identified as having received a medication recommendation from the clinical pharmacist. For patients with a CP2 score of 0 to 2, 2 of 588 charts (0.3%) reviewed received a recommendation (level 1). The proportion increased to 37 of 358 (10.3%) for scores of 3 to 7 (level 2), 40 of 119 (33.6%) for scores of 8 to 10 (level 3), and 22 of 42 (52.4%) for scores of ≥11 (level 4). Compared with CP2 scores in level 1, patient appointments were more likely to receive a medication recommendation after CMR in level 2 (relative risk [RR], 30.4; 95% confidence interval [CI], 7.4-125.3), in level 3 (RR, 98.8; 95% CI, 24.2-403.3), and in level 4 (RR, 154; 95% CI, 37.5-632.8). CONCLUSIONS: Patients with higher CP2 scores were more likely to receive a medication recommendation after CMR by a clinical pharmacist than patients with lower scores. The CP2 score could be used by clinical pharmacists in family medicine to enhance the efficient and effective delivery of interprofessional care.

An off-site clinical pharmacy service in family medicine: development and 1-year outcomes.

BACKGROUND AND OBJECTIVES: Clinical pharmacists are a part of integrated health care teams and provide clinical medication recommendations for family physicians. On-site clinical pharmacy services are common in family medicine. This model may not be the most effective or efficient way to provide clinical pharmacy services in a small practice or in a remote location. The objectives of this study were to describe the development of an off-site clinical pharmacy service and to describe the 1-year clinical impact of this service. METHODS: The University of Colorado Park Meadows Family Medicine Clinic is located approximately 15 miles from the Anschutz Medical Campus. In July 2011, a clinical pharmacist implemented clinical pharmacy services with the goal of providing medication expertise primarily using an off-site model. The clinical pharmacist prospectively screened patients with appointments and provided medication recommendations in the electronic medical record for providers to consider at the patient appointment. RESULTS: For the first 12 months, the clinical pharmacist spent 118 hours providing the clinical pharmacy service. A total of 315 medication recommendations were made for 123 patients; 69.8% were implemented. Forty-nine vaccinations were administered, and 24 potentially dangerous major drug-drug interactions were identified and resolved. Thirty-one unnecessary high-cost drugs were discontinued, resulting in estimated annual savings of $52,215.36. CONCLUSIONS: Our data indicate that clinical pharmacy services can be implemented for smaller remote family clinics using an offsite model. Within this model, clinical pharmacy interventions optimized medication use, managed serious drug interactions, and resulted in cost avoidance.

Recruitment and retention of physicians for primary care research.

The primary objective of this report is to examine factors associated with recruitment of physicians in community-based primary care research. Reported results are based on an observational study of physician recruitment efforts undertaken in a randomized controlled trial designed to improve primary care physicians' cancer screening and counseling activities. The Partners for Prevention project was a statewide randomized controlled trial of primary care physicians selected from the state of Colorado. Two-hundred and ten eligible internal medicine and family medicine practices in both rural and urban community settings of the state of Colorado were selected into this study and a sentinel physician was chosen to represent each practice. Only 6% (13/210) of recruited practices initially declined to participate in the study, but the total refusal rate had reached 30% (59/210) by the time the intervention was implemented five months later. Study participants (n = 136) were younger (mean age 45.7 vs. 50.0, p = 0.008) and more often located in a rural area (46% vs. 31%, p = 0.04) than decliners (n = 59), but there was no association with gender of the physician (87% for females vs. 95% for males, p = 0.13). Participants were more often family practice physicians by training rather than internists (75% vs. 56%, p = 0.008), whereas there was no difference in participation rates by practice type (solo versus group, 60% vs. 64%, p = 0.52). Differences in demographic, geographic, and training characteristics between trial participants and decliners suggest the potential for better targeting of recruitment efforts. Viable strategies for recruiting community-based primary care practices to research studies are proposed.