Redesigning Residency Training: Summary Findings From the Preparing the Personal Physician for Practice (P4) Project.

BACKGROUND AND OBJECTIVES: The Preparing the Personal Physician for Practice (P4) project (2007 to 2014) involved a comparative case study of experiments conducted by 14 selected family medicine programs designed to evaluate new models of residency education that aligned with the patient-centered medical home (PCMH). Changes in length, structure, content, and location of training were studied. METHODS: We conducted both a critical review of P4 published Evaluation Center and site-specific papers and a qualitative narrative analysis of process reports compiled throughout the project. We mapped key findings from P4 to results obtained from a survey of program directors on their top 10 "need to know" areas in family medicine education. RESULTS: Collectively, 830 unique residents took part in P4, which explored 80 hypotheses regarding 44 innovations. To date, 39 papers have resulted from P4 work, with the P4 Evaluation Center producing 17 manuscripts and faculty at individual sites producing 22 manuscripts. P4 investigators delivered 21 presentations and faculty from P4 participating programs delivered 133 presentations at national meetings. For brevity, we present findings derived from the analyses of project findings according to the following categories: (1) how residency training aligned with PCMH; (2) educational redesign and assessment; (3) methods of financing new residency experiences; (4) length of training; (5) scope of practice; and (6) setting standards for conducting multisite educational research. CONCLUSIONS: The P4 project was a successful model for multisite graduate medical education research. Insights gained from the P4 project could help family medicine educators with future residency program redesign.

Impact of Residency Training Redesign on Residents' Clinical Knowledge.

BACKGROUND AND OBJECTIVES: The In-training Examination (ITE) is a frequently used method to evaluate family medicine residents' clinical knowledge. We compared family medicine ITE scores among residents who trained in the 14 programs that participated in the Preparing the Personal Physician for Practice (P4) Project to national averages over time, and according to educational innovations. METHODS: The ITE scores of 802 consenting P4 residents who trained in 2007 through 2011 were obtained from the American Board of Family Medicine. The primary analysis involved comparing scores within each academic year (2007 through 2011), according to program year (PGY) for P4 residents to all residents nationally. A secondary analysis compared ITE scores among residents in programs that experimented with length of training and compared scores among residents in programs that offered individualized education options with those that did not. RESULTS: Release of ITE scores was consented to by 95.5% of residents for this study. Scores of P4 residents were higher compared to national scores in each year. For example, in 2011, the mean P4 score for PGY1 was 401.2, compared to the national average of 386. For PGY2, the mean P4 score was 443.1, compared to the national average of 427, and for PGY3, the mean P4 score was 477.0, compared to the national PGY3 score of 456. Scores of residents in programs that experimented with length of training were similar to those in programs that did not. Scores were also similar between residents in programs with and without individualized education options. CONCLUSIONS: Family medicine residency programs undergoing substantial educational changes, including experiments in length of training and individualized education, did not appear to experience a negative effect on resident's clinical knowledge, as measured by ITE scores. Further research is needed to study the effect of a wide range of residency training innovations on ITE scores over time.

Rural Curricular Guidelines Based on Practice Scope of Recent Residency Graduates Practicing in Small Communities.

BACKGROUND AND OBJECTIVES: The optimal curriculum for training family physicians for rural practice within a traditional urban-based residency is not defined. We used the scope of practice among recent family medicine graduates of residencies associated with Preparing the Personal Physician for Practice (P4), practicing in small communities, to identify rural curriculum components. METHODS: We surveyed graduates 18 months after residency between 2007 and 2014. The survey measured self-reported practice characteristics, including community size, and scope of practice. We compared the subgroups according to practice community size. RESULTS: Compared to graduates in larger communities, those practicing in small communities were more likely to report a broader scope of clinical practice including: adult hospital care (59% vs 35%), vaginal deliveries (23% vs 12%), C sections as primary surgeon (14% vs 5%) and assistant (21% vs 8%), newborn hospital care (45% vs 24%), and procedures such as endometrial biopsy (46% vs 33%), joint injections and aspirations (89% vs 79%), and fracture care (58% vs 42%). Graduates in small communities were also more often engaged in assessing community health needs (78% vs 64%) and developing community interventions (67% vs 51%) compared to graduates in larger communities. In contrast, graduates in small communities were less likely to have integrated behavioral health (26% vs 46%) and case management support (37% vs 52%). CONCLUSIONS: A rural practice curriculum should include training toward a broad medical scope of practice as well as skills in community-oriented primary care and integrated behavioral health.

Scope of Practice Among Recent Family Medicine Residency Graduates.

BACKGROUND AND OBJECTIVES: The scope of practice among primary care providers varies, and studies have shown that family physicians' scope may be shrinking. We studied the scope of practice among graduates of residencies associated with Preparing the Personal Physician for Practice (P4) and how length of training and individualized education innovations may influence scope. METHODS: We surveyed graduates 18 months after residency between 2008 and 2014. The survey measured self-reported practice characteristics, scope of practice and career satisfaction. We assessed scope using individual practice components (25 clinical activities, 30 procedures) and a scaled score (P4-SOP) that measured breadth of practice scope. We conducted subgroup analyses according to exposure to innovations over the project period and exposure to specific innovations. RESULTS: No significant differences were found in mean P4-SOP scores between the Pre and Full P4 groups. Compared to national data, P4 graduates reported higher rates for vaginal deliveries (19.3% vs 9.2%), adult inpatient care (48.5% vs 33.7%) and nursing home care (25.4 vs 11.7%) in practice. Graduates exposed to innovations that lengthened training, compared to standard training length, were more likely to include adult hospital care (58.2% vs 38.5%, P=0.002), adult ICU care (30.6% vs 19.2%, P=0.047) and newborn resuscitation (25.6% vs 14%, P=0.028) in their practice and performed 19/30 procedures at higher rates. Graduates of programs with individualized training innovations reported no significant differences in scope compared to graduates without this innovation. CONCLUSIONS: Graduates of residencies engaged in significant educational redesign report a broad scope of practice. Innovations around the length of training may broaden scope and individualized education appears not to constrict scope.

Factors Associated With Interest in Pursuing a Fourth Year of Family Medicine Residency Training.

BACKGROUND AND OBJECTIVES: The optimal length of residency training in family medicine, as with emergency medicine, is an ongoing debate due to several factors, but perceptions of graduating residents are missing from this debate. METHODS: We used data from 3,054 family medicine residents who registered for the American Board of Family Medicine certification examination in 2014. A practice demographic questionnaire was a mandatory component of registration. The survey included the question, "If another year of training were available in your residency program, how likely would you be to pursue that?" We examined several factors potentially associated with interest in additional training. RESULTS: The sample was predominantly female (53.9%), white (65.7%), and non-Hispanic (91.7%), with a mean age of 32.5 years. Overall, 52.8% of respondents reported being not at all likely to pursue additional training if available, 26.4% reported being somewhat likely, and 21.4% reported being moderately or extremely likely to do so. Male respondents were more likely than females to express no interest in an additional year of training (57.7% versus 48.1%). As educational debt increased, the likelihood of interest in additional training decreased. Anticipated practice settings, planned scope of practice, and intention to perform specific procedures also influenced interest in pursuing additional training. CONCLUSION: Slightly less than half of graduating family medicine residents expressed interest in a fourth year of training. Interest in a fourth year was associated with several characteristics that may help the discipline decide on the optimal length of training.

Team Training in Family Medicine Residency Programs and Its Impact on Team-Based Practice Post-Graduation.

OBJECTIVE: Our objective was to examine perceptions of adequacy in team-based care training during residency and whether this influences practice choice post- residency training. METHODS: We analyzed self-administered survey data from recent residency graduates collected as part of the Preparing Personal Physicians for Practice (P4) Project to characterize residents' perceptions of adequacy of training they received on team-based care. Multivariable logistic regression was used to assess the association between adequacy of team-based care training and joining practices that use team-based care after residency graduation, adjusting for differences in demographics. RESULTS: A total of 241 residency graduates were included in these analyses with response rates to surveys of 80.8%-98.1%. They reported practicing in 31 different US states or districts and four other countries. Over 82% of residency graduates reported being adequately trained in team-based care, 9.5% reported being overtrained, and 7.9% reported receiving no team-based care training over the study period. Seventy-six percent of P4 graduates joined practices that used team-based care in 2011, which increased to 86% (81/94) in 2013. The adjusted odds of practicing in settings with team-based care was 5.7 times higher for residents who reported being adequately prepared for team-based care compared to those who reported receiving no team-based care training and was 12.5 times higher for those who reported being over-prepared compared to those who reported no training/under-prepared. CONCLUSIONS: The majority of residency graduates perceive they were well trained in team-based care, which is significantly associated with joining practices that use team-based care post graduation.

Early Career Outcomes of Family Medicine Residency Graduates Exposed to Innovative Flexible Longitudinal Tracks.

BACKGROUND AND OBJECTIVES: The Preparing the Personal Physician for Practice (P4) project used a case series design to study innovations in the content, length, structure, and location of residency training in 14 geographically diverse family medicine programs between 2007 and 2012. We aimed to explore how offering flexible longitudinal tracks (FLT) affected graduates' scope of practice, particularly in maternal child health (MCH), which included at least 17 months of focused training that increased each year over 4 years. METHODS: We administered a cross-sectional survey to graduates of P4 residencies approximately 18 months after they completed training (2011-2014) and compared graduates of the John Peter Smith (JPS) Family Medicine Residency MCH FLT to all other P4 graduates. RESULTS: The overall response rate was 81.8% (365/446). JPS graduates who completed the flexible MCH track (n=15) compared to all other P4 graduates (n=332) were more likely to deliver babies (13/15, 86.7% versus 48/324, 14.6%) and perform C-sections as the primary surgeon (12/15, 80.0% versus 15/322, 4.7%). Additional areas of expanded scope associated with the MCH track included endoscopy (4/15, 26.7% versus 10/323, 3.1%), the care of hospitalized adults and associated procedures (central lines, eg: 8/15, 53.3% versus 47/322, 14.6%), and the care of hospitalized children (13/15, 86.7% versus 111/323, 34.4%). CONCLUSIONS: Graduating from the JPS MCH FLT was associated with a higher provision of maternal, child, and ill adult patient care services, including associated procedures.

Varied Rates of Implementation of Patient-Centered Medical Home Features and Residents' Perceptions of Their Importance Based on Practice Experience.

BACKGROUND AND OBJECTIVES: Little is known about how the patient-centered medical home (PCMH) is being implemented in residency practices. We describe both the trends in implementation of PCMH features and the influence that working with PCMH features has on resident attitudes toward their importance in 14 family medicine residencies associated with the P4 Project. METHODS: We assessed 24 residency continuity clinics annually between 2007-2011 on presence or absence of PCMH features. Annual resident surveys (n=690) assessed perceptions of importance of PCMH features using a 4-point scale (not at all important to very important). We used generalized estimating equations logistic regression to assess trends and ordinal-response proportional odds regression models to determine if resident ratings of importance were associated with working with those features during training. RESULTS: Implementation of electronic health record (EHR) features increased significantly from 2007-2011, such as email communication with patients (33% to 67%), preventive services registries (23% to 64%), chronic disease registries (63% to 82%), and population-based quality assurance (46% to 79%). Team-based care was the only process of care feature to change significantly (54% to 93%). Residents with any exposure to EHR-based features had higher odds of rating the features more important compared to those with no exposure. We observed consistently lower odds of the resident rating process of care features as more important with any exposure compared to no exposure. CONCLUSIONS: Residencies engaged in educational transformation were more successful in implementing EHR-based PCMH features, and exposure during training appears to positively influence resident ratings of importance, while exposure to process of care features are slower to implement with less influence on importance ratings.

Association Between Patient- Centered Medical Home Features and Satisfaction With Family Medicine Residency Training in the US.

BACKGROUND AND OBJECTIVES: Primary care residencies are undergoing dramatic changes because of changing health care systems and evolving demands for updated training models. We examined the relationships between residents' exposures to patient-centered medical home (PCMH) features in their assigned continuity clinics and their satisfaction with training. METHODS: Longitudinal surveys were collected annually from residents evaluating satisfaction with training using a 5-point Likert-type scale (1=very unsatisfied to 5=very satisfied) from 2007 through 2011, and the presence or absence of PCMH features were collected from 24 continuity clinics during the same time period. Odds ratios on residents' overall satisfaction were compared according to whether they had no exposure to PCMH features, some exposure (1-2 years), or full exposure (all 3 or more years). RESULTS: Fourteen programs and 690 unique residents provided data to this study. Resident satisfaction with training was highest with full exposure for integrated case management compared to no exposure, which occurred in 2010 (OR=2.85, 95% CI=1.40, 5.80). Resident satisfaction was consistently statistically lower with any or full exposure (versus none) to expanded clinic hours in 2007 and 2009 (eg, OR for some exposure in 2009 was 0.31 95% CI=0.19, 0.51, and OR for full exposure 0.28 95% CI=0.16, 0.49). Resident satisfaction for many electronic health record (EHR)-based features tended to be significantly lower with any exposure (some or full) versus no exposure over the study period. For example, the odds ratio for resident satisfaction was significantly lower with any exposure to electronic health records in continuity practice in 2008, 2009, and 2010 (OR for some exposure in 2008 was 0.36; 95% CI=0.19, 0.70, with comparable results in 2009, 2010). CONCLUSIONS: Resident satisfaction with training was inconsistently correlated with exposure to features of PCMH. No correlation between PCMH exposure and resident satisfaction was sustained over time.

Five Key Leadership Actions Needed to Redesign Family Medicine Residencies.

BACKGROUND: New skills are needed to properly prepare the next generation of physicians and health professionals to practice in medical homes. Transforming residency training to address these new skills requires strong leadership. OBJECTIVE: We sought to increase the understanding of leadership skills useful in residency programs that plan to undertake meaningful change. METHODS: The Preparing the Personal Physician for Practice (P4) project (2007-2014) was a comparative case study of 14 family medicine residencies that engaged in innovative training redesign, including altering the scope, content, sequence, length, and location of training to align resident education with requirements of the patient-centered medical home. In 2012, each P4 residency team submitted a final summary report of innovations implemented, overall insights, and dissemination activities during the study. Six investigators conducted independent narrative analyses of these reports. A consensus meeting held in September 2012 was used to identify key leadership actions associated with successful educational redesign. RESULTS: Five leadership actions were associated with successful implementation of innovations and residency transformation: (1) manage change; (2) develop financial acumen; (3) adapt best evidence educational strategies to the local environment; (4) create and sustain a vision that engages stakeholders; and (5) demonstrate courage and resilience. CONCLUSIONS: Residency programs are expected to change to better prepare their graduates for a changing delivery system. Insights about effective leadership skills can provide guidance for faculty to develop the skills needed to face practical realities while guiding transformation.

Innovating within the ACGME regulatory environment is not an oxymoron.

BACKGROUND AND OBJECTIVES: The aim of this study was to describe the analysis of program citations and cycle length for reaccreditation in the 14 family medicine residencies participating in the P4 project. METHODS: An exploratory narrative analysis was conducted on all actions taken by the Review Committee for Family Medicine (RC-FM) between 2003 and 2012. The analysis included cycle length and types of citations associated with accreditation actions. Several validation steps were undertaken to confirm findings reported. RESULTS: Mean cycle length for all P4 programs was 4.0 before P4 (2007) and did not change significantly during P4. The average number of citations per program before P4 was 6.2, and during P4 the average was 6.8. The P4 averages were similar to national norms during the project period. The citations that most commonly decreased during the P4 project were: Continuity of Patient Care/Inpatient, FMC Patient Population/Patient Volume, Orthopedics or Sports Medicine Curriculum, Resident Final Evaluation, Resident Workload/Duty Hours, and Resident Attrition. The citations that most commonly increased during the P4 project were FMC Patient Population/Demographics, Certifying Exam Scores, and Management of Health Systems Curriculum. CONCLUSIONS: Innovation and redesign of residency training in the P4 programs appears not to have affected the average cycle length or number of citations per program. The current regulatory environment in family medicine residency education appears to allow for innovation and experimentation.

Financing Residency Training Redesign.

BACKGROUND: Redesign in the health care delivery system creates a need to reorganize resident education. How residency programs fund these redesign efforts is not known. METHODS: Family medicine residency program directors participating in the Preparing Personal Physicians for Practice (P(4)) project were surveyed between 2006 and 2011 on revenues and expenses associated with training redesign. RESULTS: A total of 6 university-based programs in the study collectively received $5,240,516 over the entire study period, compared with $4,718,943 received by 8 community-based programs. Most of the funding for both settings came from grants, which accounted for 57.8% and 86.9% of funding for each setting, respectively. Department revenue represented 3.4% of university-based support and 13.1% of community-based support. The total average revenue (all years combined) per program for university-based programs was just under $875,000, and the average was nearly $590,000 for community programs. The vast majority of funds were dedicated to salary support (64.8% in university settings versus 79.3% in community-based settings). Based on the estimated ratio of new funding relative to the annual costs of training using national data for a 3-year program with 7 residents per year, training redesign added 3% to budgets for university-based programs and about 2% to budgets for community-based programs. CONCLUSIONS: Residencies undergoing training redesign used a variety of approaches to fund these changes. The costs of innovations marginally increased the estimated costs of training. Federal and local funding sources were most common, and costs were primarily salary related. More research is needed on the costs of transforming residency training.

Measuring family physician identity: the development of a new instrument.

OBJECTIVE: Our objective was to describe the development and psychometric assessment of an instrument designed to assess family medicine identity in residency training sites and compare responses from physician faculty and residents. METHODS: We conducted 28 focus groups between 2007--2008, 14 with faculty and 14 with residents who were part of the Preparing Personal Physicians for Practice (P4) Project. The first 22 focus groups were exploratory, and the second six were confirmatory where we shared working variable statements scored using a 5-point Likert scale. We then administered the survey to 223 faculty and 147 residents who were part of the P4 Project, followed by a principal component (factor) analysis, retaining items that reflected domains with eigenvalues higher than 1.0. RESULTS: A total of 223 family physician faculty and 147 residents completed the identity survey. The item analysis extraction loadings ranged from 0.36 to 0.70. Based on item grouping patterns, five domains were reflected in the data: Patient/Family Relationships, Patient Advocacy, Career Flexibility, Balancing the Breadth and Depth in Practice, and Comprehensive Nature of Patient Care. Compared to residents, faculty conveyed stronger agreement about being comfortable balancing the breadth and depth of medical knowledge needed in practice and using a variety of approaches to supplement their medical knowledge about patient care compared to residents (90.6% versus 68.7% for breadth and depth, 95.9% versus 88.3 for using a variety of approaches). Compared to faculty, residents agreed more strongly that the ability to choose many options in how to build their practice appeals to them compared to faculty (89.1% versus 82.9%). CONCLUSIONS: We successfully developed and tested a survey designed to measure family medicine identity in residencies, with five domains. Survey item responses were different between residents and faculty, which indicates the instrument may be sensitive to important changes over time.

Integrating behavioral and physical health care in the real world: early lessons from advancing care together.

BACKGROUND: More than 20 years ago the Institute of Medicine advocated for integration of physical and behavioral health care. Today, practices are integrating care in response to recent policy initiatives. However, few studies describe how integration is accomplished in real-world practices without the financial or research support available for most randomized controlled trials. METHODS: To study how practices integrate care, we are conducting a cross-case comparative, mixed-methods study of 11 practices participating in Advancing Care Together (ACT). Using a grounded theory approach, we analyzed multiple sources of data (eg, documents, practice surveys, field notes from observation visits, semistructured interviews, online diaries) collected from each ACT innovator. RESULTS: Integration requires making changes in organization and interpersonal relationships. During early integration efforts, challenges related to workflow and access, leadership and culture change, and tracking and using data to evaluate patient- and practice-level improvement emerged for ACT innovators. We describe the strategies innovators are developing to address these challenges. CONCLUSION: Integrating care is a fundamental and difficult change for practices and health care professionals. Research identifying common challenges that manifest in early efforts can help others attempting integration and inform state, local, and federal policies aimed at achieving wide-spread implementation.

Faculty development needs in residency redesign for practice in patient-centered medical homes: a P4 report.

OBJECTIVES: The study's objective was to describe faculty development skills needed for residency redesign in 14 family medicine residencies associated with the Preparing the Personal Physician for Practice (P4) project. METHODS: We used self-administered surveys to assess ratings of existing faculty development efforts and resident attitudes about faculty teaching between 2007 and 2011. Telephone interviews were conducted to assess faculty development activities and needs at baseline. Early project faculty development needs were addressed using tailored sessions delivered during site visits. We conducted a detailed content analysis of 14 site-specific comprehensive reports to characterize ongoing faculty development needs and faculty themes related to residency redesign. RESULTS: Early in the P4 project, faculty needs included skills in using the electronic health record (EHR) in teaching, change management, curriculum design, evaluation, learning portfolios and individualized learning plans, career coaching, qualitative research, competency-based assessment, and leadership. As the project progressed, the need for a "learning together" approach when training residents in transformed practices emerged. Using the EHR more effectively, evaluation and competency-based assessment skills, individualized curriculum design, better career coaching skills, shared leadership, and team-based care skills were consistent faculty development needs. Redesign strategies included having a committed core faculty group, faculty retreats, curricular change process management, intra-residency collaboration, and providing adequate support for key individuals. CONCLUSIONS: Faculty attempting to redesign residencies to train residents in patient-centered medical homes need new skills, and understanding these needs can inform faculty development programs nationally to achieve the crucial mission of training the workforce to accomplish this transformation.

Assessing the impact of innovative training of family physicians for the patient-centered medical home.

BACKGROUND: New approaches to enhance access in primary care necessitate change in the model for residency education. PURPOSE: To describe instrument design, development and testing, and data collection strategies for residency programs, continuity clinics, residents, and program graduates participating in the Preparing the Personal Physician for Practice (P(4)) project. METHODS: We developed and pilot-tested surveys to assess demographic characteristics of residents, clinical and operational features of the continuity clinics and educational programs, and attitudes about and implementation status of Patient Centered Medical Home (PCMH) characteristics. Surveys were administered annually to P(4) residency programs since the project started in 2007. Descriptive statistics were used to profile data from the P(4) baseline year. RESULTS: Most P(4) residents were non-Hispanic white women (60.7%), married or partnered, attended medical school in the United States and were the first physicians in their families to attend medical school. Nearly 85% of residency continuity clinics were family health centers, and about 8% were federally qualified health centers. The most likely PCMH features in continuity clinics were having an electronic health record and having fully secure remote access available; both of which were found in more than 50% of continuity clinics. Approximately one-half of continuity clinics used the electronic health record for safety projects, and nearly 60% used it for quality-improvement projects. CONCLUSIONS: We created a collaborative evaluation model in all 14 P(4) residencies. Successful implementation of new surveys revealed important baseline features of residencies and residents that are pertinent to studying the effects of new training models for the PCMH.

A p4 report: effect of curriculum innovation on residency applications and match performance.

BACKGROUND AND OBJECTIVES: Family medicine is actively engaged in residency redesign, but it is unclear how curricular innovation and restructuring of residency programs will affect their performance in the National Resident Matching Program (NRMP). METHODS: The Preparing the Personal Physician for Practice (P4) Project is a residency redesign initiative of 14 family medicine residency programs. Applicant and Match data provided by P4 programs were analyzed to determine if Match performance improved between the pre- (2006--2007) and post- (2008--2010) P4 program years and were compared to national applicant and Match data obtained from Electronic Residency Application System (ERAS) data and the NRMP. RESULTS: The mean number of US MD senior applicants per program increased from 53 before P4 to 81 after P4 implementation. The mean number of applicants interviewed per program increased nearly 40% in the post-P4 period. The mean percent of positions filled in the Match increased from 72.6% before P4 to 86.8% post-P4. Programs that implemented individualized training significantly improved the percent of positions filled in the Match compared to those that did not, 90% versus 83. CONCLUSIONS: In the family medicine P4 programs, innovations in residency curriculum, especially those with individualized training, appear to have a positive influence on student interest and program performance in the Match.