Using the Homeland Security Exercise and Evaluation Program (HSEEP) Building Block Approach to Implement System Evaluation Theory (SET).

This article shares lessons learned in applying system evaluation theory (SET) to evaluate a Clinical and Translational Research Center (CTR) funded by the National Institutes of Health. After describing how CTR support cores are intended to work interdependently as a system, the case is made for SET as the best fit for evaluating this evaluand. The article then details how the evaluation was also challenged to facilitate a CTR culture shift, helping support cores to move from working autonomously to working together and understanding how the cores' individual operating processes impact each other. This was achieved by incorporating the Homeland Security Exercise and Evaluation Program (HSEEP) building block approach to implement SET. Each of the seven HSEEP building blocks is examined for alignment with each of SET's three steps and the ability to systematically support the goal of moving CTR cores toward working interdependently. The implications of using HSEEP to support SET implementation for future evaluations are discussed.

Lessons Learned in Evaluating the Infrastructure of a Centre for Translational Research (CTR).

This paper shares lessons learned while evaluating the implementation of a Clinical and Translational Research Center (CTR). To meet its overarching goals the CTR consists of numerous research support units (e.g., biostats, community engagement, professional development, etc.) that are intended to work together collaboratively. It is then argued that an evaluation approach grounded in system thinking was the best fit to evaluate this key CTR design feature. The rationale for selecting Systems Evaluation Theory (SET) as the evaluation framework best suited to evaluate the CTR infrastructure is then presented. The application of SET and the lessons learned are then shared. The paper concludes that there are many similarly structured programs worldwide to which the lessons learned can be applied and upfront investments in using a system approach are rewarded by providing meaningful and useful evaluation recommendations for system change.

Will rural family medicine residency training survive?

BACKGROUND AND OBJECTIVES: Rural family medicine residencies may be more threatened by declining interest in family medicine than their urban counterparts. This study examines the recent performance of rural residencies in the National Resident Matching Program as an indicator of their viability. METHODS: We surveyed all 30 family medicine residencies located in rural areas during the summer of 2004 and a geographically matched sample of 31 urban residencies. We gathered information about the matching process for 2002, 2003, and 2004. The response rate was 70.5%. RESULTS: Rural programs offer about one third fewer first-year (postgraduate year 1 [PGY-1]) positions than their urban counterparts. Rural programs had lower Match rates (60.1%) than urban programs (72.5%) in 2004 but no meaningful differences in the proportion of international medical graduates (IMGs) or osteopathic physicians (DOs) who ultimately accepted positions. The 44.2% of residencies that predicted they would be thriving 2 years in the future filled an average of 81.3% of their slots on Match Day; there were no rural/urban differences. Programs with less-optimistic appraisals of their future had much lower Match rates. Two factors were associated with lower Match rates when other variables were taken into account: the proportion of IMGs in the 2 previous entering years and a stated rural mission. CONCLUSIONS: Rural programs appear to be slightly less stable than their urban counterparts, but the differences are minor. The viability of rural family medicine residency programs is probably affected more by the overall attractiveness of family medicine as a discipline rather than the rural or urban location of the residency.

Increasing substance abuse treatment compliance for persons with traumatic brain injury.

This study compared 3 methods of increasing participation in substance abuse treatment for clients with traumatic brain injury. Participants (N = 195) were randomly assigned to 4 conditions: (a) motivational interview, (b) reduction of logistical barriers to attendance, (c) financial incentive, and (d) attention control. Four interviewers conducted structured, brief telephone interventions targeting the timeliness of signing an individualized service plan. Participants assigned to the barrier reduction (74%) and financial incentive (83%) groups were more likely to sign within 30 days compared with the motivational interview (45%) and attention control (45%) groups. Similar results were observed for time to signing, perfect attendance at appointments, and premature termination during the following 6 months. Extent of psychiatric symptoms was the only significant covariate.

Rural and urban primary care physician professional beliefs and quality improvement behaviors.

PURPOSE: We evaluated whether primary care physicians (PCPs) from urban and rural practices differ on attitudes and behaviors related to quality improvement (QI) activities, patient relationships, and professionalism/self-regulation. METHODS: Data from a national survey that assessed physician attitudes and behaviors based on the Physician Charter on Medical Professionalism were used. Of the 1,891 survey respondents, N = 840 were PCPs (n = 274 family medicine (response rate = 67.5%); n = 257 general internal medicine (60.8%); and n = 309 pediatricians (72.7%)). Using Rural-Urban Commuting Area (RUCA) codes, PCPs were classified as urban and rural according to their practice ZIP code. FINDINGS: A total of n = 691 physicians were urban and n = 127 rural. Attitudes regarding participating in QI did not differ by practice location; however, rural PCPs were more likely to have reviewed an other physician's records for QI than urban PCPs (65.6% vs 48.0%, P < .001). Rural physicians were more likely to agree that physicians should talk with their patients about the cost of care than urban PCPs (40.5% vs 29.2%, P = .02). While all PCPs endorsed attitudes regarding the importance of professional behaviors (eg, reporting impaired/incompetent colleagues, disclosing medical errors) at generally similar levels, their behaviors differed. More rural physicians had a personal knowledge of an impaired/incompetent physician than urban physicians (20.7% vs 12.7%, P = .02). CONCLUSIONS: PCPs from rural and urban areas share similar attitudes regarding the importance of participating in QI and fulfilling professional responsibilities. However, certain behaviors (eg, knowledge of impaired colleagues) do differ. These results should be confirmed in larger studies of rural PCPs.

The National Health Service Corps and Medicaid inpatient care: experience in a southern state.

BACKGROUND: Since 1970, the National Health Service Corps (NHSC) has worked to increase primary care access among underserved groups. This study examined whether NHSC alumni physicians were likely to treat a high proportion of Medicaid patients in their practices. METHODS: Using licensure files and hospital discharge data, we identified all physicians practicing in South Carolina who attended at least one discharge in 1998, excluding physicians who graduated before 1969, residents, and current NHSC-obligated physicians. The outcome studied was ranking in the highest quartile for Medicaid participation. RESULTS: Former NHSC participants, after adjustment for personal characteristics, education, and specialty, were nearly twice as likely to fall into the category of high Medicaid participation. NHSC physicians were more likely to practice in community health centers and to locate in areas with a health professions shortage and counties with high percentages of minorities and people living in poverty. CONCLUSION: NHSC alumni make career choices leading them to serve low-income patients.