Assessing risk of bias in the meta-analysis of round 1 of the Health Care Innovation Awards.

BACKGROUND: Systematic reviews of observational studies can be affected by biases that lead to under- or over-estimates of true intervention effects. Several tools have been reported in the literature that attempt to characterize potential bias. Our objective in this study was to determine the extent to which study-specific bias may have influenced intervention impacts on total costs of care (TCOC) in round 1 of the Health Care Innovation Awards. METHODS: We reviewed 82 statistical evaluations of innovation impacts on Medicare TCOC. We developed five risk-of-bias measures and assessed their influence on TCOC impacts using meta-regression. RESULTS: The majority of evaluations used propensity score matching to create their comparison groups. One third of the non-randomized interventions were judged to have some risk of biased effects due largely to the way they recruited their treatment groups, and 35% had some degree of covariate imbalance remaining after propensity score adjustments. However, in the multivariable analysis of TCOC effects, none of the bias threats we examined (comparison group construction method, risk of bias, or degree of covariate imbalance) had a major impact on the magnitude of HCIA1 innovation effects. Evaluations using propensity score weighting produced larger but imprecise savings effects compared to propensity score matching. DISCUSSION: Our results suggest that it is unlikely that HCIA1 TCOC effect sizes were systematically affected by the types of bias we considered. Assessing the risk of bias based on specific study design features is likely to be more useful for identifying problematic characteristics than the subjective quality ratings used by existing risk tools.

Meta-Analysis of the Impact of Four Advanced Primary Care Redesign Initiatives on Medicare Expenditures.

We conducted a secondary analysis of the evaluations of 22 sites participating in four primary care redesign initiatives funded by the Centers for Medicare and Medicaid Services or the Center for Medicare and Medicaid Innovation. Our objectives were to determine the overall impact of the initiatives on Medicare expenditures and whether specific site-level program features influenced expenditure findings. Averaged over sites, the mean intervention effect was a statistically insignificant US$26 per beneficiary per year. Policy implications from meta-regression results suggest that funders should consider supporting technical assistance efforts and pay for performance incentives to increase savings. There was no evidence that paying for medical home transformation produced savings in total cost of care. We estimate that in future evaluations, data from 35 sites would be needed to detect feature effects of US$300 per beneficiary per year.

Making Evidence Actionable: Interactive Dashboards, Bayes, and Health Care Innovation.

The results of many large-scale federal or multi-site evaluations are typically compiled into long reports which end up sitting on policymaker's shelves. Moreover, the information policymakers need from these reports is often buried in the report, may not be remembered, understood, or readily accessible to the policymaker when it is needed. This is not a new challenge for evaluators, and advances in statistical methodology, while they have created greater opportunities for insight, may compound the challenge by creating multiple lenses through which evidence can be viewed. The descriptive evidence from traditional frequentist models, while familiar, are frequently misunderstood, while newer Bayesian methods provide evidence which is intuitive, but less familiar. These methods are complementary but presenting both increases the amount of evidence stakeholders and policymakers may find useful. In response to these challenges, we developed an interactive dashboard that synthesizes quantitative and qualitative data and allows users to access the evidence they want, when they want it, allowing each user a customized, and customizable view into the data collected for one large-scale federal evaluation. This offers the opportunity for policymakers to select the specifics that are most relevant to them at any moment, and also apply their own risk tolerance to the probabilities of various outcomes.

Profile of Justice-Involved Marijuana and Other Substance Users: Demographics, Health and Health Care, Family, and Justice System Experiences.

Substance users are more likely to have co-occurring health problems, and this pattern is intensified among those involved with the criminal justice system. Interview data for 1977 incarcerated men in 5 states from the Multi-site Family Study on Incarceration, Parenting, and Partnering that was conducted between December 2008 and August 2011 were analyzed to compare pre-incarceration substance use patterns and health outcomes between men who primarily used marijuana, primarily used alcohol, primarily used other drugs, and did not use any illicit substances during that time. Using regression modeling, we examined the influence of substance use patterns on physical and mental health. Primary marijuana users comprised the largest portion of the sample (31.5%), closely followed by nonusers (30.0%), and those who primarily used other drugs (30.0%); primary alcohol users comprised the smallest group (19.6%). The substance user groups differed significantly from the nonuser group on many aspects of physical and mental health. Findings suggest that even among justice-involved men who are not using "hard" drugs, substance use merits serious attention. Expanding the availability of substance use treatment during and after incarceration might help to promote physical and mental health during incarceration and reentry.

Impact Of Health Care Delivery System Innovations On Total Cost Of Care.

Using delivery system innovations to advance health care reform continues to be of widespread interest. However, it is difficult to generalize about the success of specific types of innovations, since they have been examined in only a few studies. To gain a broader perspective, we analyzed the results of forty-three ambulatory care programs funded by the first round of the Center for Medicare and Medicaid Innovation's Health Care Innovations Awards. The innovations' impacts on total cost of care were estimated by independent evaluators using multivariable difference-in-differences models. Through the first two years, most of the innovations did not show a significant effect on total cost of care. Using meta-regression, we assessed the effects on costs of five common components of these innovations. Innovations that used health information technology or community health workers achieved the greatest cost savings. Savings were also relatively large in programs that targeted clinically fragile patients-clinically complex populations at risk for disease progression. While the magnitude of these effects was often substantial, none achieved conventional levels of significance in our analyses. Meta-analyses of a larger number of delivery system innovations are needed to more clearly establish their potential for patient care cost savings.

Measuring job satisfaction among healthcare staff in the United States: a confirmatory factor analysis of the Satisfaction of Employees in Health Care (SEHC) survey.

OBJECTIVE: To validate the Satisfaction of Employees in Health Care (SEHC) survey with multidisciplinary, healthcare staff in the United States (U.S.). DESIGN: A cross-sectional psychometric study using confirmatory factor analysis. The original three-factor model was tested and modified using half-samples. Models were assessed using goodness-of-fit measures. Scale reliability and validity were tested with Cronbach's α coefficient and correlation of total SEHC score with two global satisfaction items, respectively. SETTING: We administered a web-based survey from January to May 2015 to healthcare staff participating in initiatives aimed at delivering better care and reducing costs. PARTICIPANTS: The overall response rate was 38% (N = 1089), and respondents were from 86 healthcare projects. A total of 928 respondents completed the SEHC survey in full and were used in this study. MAIN OUTCOME MEASURES: Model fit of 18 SEHC items and total SEHC score. RESULTS: The mean SEHC score was 77.6 (SD: 19.0). A one-factor model of job satisfaction had high loadings on all items, and demonstrated adequate model fit (second half-sample RMSEA: 0.069). The scale demonstrated high reliability (Cronbach's alpha = 0.942) and validity (r = 0.77 and 0.76, both P < 0.05). CONCLUSIONS: The SEHC appears to measure a single general job satisfaction construct. The scale has adequate reliability and validity to recommend its use to assess satisfaction among multidisciplinary, U.S. healthcare staff. Our findings suggest that this survey is a good candidate for reduction to a short-form, and future research should validate this survey in other healthcare populations.

Physician dual practice.

Physicians employed in government clinics and hospitals also frequently have private practices. The economic theory of such dual practice is relatively limited and recent. We provide a summary and comparison of five models of dual practice, including one we have developed based on total compensation theory and contracting limitations. We also discuss whether theoretical predictions are consistent with empirical evidence from developed and developing countries. We argue that the social trade-off between the benefits and costs of dual practice hinge on the quality of a country's contracting institutions. The conclusion outlines a proposed research agenda for better understanding this widespread phenomenon in the health sector and in other segments of society.

The medical treatment of depression, 1991-1996: productive inefficiency, expected outcome variations, and price indexes.

We examine the price of treating episodes of acute phase major depression over the 1991-1996 time period. We combine data from a large retrospective medical claims data base (MarketScan, from the Medstat Group) with clinical literature and expert clinical opinion elicited from a two-stage Delphi procedure. This enables us to construct a variety of treatment price indexes that include variations over time in the proportion of the "off-frontier" production, as well as the corresponding variations in expected treatment outcomes. We find that in general the incremental cost of successfully treating an episode of acute phase major depression has generally fallen over the 1991-1996 time period. Based on hedonic regression equations that account for the effects of changing patient mix, we find reductions that range from about -1.66 to -2.13% per year.