Measuring overfitting in nonlinear models: a new method and an application to health expenditures.

When fitting an econometric model, it is well known that we pick up part of the idiosyncratic characteristics of the data along with the systematic relationship between dependent and explanatory variables. This phenomenon is known as overfitting and generally occurs when a model is excessively complex relative to the amount of data available. Overfitting is a major threat to regression analysis in terms of both inference and prediction. We start by showing that the Copas measure becomes confounded by shrinkage or expansion arising from in-sample bias when applied to the untransformed scale of nonlinear models, which is typically the scale of interest when assessing behaviors or analyzing policies. We then propose a new measure of overfitting that is both expressed on the scale of interest and immune to this problem. We also show how to measure the respective contributions of in-sample bias and overfitting to the overall predictive bias when applying an estimated model to new data. We finally illustrate the properties of our new measure through both a simulation study and a real-data illustration based on inpatient healthcare expenditure data, which shows that the distinctions can be important.

The causal effects of home care use on institutional long-term care utilization and expenditures.

Limited evidence exists on whether expanding home care saves money overall or how much institutional long-term care can be reduced. This paper estimates the causal effect of Medicaid-financed home care services on the costs and utilization of institutional long-term care using Medicaid claims data. A unique instrumental variable was applied to address the potential bias caused by omitted variables or reverse effect of institutional care use. We find that the use of Medicaid-financed home care services significantly reduced but only partially offset utilization and Medicaid expenditures on nursing facility services. A $1000 increase in Medicaid home care expenditures avoided 2.75 days in nursing facilities and reduced annual Medicaid nursing facility costs by $351 among people over age 65 when selection bias is addressed. Failure to address selection biases would misestimate the substitution and offset effects.

Is personality associated with health care use by older adults?

CONTEXT: The patterns of health care utilization in the United States pose well-established challenges for public policy. Although economic and sociological research has resulted in considerable knowledge about what influences the use of health services, the psychological literature in this area is underdeveloped. Importantly, it is not known whether personality traits are associated with older adults' use of acute and long-term care services. METHODS: Data were collected from 1,074 community-dwelling seniors participating in a Medicare demonstration. First they completed a self-report questionnaire measuring the "Big Five" personality traits: Neuroticism, Extraversion, Openness to Experience, Agreeableness, and Conscientiousness. During the next two years, the participants maintained daily journals of their use of health care services. We used regression models based on the Andersen behavioral model of health care utilization to test for associations. FINDINGS: Our hypothesis that higher Neuroticism would be associated with greater health care use was confirmed for three services-probability of any emergency department (ED) use, likelihood of any custodial nursing home use, and more skilled nursing facility (SNF) days for SNF users-but was disconfirmed for hospital days for those hospitalized. Higher Openness to Experience was associated with a greater likelihood of custodial home care use, and higher Agreeableness and lower Conscientiousness with a higher probability of custodial nursing home use. For users, lower Openness was associated with more ED visits and SNF days, and lower Conscientiousness with more ED visits. For many traits with significant associations, the predicted use was 16 to 30 percent greater for people high (low) versus low (high) in specific traits. CONCLUSIONS: Personality traits are associated with Medicare beneficiaries' use of many expensive health care services, findings that have implications for health services research and policy. Accordingly, person-centered interventions, population-based translational effectiveness programs, and other personalized approaches that leverage the profound advances in personality psychology in recent decades should be considered.

Reliance on Veterans Affairs outpatient care by Medicare-eligible veterans.

OBJECTIVE: To examine longitudinal changes in Medicare-eligible veterans' reliance on the Department of Veterans Affairs (VA) healthcare system for primary and specialty care over 4 years. METHODS: We merged VA administrative and Medicare claims data to examine outpatient use during fiscal years (FY) 2001 to 2004 by 15,520 Medicare-eligible veterans who used VA primary care in FY2000. Reliance on VA outpatient care was defined as the proportion of total (VA/Medicare) visits received in VA for primary or specialty care. RESULTS: Of 869,000 primary and specialty care visits in the study period, 39% occurred within VA and 77% were specialty care. Reliance on VA primary care was substantially higher than specialty care (66% vs. 50% in FY2001; P<0.001). Reliance on VA primary and specialty care decreased over time (57% vs. 31% in FY2004; P<0.001). Significant shifts occurred at both extremes of VA reliance. From FY2001 to FY2004, the proportion of patients in the top decile of reliance on VA primary care decreased from 39% to 31%, whereas the proportion in the bottom decile doubled from 8% to 18%. Similarly, the proportion of patients in the top decile of reliance on VA specialty care decreased from 24% to 13%, whereas the proportion in the bottom decile doubled from 22% to 47%. CONCLUSIONS: Reliance on VA primary and specialty care among VA primary care patients decreased substantially over time, particularly for specialty care. Increasing use of non-VA services may complicate VA's implementation of patient-centered medical home models and performance measurement.

Estimating lifetime or episode-of-illness costs under censoring.

Many analyses of healthcare costs involve use of data with varying periods of observation and right censoring of cases before death or at the end of the episode of illness. The prominence of observations with no expenditure for some short periods of observation and the extreme skewness typical of these data raise concerns about the robustness of estimators based on inverse probability weighting (IPW) with the survival from censoring probabilities. These estimators also cannot distinguish between the effects of covariates on survival and intensity of utilization, which jointly determine costs. In this paper, we propose a new estimator that extends the class of two-part models to deal with random right censoring and for continuous death and censoring times. Our model also addresses issues about the time to death in these analyses and separates the survival effects from the intensity effects. Using simulations, we compare our proposed estimator to the inverse probability estimator, which shows bias when censoring is large and covariates affect survival. We find our estimator to be unbiased and also more efficient for these designs. We apply our method and compare it with the IPW method using data from the Medicare-SEER files on prostate cancer.

Issues for the next generation of health care cost analyses.

BACKGROUND: Given the characteristics of health care expenditure/cost data-a mass of observations at zero, and skewed positive expenditures, various alternative estimators have been developed that can address the analytical issues these characteristics raise. The field continues to develop new approaches and to evaluate the performance of the existing ones. OBJECTIVES: We discuss the strengths and limitations in existing methods for estimation and for model specification and checking. We suggest some areas that need fuller development or a better understanding of how the estimation approach performs when the outcome exhibits the skewness and heavy right tails that are typical of health care data. We also address various other aspects of cost analysis that include dealing with induced censoring, estimating casual effects, and generating reliable predictions that may apply to many studies. RESULTS: No current method is optimal or dominant for all cost applications. Many of the diagnostics used in choosing among alternatives have limitations that need more careful study. Several avenues in modeling cost data remain unexplored. CONCLUSIONS: Taken together, we hope that this essay would serve as a guide to the choice among methods and to the next generation of methodological research in this field.

Cost-effectiveness of systematic depression treatment among people with diabetes mellitus.

CONTEXT: Depression co-occurring with diabetes mellitus is associated with higher health services costs, suggesting that more effective depression treatment might reduce use of other medical services. OBJECTIVE: To evaluate the incremental cost and cost-effectiveness of a systematic depression treatment program among outpatients with diabetes. DESIGN: Randomized controlled trial comparing systematic depression treatment program with care as usual. SETTING: Primary care clinics of group-model prepaid health plan. PATIENTS: A 2-stage screening process identified 329 adults with diabetes and current depressive disorder. INTERVENTION: Specialized nurses delivered a 12-month, stepped-care depression treatment program beginning with either problem-solving treatment psychotherapy or a structured antidepressant pharmacotherapy program. Subsequent treatment (combining psychotherapy and medication, adjustments to medication, and specialty referral) was adjusted according to clinical response. MAIN OUTCOME MEASURES: Depressive symptoms were assessed by blinded telephone assessments at 3, 6, 12, and 24 months. Health service costs were assessed using health plan accounting records. RESULTS: Over 24 months, patients assigned to the intervention accumulated a mean of 61 additional days free of depression (95% confidence interval [CI], 11 to 82 days) and had outpatient health services costs that averaged $314 less (95% CI, $1007 less to $379 more) compared with patients continuing in usual care. When an additional day free of depression is valued at $10, the net economic benefit of the intervention is $952 per patient treated (95% CI, $244 to $1660). CONCLUSIONS: For adults with diabetes, systematic depression treatment significantly increases time free of depression and appears to have significant economic benefits from the health plan perspective. Depression screening and systematic depression treatment should become routine components of diabetes care.

Optimal health insurance for prevention and treatment.

This paper reexamines the efficiency-based arguments for optimal health insurance, extending the classic analysis to consider optimal coverage for prevention and treatment separately. Our paper considers the tradeoff between individuals' risk reduction on the one hand, and both ex ante and ex post moral hazard on the other. We demonstrate that it is always desirable to offer at least some insurance coverage for preventive care if individual consumers ignore the impact of their preventive care on the health premium. Using a utility-based framework, we reconfirm the conventional tradeoff between risk avoidance (by risk sharing) and moral hazard for insuring treatment goods. Uncompensated losses that reduce effective income provide a new efficiency-based argument for more generous insurance coverage for prevention and treatment of health conditions. The optimal coinsurance rates for prevention and for treatment are not identical.

The impact of welfare reform on insurance coverage before pregnancy and the timing of prenatal care initiation.

OBJECTIVE: This study investigates the impact of welfare reform on insurance coverage before pregnancy and on first-trimester initiation of prenatal care (PNC) among pregnant women eligible for Medicaid under welfare-related eligibility criteria. DATA SOURCES: We used pooled data from the Pregnancy Risk Assessment Monitoring System for eight states (AL, FL, ME, NY, OK, SC, WA, and WV) from 1996 through 1999. STUDY DESIGN: We estimated a two-part logistic model of insurance coverage before pregnancy and first-trimester PNC initiation. The impact of welfare reform on insurance coverage before pregnancy was measured by marginal effects computed from coefficients of an interaction term for the postreform period and welfare-related eligibility and on PNC initiation by the same interaction term and the coefficients of insurance coverage adjusted for potential simultaneous equation bias. We compared the estimates from this model with results from simple logistic, ordinary least squares, and two-stage least squares models. PRINCIPAL FINDINGS: Welfare reform had a significant negative impact on Medicaid coverage before pregnancy among welfare-related Medicaid eligibles. This drop resulted in a small decline in their first-trimester PNC initiation. Enrollment in Medicaid before pregnancy was independent of the decision to initiate PNC, and estimates of the effect of a reduction in Medicaid coverage before pregnancy on PNC initiation were consistent over the single- and two-stage models. Effects of private coverage were mixed. Welfare reform had no impact on first-trimester PNC beyond that from reduced Medicaid coverage in the pooled regression but separate state-specific regressions suggest additional effects from time and income constraints induced by welfare reform may have occurred in some states. CONCLUSIONS: Welfare reform had significant adverse effects on insurance coverage and first-trimester PNC initiation among our nation's poorest women of childbearing age. Improved outreach and insurance options for these women are needed to meet national health goals.

The Drivers of Discretionary Utilization: Clinical History Versus Physician Supply.

PURPOSE: Because the effect of physician supply on utilization remains controversial, literature based on non-Medicare populations is sparse, and a physician supply expansion is under way, the potential for physician-induced demand across diverse populations is important to understand. A substantial proportion of gastrointestinal endoscopies may be inappropriate. The authors analyzed the impact of physician supply, practice patterns, and clinical history on esophagogastroduodenoscopy (EGD, defined as discretionary) among patients hospitalized with lower gastrointestinal bleeding (LGIB). METHOD: Among 34,344 patients hospitalized for LGIB from 2004 to 2009, 43.1% and 21.3% had a colonoscopy or EGD, respectively, during the index hospitalization or within 6 months after. Linking to the Dartmouth Atlas via patients' hospital referral region, gastroenterologist density and hospital care intensity (HCI) index were ascertained. Adjusting for age, gender, comorbidities, and race/education indicators, the association of gastroenterologist density, HCI index, and history of upper gastrointestinal disease with EGD was estimated using logistic regression. RESULTS: EGD was not associated with gastroenterologist density or HCI index, but was associated with a history of upper gastrointestinal disease (OR 2.30; 95% CI 2.17-2.43), peptic ulcer disease (OR 4.82; 95% CI 4.26-5.45), and liver disease (OR 1.34; 95% CI 1.18-1.54). CONCLUSIONS: Among patients hospitalized with LGIB, large variation in gastroenterologist density did not predict EGD, but relevant clinical history did, with association strengths commensurate with risk for upper gastrointestinal bleeding. In the scenario studied, no evidence was found that specialty physician supply increases will result in more discretionary care within commercially insured populations.

Cardiovascular risk profile earlier in life and Medicare costs in the last year of life.

BACKGROUND: Health care costs are generally highest in the year before death, and much attention has been directed toward reducing costs for end-of-life care. However, it is unknown whether cardiovascular risk profile earlier in life influences health care costs in the last year of life. This study addresses this question. METHODS: Prospective cohort of adults from the Chicago Heart Association Detection Project in Industry included 6582 participants (40% women), aged 33 to 64 years at baseline examination (1967-1973), who died at ages 66 to 99 years. Medicare billing records (1984-2002) were used to obtain cardiovascular disease-related and total charges (adjusted to year 2002 dollars) for inpatient and outpatient services during the last year of life. Participants were classified as having favorable levels of all major cardiovascular risk factors (low risk), that is, serum cholesterol level lower than 200 mg/dL (<5.2 mmol/L), blood pressure 120/80 mm Hg or lower and no antihypertensive medication, body mass index (calculated as weight in kilograms divided by the square of height in meters) lower than 25, no current smoking, no diabetes, and no electrocardiographic abnormalities, or unfavorable levels of any 1 only, any 2 only, any 3 only, or 4 or more of these risk factors. RESULTS: In the last year of life, average Medicare charges were lowest for low-risk persons. For example, cardiovascular disease-related and total charges were lower by 10,367 dollars and 15,318 dollars compared with those with 4 or more unfavorable risk factors; the fewer the unfavorable risk factors, the lower the Medicare charges (P for trends <.001). Analyses by sex showed similar patterns. CONCLUSION: Favorable cardiovascular risk profile earlier in life is associated with lower Medicare charges at the end of life.

One-hour postload plasma glucose in middle age and Medicare expenditures in older age among nondiabetic men and women: the Chicago Heart Association Detection Project in Industry.

OBJECTIVE: To examine associations in nondiabetic individuals of 1-h postload plasma glucose measured in young adulthood and middle age with subsequent Medicare expenditures for cardiovascular disease (CVD), diabetes, cancer, and all health care at age 65 years or older using data from the Chicago Heart Association Detection Project in Industry (CHA). RESEARCH DESIGN AND METHODS: Medicare data (1984-2000) were linked with CHA baseline records (1967-1973) for 8,580 men and 6,723 women ages 33-64 years who were free of coronary heart disease, diabetes, and major electrocardiogram (ECG) abnormalities and who were Medicare eligible (65+ years) for at least 2 years. Participants were classified based on 1-h postload plasma glucose levels <120, 120-199, or > or =200 mg/dl. RESULTS: With adjustment for baseline age, cigarette smoking, serum cholesterol, systolic blood pressure, BMI, ethnicity, education, and minor ECG abnormalities, the average annual and cumulative Medicare, total, and diabetes- and CVD-related charges were significantly higher with higher baseline plasma glucose in women, while only diabetes-related charges were significantly higher in men. For example, in women, multivariate-adjusted CVD-related cumulative charges were, respectively, USD 14,260, 18,909, and 21,183 for the three postload plasma glucose categories (P value for trend = 0.035). CONCLUSIONS: These findings suggest that maintaining low glucose levels early in life has the potential to reduce health care costs in older age.

Generalized modeling approaches to risk adjustment of skewed outcomes data.

There are two broad classes of models used to address the econometric problems caused by skewness in data commonly encountered in health care applications: (1) transformation to deal with skewness (e.g., ordinary least square (OLS) on ln(y)); and (2) alternative weighting approaches based on exponential conditional models (ECM) and generalized linear model (GLM) approaches. In this paper, we encompass these two classes of models using the three parameter generalized Gamma (GGM) distribution, which includes several of the standard alternatives as special cases-OLS with a normal error, OLS for the log-normal, the standard Gamma and exponential with a log link, and the Weibull. Using simulation methods, we find the tests of identifying distributions to be robust. The GGM also provides a potentially more robust alternative estimator to the standard alternatives. An example using inpatient expenditures is also analyzed.

Relationship of fruit and vegetable consumption in middle-aged men to medicare expenditures in older age: the Chicago Western Electric Study.

BACKGROUND: High fruit and vegetable intake is associated with lower risk of hypertension, cardiovascular disease, and cancer. Little is known about the relationship of fruit and vegetable intake to health care expenditures. OBJECTIVE: Examine whether fruit and vegetable intake among middle-aged adults is related to Medicare charges-total, cardiovascular disease, cancer-related-in older age. DESIGN: Participants were grouped into one of three strata according to fruit and vegetable intake, determined from detailed dietary history (1958-1959): less than 14 cups per month, 14 to 42 cups per month, or more than 42 cups per month. Combined intake was classified as low, medium, or high. Medicare claims data (1984-2000) were used to estimate mean annual spending for eligible surviving participants (65 years and older) from the Chicago Western Electric Study: 1,063 men age 40 to 55 and without coronary heart disease, diabetes, and cancer at baseline (1957-1958). Cumulative charges before death (n = 401) were also calculated. RESULTS: Higher fruit and fruit plus vegetable intakes were associated with lower mean annual and cumulative Medicare charges (P values for trend .019 to .862). For example, with adjustment for baseline age, education, total energy intake, and multiple baseline risk factors, annual cardiovascular disease-related charges were 3,128 dollars vs 4,223 dollars for men with high vs low intake of fruit plus vegetables. Corresponding figures were 1,352 dollars vs 1,640 dollars for cancer-related charges and 10,024 dollars vs 12,211 dollars for total charges. Results were generally similar for vegetable intake. CONCLUSION: These findings, albeit mostly not statistically significant, suggest that for men high intake of fruits and fruits plus vegetables earlier in life has potential not only for better health status but also for lower health care costs in older age.

Effects of physician experience on costs and outcomes on an academic general medicine service: results of a trial of hospitalists.

BACKGROUND: Hospitalists may decrease costs and improve outcomes in hospitalized patients, but existing evidence is limited and has not identified mechanisms for such effects. OBJECTIVE: To study the costs and outcomes for patients on an academic general medicine service assigned to teams led by hospitalists and nonhospitalists. DESIGN: Cohort study. SETTING: Academic general medicine service. PATIENTS: 6511 patients admitted to the hospital from July 1997 through June 1999. INTERVENTION: All patients admitted every fourth day were assigned to 1 of 2 hospitalists caring for inpatients 6 months each year or 1 of 58 nonhospitalists caring for inpatients 1 to 2 months each year. MEASUREMENTS: Length of stay; inpatient costs; and 30-, 60-, and 365-day mortality. RESULTS: Patients assigned to hospitalists (24.8%) and nonhospitalists (75.2%) did not differ in age, race, sex, diagnosis mix, or Charlson index score. In year 1, average adjusted length of stay was 0.29 day shorter for patients cared for by hospitalists than by nonhospitalists (95% CI, -0.66 to 0.06 day; P = 0.06); in year 2, average adjusted length of stay was 0.49 day shorter for patients cared for by hospitalists (CI, -0.79 to -0.15 day; P = 0.01). Average adjusted costs were not significantly reduced for hospitalists compared with nonhospitalists in year 1 but were reduced by $782 in year 2 (CI, -$1313 to -$187; P = 0.01). When years 1 and 2 were combined or when year 1 was analyzed alone, 30-day mortality was not significantly different for hospitalists and nonhospitalists; however, 30-day mortality was 4.2% for hospitalists compared with 6.0% for nonhospitalists in year 2 (CI for difference, 1.8 percentage points [-3.6 to -0.1 percentage points]; P = 0.04) and the adjusted relative risk was 0.65 (CI, 0.44 to 0.96; P = 0.03). In multivariate analyses, resource use decreased with the physician's cumulative experience in caring for a patient's primary diagnosis. Mortality showed a similar pattern. CONCLUSIONS: Hospitalist care was associated with lower costs and short-term mortality in the second but not the first year of hospitalists' experience. Disease-specific physician experience may reduce resource use and improve patient outcomes; in addition, it may be an important determinant of the effectiveness of hospitalists.

Welfare reform, insurance coverage pre-pregnancy, and timely enrollment: an eight-state study.

Implementation of the Personal Responsibility and Work Opportunity Reconciliation Act (PRWORA) broke the automatic linkage between Medicaid eligibility/enrollment and welfare cash assistance for women eligible at welfare income levels. This study used data from the Pregnancy Risk Assessment Monitoring System (PRAMS) for the period 1996-1999 to examine insurance coverage of these and other pregnant women pre- and post-PRWORA. Controlling for individual characteristics and economic growth, the relative odds of having private insurance did not change while the odds of being Medicaid enrolled versus uninsured pre-pregnancy declined for welfare-eligible women post-PRWORA. The absolute effect was a decline of 7.9 percentage points in the probability of welfare-eligible women being insured. While these results apply to the early years of welfare reform, it is still likely that states can improve Medicaid outreach and enrollment of women eligible prior to pregnancy.

Optimal health insurance for multiple goods and time periods.

We examine the efficiency-based arguments for second-best optimal health insurance with multiple treatment goods and multiple time periods. Correlated shocks across health care goods and over time interact with complementarity and substitutability to affect optimal cost sharing. Health care goods that are substitutes or have positively correlated demand shocks should have lower optimal patient cost sharing. Positive serial correlations of demand shocks and uncompensated losses that are positively correlated with covered health services also reduce optimal cost sharing. Our results rationalize covering pharmaceuticals and outpatient spending more fully than is implied by static, one good, or one period models.

Covering the uninsured: what is it worth?

One out of six Americans under age sixty-five lacks health insurance, a situation that imposes sizable hidden costs upon society. The poorer health and shorter lives of those without coverage account for most of these costs. Other impacts are manifested by Medicare and disability support payments, demands on the public health infrastructure, and losses of local health service capacity. We conclude that the estimated value of health forgone each year because of uninsurance (dollars 65-dollars 130 billion) constitutes a lower-bound estimate of economic losses resulting from the present level of uninsurance nationally.

Alcohol dependence and the price of alcoholic beverages.

This study estimates the impact of the price of alcoholic beverages on latent dimensions of current alcohol dependence and abuse. A three-part econometric model is used to estimate the impact of price on three latent dimensions (factors). For heavier drinking, the estimated price elasticity is -1.325 (P = 0.027); for physical and other consequences of drinking, -1.895 (P = 0.003); for increased salience of drinking, -0.411 (P = 0.339). For a single latent factor characterized simply as dependence/abuse, estimated price elasticity is -1.487 (P = 0.012). These results suggest that higher prices for alcohol reduce important dimensions of current alcohol dependence and abuse.