Estimating cost-offsets of new medications: use of new antipsychotics and mental health costs for schizophrenia.

Estimation of the effect of one treatment compared to another in the absence of randomization is a common problem in biostatistics. An increasingly popular approach involves instrumental variables-variables that are predictive of who received a treatment yet not directly predictive of the outcome. When treatment is binary, many estimators have been proposed: method-of-moments estimators using a two-stage least-squares procedure, generalized-method-of-moments estimators using two-stage predictor substitution or two-stage residual inclusion procedures, and likelihood-based latent variable approaches. The critical assumptions to the consistency of two-stage procedures and of the likelihood-based procedures differ. Because neither set of assumptions can be completely tested from the observed data alone, comparing the results from the different approaches is an important sensitivity analysis. We provide a general statistical framework for estimation of the casual effect of a binary treatment on a continuous outcome using simultaneous equations to specify models. A comparison of health care costs for adults with schizophrenia treated with newer atypical antipsychotics and those treated with conventional antipsychotic medications illustrates our methods. Surprisingly large differences in the results among the methods are investigated using a simulation study. Several new findings concerning the performance in terms of precision and robustness of each approach in different situations are obtained. We illustrate that in general supplemental information is needed to determine which analysis, if any, is trustworthy and reaffirm that comparing results from different approaches is a valuable sensitivity analysis.

Short screening scales to monitor population prevalences and trends in non-specific psychological distress.

BACKGROUND: A 10-question screening scale of psychological distress and a six-question short-form scale embedded within the 10-question scale were developed for the redesigned US National Health Interview Survey (NHIS). METHODS: Initial pilot questions were administered in a US national mail survey (N = 1401). A reduced set of questions was subsequently administered in a US national telephone survey (N = 1574). The 10-question and six-question scales, which we refer to as the K10 and K6, were constructed from the reduced set of questions based on Item Response Theory models. The scales were subsequently validated in a two-stage clinical reappraisal survey (N = 1000 telephone screening interviews in the first stage followed by N = 153 face-to-face clinical interviews in the second stage that oversampled first-stage respondents who screened positive for emotional problems) in a local convenience sample. The second-stage sample was administered the screening scales along with the Structured Clinical Interview for DSM-IV (SCID). The K6 was subsequently included in the 1997 (N = 36116) and 1998 (N = 32440) US National Health Interview Survey, while the K10 was included in the 1997 (N = 10641) Australian National Survey of Mental Health and Well-Being. RESULTS: Both the K10 and K6 have good precision in the 90th-99th percentile range of the population distribution (standard errors of standardized scores in the range 0.20-0.25) as well as consistent psychometric properties across major sociodemographic subsamples. The scales strongly discriminate between community cases and non-cases of DSM-IV/SCID disorders, with areas under the Receiver Operating Characteristic (ROC) curve of 0.87-0.88 for disorders having Global Assessment of Functioning (GAF) scores of 0-70 and 0.95-0.96 for disorders having GAF scores of 0-50. CONCLUSIONS: The brevity, strong psychometric properties, and ability to discriminate DSM-IV cases from non-cases make the K10 and K6 attractive for use in general-purpose health surveys. The scales are already being used in annual government health surveys in the US and Canada as well as in the WHO World Mental Health Surveys. Routine inclusion of either the K10 or K6 in clinical studies would create an important, and heretofore missing, crosswalk between community and clinical epidemiology.

Comparison of use of medications after acute myocardial infarction in the Veterans Health Administration and Medicare.

BACKGROUND: There is concern that care provided in the Veterans Health Administration (VA) may be of poorer quality than non-VA health care. We compared use of medications after acute myocardial infarction in the VA with that in non-VA healthcare settings under fee-for-service (FFS) Medicare financing. METHODS AND RESULTS: We used clinical data from 2486 VA and 29 249 FFS men >65 years old discharged with a confirmed diagnosis of acute myocardial infarction from 81 VA hospitals and 1530 non-VA hospitals. We reported odds ratios (ORs) for use of thrombolytics, beta-blockers, ACE inhibitors, or aspirin among ideal candidates adjusted for age, sample design (hospital academic affiliation, availability of cardiac procedures, and volume), and within-hospital clustering. Ideal VA candidates were more likely to undergo thrombolytic therapy at arrival (OR [VA relative to Medicare] 1.40 [1.05, 1.74]) or to receive ACE inhibitors (OR 1.67 [1.12, 2.45]) or aspirin (OR 2.32 [1.81, 3.01]) at discharge and equally likely to receive beta-blockers (OR 1.09 [1.03, 1.40]) at discharge. CONCLUSIONS: Ideal candidates in VA were at least as likely as those in FFS to receive medical therapies of known benefit for acute myocardial infarction.

Pitfalls in assessing the quality of care for patients with cardiovascular disease.

PURPOSE: There are no clinical performance measures for cardiovascular diseases that span the continuum of hospital through postdischarge ambulatory care. We tested the feasibility of developing and implementing such measures for patients with acute myocardial infarction, congestive heart failure, or hypertension. SUBJECTS AND METHODS: After reviewing practice guidelines and the medical literature, we developed potential measures related to therapy, diagnostic evaluation, and communication. We tested the feasibility of implementing the selected measures for 518 patients with myocardial infarction, 396 with heart failure, and 601 with hypertension who were enrolled in four major U.S. managed care plans at six geographic sites, using data from administrative claims, medical records, and patient surveys. RESULTS: Difficulties in obtaining timely data and small numbers of cases adversely affected measurement. We encountered 6- to 12-month delays, disagreement between principal discharge diagnosis as coded in administrative and records data (for 9% of myocardial infarction and 21% of heart failure patients), missing medical records (20% for both myocardial infarction and heart failure patients), and problems in identifying physicians accountable for care. Low rates of performing key diagnostic tests (e.g., ejection fraction) excluded many cases from measures of appropriate therapy that were conditional on test results. Patient survey response rates were low. CONCLUSIONS: Constructing meaningful clinical performance measures is straightforward, but implementing them on a large scale will require improved data systems. Lack of standardized data captured at the point of clinical care and low rates of eligibility for key measures hamper measurement of quality of care.

Identifying a cohort of patients with early-stage breast cancer: a comparison of hospital discharge and primary data.

BACKGROUND: Hospital discharge data are a potential source of information for quality of care; however, they lack detailed clinical data. OBJECTIVES: To assess the usefulness of hospital discharge data for describing patterns of care. RESEARCH DESIGN: Cohort study comparing hospital discharge data with data collected from medical records and patients. PATIENTS: Women diagnosed with early-stage breast cancer in Massachusetts and Minnesota (1993-1995). MEASURES: The percentage of patients in the primary data set who did not match a record in the discharge data set, and the percentage of patients in the discharge data set who did not match a record in the primary data set. Odds ratios for appearing in one data set, but not the other according to patient and hospital characteristics. RESULTS: For patients in the primary data set, 26.9% from Massachusetts and 13.2% from Minnesota did not match a record in the discharge data set. In both states, factors associated with failure to match to the discharge data included receipt of breast conserving surgery, shorter length of stay, and treatment hospital. For patients in the discharge data set, 43.4% in Massachusetts and 30.3% in Minnesota did not match a patient in the primary data set. In both states, factors associated with failure to match to the primary data included treatment hospital and the presence of positive lymph nodes. CONCLUSIONS: Hospital discharge data were fairly sensitive when linked to patients with early-stage breast cancer who were identified through hospital records. The discharge data lacked specificity, however. If discharge data are used to characterize patterns care for inpatients with early stage disease, estimates are likely to be inaccurate due to the inclusion of unsuitable patients in the denominator used to calculate procedure rates.

On determination of sample size in hierarchical binomial models.

We consider a two- and a three-stage hierarchical design containing the effects of k clusters with n units per cluster. In the two-stage model, the conditional distribution of the discrete response Y(i) is assumed to be independent binomial with mean n(straight theta)i (I=1,....k). The success probabilities, straight theta(i)'s, are assumed exchangeable across the k clusters, each arising from a beta distribution. In the three-stage model, the parameters in the beta distribution are assumed to have independent gamma distributions. The size of each cluster, n, is determined for functions of straight theta(i). Lengths of central posterior intervals are computed for various functions of the straight theta(i)'s using Markov chain Monte Carlo and Monte Carlo simulations. Several prior distributions are characterized and tables are provided for n with given k. Methods for sample size calculations under the two- and three-stage models are illustrated and compared for the design of a multi-institutional study to evaluate the appropriateness of discharge planning rates for a cohort of patients with congestive heart failure.

Partial versus full hospitalization for adults in psychiatric distress: a systematic review of the published literature (1957-1997).

OBJECTIVE: The authors reviewed published research that compared partial and full hospitalization as alternative programs for the care of mentally ill adults, with the goal of both systematizing the knowledge base and providing directions for future research. METHOD: Studies published since 1950 were obtained through manual and electronic searches. Results were stratified by outcome domain, type of measure used to report between-group differences (global, partial, or rate-based), and time of assessment. Effect sizes were computed and combined within a random-effects framework. RESULTS: Eighteen investigations published between 1957 and 1997 were systematically reviewed. Over half of eligible patients were excluded a priori; diagnostic severity of enrollees varied widely. On measures of psychopathology, social functioning, family burden, and service utilization, the authors found no evidence of differential outcome in the selected patient population admitted to the studies reviewed. Rates of satisfaction with services suggested an advantage for partial hospitalization within 1 year of discharge, with the gap being largest at 7-12 months. CONCLUSIONS: Although partial hospitalization is not an option for all patients requiring intensive services, outcomes of partial hospitalization patients in these studies were no different from those of inpatients. Further, patients and families were more satisfied with partial hospitalization in the short term. Weaknesses of the studies limited the scope of our inquiry and the generalizability of findings. Positive findings require replication under the present circumstances of mental health care, and more research is needed to identify predictors of differential outcome and successful partial hospitalization. A clearer definition of partial hospitalization will help consolidate its role in the continuum of mental health services.

Impact of underuse, overuse, and discretionary use on geographic variation in the use of coronary angiography after acute myocardial infarction.

BACKGROUND: Geographic variation in the use of medical procedures has been well documented. However, it is not known whether this variation is due to differences in use when procedures are indicated, discretionary, or contraindicated. OBJECTIVES: To examine whether use of coronary angiography after acute myocardial infarction (AMI) according to appropriateness criteria varied across geographic regions and whether underuse, overuse, or discretionary use accounted for variation in overall use. DESIGN: Retrospective cohort study using data from the Cooperative Cardiovascular Project. SETTING: Ninety-five hospital referral regions. PATIENTS: There were 44,294 Medicare patients hospitalized with AMI during 1994 or 1995, classified according to appropriateness for angiography. MAIN OUTCOME MEASURE: Variation in use of angiography, as measured by the difference between high and low rates of use across regions. RESULTS: Across regions, variation in the use of angiography was similar for indications judged necessary; appropriate, but not necessary; or uncertain. Variation was lowest for indications judged unsuitable (difference between high rate and low rate across regions = 16.3%; 95% CI = 12.6%; 20.6%). The primary cause of variation in the overall rate of angiography was due to use for indications judged appropriate, but not necessary or uncertain. When variation associated with these indications was accounted for, the difference between the resulting high and low overall rates was 10.8% (9.4%, 12.4%). In contrast, variation in the overall rate remained high when underuse in necessary situations or overuse in unsuitable situations was accounted for. CONCLUSIONS: Across regions, practice was more similar for patients categorized unsuitable for angiography than for patients with other indications. Variation in overall use of angiography appeared to be driven by utilization for discretionary indications rather than by underuse or overuse. If equivalent rates across geographic areas are judged desirable, then greater effort must be directed toward defining care for patients with discretionary indications.

Managed care and children's behavioral health services in Massachusetts.

OBJECTIVE: The authors investigated changes in treatment patterns and costs of care for children after the implementation of the Massachusetts Medicaid carve-out managed care plan. METHODS: The authors hypothesized that after the introduction of managed care, per-child expenditures would be reduced, continuity of care would not improve, and per-child mental health expenditures would undergo larger reductions for disabled children, compared with children enrolled in the Aid to Families With Dependent Children program. Using data from Medicaid and the Massachusetts Department of Mental Health, the authors studied 16,664 Massachusetts Medicaid beneficiaries aged one to 17 years for whom reimbursement claims were submitted for psychiatric or substance use disorder treatment at least once during the two years before the introduction of managed care (1991 to 1992) or during the two years afterward (1994 to 1995). Multivariate analysis was used to estimate changes in probability of admission, and, among patients admitted, to identify factors accounting for variation in length of stay. To assess the variation in expenditures, we regressed the same variables, using the natural logarithm function to transform total mental health expenditures data and inpatient expenditures data to reduce skewness. RESULTS: After the introduction of managed care, per-child expenditures were lower, especially for disabled children, and the Department of Mental Health was used as a safety net for the most seriously ill children without increasing state expenditures. Continuity of care appeared to decline for disabled children. CONCLUSIONS: It is likely that a combination of factors related to the reported changes in patterns of care and expenditures were responsible for the overall per-child expenditures.

Cardiac surgery report cards: comprehensive review and statistical critique.

Public report cards and confidential, collaborative peer education represent distinctly different approaches to cardiac surgery quality assessment and improvement. This review discusses the controversies regarding their methodology and relative effectiveness. Report cards have been the more commonly used approach, typically as a result of state legislation. They are based on the presumption that publication of outcomes effectively motivates providers, and that market forces will reward higher quality. Numerous studies have challenged the validity of these hypotheses. Furthermore, although states with report cards have reported significant decreases in risk-adjusted mortality, it is unclear whether this improvement resulted from public disclosure or, rather, from the development of internal quality programs by hospitals. An additional confounding factor is the nationwide decline in heart surgery mortality, including states without quality monitoring. Finally, report cards may engender negative behaviors such as high-risk case avoidance and "gaming" of the reporting system, especially if individual surgeon results are published. The alternative approach, continuous quality improvement, may provide an opportunity to enhance performance and reduce interprovider variability while avoiding the unintended negative consequences of report cards. This collaborative method, which uses exchange visits between programs and determination of best practice, has been highly effective in northern New England and in the Veterans Affairs Administration. However, despite their potential advantages, quality programs based solely on confidential continuous quality improvement do not address the issue of public accountability. For this reason, some states may continue to mandate report cards. In such instances, it is imperative that appropriate statistical techniques and report formats are used, and that professional organizations simultaneously implement continuous quality improvement programs. The statistical methodology underlying current report cards is flawed, and does not justify the degree of accuracy presented to the public. All existing risk-adjustment methods have substantial inherent imprecision, and this is compounded when the results of such patient-level models are aggregated and used inappropriately to assess provider performance. Specific problems include sample size differences, clustering of observations, multiple comparisons, and failure to account for the random component of interprovider variability. We advocate the use of hierarchical or multilevel statistical models to address these concerns, as well as report formats that emphasize the statistical uncertainty of the results.

Survival after acute myocardial infarction in patients with end-stage renal disease: results from the cooperative cardiovascular project.

Cardiovascular disease (CVD) is the most common cause of death in patients with end-stage renal disease (ESRD). The optimal management strategy in this population is unknown. We studied 640 patients with ESRD and acute myocardial infarction during 1994 to 1995 as part of the Health Care Financing Administration's Cooperative Cardiovascular Project. The majority of patients were treated with medical therapy alone, 46 patients (7%) were treated with percutaneous transluminal coronary angioplasty (PTCA), and 29 patients (5%) underwent coronary artery bypass grafting (CABG). Patient characteristics and comorbid conditions were similar among the three groups. The overall 1-year mortality rate was 53%. Advanced age, low or high body mass index, history of peripheral vascular disease or stroke, the inability to walk independently, and several indicators of cardiac dysfunction were associated with an increased relative risk (RR) for death. Survival curves differed significantly by treatment modality, with 1-year survival rates of 45%, 54%, and 69% in the medical therapy alone, PTCA, and CABG groups, respectively (P = 0.03). After adjustment for confounding variables, the RR for death was less (but not significantly so) in the CABG group (RR, 0.6; 95% confidence interval, 0.3 to 1.1). There are no randomized clinical trial data to guide therapy of CVD in patients with ESRD. On the basis of these and other available data, CABG may be the optimal therapy for CVD in ESRD. In light of the exceptionally poor outcomes observed for patients treated with medical therapy alone, it may be premature to dismiss PTCA as a therapeutic option in this population.

Outcome of myocardial infarction in Veterans Health Administration patients as compared with medicare patients.

BACKGROUND: Some have the opinion that patients cared for in Veterans Health Administration (VHA) hospitals receive care of poorer quality than those cared for in non-VHA institutions. To assess the quality of care in VHA hospitals, we compared the outcome of acute myocardial infarction among patients in VHA and non-VHA institutions while controlling for potential confounders, including coexisting conditions and severity of illness. METHODS: We studied 2486 veterans discharged from 81 VHA hospitals and 29,249 Medicare patients discharged from 1530 non-VHA hospitals, restricting our samples to men at least 65 years of age who were discharged with confirmed acute myocardial infarction. We compared coexisting conditions, severity of illness, and 30-day and 1-year mortality in the two samples. RESULTS: VHA patients were significantly more likely than Medicare patients to have a recorded history of hypertension (64.3 percent vs. 57.3 percent), chronic obstructive pulmonary disease or asthma (30.9 percent vs. 23.5 percent), diabetes (34.8 percent vs. 29.0 percent), stroke (20.4 percent vs. 14.2 percent), or dementia (7.2 percent vs. 4.8 percent) (P<0.001 for all comparisons). According to both multivariate logistic regression and an analysis using 2265 matched pairs of VHA and Medicare patients, there were no significant differences in 30-day or 1-year mortality. The matched-pairs analysis found that the difference in mortality at 30 days (the mortality rate among Medicare patients minus the mortality rate among VHA patients), averaged over the 5-year age groups, was -0.8 percent (95 percent confidence interval, -2.8 percent to 1.3 percent), and the difference in mortality at 1 year was -1.3 percent (95 percent confidence interval, -3.9 percent to 1.3 percent). CONCLUSIONS: VHA patients had more coexisting conditions than Medicare patients. Nevertheless, we found no significant difference in mortality between VHA and Medicare patients, a result that suggests a similar quality of care for acute myocardial infarction.

Positive predictive value of the diagnosis of acute myocardial infarction in an administrative database.

OBJECTIVE: To determine the positive predictive value of ICD-9-CM coding of acute myocardial infarction and cardiac procedures. METHODS: Using chart-abstracted data as the standard, we examined administrative data from the Veterans Health Administration for a national random sample of 5,151 discharges. MAIN RESULTS: The positive predictive value of acute myocardial infarction coding in the primary position was 96.9%. The sensitivity and specificity of coding were, respectively, 96% and 99% for catheterization, 95.7% and 100% for coronary artery bypass graft surgery, and 90.3% and 99. 7% for percutaneous transluminal coronary angioplasty. CONCLUSIONS: The positive predictive value of acute myocardial infarction and related procedure coding is comparable to or better than previously reported observations of administrative databases.

The value of mental health care at the system level: the case of treating depression.

The value of mental health services is regularly questioned in health policy debates. Although all health services are being asked to demonstrate their value, there are special concerns about this set of services because spending on mental health care has grown markedly over the past twenty years. We propose a method for using administrative data to develop a comprehensive assessment of value for mental health care, which we call systems cost-effectiveness (SCE). We apply the method to acute-phase treatment of depression in a large insured population. Our results show that SCE of treatment for depression has improved during the 1990s.

Understanding variability in physician ratings of the appropriateness of coronary angiography after acute myocardial infarction.

We examined variability in ratings of the appropriateness of coronary angiography for 890 clinical scenarios (indications) after an acute myocardial infarction (AMI) from a nine-member multispecialty panel as a function of panel characteristics and the attributes of the clinical indications. We documented a substantial degree of reliability in the ratings. However, key differences among the experts in terms of both their overall propensity to score high and their beliefs regarding the impact of clinical factors on appropriateness were identified. Age, cardiac complications, post-AMI angina, and noninvasive test results were the clinical factors most strongly related to appropriateness ratings for coronary angiography. Further research on the effectiveness of coronary angiography in older patients and in patients with shock, pulmonary edema, and silent ischemia is needed to improve our knowledge about the appropriateness of this procedure in these patients.

Meta-analysis: formulating, evaluating, combining, and reporting.

Meta-analysis involves combining summary information from related but independent studies. The objectives of a meta-analysis include increasing power to detect an overall treatment effect, estimation of the degree of benefit associated with a particular study treatment, assessment of the amount of variability between studies, or identification of study characteristics associated with particularly effective treatments. This article presents a tutorial on meta-analysis intended for anyone with a mathematical statistics background. Search strategies and review methods of the literature are discussed. Emphasis is focused on analytic methods for estimation of the parameters of interest. Three modes of inference are discussed: maximum likelihood; restricted maximum likelihood, and Bayesian. Finally, software for performing inference using restricted maximum likelihood and fully Bayesian methods are demonstrated. Methods are illustrated using two examples: an evaluation of mortality from prophylactic use of lidocaine after a heart attack, and a comparison of length of hospital stay for stroke patients under two different management protocols.

Applying Bayesian ideas to the development of medical guidelines.

Measurements of the quality of health care, in particular the underuse and overuse of medical therapies and diagnostic tests, often involve employment of medical practice guidelines to assess the appropriateness of treatments. This paper presents a case study of a Bayesian analysis for the development of medical guidelines based on expert opinion, using ordinal categorical rater data. We develop guidelines for the use of coronary angiography following an acute myocardial infarction (AMI) for 890 clinical indications using statistical models fit to appropriateness ratings obtained from a nine-member expert panel. The main foci of our analyses were on the estimation of an appropriateness score for each of the clinical indications, an associated measure of precision, and functions of the underlying score. We considered two classes of models that assume the ratings are either in the form of grouped normal data or are ungrouped variables arising from a normal distribution, while permitting rater effects and indication heterogeneity in both. We estimated models using Markov chain Monte Carlo methods and constructed indices quantifying appropriateness based on posterior probabilities of selected model parameters. We compared our model-based approach to the standard approach currently employed in medical guideline development and found that the standard approach correctly identified 99 per cent of the appropriate indications while overestimating appropriateness 18 per cent of the time compared to our model-based approach.

Eliciting expert opinion using the Delphi technique: identifying performance indicators for cardiovascular disease.

Combining opinion from expert panels is becoming a more common method of selecting criteria to define quality of health care. The Rand Corporation pioneered this method is the 1950s and 1960s in the context of forecasting technological events. Since then, numerous organizations have adopted the methodology to develop local and national policy. In the context of quality of care, opinion is typically elicited from a sample of experts regarding the appropriateness or importance of a medical treatment for several well-defined clinical cohorts. The information from the experts is then combined in order to create a standard or performance measure of care. This article describes how to use the panel process to elicit information from diverse panels of experts. Methods are demonstrated using the data from five distinct panels convened as part of the Harvard Q-SPAN-CD study, a nationally-funded project whose goal is to identify a set of cardiovascular-related performance measures.

Rating the appropriateness of coronary angiography--do practicing physicians agree with an expert panel and with each other?

BACKGROUND: Evaluations of the appropriateness of medical care are important to monitor the quality of care and to contain costs and enhance safety by reducing inappropriate care. Experts' views are usually incorporated into evaluations of appropriateness. However, practicing physicians may not concur with these views, and physicians' clinical backgrounds may influence their beliefs. METHODS: We asked 1058 internists, family practitioners, and cardiologists in California, Florida, New York, Pennsylvania, and Texas to rate the appropriateness of coronary angiography after acute myocardial infarction for 20 common indications. Nine clinical experts also rated these indications using an established consensus method. RESULTS: For 17 of the 20 indications, median ratings of surveyed physicians and the expert panel agreed within 1 unit on a 9-unit scale. Patients' older age had a negative effect on ratings by the expert panel but not on ratings by surveyed physicians. In multivariable analyses of surveyed physicians, cardiologists rated angiography as significantly more appropriate than did primary care physicians for complicated indications, and for uncomplicated indications cardiologists who performed invasive procedures gave higher appropriateness ratings for angiography than did cardiologists who did not perform such procedures and primary care physicians. For uncomplicated indications, physicians from hospitals providing coronary angioplasty and bypass surgery rated angiography as more appropriate than physicians from other hospitals. Physicians from New York and those employed by health maintenance organizations rated angiography as less appropriate than other physicians. CONCLUSIONS: Surveyed physicians agreed with clinical experts about the appropriateness of coronary angiography after myocardial infarction for most indications, indicating that well-designed expert panels can closely reflect the views of practicing physicians. Variations in beliefs among practicing physicians suggest that evaluations of medical practice should incorporate the views of a range of relevant types of physicians.