Principles for planning and conducting comparative effectiveness research.

AIMS: To develop principles for planning and conducting comparative effectiveness research (CER). METHODS: Beginning with a modified existing list of health technology assessment principles, we developed a set of CER principles using literature review, engagement of multiple experts and broad stakeholder feedback. RESULTS & CONCLUSION: Thirteen principles and actions to fulfill their intent are proposed. Principles include clarity of objectives, transparency, engagement of stakeholders, consideration of relevant perspectives, use of relevant comparators, and evaluation of relevant outcomes and treatment heterogeneity. Should these principles be found appropriate and useful, CER studies should be audited for adherence to them and monitored for their impact on care management, patient relevant outcomes and clinical guidelines.

EBM, HTA, and CER: clearing the confusion.

CONTEXT: The terms evidence-based medicine (EBM), health technology assessment (HTA), comparative effectiveness research (CER), and other related terms lack clarity and so could lead to miscommunication, confusion, and poor decision making. The objective of this article is to clarify their definitions and the relationships among key terms and concepts. METHODS: This article used the relevant methods and policy literature as well as the websites of organizations engaged in evidence-based activities to develop a framework to explain the relationships among the terms EBM, HTA, and CER. FINDINGS: This article proposes an organizing framework and presents a graphic demonstrating the differences and relationships among these terms and concepts. CONCLUSIONS: More specific terminology and concepts are necessary for an informed and clear public policy debate. They are even more important to inform decision making at all levels and to engender more accountability by the organizations and individuals responsible for these decisions.

Bayesian meta-analyses for comparative effectiveness and informing coverage decisions.

BACKGROUND: Evidence-based medicine is increasingly expected in health care decision-making. The Centers for Medicare and Medicaid have initiated efforts to understand the applicability of Bayesian techniques for synthesizing evidence. As a case study, a Bayesian analysis of clinical trials of implantable cardioverter defibrillators was undertaken using patient-level data not typically available for analysis. PURPOSE: Conduct Bayesian meta-analyses of the defibrillator trials using published results to demonstrate a Bayesian approach useful to policy makers. DATA SOURCES, STUDY SELECTION, DATA EXTRACTION: We reconsidered trials in a 2007 systematic review by Ezekowitz et al (Ann Intern Med. 2007;147:251-262) and extracted information from the original published articles. Employing a Bayesian hierarchical approach, we developed a base model and 2 variants, and modeled hazard ratios separately within each year of follow-up. We considered sequential meta-analyses over time and found the predictive distribution of the results of the next trial, given its sample size. DATA SYNTHESIS: For the most robust of 3 models, the probability that the mean defibrillator effect (in the population of trials) is beneficial is greater than 0.999. In that model, about 5% of trials in the population of trials would have a detrimental effect. Despite the moderate amount of heterogeneity across the trials, there was stability of conclusions after the first 3 of the 12 total trials had been conducted. This stability enabled reasonable predictions for the results of future trials. LIMITATIONS: Inability to assess treatment effects within subsets of patients. CONCLUSIONS: Bayesian meta-analyses based on literature surveys can effectively inform coverage decisions. Bayesian modeling for endpoints such as mortality can elucidate treatment effects over time. The Bayesian approach used in a sequential manner over time can predict results and help assess the utility of future clinical trials.

Are Key Principles for improved health technology assessment supported and used by health technology assessment organizations?

Previously, our group-the International Working Group for HTA Advancement-proposed a set of fifteen Key Principles that could be applied to health technology assessment (HTA) programs in different jurisdictions and across a range of organizations and perspectives. In this commentary, we investigate the extent to which these principles are supported and used by fourteen selected HTA organizations worldwide. We find that some principles are broadly supported: examples include being explicit about HTA goals and scope; considering a wide range of evidence and outcomes; and being unbiased and transparent. Other principles receive less widespread support: examples are addressing issues of generalizability and transferability; being transparent on the link between HTA findings and decision-making processes; considering a full societal perspective; and monitoring the implementation of HTA findings. The analysis also suggests a lack of consensus in the field about some principles--for example, considering a societal perspective. Our study highlights differences in the uptake of key principles for HTA and indicates considerable room for improvement for HTA organizations to adopt principles identified to reflect good HTA practices. Most HTA organizations espouse certain general concepts of good practice--for example, assessments should be unbiased and transparent. However, principles that require more intensive follow-up--for example, monitoring the implementation of HTA findings--have received little support and execution.

Value of biologic therapy: a forecasting model in three disease areas.

OBJECTIVE: Forecast the return on investment (ROI) for advances in biologic therapies in years 2015 and 2030, based upon impact on disease prevalence, morbidity, and mortality for asthma, diabetes, and colorectal cancer. METHODS: A deterministic, spreadsheet-based, forecasting model was developed based on trends in demographics and disease epidemiology. 'Return' was defined as reductions in disease burden (prevalence, morbidity, mortality) translated into monetary terms; 'investment' was defined as the incremental costs of biologic therapy advances. Data on disease prevalence, morbidity, mortality, and associated costs were obtained from government survey statistics or published literature. Expected impact of advances in biologic therapies was based on expert opinion. Gains in quality-adjusted life years (QALYs) were valued at $100,000 per QALY. RESULTS: The base case analysis, in which reductions in disease prevalence and mortality predicted by the expert panel are not considered, shows the resulting ROIs remain positive for asthma and diabetes but fall below $1 for colorectal cancer. Analysis involving expert panel predictions indicated positive ROI results for all three diseases at both time points, ranging from $207 for each incremental dollar spent on biologic therapies to treat asthma in 2030, to $4 for each incremental dollar spent on biologic therapies to treat colorectal cancer in 2015. If QALYs are not considered, the resulting ROIs remain positive for all three diseases at both time points. CONCLUSIONS: Society may expect substantial returns from investments in innovative biologic therapies. These benefits are most likely to be realized in an environment of appropriate use of new molecules. LIMITATIONS: The potential variance between forecasted (from expert opinion) and actual future health outcomes could be significant. Similarly, the forecasted growth in use of biologic therapies relied upon unvalidated market forecasts.

Willingness to pay for cancer prevention.

Cancer inflicts great pain, burden and cost upon American society, and preventing cancer is important but not costless. The aim of this review was to explore the upper limits that American society is paying and appears willing to pay to prevent cancer, by enforced environmental regulations and implemented clinical practice guidelines. Cost-effectiveness studies of clinical and environmental cancer-prevention policies and programmes were identified through a comprehensive literature review and confirmed to be officially sanctioned and implemented, enforced or funded. Data were collected in 2005-6 and analysed in 2007. The incremental cost-effectiveness ratios (ICERs) for clinical prevention policies ranged from under $US2000 to over $US6 000 000 per life-year saved (LYS), exceeding $US100 000 per LYS for only 11 of 101 guidelines. Median ICERs for tobacco-related ($US3978/LYS), colorectal ($US22 694/LYS) and breast ($US25 687/LYS) cancer prevention were within generally accepted ranges and tended not to vary greatly, whereas those for prostate ($US73 603/LYS) and cervical ($US125 157/LYS) cancer-prevention policies were considerably higher and varied substantially more. In contrast, both the median and range of the environmental policies were enormous, with 90% exceeding $US100 000 per LYS, and ICERs ranging from $US61 004 to over $US24 billion per LYS. Notwithstanding a relatively large and accessible literature evaluating the cost effectiveness of clinical and environmental cancer-prevention policies as well as the availability of ICERs for the policies identified in this study, the apparent willingness to pay to prevent cancer in the US still varies greatly and can be extremely high, particularly for many of the environmental cancer-prevention policies.

Key principles for the improved conduct of health technology assessments for resource allocation decisions.

Health technology assessment (HTA) is a dynamic, rapidly evolving process, embracing different types of assessments that inform real-world decisions about the value (i.e., benefits, risks, and costs) of new and existing technologies. Historically, most HTA agencies have focused on producing high quality assessment reports that can be used by a range of decision makers. However, increasingly organizations are undertaking or commissioning HTAs to inform a particular resource allocation decision, such as listing a drug on a national or local formulary, defining the range of coverage under insurance plans, or issuing mandatory guidance on the use of health technologies in a particular healthcare system. A set of fifteen principles that can be used in assessing existing or establishing new HTA activities is proposed, providing examples from existing HTA programs. The principal focus is on those HTA activities that are linked to, or include, a particular resource allocation decision. In these HTAs, the consideration of both costs and benefits, in an economic evaluation, is critical. It is also important to consider the link between the HTA and the decision that will follow. The principles are organized into four sections: (i) "Structure" of HTA programs; (ii) "Methods" of HTA; (iii) "Processes for Conduct" of HTA; and (iv) "Use of HTAs in Decision Making."

Cost-effectiveness of live attenuated influenza vaccine versus inactivated influenza vaccine among children aged 24-59 months in the United States.

BACKGROUND: The US Advisory Committee on Immunization Practices (ACIP) recently expanded the influenza vaccine recommendation to include children 24-59 months of age. In a large head-to-head randomized controlled trial, live attenuated influenza vaccine, trivalent (LAIV) demonstrated a 54% relative reduction in culture-confirmed influenza illness compared with trivalent inactivated influenza vaccine (TIV) among children aged 24-59 months. OBJECTIVE: To evaluate the relative cost and benefit between two influenza vaccines (LAIV and TIV) for healthy children 24-59 months of age. METHODS: Using patient-level data from the clinical trial supplemented with cost data from published literature, we modeled the cost-effectiveness of these two vaccines. Effectiveness was measured in quality-adjusted life years (QALY) and cases of influenza avoided. The analysis used the societal perspective. RESULTS: Due to its higher acquisition cost, LAIV increased vaccination costs by USD7.72 per child compared with TIV. However, compared with TIV, LAIV reduced the number of influenza illness cases and lowered the subsequent healthcare use of children and productivity losses of parents. The estimated offsets in direct and indirect costs saved USD15.80 and USD37.72 per vaccinated child, respectively. LAIV had a net total cost savings of USD45.80 per child relative to TIV. One-way and probabilistic sensitivity analyses indicated that the model was robust across a wide range of relative vaccine efficacy and cost estimates. CONCLUSIONS: Due to its increased relative vaccine efficacy over TIV, LAIV reduced the burden of influenza and lowered both direct health care and societal costs among children 24-59 months of age.

Can managed care organizations partner with manufacturers for comparative effectiveness research?

OBJECTIVE: To describe 2 published pragmatic or practical clinical trials (PCTs) as case studies illustrating successful partnerships between managed care organizations (MCOs) and pharmaceutical manufacturers. STUDY DESIGN: In today's environment, there is increasing concern about the comparative effectiveness of medical interventions. Various opinion leaders and stakeholders lament the dearth of such evidence and are calling for the public and private sectors to invest up to billions of dollars to create better comparative evidence. METHODS: We selected 2 PCTs conducted at different points in the drug life cycle to highlight strengths, limitations, and policy implications. The phase IV study compared fluoxetine hydrochloride vs 2 generic tricyclic antidepressants in selected primary care clinics of a health maintenance organization from 1992 through 1994. The phase IIIb study compared daily budesonide via dry powder inhaler vs triamcinolone acetonide metered-dose inhaler in adult patients with persistent asthma in 25 MCOs from 1995 through 1998. RESULTS: Both PCTs were successfully sponsored and funded by pharmaceutical manufacturers in collaboration with MCOs and provided potentially useful evidence of real-world effectiveness and evidence of value to healthcare decision makers. CONCLUSIONS: Industry-sponsored PCTs in managed care are feasible when manufacturer and MCO incentives align and can provide real-world evidence of comparative effectiveness and value for money. These trials can be conducted successfully in the phase IIIb and phase IV environments.

The return on investment in health care: from 1980 to 2000.

OBJECTIVE: To estimate the return on US investment (ROI) in overall health as well as four specific conditions. METHODS: The study utilized three distinct approaches to "triangulate" the evidence as related to ROI in health care: 1) an estimation of the average ROI in additional health-care service expenditures in the United States for the year 2000 compared with the year 1980, based on US summaries of health expenditures and health outcomes; 2) an estimate of the ROI in Medicare services for the period from 1985 to 2000 for treatment of heart attack, stroke, type 2 diabetes, and breast cancer, based on National Long-term Care Survey data and Medicare claims; and 3) an estimate of the ROI for selected major treatment innovations for the same four conditions during the period from 1975 to 2000. RESULTS: We calculated that each additional dollar spent on overall health-care services produced health gains valued at Dollars 1.55 to Dollars 1.94 under our base case assumptions. The return on health gains associated with treatment for heart attack, stroke, type 2 diabetes, and breast cancer were Dollars 1.10, Dollars 1.49, Dollars 1.55, and Dollars 4.80, respectively, for every additional dollar spent by Medicare. The ROI for specific treatment innovations ranged from both savings in treatment costs and gains in health to gains in health valued at Dollars 1.12 to Dollars 38.00 for every additional dollar spent. CONCLUSION: The value of improved health in the US population in 2000 compared with 1980 significantly outweighs the additional health-care expenditures in 2000 compared with 1980.

Patient satisfaction with budesonide Turbuhaler versus triamcinolone acetonide administered via pressurized metered-dose inhaler in a managed care setting.

Dissatisfaction with medication may negatively affect compliance and thus the effectiveness of the treatment. However, no prospective well-controlled studies have assessed the relative patient satisfaction with competing inhaled corticosteroids in a real-life setting. The objective of the current study was to compare the relative patient satisfaction with budesonide inhalation powder administered via Turbuhaler (AstraZeneca LP, Wilmington, DE) (200 to 1600 microg/d using one of 3 dosing strengths: 100, 200, or 400 microg per inhalation) and triamcinolone acetonide administered via pressurized metered-dose inhaler (200 to 1600 microg/d) among persons treated in managed care settings. A total of 945 subjects 18 years of age or older diagnosed with asthma and enrolled in 25 managed care organizations participated in this prospective, randomized, open-label, parallel-group, 12-month study. As part of the study, subjects completed a self-administered, 17-item patient satisfaction questionnaire that addressed 4 domains: side effects, knowledge/ease of use, convenience, and overall satisfaction. Questionnaire reliability was assessed using Cronbach's alpha, and validity was examined by correlating subscale scores with symptom-free days and Medical Outcomes Study 36-Item Short-Form questionnaire and Asthma Quality of Life Questionnaire scores. The satisfaction questionnaire also included a previously validated section addressing patient compliance. Patients receiving budesonide had significantly higher scores for all four satisfaction subscales throughout the study period than did those receiving triamcinolone acetonide. Similarly, compliance scores were consistently higher for the budesonide group. The difference between the treatment groups in overall satisfaction scores at the end of the study was clinically meaningful. Patients treated with budesonide were significantly more satisfied and compliant with their inhaled corticosteroid regimen compared with patients treated with triamcinolone acetonide.

What will it take to make cost-effectiveness analysis acceptable in the United States?

Many believe cost-effectiveness analysis (CEA) to be an underused tool to assist healthcare decision-makers. Reasons given for its underuse have largely focused on unstandardized methods, potential for bias, lack of training particularly among potential consumers of studies, and lack of trust between sponsors and users of analyses. This commentary reflects on these and related issues, including legal and political constraints. It discounts many of the conventional arguments regarding the real obstacles to using CEA and suggests steps needed to make CEA more acceptable to US healthcare decision-makers.

Evidence based? Caveat emptor!

Medical practices, clinical practice guidelines, clinical performance measures and measurements, and a variety of health care-related administrative decisions, such as insurance coverage decisions, are claiming to be "evidence based" with increasing frequency. In this paper we examine the "evidence based" label; discuss how evidence ought to have been assembled, evaluated, and synthesized; and when evidence is sufficient for the "evidence-based" moniker to rightfully apply. We also highlight several considerations other than the strength of evidence that are relevant to several common types of health care-related administrative decisions and that influence the extent to which the resulting decisions are truly evidence based.

Economic burden of heart failure: a summary of recent literature.

OBJECTIVE: We reviewed literature published from 1995 to 2002 to highlight findings on the economic burden of heart failure (HF). Methods A key-word search of literature indexes for relevant citations identified 54 articles that were then summarized for findings on HF economics. RESULTS: Results were described in terms of burden of illness, cost-effectiveness analysis, and resource utilization and costs. Hospitalization of the elderly is the driving force behind HF costs. Interventions that decrease the frequency of hospital admissions while maintaining clinical and patient reported outcomes are considered a high priority among decision makers and clinicians. Although the cost-effectiveness of therapy with beta-adrenergic blocking agents has been well established in the literature, the cost-effectiveness of hospital- or home-based HF management programs is still under debate. The issues of payer status and physician specialty impact on decreased hospital admission and cost have been inconclusive. CONCLUSIONS: Any intervention capable of decreasing even a small fraction of adverse outcomes, most notably hospital admission and length of stay, could trigger significant cost savings in the management of HF. Public policy makers, together with clinicians identifying cost saving or cost-effective interventions in their practice, are expected to increase their efforts to evaluate the cost-effectiveness and outcomes of medical and pharmacologic interventions in HF.

Effectiveness of budesonide administered via dry-powder inhaler versus triamcinolone acetonide administered via pressurized metered-dose inhaler for adults with persistent asthma in managed care settings.

BACKGROUND: Clinical studies have demonstrated the efficacy and relative safety of inhaled corticosteroids in the treatment of asthma. However, effectiveness and cost-effectiveness comparisons of available inhaled corticosteroids in real-life clinical settings are lacking. OBJECTIVE: This study compared the effectiveness and safety of budesonide administered via dry-powder inhaler versus that of triamcinolone acetonide administered via pressurized metered-dose inhaler in the treatment of adult patients with persistent asthma treated in a managed care setting. METHODS: This was a randomized, open-labe, 52-week study of adult patients (aged >or= 18 years) with persistent asthma enrolled in 25 US health plans. The primary study outcome was mean change from baseline to the end of treatment in symptom-free days. Secondary variables were changes from baseline in number of episode-free days, episode-free days at 52 weeks, forced expiratory volume in 1 second (FEV(1)), forced vital capacity, asthma symptom scores, breakthrough bronchdilator use, patient discontinuations, and health-related quality of life. Patients were issued diaries in which to record use of study medication and concomitant asthma medication use, as well as daytime and nighttime asthma symptom severity. Patients were assessed at weeks 4, 13, 26, 39, and 52. Safety was assessed based on adverse events and changes in laboratory tests, vital signs, and physical examinations. RESULTS: A total of 945 patients (344 men, 601 women; mean [SD] age, 46.8 [14] years) were enrolled; 631 received budesonide and 314 received triacinolane acetonide. Improvements in all effectiveness variables were observed with both treatments. The mean increase from baseline in the number of symptom-free days per month assessed at month 12 was 7.74 (95% CI, 6.81-8.66) for patients receiving budesonide and 3.78 (95% CI, 2.47-5.09) for patients receiving triamcinoline acetonide ( P<0.001). The estimated annual mean (SD) number symptom-free days for patients receiving budesonide was 141.1 (125.0) over the treatment phase, compared with 99.3 (112.1) for those receiving triamcinolone acetonide (P<0.001). Patients receiving budesonide demonstrated significant improvements (compared with those receiving triamcinolone acetonide) in overall quality of life, daytime and nighttime asthma symptom severity, breakthrough bronchodilator use, and FEV(1) (all P<0.001). Safety measures were similar between groups. CONCLUSION: In these managed care settings, budesonide inhalation powder administered via dry-powder inhaler was significantly more effective than triamcinolone acetonide administered via pressurized metered-dose inhaler in the treatment of adults with persistent asthma.

Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices--Modeling Studies.

OBJECTIVES: Mathematical modeling is used widely in economic evaluations of pharmaceuticals and other health-care technologies. Users of models in government and the private sector need to be able to evaluate the quality of models according to scientific criteria of good practice. This report describes the consensus of a task force convened to provide modelers with guidelines for conducting and reporting modeling studies. METHODS: The task force was appointed with the advice and consent of the Board of Directors of ISPOR. Members were experienced developers or users of models, worked in academia and industry, and came from several countries in North America and Europe. The task force met on three occasions, conducted frequent correspondence and exchanges of drafts by electronic mail, and solicited comments on three drafts from a core group of external reviewers and more broadly from the membership of ISPOR. RESULTS: Criteria for assessing the quality of models fell into three areas: model structure, data used as inputs to models, and model validation. Several major themes cut across these areas. Models and their results should be represented as aids to decision making, not as statements of scientific fact; therefore, it is inappropriate to demand that models be validated prospectively before use. However, model assumptions regarding causal structure and parameter estimates should be continually assessed against data, and models should be revised accordingly. Structural assumptions and parameter estimates should be reported clearly and explicitly, and opportunities for users to appreciate the conditional relationship between inputs and outputs should be provided through sensitivity analyses. CONCLUSIONS: Model-based evaluations are a valuable resource for health-care decision makers. It is the responsibility of model developers to conduct modeling studies according to the best practicable standards of quality and to communicate results with adequate disclosure of assumptions and with the caveat that conclusions are conditional upon the assumptions and data on which the model is built.

The effect on social welfare of a switch of second-generation antihistamines from prescription to over-the-counter status: a microeconomic analysis.

BACKGROUND: The US Food and Drug Administration (FDA) recently held a meeting to determine whether the status of second-generation antihistamines (SGAs) should be switched from prescription (Rx) to over-the-counter (OTC) status. OBJECTIVE: This article provides a conceptual microeconomic framework for addressing issues regarding the impact of such a switch on social welfare. METHODS: A review of the economic literature on Rx-to-OTC switches was conducted. Relevant articles published in English between 1990 and 2001 were identified through searches of ABI Info, EconLit, PsychInfo, MEDLINE, CANCERLIT, AIDSLINE, and HealthStar, as well as a general Internet search for statements in the press or on the FDA Web site. The search terms used were Rx, prescription, OTC, over-the-counter, second-generation antihistamines, nonsedating antihistamines, first-generation antihistamines, and sedating antihistamines. Microeconomic models focusing on consumer surplus were employed to determine the potential price response and social-welfare implications of a switch of SGAs to OTC status. RESULTS: Unlike the agents involved in previous Rx-to-OTC switches, SGAs are still under patent protection. Economic theory suggests that a firm that is protected by a patent will price aggressively. The market for OTC SGAs is likely to be more elastic due to a lack of insurance coverage for OTC products; hence, drug manufacturers would be likely to charge a lower price if SGAs were sold OTC. However, a lower price does not necessarily guarantee an improvement in social welfare; the net impact is determined by whether the increase in consumer surplus outweighs the deadweight loss (losses of consumer and producer surplus not transferred to other parties). Additionally, the assumption of a price reduction would be called into question if there were inequalities in marginal costs between the Rx and OTC markets. In this situation, the postswitch price might increase or not be reduced significantly. CONCLUSIONS: It is uncertain whether granting OTC status to SGAs would be cost saving to society, particularly as these drugs are patent protected. The social-welfare implications of such a switch would depend heavily on pricing strategies and consumer behavior. Further analyses are needed to determine how both factors influence social welfare; only then can the costs and benefits of a switch be understood completely.