Systematic bias in predictions of new drugs' budget impact: analysis of a sample of recent US drug launches.

OBJECTIVE: Expectations about the budget impact of new drug launches may affect payer behavior and ultimately consumer costs. Therefore, we evaluated the accuracy of pre-launch US budget impact estimates for a sample of new drugs. METHODS: We searched for publicly available budget impact estimates made pre-launch for drugs approved in the US from 1 September 2010 to 1 September 2015 and compared them to actual sales. Accuracy was calculated as the ratio of pre-launch estimate to actual sales. Quantitative analyses, including multivariate regressions, were used to identify factors associated with accuracy. RESULTS: We identified 25 budget impact estimates: 23 for one of 14 individual drugs and 2 for the category of PCSK9 inhibitors. The ratios of predicted to actual budget impact ranged from 0.2 (estimate was 20% of sales) for secukinumab to 37.5 (estimate was 37.5 × sales) for PCSK9 inhibitors. Mean ratio was 5.5. In multivariate analyses, larger eligible population, more recent estimate year (e.g. 2015 vs. 2012), and being first in class, were associated with statistically significant, greater overestimation of budget impact. CONCLUSIONS: For every $5.5 of predicted cost, there was $1 of actual cost to the healthcare system. This study, although based on a small, non-random sample, suggests possible cognitive bias on the part of the estimators. Overestimating budget impact may lead to early access restrictions, higher copays, and other changes that ultimately impact patients. Analysts and non-profits should be attuned to likely sources of error in order to improve their predictions.

Measuring the Value of New Drugs: Validity and Reliability of 4 Value Assessment Frameworks in the Oncology Setting.

BACKGROUND: Several organizations have developed frameworks to systematically assess the value of new drugs. OBJECTIVE: To evaluate the convergent validity and interrater reliability of 4 value frameworks to understand the extent to which these tools can facilitate value-based treatment decisions in oncology. METHODS: Eight panelists used the American Society of Clinical Oncology (ASCO), European Society for Medical Oncology (ESMO), Institute for Clinical and Economic Review (ICER), and National Comprehensive Cancer Network (NCCN) frameworks to conduct value assessments of 15 drugs for advanced lung and breast cancers and castration-refractory prostate cancer. Panelists received instructions and published clinical data required to complete the assessments, assigning each drug a numeric or letter score. Kendall's Coefficient of Concordance for Ranks (Kendall's W) was used to measure convergent validity by cancer type among the 4 frameworks. Intraclass correlation coefficients (ICCs) were used to measure interrater reliability for each framework across cancers. Panelists were surveyed on their experiences. RESULTS: Kendall's W across all 4 frameworks for breast, lung, and prostate cancer drugs was 0.560 (P= 0.010), 0.562 (P = 0.010), and 0.920 (P < 0.001), respectively. Pairwise, Kendall's W for breast cancer drugs was highest for ESMO-ICER and ICER-NCCN (W = 0.950, P = 0.019 for both pairs) and lowest for ASCO-NCCN (W = 0.300, P = 0.748). For lung cancer drugs, W was highest pairwise for ESMO-ICER (W = 0.974, P = 0.007) and lowest for ASCO-NCCN (W = 0.218, P = 0.839); for prostate cancer drugs, pairwise W was highest for ICER-NCCN (W = 1.000, P < 0.001) and lowest for ESMO-ICER and ESMO-NCCN (W = 0.900, P = 0.052 for both pairs). When ranking drugs on distinct framework subdomains, Kendall's W among breast cancer drugs was highest for certainty (ICER, NCCN: W = 0.908, P = 0.046) and lowest for clinical benefit (ASCO, ESMO, NCCN: W = 0.345, P = 0.436). Among lung cancer drugs, W was highest for toxicity (ASCO, ESMO, NCCN: W = 0. 944, P < 0.001) and lowest for certainty (ICER, NCCN: W = 0.230, P = 0.827); and among prostate cancer drugs, it was highest for quality of life (ASCO, ESMO: W = 0.986, P = 0.003) and lowest for toxicity (ASCO, ESMO, NCCN: W = 0.200, P = 0.711). ICC (95% CI) for ASCO, ESMO, ICER, and NCCN were 0.800 (0.660-0.913), 0.818 (0.686-0.921), 0.652 (0.466-0.834), and 0.153 (0.045-0.371), respectively. When scores were rescaled to 0-100, NCCN provided the narrowest band of scores. When asked about their experiences using the ASCO, ESMO, ICER, and NCCN frameworks, panelists generally agreed that the frameworks were logically organized and reasonably easy to use, with NCCN rated somewhat easier. CONCLUSIONS: Convergent validity among the ASCO, ESMO, ICER, and NCCN frameworks was fair to excellent, increasing with clinical benefit subdomain concordance and simplicity of drug trial data. Interrater reliability, highest for ASCO and ESMO, improved with clarity of instructions and specificity of score definitions. Continued use, analyses, and refinements of these frameworks will bring us closer to the ultimate goal of using value-based treatment decisions to improve patient care and outcomes. DISCLOSURES: This work was funded by Eisai Inc. Copher and Knoth are employees of Eisai Inc. Bentley, Lee, Zambrano, and Broder are employees of Partnership for Health Analytic Research, a health services research company paid by Eisai Inc. to conduct this research. For this study, Cohen, Huynh, and Neville report fees from Partnership for Health Analytic Research. Outside of this study, Cohen receives grants and direct consulting fees from various companies that manufacture and market pharmaceuticals. Mei reports a grant from Eisai Inc. during this study. The other authors have no disclosures to report. Study concept and design were contributed by Bentley and Broder, with assistance from Elkin and Cohen. Bentley took the lead in data collection, along with Elkin, Huynh, Mukherjea, Neville, Mei, Popescu, Lee, and Zambrano. Data interpretation was performed by Bentley and Broder, along with Elkin, Cohen, Copher, and Knoth. The manuscript was written primarily by Bentley, along with Elkin and Broder, and revised by Bentley, Broder, Elkin, Cohen, Copher, and Knoth. Select components of this work's methods were presented at ISPOR 19th Annual European Congress held in Vienna, Austria, October 29-November 2, 2016, and Society for Medical Decision Making 38th Annual North American Meeting held in Vancouver, Canada, October 23-26, 2016.

Validity and Reliability of Value Assessment Frameworks for New Cancer Drugs.

BACKGROUND: Several organizations have developed frameworks to systematically assess the value of new drugs. These organizations include the American Society of Clinical Oncology (ASCO), the European Society for Medical Oncology (ESMO), the Institute for Clinical and Economic Review (ICER), and the National Comprehensive Cancer Network (NCCN). OBJECTIVES: To understand the extent to which these four tools can facilitate value-based treatment decisions in oncology. METHODS: In this pilot study, eight panelists conducted value assessments of five advanced lung cancer drugs using the ASCO, ESMO, and ICER frameworks. The panelists received instructions and published clinical data required to complete the assessments. Published NCCN framework scores were abstracted. The Kendall's W coefficient was used to measure convergent validity among the four frameworks. Intraclass correlation coefficients were used to measure inter-rater reliability among the ASCO, ESMO, and ICER frameworks. Sensitivity analyses were conducted. RESULTS: Drugs were ranked similarly by the four frameworks, with Kendall's W of 0.703 (P = 0.006) across all the four frameworks. Pairwise, Kendall's W was the highest for ESMO-ICER (W = 0.974; P = 0.007) and ASCO-NCCN (W = 0.944; P = 0.022) and the lowest for ICER-NCCN (W = 0.647; P = 0.315) and ESMO-NCCN (W = 0.611; P = 0.360). Intraclass correlation coefficients (confidence interval [CI]) for the ASCO, ESMO, and ICER frameworks were 0.786 (95% CI 0.517-0.970), 0.804 (95% CI 0.545-0.973), and 0.281 (95% CI 0.055-0.799), respectively. When scores were rescaled to 0 to 100, the ICER framework provided the narrowest band of scores. CONCLUSIONS: The ASCO, ESMO, ICER, and NCCN frameworks demonstrated convergent validity, despite differences in conceptual approaches used. The ASCO inter-rater reliability was high, although potentially at the cost of user burden. The ICER inter-rater reliability was poor, possibly because of its failure to distinguish differential value among the sample of drugs tested. Refinements of all frameworks should continue on the basis of further testing and stakeholder feedback.

Nutrition-based interventions to address metabolic syndrome in the Navajo: a systematic review.

AIMS AND OBJECTIVES: The objective of this systematic review is to identify nutrition-based interventions that may be effective for the prevention and treatment of metabolic syndrome in the Navajo. BACKGROUND: Metabolic syndrome, a major risk factor for cardiovascular disease, affects almost half of the Navajo population. The diet of the Navajo, heavy in fat and refined carbohydrates, has been identified as an important contributing factor to the high rates of metabolic syndrome in this population. DESIGN: A search was conducted on PubMed, EMBASE and CINAHL to identify studies published before October, 2013, involving nutrition-based interventions in adult populations similar to the Navajo targeting at least one measure of metabolic syndrome. METHODS: Data on efficacy and participation were gathered and synthesised qualitatively. RESULTS: Out of 19 studies included in this systematic review, 11 interventions were identified to be effective at improving at least one measure of metabolic syndrome. Level of exposure to the intervention, frequency of intervention activities, family and social support, cultural adaptation and case management were identified as factors that may improve the efficacy of an intervention. CONCLUSIONS: Multiple nutrition-based interventions have been found to be effective in populations similar to the Navajo. RELEVANCE TO CLINICAL PRACTICE: Development of a strategy to address metabolic syndrome in the Navajo may involve aspects from multiple interventions to increase efficacy and maximise participation.