Tools for assessing quality of studies investigating health interventions using real-world data: a literature review and content analysis.

OBJECTIVES: We aimed to identify existing appraisal tools for non-randomised studies of interventions (NRSIs) and to compare the criteria that the tools provide at the quality-item level. DESIGN: Literature review through three approaches: systematic search of journal articles, snowballing search of reviews on appraisal tools and grey literature search on websites of health technology assessment (HTA) agencies. DATA SOURCES: Systematic search: Medline; Snowballing: starting from three articles (D'Andrea et al, Quigley et al and Faria et al); Grey literature: websites of European HTA agencies listed by the International Network of Agencies for Health Technology Assessment. Appraisal tools were searched through April 2022. ELIGIBILITY CRITERIA FOR SELECTING STUDIES: We included a tool, if it addressed quality concerns of NRSIs and was published in English (unless from grey literature). A tool was excluded, if it was only for diagnostic, prognostic, qualitative or secondary studies. DATA EXTRACTION AND SYNTHESIS: Two independent researchers searched, screened and reviewed all included studies and tools, summarised quality items and scored whether and to what extent a quality item was described by a tool, for either methodological quality or reporting. RESULTS: Forty-nine tools met inclusion criteria and were included for the content analysis. Concerns regarding the quality of NRSI were categorised into 4 domains and 26 items. The Research Triangle Institute Item Bank (RTI Item Bank) and STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) were the most comprehensive tools for methodological quality and reporting, respectively, as they addressed (n=20; 17) and sufficiently described (n=18; 13) the highest number of items. However, none of the tools covered all items. CONCLUSION: Most of the tools have their own strengths, but none of them could address all quality concerns relevant to NRSIs. Even the most comprehensive tools can be complemented by several items. We suggest decision-makers, researchers and tool developers consider the quality-item level heterogeneity, when selecting a tool or identifying a research gap. OSF REGISTRATION NUMBER: OSF registration DOI (https://doi.org/10.17605/OSF.IO/KCSGX).

Cross-Regional Data Initiative for the Assessment and Development of Treatment for Neurological and Mental Disorders.

Purpose: To describe and categorize detailed components of databases in the Neurological and Mental Health Global Epidemiology Network (NeuroGEN). Methods: An online 132-item questionnaire was sent to key researchers and data custodians of NeuroGEN in North America, Europe, Asia and Oceania. From the responses, we assessed data characteristics including population coverage, data follow-up, clinical information, validity of diagnoses, medication use and data latency. We also evaluated the possibility of conversion into a common data model (CDM) to implement a federated network approach. Moreover, we used radar charts to visualize the data capacity assessments, based on different perspectives. Results: The results indicated that the 15 databases covered approximately 320 million individuals, included in 7 nationwide claims databases from Australia, Finland, South Korea, Taiwan and the US, 6 population-based electronic health record databases from Hong Kong, Scotland, Taiwan, the Netherlands and the UK, and 2 biomedical databases from Taiwan and the UK. Conclusion: The 15 databases showed good potential for a federated network approach using a common data model. Our study provided publicly accessible information on these databases for those seeking to employ real-world data to facilitate current assessment and future development of treatments for neurological and mental disorders.

Key Considerations in the Health Technology Assessment of Advanced Therapy Medicinal Products in Scotland, The Netherlands, and England.

OBJECTIVES: Advanced therapy medicinal products (ATMPs) are highly innovative therapies. Their costs and uncertain value claims have raised concerns among health technology assessment (HTA) bodies and payers. Little is known about how underlying considerations in HTA of ATMPs shape assessment and reimbursement recommendations. We aim to identify and assess key considerations that played a role in HTA of ATMPs underlying reimbursement recommendations. METHODS: A review of HTA reports was conducted of all authorized ATMPs in Scotland, The Netherlands, and England. Considerations were extracted and categorized into EUnetHTA Core Model domains. Per jurisdiction, considerations were aggregated and key considerations identified (defined as occurring in >1/assessment per jurisdiction). A narrative analysis was conducted comparing key considerations between jurisdictions and different reimbursement recommendations. RESULTS: We identified 15 ATMPs and 18 HTA reports. In The Netherlands and England most key considerations were identified in clinical effectiveness (EFF) and cost- and economic effectiveness (ECO) domains. In Scotland, the social aspects domain yielded most key considerations, followed by ECO and EFF. More uncertainty in evidence and assessment outcomes was accepted when orphan or end-of-life criteria were applied. A higher percentage of considerations supporting recommendations were identified for products with positive recommendations compared with restricted and negative recommendations. CONCLUSIONS: This is the first empirical review of HTA's using the EUnetHTA Core Model to identify and structure key considerations retrospectively. It provides insights in supporting and opposing considerations for reimbursement of individual products and differences between jurisdictions. Besides the EFF and ECO domain, the social, ethical, and legal domains seem to bear considerable weight in assessment of ATMPs.

The Role of Regulator-Imposed Post-Approval Studies in Health Technology Assessments for Conditionally Approved Drugs.

BACKGROUND: The European Medicines Agency (EMA) aims to resolve uncertainties associated with conditionally approved drugs by imposing post-approval studies. Results from these studies may be relevant for health technology assessment (HTA) organizations. This study investigated the role of regulator-imposed post-approval studies within HTA. METHODS: For all conditionally approved drugs up to December 2018, regulator-imposed post-approval studies were identified from EMA's public assessment reports. The availability for and inclusion of study results in relative effectiveness (re)assessments were analyzed for 4 European HTA organizations: NICE (National Institute for Health and Care Excellence, England/Wales), HAS (Haute Autorité de Santé, France), ZIN (Zorginstituut Nederland, the Netherlands) and the European Network for Health Technology Assessment (EUnetHTA, Europe). When study results became available between an HTA organization's initial assessment and reassessment, it was evaluated whether and how they affected the assessment and its outcome. RESULTS: For 36 conditionally approved drugs, 98 post-approval studies were imposed. In total, 81 initial relative effectiveness assessments (REAs) and 13 reassessments were available, with numbers of drugs (re)assessed varying greatly between jurisdictions. Study results were available for 16 initial REAs (20%) and included in 14 (88%), and available for 10 reassessments (77%) and included in all (100%). Five reassessments had an outcome different from the initial REA, with 4 (2 positive and 2 negative changes) relating directly to the new study results. Reassessments often cited the inability of post-approval studies to resolve the concerns reported in the initial REA. CONCLUSION: Results from regulator-imposed post-approval studies for conditionally approved drugs were not often used in REAs by HTA organizations, because they were often not yet available at the time of initial assessment and because reassessments were scarce. When available, results from post-approval studies were almost always used within HTA, and they have led to changes in conclusions about drugs' relative effectiveness. Post-approval studies can be relevant within HTA but the current lack of alignment between regulators and HTA organizations limits their potential.

Pre-approval and post-approval availability of evidence and clinical benefit of conditionally approved cancer drugs in Europe: A comparison with standard approved cancer drugs.

AIMS: Cancer drugs are increasingly approved through expedited regulatory pathways including the European conditional marketing authorization (CMA). Whether, when taking CMA post-approval confirmatory trials into account, the level of evidence and clinical benefit between CMA and standard approved (SMA) drugs differs remains unknown. METHODS: We identified all CMA cancer indications converted to SMA in 2006-2020 and compared these to similar SMA indications with regard to pivotal trial and CMA post-approval confirmatory trial design, outcomes and demonstrated clinical benefit (per the European Society for Medical Oncology Magnitude of Clinical Benefit Scale). We tested for differences in clinical benefit and whether substantial clinical benefit was demonstrated. To account for the clinical benefit of unconverted CMA indications, we performed sensitivity analyses. RESULTS: We included 15 SMA and 15 converted CMA cancer indications (17 remained unconverted). Approval of 11 SMA (73%) and four CMA indications (27%) was supported by a controlled trial. Improved overall survival (OS) was demonstrated for four SMA indications (27%). Improved quality of life (QoL) was demonstrated for three SMA (20%) and one CMA indication(s) (7%). Of subsequent CMA post-approval confirmatory trials, 11 were controlled (79%), one demonstrated improved OS (7%) and five improved QoL (36%). After conversion, CMA indications were associated with similar clinical benefit (P = .31) and substantial clinical benefit as SMA indications (risk ratio 1.4, 95% confidence interval 0.57-3.4). CONCLUSION: While CMA cancer indications are initially associated with less comprehensive evidence than SMA indications, levels of evidence and clinical benefit are similar after conversion from CMA to SMA.

Associations between uncertainties identified by the European Medicines Agency and national decision making on reimbursement by HTA agencies.

We aimed to determine whether uncertainties identified by the European Medicines Agency (EMA) were associated with negative relative effectiveness assessments (REAs) and negative overall reimbursement recommendations by national health technology assessment (HTA) agencies. Therefore, we identified all HTA reports from Haute Autorité de Santé (HAS; France), National Institute for Health and Care Excellence (NICE; England/Wales), Scottish Medicine Consortium (SMC; Scotland), and Zorginstituut Nederland (ZIN; The Netherlands) for a cohort of innovative medicines that the EMA had approved in 2009 to 2010 (excluding vaccines). Uncertainty regarding pivotal trial methodology, clinical outcomes, and their clinical relevance were combined to reflect a low, medium, or high level of uncertainty. We assessed associations by calculating risk ratios (RRs) and 95% confidence intervals (CIs), and agreement between REA and overall reimbursement recommendation outcomes. We identified 36 medicines for which 121 reimbursement recommendations had been issued by the HTA agencies between September 2009 and July 2018. High versus low uncertainty was associated with an increased risk for negative REAs and negative overall reimbursement recommendations: RRs 1.9 (95% CI 0.9-3.9) and 1.6 (95% CI 0.7-3.5), respectively, which was supported by further sensitivity analyses. We identified a lack of agreement between 33 (27%) REA and overall reimbursement recommendation outcomes, which were mostly restricted recommendations that followed on negative REAs in case of low or medium uncertainty. In conclusion, high uncertainty identified by the EMA was associated with negative REAs and negative overall reimbursement recommendations. To reduce uncertainty and ultimately facilitate efficient patient access, regulators, HTA agencies, and other stakeholders should discuss how uncertainties should be weighed and addressed early in the drug life cycle of innovative treatments.

Greedy caliper propensity score matching can yield variable estimates of the treatment-outcome association-A simulation study.

PURPOSE: Greedy caliper propensity score (PS) matching is dependent on randomness, which can ultimately affect causal estimates. We sought to investigate the variation introduced by this randomness. METHODS: Based on a literature search to define the simulation parameters, we simulated 36 cohorts of different sizes, treatment prevalence, outcome prevalence, treatment-outcome-association. We performed 1:1 caliper and nearest neighbor (NN) caliper PS-matching and repeated this 1000 times in the same cohort, before calculating the treatment-outcome association. RESULTS: Repeating caliper and NN caliper matching in the same cohort yielded large variations in effect estimates, in all 36 scenarios, with both types of matching. The largest variation was found in smaller cohorts, where the odds ratio (OR) ranged from 0.53 to 10.00 (IQR of ORs: 1.11-1.67). The 95% confidence interval was not consistently overlapping a neutral association after repeating the matching with both algorithms. We confirmed these findings in a noninterventional example study. CONCLUSION: Caliper PS-matching can yield highly variable estimates of the treatment-outcome association if the analysis is repeated.

Estimation of manufacturing development costs of cell-based therapies: a feasibility study.

BACKGROUND AIMS: Cell-based therapies (CBTs) provide opportunities to treat rare and high-burden diseases. Manufacturing development of these innovative products is said to be complex and costly. However, little research is available providing insight into resource use and cost drivers. Therefore, this study aimed to assess the feasibility of estimating the cost of manufacturing development of two cell-based therapy case studies using a CBT cost framework specifically designed for small-scale cell-based therapies. METHODS: A retrospective costing study was conducted in which the cost of developing an adoptive immunotherapy of Epstein-Barr virus-specific cytotoxic T lymphocytes (CTLs) and a pluripotent stem cell (PSC) master cell bank was estimated. Manufacturing development was defined as products advancing from technology readiness level 3 to 6. The study was conducted in a Scottish facility. Development steps were recreated via developer focus groups. Data were collected from facility administrative and financial records and developer interviews. RESULTS: Application of the manufacturing cost framework to retrospectively estimate the manufacturing design cost of two case studies in one Scottish facility appeared feasible. Manufacturing development cost was estimated at £1,201,016 for CTLs and £494,456 for PSCs. Most costs were accrued in the facility domain (56% and 51%), followed by personnel (20% and 32%), materials (19% and 15%) and equipment (4% and 2%). CONCLUSIONS: Based on this study, it seems feasible to retrospectively estimate resources consumed in manufacturing development of cell-based therapies. This fosters inclusion of cost in the formulation and dissemination of best practices to facilitate early and sustainable patient access and inform future cost-conscious manufacturing design decisions.

Increasing the information provided by probabilistic sensitivity analysis: The relative density plot.

BACKGROUND: Results of probabilistic sensitivity analyses (PSA) are frequently visualized as a scatterplot, which is limited through overdrawing and a lack of insight in relative density. To overcome these limitations, we have developed the Relative Density plot (PSA-ReD). METHODS: The PSA-ReD combines a density plot and a contour plot to visualize and quantify PSA results. Relative density, depicted using a color gradient, is transformed to a cumulative probability. Contours are then plotted over regions with a specific cumulative probability. We use two real-world case studies to demonstrate the value of the PSA-ReD plot. RESULTS: The PSA-ReD method demonstrates proof-of-concept and feasibility. In the real-world case-studies, PSA-ReD provided additional visual information that could not be understood from the traditional scatterplot. High density areas were identified by color-coding and the contour plot allowed for quantification of PSA iterations within areas of the cost-effectiveness plane, diminishing overdrawing and putting infrequent iterations in perspective. Critically, the PSA-ReD plot informs modellers about non-linearities within their model. CONCLUSIONS: The PSA-ReD plot is easy to implement, presents more of the information enclosed in PSA data, and prevents inappropriate interpretation of PSA results. It gives modelers additional insight in model functioning and the distribution of uncertainty around the cost-effectiveness estimate.

A Review of Methodological Considerations for Economic Evaluations of Gene Therapies and Their Application in Literature.

OBJECTIVES: To identify methodological considerations discussed in literature addressing economic evaluations (EEs) of gene therapies (GTs). Additionally, we assessed if these considerations are applied in published GT EEs to increase understanding and explore impact. METHODS: First a peer-reviewed literature review was performed to identify research addressing methodological considerations of GT EEs until August 2019. Identified considerations were grouped in themes using thematic content analysis. A second literature search was conducted in which we identified published evaluations. The EE quality of reporting was assessed using Consolidated Health Economic Evaluation Reporting Standards. RESULTS: The first literature search yielded 13 articles discussing methodological considerations. The second search provided 12 EEs. Considerations identified were payment models, definition of perspectives, addressing uncertainty, data extrapolation, discount rates, novel value elements, and use of indirect and surrogate endpoints. All EEs scored satisfactory to good according to Consolidated Health Economic Evaluation Reporting Standards. Regarding methodological application, we found 1 methodological element (payment models) was applied in 2 base cases. Scenarios explored alternative perspectives, survival assumptions, and extrapolation methods in 10 EEs. CONCLUSIONS: Although EE quality of reporting was considered good, their informativeness for health technology assessment and decision makers seemed limited owing to many uncertainties. We suggest accepted EE methods can broadly be applied to GTs, but few elements may need adjustment. Further research and multi-stakeholder consensus is needed to determine appropriateness and application of individual methodological considerations. For now, we recommend including scenario analyses to explore impact of methodological choices and (clinical) uncertainties. This study contributes to better understanding of perceived appropriate evaluation of GTs and informs best modeling practices.

Assessment of the Regulatory Dialogue Between Pharmaceutical Companies and the European Medicines Agency on the Choice of Noninferiority Margins.

Choosing a noninferiority margin is one of the main challenges when designing a noninferiority trial. The European Medicines Agency (EMA) published a guidance report on the choice of margins in 2005. Nonetheless, in 2008 and 2009 they did not accept 41% (35 of 86) of the noninferiority margins that were proposed by pharmaceutical companies in the context of scientific-advice letters. In this study, we focus on whether the EMA's recommendations were followed by pharmaceutical companies, and on a possible relationship with eventual drug approval. Five of the 35 unaccepted margins were equivalence margins; we considered only the 30 unaccepted noninferiority margins in our analysis. Twelve of these margins were defined based on clinical and statistical considerations (the approach recommended by the EMA) and were rejected due to unacceptable clinical considerations. The other 18 margins were rejected because they were considered too wide. The EMA's recommendations were followed in the cases of 10 of the 15 margins (67%) for which information on follow-through of recommendations was available. The main reason for ignoring the EMA's recommendation in the other 5 cases was that the margins had been accepted by the US Food and Drug Administration. The proportions of approved drugs for which recommendations were and were not followed were similar, yet numbers were too low for formal statistical testing. This study shows that the main concern of regulators with regard to noninferiority trials was the strictness of margins from a clinical perspective. Future studies using more recent data, including data on the US Food and Drug Administration, may help in assessing the impact of guideline recommendations on noninferiority margins used for drug approval and may assist in reaching consensus among regulators about the choice of margins.

Application of Healthcare 'Big Data' in CNS Drug Research: The Example of the Neurological and mental health Global Epidemiology Network (NeuroGEN).

Neurological and psychiatric (mental health) disorders have a large impact on health burden globally. Cognitive disorders (including dementia) and stroke are leading causes of disability. Mental health disorders, including depression, contribute up to one-third of total years lived with disability. The Neurological and mental health Global Epidemiology Network (NeuroGEN) is an international multi-database network that harnesses administrative and electronic medical records from Australia, Asia, Europe and North America. Using these databases NeuroGEN will investigate medication use and health outcomes in neurological and mental health disorders. A key objective of NeuroGEN is to facilitate high-quality observational studies to address evidence-practice gaps where randomized controlled trials do not provide sufficient information on medication benefits and risks that is specific to vulnerable population groups. International multi-database research facilitates comparisons across geographical areas and jurisdictions, increases statistical power to investigate small subpopulations or rare outcomes, permits early post-approval assessment of safety and effectiveness, and increases generalisability of results. Through bringing together international researchers in pharmacoepidemiology, NeuroGEN has the potential to be paradigm-changing for observational research to inform evidence-based prescribing. The first focus of NeuroGEN will be to address evidence-gaps in the treatment of chronic comorbidities in people with dementia.

What does cell therapy manufacturing cost? A framework and methodology to facilitate academic and other small-scale cell therapy manufacturing costings.

BACKGROUND AIMS: Recent technical and clinical advances with cell-based therapies (CBTs) hold great promise in the treatment of patients with rare diseases and those with high unmet medical need. Currently the majority of CBTs are developed and manufactured in specialized academic facilities. Due to small scale, unique characteristics and specific supply chain, CBT manufacturing is considered costly compared to more conventional medicinal products. As a result, biomedical researchers and clinicians are increasingly faced with cost considerations in CBT development. The objective of this research was to develop a costing framework and methodology for academic and other small-scale facilities that manufacture cell-based therapies. METHODS: We conducted an international multi-center costing study in four facilities in Europe using eight CBTs as case studies. This study includes costs from cell or tissue procurement to release of final product for clinical use. First, via interviews with research scientists, clinicians, biomedical scientists, pharmacists and technicians, we designed a high-level costing framework. Next, we developed a more detailed uniform methodology to allocate cost items. Costs were divided into steps (tissue procurement, manufacturing and fill-finish). The steps were each subdivided into cost categories (materials, equipment, personnel and facility), and each category was broken down into facility running (fixed) costs and operational (variable) costs. The methodology was tested via the case studies and validated in developer interviews. Costs are expressed in 2018 euros (€). RESULTS: The framework and methodology were applicable across facilities and proved sensitive to differences in product and facility characteristics. Case study cost estimates ranged between €23 033 and €190 799 Euros per batch, with batch yield varying between 1 and 88 doses. The cost estimations revealed hidden costs to developers and provided insights into cost drivers to help design manufacturing best practices. CONCLUSIONS: This framework and methodology provide step-by-step guidance to estimate manufacturing costs specifically for cell-based therapies manufactured in academic and other small-scale enterprises. The framework and methodology can be used to inform and plan cost-conscious strategies for CBTs.

A novel method for predicting the budget impact of innovative medicines: validation study for oncolytics.

BACKGROUND: High budget impact (BI) estimates of new drugs have led to decision-making challenges potentially resulting in restrictions in patient access. However, current BI predictions are rather inaccurate and short term. We therefore developed a new approach for BI prediction. Here, we describe the validation of our BI prediction approach using oncology drugs as a case study. METHODS: We used Dutch population-level data to estimate BI where BI is defined as list price multiplied by volume. We included drugs in the antineoplastic agents ATC category which the European Medicines Agency (EMA) considered a New Active Substance and received EMA marketing authorization (MA) between 2000 and 2017. A mixed-effects model was used for prediction and included tumor site, orphan, first in class or conditional approval designation as covariates. Data from 2000 to 2012 were the training set. BI was predicted monthly from 0 to 45 months after MA. Cross-validation was performed using a rolling forecasting origin with e^|Ln(observed BI/predicted BI)| as outcome. RESULTS: The training set and validation set included 25 and 44 products, respectively. Mean error, composed of all validation outcomes, was 2.94 (median 1.57). Errors are higher with less available data and at more future predictions. Highest errors occur without any prior data. From 10 months onward, error remains constant. CONCLUSIONS: The validation shows that the method can relatively accurately predict BI. For payers or policymakers, this approach can yield a valuable addition to current BI predictions due to its ease of use, independence of indications and ability to update predictions to the most recent data.

Managing Uncertainties Due to Limited Evidence in Economic Evaluations of Novel Anti-Tuberculosis Regimens: A Systematic Review.

BACKGROUND: Limited evidence for the implementation of new health technologies in low- and middle-income countries (LMICs) may lead to uncertainties in economic evaluations and cause the evaluations to produce inaccurate information for decision making. We performed a systematic review of economic evaluations on implementing new short-course regimens (SCR) for drug-sensitive and drug-resistant tuberculosis (TB), to explore how uncertainties due to the limited evidence in the studies were dealt with and to identify useful information for decision making from these studies. METHODS: We searched in electronic databases PubMed, EMBASE, NHSEED, and CEA registry for economic evaluations addressing the implementation of new anti-TB SCRs in LMICs published until September 2018. We included studies addressing both the cost and outcomes of implementing a new regimen for drug-sensitive and drug-resistant TB with a shorter treatment duration than the currently used regimens. The quality of the included studies was assessed using The Consensus Health Economic Criteria checklist. We extracted information from the included studies on uncertainties and how they were managed. The management of uncertainties was compared with approaches used in early health technology assessments (HTAs), including sensitivity analyses and pragmatic scenario analyses. We extracted information that could be useful for decision making such as cost-effectiveness conclusions, and barriers to implementing the intervention. RESULTS: Four of the 322 studies found in the search met the eligibility criteria. Three studies were model-based studies that investigated the cost effectiveness of a new first-line SCR. One study was an empirical study investigating the cost effectiveness of new regimens for drug-resistant TB. The model-based studies addressed uncertainties due to limited evidence through various sensitivity analyses as in early HTAs. They performed a deterministic sensitivity analysis and found the main drivers of the cost-effectiveness outcomes, that is, the rate of treatment default and treatment delivery costs. Additionally, two of the model-based studies performed a pragmatic scenario analysis and found a potential barrier to implementing the new first-line SCR, that is, a weak health system with a low TB care utilization rate. The empirical study only performed a few scenario analyses with different regimen prices and volumes of TB care utilization. Therefore, the study could only provide information on the main cost drivers. CONCLUSION: Using an approach similar to that used in early HTAs, where uncertainties due to the limited evidence are rigorously explored upfront, the economic evaluations could inform not only the decision to implement the intervention but also how to manage risks and implementation barriers.

Affordability of oncology drugs: accuracy of budget impact estimations.

Background: In many countries, Budget Impact (BI) informs reimbursement decisions. Evidence has shown that decision-makers have restricted access based on high BI estimates but studies show that BI estimates are often inaccurate. Objective: To assess the accuracy of BI estimations used for informing access decisions on oncology drugs in the Netherlands. Study Design: Oncology products for which European Medicines Agency Marketing Authorisation was granted between 1-1-2000 and 1-10-2017 were selected. Observed BI data were provided by FarmInform. BI estimates were extracted from the reimbursement dossiers of the Dutch Healthcare Institute. Products without an estimated BI in the reimbursement dossier were excluded. Accuracy is defined as the ratio observed BI/estimated BI. Setting: General community, the Netherlands. Results: Ten products were included in the base case analysis. Mean accuracy was 0.64 and observed BI deviated by more than 40% and 100% from the estimated BI for 4 and 5 products, respectively. For all products together, €141 million BI was estimated and €82 million BI was observed, a €59 million difference. Conclusions: The findings indicate that BI estimates for oncology drugs in the Netherlands are inaccurate. The role and use of BI in reimbursement decisions for these potentially life-saving drugs should therefore be considered carefully, as well as BI estimation methodology.

Conditional Financing of Drugs in the Netherlands: Past, Present, and Future-Results From Stakeholder Interviews.

BACKGROUND: Conditional financing (CF) of hospital drugs was implemented in the Netherlands as a form of managed entry agreements between 2006 and 2012. CF was a 4-year process comprising 3 stages: initial health technology assessment of the drug (T = 0), conduct of outcomes research studies, and reassessment of the drug (T = 4). OBJECTIVES: To analyze stakeholder experiences in implementing CF in practice. METHODS: Public and private stakeholders were approached for participation in stakeholder interviews through standardized email invitations. An interview guide was developed to guide discussions that covered the following topics: perceived aims of CF, functioning of CF, impact of CF, and conclusions and future perspectives. Extensive summaries were generated for each interview and subsequently used for directed content analysis. RESULTS: Thirty stakeholders were interviewed. Differences emerged among the stakeholders on the perceived aims of CF. Conversely, there was some agreement among stakeholders on the shortcomings in the functioning of CF, the positive impact of CF on the Dutch healthcare setting, and improvement points for CF. CONCLUSIONS: Despite stakeholders' belief that CF either did not meet its aims or only partially did so, there was agreement on the need for new policy to address the same aims of CF in the future. Nevertheless, stakeholders diverged on whether CF should be improved on the basis of learnings identified and reintroduced into practice or replaced with new policy schemes.

Accuracy of budget impact estimations and impact on patient access: a hepatitis C case study.

BACKGROUND: High budget impact (BI) estimates of new drugs limit access to patients due to concerns regarding affordability and displacement effects. The accuracy and methodological quality of BI analyses are often low, potentially mis-informing reimbursement decision making. Using hepatitis C as a case study, we aim to quantify the accuracy of the BI predictions used in Dutch reimbursement decision-making and to characterize the influence of market-dynamics on actual BI. METHODS: We selected hepatitis C direct-acting antivirals (DAAs) that were introduced in the Netherlands between January 2014 and March 2018. Dutch National Health Care Institute (ZIN) BI estimates were derived from the reimbursement dossiers. Actual Dutch BI data were provided by FarmInform. BI prediction accuracy was assessed by comparing the ZIN BI estimates with the actual BI data. RESULTS: Actual BI, from 1 Jan 2014 to 1 March 2018, was €248 million whilst the BI estimates ranged from €388-€510 million. The latter figure represents the estimated BI for the reimbursement scenario that was adopted, implying a €275 million overestimation. Absent incorporation of timing of regulatory decisions and inadequate correction for the introduction of new products were main drivers of BI overestimation, as well as uncertainty regarding the patient population size and the impact of the final reimbursement decision. DISCUSSION: BI in reimbursement dossiers largely overestimated actual BI of hepatitis C DAAs. When BI analysis is performed according to existing guidelines, the resulting more accurate BI estimates may lead to better informed reimbursement decisions.

Disparities in model-based cost-effectiveness analyses of tuberculosis diagnosis: A systematic review.

BACKGROUND: Structural approach disparities were minimally addressed in past systematic reviews of model-based cost-effectiveness analyses addressing Tuberculosis management strategies. This review aimed to identify the structural approach disparities in model-based cost-effectiveness analysis studies addressing Tuberculosis diagnosis and describe potential hazards caused by those disparities. METHODS: A systematic search to identify studies published before October 2015 was performed in five electronic databases. After removal of duplication, studies' titles and abstracts were screened based on predetermined criteria. The full texts of potentially relevant studies were subsequently screened and excluded when they did not address active pulmonary Tuberculosis diagnosis. Quality of the studies was assessed using the "Philips' checklist." Various data regarding general information, cost-effectiveness results, and disease modeling were extracted using standardized data extraction forms. Data pertaining to models' structural approaches were compared and analyzed qualitatively for their applicability in various study settings, as well as their potential influence on main outcomes and cost-effectiveness conclusion. RESULTS: A total of 27 studies were included in the review. Most studies utilized a static model, which could underestimate the cost-effectiveness of the diagnostic tools strategies, due to the omission of indirect diagnosis effects, i.e. transmission reduction. A few structural assumption disparities were found in the dynamic models. Extensive disparities were found in the static models, consisting of varying structural assumptions regarding treatment outcomes, clinical diagnosis and empirical treatment, inpatient discharge decision, and re-diagnosis of false negative patients. CONCLUSION: In cost-effectiveness analysis studies addressing active pulmonary Tuberculosis diagnosis, models showed numerous disparities in their structural approaches. Several structural approaches could be inapplicable in certain settings. Furthermore, they could contribute to under- or overestimation of the cost-effectiveness of the diagnosis tools or strategies. They could thus lead to ambiguities and difficulties when interpreting a study result. A set of recommendations is proposed to manage issues related to these structural disparities.

Using Real-World Data in Health Technology Assessment (HTA) Practice: A Comparative Study of Five HTA Agencies.

BACKGROUND: Reimbursement decisions are conventionally based on evidence from randomised controlled trials (RCTs), which often have high internal validity but low external validity. Real-world data (RWD) may provide complimentary evidence for relative effectiveness assessments (REAs) and cost-effectiveness assessments (CEAs). This study examines whether RWD is incorporated in health technology assessment (HTA) of melanoma drugs by European HTA agencies, as well as differences in RWD use between agencies and across time. METHODS: HTA reports published between 1 January 2011 and 31 December 2016 were retrieved from websites of agencies representing five jurisdictions: England [National Institute for Health and Care Excellence (NICE)], Scotland [Scottish Medicines Consortium (SMC)], France [Haute Autorité de santé (HAS)], Germany [Institute for Quality and Efficacy in Healthcare (IQWiG)] and The Netherlands [Zorginstituut Nederland (ZIN)]. A standardized data extraction form was used to extract information on RWD inclusion for both REAs and CEAs. RESULTS: Overall, 52 reports were retrieved, all of which contained REAs; CEAs were present in 25 of the reports. RWD was included in 28 of the 52 REAs (54%), mainly to estimate melanoma prevalence, and in 22 of the 25 (88%) CEAs, mainly to extrapolate long-term effectiveness and/or identify drug-related costs. Differences emerged between agencies regarding RWD use in REAs; the ZIN and IQWiG cited RWD for evidence on prevalence, whereas the NICE, SMC and HAS additionally cited RWD use for drug effectiveness. No visible trend for RWD use in REAs and CEAs over time was observed. CONCLUSION: In general, RWD inclusion was higher in CEAs than REAs, and was mostly used to estimate melanoma prevalence in REAs or to predict long-term effectiveness in CEAs. Differences emerged between agencies' use of RWD; however, no visible trends for RWD use over time were observed.