Acceptability of Using Real-World Data to Estimate Relative Treatment Effects in Health Technology Assessments: Barriers and Future Steps.

OBJECTIVES: Evidence about the comparative effects of new treatments is typically collected in randomized controlled trials (RCTs). In some instances, RCTs are not possible, or their value is limited by an inability to capture treatment effects over the longer term or in all relevant population subgroups. In these cases, nonrandomized studies (NRS) using real-world data (RWD) are increasingly used to complement trial evidence on treatment effects for health technology assessment (HTA). However, there have been concerns over a lack of acceptability of this evidence by HTA agencies. This article aims to identify the barriers to the acceptance of NRS and steps that may facilitate increases in the acceptability of NRS in the future. METHODS: Opinions of the authorship team based on their experience in real-world evidence research in academic, HTA, and industry settings, supported by a critical assessment of existing studies. RESULTS: Barriers were identified that are applicable to key stakeholder groups, including HTA agencies (eg, the lack of comprehensive methodological guidelines for using RWD), evidence generators (eg, avoidable deviations from best practices), and external stakeholders (eg, data controllers providing timely access to high-quality RWD). Future steps that may facilitate future acceptability of NRS include improvements in the quality, integration, and accessibility of RWD, wider use of demonstration projects to highlight the value and applicability of nonrandomized designs, living, and more detailed HTA guidelines, and improvements in HTA infrastructure relating to RWD. CONCLUSION: NRS can represent a crucial source of evidence on treatment effects for use in HTA when RCT evidence is limited.

Transporting Comparative Effectiveness Evidence Between Countries: Considerations for Health Technology Assessments.

Internal validity is often the primary concern for health technology assessment agencies when assessing comparative effectiveness evidence. However, the increasing use of real-world data from countries other than a health technology assessment agency's target population in effectiveness research has increased concerns over the external validity, or "transportability", of this evidence, and has led to a preference for local data. Methods have been developed to enable a lack of transportability to be addressed, for example by accounting for cross-country differences in disease characteristics, but their consideration in health technology assessments is limited. This may be because of limited knowledge of the methods and/or uncertainties in how best to utilise them within existing health technology assessment frameworks. This article aims to provide an introduction to transportability, including a summary of its assumptions and the methods available for identifying and adjusting for a lack of transportability, before discussing important considerations relating to their use in health technology assessment settings, including guidance on the identification of effect modifiers, guidance on the choice of target population, estimand, study sample and methods, and how evaluations of transportability can be integrated into health technology assessment submission and decision processes.

Unmeasured confounding in nonrandomized studies: quantitative bias analysis in health technology assessment.

Evidence generated from nonrandomized studies (NRS) is increasingly submitted to health technology assessment (HTA) agencies. Unmeasured confounding is a primary concern with this type of evidence, as it may result in biased treatment effect estimates, which has led to much criticism of NRS by HTA agencies. Quantitative bias analyses are a group of methods that have been developed in the epidemiological literature to quantify the impact of unmeasured confounding and adjust effect estimates from NRS. Key considerations for application in HTA proposed in this article reflect the need to balance methodological complexity with ease of application and interpretation, and the need to ensure the methods fit within the existing frameworks used to assess nonrandomized evidence by HTA bodies.

Application of quantitative bias analysis for unmeasured confounding in cost-effectiveness modelling.

Due to uncertainty regarding the potential impact of unmeasured confounding, health technology assessment (HTA) agencies often disregard evidence from nonrandomized studies when considering new technologies. Quantitative bias analysis (QBA) methods provide a means to quantify this uncertainty but have not been widely used in the HTA setting, particularly in the context of cost-effectiveness modelling (CEM). This study demonstrated the application of an aggregate and patient-level QBA approach to quantify and adjust for unmeasured confounding in a simulated nonrandomized comparison of survival outcomes. Application of the QBA output within a CEM through deterministic and probabilistic sensitivity analyses and under different scenarios of knowledge of an unmeasured confounder demonstrates the potential value of QBA in HTA.

The use of UK primary care databases in health technology assessments carried out by the National Institute for health and care excellence (NICE).

BACKGROUND: Real world evidence (RWE) is becoming more frequently used in technology appraisals (TAs). This study sought to explore the use and acceptance of evidence from primary care databases, a key source of RWE in the UK, in National Institute for Health and Care Excellence (NICE) technology assessments and to provide recommendations regarding their use in future submissions. METHODS: A keyword search was conducted relating to the main primary care databases in the UK on the NICE website. All NICE TAs identified through this search were screened, assessed for duplication and information on the data source and the way the data was used in the submission were extracted. Comments by the evidence review group (ERG) and the appraisal committee were also extracted and reviewed. All data extraction was performed by two independent reviewers and all decisions were reached by consensus with an additional third reviewer. RESULTS: A total of 52 NICE TAs were identified, 47 used the General Practice Research Database /Clinical Practice Research Datalink (GPRD/CPRD) database, 10 used The Health Improvement Network (THIN) database and 3 used the QResearch databases. Data from primary care databases were used to support arguments regarding clinical need and current treatment in 33 NICE TAs while 36 were used to inform input parameters for economic models. The databases were sometimes used for more than one purpose. The data from the three data sources were generally well received by the ERGs/committees. Criticisms of the data typically occurred where the results had been repurposed from a published study or had not been applied appropriately. CONCLUSIONS: The potential of UK primary care databases in NICE submissions is increasingly being realised, particularly in informing the parameters of economic models. Purpose conducted studies are less likely to receive criticism from ERGs/committees, particularly when providing clinical input into cost effectiveness models.

Costs of gastrointestinal bleeding events in atrial fibrillation: a UK Clinical Practice Research Datalink study.

Aim: To estimate the healthcare costs attributable to gastrointestinal (GI) bleeds in nonvalvular atrial fibrillation (NVAF) patients. Material & methods: A difference-in-differences approach was used in which NVAF patients suffering a (GI) bleed were propensity score matched to those not suffering a GI bleed, and the difference in healthcare costs in the year prior to the GI bleed and the subsequent 3 years was compared between the two groups. Results: The mean cost attributable to GI bleeds was £3989 (p < 0.0001) in the year of the bleed and £1816 (p = 0.001) in the subsequent year. Attributable costs arose primarily from inpatient visits. Conclusion: GI bleeds among NVAF patients are associated with significant healthcare costs up to 2 years following the bleed.

Risks and benefits of psychotropic medication in pregnancy: cohort studies based on UK electronic primary care health records.

BACKGROUND: Although many women treated with psychotropic medication become pregnant, no psychotropic medication has been licensed for use in pregnancy. This leaves women and their health-care professionals in a treatment dilemma, as they need to balance the health of the woman with that of the unborn child. The aim of this project was to investigate the risks and benefits of psychotropic medication in women treated for psychosis who become pregnant. OBJECTIVE(S): (1) To provide a descriptive account of psychotropic medication prescribed before pregnancy, during pregnancy and up to 15 months after delivery in UK primary care from 1995 to 2012; (2) to identify risk factors predictive of discontinuation and restarting of lithium (multiple manufacturers), anticonvulsant mood stabilisers and antipsychotic medication; (3) to examine the extent to which pregnancy is a determinant for discontinuation of psychotropic medication; (4) to examine prevalence of records suggestive of adverse mental health, deterioration or relapse 18 months before and during pregnancy, and up to 15 months after delivery; and (5) to estimate absolute and relative risks of adverse maternal and child outcomes of psychotropic treatment in pregnancy. DESIGN: Retrospective cohort studies. SETTING: Primary care. PARTICIPANTS: Women treated for psychosis who became pregnant, and their children. INTERVENTIONS: Treatment with antipsychotics, lithium or anticonvulsant mood stabilisers. MAIN OUTCOME MEASURES: Discontinuation and restarting of treatment; worsening of mental health; acute pre-eclampsia/gestational hypertension; gestational diabetes; caesarean section; perinatal death; major congenital malformations; poor birth outcome (low birthweight, preterm birth, small for gestational age, low Apgar score); transient poor birth outcomes (tremor, agitation, breathing and muscle tone problems); and neurodevelopmental and behavioural disorders. DATA SOURCES: Clinical Practice Research Datalink database and The Health Improvement Network primary care database. RESULTS: Prescribing of psychotropic medication was relatively constant before pregnancy, decreased sharply in early pregnancy and peaked after delivery. Antipsychotic and anticonvulsant treatment increased over the study period. The recording of markers of worsening mental health peaked after delivery. Pregnancy was a strong determinant for discontinuation of psychotropic medication. However, between 40% and 76% of women who discontinued psychotropic medication before or in early pregnancy restarted treatment by 15 months after delivery. The risk of major congenital malformations, and neurodevelopmental and behavioural outcomes in valproate (multiple manufacturers) users was twice that in users of other anticonvulsants. The risks of adverse maternal and child outcomes in women who continued antipsychotic use in pregnancy were not greater than in those who discontinued treatment before pregnancy. LIMITATIONS: A few women would have received parts of their care outside primary care, which may not be captured in this analysis. Likewise, the analyses were based on prescribing data, which may differ from usage. CONCLUSIONS: Psychotropic medication is prescribed before, during and after pregnancy. Many women discontinue treatment before or during early pregnancy and then restart again in late pregnancy or after delivery. Our results support previous associations between valproate and adverse child outcomes but we found no evidence of such an association for antipsychotics. FUTURE WORK: Future research should focus on (1) curtailing the use of sodium valproate; (2) estimating the benefits of psychotropic drug use in pregnancy; and (3) investigating the risks associated with lifestyle choices that are more prevalent among women using psychotropic drugs. FUNDING DETAILS: The National Institute for Health Research Health Technology Assessment programme.