Acceptability of Using Real-World Data to Estimate Relative Treatment Effects in Health Technology Assessments: Barriers and Future Steps.

OBJECTIVES: Evidence about the comparative effects of new treatments is typically collected in randomized controlled trials (RCTs). In some instances, RCTs are not possible, or their value is limited by an inability to capture treatment effects over the longer term or in all relevant population subgroups. In these cases, nonrandomized studies (NRS) using real-world data (RWD) are increasingly used to complement trial evidence on treatment effects for health technology assessment (HTA). However, there have been concerns over a lack of acceptability of this evidence by HTA agencies. This article aims to identify the barriers to the acceptance of NRS and steps that may facilitate increases in the acceptability of NRS in the future. METHODS: Opinions of the authorship team based on their experience in real-world evidence research in academic, HTA, and industry settings, supported by a critical assessment of existing studies. RESULTS: Barriers were identified that are applicable to key stakeholder groups, including HTA agencies (eg, the lack of comprehensive methodological guidelines for using RWD), evidence generators (eg, avoidable deviations from best practices), and external stakeholders (eg, data controllers providing timely access to high-quality RWD). Future steps that may facilitate future acceptability of NRS include improvements in the quality, integration, and accessibility of RWD, wider use of demonstration projects to highlight the value and applicability of nonrandomized designs, living, and more detailed HTA guidelines, and improvements in HTA infrastructure relating to RWD. CONCLUSION: NRS can represent a crucial source of evidence on treatment effects for use in HTA when RCT evidence is limited.

Consolidated Health Economic Evaluation Reporting Standards for Interventions That Use Artificial Intelligence (CHEERS-AI).

OBJECTIVES: Economic evaluations (EEs) are commonly used by decision makers to understand the value of health interventions. The Consolidated Health Economic Evaluation Reporting Standards (CHEERS 2022) provide reporting guidelines for EEs. Healthcare systems will increasingly see new interventions that use artificial intelligence (AI) to perform their function. We developed Consolidated Health Economic Evaluation Reporting Standards for Interventions that use AI (CHEERS-AI) to ensure EEs of AI-based health interventions are reported in a transparent and reproducible manner. METHODS: Potential CHEERS-AI reporting items were informed by 2 published systematic literature reviews of EEs and a contemporary update. A Delphi study was conducted using 3 survey rounds to elicit multidisciplinary expert views on 26 potential items, through a 9-point Likert rating scale and qualitative comments. An online consensus meeting was held to finalize outstanding reporting items. A digital health patient group reviewed the final checklist from a patient perspective. RESULTS: A total of 58 participants responded to survey round 1, 42, and 31 of whom responded to rounds 2 and 3, respectively. Nine participants joined the consensus meeting. Ultimately, 38 reporting items were included in CHEERS-AI. They comprised the 28 original CHEERS 2022 items, plus 10 new AI-specific reporting items. Additionally, 8 of the original CHEERS 2022 items were elaborated on to ensure AI-specific nuance is reported. CONCLUSIONS: CHEERS-AI should be used when reporting an EE of an intervention that uses AI to perform its function. CHEERS-AI will help decision makers and reviewers to understand important AI-specific details of an intervention, and any implications for the EE methods used and cost-effectiveness conclusions.

Measuring quality of life of adults with intellectual disabilities: Psychometric evaluation of the personal outcomes scale in the United Kingdom.

BACKGROUND: The Personal Outcomes Scale (POS) is a scale developed to measure quality of life of adults (18+) with intellectual disability. Previous studies have reported good fit for Spanish and Portuguese language versions of POS. AIMS: This study aimed to evaluate the factor structure of the English language version of POS when used to measure the quality of life of adults (18+) with intellectual disability in the UK. MATERIALS AND METHODS: Analysis was conducted on POS data from 310 adults with an intellectual disability. First and second order factor models and multi-level models were used to assess fit. RESULTS: There was poor fit to the data for all tested models. We estimated that 23% of variance in POS scores was accounted for by interviewer cluster. DISCUSSION: This was the first UK-based evaluation of POS and our data did not confirm the factor structure of the POS measure. The identification of systematic variability within the dataset indicates that inter-rater reliability is a potential limitation of the POS tool. CONCLUSION: Further research is needed to investigate inter-rater reliability of POS interviewers and to explore factor structure.

Addressing missing data in the estimation of time-varying treatments in comparative effectiveness research.

Comparative effectiveness research is often concerned with evaluating treatment strategies sustained over time, that is, time-varying treatments. Inverse probability weighting (IPW) is often used to address the time-varying confounding by re-weighting the sample according to the probability of treatment receipt at each time point. IPW can also be used to address any missing data by re-weighting individuals according to the probability of observing the data. The combination of these two distinct sets of weights may lead to inefficient estimates of treatment effects due to potentially highly variable total weights. Alternatively, multiple imputation (MI) can be used to address the missing data by replacing each missing observation with a set of plausible values drawn from the posterior predictive distribution of the missing data given the observed data. Recent studies have compared IPW and MI for addressing the missing data in the evaluation of time-varying treatments, but they focused on missing confounders and monotone missing data patterns. This article assesses the relative advantages of MI and IPW to address missing data in both outcomes and confounders measured over time, and across monotone and non-monotone missing data settings. Through a comprehensive simulation study, we find that MI consistently provided low bias and more precise estimates compared to IPW across a wide range of scenarios. We illustrate the implications of method choice in an evaluation of biologic drugs for patients with severe rheumatoid arthritis, using the US National Databank for Rheumatic Diseases, in which 25% of participants had missing health outcomes or time-varying confounders.

Economic Evaluations of Digital Health Interventions for Children and Adolescents: Systematic Review.

BACKGROUND: Digital health interventions (DHIs) are defined as digital technologies such as digital health applications and information and communications technology systems (including SMS text messages) implemented to meet health objectives. DHIs implemented using various technologies, ranging from electronic medical records to videoconferencing systems and mobile apps, have experienced substantial growth and uptake in recent years. Although the clinical effectiveness of DHIs for children and adolescents has been relatively well studied, much less is known about the cost-effectiveness of these interventions. OBJECTIVE: This study aimed to systematically review economic evaluations of DHIs for pediatric and adolescent populations. This study also reviewed methodological issues specific to economic evaluations of DHIs to inform future research priorities. METHODS: We conducted a database search in PubMed from 2011 to 2021 using the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist. In total, 2 authors independently screened the titles and abstracts of the search results to identify studies eligible for full-text review. We generated a data abstraction procedure based on recommendations from the Panel on Cost-Effectiveness in Health and Medicine. The types of economic evaluations included in this review were cost-effectiveness analyses (costs per clinical effect), cost-benefit analyses (costs and effects expressed in monetary terms as net benefit), and cost-utility analyses (cost per quality-adjusted life year or disability-adjusted life year). Narrative analysis was used to synthesize the quantitative data because of heterogeneity across the studies. We extracted methodological issues related to study design, analysis framework, cost and outcome measurement, and methodological assumptions regarding the health economic evaluation. RESULTS: We included 22 articles assessing the cost-effectiveness of DHI interventions for children and adolescents. Most articles (14/22, 64%) evaluated interventions delivered through web-based portals or SMS text messaging, most frequently within the health care specialties of mental health and maternal, newborn, and child health. In 82% (18/22) of the studies, DHIs were found to be cost-effective or cost saving compared with the nondigital standard of care. The key drivers of cost-effectiveness included population coverage, cost components, intervention effect size and scale-up, and study perspective. The most frequently identified methodological challenges were related to study design (17/22, 77%), costing (11/22, 50%), and economic modeling (9/22, 41%). CONCLUSIONS: This is the first systematic review of economic evaluations of DHIs targeting pediatric and adolescent populations. We found that most DHIs (18/22, 82%) for children and adolescents were cost-effective or cost saving compared with the nondigital standard of care. In addition, this review identified key methodological challenges directly related to the conduct of economic evaluations of DHIs and highlighted areas where further methodological research is required to address these challenges. These included the need for measurement of user involvement and indirect effects of DHIs and the development of children-specific, generic quality-of-life outcomes.

ATOM Calibration Framework: Interaction and Visualization Functionalities.

Robotic systems are evolving to include a large number of sensors and diverse sensor modalities. In order to operate a system with multiple sensors, the geometric transformations between those sensors must be accurately estimated. The process by which these transformations are estimated is known as sensor calibration. Behind every sensor calibration approach is a formulation and a framework. The formulation is the method by which the transformations are estimated. The framework is the set of operations required to carry out the calibration procedure. This paper proposes a novel calibration framework that gives more flexibility, control and information to the user, enhancing the user interface and the user experience of calibrating a robotic system. The framework consists of several visualization and interaction functionalities useful for a calibration procedure, such as the estimation of the initial pose of the sensors, the data collection and labeling, the data review and correction and the visualization of the estimation of the extrinsic and intrinsic parameters. This framework is supported by the Atomic Transformations Optimization Method formulation, referred to as ATOM. Results show that this framework is applicable to various robotic systems with different configurations, number of sensors and sensor modalities. In addition to this, a survey comparing the frameworks of different calibration approaches shows that ATOM provides a very good user experience.

Weight Change and the Onset of Cardiovascular Diseases: Emulating Trials Using Electronic Health Records.

BACKGROUND: Cross-sectional measures of body mass index (BMI) are associated with cardiovascular disease (CVD) incidence, but less is known about whether weight change affects the risk of CVD. METHODS: We estimated the effect of 2-y weight change interventions on 7-y risk of CVD (CVD death, myocardial infarction, stroke, hospitalization from coronary heart disease, and heart failure) by emulating hypothetical interventions using electronic health records. We identified 138,567 individuals with 45-69 years of age without chronic disease in England from 1998 to 2016. We performed pooled logistic regression, using inverse-probability weighting to adjust for baseline and time-varying confounders. We categorized each individual into a weight loss, maintenance, or gain group. RESULTS: Among those of normal weight, both weight loss [risk difference (RD) vs. weight maintenance = 1.5% (0.3% to 3.0%)] and gain [RD = 1.3% (0.5% to 2.2%)] were associated with increased risk for CVD compared with weight maintenance. Among overweight individuals, we observed moderately higher risk of CVD in both the weight loss [RD = 0.7% (-0.2% to 1.7%)] and the weight gain group [RD = 0.7% (-0.1% to 1.7%)], compared with maintenance. In the obese, those losing weight showed lower risk of coronary heart disease [RD = -1.4% (-2.4% to -0.6%)] but not of stroke. When we assumed that chronic disease occurred 1-3 years before the recorded date, estimates for weight loss and gain were attenuated among overweight individuals; estimates for loss were lower among obese individuals. CONCLUSION: Among individuals with obesity, the weight-loss group had a lower risk of coronary heart disease but not of stroke. Weight gain was associated with increased risk of CVD across BMI groups. See video abstract at, http://links.lww.com/EDE/B838.

An International Comparison of EQ-5D-5L and EQ-5D-3L for Use in Cost-Effectiveness Analysis.

OBJECTIVES: To estimate the impact of using EQ5D-5L (5L) compared with EQ5D-3L (3L) in cost-effectiveness analyses in 6 countries with 3L and 5L values: Germany, Japan, Korea, The Netherlands, China, and Spain. METHODS: Eight cost-effectiveness analyses based on clinical studies with 3L provided 11 pairwise comparisons. We estimated cost-effectiveness by applying the appropriate country values for 3L to observed responses. We re-estimated cost-effectiveness for each country by predicting the 5L tariff score for each respondent, for each country, using a previously published mapping method. We compared results in terms of impact on estimated incremental quality-adjusted life-year (QALY) gain and cost-effectiveness ratios. RESULTS: For most countries the impact of moving from 3L to 5L is to lower the incremental QALY gain in the majority of comparisons. The only exception to this was Japan, where 4 out of 11 cases (37%) saw lower QALYs gained when using 5L. The mean and median reductions in health gain, in those case studies where 5L does lead to lower health gain, are largest in The Netherlands (84% mean reduction, 41% median reduction), Germany (68% and 27%), and Spain (30% and 31%). For most countries, those studies where 5L leads to lower health gain see larger reductions than the gains in studies showing the opposite tendency. CONCLUSIONS: Overall, 3L and 5L are not interchangeable in these countries. Differences between results are large, but the direction of change can be unpredictable. These findings should prompt further investigation into the reasons for differences.

Flexible Bayesian longitudinal models for cost-effectiveness analyses with informative missing data.

Cost-effectiveness analyses (CEA) are recommended to include sensitivity analyses which make a range of contextually plausible assumptions about missing data. However, with longitudinal data on, for example, patients' health-related quality of life (HRQoL), the missingness patterns can be complicated because data are often missing both at specific timepoints (interim missingness) and following loss to follow-up. Methods to handle these complex missing data patterns have not been developed for CEA, and must recognize that data may be missing not at random, while accommodating both the correlation between costs and health outcomes and the non-normal distribution of these endpoints. We develop flexible Bayesian longitudinal models that allow the impact of interim missingness and loss to follow-up to be disentangled. This modeling framework enables studies to undertake sensitivity analyses according to various contextually plausible missing data mechanisms, jointly model costs and outcomes using appropriate distributions, and recognize the correlation among these endpoints over time. We exemplify these models in the REFLUX study in which 52% of participants had HRQoL data missing for at least one timepoint over the 5-year follow-up period. We provide guidance for sensitivity analyses and accompanying code to help future studies handle these complex forms of missing data.

COVID-19 Lockdowns May Reduce Resistance Genes Diversity in the Human Microbiome and the Need for Antibiotics.

Recently, much attention has been paid to the COVID-19 pandemic. Yet bacterial resistance to antibiotics remains a serious and unresolved public health problem that kills hundreds of thousands of people annually, being an insidious and silent pandemic. To contain the spreading of the SARS-CoV-2 virus, populations confined and tightened hygiene measures. We performed this study with computer simulations and by using mobility data of mobile phones from Google in the region of Lisbon, Portugal, comprising 3.7 million people during two different lockdown periods, scenarios of 40 and 60% mobility reduction. In the simulations, we assumed that the network of physical contact between people is that of a small world and computed the antibiotic resistance in human microbiomes after 180 days in the simulation. Our simulations show that reducing human contacts drives a reduction in the diversity of antibiotic resistance genes in human microbiomes. Kruskal-Wallis and Dunn's pairwise tests show very strong evidence (p < 0.000, adjusted using the Bonferroni correction) of a difference between the four confinement regimes. The proportion of variability in the ranked dependent variable accounted for by the confinement variable was η2 = 0.148, indicating a large effect of confinement on the diversity of antibiotic resistance. We have shown that confinement and hygienic measures, in addition to reducing the spread of pathogenic bacteria in a human network, also reduce resistance and the need to use antibiotics.

Estimating treatment effects under untestable assumptions with nonignorable missing data.

Nonignorable missing data poses key challenges for estimating treatment effects because the substantive model may not be identifiable without imposing further assumptions. For example, the Heckman selection model has been widely used for handling nonignorable missing data but requires the study to make correct assumptions, both about the joint distribution of the missingness and outcome and that there is a valid exclusion restriction. Recent studies have revisited how alternative selection model approaches, for example estimated by multiple imputation (MI) and maximum likelihood, relate to Heckman-type approaches in addressing the first hurdle. However, the extent to which these different selection models rely on the exclusion restriction assumption with nonignorable missing data is unclear. Motivated by an interventional study (REFLUX) with nonignorable missing outcome data in half of the sample, this article critically examines the role of the exclusion restriction in Heckman, MI, and full-likelihood selection models when addressing nonignorability. We explore the implications of the different methodological choices concerning the exclusion restriction for relative bias and root-mean-squared error in estimating treatment effects. We find that the relative performance of the methods differs in practically important ways according to the relevance and strength of the exclusion restriction. The full-likelihood approach is less sensitive to alternative assumptions about the exclusion restriction than Heckman-type models and appears an appropriate method for handling nonignorable missing data. We illustrate the implications of method choice for inference in the REFLUX study, which evaluates the effect of laparoscopic surgery on long-term quality of life for patients with gastro-oseophageal reflux disease.

Reference-based multiple imputation for missing data sensitivity analyses in trial-based cost-effectiveness analysis.

Missing data are a common issue in cost-effectiveness analysis (CEA) alongside randomised trials and are often addressed assuming the data are 'missing at random'. However, this assumption is often questionable, and sensitivity analyses are required to assess the implications of departures from missing at random. Reference-based multiple imputation provides an attractive approach for conducting such sensitivity analyses, because missing data assumptions are framed in an intuitive way by making reference to other trial arms. For example, a plausible not at random mechanism in a placebo-controlled trial would be to assume that participants in the experimental arm who dropped out stop taking their treatment and have similar outcomes to those in the placebo arm. Drawing on the increasing use of this approach in other areas, this paper aims to extend and illustrate the reference-based multiple imputation approach in CEA. It introduces the principles of reference-based imputation and proposes an extension to the CEA context. The method is illustrated in the CEA of the CoBalT trial evaluating cognitive behavioural therapy for treatment-resistant depression. Stata code is provided. We find that reference-based multiple imputation provides a relevant and accessible framework for assessing the robustness of CEA conclusions to different missing data assumptions.

Copula selection models for non-Gaussian outcomes that are missing not at random.

Missing not at random (MNAR) data pose key challenges for statistical inference because the substantive model of interest is typically not identifiable without imposing further (eg, distributional) assumptions. Selection models have been routinely used for handling MNAR by jointly modeling the outcome and selection variables and typically assuming that these follow a bivariate normal distribution. Recent studies have advocated parametric selection approaches, for example, estimated by multiple imputation and maximum likelihood, that are more robust to departures from the normality assumption compared with those assuming that nonresponse and outcome are jointly normally distributed. However, the proposed methods have been mostly restricted to a specific joint distribution (eg, bivariate t-distribution). This paper discusses a flexible copula-based selection approach (which accommodates a wide range of non-Gaussian outcome distributions and offers great flexibility in the choice of functional form specifications for both the outcome and selection equations) and proposes a flexible imputation procedure that generates plausible imputed values from the copula selection model. A simulation study characterizes the relative performance of the copula model compared with the most commonly used selection models for estimating average treatment effects with MNAR data. We illustrate the methods in the REFLUX study, which evaluates the effect of laparoscopic surgery on long-term quality of life in patients with reflux disease. We provide software code for implementing the proposed copula framework using the R package GJRM.

Cost-effectiveness analysis of English memory assessment services 2 years after first consultation for patients with dementia.

OBJECTIVES: This paper aims to compare changes over 2 years in patients' health-related quality of life (HRQL) with the health and social care costs of diagnosis and treatment of people newly referred to memory assessment services (MAS). METHODS: We analysed observational data from 1318 patients referred to 69 MAS who completed resource use and HRQL questionnaires at baseline 3, 6, 12, and 24 months. We reported mean differences in HRQL (disease-specific DEMQOL and generic EQ-5D-3 L), quality-adjusted life years (QALYs), costs and cost-effectiveness between baseline, and 2-year follow-up. RESULTS: Two years after referral to MAS, patients reported a higher DEMQOL score (mean gain 4.47, 95% confidence interval, 3.08-5.90) and EQ-5D-3 L (0.014, -0.011 to 0.039). Mean total costs and QALYs over 24 months was £2411 (£1721-£2873) and 0.027 (0.003-0.051), respectively. Assuming that patients' HRQL would not have altered over the 2 years had they not attended MAS, these outcomes suggest an incremental cost-effectiveness ratio of £89 546 (£38 123-£145 864) based on changes in EQ-5D-3 L. If we assumed that patients' HRQL would have declined by about 10% over this period had they not attended MAS, the cost-effectiveness ratio would be £25 056. The largest MAS (N = 32; 46%) with over 50 new patients a month were more likely to be cost-effective than smaller ones (P < 0.01). CONCLUSIONS: MAS are effective and can be cost-effective for diagnosing and treating people with suspected dementia. Large variations in costs between clinics suggest that many MAS could improve their cost-effectiveness.

Using Resistin, glucose, age and BMI to predict the presence of breast cancer.

BACKGROUND: The goal of this exploratory study was to develop and assess a prediction model which can potentially be used as a biomarker of breast cancer, based on anthropometric data and parameters which can be gathered in routine blood analysis. METHODS: For each of the 166 participants several clinical features were observed or measured, including age, BMI, Glucose, Insulin, HOMA, Leptin, Adiponectin, Resistin and MCP-1. Machine learning algorithms (logistic regression, random forests, support vector machines) were implemented taking in as predictors different numbers of variables. The resulting models were assessed with a Monte Carlo Cross-Validation approach to determine 95% confidence intervals for the sensitivity, specificity and AUC of the models. RESULTS: Support vector machines models using Glucose, Resistin, Age and BMI as predictors allowed predicting the presence of breast cancer in women with sensitivity ranging between 82 and 88% and specificity ranging between 85 and 90%. The 95% confidence interval for the AUC was [0.87, 0.91]. CONCLUSIONS: These findings provide promising evidence that models combining age, BMI and metabolic parameters may be a powerful tool for a cheap and effective biomarker of breast cancer.

EQ-5D-5L versus EQ-5D-3L: The Impact on Cost Effectiveness in the United Kingdom.

OBJECTIVES: To model the relationship between the three-level (3L) and the five-level (5L) EuroQol five-dimensional questionnaire and examine how differences have an impact on cost effectiveness in case studies. METHODS: We used two data sets that included the 3L and 5L versions from the same respondents. The EuroQol Group data set (n = 3551) included patients with different diseases and a healthy cohort. The National Data Bank data set included patients with rheumatoid disease (n = 5205). We estimated a system of ordinal regressions in each data set using copula models to link responses of the 3L instrument to those of the 5L instrument and its UK tariff, and vice versa. Results were applied to nine cost-effectiveness studies. RESULTS: Best-fitting models differed between the EuroQol Group and the National Data Bank data sets in terms of the explanatory variables, copulas, and coefficients. In both cases, the coefficients of the covariates and latent factors between the 3L and the 5L instruments were significantly different, indicating that moving between instruments is not simply a uniform re-alignment of the response levels for most dimensions. In the case studies, moving from the 3L to the 5L caused a decrease of up to 87% in incremental quality-adjusted life-years gained from effective technologies in almost all cases. Incremental cost-effectiveness ratios increased, often substantially. Conversely, one technology with a significant mortality gain saw increased incremental quality-adjusted life-years. CONCLUSIONS: The 5L shifts mean utility scores up the utility scale toward full health and compresses them into a smaller range, compared with the 3L. Improvements in quality of life are valued less using the 5L than using the 3L. The 3L and the 5L can produce substantially different estimates of cost effectiveness. There is no simple proportional adjustment that can be made to reconcile these differences.

The cost of diagnosis and early support in patients with cognitive decline.

OBJECTIVE: Recent research indicates considerable heterogeneity in the provision of memory assessment services (MAS). However, little is known on the extent of variation in the costs of the services MAS provide. We investigated the costs of supporting patients with suspected dementia, including assessment and support over the following 6 months. METHODS: Clinic costs were estimated on the basis of an organisational survey reporting staff roll, grade and activities. Costs of primary health and social care were estimated from questionnaire data reported by carers of patients at baseline, 3 and 6 months after referral. RESULTS: Mean monthly staff costs at MAS were £73 000. Imaging at assessment costs an additional £3500 per month. Monthly clinic cost per new patient assessed varied from £320 to £5400 across clinics. Additional primary health and social care costs of £130-220 a month between baseline and 6 months were reported by carers. Costs of pharmacological and non-pharmacological treatments reported by carers were small. Informal care costs dwarfed health and social care costs when valued at a modest unit cost. The overall mean cost of supporting a patient for 6 months varied from £1600 to £2500 dependent on assumptions regarding the proportion of MAS intervention and review costs accrued at 6 months. CONCLUSIONS: There is considerable variation in the intensity and associated costs of services provided by MAS. Further research should ascertain to what extent such variation is associated with differences in patient outcomes. Copyright © 2016 John Wiley & Sons, Ltd.