Model-based standardization using multiple imputation.

BACKGROUND: When studying the association between treatment and a clinical outcome, a parametric multivariable model of the conditional outcome expectation is often used to adjust for covariates. The treatment coefficient of the outcome model targets a conditional treatment effect. Model-based standardization is typically applied to average the model predictions over the target covariate distribution, and generate a covariate-adjusted estimate of the marginal treatment effect. METHODS: The standard approach to model-based standardization involves maximum-likelihood estimation and use of the non-parametric bootstrap. We introduce a novel, general-purpose, model-based standardization method based on multiple imputation that is easily applicable when the outcome model is a generalized linear model. We term our proposed approach multiple imputation marginalization (MIM). MIM consists of two main stages: the generation of synthetic datasets and their analysis. MIM accommodates a Bayesian statistical framework, which naturally allows for the principled propagation of uncertainty, integrates the analysis into a probabilistic framework, and allows for the incorporation of prior evidence. RESULTS: We conduct a simulation study to benchmark the finite-sample performance of MIM in conjunction with a parametric outcome model. The simulations provide proof-of-principle in scenarios with binary outcomes, continuous-valued covariates, a logistic outcome model and the marginal log odds ratio as the target effect measure. When parametric modeling assumptions hold, MIM yields unbiased estimation in the target covariate distribution, valid coverage rates, and similar precision and efficiency than the standard approach to model-based standardization. CONCLUSION: We demonstrate that multiple imputation can be used to marginalize over a target covariate distribution, providing appropriate inference with a correctly specified parametric outcome model and offering statistical performance comparable to that of the standard approach to model-based standardization.

A Scoping Review of Item-Level Missing Data in Within-Trial Cost-Effectiveness Analysis.

OBJECTIVES: Cost-effectiveness analysis (CEA) alongside randomized controlled trials often relies on self-reported multi-item questionnaires that are invariably prone to missing item-level data. The purpose of this study is to review how missing multi-item questionnaire data are handled in trial-based CEAs. METHODS: We searched the National Institute for Health Research journals to identify within-trial CEAs published between January 2016 and April 2021 using multi-item instruments to collect costs and quality of life (QOL) data. Information on missing data handling and methods, with a focus on the level and type of imputation, was extracted. RESULTS: A total of 87 trial-based CEAs were included in the review. Complete case analysis or available case analysis and multiple imputation (MI) were the most popular methods, selected by similar numbers of studies, to handle missing costs and QOL in base-case analysis. Nevertheless, complete case analysis or available case analysis dominated sensitivity analysis. Once imputation was chosen, missing costs were widely imputed at item-level via MI, whereas missing QOL was usually imputed at the more aggregated time point level during the follow-up via MI. CONCLUSIONS: Missing costs and QOL tend to be imputed at different levels of missingness in current CEAs alongside randomized controlled trials. Given the limited information provided by included studies, the impact of applying different imputation methods at different levels of aggregation on CEA decision making remains unclear.

Using epidemiological evidence to forecast population need for early treatment programmes in mental health: a generalisable Bayesian prediction methodology applied to and validated for first-episode psychosis in England.

Background: Mental health policy makers require evidence-based information to optimise effective care provision based on local need, but tools are unavailable. Aims: To develop and validate a population-level prediction model for need for early intervention in psychosis (EIP) care for first-episode psychosis (FEP) in England up to 2025, based on epidemiological evidence and demographic projections. Method: We used Bayesian Poisson regression to model small-area-level variation in FEP incidence for people aged 16-64 years. We compared six candidate models, validated against observed National Health Service FEP data in 2017. Our best-fitting model predicted annual incidence case-loads for EIP services in England up to 2025, for probable FEP, treatment in EIP services, initial assessment by EIP services and referral to EIP services for 'suspected psychosis'. Forecasts were stratified by gender, age and ethnicity, at national and Clinical Commissioning Group levels. Results: A model with age, gender, ethnicity, small-area-level deprivation, social fragmentation and regional cannabis use provided best fit to observed new FEP cases at national and Clinical Commissioning Group levels in 2017 (predicted 8112, 95% CI 7623-8597; observed 8038, difference of 74 [0.92%]). By 2025, the model forecasted 11 067 new treated cases per annum (95% CI 10383-11740). For every 10 new treated cases, 21 and 23 people would be assessed by and referred to EIP services for suspected psychosis, respectively. Conclusions: Our evidence-based methodology provides an accurate, validated tool to inform clinical provision of EIP services about future population need for care, based on local variation of major social determinants of psychosis.

Conflating marginal and conditional treatment effects: Comments on "Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study".

In this commentary, we highlight the importance of: (1) carefully considering and clarifying whether a marginal or conditional treatment effect is of interest in a population-adjusted indirect treatment comparison; and (2) developing distinct methodologies for estimating the different measures of effect. The appropriateness of each methodology depends on the preferred target of inference.

Joint Longitudinal Models for Dealing With Missing at Random Data in Trial-Based Economic Evaluations.

OBJECTIVES: In trial-based economic evaluation, some individuals are typically associated with missing data at some time point, so that their corresponding aggregated outcomes (eg, quality-adjusted life-years) cannot be evaluated. Restricting the analysis to the complete cases is inefficient and can result in biased estimates, while imputation methods are often implemented under a missing at random (MAR) assumption. We propose the use of joint longitudinal models to extend standard approaches by taking into account the longitudinal structure to improve the estimation of the targeted quantities under MAR. METHODS: We compare the results from methods that handle missingness at an aggregated (case deletion, baseline imputation, and joint aggregated models) and disaggregated (joint longitudinal models) level under MAR. The methods are compared using a simulation study and applied to data from 2 real case studies. RESULTS: Simulations show that, according to which data affect the missingness process, aggregated methods may lead to biased results, while joint longitudinal models lead to valid inferences under MAR. The analysis of the 2 case studies support these results as both parameter estimates and cost-effectiveness results vary based on the amount of data incorporated into the model. CONCLUSIONS: Our analyses suggest that methods implemented at the aggregated level are potentially biased under MAR as they ignore the information from the partially observed follow-up data. This limitation can be overcome by extending the analysis to a longitudinal framework using joint models, which can incorporate all the available evidence.

Comparison of Parametric Survival Extrapolation Approaches Incorporating General Population Mortality for Adequate Health Technology Assessment of New Oncology Drugs.

OBJECTIVES: Survival extrapolation of trial outcomes is required for health economic evaluation. Generally, all-cause mortality (ACM) is modeled using standard parametric distributions, often without distinguishing disease-specific/excess mortality and general population background mortality (GPM). Recent National Institute for Health and Care Excellence guidance (Technical Support Document 21) recommends adding GPM hazards to disease-specific/excess mortality hazards in the log-likelihood function ("internal additive hazards"). This article compares alternative extrapolation approaches with and without GPM adjustment. METHODS: Survival extrapolations using the internal additive hazards approach (1) are compared to no GPM adjustment (2), applying GPM hazards once ACM hazards drop below GPM hazards (3), adding GPM hazards to ACM hazards (4), and proportional hazards for ACM versus GPM hazards (5). The fit, face validity, mean predicted life-years, and corresponding uncertainty measures are assessed for the active versus control arms of immature and mature (30- and 75-month follow-up) multiple myeloma data and mature (64-month follow-up) breast cancer data. RESULTS: The 5 approaches yielded considerably different outcomes. Incremental mean predicted life-years vary most in the immature multiple myeloma data set. The lognormal distribution (best statistical fit for approaches 1-4) produces survival increments of 3.5 (95% credible interval: 1.4-5.3), 8.5 (3.1-13.0), 3.5 (1.3-5.4), 2.9 (1.1-4.5), and 1.6 (0.4-2.8) years for approaches 1 to 5, respectively. Approach 1 had the highest face validity for all data sets. Uncertainty over parametric distributions was comparable for GPM-adjusted approaches 1, 3, and 4, and much larger for approach 2. CONCLUSION: This study highlights the importance of GPM adjustment, and particularly of incorporating GPM hazards in the log-likelihood function of standard parametric distributions.

The Effects of Model Misspecification in Unanchored Matching-Adjusted Indirect Comparison: Results of a Simulation Study.

OBJECTIVES: To assess the performance of unanchored matching-adjusted indirect comparison (MAIC) by matching on first moments or higher moments in a cross-study comparisons under a variety of conditions. A secondary objective was to gauge the performance of the method relative to propensity score weighting (PSW). METHODS: A simulation study was designed based on an oncology example, where MAIC was used to account for differences between a contemporary trial in which patients had more favorable characteristics and a historical control. A variety of scenarios were then tested varying the setup of the simulation study, including violating the implicit or explicit assumptions of MAIC. RESULTS: Under ideal conditions and under a variety of scenarios, MAIC performed well (shown by a low mean absolute error [MAE]) and was unbiased (shown by a mean error [ME] of about zero). The performance of the method deteriorated where the matched characteristics had low explanatory power or there was poor overlap between studies. Only when important characteristics are not included in the matching did the method become biased (nonzero ME). Where the method showed poor performance, this was exaggerated if matching was also performed on the variance (ie, higher moments). Relative to PSW, MAIC provided similar results in most circumstances, although it exhibited slightly higher MAE and a higher chance of exaggerating bias. CONCLUSIONS: MAIC appears well suited to adjust for cross-trial comparisons provided the assumptions underpinning the model are met, with relatively little efficiency loss compared with PSW.

Computing the Expected Value of Sample Information Efficiently: Practical Guidance and Recommendations for Four Model-Based Methods.

Value of information (VOI) analyses can help policy makers make informed decisions about whether to conduct and how to design future studies. Historically a computationally expensive method to compute the expected value of sample information (EVSI) restricted the use of VOI to simple decision models and study designs. Recently, 4 EVSI approximation methods have made such analyses more feasible and accessible. Members of the Collaborative Network for Value of Information (ConVOI) compared the inputs, the analyst's expertise and skills, and the software required for the 4 recently developed EVSI approximation methods. Our report provides practical guidance and recommendations to help inform the choice between the 4 efficient EVSI estimation methods. More specifically, this report provides: (1) a step-by-step guide to the methods' use, (2) the expertise and skills required to implement the methods, and (3) method recommendations based on the features of decision-analytic problems.

Summarising salient information on historical controls: A structured assessment of validity and comparability across studies.

BACKGROUND: While placebo-controlled randomised controlled trials remain the standard way to evaluate drugs for efficacy, historical data are used extensively across the development cycle. This ranges from supplementing contemporary data to increase the power of trials to cross-trial comparisons in estimating comparative efficacy. In many cases, these approaches are performed without in-depth review of the context of data, which may lead to bias and incorrect conclusions. METHODS: We discuss the original 'Pocock' criteria for the use of historical data and how the use of historical data has evolved over time. Based on these factors and personal experience, we created a series of questions that may be asked of historical data, prior to their use. Based on the answers to these questions, various statistical approaches are recommended. The strategy is illustrated with a case study in colorectal cancer. RESULTS: A number of areas need to be considered with historical data, which we split into three categories: outcome measurement, study/patient characteristics (including setting and inclusion/exclusion criteria), and disease process/intervention effects. Each of these areas may introduce issues if not appropriately handled, while some may preclude the use of historical data entirely. We present a tool (in the form of a table) for highlighting any such issues. Application of the tool to a colorectal cancer data set demonstrates under what conditions historical data could be used and what the limitations of such an analysis would be. CONCLUSION: Historical data can be a powerful tool to augment or compare with contemporary trial data, though caution is required. We present some of the issues that may be considered when involving historical data and what (if any) statistical approaches may account for differences between studies. We recommend that, where historical data are to be used in analyses, potential differences between studies are addressed explicitly.

A comparative review of network meta-analysis models in longitudinal randomized controlled trial.

Network meta-analysis (NMA) technique extends the standard meta-analysis methods, allowing pairwise comparison of all treatments in a network in the absence of head-to-head comparisons. Traditional NMA models consider a single endpoint for each trial. However, in many cases, trials in the network have different durations and/or report data at multiple time points. Moreover, these time points are often not the same for all trials. In this work, we review the most relevant methods that incorporate multiple time points and allow indirect comparisons of treatment effects across different longitudinal studies. In particular, we focus on the mixed treatment comparison developed by Dakin et al,[10] on the Bayesian evidence synthesis techniques-integrated two-component prediction developed by Ding et al,[11] and on the more recent method based on fractional polynomials by Jansen et al.[12] We highlight the main features of each model and illustrate them in simulations and in a real data application. Our study shows that methods based on fractional polynomials offer a flexible modeling strategy in most applications.

A full Bayesian model to handle structural ones and missingness in economic evaluations from individual-level data.

Economic evaluations from individual-level data are an important component of the process of technology appraisal, with a view to informing resource allocation decisions. A critical problem in these analyses is that both effectiveness and cost data typically present some complexity (eg, nonnormality, spikes, and missingness) that should be addressed using appropriate methods. However, in routine analyses, standardised approaches are typically used, possibly leading to biassed inferences. We present a general Bayesian framework that can handle the complexity. We show the benefits of using our approach with a motivating example, the MenSS trial, for which there are spikes at one in the effectiveness and missingness in both outcomes. We contrast a set of increasingly complex models and perform sensitivity analysis to assess the robustness of the conclusions to a range of plausible missingness assumptions. We demonstrate the flexibility of our approach with a second example, the PBS trial, and extend the framework to accommodate the characteristics of the data in this study. This paper highlights the importance of adopting a comprehensive modelling approach to economic evaluations and the strategic advantages of building these complex models within a Bayesian framework.

R You Still Using Excel? The Advantages of Modern Software Tools for Health Technology Assessment.

Economic models are used in health technology assessments (HTAs) to evaluate the cost-effectiveness of competing medical technologies and inform the efficient use of healthcare resources. Historically, these models have been developed with specialized commercial software (such as TreeAge) or more commonly with spreadsheet software (almost always Microsoft Excel). Although these tools may be sufficient for relatively simple analyses, they put unnecessary constraints on the analysis that may ultimately limit its credibility and relevance. In contrast, modern programming languages such as R, Python, Matlab, and Julia facilitate the development of models that are (i) clinically realistic, (ii) capable of quantifying decision uncertainty, (iii) transparent and reproducible, and (iv) reusable and adaptable. An HTA environment that encourages use of modern software can therefore help ensure that coverage and pricing decisions confer greatest possible benefit and capture all scientific uncertainty, thus enabling correct prioritization of future research.

Frequentist and Bayesian meta-regression of health state utilities for multiple myeloma incorporating systematic review and analysis of individual patient data.

This analysis presents the results of a systematic review for health state utilities in multiple myeloma, as well as analysis of over 9,000 observations taken from registry and trial data. The 27 values identified from 13 papers are then synthesised in a frequentist nonparametric bootstrap model and a Bayesian meta-regression. Results were similar between the frequentist and Bayesian models with low utility on disease diagnosis (approximately 0.55), raising to approximately 0.65 on first line treatment and declining slightly with each subsequent line. Stem cell transplant was also found to be a significant predictor of health-related quality of life in both individual patient data and meta-regression, with an increased utility of approximately 0.06 across different models. The work presented demonstrates the feasibility of Bayesian methods for utility meta-regression, whilst also presenting an internally consistent set of data from the analysis of registry data. To facilitate easy updating of the data and model, data extraction tables and model code are provided as Data S1. The main limitations of the model relate to the low number of studies available, particularly in highly pretreated patients.

Incidence and prevalence of diabetes and cost of illness analysis of polycystic ovary syndrome: a Bayesian modelling study.

STUDY QUESTION: What is the incidence/prevalence of type 2 diabetes in women with polycystic ovary syndrome (PCOS) and the economic burden associated with PCOS in the UK? SUMMARY ANSWER: The incidence and prevalence of type 2 diabetes in women with PCOS are 3-33 per 1000 person years and 26.5%, respectively, with an associated annual healthcare burden of at least £237 million in the UK. WHAT IS KNOWN ALREADY: Although observational studies have been designed to assess the incidence of diabetes in women with PCOS, these have been open to criticism because of short periods of follow-up, small sample sizes or invalidated diagnosis of PCOS. Only one study has estimated the healthcare-related economic burden of PCOS, reporting a cost of $4.36 billion per year in the USA. STUDY DESIGN, SIZE, DURATION: This was a modelling study using individual patient data from a UK primary care database between 2004 and 2014 and aggregate data from the literature to obtain conversion rates through disease progression of PCOS. A simulation approach was applied to model the population dynamics of PCOS over a follow-up period of 25 years. PARTICIPANTS/MATERIALS, SETTING, METHODS: A total of 14 135 women with PCOS or symptoms indicative of PCOS were selected from the primary care database to estimate the incidence of confirmed diagnosis of PCOS and diagnosis of type 2 diabetes. A 'virtual' cohort including the entire PCOS population (size estimated from the UK census data) was simulated to model the population dynamics of PCOS. The economic and utility analyses were further conducted from a healthcare perspective. MAIN RESULTS AND THE ROLE OF CHANCE: The peak conversion rate from possible to diagnosed PCOS was 121 per 1000 person-year (PY). The maximal incidence of type 2 diabetes was 33 per 1000 PY. The estimated prevalence of diabetes in the PCOS population was 26.5% (95% interval: 25.4-27.8%) during a 25-year follow-up. The annual healthcare burden of PCOS based on our conservative estimate is at least £237 million for the follow-up period examined. LIMITATIONS, REASONS FOR CAUTION: Due to lack of data, a full economic evaluation including healthcare costs of all the comorbidities associated with PCOS was not possible. Simplification of the real-world situation represented by the model may be a concern. WIDER IMPLICATIONS OF THE FINDINGS: This study suggests that a large number of women with symptoms indicative of PCOS never receive a definitive diagnosis yet can suffer from a rapid conversion to diabetes. This significantly reduces the quality of life for individual patients and incurs high costs for healthcare providers. As the risk of diabetes in women with PCOS is similar to that seen in populations at high risks of diabetes, it is possible that including them in national screening programmes may be cost effective. STUDY FUNDING/COMPETING INTEREST(S): There was no funding for the current study. There are no conflicts of interest. TRIAL REGISTRATION NUMBER: Not applicable.

Calculating the Expected Value of Sample Information Using Efficient Nested Monte Carlo: A Tutorial.

OBJECTIVE: The expected value of sample information (EVSI) quantifies the economic benefit of reducing uncertainty in a health economic model by collecting additional information. This has the potential to improve the allocation of research budgets. Despite this, practical EVSI evaluations are limited partly due to the computational cost of estimating this value using the gold-standard nested simulation methods. Recently, however, Heath et al. developed an estimation procedure that reduces the number of simulations required for this gold-standard calculation. Up to this point, this new method has been presented in purely technical terms. STUDY DESIGN: This study presents the practical application of this new method to aid its implementation. We use a worked example to illustrate the key steps of the EVSI estimation procedure before discussing its optimal implementation using a practical health economic model. METHODS: The worked example is based on a three-parameter linear health economic model. The more realistic model evaluates the cost-effectiveness of a new chemotherapy treatment, which aims to reduce the number of side effects experienced by patients. We use a Markov model structure to evaluate the health economic profile of experiencing side effects. RESULTS: This EVSI estimation method offers accurate estimation within a feasible computation time, seconds compared to days, even for more complex model structures. The EVSI estimation is more accurate if a greater number of nested samples are used, even for a fixed computational cost. CONCLUSIONS: This new method reduces the computational cost of estimating the EVSI by nested simulation.

A dynamic Bayesian Markov model for health economic evaluations of interventions in infectious disease.

BACKGROUND: Health economic evaluations of interventions in infectious disease are commonly based on the predictions of ordinary differential equation (ODE) systems or Markov models (MMs). Standard MMs are static, whereas ODE systems are usually dynamic and account for herd immunity which is crucial to prevent overestimation of infection prevalence. Complex ODE systems including distributions on model parameters are computationally intensive. Thus, mainly ODE-based models including fixed parameter values are presented in the literature. These do not account for parameter uncertainty. As a consequence, probabilistic sensitivity analysis (PSA), a crucial component of health economic evaluations, cannot be conducted straightforwardly. METHODS: We present a dynamic MM under a Bayesian framework. We extend a static MM by incorporating the force of infection into the state allocation algorithm. The corresponding output is based on dynamic changes in prevalence and thus accounts for herd immunity. In contrast to deterministic ODE-based models, PSA can be conducted straightforwardly. We introduce a case study of a fictional sexually transmitted infection and compare our dynamic Bayesian MM to a deterministic and a Bayesian ODE system. The models are calibrated to simulated time series data. RESULTS: By means of the case study, we show that our methodology produces outcome which is comparable to the "gold standard" of the Bayesian ODE system. CONCLUSIONS: In contrast to ODE systems in the literature, the dynamic MM includes distributions on all model parameters at manageable computational effort (including calibration). The run time of the Bayesian ODE system is 15 times longer.

Effectiveness of an intervention to facilitate prompt referral to memory clinics in the United Kingdom: Cluster randomised controlled trial.

BACKGROUND: Most people with dementia do not receive timely diagnosis, preventing them from making informed plans about their future and accessing services. Many countries have a policy to increase timely diagnosis, but trials aimed at changing general practitioner (GP) practice have been unsuccessful. We aimed to assess whether a GP's personal letter, with an evidence-based leaflet about overcoming barriers to accessing help for memory problems-aimed at empowering patients and families-increases timely dementia diagnosis and patient presentation to general practice. METHODS AND FINDING: Multicentre, cluster-randomised controlled trial with raters masked to an online computer-generated randomisation system assessing 1 y outcome. We recruited 22 general practices (August 2013-September 2014) and 13 corresponding secondary care memory services in London, Hertfordshire, and Essex, United Kingdom. Eligible patients were aged ≥70 y, without a known diagnosis of dementia, living in their own homes. There were 6,387 such patients in 11 intervention practices and 8,171 in the control practices. The primary outcome was cognitive severity on Mini Mental State Examination (MMSE). Main secondary outcomes were proportion of patients consulting their GP with suspected memory disorders and proportion of those referred to memory clinics. There was no between-group difference in cognitive severity at diagnosis (99 intervention, mean MMSE = 22.04, 95% confidence intervals (CIs) = 20.95 to 23.13; 124 control, mean MMSE = 22.59, 95% CI = 21.58 to 23.6; p = 0.48). GP consultations with patients with suspected memory disorders increased in intervention versus control group (odds ratio = 1.41; 95% CI = 1.28, 1.54). There was no between-group difference in the proportions of patients referred to memory clinics (166, 2.5%; 220, 2.7%; p = .077 respectively). The study was limited as we do not know whether the additional patients presenting to GPs had objective as well as subjective memory problems and therefore should have been referred. In addition, we aimed to empower patients but did not do anything to change GP practice. CONCLUSIONS: Our intervention to access timely dementia diagnosis resulted in more patients presenting to GPs with memory problems, but no diagnoses increase. We are uncertain as to the reason for this and do not know whether empowering the public and targeting GPs would have resulted in a successful intervention. Future interventions should be targeted at both patients and GPs. TRIAL REGISTRATION: Current Controlled Trials ISRCTN19216873.

Cost-effectiveness analysis of the nine-valent HPV vaccine in Italy.

BACKGROUND: In Italy HPV vaccination with the quadrivalent vaccine (Gardasil®) is offered actively and free of charge to girls aged 12 since 2007. A nine-valent vaccine (Gardasil 9®) received the European market authorization in 2015 to protect, with only 2 doses, against around 90% of all HPV positive cancers, over 80% of high-grade precancerous lesions and 90% of genital warts caused by HPV types 6/11. METHODS: A dynamic transmission model simulating the natural history of HPV-infections was calibrated to the Italian setting and used to estimate costs and QALYs associated with vaccination strategies. The analyses compared two strategies with the nine-valent vaccine (cervical cancer screening and vaccination in girls only or vaccination in boys and girls) to four alternative strategies (cervical cancer screening and vaccination with quadrialent vaccine in girls only, in both boys and girls, with bivalent vaccine in girls and screening strategy only). The National Health Service perspective was considered. CONCLUSION: The switch to the nine-valent vaccine in Italy can further reduce the burden associated to cervical cancer and HPV-related diseases and is highly cost-effective. RESULTS: Compared to the current vaccination program with quadrivalent vaccine, the nine-valent vaccine in a programme including girls and boys shows further reductions of 17% in the incidence of cervical cancer, 35 and 14% in anal cancer for males and females, as well as over a million cases of genital warts avoided after 100 years. The new technology is associated with an ICER of 10,463€ per QALY gained in universal vaccination, decreasing to 4483€ when considering the vaccine switch for girls-only.