Anti-Vascular Endothelial Growth Factor Drugs Compared With Panretinal Photocoagulation for the Treatment of Proliferative Diabetic Retinopathy: A Cost-Effectiveness Analysis.

OBJECTIVES: This study aimed to evaluate the cost-effectiveness of anti-vascular endothelial growth factor drugs (anti-VEGFs) compared with panretinal photocoagulation (PRP) for treating proliferative diabetic retinopathy (PDR) in the United Kingdom. METHODS: A discrete event simulation model was developed, informed by individual participant data meta-analysis. The model captures treatment effects on best corrected visual acuity in both eyes, and the occurrence of diabetic macular edema and vitreous hemorrhage. The model also estimates the value of undertaking further research to resolve decision uncertainty. RESULTS: Anti-VEGFs are unlikely to generate clinically meaningful benefits over PRP. The model predicted anti-VEGFs be more costly and similarly effective as PRP, generating 0.029 fewer quality-adjusted life-years at an additional cost of £3688, with a net health benefit of -0.214 at a £20 000 willingness-to-pay threshold. Scenario analysis results suggest that only under very select conditions may anti-VEGFs offer potential for cost-effective treatment of PDR. The consequences of loss to follow-up were an important driver of model outcomes. CONCLUSIONS: Anti-VEGFs are unlikely to be a cost-effective treatment for early PDR compared with PRP. Anti-VEGFs are generally associated with higher costs and similar health outcomes across various scenarios. Although anti-VEGFs were associated with lower diabetic macular edema rates, the number of cases avoided is insufficient to offset the additional treatment costs. Key uncertainties relate to the long-term comparative effectiveness of anti-VEGFs, particularly considering the real-world rates and consequences of treatment nonadherence. Further research on long-term visual acuity and rates of vision-threatening complications may be beneficial in resolving uncertainties.

Reflecting Parameter Uncertainty in Addition to Variability in Constrained Healthcare Resource Discrete Event Simulations: Worth Going the Extra Mile or a Road to Nowhere?

OBJECTIVES: Probabilistic sensitivity analysis (PSA) has been shown to reduce bias in outcomes of health economic models. However, only 1 existing study has been identified that incorporates PSA within a resource-constrained discrete event simulation (DES) model. This article aims to assess whether it is feasible and appropriate to use PSA to characterize parameter uncertainty in DES models that are primarily constructed to explore the impact of constrained resources. METHODS: PSA is incorporated into a new case study of an Emergency Department DES. Structured expert elicitation is used to derive the variability and uncertainty input distributions associated with length of time taken to complete key activities within the Emergency Department. Potential challenges of implementation and analysis are explored. RESULTS: The results of a trial of the model, which used the best estimates of the elicited means and variability around the time taken to complete activities, provided a reasonable fit to the data for length of time within the Emergency Department. However, there was substantial and skewed uncertainty around the activity times estimated from the elicitation exercise. This led to patients taking almost 3 weeks to leave the Emergency Department in some PSA runs, which would not occur in practice. CONCLUSIONS: Structured expert elicitation can be used to derive plausible estimates of activity times and their variability, but experts' uncertainty can be substantial. For parameters that have an impact on interactions within a resource-constrained simulation model, PSA can lead to implausible model outputs; hence, other methods may be needed.

Priorities for research to support local authority action on health and climate change: a study in England.

BACKGROUND: Evidence is needed to support local action to reduce the adverse health impacts of climate change and maximise the health co-benefits of climate action. Focused on England, the study identifies priority areas for research to inform local decision making. METHODS: Firstly, potential priority areas for research were identified from a brief review of UK policy documents, and feedback invited from public and policy stakeholders. This included a survey of Directors of Public Health (DsPH) in England, the local government officers responsible for public health. Secondly, rapid reviews of research evidence examined whether there was UK evidence relating to the priorities identified in the survey. RESULTS: The brief policy review pointed to the importance of evidence in two broad areas: (i) community engagement in local level action on the health impacts of climate change and (ii) the economic (cost) implications of such action. The DsPH survey (n = 57) confirmed these priorities. With respect to community engagement, public understanding of climate change's health impacts and the public acceptability of local climate actions were identified as key evidence gaps. With respect to economic implications, the gaps related to evidence on the health and non-health-related costs and benefits of climate action and the short, medium and longer-term budgetary implications of such action, particularly with respect to investments in the built environment. Across both areas, the need for evidence relating to impacts across income groups was highlighted, a point also emphasised by the public involvement panel. The rapid reviews confirmed these evidence gaps (relating to public understanding, public acceptability, economic evaluation and social inequalities). In addition, public and policy stakeholders pointed to other barriers to action, including financial pressures, noting that better evidence is insufficient to enable effective local action. CONCLUSIONS: There is limited evidence to inform health-centred local action on climate change. More evidence is required on public perspectives on, and the economic dimensions of, local climate action. Investment in locally focused research is urgently needed if local governments are to develop and implement evidence-based policies to protect public health from climate change and maximise the health co-benefits of local action.

Study Protocol. Evaluating the life-course health impact of a city-wide system approach to improve air quality in Bradford, UK: A quasi-experimental study with implementation and process evaluation.

BACKGROUND: Air quality is a major public health threat linked to poor birth outcomes, respiratory and cardiovascular disease, and premature mortality. Deprived groups and children are disproportionately affected. Bradford will implement a Clean Air Zone (CAZ) as part of the Bradford Clean Air Plan (B-CAP) in 2022 to reduce pollution, providing a natural experiment. The aim of the current study is to evaluate the impact of the B-CAP on health outcomes and air quality, inequalities and explore value for money. An embedded process and implementation evaluation will also explore barriers and facilitators to implementation, impact on attitudes and behaviours, and any adverse consequences. METHODS: The study is split into 4 work packages (WP). WP1A: 20 interviews with decision makers, 20 interviews with key stakeholders; 10 public focus groups and documentary analysis of key reports will assess implementation barriers, acceptability and adverse or unanticipated consequences at 1 year post-implementation (defined as point at which charging CAZ goes 'live'). WP1B: A population survey (n = 2000) will assess travel behaviour and attitudes at baseline and change at 1 year post-implementation). WP2: Routine air quality measurements will be supplemented with data from mobile pollution sensors in 12 schools collected by N = 240 pupil citizen scientists (4 within, 4 bordering and 4 distal to CAZ boundary). Pupils will carry sensors over four monitoring periods over a 12 month period (two pre, and two post-implementation). We will explore whether reductions in pollution vary by CAZ proximity. WP3A: We will conduct a quasi-experimental interrupted time series analysis using a longitudinal routine health dataset of > 530,000 Bradford residents comparing trends (3 years prior vs 3 years post) in respiratory health (assessed via emergency/GP attendances. WP3B: We will use the richly-characterised Born in Bradford cohort (13,500 children) to explore health inequalities in respiratory health using detailed socio-economic data. WP4: will entail a multi-sectoral health economic evaluation to determine value for money of the B-CAP. DISCUSSION: This will be first comprehensive quasi-experimental evaluation of a city-wide policy intervention to improve air quality. The findings will be of value for other areas implementing this type of approach. TRIAL REGISTRATION: ISRCTN67530835 https://doi.org/10.1186/ISRCTN67530835.

Eliciting uncertainty for complex parameters in model-based economic evaluations: quantifying a temporal change in the treatment effect.

BACKGROUND: In model-based economic evaluations, the effectiveness parameter is often informed by studies with a limited duration of follow-up, requiring extrapolation of the treatment effect over a longer time horizon. Extrapolation from short-term data alone may not adequately capture uncertainty in that extrapolation. This study aimed to use structured expert elicitation to quantify uncertainty associated with extrapolation of the treatment effect observed in a clinical trial. METHODS: A structured expert elicitation exercise was conducted for an applied study of a podiatry intervention designed to reduce the rate of falls and fractures in the elderly. A bespoke web application was used to elicit experts' beliefs about two outcomes (rate of falls and odds of fracture) as probability distributions (priors), for two treatment options (intervention and treatment as usual) at multiple time points. These priors were used to derive the temporal change in the treatment effect of the intervention, to extrapolate outcomes observed in a trial. The results were compared with extrapolation without experts' priors. RESULTS: The study recruited thirty-eight experts (geriatricians, general practitioners, physiotherapists, nurses, and academics) from England and Wales. The majority of experts (32/38) believed that the treatment effect would depreciate over time and expressed greater uncertainty than that extrapolated from a trial-based outcome alone. The between-expert variation in predicted outcomes was relatively small. CONCLUSIONS: This study suggests that uncertainty in extrapolation can be informed using structured expert elicitation methods. Using structured elicitation to attach values to complex parameters requires key assumptions and simplifications to be considered.

Delayed transfers of care for older people: a wider perspective.

Delayed transfers of care (DTOC), often unhelpfully referred to as 'bed blocking', has become a byword for waste and inefficiency in healthcare systems throughout the world. An estimated 2.7 million bed days are occupied each year in England by older people no longer in need of acute treatment, estimated to cost £820 million (2014/15) in inpatient care. Policy and media attention have often been drawn to this narrative of financial waste, resulting in policy setting that directly targets the level of DTOC, but has done little to put patient health first. These figures and policies portray a misleading image of the delays as primarily of concern in terms of their financial burden on acute hospital care, with little consideration given to the quantification on patient health or wider societal impacts. In spite of the multi-factorial decision-making process that occurs for each patient discharge, current evaluation frameworks and national policy setting fail to reflect the complexity of the process. In this commentary, we interrogate the current approach to the quantification of the DTOC impact and explore how policies and evaluation methods can do more to reflect the true impact of the delays.

Cost-effectiveness of a proportionate universal offer of free exercise: Leeds Let's Get Active.

BACKGROUND: The purpose of this paper is to assess the cost-effectiveness of a proportionate universal programme to reduce physical inactivity (Leeds Let us Get Active (LLGA)) in adults. METHODS: A continuous-time Markov chain model was developed to assess the cost implications and QALY gains associated with increases in physical activity levels across the adult population. A parametric survival analysis approach was applied to estimate the decay of intervention effect over time. Baseline model data were obtained from previous economic models, population-based surveys and other published literature. A cost-utility analysis was conducted from a health care sector perspective over the programme duration (39 months). Scenario and probabilistic sensitivity analyses were performed to test the robustness of cost-effectiveness results. RESULTS: In total, 51 874 adult residents registered to the programme and provided baseline data,19.5% of which were living in deprived areas. Under base case assumptions, LLGA was found to be likely to be cost-effective. However, variations in key structural assumptions showed sensitivity of the results. CONCLUSIONS: Results from this study suggest a non-negligible level of uncertainty regarding the effectiveness, and therefore, cost-effectiveness of a universal offer of free leisure centre-based exercise that targets hard to reach groups. Further data collection and a shift towards prospective evaluations are needed.

Experiences of Structured Elicitation for Model-Based Cost-Effectiveness Analyses.

BACKGROUND: Empirical evidence supporting the cost-effectiveness estimates of particular health care technologies may be limited, or it may even be missing entirely. In these situations, additional information, often in the form of expert judgments, is needed to reach a decision. There are formal methods to quantify experts' beliefs, termed as structured expert elicitation (SEE), but only limited research is available in support of methodological choices. Perhaps as a consequence, the use of SEE in the context of cost-effectiveness modelling is limited. OBJECTIVES: This article reviews applications of SEE in cost-effectiveness modelling with the aim of summarizing the basis for methodological choices made in each application and recording the difficulties and challenges reported by the authors in the design, conduct, and analyses. METHODS: The methods used in each application were extracted along with the criteria used to support methodological and practical choices and any issues or challenges discussed in the text. Issues and challenges were extracted using an open field, and then categorised and grouped for reporting. RESULTS: The review demonstrates considerable heterogeneity in methods used, and authors acknowledge great methodological uncertainty in justifying their choices. Specificities of the context area emerging as potentially important in determining further methodological research in elicitation are between- expert variation and its interpretation, the fact that substantive experts in the area may not be trained in quantitative subjects, that judgments are often needed on various parameter types, the need for some form of assessment of validity, and the need for more integration with behavioural research to devise relevant debiasing strategies. CONCLUSIONS: This review of experiences of SEE highlights a number of specificities/constraints that can shape the development of guidance and target future research efforts in this area.

A Comprehensive Algorithm for Approval of Health Technologies With, Without, or Only in Research: The Key Principles for Informing Coverage Decisions.

BACKGROUND: The value of evidence about the performance of a technology and the value of access to a technology are central to policy decisions regarding coverage with, without, or only in research and managed entry (or risk-sharing) agreements. OBJECTIVES: We aim to outline the key principles of what assessments are needed to inform "only in research" (OIR) or "approval with research" (AWR) recommendations, in addition to approval or rejection. METHODS: We developed a comprehensive algorithm to inform the sequence of assessments and judgments that lead to different types of guidance: OIR, AWR, Approve, or Reject. This algorithm identifies the order in which assessments might be made, how similar guidance might be arrived at through different combinations of considerations, and when guidance might change. RESULTS: The key principles are whether the technology is expected to be cost-effective; whether the technology has significant irrecoverable costs; whether additional research is needed; whether research is possible with approval and whether there are opportunity costs that once committed by approval cannot be recovered; and whether there are effective price reductions. Determining expected cost-effectiveness is only a first step. In addition to AWR for technologies expected to be cost-effective and OIR for those not expected to be cost-effective, there are other important circumstances when OIR should be considered. CONCLUSIONS: These principles demonstrate that cost-effectiveness is a necessary but not sufficient condition for approval. Even when research is possible with approval, OIR may be appropriate when a technology is expected to be cost-effective due to significant irrecoverable costs.

Unifying Research and Reimbursement Decisions: Case Studies Demonstrating the Sequence of Assessment and Judgments Required.

BACKGROUND: The key principles regarding what assessments lead to different types of guidance about the use of health technologies (Only in Research, Approval with Research, Approve, or Reject) provide an explicit and transparent framework for technology appraisal. OBJECTIVE: We aim to demonstrate how these principles and assessments can be applied in practice through the use of a seven-point checklist of assessment. METHODS: The value of access to a technology and the value of additional evidence are explored through the application of the checklist to the case studies of enhanced external counterpulsation for chronic stable angina and clopidogrel for the management of patients with non-ST-segment elevation acute coronary syndromes. RESULTS: The case studies demonstrate the importance of considering 1) the expected cost-effectiveness and population net health effects; 2) the need for evidence and whether the type of research required can be conducted once a technology is approved for widespread use; 3) whether there are sources of uncertainty that cannot be resolved by research but only over time; and 4) whether there are significant (opportunity) costs that once committed by approval cannot be recovered. CONCLUSIONS: The checklist demonstrates that cost-effectiveness is a necessary but not sufficient condition for approval. Only in Research may be appropriate when a technology is expected to be cost-effective due to significant irrecoverable costs. It is only approval that can be ruled out if a technology is not expected to be cost-effective. Lack of cost-effectiveness is not a necessary or sufficient condition for rejection.

Cefiderocol for treating severe aerobic Gram-negative bacterial infections: technology evaluation to inform a novel subscription-style payment model.

Background: To limit the use of antimicrobials without disincentivising the development of novel antimicrobials, there is interest in establishing innovative models that fund antimicrobials based on an evaluation of their value as opposed to the volumes used. The aim of this project was to evaluate the population-level health benefit of cefiderocol in the NHS in England, for the treatment of severe aerobic Gram-negative bacterial infections when used within its licensed indications. The results were used to inform the National Institute for Health and Care Excellence guidance in support of commercial discussions regarding contract value between the manufacturer and NHS England. Methods: The health benefit of cefiderocol was first derived for a series of high-value clinical scenarios. These represented uses that were expected to have a significant impact on patients' mortality risks and health-related quality of life. The clinical effectiveness of cefiderocol relative to its comparators was estimated by synthesising evidence on susceptibility of the pathogens of interest to the antimicrobials in a network meta-analysis. Patient-level costs and health outcomes of cefiderocol under various usage scenarios compared with alternative management strategies were quantified using decision modelling. Results were reported as incremental net health effects expressed in quality-adjusted life-years, which were scaled to 20-year population values using infection number forecasts based on data from Public Health England. The outcomes estimated for the high-value clinical scenarios were extrapolated to other expected uses for cefiderocol. Results: Among Enterobacterales isolates with the metallo-beta-lactamase resistance mechanism, the base-case network meta-analysis found that cefiderocol was associated with a lower susceptibility relative to colistin (odds ratio 0.32, 95% credible intervals 0.04 to 2.47), but the result was not statistically significant. The other treatments were also associated with lower susceptibility than colistin, but the results were not statistically significant. In the metallo-beta-lactamase Pseudomonas aeruginosa base-case network meta-analysis, cefiderocol was associated with a lower susceptibility relative to colistin (odds ratio 0.44, 95% credible intervals 0.03 to 3.94), but the result was not statistically significant. The other treatments were associated with no susceptibility. In the base case, patient-level benefit of cefiderocol was between 0.02 and 0.15 quality-adjusted life-years, depending on the site of infection, the pathogen and the usage scenario. There was a high degree of uncertainty surrounding the benefits of cefiderocol across all subgroups. There was substantial uncertainty in the number of infections that are suitable for treatment with cefiderocol, so population-level results are presented for a range of scenarios for the current infection numbers, the expected increases in infections over time and rates of emergence of resistance. The population-level benefits varied substantially across the base-case scenarios, from 896 to 3559 quality-adjusted life-years over 20 years. Conclusion: This work has provided quantitative estimates of the value of cefiderocol within its areas of expected usage within the NHS. Limitations: Given existing evidence, the estimates of the value of cefiderocol are highly uncertain. Future work: Future evaluations of antimicrobials would benefit from improvements to NHS data linkages; research to support appropriate synthesis of susceptibility studies; and application of routine data and decision modelling to assess enablement value. Study registration: No registration of this study was undertaken. Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment Policy Research Programme (NIHR award ref: NIHR135591), conducted through the Policy Research Unit in Economic Methods of Evaluation in Health and Social Care Interventions, PR-PRU-1217-20401, and is published in full in Health Technology Assessment; Vol. 28, No. 28. See the NIHR Funding and Awards website for further award information.

This project tested new methods for estimating the value to the NHS of an antimicrobial, cefiderocol, so its manufacturer could be paid fairly even if very little drug is used in order to reduce the risk of bacteria becoming resistant to the product. Clinicians said that the greatest benefit of cefiderocol is when used for complicated urinary tract infections and pneumonia acquired within hospitals caused by two types of bacteria (called Enterobacterales and Pseudomonas aeruginosa), with a resistance mechanism called metallo-beta-lactamase. Because there were no relevant clinical trial data, we estimated how effective cefiderocol and alternative treatments were by doing a systematic literature review of studies that grew bacteria from infections in the laboratory and tested the drugs on them. We linked this to data estimating the long-term health and survival of patients. Some evidence was obtained by asking clinicians detailed questions about what they thought the effects would be based on their experience and the available evidence. We included the side effects of the alternative treatments, some of which can cause kidney damage. We estimated how many infections there would be in the UK, whether they would increase over time and how resistance to treatments may change over time. Clinicians told us that they would also use cefiderocol to treat intra-abdominal and bloodstream infections, and some infections caused by another bacteria called Stenotrophomonas. We estimated how many of these infections there would be, and assumed the same health benefits as for other types of infections. The total value to the NHS was calculated using these estimates. We also considered whether we had missed any additional elements of value. We estimated that the value to the NHS was £18­71 million over 20 years. This reflects the maximum the NHS could pay for use of cefiderocol if the health lost as a result of making these payments rather than funding other NHS services is not to exceed the health benefits of using this antimicrobial. However, these estimates are uncertain due to limitations with the evidence used to produce them and assumptions that had to be made.

Capturing all of the costs in NICE appraisals: the impact of inflammatory rheumatic diseases on productivity.

Inflammatory rheumatic diseases are common. It is estimated that ∼2.1% of the population has an inflammatory rheumatic disease (Andrianakos A, Trontzas P, Christoyannis F et al. Prevalence of rheumatic diseases in Greece: a cross-sectional population based epidemiological study. The ESORDIG Study. J Rheumatol 2003;30:1589-601). For diseases such as RA, PsA and AS, onset is most frequent between the ages of 30 and 50 years. The impact of inflammatory rheumatic diseases on physical functioning can be significant. Patients can suffer from swollen joints that cause pain and disability. This can reduce sufferers' ability to lead fully productive lives. This has major financial consequences for sufferers and their families and there is an economic impact on society. The National Institute for Health and Clinical Excellence (NICE) technology appraisal process has typically ignored any improvements in productivity that may result from treatment. There have been calls to extend the perspective of economic evaluations to include productivity costs as one aspect of wider social effects. However, there are a number of issues that must be resolved before productivity costs can become a routine input into the calculation of cost-effectiveness of treatments. First, there is limited agreement regarding the practical details and appropriate methods for their inclusion in economic evaluation. Second, there are issues that must be addressed regarding society's preference for equity. This issue arises when considering individual's economic productivity, that is to say, how we weigh individuals who are more productive (e.g. those in employment) against those who are less so (e.g. the unemployed). Finally, it is important to consider cross-budgetary effects, since productivity has cost and benefit implications outside of health.

Does providing everyone with free-of-charge organised exercise opportunities work in public health?

BACKGROUND AND PURPOSE: Population-level initiatives of free-of-charge organised exercise have been implemented to encourage residents to take up regular physical activity. However, there exists a paucity of evidence on the ability of these interventions to attract and engage residents, especially targeted subgroups. Seeking to contribute to this evidence base, we evaluated a proportionate universal programme providing free exercise sessions, Leeds Let's Get Active. METHODS: Descriptive statistics were used to summarise the programme data and participants. Time to event, count and logistic regression models examined how different population subgroups engaged with the programme in terms of number of entries, weekly participation rates and drop-off patterns. RESULTS: 51,874 adult residents registered to the programme and provided baseline data (2013-2016). A small proportion (1.6%) attended the free sessions on a weekly basis. Higher participation rates were estimated for the groups of males, retired and non-inactive participants. A neighbourhood-level deprivation status was found to have no marginal effect on the level and frequency of participation, but to be negatively associated with participation drop-off (HR 0.93, 95% CI 0.89-0.97, p = 0.001). CONCLUSIONS: Providing everyone with free-of-charge organised exercise opportunities in public leisure centres located in deprived areas can attract large volumes of residents, but may not sufficiently encourage adults, especially inactive residents and those living in disadvantaged neighbourhoods, to take up regular exercise.

Modelling decay in effectiveness for evaluation of behaviour change interventions: a tutorial for public health economists.

BACKGROUND AND PURPOSE: Recent methodological reviews of evaluations of behaviour change interventions in public health have highlighted that the decay in effectiveness over time has been mostly overlooked, potentially leading to suboptimal decision-making. While, in principle, discrete-time Markov chains-the most commonly used modelling approach-can be adapted to account for decay in effectiveness, this framework inherently lends itself to strong model simplifications. The application of formal and more appropriate modelling approaches has been supported, but limited progress has been made to date. The purpose of this paper is to encourage this shift by offering a practical guide on how to model decay in effectiveness using a continuous-time Markov chain (CTMC)-based approach. METHODS: A CTMC approach is demonstrated, with a contextualized tutorial being presented to facilitate learning and uptake. A worked example based on the stylized case study in physical activity promotion is illustrated with accompanying R code. DISCUSSION: The proposed framework presents a relatively small incremental change from the current modelling practice. CTMC represents a technical solution which, in absence of relevant data, allows for formally testing the sensitivity of results to assumptions regarding the long-term sustainability of intervention effects and improving model transparency. CONCLUSIONS: The use of CTMC should be considered in evaluations where decay in effectiveness is likely to be a key factor to consider. This would enable more robust model-based evaluations of population-level programmes to promote behaviour change and reduce the uncertainty surrounding the decision to invest in these public health interventions.

Digital interventions in mental health: evidence syntheses and economic modelling.

BACKGROUND: Economic evaluations provide evidence on whether or not digital interventions offer value for money, based on their costs and outcomes relative to the costs and outcomes of alternatives. OBJECTIVES: (1) Evaluate and summarise published economic studies about digital interventions across different technologies, therapies, comparators and mental health conditions; (2) synthesise clinical evidence about digital interventions for an exemplar mental health condition; (3) construct an economic model for the same exemplar mental health condition using the previously synthesised clinical evidence; and (4) consult with stakeholders about how they understand and assess the value of digital interventions. METHODS: We completed four work packages: (1) a systematic review and quality assessment of economic studies about digital interventions; (2) a systematic review and network meta-analysis of randomised controlled trials on digital interventions for generalised anxiety disorder; (3) an economic model and value-of-information analysis on digital interventions for generalised anxiety disorder; and (4) a series of knowledge exchange face-to-face and digital seminars with stakeholders. RESULTS: In work package 1, we reviewed 76 economic evaluations: 11 economic models and 65 within-trial analyses. Although the results of the studies are not directly comparable because they used different methods, the overall picture suggests that digital interventions are likely to be cost-effective, compared with no intervention and non-therapeutic controls, whereas the value of digital interventions compared with face-to-face therapy or printed manuals is unclear. In work package 2, we carried out two network meta-analyses of 20 randomised controlled trials of digital interventions for generalised anxiety disorder with a total of 2350 participants. The results were used to inform our economic model, but when considered on their own they were inconclusive because of the very wide confidence intervals. In work package 3, our decision-analytic model found that digital interventions for generalised anxiety disorder were associated with lower net monetary benefit than medication and face-to-face therapy, but greater net monetary benefit than non-therapeutic controls and no intervention. Value for money was driven by clinical outcomes rather than by intervention costs, and a value-of-information analysis suggested that uncertainty in the treatment effect had the greatest value (£12.9B). In work package 4, stakeholders identified several areas of benefits and costs of digital interventions that are important to them, including safety, sustainability and reducing waiting times. Four factors may influence their decisions to use digital interventions, other than costs and outcomes: increasing patient choice, reaching underserved populations, enabling continuous care and accepting the 'inevitability of going digital'. LIMITATIONS: There was substantial uncertainty around effect estimates of digital interventions compared with alternatives. This uncertainty was driven by the small number of studies informing most comparisons, the small samples in some of these studies and the studies' high risk of bias. CONCLUSIONS: Digital interventions may offer good value for money as an alternative to 'doing nothing' or 'doing something non-therapeutic' (e.g. monitoring or having a general discussion), but their added value compared with medication, face-to-face therapy and printed manuals is uncertain. Clinical outcomes rather than intervention costs drive 'value for money'. FUTURE WORK: There is a need to develop digital interventions that are more effective, rather than just cheaper, than their alternatives. STUDY REGISTRATION: This study is registered as PROSPERO CRD42018105837. FUNDING: This project was funded by the National Institute for Health Research (NIHR) Health Technology Assessment programme and will be published in full in Health Technology Assessment; Vol. 26, No. 1. See the NIHR Journals Library website for further project information.

Digital interventions are activities accessed via technology platforms (e.g. computers, smartphones and virtual reality) that can improve users' mental health and reduce addiction problems. To assess whether or not digital interventions offer 'value for money', we needed to compare their costs and outcomes with the costs and outcomes of alternatives, such as face-to-face therapy and medication. This was done through economic evaluations. This project consisted of four work packages. In work package 1, we reviewed 76 published economic evaluations of digital interventions for different mental health and addiction problems. We could not directly compare their results because of differences in the methods that were used, but the overall picture suggested that digital interventions could offer good value for money as an alternative to 'doing nothing' or simply monitoring someone or giving them general information. The picture was unclear when digital interventions were compared with face-to-face therapy. In work package 2, we pooled research studies that evaluated the outcomes of digital interventions in reducing anxiety and worry; the results were inconclusive because we were uncertain about the differences in outcomes between digital interventions and alternatives. In work package 3, an economic model suggested that value for money in digital interventions is driven by how good they are and not by how much they cost. In work package 4, we presented our methods and results to service users, mental health professionals and researchers who wanted to know more about the value of digital interventions for specific groups (e.g. children and older adults) and for outcomes other than reducing symptoms (e.g. reducing waiting times for treatment and improving attendance for therapy). Finally, the stakeholders highlighted four factors that may influence their decisions to use digital interventions, other than costs and outcomes: increasing choice, reaching underserved populations, enabling continuous care and accepting the 'inevitability of going digital'.

Cost Effectiveness of Digital Interventions for Generalised Anxiety Disorder: A Model-Based Analysis.

BACKGROUND: Digital interventions (DIs) are increasingly being used in mental health care, despite limited evidence regarding their value for money. This study aimed to evaluate the cost effectiveness of DIs for generalised anxiety disorder (GAD), in comparison with alternative care options, from the perspective of the UK health care system. METHODS: An open-source decision analytic cohort model was used to extrapolate the results of a network meta-analysis over a patient's lifetime and estimate the costs and outcomes (quality-adjusted life-years) of DIs and their comparators. The net monetary benefit (NMB) and probability of cost effectiveness was estimated for each comparator, and we conducted a Value of Information analysis to evaluate the scale and drivers of uncertainty. RESULTS: DIs were associated with lower NMB compared with medication and with group therapy, but greater NMB compared with non-therapeutic controls and with usual care. DIs that were supported by a clinician, an assistant or a lay person had higher delivery costs than purely patient-self-directed DIs, yielding a greater NMB when opportunity cost was above £3000/QALY. There was considerable uncertainty in the findings driven largely by uncertainty in the estimated treatment effects. The value of further research to establish the effectiveness of DIs for GAD was substantial, at least £12.9 billion. CONCLUSIONS: The high uncertainty about these results does not allow for recommendations based on the cost effectiveness of DIs. However, the analysis highlights areas for future research, and demonstrates that apparent cost savings associated with DIs can be offset by reduced effectiveness.

Reference Case Methods for Expert Elicitation in Health Care Decision Making.

BACKGROUND: The evidence used to inform health care decision making (HCDM) is typically uncertain. In these situations, the experience of experts is essential to help decision makers reach a decision. Structured expert elicitation (referred to as elicitation) is a quantitative process to capture experts' beliefs. There is heterogeneity in the existing elicitation methodology used in HCDM, and it is not clear if existing guidelines are appropriate for use in this context. In this article, we seek to establish reference case methods for elicitation to inform HCDM. METHODS: We collated the methods available for elicitation using reviews and critique. In addition, we conducted controlled experiments to test the accuracy of alternative methods. We determined the suitability of the methods choices for use in HCDM according to a predefined set of principles for elicitation in HCDM, which we have also generated. We determined reference case methods for elicitation in HCDM for health technology assessment (HTA). RESULTS: In almost all methods choices available for elicitation, we found a lack of empirical evidence supporting recommendations. Despite this, it is possible to define reference case methods for HTA. The reference methods include a focus on gathering experts with substantive knowledge of the quantities being elicited as opposed to those trained in probability and statistics, eliciting quantities that the expert might observe directly, and individual elicitation of beliefs, rather than solely consensus methods. It is likely that there are additional considerations for decision makers in health care outside of HTA. CONCLUSIONS: The reference case developed here allows the use of different methods, depending on the decision-making setting. Further applied examples of elicitation methods would be useful. Experimental evidence comparing methods should be generated.

Modelling the cost-effectiveness of biologic treatments for psoriatic arthritis.

OBJECTIVES: A probabilistic model was developed to determine the cost-effectiveness of three biologics, etanercept, infliximab and adalimumab, compared with palliative care for the treatment of active and progressive PsA in patients who have an inadequate response to standard treatment (including DMARDs). METHODS: A previous model was revised to evaluate the impact of biologics on both skin and joint disease and to include new evidence from the clinical review and evidence synthesis. Initial response to biologics was determined using the PsA response criteria. The impact of biologics on the arthritis component of the disease is then modelled via a change in the HAQ and the impact of the psoriasis component measured using the Psoriasis Area and Severity Index. RESULTS: For PsA patients with mild to moderate skin disease, the incremental cost-effectiveness ratio (ICER) for etanercept vs palliative care is around £18 000, and the ICER for infliximab vs etanercept is around £44 000 per quality-adjusted life year (QALY). Adalimumab is extendedly dominated. The probability that etanercept is cost effective is 0.436 at a threshold of £20 000 per QALY. Etanercept is also likely to be cost effective for patients with moderate to severe psoriasis or negligible skin involvement. CONCLUSIONS: Further investigation is required to reduce uncertainties around a number of model parameters, in particular the length of time over which biologics are assumed to be effective and the progression of HAQ on and off treatment.

Methods to elicit experts' beliefs over uncertain quantities: application to a cost effectiveness transition model of negative pressure wound therapy for severe pressure ulceration.

We can use decision models to estimate cost effectiveness, quantify uncertainty regarding the adoption decision and provide estimates of the value of further research. In many cases, the existence of only limited data with which to populate a decision model can mean that a cost-effectiveness analysis either does not proceed or may misrepresent the degree of uncertainty associated with model inputs. An example is the case of negative pressure wound therapy (NPWT) used to treat severe pressure ulceration, for which the evidence base is limited and sparse. There is, however, substantial practical experience of using this treatment and its comparators. We can capture this knowledge quantitatively to inform a cost-effectiveness model by eliciting beliefs from experts. This paper describes the design and conduct of an elicitation exercise to generate estimates of multiple uncertain model inputs and validate analytical assumptions for a decision model on the use of NPWT. In designing the exercise, the primary focus was the use of elicitation to inform decision models (multistate models), where representations of uncertain beliefs need to be probabilistically coherent. This paper demonstrates that it is feasible to collect formally elicited evidence to inform decision models.

Bridging the gap between methods research and the needs of policy makers: a review of the research priorities of the National Institute for Health and Clinical Excellence.

OBJECTIVES: The aim of this study was to establish a list of priority topics for methods research to support decision making at the National Institute for Health and Clinical Excellence (NICE). METHODS: Potential priorities for methods research topics were identified through a focused literature review, interviews, an email survey, a workshop and a Web-based feedback exercise. Participants were members of the NICE secretariat and its advisory bodies, representatives from academia, industry, and other organizations working closely with NICE. The Web exercise was open to anyone to complete but publicized among the above groups. RESULTS: A list of potential topics was collated. Priorities for further research differed according to the type of respondent and the extent to which they work directly with NICE. Priorities emerging from the group closest to NICE included: methodology for indirect and mixed treatment comparisons; synthesis of qualitative evidence; research relating to the use of quality-adjusted life-years (QALYs) in decision making; methods and empirical research for establishing the cost-effectiveness threshold; and determining how data on the uncertainty of effectiveness and cost-effectiveness data should be taken into account in the decision-making process. Priorities emerging from the broadest group of respondents (through the Web exercise) included: methods for extrapolating beyond evidence observed in trials, methods for capturing benefits not included in the QALY and methods to assess when technologies should be recommended in the context of further evidence gathering. CONCLUSIONS: Consideration needs to be given to the needs of those who use the outputs of research for decision making when determining priorities for future methods research.