Guidance on the use of complex systems models for economic evaluations of public health interventions.

To help health economic modelers respond to demands for greater use of complex systems models in public health. To propose identifiable features of such models and support researchers to plan public health modeling projects using these models. A working group of experts in complex systems modeling and economic evaluation was brought together to develop and jointly write guidance for the use of complex systems models for health economic analysis. The content of workshops was informed by a scoping review. A public health complex systems model for economic evaluation is defined as a quantitative, dynamic, non-linear model that incorporates feedback and interactions among model elements, in order to capture emergent outcomes and estimate health, economic and potentially other consequences to inform public policies. The guidance covers: when complex systems modeling is needed; principles for designing a complex systems model; and how to choose an appropriate modeling technique. This paper provides a definition to identify and characterize complex systems models for economic evaluations and proposes guidance on key aspects of the process for health economics analysis. This document will support the development of complex systems models, with impact on public health systems policy and decision making.

A model-based economic analysis of the CFHealthHub intervention to support adherence to inhaled medications for people with cystic fibrosis in the UK.

BACKGROUND: Adherence to preventative inhaled therapies in people with cystic fibrosis (CF) is low, resulting in potentially avoidable health losses and the need for costly rescue therapies. OBJECTIVES: To estimate the cost-effectiveness of the CFHealthHub (CFHH) intervention to support adherence to inhaled medications. METHODS: A state transition model was developed to assess the cost-effectiveness of the CFHH intervention versus usual care from the perspective of the UK National Health Service and Personal Social Services over a lifetime horizon. Costs and health outcomes were discounted at a rate of 3.5 percent per annum. Costs were valued at 2021/22 prices. The model structure includes health states defined by survival status, level of lung function, and transplant history. Treatment effects were modeled by changing the probabilities of transitioning between lung function states and reducing exacerbation rates. Model parameters were informed by the CFHH trial, CF Registry data, routine cost databases, literature, and expert opinion. Deterministic and probabilistic sensitivity analyses were undertaken to assess uncertainty. RESULTS: The CFHH intervention is expected to generate additional health gains and cost savings compared with usual care. Assuming that it is delivered for 10 years, the CFHH intervention is expected to generate 0.17 additional quality-adjusted life years and cost savings of GBP 1,600 (EUR 1,662) per patient. CONCLUSIONS: The CFHH intervention is expected to dominate usual care, irrespective of the duration over which the intervention is delivered. The modeled benefits and cost savings are smaller than initially expected and are sensitive to relative treatment effects on lung function.

A systematic review of meta-analyses assessing the validity of tumour response endpoints as surrogates for progression-free or overall survival in cancer.

BACKGROUND: Tumour response endpoints, such as overall response rate (ORR) and complete response (CR), are increasingly used in cancer trials. However, the validity of response-based surrogates is unclear. This systematic review summarises meta-analyses assessing the association between response-based outcomes and overall survival (OS), progression-free survival (PFS) or time-to-progression (TTP). METHODS: Five databases were searched to March 2019. Meta-analyses reporting correlation or regression between response-based outcomes and OS, PFS or TTP were summarised. RESULTS: The systematic review included 63 studies across 20 cancer types, most commonly non-small cell lung cancer (NSCLC), colorectal cancer (CRC) and breast cancer. The strength of association between ORR or CR and either PFS or OS varied widely between and within studies, with no clear pattern by cancer type. The association between ORR and OS appeared weaker and more variable than that between ORR and PFS, both for associations between absolute endpoints and associations between treatment effects. CONCLUSIONS: This systematic review suggests that response-based endpoints, such as ORR and CR, may not be reliable surrogates for PFS or OS. Where it is necessary to use tumour response to predict treatment effects on survival outcomes, it is important to fully reflect all statistical uncertainty in the surrogate relationship.

Prehospital early warning scores for adults with suspected sepsis: the PHEWS observational cohort and decision-analytic modelling study.

Background: Guidelines for sepsis recommend treating those at highest risk within 1 hour. The emergency care system can only achieve this if sepsis is recognised and prioritised. Ambulance services can use prehospital early warning scores alongside paramedic diagnostic impression to prioritise patients for treatment or early assessment in the emergency department. Objectives: To determine the accuracy, impact and cost-effectiveness of using early warning scores alongside paramedic diagnostic impression to identify sepsis requiring urgent treatment. Design: Retrospective diagnostic cohort study and decision-analytic modelling of operational consequences and cost-effectiveness. Setting: Two ambulance services and four acute hospitals in England. Participants: Adults transported to hospital by emergency ambulance, excluding episodes with injury, mental health problems, cardiac arrest, direct transfer to specialist services, or no vital signs recorded. Interventions: Twenty-one early warning scores used alongside paramedic diagnostic impression, categorised as sepsis, infection, non-specific presentation, or other specific presentation. Main outcome measures: Proportion of cases prioritised at the four hospitals; diagnostic accuracy for the sepsis-3 definition of sepsis and receiving urgent treatment (primary reference standard); daily number of cases with and without sepsis prioritised at a large and a small hospital; the minimum treatment effect associated with prioritisation at which each strategy would be cost-effective, compared to no prioritisation, assuming willingness to pay £20,000 per quality-adjusted life-year gained. Results: Data from 95,022 episodes involving 71,204 patients across four hospitals showed that most early warning scores operating at their pre-specified thresholds would prioritise more than 10% of cases when applied to non-specific attendances or all attendances. Data from 12,870 episodes at one hospital identified 348 (2.7%) with the primary reference standard. The National Early Warning Score, version 2 (NEWS2), had the highest area under the receiver operating characteristic curve when applied only to patients with a paramedic diagnostic impression of sepsis or infection (0.756, 95% confidence interval 0.729 to 0.783) or sepsis alone (0.655, 95% confidence interval 0.63 to 0.68). None of the strategies provided high sensitivity (> 0.8) with acceptable positive predictive value (> 0.15). NEWS2 provided combinations of sensitivity and specificity that were similar or superior to all other early warning scores. Applying NEWS2 to paramedic diagnostic impression of sepsis or infection with thresholds of > 4, > 6 and > 8 respectively provided sensitivities and positive predictive values (95% confidence interval) of 0.522 (0.469 to 0.574) and 0.216 (0.189 to 0.245), 0.447 (0.395 to 0.499) and 0.274 (0.239 to 0.313), and 0.314 (0.268 to 0.365) and 0.333 (confidence interval 0.284 to 0.386). The mortality relative risk reduction from prioritisation at which each strategy would be cost-effective exceeded 0.975 for all strategies analysed. Limitations: We estimated accuracy using a sample of older patients at one hospital. Reliable evidence was not available to estimate the effectiveness of prioritisation in the decision-analytic modelling. Conclusions: No strategy is ideal but using NEWS2, in patients with a paramedic diagnostic impression of infection or sepsis could identify one-third to half of sepsis cases without prioritising unmanageable numbers. No other score provided clearly superior accuracy to NEWS2. Research is needed to develop better definition, diagnosis and treatments for sepsis. Study registration: This study is registered as Research Registry (reference: researchregistry5268). Funding: This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment programme (NIHR award ref: 17/136/10) and is published in full in Health Technology Assessment; Vol. 28, No. 16. See the NIHR Funding and Awards website for further award information.

Sepsis is a life-threatening condition in which an abnormal response to infection causes heart, lung or kidney failure. People with sepsis need urgent treatment. They need to be prioritised at the emergency department rather than waiting in the queue. Paramedics attempt to identify people with possible sepsis using an early warning score (based on simple measurements, such as blood pressure and heart rate) alongside their impression of the patient's diagnosis. They can then alert the hospital to assess the patient quickly. However, an inaccurate early warning score might miss cases of sepsis or unnecessarily prioritise people without sepsis. We aimed to measure how accurately early warning scores identified people with sepsis when used alongside paramedic diagnostic impression. We collected data from 71,204 people that two ambulance services transported to four different hospitals in 2019. We recorded paramedic diagnostic impressions and calculated early warning scores for each patient. At one hospital, we linked ambulance records to hospital records and identified who had sepsis. We then calculated the accuracy of using the scores alongside diagnostic impression to diagnose sepsis. Finally, we used modelling to predict how many patients (with and without sepsis) paramedics would prioritise using different strategies based on early warning scores and diagnostic impression. We found that none of the currently available early warning scores were ideal. When they were applied to all patients, they prioritised too many people. When they were only applied to patients whom the paramedics thought had infection, they missed many cases of sepsis. The NEWS2, score, which ambulance services already use, was as good as or better than all the other scores we studied. We found that using the NEWS2, score in people with a paramedic impression of infection could achieve a reasonable balance between prioritising too many patients and avoiding missing patients with sepsis.

Modelling approaches for histology-independent cancer drugs to inform NICE appraisals: a systematic review and decision-framework.

BACKGROUND: The first histology-independent marketing authorisation in Europe was granted in 2019. This was the first time that a cancer treatment was approved based on a common biomarker rather than the location in the body at which the tumour originated. This research aims to explore the implications for National Institute for Health and Care Excellence appraisals. METHODS: Targeted reviews were undertaken to determine the type of evidence that is likely to be available at the point of marketing authorisation and the analyses required to support National Institute for Health and Care Excellence appraisals. Several challenges were identified concerning the design and conduct of trials for histology-independent products, the greater levels of heterogeneity within the licensed population and the use of surrogate end points. We identified approaches to address these challenges by reviewing key statistical literature that focuses on the design and analysis of histology-independent trials and by undertaking a systematic review to evaluate the use of response end points as surrogate outcomes for survival end points. We developed a decision framework to help to inform approval and research policies for histology-independent products. The framework explored the uncertainties and risks associated with different approval policies, including the role of further data collection, pricing schemes and stratified decision-making. RESULTS: We found that the potential for heterogeneity in treatment effects, across tumour types or other characteristics, is likely to be a central issue for National Institute for Health and Care Excellence appraisals. Bayesian hierarchical methods may serve as a useful vehicle to assess the level of heterogeneity across tumours and to estimate the pooled treatment effects for each tumour, which can inform whether or not the assumption of homogeneity is reasonable. Our review suggests that response end points may not be reliable surrogates for survival end points. However, a surrogate-based modelling approach, which captures all relevant uncertainty, may be preferable to the use of immature survival data. Several additional sources of heterogeneity were identified as presenting potential challenges to National Institute for Health and Care Excellence appraisal, including the cost of testing, baseline risk, quality of life and routine management costs. We concluded that a range of alternative approaches will be required to address different sources of heterogeneity to support National Institute for Health and Care Excellence appraisals. An exemplar case study was developed to illustrate the nature of the assessments that may be required. CONCLUSIONS: Adequately designed and analysed basket studies that assess the homogeneity of outcomes and allow borrowing of information across baskets, where appropriate, are recommended. Where there is evidence of heterogeneity in treatment effects and estimates of cost-effectiveness, consideration should be given to optimised recommendations. Routine presentation of the scale of the consequences of heterogeneity and decision uncertainty may provide an important additional approach to the assessments specified in the current National Institute for Health and Care Excellence methods guide. FURTHER RESEARCH: Further exploration of Bayesian hierarchical methods could help to inform decision-makers on whether or not there is sufficient evidence of homogeneity to support pooled analyses. Further research is also required to determine the appropriate basis for apportioning genomic testing costs where there are multiple targets and to address the challenges of uncontrolled Phase II studies, including the role and use of surrogate end points. FUNDING: This project was funded by the National Institute for Health Research (NIHR) Evidence Synthesis programme and will be published in full in Health Technology Assessment; Vol. 25, No. 76. See the NIHR Journals Library website for further project information.

In May 2017, the US Food and Drug Administration granted the first approval for a cancer treatment based on a common biomarker rather than the location in the body at which the tumour originated (the tumour site); that is, a site-agnostic or 'histology-independent' indication was granted. Research from the National Institute for Health and Care Excellence suggests that there are approximately 20 technologies currently in development for histology-independent indications. The first marketing authorisation was granted in Europe in 2019. Histology-independent treatments have the potential to have important effects in patient populations for whom there are currently limited or no available treatment options. However, it is also important to ensure that the use of these treatments in the NHS is supported by systematic and robust assessments of clinical evidence (i.e. how well the medicine or treatment works) and economic evidence (i.e. the medicine's value for money). These assessments are undertaken by the National Institute for Health and Care Excellence, usually for treatments targeting specific tumours sites. However, a histology-independent marketing authorisation would probably include many tumour sites and it is not possible for the National Institute for Health and Care Excellence to conduct a separate assessment for each tumour site for which the treatment could be beneficial. As a result, the National Institute for Health and Care Excellence needs to consider how these products can be appropriately assessed without creating unnecessary delays in patient access. This research explores the extent to which the National Institute for Health and Care Excellence's existing approaches for assessing clinical and economic value can be applied to histology-independent indications, and any changes that might be required. We explore the nature and amount of evidence that is typically available at the point of initial marketing authorisation and develop recommendations to establish the evidence and analyses required to help inform the National Institute for Health and Care Excellence's decisions. We use case studies to highlight possible challenges and to explore ways that these challenges might be addressed. This research will help to inform future National Institute for Health and Care Excellence policy on how to appraise cancer drugs with histology-independent indications. It will also inform the development of guidance for those developing these treatments to help their understanding of the clinical effectiveness and cost-effectiveness assessments that will be required to inform the National Institute for Health and Care Excellence's appraisals.

Correction to: The Future of Precision Medicine: Potential Impacts for Health Technology Assessment.

The article The Future of Precision Medicine: Potential Impacts for Health Technology Assessment written by James Love­Koh, Alison Peel Juan, Carlos Rejon­Parrilla, KateAnastasia Chalkidou, Hannah Wood, Matthew Taylor was originally published electronically on the publisher's internet portal (currently Springer Link) on [13th July, 2018] with incorrect spelling of the co-author "Juan Carlos Rejon-Parilla". The correct spelling is "Juan Carlos Rejon-Parrilla".

The Future of Precision Medicine: Potential Impacts for Health Technology Assessment.

OBJECTIVE: Precision medicine allows healthcare interventions to be tailored to groups of patients based on their disease susceptibility, diagnostic or prognostic information, or treatment response. We analysed what developments are expected in precision medicine over the next decade and considered the implications for health technology assessment (HTA) agencies. METHODS: We performed a pragmatic literature search to account for the large size and wide scope of the precision medicine literature. We refined and enriched these results with a series of expert interviews up to 1 h in length, including representatives from HTA agencies, research councils and researchers designed to cover a wide spectrum of precision medicine applications and research. RESULTS: We identified 31 relevant papers and interviewed 13 experts. We found that three types of precision medicine are expected to emerge in clinical practice: complex algorithms, digital health applications and 'omics'-based tests. These are expected to impact upon each stage of the HTA process, from scoping and modelling through to decision-making and review. The complex and uncertain treatment pathways associated with patient stratification and fast-paced technological innovation are central to these effects. DISCUSSION: Innovation in precision medicine promises substantial benefits but will change the way in which some health services are delivered and evaluated. The shelf life of guidance may decrease, structural uncertainty may increase and new equity considerations will emerge. As biomarker discovery accelerates and artificial intelligence-based technologies emerge, refinements to the methods and processes of evidence assessments will help to adapt and maintain the objective of investing in healthcare that is value for money.