A structured process for the validation of a decision-analytic model: application to a cost-effectiveness model for risk-stratified national breast screening.

BACKGROUND: Decision-makers require knowledge of the strengths and weaknesses of decision-analytic models used to evaluate healthcare interventions to be able to confidently use the results of such models to inform policy. A number of aspects of model validity have previously been described, but no systematic approach to assessing the validity of a model has been proposed. This study aimed to consolidate the different aspects of model validity into a step-by-step approach to assessing the strengths and weaknesses of a decision-analytic model. METHODS: A pre-defined set of steps were used to conduct the validation process of an exemplar early decision-analytic-model-based cost-effectiveness analysis of a risk-stratified national breast cancer screening programme [UK healthcare perspective; lifetime horizon; costs (£; 2021)]. Internal validation was assessed in terms of descriptive validity, technical validity and face validity. External validation was assessed in terms of operational validation, convergent validity (or corroboration) and predictive validity. RESULTS: The results outline the findings of each step of internal and external validation of the early decision-analytic-model and present the validated model (called 'MANC-RISK-SCREEN'). The positive aspects in terms of meeting internal validation requirements are shown together with the remaining limitations of MANC-RISK-SCREEN. CONCLUSION: Following a transparent and structured validation process, MANC-RISK-SCREEN has been shown to have satisfactory internal and external validity for use in informing resource allocation decision-making. We suggest that MANC-RISK-SCREEN can be used to assess the cost-effectiveness of exemplars of risk-stratified national breast cancer screening programmes (NBSP) from the UK perspective. IMPLICATIONS: A step-by-step process for conducting the validation of a decision-analytic model was developed for future use by health economists. Using this approach may help researchers to fully demonstrate the strengths and limitations of their model to decision-makers.

Estimating the Cost of 3 Risk Prediction Strategies for Potential Use in the United Kingdom National Breast Screening Program.

Background: Economic evaluations have suggested that risk-stratified breast cancer screening may be cost-effective but have used assumptions to estimate the cost of risk prediction. The aim of this study was to identify and quantify the resource use and associated costs required to introduce a breast cancer risk-stratification approach into the English national breast screening program. Methods: A micro-costing study, conducted alongside a cohort-based prospective trial (BC-PREDICT), identified the resource use and cost per individual (£; 2021 price year) of providing a risk-stratification strategy at a woman's first mammography. Costs were calculated for 3 risk-stratification approaches: Tyrer-Cuzick survey, Tyrer-Cuzick with Volpara breast-density measurement, and Tyrer-Cuzick with Volpara breast-density measurement and testing for 142 single nucleotide polymorphisms (SNP). Costs were determined for the intervention as implemented in the trial and in the health service. Results: The cost of providing the risk-stratification strategy was calculated to be £16.45 for the Tyrer-Cuzick survey approach, £21.82 for the Tyrer-Cuzick with Volpara breast-density measurement, and £102.22 for the Tyrer-Cuzick with Volpara breast-density measurement and SNP testing. Limitations: This study did not use formal expert elicitation methods to synthesize estimates. Conclusion: The costs of risk prediction using a survey and breast density measurement were low, but adding SNP testing substantially increases costs. Implementation issues present in the trial may also significantly increase the cost of risk prediction. Implications: This is the first study to robustly estimate the cost of risk-stratification for breast cancer screening. The cost of risk prediction using questionnaires and automated breast density measurement was low, but full economic evaluations including accurate costs are required to provide evidence of the cost-effectiveness of risk-stratified breast cancer screening. Highlights: Economic evaluations have suggested that risk-stratified breast cancer screening may be a cost-effective use of resources in the United Kingdom.Current estimates of the cost of risk stratification are based on pragmatic assumptions.This study provides estimates of the cost of risk stratification using 3 strategies and when these strategies are implemented perfectly and imperfectly in the health system.The cost of risk stratification is relatively low unless single nucleotide polymorphisms are included in the strategy.

Capturing the Impact of Constraints on the Cost-Effectiveness of Cell and Gene Therapies: A Systematic Review.

OBJECTIVE: Decision-makers need to resolve constraints on delivering cell and gene therapies to patients as these treatments move into routine care. This study aimed to investigate if, and how, constraints that affect the expected cost and health consequences of cell and gene therapies have been included in published examples of cost-effectiveness analyses (CEAs). METHOD: A systematic review identified CEAs of cell and gene therapies. Studies were identified from previous systematic reviews and by searching Medline and Embase until 21 January 2022. Constraints described qualitatively were categorised by theme and summarised by a narrative synthesis. Constraints evaluated in quantitative scenario analyses were appraised by whether they changed the decision to recommend treatment. RESULTS: Thirty-two CEAs of cell (n = 20) and gene therapies (n = 12) were included. Twenty-one studies described constraints qualitatively (70% cell therapy CEAs; 58% gene therapy CEAs). Qualitative constraints were categorised by four themes: single payment models; long-term affordability; delivery by providers; manufacturing capability. Thirteen studies assessed constraints quantitatively (60% cell therapy CEAs; 8% gene therapy CEAs). Two types of constraint were assessed quantitatively across four jurisdictions (USA, Canada, Singapore, The Netherlands): alternatives to single payment models (n = 9 scenario analyses); improving manufacturing (n = 12 scenario analyses). The impact on decision-making was determined by whether the estimated incremental cost-effectiveness ratios crossed a relevant cost-effectiveness threshold for each jurisdiction (outcome-based payment models: n = 25 threshold comparisons made, 28% decisions changed; improving manufacturing: n = 24 threshold comparisons made, 4% decisions changed). CONCLUSION: The net health impact of constraints is vital evidence to help decision-makers scale up the delivery of cell and gene therapies as patient volume increases and more advanced therapy medicinal products are launched. CEAs will be essential to quantify how constraints affect the cost-effectiveness of care, prioritise constraints to be resolved, and establish the value of strategies to implement cell and gene therapies by accounting for their health opportunity cost.

Quantifying the Impact of Capacity Constraints in Economic Evaluations: An Application in Precision Medicine.

BACKGROUND: Examples of precision medicine are complex interventions featuring both testing and treatment components. Because of this complexity, there are often barriers to the introduction of such interventions. Few economic evaluations attempt to determine the impact of these barriers on the cost-effectiveness of the intervention. This study presents a case study economic evaluation that illustrates how the value of implementation methods may be used to quantify the impact of capacity constraints in a decision-analytic model. METHODS: A baseline decision-analytic model-based economic evaluation of ALK mutation testing was reproduced from a published technology appraisal. Three constraints (commissioning awareness, localization of testing, and pathology laboratory capacity) were identified using qualitative interviews, parameterized, and incorporated into the model. Value of implementation methods were used alongside incremental cost-effectiveness ratios (ICERs) to quantify the impact on the cost-effectiveness and net monetary benefit (NMB) of each capacity constraint and from the 3 constraints combined. RESULTS: Each of the 3 capacity constraints resulted in a loss of NMB ranging from £7773 (0.1% of the total) per year for localized testing to £4,907,893 (77%) for a lack of awareness about commissioning ALK testing. When combined, the constraints resulted in a loss of NMB of £5,289,414 (83%). The localization and limited pathology capacity constraints slightly increased the ICER, but the lack of commissioning awareness constraint did not change the ICER. CONCLUSIONS: Capacity constraints may have a significant impact on the NMB produced by examples of precision medicine. Value of implementation methods can be used to quantify the impact of such constraints by combining the impact of the constraints on the cost-effectiveness of the intervention with the impact on the number of patients receiving the intervention. HIGHLIGHTS: While capacity constraints may prevent the use of precision medicine in clinical practice, economic evaluations rarely account for the impact of such barriers.This study demonstrates how constraints can be identified using qualitative methods and subsequently incorporated into decision-analytic models using quantitative value of implementation methods.In addition, this article demonstrates how value of implementation methods can be used to account for the impact of capacity constraints on the costs and benefits of an intervention as well as the number of patients receiving the intervention.In the case study presented herein, a capacity constraint reducing patient access to an example of precision medicine caused the biggest loss of net monetary benefit.Health economists should consider moving beyond incremental cost-effectiveness ratios to measures of total net monetary benefit to fully capture the impact of implementing precision medicine.

Implementing Interventions with Varying Marginal Cost-Effectiveness: An Application in Precision Medicine.

Purpose. A range of barriers may constrain the effective implementation of strategies to deliver precision medicine. If the marginal costs and consequences of precision medicine vary at different levels of implementation, then such variation will have an impact on relative cost-effectiveness. This study aimed to illustrate the importance and quantify the impact of varying marginal costs and benefits on the value of implementation for a case study in precision medicine. Methods. An existing method to calculate the value of implementation was adapted to allow marginal costs and consequences of introducing precision medicine into practice to vary across differing levels of implementation. This illustrative analysis used a case study based on a published decision-analytic model-based cost-effectiveness analysis of a 70-gene recurrence score (MammaPrint) for breast cancer. The impact of allowing for varying costs and benefits for the value of the precision medicine and of implementation strategies was illustrated graphically and numerically in both static and dynamic forms. Results. The increasing returns to scale exhibited by introducing this specific example of precision medicine mean that a minimum level of implementation (51%) is required for using the 70-gene recurrence score to be cost-effective at a defined threshold of €20,000 per quality-adjusted life year. The observed variation in net monetary benefit implies that the value of implementation strategies was dependent on the initial and ending levels of implementation in addition to the magnitude of the increase in patients receiving the 70-gene recurrence score. In dynamic models, incremental losses caused by low implementation accrue over time unless implementation is improved. Conclusions. Poor implementation of approaches to deliver precision medicine, identified to be cost-effective using decision-analytic model-based cost-effectiveness analysis, can have a significant economic impact on health systems. Developing and evaluating the economic impact of strategies to improve the implementation of precision medicine will potentially realize the more cost-effective use of health care budgets.

The role of information provision in economic evaluations of non-invasive prenatal testing: a systematic review.

BACKGROUND: Technological progress has led to changes in the antenatal screening programmes, most significantly the introduction of non-invasive prenatal testing (NIPT). The availability of a new type of testing changes the type of information that the parent(s) require before, during and after screening to mitigate anxiety about the testing process and results. OBJECTIVES: To identify the extent to which economic evaluations of NIPT have accounted for the need to provide information alongside testing and the associated costs and health outcomes of information provision. METHODS: A systematic review of economic evaluations of NIPTs (up to February 2018) was conducted. Medline, Embase, CINAHL and PsychINFO were searched using an electronic search strategy combining a published economic search filter (from NHS economic evaluations database) with terms related to NIPT and screening-related technologies. Data were extracted using the Consolidated Health Economic Evaluation Reporting Standards framework and the results were summarised as part of a narrative synthesis. RESULTS: A total of 12 economic evaluations were identified. The majority of evaluations (n = 10; 83.3%) involved cost effectiveness analysis. Only four studies (33.3%) included the cost of providing information about NIPT in their economic evaluation. Two studies considered the impact of test results on parents' quality of life by allowing utility decrements for different outcomes. Some studies suggested that the challenges of valuing information prohibited their inclusion in an economic evaluation. CONCLUSION: Economic evaluations of NIPTs need to account for the costs and outcomes associated with information provision, otherwise estimates of cost effectiveness may prove inaccurate.

Accounting for Capacity Constraints in Economic Evaluations of Precision Medicine: A Systematic Review.

BACKGROUND AND OBJECTIVE: Precision (stratified or personalised) medicine is underpinned by the premise that it is feasible to identify known heterogeneity using a specific test or algorithm in patient populations and to use this information to guide patient care to improve health and well-being. This study aimed to understand if, and how, previous economic evaluations of precision medicine had taken account of the impact of capacity constraints. METHODS: A meta-review was conducted of published systematic reviews of economic evaluations of precision medicine (test-treat interventions) and individual studies included in these reviews. Due to the volume of studies identified, a sample of papers published from 2007 to 2015 was collated. A narrative analysis identified whether potential capacity constraints were discussed qualitatively in the studies and, if relevant, which quantitative methods were used to account for capacity constraints. RESULTS: A total of 45 systematic reviews of economic evaluations of precision medicine were identified, from which 222 studies focusing on test-treat interventions, published between 2007 and 2015, were extracted. Of these studies, 33 (15%) qualitatively discussed the potential impact of capacity constraints, including budget constraints; quality of tests and the testing process; ease of use of tests in clinical practice; and decision uncertainty. Quantitative methods (nine studies) to account for capacity constraints included static methods such as capturing inefficiencies in trials or models and sensitivity analysis around model parameters; and dynamic methods, which allow the impact of capacity constraints on cost effectiveness to change over time. CONCLUSIONS: Understanding the cost effectiveness of precision medicine is necessary, but not sufficient, evidence for its successful implementation. There are currently few examples of evaluations that have quantified the impact of capacity constraints, which suggests an area of focus for future research.

Cost-effectiveness analyses of genetic and genomic diagnostic tests.

Developments in next-generation sequencing technologies have driven the clinical application of diagnostic tests that interrogate the whole genome, which offer the chance to diagnose rare inherited diseases or inform the targeting of therapies. New genomic diagnostic tests compete with traditional approaches to diagnosis, including the genetic testing of single genes and other clinical strategies, for finite health-care budgets. In this context, decision analytic model-based cost-effectiveness analysis is a useful method to help evaluate the costs versus consequences of introducing new health-care interventions. This Perspective presents key methodological, technical, practical and organizational challenges that must be considered by decision-makers responsible for the allocation of health-care resources to obtain robust and timely information about the relative cost-effectiveness of the increasing numbers of emerging genomic tests.

The Role of Information Provision in Economic Evaluations of Newborn Bloodspot Screening: A Systematic Review.

BACKGROUND: The extent to which economic evaluations have included the healthcare resource and outcome-related implications of information provision in national newborn bloodspot screening programmes (NBSPs) is not currently known. OBJECTIVES: To identify if, and how, information provision has been incorporated into published economic evaluations of NBSPs. METHODS: A systematic review of economic evaluations of NBSPs (up to November 2014) was conducted. Three electronic databases were searched (Ovid: Medline, Embase, CINAHL) using an electronic search strategy combining a published economic search filter with terms related to national NBSPs and screening-related technologies. These electronic searches were supplemented by searching the NHS Economic Evaluations Database (NHS EED) and hand-searching identified study reference lists. The results were tabulated and summarised as part of a narrative synthesis. RESULTS: A total of 27 economic evaluations [screening-related technologies (n = 11) and NBSPs (n = 16)] were identified. The majority of economic evaluations did not quantify the impact of information provision in terms of healthcare costs or outcomes. Five studies did include an estimate of the time cost associated with information provision. Four studies included a value to reflect the disutility associated with parental anxiety caused by false-positive results, which was used as a proxy for the impact of imperfect information. CONCLUSION: A limited evidence base currently quantifies the impact of information provision on the healthcare costs and impact on the users of NBSPs; the parents of newborns. We suggest that economic evaluations of expanded NBSPs need to take account of information provision otherwise the impact on healthcare costs and the outcomes for newborns and their parents may be underestimated.