Net Monetary Benefit Lines Augmented with Value-of-Information Measures to Present the Results of Economic Evaluations under Uncertainty.

BACKGROUND: Methods to present the result of cost-effectiveness analyses under parameter uncertainty include cost-effectiveness planes (CEPs), cost-effectiveness acceptability curves/frontier (CEACs/CEAF), expected loss curves (ELCs), and net monetary benefit (NMB) lines. We describe how NMB lines can be augmented to present NMB values that could be achieved by reducing or resolving parameter uncertainty. We evaluated the ability of these methods to correctly 1) identify the alternative with the highest expected NMB and 2) communicate the magnitude of parameter and decision uncertainty. METHODS: We considered 4 hypothetical decision problems representing scenarios with high variance or correlated cost and effect estimates and alternatives with similar cost-effectiveness ratios. We used these decision problems to demonstrate the limitations of existing methods and the potential of augmented NMB lines to resolve these issues. RESULTS: CEPs and CEACs/CEAF could falsely imply the lack of sufficient evidence to identify the optimal option if cost and effect estimates have high variance, are correlated across alternatives, or when alternatives have similar cost-effectiveness ratios. The augmented NMB lines and ELCs can correctly identify the option with the highest expected NMB and communicate the potential benefit of resolving uncertainties. Like ELCs, the augmented NMB lines provide information about the value of resolving parameter uncertainties, but augmented NMB lines may be easier to interpret for decision makers. CONCLUSIONS: Our analysis supports recommending the augment NMB lines as an important method to present the results of economic evaluation studies under parameter uncertainty. HIGHLIGHTS: The results of cost-effectiveness analyses (CEAs) when the cost and effect estimates of alternatives are uncertain are commonly presented using cost-effectiveness planes (CEPs), cost-effectiveness acceptability curves/frontier (CEACs/CEAF), and expected loss curves (ELCs).Although currently not often used, net monetary benefit (NMB) lines could present the results of cost-effectiveness to identify the alternative with the highest expected NMB values given the current level of uncertainty. Furthermore, NMB lines can be augmented to 1) show metrics of value of information, which measure the value of additional research to reduce or eliminate the decision uncertainty, and 2) display the confidence intervals along the NMB lines to ensure that NMB values are estimated accurately using a sufficiently large number of parameter samples.Using several decision problems, we demonstrate the limitation of existing methods to present the results of CEAs under parameter uncertainty and how augmented NMB lines could resolve these issues.Our analysis supports recommending augmented NMB lines as an important method to present the results of CEA under uncertainty since they 1) correctly identify the alternative with the highest expected NMB value given the current evidence, 2) provide information about the potential value of additional research to improve the decision by reducing or resolving uncertainty in model parameters, 3) assist the analysis to visually ensure that enough parameter samples are used to estimate the expected NMB of alternatives, and 4) are easier to interpret for decision makers compared with other methods.

Making Drug Approval Decisions in the Face of Uncertainty: Cumulative Evidence versus Value of Information.

BACKGROUND: The COVID-19 pandemic underscored the criticality and complexity of decision making for novel treatment approval and further research. Our study aims to assess potential decision-making methodologies, an evaluation vital for refining future public health crisis responses. METHODS: We compared 4 decision-making approaches to drug approval and research: the Food and Drug Administration's policy decisions, cumulative meta-analysis, a prospective value-of-information (VOI) approach (using information available at the time of decision), and a reference standard (retrospective VOI analysis using information available in hindsight). Possible decisions were to reject, accept, provide emergency use authorization, or allow access to new therapies only in research settings. We used monoclonal antibodies provided to hospitalized COVID-19 patients as a case study, examining the evidence from September 2020 to December 2021 and focusing on each method's capacity to optimize health outcomes and resource allocation. RESULTS: Our findings indicate a notable discrepancy between policy decisions and the reference standard retrospective VOI approach with expected losses up to $269 billion USD, suggesting suboptimal resource use during the wait for emergency use authorization. Relying solely on cumulative meta-analysis for decision making results in the largest expected loss, while the policy approach showed a loss up to $16 billion and the prospective VOI approach presented the least loss (up to $2 billion). CONCLUSION: Our research suggests that incorporating VOI analysis may be particularly useful for research prioritization and treatment implementation decisions during pandemics. While the prospective VOI approach was favored in this case study, further studies should validate the ideal decision-making method across various contexts. This study's findings not only enhance our understanding of decision-making strategies during a health crisis but also provide a potential framework for future pandemic responses. HIGHLIGHTS: This study reviews discrepancies between a reference standard (retrospective VOI, using hindsight information) and 3 conceivable real-time approaches to research-treatment decisions during a pandemic, suggesting suboptimal use of resources.Of all prospective decision-making approaches considered, VOI closely mirrored the reference standard, yielding the least expected value loss across our study timeline.This study illustrates the possible benefit of VOI results and the need for evidence accumulation accompanied by modeling in health technology assessment for emerging therapies.

A Guide to an Iterative Approach to Model-Based Decision Making in Health and Medicine: An Iterative Decision-Making Framework.

Decision makers frequently face decisions about optimal resource allocation. A model-based economic evaluation can be used to guide decision makers in their choices by systematically evaluating the magnitude of expected health effects and costs of decision options and by making trade-offs explicit. We provide a guide to an iterative approach to the medical decision-making process by following a coherent framework, and outline the overarching iterative steps of model-based decision making. We systematized the framework by performing three steps. First, we compiled the existing guidelines provided by the ISPOR-SMDM Modeling Good Research Practices Task Force, and the ISPOR Value of Information Task Force. Second, we identified other previous work related to frameworks and guidelines for model-based decision analyses through a literature search in PubMed. Third, we assessed the role of the evidence and iterative process in decision making and formalized key steps in a model-based decision-making framework. We provide guidance on an iterative approach to medical decision making by applying the compiled iterative model-based decision-making framework. The framework formally combines the decision problem conceptualization (Part I), the model conceptualization and development (Part II), and the process of model-based decision analysis (Part III). Following the overarching steps of the framework ensures compliance to the principles of evidence-based medicine and regular updates of the evidence, given that value of information analysis represents an essential component of model-based decision analysis in the framework. Following the provided guide and the steps outlined in the framework can help inform various health care decisions, and therefore it has the potential to improve decision making.