Choosing a Metamodel of a Simulation Model for Uncertainty Quantification.

BACKGROUND: Metamodeling may substantially reduce the computational expense of individual-level state transition simulation models (IL-STM) for calibration, uncertainty quantification, and health policy evaluation. However, because of the lack of guidance and readily available computer code, metamodels are still not widely used in health economics and public health. In this study, we provide guidance on how to choose a metamodel for uncertainty quantification. METHODS: We built a simulation study to evaluate the prediction accuracy and computational expense of metamodels for uncertainty quantification using life-years gained (LYG) by treatment as the IL-STM outcome. We analyzed how metamodel accuracy changes with the characteristics of the simulation model using a linear model (LM), Gaussian process regression (GP), generalized additive models (GAMs), and artificial neural networks (ANNs). Finally, we tested these metamodels in a case study consisting of a probabilistic analysis of a lung cancer IL-STM. RESULTS: In a scenario with low uncertainty in model parameters (i.e., small confidence interval), sufficient numbers of simulated life histories, and simulation model runs, commonly used metamodels (LM, ANNs, GAMs, and GP) have similar, good accuracy, with errors smaller than 1% for predicting LYG. With a higher level of uncertainty in model parameters, the prediction accuracy of GP and ANN is superior to LM. In the case study, we found that in the worst case, the best metamodel had an error of about 2.1%. CONCLUSION: To obtain good prediction accuracy, in an efficient way, we recommend starting with LM, and if the resulting accuracy is insufficient, we recommend trying ANNs and eventually also GP regression.

Comparative effectiveness of prostate cancer screening between the ages of 55 and 69 years followed by active surveillance.

BACKGROUND: Because of the recent grade C draft recommendation by the US Preventive Services Task Force (USPSTF) for prostate cancer screening between the ages of 55 and 69 years, there is a need to determine whether this could be cost-effective in a US population setting. METHODS: This study used a microsimulation model of screening and active surveillance (AS), based on data from the European Randomized Study of Screening for Prostate Cancer and the Surveillance, Epidemiology, and End Results Program, for the natural history of prostate cancer and Johns Hopkins AS cohort data to inform the probabilities of referral to treatment during AS. A cohort of 10 million men, based on US life tables, was simulated. The lifetime costs and effects of screening between the ages of 55 and 69 years with different screening frequencies and AS protocols were projected, and their cost-effectiveness was determined. RESULTS: Quadrennial screening between the ages of 55 and 69 years (55, 59, 63, and 67 years) with AS for men with low-risk cancers (ie, those with a Gleason score of 6 or lower) and yearly biopsies or triennial biopsies resulted in an incremental cost per quality-adjusted life-year (QALY) of $51,918 or $69,380, respectively. Most policies in which screening was followed by immediate treatment were dominated. In most sensitivity analyses, this study found a policy with which the cost per QALY remained below $100,000. CONCLUSIONS: Prostate-specific antigen-based prostate cancer screening in the United States between the ages of 55 and 69 years, as recommended by the USPSTF, may be cost-effective at a $100,000 threshold but only with a quadrennial screening frequency and with AS offered to all low-risk men. Cancer 2018;124:507-13. © 2017 American Cancer Society.