Characteristics of a cost-effective blood test for colorectal cancer screening.

BACKGROUND: Blood-based biomarker tests can potentially change the landscape of colorectal cancer (CRC) screening. We characterize the conditions under which blood test screening would be as effective and cost-effective as annual fecal immunochemical testing (FIT) or decennial colonoscopy. METHODS: We used the three CISNET-Colon models to compare scenarios of no screening, annual FIT, decennial colonoscopy, and a blood test meeting CMS coverage criteria (74% CRC sensitivity and 90% specificity). We varied the sensitivity to detect CRC (74%-92%), advanced adenomas (AAs, 10%-50%), screening interval (1-3 years), and test cost ($25-$500). Primary outcomes included quality-adjusted life-years gained (QALYG) from screening and costs for an US average-risk 45-year-old cohort. RESULTS: Annual FIT yielded 125-163 QALYG per 1,000 at a cost of $3,811-5,384 per person, whereas colonoscopy yielded 132-177 QALYG at a cost of $5,375-7,031 per person. A blood test with 92% CRC sensitivity and 50% AA sensitivity yielded 117-162 QALYG if used every three years and 133-173 QALYG if used every year but would not be cost-effective if priced above $125 per test. If used every three years, a $500 blood test only meeting CMS coverage criteria yielded 83-116 QALYG, at a cost of $8,559-9,413 per person. CONCLUSION: Blood tests that only meet CMS coverage requirements should not be recommended to patients who would otherwise undergo screening by colonoscopy or FIT due to lower benefit. Blood tests need higher AA sensitivity (above 40%) and lower costs (below $125) to be cost-effective.

Emulator-Based Bayesian Calibration of the CISNET Colorectal Cancer Models.

PURPOSE: To calibrate Cancer Intervention and Surveillance Modeling Network (CISNET)'s SimCRC, MISCAN-Colon, and CRC-SPIN simulation models of the natural history colorectal cancer (CRC) with an emulator-based Bayesian algorithm and internally validate the model-predicted outcomes to calibration targets. METHODS: We used Latin hypercube sampling to sample up to 50,000 parameter sets for each CISNET-CRC model and generated the corresponding outputs. We trained multilayer perceptron artificial neural networks (ANNs) as emulators using the input and output samples for each CISNET-CRC model. We selected ANN structures with corresponding hyperparameters (i.e., number of hidden layers, nodes, activation functions, epochs, and optimizer) that minimize the predicted mean square error on the validation sample. We implemented the ANN emulators in a probabilistic programming language and calibrated the input parameters with Hamiltonian Monte Carlo-based algorithms to obtain the joint posterior distributions of the CISNET-CRC models' parameters. We internally validated each calibrated emulator by comparing the model-predicted posterior outputs against the calibration targets. RESULTS: The optimal ANN for SimCRC had 4 hidden layers and 360 hidden nodes, MISCAN-Colon had 4 hidden layers and 114 hidden nodes, and CRC-SPIN had 1 hidden layer and 140 hidden nodes. The total time for training and calibrating the emulators was 7.3, 4.0, and 0.66 h for SimCRC, MISCAN-Colon, and CRC-SPIN, respectively. The mean of the model-predicted outputs fell within the 95% confidence intervals of the calibration targets in 98 of 110 for SimCRC, 65 of 93 for MISCAN, and 31 of 41 targets for CRC-SPIN. CONCLUSIONS: Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis, such as the CISNET CRC models. In this work, we present a step-by-step guide to constructing emulators for calibrating 3 realistic CRC individual-level models using a Bayesian approach. HIGHLIGHTS: We use artificial neural networks (ANNs) to build emulators that surrogate complex individual-based models to reduce the computational burden in the Bayesian calibration process.ANNs showed good performance in emulating the CISNET-CRC microsimulation models, despite having many input parameters and outputs.Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis.This work aims to support health decision scientists who want to quantify the uncertainty of calibrated parameters of computationally intensive simulation models under a Bayesian framework.

Emulator-based Bayesian calibration of the CISNET colorectal cancer models.

Purpose: To calibrate Cancer Intervention and Surveillance Modeling Network (CISNET) 's SimCRC, MISCAN-Colon, and CRC-SPIN simulation models of the natural history colorectal cancer (CRC) with an emulator-based Bayesian algorithm and internally validate the model-predicted outcomes to calibration targets. Methods: We used Latin hypercube sampling to sample up to 50,000 parameter sets for each CISNET-CRC model and generated the corresponding outputs. We trained multilayer perceptron artificial neural networks (ANN) as emulators using the input and output samples for each CISNET-CRC model. We selected ANN structures with corresponding hyperparameters (i.e., number of hidden layers, nodes, activation functions, epochs, and optimizer) that minimize the predicted mean square error on the validation sample. We implemented the ANN emulators in a probabilistic programming language and calibrated the input parameters with Hamiltonian Monte Carlo-based algorithms to obtain the joint posterior distributions of the CISNET-CRC models' parameters. We internally validated each calibrated emulator by comparing the model-predicted posterior outputs against the calibration targets. Results: The optimal ANN for SimCRC had four hidden layers and 360 hidden nodes, MISCAN-Colon had 4 hidden layers and 114 hidden nodes, and CRC-SPIN had one hidden layer and 140 hidden nodes. The total time for training and calibrating the emulators was 7.3, 4.0, and 0.66 hours for SimCRC, MISCAN-Colon, and CRC-SPIN, respectively. The mean of the model-predicted outputs fell within the 95% confidence intervals of the calibration targets in 98 of 110 for SimCRC, 65 of 93 for MISCAN, and 31 of 41 targets for CRC-SPIN. Conclusions: Using ANN emulators is a practical solution to reduce the computational burden and complexity for Bayesian calibration of individual-level simulation models used for policy analysis, like the CISNET CRC models. In this work, we present a step-by-step guide to constructing emulators for calibrating three realistic CRC individual-level models using a Bayesian approach.

Projected long-term effects of colorectal cancer screening disruptions following the COVID-19 pandemic.

The aftermath of the initial phase of the COVID-19 pandemic may contribute to the widening of disparities in colorectal cancer (CRC) outcomes due to differential disruptions to CRC screening. This comparative microsimulation analysis uses two CISNET CRC models to simulate the impact of ongoing screening disruptions induced by the COVID-19 pandemic on long-term CRC outcomes. We evaluate three channels through which screening was disrupted: delays in screening, regimen switching, and screening discontinuation. The impact of these disruptions on long-term CRC outcomes was measured by the number of life-years lost due to CRC screening disruptions compared to a scenario without any disruptions. While short-term delays in screening of 3-18 months are predicted to result in minor life-years loss, discontinuing screening could result in much more significant reductions in the expected benefits of screening. These results demonstrate that unequal recovery of screening following the pandemic can widen disparities in CRC outcomes and emphasize the importance of ensuring equitable recovery to screening following the pandemic.

Robustness Analysis of Colorectal Cancer Colonoscopy Screening Strategies.

Colorectal Cancer (CRC) is a leading cause of cancer deaths in the United States. Despite significant overall declines in CRC incidence and mortality, there has been an alarming increase in CRC among people younger than 50. This study uses an established microsimulation model, CRC-SPIN, to perform a 'stress test' of colonoscopy screening strategies. First, we expand CRC-SPIN to include birth-cohort effects. Second, we estimate natural history model parameters via Incremental Mixture Approximate Bayesian Computation (IMABC) for two model versions to characterize uncertainty while accounting for increased early CRC onset. Third, we simulate 26 colonoscopy screening strategies across the posterior distribution of estimated model parameters, assuming four different colonoscopy sensitivities (104 total scenarios). We find that model projections of screening benefit are highly dependent on natural history and test sensitivity assumptions, but in this stress test, the policy recommendations are robust to the uncertainties considered.

Expanding Medicaid to Reduce Human Immunodeficiency Virus Transmission in Houston, Texas: Insights From a Modeling Study.

CONTEXT: Medicaid expansion has been nationally shown to improve engagement in the human immunodeficiency virus (HIV) treatment and prevention continua, which are vital steps to stopping the HIV epidemic. New HIV infections in the United States are disproportionately concentrated among young Black men who have sex with men (YBMSM). Houston, TX, is the most populous city in the Southern United States with a racially/ethnically diverse population that is located in 1 of 11 US states that have not yet expanded Medicaid coverage as of 2021. METHODS: An agent-based model that incorporated the sexual networks of YBMSM was used to simulate improved antiretroviral treatment and pre-exposure prophylaxis (PrEP) engagement through Medicaid expansion in Houston, TX. Analyses considered the HIV incidence (number of new infections and as a rate metric) among YBMSM over the next 10 years under Medicaid expansion as the primary outcome. Additional scenarios, involving viral suppression and PrEP uptake above the projected levels achieved under Medicaid expansion, were also simulated. RESULTS: The baseline model projected an HIV incidence rate of 4.96 per 100 person years (py) and about 368 new annual HIV infections in the 10th year. Improved HIV treatment and prevention continua engagement under Medicaid expansion resulted in a 14.9% decline in the number of annual new HIV infections in the 10th year. Increasing viral suppression by an additional 15% and PrEP uptake by 30% resulted in a 44.0% decline in new HIV infections in the 10th year, and a 27.1% decline in cumulative infections across the 10 years of the simulated intervention. FINDINGS: Simulation results indicate that Medicaid expansion has the potential to reduce HIV incidence among YBMSM in Houston. Achieving HIV elimination objectives, however, might require additional effective measures to increase antiretroviral treatment and PrEP uptake beyond the projected improvements under expanded Medicaid.

Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models.

Background: We evaluated the implications of different approaches to characterize the uncertainty of calibrated parameters of microsimulation decision models (DMs) and quantified the value of such uncertainty in decision making. Methods: We calibrated the natural history model of CRC to simulated epidemiological data with different degrees of uncertainty and obtained the joint posterior distribution of the parameters using a Bayesian approach. We conducted a probabilistic sensitivity analysis (PSA) on all the model parameters with different characterizations of the uncertainty of the calibrated parameters. We estimated the value of uncertainty of the various characterizations with a value of information analysis. We conducted all analyses using high-performance computing resources running the Extreme-scale Model Exploration with Swift (EMEWS) framework. Results: The posterior distribution had a high correlation among some parameters. The parameters of the Weibull hazard function for the age of onset of adenomas had the highest posterior correlation of -0.958. When comparing full posterior distributions and the maximum-a-posteriori estimate of the calibrated parameters, there is little difference in the spread of the distribution of the CEA outcomes with a similar expected value of perfect information (EVPI) of $653 and $685, respectively, at a willingness-to-pay (WTP) threshold of $66,000 per quality-adjusted life year (QALY). Ignoring correlation on the calibrated parameters' posterior distribution produced the broadest distribution of CEA outcomes and the highest EVPI of $809 at the same WTP threshold. Conclusion: Different characterizations of the uncertainty of calibrated parameters affect the expected value of eliminating parametric uncertainty on the CEA. Ignoring inherent correlation among calibrated parameters on a PSA overestimates the value of uncertainty.

Sequentially calibrating a Bayesian microsimulation model to incorporate new information and assumptions.

BACKGROUND: Microsimulation models are mathematical models that simulate event histories for individual members of a population. They are useful for policy decisions because they simulate a large number of individuals from an idealized population, with features that change over time, and the resulting event histories can be summarized to describe key population-level outcomes. Model calibration is the process of incorporating evidence into the model. Calibrated models can be used to make predictions about population trends in disease outcomes and effectiveness of interventions, but calibration can be challenging and computationally expensive. METHODS: This paper develops a technique for sequentially updating models to take full advantage of earlier calibration results, to ultimately speed up the calibration process. A Bayesian approach to calibration is used because it combines different sources of evidence and enables uncertainty quantification which is appealing for decision-making. We develop this method in order to re-calibrate a microsimulation model for the natural history of colorectal cancer to include new targets that better inform the time from initiation of preclinical cancer to presentation with clinical cancer (sojourn time), because model exploration and validation revealed that more information was needed on sojourn time, and that the predicted percentage of patients with cancers detected via colonoscopy screening was too low. RESULTS: The sequential approach to calibration was more efficient than recalibrating the model from scratch. Incorporating new information on the percentage of patients with cancers detected upon screening changed the estimated sojourn time parameters significantly, increasing the estimated mean sojourn time for cancers in the colon and rectum, providing results with more validity. CONCLUSIONS: A sequential approach to recalibration can be used to efficiently recalibrate a microsimulation model when new information becomes available that requires the original targets to be supplemented with additional targets.

Unequal Recovery in Colorectal Cancer Screening Following the COVID-19 Pandemic: A Comparative Microsimulation Analysis.

The aftermath of the initial phase of the COVID-19 pandemic may contribute to the widening of disparities in access to colorectal cancer (CRC) screening due to differential disruptions to CRC screening. This comparative microsimulation analysis uses two CISNET CRC models to simulate the impact of ongoing screening disruptions induced by the COVID-19 pandemic on long-term CRC outcomes. We evaluate three channels through which screening was disrupted: delays in screening, regimen switching, and screening discontinuation. The impact of these disruptions on long-term colorectal cancer (CRC) outcomes was measured by the number of Life-years lost due to CRC screening disruptions compared to a scenario without any disruptions. While short-term delays in screening of 3-18 months are predicted to result in minor life-years loss, discontinuing screening could result in much more significant reductions in the expected benefits of screening. These results demonstrate that unequal recovery of screening following the pandemic can widen disparities in colorectal cancer outcomes and emphasize the importance of ensuring equitable recovery to screening following the pandemic.

Too Good to Be True? Evaluation of Colonoscopy Sensitivity Assumptions Used in Policy Models.

BACKGROUND: Models can help guide colorectal cancer screening policy. Although models are carefully calibrated and validated, there is less scrutiny of assumptions about test performance. METHODS: We examined the validity of the CRC-SPIN model and colonoscopy sensitivity assumptions. Standard sensitivity assumptions, consistent with published decision analyses, assume sensitivity equal to 0.75 for diminutive adenomas (<6 mm), 0.85 for small adenomas (6-10 mm), 0.95 for large adenomas (≥10 mm), and 0.95 for preclinical cancer. We also selected adenoma sensitivity that resulted in more accurate predictions. Targets were drawn from the Wheat Bran Fiber study. We examined how well the model predicted outcomes measured over a three-year follow-up period, including the number of adenomas detected, the size of the largest adenoma detected, and incident colorectal cancer. RESULTS: Using standard sensitivity assumptions, the model predicted adenoma prevalence that was too low (42.5% versus 48.9% observed, with 95% confidence interval 45.3%-50.7%) and detection of too few large adenomas (5.1% versus 14.% observed, with 95% confidence interval 11.8%-17.4%). Predictions were close to targets when we set sensitivities to 0.20 for diminutive adenomas, 0.60 for small adenomas, 0.80 for 10- to 20-mm adenomas, and 0.98 for adenomas 20 mm and larger. CONCLUSIONS: Colonoscopy may be less accurate than currently assumed, especially for diminutive adenomas. Alternatively, the CRC-SPIN model may not accurately simulate onset and progression of adenomas in higher-risk populations. IMPACT: Misspecification of either colonoscopy sensitivity or disease progression in high-risk populations may affect the predicted effectiveness of colorectal cancer screening. When possible, decision analyses used to inform policy should address these uncertainties.See related commentary by Etzioni and Lange, p. 702.

An Evolutionary Algorithm to Personalize Stool-Based Colorectal Cancer Screening.

BACKGROUND: Fecal immunochemical testing (FIT) is an established method for colorectal cancer (CRC) screening. Measured FIT-concentrations are associated with both present and future risk of CRC, and may be used for personalized screening. However, evaluation of personalized screening is computationally challenging. In this study, a broadly applicable algorithm is presented to efficiently optimize personalized screening policies that prescribe screening intervals and FIT-cutoffs, based on age and FIT-history. METHODS: We present a mathematical framework for personalized screening policies and a bi-objective evolutionary algorithm that identifies policies with minimal costs and maximal health benefits. The algorithm is combined with an established microsimulation model (MISCAN-Colon), to accurately estimate the costs and benefits of generated policies, without restrictive Markov assumptions. The performance of the algorithm is demonstrated in three experiments. RESULTS: In Experiment 1, a relatively small benchmark problem, the optimal policies were known. The algorithm approached the maximum feasible benefits with a relative difference of 0.007%. Experiment 2 optimized both intervals and cutoffs, Experiment 3 optimized cutoffs only. Optimal policies in both experiments are unknown. Compared to policies recently evaluated for the USPSTF, personalized screening increased health benefits up to 14 and 4.3%, for Experiments 2 and 3, respectively, without adding costs. Generated policies have several features concordant with current screening recommendations. DISCUSSION: The method presented in this paper is flexible and capable of optimizing personalized screening policies evaluated with computationally-intensive but established simulation models. It can be used to inform screening policies for CRC or other diseases. For CRC, more debate is needed on what features a policy needs to exhibit to make it suitable for implementation in practice.

Impact of a Low-Intensity Resource Referral Intervention on Patients' Knowledge, Beliefs, and Use of Community Resources: Results from the CommunityRx Trial.

BACKGROUND: Connecting patients to community-based resources is now a cornerstone of modern healthcare that supports self-management of health. The mechanisms that link resource information to behavior change, however, remain poorly understood. OBJECTIVE: To evaluate the impact of CommunityRx, an automated, low-intensity resource referral intervention, on patients' knowledge, beliefs, and use of community resources. DESIGN: Real-world controlled clinical trial at an urban academic medical center in 2015-2016; participants were assigned by alternating week to receive the CommunityRx intervention or usual care. Surveys were administered at baseline, 1 week, 1 month, and 3 months. PARTICIPANTS: Publicly insured adults, ages 45-74 years. INTERVENTION: CommunityRx generated an automated, personalized list of resources, known as HealtheRx, near each participant's home using condition-specific, evidence-based algorithms. Algorithms used patient demographic and health characteristics documented in the electronic health record to identify relevant resources from a comprehensive, regularly updated database of health-related resources in the study area. MAIN MEASURES: Using intent-to-treat analysis, we examined the impact of HealtheRx referrals on (1) knowledge of the most commonly referred resource types, including healthy eating classes, individual counseling, mortgage assistance, smoking cessation, stress management, and weight loss classes or groups, and (2) beliefs about having resources in the community to manage health. KEY RESULTS: In a real-world controlled trial of 374 adults, intervention recipients improved knowledge (AOR = 2.15; 95% CI, 1.29-3.58) and beliefs (AOR = 1.68; 95% CI, 1.07-2.64) about common resources in the community to manage health, specifically gaining knowledge about smoking cessation (AOR = 2.76; 95% CI, 1.07-7.12) and weight loss resources (AOR = 2.26; 95% CI 1.05-4.84). Positive changes in both knowledge and beliefs about community resources were associated with higher resource use (P = 0.02). CONCLUSIONS: In a middle-age and older population with high morbidity, a low-intensity health IT intervention to deliver resource referrals promoted behavior change by increasing knowledge and positive beliefs about community resources for self-management of health. NIH TRIAL REGISTRY: NCT02435511.

Learning-accelerated discovery of immune-tumour interactions.

We present an integrated framework for enabling dynamic exploration of design spaces for cancer immunotherapies with detailed dynamical simulation models on high-performance computing resources. Our framework combines PhysiCell, an open source agent-based simulation platform for cancer and other multicellular systems, and EMEWS, an open source platform for extreme-scale model exploration. We build an agent-based model of immunosurveillance against heterogeneous tumours, which includes spatial dynamics of stochastic tumour-immune contact interactions. We implement active learning and genetic algorithms using high-performance computing workflows to adaptively sample the model parameter space and iteratively discover optimal cancer regression regions within biological and clinical constraints.

MICROSIMULATION MODEL CALIBRATION USING INCREMENTAL MIXTURE APPROXIMATE BAYESIAN COMPUTATION.

Microsimulation models (MSMs) are used to inform policy by predicting population-level outcomes under different scenarios. MSMs simulate individual-level event histories that mark the disease process (such as the development of cancer) and the effect of policy actions (such as screening) on these events. MSMs often have many unknown parameters; calibration is the process of searching the parameter space to select parameters that result in accurate MSM prediction of a wide range of targets. We develop Incremental Mixture Approximate Bayesian Computation (IMABC) for MSM calibration, which results in a simulated sample from the posterior distribution of model parameters given calibration targets. IMABC begins with a rejection-based ABC step, drawing a sample of points from the prior distribution of model parameters and accepting points that result in simulated targets that are near observed targets. Next, the sample is iteratively updated by drawing additional points from a mixture of multivariate normal distributions and accepting points that result in accurate predictions. Posterior estimates are obtained by weighting the final set of accepted points to account for the adaptive sampling scheme. We demonstrate IMABC by calibrating CRC-SPIN 2.0, an updated version of a MSM for colorectal cancer (CRC) that has been used to inform national CRC screening guidelines.

CANDLE/Supervisor: a workflow framework for machine learning applied to cancer research.

BACKGROUND: Current multi-petaflop supercomputers are powerful systems, but present challenges when faced with problems requiring large machine learning workflows. Complex algorithms running at system scale, often with different patterns that require disparate software packages and complex data flows cause difficulties in assembling and managing large experiments on these machines. RESULTS: This paper presents a workflow system that makes progress on scaling machine learning ensembles, specifically in this first release, ensembles of deep neural networks that address problems in cancer research across the atomistic, molecular and population scales. The initial release of the application framework that we call CANDLE/Supervisor addresses the problem of hyper-parameter exploration of deep neural networks. CONCLUSIONS: Initial results demonstrating CANDLE on DOE systems at ORNL, ANL and NERSC (Titan, Theta and Cori, respectively) demonstrate both scaling and multi-platform execution.

High-throughput cancer hypothesis testing with an integrated PhysiCell-EMEWS workflow.

BACKGROUND: Cancer is a complex, multiscale dynamical system, with interactions between tumor cells and non-cancerous host systems. Therapies act on this combined cancer-host system, sometimes with unexpected results. Systematic investigation of mechanistic computational models can augment traditional laboratory and clinical studies, helping identify the factors driving a treatment's success or failure. However, given the uncertainties regarding the underlying biology, these multiscale computational models can take many potential forms, in addition to encompassing high-dimensional parameter spaces. Therefore, the exploration of these models is computationally challenging. We propose that integrating two existing technologies-one to aid the construction of multiscale agent-based models, the other developed to enhance model exploration and optimization-can provide a computational means for high-throughput hypothesis testing, and eventually, optimization. RESULTS: In this paper, we introduce a high throughput computing (HTC) framework that integrates a mechanistic 3-D multicellular simulator (PhysiCell) with an extreme-scale model exploration platform (EMEWS) to investigate high-dimensional parameter spaces. We show early results in applying PhysiCell-EMEWS to 3-D cancer immunotherapy and show insights on therapeutic failure. We describe a generalized PhysiCell-EMEWS workflow for high-throughput cancer hypothesis testing, where hundreds or thousands of mechanistic simulations are compared against data-driven error metrics to perform hypothesis optimization. CONCLUSIONS: While key notational and computational challenges remain, mechanistic agent-based models and high-throughput model exploration environments can be combined to systematically and rapidly explore key problems in cancer. These high-throughput computational experiments can improve our understanding of the underlying biology, drive future experiments, and ultimately inform clinical practice.