Comparing Lung Cancer Screening Strategies in a Nationally Representative US Population Using Transportability Methods for the National Lung Cancer Screening Trial.

Importance: The National Lung Screening Trial (NLST) found that screening for lung cancer with low-dose computed tomography (CT) reduced lung cancer-specific and all-cause mortality compared with chest radiography. It is uncertain whether these results apply to a nationally representative target population. Objective: To extend inferences about the effects of lung cancer screening strategies from the NLST to a nationally representative target population of NLST-eligible US adults. Design, Setting, and Participants: This comparative effectiveness study included NLST data from US adults at 33 participating centers enrolled between August 2002 and April 2004 with follow-up through 2009 along with National Health Interview Survey (NHIS) cross-sectional household interview survey data from 2010. Eligible participants were adults aged 55 to 74 years, and were current or former smokers with at least 30 pack-years of smoking (former smokers were required to have quit within the last 15 years). Transportability analyses combined baseline covariate, treatment, and outcome data from the NLST with covariate data from the NHIS and reweighted the trial data to the target population. Data were analyzed from March 2020 to May 2023. Interventions: Low-dose CT or chest radiography screening with a screening assessment at baseline, then yearly for 2 more years. Main Outcomes and Measures: For the outcomes of lung-cancer specific and all-cause death, mortality rates, rate differences, and ratios were calculated at a median (25th percentile and 75th percentile) follow-up of 5.5 (5.2-5.9) years for lung cancer-specific mortality and 6.5 (6.1-6.9) years for all-cause mortality. Results: The transportability analysis included 51 274 NLST participants and 685 NHIS participants representing the target population (of approximately 5 700 000 individuals after survey-weighting). Compared with the target population, NLST participants were younger (median [25th percentile and 75th percentile] age, 60 [57 to 65] years vs 63 [58 to 67] years), had fewer comorbidities (eg, heart disease, 6551 of 51 274 [12.8%] vs 1 025 951 of 5 739 532 [17.9%]), and were more educated (bachelor's degree or higher, 16 349 of 51 274 [31.9%] vs 859 812 of 5 739 532 [15.0%]). In the target population, for lung cancer-specific mortality, the estimated relative rate reduction was 18% (95% CI, 1% to 33%) and the estimated absolute rate reduction with low-dose CT vs chest radiography was 71 deaths per 100 000 person-years (95% CI, 4 to 138 deaths per 100 000 person-years); for all-cause mortality the estimated relative rate reduction was 6% (95% CI, -2% to 12%). In the NLST, for lung cancer-specific mortality, the estimated relative rate reduction was 21% (95% CI, 9% to 32%) and the estimated absolute rate reduction was 67 deaths per 100 000 person-years (95% CI, 27 to 106 deaths per 100 000 person-years); for all-cause mortality, the estimated relative rate reduction was 7% (95% CI, 0% to 12%). Conclusions and Relevance: Estimates of the comparative effectiveness of low-dose CT screening compared with chest radiography in a nationally representative target population were similar to those from unweighted NLST analyses, particularly on the relative scale. Increased uncertainty around effect estimates for the target population reflects large differences in the observed characteristics of trial participants and the target population.

Breast Multiparametric MRI for Prediction of Neoadjuvant Chemotherapy Response in Breast Cancer: The BMMR2 Challenge.

Purpose To describe the design, conduct, and results of the Breast Multiparametric MRI for prediction of neoadjuvant chemotherapy Response (BMMR2) challenge. Materials and Methods The BMMR2 computational challenge opened on May 28, 2021, and closed on December 21, 2021. The goal of the challenge was to identify image-based markers derived from multiparametric breast MRI, including diffusion-weighted imaging (DWI) and dynamic contrast-enhanced (DCE) MRI, along with clinical data for predicting pathologic complete response (pCR) following neoadjuvant treatment. Data included 573 breast MRI studies from 191 women (mean age [±SD], 48.9 years ± 10.56) in the I-SPY 2/American College of Radiology Imaging Network (ACRIN) 6698 trial (ClinicalTrials.gov: NCT01042379). The challenge cohort was split into training (60%) and test (40%) sets, with teams blinded to test set pCR outcomes. Prediction performance was evaluated by area under the receiver operating characteristic curve (AUC) and compared with the benchmark established from the ACRIN 6698 primary analysis. Results Eight teams submitted final predictions. Entries from three teams had point estimators of AUC that were higher than the benchmark performance (AUC, 0.782 [95% CI: 0.670, 0.893], with AUCs of 0.803 [95% CI: 0.702, 0.904], 0.838 [95% CI: 0.748, 0.928], and 0.840 [95% CI: 0.748, 0.932]). A variety of approaches were used, ranging from extraction of individual features to deep learning and artificial intelligence methods, incorporating DCE and DWI alone or in combination. Conclusion The BMMR2 challenge identified several models with high predictive performance, which may further expand the value of multiparametric breast MRI as an early marker of treatment response. Clinical trial registration no. NCT01042379 Keywords: MRI, Breast, Tumor Response Supplemental material is available for this article. © RSNA, 2024.

Systematically missing data in causally interpretable meta-analysis.

Causally interpretable meta-analysis combines information from a collection of randomized controlled trials to estimate treatment effects in a target population in which experimentation may not be possible but from which covariate information can be obtained. In such analyses, a key practical challenge is the presence of systematically missing data when some trials have collected data on one or more baseline covariates, but other trials have not, such that the covariate information is missing for all participants in the latter. In this article, we provide identification results for potential (counterfactual) outcome means and average treatment effects in the target population when covariate data are systematically missing from some of the trials in the meta-analysis. We propose three estimators for the average treatment effect in the target population, examine their asymptotic properties, and show that they have good finite-sample performance in simulation studies. We use the estimators to analyze data from two large lung cancer screening trials and target population data from the National Health and Nutrition Examination Survey (NHANES). To accommodate the complex survey design of the NHANES, we modify the methods to incorporate survey sampling weights and allow for clustering.

Survival analysis using deep learning with medical imaging.

There is widespread interest in using deep learning to build prediction models for medical imaging data. These deep learning methods capture the local structure of the image and require no manual feature extraction. Despite the importance of modeling survival in the context of medical data analysis, research on deep learning methods for modeling the relationship of imaging and time-to-event data is still under-developed. We provide an overview of deep learning methods for time-to-event outcomes and compare several deep learning methods to Cox model based methods through the analysis of a histology dataset of gliomas.

Regression-based estimation of heterogeneous treatment effects when extending inferences from a randomized trial to a target population.

Most work on extending (generalizing or transporting) inferences from a randomized trial to a target population has focused on estimating average treatment effects (i.e., averaged over the target population's covariate distribution). Yet, in the presence of strong effect modification by baseline covariates, the average treatment effect in the target population may be less relevant for guiding treatment decisions. Instead, the conditional average treatment effect (CATE) as a function of key effect modifiers may be a more useful estimand. Recent work on estimating target population CATEs using baseline covariate, treatment, and outcome data from the trial and covariate data from the target population only allows for the examination of heterogeneity over distinct subgroups. We describe flexible pseudo-outcome regression modeling methods for estimating target population CATEs conditional on discrete or continuous baseline covariates when the trial is embedded in a sample from the target population (i.e., in nested trial designs). We construct pointwise confidence intervals for the CATE at a specific value of the effect modifiers and uniform confidence bands for the CATE function. Last, we illustrate the methods using data from the Coronary Artery Surgery Study (CASS) to estimate CATEs given history of myocardial infarction and baseline ejection fraction value in the target population of all trial-eligible patients with stable ischemic heart disease.

Extending prediction models for use in a new target population with failure time outcomes.

Prediction models are often built and evaluated using data from a population that differs from the target population where model-derived predictions are intended to be used in. In this article, we present methods for evaluating model performance in the target population when some observations are right censored. The methods assume that outcome and covariate data are available from a source population used for model development and covariates, but no outcome data, are available from the target population. We evaluate the finite sample performance of the proposed estimators using simulations and apply the methods to transport a prediction model built using data from a lung cancer screening trial to a nationally representative population of participants eligible for lung cancer screening.

Estimating the area under the ROC curve when transporting a prediction model to a target population.

We propose methods for estimating the area under the receiver operating characteristic (ROC) curve (AUC) of a prediction model in a target population that differs from the source population that provided the data used for original model development. If covariates that are associated with model performance, as measured by the AUC, have a different distribution in the source and target populations, then AUC estimators that only use data from the source population will not reflect model performance in the target population. Here, we provide identification results for the AUC in the target population when outcome and covariate data are available from the sample of the source population, but only covariate data are available from the sample of the target population. In this setting, we propose three estimators for the AUC in the target population and show that they are consistent and asymptotically normal. We evaluate the finite-sample performance of the estimators using simulations and use them to estimate the AUC in a nationally representative target population from the National Health and Nutrition Examination Survey for a lung cancer risk prediction model developed using source population data from the National Lung Screening Trial.

Transporting a Prediction Model for Use in a New Target Population.

We considered methods for transporting a prediction model for use in a new target population, both when outcome and covariate data for model development are available from a source population that has a different covariate distribution compared with the target population and when covariate data (but not outcome data) are available from the target population. We discuss how to tailor the prediction model to account for differences in the data distribution between the source population and the target population. We also discuss how to assess the model's performance (e.g., by estimating the mean squared prediction error) in the target population. We provide identifiability results for measures of model performance in the target population for a potentially misspecified prediction model under a sampling design where the source and the target population samples are obtained separately. We introduce the concept of prediction error modifiers that can be used to reason about tailoring measures of model performance to the target population. We illustrate the methods in simulated data and apply them to transport a prediction model for lung cancer diagnosis from the National Lung Screening Trial to the nationally representative target population of trial-eligible individuals in the National Health and Nutrition Examination Survey.

Impact of Alternate b-Value Combinations and Metrics on the Predictive Performance and Repeatability of Diffusion-Weighted MRI in Breast Cancer Treatment: Results from the ECOG-ACRIN A6698 Trial.

In diffusion-weighted MRI (DW-MRI), choice of b-value influences apparent diffusion coefficient (ADC) values by probing different aspects of the tissue microenvironment. As a secondary analysis of the multicenter ECOG-ACRIN A6698 trial, the purpose of this study was to investigate the impact of alternate b-value combinations on the performance and repeatability of tumor ADC as a predictive marker of breast cancer treatment response. The final analysis included 210 women who underwent standardized 4-b-value DW-MRI (b = 0/100/600/800 s/mm2) at multiple timepoints during neoadjuvant chemotherapy treatment and a subset (n = 71) who underwent test−retest scans. Centralized tumor ADC and perfusion fraction (fp) measures were performed using variable b-value combinations. Prediction of pathologic complete response (pCR) based on the mid-treatment/12-week percent change in each metric was estimated by area under the receiver operating characteristic curve (AUC). Repeatability was estimated by within-subject coefficient of variation (wCV). Results show that two-b-value ADC calculations provided non-inferior predictive value to four-b-value ADC calculations overall (AUCs = 0.60−0.61 versus AUC = 0.60) and for HR+/HER2− cancers where ADC was most predictive (AUCs = 0.75−0.78 versus AUC = 0.76), p < 0.05. Using two b-values (0/600 or 0/800 s/mm2) did not reduce ADC repeatability over the four-b-value calculation (wCVs = 4.9−5.2% versus 5.4%). The alternate metrics ADCfast (b ≤ 100 s/mm2), ADCslow (b ≥ 100 s/mm2), and fp did not improve predictive performance (AUCs = 0.54−0.60, p = 0.08−0.81), and ADCfast and fp demonstrated the lowest repeatability (wCVs = 6.71% and 12.4%, respectively). In conclusion, breast tumor ADC calculated using a simple two-b-value approach can provide comparable predictive value and repeatability to full four-b-value measurements as a marker of treatment response.

Estimating subgroup effects in generalizability and transportability analyses.

Methods for extending - generalizing or transporting - inferences from a randomized trial to a target population involve conditioning on a large set of covariates that is sufficient for rendering the randomized and non-randomized groups exchangeable. Yet, decision-makers are often interested in examining treatment effects in subgroups of the target population defined in terms of only a few discrete covariates. Here, we propose methods for estimating subgroup-specific potential outcome means and average treatment effects in generalizability and transportability analyses, using outcome model-based (g-formula), weighting, and augmented weighting estimators. We consider estimating subgroup-specific average treatment effects in the target population and its non-randomized subset, and provide methods that are appropriate both for nested and non-nested trial designs. As an illustration, we apply the methods to data from the Coronary Artery Surgery Study to compare the effect of surgery plus medical therapy versus medical therapy alone for chronic coronary artery disease in subgroups defined by history of myocardial infarction.

Development and internal validation of a predictive model of cognitive decline 36 months following elective surgery.

INTRODUCTION: Our goal was to determine if features of surgical patients, easily obtained from the medical chart or brief interview, could be used to predict those likely to experience more rapid cognitive decline following surgery. METHODS: We analyzed data from an observational study of 560 older adults (≥70 years) without dementia undergoing major elective non-cardiac surgery. Cognitive decline was measured using change in a global composite over 2 to 36 months following surgery. Predictive features were identified as variables readily obtained from chart review or a brief patient assessment. We developed predictive models for cognitive decline (slope) and predicting dichotomized cognitive decline at a clinically determined cut. RESULTS: In a hold-out testing set, the regularized regression predictive model achieved a root mean squared error (RMSE) of 0.146 and a model r-square (R2 ) of .31. Prediction of "rapid" decliners as a group achieved an area under the curve (AUC) of .75. CONCLUSION: Some of our models could predict persons with increased risk for accelerated cognitive decline with greater accuracy than relying upon chance, and this result might be useful for stratification of surgical patients for inclusion in future clinical trials.

Machine Learning to Develop and Internally Validate a Predictive Model for Post-operative Delirium in a Prospective, Observational Clinical Cohort Study of Older Surgical Patients.

BACKGROUND: Our objective was to assess the performance of machine learning methods to predict post-operative delirium using a prospective clinical cohort. METHODS: We analyzed data from an observational cohort study of 560 older adults (≥ 70 years) without dementia undergoing major elective non-cardiac surgery. Post-operative delirium was determined by the Confusion Assessment Method supplemented by a medical chart review (N = 134, 24%). Five machine learning algorithms and a standard stepwise logistic regression model were developed in a training sample (80% of participants) and evaluated in the remaining hold-out testing sample. We evaluated three overlapping feature sets, restricted to variables that are readily available or minimally burdensome to collect in clinical settings, including interview and medical record data. A large feature set included 71 potential predictors. A smaller set of 18 features was selected by an expert panel using a consensus process, and this smaller feature set was considered with and without a measure of pre-operative mental status. RESULTS: The area under the receiver operating characteristic curve (AUC) was higher in the large feature set conditions (range of AUC, 0.62-0.71 across algorithms) versus the selected feature set conditions (AUC range, 0.53-0.57). The restricted feature set with mental status had intermediate AUC values (range, 0.53-0.68). In the full feature set condition, algorithms such as gradient boosting, cross-validated logistic regression, and neural network (AUC = 0.71, 95% CI 0.58-0.83) were comparable with a model developed using traditional stepwise logistic regression (AUC = 0.69, 95% CI 0.57-0.82). Calibration for all models and feature sets was poor. CONCLUSIONS: We developed machine learning prediction models for post-operative delirium that performed better than chance and are comparable with traditional stepwise logistic regression. Delirium proved to be a phenotype that was difficult to predict with appreciable accuracy.

Deep learning for survival outcomes.

Deep learning is a class of machine learning algorithms that are popular for building risk prediction models. When observations are censored, the outcomes are only partially observed and standard deep learning algorithms cannot be directly applied. We develop a new class of deep learning algorithms for outcomes that are potentially censored. To account for censoring, the unobservable loss function used in the absence of censoring is replaced by a censoring unbiased transformation. The resulting class of algorithms can be used to estimate both survival probabilities and restricted mean survival. We show how the deep learning algorithms can be implemented by adapting software for uncensored data by using a form of response transformation. We provide comparisons of the proposed deep learning algorithms to existing risk prediction algorithms for predicting survival probabilities and restricted mean survival through both simulated datasets and analysis of data from breast cancer patients.

Extending inferences from a randomized trial to a new target population.

When treatment effect modifiers influence the decision to participate in a randomized trial, the average treatment effect in the population represented by the randomized individuals will differ from the effect in other populations. In this tutorial, we consider methods for extending causal inferences about time-fixed treatments from a trial to a new target population of nonparticipants, using data from a completed randomized trial and baseline covariate data from a sample from the target population. We examine methods based on modeling the expectation of the outcome, the probability of participation, or both (doubly robust). We compare the methods in a simulation study and show how they can be implemented in software. We apply the methods to a randomized trial nested within a cohort of trial-eligible patients to compare coronary artery surgery plus medical therapy versus medical therapy alone for patients with chronic coronary artery disease. We conclude by discussing issues that arise when using the methods in applied analyses.

Tumor Deposits in Stage III Colon Cancer: Correlation With Other Histopathologic Variables, Prognostic Value, and Risk Stratification-Time to Consider "N2c".

OBJECTIVES: National Comprehensive Cancer Network (NCCN) guidelines for stage III colon cancer define low-risk versus high-risk patients based on T (1 to 3 vs. 4) and N (1 vs. 2) status, with some variations in treatment. This study analyzes the impact of tumor deposits (TDs), T and N status, poor differentiation (PD), perineural invasion (PNI), and lymphovascular invasion (LVI) on survival. MATERIALS AND METHODS: A retrospective analysis (2010-2015) of the National Cancer Database of stage III colon cancer patients treated with both surgery and chemotherapy was conducted. Data was extracted on sex, race, age at diagnosis, Charlson-Deyo Score, histopathologic variables, and survival rates. Statistical analysis used the test of proportions, log-rank test for Kaplan-Meier curves, and Cox proportional hazard models. RESULTS: For the 42,901 patients analyzed, 5-year survival rates were similar for LNTD (59.8%) and LNTD (58.2%), but significantly worse for LNTD (41.5%) (P<0.001). The presence of LNTD was more often associated with T4 (36.9%), N2 (55.1%), PD (37.4%), PNI (34.5%), and LVI (69.1%), than LNTD or LNTD (P<0.001). The hazard ratios for each variable were: TD: 1.34; T4: 1.71; N2: 1.44; PD: 1.37; PNI: 1.11; LVI: 1.18. LN patients with ≥3 TD (N1c) had worse overall survival than those with 1 to 2 TD (P<0.01), but similar to ≥4 LNTD (N2) and 1 to 3 LNTD (N1a-b). In our model, 5-year survival ranged from 23.4% for high-risk to 78.1% for low-risk patients (P<0.001). CONCLUSION: This National Cancer Database (NCDB) analysis offers greater risk stratification and may prompt consideration of changes in American Joint Committee on Cancer (AJCC) classification (N2c, in addition to N1c) to reflect the different prognosis and guide management, as well as survivorship strategies, for TD stage III colon cancer patients.

Personalized Risk Prediction in Clinical Oncology Research: Applications and Practical Issues Using Survival Trees and Random Forests.

A crucial component of making individualized treatment decisions is to accurately predict each patient's disease risk. In clinical oncology, disease risks are often measured through time-to-event data, such as overall survival and progression/recurrence-free survival, and are often subject to censoring. Risk prediction models based on recursive partitioning methods are becoming increasingly popular largely due to their ability to handle nonlinear relationships, higher-order interactions, and/or high-dimensional covariates. The most popular recursive partitioning methods are versions of the Classification and Regression Tree (CART) algorithm, which builds a simple interpretable tree structured model. With the aim of increasing prediction accuracy, the random forest algorithm averages multiple CART trees, creating a flexible risk prediction model. Risk prediction models used in clinical oncology commonly use both traditional demographic and tumor pathological factors as well as high-dimensional genetic markers and treatment parameters from multimodality treatments. In this article, we describe the most commonly used extensions of the CART and random forest algorithms to right-censored outcomes. We focus on how they differ from the methods for noncensored outcomes, and how the different splitting rules and methods for cost-complexity pruning impact these algorithms. We demonstrate these algorithms by analyzing a randomized Phase III clinical trial of breast cancer. We also conduct Monte Carlo simulations to compare the prediction accuracy of survival forests with more commonly used regression models under various scenarios. These simulation studies aim to evaluate how sensitive the prediction accuracy is to the underlying model specifications, the choice of tuning parameters, and the degrees of missing covariates.