Assessing Performance and Clinical Usefulness in Prediction Models With Survival Outcomes: Practical Guidance for Cox Proportional Hazards Models.

Risk prediction models need thorough validation to assess their performance. Validation of models for survival outcomes poses challenges due to the censoring of observations and the varying time horizon at which predictions can be made. This article describes measures to evaluate predictions and the potential improvement in decision making from survival models based on Cox proportional hazards regression.As a motivating case study, the authors consider the prediction of the composite outcome of recurrence or death (the "event") in patients with breast cancer after surgery. They developed a simple Cox regression model with 3 predictors, as in the Nottingham Prognostic Index, in 2982 women (1275 events over 5 years of follow-up) and externally validated this model in 686 women (285 events over 5 years). Improvement in performance was assessed after the addition of progesterone receptor as a prognostic biomarker.The model predictions can be evaluated across the full range of observed follow-up times or for the event occurring by the end of a fixed time horizon of interest. The authors first discuss recommended statistical measures that evaluate model performance in terms of discrimination, calibration, or overall performance. Further, they evaluate the potential clinical utility of the model to support clinical decision making according to a net benefit measure. They provide SAS and R code to illustrate internal and external validation.The authors recommend the proposed set of performance measures for transparent reporting of the validity of predictions from survival models.

Breast Cancer Polygenic Risk Score and Contralateral Breast Cancer Risk.

Previous research has shown that polygenic risk scores (PRSs) can be used to stratify women according to their risk of developing primary invasive breast cancer. This study aimed to evaluate the association between a recently validated PRS of 313 germline variants (PRS313) and contralateral breast cancer (CBC) risk. We included 56,068 women of European ancestry diagnosed with first invasive breast cancer from 1990 onward with follow-up from the Breast Cancer Association Consortium. Metachronous CBC risk (N = 1,027) according to the distribution of PRS313 was quantified using Cox regression analyses. We assessed PRS313 interaction with age at first diagnosis, family history, morphology, ER status, PR status, and HER2 status, and (neo)adjuvant therapy. In studies of Asian women, with limited follow-up, CBC risk associated with PRS313 was assessed using logistic regression for 340 women with CBC compared with 12,133 women with unilateral breast cancer. Higher PRS313 was associated with increased CBC risk: hazard ratio per standard deviation (SD) = 1.25 (95%CI = 1.18-1.33) for Europeans, and an OR per SD = 1.15 (95%CI = 1.02-1.29) for Asians. The absolute lifetime risks of CBC, accounting for death as competing risk, were 12.4% for European women at the 10th percentile and 20.5% at the 90th percentile of PRS313. We found no evidence of confounding by or interaction with individual characteristics, characteristics of the primary tumor, or treatment. The C-index for the PRS313 alone was 0.563 (95%CI = 0.547-0.586). In conclusion, PRS313 is an independent factor associated with CBC risk and can be incorporated into CBC risk prediction models to help improve stratification and optimize surveillance and treatment strategies.

Impute-then-exclude versus exclude-then-impute: Lessons when imputing a variable used both in cohort creation and as an independent variable in the analysis model.

We examined the setting in which a variable that is subject to missingness is used both as an inclusion/exclusion criterion for creating the analytic sample and subsequently as the primary exposure in the analysis model that is of scientific interest. An example is cancer stage, where patients with stage IV cancer are often excluded from the analytic sample, and cancer stage (I to III) is an exposure variable in the analysis model. We considered two analytic strategies. The first strategy, referred to as "exclude-then-impute," excludes subjects for whom the observed value of the target variable is equal to the specified value and then uses multiple imputation to complete the data in the resultant sample. The second strategy, referred to as "impute-then-exclude," first uses multiple imputation to complete the data and then excludes subjects based on the observed or filled-in values in the completed samples. Monte Carlo simulations were used to compare five methods (one based on "exclude-then-impute" and four based on "impute-then-exclude") along with the use of a complete case analysis. We considered both missing completely at random and missing at random missing data mechanisms. We found that an impute-then-exclude strategy using substantive model compatible fully conditional specification tended to have superior performance across 72 different scenarios. We illustrated the application of these methods using empirical data on patients hospitalized with heart failure when heart failure subtype was used for cohort creation (excluding subjects with heart failure with preserved ejection fraction) and was also an exposure in the analysis model.

PredictCBC-2.0: a contralateral breast cancer risk prediction model developed and validated in ~ 200,000 patients.

BACKGROUND: Prediction of contralateral breast cancer (CBC) risk is challenging due to moderate performances of the known risk factors. We aimed to improve our previous risk prediction model (PredictCBC) by updated follow-up and including additional risk factors. METHODS: We included data from 207,510 invasive breast cancer patients participating in 23 studies. In total, 8225 CBC events occurred over a median follow-up of 10.2 years. In addition to the previously included risk factors, PredictCBC-2.0 included CHEK2 c.1100delC, a 313 variant polygenic risk score (PRS-313), body mass index (BMI), and parity. Fine and Gray regression was used to fit the model. Calibration and a time-dependent area under the curve (AUC) at 5 and 10 years were assessed to determine the performance of the models. Decision curve analysis was performed to evaluate the net benefit of PredictCBC-2.0 and previous PredictCBC models. RESULTS: The discrimination of PredictCBC-2.0 at 10 years was higher than PredictCBC with an AUC of 0.65 (95% prediction intervals (PI) 0.56-0.74) versus 0.63 (95%PI 0.54-0.71). PredictCBC-2.0 was well calibrated with an observed/expected ratio at 10 years of 0.92 (95%PI 0.34-2.54). Decision curve analysis for contralateral preventive mastectomy (CPM) showed the potential clinical utility of PredictCBC-2.0 between thresholds of 4 and 12% 10-year CBC risk for BRCA1/2 mutation carriers and non-carriers. CONCLUSIONS: Additional genetic information beyond BRCA1/2 germline mutations improved CBC risk prediction and might help tailor clinical decision-making toward CPM or alternative preventive strategies. Identifying patients who benefit from CPM, especially in the general breast cancer population, remains challenging.

Long-term risk of subsequent ipsilateral lesions after surgery with or without radiotherapy for ductal carcinoma in situ of the breast.

BACKGROUND: Radiotherapy (RT) following breast-conserving surgery (BCS) for ductal carcinoma in situ (DCIS) reduces ipsilateral breast event rates in clinical trials. This study assessed the impact of DCIS treatment on a 20-year risk of ipsilateral DCIS (iDCIS) and ipsilateral invasive breast cancer (iIBC) in a population-based cohort. METHODS: The cohort comprised all women diagnosed with DCIS in the Netherlands during 1989-2004 with follow-up until 2017. Cumulative incidence of iDCIS and iIBC following BCS and BCS + RT were assessed. Associations of DCIS treatment with iDCIS and iIBC risk were estimated in multivariable Cox models. RESULTS: The 20-year cumulative incidence of any ipsilateral breast event was 30.6% (95% confidence interval (CI): 28.9-32.6) after BCS compared to 18.2% (95% CI 16.3-20.3) following BCS + RT. Women treated with BCS compared to BCS + RT had higher risk of developing iDCIS and iIBC within 5 years after DCIS diagnosis (for iDCIS: hazard ratio (HR)age < 50 3.2 (95% CI 1.6-6.6); HRage ≥ 50 3.6 (95% CI 2.6-4.8) and for iIBC: HRage<50 2.1 (95% CI 1.4-3.2); HRage ≥ 50 4.3 (95% CI 3.0-6.0)). After 10 years, the risk of iDCIS and iIBC no longer differed for BCS versus BCS + RT (for iDCIS: HRage < 50 0.7 (95% CI 0.3-1.5); HRage ≥ 50 0.7 (95% CI 0.4-1.3) and for iIBC: HRage < 50 0.6 (95% CI 0.4-0.9); HRage ≥ 50 1.2 (95% CI 0.9-1.6)). CONCLUSION: RT is associated with lower iDCIS and iIBC risk up to 10 years after BCS, but this effect wanes thereafter.

Prediction of contralateral breast cancer: external validation of risk calculators in 20 international cohorts.

BACKGROUND: Three tools are currently available to predict the risk of contralateral breast cancer (CBC). We aimed to compare the performance of the Manchester formula, CBCrisk, and PredictCBC in patients with invasive breast cancer (BC). METHODS: We analyzed data of 132,756 patients (4682 CBC) from 20 international studies with a median follow-up of 8.8 years. Prediction performance included discrimination, quantified as a time-dependent Area-Under-the-Curve (AUC) at 5 and 10 years after diagnosis of primary BC, and calibration, quantified as the expected-observed (E/O) ratio at 5 and 10 years and the calibration slope. RESULTS: The AUC at 10 years was: 0.58 (95% confidence intervals [CI] 0.57-0.59) for CBCrisk; 0.60 (95% CI 0.59-0.61) for the Manchester formula; 0.63 (95% CI 0.59-0.66) and 0.59 (95% CI 0.56-0.62) for PredictCBC-1A (for settings where BRCA1/2 mutation status is available) and PredictCBC-1B (for the general population), respectively. The E/O at 10 years: 0.82 (95% CI 0.51-1.32) for CBCrisk; 1.53 (95% CI 0.63-3.73) for the Manchester formula; 1.28 (95% CI 0.63-2.58) for PredictCBC-1A and 1.35 (95% CI 0.65-2.77) for PredictCBC-1B. The calibration slope was 1.26 (95% CI 1.01-1.50) for CBCrisk; 0.90 (95% CI 0.79-1.02) for PredictCBC-1A; 0.81 (95% CI 0.63-0.99) for PredictCBC-1B, and 0.39 (95% CI 0.34-0.43) for the Manchester formula. CONCLUSIONS: Current CBC risk prediction tools provide only moderate discrimination and the Manchester formula was poorly calibrated. Better predictors and re-calibration are needed to improve CBC prediction and to identify low- and high-CBC risk patients for clinical decision-making.

Cancer-immune interactions in ER-positive breast cancers: PI3K pathway alterations and tumor-infiltrating lymphocytes.

INTRODUCTION: The presence of tumor-infiltrating lymphocytes (TILs) is correlated with good prognosis and outcome after (immuno)therapy in triple-negative and HER2-positive breast cancer. However, the role of TILs in luminal breast cancer is less clear. Emerging evidence has now demonstrated that genetic aberrations in malignant cells influence the immune landscape of tumors. Phosphatidylinositol 3-kinase (PI3K) is the most common altered pathway in ER-positive breast cancer. It is unknown whether changes in the PI3K pathway result in a different composition of the breast tumor microenvironment. Here we present the retrospective analysis of a prospective randomized trial in ER-positive breast cancer on the prognostic and predictive value of specific tumor-associated lymphocytes in the context of PI3K alterations. METHODS: We included 563 ER-positive tumors from a multicenter trial for stage I to III postmenopausal breast cancer patients, who were randomized to tamoxifen or no adjuvant therapy. The amount of CD8-, CD4-, and FOXP3-positive cells was evaluated by immunohistochemistry and quantified by imaging-analysis software. We analyzed the associations between PIK3CA hotspot mutations, PTEN expression, phosphorylated proteins of the PI3K and MAPK pathway (p-AKT, p-ERK1/2, p-4EBP1, p-p70S6K), and recurrence-free interval after adjuvant tamoxifen or no adjuvant treatment. RESULTS: CD8-positive lymphocytes were significantly more abundant in PIK3CA-mutated tumors (OR = 1.65; 95% CI 1.03-2.68). While CD4 and FOXP3 were not significantly associated with prognosis, patients with tumors classified as CD8-high had increased risk of recurrence (HR = 1.98; 95% CI 1.14-3.41; multivariable model including PIK3CA status, treatment arm, and other standard clinicopathological variables). Lymphocytes were more often present in tumors with increased PI3K downstream phosphorylation. This was most pronounced for FOXP3-positive cells. CONCLUSION: These exploratory analyses of a prospective trial in luminal breast cancer suggest high CD8 infiltration is associated with unfavorable outcome and that PI3K pathway alterations might be associated with the composition of the tumor microenvironment.

Prediction and clinical utility of a contralateral breast cancer risk model.

BACKGROUND: Breast cancer survivors are at risk for contralateral breast cancer (CBC), with the consequent burden of further treatment and potentially less favorable prognosis. We aimed to develop and validate a CBC risk prediction model and evaluate its applicability for clinical decision-making. METHODS: We included data of 132,756 invasive non-metastatic breast cancer patients from 20 studies with 4682 CBC events and a median follow-up of 8.8 years. We developed a multivariable Fine and Gray prediction model (PredictCBC-1A) including patient, primary tumor, and treatment characteristics and BRCA1/2 germline mutation status, accounting for the competing risks of death and distant metastasis. We also developed a model without BRCA1/2 mutation status (PredictCBC-1B) since this information was available for only 6% of patients and is routinely unavailable in the general breast cancer population. Prediction performance was evaluated using calibration and discrimination, calculated by a time-dependent area under the curve (AUC) at 5 and 10 years after diagnosis of primary breast cancer, and an internal-external cross-validation procedure. Decision curve analysis was performed to evaluate the net benefit of the model to quantify clinical utility. RESULTS: In the multivariable model, BRCA1/2 germline mutation status, family history, and systemic adjuvant treatment showed the strongest associations with CBC risk. The AUC of PredictCBC-1A was 0.63 (95% prediction interval (PI) at 5 years, 0.52-0.74; at 10 years, 0.53-0.72). Calibration-in-the-large was -0.13 (95% PI: -1.62-1.37), and the calibration slope was 0.90 (95% PI: 0.73-1.08). The AUC of Predict-1B at 10 years was 0.59 (95% PI: 0.52-0.66); calibration was slightly lower. Decision curve analysis for preventive contralateral mastectomy showed potential clinical utility of PredictCBC-1A between thresholds of 4-10% 10-year CBC risk for BRCA1/2 mutation carriers and non-carriers. CONCLUSIONS: We developed a reasonably calibrated model to predict the risk of CBC in women of European-descent; however, prediction accuracy was moderate. Our model shows potential for improved risk counseling, but decision-making regarding contralateral preventive mastectomy, especially in the general breast cancer population where limited information of the mutation status in BRCA1/2 is available, remains challenging.

Impact of Comorbidities and Age on Cause-Specific Mortality in Postmenopausal Patients with Breast Cancer.

BACKGROUND: The aim was to study the impact of comorbidities and age on breast cancer mortality, taking into account competing causes of death. SUBJECTS, MATERIALS, AND METHODS: Cohort analysis of Dutch and Belgian patients with postmenopausal, early hormone receptor-positive breast cancer included in the Tamoxifen and Exemestane Adjuvant Multinational (TEAM) trial between 2001 and 2006. This is a randomized controlled trial of patients who had completed local treatment with curative intent and were randomized to receive exemestane for 5 years, or sequential treatment of tamoxifen followed by exemestane for a duration of 5 years. Patients were categorized by number of comorbidities (no comorbidities, 1-2 comorbidities, and >2 comorbidities) and age (<70 years and ≥70 years). Main outcome was breast cancer mortality considering other-cause mortality as competing event; cumulative incidences were calculated using the Cumulative Incidence Competing Risk Methods, and the Fine and Gray model was used to calculate the effect of age and comorbidities for the cause-specific incidences of breast cancer death, taking into account the effect of competing causes of death. RESULTS: Overall, 3,159 patients were included, of which 2,203 (69.7%) were aged <70 years and 956 (30.3%) were aged ≥70 years at diagnosis. Cumulative incidence of breast cancer mortality was higher among patients ≥70 without comorbidities (22.2%, 95% CI, 17.5-26.9) compared with patients <70 without comorbidities (15.6%, 95% CI, 13.6-17.7, reference group), multivariable subdistribution hazard ratio (sHR) 1.49 (95% CI, 1.12-1.97, p = .005) after a median follow-up of 10 years. Use of chemotherapy was lower in older patients (1%, irrespective of the number of comorbidities) compared with younger patients (50%, 44%, and 38% for patients with no, 1-2, or >2 comorbidities, p < .001). CONCLUSION: Older patients without comorbidities have a higher risk of dying due to breast cancer than younger counterparts, even when taking into account higher competing mortality, while use of chemotherapy in this group was low. These findings underline the need to take into account comorbidities, age, and competing mortality in the prognosis of breast cancer for accurate decision making. IMPLICATIONS FOR PRACTICE: Older patients without comorbidity are at increased risk of dying from breast cancer, despite a higher other-cause mortality. This study shows that including age and comorbidity for the assessment of breast cancer mortality and other-cause mortality is indispensable for treatment decision making in older patients. Future prognostic tools for breast cancer prognosis should incorporate these items as well as risk of toxicity of adjuvant chemotherapy to adequately predict outcomes to optimize personalized treatment for older patients with early breast cancer.

Immunomodulatory Factors Control the Fate of Melanoma Tumor Initiating Cells.

Melanoma is a highly heterogeneous tumor for which recent evidence supports a model of dynamic stemness. Melanoma cells might temporally acquire tumor-initiating properties or switch from a status of tumor-initiating cells (TICs) to a more differentiated one depending on the tumor context. However, factors driving these functional changes are still unknown. We focused on the role of cyto/chemokines in shaping TICs isolated directly from tumor specimens of two melanoma patients, namely Me14346S and Me15888S. We analyzed the secretion profile of TICs and of their corresponding melanoma differentiated cells and we tested the ability of cyto/chemokines to influence TIC self-renewal and differentiation. We found that TICs, grown in vitro as melanospheres, had a complex secretory profile as compared to their differentiated counterparts. Some factors, such as CCL-2 and IL-8, also produced by adherent melanoma cells and melanocytes did not influence TIC properties. Conversely, IL-6, released by differentiated cells, reduced TIC self-renewal and induced TIC differentiation while IL-10, produced by Me15888S, strongly promoted TIC self-renewal through paracrine/autocrine actions. Complete neutralization of IL-10 activity by gene silencing and antibody-mediated blocking of the IL-10Rα was required to sensitize Me15888S to IL-6-induced differentiation. For the first time these results show that functional heterogeneity of melanoma could be directly influenced by inflammatory and suppressive soluble factors, with IL-6 favoring TIC differentiation, and IL-10 supporting TIC self-renewal. Thus, understanding the tumor microenvironment (TME) role in modulating melanoma TIC phenotype is fundamental to identifying novel therapeutic targets to achieve long-lasting regression of metastatic melanoma. Stem Cells 2016;34:2449-2460.

The Sooner the Better? How Symptom Interval Correlates With Outcome in Children and Adolescents With Solid Tumors: Regression Tree Analysis of the Findings of a Prospective Study.

BACKGROUND: The potential impact of diagnostic delays on patients' outcomes is a debated issue in pediatric oncology and discordant results have been published so far. We attempted to tackle this issue by analyzing a prospective series of 351 consecutive children and adolescents with solid malignancies using innovative statistical tools. METHODS: To address the nonlinear complexity of the association between symptom interval and overall survival (OS), a regression tree algorithm was constructed with sequential binary splitting rules and used to identify homogeneous patient groups vis-à-vis functional relationship between diagnostic delay and OS. RESULTS: Three different groups were identified: group A, with localized disease and good prognosis (5-year OS 85.4%); group B, with locally or regionally advanced, or metastatic disease and intermediate prognosis (5-year OS 72.9%), including neuroblastoma, Wilms tumor, non-rhabdomyosarcoma soft tissue sarcoma, and germ cell tumor; and group C, with locally or regionally advanced, or metastatic disease and poor prognosis (5-year OS 45%), including brain tumors, rhabdomyosarcoma, and bone sarcoma. The functional relationship between symptom interval and mortality risk differed between the three subgroups, there being no association in group A (hazard ratio [HR]: 0.96), a positive linear association in group B (HR: 1.48), and a negative linear association in group C (HR: 0.61). CONCLUSIONS: Our analysis suggests that at least a subset of patients can benefit from an earlier diagnosis in terms of survival. For others, intrinsic aggressiveness may mask the potential effect of diagnostic delays. Based on these findings, early diagnosis should remain a goal for pediatric cancer patients.

Integrating multiple kidney function markers to predict all-cause and cardiovascular disease mortality: prospective analysis of 366 758 UK Biobank participants.

Background: Reduced kidney function is a risk factor of cardiovascular and all-cause mortality. This association was demonstrated for several kidney function markers, but it is unclear whether integrating multiple measured markers may improve mortality risk prediction. Methods: We conducted an exploratory factor analysis (EFA) of serum creatinine- and cystatin C-based estimated glomerular filtration rate [eGFRcre and eGFRcys; derived by the Chronic Kidney Disease Epidemiology Collaboration (CKD-EPI) and European Kidney Function Consortium (EKFC) equations], blood urea nitrogen (BUN), uric acid and serum albumin among 366 758 participants in the UK Biobank without a history of kidney failure. Fitting Cox proportional hazards models, we compared the ability of the identified latent factors to predict overall mortality and mortality by cardiovascular disease (CVD), also considering CVD-specific causes like coronary heart disease (CHD) and cerebrovascular disease. Results: During 12.5 years of follow-up, 26 327 participants died from any cause, 5376 died from CVD, 2908 died from CHD and 1116 died from cerebrovascular disease. We identified two latent factors, EFA1 and EFA2, both representing kidney function variations. When using the CKD-EPI equation, EFA1 performed like eGFRcys, with EFA1 showing slightly larger hazard ratios for overall and CVD-related mortality. At 10 years of follow-up, EFA1 and eGFRcys showed moderate discrimination performance for CVD-related mortality, outperforming all other kidney indices. eGFRcre was the least predictive marker across all outcomes. When using the EKFC equation, eGFRcys performed better than EFA1 while all other results remaining similar. Conclusions: While EFA is an attractive approach to capture the complex effects of kidney function, eGFRcys remains the most practical and effective measurement for all-cause and CVD mortality risk prediction.

Determinants of SARS-CoV-2 nasopharyngeal testing in a rural community sample susceptible of first infection: the CHRIS COVID-19 study.

To characterize COVID-19 epidemiology, numerous population-based studies have been undertaken to model the risk of SARS-CoV-2 infection. Less is known about what may drive the probability to undergo testing. Understanding how much testing is driven by contextual or individual conditions is important to delineate the role of individual behavior and to shape public health interventions and resource allocation. In the Val Venosta/Vinschgau district (South Tyrol, Italy), we conducted a population-representative longitudinal study on 697 individuals susceptible to first infection who completed 4,512 repeated online questionnaires at four-week intervals between September 2020 and May 2021. Mixed-effects logistic regression models were fitted to investigate associations of self-reported SARS-CoV-2 testing with individual characteristics (social, demographic, and biological) and contextual determinants. Testing was associated with month of reporting, reflecting the timing of both the pandemic intensity and public health interventions, COVID-19-related symptoms (odds ratio, OR:8.26; 95% confidence interval, CI:6.04-11.31), contacts with infected individuals within home (OR:7.47, 95%CI:3.81-14.62) or outside home (OR:9.87, 95%CI:5.78-16.85), and being retired (OR:0.50, 95%CI:0.34-0.73). Symptoms and next within- and outside-home contacts were the leading determinants of swab testing predisposition in the most acute phase of the pandemics. Testing was not associated with age, sex, education, comorbidities, or lifestyle factors. In the study area, contextual determinants reflecting the course of the pandemic were predominant compared to individual sociodemographic characteristics in explaining the SARS-CoV-2 probability of testing. Decision makers should evaluate whether the intended target groups were correctly prioritized by the testing campaign.

Trends and symptoms of SARS-CoV-2 infection: a longitudinal study on an Alpine population representative sample.

OBJECTIVES: The continuous monitoring of SARS-CoV-2 infection waves and the emergence of novel pathogens pose a challenge for effective public health surveillance strategies based on diagnostics. Longitudinal population representative studies on incident events and symptoms of SARS-CoV-2 infection are scarce. We aimed at describing the evolution of the COVID-19 pandemic during 2020 and 2021 through regular monitoring of self-reported symptoms in an Alpine community sample. DESIGN: To this purpose, we designed a longitudinal population representative study, the Cooperative Health Research in South Tyrol COVID-19 study. PARTICIPANTS AND OUTCOME MEASURES: A sample of 845 participants was retrospectively investigated for active and past infections with swab and blood tests, by August 2020, allowing adjusted cumulative incidence estimation. Of them, 700 participants without previous infection or vaccination were followed up monthly until July 2021 for first-time infection and symptom self-reporting: COVID-19 anamnesis, social contacts, lifestyle and sociodemographic data were assessed remotely through digital questionnaires. Temporal symptom trajectories and infection rates were modelled through longitudinal clustering and dynamic correlation analysis. Negative binomial regression and random forest analysis assessed the relative importance of symptoms. RESULTS: At baseline, the cumulative incidence of SARS-CoV-2 infection was 1.10% (95% CI 0.51%, 2.10%). Symptom trajectories mimicked both self-reported and confirmed cases of incident infections. Cluster analysis identified two groups of high-frequency and low-frequency symptoms. Symptoms like fever and loss of smell fell in the low-frequency cluster. Symptoms most discriminative of test positivity (loss of smell, fatigue and joint-muscle aches) confirmed prior evidence. CONCLUSIONS: Regular symptom tracking from population representative samples is an effective screening tool auxiliary to laboratory diagnostics for novel pathogens at critical times, as manifested in this study of COVID-19 patterns. Integrated surveillance systems might benefit from more direct involvement of citizens' active symptom tracking.

External validation and clinical utility assessment of PREDICT breast cancer prognostic model in young, systemic treatment-naïve women with node-negative breast cancer.

BACKGROUND: The validity of the PREDICT breast cancer prognostic model is unclear for young patients without adjuvant systemic treatment. This study aimed to validate PREDICT and assess its clinical utility in young women with node-negative breast cancer who did not receive systemic treatment. METHODS: We selected all women from the Netherlands Cancer Registry who were diagnosed with node-negative breast cancer under age 40 between 1989 and 2000, a period when adjuvant systemic treatment was not standard practice for women with node-negative disease. We evaluated the calibration and discrimination of PREDICT using the observed/expected (O/E) mortality ratio, and the area under the receiver operating characteristic curve (AUC), respectively. Additionally, we compared the potential clinical utility of PREDICT for selectively administering chemotherapy to the chemotherapy-to-all strategy using decision curve analysis at predefined thresholds. RESULTS: A total of 2264 women with a median age at diagnosis of 36 years were included. Of them, 71.2% had estrogen receptor (ER)-positive tumors and 44.0% had grade 3 tumors. Median tumor size was 16 mm. PREDICT v2.2 underestimated 10-year all-cause mortality by 33% in all women (O/E ratio:1.33, 95%CI:1.22-1.43). Model discrimination was moderate overall (AUC10-year:0.65, 95%CI:0.62-0.68), and poor for women with ER-negative tumors (AUC10-year:0.56, 95%CI:0.51-0.62). Compared to the chemotherapy-to-all strategy, PREDICT only showed a slightly higher net benefit in women with ER-positive tumors, but not in women with ER-negative tumors. CONCLUSIONS: PREDICT yields unreliable predictions for young women with node-negative breast cancer. Further model updates are needed before PREDICT can be routinely used in this patient subset.

Graphical calibration curves and the integrated calibration index (ICI) for competing risk models.

BACKGROUND: Assessing calibration-the agreement between estimated risk and observed proportions-is an important component of deriving and validating clinical prediction models. Methods for assessing the calibration of prognostic models for use with competing risk data have received little attention. METHODS: We propose a method for graphically assessing the calibration of competing risk regression models. Our proposed method can be used to assess the calibration of any model for estimating incidence in the presence of competing risk (e.g., a Fine-Gray subdistribution hazard model; a combination of cause-specific hazard functions; or a random survival forest). Our method is based on using the Fine-Gray subdistribution hazard model to regress the cumulative incidence function of the cause-specific outcome of interest on the predicted outcome risk of the model whose calibration we want to assess. We provide modifications of the integrated calibration index (ICI), of E50 and of E90, which are numerical calibration metrics, for use with competing risk data. We conducted a series of Monte Carlo simulations to evaluate the performance of these calibration measures when the underlying model has been correctly specified and when the model was mis-specified and when the incidence of the cause-specific outcome differed between the derivation and validation samples. We illustrated the usefulness of calibration curves and the numerical calibration metrics by comparing the calibration of a Fine-Gray subdistribution hazards regression model with that of random survival forests for predicting cardiovascular mortality in patients hospitalized with heart failure. RESULTS: The simulations indicated that the method for constructing graphical calibration curves and the associated calibration metrics performed as desired. We also demonstrated that the numerical calibration metrics can be used as optimization criteria when tuning machine learning methods for competing risk outcomes. CONCLUSIONS: The calibration curves and numeric calibration metrics permit a comprehensive comparison of the calibration of different competing risk models.

Predicting 10-year survival after resection of colorectal liver metastases; an international study including biomarkers and perioperative treatment.

BACKGROUND: The aim of this study was to develop a prediction model for 10-year overall survival (OS) after resection of colorectal liver metastasis (CRLM) based on patient, tumour and treatment characteristics. METHODS: Consecutive patients after complete resection of CRLM were included from two centres (1992-2019). A prediction model providing 10-year OS probabilities was developed using Cox regression analysis, including KRAS, BRAF and histopathological growth patterns. Discrimination and calibration were assessed using cross-validation. A web-based calculator was built to predict individual 10-year OS probabilities. RESULTS: A total of 4112 patients were included. The estimated 10-year OS was 30% (95% CI 29-32). Fifteen patient, tumour and treatment characteristics were independent prognostic factors for 10-year OS; age, gender, location and nodal status of the primary tumour, disease-free interval, number and diameter of CRLM, preoperative CEA, resection margin, extrahepatic disease, KRAS and BRAF mutation status, histopathological growth patterns, perioperative systemic chemotherapy and hepatic arterial infusion pump chemotherapy. The discrimination at 10-years was 0.73 for both centres. A simplified risk score identified four risk groups with a 10-year OS of 57%, 38%, 24%, and 12%. CONCLUSIONS: Ten-year OS after resection of CRLM is best predicted with a model including 15 patient, tumour, and treatment characteristics. The web-based calculator can be used to inform patients. This model serves as a benchmark to determine the prognostic value of novel biomarkers.

Development and validation of the PORTRET tool to predict recurrence, overall survival, and other-cause mortality in older patients with breast cancer in the Netherlands: a population-based study.

BACKGROUND: Current prediction tools for breast cancer outcomes are not tailored to the older patient, in whom competing risk strongly influences treatment effects. We aimed to develop and validate a prediction tool for 5-year recurrence, overall mortality, and other-cause mortality for older patients (aged ≥65 years) with early invasive breast cancer and to estimate individualised expected benefits of adjuvant systemic treatment. METHODS: We selected surgically treated patients with early invasive breast cancer (stage I-III) aged 65 years or older from the population-based FOCUS cohort in the Netherlands. We developed prediction models for 5-year recurrence, overall mortality, and other-cause mortality using cause-specific Cox proportional hazard models. External validation was performed in a Dutch Cancer registry cohort. Performance was evaluated with discrimination accuracy and calibration plots. FINDINGS: We included 2744 female patients in the development cohort and 13631 female patients in the validation cohort. Median age was 74·8 years (range 65-98) in the development cohort and 76·0 years (70-101) in the validation cohort. 5-year follow-up was complete for more than 99% of all patients. We observed 343 and 1462 recurrences, and 831 and 3594 deaths, of which 586 and 2565 were without recurrence, in the development and validation cohort, respectively. The area under the receiver-operating-characteristic curve at 5 years in the external dataset was 0·76 (95% CI 0·75-0·76) for overall mortality, 0·76 (0·76-0·77) for recurrence, and 0·75 (0·74-0·75) for other-cause mortality. INTERPRETATION: The PORTRET tool can accurately predict 5-year recurrence, overall mortality, and other-cause mortality in older patients with breast cancer. The tool can support shared decision making, especially since it provides individualised estimated benefits of adjuvant treatment. FUNDING: Dutch Cancer Foundation and ZonMw.