Using temporal recalibration to improve the calibration of risk prediction models in competing risk settings when there are trends in survival over time.

We have previously proposed temporal recalibration to account for trends in survival over time to improve the calibration of predictions from prognostic models for new patients. This involves first estimating the predictor effects using data from all individuals (full dataset) and then re-estimating the baseline using a subset of the most recent data whilst constraining the predictor effects to remain the same. In this article, we demonstrate how temporal recalibration can be applied in competing risk settings by recalibrating each cause-specific (or subdistribution) hazard model separately. We illustrate this using an example of colon cancer survival with data from the Surveillance Epidemiology and End Results (SEER) program. Data from patients diagnosed in 1995-2004 were used to fit two models for deaths due to colon cancer and other causes respectively. We discuss considerations that need to be made in order to apply temporal recalibration such as the choice of data used in the recalibration step. We also demonstrate how to assess the calibration of these models in new data for patients diagnosed subsequently in 2005. Comparison was made to a standard analysis (when improvements over time are not taken into account) and a period analysis which is similar to temporal recalibration but differs in the data used to estimate the predictor effects. The 10-year calibration plots demonstrated that using the standard approach over-estimated the risk of death due to colon cancer and the total risk of death and that calibration was improved using temporal recalibration or period analysis.

Minimum sample size calculations for external validation of a clinical prediction model with a time-to-event outcome.

Previous articles in Statistics in Medicine describe how to calculate the sample size required for external validation of prediction models with continuous and binary outcomes. The minimum sample size criteria aim to ensure precise estimation of key measures of a model's predictive performance, including measures of calibration, discrimination, and net benefit. Here, we extend the sample size guidance to prediction models with a time-to-event (survival) outcome, to cover external validation in datasets containing censoring. A simulation-based framework is proposed, which calculates the sample size required to target a particular confidence interval width for the calibration slope measuring the agreement between predicted risks (from the model) and observed risks (derived using pseudo-observations to account for censoring) on the log cumulative hazard scale. Precise estimation of calibration curves, discrimination, and net-benefit can also be checked in this framework. The process requires assumptions about the validation population in terms of the (i) distribution of the model's linear predictor and (ii) event and censoring distributions. Existing information can inform this; in particular, the linear predictor distribution can be approximated using the C-index or Royston's D statistic from the model development article, together with the overall event risk. We demonstrate how the approach can be used to calculate the sample size required to validate a prediction model for recurrent venous thromboembolism. Ideally the sample size should ensure precise calibration across the entire range of predicted risks, but must at least ensure adequate precision in regions important for clinical decision-making. Stata and R code are provided.

Estimating causal effects in the presence of competing events using regression standardisation with the Stata command standsurv.

BACKGROUND: When interested in a time-to-event outcome, competing events that prevent the occurrence of the event of interest may be present. In the presence of competing events, various estimands have been suggested for defining the causal effect of treatment on the event of interest. Depending on the estimand, the competing events are either accommodated or eliminated, resulting in causal effects with different interpretations. The former approach captures the total effect of treatment on the event of interest while the latter approach captures the direct effect of treatment on the event of interest that is not mediated by the competing event. Separable effects have also been defined for settings where the treatment can be partitioned into two components that affect the event of interest and the competing event through different causal pathways. METHODS: We outline various causal effects that may be of interest in the presence of competing events, including total, direct and separable effects, and describe how to obtain estimates using regression standardisation with the Stata command standsurv. Regression standardisation is applied by obtaining the average of individual estimates across all individuals in a study population after fitting a survival model. RESULTS: With standsurv several contrasts of interest can be calculated including differences, ratios and other user-defined functions. Confidence intervals can also be obtained using the delta method. Throughout we use an example analysing a publicly available dataset on prostate cancer to allow the reader to replicate the analysis and further explore the different effects of interest. CONCLUSIONS: Several causal effects can be defined in the presence of competing events and, under assumptions, estimates of those can be obtained using regression standardisation with the Stata command standsurv. The choice of which causal effect to define should be given careful consideration based on the research question and the audience to which the findings will be communicated.

Estimating restricted mean survival time and expected life-years lost in the presence of competing risks within flexible parametric survival models.

BACKGROUND: Royston-Parmar flexible parametric survival models (FPMs) can be fitted on either the cause-specific hazards or cumulative incidence scale in the presence of competing risks. An advantage of modelling within this framework for competing risks data is the ease at which alternative predictions to the (cause-specific or subdistribution) hazard ratio can be obtained. Restricted mean survival time (RMST), or restricted mean failure time (RMFT) on the mortality scale, is one such measure. This has an attractive interpretation, especially when the proportionality assumption is violated. Compared to similar measures, fewer assumptions are required and it does not require extrapolation. Furthermore, one can easily obtain the expected number of life-years lost, or gained, due to a particular cause of death, which is a further useful prognostic measure as introduced by Andersen. METHODS: In the presence of competing risks, prediction of RMFT and the expected life-years lost due to a cause of death are presented using Royston-Parmar FPMs. These can be predicted for a specific covariate pattern to facilitate interpretation in observational studies at the individual level, or at the population-level using standardisation to obtain marginal measures. Predictions are illustrated using English colorectal data and are obtained using the Stata post-estimation command, standsurv. RESULTS: Reporting such measures facilitate interpretation of a competing risks analysis, particularly when the proportional hazards assumption is not appropriate. Standardisation provides a useful way to obtain marginal estimates to make absolute comparisons between two covariate groups. Predictions can be made at various time-points and presented visually for each cause of death to better understand the overall impact of different covariate groups. CONCLUSIONS: We describe estimation of RMFT, and expected life-years lost partitioned by each competing cause of death after fitting a single FPM on either the log-cumulative subdistribution, or cause-specific hazards scale. These can be used to facilitate interpretation of a competing risks analysis when the proportionality assumption is in doubt.

Conditional crude probabilities of death for English cancer patients.

BACKGROUND: Cancer survival statistics are typically reported by using measures discounting the impact of other-cause mortality, such as net survival. This is a hypothetical measure and is interpreted as excluding the possibility of cancer patients dying from other causes. Crude probability of death partitions the all-cause probability of death into deaths from cancer and other causes. METHODS: The National Cancer Registration and Analysis Service is the single cancer registry for England. In 2006-2015, 1,590,477 malignant tumours were diagnosed for breast, colorectal, lung, melanoma and prostate cancer in adults. We used a relative survival framework, with a period approach, providing estimates for up to 10-year survival. Mortality was partitioned into deaths due to cancer or other causes. Unconditional and conditional (on surviving 1-years and 5-years) crude probability of death were estimated for the five cancers. RESULTS: Elderly patients who survived for a longer period before dying were more likely to die from other causes of death (except for lung cancer). For younger patients, deaths were almost entirely due to the cancer. CONCLUSION: There are different measures of survival, each with their own strengths and limitations. Careful choices of survival measures are needed for specific scenarios to maximise the understanding of the data.

Robustness of individual and marginal model-based estimates: A sensitivity analysis of flexible parametric models.

BACKGROUND: Flexible parametric survival models (FPMs) are commonly used in epidemiology. These are preferred as a wide range of hazard shapes can be captured using splines to model the log-cumulative hazard function and can include time-dependent effects for more flexibility. An important issue is the number of knots used for splines. The reliability of estimates are assessed using English data for 10 cancer types and the use of online interactive graphs to enable a more comprehensive sensitivity analysis at the control of the user is demonstrated. METHODS: Sixty FPMs were fitted to each cancer type with varying degrees of freedom to model the baseline excess hazard and the main and time-dependent effect of age. For each model, we obtained age-specific, age-group and internally age-standardised relative survival estimates. The Akaike Information Criterion and Bayesian Information Criterion were also calculated and comparative estimates were obtained using the Ederer II and Pohar Perme methods. Web-based interactive graphs were developed to present results. RESULTS: Age-standardised estimates were very insensitive to the exact number of knots for the splines. Age-group survival is also stable with negligible differences between models. Age-specific estimates are less stable especially for the youngest and oldest patients, of whom there are very few, but for most scenarios perform well. CONCLUSION: Although estimates do not depend heavily on the number of knots, too few knots should be avoided, as they can result in a poor fit. Interactive graphs engage researchers in assessing model sensitivity to a wide range of scenarios and their use is highly encouraged.