Development and External Validation of Prediction Models for 10-Year Survival of Invasive Breast Cancer. Comparison with PREDICT and CancerMath.

Purpose: To compare PREDICT and CancerMath, two widely used prognostic models for invasive breast cancer, taking into account their clinical utility. Furthermore, it is unclear whether these models could be improved.Experimental Design: A dataset of 5,729 women was used for model development. A Bayesian variable selection algorithm was implemented to stochastically search for important interaction terms among the predictors. The derived models were then compared in three independent datasets (n = 5,534). We examined calibration, discrimination, and performed decision curve analysis.Results: CancerMath demonstrated worse calibration performance compared with PREDICT in estrogen receptor (ER)-positive and ER-negative tumors. The decline in discrimination performance was -4.27% (-6.39 to -2.03) and -3.21% (-5.9 to -0.48) for ER-positive and ER-negative tumors, respectively. Our new models matched the performance of PREDICT in terms of calibration and discrimination, but offered no improvement. Decision curve analysis showed predictions for all models were clinically useful for treatment decisions made at risk thresholds between 5% and 55% for ER-positive tumors and at thresholds of 15% to 60% for ER-negative tumors. Within these threshold ranges, CancerMath provided the lowest clinical utility among all the models.Conclusions: Survival probabilities from PREDICT offer both improved accuracy and discrimination over CancerMath. Using PREDICT to make treatment decisions offers greater clinical utility than CancerMath over a range of risk thresholds. Our new models performed as well as PREDICT, but no better, suggesting that, in this setting, including further interaction terms offers no predictive benefit. Clin Cancer Res; 24(9); 2110-5. ©2018 AACR.

Calibration of complex models through Bayesian evidence synthesis: a demonstration and tutorial.

Decision-analytic models must often be informed using data that are only indirectly related to the main model parameters. The authors outline how to implement a Bayesian synthesis of diverse sources of evidence to calibrate the parameters of a complex model. A graphical model is built to represent how observed data are generated from statistical models with unknown parameters and how those parameters are related to quantities of interest for decision making. This forms the basis of an algorithm to estimate a posterior probability distribution, which represents the updated state of evidence for all unknowns given all data and prior beliefs. This process calibrates the quantities of interest against data and, at the same time, propagates all parameter uncertainties to the results used for decision making. To illustrate these methods, the authors demonstrate how a previously developed Markov model for the progression of human papillomavirus (HPV-16) infection was rebuilt in a Bayesian framework. Transition probabilities between states of disease severity are inferred indirectly from cross-sectional observations of prevalence of HPV-16 and HPV-16-related disease by age, cervical cancer incidence, and other published information. Previously, a discrete collection of plausible scenarios was identified but with no further indication of which of these are more plausible. Instead, the authors derive a Bayesian posterior distribution, in which scenarios are implicitly weighted according to how well they are supported by the data. In particular, we emphasize the appropriate choice of prior distributions and checking and comparison of fitted models.

Optimal timing of allogeneic hematopoietic stem cell transplantation in patients with myelodysplastic syndrome.

Allogeneic hematopoietic stem cell transplantation (HSCT) represents the only curative treatment for patients with myelodysplastic syndrome (MDS), but involves non-negligible morbidity and mortality. Registry studies have shown that advanced disease stage at transplantation is associated with inferior overall survival. To define the optimal timing of allogeneic HSCT, we carried out a decision analysis by studying 660 patients who received best supportive care and 449 subjects who underwent transplantation. Risk assessment was based on both the International Prognostic Scoring System (IPSS) and the World Health Organization classification-based Prognostic Scoring System (WPSS). We used a continuous-time multistate Markov model to describe the natural history of disease and evaluate the effect of allogeneic HSCT on survival. This model estimated life expectancy from diagnosis according to treatment policy at different risk stages. Relative to supportive care, estimated life expectancy increased when transplantation was delayed from the initial stages until progression to intermediate-1 IPSS-risk or to intermediate WPSS-risk stage, and then decreased for higher risks. Modeling decision analysis on WPSS versus IPSS allowed better estimation of the optimal timing of transplantation. These observations indicate that allogeneic HSCT offers optimal survival benefits when the procedure is performed before MDS patients progress to advanced disease stages.

Predictors of outcome for anal fistula surgery.

OBJECTIVES: To review our experience with patients treated for anal fistula secondary to cryptoglandular disease and to determine factors that influence postoperative outcome. DESIGN: Retrospective review. SETTING: A regional tertiary referral center. PATIENTS: Adult patients with anal fistula secondary to cryptoglandular disease. INTERVENTIONS: Fistulotomy, advancement flap, and fistula plugging. MAIN OUTCOME MEASURES: Rates of operative failure (persistent fistula), incontinence, and septic complications. We evaluated age, sex, previous operation, fistula type, number of fistula tracts, horseshoe fistula, and intervention type to determine their independent influence on outcomes. RESULTS: One hundred seventy-nine patients (79.3% male) underwent fistula operation from October 1, 2003, through December 31, 2008. Median age was 45 years. Fistulotomy was undertaken in 82.7% of patients, advancement flap in 10.6%, and plugging in 6.7%. The rates of operative failure, postoperative incontinence, and septic complications were 15.6%, 15.6%, and 7.3%, respectively. Plugging carried the highest failure rate (83.3%) compared with fistulotomy (10.1%) (odds ratio [OR], 44.3 [95% confidence interval (CI), 8.9-221.0; P < .001]) and was the only independent predictor for failure after adjusting for all variables. Being older than 45 years was associated with a higher postoperative incontinence rate compared with the younger group (adjusted OR, 2.8 [95% CI, 1.0-7.7; P = .04]). High transsphincteric and suprasphincteric fistulas were predictors of incontinence compared with subcutaneous fistulas (adjusted OR, 22.9 [95% CI, 2.2-242.0; P = .009] and 61.5 [4.5-844.0; P = .002], respectively). The only predictor of septic complications was plugging compared with fistulotomy (adjusted OR, 15.1 [95% CI, 2.3-97.7; P = .004]). CONCLUSIONS: Fistulotomy is the preferred operation for anal fistula. Plugging is associated with the highest operative failure and septic complication rates. Incontinence was influenced more by fistula type and age rather than procedure.

Survival models in health economic evaluations: balancing fit and parsimony to improve prediction.

Health economic decision models compare costs and health effects of different interventions over the long term and usually incorporate survival data. Since survival is often extrapolated beyond the range of the data, inaccurate model specification can result in very different policy decisions. However, in this area, flexible survival models are rarely considered, and model uncertainty is rarely accounted for. In this article, various survival distributions are applied in a decision model for oral cancer screening. Flexible parametric models are compared with Bayesian semiparametric models, in which the baseline hazard can be made arbitrarily complex while still enabling survival to be extrapolated. A fully Bayesian framework is used for all models so that uncertainties can be easily incorporated in estimates of long-term costs and effects. The fit and predictive ability of both parametric and semiparametric models are compared using the deviance information criterion in order to account for model uncertainty in the cost-effectiveness analysis. Under the Bayesian semiparametric models, some smoothing of the hazard function is required to obtain adequate predictive ability and avoid sensitivity to the choice of prior. We determine that one flexible parametric survival model fits substantially better than the others considered in the oral cancer example.

A simple model to predict coronary disease in patients undergoing operation for mitral regurgitation.

BACKGROUND: Coexistent coronary disease can be identified in a third of patients with mitral valve disease. This study aims to evaluate candidate selection strategy using risk factor identification and logistic regression and to develop an additive model for the prediction of coexistent coronary disease. METHODS: The sample is a consecutive series of patients who had mitral repair from 1987 to 1999. Sensitivities and specificities were calculated for each risk factor. Variables for prediction of coronary disease were entered into a univariate analysis, and predictors were entered into a forward and backward stepwise multivariate logistic regression model to form a predictive score. An additive model was derived from transformation of the logistic model. Receiver operating characteristic curves were used to compare discrimination and precision quantified by the Hosmer-Lemeshow statistic. RESULTS: The American Heart Association and American College of Cardiology risk factor identification selection criteria for the 359 patients who had screening coronary angiography yielded 100% sensitivity and 1% specificity. Risk prediction with our logistic model produced a receiver operating characteristic curve area of 0.91 and Hosmer-Lemeshow score of 3.4 (p = 0.9). Similar discriminating ability for our patients was achieved by the Cleveland Clinic logistic model (receiver operator characteristic curve area of 0.79; Hosmer-Lemeshow score of 12; p = 0.1). Our five-item additive model produced receiver operating characteristic curve area of 0.91 and Hosmer-Lemeshow score of 3.81 (p = 0.80). CONCLUSIONS: Simple risk factor identification has excellent sensitivity but is limited by specificity. Logistic regression modeling is an accurate risk prediction method but is difficult to apply at the bedside. Simplicity and accuracy may be achieved by the logistic regression-derived simple additive model.