Decision curve analysis confirms higher clinical utility of multi-domain versus single-domain prediction models in patients with open abdomen treatment for peritonitis.

BACKGROUND: Prediction modelling increasingly becomes an important risk assessment tool in perioperative systems approaches, e.g. in complex patients with open abdomen treatment for peritonitis. In this population, combining predictors from multiple medical domains (i.e. demographical, physiological and surgical variables) outperforms the prediction capabilities of single-domain prediction models. However, the benefit of these prediction models for clinical decision-making remains to be investigated. We therefore examined the clinical utility of mortality prediction models in patients suffering from peritonitis with a decision curve analysis. METHODS: In this secondary analysis of a large dataset, a traditional logistic regression approach, three machine learning methods and a stacked ensemble were employed to examine the predictive capability of demographic, physiological and surgical variables in predicting mortality under open abdomen treatment for peritonitis. Calibration was examined with calibration belts and predictive performance was assessed with the area both under the receiver operating characteristic curve (AUROC) and under the precision recall curve (AUPRC) and with the Brier Score. Clinical utility of the prediction models was examined by means of a decision curve analysis (DCA) within a treatment threshold range of interest of 0-30%, where threshold probabilities are traditionally defined as the minimum probability of disease at which further intervention would be warranted. RESULTS: Machine learning methods supported available evidence of a higher prediction performance of a multi- versus single-domain prediction models. Interestingly, their prediction performance was similar to a logistic regression model. The DCA demonstrated that the overall net benefit is largest for a multi-domain prediction model and that this benefit is larger compared to the default "treat all" strategy only for treatment threshold probabilities above about 10%. Importantly, the net benefit for low threshold probabilities is dominated by physiological predictors: surgical and demographics predictors provide only secondary decision-analytic benefit. CONCLUSIONS: DCA provides a valuable tool to compare single-domain and multi-domain prediction models and demonstrates overall higher decision-analytic value of the latter. Importantly, DCA provides a means to clinically differentiate the risks associated with each of these domains in more depth than with traditional performance metrics and highlighted the importance of physiological predictors for conservative intervention strategies for low treatment thresholds. Further, machine learning methods did not add significant benefit either in prediction performance or decision-analytic utility compared to logistic regression in these data.

Outcome prediction with serial neuron-specific enolase and machine learning in anoxic-ischaemic disorders of consciousness.

BACKGROUND: The continuation of life-sustaining therapy in critical care patients with anoxic-ischemic disorders of consciousness (AI-DOC) depends on prognostic tests such as serum neuron-specific enolase (NSE) concentration levels. OBJECTIVES: To apply predictive models using machine learning methods to examine, one year after onset, the prognostic power of serial measurements of NSE in patients with AI-DOC. To compare the discriminative accuracy of this method to both standard single-day, absolute, and difference-between-days, relative NSE levels. METHODS: Classification algorithms were implemented and K-nearest neighbours (KNN) imputation was used to avoid complete case elimination of patients with missing NSE values. Non-imputed measurements from Day 0 to Day 6 were used for single day and difference-between-days. RESULTS: The naive Bayes classifier on imputed serial NSE measurements returned an AUC of (0.81±0.07) for n=126 patients (100 poor outcome). This was greater than logistic regression (0.73±0.08) and all other classifiers. Naive Bayes gave a specificity and sensitivity of 96% and 49%, respectively, for an (uncalibrated) probability decision threshold of 90%. The maximum AUC for a single day was Day 3 (0.75) for a subset of n=79 (61 poor outcome) patients, and for differences between Day 1 and Day 4 (0.81) for a subset of n=46 (39 poor outcome) patients. CONCLUSION: Imputation avoided the elimination of patients with missing data and naive Bayes outperformed all other classifiers. Machine learning algorithms could detect automatically discriminatory features and the overall predictive power increased from standard methods due to the larger data set. CODE AVAILABILITY: Data analysis code is available under GNU at: https://github.com/emilymuller1991/outcome_prediction_nse.

Is the endoscopic third ventriculostomy success score an appropriate tool to inform clinical decision-making?

INTRODUCTION: The endoscopic third ventriculostomy success score (ETVSS) is a model, which provides each patient with a prediction of the outcome of endoscopic third ventriculostomy. The objective of this study was to determine if there is clinical value to the use of the ETVSS in the decision for ETV. METHODS: Prospectively collected data on all ETV procedures with the Republic of Ireland in children ≤16 years of age, totalling 112, from 2008 to 2014 was analysed. The percentage chance of success at six months was retrospectively calculated according to the ETVSS. A multivariable model, comprising the risk factors from the ETVSS - age, aetiology and previous shunt - was created and its performance compared to that of the ETVSS. RESULTS: The ETVSS achieved an AUC of 0.61 (95% CI: 0.49-0.71) with a sensitivity and specificity of 50% and 76%, respectively, at its optimal cutoff. The ETVSS was not significantly well calibrated in this cohort and there was a limited net benefit on decision curve analysis in comparison with the strategy of performing ETV in all patients. The multivariable model achieved an AUC of 0.67 (95% CI: 0.56-0.78), was well calibrated and was associated with a superior net benefit over that of the ETVSS. CONCLUSION: The ETVSS represents the future of patient risk stratification with an easy to use, individualised approach for each patient. The ETVSS has performed adequately in this study. However, through the addition of novel risk factors, the continuous updating of the model and recalibration where needed, the ETVSS can become a tool that the paediatric neurosurgeon cannot do without.