Comparing clinical judgment with the MySurgeryRisk algorithm for preoperative risk assessment: A pilot usability study.

BACKGROUND: Major postoperative complications are associated with increased cost and mortality. The complexity of electronic health records overwhelms physicians' abilities to use the information for optimal and timely preoperative risk assessment. We hypothesized that data-driven, predictive-risk algorithms implemented in an intelligent decision-support platform simplify and augment physicians' risk assessments. METHODS: This prospective, nonrandomized pilot study of 20 physicians at a quaternary academic medical center compared the usability and accuracy of preoperative risk assessment between physicians and MySurgeryRisk, a validated, machine-learning algorithm, using a simulated workflow for the real-time, intelligent decision-support platform. We used area under the receiver operating characteristic curve to compare the accuracy of physicians' risk assessment for six postoperative complications before and after interaction with the algorithm for 150 clinical cases. RESULTS: The area under the receiver operating characteristic curve of the MySurgeryRisk algorithm ranged between 0.73 and 0.85 and was significantly better than physicians' initial risk assessments (area under the receiver operating characteristic curve between 0.47 and 0.69) for all postoperative complications except cardiovascular. After interaction with the algorithm, the physicians significantly improved their risk assessment for acute kidney injury and for an intensive care unit admission greater than 48 hours, resulting in a net improvement of reclassification of 12% and 16%, respectively. Physicians rated the algorithm as easy to use and useful. CONCLUSION: Implementation of a validated, MySurgeryRisk computational algorithm for real-time predictive analytics with data derived from the electronic health records to augment physicians' decision-making is feasible and accepted by physicians. Early involvement of physicians as key stakeholders in both design and implementation of this technology will be crucial for its future success.

MySurgeryRisk: Development and Validation of a Machine-learning Risk Algorithm for Major Complications and Death After Surgery.

OBJECTIVE: To accurately calculate the risk for postoperative complications and death after surgery in the preoperative period using machine-learning modeling of clinical data. BACKGROUND: Postoperative complications cause a 2-fold increase in the 30-day mortality and cost, and are associated with long-term consequences. The ability to precisely forecast the risk for major complications before surgery is limited. METHODS: In a single-center cohort of 51,457 surgical patients undergoing major inpatient surgery, we have developed and validated an automated analytics framework for a preoperative risk algorithm (MySurgeryRisk) that uses existing clinical data in electronic health records to forecast patient-level probabilistic risk scores for 8 major postoperative complications (acute kidney injury, sepsis, venous thromboembolism, intensive care unit admission >48 hours, mechanical ventilation >48 hours, wound, neurologic, and cardiovascular complications) and death up to 24 months after surgery. We used the area under the receiver characteristic curve (AUC) and predictiveness curves to evaluate model performance. RESULTS: MySurgeryRisk calculates probabilistic risk scores for 8 postoperative complications with AUC values ranging between 0.82 and 0.94 [99% confidence intervals (CIs) 0.81-0.94]. The model predicts the risk for death at 1, 3, 6, 12, and 24 months with AUC values ranging between 0.77 and 0.83 (99% CI 0.76-0.85). CONCLUSIONS: We constructed an automated predictive analytics framework for machine-learning algorithm with high discriminatory ability for assessing the risk of surgical complications and death using readily available preoperative electronic health records data. The feasibility of this novel algorithm implemented in real time clinical workflow requires further testing.

Extended vertical lists for temporal pattern mining from multivariate time series.

In this paper, the problem of mining complex temporal patterns in the context of multivariate time series is considered. A new method called the Fast Temporal Pattern Mining with Extended Vertical Lists is introduced. The method is based on an extension of the level-wise property, which requires a more complex pattern to start at positions within a record where all of the subpatterns of the pattern start. The approach is built around a novel data structure called the Extended Vertical List that tracks positions of the first state of the pattern inside records and links them to appropriate positions of a specific subpattern of the pattern called the prefix. Extensive computational results indicate that the new method performs significantly faster than the previous version of the algorithm for Temporal Pattern Mining; however, the increase in speed comes at the expense of increased memory usage.

Preoperative assessment of the risk for multiple complications after surgery.

BACKGROUND: The association between preoperative patient characteristics and the number of major postoperative complications after a major operation is not well defined. METHODS: In a retrospective, single-center cohort of 50,314 adult surgical patients, we used readily available preoperative clinical data to model the number of major postoperative complications from none to ≥3. We included acute kidney injury; prolonged stay (>48 hours) in an intensive care unit; need for prolonged (>48 hours) mechanical ventilation; severe sepsis; and cardiovascular, wound, and neurologic complications. Risk probability scores generated from the multinomial logistic models were used to develop an online calculator. We stratified patients based on their risk of having ≥3 postoperative complications. RESULTS: Patients older than 65 years (odds ratio 1.5, 95% confidence interval, 1.4-1.6), males (odds ratio 1.2, 95% confidence interval, 1.2-1.3), patients with a greater Charlson comorbidity index (odds ratio 3.9, 95% confidence interval, 3.6-4.2), patients requiring emergency operation (odds ratio 3.5, 95% confidence interval, 3.3.-3.7), and patients admitted on a weekend (odds ratio 1.4, 95% confidence interval, 1.3-1.5) were more likely to have ≥3 postoperative complications than they were to have none. Patients in the medium- and high-risk categories were 3.7 and 6.3 times more likely to have ≥3 postoperative complications, respectively. High-risk patients were 5.8 and 4.4 times more likely to die within 30 and 90 days of admission, respectively. CONCLUSION: Readily available, preoperative clinical and sociodemographic factors are associated with a greater number of postoperative complications and adverse surgical outcomes. We developed an online calculator that predicts probability of developing each number of complications after a major operation.

Application of Machine Learning Techniques to High-Dimensional Clinical Data to Forecast Postoperative Complications.

OBJECTIVE: To compare performance of risk prediction models for forecasting postoperative sepsis and acute kidney injury. DESIGN: Retrospective single center cohort study of adult surgical patients admitted between 2000 and 2010. PATIENTS: 50,318 adult patients undergoing major surgery. MEASUREMENTS: We evaluated the performance of logistic regression, generalized additive models, naïve Bayes and support vector machines for forecasting postoperative sepsis and acute kidney injury. We assessed the impact of feature reduction techniques on predictive performance. Model performance was determined using the area under the receiver operating characteristic curve, accuracy, and positive predicted value. The results were reported based on a 70/30 cross validation procedure where the data were randomly split into 70% used for training the model and the 30% for validation. MAIN RESULTS: The areas under the receiver operating characteristic curve for different models ranged between 0.797 and 0.858 for acute kidney injury and between 0.757 and 0.909 for severe sepsis. Logistic regression, generalized additive model, and support vector machines had better performance compared to Naïve Bayes model. Generalized additive models additionally accounted for non-linearity of continuous clinical variables as depicted in their risk patterns plots. Reducing the input feature space with LASSO had minimal effect on prediction performance, while feature extraction using principal component analysis improved performance of the models. CONCLUSIONS: Generalized additive models and support vector machines had good performance as risk prediction model for postoperative sepsis and AKI. Feature extraction using principal component analysis improved the predictive performance of all models.

The Pattern of Longitudinal Change in Serum Creatinine and 90-Day Mortality After Major Surgery.

OBJECTIVE: Calculate mortality risk that accounts for both severity and recovery of postoperative kidney dysfunction using the pattern of longitudinal change in creatinine. BACKGROUND: Although the importance of renal recovery after acute kidney injury (AKI) is increasingly recognized, the complex association that accounts for longitudinal creatinine changes and mortality is not fully described. METHODS: We used routinely collected clinical information for 46,299 adult patients undergoing major surgery to develop a multivariable probabilistic model optimized for nonlinearity of serum creatinine time series that calculates the risk function for 90-day mortality. We performed a 70/30 cross validation analysis to assess the accuracy of the model. RESULTS: All creatinine time series exhibited nonlinear risk function in relation to 90-day mortality and their addition to other clinical factors improved the model discrimination. For any given severity of AKI, patients with complete renal recovery, as manifested by the return of the discharge creatinine to the baseline value, experienced a significant decrease in the odds of dying within 90 days of admission compared with patients with partial recovery. Yet, for any severity of AKI, even complete renal recovery did not entirely mitigate the increased odds of dying, as patients with mild AKI and complete renal recovery still had significantly increased odds for dying compared with patients without AKI [odds ratio: 1.48 (95% confidence interval: 1.30-1.68)]. CONCLUSIONS: We demonstrate the nonlinear relationship between both severity and recovery of renal dysfunction and 90-day mortality after major surgery. We have developed an easily applicable computer algorithm that calculates this complex relationship.