Comparing Clinician Estimates versus a Statistical Tool for Predicting Risk of Death within 45 Days of Admission for Cancer Patients.

OBJECTIVES: While clinical practice guidelines recommend that oncologists discuss goals of care with patients who have advanced cancer, it is estimated that less than 20% of individuals admitted to the hospital with high-risk cancers have end-of-life discussions with their providers. While there has been interest in developing models for mortality prediction to trigger such discussions, few studies have compared how such models compare with clinical judgment to determine a patient's mortality risk. METHODS: This study is a prospective analysis of 1,069 solid tumor medical oncology hospital admissions (n = 911 unique patients) from February 7 to June 7, 2022, at Memorial Sloan Kettering Cancer Center. Electronic surveys were sent to hospitalists, advanced practice providers, and medical oncologists the first afternoon following a hospital admission and they were asked to estimate the probability that the patient would die within 45 days. Provider estimates of mortality were compared with those from a predictive model developed using a supervised machine learning methodology, and incorporated routine laboratory, demographic, biometric, and admission data. Area under the receiver operating characteristic curve (AUC), calibration and decision curves were compared between clinician estimates and the model predictions. RESULTS: Within 45 days following hospital admission, 229 (25%) of 911 patients died. The model performed better than the clinician estimates (AUC 0.834 vs. 0.753, p < 0.0001). Integrating clinician predictions with the model's estimates further increased the AUC to 0.853 (p < 0.0001). Clinicians overestimated risk whereas the model was extremely well-calibrated. The model demonstrated net benefit over a wide range of threshold probabilities. CONCLUSION: The inpatient prognosis at admission model is a robust tool to assist clinical providers in evaluating mortality risk, and it has recently been implemented in the electronic medical record at our institution to improve end-of-life care planning for hospitalized cancer patients.

Building a Clinically Relevant Risk Model: Predicting Risk of a Potentially Preventable Acute Care Visit for Patients Starting Antineoplastic Treatment.

PURPOSE: To create a risk prediction model that identifies patients at high risk for a potentially preventable acute care visit (PPACV). PATIENTS AND METHODS: We developed a risk model that used electronic medical record data from initial visit to first antineoplastic administration for new patients at Memorial Sloan Kettering Cancer Center from January 2014 to September 2018. The final time-weighted least absolute shrinkage and selection operator model was chosen on the basis of clinical and statistical significance. The model was refined to predict risk on the basis of 270 clinically relevant data features spanning sociodemographics, malignancy and treatment characteristics, laboratory results, medical and social history, medications, and prior acute care encounters. The binary dependent variable was occurrence of a PPACV within the first 6 months of treatment. There were 8,067 observations for new-start antineoplastic therapy in our training set, 1,211 in the validation set, and 1,294 in the testing set. RESULTS: A total of 3,727 patients experienced a PPACV within 6 months of treatment start. Specific features that determined risk were surfaced in a web application, riskExplorer, to enable clinician review of patient-specific risk. The positive predictive value of a PPACV among patients in the top quartile of model risk was 42%. This quartile accounted for 35% of patients with PPACVs and 51% of potentially preventable inpatient bed days. The model C-statistic was 0.65. CONCLUSION: Our clinically relevant model identified the patients responsible for 35% of PPACVs and more than half of the inpatient beds used by the cohort. Additional research is needed to determine whether targeting these high-risk patients with symptom management interventions could improve care delivery by reducing PPACVs.