BACKGROUND: Identification of patients with high-risk of experiencing inability to walk after surgery is important for surgeons to make therapeutic strategies for patients with metastatic spinal disease. However, there is a lack of clinical tool to assess postoperative ambulatory status for those patients. The emergence of artificial intelligence (AI) brings a promising opportunity to develop accurate prediction models. METHODS: This study collected 455 patients with metastatic spinal disease who underwent posterior decompressive surgery at three tertiary medical institutions. Of these, 220 patients were collected from one medical institution to form the model derivation cohort, while 89 and 146 patients were collected from two other medical institutions to form the external validation cohorts 1 and 2, respectively. Patients in the model derivation cohort were used to develop and internally validate models. To establish the interactive AI platform, machine learning techniques were used to develop prediction models, including logistic regression (LR), decision tree (DT), random forest (RF), extreme gradient boosting machine (eXGBM), support vector machine (SVM), and neural network (NN). Furthermore, to enhance the resilience of the study's model, an ensemble machine learning approach was employed using a soft-voting method by combining the results of the above six algorithms. A scoring system incorporating 10 evaluation metrics was used to comprehensively assess the prediction performance of the developed models. The scoring system had a total score of 0 to 60, with higher scores denoting better prediction performance. An interactive AI platform was further deployed via Streamlit. The prediction performance was compared between medical experts and the AI platform in assessing the risk of experiencing postoperative inability to walk among patients with metastatic spinal disease. RESULTS: Among all developed models, the ensemble model outperformed the six other models with the highest score of 57, followed by the eXGBM model (54), SVM model (50), and NN model (50). The ensemble model had the best performance in accuracy and calibration slope, and the second-best performance in precise, recall, specificity, area under the curve (AUC), Brier score, and log loss. The scores of the LR model, RF model, and DT model were 39, 46, and 26, respectively. External validation demonstrated that the ensemble model had an AUC value of 0.873 (95% CI: 0.809-0.936) in the external validation cohort 1 and 0.924 (95% CI: 0.890-0.959) in the external validation cohort 2. In the new ensemble machine learning model excluding the feature of the number of comorbidities, the AUC value was still as high as 0.916 (95% CI: 0.863-0.969). In addition, the AUC values of the new model were 0.880 (95% CI: 0.819-0.940) in the external validation cohort 1 and 0.922 (95% CI: 0.887-0.958) in the external validation cohort 2, indicating favorable generalization of the model. The interactive AI platform was further deployed online based on the final machine learning model, and it was available at https://postoperativeambulatory-izpdr6gsxxwhitr8fubutd.streamlit.app/ . By using the AI platform, researchers were able to obtain the individual predicted risk of postoperative inability to walk, gain insights into the key factors influencing the outcome, and find the stratified therapeutic recommendations. The AUC value obtained from the AI platform was significantly higher than the average AUC value achieved by the medical experts ( P <0.001), denoting that the AI platform obviously outperformed the individual medical experts. CONCLUSIONS: The study successfully develops and validates an interactive AI platform for evaluating the risk of postoperative loss of ambulatory ability in patients with metastatic spinal disease. This AI platform has the potential to serve as a valuable model for guiding healthcare professionals in implementing surgical plans and ultimately enhancing patient outcomes.
Artificial Intelligence , Spinal Neoplasms , Humans , Female , Male , Middle Aged , Spinal Neoplasms/secondary , Spinal Neoplasms/surgery , Aged , Adult , Machine Learning , Walking/physiology
Purpose: Using an ensemble machine learning technique that incorporates the results of multiple machine learning algorithms, the study's objective is to build a reliable model to predict the early mortality among hepatocellular carcinoma (HCC) patients with bone metastases. Methods: We extracted a cohort of 124,770 patients with a diagnosis of hepatocellular carcinoma from the Surveillance, Epidemiology, and End Results (SEER) program and enrolled a cohort of 1897 patients who were diagnosed as having bone metastases. Patients with a survival time of 3 months or less were considered to have had early death. To compare patients with and without early mortality, subgroup analysis was used. Patients were randomly divided into two groups: a training cohort (n = 1509, 80%) and an internal testing cohort (n = 388, 20%). In the training cohort, five machine learning techniques were employed to train and optimize models for predicting early mortality, and an ensemble machine learning technique was used to generate risk probability in a way of soft voting, and it was able to combine the results from the multiply machine learning algorithms. The study employed both internal and external validations, and the key performance indicators included the area under the receiver operating characteristic curve (AUROC), Brier score, and calibration curve. Patients from two tertiary hospitals were chosen as the external testing cohorts (n = 98). Feature importance and reclassification were both operated in the study. Results: The early mortality was 55.5% (1052/1897). Eleven clinical characteristics were included as input features of machine learning models: sex (p = 0.019), marital status (p = 0.004), tumor stage (p = 0.025), node stage (p = 0.001), fibrosis score (p = 0.040), AFP level (p = 0.032), tumor size (p = 0.001), lung metastases (p < 0.001), cancer-directed surgery (p < 0.001), radiation (p < 0.001), and chemotherapy (p < 0.001). Application of the ensemble model in the internal testing population yielded an AUROC of 0.779 (95% confidence interval [CI]: 0.727-0.820), which was the largest AUROC among all models. Additionally, the ensemble model (0.191) outperformed the other five machine learning models in terms of Brier score. In terms of decision curves, the ensemble model also showed favorable clinical usefulness. External validation showed similar results; with an AUROC of 0.764 and Brier score of 0.195, the prediction performance was further improved after revision of the model. Feature importance demonstrated that the top three most crucial features were chemotherapy, radiation, and lung metastases based on the ensemble model. Reclassification of patients revealed a substantial difference in the two risk groups' actual probabilities of early mortality (74.38% vs. 31.35%, p < 0.001). Patients in the high-risk group had significantly shorter survival time than patients in the low-risk group (p < 0.001), according to the Kaplan-Meier survival curve. Conclusions: The ensemble machine learning model exhibits promising prediction performance for early mortality among HCC patients with bone metastases. With the aid of routinely accessible clinical characteristics, this model can be a trustworthy prognostic tool to predict the early death of those patients and facilitate clinical decision-making.
Purpose: Bone is one of the most common sites for the spread of malignant tumors. Patients with bone metastases whose prognosis was shorter than 3 months (early death) were considered as surgical contraindications. However, the information currently available in the literature limits our capacity to assess the risk likelihood of 3 month mortality. As a result, the study's objective is to create an accurate prediction model utilizing machine-learning techniques to predict 3 month mortality specifically among lung cancer patients with bone metastases according to easily available clinical data. Methods: This study enrolled 19,887 lung cancer patients with bone metastases between 2010 and 2018 from a large oncologic database in the United States. According to a ratio of 8:2, the entire patient cohort was randomly assigned to a training (n = 15881, 80%) and validation (n = 4,006, 20%) group. In the training group, prediction models were trained and optimized using six approaches, including logistic regression, XGBoosting machine, random forest, neural network, gradient boosting machine, and decision tree. There were 13 metrics, including the Brier score, calibration slope, intercept-in-large, area under the curve (AUC), and sensitivity, used to assess the model's prediction performance in the validation group. In each metric, the best prediction effectiveness was assigned six points, while the worst was given one point. The model with the highest sum score of the 13 measures was optimal. The model's explainability was performed using the local interpretable model-agnostic explanation (LIME) according to the optimal model. Predictor importance was assessed using H2O automatic machine learning. Risk stratification was also evaluated based on the optimal threshold. Results: Among all recruited patients, the 3 month mortality was 48.5%. Twelve variables, including age, primary site, histology, race, sex, tumor (T) stage, node (N) stage, brain metastasis, liver metastasis, cancer-directed surgery, radiation, and chemotherapy, were significantly associated with 3 month mortality based on multivariate analysis, and these variables were included for developing prediction models. With the highest sum score of all the measurements, the gradient boosting machine approach outperformed all the other models (62 points), followed by the XGBooting machine approach (59 points) and logistic regression (53). The area under the curve (AUC) was 0.820 (95% confident interval [CI]: 0.807-0.833), 0.820 (95% CI: 0.807-0.833), and 0.815 (95% CI: 0.801-0.828), respectively, calibration slope was 0.97, 0.95, and 0.96, respectively, and accuracy was all 0.772. Explainability of models was conducted to rank the predictors and visualize their contributions to an individual's mortality outcome. The top four important predictors in the population according to H2O automatic machine learning were chemotherapy, followed by liver metastasis, radiation, and brain metastasis. Compared to patients in the low-risk group, patients in the high-risk group were more than three times the odds of dying within 3 months (P < 0.001). Conclusions: Using machine learning techniques, this study offers a number of models, and the optimal model is found after thoroughly assessing and contrasting the prediction performance of each model. The optimal model can be a pragmatic risk prediction tool and is capable of identifying lung cancer patients with bone metastases who are at high risk for 3 month mortality, informing risk counseling, and aiding clinical treatment decision-making. It is better advised for patients in the high-risk group to have radiotherapy alone, the best supportive care, or minimally invasive procedures like cementoplasty.
Bone Neoplasms , Brain Neoplasms , Liver Neoplasms , Lung Neoplasms , Humans , Machine Learning , Bone Neoplasms/secondary , Bone Neoplasms/surgery
