Comparison of Classically and Machine Learning Generated Survival Prediction Models for Patients With Spinal Metastasis - A meta-Analysis of Two Recently Developed Algorithms.

STUDY DESIGN: A systemic review and a meta-analysis. We also provided a retrospective cohort for validation in this study. OBJECTIVE: (1) Using a meta-analysis to determine the pooled discriminatory ability of The Skeletal Oncology Research Group (SORG) classical algorithm (CA) and machine learning algorithms (MLA); and (2) test the hypothesis that SORG-CA has less variability in performance than SORG-MLA in non-American validation cohorts as SORG-CA does not incorporates regional-specific variables such as body mass index as input. METHODS: After data extraction from the included studies, logit-transformation was applied for extracted AUCs for further analysis. The discriminatory abilities of both algorithms were directly compared by their logit (AUC)s. Further subgroup analysis by region (America vs non-America) was also conducted by comparing the corresponding logit (AUC). RESULTS: The pooled logit (AUC)s of 90-day SORG-CA was .82 (95% confidence interval [CI], .53-.11), 1-year SORG-CA was 1.11 (95% CI, .74-1.48), 90-day SORG-MLA was 1.36 (95% CI, 1.09-1.63), and 1-year SORG-MLA was 1.57 (95% CI, 1.17-1.98). All the algorithms performed better in United States than in Taiwan (P < .001). The performance of SORG-CA was less influenced by a non-American cohort than SORG-MLA. CONCLUSION: These observations might highlight the importance of incorporating region-specific variables into existing models to make them generalizable to racially or geographically distinct regions.

Impact of intraoperative imaging on decision-making during spine surgery: a survey among spine surgeons using simulated intraoperative images.

PURPOSE: To assess whether the intention to intraoperatively reposition pedicle screws differs when spine surgeons evaluate the same screws with 2D imaging or 3D imaging. METHODS: In this online survey study, 21 spine surgeons evaluated eight pedicle screws from patients who had undergone posterior spinal fixation. In a simulated intraoperative setting, surgeons had to decide if they would reposition a marked pedicle screw based on its position in the provided radiologic imaging. The eight assessed pedicle screws varied in radiologic position, including two screws positioned within the pedicle, two breaching the pedicle cortex < 2 mm, two breaching the pedicle cortex 2-4 mm, and two positioned completely outside the pedicle. Surgeons assessed each pedicle screw twice without knowing and in random order: once with a scrollable three-dimensional (3D) image and once with two oblique fluoroscopic two-dimensional (2D) images. RESULTS: Almost all surgeons (19/21) intended to reposition more pedicle screws based on 3D imaging than on 2D imaging, with a mean number of pedicle screws to be repositioned of, respectively, 4.1 (± 1.3) and 2.0 (± 1.3; p < 0.001). Surgeons intended to reposition two screws placed completely outside the pedicle, one breaching 2-4mm, and one breaching < 2 mm more often based on 3D imaging. CONCLUSION: When provided with 3D imaging, spine surgeons not only intend to intraoperatively reposition pedicle screws at risk of causing postoperative complications more often but also screws with acceptable positions. This study highlights the potential of intraoperative 3D imaging as well as the need for consensus on how to act on intraoperative 3D information.

Stereotactic Body and Conventional Radiotherapy for Painful Bone Metastases: A Systematic Review and Meta-Analysis.

Importance: Conventional external beam radiotherapy (cEBRT) and stereotactic body radiotherapy (SBRT) are commonly used treatment options for relieving metastatic bone pain. The effectiveness of SBRT compared with cEBRT in pain relief has been a subject of debate, and conflicting results have been reported. Objective: To compare the effectiveness associated with SBRT vs cEBRT for relieving metastatic bone pain. Data Sources: A structured search was performed in the PubMed, Embase, and Cochrane databases on June 5, 2023. Additionally, results were added from a new randomized clinical trial (RCT) and additional unpublished data from an already published RCT. Study Selection: Comparative studies reporting pain response after SBRT vs cEBRT in patients with painful bone metastases. Data Extraction and Synthesis: Two independent reviewers extracted data from eligible studies. Data were extracted for the intention-to-treat (ITT) and per-protocol (PP) populations. The study is reported following the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline. Main Outcomes and Measures: Overall and complete pain response at 1, 3, and 6 months after radiotherapy, according to the study's definition. Relative risk ratios (RRs) with 95% CIs were calculated for each study. A random-effects model using a restricted maximum likelihood estimator was applied for meta-analysis. Results: There were 18 studies with 1685 patients included in the systematic review and 8 RCTs with 1090 patients were included in the meta-analysis. In 7 RCTs, overall pain response was defined according to the International Consensus on Palliative Radiotherapy Endpoints in clinical trials (ICPRE). The complete pain response was reported in 6 RCTs, all defined according to the ICPRE. The ITT meta-analyses showed that the overall pain response rates did not differ between cEBRT and SBRT at 1 (RR, 1.14; 95% CI, 0.99-1.30), 3 (RR, 1.19; 95% CI, 0.96-1.47), or 6 (RR, 1.22; 95% CI, 0.96-1.54) months. However, SBRT was associated with a higher complete pain response at 1 (RR, 1.43; 95% CI, 1.02-2.01), 3 (RR, 1.80; 95% CI, 1.16-2.78), and 6 (RR, 2.47; 95% CI, 1.24-4.91) months after radiotherapy. The PP meta-analyses showed comparable results. Conclusions and Relevance: In this systematic review and meta-analysis, patients with painful bone metastases experienced similar overall pain response after SBRT compared with cEBRT. More patients had complete pain alleviation after SBRT, suggesting that selected subgroups will benefit from SBRT.

Comparison of eight modern preoperative scoring systems for survival prediction in patients with extremity metastasis.

BACKGROUND: Survival is an important factor to consider when clinicians make treatment decisions for patients with skeletal metastasis. Several preoperative scoring systems (PSSs) have been developed to aid in survival prediction. Although we previously validated the Skeletal Oncology Research Group Machine-learning Algorithm (SORG-MLA) in Taiwanese patients of Han Chinese descent, the performance of other existing PSSs remains largely unknown outside their respective development cohorts. We aim to determine which PSS performs best in this unique population and provide a direct comparison between these models. METHODS: We retrospectively included 356 patients undergoing surgical treatment for extremity metastasis at a tertiary center in Taiwan to validate and compare eight PSSs. Discrimination (c-index), decision curve (DCA), calibration (ratio of observed:expected survivors), and overall performance (Brier score) analyses were conducted to evaluate these models' performance in our cohort. RESULTS: The discriminatory ability of all PSSs declined in our Taiwanese cohort compared with their Western validations. SORG-MLA is the only PSS that still demonstrated excellent discrimination (c-indexes>0.8) in our patients. SORG-MLA also brought the most net benefit across a wide range of risk probabilities on DCA with its 3-month and 12-month survival predictions. CONCLUSIONS: Clinicians should consider potential ethnogeographic variations of a PSS's performance when applying it onto their specific patient populations. Further international validation studies are needed to ensure that existing PSSs are generalizable and can be integrated into the shared treatment decision-making process. As cancer treatment keeps advancing, researchers developing a new prediction model or refining an existing one could potentially improve their algorithm's performance by using data gathered from more recent patients that are reflective of the current state of cancer care.

Assessing the accuracy of a new 3D2D registration algorithm based on a non-invasive skin marker model for navigated spine surgery.

PURPOSE: We assessed the accuracy of a new 3D2D registration algorithm to be used for navigated spine surgery and explored anatomical and radiologic parameters affecting the registration accuracy. Compared to existing 3D2D registration algorithms, the algorithm does not need bone-mounted or table-mounted instruments for registration. Neither does the intraoperative imaging device have to be tracked or calibrated. METHODS: The rigid registration algorithm required imaging data (a pre-existing CT scan (3D) and two angulated fluoroscopic images (2D)) to register positions of vertebrae in 3D and is based on non-invasive skin markers. The algorithm registered five adjacent vertebrae and was tested in the thoracic and lumbar spine from three human cadaveric specimens. The registration accuracy was calculated for each registered vertebra and measured with the target registration error (TRE) in millimeters. We used multivariable analysis to identify parameters independently affecting the algorithm's accuracy such as the angulation between the two fluoroscopic images (between 40° and 90°), the detector-skin distance, the number of skin markers applied, and waist circumference. RESULTS: The algorithm registered 780 vertebrae with a median TRE of 0.51 mm [interquartile range 0.32-0.73 mm] and a maximum TRE of 2.06 mm. The TRE was most affected by the angulation between the two fluoroscopic images obtained (p < 0.001): larger angulations resulted in higher accuracy. The algorithm was more accurate in thoracic vertebrae (p = 0.004) and in the specimen with the smallest waist circumference (p = 0.003). The algorithm registered all five adjacent vertebrae with similar accuracy. CONCLUSION: We studied the accuracy of a new 3D2D registration algorithm based on non-invasive skin markers. The algorithm registered five adjacent vertebrae with similar accuracy in the thoracic and lumbar spine and showed a maximum target registration error of approximately 2 mm. To further evaluate its potential for navigated spine surgery, the algorithm may now be integrated into a complete navigation system.

Machine learning prediction models in orthopedic surgery: A systematic review in transparent reporting.

Machine learning (ML) studies are becoming increasingly popular in orthopedics but lack a critically appraisal of their adherence to peer-reviewed guidelines. The objective of this review was to (1) evaluate quality and transparent reporting of ML prediction models in orthopedic surgery based on the transparent reporting of multivariable prediction models for individual prognosis or diagnosis (TRIPOD), and (2) assess risk of bias with the Prediction model Risk Of Bias ASsessment Tool. A systematic review was performed to identify all ML prediction studies published in orthopedic surgery through June 18th, 2020. After screening 7138 studies, 59 studies met the study criteria and were included. Two reviewers independently extracted data and discrepancies were resolved by discussion with at least two additional reviewers present. Across all studies, the overall median completeness for the TRIPOD checklist was 53% (interquartile range 47%-60%). The overall risk of bias was low in 44% (n = 26), high in 41% (n = 24), and unclear in 15% (n = 9). High overall risk of bias was driven by incomplete reporting of performance measures, inadequate handling of missing data, and use of small datasets with inadequate outcome numbers. Although the number of ML studies in orthopedic surgery is increasing rapidly, over 40% of the existing models are at high risk of bias. Furthermore, over half incompletely reported their methods and/or performance measures. Until these issues are adequately addressed to give patients and providers trust in ML models, a considerable gap remains between the development of ML prediction models and their implementation in orthopedic practice.

The Systemic Immune Response in COVID-19 Is Associated with a Shift to Formyl-Peptide Unresponsive Eosinophils.

A malfunction of the innate immune response in COVID-19 is associated with eosinopenia, particularly in more severe cases. This study tested the hypothesis that this eosinopenia is COVID-19 specific and is associated with systemic activation of eosinophils. Blood of 15 healthy controls and 75 adult patients with suspected COVID-19 at the ER were included before PCR testing and analyzed by point-of-care automated flow cytometry (CD10, CD11b, CD16, and CD62L) in the absence or presence of a formyl peptide (fNLF). Forty-five SARS-CoV-2 PCR positive patients were grouped based on disease severity. PCR negative patients with proven bacterial (n = 20) or other viral (n = 10) infections were used as disease controls. Eosinophils were identified with the use of the FlowSOM algorithm. Low blood eosinophil numbers (<100 cells/µL; p < 0.005) were found both in patients with COVID-19 and with other infectious diseases, albeit less pronounced. Two discrete eosinophil populations were identified in healthy controls both before and after activation with fNLF based on the expression of CD11b. Before activation, the CD11bbright population consisted of 5.4% (CI95% = 3.8, 13.4) of total eosinophils. After activation, this population of CD11bbright cells comprised nearly half the population (42.21%, CI95% = 35.9, 54.1). Eosinophils in COVID-19 had a similar percentage of CD11bbright cells before activation (7.6%, CI95% = 4.5, 13.6), but were clearly refractory to activation with fNLF as a much lower percentage of cells end up in the CD11bbright fraction after activation (23.7%, CI95% = 18.5, 27.6; p < 0.001). Low eosinophil numbers in COVID-19 are associated with refractoriness in responsiveness to fNLF. This might be caused by migration of fully functional cells to the tissue.

Availability and reporting quality of external validations of machine-learning prediction models with orthopedic surgical outcomes: a systematic review.

Background and purpose - External validation of machine learning (ML) prediction models is an essential step before clinical application. We assessed the proportion, performance, and transparent reporting of externally validated ML prediction models in orthopedic surgery, using the Transparent Reporting for Individual Prognosis or Diagnosis (TRIPOD) guidelines.Material and methods - We performed a systematic search using synonyms for every orthopedic specialty, ML, and external validation. The proportion was determined by using 59 ML prediction models with only internal validation in orthopedic surgical outcome published up until June 18, 2020, previously identified by our group. Model performance was evaluated using discrimination, calibration, and decision-curve analysis. The TRIPOD guidelines assessed transparent reporting.Results - We included 18 studies externally validating 10 different ML prediction models of the 59 available ML models after screening 4,682 studies. All external validations identified in this review retained good discrimination. Other key performance measures were provided in only 3 studies, rendering overall performance evaluation difficult. The overall median TRIPOD completeness was 61% (IQR 43-89), with 6 items being reported in less than 4/18 of the studies.Interpretation - Most current predictive ML models are not externally validated. The 18 available external validation studies were characterized by incomplete reporting of performance measures, limiting a transparent examination of model performance. Further prospective studies are needed to validate or refute the myriad of predictive ML models in orthopedics while adhering to existing guidelines. This ensures clinicians can take full advantage of validated and clinically implementable ML decision tools.

Flow cytometric evaluation of the neutrophil compartment in COVID-19 at hospital presentation: A normal response to an abnormal situation.

Coronavirus disease 2019 (COVID-19) is a rapidly emerging pandemic disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Critical COVID-19 is thought to be associated with a hyper-inflammatory process that can develop into acute respiratory distress syndrome, a critical disease normally mediated by dysfunctional neutrophils. This study tested the hypothesis whether the neutrophil compartment displays characteristics of hyperinflammation in COVID-19 patients. Therefore, a prospective study was performed on all patients with suspected COVID-19 presenting at the emergency room of a large academic hospital. Blood drawn within 2 d after hospital presentation was analyzed by point-of-care automated flow cytometry and compared with blood samples collected at later time points. COVID-19 patients did not exhibit neutrophilia or eosinopenia. Unexpectedly neutrophil activation markers (CD11b, CD16, CD10, and CD62L) did not differ between COVID-19-positive patients and COVID-19-negative patients diagnosed with other bacterial/viral infections, or between COVID-19 severity groups. In all patients, a decrease was found in the neutrophil maturation markers indicating an inflammation-induced left shift of the neutrophil compartment. In COVID-19 this was associated with disease severity.

An increase in CD62Ldim neutrophils precedes the development of pulmonary embolisms in COVID-19 patients.

OBJECTIVES: A high incidence of pulmonary embolism (PE) is reported in patients with critical coronavirus disease 2019 (COVID-19). Neutrophils may contribute to this through a process referred to as immunothrombosis. The aim of this study was to investigate the occurrence of neutrophil subpopulations in blood preceding the development of COVID-19 associated PE. METHODS: We studied COVID-19 patients admitted to the ICU of our tertiary hospital between 19-03-2020 and 17-05-2020. Point-of-care fully automated flow cytometry was performed prior to ICU admission, measuring the neutrophil activation/maturation markers CD10, CD11b, CD16 and CD62L. Neutrophil receptor expression was compared between patients who did or did not develop PE (as diagnosed on CT angiography) during or after their ICU stay. RESULTS: Among 25 eligible ICU patients, 22 subjects were included for analysis, of whom nine developed PE. The median (IQR) time between neutrophil phenotyping and PE occurrence was 9 (7-12) days. A significant increase in the immune-suppressive neutrophil phenotype CD16bright /CD62Ldim was observed on the day of ICU admission (P = 0.014) in patients developing PE compared to patients who did not. CONCLUSION: The increase in this neutrophil phenotype indicates that the increased number of CD16bright /CD62Ldim neutrophils might be used as prognostic marker to predict those patients that will develop PE in critical COVID-19 patients.

Thirty-day Postoperative Complications After Surgery For Metastatic Long Bone Disease Are Associated With Higher Mortality at 1 Year.

BACKGROUND: The benefits of surgical treatment of a metastasis of the extremities may be offset by drawbacks such as potential postoperative complications. For this group of patients, the primary goal of surgery is to improve quality of life in a palliative setting. A better comprehension of factors associated with complications and the impact of postoperative complications on mortality may prevent negative outcomes and help surgeons in surgical decision-making. QUESTIONS/PURPOSES: (1) What is the risk of 30-day postoperative complications after surgical treatment of osseous metastatic disease of the extremities? (2) What predisposing factors are associated with a higher risk of 30-day complications? (3) Are minor and major 30-day complications associated with higher mortality at 1 year? METHODS: Between 1999 and 2016, 1090 patients with osseous metastatic disease of the long bones treated surgically at our institution were retrospectively included in the study. Surgery included intramedullary nailing (58%), endoprosthetic reconstruction (22%), plate-screw fixation (14%), dynamic hip screw fixation (2%), and combined approaches (4%). Surgery was performed if patients were deemed healthy enough to proceed to surgery and wished to undergo surgery. All data were retrieved by manually reviewing patients' records. The overall frequency of complications, which were defined using the Clavien-Dindo classification system, was calculated. We did not include Grade I complications as postoperative complications and complications were divided into minor (Grade II) and major (Grades III-V) complications. A multivariate logistic regression analysis was used to identify factors associated with 30-day postoperative complications. A Cox regression analysis was used to assess the association between postoperative complications and overall survival. RESULTS: Overall, 31% of the patients (333 of 1090) had a postoperative complication within 30 days. The following factors were independently associated with 30-day postoperative complications: rapidly growing primary tumors classified according to the modified Katagiri classification (odds ratio 1.6; 95% confidence interval, 1.1-2.2; p = 0.011), multiple bone metastases (OR 1.6; 95% CI, 1.1-2.3; p = 0.008), pathologic fracture (OR 1.5; 95% CI, 1.1-2.0; p = 0.010), lower-extremity location (OR 2.2; 95% CI, 1.6-3.2; p < 0.001), hypoalbuminemia (OR 1.7; 95% CI, 1.2-2.4; p = 0.002), hyponatremia (OR 1.5; 95% CI, 1.0-2.2; p = 0.044), and elevated white blood cell count (OR 1.6; 95% CI, 1.1-2.4; p = 0.007). Minor and major postoperative complications within 30 days after surgery were both associated with greater 1-year mortality (hazard ratio 1.6; 95% CI, 1.3-1.8; p < 0.001 and HR 3.4; 95% CI, 2.8-4.2, respectively; p < 0.001). CONCLUSION: Patients with metastatic disease in the long bones are vulnerable to postoperative adverse events. When selecting patients for surgery, surgeons should carefully assess a patient's cancer status, and several preoperative laboratory values should be part of the standard work-up before surgery. Furthermore, 30-day postoperative complications decrease survival within 1 year after surgery. Therefore, patients at a high risk of having postoperative complications are less likely to profit from surgery and should be considered for nonoperative treatment or be monitored closely after surgery. LEVEL OF EVIDENCE: Level III, therapeutic study.

Development and Internal Validation of Machine Learning Algorithms for Preoperative Survival Prediction of Extremity Metastatic Disease.

BACKGROUND: A preoperative estimation of survival is critical for deciding on the operative management of metastatic bone disease of the extremities. Several tools have been developed for this purpose, but there is room for improvement. Machine learning is an increasingly popular and flexible method of prediction model building based on a data set. It raises some skepticism, however, because of the complex structure of these models. QUESTIONS/PURPOSES: The purposes of this study were (1) to develop machine learning algorithms for 90-day and 1-year survival in patients who received surgical treatment for a bone metastasis of the extremity, and (2) to use these algorithms to identify those clinical factors (demographic, treatment related, or surgical) that are most closely associated with survival after surgery in these patients. METHODS: All 1090 patients who underwent surgical treatment for a long-bone metastasis at two institutions between 1999 and 2017 were included in this retrospective study. The median age of the patients in the cohort was 63 years (interquartile range [IQR] 54 to 72 years), 56% of patients (610 of 1090) were female, and the median BMI was 27 kg/m (IQR 23 to 30 kg/m). The most affected location was the femur (70%), followed by the humerus (22%). The most common primary tumors were breast (24%) and lung (23%). Intramedullary nailing was the most commonly performed type of surgery (58%), followed by endoprosthetic reconstruction (22%), and plate screw fixation (14%). Missing data were imputed using the missForest methods. Features were selected by random forest algorithms, and five different models were developed on the training set (80% of the data): stochastic gradient boosting, random forest, support vector machine, neural network, and penalized logistic regression. These models were chosen as a result of their classification capability in binary datasets. Model performance was assessed on both the training set and the validation set (20% of the data) by discrimination, calibration, and overall performance. RESULTS: We found no differences among the five models for discrimination, with an area under the curve ranging from 0.86 to 0.87. All models were well calibrated, with intercepts ranging from -0.03 to 0.08 and slopes ranging from 1.03 to 1.12. Brier scores ranged from 0.13 to 0.14. The stochastic gradient boosting model was chosen to be deployed as freely available web-based application and explanations on both a global and an individual level were provided. For 90-day survival, the three most important factors associated with poorer survivorship were lower albumin level, higher neutrophil-to-lymphocyte ratio, and rapid growth primary tumor. For 1-year survival, the three most important factors associated with poorer survivorship were lower albumin level, rapid growth primary tumor, and lower hemoglobin level. CONCLUSIONS: Although the final models must be externally validated, the algorithms showed good performance on internal validation. The final models have been incorporated into a freely accessible web application that can be found at https://sorg-apps.shinyapps.io/extremitymetssurvival/. Pending external validation, clinicians may use this tool to predict survival for their individual patients to help in shared treatment decision making. LEVEL OF EVIDENCE: Level III, therapeutic study.