Using machine learning to predict internal rotation after anatomic and reverse total shoulder arthroplasty.

BACKGROUND: Improvement in internal rotation (IR) after anatomic (aTSA) and reverse (rTSA) total shoulder arthroplasty is difficult to predict, with rTSA patients experiencing greater variability and more limited IR improvements than aTSA patients. The purpose of this study is to quantify and compare the IR score for aTSA and rTSA patients and create supervised machine learning that predicts IR after aTSA and rTSA at multiple postoperative time points. METHODS: Clinical data from 2270 aTSA and 4198 rTSA patients were analyzed using 3 supervised machine learning techniques to create predictive models for internal rotation as measured by the IR score at 6 postoperative time points. Predictions were performed using the full input feature set and 2 minimal input feature sets. The mean absolute error (MAE) quantified the difference between actual and predicted IR scores for each model at each time point. The predictive accuracy of the XGBoost algorithm was also quantified by its ability to distinguish which patients would achieve clinical improvement greater than the minimal clinically important difference (MCID) and substantial clinical benefit (SCB) patient satisfaction thresholds for IR score at 2-3 years after surgery. RESULTS: rTSA patients had significantly lower mean IR scores and significantly less mean IR score improvement than aTSA patients at each postoperative time point. Both aTSA and rTSA patients experienced significant improvements in their ability to perform activities of daily living (ADLs); however, aTSA patients were significantly more likely to perform these ADLs. Using a minimal feature set of preoperative inputs, our machine learning algorithms had equivalent accuracy when predicting IR score for both aTSA (0.92-1.18 MAE) and rTSA (1.03-1.25 MAE) from 3 months to >5 years after surgery. Furthermore, these predictive algorithms identified with 90% accuracy for aTSA and 85% accuracy for rTSA which patients will achieve MCID IR score improvement and predicted with 85% accuracy for aTSA patients and 77% accuracy for rTSA which patients will achieve SCB IR score improvement at 2-3 years after surgery. DISCUSSION: Our machine learning study demonstrates that active internal rotation can be accurately predicted after aTSA and rTSA at multiple postoperative time points using a minimal feature set of preoperative inputs. These predictive algorithms accurately identified which patients will, and will not, achieve clinical improvement in IR score that exceeds the MCID and SCB patient satisfaction thresholds.

Validation of a machine learning-derived clinical metric to quantify outcomes after total shoulder arthroplasty.

BACKGROUND: We propose a new clinical assessment tool constructed using machine learning, called the Shoulder Arthroplasty Smart (SAS) score to quantify outcomes following total shoulder arthroplasty (TSA). METHODS: Clinical data from 3667 TSA patients with 8104 postoperative follow-up reports were used to quantify the psychometric properties of validity, responsiveness, and clinical interpretability for the proposed SAS score and each of the Simple Shoulder Test (SST), Constant, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form (ASES), University of California Los Angeles (UCLA), and Shoulder Pain and Disability Index (SPADI) scores. RESULTS: Convergent construct validity was demonstrated, with all 6 outcome measures being moderately to highly correlated preoperatively and highly correlated postoperatively when quantifying TSA outcomes. The SAS score was most correlated with the UCLA score and least correlated with the SST. No clinical outcome score exhibited significant floor effects preoperatively or postoperatively or significant ceiling effects preoperatively; however, significant ceiling effects occurred postoperatively for each of the SST (44.3%), UCLA (13.9%), ASES (18.7%), and SPADI (19.3%) measures. Ceiling effects were more pronounced for anatomic than reverse TSA, and generally, men, younger patients, and whites who received TSA were more likely to experience a ceiling effect than TSA patients who were female, older, and of non-white race or ethnicity. The SAS score had the least number of patients with floor and ceiling effects and also exhibited no response bias in any patient characteristic analyzed in this study. Regarding clinical interpretability, patient satisfaction anchor-based thresholds for minimal clinically importance difference and substantial clinical benefit were quantified for all 6 outcome measures; the SAS score thresholds were most similar in magnitude to the Constant score. Regarding responsiveness, all 6 outcome measures detected a large effect, with the UCLA exhibiting the most responsiveness and the SST exhibiting the least. Finally, each of the SAS, ASES, Constant, and SPADI scores had similarly large standardized response mean and effect size responsiveness. DISCUSSION: The 6-question SAS score is an efficient TSA-specific outcome measure with equivalent or better validity, responsiveness, and clinical interpretability as 5 other historical assessment tools. The SAS score has an appropriate response range without floor or ceiling effects and without bias in any target patient characteristic, unlike the age, gender, or race/ethnicity bias observed in the ceiling scores with the other outcome measures. Because of these substantial benefits, we recommend the use of the new SAS score for quantifying TSA outcomes.

Using machine learning to predict clinical outcomes after shoulder arthroplasty with a minimal feature set.

BACKGROUND: A machine learning analysis was conducted on 5774 shoulder arthroplasty patients to create predictive models for multiple clinical outcome measures after anatomic total shoulder arthroplasty (aTSA) and reverse total shoulder arthroplasty (rTSA). The goal of this study was to compare the accuracy associated with a full-feature set predictive model (ie, full model, comprising 291 parameters) and a minimal-feature set model (ie, abbreviated model, comprising 19 input parameters) to predict clinical outcomes to assess the efficacy of using a minimal feature set of inputs as a shoulder arthroplasty clinical decision-support tool. METHODS: Clinical data from 2153 primary aTSA patients and 3621 primary rTSA patients were analyzed using the XGBoost machine learning technique to create and test predictive models for multiple outcome measures at different postoperative time points via the full and abbreviated models. Mean absolute errors (MAEs) quantified the difference between actual and predicted outcomes, and each model also predicted whether a patient would experience clinical improvement greater than the patient satisfaction anchor-based thresholds of the minimal clinically important difference and substantial clinical benefit for each outcome measure at 2-3 years after surgery. RESULTS: Across all postoperative time points analyzed, the full and abbreviated models had similar MAEs for the American Shoulder and Elbow Surgeons score (±11.7 with full model vs. ±12.0 with abbreviated model), Constant score (±8.9 vs. ±9.8), Global Shoulder Function score (±1.4 vs. ±1.5), visual analog scale pain score (±1.3 vs. ±1.4), active abduction (±20.4° vs. ±21.8°), forward elevation (±17.6° vs. ±19.2°), and external rotation (±12.2° vs. ±12.6°). Marginal improvements in MAEs were observed for each outcome measure prediction when the abbreviated model was supplemented with data on implant size and/or type and measurements of native glenoid anatomy. The full and abbreviated models each effectively risk stratified patients using only preoperative data by accurately identifying patients with improvement greater than the minimal clinically important difference and substantial clinical benefit thresholds. DISCUSSION: Our study showed that the full and abbreviated machine learning models achieved similar accuracy in predicting clinical outcomes after aTSA and rTSA at multiple postoperative time points. These promising results demonstrate an efficient utilization of machine learning algorithms to predict clinical outcomes. Our findings using a minimal feature set of only 19 preoperative inputs suggest that this tool may be easily used during a surgical consultation to improve decision making related to shoulder arthroplasty.

What Is the Accuracy of Three Different Machine Learning Techniques to Predict Clinical Outcomes After Shoulder Arthroplasty?

BACKGROUND: Machine learning techniques can identify complex relationships in large healthcare datasets and build prediction models that better inform physicians in ways that can assist in patient treatment decision-making. In the domain of shoulder arthroplasty, machine learning appears to have the potential to anticipate patients' results after surgery, but this has not been well explored. QUESTIONS/PURPOSES: (1) What is the accuracy of machine learning to predict the American Shoulder and Elbow Surgery (ASES), University of California Los Angeles (UCLA), Constant, global shoulder function, and VAS pain scores, as well as active abduction, forward flexion, and external rotation at 1 year, 2 to 3 years, 3 to 5 years, and more than 5 years after anatomic total shoulder arthroplasty (aTSA) or reverse total shoulder arthroplasty (rTSA)? (2) What is the accuracy of machine learning to identify whether a patient will achieve clinical improvement that exceeds the minimum clinically important difference (MCID) threshold for each outcome measure? (3) What is the accuracy of machine learning to identify whether a patient will achieve clinical improvement that exceeds the substantial clinical benefit threshold for each outcome measure? METHODS: A machine learning analysis was conducted on a database of 7811 patients undergoing shoulder arthroplasty of one prosthesis design to create predictive models for multiple clinical outcome measures. Excluding patients with revisions, fracture indications, and hemiarthroplasty resulted in 6210 eligible primary aTSA and rTSA patients, of whom 4782 patients with 11,198 postoperative follow-up visits had sufficient preoperative, intraoperative, and postoperative data to train and test the predictive models. Preoperative clinical data from 1895 primary aTSA patients and 2887 primary rTSA patients were analyzed using three commercially available supervised machine learning techniques: linear regression, XGBoost, and Wide and Deep, to train and test predictive models for the ASES, UCLA, Constant, global shoulder function, and VAS pain scores, as well as active abduction, forward flexion, and external rotation. Our primary study goal was to quantify the accuracy of three machine learning techniques to predict each outcome measure at multiple postoperative timepoints after aTSA and rTSA using the mean absolute error between the actual and predicted values. Our secondary study goals were to identify whether a patient would experience clinical improvement greater than the MCID and substantial clinical benefit anchor-based thresholds of patient satisfaction for each outcome measure as quantified by the model classification parameters of precision, recall, accuracy, and area under the receiver operating curve. RESULTS: Each machine learning technique demonstrated similar accuracy to predict each outcome measure at each postoperative point for both aTSA and rTSA, though small differences in prediction accuracy were observed between techniques. Across all postsurgical timepoints, the Wide and Deep technique was associated with the smallest mean absolute error and predicted the postoperative ASES score to ± 10.1 to 11.3 points, the UCLA score to ± 2.5 to 3.4, the Constant score to ± 7.3 to 7.9, the global shoulder function score to ± 1.0 to 1.4, the VAS pain score to ± 1.2 to 1.4, active abduction to ± 18 to 21°, forward elevation to ± 15 to 17°, and external rotation to ± 10 to 12°. These models also accurately identified the patients who did and did not achieve clinical improvement that exceeded the MCID (93% to 99% accuracy for patient-reported outcome measures (PROMs) and 85% to 94% for pain, function, and ROM measures) and substantial clinical benefit (82% to 93% accuracy for PROMs and 78% to 90% for pain, function, and ROM measures) thresholds. CONCLUSIONS: Machine learning techniques can use preoperative data to accurately predict clinical outcomes at multiple postoperative points after shoulder arthroplasty and accurately risk-stratify patients by preoperatively identifying who may and who may not achieve MCID and substantial clinical benefit improvement thresholds for each outcome measure. CLINICAL RELEVANCE: Three different commercially available machine learning techniques were used to train and test models that predicted clinical outcomes after aTSA and rTSA; this device-type comparison was performed to demonstrate how predictive modeling techniques can be used in the near future to help answer unsolved clinical questions and augment decision-making to improve outcomes after shoulder arthroplasty.

Development and Prospective Validation of a Machine Learning-Based Risk of Readmission Model in a Large Military Hospital.

BACKGROUND: Thirty-day hospital readmissions are a quality metric for health care systems. Predictive models aim to identify patients likely to readmit to more effectively target preventive strategies. Many risk of readmission models have been developed on retrospective data, but prospective validation of readmission models is rare. To the best of our knowledge, none of these developed models have been evaluated or prospectively validated in a military hospital. OBJECTIVES: The objectives of this study are to demonstrate the development and prospective validation of machine learning (ML) risk of readmission models to be utilized by clinical staff at a military medical facility and demonstrate the collaboration between the U.S. Department of Defense's integrated health care system and a private company. METHODS: We evaluated multiple ML algorithms to develop a predictive model for 30-day readmissions using data from a retrospective cohort of all-cause inpatient readmissions at Madigan Army Medical Center (MAMC). This predictive model was then validated on prospective MAMC patient data. Precision, recall, accuracy, and the area under the receiver operating characteristic curve (AUC) were used to evaluate model performance. The model was revised, retrained, and rescored on additional retrospective MAMC data after the prospective model's initial performance was evaluated. RESULTS: Within the initial retrospective cohort, which included 32,659 patient encounters, the model achieved an AUC of 0.68. During prospective scoring, 1,574 patients were scored, of whom 152 were readmitted within 30 days of discharge, with an all-cause readmission rate of 9.7%. The AUC of the prospective predictive model was 0.64. The model achieved an AUC of 0.76 after revision and addition of further retrospective data. CONCLUSION: This work reflects significant collaborative efforts required to operationalize ML models in a complex clinical environment such as that seen in an integrated health care system and the importance of prospective model validation.