Evaluating the fairness and accuracy of machine learning-based predictions of clinical outcomes after anatomic and reverse total shoulder arthroplasty.

BACKGROUND: Machine learning (ML)-based clinical decision support tools (CDSTs) make personalized predictions for different treatments; by comparing predictions of multiple treatments, these tools can be used to optimize decision making for a particular patient. However, CDST prediction accuracy varies for different patients and also for different treatment options. If these differences are sufficiently large and consistent for a particular subcohort of patients, then that bias may result in those patients not receiving a particular treatment. Such level of bias would deem the CDST "unfair." The purpose of this study is to evaluate the "fairness" of ML CDST-based clinical outcomes predictions after anatomic (aTSA) and reverse total shoulder arthroplasty (rTSA) for patients of different demographic attributes. METHODS: Clinical data from 8280 shoulder arthroplasty patients with 19,249 postoperative visits was used to evaluate the prediction fairness and accuracy associated with the following patient demographic attributes: ethnicity, sex, and age at the time of surgery. Performance of clinical outcome and range of motion regression predictions were quantified by the mean absolute error (MAE) and performance of minimal clinically important difference (MCID) and substantial clinical benefit classification predictions were quantified by accuracy, sensitivity, and the F1 score. Fairness of classification predictions leveraged the "four-fifths" legal guideline from the US Equal Employment Opportunity Commission and fairness of regression predictions leveraged established MCID thresholds associated with each outcome measure. RESULTS: For both aTSA and rTSA clinical outcome predictions, only minor differences in MAE were observed between patients of different ethnicity, sex, and age. Evaluation of prediction fairness demonstrated that 0 of 486 MCID (0%) and only 3 of 486 substantial clinical benefit (0.6%) classification predictions were outside the 20% fairness boundary and only 14 of 972 (1.4%) regression predictions were outside of the MCID fairness boundary. Hispanic and Black patients were more likely to have ML predictions out of fairness tolerance for aTSA and rTSA. Additionally, patients <60 years old were more likely to have ML predictions out of fairness tolerance for rTSA. No disparate predictions were identified for sex and no disparate regression predictions were observed for forward elevation, internal rotation score, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form score, or global shoulder function. CONCLUSION: The ML algorithms analyzed in this study accurately predict clinical outcomes after aTSA and rTSA for patients of different ethnicity, sex, and age, where only 1.4% of regression predictions and only 0.3% of classification predictions were out of fairness tolerance using the proposed fairness evaluation method and acceptance criteria. Future work is required to externally validate these ML algorithms to ensure they are equally accurate for all legally protected patient groups.

Using machine learning to predict clinical outcomes after shoulder arthroplasty with a minimal feature set.

BACKGROUND: A machine learning analysis was conducted on 5774 shoulder arthroplasty patients to create predictive models for multiple clinical outcome measures after anatomic total shoulder arthroplasty (aTSA) and reverse total shoulder arthroplasty (rTSA). The goal of this study was to compare the accuracy associated with a full-feature set predictive model (ie, full model, comprising 291 parameters) and a minimal-feature set model (ie, abbreviated model, comprising 19 input parameters) to predict clinical outcomes to assess the efficacy of using a minimal feature set of inputs as a shoulder arthroplasty clinical decision-support tool. METHODS: Clinical data from 2153 primary aTSA patients and 3621 primary rTSA patients were analyzed using the XGBoost machine learning technique to create and test predictive models for multiple outcome measures at different postoperative time points via the full and abbreviated models. Mean absolute errors (MAEs) quantified the difference between actual and predicted outcomes, and each model also predicted whether a patient would experience clinical improvement greater than the patient satisfaction anchor-based thresholds of the minimal clinically important difference and substantial clinical benefit for each outcome measure at 2-3 years after surgery. RESULTS: Across all postoperative time points analyzed, the full and abbreviated models had similar MAEs for the American Shoulder and Elbow Surgeons score (±11.7 with full model vs. ±12.0 with abbreviated model), Constant score (±8.9 vs. ±9.8), Global Shoulder Function score (±1.4 vs. ±1.5), visual analog scale pain score (±1.3 vs. ±1.4), active abduction (±20.4° vs. ±21.8°), forward elevation (±17.6° vs. ±19.2°), and external rotation (±12.2° vs. ±12.6°). Marginal improvements in MAEs were observed for each outcome measure prediction when the abbreviated model was supplemented with data on implant size and/or type and measurements of native glenoid anatomy. The full and abbreviated models each effectively risk stratified patients using only preoperative data by accurately identifying patients with improvement greater than the minimal clinically important difference and substantial clinical benefit thresholds. DISCUSSION: Our study showed that the full and abbreviated machine learning models achieved similar accuracy in predicting clinical outcomes after aTSA and rTSA at multiple postoperative time points. These promising results demonstrate an efficient utilization of machine learning algorithms to predict clinical outcomes. Our findings using a minimal feature set of only 19 preoperative inputs suggest that this tool may be easily used during a surgical consultation to improve decision making related to shoulder arthroplasty.

Validation of a machine learning-derived clinical metric to quantify outcomes after total shoulder arthroplasty.

BACKGROUND: We propose a new clinical assessment tool constructed using machine learning, called the Shoulder Arthroplasty Smart (SAS) score to quantify outcomes following total shoulder arthroplasty (TSA). METHODS: Clinical data from 3667 TSA patients with 8104 postoperative follow-up reports were used to quantify the psychometric properties of validity, responsiveness, and clinical interpretability for the proposed SAS score and each of the Simple Shoulder Test (SST), Constant, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form (ASES), University of California Los Angeles (UCLA), and Shoulder Pain and Disability Index (SPADI) scores. RESULTS: Convergent construct validity was demonstrated, with all 6 outcome measures being moderately to highly correlated preoperatively and highly correlated postoperatively when quantifying TSA outcomes. The SAS score was most correlated with the UCLA score and least correlated with the SST. No clinical outcome score exhibited significant floor effects preoperatively or postoperatively or significant ceiling effects preoperatively; however, significant ceiling effects occurred postoperatively for each of the SST (44.3%), UCLA (13.9%), ASES (18.7%), and SPADI (19.3%) measures. Ceiling effects were more pronounced for anatomic than reverse TSA, and generally, men, younger patients, and whites who received TSA were more likely to experience a ceiling effect than TSA patients who were female, older, and of non-white race or ethnicity. The SAS score had the least number of patients with floor and ceiling effects and also exhibited no response bias in any patient characteristic analyzed in this study. Regarding clinical interpretability, patient satisfaction anchor-based thresholds for minimal clinically importance difference and substantial clinical benefit were quantified for all 6 outcome measures; the SAS score thresholds were most similar in magnitude to the Constant score. Regarding responsiveness, all 6 outcome measures detected a large effect, with the UCLA exhibiting the most responsiveness and the SST exhibiting the least. Finally, each of the SAS, ASES, Constant, and SPADI scores had similarly large standardized response mean and effect size responsiveness. DISCUSSION: The 6-question SAS score is an efficient TSA-specific outcome measure with equivalent or better validity, responsiveness, and clinical interpretability as 5 other historical assessment tools. The SAS score has an appropriate response range without floor or ceiling effects and without bias in any target patient characteristic, unlike the age, gender, or race/ethnicity bias observed in the ceiling scores with the other outcome measures. Because of these substantial benefits, we recommend the use of the new SAS score for quantifying TSA outcomes.

What Is the Accuracy of Three Different Machine Learning Techniques to Predict Clinical Outcomes After Shoulder Arthroplasty?

BACKGROUND: Machine learning techniques can identify complex relationships in large healthcare datasets and build prediction models that better inform physicians in ways that can assist in patient treatment decision-making. In the domain of shoulder arthroplasty, machine learning appears to have the potential to anticipate patients' results after surgery, but this has not been well explored. QUESTIONS/PURPOSES: (1) What is the accuracy of machine learning to predict the American Shoulder and Elbow Surgery (ASES), University of California Los Angeles (UCLA), Constant, global shoulder function, and VAS pain scores, as well as active abduction, forward flexion, and external rotation at 1 year, 2 to 3 years, 3 to 5 years, and more than 5 years after anatomic total shoulder arthroplasty (aTSA) or reverse total shoulder arthroplasty (rTSA)? (2) What is the accuracy of machine learning to identify whether a patient will achieve clinical improvement that exceeds the minimum clinically important difference (MCID) threshold for each outcome measure? (3) What is the accuracy of machine learning to identify whether a patient will achieve clinical improvement that exceeds the substantial clinical benefit threshold for each outcome measure? METHODS: A machine learning analysis was conducted on a database of 7811 patients undergoing shoulder arthroplasty of one prosthesis design to create predictive models for multiple clinical outcome measures. Excluding patients with revisions, fracture indications, and hemiarthroplasty resulted in 6210 eligible primary aTSA and rTSA patients, of whom 4782 patients with 11,198 postoperative follow-up visits had sufficient preoperative, intraoperative, and postoperative data to train and test the predictive models. Preoperative clinical data from 1895 primary aTSA patients and 2887 primary rTSA patients were analyzed using three commercially available supervised machine learning techniques: linear regression, XGBoost, and Wide and Deep, to train and test predictive models for the ASES, UCLA, Constant, global shoulder function, and VAS pain scores, as well as active abduction, forward flexion, and external rotation. Our primary study goal was to quantify the accuracy of three machine learning techniques to predict each outcome measure at multiple postoperative timepoints after aTSA and rTSA using the mean absolute error between the actual and predicted values. Our secondary study goals were to identify whether a patient would experience clinical improvement greater than the MCID and substantial clinical benefit anchor-based thresholds of patient satisfaction for each outcome measure as quantified by the model classification parameters of precision, recall, accuracy, and area under the receiver operating curve. RESULTS: Each machine learning technique demonstrated similar accuracy to predict each outcome measure at each postoperative point for both aTSA and rTSA, though small differences in prediction accuracy were observed between techniques. Across all postsurgical timepoints, the Wide and Deep technique was associated with the smallest mean absolute error and predicted the postoperative ASES score to ± 10.1 to 11.3 points, the UCLA score to ± 2.5 to 3.4, the Constant score to ± 7.3 to 7.9, the global shoulder function score to ± 1.0 to 1.4, the VAS pain score to ± 1.2 to 1.4, active abduction to ± 18 to 21°, forward elevation to ± 15 to 17°, and external rotation to ± 10 to 12°. These models also accurately identified the patients who did and did not achieve clinical improvement that exceeded the MCID (93% to 99% accuracy for patient-reported outcome measures (PROMs) and 85% to 94% for pain, function, and ROM measures) and substantial clinical benefit (82% to 93% accuracy for PROMs and 78% to 90% for pain, function, and ROM measures) thresholds. CONCLUSIONS: Machine learning techniques can use preoperative data to accurately predict clinical outcomes at multiple postoperative points after shoulder arthroplasty and accurately risk-stratify patients by preoperatively identifying who may and who may not achieve MCID and substantial clinical benefit improvement thresholds for each outcome measure. CLINICAL RELEVANCE: Three different commercially available machine learning techniques were used to train and test models that predicted clinical outcomes after aTSA and rTSA; this device-type comparison was performed to demonstrate how predictive modeling techniques can be used in the near future to help answer unsolved clinical questions and augment decision-making to improve outcomes after shoulder arthroplasty.

The cognitive effects of opioids.

Successful opioid therapy often depends on achieving a balance between analgesic effectiveness and side effects. The risk of opioid-induced cognitive impairment often hinders clinicians and patients from initiating or optimizing opioid therapy. Despite subjective experiences of mental dullness and sedation, objective tests of cognitive functioning do not always demonstrate marked changes following opioid administration. To guide clinical practice, as well as patient and family teaching, pain management nurses should be familiar with literature regarding this topic. The purpose of this article is to review the empiric literature on opioids and cognitive functioning, including the relationships among pain, cognition, delirium, and opioids. In general, research reflects minimal to no significant impairments in cognitive functioning. If impairment does occur, it is most often associated with parenteral opioids administered to opioid-naive individuals. Some evidence suggests that opioids may actually enhance cognitive function and decrease delirium in some patient populations. This article describes this research and explores the clinical implications of the research in this area.

Etanercept in the treatment of adult patients with Still's disease.

OBJECTIVE: To evaluate the safety and efficacy of etanercept in the treatment of adult patients with Still's disease. METHODS: Twelve adult patients who met criteria for Still's disease and had active arthritis were enrolled in a 6-month open-label trial of etanercept given in biweekly doses of 25 mg. The mean disease duration at study entry was 10.7 years. All patients had been treated unsuccessfully with other disease-modifying antirheumatic drugs. Efficacy was evaluated according to American College of Rheumatology (ACR) improvement criteria, and adverse events were recorded. RESULTS: Ten patients successfully completed the study; 2 withdrew due to disease flare. In 4 patients, the dosage of etanercept was increased from 25 mg biweekly to 25 mg 3 times per week. Seven patients met ACR 20% response criteria. Of these 7 responders, 4 met ACR 50% response criteria and 2 met ACR 70% response criteria. Among the 3 patients with systemic features of Still's disease (fever and rash), improvement in these features was seen in 1; the arthritis did not improve in any of these 3 patients. Except in the 2 patients who withdrew due to disease flare (rash, fever, and arthritis), no other significant adverse events occurred. CONCLUSION: In this initial study of etanercept therapy for Still's disease in the adult, this treatment resulted in improvement in the arthritis and was well tolerated. Additional trials should be performed to elucidate the effects of tumor necrosis factor inhibitors in Still's disease.

Rapid magnetic resonance imaging vs radiographs for patients with low back pain: a randomized controlled trial.

CONTEXT: Faster magnetic resonance imaging (MRI) scanning has made MRI a potential cost-effective replacement for radiographs for patients with low back pain. However, whether rapid MRI scanning results in better patient outcomes than radiographic evaluation or a cost-effective alternative is unknown. OBJECTIVE: To determine the clinical and economic consequences of replacing spine radiographs with rapid MRI for primary care patients. DESIGN, SETTING, AND PATIENTS: Randomized controlled trial of 380 patients aged 18 years or older whose primary physicians had ordered that their low back pain be evaluated by radiographs. The patients were recruited between November 1998 and June 2000 from 1 of 4 imaging centers in the Seattle, Wash, area: a university-based teaching program, a nonuniversity-based teaching program, and 2 private clinics. INTERVENTION: Patients were randomly assigned to receive lumbar spine evaluation by rapid MRI or by radiograph. MAIN OUTCOME MEASURES: Back-related disability measured by the modified Roland questionnaire. Secondary outcomes included Medical Outcomes Study 36-Item Short Form Health Survey (SF-36), pain, preference scores, satisfaction, and costs. RESULTS: At 12 months, primary outcomes of functional disability were obtained from 337 (89%) of the 380 patients enrolled. The mean back-related disability modified Roland score for the 170 patients assigned to the radiograph evaluation group was 8.75 vs 9.34 for the 167 patients assigned the rapid MRI evaluation group (mean difference, -0.59; 95% CI, -1.69 to 0.87). The mean differences in the secondary outcomes were not statistically significant : pain bothersomeness (0.07; 95% CI -0.88 to 1.22), pain frequency (0.12; 95% CI, -0.69 to 1.37), and SF-36 subscales of bodily pain (1.25; 95% CI, -4.46 to 4.96), and physical functioning (2.73, 95% CI -4.09 to 6.22). Ten patients in the rapid MRI group vs 4 in the radiograph group had lumbar spine operations (risk difference, 0.34; 95% CI, -0.06 to 0.73). The rapid MRI strategy had a mean cost of 2380 dollars vs 2059 dollars for the radiograph strategy (mean difference, 321 dollars; 95% CI, -1100 to 458). CONCLUSIONS: Rapid MRIs and radiographs resulted in nearly identical outcomes for primary care patients with low back pain. Although physicians and patients preferred the rapid MRI, substituting rapid MRI for radiographic evaluations in the primary care setting may offer little additional benefit to patients, and it may increase the costs of care because of the increased number of spine operations that patients are likely to undergo.