"Dr. AI Will See You Now": How Do ChatGPT-4 Treatment Recommendations Align With Orthopaedic Clinical Practice Guidelines?

BACKGROUND: Artificial intelligence (AI) is engineered to emulate tasks that have historically required human interaction and intellect, including learning, pattern recognition, decision-making, and problem-solving. Although AI models like ChatGPT-4 have demonstrated satisfactory performance on medical licensing exams, suggesting a potential for supporting medical diagnostics and decision-making, no study of which we are aware has evaluated the ability of these tools to make treatment recommendations when given clinical vignettes and representative medical imaging of common orthopaedic conditions. As AI continues to advance, a thorough understanding of its strengths and limitations is necessary to inform safe and helpful integration into medical practice. QUESTIONS/PURPOSES: (1) What is the concordance between ChatGPT-4-generated treatment recommendations for common orthopaedic conditions with both the American Academy of Orthopaedic Surgeons (AAOS) clinical practice guidelines (CPGs) and an orthopaedic attending physician's treatment plan? (2) In what specific areas do the ChatGPT-4-generated treatment recommendations diverge from the AAOS CPGs? METHODS: Ten common orthopaedic conditions with associated AAOS CPGs were identified: carpal tunnel syndrome, distal radius fracture, glenohumeral joint osteoarthritis, rotator cuff injury, clavicle fracture, hip fracture, hip osteoarthritis, knee osteoarthritis, ACL injury, and acute Achilles rupture. For each condition, the medical records of 10 deidentified patients managed at our facility were used to construct clinical vignettes that each had an isolated, single diagnosis with adequate clarity. The vignettes also encompassed a range of diagnostic severity to evaluate more thoroughly adherence to the treatment guidelines outlined by the AAOS. These clinical vignettes were presented alongside representative radiographic imaging. The model was prompted to provide a single treatment plan recommendation. Each treatment plan was compared with established AAOS CPGs and to the treatment plan documented by the attending orthopaedic surgeon treating the specific patient. Vignettes where ChatGPT-4 recommendations diverged from CPGs were reviewed to identify patterns of error and summarized. RESULTS: ChatGPT-4 provided treatment recommendations in accordance with the AAOS CPGs in 90% (90 of 100) of clinical vignettes. Concordance between ChatGPT-generated plans and the plan recommended by the treating orthopaedic attending physician was 78% (78 of 100). One hundred percent (30 of 30) of ChatGPT-4 recommendations for fracture vignettes and hip and knee arthritis vignettes matched with CPG recommendations, whereas the model struggled most with recommendations for carpal tunnel syndrome (3 of 10 instances demonstrated discordance). ChatGPT-4 recommendations diverged from AAOS CPGs for three carpal tunnel syndrome vignettes; two ACL injury, rotator cuff injury, and glenohumeral joint osteoarthritis vignettes; as well as one acute Achilles rupture vignette. In these situations, ChatGPT-4 most often struggled to correctly interpret injury severity and progression, incorporate patient factors (such as lifestyle or comorbidities) into decision-making, and recognize a contraindication to surgery. CONCLUSION: ChatGPT-4 can generate accurate treatment plans aligned with CPGs but can also make mistakes when it is required to integrate multiple patient factors into decision-making and understand disease severity and progression. Physicians must critically assess the full clinical picture when using AI tools to support their decision-making. CLINICAL RELEVANCE: ChatGPT-4 may be used as an on-demand diagnostic companion, but patient-centered decision-making should continue to remain in the hands of the physician.

Development and Implementation of an Experiential Longitudinal Health Systems Science Thread Into an Existing Medical School Curriculum.

PROBLEM: Medical school graduates enter a complex health care delivery system involving interprofessional teamwork and multifaceted value-based patient care decisions. However, current curricula on health systems science (HSS) are piecemeal, lecture based, and confined to preclinical training. APPROACH: The VISTA program is a longitudinal, immersive learning curriculum integrated into the University of Chicago Pritzker School of Medicine curriculum between 2016 and 2018. Key components include a unit-based nursing interprofessional team experience, a discharge objective structured clinical examination (OSCE), a patient safety simulation, and the implementation of a Choosing Wisely SmartPhrase. Graduates before 2016-2017 and after 2018-2020 VISTA implementation completed a Likert-style survey assessing attitudes, knowledge, and behaviors on HSS topics. A free response question solicited improvement areas. The Association of American Medical Colleges (AAMC) Graduation Questionnaire (GQ) was also examined. OUTCOMES: The overall VISTA survey response rate was 59%, with 126 fourth-year medical student respondents before VISTA and 120 after VISTA. Compared with pre-VISTA graduates, post-VISTA graduates reported a significantly higher rate of competence on the HSS questions, with the greatest increases seen in effective communication at discharge (n = 73/126 [57.9%] to 116/120 [96.7%], P < .001), knowledge on safety event reporting (n = 53/126 [42.1%] to 96/120 [79.8%], P < .001), and considering costs in making health care decisions (n = 76/126 [60.3%] to 117/120 [97.5%], P < .001). All were directly addressed through experiential learning interventions, and 2 were intended practice behaviors. VISTA graduate responses to free-text questions demonstrated a more nuanced understanding of HSS compared with pre-VISTA responses. The AAMC GQ data showed increased agreement with an item that mapped to HSS understanding. NEXT STEPS: The VISTA program provides a model for institutions to enhance HSS education between curricular overhauls. Next steps include implementing value-added roles and additional immersive learning exercises.

Preference Signaling in the Orthopaedic Surgery Match: Applicant and Residency Program Attitudes, Behaviors, and Outcomes.

Introduction: The orthopaedic surgery match has experienced a consistent increase in both the number of applicants and applications submitted per applicant. Preference signaling was implemented during the 2022 to 2023 application cycle in part to curtail the rising application burden on both applicants and residency programs. Our aim was to explore the impact of the preference signaling system on applicant and residency program leader attitudes, behaviors, and outcomes. Methods: We distributed surveys to American Orthopaedic Association/Council of Orthopaedic Residency Directors member program leaders (program directors, assistant program directors, and program coordinators) and applicants registered for the Electronic Standardized Letter of Recommendation after Universal Interview Offer Day 2022 (Fall Survey) and Match Day 2023 (Spring Survey). The surveys contained multiple-choice and numeric response questions on attitudes, behaviors, and outcomes that were analyzed and reported as percentages and medians, respectively. Open-text responses were reviewed for dominant themes. Results: One hundred program leaders and 378 applicants (47%) completed the Fall Survey, and 146 program leaders and 290 applicants (36%) completed the Spring Survey. A majority of applicants (71%) and program leadership (91%) support the continued use of signaling. Applicants reported a 16% reduction in the number of programs to which they applied. Program directors largely used signaling as a tool for screening applications (75%), with few programs using signaling in the ranking process (20%). Applicants reported that 81% of their interviews were from programs they signaled. Slightly more than half of programs (53%) reported filling their last slot at a higher rank order position than the average of the previous 5 years. Qualitative analysis suggests a need for more transparency in the use of signals, consideration of application and/or interview caps, and reconsideration of the other components of the application. Conclusion: Preference signaling in the orthopaedic surgery match was met with positive feedback and led to a reduction in the number of applications. Future research will examine the continued impact of preference signaling and assess alterations for optimizing the match process.

The Ankle Ligament Reconstruction-Return to Sport after Injury (ALR-RSI) is a valid and reliable measure to assess psychological readiness before returning to sport following peroneal tendon pathology surgery.

PURPOSE: The objective of this study was to validate a scale that could help surgeons evaluate patients' psychological readiness to return to sport (RTS) after peroneal tendon pathology surgery. METHODS: The Ankle Ligament Reconstruction-Return to Sport after Injury (ALR-RSI) scale, which had previously been validated in ankle ligament reconstruction patients, was adapted to evaluate the psychological preparedness for RTS in athletic patients who underwent peroneal tendinopathy surgery. The Foot and Ankle Outcome Score (FAOS) and Foot Ankle Ability Measurement (FAAM) scores were employed as patient-related outcome measurement (PROM) instruments. RESULTS: This study included 57 patients. There was a strong correlation between ALR-RSI and both FAOS and FAAM (r = -0.68 and 0.74, respectively). ALR-RSI was considerably higher in patients who returned to sports than in those who did not. The mean score was 72.9 ± 19.0 in patients who returned to the same preinjury level, 48.5 ± 24.0 in those who returned to a lower level and 53.6 ± 31.1 in patients who changed their athletic activity (p < 0.0001). Furthermore, ALR-RSI showed at least a similar discrimination ability when compared to FAOS and FAAM. The test-retest intraclass correlation coefficient was 0.95. The Cronbach's α statistic used to measure the internal consistency was high (0.95). A Youden index of 0.65 was observed for a cut-off score of 68 points. CONCLUSION: ALR-RSI is a valid instrument for assessing psychological readiness to RTS in an athletic population following peroneal tendon surgery. When compared to the most commonly used PROMs, it was strongly correlated and demonstrated at least similar discrimination capacity. This could assist surgeons in identifying athletes who will have poor postoperative results and advising them on their capability to RTS. LEVEL OF EVIDENCE: Level III.

The comparative efficacy of nerve transfer versus tendon transfer in the management of radial palsy: A systematic review and meta-analysis.

Background: There is no clear census as to which operative technique provides better recovery for radial nerve injuries. Therefore, in this systematic review, we examined the functional recovery, patient-reported outcomes, and complications of tendon transfer (TT) and nerve transfer (NT). Methods: Five electronic databases were searched for studies (>10 cases per study) comparing NT and TT regardless of the study design (observational or experimental). Manual search was also conducted. The quality was assessed by the NIH tool. Outcomes included functional recovery, patient-reported outcomes (DASH score, satisfaction, and inability to return to work), and complications. The prevalence was pooled across studies using STATA software, and then, a subgroup analysis based on the intervention type. Results: Twenty-one studies (542 patients) were analyzed. Excellent recovery, assessed by the Bincaz scale, was higher in the TT group (29 % vs. 11 %) as well as failure to extend the fingers (49 % vs. 9 %). No significant difference was noted between both groups regarding DASH score (mean difference = -2.76; 95 % CI: -12.66: 6.93). Satisfaction was great in the TT group (89 %) with a limited proportion of patients unable to return to work (7 %). Complications were slightly higher in the TT group (8 % vs. 7 %) while 18 % of patients undergoing TT requiring revision surgery. Radial deviation was encountered in 18 % of patients in the TT group and 0 % in the NT group. The quality was good, fair, and poor in 2, 13, and 6, respectively. Conclusions: In radial nerve injuries, although tendon transfer may seem to provide better functional motor recovery than nerve transfer, it is associated with a higher rate of failure to extend the finger. Given the large confidence interval, the accuracy of this finding is questioned. However, a great proportion of those patients require revision surgery afterward. Additionally, tendon transfer is associated with a greater complication rate than nerve transfer, particularly radial deviation.

Validation of the Ankle Ligament Reconstruction-Return to Sports after Injury (ALR-RSI) Score as a Tool to Assess Psychological Readiness to Return to Sport in an Active Population After Ankle Fracture Surgery: A Cross-sectional Study.

Ankle injuries account for 15% to 25% of all sports injuries resulting in significant pain and loss of function. The purpose of this cross-sectional study was to validate a scale to help surgeons quantify the psychological readiness to Return To Sport (RTS) in patients undergoing ankle fracture surgery. ALR-RSI was used to assess the psychological readiness for RTS in athletic patients who underwent ankle fracture fixation between January 2020 and January 2021. Participants filled out ALR-RSI and 2 Patient-Related Outcome Measurement (PROM) tools: Olerud-Molander Ankle Score (OMAS) and Self-Reported Foot and Ankle Score (SEFAS). A total of 93 patients were included. There was a strong correlation between ALR-RSI and both OMAS and SEFAS, with Pearson coefficients of r = 0.58 and 0.53, respectively. ALR-RSI was significantly higher in the RTS group than in those who no longer practiced their main preinjury sport. Moreover, the discriminant validity of ALR-RSI (AUC = 0.81) was better than that of the SEFAS and OMAS (AUC = 0.64 and 0.65, respectively, p = .001). The intra-class correlation coefficient ρ of 0.94 showed excellent reproducibility. At an optimal cutoff value of 76.7, ALR-RSI had a sensitivity of 81% and a specificity of 75% with a Youden index of 0.56. In conclusion, ALR-RSI was a valid and reproducible tool to evaluate the psychological readiness for RTS in an active population after an ankle fracture. This score could help surgeons identify athletes who may have unfavorable postoperative outcomes and provide support on the ability to RTS.

Rice Bodies Accompanied by Tenosynovitis of the Wrist: A Case Report and Literature Review.

Rice bodies, a rare finding in clinical practice, are commonly observed in the shoulders and knees of affected individuals. However, they can occur in the wrist as well. Herein, we report a case of a female presenting with painful swelling in the right wrist that lasted with a history of carpal tunnel syndrome, rheumatoid arthritis, and history of median nerve decompression two years ago. A potential diagnosis of infectious diseases and gout was excluded through negative cultures and laboratory findings. X-rays showed no significant findings; however, magnetic resonance imaging revealed findings suggestive of rice bodies that were confirmed by additional proton dense fat-saturated imaging. The mass was then removed by extensive debridement and sent for pathological assessment, which showed multiple nodules containing fibrin and polymorphonuclear cells. The patient did not experience recurrence during the follow-up period. Rice bodies, although rare, can occur in the wrist, and this imposes several challenges associated with their diagnostic and management protocols.