A Deep Learning Model Enhances Clinicians' Diagnostic Accuracy to More Than 96% for Anterior Cruciate Ligament Ruptures on Magnetic Resonance Imaging.

PURPOSE: To develop a deep learning model to accurately detect anterior cruciate ligament (ACL) ruptures on magnetic resonance imaging (MRI) and to evaluate its effect on the diagnostic accuracy and efficiency of clinicians. METHODS: A training dataset was built from MRIs acquired from January 2017 to June 2021, including patients with knee symptoms, irrespective of ACL ruptures. An external validation dataset was built from MRIs acquired from January 2021 to June 2022, including patients who underwent knee arthroscopy or arthroplasty. Patients with fractures or prior knee surgeries were excluded in both datasets. Subsequently, a deep learning model was developed and validated using these datasets. Clinicians of varying expertise levels in sports medicine and radiology were recruited, and their capacities in diagnosing ACL injuries in terms of accuracy and diagnosing time were evaluated both with and without artificial intelligence (AI) assistance. RESULTS: A deep learning model was developed based on the training dataset of 22,767 MRIs from 5 centers and verified with external validation dataset of 4,086 MRIs from 6 centers. The model achieved an area under the receiver operating characteristic curve of 0.987 and a sensitivity and specificity of 95.1%. Thirty-eight clinicians from 25 centers were recruited to diagnose 3,800 MRIs. The AI assistance significantly improved the accuracy of all clinicians, exceeding 96%. Additionally, a notable reduction in diagnostic time was observed. The most significant improvements in accuracy and time efficiency were observed in the trainee groups, suggesting that AI support is particularly beneficial for clinicians with moderately limited diagnostic expertise. CONCLUSIONS: This deep learning model demonstrated expert-level diagnostic performance for ACL ruptures, serving as a valuable tool to assist clinicians of various specialties and experience levels in making accurate and efficient diagnoses. LEVEL OF EVIDENCE: Level III, retrospective comparative case series.

Artificial intelligence suppression as a strategy to mitigate artificial intelligence automation bias.

BACKGROUND: Incorporating artificial intelligence (AI) into clinics brings the risk of automation bias, which potentially misleads the clinician's decision-making. The purpose of this study was to propose a potential strategy to mitigate automation bias. METHODS: This was a laboratory study with a randomized cross-over design. The diagnosis of anterior cruciate ligament (ACL) rupture, a common injury, on magnetic resonance imaging (MRI) was used as an example. Forty clinicians were invited to diagnose 200 ACLs with and without AI assistance. The AI's correcting and misleading (automation bias) effects on the clinicians' decision-making processes were analyzed. An ordinal logistic regression model was employed to predict the correcting and misleading probabilities of the AI. We further proposed an AI suppression strategy that retracted AI diagnoses with a higher misleading probability and provided AI diagnoses with a higher correcting probability. RESULTS: The AI significantly increased clinicians' accuracy from 87.2%±13.1% to 96.4%±1.9% (P < .001). However, the clinicians' errors in the AI-assisted round were associated with automation bias, accounting for 45.5% of the total mistakes. The automation bias was found to affect clinicians of all levels of expertise. Using a logistic regression model, we identified an AI output zone with higher probability to generate misleading diagnoses. The proposed AI suppression strategy was estimated to decrease clinicians' automation bias by 41.7%. CONCLUSION: Although AI improved clinicians' diagnostic performance, automation bias was a serious problem that should be addressed in clinical practice. The proposed AI suppression strategy is a practical method for decreasing automation bias.