The Large Language Model ChatGPT-4 Exhibits Excellent Triage Capabilities and Diagnostic Performance for Patients Presenting With Various Causes of Knee Pain.

PURPOSE: To provide a proof-of-concept analysis of the appropriateness and performance of ChatGPT-4 to triage, synthesize differential diagnoses, and generate treatment plans concerning common presentations of knee pain. METHODS: Twenty knee complaints warranting triage and expanded scenarios were input into ChatGPT-4, with memory cleared prior to each new input to mitigate bias. For the 10 triage complaints, ChatGPT-4 was asked to generate a differential diagnosis that was graded for accuracy and suitability in comparison to a differential created by 2 orthopaedic sports medicine physicians. For the 10 clinical scenarios, ChatGPT-4 was prompted to provide treatment guidance for the patient, which was again graded. To test the higher-order capabilities of ChatGPT-4, further inquiry into these specific management recommendations was performed and graded. RESULTS: All ChatGPT-4 diagnoses were deemed appropriate within the spectrum of potential pathologies on a differential. The top diagnosis on the differential was identical between surgeons and ChatGPT-4 for 70% of scenarios, and the top diagnosis provided by the surgeon appeared as either the first or second diagnosis in 90% of scenarios. Overall, 16 of 30 diagnoses (53.3%) in the differential were identical. When provided with 10 expanded vignettes with a single diagnosis, the accuracy of ChatGPT-4 increased to 100%, with the suitability of management graded as appropriate in 90% of cases. Specific information pertaining to conservative management, surgical approaches, and related treatments was appropriate and accurate in 100% of cases. CONCLUSIONS: ChatGPT-4 provided clinically reasonable diagnoses to triage patient complaints of knee pain due to various underlying conditions that were generally consistent with differentials provided by sports medicine physicians. Diagnostic performance was enhanced when providing additional information, allowing ChatGPT-4 to reach high predictive accuracy for recommendations concerning management and treatment options. However, ChatGPT-4 may show clinically important error rates for diagnosis depending on prompting strategy and information provided; therefore, further refinements are necessary prior to implementation into clinical workflows. CLINICAL RELEVANCE: Although ChatGPT-4 is increasingly being used by patients for health information, the potential for ChatGPT-4 to serve as a clinical support tool is unclear. In this study, we found that ChatGPT-4 was frequently able to diagnose and triage knee complaints appropriately as rated by sports medicine surgeons, suggesting that it may eventually be a useful clinical support tool.

A Machine Learning Model Demonstrates Excellent Performance in Predicting Subscapularis Tears Based on Pre-Operative Imaging Parameters Alone.

PURPOSE: To develop a machine learning model capable of identifying subscapularis tears before surgery based on imaging and physical examination findings. METHODS: Between 2010 and 2020, 202 consecutive shoulders underwent arthroscopic rotator cuff repair by a single surgeon. Patient demographics, physical examination findings (including range of motion, weakness with internal rotation, lift/push-off test, belly press test, and bear hug test), and imaging (including direct and indirect signs of tearing, biceps status, fatty atrophy, cystic changes, and other similar findings) were included for model creation. RESULTS: Sixty percent of the shoulders had partial or full thickness tears of the subscapularis verified during surgery (83% of these were upper third). Using only preoperative imaging-related parameters, the XGBoost model demonstrated excellent performance at predicting subscapularis tears (c-statistic, 0.84; accuracy, 0.85; F1 score, 0.87). The top 5 features included direct signs related to the presence of tearing as evidenced on magnetic resonance imaging (MRI) (changes in tendon morphology and signal), as well as the quality of the MRI and biceps pathology. CONCLUSIONS: In this study, machine learning was successful in predicting subscapularis tears by MRI alone in 85% of patients, and this accuracy did not decrease by isolating the model to the top features. The top five features included direct signs related to the presence of tearing as evidenced on MRI (changes in tendon morphology and signal), as well as the quality of the MRI and biceps pathology. Last, in advanced modeling, the addition of physical examination or patient characteristics did not make a significant difference in the predictive ability of this model. LEVEL OF EVIDENCE: Level III, diagnostic case-control study.

External validation of the Norwegian anterior cruciate ligament reconstruction revision prediction model using patients from the STABILITY 1 Trial.

PURPOSE: A machine learning-based anterior cruciate ligament (ACL) revision prediction model has been developed using Norwegian Knee Ligament Register (NKLR) data, but lacks external validation outside Scandinavia. This study aimed to assess the external validity of the NKLR model (https://swastvedt.shinyapps.io/calculator_rev/) using the STABILITY 1 randomized clinical trial (RCT) data set. The hypothesis was that model performance would be similar. METHODS: The NKLR Cox Lasso model was selected for external validation owing to its superior performance in the original study. STABILITY 1 patients with all five predictors required by the Cox Lasso model were included. The STABILITY 1 RCT was a prospective study which randomized patients to receive either a hamstring tendon autograft (HT) alone or HT plus a lateral extra-articular tenodesis (LET). Since all patients in the STABILITY 1 trial received HT ± LET, three configurations were tested: 1: all patients coded as HT, 2: HT + LET group coded as bone-patellar tendon-bone (BPTB) autograft, 3: HT + LET group coded as unknown/other graft choice. Model performance was assessed via concordance and calibration. RESULTS: In total, 591/618 (95.6%) STABILITY 1 patients were eligible for inclusion, with 39 undergoing revisions within 2 years (6.6%). Model performance was best when patients receiving HT + LET were coded as BPTB. Concordance was similar to the original NKLR prediction model for 1- and 2-year revision prediction (STABILITY: 0.71; NKLR: 0.68-0.69). Concordance 95% confidence interval (CI) ranged from 0.63 to 0.79. The model was well calibrated for 1-year prediction while the 2-year prediction demonstrated evidence of miscalibration. CONCLUSION: When patients in STABILITY 1 who received HT + LET were coded as BPTB in the NKLR prediction model, concordance was similar to the index study. However, due to a wide 95% CI, the true performance of the prediction model with this Canadian and European cohort is unclear and a larger data set is required to definitively determine the external validity. Further, better calibration for 1-year predictions aligns with general prediction modelling challenges over longer periods. While not a large enough sample size to elicit the true accuracy and external validity of the prediction model when applied to North American patients, this analysis provides more support for the notion that HT plus LET performs similarly to BPTB reconstruction. In addition, despite the wide confidence interval, this study suggests optimism regarding the accuracy of the model when applied outside of Scandinavia. LEVEL OF EVIDENCE: Level 3, cohort study.

Limited clinical utility of a machine learning revision prediction model based on a national hip arthroscopy registry.

PURPOSE: Accurate prediction of outcome following hip arthroscopy is challenging and machine learning has the potential to improve our predictive capability. The purpose of this study was to determine if machine learning analysis of the Danish Hip Arthroscopy Registry (DHAR) can develop a clinically meaningful calculator for predicting the probability of a patient undergoing subsequent revision surgery following primary hip arthroscopy. METHODS: Machine learning analysis was performed on the DHAR. The primary outcome for the models was probability of revision hip arthroscopy within 1, 2, and/or 5 years after primary hip arthroscopy. Data were split randomly into training (75%) and test (25%) sets. Four models intended for these types of data were tested: Cox elastic net, random survival forest, gradient boosted regression (GBM), and super learner. These four models represent a range of approaches to statistical details like variable selection and model complexity. Model performance was assessed by calculating calibration and area under the curve (AUC). Analysis was performed using only variables available in the pre-operative clinical setting and then repeated to compare model performance using all variables available in the registry. RESULTS: In total, 5581 patients were included for analysis. Average follow-up time or time-to-revision was 4.25 years (± 2.51) years and overall revision rate was 11%. All four models were generally well calibrated and demonstrated concordance in the moderate range when restricted to only pre-operative variables (0.62-0.67), and when considering all variables available in the registry (0.63-0.66). The 95% confidence intervals for model concordance were wide for both analyses, ranging from a low of 0.53 to a high of 0.75, indicating uncertainty about the true accuracy of the models. CONCLUSION: The association between pre-surgical factors and outcome following hip arthroscopy is complex. Machine learning analysis of the DHAR produced a model capable of predicting revision surgery risk following primary hip arthroscopy that demonstrated moderate accuracy but likely limited clinical usefulness. Prediction accuracy would benefit from enhanced data quality within the registry and this preliminary study holds promise for future model generation as the DHAR matures. Ongoing collection of high-quality data by the DHAR should enable improved patient-specific outcome prediction that is generalisable across the population. LEVEL OF EVIDENCE: Level III.

MCID and PASS in Knee Surgeries. Theoretical Aspects and Clinical Relevance References.

The application and interpretation of patient-reported outcome measures (PROM), following knee injuries, pathologies, and interventions, can be challenging. In recent years, the literature has been enriched with metrics to facilitate our understanding and interpretation of these outcome measures. Two commonly utilized tools include the minimal clinically important difference (MCID) and the patient acceptable symptoms state (PASS). These measures have demonstrated clinical value, however, they have often been under- or mis-reported. It is paramount to use them to understand the clinical significance of any statistically significant results. Still, it remains important to know their caveats and limitations. In this focused report on MCID and PASS, their definitions, methods of calculations, clinical relevance, interpretations, and limitations are reviewed and presented in a simple approach.

Multivariable regression: understanding one of medicine's most fundamental statistical tools.

Multivariable regression is a fundamental tool that drives observational research in orthopaedic surgery. However, regression analyses are not always implemented correctly. This study presents a basic overview of regression analyses and reviews frequent points of confusion. Topics include linear, logistic, and time-to-event regressions, causal inference, confounders, overfitting, missing data, multicollinearity, interactions, and key differences between multivariable versus multivariate regression. The goal is to provide clarity regarding the use and interpretation of multivariable analyses for those attempting to increase their statistical literacy in orthopaedic research.

A practical guide to the development and deployment of deep learning models for the Orthopedic surgeon: part I.

Deep learning has a profound impact on daily life. As Orthopedics makes use of this rapid escalation in technology, Orthopedic surgeons will need to take leadership roles on deep learning projects. Moreover, surgeons must possess an understanding of what is necessary to design and implement deep learning-based project pipelines. This review provides a practical guide for the Orthopedic surgeon to understand the steps needed to design, develop, and deploy a deep learning pipeline for clinical applications. A detailed description of the processes involved in defining the problem, building the team, acquiring and curating the data, labeling the data, establishing the ground truth, pre-processing and augmenting the data, and selecting the required hardware is provided. In addition, an overview of unique considerations involved in the training and evaluation of deep learning models is provided. This review strives to provide surgeons with the groundwork needed to identify gaps in the clinical landscape that deep learning models may be able to fill and equips them with the knowledge needed to lead an interdisciplinary team through the process of creating novel deep-learning-based solutions to fill those gaps.

Natural language processing: using artificial intelligence to understand human language in orthopedics.

Natural language processing (NLP) describes the broad field of artificial intelligence by which computers are trained to understand and generate human language. Within healthcare research, NLP is commonly used for variable extraction and classification/cohort identification tasks. While these tools are becoming increasingly popular and available as both open-source and commercial products, there is a paucity of the literature within the orthopedic space describing the key tasks within these powerful pipelines. Curation and navigation of the electronic medical record are becoming increasingly onerous, and it is important for physicians and other healthcare professionals to understand potential methods of harnessing this large data resource. The purpose of this study is to provide an overview of the tasks required to develop an NLP pipeline for orthopedic research and present recent examples of successful implementations.

A practical guide to the development and deployment of deep learning models for the orthopedic surgeon: part II.

Deep learning has the potential to be one of the most transformative technologies to impact orthopedic surgery. Substantial innovation in this area has occurred over the past 5 years, but clinically meaningful advancements remain limited by a disconnect between clinical and technical experts. That is, it is likely that few orthopedic surgeons possess both the clinical knowledge necessary to identify orthopedic problems, and the technical knowledge needed to implement deep learning-based solutions. To maximize the utilization of rapidly advancing technologies derived from deep learning models, orthopedic surgeons should understand the steps needed to design, organize, implement, and evaluate a deep learning project and its workflow. Equipping surgeons with this knowledge is the objective of this three-part editorial review. Part I described the processes involved in defining the problem, team building, data acquisition, curation, labeling, and establishing the ground truth. Building on that, this review (Part II) provides guidance on pre-processing and augmenting the data, making use of open-source libraries/toolkits, and selecting the required hardware to implement the pipeline. Special considerations regarding model training and evaluation unique to deep learning models relative to "shallow" machine learning models are also reviewed. Finally, guidance pertaining to the clinical deployment of deep learning models in the real world is provided. As in Part I, the focus is on applications of deep learning for computer vision and imaging.

Supervised machine learning and associated algorithms: applications in orthopedic surgery.

Supervised learning is the most common form of machine learning utilized in medical research. It is used to predict outcomes of interest or classify positive and/or negative cases with a known ground truth. Supervised learning describes a spectrum of techniques, ranging from traditional regression modeling to more complex tree boosting, which are becoming increasingly prevalent as the focus on "big data" develops. While these tools are becoming increasingly popular and powerful, there is a paucity of literature available that describe the strengths and limitations of these different modeling techniques. Typically, there is no formal training for health care professionals in the use of machine learning models. As machine learning applications throughout medicine increase, it is important that physicians and other health care professionals better understand the processes underlying application of these techniques. The purpose of this study is to provide an overview of commonly used supervised learning techniques with recent case examples within the orthopedic literature. An additional goal is to address disparities in the understanding of these methods to improve communication within and between research teams.

Injuries in elite women's ski jumping: a cohort study following three International Ski Federation (FIS) World Cup seasons from 2017-2018 to 2019-2020.

OBJECTIVES: To define incidence and injury patterns of International Ski Federation (FIS) World Cup (WC) women ski jumpers over three seasons. METHODS: Ski jump athletes competing in the Women's FIS WC were recruited for prospective injury surveillance from 2017-2018 to 2019-2020. Team representatives recruited the athletes annually and prospectively recorded all injuries requiring medical attention. Retrospective end-of-season interviews corroborated injury surveillance. Medical doctors collected and processed the data. The 4-month competitive season was used to calculate the annual incidence of injuries per 100 athletes per season. Injury type, location, severity and aetiology were reported. RESULTS: Athletes from 19 nations were enrolled equalling 205 athlete-seasons. Mean age was 21.2 years (SD=3.8). Thirty-nine injury events resulted in 54 total injuries (26.3 injuries/100 athletes/season). Injuries were mostly acute (83%) and occurred on the ski jump hill (78%). The most common injury location was the knee (n=18, 33%). Crash landings were the most common cause of injury events (70%). Nearly half of the acute ski jump injury events occurred in snowy, windy or cloudy conditions (44%) and/or during telemark landings (46%), and most jumps (96%) were shorter than hill size. One third of the injuries were severe, and 78% of severe injuries involved the knee. CONCLUSION: Acute injury events occur relatively frequently in elite women ski jumpers, most resulting in time-loss from sport and a significant proportion involving serious knee injuries. Crash landing was the leading cause of injury. This baseline information can be used to guide and evaluate future efforts at injury prevention.

Editorial Commentary: Machine Learning in Medicine Requires Clinician Input, Faces Barriers, and High-Quality Evidence Is Required to Demonstrate Improved Patient Outcomes.

Machine learning (ML) and artificial intelligence (AI) may be described as advanced statistical techniques using algorithms to "learn" to evaluate and predict relationships between input and results without explicit human programming, often with high accuracy. The potentials and pitfalls of ML continue to be explored as predictive modeling grows in popularity. While use of and optimism for AI continues to increase in orthopaedic surgery, there remains little high-quality evidence of its ability to improve patient outcome. It is up to us as clinicians to provide context for ML models and guide the use of these technologies to optimize the outcome for our patients. Barriers to widespread adoption of ML include poor quality data, limits to compliant data sharing, few clinicians who are expert in ML statistical techniques, and computing costs including technology, infrastructure, personnel, energy, and updates.

Machine learning and conventional statistics: making sense of the differences.

The application of machine learning (ML) to the field of orthopaedic surgery is rapidly increasing, but many surgeons remain unfamiliar with the nuances of this novel technique. With this editorial, we address a fundamental topic-the differences between ML techniques and traditional statistics. By doing so, we aim to further familiarize the reader with the new opportunities available thanks to the ML approach.

Artificial intelligence and machine learning: an introduction for orthopaedic surgeons.

The application of artificial intelligence (AI) and machine learning to the field of orthopaedic surgery is rapidly increasing. While this represents an important step in the advancement of our specialty, the concept of AI is rich with statistical jargon and techniques unfamiliar to many clinicians. This knowledge gap may limit the impact and potential of these novel techniques. We aim to narrow this gap in a way that is accessible for all orthopaedic surgeons. With this manuscript, we introduce the concept of AI and machine learning and give examples of how it can impact clinical practice and patient care.Level of evidence VI.

Low annual hospital volume of anterior cruciate ligament reconstruction is not associated with higher revision rates.

PURPOSE: Surgery performed in low-volume centres has been associated with longer operating time, longer hospital stays, lower functional outcomes, and higher rates of revision surgery, complications and mortality. This has been reported consistently in the arthroplasty literature, but there is a paucity of data regarding the relationship between surgical volume and outcome following anterior cruciate ligament (ACL) reconstruction. The purpose was to compare ACL reconstruction failure rates between hospitals performing different annual surgical volumes. METHODS: All patients from the Norwegian Knee Ligament Register having primary autograft ACL reconstruction between 2004 and 2016 were included. Hospital volume was divided into quintiles based on the number of ACL reconstructions performed annually, defined arbitrarily as: 1-12 (V1), 13-24 (V2), 25-49 (V3), 50-99 (V4) and ≥ 100 (V5) annual procedures. Kaplan-Meier estimated survival curves and survival percentages were calculated with revision ACL reconstruction as the end point. Secondary outcome measures included (1) mean change in Knee Injury and Osteoarthritis Outcome Score (KOOS) Quality of Life (QoL) and Sport subsections from pre-operative to 5-year follow-up and (2) subjective failure defined as KOOS QoL < 44. RESULTS: Twenty thousand eight hundred and fifty patients met the inclusion criteria and 1195 (5.7%) underwent subsequent revision ACL reconstruction over the study period. Revision rates were lower in the lower volume hospitals compared with the higher volume hospitals (p < 0.001). There was no clinically significant difference in improvement between pre-operative and 5-year follow-up KOOS scores between hospital volume categories, but a higher proportion of patients having surgery at lower volume hospitals reported a subjective failure. Patients in the lower volume categories (V1-3) were more often male and older compared to the higher volume hospitals (V4-5). Concomitant meniscal injuries and participation in pivoting sports were most common in V5 compared with V1 (p < 0.001). Median operative time decreased as hospital volume increased, ranging from 90 min at V1 hospitals to 56 min at V5 hospitals (p < 0.001). CONCLUSION: Patients having ACL reconstruction at lower volume hospitals had a lower rate of subsequent revision surgery relative to higher volume hospitals. However, complications occurred more frequently, operative duration was longer, and the number of patients reporting a subjective failure of ACL reconstruction was highest at these lower volume hospitals. LEVEL OF EVIDENCE: Level III.

Machine learning algorithm to predict anterior cruciate ligament revision demonstrates external validity.

PURPOSE: External validation of machine learning predictive models is achieved through evaluation of model performance on different groups of patients than were used for algorithm development. This important step is uncommonly performed, inhibiting clinical translation of newly developed models. Machine learning analysis of the Norwegian Knee Ligament Register (NKLR) recently led to the development of a tool capable of estimating the risk of anterior cruciate ligament (ACL) revision ( https://swastvedt.shinyapps.io/calculator_rev/ ). The purpose of this study was to determine the external validity of the NKLR model by assessing algorithm performance when applied to patients from the Danish Knee Ligament Registry (DKLR). METHODS: The primary outcome measure of the NKLR model was probability of revision ACL reconstruction within 1, 2, and/or 5 years. For external validation, all DKLR patients with complete data for the five variables required for NKLR prediction were included. The five variables included graft choice, femur fixation device, KOOS QOL score at surgery, years from injury to surgery, and age at surgery. Predicted revision probabilities were calculated for all DKLR patients. The model performance was assessed using the same metrics as the NKLR study: concordance and calibration. RESULTS: In total, 10,922 DKLR patients were included for analysis. Average follow-up time or time-to-revision was 8.4 (± 4.3) years and overall revision rate was 6.9%. Surgical technique trends (i.e., graft choice and fixation devices) and injury characteristics (i.e., concomitant meniscus and cartilage pathology) were dissimilar between registries. The model produced similar concordance when applied to the DKLR population compared to the original NKLR test data (DKLR: 0.68; NKLR: 0.68-0.69). Calibration was poorer for the DKLR population at one and five years post primary surgery but similar to the NKLR at two years. CONCLUSION: The NKLR machine learning algorithm demonstrated similar performance when applied to patients from the DKLR, suggesting that it is valid for application outside of the initial patient population. This represents the first machine learning model for predicting revision ACL reconstruction that has been externally validated. Clinicians can use this in-clinic calculator to estimate revision risk at a patient specific level when discussing outcome expectations pre-operatively. While encouraging, it should be noted that the performance of the model on patients undergoing ACL reconstruction outside of Scandinavia remains unknown. LEVEL OF EVIDENCE: III.

The development and deployment of machine learning models.

Applications of artificial intelligence, specifically machine learning, are becoming increasingly popular in Orthopaedic Surgery, and medicine as a whole. This growing interest is shared by data scientists and physicians alike. However, there is an asymmetry of understanding of the developmental process and potential applications of machine learning. As new technology will undoubtedly affect clinical practice in the coming years, it is important for physicians to understand how these processes work. The purpose of this paper is to provide clarity and a general framework for building and assessing machine learning models.

Picking apart p values: common problems and points of confusion.

Due to its frequent misuse, the p value has become a point of contention in the research community. In this editorial, we seek to clarify some of the common misconceptions about p values and the hazardous implications associated with misunderstanding this commonly used statistical concept. This article will discuss issues related to p value interpretation in addition to problems such as p-hacking and statistical fragility; we will also offer some thoughts on addressing these issues. The aim of this editorial is to provide clarity around the concept of statistical significance for those attempting to increase their statistical literacy in Orthopedic research.

Editorial Commentary: Anatomic Posterolateral Corner Reconstruction: All-Autograft Technique.

Several techniques for posterolateral corner reconstruction have been described in the literature, typically using allogeneic tissue. Autograft reconstruction has potential value because of decreased cost and limited allograft supply in some locations. Initial results of this hamstring autograft tendon technique are promising, but further research is needed to directly compare reconstruction graft sources.