Characteristics and Outcomes of US Veterans With Immunocompromised Conditions at High Risk of SARS-CoV-2 Infection With or Without Receipt of Oral Antiviral Agents.

OBJECTIVES: Molnupiravir and nirmatrelvir-ritonavir were the first oral antiviral agents to demonstrate reduced hospitalization or death in patients with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), but patients with immunocompromised conditions were not well-represented. The objective of this study was to characterize and compare the clinical outcomes of US veterans with immunocompromised conditions prescribed oral antivirals with those who did not receive oral antivirals for mild-to-moderate SARS-CoV-2 active infection. METHODS: This was a retrospective, observational, nationwide propensity-matched analysis of US veterans with immunocompromised conditions who developed documented SARS-CoV-2 infection. The primary outcome was the composite of any hospitalization or death within 30 days of diagnosis. Secondary outcomes included 30-day comparative rates of (1) any hospitalization, (2) death, (3) intensive care requirement, and (4) subset analyses of outcomes by oral antiviral used and vaccination status. RESULTS: The composite primary outcome was significantly lower in patients receiving oral antiviral therapy compared with those who did not (23/390 [5.9%] vs 57/390 [14.6%]; odds ratio, 0.37; 95% confidence interval, .22-.61). This difference was driven largely by fewer deaths in the oral antiviral group (1/390 [0.3%] vs 19/390 [4.9%]; odds ratio, 0.05; 95% confidence interval, .007-.38). There was no significant difference in rate of intensive care requirement. The composite outcome was improved in vaccinated patients (completing the first series or first booster dose) who received oral antiviral agents compared with those who did not receive oral antiviral agents. Compared with those prescribed nirmatrelvir-ritonavir, patients given molnupiravir were older, had a higher incidence of cautions/contraindications, greater prevalence of tobacco use, and more cardiovascular complications. CONCLUSIONS: Use of molnupiravir or nirmatrelvir-ritonavir was associated with lower incidences of hospitalization or death within 30 days of diagnosis in US veterans with immunocompromised conditions, regardless of vaccination status. These findings support the use of either oral antiviral in this patient population.

Currently Available Large Language Models Do Not Provide Musculoskeletal Treatment Recommendations That Are Concordant With Evidence-Based Clinical Practice Guidelines.

PURPOSE: To determine whether several leading, commercially available large language models (LLMs) provide treatment recommendations concordant with evidence-based clinical practice guidelines (CPGs) developed by the American Academy of Orthopaedic Surgeons (AAOS). METHODS: All CPGs concerning the management of rotator cuff tears (n = 33) and anterior cruciate ligament injuries (n = 15) were extracted from the AAOS. Treatment recommendations from Chat-Generative Pretrained Transformer version 4 (ChatGPT-4), Gemini, Mistral-7B, and Claude-3 were graded by 2 blinded physicians as being concordant, discordant, or indeterminate (i.e., neutral response without definitive recommendation) with respect to AAOS CPGs. The overall concordance between LLM and AAOS recommendations was quantified, and the comparative overall concordance of recommendations among the 4 LLMs was evaluated through the Fisher exact test. RESULTS: Overall, 135 responses (70.3%) were concordant, 43 (22.4%) were indeterminate, and 14 (7.3%) were discordant. Inter-rater reliability for concordance classification was excellent (κ = 0.92). Concordance with AAOS CPGs was most frequently observed with ChatGPT-4 (n = 38, 79.2%) and least frequently observed with Mistral-7B (n = 28, 58.3%). Indeterminate recommendations were most frequently observed with Mistral-7B (n = 17, 35.4%) and least frequently observed with Claude-3 (n = 8, 6.7%). Discordant recommendations were most frequently observed with Gemini (n = 6, 12.5%) and least frequently observed with ChatGPT-4 (n = 1, 2.1%). Overall, no statistically significant difference in concordant recommendations was observed across LLMs (P = .12). Of all recommendations, only 20 (10.4%) were transparent and provided references with full bibliographic details or links to specific peer-reviewed content to support recommendations. CONCLUSIONS: Among leading commercially available LLMs, more than 1-in-4 recommendations concerning the evaluation and management of rotator cuff and anterior cruciate ligament injuries do not reflect current evidence-based CPGs. Although ChatGPT-4 showed the highest performance, clinically significant rates of recommendations without concordance or supporting evidence were observed. Only 10% of responses by LLMs were transparent, precluding users from fully interpreting the sources from which recommendations were provided. CLINICAL RELEVANCE: Although leading LLMs generally provide recommendations concordant with CPGs, a substantial error rate exists, and the proportion of recommendations that do not align with these CPGs suggests that LLMs are not trustworthy clinical support tools at this time. Each off-the-shelf, closed-source LLM has strengths and weaknesses. Future research should evaluate and compare multiple LLMs to avoid bias associated with narrow evaluation of few models as observed in the current literature.

The Large Language Model ChatGPT-4 Exhibits Excellent Triage Capabilities and Diagnostic Performance for Patients Presenting With Various Causes of Knee Pain.

PURPOSE: To provide a proof-of-concept analysis of the appropriateness and performance of ChatGPT-4 to triage, synthesize differential diagnoses, and generate treatment plans concerning common presentations of knee pain. METHODS: Twenty knee complaints warranting triage and expanded scenarios were input into ChatGPT-4, with memory cleared prior to each new input to mitigate bias. For the 10 triage complaints, ChatGPT-4 was asked to generate a differential diagnosis that was graded for accuracy and suitability in comparison to a differential created by 2 orthopaedic sports medicine physicians. For the 10 clinical scenarios, ChatGPT-4 was prompted to provide treatment guidance for the patient, which was again graded. To test the higher-order capabilities of ChatGPT-4, further inquiry into these specific management recommendations was performed and graded. RESULTS: All ChatGPT-4 diagnoses were deemed appropriate within the spectrum of potential pathologies on a differential. The top diagnosis on the differential was identical between surgeons and ChatGPT-4 for 70% of scenarios, and the top diagnosis provided by the surgeon appeared as either the first or second diagnosis in 90% of scenarios. Overall, 16 of 30 diagnoses (53.3%) in the differential were identical. When provided with 10 expanded vignettes with a single diagnosis, the accuracy of ChatGPT-4 increased to 100%, with the suitability of management graded as appropriate in 90% of cases. Specific information pertaining to conservative management, surgical approaches, and related treatments was appropriate and accurate in 100% of cases. CONCLUSIONS: ChatGPT-4 provided clinically reasonable diagnoses to triage patient complaints of knee pain due to various underlying conditions that were generally consistent with differentials provided by sports medicine physicians. Diagnostic performance was enhanced when providing additional information, allowing ChatGPT-4 to reach high predictive accuracy for recommendations concerning management and treatment options. However, ChatGPT-4 may show clinically important error rates for diagnosis depending on prompting strategy and information provided; therefore, further refinements are necessary prior to implementation into clinical workflows. CLINICAL RELEVANCE: Although ChatGPT-4 is increasingly being used by patients for health information, the potential for ChatGPT-4 to serve as a clinical support tool is unclear. In this study, we found that ChatGPT-4 was frequently able to diagnose and triage knee complaints appropriately as rated by sports medicine surgeons, suggesting that it may eventually be a useful clinical support tool.

Defining clinically relevant outcome thresholds for pain and function after osteochondral autograft transplantation of the knee.

PURPOSE: To define the minimal clinically important difference (MCID) for measures of pain and function at 2, 5 and 10 years after osteochondral autograft transplantations (OATs). METHODS: Patients undergoing OATs of the knee were identified from a prospectively maintained cartilage surgery registry. Baseline demographic, injury and surgical factors were collected. Patient-reported outcome scores (PROMs) were collected at baseline, 2-, 5- and 10-year follow-up, including the International Knee Documentation Committee (IKDC) score, Knee Outcome Survey Activities of Daily Living Scale (KOS-ADLS), Marx activity scale and Visual Analogue Scale (VAS) for pain. The MCIDs were quantified for each metric utilizing a distribution-based method equivalent to one-half the standard deviation of the mean change in outcome score. The percentage of patients achieving MCID as a function of time was assessed. RESULTS: Of 63 consecutive patients who underwent OATs, 47 (74.6%) patients were eligible for follow-up (surgical date before October 2021) and had fully completed preoperative PROMs. A total of 39 patients (83%) were available for a minimum 2-year follow-up, with a mean (±standard deviation) follow-up of 5.8 ± 3.4 years. The MCIDs were determined to be 9.3 for IKDC, 2.5 for Marx, 7.4 for KOS-ADLS and 12.9 for pain. At 2 years, 78.1% of patients achieved MCID for IKDC, 77.8% for Marx, 75% for KOS-ADLS and 57.9% for pain. These results were generally maintained through 10-year follow-ups, with 75% of patients achieving MCID for IKDC, 80% for Marx, 80% for KOS-ADLS and 69.8% for pain. CONCLUSIONS: The majority of patients achieved a clinically relevant outcome improvement after OATs of the knee, with results sustained through 10-year follow-up. Patients who experience clinically relevant outcome improvement after OATs in the short term continue to experience sustained benefits at longer-term follow-up. These data provide valuable prognostic information when discussing patient candidacy and the expected trajectory of recovery. LEVEL OF EVIDENCE: Level III.

Editorial Commentary: Machine Learning Is Just a Statistical Technique, Not a Mystical Methodology or Peer Review Panacea.

Orthopaedic and sports medicine research surrounding artificial intelligence (AI) has dramatically risen over the last 4 years. Meaningful application and methodologic rigor in the scientific literature are critical to ensure appropriate use of AI. Common but critical errors for those engaging in AI-related research include failure to 1) ensure the question is important and previously unknown or unanswered; 2) establish that AI is necessary to answer the question; and 3) recognize model performance is more commonly a reflection of the data than the AI itself. We must take care to ensure we are not repackaging and internally validating registry data. Instead, we should be critically appraising our data-not the AI-based statistical technique. Without appropriate guardrails surrounding the use of artificial intelligence in Orthopaedic research, there is a risk of repackaging registry data and low-quality research in a recursive peer-reviewed loop.

Lower donor site morbidity with hamstring and quadriceps tendon autograft compared with bone-patellar tendon-bone autograft after anterior cruciate ligament reconstruction: a systematic review and network meta-analysis of randomized controlled trials.

PURPOSE: To perform a meta-analysis of RCTs evaluating donor site morbidity after bone-patellar tendon-bone (BTB), hamstring tendon (HT) and quadriceps tendon (QT) autograft harvest for anterior cruciate ligament reconstruction (ACLR). METHODS: PubMed, OVID/Medline and Cochrane databases were queried in July 2022. All level one articles reporting the frequency of specific donor-site morbidity were included. Frequentist model network meta-analyses with P-scores were conducted to compare the prevalence of donor-site morbidity, complications, all-cause reoperations and revision ACLR among the three treatment groups. RESULTS: Twenty-one RCTs comprising the outcomes of 1726 patients were included. The overall pooled rate of donor-site morbidity (defined as anterior knee pain, difficulty/impossibility kneeling, or combination) was 47.3% (range, 3.8-86.7%). A 69% (95% confidence interval [95% CI]: 0.18-0.56) and 88% (95% CI: 0.04-0.33) lower odds of incurring donor-site morbidity was observed with HT and QT autografts, respectively (p < 0.0001, both), when compared to BTB autograft. QT autograft was associated with a non-statistically significant reduction in donor-site morbidity compared with HT autograft (OR: 0.37, 95% CI: 0.14-1.03, n.s.). Treatment rankings (ordered from best-to-worst autograft choice with respect to donor-site morbidity) were as follows: (1) QT (P-score = 0.99), (2) HT (P-score = 0.51) and (3) BTB (P-score = 0.00). No statistically significant associations were observed between autograft and complications (n.s.), reoperations (n.s.) or revision ACLR (n.s.). CONCLUSION: ACLR using HT and QT autograft tissue was associated with a significant reduction in donor-site morbidity compared to BTB autograft. Autograft selection was not associated with complications, all-cause reoperations, or revision ACLR. Based on the current data, there is sufficient evidence to recommend that autograft selection should be personalized through considering differential rates of donor-site morbidity in the context of patient expectations and activity level without concern for a clinically important change in the rate of adverse events. LEVEL OF EVIDENCE: Level I.

Unsupervised machine learning methods and emerging applications in healthcare.

Unsupervised machine learning methods are important analytical tools that can facilitate the analysis and interpretation of high-dimensional data. Unsupervised machine learning methods identify latent patterns and hidden structures in high-dimensional data and can help simplify complex datasets. This article provides an overview of key unsupervised machine learning techniques including K-means clustering, hierarchical clustering, principal component analysis, and factor analysis. With a deeper understanding of these analytical tools, unsupervised machine learning methods can be incorporated into health sciences research to identify novel risk factors, improve prevention strategies, and facilitate delivery of personalized therapies and targeted patient care.Level of evidence: I.

Choosing the appropriate measure of central tendency: mean, median, or mode?

Mean, median, and mode are among the most basic and consistently used measures of central tendency in statistical analysis and are crucial for simplifying data sets to a single value. However, there is a lack of understanding of when to use each metric and how various factors can impact these values. The aim of this article is to clarify some of the confusion related to each measure and explain how to select the appropriate metric for a given data set. The authors present this work as an educational resource, ensuring that these common statistical concepts are better understood throughout the Orthopedic research community.

Multivariable regression: understanding one of medicine's most fundamental statistical tools.

Multivariable regression is a fundamental tool that drives observational research in orthopaedic surgery. However, regression analyses are not always implemented correctly. This study presents a basic overview of regression analyses and reviews frequent points of confusion. Topics include linear, logistic, and time-to-event regressions, causal inference, confounders, overfitting, missing data, multicollinearity, interactions, and key differences between multivariable versus multivariate regression. The goal is to provide clarity regarding the use and interpretation of multivariable analyses for those attempting to increase their statistical literacy in orthopaedic research.

Survival analyses and their applications in orthopaedics.

Survival analyses are a powerful statistical tool used to analyse data when the outcome of interest involves the time until an event. There is an array of models fit for this goal; however, there are subtle differences in assumptions, as well as a number of pitfalls, that can lead to biased results if researchers are unaware of the subtleties. As larger amounts of data become available, and more survival analyses are published every year, it is important that healthcare professionals understand how to evaluate these models and apply them into their practice. Therefore, the purpose of this study was to present an overview of survival analyses, including required assumptions and important pitfalls, as well as examples of their use within orthopaedic surgery.

A guide to appropriately planning and conducting meta-analyses: part 3. Special considerations-the network meta-analysis.

The meta-analysis has become one of the predominant studies designs in orthopaedic literature. Within recent years, the network meta-analysis has been implicated as a powerful approach to comparing multiple treatments for an outcome of interest when conducting a meta-analysis (as opposed to two competing treatments which is typical of a traditional meta-analysis). With the increasing use of the network meta-analysis, it is imperative for readers to possess the ability to independently and critically evaluate these types of studies. The purpose of this article is to provide the necessary foundation of knowledge to both properly conduct and interpret the results of a network meta-analysis.

A guide to appropriately planning and conducting meta-analyses: part 2-effect size estimation, heterogeneity and analytic approaches.

Meta-analyses by definition are a subtype of systematic review intended to quantitatively assess the strength of evidence present on an intervention or treatment. Such analyses may use individual-level data or aggregate data to produce a point estimate of an effect, also known as the combined effect, and measure precision of the calculated estimate. The current article will review several important considerations during the analytic phase of a meta-analysis, including selection of effect estimators, heterogeneity and various sub-types of meta-analytic approaches.

Supervised machine learning and associated algorithms: applications in orthopedic surgery.

Supervised learning is the most common form of machine learning utilized in medical research. It is used to predict outcomes of interest or classify positive and/or negative cases with a known ground truth. Supervised learning describes a spectrum of techniques, ranging from traditional regression modeling to more complex tree boosting, which are becoming increasingly prevalent as the focus on "big data" develops. While these tools are becoming increasingly popular and powerful, there is a paucity of literature available that describe the strengths and limitations of these different modeling techniques. Typically, there is no formal training for health care professionals in the use of machine learning models. As machine learning applications throughout medicine increase, it is important that physicians and other health care professionals better understand the processes underlying application of these techniques. The purpose of this study is to provide an overview of commonly used supervised learning techniques with recent case examples within the orthopedic literature. An additional goal is to address disparities in the understanding of these methods to improve communication within and between research teams.

A practical guide to the development and deployment of deep learning models for the Orthopedic surgeon: part I.

Deep learning has a profound impact on daily life. As Orthopedics makes use of this rapid escalation in technology, Orthopedic surgeons will need to take leadership roles on deep learning projects. Moreover, surgeons must possess an understanding of what is necessary to design and implement deep learning-based project pipelines. This review provides a practical guide for the Orthopedic surgeon to understand the steps needed to design, develop, and deploy a deep learning pipeline for clinical applications. A detailed description of the processes involved in defining the problem, building the team, acquiring and curating the data, labeling the data, establishing the ground truth, pre-processing and augmenting the data, and selecting the required hardware is provided. In addition, an overview of unique considerations involved in the training and evaluation of deep learning models is provided. This review strives to provide surgeons with the groundwork needed to identify gaps in the clinical landscape that deep learning models may be able to fill and equips them with the knowledge needed to lead an interdisciplinary team through the process of creating novel deep-learning-based solutions to fill those gaps.

Natural language processing: using artificial intelligence to understand human language in orthopedics.

Natural language processing (NLP) describes the broad field of artificial intelligence by which computers are trained to understand and generate human language. Within healthcare research, NLP is commonly used for variable extraction and classification/cohort identification tasks. While these tools are becoming increasingly popular and available as both open-source and commercial products, there is a paucity of the literature within the orthopedic space describing the key tasks within these powerful pipelines. Curation and navigation of the electronic medical record are becoming increasingly onerous, and it is important for physicians and other healthcare professionals to understand potential methods of harnessing this large data resource. The purpose of this study is to provide an overview of the tasks required to develop an NLP pipeline for orthopedic research and present recent examples of successful implementations.

A practical guide to the development and deployment of deep learning models for the orthopedic surgeon: part II.

Deep learning has the potential to be one of the most transformative technologies to impact orthopedic surgery. Substantial innovation in this area has occurred over the past 5 years, but clinically meaningful advancements remain limited by a disconnect between clinical and technical experts. That is, it is likely that few orthopedic surgeons possess both the clinical knowledge necessary to identify orthopedic problems, and the technical knowledge needed to implement deep learning-based solutions. To maximize the utilization of rapidly advancing technologies derived from deep learning models, orthopedic surgeons should understand the steps needed to design, organize, implement, and evaluate a deep learning project and its workflow. Equipping surgeons with this knowledge is the objective of this three-part editorial review. Part I described the processes involved in defining the problem, team building, data acquisition, curation, labeling, and establishing the ground truth. Building on that, this review (Part II) provides guidance on pre-processing and augmenting the data, making use of open-source libraries/toolkits, and selecting the required hardware to implement the pipeline. Special considerations regarding model training and evaluation unique to deep learning models relative to "shallow" machine learning models are also reviewed. Finally, guidance pertaining to the clinical deployment of deep learning models in the real world is provided. As in Part I, the focus is on applications of deep learning for computer vision and imaging.

High variability and lack of standardization in the evaluation of return to sport after ACL reconstruction: a systematic review.

PURPOSE: Return to sport (RTS) after ACL reconstruction (ACLR) has been recognized as an important outcome, which is associated with success of the surgery. This study aimed to assess the methods used to determine return to sport after ACLR in the published literature, report on variability of methods and evaluate their strength in establishing accurate RTS data. METHODS: Electronic databases (PubMed, Cochrane Library and Embase) were searched via a defined search strategy with no limits, to identify relevant studies from January 2008 to December 2020 for inclusion in the review. Defined eligibility criteria included studies specifically measuring and reporting on return to sport after ACLR with a clear methodology. Each included study was assessed for the definition of successful RTS, successful return to pre-injury level of sport and for methods used to determine RTS. RESULTS: One hundred and seventy-one studies were included. Of the included studies, six studies (4%) were level of evidence 1 and seventy-two studies (42%) were level of evidence 4. Forty-one studies (24%) reported on return to a specific sport and 130 studies (76%) reported on return to multiple sports or general sport. Sixteen studies (9%) reported on RTS in the pediatric population, 36 (21%) in the adult population and 119 (70%) reported on a mixed-aged population. The most commonly used definition of successful RTS was return to the same sport (44 of 125 studies, 35%). The most common method used to determine RTS was a non-validated study-specific questionnaire (73 studies, 43%), which was administered in various ways to the patients. Time of RTS assessment was variable and ranged between 6 months and 27 years post-surgery. CONCLUSION: This review demonstrates high variability in defining, evaluating and reporting RTS following ACLR. The findings of this study reveal low reliability and unproven validity of methods used to evaluate RTS and highlight the challenges in interpreting and using RTS data reported in literature. LEVEL OF EVIDENCE: IV.

The development and deployment of machine learning models.

Applications of artificial intelligence, specifically machine learning, are becoming increasingly popular in Orthopaedic Surgery, and medicine as a whole. This growing interest is shared by data scientists and physicians alike. However, there is an asymmetry of understanding of the developmental process and potential applications of machine learning. As new technology will undoubtedly affect clinical practice in the coming years, it is important for physicians to understand how these processes work. The purpose of this paper is to provide clarity and a general framework for building and assessing machine learning models.

Fragility Part I: a guide to understanding statistical power.

The aim of this paper is to close the knowledge-to-practice gap around statistical power. We demonstrate how four factors affect power: p value, effect size, sample size, and variance. This article further delves into the advantages and disadvantages of a priori versus post hoc power analyses, though we believe only understanding of the former is essential to addressing the present-day issue of reproducibility in research. Upon reading this paper, physician-scientists should have expanded their arsenal of statistical tools and have the necessary context to understand statistical fragility.

Picking apart p values: common problems and points of confusion.

Due to its frequent misuse, the p value has become a point of contention in the research community. In this editorial, we seek to clarify some of the common misconceptions about p values and the hazardous implications associated with misunderstanding this commonly used statistical concept. This article will discuss issues related to p value interpretation in addition to problems such as p-hacking and statistical fragility; we will also offer some thoughts on addressing these issues. The aim of this editorial is to provide clarity around the concept of statistical significance for those attempting to increase their statistical literacy in Orthopedic research.