How Does ChatGPT Use Source Information Compared With Google? A Text Network Analysis of Online Health Information.

BACKGROUND: The lay public is increasingly using ChatGPT (a large language model) as a source of medical information. Traditional search engines such as Google provide several distinct responses to each search query and indicate the source for each response, but ChatGPT provides responses in paragraph form in prose without providing the sources used, which makes it difficult or impossible to ascertain whether those sources are reliable. One practical method to infer the sources used by ChatGPT is text network analysis. By understanding how ChatGPT uses source information in relation to traditional search engines, physicians and physician organizations can better counsel patients on the use of this new tool. QUESTIONS/PURPOSES: (1) In terms of key content words, how similar are ChatGPT and Google Search responses for queries related to topics in orthopaedic surgery? (2) Does the source distribution (academic, governmental, commercial, or form of a scientific manuscript) differ for Google Search responses based on the topic's level of medical consensus, and how is this reflected in the text similarity between ChatGPT and Google Search responses? (3) Do these results vary between different versions of ChatGPT? METHODS: We evaluated three search queries relating to orthopaedic conditions: "What is the cause of carpal tunnel syndrome?," "What is the cause of tennis elbow?," and "Platelet-rich plasma for thumb arthritis?" These were selected because of their relatively high, medium, and low consensus in the medical evidence, respectively. Each question was posed to ChatGPT version 3.5 and version 4.0 20 times for a total of 120 responses. Text network analysis using term frequency-inverse document frequency (TF-IDF) was used to compare text similarity between responses from ChatGPT and Google Search. In the field of information retrieval, TF-IDF is a weighted statistical measure of the importance of a key word to a document in a collection of documents. Higher TF-IDF scores indicate greater similarity between two sources. TF-IDF scores are most often used to compare and rank the text similarity of documents. Using this type of text network analysis, text similarity between ChatGPT and Google Search can be determined by calculating and summing the TF-IDF for all keywords in a ChatGPT response and comparing it with each Google search result to assess their text similarity to each other. In this way, text similarity can be used to infer relative content similarity. To answer our first question, we characterized the text similarity between ChatGPT and Google Search responses by finding the TF-IDF scores of the ChatGPT response and each of the 20 Google Search results for each question. Using these scores, we could compare the similarity of each ChatGPT response to the Google Search results. To provide a reference point for interpreting TF-IDF values, we generated randomized text samples with the same term distribution as the Google Search results. By comparing ChatGPT TF-IDF to the random text sample, we could assess whether TF-IDF values were statistically significant from TF-IDF values obtained by random chance, and it allowed us to test whether text similarity was an appropriate quantitative statistical measure of relative content similarity. To answer our second question, we classified the Google Search results to better understand sourcing. Google Search provides 20 or more distinct sources of information, but ChatGPT gives only a single prose paragraph in response to each query. So, to answer this question, we used TF-IDF to ascertain whether the ChatGPT response was principally driven by one of four source categories: academic, government, commercial, or material that took the form of a scientific manuscript but was not peer-reviewed or indexed on a government site (such as PubMed). We then compared the TF-IDF similarity between ChatGPT responses and the source category. To answer our third research question, we repeated both analyses and compared the results when using ChatGPT 3.5 versus ChatGPT 4.0. RESULTS: The ChatGPT response was dominated by the top Google Search result. For example, for carpal tunnel syndrome, the top result was an academic website with a mean TF-IDF of 7.2. A similar result was observed for the other search topics. To provide a reference point for interpreting TF-IDF values, a randomly generated sample of text compared with Google Search would have a mean TF-IDF of 2.7 ± 1.9, controlling for text length and keyword distribution. The observed TF-IDF distribution was higher for ChatGPT responses than for random text samples, supporting the claim that keyword text similarity is a measure of relative content similarity. When comparing source distribution, the ChatGPT response was most similar to the most common source category from Google Search. For the subject where there was strong consensus (carpal tunnel syndrome), the ChatGPT response was most similar to high-quality academic sources rather than lower-quality commercial sources (TF-IDF 8.6 versus 2.2). For topics with low consensus, the ChatGPT response paralleled lower-quality commercial websites compared with higher-quality academic websites (TF-IDF 14.6 versus 0.2). ChatGPT 4.0 had higher text similarity to Google Search results than ChatGPT 3.5 (mean increase in TF-IDF similarity of 0.80 to 0.91; p < 0.001). The ChatGPT 4.0 response was still dominated by the top Google Search result and reflected the most common search category for all search topics. CONCLUSION: ChatGPT responses are similar to individual Google Search results for queries related to orthopaedic surgery, but the distribution of source information can vary substantially based on the relative level of consensus on a topic. For example, for carpal tunnel syndrome, where there is widely accepted medical consensus, ChatGPT responses had higher similarity to academic sources and therefore used those sources more. When fewer academic or government sources are available, especially in our search related to platelet-rich plasma, ChatGPT appears to have relied more heavily on a small number of nonacademic sources. These findings persisted even as ChatGPT was updated from version 3.5 to version 4.0. CLINICAL RELEVANCE: Physicians should be aware that ChatGPT and Google likely use the same sources for a specific question. The main difference is that ChatGPT can draw upon multiple sources to create one aggregate response, while Google maintains its distinctness by providing multiple results. For topics with a low consensus and therefore a low number of quality sources, there is a much higher chance that ChatGPT will use less-reliable sources, in which case physicians should take the time to educate patients on the topic or provide resources that give more reliable information. Physician organizations should make it clear when the evidence is limited so that ChatGPT can reflect the lack of quality information or evidence.

Complications and Unplanned Reoperation After Thumb Metacarpophalangeal Arthrodesis.

PURPOSE: Arthrodesis of the metacarpophalangeal (MCP) joint of the thumb is a common procedure to treat arthritis or instability. Studies reporting hardware complications and nonunion rates after thumb MCP joint arthrodesis report on small sample sizes. We aimed to describe the hardware complication rate, the nonunion rate, and the number of thumbs that achieve union among patients undergoing thumb MCP joint arthrodesis. METHODS: A database spanning 5 urban hospitals in a single metropolitan region in the United States was searched for patients who underwent thumb MCP joint arthrodesis between January 1, 2004 and January 1, 2020. After reviewing patient records, we identified 122 thumbs that underwent MCP joint arthrodesis and had a minimum follow-up of 90 days. The primary outcome was unplanned reoperation after hardware complications and nonunion. Second, the number of thumbs that achieved radiographic union was reported for the tension band and screw fixation arthrodesis group. RESULTS: Twenty-two (18%) out of 122 thumbs had hardware complications after thumb MCP joint arthrodesis, and 11 (9%) out of 122 thumbs developed a nonunion. Patients who underwent screw fixation arthrodesis had no events of hardware complications and subsequent hardware removal. The nonunion rate was 9/65 (14%) in the tension band arthrodesis group and 2 (4%) of 45 in the screw fixation arthrodesis group. Of the thumbs that had available radiographs to assess the healing of the arthrodesis, 34 (81%) of 42 were radiographically united in the tension band arthrodesis group and 29 (91%) of 32 in the screw fixation group. CONCLUSIONS: Our data suggest that screw fixation has fewer hardware complications and a lower reoperation rate than tension band arthrodesis. TYPE OF STUDY/LEVEL OF EVIDENCE: Prognosis IV.

Factors Associated With Reoperation After Distal Radius Nonunion Repair.

PURPOSE: This study aimed to evaluate the incidence of, and factors associated with, reoperation after distal radius nonunion repair. METHODS: We conducted a retrospective cohort study at a multicenter academic institution and identified adult patients who underwent open reduction and internal fixation for distal radius nonunion between January 2005 and August 2021. Thirty-three patients were included in this study. The cohort consisted of 13 males (13/33) and had a median age of 56 years (interquartile ranges: 49-64). Median follow-up was 59 months (interquartile ranges: 23-126). RESULTS: Unplanned reoperations occurred in eight of 33 patients. The most common reasons for reoperation were irrigation and debridement for infection, revision surgery for persistent nonunion, and unplanned hardware removal. In total, 10 complications occurred in nine patients. The most common complications were infection and persistent nonunion; both occurred in three cases. CONCLUSIONS: Complications after distal radius nonunion repair are common. Reoperation after distal radius nonunion repair is required in approximately one of four cases. TYPE OF STUDY/LEVEL OF EVIDENCE: Prognosis IV.

Anatomy of the Lumbrical Muscles: Implications for Mechanical Advantage.

BACKGROUND: The lumbrical muscles comprise 4 intrinsic muscles of the hand and are involved in flexion of the metacarpophalangeal joint (MCPJ) and extension of the proximal interphalangeal and distal interphalangeal joints. The purpose of this study was to investigate the anatomical mechanics of the lumbrical muscles of the index, middle, ring, and small fingers. METHODS: We evaluated 25 cadaver arms and measured the distance between the MCPJ and fingertip, the distance between the MCPJ and lumbrical muscle insertion, and the distance between the MCPJ and the most proximal lumbrical muscle origin. With these measurements we calculated the needed force, insertion ratio (length of the proximal, middle, and distal phalanx divided by the MCPJ to insertion distance), and lumbrical muscle length. RESULTS: We found that the force was significantly different between all fingers, except for the comparison of the index and ring finger (P = .34). In addition, we found that muscle length was significantly different between most the fingers, except for the comparison between the index and middle fingers (P = .24), and index and ring fingers (P = .20). There was no significant difference in insertion ratio. CONCLUSIONS: Our study suggests that the anatomical mechanics for the motor function of the lumbrical muscles are similar in all fingers. This could further imply that movements are equally precise in all fingers resulting in coordination with one another and, therefore, adequate hand function. LEVEL OF EVIDENCE: IV.

Cause of Extensor Pollicis Longus Ruptures After Distal Radius Fracture Fixation Using a Volar Plate.

BACKGROUND: This research sought to analyze a cohort of patients with extensor pollicis longus (EPL) ruptures after volar locked plating of a distal radius fracture (DRF) to characterize the incidence of ruptures that are unlikely to be related to dorsal screw prominence. METHODS: This is a retrospective, observational, descriptive cohort study of adults with operative fixation of a closed DRF and an EPL rupture between 2002 and 2022. Eighteen patients with operative fixation using a volar plate of a closed DRF had an EPL rupture. The cohort consisted of 66% women with an average age of 57.5 years. Median follow-up was 14.5 months. RESULTS: The incidence of EPL rupture was 0.4% (18/4768). The average time from DRF and DRF fixation to EPL rupture was 3.7 and 3.4 months, respectively. Based on the operative record, in 2 of the 18 patients (11%), the rupture was directly attributable to prominent hardware; however, in 4 of the 18 patients (22%), the rupture was not related to prominent hardware, and the cause was indeterminate in 12 patients (67%). Radiologic analysis of those in the indeterminate group demonstrated that 5 of the 12 patients had screws that had a high probability of being prominent. CONCLUSIONS: The incidence of EPL rupture after volar plating of DRF is between 0% and 1% and usually occurs about 3 months after fixation. Approximately 50% of EPL ruptures are attributable to prominent dorsal screws. Although screw prominence is an important cause of EPL rupture, it is not the sole cause of rupture.

Psychosocial and functional impact of successful digital replantation - A cohort study of 36 patients with a median follow-up of 6 years.

INTRODUCTION: This study aims to evaluate the long-term psychosocial and functional outcomes of successful digital replantation following traumatic amputation. METHODS: Patients that underwent successful replantation (i.e. no secondary amputation following replantation) of one or more traumatically amputated digits between January 2009 and April 2019 were invited to participate in this study. In addition to a custom questionnaire on psychosocial and socioeconomic aspects of life, various Patient-Reported Outcomes Measurement Information System (PROMIS) questionnaires regarding global health, upper extremity function, and depressive symptoms were completed. Bivariate analyses were performed to identify significant associations between outcomes and explanatory variables. RESULTS: Thirty-six patients were successfully enrolled and completed the questionnaires at a median follow-up of 6.1 years. The median PROMIS score for Upper Extremity Function (40.6) was considerably different from the score that is typically found in the general population (all PROMIS instruments are calibrated with a control group score of 50.0), but the median PROMIS scores for Global Health - Physical (49.0), Global Health - Mental (50.7), and Depression (45.6) were comparable to those among the general population. Dominant hand injury, a greater number of injured digits, higher age at the time of injury, and the need for neuropathic pain medication were associated with lower Upper Extremity Function scores (all p < 0.05). Additionally, the presence of neuroma was associated with negative changes in both household finances and mental well-being (p < 0.05). CONCLUSIONS: At long-term follow-up, a majority of patients that underwent replantation of traumatically amputated digits seem to cope well based on psychosocial and functional outcomes. However, neuropathic pain and the presence of neuroma are strong negative factors. Specific attention to digital nerves at the time of surgery is crucial in the management of traumatic amputations.