How Does ChatGPT Use Source Information Compared With Google? A Text Network Analysis of Online Health Information.

BACKGROUND: The lay public is increasingly using ChatGPT (a large language model) as a source of medical information. Traditional search engines such as Google provide several distinct responses to each search query and indicate the source for each response, but ChatGPT provides responses in paragraph form in prose without providing the sources used, which makes it difficult or impossible to ascertain whether those sources are reliable. One practical method to infer the sources used by ChatGPT is text network analysis. By understanding how ChatGPT uses source information in relation to traditional search engines, physicians and physician organizations can better counsel patients on the use of this new tool. QUESTIONS/PURPOSES: (1) In terms of key content words, how similar are ChatGPT and Google Search responses for queries related to topics in orthopaedic surgery? (2) Does the source distribution (academic, governmental, commercial, or form of a scientific manuscript) differ for Google Search responses based on the topic's level of medical consensus, and how is this reflected in the text similarity between ChatGPT and Google Search responses? (3) Do these results vary between different versions of ChatGPT? METHODS: We evaluated three search queries relating to orthopaedic conditions: "What is the cause of carpal tunnel syndrome?," "What is the cause of tennis elbow?," and "Platelet-rich plasma for thumb arthritis?" These were selected because of their relatively high, medium, and low consensus in the medical evidence, respectively. Each question was posed to ChatGPT version 3.5 and version 4.0 20 times for a total of 120 responses. Text network analysis using term frequency-inverse document frequency (TF-IDF) was used to compare text similarity between responses from ChatGPT and Google Search. In the field of information retrieval, TF-IDF is a weighted statistical measure of the importance of a key word to a document in a collection of documents. Higher TF-IDF scores indicate greater similarity between two sources. TF-IDF scores are most often used to compare and rank the text similarity of documents. Using this type of text network analysis, text similarity between ChatGPT and Google Search can be determined by calculating and summing the TF-IDF for all keywords in a ChatGPT response and comparing it with each Google search result to assess their text similarity to each other. In this way, text similarity can be used to infer relative content similarity. To answer our first question, we characterized the text similarity between ChatGPT and Google Search responses by finding the TF-IDF scores of the ChatGPT response and each of the 20 Google Search results for each question. Using these scores, we could compare the similarity of each ChatGPT response to the Google Search results. To provide a reference point for interpreting TF-IDF values, we generated randomized text samples with the same term distribution as the Google Search results. By comparing ChatGPT TF-IDF to the random text sample, we could assess whether TF-IDF values were statistically significant from TF-IDF values obtained by random chance, and it allowed us to test whether text similarity was an appropriate quantitative statistical measure of relative content similarity. To answer our second question, we classified the Google Search results to better understand sourcing. Google Search provides 20 or more distinct sources of information, but ChatGPT gives only a single prose paragraph in response to each query. So, to answer this question, we used TF-IDF to ascertain whether the ChatGPT response was principally driven by one of four source categories: academic, government, commercial, or material that took the form of a scientific manuscript but was not peer-reviewed or indexed on a government site (such as PubMed). We then compared the TF-IDF similarity between ChatGPT responses and the source category. To answer our third research question, we repeated both analyses and compared the results when using ChatGPT 3.5 versus ChatGPT 4.0. RESULTS: The ChatGPT response was dominated by the top Google Search result. For example, for carpal tunnel syndrome, the top result was an academic website with a mean TF-IDF of 7.2. A similar result was observed for the other search topics. To provide a reference point for interpreting TF-IDF values, a randomly generated sample of text compared with Google Search would have a mean TF-IDF of 2.7 ± 1.9, controlling for text length and keyword distribution. The observed TF-IDF distribution was higher for ChatGPT responses than for random text samples, supporting the claim that keyword text similarity is a measure of relative content similarity. When comparing source distribution, the ChatGPT response was most similar to the most common source category from Google Search. For the subject where there was strong consensus (carpal tunnel syndrome), the ChatGPT response was most similar to high-quality academic sources rather than lower-quality commercial sources (TF-IDF 8.6 versus 2.2). For topics with low consensus, the ChatGPT response paralleled lower-quality commercial websites compared with higher-quality academic websites (TF-IDF 14.6 versus 0.2). ChatGPT 4.0 had higher text similarity to Google Search results than ChatGPT 3.5 (mean increase in TF-IDF similarity of 0.80 to 0.91; p < 0.001). The ChatGPT 4.0 response was still dominated by the top Google Search result and reflected the most common search category for all search topics. CONCLUSION: ChatGPT responses are similar to individual Google Search results for queries related to orthopaedic surgery, but the distribution of source information can vary substantially based on the relative level of consensus on a topic. For example, for carpal tunnel syndrome, where there is widely accepted medical consensus, ChatGPT responses had higher similarity to academic sources and therefore used those sources more. When fewer academic or government sources are available, especially in our search related to platelet-rich plasma, ChatGPT appears to have relied more heavily on a small number of nonacademic sources. These findings persisted even as ChatGPT was updated from version 3.5 to version 4.0. CLINICAL RELEVANCE: Physicians should be aware that ChatGPT and Google likely use the same sources for a specific question. The main difference is that ChatGPT can draw upon multiple sources to create one aggregate response, while Google maintains its distinctness by providing multiple results. For topics with a low consensus and therefore a low number of quality sources, there is a much higher chance that ChatGPT will use less-reliable sources, in which case physicians should take the time to educate patients on the topic or provide resources that give more reliable information. Physician organizations should make it clear when the evidence is limited so that ChatGPT can reflect the lack of quality information or evidence.

Complications and Unplanned Reoperation After Thumb Metacarpophalangeal Arthrodesis.

PURPOSE: Arthrodesis of the metacarpophalangeal (MCP) joint of the thumb is a common procedure to treat arthritis or instability. Studies reporting hardware complications and nonunion rates after thumb MCP joint arthrodesis report on small sample sizes. We aimed to describe the hardware complication rate, the nonunion rate, and the number of thumbs that achieve union among patients undergoing thumb MCP joint arthrodesis. METHODS: A database spanning 5 urban hospitals in a single metropolitan region in the United States was searched for patients who underwent thumb MCP joint arthrodesis between January 1, 2004 and January 1, 2020. After reviewing patient records, we identified 122 thumbs that underwent MCP joint arthrodesis and had a minimum follow-up of 90 days. The primary outcome was unplanned reoperation after hardware complications and nonunion. Second, the number of thumbs that achieved radiographic union was reported for the tension band and screw fixation arthrodesis group. RESULTS: Twenty-two (18%) out of 122 thumbs had hardware complications after thumb MCP joint arthrodesis, and 11 (9%) out of 122 thumbs developed a nonunion. Patients who underwent screw fixation arthrodesis had no events of hardware complications and subsequent hardware removal. The nonunion rate was 9/65 (14%) in the tension band arthrodesis group and 2 (4%) of 45 in the screw fixation arthrodesis group. Of the thumbs that had available radiographs to assess the healing of the arthrodesis, 34 (81%) of 42 were radiographically united in the tension band arthrodesis group and 29 (91%) of 32 in the screw fixation group. CONCLUSIONS: Our data suggest that screw fixation has fewer hardware complications and a lower reoperation rate than tension band arthrodesis. TYPE OF STUDY/LEVEL OF EVIDENCE: Prognosis IV.

Functional outcomes and complications after treatment of distal radius fracture in patients sixty years and over: A systematic review and network meta-analysis.

AIM: This network meta-analysis aims to compare functional outcomes and complications between conservative treatment and surgery for distal radius fractures in patients aged 60 years and over. METHODS: We searched the PubMed, EMBASE, and Web of Science databases for randomized controlled trials (RCTs) assessing the effect of conservative treatment and surgery for distal radius fractures in patients aged 60 years and over. Primary outcomes included grip strength and overall complications. Secondary outcomes included Disabilities of the Arm, Shoulder, and Hand (DASH) scores, Patient-Rated Wrist Evaluation (PRWE) scores, wrist range of motion and forearm rotation, and radiographic assessment. All continuous outcomes were assessed using standardized mean differences (SMDs) with 95% confidence intervals (CIs), and binary outcomes were assessed using odds ratio (OR) with 95% CIs. The surface under the cumulative ranking curve (SUCRA) was used to determine a hierarchy of treatments. Cluster analysis was performed for grouping treatments based on the SUCRA values of primary outcomes. RESULTS: Fourteen RCTs were included to compare conservative treatment, volar lockedplate (VLP), K-wires fixation, and external-fixation. VLP outperformed conservative treatment for 1-year and minimum 2-year grip strength (SMD; 0.28 [0.07 to 0.48] and 0.27 [0.02 to 0.53], respectively). VLP yielded the optimal grip strength at 1-year and minimum 2-year follow-up (SUCRA; 89.8% and 86.7%, respectively). In a subgroup analysis of patients aged 60 to 80 years old, VLP outperformed conservative treatment in DASH and PRWE scores (SMD, 0.33 [0.10, 0.56] and 0.23 [0.01, 0.45], respectively). In addition, VLP had the fewest complications (SUCRA = 84.3%). Cluster analysis suggested that VLP and K-wire fixation were more effective treatment groups. CONCLUSION: Evidence to date demonstrates that VLP provides measurable benefits in grip strength and fewer complications to those 60 years of age and over, and that benefit is not reflected in current practice guidelines. There is a subgroup of patients where K-wire fixation outcomes are similar to those of VLP; defining this subgroup may yield substantial societal benefits.

Popliteal pseudoaneurysm in a young patient with multiple hereditary exostosis.

Multiple hereditary exostosis is an osteogenic disorder that causes outgrowths of cartilaginous bone tumors that are associated with adjacent neurovascular compressive injuries. We present the case of an adolescent male with multiple hereditary exostosis complicated by popliteal pseudoaneurysm formation who underwent excision of the osteochondroma and vein patch angioplasty repair of the artery. We highlight the rare association between this genetic disease and subsequent vascular complications and review the available literature of arterial complications of this disease.

Demystifying the long noncoding RNA landscape of small EVs derived from human mesenchymal stromal cells.

INTRODUCTION: The regenerative capacity of mesenchymal stromal cells or medicinal signaling cells (MSCs) is largely mediated by their secreted small extracellular vesicles (sEVs), and the therapeutic efficacy of sEVs can be enhanced by licensing approaches (e.g., cytokines, hypoxia, chemicals, and genetic modification). Noncoding RNAs within MSC-derived sEVs (MSC-sEVs) have been demonstrated to be responsible for tissue regeneration. However, unlike miRNA fingerprints, which have been explored, the landscape of long noncoding RNAs (lncRNAs) in MSC-sEVs remains to be described. OBJECTIVES: To characterize lncRNA signatures in sEVs of human adipose-derived MSCs with or without inflammatory cytokine licensing and depict MSC-sEV-specific and MSC-enriched lncRNA repertoires. METHODS: sEVs were isolated from MSCs with or without TNF-α and IFN-γ (20 ng/mL) stimulation. High-throughput lncRNA sequencing and an in silico approach were employed to analyze the profile of lncRNAs in sEVs and predict lncRNA-protein interactomes. RESULTS: sEVs derived from human MSCs and fibroblasts carried a unique landscape of lncRNAs distinct from the lncRNAs inside these cells. Compared with fibroblast-derived sEVs (F-sEVs), 194 MSC-sEV-specific and 8 upregulated lncRNAs in MSC-sEVs were considered "medicinal signaling lncRNAs"; inflammatory cytokines upregulated 27 lncRNAs in MSC-sEVs, which were considered "licensing-responsive lncRNAs". Based on lncRNA-protein interactome prediction and enrichment analysis, we found that the proteins interacting with medicinal signaling lncRNAs or licensing-responsive lncRNAs have a tight interaction network involved in chromatin remodeling, SWI/SNF superfamily type complexes, and histone binding. CONCLUSION: In summary, our study depicts the landscape of lncRNAs in MSC-sEVs and predicts their potential functions via the lncRNA-protein interactome. Elucidation of the lncRNA landscape of MSC-sEVs will facilitate defining the therapeutic potency of MSC-sEVs and the development of sEV-based therapeutics.

The Efficacy of Naïve versus Modified Mesenchymal Stem Cells in Improving Muscle Function in Duchenne Muscular Dystrophy: A Systematic Review.

As one of the most common genetic conditions, Duchenne muscular dystrophy (DMD) is a fatal disease caused by a recessive mutation resulting in muscle weakness in both voluntary and involuntary muscles and, eventually, in death because of cardiovascular failure. Currently, there is no pharmacologically curative treatment of DMD, but there is evidence supporting that mesenchymal stem cells (MSCs) are a novel solution for treating DMD. This systematic review focused on elucidating the therapeutic efficacy of MSCs on the DMD in vivo model. A key issue of previous studies was the material-choice, naïve MSCs or modified MSCs; modified MSCs are activated by culture methods or genetic modification. In summary, MSCs seem to improve pulmonary and cardiac functions and thereby improve survival regardless of them being naïve or modified. The improved function of distal skeletal muscles was observed only with primed MSCs treatment but not naïve MSCs. While MSCs can provide significant benefits to DMD mouse models, there is little to no data on the results in human patients. Due to the limited number of human studies, the differences in study design, and the insufficient understanding of mechanisms of action, more rigorous comparative trials are needed to elucidate which types of MSCs and modifications have optimal therapeutic potential.

The Crosstalk between Mesenchymal Stem Cells and Macrophages in Bone Regeneration: A Systematic Review.

Bone regeneration is a complex and well-coordinated process that involves crosstalk between immune cells and resident cells in the injury site. Transplantation of mesenchymal stem cells (MSCs) is a promising strategy to enhance bone regeneration. Growing evidence suggests that macrophages have a significant impact on osteogenesis during bone regeneration. However, the precise mechanisms by which macrophage subtypes influence bone regeneration and how MSCs communicate with macrophages have not yet been fully elucidated. In this systematic literature review, we gathered evidence regarding the crosstalk between MSCs and macrophages during bone regeneration. According to the PRISMA protocol, we extracted literature from PubMed and Embase databases by using "mesenchymal stem cells" and "macrophages" and "bone regeneration" as keywords. Thirty-three studies were selected for this review. MSCs isolated from both bone marrow and adipose tissue and both primary macrophages and macrophage cell lines were used in the selected studies. In conclusion, anti-inflammatory macrophages (M2) have significantly more potential to strengthen bone regeneration compared with naïve (M0) and classically activated macrophages (M1). Transplantation of MSCs induced M1-to-M2 transition and transformed the skeletal microenvironment to facilitate bone regeneration in bone fracture and bone defect models. This review highlights the complexity between MSCs and macrophages, providing more insight into the polarized macrophage behavior in this evolving field of osteoimmunology. The results may serve as a useful reference for definite success in MSC-based therapy based on the critical interaction with macrophages.