Accuracy, readability, and understandability of large language models for prostate cancer information to the public.

BACKGROUND: Generative Pretrained Model (GPT) chatbots have gained popularity since the public release of ChatGPT. Studies have evaluated the ability of different GPT models to provide information about medical conditions. To date, no study has assessed the quality of ChatGPT outputs to prostate cancer related questions from both the physician and public perspective while optimizing outputs for patient consumption. METHODS: Nine prostate cancer-related questions, identified through Google Trends (Global), were categorized into diagnosis, treatment, and postoperative follow-up. These questions were processed using ChatGPT 3.5, and the responses were recorded. Subsequently, these responses were re-inputted into ChatGPT to create simplified summaries understandable at a sixth-grade level. Readability of both the original ChatGPT responses and the layperson summaries was evaluated using validated readability tools. A survey was conducted among urology providers (urologists and urologists in training) to rate the original ChatGPT responses for accuracy, completeness, and clarity using a 5-point Likert scale. Furthermore, two independent reviewers evaluated the layperson summaries on correctness trifecta: accuracy, completeness, and decision-making sufficiency. Public assessment of the simplified summaries' clarity and understandability was carried out through Amazon Mechanical Turk (MTurk). Participants rated the clarity and demonstrated their understanding through a multiple-choice question. RESULTS: GPT-generated output was deemed correct by 71.7% to 94.3% of raters (36 urologists, 17 urology residents) across 9 scenarios. GPT-generated simplified layperson summaries of this output was rated as accurate in 8 of 9 (88.9%) scenarios and sufficient for a patient to make a decision in 8 of 9 (88.9%) scenarios. Mean readability of layperson summaries was higher than original GPT outputs ([original ChatGPT v. simplified ChatGPT, mean (SD), p-value] Flesch Reading Ease: 36.5(9.1) v. 70.2(11.2), <0.0001; Gunning Fog: 15.8(1.7) v. 9.5(2.0), p < 0.0001; Flesch Grade Level: 12.8(1.2) v. 7.4(1.7), p < 0.0001; Coleman Liau: 13.7(2.1) v. 8.6(2.4), 0.0002; Smog index: 11.8(1.2) v. 6.7(1.8), <0.0001; Automated Readability Index: 13.1(1.4) v. 7.5(2.1), p < 0.0001). MTurk workers (n = 514) rated the layperson summaries as correct (89.5-95.7%) and correctly understood the content (63.0-87.4%). CONCLUSION: GPT shows promise for correct patient education for prostate cancer-related contents, but the technology is not designed for delivering patients information. Prompting the model to respond with accuracy, completeness, clarity and readability may enhance its utility when used for GPT-powered medical chatbots.

Evaluation of YouTube As A Source For Graves' Disease Information: Is High-Quality Guideline-Based Information Available?

Objective: To understand the quality of informational Graves' disease (GD) videos on YouTube for treatment decision-making quality and inclusion of American Thyroid Association (ATA) treatment guidelines. Study Design: Cross-sectional cohort. Setting: Informational YouTube videos with subject matter "Graves' Disease treatment." Method: The top 50 videos based on our query were assessed using the DISCERN instrument. This validated algorithm discretely rates treatment-related information from excellent (≥4.5) to very poor (<1.9). Videos were also screened for ATA guideline inclusion. Descriptive statistics were used for cohort characterization. Univariate and multivariate linear regressions characterized factors associated with DISCERN scores. Significance was set at P < .05. Results: The videos featured 57,513.43 views (SD = 162,579.25), 1054.70 likes (SD = 2329.77), and 168.80 comments (SD = 292.97). Most were patient education (52%) or patient experience (24%). A minority (40%) were made by thyroid specialists (endocrinologists, endocrine surgeons, or otolaryngologists). Under half did not mention all 3 treatment modalities (44%), and 54% did not mention any ATA recommendations. Overall, videos displayed poor reliability (mean = 2.26, SD = 0.67), treatment information quality (mean = 2.29, SD = 0.75), and overall video quality (mean = 2.47, SD = 1.07). Physician videos were associated with lower likes, views, and comments (P < .001) but higher DISCERN reliability (P = .015) and overall score (P = .019). Longer videos (P = .015), patient accounts (P = .013), and patient experience (P = .002) were associated with lower scores. Conclusion: The most available GD treatment content on YouTube varies significantly in the quality of medical information. This may contribute to suboptimal disease understanding, especially for patients highly engaged with online health information sources.

Associations Between Risk Perception and HPV Knowledge and Vaccine Uptake: Highlighting the LGBTQ+ Population.

INTRODUCTION: Gay and bisexual males and other LGBTQ+ communities are more frequently exposed to factors associated with an increased risk of human papillomavirus (HPV) acquisition. Vaccination is critical to protect against HPV+ head and neck cancer (HNC). We characterized the association of perceived level of risk of contraction with HPV knowledge, and vaccine decision-making. STUDY DESIGN: Cross-sectional cohort. SETTING: LGBTQ and general survey Reddit forums (control). METHODS: A survey was shared amongst the online forums. Descriptive statistics characterized the data. Multivariable logistic regression was used to understand factors associated with vaccination, self-perceived high risk, and knowledge of HPV + HNC. RESULTS: Of 718 respondents, most were female (41.09%), Caucasian (59.89%), college-educated (33.01%), and insured (77.15%) with a mean age of 30.75 years. Half were vaccinated (49.16%), with most unvaccinated endorsing interest (60.58%). Few dependents were vaccinated (25.91%), with interest in vaccination among parents of unvaccinated children (38.58%). Knowledge of HIV's association with HPV (62.95%), HPV causing HNC (55.57%), and the vaccine's efficacy against HNC (55.57%) was also moderate. Identifying female (P = .042), a self-perceived high-risk (P < .001), and having vaccinated children (P < .001) increased vaccination likelihood; transgender (P = .021), or lesbian or gay sexual identity (P < .001) decreased likelihood. Personal HNC diagnosis (P < .001), self-vaccination (P < .001), having vaccinated children (P < .001), having anal sex (P = .001) or no knowledge of past HPV status (P < .001) increased likelihood of high self-perceived risk. CONCLUSION: Efforts to improve public education regarding the association between HPV and HNC and vaccination efficacy are required to better inform vaccine decision-making among individuals at risk for HPV infection.

Evaluation of Oropharyngeal Cancer Information from Revolutionary Artificial Intelligence Chatbot.

OBJECTIVE: With burgeoning popularity of artificial intelligence-based chatbots, oropharyngeal cancer patients now have access to a novel source of medical information. Because chatbot information is not reviewed by experts, we sought to evaluate an artificial intelligence-based chatbot's oropharyngeal cancer-related information for accuracy. METHODS: Fifteen oropharyngeal cancer-related questions were developed and input into ChatGPT version 3.5. Four physician-graders independently assessed accuracy, comprehensiveness, and similarity to a physician response using 5-point Likert scales. Responses graded lower than three were then critiqued by physician-graders. Critiques were analyzed using inductive thematic analysis. Readability of responses was assessed using Flesch Reading Ease (FRE) and Flesch-Kincaid Reading Grade Level (FKRGL) scales. RESULTS: Average accuracy, comprehensiveness, and similarity to a physician response scores were 3.88 (SD = 0.99), 3.80 (SD = 1.14), and 3.67 (SD = 1.08), respectively. Posttreatment-related questions were most accurate, comprehensive, and similar to a physician response, followed by treatment-related, then diagnosis-related questions. Posttreatment-related questions scored significantly higher than diagnosis-related questions in all three domains (p < 0.01). Two themes of the physician critiques were identified: suboptimal education value and potential to misinform patients. The mean FRE and FKRGL scores both indicated greater than an 11th grade readability level-higher than the 6th grade level recommended for patients. CONCLUSION: ChatGPT responses may not educate patients to an appropriate degree, could outright misinform them, and read at a more difficult grade level than is recommended for patient material. As oropharyngeal cancer patients represent a vulnerable population facing complex, life-altering diagnoses, and treatments, they should be cautious when consuming chatbot-generated medical information. LEVEL OF EVIDENCE: NA Laryngoscope, 134:2252-2257, 2024.

Evaluation of the quality of thyroidectomy-related posts on a video-based social media platform.

Objective: To evaluate the quality of thyroidectomy-related posts on TikTok, the fastest-growing social media platform worldwide. Methods: Videos posted from April 2020 to September 2022 were queried on TikTok using the search terms "thyroidsurgery," "thyroidectomy," and "thyroidremoval." Two reviewers recorded thematic, demographic, and performance data of these posts. The DISCERN instrument was used to evaluate the quality and reliability of the information contained in the videos. Descriptive statistics were used to characterize post-submitter demographics and video content. Simple and multiple linear regression analyses were used to evaluate the association between DISCERN scores and video characteristics. Univariate analysis of variance was performed to compare DISCERN scores between author types. Results: In this study, 228 TikTok videos were included which totaled over 23 million views. On average, each video accumulated more than 6000 "likes," 300 comments, and 70 shares. The average total DISCERN score was 27.46, which is deemed to be of poor overall quality. Upon multiple linear regression, video duration (ß = 4.66, p < .001) and educational subject type (ß = 3.97, p < .001) significantly positively predicted aggregate DISCERN scores, while journey subject type (ß = -3.19, p = .006), and reassurance subject type (ß = -2.52, p = .035) significantly negatively predicted aggregate DISCERN scores. Aggregate DISCERN scores varied significantly (p < .05) between author types. Conclusion: Social media posts on TikTok about thyroidectomy are mostly of poor quality and reliability but vary by authorship, subject type, and video characteristics. Given its widespread popularity, TikTok videos may have an increasing role in shaping patient perception of thyroidectomy and may represent an opportunity to provide education. Lay summary: TikTok posts about thyroidectomy are mostly of poor quality but vary by authorship, subject, and video characteristics. Given its popularity, TikTok videos may have a role in shaping the patient perception of thyroidectomy and may represent an opportunity to provide education. Level of evidence: Level 4.

Bridging the Gap Between Urological Research and Patient Understanding: The Role of Large Language Models in Automated Generation of Layperson's Summaries.

INTRODUCTION: This study assessed ChatGPT's ability to generate readable, accurate, and clear layperson summaries of urological studies, and compared the performance of ChatGPT-generated summaries with original abstracts and author-written patient summaries to determine its effectiveness as a potential solution for creating accessible medical literature for the public. METHODS: Articles from the top 5 ranked urology journals were selected. A ChatGPT prompt was developed following guidelines to maximize readability, accuracy, and clarity, minimizing variability. Readability scores and grade-level indicators were calculated for the ChatGPT summaries, original abstracts, and patient summaries. Two MD physicians independently rated the accuracy and clarity of the ChatGPT-generated layperson summaries. Statistical analyses were conducted to compare readability scores. Cohen's κ coefficient was used to assess interrater reliability for correctness and clarity evaluations. RESULTS: A total of 256 journal articles were included. The ChatGPT-generated summaries were created with an average time of 17.5 (SD 15.0) seconds. The readability scores of the ChatGPT-generated summaries were significantly better than the original abstracts, with Global Readability Score 54.8 (12.3) vs 29.8 (18.5), Flesch Kincade Reading Ease 54.8 (12.3) vs 29.8 (18.5), Flesch Kincaid Grade Level 10.4 (2.2) vs 13.5 (4.0), Gunning Fog Score 12.9 (2.6) vs 16.6 (4.1), Smog Index 9.1 (2.0) vs 12.0 (3.0), Coleman Liau Index 12.9 (2.1) vs 14.9 (3.7), and Automated Readability Index 11.1 (2.5) vs 12.0 (5.7; P < .0001 for all except Automated Readability Index, which was P = .037). The correctness rate of ChatGPT outputs was >85% across all categories assessed, with interrater agreement (Cohen's κ) between 2 independent physician reviewers ranging from 0.76-0.95. CONCLUSIONS: ChatGPT can create accurate summaries of scientific abstracts for patients, with well-crafted prompts enhancing user-friendliness. Although the summaries are satisfactory, expert verification is necessary for improved accuracy.

Pan-cancer transcriptional signatures predictive of oncogenic mutations reveal that Fbw7 regulates cancer cell oxidative metabolism.

The Fbw7 (F-box/WD repeat-containing protein 7) ubiquitin ligase targets multiple oncoproteins for degradation and is commonly mutated in cancers. Like other pleiotropic tumor suppressors, Fbw7's complex biology has impeded our understanding of how Fbw7 mutations promote tumorigenesis and hindered the development of targeted therapies. To address these needs, we employed a transfer learning approach to derive gene-expression signatures from The Cancer Gene Atlas datasets that predict Fbw7 mutational status across tumor types and identified the pathways enriched within these signatures. Genes involved in mitochondrial function were highly enriched in pan-cancer signatures that predict Fbw7 mutations. Studies in isogenic colorectal cancer cell lines that differed in Fbw7 mutational status confirmed that Fbw7 mutations increase mitochondrial gene expression. Surprisingly, Fbw7 mutations shifted cellular metabolism toward oxidative phosphorylation and caused context-specific metabolic vulnerabilities. Our approach revealed unexpected metabolic reprogramming and possible therapeutic targets in Fbw7-mutant cancers and provides a framework to study other complex, oncogenic mutations.

The PP2A-B56 Phosphatase Opposes Cyclin E Autocatalytic Degradation via Site-Specific Dephosphorylation.

Cyclin E, in conjunction with its catalytic partner cyclin-dependent kinase 2 (CDK2), regulates cell cycle progression as cells exit quiescence and enter S-phase. Multiple mechanisms control cyclin E periodicity during the cell cycle, including phosphorylation-dependent cyclin E ubiquitylation by the SCFFbw7 ubiquitin ligase. Serine 384 (S384) is the critical cyclin E phosphorylation site that stimulates Fbw7 binding and cyclin E ubiquitylation and degradation. Because S384 is autophosphorylated by bound CDK2, this presents a paradox as to how cyclin E can evade autocatalytically induced degradation in order to phosphorylate its other substrates. We found that S384 phosphorylation is dynamically regulated in cells and that cyclin E is specifically dephosphorylated at S384 by the PP2A-B56 phosphatase, thereby uncoupling cyclin E degradation from cyclin E-CDK2 activity. Furthermore, the rate of S384 dephosphorylation is high in interphase but low in mitosis. This provides a mechanism whereby interphase cells can oppose autocatalytic cyclin E degradation and maintain cyclin E-CDK2 activity while also enabling cyclin E destruction in mitosis, when inappropriate cyclin E expression is genotoxic.

Genome-wide CRISPR-Cas9 Screens Reveal Loss of Redundancy between PKMYT1 and WEE1 in Glioblastoma Stem-like Cells.

To identify therapeutic targets for glioblastoma (GBM), we performed genome-wide CRISPR-Cas9 knockout (KO) screens in patient-derived GBM stem-like cells (GSCs) and human neural stem/progenitors (NSCs), non-neoplastic stem cell controls, for genes required for their in vitro growth. Surprisingly, the vast majority GSC-lethal hits were found outside of molecular networks commonly altered in GBM and GSCs (e.g., oncogenic drivers). In vitro and in vivo validation of GSC-specific targets revealed several strong hits, including the wee1-like kinase, PKMYT1/Myt1. Mechanistic studies demonstrated that PKMYT1 acts redundantly with WEE1 to inhibit cyclin B-CDK1 activity via CDK1-Y15 phosphorylation and to promote timely completion of mitosis in NSCs. However, in GSCs, this redundancy is lost, most likely as a result of oncogenic signaling, causing GBM-specific lethality.

Tumor suppression by the Fbw7 ubiquitin ligase: mechanisms and opportunities.

Tumor suppressors with widespread impact on carcinogenesis control broad spectra of oncogenic pathways. Protein degradation is an emerging mechanism by which tumor suppressors regulate a diversity of pathways and is exemplified by the SCF(Fbw7) ubiquitin ligase. Rapidly accumulating data indicate that SCF(Fbw7) regulates a network of crucial oncoproteins. Importantly, the FBXW7 gene, which encodes Fbw7, is one of the most frequently mutated genes in human cancers. These studies are yielding important new insights into tumorigenesis and may soon enable therapies targeting the Fbw7 pathway. Here, we focus on the mechanisms and consequences of Fbw7 deregulation in cancers and discuss possible therapeutic approaches.

Identification of PTPsigma as an autophagic phosphatase.

Macroautophagy is a dynamic process whereby portions of the cytosol are encapsulated in double-membrane vesicles and delivered to the lysosome for degradation. Phosphatidylinositol-3-phosphate (PtdIns3P) is concentrated on autophagic vesicles and recruits effector proteins that are crucial for this process. The production of PtdIns3P by the class III phosphatidylinositol 3-kinase Vps34, has been well established; however, protein phosphatases that antagonize this early step in autophagy remain to be identified. To identify such enzymes, we screened human phosphatase genes by RNA interference and found that loss of PTPσ, a dual-domain protein tyrosine phosphatase (PTP), increases levels of cellular PtdIns3P. The abundant PtdIns3P-positive vesicles conferred by loss of PTPσ strikingly phenocopied those observed in cells starved of amino acids. Accordingly, we discovered that loss of PTPσ hyperactivates both constitutive and induced autophagy. Finally, we found that PTPσ localizes to PtdIns3P-positive membranes in cells, and this vesicular localization is enhanced during autophagy. We therefore describe a novel role for PTPσ and provide insight into the regulation of autophagy. Mechanistic knowledge of this process is crucial for understanding and targeting therapies for several human diseases, including cancer and Alzheimer's disease, in which abnormal autophagy might be pathological.