Accuracy, readability, and understandability of large language models for prostate cancer information to the public.

BACKGROUND: Generative Pretrained Model (GPT) chatbots have gained popularity since the public release of ChatGPT. Studies have evaluated the ability of different GPT models to provide information about medical conditions. To date, no study has assessed the quality of ChatGPT outputs to prostate cancer related questions from both the physician and public perspective while optimizing outputs for patient consumption. METHODS: Nine prostate cancer-related questions, identified through Google Trends (Global), were categorized into diagnosis, treatment, and postoperative follow-up. These questions were processed using ChatGPT 3.5, and the responses were recorded. Subsequently, these responses were re-inputted into ChatGPT to create simplified summaries understandable at a sixth-grade level. Readability of both the original ChatGPT responses and the layperson summaries was evaluated using validated readability tools. A survey was conducted among urology providers (urologists and urologists in training) to rate the original ChatGPT responses for accuracy, completeness, and clarity using a 5-point Likert scale. Furthermore, two independent reviewers evaluated the layperson summaries on correctness trifecta: accuracy, completeness, and decision-making sufficiency. Public assessment of the simplified summaries' clarity and understandability was carried out through Amazon Mechanical Turk (MTurk). Participants rated the clarity and demonstrated their understanding through a multiple-choice question. RESULTS: GPT-generated output was deemed correct by 71.7% to 94.3% of raters (36 urologists, 17 urology residents) across 9 scenarios. GPT-generated simplified layperson summaries of this output was rated as accurate in 8 of 9 (88.9%) scenarios and sufficient for a patient to make a decision in 8 of 9 (88.9%) scenarios. Mean readability of layperson summaries was higher than original GPT outputs ([original ChatGPT v. simplified ChatGPT, mean (SD), p-value] Flesch Reading Ease: 36.5(9.1) v. 70.2(11.2), <0.0001; Gunning Fog: 15.8(1.7) v. 9.5(2.0), p < 0.0001; Flesch Grade Level: 12.8(1.2) v. 7.4(1.7), p < 0.0001; Coleman Liau: 13.7(2.1) v. 8.6(2.4), 0.0002; Smog index: 11.8(1.2) v. 6.7(1.8), <0.0001; Automated Readability Index: 13.1(1.4) v. 7.5(2.1), p < 0.0001). MTurk workers (n = 514) rated the layperson summaries as correct (89.5-95.7%) and correctly understood the content (63.0-87.4%). CONCLUSION: GPT shows promise for correct patient education for prostate cancer-related contents, but the technology is not designed for delivering patients information. Prompting the model to respond with accuracy, completeness, clarity and readability may enhance its utility when used for GPT-powered medical chatbots.

Study protocol for the Intraoperative Complications Assessment and Reporting with Universal Standards (ICARUS) global cross-specialty surveys and consensus.

Annually, about 300 million surgeries lead to significant intraoperative adverse events (iAEs), impacting patients and surgeons. Their full extent is underestimated due to flawed assessment and reporting methods. Inconsistent adoption of new grading systems and a lack of standardization, along with litigation concerns, contribute to underreporting. Only half of relevant journals provide guidelines on reporting these events, with a lack of standards in surgical literature. To address these issues, the Intraoperative Complications Assessment and Reporting with Universal Standard (ICARUS) Global Surgical Collaboration was established in 2022. The initiative involves conducting global surveys and a Delphi consensus to understand the barriers for poor reporting of iAEs, validate shared criteria for reporting, define iAEs according to surgical procedures, evaluate the existing grading systems' reliability, and identify strategies for enhancing the collection, reporting, and management of iAEs. Invitation to participate are extended to all the surgical specialties, interventional cardiology, interventional radiology, OR Staffs and anesthesiology. This effort represents an essential step towards improved patient safety and the well-being of healthcare professionals in the surgical field.

Bridging the Gap Between Urological Research and Patient Understanding: The Role of Large Language Models in Automated Generation of Layperson's Summaries.

INTRODUCTION: This study assessed ChatGPT's ability to generate readable, accurate, and clear layperson summaries of urological studies, and compared the performance of ChatGPT-generated summaries with original abstracts and author-written patient summaries to determine its effectiveness as a potential solution for creating accessible medical literature for the public. METHODS: Articles from the top 5 ranked urology journals were selected. A ChatGPT prompt was developed following guidelines to maximize readability, accuracy, and clarity, minimizing variability. Readability scores and grade-level indicators were calculated for the ChatGPT summaries, original abstracts, and patient summaries. Two MD physicians independently rated the accuracy and clarity of the ChatGPT-generated layperson summaries. Statistical analyses were conducted to compare readability scores. Cohen's κ coefficient was used to assess interrater reliability for correctness and clarity evaluations. RESULTS: A total of 256 journal articles were included. The ChatGPT-generated summaries were created with an average time of 17.5 (SD 15.0) seconds. The readability scores of the ChatGPT-generated summaries were significantly better than the original abstracts, with Global Readability Score 54.8 (12.3) vs 29.8 (18.5), Flesch Kincade Reading Ease 54.8 (12.3) vs 29.8 (18.5), Flesch Kincaid Grade Level 10.4 (2.2) vs 13.5 (4.0), Gunning Fog Score 12.9 (2.6) vs 16.6 (4.1), Smog Index 9.1 (2.0) vs 12.0 (3.0), Coleman Liau Index 12.9 (2.1) vs 14.9 (3.7), and Automated Readability Index 11.1 (2.5) vs 12.0 (5.7; P < .0001 for all except Automated Readability Index, which was P = .037). The correctness rate of ChatGPT outputs was >85% across all categories assessed, with interrater agreement (Cohen's κ) between 2 independent physician reviewers ranging from 0.76-0.95. CONCLUSIONS: ChatGPT can create accurate summaries of scientific abstracts for patients, with well-crafted prompts enhancing user-friendliness. Although the summaries are satisfactory, expert verification is necessary for improved accuracy.

Multiparametric ultrasound of prostate: role in prostate cancer diagnosis.

Recent advances in ultrasonography (US) technology established modalities, such as Doppler-US, HistoScanning, contrast-enhanced ultrasonography (CEUS), elastography, and micro-ultrasound. The early results of these US modalities have been promising, although there are limitations including the need for specialized equipment, inconsistent results, lack of standardizations, and external validation. In this review, we identified studies evaluating multiparametric ultrasonography (mpUS), the combination of multiple US modalities, for prostate cancer (PCa) diagnosis. In the past 5 years, a growing number of studies have shown that use of mpUS resulted in high PCa and clinically significant prostate cancer (CSPCa) detection performance using radical prostatectomy histology as the reference standard. Recent studies have demonstrated the role mpUS in improving detection of CSPCa and guidance for prostate biopsy and therapy. Furthermore, some aspects including lower costs, real-time imaging, applicability for some patients who have contraindication for magnetic resonance imaging (MRI) and availability in the office setting are clear advantages of mpUS. Interobserver agreement of mpUS was overall low; however, this limitation can be improved using standardized and objective evaluation systems such as the machine learning model. Whether mpUS outperforms MRI is unclear. Multicenter randomized controlled trials directly comparing mpUS and multiparametric MRI are warranted.