Accuracy, readability, and understandability of large language models for prostate cancer information to the public.

BACKGROUND: Generative Pretrained Model (GPT) chatbots have gained popularity since the public release of ChatGPT. Studies have evaluated the ability of different GPT models to provide information about medical conditions. To date, no study has assessed the quality of ChatGPT outputs to prostate cancer related questions from both the physician and public perspective while optimizing outputs for patient consumption. METHODS: Nine prostate cancer-related questions, identified through Google Trends (Global), were categorized into diagnosis, treatment, and postoperative follow-up. These questions were processed using ChatGPT 3.5, and the responses were recorded. Subsequently, these responses were re-inputted into ChatGPT to create simplified summaries understandable at a sixth-grade level. Readability of both the original ChatGPT responses and the layperson summaries was evaluated using validated readability tools. A survey was conducted among urology providers (urologists and urologists in training) to rate the original ChatGPT responses for accuracy, completeness, and clarity using a 5-point Likert scale. Furthermore, two independent reviewers evaluated the layperson summaries on correctness trifecta: accuracy, completeness, and decision-making sufficiency. Public assessment of the simplified summaries' clarity and understandability was carried out through Amazon Mechanical Turk (MTurk). Participants rated the clarity and demonstrated their understanding through a multiple-choice question. RESULTS: GPT-generated output was deemed correct by 71.7% to 94.3% of raters (36 urologists, 17 urology residents) across 9 scenarios. GPT-generated simplified layperson summaries of this output was rated as accurate in 8 of 9 (88.9%) scenarios and sufficient for a patient to make a decision in 8 of 9 (88.9%) scenarios. Mean readability of layperson summaries was higher than original GPT outputs ([original ChatGPT v. simplified ChatGPT, mean (SD), p-value] Flesch Reading Ease: 36.5(9.1) v. 70.2(11.2), <0.0001; Gunning Fog: 15.8(1.7) v. 9.5(2.0), p < 0.0001; Flesch Grade Level: 12.8(1.2) v. 7.4(1.7), p < 0.0001; Coleman Liau: 13.7(2.1) v. 8.6(2.4), 0.0002; Smog index: 11.8(1.2) v. 6.7(1.8), <0.0001; Automated Readability Index: 13.1(1.4) v. 7.5(2.1), p < 0.0001). MTurk workers (n = 514) rated the layperson summaries as correct (89.5-95.7%) and correctly understood the content (63.0-87.4%). CONCLUSION: GPT shows promise for correct patient education for prostate cancer-related contents, but the technology is not designed for delivering patients information. Prompting the model to respond with accuracy, completeness, clarity and readability may enhance its utility when used for GPT-powered medical chatbots.

Study protocol for the Intraoperative Complications Assessment and Reporting with Universal Standards (ICARUS) global cross-specialty surveys and consensus.

Annually, about 300 million surgeries lead to significant intraoperative adverse events (iAEs), impacting patients and surgeons. Their full extent is underestimated due to flawed assessment and reporting methods. Inconsistent adoption of new grading systems and a lack of standardization, along with litigation concerns, contribute to underreporting. Only half of relevant journals provide guidelines on reporting these events, with a lack of standards in surgical literature. To address these issues, the Intraoperative Complications Assessment and Reporting with Universal Standard (ICARUS) Global Surgical Collaboration was established in 2022. The initiative involves conducting global surveys and a Delphi consensus to understand the barriers for poor reporting of iAEs, validate shared criteria for reporting, define iAEs according to surgical procedures, evaluate the existing grading systems' reliability, and identify strategies for enhancing the collection, reporting, and management of iAEs. Invitation to participate are extended to all the surgical specialties, interventional cardiology, interventional radiology, OR Staffs and anesthesiology. This effort represents an essential step towards improved patient safety and the well-being of healthcare professionals in the surgical field.

Publishers' and journals' instructions to authors on use of generative artificial intelligence in academic and scientific publishing: bibliometric analysis.

OBJECTIVES: To determine the extent and content of academic publishers' and scientific journals' guidance for authors on the use of generative artificial intelligence (GAI). DESIGN: Cross sectional, bibliometric study. SETTING: Websites of academic publishers and scientific journals, screened on 19-20 May 2023, with the search updated on 8-9 October 2023. PARTICIPANTS: Top 100 largest academic publishers and top 100 highly ranked scientific journals, regardless of subject, language, or country of origin. Publishers were identified by the total number of journals in their portfolio, and journals were identified through the Scimago journal rank using the Hirsch index (H index) as an indicator of journal productivity and impact. MAIN OUTCOME MEASURES: The primary outcomes were the content of GAI guidelines listed on the websites of the top 100 academic publishers and scientific journals, and the consistency of guidance between the publishers and their affiliated journals. RESULTS: Among the top 100 largest publishers, 24% provided guidance on the use of GAI, of which 15 (63%) were among the top 25 publishers. Among the top 100 highly ranked journals, 87% provided guidance on GAI. Of the publishers and journals with guidelines, the inclusion of GAI as an author was prohibited in 96% and 98%, respectively. Only one journal (1%) explicitly prohibited the use of GAI in the generation of a manuscript, and two (8%) publishers and 19 (22%) journals indicated that their guidelines exclusively applied to the writing process. When disclosing the use of GAI, 75% of publishers and 43% of journals included specific disclosure criteria. Where to disclose the use of GAI varied, including in the methods or acknowledgments, in the cover letter, or in a new section. Variability was also found in how to access GAI guidelines shared between journals and publishers. GAI guidelines in 12 journals directly conflicted with those developed by the publishers. The guidelines developed by top medical journals were broadly similar to those of academic journals. CONCLUSIONS: Guidelines by some top publishers and journals on the use of GAI by authors are lacking. Among those that provided guidelines, the allowable uses of GAI and how it should be disclosed varied substantially, with this heterogeneity persisting in some instances among affiliated publishers and journals. Lack of standardization places a burden on authors and could limit the effectiveness of the regulations. As GAI continues to grow in popularity, standardized guidelines to protect the integrity of scientific output are needed.

Bridging the Gap Between Urological Research and Patient Understanding: The Role of Large Language Models in Automated Generation of Layperson's Summaries.

INTRODUCTION: This study assessed ChatGPT's ability to generate readable, accurate, and clear layperson summaries of urological studies, and compared the performance of ChatGPT-generated summaries with original abstracts and author-written patient summaries to determine its effectiveness as a potential solution for creating accessible medical literature for the public. METHODS: Articles from the top 5 ranked urology journals were selected. A ChatGPT prompt was developed following guidelines to maximize readability, accuracy, and clarity, minimizing variability. Readability scores and grade-level indicators were calculated for the ChatGPT summaries, original abstracts, and patient summaries. Two MD physicians independently rated the accuracy and clarity of the ChatGPT-generated layperson summaries. Statistical analyses were conducted to compare readability scores. Cohen's κ coefficient was used to assess interrater reliability for correctness and clarity evaluations. RESULTS: A total of 256 journal articles were included. The ChatGPT-generated summaries were created with an average time of 17.5 (SD 15.0) seconds. The readability scores of the ChatGPT-generated summaries were significantly better than the original abstracts, with Global Readability Score 54.8 (12.3) vs 29.8 (18.5), Flesch Kincade Reading Ease 54.8 (12.3) vs 29.8 (18.5), Flesch Kincaid Grade Level 10.4 (2.2) vs 13.5 (4.0), Gunning Fog Score 12.9 (2.6) vs 16.6 (4.1), Smog Index 9.1 (2.0) vs 12.0 (3.0), Coleman Liau Index 12.9 (2.1) vs 14.9 (3.7), and Automated Readability Index 11.1 (2.5) vs 12.0 (5.7; P < .0001 for all except Automated Readability Index, which was P = .037). The correctness rate of ChatGPT outputs was >85% across all categories assessed, with interrater agreement (Cohen's κ) between 2 independent physician reviewers ranging from 0.76-0.95. CONCLUSIONS: ChatGPT can create accurate summaries of scientific abstracts for patients, with well-crafted prompts enhancing user-friendliness. Although the summaries are satisfactory, expert verification is necessary for improved accuracy.

Clinical Patient Summaries Not Fit for Purpose: A Study in Urology.

We evaluated the comprehensibility of patient summaries provided by urology journals for the general public. The WebFX online tool was used to assess the readability of abstracts and patient summaries by scoring the text according to established readability indices. A total of 266 articles were included and statistical analysis was performed to compare the readability of abstracts and patient summaries, stratified by article type and text type. The results show that patient summaries consistently performed worse than abstracts for all readability metrics, and the readability levels for both abstracts and patient summaries were more advanced than the recommended guidelines on average. This study suggests that patient summaries provided by these urology journals may not be easily understood by the general population, and tools should be developed to help urological researchers improve the accessibility of their work. PATIENT SUMMARY: We checked how easy it is to read and understand patient summaries and abstracts of research articles from four urology journals. We found that the summaries and abstracts were too hard to read. This study shows that we need to make these summaries easier to read for everyone.

Automated Capture of Intraoperative Adverse Events Using Artificial Intelligence: A Systematic Review and Meta-Analysis.

Intraoperative adverse events (iAEs) impact the outcomes of surgery, and yet are not routinely collected, graded, and reported. Advancements in artificial intelligence (AI) have the potential to power real-time, automatic detection of these events and disrupt the landscape of surgical safety through the prediction and mitigation of iAEs. We sought to understand the current implementation of AI in this space. A literature review was performed to PRISMA-DTA standards. Included articles were from all surgical specialties and reported the automatic identification of iAEs in real-time. Details on surgical specialty, adverse events, technology used for detecting iAEs, AI algorithm/validation, and reference standards/conventional parameters were extracted. A meta-analysis of algorithms with available data was conducted using a hierarchical summary receiver operating characteristic curve (ROC). The QUADAS-2 tool was used to assess the article risk of bias and clinical applicability. A total of 2982 studies were identified by searching PubMed, Scopus, Web of Science, and IEEE Xplore, with 13 articles included for data extraction. The AI algorithms detected bleeding (n = 7), vessel injury (n = 1), perfusion deficiencies (n = 1), thermal damage (n = 1), and EMG abnormalities (n = 1), among other iAEs. Nine of the thirteen articles described at least one validation method for the detection system; five explained using cross-validation and seven divided the dataset into training and validation cohorts. Meta-analysis showed the algorithms were both sensitive and specific across included iAEs (detection OR 14.74, CI 4.7-46.2). There was heterogeneity in reported outcome statistics and article bias risk. There is a need for standardization of iAE definitions, detection, and reporting to enhance surgical care for all patients. The heterogeneous applications of AI in the literature highlights the pluripotent nature of this technology. Applications of these algorithms across a breadth of urologic procedures should be investigated to assess the generalizability of these data.

Assessment and Reporting of Perioperative Adverse Events and Complications in Patients Undergoing Inguinal Lymphadenectomy for Melanoma, Vulvar Cancer, and Penile Cancer: A Systematic Review and Meta-analysis.

BACKGROUND: Inguinal lymph node dissection (ILND) plays a crucial role in the oncological management of patients with melanoma, penile, and vulvar cancer. This study aims to systematically evaluate perioperative adverse events (AEs) in patients undergoing ILND and its reporting. METHODS: A systematic review was conducted according to PRISMA. PubMed, MEDLINE, Scopus, and Embase were queried to identify studies discussing perioperative AEs in patients with melanoma, penile, and vulvar cancer following ILND. RESULTS: Our search generated 3.469 publications, with 296 studies meeting the inclusion criteria. Details of 14.421 patients were analyzed. Of these studies, 58 (19.5%) described intraoperative AEs (iAEs) as an outcome of interest. Overall, 68 (2.9%) patients reported at least one iAE. Postoperative AEs were reported in 278 studies, combining data on 10.898 patients. Overall, 5.748 (52.7%) patients documented ≥1 postoperative AEs. The most reported ILND-related AEs were lymphatic AEs, with a total of 4.055 (38.8%) events. The pooled meta-analysis confirmed that high BMI (RR 1.09; p = 0.006), ≥1 comorbidities (RR 1.79; p = 0.01), and diabetes (RR 1.81; p = < 0.00001) are independent predictors for any AEs after ILND. When assessing the quality of the AEs reporting, we found 25% of studies reported at least 50% of the required criteria. CONCLUSION: ILND performed in melanoma, penile, and vulvar cancer patients is a morbid procedure. The quality of the AEs reporting is suboptimal. A more standardized AEs reporting system is needed to produce comparable data across studies for furthering the development of strategies to decrease AEs.

The Impact of Facility Surgical Caseload Volumes on Survival Outcomes in Patients Undergoing Radical Cystectomy.

The role of surgical experience and its impact on the survival requires further investigation. A cohort of patients undergoing radical cystectomy or anterior pelvic exenteration for localized bladder cancer between 2006 and 2013 at 1143 facilities across the United States was identified using the National Cancer Database and analyzed. Using overall survival (OS) as the primary outcome, the relationship between facility annual caseload (FAC) and facility annual surgical caseload (FASC) for those undergoing curative surgery was examined. Four volume groups (VG) depending on caseload using both FAC and FASC were defined. These included VG1: below 50th percentile, VG2: 50th−74th percentile, VG3: 75th−89th percentile, and VG4: 90th and above. Between 2006 and 2013, 27,272 patients underwent surgery for localized bladder cancer. The median OS was 59.66 months (95% CI: 57.79−61.77). OS improved significantly as caseload increased. The unadjusted median OS difference between VG1 and VG4 was 15.35 months (64.3 vs. 48.95 months, HR 1.19 95% CI: 1.13−1.25, p < 0.001) for FAC. This figure was 19.84 months (66.89 vs. 47.05 months, HR 1.25 95% CI: 1.18−1.32, p < 0.0001) for FASC. This analysis revealed a significant and clinically important survival advantage for curative bladder cancer surgery at highly experienced centers.

Superior Capsular Reconstruction of the Shoulder.

For irreparable rotator cuff tears, superior capsular reconstruction (SCR) has become an option for restoring glenohumeral joint stability and reversing proximal humeral migration. Signs of irreparable rotator cuff tears include pain from subacromial impingement, muscle weakness, and pseudoparalysis. In biomechanical studies, Mihata et al. showed SCR with fascia lata graft and side-to-side suturing to remaining infraspinatus tendon restored superior stability of the shoulder joint. Adding acromioplasty decreased the subacromial contact area without altering the humeral head position, superior translation, or subacromial peak contact pressure. The same research group showed that using an 8-mm thick fascia lata graft attached at 15° to 45° of shoulder abduction optimized superior stability of the shoulder joint. Adams et al. performed SCR using a dermal allograft and found that greater glenohumeral abduction angle (60°) decreased applied deltoid force. SCR can be performed with the patient in the lateral decubitus or beach chair position. Arthroscopic exploration, debridement, and infraspinatus and supraspinatus repair attempt is completed before proceeding with SCR. To restore the superior capsule of the shoulder, the graft (fascia lata autograft, dermal allograft) can be attached to the superior glenoid medially and the rotator cuff footprint on the greater tuberosity of the humerus laterally, after debriding bone to enhance healing. SCR with side-to-side suturing to the remnant rotator cuff yields promising clinical results. Using a fascia lata autograft, Mihata et al. showed a reversal of pseudoparalysis in 93% to 96% of patients and mean active elevation, external rotation, and acromiohumeral distance on radiography all improved. Using a dermal allograft and a unique graft delivery technique, Burkhart et al. reversed pseudoparalysis in 9 of 10 patients and 70% of patients had completely intact grafts. Recommendations for rehabilitation and return to activity vary, but adequate time for graft healing is recommended.