Exploring the Feasibility of GPT-4 as a Data Extraction Tool for Renal Surgery Operative Notes.

INTRODUCTION: GPT-4 is a large language model with potential for multiple applications in urology. Our study sought to evaluate GPT-4's performance in data extraction from renal surgery operative notes. METHODS: GPT-4 was queried to extract information on laterality, surgery, approach, estimated blood loss, and ischemia time from deidentified operative notes. Match rates were determined by the number of "matched" data points between GPT-4 and human-curated extraction. Accuracy rates were calculated after manually reviewing "not matched" data points. Cohen's kappa and the intraclass coefficient were used to evaluate interrater agreement/reliability. RESULTS: Our cohort consisted of 1498 renal surgeries from 2003 to 2023. Match rates were high for laterality (94.4%), surgery (92.5%), and approach (89.4%), but lower for estimated blood loss (77.1%) and ischemia time (25.6%). GPT-4 was more accurate for estimated blood loss (90.3% vs 85.5% human curated) and similarly accurate for laterality (95.2% vs 95.3% human curated). Human-curated accuracy rates were higher for surgery (99.3% vs 93% GPT-4), approach (97.9% vs 90.8% GPT-4), and ischemia time (95.6% vs 30.7% GPT-4). Cohen's kappa was 0.96 for laterality, 0.83 for approach, and 0.71 for surgery. The intraclass coefficient was 0.62 for estimated blood loss and 0.09 for ischemia time. CONCLUSIONS: Match and accuracy rates were higher for categorical variables. GPT-4 data extraction was particularly error prone for variables with heterogenous documentation styles. The role of a standard operative template to aid data extraction will be explored in the future. GPT-4 can be utilized as a helpful and efficient data extraction tool with manual feedback.

Investigating the clinical reasoning abilities of large language model GPT-4: an analysis of postoperative complications from renal surgeries.

PURPOSE: Large language models, a subset of artificial intelligence, have immense potential to support human tasks. The role of these models in science and medicine is unclear, requiring strong critical thinking and analysis skills. The objective of our study was to evaluate GPT-4's abilities to assess postoperative complications after renal surgeries. MATERIALS AND METHODS: Discharge summaries were compiled, and patient information was deidentified in a Python-based program. Prompts were engineered in GPT-4 to assess for the presence of postoperative complications. GPT-4 was further asked to interpret each complication's Clavien-Dindo classification and institutional-specific category. GPT-4's database was compared to a human-curated database. Discrepancies were manually reviewed to calculate match and accuracy rates. RESULTS: Approximately 944 renal surgeries were conducted from August 2005 to March 2022. There was a 79.6% match rate between GPT-4 and human-curated data in detecting postoperative complications. Accuracy rates were 86.7% for GPT-4 and 92.9% for human-curated. A subgroup of 139 patients had a complication detected by both GPT-4 and human with available Clavien-Dindo classification and category information. There was a 37.4% overall match rate for Clavien-Dindo grade and 55.4% match rate for category. CONCLUSIONS: GPT-4 was able to accurately detect if there were any postoperative complications. It struggled with the complex task of further analyzing complications, especially with Clavien-Dindo classification, which requires more critical thinking and interpretation. While GPT-4 is not yet ready for advanced postoperative complication analysis, it can still be used to support clinicians in this endeavor.

Evaluating Diagnostic Accuracy and Inter-reader Agreement of the Prostate Imaging After Focal Ablation Scoring System.

Background and objective: Focal therapy (FT) is increasingly recognized as a promising approach for managing localized prostate cancer (PCa), notably reducing treatment-related morbidities. However, post-treatment anatomical changes present significant challenges for surveillance using current imaging techniques. This study aimed to evaluate the inter-reader agreement and efficacy of the Prostate Imaging after Focal Ablation (PI-FAB) scoring system in detecting clinically significant prostate cancer (csPCa) on post-FT multiparametric magnetic resonance imaging (mpMRI). Methods: A retrospective cohort study was conducted involving patients who underwent primary FT for localized csPCa between 2013 and 2023, followed by post-FT mpMRI and a prostate biopsy. Two expert genitourinary radiologists retrospectively evaluated post-FT mpMRI using PI-FAB. The key measures included inter-reader agreement of PI-FAB scores, assessed by quadratic weighted Cohen's kappa (κ), and the system's efficacy in predicting in-field recurrence of csPCa, with a PI-FAB score cutoff of 3. Additional diagnostic metrics including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and overall accuracy were also evaluated. Key findings and limitations: Scans from 38 patients were analyzed, revealing a moderate level of agreement in PI-FAB scoring (κ = 0.56). Both radiologists achieved sensitivity of 93% in detecting csPCa, although specificity, PPVs, NPVs, and accuracy varied. Conclusions and clinical implications: The PI-FAB scoring system exhibited high sensitivity with moderate inter-reader agreement in detecting in-field recurrence of csPCa. Despite promising results, its low specificity and PPV necessitate further refinement. These findings underscore the need for larger studies to validate the clinical utility of PI-FAB, potentially aiding in standardizing post-treatment surveillance. Patient summary: Focal therapy has emerged as a promising approach for managing localized prostate cancer, but limitations in current imaging techniques present significant challenges for post-treatment surveillance. The Prostate Imaging after Focal Ablation (PI-FAB) scoring system showed high sensitivity for detecting in-field recurrence of clinically significant prostate cancer. However, its low specificity and positive predictive value necessitate further refinement. Larger, more comprehensive studies are needed to fully validate its clinical utility.

Evaluating the performance of Generative Pre-trained Transformer-4 (GPT-4) in standardizing radiology reports.

OBJECTIVE: Radiology reporting is an essential component of clinical diagnosis and decision-making. With the advent of advanced artificial intelligence (AI) models like GPT-4 (Generative Pre-trained Transformer 4), there is growing interest in evaluating their potential for optimizing or generating radiology reports. This study aimed to compare the quality and content of radiologist-generated and GPT-4 AI-generated radiology reports. METHODS: A comparative study design was employed in the study, where a total of 100 anonymized radiology reports were randomly selected and analyzed. Each report was processed by GPT-4, resulting in the generation of a corresponding AI-generated report. Quantitative and qualitative analysis techniques were utilized to assess similarities and differences between the two sets of reports. RESULTS: The AI-generated reports showed comparable quality to radiologist-generated reports in most categories. Significant differences were observed in clarity (p = 0.027), ease of understanding (p = 0.023), and structure (p = 0.050), favoring the AI-generated reports. AI-generated reports were more concise, with 34.53 fewer words and 174.22 fewer characters on average, but had greater variability in sentence length. Content similarity was high, with an average Cosine Similarity of 0.85, Sequence Matcher Similarity of 0.52, BLEU Score of 0.5008, and BERTScore F1 of 0.8775. CONCLUSION: The results of this proof-of-concept study suggest that GPT-4 can be a reliable tool for generating standardized radiology reports, offering potential benefits such as improved efficiency, better communication, and simplified data extraction and analysis. However, limitations and ethical implications must be addressed to ensure the safe and effective implementation of this technology in clinical practice. CLINICAL RELEVANCE STATEMENT: The findings of this study suggest that GPT-4 (Generative Pre-trained Transformer 4), an advanced AI model, has the potential to significantly contribute to the standardization and optimization of radiology reporting, offering improved efficiency and communication in clinical practice. KEY POINTS: • Large language model-generated radiology reports exhibited high content similarity and moderate structural resemblance to radiologist-generated reports. • Performance metrics highlighted the strong matching of word selection and order, as well as high semantic similarity between AI and radiologist-generated reports. • Large language model demonstrated potential for generating standardized radiology reports, improving efficiency and communication in clinical settings.

Initial experience and cancer detection rates of office-based transperineal magnetic resonance imaging-ultrasound fusion prostate biopsy under local anesthesia.

INTRODUCTION: We aimed to demonstrate feasibility and cancer detection rates of office-based ultrasound-guided transperineal magnetic resonance imaging-ultrasound (MRI-US) fusion (TFB) prostate biopsy under local anesthesia. METHODS: With institutional review board approval, records of men undergoing TFB in the office setting under local anesthesia were reviewed. Baseline patient characteristics, MRI findings, cancer detection rates, and complications were recorded. The PrecisionPoint Transperineal Access System (Perineologic, Cumberland, MD, U.S.), along with UroNav 3.0 image-fusion system (Invivo International, Best, The Netherlands) were used for all procedures. Following biopsy, men were surveyed to assess patient experience. RESULTS: Between January 2019 and February 2020, 200 TFBs were performed, of which 141 (71%) were positive for prostate cancer, with 117 (83%) Gleason grade group 2 or higher. A total of 259 of 265 MRI lesions were biopsied, with 127 (49%) positive overall. Prostate Imaging-Reporting and Data System (PI-RADS) 4-5 lesions were positive for prostate cancer in 59% of cases. The mean procedural time was 20 minutes, with a patient enter-to-exit room time of 54 minutes. There were no septic complications, no patients required post-procedure hospital admission, and all procedures were successfully completed. Seventy-five percent of patients surveyed reported complete resolution of pain at three days following the procedure. CONCLUSIONS: Office-based TFB represents a viable approach to prostate cancer detection following prostate MRI. Larger-scale assessment is needed to categorize cancer detection rates more accurately by PI-RADs subset, patient selection factors, complication rate, and cost relative to TFB under anesthesia.

Feasibility and Utility of Mobile Applications for the Evaluation of Urology Residents' Surgical Competence.

OBJECTIVE: To provide real-time assessment and feedback on the competency of urology residents' surgical skill via mobile applications and examine their feasibility and utility. MATERIALS AND METHODS: Two mobile application-based systems (SIMPL and myTIPreport) were sequentially implemented for the case-by-case assessment of residents' performance of surgical skills at a single institution. Data was collected regarding residents' perception of their feedback pre- and post-implementation of the applications. Faculty were surveyed after their implementation to determine their feasibility and utility. RESULTS: 297 individual evaluations were completed with SIMPL and 822 with myTIPreport over four and eleven months respectively. Post-implementation, residents showed significantly improved perceptions regarding the quantity and personalization of surgical skill feedback (P = .043 and .005 respectively). A majority (75%) of the faculty found the mobile applications feasible to use, an improvement compared to prior methods of resident evaluation, and would recommend continued use. CONCLUSION: This study represents the first documented use of real-time surgical competency assessment in urology. The use of mobile applications to evaluate urology residents' surgical competency in clinical practice is both feasible and useful. Their use may allow for more individualized surgical skill teaching during training and for the verification of the surgical skills necessary to practice autonomously.

Management and treatment options when facing malignant ureteral obstruction.

Malignant ureteral obstruction is an unfortunate finding that can be caused by a wide-ranging number of malignancies with a prognosis of limited survival. Given its presentation and progression, it can be refractory to treatment by traditional single polymeric ureteral stents. With a higher failure rate than causes of benign ureteral obstruction, a number of other options are available for initial management, as well as in cases of first-line therapy failure, including tandem stents, metallic stents, percutaneous nephrostomies and extra-anatomic stents. We reviewed the literature and carried out a PubMed search including the following keywords and phrases: "malignant ureteral obstruction," "tandem ureteral stents," "metallic ureteral stents," "resonance stent," "metal mesh ureteral stents" and "extra-anatomic stents." The vast majority of studies were small and retrospective, with a large number of studies related to metallic stents. Given the heterogenous patient population and diversity of practice, it is difficult to truly assess the efficacy of each method. As there are no guidelines or major head-to-head prospective trials involving these techniques, it makes practicing up to the specific provider. However, this article attempts to provide a framework with which the urologist who is presented with malignant ureteral obstruction can plan in order to provide the individualized care on a case-by-case basis. What is clear is that prospective, randomized clinical trials are necessary to help bring evidence-based medicine and guidelines for patients with malignant ureteral obstruction.

Triple Antiplatelet Therapy and Combinations with Oral Anticoagulants After Stent Implantation.

Triple oral anticoagulation or triple antiplatelet therapies may be administered for various reasons. They reduce cardiac complications following percutaneous coronary intervention and stroke or other thromboembolic phenomenon in conditions such as atrial fibrillation. There is an elevated risk of severe bleeding, so it is necessary to balance risk and benefits. Newer oral anticoagulants and antiplatelet drugs may be considered; the number of options is increasing. This article examines triple therapies and the efficacy and safety of combinations of traditional anticoagulant and antiplatelet drugs, and reviews clinical trial data on novel agents. Guidelines to inform clinical decision-making are presented.