Performance of ChatGPT-3.5 and ChatGPT-4 on the European Board of Urology (EBU) exams: a comparative analysis.

BACKGROUND AND OBJECTIVE: In the transformative era of artificial intelligence, its integration into various spheres, especially healthcare, has been promising. The objective of this study was to analyze the performance of ChatGPT, as open-source Large Language Model (LLM), in its different versions on the recent European Board of Urology (EBU) in-service assessment questions. DESIGN AND SETTING: We asked multiple choice questions of the official EBU test books to ChatGPT-3.5 and ChatGPT-4 for the following exams: exam 1 (2017-2018), exam 2 (2019-2020) and exam 3 (2021-2022). Exams were passed with ≥60% correct answers. RESULTS: ChatGPT-4 provided significantly more correct answers in all exams compared to the prior version 3.5 (exam 1: ChatGPT-3.5 64.3% vs. ChatGPT-4 81.6%; exam 2: 64.5% vs. 80.5%; exam 3: 56% vs. 77%, p < 0.001, respectively). Test exam 3 was the only exam ChatGPT-3.5 did not pass. Within the different subtopics, there were no significant differences of provided correct answers by ChatGPT-3.5. Concerning ChatGPT-4, the percentage in test exam 3 was significantly decreased in the subtopics Incontinence (exam 1: 81.6% vs. exam 3: 53.6%; p = 0.026) and Transplantation (exam 1: 77.8% vs. exam 3: 0%; p = 0.020). CONCLUSION: Our findings indicate that ChatGPT, especially ChatGPT-4, has the general ability to answer complex medical questions and might pass FEBU exams. Nevertheless, there is still the indispensable need for human validation of LLM answers, especially concerning health care issues.

German specialists treating testicular cancer follow different guidelines with resulting inconsistency in assessment of retroperitoneal lymph-node metastasis: clinical implications and possible corrective measures.

BACKGROUND: Testicular germ cell tumors (GCTs) are aggressive but highly curable tumors. To avoid over/undertreatment, reliable clinical staging of retroperitoneal lymph-node metastasis is necessary. Current clinical guidelines, in their different versions, lack specific recommendations on how to measure lymph-node metastasis. OBJECTIVE: We aimed to assess the practice patterns of German institutions frequently treating testicular cancer for measuring retroperitoneal lymph-node size. METHODS: An 8-item survey was distributed among German university hospitals and members of the German Testicular Cancer Study Group. RESULTS: In the group of urologists, 54.7% assessed retroperitoneal lymph nodes depending on their short-axis diameter (SAD) (33.3% in any plane, 21.4% in the axial plane), while 45.3% used long-axis diameter (LAD) for the assessment (42.9% in any plane, 2.4% in the axial plane). Moreover, the oncologists mainly assessed lymph-node size based on the SAD (71.4%). Specifically, 42.9% of oncologists assessed the SAD in any plane, while 28.5% measured this dimension in the axial plane. Only 28.6% of oncologists considered the LAD (14.3% in any plane, 14.3% in the axial plane). None of the oncologists and 11.9% of the urologists (n = 5) always performed an MRI for the initial assessment, while for follow-up imaging, the use increased to 36.5% of oncologists and 31% of urologists. Furthermore, only 17% of the urologists, and no oncologists, calculated lymph-node volume in their assessment (p = 0.224). CONCLUSION: Clear and consistent measurement instructions are urgently needed to be present in all guidelines across different specialistic fields involved in testicular cancer management.