ABSTRACT
PURPOSE: The diagnostic performance of large language artificial intelligence (AI) models when utilizing radiological images has yet to be investigated. We employed Claude 3 Opus (released on March 4, 2024) and Claude 3.5 Sonnet (released on June 21, 2024) to investigate their diagnostic performances in response to the Radiology's Diagnosis Please quiz questions. MATERIALS AND METHODS: In this study, the AI models were tasked with listing the primary diagnosis and two differential diagnoses for 322 quiz questions from Radiology's "Diagnosis Please" cases, which included cases 1 to 322, published from 1998 to 2023. The analyses were performed under the following conditions: (1) Condition 1: submitter-provided clinical history (text) alone. (2) Condition 2: submitter-provided clinical history and imaging findings (text). (3) Condition 3: clinical history (text) and key images (PNG file). We applied McNemar's test to evaluate differences in the correct response rates for the overall accuracy under Conditions 1, 2, and 3 for each model and between the models. RESULTS: The correct diagnosis rates were 58/322 (18.0%) and 69/322 (21.4%), 201/322 (62.4%) and 209/322 (64.9%), and 80/322 (24.8%) and 97/322 (30.1%) for Conditions 1, 2, and 3 for Claude 3 Opus and Claude 3.5 Sonnet, respectively. The models provided the correct answer as a differential diagnosis in up to 26/322 (8.1%) for Opus and 23/322 (7.1%) for Sonnet. Statistically significant differences were observed in the correct response rates among all combinations of Conditions 1, 2, and 3 for each model (p < 0.01). Claude 3.5 Sonnet outperformed in all conditions, but a statistically significant difference was observed only in the comparison for Condition 3 (30.1% vs. 24.8%, p = 0.028). CONCLUSION: Two AI models demonstrated a significantly improved diagnostic performance when inputting both key images and clinical history. The models' ability to identify important differential diagnoses under these conditions was also confirmed.
ABSTRACT
This study aimed to compare the image quality and detection performance of pancreatic cystic lesions between computed tomography (CT) images reconstructed by deep learning reconstruction (DLR) and filtered back projection (FBP). This retrospective study included 54 patients (mean age: 67.7 ± 13.1) who underwent contrast-enhanced CT from May 2023 to August 2023. Among eligible patients, 30 and 24 were positive and negative for pancreatic cystic lesions, respectively. DLR and FBP were used to reconstruct portal venous phase images. Objective image quality analyses calculated quantitative image noise, signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR) using regions of interest on the abdominal aorta, pancreatic lesion, and pancreatic parenchyma. Three blinded radiologists performed subjective image quality assessment and lesion detection tests. Lesion depiction, normal structure illustration, subjective image noise, and overall image quality were utilized as subjective image quality indicators. DLR significantly reduced quantitative image noise compared with FBP (p < 0.001). SNR and CNR were significantly improved in DLR compared with FBP (p < 0.001). Three radiologists rated significantly higher scores for DLR in all subjective image quality indicators (p ≤ 0.029). Performance of DLR and FBP were comparable in lesion detection, with no statistically significant differences in the area under the receiver operating characteristic curve, sensitivity, specificity and accuracy. DLR reduced image noise and improved image quality with a clearer depiction of pancreatic structures. These improvements may have a positive effect on evaluating pancreatic cystic lesions, which can contribute to appropriate management of these lesions.
ABSTRACT
PURPOSE: Large language models (LLMs) are rapidly advancing and demonstrating high performance in understanding textual information, suggesting potential applications in interpreting patient histories and documented imaging findings. As LLMs continue to improve, their diagnostic abilities are expected to be enhanced further. However, there is a lack of comprehensive comparisons between LLMs from different manufacturers. In this study, we aimed to test the diagnostic performance of the three latest major LLMs (GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro) using Radiology Diagnosis Please Cases, a monthly diagnostic quiz series for radiology experts. MATERIALS AND METHODS: Clinical history and imaging findings, provided textually by the case submitters, were extracted from 324 quiz questions originating from Radiology Diagnosis Please cases published between 1998 and 2023. The top three differential diagnoses were generated by GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro, using their respective application programming interfaces. A comparative analysis of diagnostic performance among these three LLMs was conducted using Cochrane's Q and post hoc McNemar's tests. RESULTS: The respective diagnostic accuracies of GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro for primary diagnosis were 41.0%, 54.0%, and 33.9%, which further improved to 49.4%, 62.0%, and 41.0%, when considering the accuracy of any of the top three differential diagnoses. Significant differences in the diagnostic performance were observed among all pairs of models. CONCLUSION: Claude 3 Opus outperformed GPT-4o and Gemini 1.5 Pro in solving radiology quiz cases. These models appear capable of assisting radiologists when supplied with accurate evaluations and worded descriptions of imaging findings.
ABSTRACT
Chronic lung allograft dysfunction (CLAD) continues to be the leading cause of death in the long term after lung transplantation (LTx). CLAD has the following two main subtypes: bronchiolitis obliterans syndrome (BOS) and restrictive allograft syndrome (RAS). BOS features obstructive lung dysfunction, while RAS features restrictive lung dysfunction. Overall, RAS has a worse prognosis. The pathophysiology of CLAD is not fully understood; however, pulmonary infections can trigger CLAD, including coronavirus disease 2019 (COVID-19) pneumonia. Here, we describe a case of a 55-year-old female who received LTx about seven years ago and developed RAS after COVID-19 pneumonia. RAS was ultimately diagnosed based on the clinical course and imaging findings. Steroid pulse therapy and empirical antimicrobial therapy were initiated, but respiratory failure progressed, and the patient died 139 days after COVID-19 diagnosis, and 83 days after dyspnea progression. Clinicians should be aware of unusual stair-step clinical courses and imaging features in a given setting of pulmonary infection including COVID-19 to suspect CLAD in lung transplant patients.