Comparing Diagnostic Accuracy of Radiologists versus GPT-4V and Gemini Pro Vision Using Image Inputs from Diagnosis Please Cases.

Background The diagnostic abilities of multimodal large language models (LLMs) using direct image inputs and the impact of the temperature parameter of LLMs remain unexplored. Purpose To investigate the ability of GPT-4V and Gemini Pro Vision in generating differential diagnoses at different temperatures compared with radiologists using Radiology Diagnosis Please cases. Materials and Methods This retrospective study included Diagnosis Please cases published from January 2008 to October 2023. Input images included original images and captures of the textual patient history and figure legends (without imaging findings) from PDF files of each case. The LLMs were tasked with providing three differential diagnoses, repeated five times at temperatures 0, 0.5, and 1. Eight subspecialty-trained radiologists solved cases. An experienced radiologist compared generated and final diagnoses, considering the result correct if the generated diagnoses included the final diagnosis after five repetitions. Accuracy was assessed across models, temperatures, and radiology subspecialties, with statistical significance set at P < .007 after Bonferroni correction for multiple comparisons across the LLMs at the three temperatures and with radiologists. Results A total of 190 cases were included in neuroradiology (n = 53), multisystem (n = 27), gastrointestinal (n = 25), genitourinary (n = 23), musculoskeletal (n = 17), chest (n = 16), cardiovascular (n = 12), pediatric (n = 12), and breast (n = 5) subspecialties. Overall accuracy improved with increasing temperature settings (0, 0.5, 1) for both GPT-4V (41% [78 of 190 cases], 45% [86 of 190 cases], 49% [93 of 190 cases], respectively) and Gemini Pro Vision (29% [55 of 190 cases], 36% [69 of 190 cases], 39% [74 of 190 cases], respectively), although there was no evidence of a statistically significant difference after Bonferroni adjustment (GPT-4V, P = .12; Gemini Pro Vision, P = .04). The overall accuracy of radiologists (61% [115 of 190 cases]) was higher than that of Gemini Pro Vision at temperature 1 (T1) (P < .001), while no statistically significant difference was observed between radiologists and GPT-4V at T1 after Bonferroni adjustment (P = .02). Radiologists (range, 45%-88%) outperformed the LLMs at T1 (range, 24%-75%) in most subspecialties. Conclusion Using direct radiologic image inputs, GPT-4V and Gemini Pro Vision showed improved diagnostic accuracy with increasing temperature settings. Although GPT-4V slightly underperformed compared with radiologists, it nonetheless demonstrated promising potential as a supportive tool in diagnostic decision-making. © RSNA, 2024 See also the editorial by Nishino and Ballard in this issue.

Mammographic density assessment: comparison of radiologists, automated volumetric measurement, and artificial intelligence-based computer-assisted diagnosis.

BACKGROUND: Artificial intelligence-based computer-assisted diagnosis (AI-CAD) is increasingly used for mammographic exams, and its role in mammographic density assessment should be evaluated. PURPOSE: To assess the inter-modality agreement between radiologists, automated volumetric density measurement program (Volpara), and AI-CAD system in breast density categorization using the Breast Imaging-Reporting and Data System (BI-RADS) density categories. MATERIAL AND METHODS: A retrospective review was conducted on 1015 screening digital mammograms that were performed in Asian female patients (mean age = 56 years ± 10 years) in our health examination center between December 2022 and January 2023. Four radiologists with two different levels of experience (expert and general radiologists) performed density assessments. Agreement between the radiologists, Volpara, and AI-CAD (Lunit INSIGHT MMG) was evaluated using weighted kappa statistics and matched rates. RESULTS: Inter-reader agreement between expert and general radiologists was substantial (k = 0.65) with a matched rate of 72.8%. The agreement was substantial between expert or general radiologists and Volpara (k = 0.64-0.67) with a matched rate of 72.0% but moderate between expert or general radiologists and AI-CAD (k = 0.45-0.58) with matched rates of 56.7%-67.0%. The agreement between Volpara and AI-CAD was moderate (k = 0.53) with a matched rate of 60.8%. CONCLUSION: The agreement in breast density categorization between radiologists and automated volumetric density measurement program (Volpara) was higher than the agreement between radiologists and AI-CAD (Lunit INSIGHT MMG).

AI-based improvement in lung cancer detection on chest radiographs: results of a multi-reader study in NLST dataset.

OBJECTIVE: Assess if deep learning-based artificial intelligence (AI) algorithm improves reader performance for lung cancer detection on chest X-rays (CXRs). METHODS: This reader study included 173 images from cancer-positive patients (n = 98) and 346 images from cancer-negative patients (n = 196) selected from National Lung Screening Trial (NLST). Eight readers, including three radiology residents, and five board-certified radiologists, participated in the observer performance test. AI algorithm provided image-level probability of pulmonary nodule or mass on CXRs and a heatmap of detected lesions. Reader performance was compared with AUC, sensitivity, specificity, false-positives per image (FPPI), and rates of chest CT recommendations. RESULTS: With AI, the average sensitivity of readers for the detection of visible lung cancer increased for residents, but was similar for radiologists compared to that without AI (0.61 [95% CI, 0.55-0.67] vs. 0.72 [95% CI, 0.66-0.77], p = 0.016 for residents, and 0.76 [95% CI, 0.72-0.81] vs. 0.76 [95% CI, 0.72-0.81, p = 1.00 for radiologists), while false-positive findings per image (FPPI) was similar for residents, but decreased for radiologists (0.15 [95% CI, 0.11-0.18] vs. 0.12 [95% CI, 0.09-0.16], p = 0.13 for residents, and 0.24 [95% CI, 0.20-0.29] vs. 0.17 [95% CI, 0.13-0.20], p < 0.001 for radiologists). With AI, the average rate of chest CT recommendation in patients positive for visible cancer increased for residents, but was similar for radiologists (54.7% [95% CI, 48.2-61.2%] vs. 70.2% [95% CI, 64.2-76.2%], p < 0.001 for residents and 72.5% [95% CI, 68.0-77.1%] vs. 73.9% [95% CI, 69.4-78.3%], p = 0.68 for radiologists), while that in cancer-negative patients was similar for residents, but decreased for radiologists (11.2% [95% CI, 9.6-13.1%] vs. 9.8% [95% CI, 8.0-11.6%], p = 0.32 for residents and 16.4% [95% CI, 14.7-18.2%] vs. 11.7% [95% CI, 10.2-13.3%], p < 0.001 for radiologists). CONCLUSIONS: AI algorithm can enhance the performance of readers for the detection of lung cancers on chest radiographs when used as second reader. KEY POINTS: • Reader study in the NLST dataset shows that AI algorithm had sensitivity benefit for residents and specificity benefit for radiologists for the detection of visible lung cancer. • With AI, radiology residents were able to recommend more chest CT examinations (54.7% vs 70.2%, p < 0.001) for patients with visible lung cancer. • With AI, radiologists recommended significantly less proportion of unnecessary chest CT examinations (16.4% vs. 11.7%, p < 0.001) in cancer-negative patients.

Comparison of variability in breast density assessment by BI-RADS category according to the level of experience.

Background Only few studies have assessed variability in the results obtained by the readers with different experience levels in comparison with automated volumetric breast density measurements. Purpose To examine the variations in breast density assessment according to BI-RADS categories among readers with different experience levels and to compare it with the results of automated quantitative measurements. Material and Methods Density assignment was done for 1000 screening mammograms by six readers with three different experience levels (breast-imaging experts, general radiologists, and students). Agreement level between the results obtained by the readers and the Volpara automated volumetric breast density measurements was assessed. The agreement analysis using two categories-non-dense and dense breast tissue-was also performed. Results Intra-reader agreement for experts, general radiologists, and students were almost perfect or substantial (k = 0.74-0.95). The agreement between visual assessments of the breast-imaging experts and volumetric assessments by Volpara was substantial (k = 0.77). The agreement was moderate between the experts and general radiologists (k = 0.67) and slight between the students and Volpara (k = 0.01). The agreement for the two category groups (nondense and dense) was almost perfect between the experts and Volpara (k = 0.83). The agreement was substantial between the experts and general radiologists (k = 0.78). Conclusion We observed similar high agreement levels between visual assessments of breast density performed by radiologists and the volumetric assessments. However, agreement levels were substantially lower for the untrained readers.

Predictive Clinicopathologic and Dynamic Contrast-Enhanced MRI Findings for Tumor Response to Neoadjuvant Chemotherapy in Triple-Negative Breast Cancer.

OBJECTIVE: The purpose of this study is to investigate whether clinicopathologic factors and dynamic contrast-enhanced MRI (DCE-MRI) features are associated with pathologic tumor response to neoadjuvant chemotherapy (NAC) in patients with triple-negative breast cancer (TNBC). MATERIALS AND METHODS: Seventy-three patients with TNBC who underwent pre-NAC MRI, completed NAC, and underwent surgery between January 2009 and December 2010 were included in the study. MRI features and clinicopathologic factors for predicting pathologic responses were analyzed, and residual tumor sizes, as measured using MRI and surgical specimens, were evaluated. RESULTS: Of 73 study patients, 20 (27%) had a pathologic complete response (pCR). Homogeneous enhancement on pre-NAC MRI (odds ratio from multivariate analysis, 14.66) and a concentric shrinkage pattern of tumor volume reduction on post-NAC MRI (odds ratio, 8.63) were independently associated with pCR. Residual tumor sizes, as measured using MRI and surgical specimens, showed a strong correlation (r = 0.652, p < 0.001). The correlation for residual tumor sizes was stronger for patients with pCR (r = 0.600, p < 0.001) and those with a concentric shrinkage pattern (r = 0.818, p < 0.001) than for patients with a response other than near pCR or pCR (i.e., the non-pCR group) (r = -0.128, p = 0.590) and patients with a dendritic shrinkage pattern of tumor volume reduction (r = 0.270, p = 0.182). CONCLUSION: Homogeneous enhancement of tumors on pre-NAC MRI and the presence of a concentric shrinkage pattern after NAC are associated with pCR in patients with TNBC. Residual tumor sizes on MRI and surgical specimens tended to show a stronger correlation in the pCR group or the concentric shrinkage group than in the non-pCR group or the dendritic shrinkage group.

Preoperative cardiac computed tomography for demonstration of congenital cardiac septal defect in adults.

OBJECTIVES: We aimed to evaluate the role of preoperative cardiac computed tomography (CT) for adults with congenital cardiac septal defect (CSD). METHODS: Sixty-five consecutive patients who underwent preoperative CT and surgery for CSD were included. The diagnostic accuracy of CT and the concordance rate of the subtype classification of CSD were evaluated using surgical findings as the reference standard. Sixty-five patients without CSD who underwent cardiac valve surgery were used as a control group. An incremental value of CT over echocardiography was described retrospectively. RESULTS: Sensitivity and specificity of CT for diagnosis of CSD were 95 % and 100 %, respectively. The concordance rate of subtype classification was 91 % in CT and 92 % in echocardiography. The maximum size of the defect measured by CT correlated well with surgical measurement (r = 0.82), and the limit of agreement was -0.9 ± 7.42 mm. In comparison with echocardiography, CT was able to detect combined abnormalities in three cases, and exclusively provided correct subtype classification or clarified suspected abnormal findings found on echocardiography in seven cases. CONCLUSIONS: Cardiac CT can accurately demonstrates CSD in preoperative adult patients. CT may have an incremental role in preoperative planning, particularly in those with more complex anatomy. KEY POINTS: • Cardiac CT can demonstrate cardiac septal defect accurately in preoperative planning. • Cardiac CT can demonstrate combined abnormalities of cardiac septal defect. • Cardiac CT may have an incremental role over echocardiography in complex anatomy.

Optimizing angular range in digital breast tomosynthesis: A phantom study investigating lesion detection across varied breast density and thickness.

PURPOSE: To determine the optimal angular range (AR) for digital breast tomosynthesis (DBT) systems that provides highest lesion visibility across various breast densities and thicknesses. METHOD: A modular DBT phantom, consisting of tissue-equivalent adipose and glandular modules, along with a module embedded with test objects (speckles, masses, fibers), was used to create combinations simulating different breast thicknesses, densities, and lesion locations. A prototype DBT system operated at four ARs (AR±7.5°, AR±12.5°, AR±19°, and AR±25°) to acquire 11 projection images for each combination, with separate fixed doses for thin and thick combinations. Three blinded radiologists independently assessed lesion visibility in reconstructed images; assessments were averaged and compared using linear mixed models. RESULTS: Speckle visibility was highest with AR±7.5° or AR±12.5°, decreasing with wider ARs in all density and thickness combinations. The difference between AR±7.5° and AR±12.5° was not statistically significant, except for the tube-side speckles in thin-fatty combinations (5.83 [AR±7.5°] vs. 5.39 [AR±12.5°], P = 0.019). Mass visibility was not affected by AR in thick combinations, while AR±12.5° exhibited the highest mass visibility for both thin-fatty and thin-dense combinations (P = 0.032 and 0.007, respectively). Different ARs provided highest fiber visibility for different combinations; however, AR±12.5° consistently provided highest or comparable visibility. AR±12.5° showed highest overall lesion visibility for all density and thickness combinations. CONCLUSIONS: AR±12.5° exhibited the highest overall lesion visibility across various phantom thicknesses and densities using a projection number of 11.

Assessing internal mammary lymph node metastasis by breast magnetic resonance imaging in breast cancer.

The internal mammary lymph nodes (IMLNs) are a main pathway of metastasis in breast cancer, and breast magnetic resonance imaging (MRI) plays an important role in staging that disease. We investigated the MRI parameters that can predict metastatic IMLNs and evaluated their diagnostic performance by comparing the breast MRI findings for metastatic and benign IMLNs. From January 2016 to December 2020, 474 cases of enlarged IMLNs on breast MRI were identified. By cytopathology or integrated positron emission tomography/computed tomography (PET/CT), 168 IMLNs were confirmed as metastatic, and 81 were confirmed as benign. Breast MRIs were reviewed by 2 radiologists, and various parameters (node axes, fatty hilum, necrosis, margin characteristics, restricted diffusion, and involved levels; primary tumor location and skin involvement) were assessed. Independent t-tests, receiver operating characteristic (ROC) curve analyses, chi-square tests, and Fisher exact tests were performed to compare and evaluate the diagnostic accuracy of the imaging findings. Significant differences in the breast MRI findings for the short and long axes, fatty hilum, necrosis, margin characteristics, diffusion restriction, and tumor location were observed between benign and metastatic IMLNs. Compared with the long axis and the ratio of the axes, the short axis had the best diagnostic value (higher area under the ROC curve) for predicting metastatic IMLNs. In conclusion, breast MRI parameters such as short axis, presence of fatty hilum, necrosis, margin characteristics, and diffusion restriction can be used to evaluate and differentiate benign from metastatic IMLNs, offering valuable insights to improve diagnosis and treatment planning in breast cancer.

Feasibility of supine MRI (Magnetic Resonance Imaging)-navigated ultrasound in breast cancer patients.

OBJECTIVE: The purpose of this study was to evaluate the feasibility of image fusion between US and supine MRI in breast cancer patients, and to evaluate differences in tumor location between prone and supine positions. METHODS: This prospective study included 88 patients who underwent an additional supine MRI (MRsup) sequence following routine prone MRI (MRpro) for breast cancer between May 2016 and December 2017. The location of the tumor and discrepancies in the distances from nipple to lesion (NLD), skin to lesion (SLD), and chest wall to lesion (CLD) were evaluated between MRpro and MRsup (MRpro-sup), MRpro and MRsup-navigated US (MRpro-USnav), and MRsup and USnav (MRsup-USnav). Associations between breast thickness and measurement discrepancies were analyzed. RESULTS: Total 91 index lesions were evaluated. The intraclass correlation coefficients (ICCs) for the location of MRpro and MRsup compared with USnav were 0.994 (range: 0.990-0.996) and 0.998 (range: 0.996-0.999), respectively. The mean MRpro-sup and MRpro-USnav measurement discrepancies were greater than those of MRsup-USnav, significantly. Most outer locations showed greater mean measurement discrepancies than inner locations, and each NLD, SLD, and CLD mean measurement discrepancy showed different tendencies according to location (upper or lower) and lesion depth (superficial, middle, or deep). High breast thickness showed significantly greater mean measurement discrepancies than low breast thickness. CONCLUSION: Image fusion between US and supine MRI is feasible in breast cancer patients, although there is a considerable difference in tumor location measurements between prone and supine positions, especially with thicker breasts.

Ectopic Male Breast Cancer in the Perineum: A Case Report.

Ectopic breast tissue and male breast cancer are both very rare diseases with only a few reports in the literature. Here, we present the first case of ectopic male breast cancer in the perineum. The patient was a 70-year-old man with a palpable mass in the perineum. A wide local excision and inguinal lymph node dissection revealed invasive breast carcinoma of no special type involving the skin and subcutis, and inguinal lymph node metastases. Immunohistochemical staining showed that the tumor cells were strongly positive for estrogen and progesterone receptors and negative for human epidermal growth factor receptor 2. Moreover, no p53 overexpression was observed. Herein, the clinical and pathologic features, as well as a review of ectopic male breast cancer are discussed.

Coronary bifurcation stent morphology in dual-source CT: validation with micro-CT.

The aim of this study is to analyze post-bifurcation coronary stent morphology in vitro using dual-source CT and validate those findings using micro-CT. Nine silicon coronary artery phantoms simulating main branch (MB) and side branch (SB) were prepared using a 3-D printer. After bifurcation stenting using Crush, Culotte, and T-stenting technique, in vitro CT imaging was conducted using both dual-source CT and micro-CT. Morphological change of the bifurcation stent including crushed segment and floating stent strut were evaluated. Dual-source CT was able to demonstrate morphologic changes of bifurcation stents and was comparable to micro-CT. In stents with Crush and T-stenting methods, crushed segment appeared denser and thicker than other stent parts and was located at MB side in all phantoms. Floating strut was observed in half of the phantoms with Crush technique and all phantoms with T-stenting. Parameters measured in both dual-source CT and micro-CT showed good correlation and high agreement (limits of agreement and correlation for length, perimeter and area, 0.3 ± 3.5, 0.5 ± 2.1 mm, and 0.0 ± 1.5 mm², r = 0.76, 0.92, and 0.91). The morphology of post-bifurcation stent on dual-source CT correlates well with that of micro-CT in the coronary artery phantom. Coronary CT angiography may be a feasible method for the evaluation of stent morphology in patients who underwent bifurcation stenting.