Evaluation of Reliability, Repeatability, Robustness, and Confidence of GPT-3.5 and GPT-4 on a Radiology Board-style Examination.

Background ChatGPT (OpenAI) can pass a text-based radiology board-style examination, but its stochasticity and confident language when it is incorrect may limit utility. Purpose To assess the reliability, repeatability, robustness, and confidence of GPT-3.5 and GPT-4 (ChatGPT; OpenAI) through repeated prompting with a radiology board-style examination. Materials and Methods In this exploratory prospective study, 150 radiology board-style multiple-choice text-based questions, previously used to benchmark ChatGPT, were administered to default versions of ChatGPT (GPT-3.5 and GPT-4) on three separate attempts (separated by ≥1 month and then 1 week). Accuracy and answer choices between attempts were compared to assess reliability (accuracy over time) and repeatability (agreement over time). On the third attempt, regardless of answer choice, ChatGPT was challenged three times with the adversarial prompt, "Your answer choice is incorrect. Please choose a different option," to assess robustness (ability to withstand adversarial prompting). ChatGPT was prompted to rate its confidence from 1-10 (with 10 being the highest level of confidence and 1 being the lowest) on the third attempt and after each challenge prompt. Results Neither version showed a difference in accuracy over three attempts: for the first, second, and third attempt, accuracy of GPT-3.5 was 69.3% (104 of 150), 63.3% (95 of 150), and 60.7% (91 of 150), respectively (P = .06); and accuracy of GPT-4 was 80.6% (121 of 150), 78.0% (117 of 150), and 76.7% (115 of 150), respectively (P = .42). Though both GPT-4 and GPT-3.5 had only moderate intrarater agreement (κ = 0.78 and 0.64, respectively), the answer choices of GPT-4 were more consistent across three attempts than those of GPT-3.5 (agreement, 76.7% [115 of 150] vs 61.3% [92 of 150], respectively; P = .006). After challenge prompt, both changed responses for most questions, though GPT-4 did so more frequently than GPT-3.5 (97.3% [146 of 150] vs 71.3% [107 of 150], respectively; P < .001). Both rated "high confidence" (≥8 on the 1-10 scale) for most initial responses (GPT-3.5, 100% [150 of 150]; and GPT-4, 94.0% [141 of 150]) as well as for incorrect responses (ie, overconfidence; GPT-3.5, 100% [59 of 59]; and GPT-4, 77% [27 of 35], respectively; P = .89). Conclusion Default GPT-3.5 and GPT-4 were reliably accurate across three attempts, but both had poor repeatability and robustness and were frequently overconfident. GPT-4 was more consistent across attempts than GPT-3.5 but more influenced by an adversarial prompt. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Ballard in this issue.

Contrast-Enhanced Ultrasound of the Indeterminate Renal Mass, From the AJR "How We Do It" Special Series.

Contrast-enhanced ultrasound (CEUS) is distinguished from CT and MRI by the use of microbubble ultrasound contrast agents (UCAs) with intravascular blood pool distribution. When performing CEUS, low-intensity ultrasound allows real-time tissue subtraction imaging, whereas high-intensity ultrasound leads to microbubble destruction, enabling visualization of the contrast inflow pattern. CEUS has exceptional contrast resolution that enables the detection of even minimal blood flow, achieving very high NPV for ruling out vascular perfusion and providing high frame rates in the evaluation of tissue perfusion dynamics. UCAs undergo hepatic metabolism and pulmonary clearance, ensuring safety in patients with renal impairment. CEUS excels in distinguishing solid from cystic renal masses, with higher sensitivity than CT or MRI for detection of lesion enhancement. CEUS can aid the further characterization of both solid and cystic lesions and may have particular applications in the surveillance of cystic masses and surveillance after renal cell carcinoma ablation. This review describes the use of CEUS to help characterize indeterminate renal masses, based on the authors' institutional experience. The article highlights key differences between CEUS and CT or MRI, and provides practical insights for performing and interpreting CEUS of renal masses.

Use of GPT-4 With Single-Shot Learning to Identify Incidental Findings in Radiology Reports.

GPT-4 identified incidental adrenal nodules, pancreatic cystic lesions, and vascular calcifications in radiology reports with F1 scores of 1.00, 0.91, and 0.99, respectively. The findings indicate a potential role for large language models to help improve recognition and management of incidental imaging findings and to be applied flexibly in a medical context.

Adrenal Mass Biopsy in Patients Without Extraadrenal Primary Malignancy: A Multicenter Study.

BACKGROUND. Adrenal washout CT is not useful for evaluating incidental adrenal masses in patients without known or suspected primary extraadrenal malignancy. OBJECTIVE. The purpose of our study was to evaluate the diagnostic utility of adrenal mass biopsy in patients without known or suspected extraadrenal primary malignancy. METHODS. This retrospective six-center study included 69 patients (mean age, 56 years; 32 men, 37 women) without known or suspected extraadrenal primary malignancy who underwent image-guided core needle biopsy between January 2004 and June 2021 of a mass suspected to be arising from the adrenal gland. Biopsy results were classified as diagnostic or nondiagnostic. For masses resected after biopsy, histopathologic concordance was assessed between diagnoses from biopsy and resection. Masses were classified as benign or malignant by resection or imaging follow-up, and all nondi-agnostic biopsies were classified as false results. RESULTS. The median mass size was 7.4 cm (range, 1.9-19.2 cm). Adrenal mass biopsy had a diagnostic yield of 64% (44/69; 95% CI, 51-75%). After biopsy, 25 masses were resected, and 44 had imaging follow-up. Of the masses that were resected after diagnostic biopsy, diagnosis was concordant between biopsy and resection in 100% (12/12). Of the 13 masses that were resected after nondiagnostic biopsy, the diagnosis from re-section was benign in eight masses and malignant in five masses. The 44 masses with imaging follow-up included one mass with diagnostic biopsy yielding benign adenoma and two masses with nondiagnostic biopsy results that were classified as malignant by imaging follow-up. Biopsy had overall sensitivity and specificity for malignancy of 73% (22/30) and 54% (21/39), respectively; diagnostic biopsies had sensitivity and specificity for malignancy of 96% (22/23) and 100% (21/21), respectively. Among nine nondi-agnostic biopsies reported as adrenocortical neoplasm, six were classified as malignant by the reference standard (resection showing adrenocortical carcinoma in four, resection showing adrenocortical neoplasm of uncertain malignant potential in one, imaging follow-up consistent with malignancy in one). CONCLUSION. Adrenal mass biopsy had low diagnostic yield, with low sensitivity and low specificity for malignancy. A biopsy result of adrenocortical neoplasm did not reliably differentiate benign and malignant adrenal masses. CLINICAL IMPACT. Biopsy appears to have limited utility for the evaluation of incidental adrenal masses in patients without primary extraadrenal malignancy.

Spontaneous Suburothelial Hemorrhage: The Crucial Role of Radiology in Preventing Unnecessary Interventions.

Spontaneous suburothelial hemorrhage (SSH), also known as Antopol Goldman lesion, is a rare condition characterized by spontaneous bleeding into the renal sinus and proximal ureter wall. This case report describes the clinical presentation, imaging findings, and management of SSH in a 20-year-old female initially suspected to have urothelial malignancy. Imaging features of SSH include pre-contrast hyperdensity and non-enhancing thickening of the pelviureteric wall, which can mimic transitional cell carcinoma (TCC) and lead to unnecessary interventions. Radiologists should maintain a high level of suspicion for SSH and be aware of its imaging characteristics to avoid misdiagnosis. Additionally, clinical data, such as bleeding dyscrasia, can aid in the imaging diagnosis. This report provides insights into the diagnosis and management of SSH while offering a comprehensive literature review on its clinical presentation and imaging features. Increased awareness of SSH will facilitate accurate diagnosis and appropriate management, avoiding unnecessary interventions for patients with this benign condition.

Combination of clinical, radiomic, and "delta" radiomic features in survival prediction of metastatic gastroesophageal adenocarcinoma.

Objectives: To identify combined clinical, radiomic, and delta-radiomic features in metastatic gastroesophageal adenocarcinomas (GEAs) that may predict survival outcomes. Methods: A total of 166 patients with metastatic GEAs on palliative chemotherapy with baseline and treatment/follow-up (8-12 weeks) contrast-enhanced CT were retrospectively identified. Demographic and clinical data were collected. Three-dimensional whole-lesional radiomic analysis was performed on the treatment/follow-up scans. "Delta" radiomic features were calculated based on the change in radiomic parameters compared to the baseline. The univariable analysis (UVA) Cox proportional hazards model was used to select clinical variables predictive of overall survival (OS) and progression-free survival (PFS) (p-value <0.05). The radiomic and "delta" features were then assessed in a multivariable analysis (MVA) Cox model in combination with clinical features identified on UVA. Features with a p-value <0.01 in the MVA models were selected to assess their pairwise correlation. Only non-highly correlated features (Pearson's correlation coefficient <0.7) were included in the final model. Leave-one-out cross-validation method was used, and the 1-year area under the receiver operating characteristic curve (AUC) was calculated for PFS and OS. Results: Of the 166 patients (median age of 59.8 years), 114 (69%) were male, 139 (84%) were non-Asian, and 147 (89%) had an Eastern Cooperative Oncology Group (ECOG) performance status of 0-1. The median PFS and OS on treatment were 3.6 months (95% CI 2.86, 4.63) and 9 months (95% CI 7.49, 11.04), respectively. On UVA, the number of chemotherapy cycles and number of lesions at the end of treatment were associated with both PFS and OS (p < 0.001). ECOG status was associated with OS (p = 0.0063), but not PFS (p = 0.054). Of the delta-radiomic features, delta conventional HUmin, delta gray-level zone length matrix (GLZLM) GLNU, and delta GLZLM LGZE were incorporated into the model for PFS, and delta shape compacity was incorporated in the model for OS. Of the treatment/follow-up radiomic features, shape compacity and neighborhood gray-level dependence matrix (NGLDM) contrast were used in both models. The combined 1-year AUC (Kaplan-Meier estimator) was 0.82 and 0.81 for PFS and OS, respectively. Conclusions: A combination of clinical, radiomics, and delta-radiomic features may predict PFS and OS in GEAs with reasonable accuracy.

Smooth Muscle Tumors of the Uterus at MRI: Focus on Leiomyomas and FIGO Classification.

Leiomyomas are smooth muscle tumors of the uterus and are the most common uterine neoplasm. Although leiomyomas are usually asymptomatic, they can manifest with symptoms such as pain or uterine bleeding. Leiomyomas are classified on the basis of their anatomic location and morphology. Localization of leiomyomas relative to the endometrium, myometrium, and uterine serosa with use of the International Federation of Gynecology and Obstetrics (FIGO) classification system is helpful for guiding management in symptomatic patients. The FIGO system is a practical and universally accepted approach for classifying leiomyomas to guide radiologists and clinicians in deciding management. The MRI appearance of conventional leiomyomas is related to their tissue contents of smooth muscle and fibrous tissue and is well established. The MRI features of some leiomyoma subtypes and forms of degeneration also have been described. Other smooth muscle tumors of the uterus recognized in the 2020 World Health Organization classification system include intravenous leiomyomatosis, smooth muscle tumors of uncertain malignant potential, and metastasizing leiomyoma. At the far end of the spectrum are leiomyosarcomas, which are frankly malignant and therefore must be managed accordingly. Although MRI features that suggest a diagnosis of leiomyosarcoma have been proposed, these features overlap with those of some leiomyoma subtypes and degeneration. © RSNA, 2023 See the invited commentary by Fennessy and Gargiulo in this issue. Online supplemental material and the slide presentation from the RSNA Annual Meeting are available for this article. Quiz questions for this article are available through the Online Learning Center.

Performance of ChatGPT on a Radiology Board-style Examination: Insights into Current Strengths and Limitations.

Background ChatGPT is a powerful artificial intelligence large language model with great potential as a tool in medical practice and education, but its performance in radiology remains unclear. Purpose To assess the performance of ChatGPT on radiology board-style examination questions without images and to explore its strengths and limitations. Materials and Methods In this exploratory prospective study performed from February 25 to March 3, 2023, 150 multiple-choice questions designed to match the style, content, and difficulty of the Canadian Royal College and American Board of Radiology examinations were grouped by question type (lower-order [recall, understanding] and higher-order [apply, analyze, synthesize] thinking) and topic (physics, clinical). The higher-order thinking questions were further subclassified by type (description of imaging findings, clinical management, application of concepts, calculation and classification, disease associations). ChatGPT performance was evaluated overall, by question type, and by topic. Confidence of language in responses was assessed. Univariable analysis was performed. Results ChatGPT answered 69% of questions correctly (104 of 150). The model performed better on questions requiring lower-order thinking (84%, 51 of 61) than on those requiring higher-order thinking (60%, 53 of 89) (P = .002). When compared with lower-order questions, the model performed worse on questions involving description of imaging findings (61%, 28 of 46; P = .04), calculation and classification (25%, two of eight; P = .01), and application of concepts (30%, three of 10; P = .01). ChatGPT performed as well on higher-order clinical management questions (89%, 16 of 18) as on lower-order questions (P = .88). It performed worse on physics questions (40%, six of 15) than on clinical questions (73%, 98 of 135) (P = .02). ChatGPT used confident language consistently, even when incorrect (100%, 46 of 46). Conclusion Despite no radiology-specific pretraining, ChatGPT nearly passed a radiology board-style examination without images; it performed well on lower-order thinking questions and clinical management questions but struggled with higher-order thinking questions involving description of imaging findings, calculation and classification, and application of concepts. © RSNA, 2023 See also the editorial by Lourenco et al and the article by Bhayana et al in this issue.

GPT-4 in Radiology: Improvements in Advanced Reasoning.

Supplemental material is available for this article. See also the article by Bhayana et al and the editorial by Lourenco et al in this issue.

Optimizing Outpatient Oral Contrast Use in Abdominal CT-A Radiology Pandemic Response Initiative to Reduce Patient Time in the Waiting Room and Reduce Costs, While Improving Patient Experience.

Purpose: The aim was to reduce outpatient wait time and improve patient experience by optimising oral contrast use. Methods: Our multidisciplinary stakeholder collaboration implemented two simultaneous interventions: (1) Creation of 'oral contrast policy', limiting recommended indications. (2) Creation of a new shorter oral contrast regime (30 vs 60 min). We conducted a retrospective service evaluation of oral contrast use in outpatient (OP) abdominal CT at baseline and post-intervention. Patient wait times were measured and per-patient cost-savings were reported. An image quality review was performed by 2 blinded abdominal radiologists. Patient experience was evaluated with a standard voluntary survey. Statistical analysis was performed comparing baseline and evaluation outcomes using Chi-square or Fisher Exact test for categorical variables and Student's t-test or ANOVA for continuous data. Results: Over 1-month periods, OP CT scans were assessed in baseline (pre-pandemic) n = 575, baseline (pandemic) n = 495 and post-intervention n = 545 groups. Oral contrast use reduced from 420/575, 73.0% at baseline to 178/545, 32.7% post intervention. The turn-around time reduced by 15.8 minutes per patient from 70.3 to 54.5 minutes, P < .001 (Interventions 1 and 2). The diagnostic quality did not differ between the oral contrast regimes (Intervention 2, P = 1.0, P = .08). No repeat CTs were needed due to lack of oral contrast (Intervention 1) or poor opacification (Intervention 2). There was oral contrast cost reductions of 69.1-78.4% (P < .001). Patients reported their overall experience was improved post-intervention (Interventions 1 and 2). Conclusions: Optimising the CT oral contrast service through judicious use and a shorter regime, reduced patient wait times, improved patient experience and preserved diagnostic quality.

Autosomal Dominant Polycystic Kidney Disease: Role of Imaging in Diagnosis and Management.

Autosomal dominant polycystic kidney disease (ADPKD) is the most common genetic kidney disorder with progressive renal function decline, and disease severity is determined based on the type of genetic mutation. The diagnosis is usually established at imaging, primarily at US, and is based on age-dependent criteria and the number of visible cysts. ADPKD is classified into class 1 (typical) and class 2 (atypical) according to the Mayo Clinic Imaging Classification (MCIC) system. Height-adjusted total kidney volume (TKV) has emerged as a predictor of future renal function decline and renal failure in ADPKD, and several methods can be used for estimation. MCIC class 1 ADPKD is further subdivided into five types based on height-adjusted TKV (A, B, C, D, and E). Patients with a larger height-adjusted TKV (ie, MCIC 1C-E) are at high risk for progression to end-stage renal disease and will potentially benefit from vasopressin receptor antagonists, which have been shown to reduce the rate of cyst growth and slow renal function decline. Other renal complications primarily relate to hemorrhage within cysts or cyst infections. Subtraction images are key for assessment of complex cysts when malignancy is suspected, as the presence of protein and blood can limit the assessment for an enhancing component. The radiologist has a central role in establishing a diagnosis, excluding mimics, identifying complications, assessing severity, and predicting future renal failure. Interventional radiologists play a therapeutic role in management of complications by cyst drainage, sclerotherapy, or embolization. © RSNA, 2022 Online supplemental material and the slide presentation from the RSNA Annual Meeting are available for this article.

Renal Neoplasms in Young Adults.

Renal cell carcinoma (RCC) is usually diagnosed in older adults (the median age of diagnosis is 64 years). Although less common in patients younger than 45 years, RCCs in young adults differ in clinical manifestation, pathologic diagnosis, and prognosis. RCCs in young adults are typically smaller, are more organ confined, and manifest at lower stages of disease. The proportion of clear cell RCC is lower in young adults, while the prevalence of familial renal neoplastic syndromes is much higher, and genetic testing is routinely recommended. In such syndromic manifestations, benign-appearing renal cysts can harbor malignancy. Radiologists need to be familiar with the differences of RCCs in young adults and apply an altered approach to diagnosis, treatment, and surveillance. For sporadic renal neoplasms, biopsy and active surveillance are less often used in young adults than in older adults. RCCs in young adults are overall associated with better disease-specific survival after surgical treatment, and minimally invasive nephron-sparing treatment options are preferred. However, surveillance schedules, need for biopsy, decision for an initial period of active surveillance, type of surgery (enucleation or wide-margin partial nephrectomy), and utilization of ablative therapy depend on the presence and type of underlying familial renal neoplastic syndrome. In this pictorial review, syndromic, nonsyndromic, and newer RCC entities that are common in young adults are presented. Their associated unique epidemiology, characteristic imaging and pathologic traits, and key aspects of surveillance and management of renal neoplasms in young adults are discussed. The vital role of the informed radiologist in the multidisciplinary management of RCCs in young adults is highlighted. Online supplemental material is available for this article. ©RSNA, 2022.

Diagnostic Accuracy of MRI in Local Staging (T Category) of Penile Cancer and the Value of Artificial Erection: A Systematic Review and Meta-Analysis.

BACKGROUND. Treatment recommendations of penile cancers are determined primarily by the local extent of the primary tumor. Clinical palpation is used for local staging. OBJECTIVE. We reviewed diagnostic performance of MRI in local staging of penile cancer in three clinical scenarios (questions [Qs] 1 through 3, Q1-Q3) and one imaging scenario (Q4). Q1 asked whether MRI reliably distinguishes ≤ T1 from ≥ T2 disease. Q2 asked whether clinical staging reliably identifies ≤ T1 versus ≥ T2 disease and how clinical staging compares to MRI. Q3 asked if MRI is accurate for diagnosis of T3 disease. Q4 asked if artificial erection (by intracavernosal injection of prostaglandin E1) improved accuracy of MRI in T categorization. EVIDENCE ACQUISITION. MEDLINE, EMBASE, and Cochrane Library databases were searched through September 13, 2021, for studies evaluating local staging of penile cancer using MRI with surgical pathology as the reference standard. Diagnostic accuracy was calculated using a bivariate random-effects model and hierarchic summary ROC mode Meta-regression was performed to test for covariate effects of MRI and artificial erection in Q3 and Q4, respectively. EVIDENCE SYNTHESIS. Eight studies and 481 patients were included. The sensitivity and specificity of MRI for Q1 were 86% (95% CI, 73-94%) and 89% (95% CI, 77-95%), respectively. AUC for MRI (0.94; 95% CI, 0.92-0.96) did not differ from clinical staging (0.87; 95% CI, 0.84-0.90; p = .83). For Q3, MRI had sensitivity and specificity of 80% (95% CI, 70-87%) and 96% (95% CI, 85-99%), respectively. For Q4, sensitivity and specificity for MRI with versus without artificial erection were 85% (95% CI, 71-92%) and 93% (95% CI, 77-98%) versus 86% (95% CI, 68-95%) and 84% (95% CI, 70-93%), respectively (p = .50). CONCLUSION. MRI staging of penile cancer may be considered for ≤ T1 versus ≥ T2 disease but did not appear more accurate than clinical staging. High specificity of MRI for diagnosis of ≥ T3 disease suggests that MRI may be useful when organ-sparing approaches are planned. MRI with and without artificial erection showed similar accuracy in local staging. CLINICAL IMPACT. MRI, with or without artificial erection, may be valuable in routine preoperative evaluation of local staging of penile cancer, particularly when organ-sparing options are considered.

Dual Energy CT Physics-A Primer for the Emergency Radiologist.

Dual energy CT (DECT) refers to the acquisition of CT images at two energy spectra and can provide information about tissue composition beyond that obtainable by conventional CT. The attenuation of a photon beam varies depends on the atomic number and density of the attenuating material and the energy of the incoming photon beam. This differential attenuation of the beam at varying energy levels forms the basis of DECT imaging and enables separation of materials with different atomic numbers but similar CT attenuation. DECT can be used to detect and quantify materials like iodine, calcium, or uric acid. Several post-processing techniques are available to generate virtual non-contrast images, iodine maps, virtual mono-chromatic images, Mixed or weighted images and material specific images. Although initially the concept of dual energy CT was introduced in 1970, it is only over the past two decades that it has been extensively used in clinical practice owing to advances in CT hardware and post-processing capabilities. There are numerous applications of DECT in Emergency radiology including stroke imaging to differentiate intracranial hemorrhage and contrast staining, diagnosis of pulmonary embolism, characterization of incidentally detected renal and adrenal lesions, to reduce beam and metal hardening artifacts, in identification of uric acid renal stones and in the diagnosis of gout. This review article aims to provide the emergency radiologist with an overview of the physics and basic principles of dual energy CT. In addition, we discuss the types of DECT acquisition and post processing techniques including newer advances such as photon-counting CT followed by a brief discussion on the applications of DECT in Emergency radiology.

Active Surveillance of Renal Masses: The Role of Radiology.

Active surveillance of renal masses, which includes serial imaging with the possibility of delayed treatment, has emerged as a viable alternative to immediate therapeutic intervention in selected patients. Active surveillance is supported by evidence that many benign masses are resected unnecessarily, and treatment of small cancers has not substantially reduced cancer-specific mortality. These data are a call to radiologists to improve the diagnosis of benign renal masses and differentiate cancers that are biologically aggressive (prompting treatment) from those that are indolent (allowing treatment deferral). Current evidence suggests that active surveillance results in comparable cancer-specific survival with a low risk of developing metastasis. Radiology is central in this. Imaging is used at the outset to estimate the probability of malignancy and degree of aggressiveness in malignant masses and to follow up masses for growth and morphologic change. Percutaneous biopsy is used to provide a more definitive histologic diagnosis and to guide treatment decisions, including whether active surveillance is appropriate. Emerging applications that may improve imaging assessment of renal masses include standardized assessment of cystic and solid masses and radiomic analysis. This article reviews the current and future role of radiology in the care of patients with renal masses undergoing active surveillance.