Liver Observation Segmentation on Contrast-Enhanced MRI: SAM and MedSAM Performance in Patients With Probable or Definite Hepatocellular Carcinoma.

Purpose: To evaluate factors impacting the Segment Anything Model (SAM) and variant MedSAM performance for segmenting liver observations on contrast-enhanced (CE) magnetic resonance imaging (MRI) in high-risk patients with probable hepatocellular carcinoma (HCC) (LR-4) and definite HCC (LR-5). Methods: A retrospective cohort of liver observations (LR-4/LR-5) on CE-MRI from 97 patients at high-risk for HCC was derived (2013-2018). Using bounding-boxes as prompts under 5-fold cross-validation, segmentation performance was evaluated at the model and liver observation-levels for: (1) model types: SAM versus MedSAM, (2) image sizes: 256 × 256 versus 512 × 512, (3) image channel composition: CE sequences at 3 phases of enhancement independently and combined, (4) liver observation size: >10 mm versus >20 mm, (5) certainty of diagnosis: LR-4 versus LR-5, and (6) contrast-agent type: hepatobiliary versus extracellular. Segmentation performance, quantified using Dice coefficient, were compared using univariate (Wilcoxon signed-rank and t-test) and multivariable analyses (multiple correspondence analysis and subsequent linear modelling). Results: MedSAM trained on 512 × 512 combined CE sequences performed best with mean Dice coefficient 0.68 (95% confidence interval 0.66, 0.69). Overall, all factors except contrast-agent type affected performance, with larger image size resulting in the highest performance improvement (512 × 512: 0.57, 256 × 256: 0.26, P < .001) at the model-level. Contrast-agents affected performance for patients with LR-4 observations using MedSAM-based models (P < .03). Larger observation size, image size, and higher certainty of diagnosis were associated with better segmentation on multivariable analysis. Conclusion: A variety of factors were found to impact SAM/MedSAM performance for segmenting liver observations in patients with probable and definite HCC on CE-MRI. Future models may be optimized by accounting for these factors.

LI-RADS CT and MRI Ancillary Feature Association with Hepatocellular Carcinoma and Malignancy: An Individual Participant Data Meta-Analysis.

Background The independent contribution of each Liver Imaging Reporting and Data System (LI-RADS) CT or MRI ancillary feature (AF) has not been established. Purpose To evaluate the association of LI-RADS AFs with hepatocellular carcinoma (HCC) and malignancy while adjusting for LI-RADS major features through an individual participant data (IPD) meta-analysis. Materials and Methods Medline, Embase, Cochrane Central Register of Controlled Trials, and Scopus were searched from January 2014 to January 2022 for studies evaluating the diagnostic accuracy of CT and MRI for HCC using LI-RADS version 2014, 2017, or 2018. Using a one-step approach, IPD across studies were pooled. Adjusted odds ratios (ORs) and 95% CIs were derived from multivariable logistic regression models of each AF combined with major features except threshold growth (excluded because of infrequent reporting). Liver observation clustering was addressed at the study and participant levels through random intercepts. Risk of bias was assessed using a composite reference standard and Quality Assessment of Diagnostic Accuracy Studies 2. Results Twenty studies comprising 3091 observations (2456 adult participants; mean age, 59 years ± 11 [SD]; 1849 [75.3%] men) were included. In total, 89% (eight of nine) of AFs favoring malignancy were associated with malignancy and/or HCC, 80% (four of five) of AFs favoring HCC were associated with HCC, and 57% (four of seven) of AFs favoring benignity were negatively associated with HCC and/or malignancy. Nonenhancing capsule (OR = 3.50 [95% CI: 1.53, 8.01]) had the strongest association with HCC. Diffusion restriction (OR = 14.45 [95% CI: 9.82, 21.27]) and mild-moderate T2 hyperintensity (OR = 10.18 [95% CI: 7.17, 14.44]) had the strongest association with malignancy. The strongest negative associations with HCC were parallels blood pool enhancement (OR = 0.07 [95% CI: 0.01, 0.49]) and marked T2 hyperintensity (OR = 0.18 [95% CI: 0.07, 0.45]). Seventeen studies (85%) had a high risk of bias. Conclusion Most LI-RADS AFs were independently associated with HCC, malignancy, or benignity as intended when adjusting for major features. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Crivellaro in this issue.

Artificial Intelligence Chatbots' Understanding of the Risks and Benefits of Computed Tomography and Magnetic Resonance Imaging Scenarios.

PURPOSE: Patients may seek online information to better understand medical imaging procedures. The purpose of this study was to assess the accuracy of information provided by 2 popular artificial intelligence (AI) chatbots pertaining to common imaging scenarios' risks, benefits, and alternatives. METHODS: Fourteen imaging-related scenarios pertaining to computed tomography (CT) or magnetic resonance imaging (MRI) were used. Factors including the use of intravenous contrast, the presence of renal disease, and whether the patient was pregnant were included in the analysis. For each scenario, 3 prompts for outlining the (1) risks, (2) benefits, and (3) alternative imaging choices or potential implications of not using contrast were inputted into ChatGPT and Bard. A grading rubric and a 5-point Likert scale was used by 2 independent reviewers to grade responses. Prompt variability and chatbot context dependency were also assessed. RESULTS: ChatGPT's performance was superior to Bard's in accurately responding to prompts per Likert grading (4.36 ± 0.63 vs 3.25 ± 1.03 seconds, P < .0001). There was substantial agreement between independent reviewer grading for ChatGPT (κ = 0.621) and Bard (κ = 0.684). Response text length was not statistically different between ChatGPT and Bard (2087 ± 256 characters vs 2162 ± 369 characters, P = .24). Response time was longer for ChatGPT (34 ± 2 vs 8 ± 1 seconds, P < .0001). CONCLUSIONS: ChatGPT performed superior to Bard at outlining risks, benefits, and alternatives to common imaging scenarios. Generally, context dependency and prompt variability did not change chatbot response content. Due to the lack of detailed scientific reasoning and inability to provide patient-specific information, both AI chatbots have limitations as a patient information resource.

Percentage of Pancreatic Cysts on MRI With a Pancreatic Carcinoma: Systematic Review and Meta-Analysis.

BACKGROUND: Pancreatic cystic lesions (PCLs) are frequent on MRI and are thought to be associated with pancreatic adenocarcinoma (PDAC) necessitating long-term surveillance based on older studies suffering from selection bias. PURPOSE: To establish the percentage of patients with PCLs on MRI with a present or future PDAC. STUDY TYPE: Systematic review, meta-analysis. POPULATION: Adults with PCLs on MRI and a present or future diagnosis of PDAC were eligible. MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Scopus were searched to April 2022 (PROSPERO:CRD42022320502). Studies limited to PCLs not requiring surveillance, <100 patients, or those with a history/genetic risk of PDAC were excluded. FIELD STRENGTH/SEQUENCE: ≥1.5 T with ≥1 T2-weighted sequence. ASSESSMENT: Two investigators extracted data, with discrepancies resolved by a third. QUADAS-2 assessed bias. PDAC was diagnosed using a composite reference standard. STATISTICAL TESTS: A meta-analysis of proportions was performed at the patient-level with 95% confidence intervals (95% CI). RESULTS: Eight studies with 1289 patients contributed to the percentage of patients with a present diagnosis of PDAC, and 10 studies with 3422 patients to the percentage with a future diagnosis. Of patients with PCLs on MRI, 14.8% (95% CI 2.4-34.9) had a PDAC at initial MRI, which decreased to 6.0% (2.2-11.3) for studies at low risk of bias. For patients without PDAC on initial MRI, 2.0% (1.1-3.2) developed PDAC during surveillance, similar for low risk of bias studies at 1.9% (0.7-3.6), with no clear trend of increased PDAC for longer surveillance durations. For patients without worrisome features or high-risk stigmata, 0.9% (0.1-2.2) developed PDAC during surveillance. Of 10, eight studies had a median surveillance ≥3 years (range 3-157 months). Sources of bias included retrospectively limiting PCLs to those with histopathology and inconsistent surveillance protocols. DATA CONCLUSION: A low percentage of patients with PCLs on MRI develop PDAC while on surveillance. The first MRI revealing a PCL should be scrutinized for PDAC. LEVEL OF EVIDENCE: 3 TECHNICAL EFFICACY: Stage 2.

Comparative Performance of 2018 LI-RADS versus Modified LIRADS (mLI-RADS): An Individual Participant Data Meta-Analysis.

BACKGROUND: LI-RADS version 2018 (v2018) is used for non-invasive diagnosis of hepatocellular carcinoma (HCC). A recently proposed modification (known as mLI-RADS) demonstrated improved sensitivity while maintaining specificity and positive predictive value (PPV) of LI-RADS category 5 (definite HCC) for HCC. However, mLI-RADS requires multicenter validation. PURPOSE: To evaluate the performance of v2018 and mLI-RADS for liver lesions in a large, heterogeneous, multi-national cohort of patients at risk for HCC. STUDY TYPE: Systematic review and meta-analysis using individual participant data (IPD) [Study Protocol: https://osf.io/duys4]. POPULATION: 2223 observations from 1817 patients (includes all LI-RADS categories; females = 448, males = 1361, not reported = 8) at elevated risk for developing HCC (based on LI-RADS population criteria) from 12 retrospective studies. FIELD STRENGTH/SEQUENCE: 1.5T and 3T; complete liver MRI with gadoxetate disodium, including axial T2w images and dynamic axial fat-suppressed T1w images precontrast and in the arterial, portal venous, transitional, and hepatobiliary phases. Diffusion-weighted imaging was used when available. ASSESSMENT: Liver observations were categorized using v2018 and mLI-RADS. The diagnostic performance of each system's category 5 (LR-5 and mLR-5) for HCC were compared. STATISTICAL TESTS: The Quality Assessment of Diagnostic Accuracy Studies version 2 (QUADAS-2 was applied to determine risk of bias and applicability. Diagnostic performances were assessed using the likelihood ratio test for sensitivity and specificity and the Wald test for PPV. The significance level was P < 0.05. RESULTS: 17% (2/12) of the studies were considered low risk of bias (244 liver observations; 164 patients). When compared to v2018, mLR-5 demonstrated higher sensitivity (61.3% vs. 46.5%, P < 0.001), similar PPV (85.3% vs. 86.3%, P = 0.89), and similar specificity (85.8% vs. 90.8%, P = 0.16) for HCC. DATA CONCLUSION: This study confirms mLR-5 has higher sensitivity than LR-5 for HCC identification, while maintaining similar PPV and specificity, validating the mLI-RADS proposal in a heterogeneous, international cohort. LEVEL OF EVIDENCE: 3 TECHNICAL EFFICACY: Stage 2.

Individual Participant Data Meta-Analysis of LR-5 in LI-RADS Version 2018 versus Revised LI-RADS for Hepatocellular Carcinoma Diagnosis.

Background A simplification of the Liver Imaging Reporting and Data System (LI-RADS) version 2018 (v2018), revised LI-RADS (rLI-RADS), has been proposed for imaging-based diagnosis of hepatocellular carcinoma (HCC). Single-site data suggest that rLI-RADS category 5 (rLR-5) improves sensitivity while maintaining positive predictive value (PPV) of the LI-RADS v2018 category 5 (LR-5), which indicates definite HCC. Purpose To compare the diagnostic performance of LI-RADS v2018 and rLI-RADS in a multicenter data set of patients at risk for HCC by performing an individual patient data meta-analysis. Materials and Methods Multiple databases were searched for studies published from January 2014 to January 2022 that evaluated the diagnostic performance of any version of LI-RADS at CT or MRI for diagnosing HCC. An individual patient data meta-analysis method was applied to observations from the identified studies. Quality Assessment of Diagnostic Accuracy Studies version 2 was applied to determine study risk of bias. Observations were categorized according to major features and either LI-RADS v2018 or rLI-RADS assignments. Diagnostic accuracies of category 5 for each system were calculated using generalized linear mixed models and compared using the likelihood ratio test for sensitivity and the Wald test for PPV. Results Twenty-four studies, including 3840 patients and 4727 observations, were analyzed. The median observation size was 19 mm (IQR, 11-30 mm). rLR-5 showed higher sensitivity compared with LR-5 (70.6% [95% CI: 60.7, 78.9] vs 61.3% [95% CI: 45.9, 74.7]; P < .001), with similar PPV (90.7% vs 92.3%; P = .55). In studies with low risk of bias (n = 4; 1031 observations), rLR-5 also achieved a higher sensitivity than LR-5 (72.3% [95% CI: 63.9, 80.1] vs 66.9% [95% CI: 58.2, 74.5]; P = .02), with similar PPV (83.1% vs 88.7%; P = .47). Conclusion rLR-5 achieved a higher sensitivity for identifying HCC than LR-5 while maintaining a comparable PPV at 90% or more, matching the results presented in the original rLI-RADS study. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Sirlin and Chernyak in this issue.

Best Practice for MRI Diagnostic Accuracy Research With Lessons and Examples from the LI-RADS Individual Participant Data Group.

Medical imaging diagnostic test accuracy research is strengthened by adhering to best practices for study design, data collection, data documentation, and study reporting. In this review, key elements of such research are discussed, and specific recommendations provided for optimizing diagnostic accuracy study execution to improve uniformity, minimize common sources of bias and avoid potential pitfalls. Examples are provided regarding study methodology and data collection practices based on insights gained by the liver imaging reporting and data system (LI-RADS) individual participant data group, who have evaluated raw data from numerous MRI diagnostic accuracy studies for risk of bias and data integrity. The goal of this review is to outline strategies for investigators to improve research practices, and to help reviewers and readers better contextualize a study's findings while understanding its limitations. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY: Stage 3.

Comparative Performance of ChatGPT and Bard in a Text-Based Radiology Knowledge Assessment.

PURPOSE: Bard by Google, a direct competitor to ChatGPT, was recently released. Understanding the relative performance of these different chatbots can provide important insight into their strengths and weaknesses as well as which roles they are most suited to fill. In this project, we aimed to compare the most recent version of ChatGPT, ChatGPT-4, and Bard by Google, in their ability to accurately respond to radiology board examination practice questions. METHODS: Text-based questions were collected from the 2017-2021 American College of Radiology's Diagnostic Radiology In-Training (DXIT) examinations. ChatGPT-4 and Bard were queried, and their comparative accuracies, response lengths, and response times were documented. Subspecialty-specific performance was analyzed as well. RESULTS: 318 questions were included in our analysis. ChatGPT answered significantly more accurately than Bard (87.11% vs 70.44%, P < .0001). ChatGPT's response length was significantly shorter than Bard's (935.28 ± 440.88 characters vs 1437.52 ± 415.91 characters, P < .0001). ChatGPT's response time was significantly longer than Bard's (26.79 ± 3.27 seconds vs 7.55 ± 1.88 seconds, P < .0001). ChatGPT performed superiorly to Bard in neuroradiology, (100.00% vs 86.21%, P = .03), general & physics (85.39% vs 68.54%, P < .001), nuclear medicine (80.00% vs 56.67%, P < .01), pediatric radiology (93.75% vs 68.75%, P = .03), and ultrasound (100.00% vs 63.64%, P < .001). In the remaining subspecialties, there were no significant differences between ChatGPT and Bard's performance. CONCLUSION: ChatGPT displayed superior radiology knowledge compared to Bard. While both chatbots display reasonable radiology knowledge, they should be used with conscious knowledge of their limitations and fallibility. Both chatbots provided incorrect or illogical answer explanations and did not always address the educational content of the question.

MRI vs. CT for pancreatic adenocarcinoma vascular invasion: comparative diagnostic test accuracy systematic review and meta-analysis.

OBJECTIVES: To perform a systematic review comparing the diagnostic accuracy of MRI vs. CT for assessing pancreatic ductal adenocarcinoma (PDAC) vascular invasion. METHODS: MEDLINE, EMBASE, Cochrane Central, and Scopus were searched until December 2021 for diagnostic accuracy studies comparing MRI vs. CT to evaluate vascular invasion of pathologically confirmed PDAC in the same patients. Findings on resection or exploratory laparotomy were the preferred reference standard. Data extraction, risk of bias, and applicability assessment were performed by two authors using the Quality Assessment of Diagnostic Accuracy Studies-Comparative Tool. Bivariate random-effects meta-analysis and meta-regression were performed with 95% confidence intervals (95% CI). RESULTS: Three studies were included assessing 474 vessels without vascular invasion and 65 with vascular invasion in 107 patients. All patients were imaged using MRI at ≥ 1.5 T and a pancreatic protocol CT. No difference was shown between MRI and CT for diagnosing PDAC vascular invasion: MRI/CT sensitivity (95% CI) were 71% (47-87%)/74% (56-86%), and specificity were 97% (94-99%)/97% (94-98%). Sources of bias included selection bias from only a subset of CT patients undergoing MRI and verification bias from patients with unresectable disease not confirmed on surgery. No patients received neoadjuvant therapy prior to staging. CONCLUSIONS: Based on limited data, no difference was observed between MRI and pancreatic protocol CT for PDAC vascular invasion assessment. MRI may be an adequate substitute for pancreatic protocol CT in some patients, particularly those who have already had a single-phase CT. Larger and more recent cohort studies at low risk of bias, including patients who have received neoadjuvant therapy, are needed. CLINICAL RELEVANCE STATEMENT: Abdominal MRI performed similarly to pancreatic protocol CT at assessing pancreatic ductal adenocarcinoma vascular invasion, suggesting local staging is adequate in some patients using MRI. More data are needed using larger, more recent cohorts including patients with neoadjuvant treatment. KEY POINTS: • Based on limited data, no difference was found between MRI and pancreatic protocol CT sensitivity and specificity for diagnosing PDAC vascular invasion (p = 0.81, 0.73 respectively). • Risk of bias could be reduced in future PDAC MRI vs CT comparative diagnostic test accuracy research by ensuring all enrolled patients undergo both imaging modalities being compared in random order and regardless of the findings on either modality. • More studies are needed that directly compare the diagnostic performance of MRI and CT for PDAC staging after neoadjuvant therapy.

Pancreatic ductal adenocarcinoma (PDAC) regional nodal disease at standard lymphadenectomy: is MRI accurate for identifying node-positive patients?

OBJECTIVE: To determine the accuracy of qualitative and quantitative MRI features for the diagnosis of pathologic regional lymph nodes at standard lymphadenectomy in patients with pancreatic ductal adenocarcinoma (PDAC). METHODS: All adult patients with pancreatic MRI performed from 2011 to 2021 within 3 months of a pancreaticoduodenectomy were eligible for inclusion in this single-center retrospective cohort study. Regional nodes at standard lymphadenectomy were independently reviewed by two fellowship-trained abdominal radiologists for the following qualitative features: heterogeneous T2 signal, round shape, indistinct margin, peri-nodal fat stranding, and restricted diffusion greater than the spleen. Quantitative characteristics including primary tumor size, largest node short- and long-axes length, number of regional nodes, absolute apparent diffusion coefficient (ADC) values, and ADC node-to-spleen signal index were assessed. Analysis was at the patient-level with surgical pathology as the reference standard. RESULTS: Of 75 patients, 85% (64/75) were positive for regional nodal disease on histopathology. None of the qualitative variables evaluated on MRI was associated with pathologic nodes. Median primary tumor maximum diameter was slightly larger for patients with pathologic nodes compared to those without (18 mm (10-42 mm) vs 16 mm (9-22 mm), p = 0.027). None of the other quantitative features was associated with pathologic nodes. Radiologist opinion was not associated with pathologic nodes (p = 0.520). Interobserver agreement was fair (kappa = 0.257). CONCLUSIONS: Lymph node morphologic features and radiologist opinion using MRI are of limited value for diagnosing PDAC regional nodal disease. Improved diagnostic techniques are needed given the prognostic implications of pathologic lymph nodes in these patients. KEY POINTS: • Multiple lymph node morphologic features routinely assessed on MRI for malignancies elsewhere in the body are likely not applicable when assessing for pancreatic ductal adenocarcinoma nodal disease. • Interobserver agreement for the presence or absence of pancreatic ductal adenocarcinoma lymph node morphologic features on MRI is fair (kappa = 0.257). • Many more lymph nodes are resected at PDAC standard lymphadenectomy than are detectable on MRI, median 25 vs 5 (p < 0.001), suggesting improved diagnostic techniques are needed to identify PDAC nodal disease.

Checklist for Artificial Intelligence in Medical Imaging Reporting Adherence in Peer-Reviewed and Preprint Manuscripts With the Highest Altmetric Attention Scores: A Meta-Research Study.

Purpose: To establish reporting adherence to the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) in diagnostic accuracy AI studies with the highest Altmetric Attention Scores (AAS), and to compare completeness of reporting between peer-reviewed manuscripts and preprints. Methods: MEDLINE, EMBASE, arXiv, bioRxiv, and medRxiv were retrospectively searched for 100 diagnostic accuracy medical imaging AI studies in peer-reviewed journals and preprint platforms with the highest AAS since the release of CLAIM to June 24, 2021. Studies were evaluated for adherence to the 42-item CLAIM checklist with comparison between peer-reviewed manuscripts and preprints. The impact of additional factors was explored including body region, models on COVID-19 diagnosis and journal impact factor. Results: Median CLAIM adherence was 48% (20/42). The median CLAIM score of manuscripts published in peer-reviewed journals was higher than preprints, 57% (24/42) vs 40% (16/42), P < .0001. Chest radiology was the body region with the least complete reporting (P = .0352), with manuscripts on COVID-19 less complete than others (43% vs 54%, P = .0002). For studies published in peer-reviewed journals with an impact factor, the CLAIM score correlated with impact factor, rho = 0.43, P = .0040. Completeness of reporting based on CLAIM score had a positive correlation with a study's AAS, rho = 0.68, P < .0001. Conclusions: Overall reporting adherence to CLAIM is low in imaging diagnostic accuracy AI studies with the highest AAS, with preprints reporting fewer study details than peer-reviewed manuscripts. Improved CLAIM adherence could promote adoption of AI into clinical practice and facilitate investigators building upon prior works.

Pancreatic Cystic Lesions on MRI: What Is The Likelihood of a Present or Future Diagnosis of Pancreatic Carcinoma?

BACKGROUND: Pancreatic cystic lesions (PCLs) are followed for years due to older and likely biased works demonstrating a strong association with pancreatic carcinoma; more recent data are needed clarifying this relationship. PURPOSE: To determine the association between PCLs on MRI and a synchronous or future diagnosis of pancreatic carcinoma. STUDY TYPE: Single-center retrospective cohort. POPULATION: A total of 192 patients (111 female, 58%) with median age 66 years (range 26-87 years) with PCLs on abdominal MRI from 2011 to 2016. FIELD STRENGTH/SEQUENCES: 1.5 T and 3 T, including T2 WI, T1 WI, diffusion weighted imaging and contrast-enhanced T1 WI. ASSESSMENT: Each PCL was reviewed independently by 2 of 10 fellowship-trained abdominal radiologists. Fukuoka guideline worrisome features and high-risk stigmata were evaluated. Follow-up imaging and clinical notes were reviewed within a system that captures pancreatic carcinoma for the region, for a median follow-up of 67 months (interquartile range: 43-88 months). STATISTICAL TESTS: Pancreatic carcinoma prevalence and incidence rate for future carcinoma with 95% confidence intervals (95% CI). Fisher exact test, logistic regression with odds ratios (OR) and the Wilcoxon rank-sum test were used to assess PCL morphologic features with the Kolmogorov-Smirnov test used to assess for normality. P < 0.05 defined statistical significance. RESULTS: The prevalence of pancreatic carcinoma on initial MRI showing a PCL was 2.4% (95% CI: 0.9%, 5.2%). Thickened/enhancing cyst wall was associated with pancreatic carcinoma, OR 52 (95% CI: 4.5, 1203). Of 189 patients with a PCL but without pancreatic carcinoma at the time of initial MRI, one developed high-grade dysplasia and none developed invasive carcinoma for an incidence rate of 0.97 (95% CI: 0.02, 5.43) and 0 (95% CI: 0, 3.59) cases per 1000 person-years, respectively. DATA CONCLUSION: A low percentage of patients with a PCL on MRI had a pancreatic carcinoma at the time of initial evaluation and none developed carcinoma over a median 67 months of follow-up. EVIDENCE LEVEL: 3 TECHNICAL EFFICACY: 5.