BI-RADS Category Assignments by GPT-3.5, GPT-4, and Google Bard: A Multilanguage Study.

Background The performance of publicly available large language models (LLMs) remains unclear for complex clinical tasks. Purpose To evaluate the agreement between human readers and LLMs for Breast Imaging Reporting and Data System (BI-RADS) categories assigned based on breast imaging reports written in three languages and to assess the impact of discordant category assignments on clinical management. Materials and Methods This retrospective study included reports for women who underwent MRI, mammography, and/or US for breast cancer screening or diagnostic purposes at three referral centers. Reports with findings categorized as BI-RADS 1-5 and written in Italian, English, or Dutch were collected between January 2000 and October 2023. Board-certified breast radiologists and the LLMs GPT-3.5 and GPT-4 (OpenAI) and Bard, now called Gemini (Google), assigned BI-RADS categories using only the findings described by the original radiologists. Agreement between human readers and LLMs for BI-RADS categories was assessed using the Gwet agreement coefficient (AC1 value). Frequencies were calculated for changes in BI-RADS category assignments that would affect clinical management (ie, BI-RADS 0 vs BI-RADS 1 or 2 vs BI-RADS 3 vs BI-RADS 4 or 5) and compared using the McNemar test. Results Across 2400 reports, agreement between the original and reviewing radiologists was almost perfect (AC1 = 0.91), while agreement between the original radiologists and GPT-4, GPT-3.5, and Bard was moderate (AC1 = 0.52, 0.48, and 0.42, respectively). Across human readers and LLMs, differences were observed in the frequency of BI-RADS category upgrades or downgrades that would result in changed clinical management (118 of 2400 [4.9%] for human readers, 611 of 2400 [25.5%] for Bard, 573 of 2400 [23.9%] for GPT-3.5, and 435 of 2400 [18.1%] for GPT-4; P < .001) and that would negatively impact clinical management (37 of 2400 [1.5%] for human readers, 435 of 2400 [18.1%] for Bard, 344 of 2400 [14.3%] for GPT-3.5, and 255 of 2400 [10.6%] for GPT-4; P < .001). Conclusion LLMs achieved moderate agreement with human reader-assigned BI-RADS categories across reports written in three languages but also yielded a high percentage of discordant BI-RADS categories that would negatively impact clinical management. © RSNA, 2024 Supplemental material is available for this article.

Comparative anatomic and symptomatic recurrence outcomes of diaphragmatic suture cruroplasty versus biosynthetic mesh reinforcement in robotic hiatal and paraesophageal hernia repair.

BACKGROUND: Hiatal and paraesophageal hernia (HH/PEH) recurrence is the most common cause of failure after gastroesophageal anti-reflux surgery. Crural reinforcement with mesh has been suggested to address this issue, but its efficacy remains debated. In this study, we aimed to determine the impact of biosynthetic mesh reinforcement compared to suture cruroplasty on anatomic and symptomatic hernia recurrence. METHOD: Data of patients who underwent robotic HH/PEH repair with suture cruroplasty with or without biosynthetic mesh reinforcement between January 2012 and April 2024 were retrospectively reviewed. Gastroesophageal reflux disease symptoms and anatomic hernia recurrence were assessed at short-term (3 months to 1 year) and longer-term (≥ 1 year) follow-up. Symptomatic hernia recurrence was defined as having both anatomic recurrence and symptoms. RESULTS: Out of the 503 patients in the study, 308 had undergone biosynthetic mesh repair, while 195 had suture-only repair. After the surgery, both groups demonstrated comparable improvements in symptoms. Short-term anatomic hernia recurrence rates were 11.8% and 15.6% for mesh and suture groups, respectively (p = 0.609), while longer-term rates were 24.7% and 44.9% (p = 0.015). The rates of symptomatic hernia recurrence in the same group were 8.8% and 14.6% in the short-term (p = 0.256), and 17.2% and 42.2% in longer-term follow-ups (p = 0.003). In the repair of medium and large-size hernias, mesh reinforcement resulted in a 50.0% relative risk reduction in anatomic hernia recurrences and a 59.2% reduction in symptomatic hernia recurrences at ≥ 1-year follow-up. CONCLUSION: After more than a year of follow-up, it has been found that using biosynthetic mesh for medium and large hiatal or paraesophageal hernia repair significantly reduces the likelihood of both anatomic and symptomatic recurrence compared to using only suture cruroplasty. These findings strongly support the use of biosynthetic mesh to manage larger hernias. However, further long-term multicenter randomized studies are needed to provide more conclusive evidence.

A diagnostic challenge: An incidental lung nodule in a 48-year-old nonsmoker.

A 43-year-old female with a medical history of renal stones, hypertension, diabetes mellitus Type 2, and depression presented to her urologist with bilateral flank pain. She complained of worsening exertional dyspnea over the last several months with recent weight gain. She also endorsed night sweats and intermittent, scant hemoptysis over the past year. She denied fever, chills, nausea, vomiting, diarrhea, constipation, hematuria, or excessive joint or muscle pain. Physical examination was unremarkable. Computed tomography scan of abdomen and pelvis demonstrated bilateral nonobstructing renal stones and a 1.8 cm × 1.7 cm nodular opacity in the right lower lobe of the lung, not present on previous scan 1 year prior. Surgical wedge resection was performed and subsequent pathologic examination demonstrated a 1.2 cm × 0.6 cm × 0.5 cm soft, gelatinous well-demarcated mass in the right lower lobe wedge specimen without gross evidence of necrosis or hemorrhage confirming colloid adenocarcinoma of the lung.