Evaluating GPT-V4 (GPT-4 with Vision) on Detection of Radiologic Findings on Chest Radiographs.

Background Generating radiologic findings from chest radiographs is pivotal in medical image analysis. The emergence of OpenAI's generative pretrained transformer, GPT-4 with vision (GPT-4V), has opened new perspectives on the potential for automated image-text pair generation. However, the application of GPT-4V to real-world chest radiography is yet to be thoroughly examined. Purpose To investigate the capability of GPT-4V to generate radiologic findings from real-world chest radiographs. Materials and Methods In this retrospective study, 100 chest radiographs with free-text radiology reports were annotated by a cohort of radiologists, two attending physicians and three residents, to establish a reference standard. Of 100 chest radiographs, 50 were randomly selected from the National Institutes of Health (NIH) chest radiographic data set, and 50 were randomly selected from the Medical Imaging and Data Resource Center (MIDRC). The performance of GPT-4V at detecting imaging findings from each chest radiograph was assessed in the zero-shot setting (where it operates without prior examples) and few-shot setting (where it operates with two examples). Its outcomes were compared with the reference standard with regards to clinical conditions and their corresponding codes in the International Statistical Classification of Diseases, Tenth Revision (ICD-10), including the anatomic location (hereafter, laterality). Results In the zero-shot setting, in the task of detecting ICD-10 codes alone, GPT-4V attained an average positive predictive value (PPV) of 12.3%, average true-positive rate (TPR) of 5.8%, and average F1 score of 7.3% on the NIH data set, and an average PPV of 25.0%, average TPR of 16.8%, and average F1 score of 18.2% on the MIDRC data set. When both the ICD-10 codes and their corresponding laterality were considered, GPT-4V produced an average PPV of 7.8%, average TPR of 3.5%, and average F1 score of 4.5% on the NIH data set, and an average PPV of 10.9%, average TPR of 4.9%, and average F1 score of 6.4% on the MIDRC data set. With few-shot learning, GPT-4V showed improved performance on both data sets. When contrasting zero-shot and few-shot learning, there were improved average TPRs and F1 scores in the few-shot setting, but there was not a substantial increase in the average PPV. Conclusion Although GPT-4V has shown promise in understanding natural images, it had limited effectiveness in interpreting real-world chest radiographs. © RSNA, 2024 Supplemental material is available for this article.

What Physicians and Health Organizations Should Know About Mandated Imaging Appropriate Use Criteria.

The Appropriate Use Criteria Program, enacted by the Centers for Medicare & Medicaid Services in response to the Protecting Access to Medicare Act of 2014 (PAMA), aims to reduce inappropriate and unnecessary imaging by mandating use of clinical decision support (CDS) by all providers who order advanced imaging examinations (magnetic resonance imaging; computed tomography; and nuclear medicine studies, including positron emission tomography). Beginning 1 January 2020, documentation of an interaction with a certified CDS system using approved appropriate use criteria will be required on all Medicare claims for advanced imaging in all emergency department patients and outpatients as a prerequisite for payment. The Appropriate Use Criteria Program will initially cover 8 priority clinical areas, including several (such as headache and low back pain) commonly encountered by internal medicine providers. All providers and organizations that order and provide advanced imaging must understand program requirements and their options for compliance strategies. Substantial resources and planning will be needed to comply with PAMA regulations and avoid unintended negative consequences on workflow and payments. However, robust evidence supporting the desired outcome of reducing inappropriate use of advanced imaging is lacking.

Medicare Imaging Demonstration: Assessing Attributes of Appropriate Use Criteria and Their Influence on Ordering Behavior.

OBJECTIVE: Persistent concern exists about the variable and possibly inappropriate utilization of high-cost imaging tests. The purpose of this study is to assess the influence of appropriate use criteria attributes on altering ambulatory imaging orders deemed inappropriate. MATERIALS AND METHODS: This secondary analysis included Medicare Imaging Demonstration data collected from three health care systems in 2011-2013 via the use of clinical decision support (CDS) during ambulatory imaging order entry. The CDS system captured whether orders were inappropriate per the appropriate use criteria of professional societies and provided advice during the intervention period. For orders deemed inappropriate, we assessed the impact of the availability of alternative test recommendations, conflicts with local best practices, and the strength of evidence for appropriate use criteria on the primary outcome of cancellation or modification of inappropriate orders. Expert review determined conflicts with local best practices for 250 recommendations for abdominal and thoracic CT orders. Strength of evidence was assessed for the 15 most commonly triggered recommendations that were deemed inappropriate. A chi-square test was used for univariate analysis. RESULTS: A total of 1691 of 63,222 imaging test orders (2.7%) were deemed inappropriate during the intervention period; this amount decreased from 364 of 11,675 test orders (3.1%) in the baseline period (p < 0.00001). Of 270 inappropriate recommendations with alternative test recommendations, 28 (10.4%) were modified, compared with four of 1024 inappropriate recommendations without alternatives (0.4%) (p < 0.0001). Seventy-eight of 250 recommendations (31%) conflicted with local best practices, but only six of 69 inappropriate recommendations (9%) conflicted (p < 0.001). No inappropriate recommendations that conflicted with local best practices were modified. All 15 commonly triggered recommendations had an Oxford Centre for Evidence-Based Medicine level of evidence of 5 (i.e., expert opinion). CONCLUSION: Orders for imaging tests that were deemed inappropriate were modified infrequently, more often with alternative recommendations present and only for appropriate use criteria consistent with local best practices.

JOURNAL CLUB: Predictors of Provider Response to Clinical Decision Support: Lessons Learned From the Medicare Imaging Demonstration.

OBJECTIVE: The efficacy of imaging clinical decision support (CDS) varies. Our objective was to identify CDS factors contributing to imaging order cancellation or modification. SUBJECTS AND METHODS: This pre-post study was performed across four institutions participating in the Medicare Imaging Demonstration. The intervention was CDS at order entry for selected outpatient imaging procedures. On the basis of the information entered, computerized alerts indicated to providers whether orders were not covered by guidelines, appropriate, of uncertain appropriateness, or inappropriate according to professional society guidelines. Ordering providers could override or accept CDS. We considered actionable alerts to be those that could generate an immediate order behavior change in the ordering physician (i.e., cancellation of inappropriate orders or modification of orders of uncertain appropriateness that had a recommended alternative). Chi-square and logistic regression identified predictors of order cancellation or modification after an alert. RESULTS: A total of 98,894 radiology orders were entered (83,114 after the intervention). Providers ignored 98.9%, modified 1.1%, and cancelled 0.03% of orders in response to alerts. Actionable alerts had a 10 fold higher rate of modification (8.1% vs 0.7%; p < 0.0001) or cancellation (0.2% vs 0.02%; p < 0.0001) orders compared with nonactionable alerts. Orders from institutions with preexisting imaging CDS had a sevenfold lower rate of cancellation or modification than was seen at sites with newly implemented CDS (1.4% vs 0.2%; p < 0.0001). In multivariate analysis, actionable alerts were 12 times more likely to result in order cancellation or modification. Orders at sites with preexisting CDS were 7.7 times less likely to be cancelled or modified (p < 0.0001). CONCLUSION: Using results from the Medicare Imaging Demonstration project, we identified potential factors that were associated with CDS effect on provider imaging ordering; these findings may have implications for future design of such computerized systems.

Imaging Findings in Elder Abuse: A Role for Radiologists in Detection.

PURPOSE: Emergency department assessment represents a critical but often missed opportunity to identify elder abuse, which is common and has serious consequences. Among emergency care providers, diagnostic radiologists are optimally positioned to raise suspicion for mistreatment when reviewing imaging of geriatric injury victims. However, little literature exists describing relevant injury patterns, and most radiologists currently receive neither formal nor informal training in elder abuse identification. METHODS: We present 2 cases to begin characterisation of the radiographic findings in elder abuse. RESULTS: Findings from these cases demonstrate similarities to suspicious findings in child abuse including high-energy fractures that are inconsistent with reported mechanisms and the coexistence of acute and chronic injuries. Specific injuries uncommon to accidental injury are also noted, including a distal ulnar diaphyseal fracture. CONCLUSIONS: We hope to raise awareness of elder abuse among diagnostic radiologists to encourage future large-scale research, increased focus on chronic osseous findings, and the addition of elder abuse to differential diagnoses.

Regional analysis of femoral head perfusion following displaced fractures of the femoral neck.

PURPOSE: To assess regional variations in the arterial and venous blood supply to the femoral head following displaced fracture of the femoral neck using dynamic contrast enhanced (DCE)-MRI quadrant analysis. MATERIALS AND METHODS: A total of 27 subjects with displaced femoral neck fractures were enrolled in the study. Quadrant specific DCE-MRI perfusion analysis was performed on a 1.5 Tesla MRI scanner. Simultaneous imaging of control and displaced fractured hips was done for comparison. RESULTS: Quadrant specific decreases were found in the arterial (A (0.52 versus 0.27; P = 5.7E-13), Akep (1.0/min(-1) versus 0.41/min(-1) ; P = 1.3E-9) and venous (kel (0.05/min(-1) versus -0.02/min(-1) ; P = 5.1E-5) supply to the femoral head between control and injured sides using a two-factor analysis of variance test. The fractional perfusion (initial area under the curve) in the supero/inferolateral quadrants was 49% min/54% min, in the supero/inferomedial quadrants was 43% min/46% min and for the total femoral head was 39% min on the fracture versus control sides. CONCLUSION: Quadrant specific decreases in arterial and venous perfusion on the fracture side were observed when compared with control.

Manual Stress Ankle Radiography Has Poor Ability to Predict Deep Deltoid Ligament Integrity in a Supination External Rotation Fracture Cohort.

Stress ankle radiographs are routinely performed to determine deep deltoid ligament integrity in supination external rotation (SER) ankle fractures. However, variability is present in the published data regarding what medial clear space (MCS) value constitutes a positive result. The purposes of the present study were to evaluate the diagnostic accuracy of different MCS cutoff values and determine whether this clinical test could accurately discriminate between patients with and without a deep deltoid ligament disruption. MCS measurements were recorded for stress ankle injury radiographs in an SER ankle fracture cohort. Preoperative ankle magnetic resonance imaging studies, obtained for all patients, were then read independently by 2 musculoskeletal attending radiologists to determine deep deltoid ligament integrity. The MCS measurements were compared with the magnetic resonance imaging diagnosis using receiver operating characteristic analyses to determine the sensitivity, specificity, and optimal data-driven cutoff values. SER II-III patients demonstrated a mean stress MCS distance of 4.3 ± 0.98 mm compared with 5.8 ± 1.76 mm in the SER IV cohort (p < .001). An analysis of differing MCS positive cutoff thresholds revealed that a stress MCS of 5.0 mm maximized the combined sensitivity and specificity of the external rotation test: 65.8% sensitive and 76.5% specific. Using the receiver operating characteristic curve analysis of the MCS measurement, the calculated area under the curve was 0.77, indicating inadequate discriminative ability for diagnosing SER pattern fractures with or without a deep deltoid ligament tear. Judicious use of additional diagnostic testing in patients with a stress MCS result between 4.0 mm and 5.5 mm is warranted.

Ten commandments for effective clinical decision support for imaging: enabling evidence-based practice to improve quality and reduce waste.

OBJECTIVE: We describe best practices for effective imaging clinical decision support (CDS) derived from firsthand experience, extending the Ten Commandments for CDS published a decade ago. Our collective perspective is used to set expectations for providers, health systems, policy makers, payers, and health information technology developers. CONCLUSION: Highlighting unique attributes of effective imaging CDS will help radiologists to successfully lead and optimize the value of the substantial federal and local investments in health information technology in the United States.

Patient-Centered Radiology Communications: Engaging Patients as Partners.

Patient-centered care is a model in which, by bringing the patient's perspective to the design and delivery of health care, we can better meet patients' needs, enhancing the quality of care. Patient-centered care requires finding ways to communicate effectively with a diverse patient population that has various levels of health literacy, cultural backgrounds, and unique needs and preferences. Moreover, multimedia resources have the potential to inform and educate patients promoting greater independence. In this review, we discuss the fundamentals of communication with the different modes used in radiology and the key elements of effective communication. Then, we highlight five opportunities along the continuum of care in the radiology practice in which we can improve communications to empower our patients and families and strengthen this partnership. Lastly, we discuss the importance on communication training of the workforce, optimizing and seamlessly integrating technology solutions into our workflows, and the need for patient feedback in the design and delivery of care.

Clinical demand for chest/abdomen/pelvis anatomy following thoracic or lumbar spine CT.

The purpose of this study is to determine how often CT is repeated to obtain chest/abdomen/pelvis data outside the reconstructed field of view (FOV) on a prior spine CT. Radiology records of 1,239 consecutive thoracic and lumbar spine CT exams of 1,025 patients from January 1, 2006 to December 31, 2008 were retrospectively reviewed to identify patients who subsequently had CT studies of the chest, abdomen, and/or pelvis. The CT data were also evaluated for contrast enhancement, slice thickness, radiation dose, and reason for subsequent CT exam. Over 3 years, 290 of the 1,239 (24%) spine CT exams were followed by CT of the same anatomic region to evaluate extraspinal anatomy. The use or nonuse of contrast in these follow-up studies was the same as the preceding spine study in 91 cases, which were repeated on the same day (n = 37), within 7 days (n = 19), within 8-30 days (n = 15), or after 30 days (n = 20). Fourteen of 25 (56%) T spine CTs and 34 of 52 (65%) L spine CTs without contrast were followed by a chest CT or abdomen/pelvis CT without contrast within 7 days, respectively. Among 31 pediatric exams, 6 of 31 (19%) spine CTs were followed by a CT of the same anatomic region, all within 7 days. Reconstructing full FOV images of spine CT scans in addition to the standard coned down spine FOV may reduce redundant CT imaging and radiation dose.

Secondary review reduced inpatient MRI orders and avoidable hospital days.

INTRODUCTION: Medical centers have dramatically increased the use of magnetic resonance imaging (MRI). At 2 large academic tertiary care centers in New York City, nearly half of inpatient MRI orders took more than 12 h to complete, delaying patient discharge and increasing avoidable hospital days. We posited that transitioning inpatient MRIs to outpatient facilities, when safe and appropriate, could reduce inpatient MRI orders and avoidable hospital days. METHODS: We manually reviewed 59 inpatient MRI orders delayed on the estimated date of discharge (EDD). These orders were often delayed due to no standard process to escalate orders for medical reasons or no system to coordinate outpatient orders. We developed a revised workflow involving an automation platform that flagged inpatient MRI orders requested within 24 h of the EDD and emailed the care team to request a second review of the order. The care team reconsidered whether the order was (1) required for discharge, (2) non-urgent and could be converted to an outpatient order, or (3) unnecessary and could be canceled. RESULTS: Over 9 months, the automation platform flagged 618 inpatient MRI orders, of which 53.9% (333/618) were reviewed by the care team. Among the orders, 24.0% (80/333) of reviewed orders and 12.9% (80/618) of all orders were transitioned to either outpatient or canceled orders. These transitioned orders were associated with 267 fewer avoidable hospital days and a cost savings of $199,194. CONCLUSION: A standardized process and second review of inpatient MRI orders on the EDD can reduce inappropriate orders and more effectively use inpatient imaging resources. PRECIS: A standardized workflow and automation platform encouraged a second review of inpatient MRI orders to reduce inappropriate orders, avoidable hospital days, and hospital costs.

Adding value in the era of COVID-19: Increasing usage of a patient-centered radiology consultation service.

OBJECTIVE: To examine the effects of COVID-19 pandemic on our department's Radiology Consultation Service (RCS) related to breast imaging, and how utilization of the provided services may have differed as compared to prior to the pandemic. MATERIALS AND METHODS: A retrospective cohort study of patients and health care providers who consulted the RCS, as well as those patients who had a screening mammogram and/or ultrasound between January 1, 2019 and September 1, 2020. Consultations were performed by an RRA, RN and one of 17 breast imaging radiologists assigned to consults on daily. Descriptive statistics were performed to describe the study subject population. RESULTS: Between January 1, 2020 and July 31, 2020, a total of 1623 consultations were performed, in comparison to the control period from the year prior (January 1, 2019 to July 31, 2019), when a total of 1398 consultations were performed, representing a 16% increase in one year. Between March 1, 2020 and June 30, 2020, a total of 679 consultations were performed, in comparison to the control period from the year prior (March 1, 2019 to June 30, 2019), when 583 consultations were performed, representing a 16.5% increase in a four-month period. 350 out of 679 (36.8%) consultations addressed COVID concerns. CONCLUSIONS: While much of radiology experienced an unprecedented decrease in imaging studies during the initial peak of COVID-19 crisis, the RCS at our institution showed a significant increase in services provided, evolving to address pressing concerns related to COVID-19.

Unscheduled Radiologic Examination Orders in the Electronic Health Record: A Novel Resource for Targeting Ambulatory Diagnostic Errors in Radiology.

PURPOSE: The aim of this study was to assess the prevalence of unscheduled radiologic examination orders in an electronic health record, and the proportion of unscheduled orders that are clinically necessary, by modality. METHODS: This retrospective study was conducted from January to October 2016 at an academic institution. All unscheduled radiologic examination orders were retrieved for seven modalities (CT, MR, ultrasound, obstetric ultrasound, bone densitometry, mammography, and fluoroscopy). After excluding duplicates, 100 randomly selected orders from each modality were assigned to two physician reviewers who classified their clinical necessity, with 10% overlap. Interannotator agreement was assessed using κ statistics, the percentage of clinically necessary unscheduled orders was compared, and χ2 analysis was used to assess differences by modality. RESULTS: A total 494,503 radiologic examination orders were placed during the study period. After exclusions, 33,546 unscheduled orders were identified, 7% of all radiologic examination orders. Among 700 reviewed unscheduled orders, agreement was substantial (κ = 0.63). Eighty-seven percent of bone densitometric examinations and sixty-five percent of mammographic studies were considered clinically necessary, primarily for follow-up management. The majority of orders in each modality were clinically necessary, except for CT, obstetric ultrasound, and fluoroscopy (P < .0001). CONCLUSIONS: Large numbers of radiologic examination orders remain unscheduled in the electronic health record. A substantial portion are clinically necessary, representing potential delays in executing documented provider care plans. Clinically unnecessary unscheduled orders may inadvertently be scheduled and performed. Identifying and performing clinically necessary unscheduled radiologic examination orders may help reduce diagnostic errors related to diagnosis and treatment delays and enhance patient safety, while eliminating clinically unnecessary unscheduled orders will help avoid unneeded testing.

Osteochondral lesions of talus associated with ankle fractures.

BACKGROUND: Residual ankle pain and stiffness is not uncommon after ankle fractures. Proposed etiologies include ligamentous instability, joint arthrosis and osteochondral injuries. We studied the incidence of osteochondral lesions of the talus (OCLT) with various ankle fracture patterns and assessed their impact on functional outcome. MATERIALS AND METHODS: Preoperative MRI of 153 patients with ankle fractures who underwent operative fixation was studied. Ligamentous structures around the ankle and OCLT were assessed by MRI. The OCLT was graded as follows: 0, normal; 1, hyperintense but morphologically intact cartilage; 2, fibrillations or fissures not extending into the bone; 3, cartilage flap or bone exposed; 4, loose undisplaced fragment; 5, displaced fragments. Functional outcome was assessed using Foot and Ankle Outcome Scoring (FAOS) at a minimum of 6 months. Outcome between the OCLT and non OCLT group with similar fracture pattern was compared using Fischer's exact test. RESULTS: There were 26 (17%) associated OCLT; four grade I, five grade II, one grade III, eight grade IV, and eight grade V lesions. Three were associated with supination adduction, 21 with supination external rotation injuries and two with pronation external rotation injuries. In the OCLT and the non OCLT group, the average symptom score, pain score, activities of daily living score, sports/recreation score and quality of life score was 80, 72, 79, 45, 50 and 73, 73, 79, 60, 45, respectively. There was no statistically significant difference between the two groups (p > 0.1). CONCLUSION: Osteochondral lesions were frequently associated with ankle fractures; however they had no significant impact on the functional outcome when associated with ankle fractures.

The Diagnostic Accuracy of Radiographs and Magnetic Resonance Imaging in Predicting Deltoid Ligament Ruptures in Ankle Fractures.

BACKGROUND: Operative indications for supination-external rotation (SER) ankle fractures depend on the integrity of the medial structures. Despite the importance of assessing deep deltoid ligament injuries, the accuracy of common diagnostic tests has not been established. QUESTIONS/PURPOSES: The objective of this study was to compare the ability of injury (non-stress) and stress radiographs and magnetic resonance imaging (MRI) to diagnose deep deltoid ligament ruptures in operative SER ankle fractures. METHODS: Patients were included who underwent surgical fixation of SER ankle fractures and had appropriate injury and manual stress test radiographs, pre-operative ankle MRI, and intra-operative assessment of deep deltoid integrity by direct visualization. The medial clear space (MCS) was considered positive for all values over 5 mm on the injury or stress mortise radiographs. MRI analysis of the deep deltoid ligament injury was performed by blinded fellowship-trained musculoskeletal radiologists. Intra-operative direct visualization and assessment of the deltoid was performed using a direct medial ankle approach at the time of operative fracture fixation. RESULTS: Using intra-operative visualization as the gold standard, MCS measurements and MRI had differing abilities to diagnose a deep deltoid rupture. In cases where the MCS was less than 5 mm on injury radiographs and stress tests were performed, MCS measurements were much less accurate than MRI in predicting deltoid ruptures (46% versus 79%, respectively) with a high false positive rate (80%). In contrast, an MCS measurement of greater than 5 mm on injury radiographs was a strong predictor of deep deltoid rupture (accuracy of 95%). CONCLUSION: Compared with direct visualization of the deltoid ligament intra-operatively, these data support proceeding with surgery when the MCS on injury radiographs is greater than 5 mm without any additional stress tests or advanced imaging. When the MCS is less than 5 mm, we recommend MRI analysis because of its increased accuracy and decreased false positive rate. Improving our ability to diagnose deltoid ruptures will contribute to more effective management of patients with SER ankle fractures.