Evaluating GPT-V4 (GPT-4 with Vision) on Detection of Radiologic Findings on Chest Radiographs.

Background Generating radiologic findings from chest radiographs is pivotal in medical image analysis. The emergence of OpenAI's generative pretrained transformer, GPT-4 with vision (GPT-4V), has opened new perspectives on the potential for automated image-text pair generation. However, the application of GPT-4V to real-world chest radiography is yet to be thoroughly examined. Purpose To investigate the capability of GPT-4V to generate radiologic findings from real-world chest radiographs. Materials and Methods In this retrospective study, 100 chest radiographs with free-text radiology reports were annotated by a cohort of radiologists, two attending physicians and three residents, to establish a reference standard. Of 100 chest radiographs, 50 were randomly selected from the National Institutes of Health (NIH) chest radiographic data set, and 50 were randomly selected from the Medical Imaging and Data Resource Center (MIDRC). The performance of GPT-4V at detecting imaging findings from each chest radiograph was assessed in the zero-shot setting (where it operates without prior examples) and few-shot setting (where it operates with two examples). Its outcomes were compared with the reference standard with regards to clinical conditions and their corresponding codes in the International Statistical Classification of Diseases, Tenth Revision (ICD-10), including the anatomic location (hereafter, laterality). Results In the zero-shot setting, in the task of detecting ICD-10 codes alone, GPT-4V attained an average positive predictive value (PPV) of 12.3%, average true-positive rate (TPR) of 5.8%, and average F1 score of 7.3% on the NIH data set, and an average PPV of 25.0%, average TPR of 16.8%, and average F1 score of 18.2% on the MIDRC data set. When both the ICD-10 codes and their corresponding laterality were considered, GPT-4V produced an average PPV of 7.8%, average TPR of 3.5%, and average F1 score of 4.5% on the NIH data set, and an average PPV of 10.9%, average TPR of 4.9%, and average F1 score of 6.4% on the MIDRC data set. With few-shot learning, GPT-4V showed improved performance on both data sets. When contrasting zero-shot and few-shot learning, there were improved average TPRs and F1 scores in the few-shot setting, but there was not a substantial increase in the average PPV. Conclusion Although GPT-4V has shown promise in understanding natural images, it had limited effectiveness in interpreting real-world chest radiographs. © RSNA, 2024 Supplemental material is available for this article.

Patient-Centered Radiology Communications: Engaging Patients as Partners.

Patient-centered care is a model in which, by bringing the patient's perspective to the design and delivery of health care, we can better meet patients' needs, enhancing the quality of care. Patient-centered care requires finding ways to communicate effectively with a diverse patient population that has various levels of health literacy, cultural backgrounds, and unique needs and preferences. Moreover, multimedia resources have the potential to inform and educate patients promoting greater independence. In this review, we discuss the fundamentals of communication with the different modes used in radiology and the key elements of effective communication. Then, we highlight five opportunities along the continuum of care in the radiology practice in which we can improve communications to empower our patients and families and strengthen this partnership. Lastly, we discuss the importance on communication training of the workforce, optimizing and seamlessly integrating technology solutions into our workflows, and the need for patient feedback in the design and delivery of care.

Secondary review reduced inpatient MRI orders and avoidable hospital days.

INTRODUCTION: Medical centers have dramatically increased the use of magnetic resonance imaging (MRI). At 2 large academic tertiary care centers in New York City, nearly half of inpatient MRI orders took more than 12 h to complete, delaying patient discharge and increasing avoidable hospital days. We posited that transitioning inpatient MRIs to outpatient facilities, when safe and appropriate, could reduce inpatient MRI orders and avoidable hospital days. METHODS: We manually reviewed 59 inpatient MRI orders delayed on the estimated date of discharge (EDD). These orders were often delayed due to no standard process to escalate orders for medical reasons or no system to coordinate outpatient orders. We developed a revised workflow involving an automation platform that flagged inpatient MRI orders requested within 24 h of the EDD and emailed the care team to request a second review of the order. The care team reconsidered whether the order was (1) required for discharge, (2) non-urgent and could be converted to an outpatient order, or (3) unnecessary and could be canceled. RESULTS: Over 9 months, the automation platform flagged 618 inpatient MRI orders, of which 53.9% (333/618) were reviewed by the care team. Among the orders, 24.0% (80/333) of reviewed orders and 12.9% (80/618) of all orders were transitioned to either outpatient or canceled orders. These transitioned orders were associated with 267 fewer avoidable hospital days and a cost savings of $199,194. CONCLUSION: A standardized process and second review of inpatient MRI orders on the EDD can reduce inappropriate orders and more effectively use inpatient imaging resources. PRECIS: A standardized workflow and automation platform encouraged a second review of inpatient MRI orders to reduce inappropriate orders, avoidable hospital days, and hospital costs.

Adding value in the era of COVID-19: Increasing usage of a patient-centered radiology consultation service.

OBJECTIVE: To examine the effects of COVID-19 pandemic on our department's Radiology Consultation Service (RCS) related to breast imaging, and how utilization of the provided services may have differed as compared to prior to the pandemic. MATERIALS AND METHODS: A retrospective cohort study of patients and health care providers who consulted the RCS, as well as those patients who had a screening mammogram and/or ultrasound between January 1, 2019 and September 1, 2020. Consultations were performed by an RRA, RN and one of 17 breast imaging radiologists assigned to consults on daily. Descriptive statistics were performed to describe the study subject population. RESULTS: Between January 1, 2020 and July 31, 2020, a total of 1623 consultations were performed, in comparison to the control period from the year prior (January 1, 2019 to July 31, 2019), when a total of 1398 consultations were performed, representing a 16% increase in one year. Between March 1, 2020 and June 30, 2020, a total of 679 consultations were performed, in comparison to the control period from the year prior (March 1, 2019 to June 30, 2019), when 583 consultations were performed, representing a 16.5% increase in a four-month period. 350 out of 679 (36.8%) consultations addressed COVID concerns. CONCLUSIONS: While much of radiology experienced an unprecedented decrease in imaging studies during the initial peak of COVID-19 crisis, the RCS at our institution showed a significant increase in services provided, evolving to address pressing concerns related to COVID-19.

Unscheduled Radiologic Examination Orders in the Electronic Health Record: A Novel Resource for Targeting Ambulatory Diagnostic Errors in Radiology.

PURPOSE: The aim of this study was to assess the prevalence of unscheduled radiologic examination orders in an electronic health record, and the proportion of unscheduled orders that are clinically necessary, by modality. METHODS: This retrospective study was conducted from January to October 2016 at an academic institution. All unscheduled radiologic examination orders were retrieved for seven modalities (CT, MR, ultrasound, obstetric ultrasound, bone densitometry, mammography, and fluoroscopy). After excluding duplicates, 100 randomly selected orders from each modality were assigned to two physician reviewers who classified their clinical necessity, with 10% overlap. Interannotator agreement was assessed using κ statistics, the percentage of clinically necessary unscheduled orders was compared, and χ2 analysis was used to assess differences by modality. RESULTS: A total 494,503 radiologic examination orders were placed during the study period. After exclusions, 33,546 unscheduled orders were identified, 7% of all radiologic examination orders. Among 700 reviewed unscheduled orders, agreement was substantial (κ = 0.63). Eighty-seven percent of bone densitometric examinations and sixty-five percent of mammographic studies were considered clinically necessary, primarily for follow-up management. The majority of orders in each modality were clinically necessary, except for CT, obstetric ultrasound, and fluoroscopy (P < .0001). CONCLUSIONS: Large numbers of radiologic examination orders remain unscheduled in the electronic health record. A substantial portion are clinically necessary, representing potential delays in executing documented provider care plans. Clinically unnecessary unscheduled orders may inadvertently be scheduled and performed. Identifying and performing clinically necessary unscheduled radiologic examination orders may help reduce diagnostic errors related to diagnosis and treatment delays and enhance patient safety, while eliminating clinically unnecessary unscheduled orders will help avoid unneeded testing.

The Diagnostic Accuracy of Radiographs and Magnetic Resonance Imaging in Predicting Deltoid Ligament Ruptures in Ankle Fractures.

BACKGROUND: Operative indications for supination-external rotation (SER) ankle fractures depend on the integrity of the medial structures. Despite the importance of assessing deep deltoid ligament injuries, the accuracy of common diagnostic tests has not been established. QUESTIONS/PURPOSES: The objective of this study was to compare the ability of injury (non-stress) and stress radiographs and magnetic resonance imaging (MRI) to diagnose deep deltoid ligament ruptures in operative SER ankle fractures. METHODS: Patients were included who underwent surgical fixation of SER ankle fractures and had appropriate injury and manual stress test radiographs, pre-operative ankle MRI, and intra-operative assessment of deep deltoid integrity by direct visualization. The medial clear space (MCS) was considered positive for all values over 5 mm on the injury or stress mortise radiographs. MRI analysis of the deep deltoid ligament injury was performed by blinded fellowship-trained musculoskeletal radiologists. Intra-operative direct visualization and assessment of the deltoid was performed using a direct medial ankle approach at the time of operative fracture fixation. RESULTS: Using intra-operative visualization as the gold standard, MCS measurements and MRI had differing abilities to diagnose a deep deltoid rupture. In cases where the MCS was less than 5 mm on injury radiographs and stress tests were performed, MCS measurements were much less accurate than MRI in predicting deltoid ruptures (46% versus 79%, respectively) with a high false positive rate (80%). In contrast, an MCS measurement of greater than 5 mm on injury radiographs was a strong predictor of deep deltoid rupture (accuracy of 95%). CONCLUSION: Compared with direct visualization of the deltoid ligament intra-operatively, these data support proceeding with surgery when the MCS on injury radiographs is greater than 5 mm without any additional stress tests or advanced imaging. When the MCS is less than 5 mm, we recommend MRI analysis because of its increased accuracy and decreased false positive rate. Improving our ability to diagnose deltoid ruptures will contribute to more effective management of patients with SER ankle fractures.

What Physicians and Health Organizations Should Know About Mandated Imaging Appropriate Use Criteria.

The Appropriate Use Criteria Program, enacted by the Centers for Medicare & Medicaid Services in response to the Protecting Access to Medicare Act of 2014 (PAMA), aims to reduce inappropriate and unnecessary imaging by mandating use of clinical decision support (CDS) by all providers who order advanced imaging examinations (magnetic resonance imaging; computed tomography; and nuclear medicine studies, including positron emission tomography). Beginning 1 January 2020, documentation of an interaction with a certified CDS system using approved appropriate use criteria will be required on all Medicare claims for advanced imaging in all emergency department patients and outpatients as a prerequisite for payment. The Appropriate Use Criteria Program will initially cover 8 priority clinical areas, including several (such as headache and low back pain) commonly encountered by internal medicine providers. All providers and organizations that order and provide advanced imaging must understand program requirements and their options for compliance strategies. Substantial resources and planning will be needed to comply with PAMA regulations and avoid unintended negative consequences on workflow and payments. However, robust evidence supporting the desired outcome of reducing inappropriate use of advanced imaging is lacking.

Integrity of clinical information in computerized order requisitions for diagnostic imaging.

Objective: Assess information integrity (concordance and completeness of documented exam indications from the electronic health record [EHR] imaging order requisition, compared to EHR provider notes), and assess potential impact of indication inaccuracies on exam planning and interpretation. Methods: This retrospective study, approved by the Institutional Review Board, was conducted at a tertiary academic medical center. There were 139 MRI lumbar spine (LS-MRI) and 176 CT abdomen/pelvis orders performed 4/1/2016-5/31/2016 randomly selected and reviewed by 4 radiologists for concordance and completeness of relevant exam indications in order requisitions compared to provider notes, and potential impact of indication inaccuracies on exam planning and interpretation. Forty each LS-MRI and CT abdomen/pelvis were re-reviewed to assess kappa agreement. Results: Requisition indications were more likely to be incomplete (256/315, 81%) than discordant (133/315, 42%) compared to provider notes (p < 0.0001). Potential impact of discrepancy between clinical information in requisitions and provider notes was higher for radiologist's interpretation than for exam planning (135/315, 43%, vs 25/315, 8%, p < 0.0001). Agreement among radiologists for concordance, completeness, and potential impact was moderate to strong (Kappa 0.66-0.89). Indications in EHR order requisitions are frequently incomplete or discordant compared to physician notes, potentially impacting imaging exam planning, interpretation and accurate diagnosis. Such inaccuracies could also diminish the relevance of clinical decision support alerts if based on information in order requisitions. Conclusions: Improved availability of relevant documented clinical information within EHR imaging requisition is necessary for optimal exam planning and interpretation.

Detection of Botulinum Toxin Muscle Effect in Humans Using Magnetic Resonance Imaging: A Qualitative Case Series.

BACKGROUND: Although important for dosing and dilution, there are few data describing botulinum toxin (BT) movement in human muscle. OBJECTIVE: To better understand BT movement within human muscle. DESIGN: Proof-of-concept study with descriptive case series. SETTING: Outpatient academic practice. PARTICIPANTS: Five subjects with stroke who were BT naive with a mean age of 60.4 ± 14 years and time poststroke of 4.6 ± 3.7 years. METHODS: Three standardized injections were given to the lateral gastrocnemius muscle (LGM): 2 contained 25 units (U) of onabotulinumtoxinA (Botox) in 0.25 mL of saline solution and the third 0.25 mL of saline solution only. The tibialis anterior muscle (TAM) was not injected in any subject. A leg magnetic resonance image was obtained at baseline, 2 months, and 3 months later with a 3.0 Tesla Siemens scanner. Three muscles, the LGM, lateral soleus muscle (LSM), and TAM, were manually outlined on the T2 mapping sequence at each time point. A histogram of T2 relaxation times (T2-RT) for all voxels at baseline was used to calculate a mean and standard deviation (SD) T2-RT for each muscle. Botulinum toxin muscle effect (BTME) at 2 months and 3 months was defined as a subject- and muscle-specific T2-RT voxel threshold ≥3 SD above the baseline mean at or near BT injection sites. MAIN OUTCOME MEASURES: BTME volume for each leg magnetic resonance imaging slice at 3 time points and 3 muscles for all subjects. RESULTS: One subject missed the 3-month scan, leaving 18 potential observations of BTME. Little to no BTME effect was seen in the noninjected TAM. A BTME was detected in the LGM in 13 of 18 possible observations, and no effect was detected in 5 observations. Possible BTME effect was seen in the LSM in 3 subjects due to either diffusion through fascia or needle misplacement. Volume of BTME, as defined here, appeared to be substantially greater than the 0.25-mL injection volume. CONCLUSIONS: This descriptive case series is among the first attempts to quantify BTME within human muscle. Our findings are preliminary and are limited by a few inconsistencies. However, we conclude that use of magnetic resonance imaging to detect the volume of BTME is feasible and may assist researchers in modeling the spread and diffusion of BT within human muscle. LEVEL OF EVIDENCE: IV.

Improving Care and Education Through a Radiology Resident-driven Clinical Consultation Service.

RATIONALE AND OBJECTIVE: As health care moves toward bundled payment systems and merit-based incentive models, increasing awareness of the value of the radiologist is essential. A resident-driven clinical imaging rounds (CIR) program initiated at our institution allows radiologists to actively and directly participate in the team-based medical model. A retrospective review of survey data evaluated the qualitative and quantitative effects of CIR on clinical management, communication, and education of referring providers and radiology residents. MATERIALS AND METHODS: The initial 10 months of a resident-organized CIR were evaluated in a retrospective study. Twenty radiology residents and 150 internal medicine physicians and medical students participated in imaging rounds. An anonymous survey of participants was performed and results were analyzed. RESULTS: Eighty-five percent of radiology resident participants completed the survey (N = 17). Approximately 30% of internal medicine participants completed the survey (N = 45). There was an overwhelming positive review of imaging rounds, with a large majority of all groups agreeing that imaging rounds improve education, communication, and patient care. CONCLUSIONS: Resident-driven imaging rounds provide a valuable opportunity to improve communication, education, and patient care. We have created a CIR with a sustainable workflow that allows direct and regularly scheduled imaging-medicine consultation valued by both radiologists and internal medicine physicians, improving the quality of patient care and providing education to our radiology residents in value-based care.

Medicare Imaging Demonstration: Assessing Attributes of Appropriate Use Criteria and Their Influence on Ordering Behavior.

OBJECTIVE: Persistent concern exists about the variable and possibly inappropriate utilization of high-cost imaging tests. The purpose of this study is to assess the influence of appropriate use criteria attributes on altering ambulatory imaging orders deemed inappropriate. MATERIALS AND METHODS: This secondary analysis included Medicare Imaging Demonstration data collected from three health care systems in 2011-2013 via the use of clinical decision support (CDS) during ambulatory imaging order entry. The CDS system captured whether orders were inappropriate per the appropriate use criteria of professional societies and provided advice during the intervention period. For orders deemed inappropriate, we assessed the impact of the availability of alternative test recommendations, conflicts with local best practices, and the strength of evidence for appropriate use criteria on the primary outcome of cancellation or modification of inappropriate orders. Expert review determined conflicts with local best practices for 250 recommendations for abdominal and thoracic CT orders. Strength of evidence was assessed for the 15 most commonly triggered recommendations that were deemed inappropriate. A chi-square test was used for univariate analysis. RESULTS: A total of 1691 of 63,222 imaging test orders (2.7%) were deemed inappropriate during the intervention period; this amount decreased from 364 of 11,675 test orders (3.1%) in the baseline period (p < 0.00001). Of 270 inappropriate recommendations with alternative test recommendations, 28 (10.4%) were modified, compared with four of 1024 inappropriate recommendations without alternatives (0.4%) (p < 0.0001). Seventy-eight of 250 recommendations (31%) conflicted with local best practices, but only six of 69 inappropriate recommendations (9%) conflicted (p < 0.001). No inappropriate recommendations that conflicted with local best practices were modified. All 15 commonly triggered recommendations had an Oxford Centre for Evidence-Based Medicine level of evidence of 5 (i.e., expert opinion). CONCLUSION: Orders for imaging tests that were deemed inappropriate were modified infrequently, more often with alternative recommendations present and only for appropriate use criteria consistent with local best practices.

JOURNAL CLUB: Predictors of Provider Response to Clinical Decision Support: Lessons Learned From the Medicare Imaging Demonstration.

OBJECTIVE: The efficacy of imaging clinical decision support (CDS) varies. Our objective was to identify CDS factors contributing to imaging order cancellation or modification. SUBJECTS AND METHODS: This pre-post study was performed across four institutions participating in the Medicare Imaging Demonstration. The intervention was CDS at order entry for selected outpatient imaging procedures. On the basis of the information entered, computerized alerts indicated to providers whether orders were not covered by guidelines, appropriate, of uncertain appropriateness, or inappropriate according to professional society guidelines. Ordering providers could override or accept CDS. We considered actionable alerts to be those that could generate an immediate order behavior change in the ordering physician (i.e., cancellation of inappropriate orders or modification of orders of uncertain appropriateness that had a recommended alternative). Chi-square and logistic regression identified predictors of order cancellation or modification after an alert. RESULTS: A total of 98,894 radiology orders were entered (83,114 after the intervention). Providers ignored 98.9%, modified 1.1%, and cancelled 0.03% of orders in response to alerts. Actionable alerts had a 10 fold higher rate of modification (8.1% vs 0.7%; p < 0.0001) or cancellation (0.2% vs 0.02%; p < 0.0001) orders compared with nonactionable alerts. Orders from institutions with preexisting imaging CDS had a sevenfold lower rate of cancellation or modification than was seen at sites with newly implemented CDS (1.4% vs 0.2%; p < 0.0001). In multivariate analysis, actionable alerts were 12 times more likely to result in order cancellation or modification. Orders at sites with preexisting CDS were 7.7 times less likely to be cancelled or modified (p < 0.0001). CONCLUSION: Using results from the Medicare Imaging Demonstration project, we identified potential factors that were associated with CDS effect on provider imaging ordering; these findings may have implications for future design of such computerized systems.

Imaging Findings in Elder Abuse: A Role for Radiologists in Detection.

PURPOSE: Emergency department assessment represents a critical but often missed opportunity to identify elder abuse, which is common and has serious consequences. Among emergency care providers, diagnostic radiologists are optimally positioned to raise suspicion for mistreatment when reviewing imaging of geriatric injury victims. However, little literature exists describing relevant injury patterns, and most radiologists currently receive neither formal nor informal training in elder abuse identification. METHODS: We present 2 cases to begin characterisation of the radiographic findings in elder abuse. RESULTS: Findings from these cases demonstrate similarities to suspicious findings in child abuse including high-energy fractures that are inconsistent with reported mechanisms and the coexistence of acute and chronic injuries. Specific injuries uncommon to accidental injury are also noted, including a distal ulnar diaphyseal fracture. CONCLUSIONS: We hope to raise awareness of elder abuse among diagnostic radiologists to encourage future large-scale research, increased focus on chronic osseous findings, and the addition of elder abuse to differential diagnoses.

2D mammography, digital breast tomosynthesis, and ultrasound: which should be used for the different breast densities in breast cancer screening?

PURPOSE: To determine which modalities [2D mammography (2D), digital breast tomosynthesis (DBT), whole breast sonography (WBS)] are optimal for screening depending on breast density. METHODS: Institutional retrospective cohort study of 2013 screening mammograms (16,789), sorted by modalities and density. RESULTS: Cancer detection is increased by adding WBS to 2D (P=.02) for the overall study population. Recall rate was lowest with 2D+DBT (10.2%, P<.001) and highest with 2D+DBT+WBS (23.6%, P<.001) for the overall study population as well. CONCLUSION: Women with dense and nondense breasts benefit from reduced recall rate with the addition of DBT; however, this benefit is negated with the addition of WBS.