Prospective Evaluation of Artificial Intelligence Triage of Incidental Pulmonary Emboli on Contrast-Enhanced CT Examinations of the Chest or Abdomen.

BACKGROUND. Artificial intelligence (AI) algorithms improved detection of incidental pulmonary embolism (IPE) on contrast-enhanced CT (CECT) examinations in retrospective studies; however, prospective validation studies are lacking. OBJECTIVE. The purpose of this study was to assess the effect on radiologists' real-world diagnostic performance and report turnaround times of a radiology department's clinical implementation of an AI triage system for detecting IPE on CECT examinations of the chest or abdomen. METHODS. This prospective single-center study included consecutive adult patients who underwent CECT of the chest or abdomen for reasons other than pulmonary embolism (PE) detection from May 12, 2021, to June 30, 2021 (phase 1), or from September 30, 2021, to December 4, 2021 (phase 2). Before phase 1, the radiology department installed a commercially available AI triage algorithm for IPE detection that automatically processed CT examinations and notified radiologists of positive results through an interactive floating widget. In phase 1, the widget was inactive, and radiologists interpreted examinations without AI assistance. In phase 2, the widget was activated, and radiologists interpreted examinations with AI assistance. A review process involving a panel of radiologists was implemented to establish the reference standard for the presence of IPE. Diagnostic performance and report turnaround times were compared using the Pearson chi-square test and Wilcoxon rank sum test, respectively. RESULTS. Phase 1 included 1467 examinations in 1434 patients (mean age, 53.8 ± 18.5 [SD] years; 753 men, 681 women); phase 2 included 3182 examinations in 2886 patients (mean age, 55.4 ± 18.2 years; 1520 men, 1366 women). The frequency of IPE was 1.4% (20/1467) in phase 1 and 1.6% (52/3182) in phase 2. Radiologists without AI, in comparison to radiologists with AI, showed significantly lower sensitivity (80.0% vs 96.2%, respectively; p = .03), without a significant difference in specificity (99.9% vs 99.9%, p = .58), for the detection of IPE. The mean report turnaround time for IPE-positive examinations was not significantly different between radiologists without AI and radiologists with AI (78.3 vs 74.6 minutes, p = .26). CONCLUSION. An AI triage system improved radiologists' sensitivity for IPE detection on CECT examinations of the chest or abdomen without significant change in report turnaround times. CLINICAL IMPACT. This prospective real-world study supports the use of AI assistance for maximizing IPE detection.

Prospective Evaluation of Artificial Intelligence Triage of Intracranial Hemorrhage on Noncontrast Head CT Examinations.

Background: Retrospective studies evaluating artificial intelligence (AI) algorithms for intracranial hemorrhage (ICH) detection on noncontrast CT (NCCT) have shown promising results but lack prospective validation. Objective: To evaluate the impact on radiologists' real-world aggregate performance for ICH detection and report turnaround times for ICH-positive examinations of a radiology department's implementation of an AI triage and notification system for ICH detection on head NCCT examinations. Methods: This prospective single-center study included adult patients who underwent head NCCT examinations from May 12, 2021 to June 30, 2021 (phase 1) or September 30, 2021 to December 4, 2021 (phase 2). Before phase 1, the radiology department implemented a commercial AI triage system for ICH detection that processed head NCCT examinations and notified radiologists of positive results through a widget with a floating pop-up display. Examinations were interpreted by neuroradiologists or emergency radiologists, who evaluated examinations without and with AI assistance in phase 1 and phase 2, respectively. A panel of radiologists conducted a review process for all examinations with discordance between the radiology report and AI and a subset of remaining examinations, to establish the reference standard. Diagnostic performance and report turnaround times were compared using Pearson chi-square test and Wilcoxon rank-sum test, respectively. Bonferroni correction was used to account for five diagnostic performance metrics (adjusted significance threshold, .01 [α=.05/5]). Results: A total of 9954 examinations from 7371 patients (mean age, 54.8±19.8 years; 3773 female, 3598 male) were included. In phases 1 and 2, 19.8% (735/3716) and 21.9% (1368/6238) of examinations, respectively, were positive for ICH (P=.01). Radiologists without versus with AI showed no significant difference in accuracy (99.5% vs 99.2%), sensitivity (98.6% vs 98.9%), PPV (99.0% vs 99.7%), or NPV (99.7% vs 99.7%) (all P>.01); specificity was higher for radiologists without than with AI (99.8% vs 99.3%, respectively, P=.004). Mean report turnaround time for ICH-positive examinations was 147.1 minutes without AI versus 149.9 minutes with AI (P=.11). Conclusion: An AI triage system for ICH detection did not improve radiologists' diagnostic performance or report turnaround times. Clinical Impact: This large prospective real-world study does not support use of AI assistance for ICH detection.

Prospective Evaluation of AI Triage of Pulmonary Emboli on CT Pulmonary Angiograms.

Background Artificial intelligence (AI) algorithms have shown high accuracy for detection of pulmonary embolism (PE) on CT pulmonary angiography (CTPA) studies in academic studies. Purpose To determine whether use of an AI triage system to detect PE on CTPA studies improves radiologist performance or examination and report turnaround times in a clinical setting. Materials and Methods This prospective single-center study included adult participants who underwent CTPA for suspected PE in a clinical practice setting. Consecutive CTPA studies were evaluated in two phases, first by radiologists alone (n = 31) (May 2021 to June 2021) and then by radiologists aided by a commercially available AI triage system (n = 37) (September 2021 to December 2021). Sixty-two percent of radiologists (26 of 42 radiologists) interpreted studies in both phases. The reference standard was determined by an independent re-review of studies by thoracic radiologists and was used to calculate performance metrics. Diagnostic accuracy and turnaround times were compared using Pearson χ2 and Wilcoxon rank sum tests. Results Phases 1 and 2 included 503 studies (participant mean age, 54.0 years ± 17.8 [SD]; 275 female, 228 male) and 1023 studies (participant mean age, 55.1 years ± 17.5; 583 female, 440 male), respectively. In phases 1 and 2, 14.5% (73 of 503) and 15.9% (163 of 1023) of CTPA studies were positive for PE (P = .47). Mean wait time for positive PE studies decreased from 21.5 minutes without AI to 11.3 minutes with AI (P < .001). The accuracy and miss rate, respectively, for radiologist detection of any PE on CTPA studies was 97.6% and 12.3% without AI and 98.6% and 6.1% with AI, which was not significantly different (P = .15 and P = .11, respectively). Conclusion The use of an AI triage system to detect any PE on CTPA studies improved wait times but did not improve radiologist accuracy, miss rate, or examination and report turnaround times. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Murphy and Tee in this issue.

Ultrasound-Based Grading System for Radiation-Induced Acute Breast Toxicity.

OBJECTIVES: To introduce an ultrasound-based scoring system for radiation-induced breast toxicity and test its reliability. METHODS: Breast ultrasound (BUS) was performed on 32 patients receiving breast radiotherapy (RT) to assess the radiation-induced acute toxicity. For each patient, both the untreated and irradiated breasts were scanned at five locations: 12:00, 3:00, 6:00, 9:00, and tumor bed to evaluate for heterogenous responses to radiation within the entire breast. In total, 314 images were analyzed. Based on ultrasound findings such as skin thickening, dermis boundary irregularity, and subcutaneous edema, a 4-level, Likert-like grading scheme is proposed: none (G0), mild (G1), moderate (G2), and severe (G3) toxicity. Two ultrasound experts graded the severity of breast toxicity independently and reported the inter- and intra-observer reliability of the grading system. Imaging findings were compared with standard clinical toxicity assessments using Common Terminology Criteria for Adverse Events (CTCAE). RESULTS: The inter-observer Pearson correlation coefficient (PCC) was 0.87 (95% CI: 0.83-0.90, P < .001). For intra-observer repeatability, the PCC of the repeated scores was 0.83 (95% CI: 0.78-0.87, P < .001). Imaging findings were compared with standard clinical toxicity assessments using CTCAE scales. The PCC between BUS scores and CTCAE results was 0.62 (95% CI: 0.35-0.80, P < .001). Among all locations, 6:00 and tumor bed showed significantly greater toxicity compared with 12:00 (P = .04). CONCLUSIONS: BUS can investigate the cutaneous and subcutaneous tissue changes after RT. This BUS-based grading system can complement subjective clinical assessments of radiation-induced breast toxicity with cutaneous and subcutaneous sonographic information.

Lessons Learned in Change Management in Deploying Novel Informatics Solutions: Experience Implementing a Point-of-Care Patient Photography System with Radiography.

We describe implementation of a point-of-care system for simultaneous acquisition of patient photographs along with portable radiographs at a large academic hospital. During the implementation process, we observed several technical challenges in the areas of (1) hardware-automatic triggering for photograph acquisition, camera hardware enclosure, networking, and system server hardware and (2) software-post-processing of photographs. Additionally, we also faced cultural challenges involving workflow issues, communication with technologists and users, and system maintenance. We describe our solutions to address these challenges. We anticipate that these experiences will provide useful insights into deploying and iterating new technologies in imaging informatics.

Multiinstitutional U.S. Academic Radiology Perspectives on Inpatient Diagnostic Imaging of Patients With Coronavirus Disease (COVID-19) and Persons Under Investigation.

OBJECTIVE. This article presents the perspectives of radiologists in different sub-specialties at three institutions across the United States regarding inpatient imaging of patients confirmed to have coronavirus disease (COVID-19) and persons under investigation (i.e., patients suspected to have COVID-19). CONCLUSION. The COVID-19 pandemic has prompted radiologists to become aware of imaging findings related to the disease and to develop workflows for the imaging of patients with COVID-19 and persons under investigation, to optimize care for all patients and preserve the health of health care workers.

Rapid Deployment of Home PACS Workstations to Enable Social Distancing in the Coronavirus Disease (COVID-19) Era.

OBJECTIVE. Social distancing is considered an effective mitigation strategy for coronavirus disease (COVID-19), and remote interpretation of radiologic studies is one approach to social distancing within the radiology department. We describe the rapid deployment of home workstations to achieve social distancing in the radiology department at the University of Alabama at Birmingham. CONCLUSION. Transitioning from on-site interpretation to remote interpretation requires a careful balancing of hospital and departmental finances, engineering choices, and educational and philosophical workflow issues.

Initial Experience With Patient Visible Light Images Obtained Simultaneously With Portable Radiographs.

OBJECTIVE. Visible light images in the form of point-of-care photographs obtained at the time of medical imaging can be useful for detecting wrong-patient errors and providing image-related clinical context. Our goal was to implement a system to automatically obtain point-of-care patient photographs along with portable radiographs. CONCLUSION. We discuss one academic medical center's initial experience in integrating the system into the clinical workflow and initial use cases ranging from cardiothoracic and abdominal imaging to musculoskeletal imaging, for which such point-of-care photographs were deemed clinically beneficial.

Patient Photographs: Privacy Versus Protection.

OBJECTIVE: Technologies to obtain point-of-care photographs along with medical imaging studies are now available. We discuss the protections that photographs can provide in radiology and the potential privacy and legal issues that can arise with their incorporation. CONCLUSION: Point-of-care photographs that are simultaneously obtained with medical imaging studies can provide biometric identification that enables detection of wrong-patient errors. Photographs also provide image-related clinical context. However, successful implementation of such technologies requires consideration of the privacy and legal issues perceived by stakeholders.

Artificial Intelligence in Cardiothoracic Radiology.

OBJECTIVE. The goal of this article is to examine some of the current cardiothoracic radiology applications of artificial intelligence in general and deep learning in particular. CONCLUSION. Artificial intelligence has been used for the analysis of medical images for decades. Recent advances in computer algorithms and hardware, coupled with the availability of larger labeled datasets, have brought about rapid advances in this field. Many of the more notable recent advances have been in the artificial intelligence subfield of deep learning.

Interpreting Radiographs with Concurrently Obtained Patient Photographs.

A technology for automatically obtaining patient photographs along with portable radiographs was implemented clinically at a large academic hospital. This article highlights several cases in which image-related clinical context, provided by the patient photographs, provided quality control information regarding patient identification, laterality, or position and assisted the radiologist with the interpretation. The information in the photographs can easily minimize unnecessary calls to the patient's nursing staff for clarifications and can lead to new methods of physically assessing patients. Published under a CC BY 4.0 license.

Trainees May Add Value to Patient Care by Decreasing Addendum Utilization in Radiology Reports.

OBJECTIVE: The purpose of this study was to evaluate the impact of trainee involvement and other factors on addendum rates in radiology reports. MATERIALS AND METHODS: This retrospective study was performed in a tertiary care pediatric hospital. From the institutional radiology data repository, we extracted all radiology reports from January 1 to June 30, 2016, as well as trainee (resident or fellow) involvement, imaging modality, patient setting (emergency, inpatient, or outpatient), order status (routine vs immediate), time of interpretation (regular work hours vs off-hours), radiologist's years of experience, and sex. We grouped imaging modalities as advanced (CT, MRI, and PET) or nonadvanced (any modality that was not CT, MRI, or PET) and radiologist experience level as ≤ 20 years or > 20 years. Our outcome measure was the rate of addenda in radiology reports. Statistical analysis was performed using multivariate logistic regression. RESULTS: From 129,033 reports finalized during the study period, 418 (0.3%) had addenda. Reports generated without trainees were 12 times more likely than reports with trainee involvement to have addenda (odds ratio [OR] = 12.2, p < 0.001). Advanced imaging studies were more likely than nonadvanced studies to be associated with addendum use (OR = 4.7, p < 0.001). Reports generated for patients in emergency or outpatient settings had a slightly higher likelihood of addendum use than those in an inpatient setting (OR = 1.5, p = 0.04; and OR = 1.3, p = 0.04, respectively). Routine orders had a slightly higher likelihood of addendum use compared with immediate orders (OR = 1.3, p = 0.01). We found no difference in addendum use by radiologist's sex, radiologist's years of experience, emergency versus outpatient setting, or time of interpretation. CONCLUSION: Trainees may add value to patient care by decreasing addendum rates in radiology reports.

Top 100 cited articles in cardiovascular magnetic resonance: a bibliometric analysis.

BACKGROUND: With limited health care resources, bibliometric studies can help guide researchers and research funding agencies towards areas where reallocation or increase in research activity is warranted. Bibliometric analyses have been published in many specialties and sub-specialties but our literature search did not reveal a bibliometric analysis on Cardiovascular Magnetic Resonance (CMR). The main objective of the study was to identify the trends of the top 100 cited articles on CMR research. METHODS: Web of Science (WOS) search was used to create a database of all English language scientific journals. This search was then cross-referenced with a similar search term query of Scopus® to identify articles that may have been missed on the initial search. Articles were ranked by citation count and screened by two independent reviewers. RESULTS: Citations for the top 100 articles ranged from 178 to 1925 with a median of 319.5. Only 17 articles were cited more than 500 times, and the vast majority (n = 72) were cited between 200-499 times. More than half of the articles (n = 52) were from the United States of America, and more than one quarter (n = 21) from the United Kingdom. More than four fifth (n = 86) of the articles were published between the time period 2000-2014 with only 1 article published before 1990. Circulation and Journal of the American College of Cardiology made up more than half (n = 62) of the list. We found 10 authors who had greater than 5 publications in the list. CONCLUSION: Our study provides an insight on the characteristics and quality of the most highly cited CMR literature, and a list of the most influential references related to CMR.

Stakeholders' Perceptions Regarding the Use of Patient Photographs Integrated with Medical Imaging Studies.

Integrating digital facial photographs of pediatric patients as identifiers (ID) with medical imaging (integrated photographic IDs) may increase the detection of mislabeled studies. The purpose of this study was to determine how different stakeholders would receive this novel technology. Parents or guardians of patients in a children's hospital outpatient radiology department, radiology faculty and residents, and radiology technologists and nurses were asked to complete a survey. The perception about the anticipated use of integrated photographic ID in different clinical scenarios was investigated, and its predictors were determined using logistic regression analysis. Four hundred ninety-eight parents responded (response rate 83 %); 96 and 97 % supported the use of integrated photographic ID, if it improves the radiologist's imaging interpretation or decreases the rate of mislabeled errors, respectively. Thirty-eight percent were worried that photographic IDs would impact patients' privacy. Ninety-four percent believed that they should be asked for their consent prior to obtaining their child's photograph. Seventy-eight radiologists responded (response rate 39 %); 63 and 59 % believed that the use of integrated photographic ID would result in improvement in accurate interpretation of images and identification of mislabeled patient errors, respectively. Forty-nine percent of radiologists had concern that integrated photographic ID would increase interpretation time. Fifty technologists and nurses responded (response rate 59 %); 71 and 73 % supported the technology if it resulted in more acute interpretation of images and identification of mislabeled patients, respectively. A majority of stakeholders support integrated photographic ID in order to improve safety. A majority of parents believe that consent should be obtained.

JOURNAL CLUB: Evaluation of Near-Miss Wrong-Patient Events in Radiology Reports.

OBJECTIVE: The purpose of this study was to estimate the prevalence of reported near-miss wrong-patient events in radiology at two large academic hospitals and its relation to imaging modality, clinical setting, and time of occurrence. MATERIALS AND METHODS: An institutional imaging report database was searched for reports between January 1, 2009, and May 30, 2013, that contained the phrases "incorrect patient" or "wrong patient." These imaging reports were categorized into either mislabeled or misidentified patient or wrong dictation or report events. The mislabeling-misidentification events involved patients whose images were incorrectly placed in another patient's folder. In wrong dictation or report events, a patient's images were placed in the correct imaging folder, but another patient's images were used in error for dictation of the report. The time to detect each of these events was also evaluated. RESULTS: Overall, 67 eligible reports were identified among 1,717,713 examinations performed during the study period. The estimated event rate was 4 per 100,000 examinations (mislabeling-misidentification, 52%; wrong dictation, 48%). The monthly mean of mislabeling-misidentification events was 0.7 (SD, 0.9) and of wrong dictation events was 0.6 (SD, 0.7). The median time for mislabeling-misidentification reports to be identified was 22 hours and for wrong dictation reports was 0 hours. Portable chest radiography was the modality involved in 69% (24/35) of reported mislabeling-misidentification and 44% (14/32) of wrong dictation events (p = 0.08); 43% (15/35) of mislabeling-misidentification and 28% (9/32) of wrong dictation events occurred during off hours; 63% (22/35) of mislabeling-misidentification and 56% (18/32) of wrong dictation events occurred in the inpatient setting. CONCLUSION: Despite use of the dual-identifier technique mandated by The Joint Commission, the number of near-miss mislabeled patient events for imaging tests and the delay in awareness of these events were substantial, especially for radiography.

Radiographic imaging for patients with contagious infectious diseases: how to acquire chest radiographs of patients infected with the Ebola virus.

OBJECTIVE: Contagious infectious diseases add a new dimension to radiology and pose many unanswered questions. In particular, what is the safest way to image patients with contagious and potentially lethal infectious diseases? Here, we describe protocols used by Emory University to successfully acquire chest radiographs of patients with Ebola virus disease. CONCLUSION: Radiology departments need to develop new protocols for various modalities used in imaging patients with contagious and potentially lethal infectious diseases.

Performance of bedside diagnostic ultrasound in an Ebola isolation unit: the Emory University Hospital experience.

OBJECTIVE: Individuals with Ebola virus disease, a contagious and potentially lethal infection, are now being treated in specialized units in the United States. We describe Emory University's initial experience, current operating procedures, and ongoing planning with diagnostic ultrasound in the isolation unit. CONCLUSION: Ultrasound use has been limited to date. Future planning considerations include deciding what types of ultrasound studies will be performed, which personnel will acquire the images, and which ultrasound machine will be used.

Improvement in Detection of Wrong-Patient Errors When Radiologists Include Patient Photographs in Their Interpretation of Portable Chest Radiographs.

This study was conducted to determine whether facial photographs obtained simultaneously with radiographs improve radiologists' detection rate of wrong-patient errors, when they are explicitly asked to include the photographs in their evaluation. Radiograph-photograph combinations were obtained from 28 patients at the time of portable chest radiography imaging. From these, pairs of radiographs were generated. Each unique pair consisted of one new and one old (comparison) radiograph. Twelve pairs of mismatched radiographs (i.e., pairs containing radiographs of different patients) were also generated. In phase 1 of the study, 5 blinded radiologist observers were asked to interpret 20 pairs of radiographs without the photographs. In phase 2, each radiologist interpreted another 20 pairs of radiographs with the photographs. Radiologist observers were not instructed about the purpose of the photographs but were asked to include the photographs in their review. The detection rate of mismatched errors was recorded along with the interpretation time for each session for each observer. The two-tailed Fisher exact test was used to evaluate differences in mismatch detection rates between the two phases. A p value of <0.05 was considered significant. The error detection rates without (0/20 = 0%) and with (17/18 = 94.4%) photographs were different (p = 0.0001). The average interpretation times for the set of 20 radiographs were 26.45 (SD 8.69) and 20.55 (SD 3.40) min, for phase 1 and phase 2, respectively (two-tailed Student t test, p = 0.1911). When radiologists include simultaneously obtained photographs in their review of portable chest radiographs, there is a significant improvement in the detection of labeling errors. No statistically significant difference in interpretation time was observed. This may lead to improved patient safety without affecting radiologists' throughput.

A novel technology for automatically obtaining digital facial photographs near-simultaneously with portable radiographs.

Our objective is to design, implement, and phantom-test a device to automatically obtain point-of-care patient photographs along with portable radiographs. Such photographs could help with detection of wrong-patient errors. Our device consists of a camera controller (CC) and a camera that can be mounted on a portable conventional radiography (CR) machine. Radiation from the CR machine triggers an identification module (IM) embedded in the CR cassette. The IM then sends the cassette identifier--Plate_ID--to the CC along with a trigger to activate the camera. This trigger ensures simultaneous acquisition of radiograph and photograph, and the Plate_ID along with a time stamp ensures binding of the two images. We conducted phantom tests to determine if clinical portable radiography exposure settings (90 to 120 kVp and exposure time ranging from 0.63 to 8.0 ms) are sufficient to trigger the IM. Phantom experiments demonstrate acceptable sensor performance for clinical portable radiography exposures. Simultaneous acquisition of photographs is achieved by integrating a low-cost identification module containing a scintillator-detector into the radiographic cassette. Incident X-rays activate the scintillator-detector triggering photograph acquisition by a camera controller.