Detection of abdominopelvic lymph nodes in multi-parametric MRI.

Reliable localization of lymph nodes (LNs) in multi-parametric MRI (mpMRI) studies plays a major role in the assessment of lymphadenopathy and staging of metastatic disease. Radiologists routinely measure the nodal size in order to distinguish benign from malignant nodes, which require subsequent cancer staging. However, identification of lymph nodes is a cumbersome task due to their myriad appearances in mpMRI studies. Multiple sequences are acquired in mpMRI studies, including T2 fat suppressed (T2FS) and diffusion weighted imaging (DWI) sequences among others; consequently, the sizing of LNs is rendered challenging due to the variety of signal intensities in these sequences. Furthermore, radiologists can miss potentially metastatic LNs during a busy clinical day. To lighten these imaging and workflow challenges, we propose a computer-aided detection (CAD) pipeline to detect both benign and malignant LNs in the body for their subsequent measurement. We employed the recently proposed Dynamic Head (DyHead) neural network to detect LNs in mpMRI studies that were acquired using a variety of scanners and exam protocols. The T2FS and DWI series were co-registered, and a selective augmentation technique called Intra-Label LISA (ILL) was used to blend the two volumes with the interpolation factor drawn from a Beta distribution. In this way, ILL diversified the samples that the model encountered during the training phase, while the requirement for both sequences to be present at test time was nullified. Our results showed a mean average precision (mAP) of 53.5% and a sensitivity of ∼78% with ILL at 4 FP/vol. This corresponded to an improvement of ≥10% in mAP and ≥12% in sensitivity at 4FP (p ¡ 0.05) respectively over current LN detection approaches evaluated on the same dataset. We also established the out-of-distribution robustness of the DyHead model by training it on data acquired by a Siemens Aera scanner and testing it on data from the Siemens Verio, Siemens Biograph mMR, and Philips Achieva scanners. Our pilot work represents an important first step towards automated detection, segmentation, and classification of lymph nodes in mpMRI.

Automated detection of incidental abdominal aortic aneurysms on computed tomography.

PURPOSE: To detect and assess abdominal aortic aneurysms (AAAs) on CT in a large asymptomatic adult patient population using fully-automated deep learning software. MATERIALS AND METHODS: The abdominal aorta was segmented using a fully-automated deep learning model trained on 66 manually-segmented abdominal CT scans from two datasets. The axial diameters of the segmented aorta were extracted to detect the presence of AAAs-maximum axial aortic diameter greater than 3 cm were labeled as AAA positive. The trained system was then externally-validated on CT colonography scans of 9172 asymptomatic outpatients (mean age, 57 years) referred for colorectal cancer screening. Using a previously-validated automated calcified atherosclerotic plaque detector, we correlated abdominal aortic Agatston and volume scores with the presence of AAA. RESULTS: The deep learning software detected AAA on the external validation dataset with a sensitivity, specificity, and AUC of 96%, (95% CI 89%, 100%), 96% (96%, 97%), and 99% (98%, 99%) respectively. The Agatston and volume scores of reported AAA-positive cases were statistically significantly greater than those of reported AAA-negative cases (p < 0.0001). Using plaque alone as a AAA detector, at a threshold Agatston score of 2871, the sensitivity and specificity were 84% (73%, 94%) and 87% (86%, 87%), respectively. CONCLUSION: Fully-automated detection and assessment of AAA on CT is feasible and accurate. There was a strong statistical association between the presence of AAA and the quantity of abdominal aortic calcified atherosclerotic plaque.

Universal detection and segmentation of lymph nodes in multi-parametric MRI.

PURPOSE: Reliable measurement of lymph nodes (LNs) in multi-parametric MRI (mpMRI) studies of the body plays a major role in the assessment of lymphadenopathy and staging of metastatic disease. Previous approaches do not adequately exploit the complementary sequences in mpMRI to universally detect and segment lymph nodes, and they have shown fairly limited performance. METHODS: We propose a computer-aided detection and segmentation pipeline to leverage the T2 fat-suppressed (T2FS) and diffusion-weighted imaging (DWI) series from a mpMRI study. The T2FS and DWI series in 38 studies (38 patients) were co-registered and blended together using a selective data augmentation technique, such that traits of both series were visible in the same volume. A mask RCNN model was subsequently trained for universal detection and segmentation of 3D LNs. RESULTS: Experiments on 18 test mpMRI studies revealed that the proposed pipeline achieved a precision of [Formula: see text]%, sensitivity of [Formula: see text]% at 4 false positives (FP) per volume, and dice score of [Formula: see text]%. This represented an improvement of [Formula: see text]% in precision, [Formula: see text]% in sensitivity at 4 FP/volume, and [Formula: see text]% in dice score, respectively, over current approaches evaluated on the same dataset. CONCLUSION: Our pipeline universally detected and segmented both metastatic and non-metastatic nodes in mpMRI studies. At test time, the input data used by the trained model could either be the T2FS series alone or a blend of co-registered T2FS and DWI series. Contrary to prior work, this eliminated the reliance on both the T2FS and DWI series in a mpMRI study.

How Does Pruning Impact Long-Tailed Multi-label Medical Image Classifiers?

Pruning has emerged as a powerful technique for compressing deep neural networks, reducing memory usage and inference time without significantly affecting overall performance. However, the nuanced ways in which pruning impacts model behavior are not well understood, particularly for long-tailed, multi-label datasets commonly found in clinical settings. This knowledge gap could have dangerous implications when deploying a pruned model for diagnosis, where unexpected model behavior could impact patient well-being. To fill this gap, we perform the first analysis of pruning's effect on neural networks trained to diagnose thorax diseases from chest X-rays (CXRs). On two large CXR datasets, we examine which diseases are most affected by pruning and characterize class "forgettability" based on disease frequency and co-occurrence behavior. Further, we identify individual CXRs where uncompressed and heavily pruned models disagree, known as pruning-identified exemplars (PIEs), and conduct a human reader study to evaluate their unifying qualities. We find that radiologists perceive PIEs as having more label noise, lower image quality, and higher diagnosis difficulty. This work represents a first step toward understanding the impact of pruning on model behavior in deep long-tailed, multi-label medical image classification. All code, model weights, and data access instructions can be found at https://github.com/VITA-Group/PruneCXR.

How Does Pruning Impact Long-Tailed Multi-Label Medical Image Classifiers?

Pruning has emerged as a powerful technique for compressing deep neural networks, reducing memory usage and inference time without significantly affecting overall performance. However, the nuanced ways in which pruning impacts model behavior are not well understood, particularly for long-tailed, multi-label datasets commonly found in clinical settings. This knowledge gap could have dangerous implications when deploying a pruned model for diagnosis, where unexpected model behavior could impact patient well-being. To fill this gap, we perform the first analysis of pruning's effect on neural networks trained to diagnose thorax diseases from chest X-rays (CXRs). On two large CXR datasets, we examine which diseases are most affected by pruning and characterize class "forgettability" based on disease frequency and co-occurrence behavior. Further, we identify individual CXRs where uncompressed and heavily pruned models disagree, known as pruning-identified exemplars (PIEs), and conduct a human reader study to evaluate their unifying qualities. We find that radiologists perceive PIEs as having more label noise, lower image quality, and higher diagnosis difficulty. This work represents a first step toward understanding the impact of pruning on model behavior in deep long-tailed, multi-label medical image classification. All code, model weights, and data access instructions can be found at https://github.com/VITA-Group/PruneCXR.

Improving Segmentation and Detection of Lesions in CT Scans Using Intensity Distribution Supervision.

We propose a method to incorporate the intensity information of a target lesion on CT scans in training segmentation and detection networks. We first build an intensity-based lesion probability (ILP) function from an intensity histogram of the target lesion. It is used to compute the probability of being the lesion for each voxel based on its intensity. Finally, the computed ILP map of each input CT scan is provided as additional supervision for network training, which aims to inform the network about possible lesion locations in terms of intensity values at no additional labeling cost. The method was applied to improve the segmentation of three different lesion types, namely, small bowel carcinoid tumor, kidney tumor, and lung nodule. The effectiveness of the proposed method on a detection task was also investigated. We observed improvements of 41.3% -> 47.8%, 74.2% -> 76.0%, and 26.4% -> 32.7% in segmenting small bowel carcinoid tumor, kidney tumor, and lung nodule, respectively, in terms of per case Dice scores. An improvement of 64.6% -> 75.5% was achieved in detecting kidney tumors in terms of average precision. The results of different usages of the ILP map and the effect of varied amount of training data are also presented.

Radiologic reporting of MRI-proven thoracolumbar epidural metastases on body CT: 12-Year single-institution experience.

RATIONALE AND OBJECTIVES: Metastatic epidural masses are an important radiological finding. The purpose of this study is to determine factors associated with non-reporting of thoracolumbar epidural metastases on body CT. MATERIALS AND METHODS: In a study population of 166 patients from a single institution over a 12-year period, 293 body CT examinations were identified which were performed within 30 days before or after a spine MRI diagnosis of epidural metastasis. Associations were sought between patient diagnosis, CT examination characteristics, reporting radiologist (n = 17), and lesion characteristics with respect to whether an epidural metastasis was reported on CT. RESULTS: In retrospective consensus review comprised of 3 radiologists, epidural metastases reported on spine MRI were clearly visible in 80.5% (236/293) of body CT examinations, however 65.3% (154/236) of the body CT reports omitted reporting their presence, even in cases where there was a preceding MRI diagnosis within 30 days (65.4%, 74/113). The identity of the reporting radiologist was statistically significantly associated with the accurate diagnostic reporting of epidural metastasis on body CT (p = 0.04). The only lesion features which were statistically significantly associated with CT reporting were lesion volume (p = 0.03) on noncontrast CT, and lesion volume (p = 0.006) and percentage of spinal canal stenosis (p = 0.001) on intravenous contrast-enhanced CT. The presence or absence of intravenous contrast was not significantly associated with CT reporting (p = 1.0). CONCLUSION: Using spine MRI as the reference standard for the presence of epidural tumor, the majority of body CT reports omit describing thoracolumbar epidural metastases which are clearly visible in retrospect.

Improving segmentation and detection of lesions in CT scans using intensity distribution supervision.

We propose a method to incorporate the intensity information of a target lesion on CT scans in training segmentation and detection networks. We first build an intensity-based lesion probability (ILP) function from an intensity histogram of the target lesion. It is used to compute the probability of being the lesion for each voxel based on its intensity. Finally, the computed ILP map of each input CT scan is provided as additional supervision for network training, which aims to inform the network about possible lesion locations in terms of intensity values at no additional labeling cost. The method was applied to improve the segmentation of three different lesion types, namely, small bowel carcinoid tumor, kidney tumor, and lung nodule. The effectiveness of the proposed method on a detection task was also investigated. We observed improvements of 41.3% → 47.8%, 74.2% → 76.0%, and 26.4% → 32.7% in segmenting small bowel carcinoid tumor, kidney tumor, and lung nodule, respectively, in terms of per case Dice scores. An improvement of 64.6% → 75.5% was achieved in detecting kidney tumors in terms of average precision. The results of different usages of the ILP map and the effect of varied amount of training data are also presented.

Improving Small Lesion Segmentation in CT Scans using Intensity Distribution Supervision: Application to Small Bowel Carcinoid Tumor.

Finding small lesions is very challenging due to lack of noticeable features, severe class imbalance, as well as the size itself. One approach to improve small lesion segmentation is to reduce the region of interest and inspect it at a higher sensitivity rather than performing it for the entire region. It is usually implemented as sequential or joint segmentation of organ and lesion, which requires additional supervision on organ segmentation. Instead, we propose to utilize an intensity distribution of a target lesion at no additional labeling cost to effectively separate regions where the lesions are possibly located from the background. It is incorporated into network training as an auxiliary task. We applied the proposed method to segmentation of small bowel carcinoid tumors in CT scans. We observed improvements for all metrics (33.5% → 38.2%, 41.3% → 47.8%, 30.0% → 35.9% for the global, per case, and per tumor Dice scores, respectively.) compared to the baseline method, which proves the validity of our idea. Our method can be one option for explicitly incorporating intensity distribution information of a target in network training.

Fully-automated detection of small bowel carcinoid tumors in CT scans using deep learning.

BACKGROUND: Small bowel carcinoid tumor is a rare neoplasm and increasing in incidence. Patients with small bowel carcinoid tumors often experience long delays in diagnosis due to the vague symptoms, slow growth of tumors, and lack of clinician awareness. Computed tomography (CT) is the most common imaging study for diagnosis of small bowel carcinoid tumor. It is often used with positron emission tomography (PET) to capture anatomical and functional aspects of carcinoid tumors and thus to increase the sensitivity. PURPOSE: We compared three different kinds of methods for the automatic detection of small bowel carcinoid tumors on CT scans, which is the first to the best of our knowledge. METHODS: Thirty-three preoperative CT scans of 33 unique patients with surgically-proven carcinoid tumors within the small bowel were collected. Ground-truth segmentation of tumors was drawn on CT scans by referring to available 18 F-DOPA PET scans and the corresponding radiology report. These scans were split into the trainval set (n = 24) and the test positive set (n= 9). Additionally, 22 CT scans of 22 unique patients who had no evidence of the tumor were collected to comprise the test negative set. We compared three different kinds of detection methods, which are detection network, patch-based classification, and segmentation-based methods. We also investigated the usefulness of small bowel segmentation for reduction of false positives (FPs) for each method. Free-response receiver operating characteristic (FROC) curves and receiver operating characteristic (ROC) curves were used for lesion- and patient-level evaluations, respectively. Statistical analyses comparing the FROC and ROC curves were also performed. RESULTS: The detection network method performed the best among the compared methods. For lesion-level detection, the detection network method, without the small bowel segmentation-based filtering, achieved sensitivity values of (60.8%, 81.1%, 82.4%, 86.5%) at per-scan FP rates of (1, 2, 4 ,8), respectively. The use of the small bowel segmentation did not improve the performance ( p = 0.742 $p=0.742$ ). For patient-level detection, again the detection network method, but with the small bowel segmentation-based filtering, achieved the highest AUC of 0.86 with a sensitivity of 78% and specificity of 82% at the Youden point. CONCLUSIONS: The carcinoid tumors in this patient population were very small and potentially difficult to diagnose. The presented method showed reasonable sensitivity at small numbers of FPs for lesion-level detection. It also achieved a promising AUC for patient-level detection. The method may have clinical application in patients with this rare and difficult to detect disease.

Universal lymph node detection in T2 MRI using neural networks.

PURPOSE: Identification of lymph nodes (LNs) that are suspicious for metastasis in T2 Magnetic Resonance Imaging (MRI) is critical for assessment of lymphadenopathy. Prior work on LN detection has been limited to specific anatomical regions of the body (pelvis, rectum). Therefore, an approach to universally detect both benign and metastatic nodes in T2 MRI studies of the body is highly desirable. METHODS: We developed a Computer Aided Detection (CAD) pipeline to universally identify LN in T2 MRI. First, we trained various neural networks for detecting LN: Faster RCNN with and without Hard Negative Example Mining (HNEM), FCOS, FoveaBox, VFNet, and Detection Transformer (DETR). Next, we show that VFNet with Adaptive Training Sample Selection (ATSS) outperformed Faster RCNN with HNEM. Finally, we ensembled models that surpassed a 45% mAP threshold. RESULTS: Experiments on 122 test studies revealed that VFNet achieved a 51.1% mAP and 78.7% recall at 4 false positives (FP) per volume, while the one-stage model ensemble achieved a mAP of 52.3% and sensitivity of 78.7% at 4FP. We found that VFNet and the one-stage model ensemble can be interchangeably used in the CAD pipeline. CONCLUSION: Our CAD pipeline universally detected both benign and metastatic nodes in T2 MRI studies, resulting in a sensitivity improvement of [Formula: see text]14% over the current LN detection approaches (sensitivity of 78.7% at 4 FP vs. 64.6% at 5 FP per volume).

Long-Tailed Classification of Thorax Diseases on Chest X-Ray: A New Benchmark Study.

Imaging exams, such as chest radiography, will yield a small set of common findings and a much larger set of uncommon findings. While a trained radiologist can learn the visual presentation of rare conditions by studying a few representative examples, teaching a machine to learn from such a "long-tailed" distribution is much more difficult, as standard methods would be easily biased toward the most frequent classes. In this paper, we present a comprehensive benchmark study of the long-tailed learning problem in the specific domain of thorax diseases on chest X-rays. We focus on learning from naturally distributed chest X-ray data, optimizing classification accuracy over not only the common "head" classes, but also the rare yet critical "tail" classes. To accomplish this, we introduce a challenging new long-tailed chest X-ray benchmark to facilitate research on developing long-tailed learning methods for medical image classification. The benchmark consists of two chest X-ray datasets for 19- and 20-way thorax disease classification, containing classes with as many as 53,000 and as few as 7 labeled training images. We evaluate both standard and state-of-the-art long-tailed learning methods on this new benchmark, analyzing which aspects of these methods are most beneficial for long-tailed medical image classification and summarizing insights for future algorithm design. The datasets, trained models, and code are available at https://github.com/VITA-Group/LongTailCXR.

Global-Local attention network with multi-task uncertainty loss for abnormal lymph node detection in MR images.

Accurate and reliable detection of abnormal lymph nodes in magnetic resonance (MR) images is very helpful for the diagnosis and treatment of numerous diseases. However, it is still a challenging task due to similar appearances between abnormal lymph nodes and other tissues. In this paper, we propose a novel network based on an improved Mask R-CNN framework for the detection of abnormal lymph nodes in MR images. Instead of laboriously collecting large-scale pixel-wise annotated training data, pseudo masks generated from RECIST bookmarks on hand are utilized as the supervision. Different from the standard Mask R-CNN architecture, there are two main innovations in our proposed network: 1) global-local attention which encodes the global and local scale context for detection and utilizes the channel attention mechanism to extract more discriminative features and 2) multi-task uncertainty loss which adaptively weights multiple objective loss functions based on the uncertainty of each task to automatically search the optimal solution. For the experiments, we built a new abnormal lymph node dataset with 821 RECIST bookmarks of 41 different types of abnormal abdominal lymph nodes from 584 different patients. The experimental results showed the superior performance of our algorithm over compared state-of-the-art approaches.

Generalized Zero-Shot Chest X-Ray Diagnosis Through Trait-Guided Multi-View Semantic Embedding With Self-Training.

Zero-shot learning (ZSL) is one of the most promising avenues of annotation-efficient machine learning. In the era of deep learning, ZSL techniques have achieved unprecedented success. However, the developments of ZSL methods have taken place mostly for natural images. ZSL for medical images has remained largely unexplored. We design a novel strategy for generalized zero-shot diagnosis of chest radiographs. In doing so, we leverage the potential of multi-view semantic embedding, a useful yet less-explored direction for ZSL. Our design also incorporates a self-training phase to tackle the problem of noisy labels alongside improving the performance for classes not seen during training. Through rigorous experiments, we show that our model trained on one dataset can produce consistent performance across test datasets from different sources including those with very different quality. Comparisons with a number of state-of-the-art techniques show the superiority of the proposed method for generalized zero-shot chest x-ray diagnosis.

Discriminative ensemble learning for few-shot chest x-ray diagnosis.

Few-shot learning is an almost unexplored area in the field of medical image analysis. We propose a method for few-shot diagnosis of diseases and conditions from chest x-rays using discriminative ensemble learning. Our design involves a CNN-based coarse-learner in the first step to learn the general characteristics of chest x-rays. In the second step, we introduce a saliency-based classifier to extract disease-specific salient features from the output of the coarse-learner and classify based on the salient features. We propose a novel discriminative autoencoder ensemble to design the saliency-based classifier. The classification of the diseases is performed based on the salient features. Our algorithm proceeds through meta-training and meta-testing. During the training phase of meta-training, we train the coarse-learner. However, during the training phase of meta-testing, we train only the saliency-based classifier. Thus, our method is first-of-its-kind where the training phase of meta-training and the training phase of meta-testing are architecturally disjoint, making the method modular and easily adaptable to new tasks requiring the training of only the saliency-based classifier. Experiments show as high as ∼19% improvement in terms of F1 score compared to the baseline in the diagnosis of chest x-rays from publicly available datasets.

Omnidirectional beacon-localization using a catadioptric system.

We present a catadioptric beacon localization system that can provide mobile network nodes with omnidirectional situational awareness of neighboring nodes. In this system, a receiver composed of a hyperboloidal mirror and camera is used to estimate the azimuth, elevation, and range of an LED beacon. We provide a general framework for understanding the propagation of error in the angle-of-arrival estimation and then present an experimental realization of such a system. The situational awareness provided by the proposed system can enable the alignment of communication nodes in an optical wireless network, which may be particularly useful in addressing RF-denied environments.

Resonant interactions between a Mollow triplet sideband and a strongly coupled cavity.

We demonstrate resonant coupling of a Mollow triplet sideband to an optical cavity in the strong coupling regime. We show that, in this regime, the resonant sideband is strongly enhanced relative to the detuned sideband. Furthermore, the linewidth of the Mollow sidebands exhibits a highly nonlinear pump power dependence when tuned across the cavity resonance due to strong resonant interactions with the cavity mode. We compare our results to calculations using the effective phonon master equation and show that the nonlinear linewidth behavior is caused by strong coherent interaction with the cavity mode that exists only when the Mollow sideband is near cavity resonance.

Design of dual-link (wide- and narrow-beam) LED communication systems.

We explore the design of an LED-based communication system comprising two free space optical links: one narrow-beam (primary) link for bulk data transmission and one wide-beam (beacon) link for alignment and support of the narrow-beam link. Such a system combines the high throughput of a highly directional link with the robust insensitivity to pointing errors of a wider-beam link. We develop a modeling framework for this dual-link configuration and then use this framework to explore system tradeoffs in power, range, and achievable rates. The proposed design presents a low-cost, compact, robust means of communication at short- to medium-ranges, and calculations show that data rates on the order of Mb/s are achievable at hundreds of meters with only a few LEDs.

Strong coupling between two quantum dots and a photonic crystal cavity using magnetic field tuning.

We demonstrate strong coupling between two indium arsenide (InAs) quantum dots (QDs) and a photonic crystal cavity by using a magnetic field as a frequency tuning method. The magnetic field causes a red shift of an exciton spin state in one QD and a blue shift in the opposite exciton spin state of the second QD, enabling them to be simultaneously tuned to the same cavity resonance. This method can match the emission frequency of two QDs separated by detunings as large as 1.35 meV using a magnetic field of up to 7 T. By controlling the detuning between the two QDs we measure the vacuum Rabi splitting (VRS) both when the QDs are individually coupled to the cavity, as well as when they are coupled to the cavity simultaneously. In the latter case the oscillator strength of two QDs shows a collective behavior, resulting in enhancement of the VRS as compared to the individual cases. Experimental results are compared to theoretical calculations based on the solution to the full master equation and found to be in excellent agreement.