Med-cDiff: Conditional Medical Image Generation with Diffusion Models.

Conditional image generation plays a vital role in medical image analysis as it is effective in tasks such as super-resolution, denoising, and inpainting, among others. Diffusion models have been shown to perform at a state-of-the-art level in natural image generation, but they have not been thoroughly studied in medical image generation with specific conditions. Moreover, current medical image generation models have their own problems, limiting their usage in various medical image generation tasks. In this paper, we introduce the use of conditional Denoising Diffusion Probabilistic Models (cDDPMs) for medical image generation, which achieve state-of-the-art performance on several medical image generation tasks.

CAT-Net: A Cross-Slice Attention Transformer Model for Prostate Zonal Segmentation in MRI.

Prostate cancer is the second leading cause of cancer death among men in the United States. The diagnosis of prostate MRI often relies on accurate prostate zonal segmentation. However, state-of-the-art automatic segmentation methods often fail to produce well-contained volumetric segmentation of the prostate zones since certain slices of prostate MRI, such as base and apex slices, are harder to segment than other slices. This difficulty can be overcome by leveraging important multi-scale image-based information from adjacent slices, but current methods do not fully learn and exploit such cross-slice information. In this paper, we propose a novel cross-slice attention mechanism, which we use in a Transformer module to systematically learn cross-slice information at multiple scales. The module can be utilized in any existing deep-learning-based segmentation framework with skip connections. Experiments show that our cross-slice attention is able to capture cross-slice information significant for prostate zonal segmentation in order to improve the performance of current state-of-the-art methods. Cross-slice attention improves segmentation accuracy in the peripheral zones, such that segmentation results are consistent across all the prostate slices (apex, mid-gland, and base). The code for the proposed model is available at https://bit.ly/CAT-Net.

RAVIR: A Dataset and Methodology for the Semantic Segmentation and Quantitative Analysis of Retinal Arteries and Veins in Infrared Reflectance Imaging.

The retinal vasculature provides important clues in the diagnosis and monitoring of systemic diseases including hypertension and diabetes. The microvascular system is of primary involvement in such conditions, and the retina is the only anatomical site where the microvasculature can be directly observed. The objective assessment of retinal vessels has long been considered a surrogate biomarker for systemic vascular diseases, and with recent advancements in retinal imaging and computer vision technologies, this topic has become the subject of renewed attention. In this paper, we present a novel dataset, dubbed RAVIR, for the semantic segmentation of Retinal Arteries and Veins in Infrared Reflectance (IR) imaging. It enables the creation of deep learning-based models that distinguish extracted vessel type without extensive post-processing. We propose a novel deep learning-based methodology, denoted as SegRAVIR, for the semantic segmentation of retinal arteries and veins and the quantitative measurement of the widths of segmented vessels. Our extensive experiments validate the effectiveness of SegRAVIR and demonstrate its superior performance in comparison to state-of-the-art models. Additionally, we propose a knowledge distillation framework for the domain adaptation of RAVIR pretrained networks on color images. We demonstrate that our pretraining procedure yields new state-of-the-art benchmarks on the DRIVE, STARE, and CHASE_DB1 datasets. Dataset link: https://ravirdataset.github.io/data.

Image Segmentation Using Deep Learning: A Survey.

Image segmentation is a key task in computer vision and image processing with important applications such as scene understanding, medical image analysis, robotic perception, video surveillance, augmented reality, and image compression, among others, and numerous segmentation algorithms are found in the literature. Against this backdrop, the broad success of deep learning (DL) has prompted the development of new image segmentation approaches leveraging DL models. We provide a comprehensive review of this recent literature, covering the spectrum of pioneering efforts in semantic and instance segmentation, including convolutional pixel-labeling networks, encoder-decoder architectures, multiscale and pyramid-based approaches, recurrent networks, visual attention models, and generative models in adversarial settings. We investigate the relationships, strengths, and challenges of these DL-based segmentation models, examine the widely used datasets, compare performances, and discuss promising research directions.

Artificial intelligence-enabled screening for diabetic retinopathy: a real-world, multicenter and prospective study.

INTRODUCTION: Early screening for diabetic retinopathy (DR) with an efficient and scalable method is highly needed to reduce blindness, due to the growing epidemic of diabetes. The aim of the study was to validate an artificial intelligence-enabled DR screening and to investigate the prevalence of DR in adult patients with diabetes in China. RESEARCH DESIGN AND METHODS: The study was prospectively conducted at 155 diabetes centers in China. A non-mydriatic, macula-centered fundus photograph per eye was collected and graded through a deep learning (DL)-based, five-stage DR classification. Images from a randomly selected one-third of participants were used for the DL algorithm validation. RESULTS: In total, 47 269 patients (mean (SD) age, 54.29 (11.60) years) were enrolled. 15 805 randomly selected participants were reviewed by a panel of specialists for DL algorithm validation. The DR grading algorithms had a 83.3% (95% CI: 81.9% to 84.6%) sensitivity and a 92.5% (95% CI: 92.1% to 92.9%) specificity to detect referable DR. The five-stage DR classification performance (concordance: 83.0%) is comparable to the interobserver variability of specialists (concordance: 84.3%). The estimated prevalence in patients with diabetes detected by DL algorithm for any DR, referable DR and vision-threatening DR were 28.8% (95% CI: 28.4% to 29.3%), 24.4% (95% CI: 24.0% to 24.8%) and 10.8% (95% CI: 10.5% to 11.1%), respectively. The prevalence was higher in female, elderly, longer diabetes duration and higher glycated hemoglobin groups. CONCLUSION: This study performed, a nationwide, multicenter, DL-based DR screening and the results indicated the importance and feasibility of DR screening in clinical practice with this system deployed at diabetes centers. TRIAL REGISTRATION NUMBER: NCT04240652.

Door and Doorway Etiquette for Virtual Humans.

We introduce a framework for simulating a variety of nontrivial, socially motivated behaviors that underlie the orderly passage of pedestrians through doorways, especially the common courtesy of opening and holding doors open for others, an important etiquette that has been overlooked in the literature on autonomous multi-human animation. Emulating such social activity requires serious attention to the interplay of visual perception, navigation in constrained doorway environments, manipulation of a variety of door types, and high-level decision making based on social considerations. To tackle this complex human simulation problem, we take an artificial life approach to modeling autonomous pedestrians, proposing a layered architecture comprising mental, behavioral, and motor layers. The behavioral layer couples two stages: (1) a decentralized, agent-based strategy for dynamically determining the well-mannered ordering of pedestrians around doorways, and (2) a state-based model that directs and coordinates a pedestrian's interactions with the door. The mental layer is a Bayesian network decision model that dynamically selects appropriate door-holding behaviors by considering both internal and external social factors pertinent to pedestrians interacting with one another in and around doorways. Our framework addresses the various door types in common use and supports a variety of doorway etiquette scenarios with efficient, real-time performance.

Fast and Scalable Position-Based Layout Synthesis.

The arrangement of objects into a layout can be challenging for non-experts, as is affirmed by the existence of interior design professionals. Recent research into the automation of this task has yielded methods that can synthesize layouts of objects respecting aesthetic and functional constraints that are non-linear and competing. These methods usually adopt a stochastic optimization scheme, which samples from different layout configurations, a process that is slow and inefficient. We introduce an physics-motivated, continuous layout synthesis technique, which results in a significant gain in speed and is readily scalable. We demonstrate our method on a variety of examples and show that it achieves results similar to conventional layout synthesis based on Markov chain Monte Carlo (McMC) state-search, but is faster by at least an order of magnitude and can handle layouts of unprecedented size as well as tightly-packed layouts that can overwhelm McMC.

Visualization of vascular injuries in extremity trauma.

A tandem of particle-based computational methods is adapted to simulate injury and hemorrhage in the human body. In order to ensure anatomical fidelity, a three-dimensional model of a targeted portion of the human body is reconstructed from a dense sequence of CT scans of an anonymized patient. Skin, bone and muscular tissue are distinguished in the imaging data and assigned with their respective material properties. An injury geometry is then generated by simulating the mechanics of a ballistic projectile passing through the anatomical model with the material point method. From the injured vascular segments identified in the resulting geometry, smoothed particle hydrodynamics (SPH) is employed to simulate bleeding, based on inflow boundary conditions obtained from a network model of the systemic arterial tree. Computational blood particles interact with the stationary particles representing impermeable bone and skin and permeable muscular tissue through the Brinkman equations for porous media. The SPH results are rendered in post-processing for improved visual fidelity. The overall simulation strategy is demonstrated on an injury scenario in the lower leg.

Automated pericardial fat quantification from coronary magnetic resonance angiography: feasibility study.

Pericardial fat volume (PFV) is emerging as an important parameter for cardiovascular risk stratification. We propose a hybrid approach for automated PFV quantification from water/fat-resolved whole-heart noncontrast coronary magnetic resonance angiography (MRA). Ten coronary MRA datasets were acquired. Image reconstruction and phase-based water-fat separation were conducted offline. Our proposed algorithm first roughly segments the heart region on the original image using a simplified atlas-based segmentation with four cases in the atlas. To get exact boundaries of pericardial fat, a three-dimensional graph-based segmentation is used to generate fat and nonfat components on the fat-only image. The algorithm then selects the components that represent pericardial fat. We validated the quantification results on the remaining six subjects and compared them with manual quantifications by an expert reader. The PFV quantified by our algorithm was [Formula: see text], compared to [Formula: see text] by the expert reader, which were not significantly different ([Formula: see text]) and showed excellent correlation ([Formula: see text],[Formula: see text]). The mean absolute difference in PFV between the algorithm and the expert reader was [Formula: see text]. The mean value of the paired differences was [Formula: see text] (95% confidence interval: [Formula: see text] to 6.21). The mean Dice coefficient of pericardial fat voxels was [Formula: see text]. Our approach may potentially be applied in a clinical setting, allowing for accurate magnetic resonance imaging (MRI)-based PFV quantification without tedious manual tracing.

Epicardial adipose tissue volume but not density is an independent predictor for myocardial ischemia.

BACKGROUND: Epicardial adipose tissue (EAT) volume is associated with plaque formation and cardiovascular event risk, its density may reflect tissue composition and metabolic activity. OBJECTIVES: Global and regional associations between EAT volume and density, ischemia and coronary calcium were investigated using a novel automatic quantitative measurement software. METHODS: 71 patients with an intermediate pre-test probability for coronary artery disease and inducible ischemia by SPECT were matched to two same-gender controls (total of 213 patients, 90% male, age 60 ± 10 years). Non-contrast CT for assessment of EAT volume, density (in Hounsfield Unit [HU]) and coronary calcium score (CCS) was performed. RESULTS: Global EAT volume was significantly increased in ischemic patients compared to controls (96 ± 49 vs. 82 ± 36 cm(3), p = 0.04), density showed no significant difference (-75.6 ± 4.3 vs. -75.1 ± 4.1HU, p = 0.63). EAT volume and density differed significantly between coronary territories (LAD: 37 ± 18 cm(3), -77.8 ± 4.5HU; LCx: 16 ± 9 cm(3), -73.9 ± 4.1HU; RCA: 36 ± 17 cm(3), -71.7 ± 4.8HU, p < 0.001). For regional ischemia, only LCx territory showed a significantly higher EAT volume (18 ± 8 vs. 16 ± 9 cm(3), p = 0.048). Multivariable logistic regression revealed a significant association with ischemia for EAT volume (OR 2.09 (1.0; 4.3), p = 0.049) and CCS (OR 1.43 (1.1; 1.9), p = 0.006). EAT volume significantly improved discrimination of ischemia over CCS (Integrated Discrimination Improvement: 3.5%, 95%CI: 1.1-6.1%, p = 0.004). Hypertension was the only risk factor significantly influencing EAT volume and density (98 ± 48 vs. 78 ± 31 cm(3), p = 0.002, -76.0 ± 4.1 vs. -74.5 ± 4.1 HU, p = 0.01). CONCLUSIONS: EAT volume is associated with myocardial ischemia and improves the discriminative power for independent ischemia prediction over CCS. In hypertensive patients, EAT is characterized by lower density and higher volumes.

The Clutterpalette: An Interactive Tool for Detailing Indoor Scenes.

We introduce the Clutterpalette, an interactive tool for detailing indoor scenes with small-scale items. When the user points to a location in the scene, the Clutterpalette suggests detail items for that location. In order to present appropriate suggestions, the Clutterpalette is trained on a dataset of images of real-world scenes, annotated with support relations. Our experiments demonstrate that the adaptive suggestions presented by the Clutterpalette increase modeling speed and enhance the realism of indoor scenes.

Automated pericardium delineation and epicardial fat volume quantification from noncontrast CT.

PURPOSE: The authors aimed to develop and validate an automated algorithm for epicardial fat volume (EFV) quantification from noncontrast CT. METHODS: The authors developed a hybrid algorithm based on initial segmentation with a multiple-patient CT atlas, followed by automated pericardium delineation using geodesic active contours. A coregistered segmented CT atlas was created from manually segmented CT data and stored offline. The heart and pericardium in test CT data are first initialized by image registration to the CT atlas. The pericardium is then detected by a knowledge-based algorithm, which extracts only the membrane representing the pericardium. From its initial atlas position, the pericardium is modeled by geodesic active contours, which iteratively deform and lock onto the detected pericardium. EFV is automatically computed using standard fat attenuation range. RESULTS: The authors applied their algorithm on 50 patients undergoing routine coronary calcium assessment by CT. Measurement time was 60 s per-patient. EFV quantified by the algorithm (83.60 ± 32.89 cm(3)) and expert readers (81.85 ± 34.28 cm(3)) showed excellent correlation (r = 0.97, p < 0.0001), with no significant differences by comparison of individual data points (p = 0.15). Voxel overlap by Dice coefficient between the algorithm and expert readers was 0.92 (range 0.88-0.95). The mean surface distance and Hausdorff distance in millimeter between manually drawn contours and the automatically obtained contours were 0.6 ± 0.9 mm and 3.9 ± 1.7 mm, respectively. Mean difference between the algorithm and experts was 9.7% ± 7.4%, similar to interobserver variability between 2 readers (8.0% ± 5.3%, p = 0.3). CONCLUSIONS: The authors' novel automated method based on atlas-initialized active contours accurately and rapidly quantifies EFV from noncontrast CT.

Skull-stripping with machine learning deformable organisms.

BACKGROUND: Segmentation methods for medical images may not generalize well to new data sets or new tasks, hampering their utility. We attempt to remedy these issues using deformable organisms to create an easily customizable segmentation plan. We validate our framework by creating a plan to locate the brain in 3D magnetic resonance images of the head (skull-stripping). NEW METHOD: Our method borrows ideas from artificial life to govern a set of deformable models. We use control processes such as sensing, proactive planning, reactive behavior, and knowledge representation to segment an image. The image may have landmarks and features specific to that dataset; these may be easily incorporated into the plan. In addition, we use a machine learning method to make our segmentation more accurate. RESULTS: Our method had the least Hausdorff distance error, but included slightly less brain voxels (false negatives). It also had the lowest false positive error and performed on par to skull-stripping specific method on other metrics. COMPARISON WITH EXISTING METHOD(S): We tested our method on 838 T1-weighted images, evaluating results using distance and overlap error metrics based on expert gold standard segmentations. We evaluated the results before and after the learning step to quantify its benefit; we also compare our results to three other widely used methods: BSE, BET, and the Hybrid Watershed algorithm. CONCLUSIONS: Our framework captures diverse categories of information needed for brain segmentation and will provide a foundation for tackling a wealth of segmentation problems.

SKULL-STRIPPING WITH DEFORMABLE ORGANISMS.

Segmenting brain from non-brain tissue within magnetic resonance (MR) images of the human head, also known as skull-stripping, is a critical processing step in the analysis of neuroimaging data. Though many algorithms have been developed to address this problem, challenges remain. In this paper, we apply the "deformable organism" framework to the skull-stripping problem. Within this framework, deformable models are equipped with higher-level control mechanisms based on the principles of artificial life, including sensing, reactive behavior, knowledge representation, and proactive planning. Our new deformable organisms are governed by a high-level plan aimed at the fully-automated segmentation of various parts of the head in MR imagery, and they are able to cooperate in computing a robust and accurate segmentation. We applied our segmentation approach to a test set of human MRI data using manual delineations of the data as a reference "gold standard." We compare these results with results from three widely used methods using set-similarity metrics.

United snakes.

Since their debut in 1987, snakes (active contour models) have become a standard image analysis technique with several variants now in common use. We present a framework called "United Snakes", which has two key features. First, it unifies the most popular snake variants, including finite difference, B-spline, and Hermite polynomial snakes in a consistent finite element formulation, thus expanding the range of object modeling capabilities within a uniform snake construction process. Second, it embodies the idea that the heretofore presumed competing technique known as "live wire" or "intelligent scissors" is in fact complementary to snakes and that the two techniques can advantageously be combined by introducing an effective hard constraint mechanism. The United Snakes framework amplifies the efficiency and reproducibility of the component techniques, and it offers more flexible interactive control while further minimizing user interactions. We apply United Snakes to several different medical image analysis tasks, including the segmentation of neuronal dendrites in EM images, dynamic chest image analysis, the quantification of growth plates in MR images and the isolation of the breast region in mammograms, demonstrating the generality, accuracy and robustness of the tool.

Geometry-driven photorealistic facial expression synthesis.

Expression mapping (also called performance driven animation) has been a popular method for generating facial animations. A shortcoming of this method is that it does not generate expression details such as the wrinkles due to skin deformations. In this paper, we provide a solution to this problem. We have developed a geometry-driven facial expression synthesis system. Given feature point positions (the geometry) of a facial expression, our system automatically synthesizes a corresponding expression image that includes photorealistic and natural looking expression details. Due to the difficulty of point tracking, the number of feature points required by the synthesis system is, in general, more than what is directly available from a performance sequence. We have developed a technique to infer the missing feature point motions from the tracked subset by using an example-based approach. Another application of our system is expression editing where the user drags feature points while the system interactively generates facial expressions with skin deformation details.

Deformable organisms for automatic medical image analysis.

We introduce a new approach to medical image analysis that combines deformable model methodologies with concepts from the field of artificial life. In particular, we propose "deformable organisms", autonomous agents whose task is the automatic segmentation, labeling, and quantitative analysis of anatomical structures in medical images. Analogous to natural organisms capable of voluntary movement, our artificial organisms possess deformable bodies with distributed sensors, as well as (rudimentary) brains with motor, perception, behavior, and cognition centers. Deformable organisms are perceptually aware of the image analysis process. Their behaviors, which manifest themselves in voluntary movement and alteration of body shape, are based upon sensed image features, pre-stored anatomical knowledge, and a deliberate cognitive plan. We demonstrate several prototype deformable organisms based on a multiscale axisymmetric body morphology, including a "corpus callosum worm" that can overcome noise, incomplete edges, considerable anatomical variation, and interference from collateral structures to segment and label the corpus callosum in 2D mid-sagittal MR brain images.