Assessing the capacity of a denoising diffusion probabilistic model to reproduce spatial context.

Diffusion models have emerged as a popular family of deep generative models (DGMs). In the literature, it has been claimed that one class of diffusion models-denoising diffusion probabilistic models (DDPMs)-demonstrate superior image synthesis performance as compared to generative adversarial networks (GANs). To date, these claims have been evaluated using either ensemble-based methods designed for natural images, or conventional measures of image quality such as structural similarity. However, there remains an important need to understand the extent to which DDPMs can reliably learn medical imaging domain-relevant information, which is referred to as 'spatial context' in this work. To address this, a systematic assessment of the ability of DDPMs to learn spatial context relevant to medical imaging applications is reported for the first time. A key aspect of the studies is the use of stochastic context models (SCMs) to produce training data. In this way, the ability of the DDPMs to reliably reproduce spatial context can be quantitatively assessed by use of post-hoc image analyses. Error-rates in DDPM-generated ensembles are reported, and compared to those corresponding to other modern DGMs. The studies reveal new and important insights regarding the capacity of DDPMs to learn spatial context. Notably, the results demonstrate that DDPMs hold significant capacity for generating contextually correct images that are 'interpolated' between training samples, which may benefit data-augmentation tasks in ways that GANs cannot.

Report on the AAPM Grand Challenge on deep generative modeling for learning medical image statistics.

Background: The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. Purpose: The goal of this challenge was to promote the development of deep generative models for medical imaging and to emphasize the need for their domain-relevant assessments via the analysis of relevant image statistics. Methods: As part of this Grand Challenge, a common training dataset and an evaluation procedure was developed for benchmarking deep generative models for medical image synthesis. To create the training dataset, an established 3D virtual breast phantom was adapted. The resulting dataset comprised about 108,000 images of size 512×512. For the evaluation of submissions to the Challenge, an ensemble of 10,000 DGM-generated images from each submission was employed. The evaluation procedure consisted of two stages. In the first stage, a preliminary check for memorization and image quality (via the Fréchet Inception Distance (FID)) was performed. Submissions that passed the first stage were then evaluated for the reproducibility of image statistics corresponding to several feature families including texture, morphology, image moments, fractal statistics and skeleton statistics. A summary measure in this feature space was employed to rank the submissions. Additional analyses of submissions was performed to assess DGM performance specific to individual feature families, the four classes in the training data, and also to identify various artifacts. Results: Fifty-eight submissions from 12 unique users were received for this Challenge. Out of these 12 submissions, 9 submissions passed the first stage of evaluation and were eligible for ranking. The top-ranked submission employed a conditional latent diffusion model, whereas the joint runners-up employed a generative adversarial network, followed by another network for image superresolution. In general, we observed that the overall ranking of the top 9 submissions according to our evaluation method (i) did not match the FID-based ranking, and (ii) differed with respect to individual feature families. Another important finding from our additional analyses was that different DGMs demonstrated similar kinds of artifacts. Conclusions: This Grand Challenge highlighted the need for domain-specific evaluation to further DGM design as well as deployment. It also demonstrated that the specification of a DGM may differ depending on its intended use.

Assessing the Ability of Generative Adversarial Networks to Learn Canonical Medical Image Statistics.

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for potential applications in medical imaging, such as medical image synthesis, restoration, reconstruction, translation, as well as objective image quality assessment. Despite the impressive progress in generating high-resolution, perceptually realistic images, it is not clear if modern GANs reliably learn the statistics that are meaningful to a downstream medical imaging application. In this work, the ability of a state-of-the-art GAN to learn the statistics of canonical stochastic image models (SIMs) that are relevant to objective assessment of image quality is investigated. It is shown that although the employed GAN successfully learned several basic first- and second-order statistics of the specific medical SIMs under consideration and generated images with high perceptual quality, it failed to correctly learn several per-image statistics pertinent to the these SIMs, highlighting the urgent need to assess medical image GANs in terms of objective measures of image quality.

Investigating the robustness of a deep learning-based method for quantitative phase retrieval from propagation-based x-ray phase contrast measurements under laboratory conditions.

Objective.Quantitative phase retrieval (QPR) in propagation-based x-ray phase contrast imaging of heterogeneous and structurally complicated objects is challenging under laboratory conditions due to partial spatial coherence and polychromaticity. A deep learning-based method (DLBM) provides a nonlinear approach to this problem while not being constrained by restrictive assumptions about object properties and beam coherence. The objective of this work is to assess a DLBM for its applicability under practical scenarios by evaluating its robustness and generalizability under typical experimental variations.Approach.Towards this end, an end-to-end DLBM was employed for QPR under laboratory conditions and its robustness was investigated across various system and object conditions. The robustness of the method was tested via varying propagation distances and its generalizability with respect to object structure and experimental data was also tested.Main results.Although the end-to-end DLBM was stable under the studied variations, its successful deployment was found to be affected by choices pertaining to data pre-processing, network training considerations and system modeling.Significance.To our knowledge, we demonstrated for the first time, the potential applicability of an end-to-end learning-based QPR method, trained on simulated data, to experimental propagation-based x-ray phase contrast measurements acquired under laboratory conditions with a commercial x-ray source and a conventional detector. We considered conditions of polychromaticity, partial spatial coherence, and high noise levels, typical to laboratory conditions. This work further explored the robustness of this method to practical variations in propagation distances and object structure with the goal of assessing its potential for experimental use. Such an exploration of any DLBM (irrespective of its network architecture) before practical deployment provides an understanding of its potential behavior under experimental settings.

Learning stochastic object models from medical imaging measurements by use of advanced ambient generative adversarial networks.

Purpose: To objectively assess new medical imaging technologies via computer-simulations, it is important to account for the variability in the ensemble of objects to be imaged. This source of variability can be described by stochastic object models (SOMs). It is generally desirable to establish SOMs from experimental imaging measurements acquired by use of a well-characterized imaging system, but this task has remained challenging. Approach: A generative adversarial network (GAN)-based method that employs AmbientGANs with modern progressive or multiresolution training approaches is proposed. AmbientGANs established using the proposed training procedure are systematically validated in a controlled way using computer-simulated magnetic resonance imaging (MRI) data corresponding to a stylized imaging system. Emulated single-coil experimental MRI data are also employed to demonstrate the methods under less stylized conditions. Results: The proposed AmbientGAN method can generate clean images when the imaging measurements are contaminated by measurement noise. When the imaging measurement data are incomplete, the proposed AmbientGAN can reliably learn the distribution of the measurement components of the objects. Conclusions: Both visual examinations and quantitative analyses, including task-specific validations using the Hotelling observer, demonstrated that the proposed AmbientGAN method holds promise to establish realistic SOMs from imaging measurements.

Normalization of optical fluence distribution for three-dimensional functional optoacoustic tomography of the breast.

SIGNIFICANCE: In three-dimensional (3D) functional optoacoustic tomography (OAT), wavelength-dependent optical attenuation and nonuniform incident optical fluence limit imaging depth and field of view and can hinder accurate estimation of functional quantities, such as the vascular blood oxygenation. These limitations hinder OAT of large objects, such as a human female breast. AIM: We aim to develop a measurement-data-driven method for normalization of the optical fluence distribution and to investigate blood vasculature detectability and accuracy for estimating vascular blood oxygenation. APPROACH: The proposed method is based on reasonable assumptions regarding breast anatomy and optical properties. The nonuniform incident optical fluence is estimated based on the illumination geometry in the OAT system, and the depth-dependent optical attenuation is approximated using Beer-Lambert law. RESULTS: Numerical studies demonstrated that the proposed method significantly enhanced blood vessel detectability and improved estimation accuracy of the vascular blood oxygenation from multiwavelength OAT measurements, compared with direct application of spectral linear unmixing without optical fluence compensation. Experimental results showed that the proposed method revealed previously invisible structures in regions deeper than 15 mm and/or near the chest wall. CONCLUSIONS: The proposed method provides a straightforward and computationally inexpensive approximation of wavelength-dependent effective optical attenuation and, thus, enables mitigation of the spectral coloring effect in functional 3D OAT imaging.

On Hallucinations in Tomographic Image Reconstruction.

Tomographic image reconstruction is generally an ill-posed linear inverse problem. Such ill-posed inverse problems are typically regularized using prior knowledge of the sought-after object property. Recently, deep neural networks have been actively investigated for regularizing image reconstruction problems by learning a prior for the object properties from training images. However, an analysis of the prior information learned by these deep networks and their ability to generalize to data that may lie outside the training distribution is still being explored. An inaccurate prior might lead to false structures being hallucinated in the reconstructed image and that is a cause for serious concern in medical imaging. In this work, we propose to illustrate the effect of the prior imposed by a reconstruction method by decomposing the image estimate into generalized measurement and null components. The concept of a hallucination map is introduced for the general purpose of understanding the effect of the prior in regularized reconstruction methods. Numerical studies are conducted corresponding to a stylized tomographic imaging modality. The behavior of different reconstruction methods under the proposed formalism is discussed with the help of the numerical studies.

Quantification of image texture in X-ray phase-contrast-enhanced projection images of in vivo mouse lungs observed at varied inflation pressures.

To date, there are very limited noninvasive, regional assays of in vivo lung microstructure near the alveolar level. It has been suggested that x-ray phase-contrast enhanced imaging reveals information about the air volume of the lung; however, the image texture information in these images remains underutilized. Projection images of in vivo mouse lungs were acquired via a tabletop, propagation-based, X-ray phase-contrast imaging system. Anesthetized mice were mechanically ventilated in an upright position. Consistent with previously published studies, a distinct image texture was observed uniquely within lung regions. Lung regions were automatically identified using supervised machine learning applied to summary measures of the image texture data. It was found that an unsupervised clustering within predefined lung regions colocates with expected differences in anatomy along the cranial-caudal axis in upright mice. It was also found that specifically selected inflation pressures-here, a purposeful surrogate of distinct states of mechanical expansion-can be predicted from the lung image texture alone, that the prediction model itself varies from apex to base and that prediction is accurate regardless of overlap with nonpulmonary structures such as the ribs, mediastinum, and heart. Cross-validation analysis indicated low inter-animal variation in the image texture classifications. Together, these results suggest that the image texture observed in a single X-ray phase-contrast-enhanced projection image could be used across a range of pressure states to study regional variations in regional lung function.

The peroxisome proliferator-activated receptor agonist pioglitazone and 5-lipoxygenase inhibitor zileuton have no effect on lung inflammation in healthy volunteers by positron emission tomography in a single-blind placebo-controlled cohort study.

BACKGROUND: Anti-inflammatory drug development efforts for lung disease have been hampered in part by the lack of noninvasive inflammation biomarkers and the limited ability of animal models to predict efficacy in humans. We used 18F-fluorodeoxyglucose (18F-FDG) positron emission tomography (PET) in a human model of lung inflammation to assess whether pioglitazone, a peroxisome proliferator-activated receptor-γ (PPAR-γ) agonist, and zileuton, a 5-lipoxygenase inhibitor, reduce lung inflammation. METHODS: For this single center, single-blind, placebo-controlled cohort study, we enrolled healthy volunteers sequentially into the following treatment cohorts (N = 6 per cohort): pioglitazone plus placebo, zileuton plus placebo, or dual placebo prior to bronchoscopic endotoxin instillation. 18F-FDG uptake pre- and post-endotoxin was quantified as the Patlak graphical analysis-determined Ki (primary outcome measure). Secondary outcome measures included the mean standard uptake value (SUVmean), post-endotoxin bronchoalveolar lavage (BAL) cell counts and differentials and blood adiponectin and urinary leukotriene E4 (LTE4) levels, determined by enzyme-linked immunosorbent assay, to verify treatment compliance. One- or two-way analysis of variance assessed for differences among cohorts in the outcome measures (expressed as mean ± standard deviation). RESULTS: Ten females and eight males (29±6 years of age) completed all study procedures except for one volunteer who did not complete the post-endotoxin BAL. Ki and SUVmean increased in all cohorts after endotoxin instillation (Ki increased by 0.0021±0.0019, 0.0023±0.0017, and 0.0024±0.0020 and SUVmean by 0.47±0.14, 0.55±0.15, and 0.54±0.38 in placebo, pioglitazone, and zileuton cohorts, respectively, p<0.001) with no differences among treatment cohorts (p = 0.933). Adiponectin levels increased as expected with pioglitazone treatment but not urinary LTE4 levels as expected with zileuton treatment. BAL cell counts (p = 0.442) and neutrophil percentage (p = 0.773) were similar among the treatment cohorts. CONCLUSIONS: Endotoxin-induced lung inflammation in humans is not responsive to pioglitazone or zileuton, highlighting the challenge in translating anti-inflammatory drug efficacy results from murine models to humans. TRIAL REGISTRATION: ClinicalTrials.gov NCT01174056.

PET of Poly (ADP-Ribose) Polymerase Activity in Cancer: Preclinical Assessment and First In-Human Studies.

Purpose To demonstrate that positron emission tomography (PET) with fluorine 18 (18F) fluorthanatrace (FTT) depicts activated poly (adenosine diphosphate-ribose)polymerase (PARP) expression and is feasible for clinical trial evaluation. Materials and Methods All studies were conducted prospectively from February 2012 through July 2015 under protocols approved by the local animal studies committee and institutional review board. The area under the receiver operating characteristic curve (AUC, in g/mL· min) for 18F-FTT was assessed in normal mouse organs before and after treatment with olaparib (n = 14), a PARP inhibitor, or iniparib (n = 11), which has no PARP inhibitory activity. Murine biodistribution studies were performed to support human translational studies. Eight human subjects with cancer and eight healthy volunteers underwent imaging to verify the human radiation dosimetry of 18F-FTT. The Wilcoxon signed rank test was used to assess for differences among treatment groups for the mouse studies. Results In mice, olaparib, but not iniparib, significantly reduced the 18F-FTT AUC in the spine (median difference before and after treatment and interquartile range [IQR]: -17 g/mL· min and 10 g/mL · min, respectively [P = .0001], for olaparib and -3 g/mL · min and 13 g/mL · min [P = .70] for iniparib) and in nodes (median difference and interquartile range [IQR] before and after treatment: -23 g/mL · min and 13 g/mL · min [P = .0001] for olaparib; -9 g/mL · min and 17 g/mL · min [P = .05] for iniparib). The effective dose was estimated at 6.9 mSv for a 370-MBq 18F-FTT dose in humans. In humans, the organs with the highest uptake on images were the spleen and pancreas. Among five subjects with measurable tumors, increased 18F-FTT uptake was seen in one subject with pancreatic adenocarcinoma and another with liver cancer. Conclusion The results suggest that 18F-FTT uptake reflects PARP expression and that its radiation dosimetry profile is compatible with those of agents currently in clinical use. © RSNA, 2016 Online supplemental material is available for this article.

Low-order non-spatial effects dominate second-order spatial effects in the texture quantifier analysis of 18F-FDG-PET images.

BACKGROUND: There is increasing interest in applying image texture quantifiers to assess the intra-tumor heterogeneity observed in FDG-PET images of various cancers. Use of these quantifiers as prognostic indicators of disease outcome and/or treatment response has yielded inconsistent results. We study the general applicability of some well-established texture quantifiers to the image data unique to FDG-PET. METHODS: We first created computer-simulated test images with statistical properties consistent with clinical image data for cancers of the uterine cervix. We specifically isolated second-order statistical effects from low-order effects and analyzed the resulting variation in common texture quantifiers in response to contrived image variations. We then analyzed the quantifiers computed for FIGOIIb cervical cancers via receiver operating characteristic (ROC) curves and via contingency table analysis of detrended quantifier values. RESULTS: We found that image texture quantifiers depend strongly on low-effects such as tumor volume and SUV distribution. When low-order effects are controlled, the image texture quantifiers tested were not able to discern only the second-order effects. Furthermore, the results of clinical tumor heterogeneity studies might be tunable via choice of patient population analyzed. CONCLUSION: Some image texture quantifiers are strongly affected by factors distinct from the second-order effects researchers ostensibly seek to assess via those quantifiers.

The effect of small tumor volumes on studies of intratumoral heterogeneity of tracer uptake.

UNLABELLED: The number of studies in the literature involving quantification of the metabolic heterogeneity seen in (18)F-FDG PET images has increased sharply over recent years. We hypothesized that inclusion of very small regions of interest as unique data points will have deleterious effects on these studies. METHODS: Using a combination of probability theory and clinical (18)F-FDG PET data, we numerically calculated the curve describing the probability a given tumor volume is large enough to adequately sample the underlying tumor biology assayed via a PET/CT scanner at a planar resolution of 4 mm and transaxial resolution of 4 mm (64 mm(3) voxel size). We then used a computer simulation to isolate the effects of tumor volume on the image local entropy. RESULTS: We computed the underlying global intensity distribution for 70 cervical cancer tumors ranging from 4 to 248 cm(3)), which were ensemble-averaged over the same intensity scale. From this distribution, we determined that about 700 total voxels (45 cm(3)) are required to give 95% certainty that the global intensity distribution has been sufficiently sampled for common statistical comparisons of individual tumor intensity distributions to be made canonically. We demonstrated that one previously suggested measure of heterogeneity is dependent on tumor volume and that measurement of heterogeneity is about 5 times more sensitive to volume changes for volumes below the proposed minimum than for those above it. CONCLUSION: Inclusion of tumor volumes below 45 cm(3) can profoundly bias comparisons of intratumoral uptake heterogeneity metrics derived from data from the current generation of whole-body (18)F-FDG PET scanners.

FDG uptake heterogeneity in FIGO IIb cervical carcinoma does not predict pelvic lymph node involvement.

TRANSLATIONAL RELEVANCE: Many types of cancer are located and assessed via positron emission tomography (PET) using the 18F-fluorodeoxyglucose (FDG) radiotracer of glucose uptake. There is rapidly increasing interest in exploiting the intra-tumor heterogeneity observed in these FDG-PET images as an indicator of disease outcome. If this image heterogeneity is of genuine prognostic value, then it either correlates to known prognostic factors, such as tumor stage, or it indicates some as yet unknown tumor quality. Therefore, the first step in demonstrating the clinical usefulness of image heterogeneity is to explore the dependence of image heterogeneity metrics upon established prognostic indicators and other clinically interesting factors. If it is shown that image heterogeneity is merely a surrogate for other important tumor properties or variations in patient populations, then the theoretical value of quantified biological heterogeneity may not yet translate into the clinic given current imaging technology. PURPOSE: We explore the relation between pelvic lymph node status at diagnosis and the visually evident uptake heterogeneity often observed in 18F-fluorodeoxyglucose positron emission tomography (FDG-PET) images of cervical carcinomas. EXPERIMENTAL DESIGN: We retrospectively studied the FDG-PET images of 47 node negative and 38 node positive patients, each having FIGO stage IIb tumors with squamous cell histology. Imaged tumors were segmented using 40% of the maximum tumor uptake as the tumor-defining threshold and then converted into sets of three-dimensional coordinates. We employed the sphericity, extent, Shannon entropy (S) and the accrued deviation from smoothest gradients (ζ) as image heterogeneity metrics. We analyze these metrics within tumor volume strata via: the Kolmogorov-Smirnov test, principal component analysis and contingency tables. RESULTS: We found no statistically significant difference between the positive and negative lymph node groups for any one metric or plausible combinations thereof. Additionally, we observed that S is strongly dependent upon tumor volume and that ζ moderately correlates with mean FDG uptake. CONCLUSIONS: FDG uptake heterogeneity did not indicate patients with differing prognoses. Apparent heterogeneity differences between clinical groups may be an artifact arising from either the dependence of some image metrics upon other factors such as tumor volume or upon the underlying variations in the patient populations compared.

Quantification of heterogeneity observed in medical images.

BACKGROUND: There has been much recent interest in the quantification of visually evident heterogeneity within functional grayscale medical images, such as those obtained via magnetic resonance or positron emission tomography. In the case of images of cancerous tumors, variations in grayscale intensity imply variations in crucial tumor biology. Despite these considerable clinical implications, there is as yet no standardized method for measuring the heterogeneity observed via these imaging modalities. METHODS: In this work, we motivate and derive a statistical measure of image heterogeneity. This statistic measures the distance-dependent average deviation from the smoothest intensity gradation feasible. We show how this statistic may be used to automatically rank images of in vivo human tumors in order of increasing heterogeneity. We test this method against the current practice of ranking images via expert visual inspection. RESULTS: We find that this statistic provides a means of heterogeneity quantification beyond that given by other statistics traditionally used for the same purpose. We demonstrate the effect of tumor shape upon our ranking method and find the method applicable to a wide variety of clinically relevant tumor images. We find that the automated heterogeneity rankings agree very closely with those performed visually by experts. CONCLUSIONS: These results indicate that our automated method may be used reliably to rank, in order of increasing heterogeneity, tumor images whether or not object shape is considered to contribute to that heterogeneity. Automated heterogeneity ranking yields objective results which are more consistent than visual rankings. Reducing variability in image interpretation will enable more researchers to better study potential clinical implications of observed tumor heterogeneity.

Current measures of metabolic heterogeneity within cervical cancer do not predict disease outcome.

BACKGROUND: A previous study evaluated the intra-tumoral heterogeneity observed in the uptake of F-18 fluorodeoxyglucose (FDG) in pre-treatment positron emission tomography (PET) scans of cancers of the uterine cervix as an indicator of disease outcome. This was done via a novel statistic which ostensibly measured the spatial variations in intra-tumoral metabolic activity. In this work, we argue that statistic is intrinsically non-spatial, and that the apparent delineation between unsuccessfully- and successfully-treated patient groups via that statistic is spurious. METHODS: We first offer a straightforward mathematical demonstration of our argument. Next, we recapitulate an assiduous re-analysis of the originally published data which was derived from FDG-PET imagery. Finally, we present the results of a principal component analysis of FDG-PET images similar to those previously analyzed. RESULTS: We find that the previously published measure of intra-tumoral heterogeneity is intrinsically non-spatial, and actually is only a surrogate for tumor volume. We also find that an optimized linear combination of more canonical heterogeneity quantifiers does not predict disease outcome. CONCLUSIONS: Current measures of intra-tumoral metabolic activity are not predictive of disease outcome as has been claimed previously. The implications of this finding are: clinical categorization of patients based upon these statistics is invalid; more sophisticated, and perhaps innately-geometric, quantifications of metabolic activity are required for predicting disease outcome.