Report on the AAPM Grand Challenge on deep generative modeling for learning medical image statistics.

Background: The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. Purpose: The goal of this challenge was to promote the development of deep generative models for medical imaging and to emphasize the need for their domain-relevant assessments via the analysis of relevant image statistics. Methods: As part of this Grand Challenge, a common training dataset and an evaluation procedure was developed for benchmarking deep generative models for medical image synthesis. To create the training dataset, an established 3D virtual breast phantom was adapted. The resulting dataset comprised about 108,000 images of size 512×512. For the evaluation of submissions to the Challenge, an ensemble of 10,000 DGM-generated images from each submission was employed. The evaluation procedure consisted of two stages. In the first stage, a preliminary check for memorization and image quality (via the Fréchet Inception Distance (FID)) was performed. Submissions that passed the first stage were then evaluated for the reproducibility of image statistics corresponding to several feature families including texture, morphology, image moments, fractal statistics and skeleton statistics. A summary measure in this feature space was employed to rank the submissions. Additional analyses of submissions was performed to assess DGM performance specific to individual feature families, the four classes in the training data, and also to identify various artifacts. Results: Fifty-eight submissions from 12 unique users were received for this Challenge. Out of these 12 submissions, 9 submissions passed the first stage of evaluation and were eligible for ranking. The top-ranked submission employed a conditional latent diffusion model, whereas the joint runners-up employed a generative adversarial network, followed by another network for image superresolution. In general, we observed that the overall ranking of the top 9 submissions according to our evaluation method (i) did not match the FID-based ranking, and (ii) differed with respect to individual feature families. Another important finding from our additional analyses was that different DGMs demonstrated similar kinds of artifacts. Conclusions: This Grand Challenge highlighted the need for domain-specific evaluation to further DGM design as well as deployment. It also demonstrated that the specification of a DGM may differ depending on its intended use.

High-Dimensional MR Reconstruction Integrating Subspace and Adaptive Generative Models.

OBJECTIVE: To develop a new method that integrates subspace and generative image models for high-dimensional MR image reconstruction. METHODS: We proposed a formulation that synergizes a low-dimensional subspace model of high-dimensional images, an adaptive generative image prior serving as spatial constraints on the sequence of "contrast-weighted" images or spatial coefficients of the subspace model, and a conventional sparsity regularization. A special pretraining plus subject-specific network adaptation strategy was proposed to construct an accurate generative-network-based representation for images with varying contrasts. An iterative algorithm was introduced to jointly update the subspace coefficients and the multi-resolution latent space of the generative image model that leveraged an recently proposed intermediate layer optimization technique for network inversion. RESULTS: We evaluated the utility of the proposed method for two high-dimensional imaging applications: accelerated MR parameter mapping and high-resolution MR spectroscopic imaging. Improved performance over state-of-the-art subspace-based methods was demonstrated in both cases. CONCLUSION: The proposed method provided a new way to address high-dimensional MR image reconstruction problems by incorporating an adaptive generative model as a data-driven spatial prior for constraining subspace reconstruction. SIGNIFICANCE: Our work demonstrated the potential of integrating data-driven and adaptive generative priors with canonical low-dimensional modeling for high-dimensional imaging problems.

Assessing the Ability of Generative Adversarial Networks to Learn Canonical Medical Image Statistics.

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for potential applications in medical imaging, such as medical image synthesis, restoration, reconstruction, translation, as well as objective image quality assessment. Despite the impressive progress in generating high-resolution, perceptually realistic images, it is not clear if modern GANs reliably learn the statistics that are meaningful to a downstream medical imaging application. In this work, the ability of a state-of-the-art GAN to learn the statistics of canonical stochastic image models (SIMs) that are relevant to objective assessment of image quality is investigated. It is shown that although the employed GAN successfully learned several basic first- and second-order statistics of the specific medical SIMs under consideration and generated images with high perceptual quality, it failed to correctly learn several per-image statistics pertinent to the these SIMs, highlighting the urgent need to assess medical image GANs in terms of objective measures of image quality.

Stability of human stress hormones and stress hormone metabolites in wastewater under oxic and anoxic conditions.

Levels in wastewater of human stress biomarkers, such as cortisone (E), cortisol (F), tetrahydrocortisone (THE), and tetrahydrocortisol (THF) may serve as indicators of population wellbeing and overall health. This study examined the stability of these biosignature compounds in wastewater to inform on their applicability for use in wastewater-based epidemiology (WBE). Wastewater from two undisclosed U.S. municipalities were fortified with the above four biomarkers of stress to a concentration of 10 ppb, and their decay was studied at three temperatures (15, 25, and 35 °C) over 24 h in oxic and anoxic conditions. Samples were analyzed using liquid chromatography tandem mass spectrometry (LC-MS/MS) in conjunction with the isotope dilution method for absolute quantitation. Results demonstrated short-term persistence (24 h) of biomarkers at low temperatures (15 °C), and accelerating kinetics of decay that were positively correlated with temperature increases. Among the four biomarkers evaluated, the tetrahydro derivatives were the most long-lived sewage-borne stress biomarkers and these are recommended as prime analytical targets for use in WBE when tracking population stress. Statistical analyses using a non-parametric Wilcoxon test further revealed no significant differences (p > 0.05) between oxic and anoxic decay rates for all stress biomarkers in wastewater from all study locations, regardless of the prevailing temperature regime. This negative finding is worthy of reporting because it suggests the feasibility of straightforward modeling of stress hormone decay, irrespective of whether the sewerage system monitored contains fully filled, pressurized pipes or partially filled gravity flow pipes, whose filling level, and with it its redox conditions, are known to fluctuate over time with water use and storm events.

Reversible inter-degree-of-freedom optical-coherence conversion via entropy swapping.

The entropy associated with an optical field quantifies the field fluctuations and thus its coherence. Any binary optical degree-of-freedom (DoF) - such as polarization or the field at a pair of points in space - can each carry up to one bit of entropy. We demonstrate here that entropy can be reversibly swapped between different DoFs, such that coherence is converted back and forth between them without loss of energy. Specifically, starting with a spatially coherent but unpolarized field carrying one bit of entropy, we unitarily convert the coherence from the spatial DoF to polarization to produce a spatially incoherent but polarized field by swapping the entropy between the two DoFs. Next, we implement the inverse unitary operator, thus converting the coherence back to yield once again a spatially coherent yet unpolarized field. We exploit the intermediate stage between the two coherence conversions - where the spatial coherence has been converted to the polarization DoF - to verify that the field has become immune to the deleterious impact of spatial phase scrambling. Maximizing the spatial entropy protects the spatial DoF by preventing it from taking on any additional fluctuations. After the second coherence conversion, spatial coherence is readily retrieved, and the effect of spatial phase scrambling circumvented.

Impact of deep learning-based image super-resolution on binary signal detection.

Purpose: Deep learning-based image super-resolution (DL-SR) has shown great promise in medical imaging applications. To date, most of the proposed methods for DL-SR have only been assessed using traditional measures of image quality (IQ) that are commonly employed in the field of computer vision. However, the impact of these methods on objective measures of IQ that are relevant to medical imaging tasks remains largely unexplored. We investigate the impact of DL-SR methods on binary signal detection performance. Approach: Two popular DL-SR methods, the super-resolution convolutional neural network and the super-resolution generative adversarial network, were trained using simulated medical image data. Binary signal-known-exactly with background-known-statistically and signal-known-statistically with background-known-statistically detection tasks were formulated. Numerical observers (NOs), which included a neural network-approximated ideal observer and common linear NOs, were employed to assess the impact of DL-SR on task performance. The impact of the complexity of the DL-SR network architectures on task performance was quantified. In addition, the utility of DL-SR for improving the task performance of suboptimal observers was investigated. Results: Our numerical experiments confirmed that, as expected, DL-SR improved traditional measures of IQ. However, for many of the study designs considered, the DL-SR methods provided little or no improvement in task performance and even degraded it. It was observed that DL-SR improved the task performance of suboptimal observers under certain conditions. Conclusions: Our study highlights the urgent need for the objective assessment of DL-SR methods and suggests avenues for improving their efficacy in medical imaging applications.

Degradation of Polyvinyl Alcohol in US Wastewater Treatment Plants and Subsequent Nationwide Emission Estimate.

Polyvinyl alcohol (PVA) is a water-soluble plastic commercially used in laundry and dish detergent pods (LDPs) for which a complete understanding of its fate in the environment and subsequent consequences is lacking. The objective of this study was to estimate the US nationwide emissions of PVA resulting from domestic use of LDPs, corroborated by a nationwide, online consumer survey and a literature review of its fate within conventional wastewater treatment plants (WWTPs). Peer-reviewed publications focusing on the degradation of PVA in critical processes of WWTPs were shortlisted as a part of the literature review, and subsequent degradation data was extracted and applied to a model with a set of assumptions. Survey and model results estimated that approximately 17,200 ± 5000 metric ton units per year (mtu/yr) of PVA are used from LDPs in the US, with 10,500 ± 3000 mtu/yr reaching WWTPs. Literature review data, when incorporated into our model, resulted in ~61% of PVA ending up in the environment via the sludge route and ~15.7% via the aqueous phase. PVA presence in the environment, regardless of its matrix, is a threat to the ecosystem due to the potential mobilization of heavy metals and other hydrophilic contaminants.

On Hallucinations in Tomographic Image Reconstruction.

Tomographic image reconstruction is generally an ill-posed linear inverse problem. Such ill-posed inverse problems are typically regularized using prior knowledge of the sought-after object property. Recently, deep neural networks have been actively investigated for regularizing image reconstruction problems by learning a prior for the object properties from training images. However, an analysis of the prior information learned by these deep networks and their ability to generalize to data that may lie outside the training distribution is still being explored. An inaccurate prior might lead to false structures being hallucinated in the reconstructed image and that is a cause for serious concern in medical imaging. In this work, we propose to illustrate the effect of the prior imposed by a reconstruction method by decomposing the image estimate into generalized measurement and null components. The concept of a hallucination map is introduced for the general purpose of understanding the effect of the prior in regularized reconstruction methods. Numerical studies are conducted corresponding to a stylized tomographic imaging modality. The behavior of different reconstruction methods under the proposed formalism is discussed with the help of the numerical studies.

Compressible Latent-Space Invertible Networks for Generative Model-Constrained Image Reconstruction.

There remains an important need for the development of image reconstruction methods that can produce diagnostically useful images from undersampled measurements. In magnetic resonance imaging (MRI), for example, such methods can facilitate reductions in data-acquisition times. Deep learning-based methods hold potential for learning object priors or constraints that can serve to mitigate the effects of data-incompleteness on image reconstruction. One line of emerging research involves formulating an optimization-based reconstruction method in the latent space of a generative deep neural network. However, when generative adversarial networks (GANs) are employed, such methods can result in image reconstruction errors if the sought-after solution does not reside within the range of the GAN. To circumvent this problem, in this work, a framework for reconstructing images from incomplete measurements is proposed that is formulated in the latent space of invertible neural network-based generative models. A novel regularization strategy is introduced that takes advantage of the multiscale architecture of certain invertible neural networks, which can result in improved reconstruction performance over classical methods in terms of traditional metrics. The proposed method is investigated for reconstructing images from undersampled MRI data. The method is shown to achieve comparable performance to a state-of-the-art generative model-based reconstruction method while benefiting from a deterministic reconstruction procedure and easier control over regularization parameters.

Nationwide Mass Inventory and Degradation Assessment of Plastic Contact Lenses in US Wastewater.

Plastics pose ecological and human health risks, with disposable contact lenses constituting a potential high-volume pollution source. Using sales data and an online survey of lens users (n = 416) alongside laboratory and field experiments at a conventional sewage treatment plant, we determined the environmental fate and mass inventories of contact lenses in the United States. The survey results revealed that 21 ± 0.8% of lens users flush their used lenses down the drain, a loading equivalent to 44 000 ± 1700 kg y-1 of lens dry mass discharged into US wastewater. Biological treatment of wastewater did not result in a measurable loss of plastic mass (p = 0.001) and caused only very limited changes in the polymer structure, as determined by µ-Raman spectroscopy. During sewage treatment, the lenses were found to accumulate as fragments in sewage sludge, resulting in an estimated accumulation of 24 000 ± 940 kg y-1 of microplastics destined for application on US agricultural soils contained in sewage sludge. Recycling of the contact lenses and their packaging amounted to only 0.04% of the total waste volume associated with contact lens use. This is the first study to identify contact lenses and more specifically silicone hydrogels, as a previously overlooked source of plastic and microplastic pollution.

Harmonic optical tomography of nonlinear structures.

Second-harmonic generation microscopy is a valuable label-free modality for imaging non-centrosymmetric structures and has important biomedical applications from live-cell imaging to cancer diagnosis. Conventional second-harmonic generation microscopy measures intensity signals that originate from tightly focused laser beams, preventing researchers from solving the scattering inverse problem for second-order nonlinear materials. Here, we present harmonic optical tomography (HOT) as a novel modality for imaging microscopic, nonlinear and inhomogeneous objects. The HOT principle of operation relies on inter-ferometrically measuring the complex harmonic field and using a scattering inverse model to reconstruct the three-dimensional distribution of harmonophores. HOT enables strong axial sectioning via the momentum conservation of spatially and temporally broadband fields. We illustrate the HOT operation with experiments and reconstructions on a beta-barium borate crystal and various biological specimens. Although our results involve second-order nonlinear materials, we show that this approach applies to any coherent nonlinear process.

Demonstration of flat-top beam illumination in widefield multiphoton microscopy.

Multiphoton microscopy provides a suitable technique for imaging biological tissues with submicrometer resolution. Usually a Gaussian beam (GB) is used for illumination, leading to a reduced power efficiency in the multiphoton response and vignetting for a square-shaped imaging area. A flat-top beam (FTB) provides a uniform spatial intensity distribution that equalizes the probability of a multiphoton effect across the imaging area. We employ a customized widefield multiphoton microscope to compare the performance of a square-shaped FTB illumination with that based on using a GB, for both two-photon fluorescence (TPF) and second-harmonic generation (SHG) imaging. The variation in signal-to-noise ratio across TPF images of fluorescent dyes spans ∼5.6 dB for the GB and ∼1.2 dB for the FTB illumination, respectively. For the GB modality, TPF images of mouse colon and Convallaria root, and SHG images of chicken tendon and human breast biopsy tissue showcase ∼20 % area that are not imaged due to either insufficient or lack of illumination. For quantitative analysis that depends on the illuminated area, this effect can potentially lead to inaccuracies. This work emphasizes the applicability of FTB illumination to multiphoton applications.

Chemical and physical changes of microplastics during sterilization by chlorination.

Wastewater treatment plants are known to release microplastics that have been detected in aquatic and terrestrial organisms constituting part of the human diet. Chlorination of wastewater-borne microplastics was hypothesized to induce chemical and physical changes detectable by Raman spectroscopy and differential scanning calorimetry (DSC). In the laboratory, virgin plastics (∼0.05 × 2 × 2 mm) were exposed to differing sterilization conditions representative of dosages used in the disinfection of drinking water, wastewater, and heavily contaminated surfaces. Polypropylene (PP) was most resistant to chlorination, followed by high density polyethylene (HDPE) and polystyrene (PS). Polystyrene showed degradation, indicated by changes in Raman peak widths, at concentration-time regimes (CT values) as low as 75 mg min/L, whereas HDPE and PP remained unaltered even at chlorine doses characteristic of wastewater disinfection (150 mg min/L). However, HDPE and PS were not completely resistant to oxidative attack by chlorination. Under extremely harsh conditions, shifts in Raman peaks and the formation of new bonds were observed. These results show that plastics commonly used in consumer products can be chemically altered, some even under conditions prevailing during wastewater treatment. Changes in polymer properties, observed for HDPE and PP under extreme exposure conditions only, are predicted to alter the risk microplastics pose to aquatic and terrestrial biota, since an increase in carbon-chlorine (C-Cl) bonds is known to increase toxicity, rendering the polymers more hydrophobic and thus more prone to adsorb, accumulate, and transport harmful persistent pollutants to biota in both aquatic and terrestrial environments.

Second-harmonic patterned polarization-analyzed reflection confocal microscopy of stromal collagen in benign and malignant breast tissues.

We present the results of polarimetric analysis of collagen on varying pathologies of breast tissues using second-harmonic patterned polarization-analyzed reflection confocal (SPPARC) microscopy. Experiments are conducted on a breast tissue microarray having benign tissues (BT), malignant invasive lobular carcinoma (ILC), and benign stroma adjacent to the malignant tissues (called the benign adjacent tissue, or BAT). Stroma in BAT and ILC exhibit the largest parameter differences. We observe that stromal collagen readings in ILC show lower depolarization, lower diattenuation and higher linear degree-of-polarization values than stromal collagen in BAT. This suggests that the optical properties of collagen change most in the vicinity of tumors. A similar trend is also exhibited in the non-collagenous extrafibrillar matrix plus cells (EFMC) region. The three highlighted parameters show greatest sensitivity to changes in the polarization response of collagen between pathologies.

Measurement of differential modal group delay of a few-mode fiber using a Fourier domain mode-locked laser.

We propose and experimentally demonstrate a method for measuring the differential modal group delay (DMGD) of a few-mode fiber using a Fourier domain mode-locked laser (FDML). We use the fast frequency-swept, wavelength-tunable output of the FDML in order to perform time domain measurements of interference of the modes, which is further postprocessed to extract the DMGD. We demonstrate the measurement of DMGD for a commercial two-mode fiber over the C-band. This method is not limited by the magnitude of DMGD or the number of modes and is minimally affected by time-dependent polarization and mode fluctuations, environmental noise, and spectral resolution of instruments.