Radiance and photon noise: imaging in geometrical optics, physical optics, quantum optics and radiology.

The statistics of detector outputs produced by an imaging system are derived from basic radiometric concepts and definitions. We show that a fundamental way of describing a photon-limited imaging system is in terms of a Poisson random process in spatial, angular, and wavelength variables. We begin the paper by recalling the concept of radiance in geometrical optics, radiology, physical optics, and quantum optics. The propagation and conservation laws for radiance in each of these domains are reviewed. Building upon these concepts, we distinguish four categories of imaging detectors that all respond in some way to the incident radiance, including the new category of photon-processing detectors (capable of measuring radiance on a photon-by-photon basis). This allows us to rigorously show how the concept of radiance is related to the statistical properties of detector outputs and to the information content of a single detected photon. A Monte-Carlo technique, which is derived from the Boltzmann transport equation, is presented as a way to estimate probability density functions to be used in reconstruction from photon-processing data.

Physics-informed motion registration of lung parenchyma across static CT images.

Lung injuries, such as ventilator-induced lung injury and radiation-induced lung injury, can lead to heterogeneous alterations in the biomechanical behavior of the lungs. While imaging methods, e.g., X-ray and static computed tomography (CT), can point to regional alterations in lung structure between healthy and diseased tissue, they fall short of delineating timewise kinematic variations between the former and the latter. Image registration has gained recent interest as a tool to estimate the displacement experienced by the lungs during respiration via regional deformation metrics such as volumetric expansion and distortion. However, successful image registration commonly relies on a temporal series of image stacks with small displacements in the lungs across succeeding image stacks, which remains limited in static imaging. In this study, we have presented a finite element (FE) method to estimate strains from static images acquired at the end-expiration (EE) and end-inspiration (EI) timepoints, i.e., images with a large deformation between the two distant timepoints. Physiologically realistic loads were applied to the geometry obtained at EE to deform this geometry to match the geometry obtained at EI. The results indicated that the simulation could minimize the error between the two geometries. Using four-dimensional (4D) dynamic CT in a rat, the strain at an isolated transverse plane estimated by our method showed sufficient agreement with that estimated through non-rigid image registration that used all the timepoints. Through the proposed method, we can estimate the lung deformation at any timepoint between EE and EI. The proposed method offers a tool to estimate timewise regional deformation in the lungs using only static images acquired at EE and EI.

Assessing the impact of nodule features and software algorithm on pulmonary nodule measurement uncertainty for nodules sized 20 mm or less.

Background: Measurements are not exact, so that if a measurement is repeated, one would get a different value each time. The spread of these values is the measurement uncertainty. Understanding measurement uncertainty of pulmonary nodules is important for proper interpretation of size and growth measurements. Larger amounts of measurement uncertainty may require longer follow-up intervals to be confident that any observed growth is due to actual growth rather than measurement uncertainty. We examined the influence of nodule features and software algorithm on measurement uncertainty of small, solid pulmonary nodules. Methods: Volumes of 107 nodules were measured on 4-6 repeated computed tomography (CT) scans (Siemens Somatom AS, 100 kVp, 120 mA, 1.0 mm slice thickness reconstruction) prospectively obtained during CT-guided fine needle aspiration biopsy between 2015-2021 at Department of Diagnostic, Molecular, and Interventional Radiology in Icahn School of Medicine at Mount Sinai, using two different automated volumetric algorithms. For each, the coefficient of variation (standard deviation divided by the mean) of nodule volume measurements was determined. The following features were considered: diameter, location, vessel and pleural attachments, nodule surface area, and extent of the nodule in the three acquisition dimensions of the scanner. Results: Median volume of 107 nodules was 515.23 and 535.53 mm3 for algorithm #1 and #2, respectively with excellent agreement (intraclass correlation coefficient =0.98). Median coefficient of variation of nodule volume was low for the two algorithms, but significantly different (4.6% vs. 8.7%, P<0.001). Both algorithms had a trend of decreasing coefficient of variation of nodule volume with increasing nodule diameter, though only significant for algorithm #2. Coefficient of variation of nodule volume was significantly associated with nodule volume (P=0.02), attachment to blood vessels (P=0.02), and nodule surface area (P=0.001) for algorithm #2 using a multiple linear regression model. Correlation between the coefficient of variation (CoV) of nodule volume and the CoV of the x, y, z measurements for algorithm #1 were 0.29 (P=0.0021), 0.25 (P=0.009), and 0.80 (P<0.001) respectively, and for algorithm #2, 0.46 (P<0.001), 0.52 (P<0.001), and 0.58 (P<0.001), respectively. Conclusions: Even in the best-case scenario represented in this study, using the same measurement algorithm, scanner, and scanning protocol, considerable measurement uncertainty exists in nodule volume measurement for nodules sized 20 mm or less. We found that measurement uncertainty was affected by interactions between nodule volume, algorithm, and shape complexity.

Report on the AAPM Grand Challenge on deep generative modeling for learning medical image statistics.

Background: The findings of the 2023 AAPM Grand Challenge on Deep Generative Modeling for Learning Medical Image Statistics are reported in this Special Report. Purpose: The goal of this challenge was to promote the development of deep generative models for medical imaging and to emphasize the need for their domain-relevant assessments via the analysis of relevant image statistics. Methods: As part of this Grand Challenge, a common training dataset and an evaluation procedure was developed for benchmarking deep generative models for medical image synthesis. To create the training dataset, an established 3D virtual breast phantom was adapted. The resulting dataset comprised about 108,000 images of size 512×512. For the evaluation of submissions to the Challenge, an ensemble of 10,000 DGM-generated images from each submission was employed. The evaluation procedure consisted of two stages. In the first stage, a preliminary check for memorization and image quality (via the Fréchet Inception Distance (FID)) was performed. Submissions that passed the first stage were then evaluated for the reproducibility of image statistics corresponding to several feature families including texture, morphology, image moments, fractal statistics and skeleton statistics. A summary measure in this feature space was employed to rank the submissions. Additional analyses of submissions was performed to assess DGM performance specific to individual feature families, the four classes in the training data, and also to identify various artifacts. Results: Fifty-eight submissions from 12 unique users were received for this Challenge. Out of these 12 submissions, 9 submissions passed the first stage of evaluation and were eligible for ranking. The top-ranked submission employed a conditional latent diffusion model, whereas the joint runners-up employed a generative adversarial network, followed by another network for image superresolution. In general, we observed that the overall ranking of the top 9 submissions according to our evaluation method (i) did not match the FID-based ranking, and (ii) differed with respect to individual feature families. Another important finding from our additional analyses was that different DGMs demonstrated similar kinds of artifacts. Conclusions: This Grand Challenge highlighted the need for domain-specific evaluation to further DGM design as well as deployment. It also demonstrated that the specification of a DGM may differ depending on its intended use.

Dissecting contributions of pulmonary arterial remodeling to right ventricular afterload in pulmonary hypertension.

Pulmonary hypertension (PH) is defined as an elevation in the right ventricle (RV) afterload, characterized by increased hemodynamic pressure in the main pulmonary artery (PA). Elevations in RV afterload increase RV wall stress, resulting in RV remodeling and potentially RV failure. From a biomechanical standpoint, the primary drivers for RV afterload elevations include increases in pulmonary vascular resistance (PVR) in the distal vasculature and decreases in vessel compliance in the proximal PA. However, the individual contributions of the various vascular remodeling events toward the progression of PA pressure elevations and altered vascular hemodynamics remain elusive. In this study, we used a subject-specific one-dimensional (1D) fluid-structure interaction (FSI) model to investigate the alteration of pulmonary hemodynamics in PH and to quantify the contributions of vascular stiffening and increased resistance towards increased main pulmonary artery (MPA) pressure. We used a combination of subject-specific hemodynamic measurements, ex-vivo mechanical testing of arterial tissue specimens, and ex-vivo X-ray micro-tomography imaging to develop the 1D-FSI model and dissect the contribution of PA remodeling events towards alterations in the MPA pressure waveform. Both the amplitude and pulsatility of the MPA pressure waveform were analyzed. Our results indicated that increased distal resistance has the greatest effect on the increase in maximum MPA pressure, while increased stiffness caused significant elevations in the characteristic impedance. The method presented in this study will serve as an essential step toward understanding the complex interplay between PA remodeling events that leads to the most severe adverse effect on RV dysfunction.

MIDRC-MetricTree: a decision tree-based tool for recommending performance metrics in artificial intelligence-assisted medical image analysis.

Purpose: The Medical Imaging and Data Resource Center (MIDRC) was created to facilitate medical imaging machine learning (ML) research for tasks including early detection, diagnosis, prognosis, and assessment of treatment response related to the coronavirus disease 2019 pandemic and beyond. The purpose of this work was to create a publicly available metrology resource to assist researchers in evaluating the performance of their medical image analysis ML algorithms. Approach: An interactive decision tree, called MIDRC-MetricTree, has been developed, organized by the type of task that the ML algorithm was trained to perform. The criteria for this decision tree were that (1) users can select information such as the type of task, the nature of the reference standard, and the type of the algorithm output and (2) based on the user input, recommendations are provided regarding appropriate performance evaluation approaches and metrics, including literature references and, when possible, links to publicly available software/code as well as short tutorial videos. Results: Five types of tasks were identified for the decision tree: (a) classification, (b) detection/localization, (c) segmentation, (d) time-to-event (TTE) analysis, and (e) estimation. As an example, the classification branch of the decision tree includes two-class (binary) and multiclass classification tasks and provides suggestions for methods, metrics, software/code recommendations, and literature references for situations where the algorithm produces either binary or non-binary (e.g., continuous) output and for reference standards with negligible or non-negligible variability and unreliability. Conclusions: The publicly available decision tree is a resource to assist researchers in conducting task-specific performance evaluations, including classification, detection/localization, segmentation, TTE, and estimation tasks.

Complete spatiotemporal quantification of cardiac motion in mice through enhanced acquisition and super-resolution reconstruction.

The quantification of cardiac motion using cardiac magnetic resonance imaging (CMR) has shown promise as an early-stage marker for cardiovascular diseases. Despite the growing popularity of CMR-based myocardial strain calculations, measures of complete spatiotemporal strains (i.e., three-dimensional strains over the cardiac cycle) remain elusive. Complete spatiotemporal strain calculations are primarily hampered by poor spatial resolution, with the rapid motion of the cardiac wall also challenging the reproducibility of such strains. We hypothesize that a super-resolution reconstruction (SRR) framework that leverages combined image acquisitions at multiple orientations will enhance the reproducibility of complete spatiotemporal strain estimation. Two sets of CMR acquisitions were obtained for five wild-type mice, combining short-axis scans with radial and orthogonal long-axis scans. Super-resolution reconstruction, integrated with tissue classification, was performed to generate full four-dimensional (4D) images. The resulting enhanced and full 4D images enabled complete quantification of the motion in terms of 4D myocardial strains. Additionally, the effects of SRR in improving accurate strain measurements were evaluated using an in-silico heart phantom. The SRR framework revealed near isotropic spatial resolution, high structural similarity, and minimal loss of contrast, which led to overall improvements in strain accuracy. In essence, a comprehensive methodology was generated to quantify complete and reproducible myocardial deformation, aiding in the much-needed standardization of complete spatiotemporal strain calculations.

Sequestration of imaging studies in MIDRC: stratified sampling to balance demographic characteristics of patients in a multi-institutional data commons.

Purpose: The Medical Imaging and Data Resource Center (MIDRC) is a multi-institutional effort to accelerate medical imaging machine intelligence research and create a publicly available image repository/commons as well as a sequestered commons for performance evaluation and benchmarking of algorithms. After de-identification, approximately 80% of the medical images and associated metadata become part of the open commons and 20% are sequestered from the open commons. To ensure that both commons are representative of the population available, we introduced a stratified sampling method to balance the demographic characteristics across the two datasets. Approach: Our method uses multi-dimensional stratified sampling where several demographic variables of interest are sequentially used to separate the data into individual strata, each representing a unique combination of variables. Within each resulting stratum, patients are assigned to the open or sequestered commons. This algorithm was used on an example dataset containing 5000 patients using the variables of race, age, sex at birth, ethnicity, COVID-19 status, and image modality and compared resulting demographic distributions to naïve random sampling of the dataset over 2000 independent trials. Results: Resulting prevalence of each demographic variable matched the prevalence from the input dataset within one standard deviation. Mann-Whitney U test results supported the hypothesis that sequestration by stratified sampling provided more balanced subsets than naïve randomization, except for demographic subcategories with very low prevalence. Conclusions: The developed multi-dimensional stratified sampling algorithm can partition a large dataset while maintaining balance across several variables, superior to the balance achieved from naïve randomization.

Assessing the Ability of Generative Adversarial Networks to Learn Canonical Medical Image Statistics.

In recent years, generative adversarial networks (GANs) have gained tremendous popularity for potential applications in medical imaging, such as medical image synthesis, restoration, reconstruction, translation, as well as objective image quality assessment. Despite the impressive progress in generating high-resolution, perceptually realistic images, it is not clear if modern GANs reliably learn the statistics that are meaningful to a downstream medical imaging application. In this work, the ability of a state-of-the-art GAN to learn the statistics of canonical stochastic image models (SIMs) that are relevant to objective assessment of image quality is investigated. It is shown that although the employed GAN successfully learned several basic first- and second-order statistics of the specific medical SIMs under consideration and generated images with high perceptual quality, it failed to correctly learn several per-image statistics pertinent to the these SIMs, highlighting the urgent need to assess medical image GANs in terms of objective measures of image quality.

Discrimination tasks in simulated low-dose CT noise.

BACKGROUND: This study reports the results of a set of discrimination experiments using simulated images that represent the appearance of subtle lesions in low-dose computed tomography (CT) of the lungs. Noise in these images has a characteristic ramp-spectrum before apodization by noise control filters. We consider three specific diagnostic features that determine whether a lesion is considered malignant or benign, two system-resolution levels, and four apodization levels for a total of 24 experimental conditions. PURPOSE: The goal of the investigation is to better understand how well human observers perform subtle discrimination tasks like these, and the mechanisms of that performance. We use a forced-choice psychophysical paradigm to estimate observer efficiency and classification images. These measures quantify how effectively subjects can read the images, and how they use images to perform discrimination tasks across the different imaging conditions. MATERIALS AND METHODS: The simulated CT images used as stimuli in the psychophysical experiments are generated from high-resolution objects passed through a modulation transfer function (MTF) before down-sampling to the image-pixel grid. Acquisition noise is then added with a ramp noise-power spectrum (NPS), with subsequent smoothing through apodization filters. The features considered are lesion size, indistinct lesion boundary, and a nonuniform lesion interior. System resolution is implemented by an MTF with resolution (10% max.) of 0.47 or 0.58 cyc/mm. Apodization is implemented by a Shepp-Logan filter (Sinc profile) with various cutoffs. Six medically naïve subjects participated in the psychophysical studies, entailing training and testing components for each condition. Training consisted of staircase procedures to find the 80% correct threshold for each subject, and testing involved 2000 psychophysical trials at the threshold value for each subject. Human-observer performance is compared to the Ideal Observer to generate estimates of task efficiency. The significance of imaging factors is assessed using ANOVA. Classification images are used to estimate the linear template weights used by subjects to perform these tasks. Classification-image spectra are used to analyze subject weights in the spatial-frequency domain. RESULTS: Overall, average observer efficiency is relatively low in these experiments (10%-40%) relative to detection and localization studies reported previously. We find significant effects for feature type and apodization level on observer efficiency. Somewhat surprisingly, system resolution is not a significant factor. Efficiency effects of the different features appear to be well explained by the profile of the linear templates in the classification images. Increasingly strong apodization is found to both increase the classification-image weights and to increase the mean-frequency of the classification-image spectra. A secondary analysis of "Unapodized" classification images shows that this is largely due to observers undoing (inverting) the effects of apodization filters. CONCLUSIONS: These studies demonstrate that human observers can be relatively inefficient at feature-discrimination tasks in ramp-spectrum noise. Observers appear to be adapting to frequency suppression implemented in apodization filters, but there are residual effects that are not explained by spatial weighting patterns. The studies also suggest that the mechanisms for improving performance through the application of noise-control filters may require further investigation.