High-precision inversion of dynamic radiography using hydrodynamic features.

While radiography is routinely used to probe complex, evolving density fields in research areas ranging from materials science to shock physics to inertial confinement fusion and other national security applications, complications resulting from noise, scatter, complex beam dynamics, etc. prevent current methods of reconstructing density from being accurate enough to identify the underlying physics with sufficient confidence. In this work, we show that using only features that are robustly identifiable in radiographs and combining them with the underlying hydrodynamic equations of motion using a machine learning approach of a conditional generative adversarial network (cGAN) provides a new and effective approach to determine density fields from a dynamic sequence of radiographs. In particular, we demonstrate the ability of this method to outperform a traditional, direct radiograph to density reconstruction in the presence of scatter, even when relatively small amounts of scatter are present. Our experiments on synthetic data show that the approach can produce high quality, robust reconstructions. We also show that the distance (in feature space) between a testing radiograph and the training set can serve as a diagnostic of the accuracy of the reconstruction.

Physics-based iterative reconstruction for dual-source and flying focal spot computed tomography.

PURPOSE: For single-source helical Computed Tomography (CT), both Filtered-Back Projection (FBP) and statistical iterative reconstruction have been investigated. However, for dual-source CT with flying focal spot (DS-FFS CT), a statistical iterative reconstruction that accurately models the scanner geometry and acquisition physics remains unknown to researchers. Therefore, our purpose is to present a novel physics-based iterative reconstruction method for DS-FFS CT and assess its image quality. METHODS: Our algorithm uses precise physics models to reconstruct from the native cone-beam geometry and interleaved dual-source helical trajectory of a DS-FFS CT. To do so, we construct a noise physics model to represent data acquisition noise and a prior image model to represent image noise and texture. In addition, we design forward system models to compute the locations of deflected focal spots, the dimension, and sensitivity of voxels and detector units, as well as the length of intersection between x-rays and voxels. The forward system models further represent the coordinated movement between the dual sources by computing their x-ray coverage gaps and overlaps at an arbitrary helical pitch. With the above models, we reconstruct images by an advanced Consensus Equilibrium (CE) numerical method to compute the maximum a posteriori estimate to a joint optimization problem that simultaneously fits all models. RESULTS: We compared our reconstruction with Siemens ADMIRE, which is the clinical standard hybrid iterative reconstruction (IR) method for DS-FFS CT, in terms of spatial resolution, noise profile, and image artifacts through both phantoms and clinical scan datasets. Experiments show that our reconstruction has a higher spatial resolution, with a Task-Based Modulation Transfer Function (MTFtask ) consistently higher than the clinical standard hybrid IR. In addition, our reconstruction shows a reduced magnitude of image undersampling artifacts than the clinical standard. CONCLUSIONS: By modeling a precise geometry and avoiding data rebinning or interpolation, our physics-based reconstruction achieves a higher spatial resolution and fewer image artifacts with smaller magnitude than the clinical standard hybrid IR.

Multiagent Consensus Equilibrium in Molecular Structure Determination.

Multiagent consensus equilibrium (MACE) is demonstrated for the integration of experimental observables as constraints in molecular structure determination and for the systematic merging of multiple computational architectures. MACE is founded on simultaneously determining the equilibrium point between multiple experimental and/or computational agents; the returned state description (e.g., atomic coordinates for molecular structure) represents the intersection of each manifold and is not equivalent to the average optimum state for each agent. The moment of inertia, determined directly from microwave spectroscopy measurements, serves to illustrate the mechanism through which MACE evaluations merge experimental and quantum chemical modeling. MACE results reported combine gradient descent optimization of each ab initio agent with an agent that predicts the chemical structure based on root-mean-square deviation of the predicted inertia tensor with experimentally measured moments of inertia. Successful model fusion for several small molecules was achieved as well as the larger molecule solketal. Fusing a model of moment of inertia, an underdetermined predictor of structure, with low cost computational methods yielded structure determination performance comparable to standard computational methods such as MP2/cc-pVTZ and greater agreement with experimental observables.

Plug-and-Play Methods for Magnetic Resonance Imaging: Using Denoisers for Image Recovery.

Magnetic Resonance Imaging (MRI) is a non-invasive diagnostic tool that provides excellent soft-tissue contrast without the use of ionizing radiation. Compared to other clinical imaging modalities (e.g., CT or ultrasound), however, the data acquisition process for MRI is inherently slow, which motivates undersampling and thus drives the need for accurate, efficient reconstruction methods from undersampled datasets. In this article, we describe the use of "plug-and-play" (PnP) algorithms for MRI image recovery. We first describe the linearly approximated inverse problem encountered in MRI. Then we review several PnP methods, where the unifying commonality is to iteratively call a denoising subroutine as one step of a larger optimization-inspired algorithm. Next, we describe how the result of the PnP method can be interpreted as a solution to an equilibrium equation, allowing convergence analysis from the equilibrium perspective. Finally, we present illustrative examples of PnP methods applied to MRI image recovery.

Imaging through distributed-volume aberrations using single-shot digital holography.

This paper explores the use of single-shot digital holography data and a novel algorithm, referred to as multiplane iterative reconstruction (MIR), for imaging through distributed-volume aberrations. Such aberrations result in a linear, shift-varying or "anisoplanatic" physical process, where multiple-look angles give rise to different point spread functions within the field of view of the imaging system. The MIR algorithm jointly computes the maximum a posteriori estimates of the anisoplanatic phase errors and the speckle-free object reflectance from the single-shot digital holography data. Using both simulations and experiments, we show that the MIR algorithm outperforms the leading multiplane image-sharpening algorithm over a wide range of anisoplanatic conditions.

MBIR: A cryo-ET 3D reconstruction method that effectively minimizes missing wedge artifacts and restores missing information.

Cryo-Electron Tomography (cryo-ET) has become an essential technique in revealing cellular and macromolecular assembly structures in their native states. However, due to radiation damage and the limited tilt range, cryo-ET suffers from low contrast and missing wedge artifacts, which limits the tomograms to low resolution and hinders further biological interpretation. In this study, we applied the Model-Based Iterative Reconstruction (MBIR) method to obtain tomographic 3D reconstructions of experimental cryo-ET datasets and demonstrated the advantages of MBIR in contrast improvement, missing wedge artifacts reduction, missing information restoration, and subtomogram averaging compared with other reconstruction approaches. Considering the outstanding reconstruction quality, MBIR has a great potential in the determination of high resolution biological structures with cryo-ET.

Dynamic Sparse Sampling for Confocal Raman Microscopy.

The total number of data points required for image generation in Raman microscopy was greatly reduced using sparse sampling strategies, in which the preceding set of measurements informed the next most information-rich sampling location. Using this approach, chemical images of pharmaceutical materials were obtained with >99% accuracy from 15.8% sampling, representing an ∼6-fold reduction in measurement time relative to full field of view rastering with comparable image quality. This supervised learning approach to dynamic sampling (SLADS) has the distinct advantage of being directly compatible with standard confocal Raman instrumentation. Furthermore, SLADS is not limited to Raman imaging, potentially providing time-savings in image reconstruction whenever the single-pixel measurement time is the limiting factor in image generation.

Demonstration of single-shot digital holography using a Bayesian framework.

In this paper, we present experimental results for image reconstruction, with isoplanatic phase-error correction, from single-shot digital holography data. We demonstrate the utility of using a model-based iterative reconstruction (MBIR) algorithm to jointly compute the maximum a posteriori estimates of the phase errors and the real-valued object reflectance function. Specifically, we show that the MBIR algorithm is robust to noise and phase errors over a range of conditions.

Reduced electron exposure for energy-dispersive spectroscopy using dynamic sampling.

Analytical electron microscopy and spectroscopy of biological specimens, polymers, and other beam sensitive materials has been a challenging area due to irradiation damage. There is a pressing need to develop novel imaging and spectroscopic imaging methods that will minimize such sample damage as well as reduce the data acquisition time. The latter is useful for high-throughput analysis of materials structure and chemistry. In this work, we present a novel machine learning based method for dynamic sparse sampling of EDS data using a scanning electron microscope. Our method, based on the supervised learning approach for dynamic sampling algorithm and neural networks based classification of EDS data, allows a dramatic reduction in the total sampling of up to 90%, while maintaining the fidelity of the reconstructed elemental maps and spectroscopic data. We believe this approach will enable imaging and elemental mapping of materials that would otherwise be inaccessible to these analysis techniques.

Optically coherent image formation and denoising using a plug and play inversion framework.

The performance of optically coherent imaging systems can be limited by measurement and speckle noise. In this paper, we develop an image formation framework for computing the maximum a posteriori estimate of an object's reflectivity when imaged using coherent illumination and detection. The proposed approach allows for the use of Gaussian denoising algorithms (GDAs), without modification, to mitigate the exponentially distributed and signal-dependent noise that occurs in coherent imaging. Several GDAs are compared using both simulated and experimental data. The proposed framework is shown to be robust to noise and significantly reduce reconstruction error compared to the standard inversion technique.

Phase-error estimation and image reconstruction from digital-holography data using a Bayesian framework.

The estimation of phase errors from digital-holography data is critical for applications such as imaging or wavefront sensing. Conventional techniques require multiple i.i.d. data and perform poorly in the presence of high noise or large phase errors. In this paper, we propose a method to estimate isoplanatic phase errors from a single data realization. We develop a model-based iterative reconstruction algorithm that computes the maximum a posteriori estimate of the phase and the speckle-free object reflectance. Using simulated data, we show that the algorithm is robust against high noise and strong phase errors.

Second Harmonic Generation Guided Raman Spectroscopy for Sensitive Detection of Polymorph Transitions.

Second harmonic generation (SHG) was integrated with Raman spectroscopy for the analysis of pharmaceutical materials. Particulate formulations of clopidogrel bisulfate were prepared in two crystal forms (Form I and Form II). Image analysis approaches enable automated identification of particles by bright field imaging, followed by classification by SHG. Quantitative SHG microscopy enabled discrimination of crystal form on a per particle basis with 99.95% confidence in a total measurement time of ∼10 ms per particle. Complementary measurements by Raman and synchrotron XRD are in excellent agreement with the classifications made by SHG, with measurement times of ∼1 min and several seconds per particle, respectively. Coupling these capabilities with at-line monitoring may enable real-time feedback for reaction monitoring during pharmaceutical production to favor the more bioavailable but metastable Form I with limits of detection in the ppm regime.

Dynamic X-ray diffraction sampling for protein crystal positioning.

A sparse supervised learning approach for dynamic sampling (SLADS) is described for dose reduction in diffraction-based protein crystal positioning. Crystal centering is typically a prerequisite for macromolecular diffraction at synchrotron facilities, with X-ray diffraction mapping growing in popularity as a mechanism for localization. In X-ray raster scanning, diffraction is used to identify the crystal positions based on the detection of Bragg-like peaks in the scattering patterns; however, this additional X-ray exposure may result in detectable damage to the crystal prior to data collection. Dynamic sampling, in which preceding measurements inform the next most information-rich location to probe for image reconstruction, significantly reduced the X-ray dose experienced by protein crystals during positioning by diffraction raster scanning. The SLADS algorithm implemented herein is designed for single-pixel measurements and can select a new location to measure. In each step of SLADS, the algorithm selects the pixel, which, when measured, maximizes the expected reduction in distortion given previous measurements. Ground-truth diffraction data were obtained for a 5 µm-diameter beam and SLADS reconstructed the image sampling 31% of the total volume and only 9% of the interior of the crystal greatly reducing the X-ray dosage on the crystal. Using in situ two-photon-excited fluorescence microscopy measurements as a surrogate for diffraction imaging with a 1 µm-diameter beam, the SLADS algorithm enabled image reconstruction from a 7% sampling of the total volume and 12% sampling of the interior of the crystal. When implemented into the beamline at Argonne National Laboratory, without ground-truth images, an acceptable reconstruction was obtained with 3% of the image sampled and approximately 5% of the crystal. The incorporation of SLADS into X-ray diffraction acquisitions has the potential to significantly minimize the impact of X-ray exposure on the crystal by limiting the dose and area exposed for image reconstruction and crystal positioning using data collection hardware present in most macromolecular crystallography end-stations.

Synchrotron X-Ray Diffraction Dynamic Sampling for Protein Crystal Centering.

A supervised learning approach for dynamic sampling (SLADS) was developed to reduce X-ray exposure prior to data collection in protein structure determination. Implementation of this algorithm allowed reduction of the X-ray dose to the central core of the crystal by up to 20-fold compared to current raster scanning approaches. This dose reduction corresponds directly to a reduction on X-ray damage to the protein crystals prior to data collection for structure determination. Implementation at a beamline at Argonne National Laboratory suggests promise for the use of the SLADS approach to aid in the analysis of X-ray labile crystals. The potential benefits match a growing need for improvements in automated approaches for microcrystal positioning.

Distributed Signal Decorrelation and Detection in Multi View Camera Networks Using the Vector Sparse Matrix Transform.

This paper introduces the vector sparse matrix transform (vector SMT), a new decorrelating transform suitable for performing distributed processing of high-dimensional signals in sensor networks. We assume that each sensor in the network encodes its measurements into vector outputs instead of scalar ones. The proposed transform decorrelates a sequence of pairs of vector outputs, until these vectors are decorrelated. In our experiments, we simulate distributed anomaly detection by a network of cameras, monitoring a spatial region. Each camera records an image of the monitored environment from its particular viewpoint and outputs a vector encoding the image. Our results, with both artificial and real data, show that the proposed vector SMT transform effectively decorrelates image measurements from the multiple cameras in the network while maintaining low overall communication energy consumption. Since it enables joint processing of the multiple vector outputs, our method provides significant improvements to anomaly detection accuracy when compared with the baseline case when the images are processed independently.

Multi-channel beam-scanning imaging at kHz frame rates by Lissajous trajectory microscopy.

A beam-scanning microscope based on Lissajous trajectory imaging is described for achieving streaming 2D imaging with continuous frame rates up to 1.4 kHz. The microscope utilizes two fast-scan resonant mirrors to direct the optical beam on a circuitous trajectory through the field of view. By separating the full Lissajous trajectory time-domain data into sub-trajectories (partial, undersampled trajectories) effective frame-rates much higher than the repeat time of the Lissajous trajectory are achieved with many unsampled pixels present. A model-based image reconstruction (MBIR) 3D in-painting algorithm is then used to interpolate the missing data for the unsampled pixels to recover full images. The MBIR algorithm uses a maximum a posteriori estimation with a generalized Gaussian Markov random field prior model for image interpolation. Because images are acquired using photomultiplier tubes or photodiodes, parallelization for multi-channel imaging is straightforward. Preliminary results show that when combined with the MBIR in-painting algorithm, this technique has the ability to generate kHz frame rate images across 6 total dimensions of space, time, and polarization for SHG, TPEF, and confocal reflective birefringence data on a multimodal imaging platform for biomedical imaging. The use of a multi-channel data acquisition card allows for multimodal imaging with perfect image overlay. Image blur due to sample motion was also reduced by using higher frame rates.

High frame-rate multichannel beam-scanning microscopy based on Lissajous trajectories.

A simple beam-scanning optical design based on Lissajous trajectory imaging is described for achieving up to kHz frame-rate optical imaging on multiple simultaneous data acquisition channels. In brief, two fast-scan resonant mirrors direct the optical beam on a circuitous trajectory through the field of view, with the trajectory repeat-time given by the least common multiplier of the mirror periods. Dicing the raw time-domain data into sub-trajectories combined with model-based image reconstruction (MBIR) 3D in-painting algorithms allows for effective frame-rates much higher than the repeat time of the Lissajous trajectory. Since sub-trajectory and full-trajectory imaging are simply different methods of analyzing the same data, both high-frame rate images with relatively low resolution and low frame rate images with high resolution are simultaneously acquired. The optical hardware required to perform Lissajous imaging represents only a minor modification to established beam-scanning hardware, combined with additional control and data acquisition electronics. Preliminary studies based on laser transmittance imaging and polarization-dependent second harmonic generation microscopy support the viability of the approach both for detection of subtle changes in large signals and for trace-light detection of transient fluctuations.

Fast space-varying convolution using matrix source coding with applications to camera stray light reduction.

Many imaging applications require the implementation of space-varying convolution for accurate restoration and reconstruction of images. Here, we use the term space-varying convolution to refer to linear operators whose impulse response has slow spatial variation. In addition, these space-varying convolution operators are often dense, so direct implementation of the convolution operator is typically computationally impractical. One such example is the problem of stray light reduction in digital cameras, which requires the implementation of a dense space-varying deconvolution operator. However, other inverse problems, such as iterative tomographic reconstruction, can also depend on the implementation of dense space-varying convolution. While space-invariant convolution can be efficiently implemented with the fast Fourier transform, this approach does not work for space-varying operators. So direct convolution is often the only option for implementing space-varying convolution. In this paper, we develop a general approach to the efficient implementation of space-varying convolution, and demonstrate its use in the application of stray light reduction. Our approach, which we call matrix source coding, is based on lossy source coding of the dense space-varying convolution matrix. Importantly, by coding the transformation matrix, we not only reduce the memory required to store it; we also dramatically reduce the computation required to implement matrix-vector products. Our algorithm is able to reduce computation by approximately factoring the dense space-varying convolution operator into a product of sparse transforms. Experimental results show that our method can dramatically reduce the computation required for stray light reduction while maintaining high accuracy.

Quality and precision of parametric images created from PET sinogram data by direct reconstruction: proof of concept.

We have previously implemented the direct reconstruction of dense kinetic model parameter images ("parametric images") from sinogram data, and compared it to conventional image domain kinetic parameter estimation methods . Although it has been shown that the direct reconstruction algorithm estimates the kinetic model parameters with lower root mean squared error than the conventional image domain techniques, some theoretical obstacles remain. These obstacles include the difficulty of evaluating the accuracy and precision of the estimated parameters. In image domain techniques, the reconstructed time activity curve (TAC) and the model predicted TAC are compared, and the goodness-of-fit is evaluated as a measure of the accuracy and precision of the estimated parameters. This approach cannot be applied to the direct reconstruction technique as there are no reconstructed TACs. In this paper, we propose ways of evaluating the precision and goodness-of-fit of the kinetic model parameters estimated by the direct reconstruction algorithm. Specifically, precision of the estimates requires the calculation of variance images for each parameter, and goodness-of-fit is addressed by reconstructing the difference between the measured and the fitted sinograms. We demonstrate that backprojecting the difference from sinogram space to image space creates error images that can be examined for goodness-of-fit and model selection purposes. The presence of nonrandom structures in the error images may indicate an inadequacy of the kinetic model that has been incorporated into the direct reconstruction algorithm. We introduce three types of goodness-of-fit images. We propose and demonstrate a number-of-runs image as a means of quantifying the adequacy or deficiency of the model. We further propose and demonstrate images of the F statistic and the change in the Akaike Information Criterion as devices for identifying the statistical advantage of one model over another at each voxel. As direct reconstruction to parametric images proliferates, it will be essential for imagers to adopt methods such as those proposed herein to assess the accuracy and precision of their parametric images.