High-precision inversion of dynamic radiography using hydrodynamic features.

While radiography is routinely used to probe complex, evolving density fields in research areas ranging from materials science to shock physics to inertial confinement fusion and other national security applications, complications resulting from noise, scatter, complex beam dynamics, etc. prevent current methods of reconstructing density from being accurate enough to identify the underlying physics with sufficient confidence. In this work, we show that using only features that are robustly identifiable in radiographs and combining them with the underlying hydrodynamic equations of motion using a machine learning approach of a conditional generative adversarial network (cGAN) provides a new and effective approach to determine density fields from a dynamic sequence of radiographs. In particular, we demonstrate the ability of this method to outperform a traditional, direct radiograph to density reconstruction in the presence of scatter, even when relatively small amounts of scatter are present. Our experiments on synthetic data show that the approach can produce high quality, robust reconstructions. We also show that the distance (in feature space) between a testing radiograph and the training set can serve as a diagnostic of the accuracy of the reconstruction.

Multiagent Consensus Equilibrium in Molecular Structure Determination.

Multiagent consensus equilibrium (MACE) is demonstrated for the integration of experimental observables as constraints in molecular structure determination and for the systematic merging of multiple computational architectures. MACE is founded on simultaneously determining the equilibrium point between multiple experimental and/or computational agents; the returned state description (e.g., atomic coordinates for molecular structure) represents the intersection of each manifold and is not equivalent to the average optimum state for each agent. The moment of inertia, determined directly from microwave spectroscopy measurements, serves to illustrate the mechanism through which MACE evaluations merge experimental and quantum chemical modeling. MACE results reported combine gradient descent optimization of each ab initio agent with an agent that predicts the chemical structure based on root-mean-square deviation of the predicted inertia tensor with experimentally measured moments of inertia. Successful model fusion for several small molecules was achieved as well as the larger molecule solketal. Fusing a model of moment of inertia, an underdetermined predictor of structure, with low cost computational methods yielded structure determination performance comparable to standard computational methods such as MP2/cc-pVTZ and greater agreement with experimental observables.

Plug-and-Play Methods for Magnetic Resonance Imaging: Using Denoisers for Image Recovery.

Magnetic Resonance Imaging (MRI) is a non-invasive diagnostic tool that provides excellent soft-tissue contrast without the use of ionizing radiation. Compared to other clinical imaging modalities (e.g., CT or ultrasound), however, the data acquisition process for MRI is inherently slow, which motivates undersampling and thus drives the need for accurate, efficient reconstruction methods from undersampled datasets. In this article, we describe the use of "plug-and-play" (PnP) algorithms for MRI image recovery. We first describe the linearly approximated inverse problem encountered in MRI. Then we review several PnP methods, where the unifying commonality is to iteratively call a denoising subroutine as one step of a larger optimization-inspired algorithm. Next, we describe how the result of the PnP method can be interpreted as a solution to an equilibrium equation, allowing convergence analysis from the equilibrium perspective. Finally, we present illustrative examples of PnP methods applied to MRI image recovery.

MBIR: A cryo-ET 3D reconstruction method that effectively minimizes missing wedge artifacts and restores missing information.

Cryo-Electron Tomography (cryo-ET) has become an essential technique in revealing cellular and macromolecular assembly structures in their native states. However, due to radiation damage and the limited tilt range, cryo-ET suffers from low contrast and missing wedge artifacts, which limits the tomograms to low resolution and hinders further biological interpretation. In this study, we applied the Model-Based Iterative Reconstruction (MBIR) method to obtain tomographic 3D reconstructions of experimental cryo-ET datasets and demonstrated the advantages of MBIR in contrast improvement, missing wedge artifacts reduction, missing information restoration, and subtomogram averaging compared with other reconstruction approaches. Considering the outstanding reconstruction quality, MBIR has a great potential in the determination of high resolution biological structures with cryo-ET.

Imaging through distributed-volume aberrations using single-shot digital holography.

This paper explores the use of single-shot digital holography data and a novel algorithm, referred to as multiplane iterative reconstruction (MIR), for imaging through distributed-volume aberrations. Such aberrations result in a linear, shift-varying or "anisoplanatic" physical process, where multiple-look angles give rise to different point spread functions within the field of view of the imaging system. The MIR algorithm jointly computes the maximum a posteriori estimates of the anisoplanatic phase errors and the speckle-free object reflectance from the single-shot digital holography data. Using both simulations and experiments, we show that the MIR algorithm outperforms the leading multiplane image-sharpening algorithm over a wide range of anisoplanatic conditions.

Dynamic Sparse Sampling for Confocal Raman Microscopy.

The total number of data points required for image generation in Raman microscopy was greatly reduced using sparse sampling strategies, in which the preceding set of measurements informed the next most information-rich sampling location. Using this approach, chemical images of pharmaceutical materials were obtained with >99% accuracy from 15.8% sampling, representing an ∼6-fold reduction in measurement time relative to full field of view rastering with comparable image quality. This supervised learning approach to dynamic sampling (SLADS) has the distinct advantage of being directly compatible with standard confocal Raman instrumentation. Furthermore, SLADS is not limited to Raman imaging, potentially providing time-savings in image reconstruction whenever the single-pixel measurement time is the limiting factor in image generation.

Demonstration of single-shot digital holography using a Bayesian framework.

In this paper, we present experimental results for image reconstruction, with isoplanatic phase-error correction, from single-shot digital holography data. We demonstrate the utility of using a model-based iterative reconstruction (MBIR) algorithm to jointly compute the maximum a posteriori estimates of the phase errors and the real-valued object reflectance function. Specifically, we show that the MBIR algorithm is robust to noise and phase errors over a range of conditions.

Second Harmonic Generation Guided Raman Spectroscopy for Sensitive Detection of Polymorph Transitions.

Second harmonic generation (SHG) was integrated with Raman spectroscopy for the analysis of pharmaceutical materials. Particulate formulations of clopidogrel bisulfate were prepared in two crystal forms (Form I and Form II). Image analysis approaches enable automated identification of particles by bright field imaging, followed by classification by SHG. Quantitative SHG microscopy enabled discrimination of crystal form on a per particle basis with 99.95% confidence in a total measurement time of ∼10 ms per particle. Complementary measurements by Raman and synchrotron XRD are in excellent agreement with the classifications made by SHG, with measurement times of ∼1 min and several seconds per particle, respectively. Coupling these capabilities with at-line monitoring may enable real-time feedback for reaction monitoring during pharmaceutical production to favor the more bioavailable but metastable Form I with limits of detection in the ppm regime.

Dynamic X-ray diffraction sampling for protein crystal positioning.

A sparse supervised learning approach for dynamic sampling (SLADS) is described for dose reduction in diffraction-based protein crystal positioning. Crystal centering is typically a prerequisite for macromolecular diffraction at synchrotron facilities, with X-ray diffraction mapping growing in popularity as a mechanism for localization. In X-ray raster scanning, diffraction is used to identify the crystal positions based on the detection of Bragg-like peaks in the scattering patterns; however, this additional X-ray exposure may result in detectable damage to the crystal prior to data collection. Dynamic sampling, in which preceding measurements inform the next most information-rich location to probe for image reconstruction, significantly reduced the X-ray dose experienced by protein crystals during positioning by diffraction raster scanning. The SLADS algorithm implemented herein is designed for single-pixel measurements and can select a new location to measure. In each step of SLADS, the algorithm selects the pixel, which, when measured, maximizes the expected reduction in distortion given previous measurements. Ground-truth diffraction data were obtained for a 5 µm-diameter beam and SLADS reconstructed the image sampling 31% of the total volume and only 9% of the interior of the crystal greatly reducing the X-ray dosage on the crystal. Using in situ two-photon-excited fluorescence microscopy measurements as a surrogate for diffraction imaging with a 1 µm-diameter beam, the SLADS algorithm enabled image reconstruction from a 7% sampling of the total volume and 12% sampling of the interior of the crystal. When implemented into the beamline at Argonne National Laboratory, without ground-truth images, an acceptable reconstruction was obtained with 3% of the image sampled and approximately 5% of the crystal. The incorporation of SLADS into X-ray diffraction acquisitions has the potential to significantly minimize the impact of X-ray exposure on the crystal by limiting the dose and area exposed for image reconstruction and crystal positioning using data collection hardware present in most macromolecular crystallography end-stations.

Phase-error estimation and image reconstruction from digital-holography data using a Bayesian framework.

The estimation of phase errors from digital-holography data is critical for applications such as imaging or wavefront sensing. Conventional techniques require multiple i.i.d. data and perform poorly in the presence of high noise or large phase errors. In this paper, we propose a method to estimate isoplanatic phase errors from a single data realization. We develop a model-based iterative reconstruction algorithm that computes the maximum a posteriori estimate of the phase and the speckle-free object reflectance. Using simulated data, we show that the algorithm is robust against high noise and strong phase errors.

Optically coherent image formation and denoising using a plug and play inversion framework.

The performance of optically coherent imaging systems can be limited by measurement and speckle noise. In this paper, we develop an image formation framework for computing the maximum a posteriori estimate of an object's reflectivity when imaged using coherent illumination and detection. The proposed approach allows for the use of Gaussian denoising algorithms (GDAs), without modification, to mitigate the exponentially distributed and signal-dependent noise that occurs in coherent imaging. Several GDAs are compared using both simulated and experimental data. The proposed framework is shown to be robust to noise and significantly reduce reconstruction error compared to the standard inversion technique.

High frame-rate multichannel beam-scanning microscopy based on Lissajous trajectories.

A simple beam-scanning optical design based on Lissajous trajectory imaging is described for achieving up to kHz frame-rate optical imaging on multiple simultaneous data acquisition channels. In brief, two fast-scan resonant mirrors direct the optical beam on a circuitous trajectory through the field of view, with the trajectory repeat-time given by the least common multiplier of the mirror periods. Dicing the raw time-domain data into sub-trajectories combined with model-based image reconstruction (MBIR) 3D in-painting algorithms allows for effective frame-rates much higher than the repeat time of the Lissajous trajectory. Since sub-trajectory and full-trajectory imaging are simply different methods of analyzing the same data, both high-frame rate images with relatively low resolution and low frame rate images with high resolution are simultaneously acquired. The optical hardware required to perform Lissajous imaging represents only a minor modification to established beam-scanning hardware, combined with additional control and data acquisition electronics. Preliminary studies based on laser transmittance imaging and polarization-dependent second harmonic generation microscopy support the viability of the approach both for detection of subtle changes in large signals and for trace-light detection of transient fluctuations.

A machine learning decision criterion for reducing scan time for hyperspectral neutron computed tomography systems.

We present the first machine learning-based autonomous hyperspectral neutron computed tomography experiment performed at the Spallation Neutron Source. Hyperspectral neutron computed tomography allows the characterization of samples by enabling the reconstruction of crystallographic information and elemental/isotopic composition of objects relevant to materials science. High quality reconstructions using traditional algorithms such as the filtered back projection require a high signal-to-noise ratio across a wide wavelength range combined with a large number of projections. This results in scan times of several days to acquire hundreds of hyperspectral projections, during which end users have minimal feedback. To address these challenges, a golden ratio scanning protocol combined with model-based image reconstruction algorithms have been proposed. This novel approach enables high quality real-time reconstructions from streaming experimental data, thus providing feedback to users, while requiring fewer yet a fixed number of projections compared to the filtered back projection method. In this paper, we propose a novel machine learning criterion that can terminate a streaming neutron tomography scan once sufficient information is obtained based on the current set of measurements. Our decision criterion uses a quality score which combines a reference-free image quality metric computed using a pre-trained deep neural network with a metric that measures differences between consecutive reconstructions. The results show that our method can reduce the measurement time by approximately a factor of five compared to a baseline method based on filtered back projection for the samples we studied while automatically terminating the scans.

Physics-based iterative reconstruction for dual-source and flying focal spot computed tomography.

PURPOSE: For single-source helical Computed Tomography (CT), both Filtered-Back Projection (FBP) and statistical iterative reconstruction have been investigated. However, for dual-source CT with flying focal spot (DS-FFS CT), a statistical iterative reconstruction that accurately models the scanner geometry and acquisition physics remains unknown to researchers. Therefore, our purpose is to present a novel physics-based iterative reconstruction method for DS-FFS CT and assess its image quality. METHODS: Our algorithm uses precise physics models to reconstruct from the native cone-beam geometry and interleaved dual-source helical trajectory of a DS-FFS CT. To do so, we construct a noise physics model to represent data acquisition noise and a prior image model to represent image noise and texture. In addition, we design forward system models to compute the locations of deflected focal spots, the dimension, and sensitivity of voxels and detector units, as well as the length of intersection between x-rays and voxels. The forward system models further represent the coordinated movement between the dual sources by computing their x-ray coverage gaps and overlaps at an arbitrary helical pitch. With the above models, we reconstruct images by an advanced Consensus Equilibrium (CE) numerical method to compute the maximum a posteriori estimate to a joint optimization problem that simultaneously fits all models. RESULTS: We compared our reconstruction with Siemens ADMIRE, which is the clinical standard hybrid iterative reconstruction (IR) method for DS-FFS CT, in terms of spatial resolution, noise profile, and image artifacts through both phantoms and clinical scan datasets. Experiments show that our reconstruction has a higher spatial resolution, with a Task-Based Modulation Transfer Function (MTFtask ) consistently higher than the clinical standard hybrid IR. In addition, our reconstruction shows a reduced magnitude of image undersampling artifacts than the clinical standard. CONCLUSIONS: By modeling a precise geometry and avoiding data rebinning or interpolation, our physics-based reconstruction achieves a higher spatial resolution and fewer image artifacts with smaller magnitude than the clinical standard hybrid IR.

Noniterative MAP reconstruction using sparse matrix representations.

We present a method for noniterative maximum a posteriori (MAP) tomographic reconstruction which is based on the use of sparse matrix representations. Our approach is to precompute and store the inverse matrix required for MAP reconstruction. This approach has generally not been used in the past because the inverse matrix is typically large and fully populated (i.e., not sparse). In order to overcome this problem, we introduce two new ideas. The first idea is a novel theory for the lossy source coding of matrix transformations which we refer to as matrix source coding. This theory is based on a distortion metric that reflects the distortions produced in the final matrix-vector product, rather than the distortions in the coded matrix itself. The resulting algorithms are shown to require orthonormal transformations of both the measurement data and the matrix rows and columns before quantization and coding. The second idea is a method for efficiently storing and computing the required orthonormal transformations, which we call a sparse-matrix transform (SMT). The SMT is a generalization of the classical FFT in that it uses butterflies to compute an orthonormal transform; but unlike an FFT, the SMT uses the butterflies in an irregular pattern, and is numerically designed to best approximate the desired transforms. We demonstrate the potential of the noniterative MAP reconstruction with examples from optical tomography. The method requires offline computation to encode the inverse transform. However, once these offline computations are completed, the noniterative MAP algorithm is shown to reduce both storage and computation by well over two orders of magnitude, as compared to a linear iterative reconstruction methods.

Fast iterative image reconstruction methods for fully 3D multispectral bioluminescence tomography.

We investigate fast iterative image reconstruction methods for fully 3D multispectral bioluminescence tomography for applications in small animal imaging. Our forward model uses a diffusion approximation for optically inhomogeneous tissue, which we solve using a finite element method (FEM). We examine two approaches to incorporating the forward model into the solution of the inverse problem. In a conventional direct calculation approach one computes the full forward model by repeated solution of the FEM problem, once for each potential source location. We describe an alternative on-the-fly approach where one does not explicitly solve for the full forward model. Instead, the solution to the forward problem is included implicitly in the formulation of the inverse problem, and the FEM problem is solved at each iteration for the current image estimate. We evaluate the convergence speeds of several representative iterative algorithms. We compare the computation cost of those two approaches, concluding that the on-the-fly approach can lead to substantial reductions in total cost when combined with a rapidly converging iterative algorithm.

Hierarchical color correction for camera cell phone images.

In this paper, we propose a hierarchical color correction algorithm for enhancing the color of digital images obtained from low-quality digital image capture devices such as cell phone cameras. The proposed method is based on a multilayer hierarchical stochastic framework whose parameters are learned in an offline training procedure using the well-known expectation maximization (EM) algorithm. This hierarchical framework functions by first making soft assignments of images into defect classes and then processing the images in each defect class with an optimized algorithm. The hierarchical color correction is performed in three stages. In the first stage, global color attributes of the low-quality input image are used in a Gaussian mixture model (GMM) framework to perform a soft classification of the image into M predefined global image classes. In the second stage, the input image is processed with a nonlinear color correction algorithm that is designed for each of the M global classes. This color correction algorithm, which we refer to as resolution synthesis color correction (RSCC), applies a spatially varying color correction determined by the local color attributes of the input image. In the third stage, the outputs of the RSCC predictors are combined using the global classification weights to yield the color corrected output image. We compare the performance of the proposed method to other commercial color correction algorithms on cell phone camera images obtained from different sources. Both subjective and objective measures of quality indicate that the new color correction algorithm improves quality over the existing methods.

Reduced electron exposure for energy-dispersive spectroscopy using dynamic sampling.

Analytical electron microscopy and spectroscopy of biological specimens, polymers, and other beam sensitive materials has been a challenging area due to irradiation damage. There is a pressing need to develop novel imaging and spectroscopic imaging methods that will minimize such sample damage as well as reduce the data acquisition time. The latter is useful for high-throughput analysis of materials structure and chemistry. In this work, we present a novel machine learning based method for dynamic sparse sampling of EDS data using a scanning electron microscope. Our method, based on the supervised learning approach for dynamic sampling algorithm and neural networks based classification of EDS data, allows a dramatic reduction in the total sampling of up to 90%, while maintaining the fidelity of the reconstructed elemental maps and spectroscopic data. We believe this approach will enable imaging and elemental mapping of materials that would otherwise be inaccessible to these analysis techniques.

A three-dimensional statistical approach to improved image quality for multislice helical CT.

Multislice helical computed tomography scanning offers the advantages of faster acquisition and wide organ coverage for routine clinical diagnostic purposes. However, image reconstruction is faced with the challenges of three-dimensional cone-beam geometry, data completeness issues, and low dosage. Of all available reconstruction methods, statistical iterative reconstruction (IR) techniques appear particularly promising since they provide the flexibility of accurate physical noise modeling and geometric system description. In this paper, we present the application of Bayesian iterative algorithms to real 3D multislice helical data to demonstrate significant image quality improvement over conventional techniques. We also introduce a novel prior distribution designed to provide flexibility in its parameters to fine-tune image quality. Specifically, enhanced image resolution and lower noise have been achieved, concurrently with the reduction of helical cone-beam artifacts, as demonstrated by phantom studies. Clinical results also illustrate the capabilities of the algorithm on real patient data. Although computational load remains a significant challenge for practical development, superior image quality combined with advancements in computing technology make IR techniques a legitimate candidate for future clinical applications.