Hyperspectral image reconstruction via patch attention driven network.

Coded aperture snapshot spectral imaging (CASSI) captures 3D hyperspectral images (HSIs) with 2D compressive measurements. The recovery of HSIs from these measurements is an ill-posed problem. This paper proposes a novel, to our knowledge, network architecture for this inverse problem, which consists of a multilevel residual network driven by patch-wise attention and a data pre-processing method. Specifically, we propose the patch attention module to adaptively generate heuristic clues by capturing uneven feature distribution and global correlations of different regions. By revisiting the data pre-processing stage, we present a complementary input method that effectively integrates the measurements and coded aperture. Extensive simulation experiments illustrate that the proposed network architecture outperforms state-of-the-art methods.

Block-based spectral image reconstruction for compressive spectral imaging using smoothness on graphs.

A novel reconstruction method for compressive spectral imaging is designed by assuming that the spectral image of interest is sufficiently smooth on a collection of graphs. Since the graphs are not known in advance, we propose to infer them from a panchromatic image using a state-of-the-art graph learning method. Our approach leads to solutions with closed-form that can be found efficiently by solving multiple sparse systems of linear equations in parallel. Extensive simulations and an experimental demonstration show the merits of our method in comparison with traditional methods based on sparsity and total variation and more recent methods based on low-rank minimization and deep-based plug-and-play priors. Our approach may be instrumental in designing efficient methods based on deep neural networks and covariance estimation.

Static coded aperture in robotic X-ray tomography systems.

Coded aperture X-ray computed tomography is a computational imaging technique capable of reconstructing inner structures of an object from a reduced set of X-ray projection measurements. Coded apertures are placed in front of the X-ray sources from different views and thus significantly reduce the radiation dose. This paper introduces coded aperture X-ray computed tomography for robotic X-ray systems which offer positioning flexibility. While single coded-aperture 3D tomography was recently introduced for standard trajectory CT scanning, it is shown that significant gains in imaging performance can be attained by simple modifications in the CT scanning trajectories enabled by emerging dual robotic CT systems. In particular, the subject is fixed on a plane and the CT system uniformly rotates around the r -axis which is misaligned with the coordinate axes. A single stationary coded aperture is placed on front of the robotic X-ray source above the plane and the corresponding X-ray projections are measured by a two-dimensional detector on the second arm of the robotic system. The compressive measurements with misalignment enable the reconstruction of high-resolution three-dimensional volumetric images from the low-resolution coded projections on the detector at a sub-sampling rate. An efficient algorithm is proposed to generate the rotation matrix with two basic sub-matrices and thus the forward model is formulated. The stationary coded aperture is designed based on the Pearson product-moment correlation coefficient analysis and the direct binary search algorithm is used to obtain the optimized coded aperture. Simulations using simulated datasets show significant gains in reconstruction performance compared to conventional coded aperture CT systems.

Static coded illumination strategies for low-dose x-ray material decomposition.

Static coded aperture x-ray tomography was introduced recently where a static illumination pattern is used to interrogate an object with a low radiation dose, from which an accurate 3D reconstruction of the object can be attained computationally. Rather than continuously switching the pattern of illumination with each view angle, as traditionally done, static code computed tomography (CT) places a single pattern for all views. The advantages are many, including the feasibility of practical implementation. This paper generalizes this powerful framework to develop single-scan dual-energy coded aperture spectral tomography that enables material characterization at a significantly reduced exposure level. Two sensing strategies are explored: rapid kV switching with a single-static block/unblock coded aperture, and coded apertures with non-uniform thickness. Both systems rely on coded illumination with a plurality of x-ray spectra created by kV switching or 3D coded apertures. The structured x-ray illumination is projected through the objects of interest and measured with standard x-ray energy integrating detectors. Then, based on the tensor representation of projection data, we develop an algorithm to estimate a full set of synthesized measurements that can be used with standard reconstruction algorithms to accurately recover the object in each energy channel. Simulation and experimental results demonstrate the effectiveness of the proposed cost-effective solution to attain material characterization in low-dose dual-energy CT.

A Multigraph-Defined Distribution Function in a Simulation Model of a Communication Network.

We presented a method based on multigraphs to mathematically define a distribution function in time for the generation of data exchange in a special-purpose communication network. This is needed for the modeling and design of communication networks (CNs) consisting of integrated telecommunications and computer networks (ITCN). Simulation models require a precise definition of network traffic communication. An additional problem for describing the network traffic in simulation models is the mathematical model of data distribution, according to which the generation and exchange of certain types and quantities of data are realized. The application of multigraphs enabled the time and quantity of the data distribution to be displayed as operational procedures for a special-purpose communication unit. A multigraph was formed for each data-exchange time and allowed its associated adjacency matrix to be defined. Using the matrix estimation method allowed the mathematical definition of the distribution function values. The application of the described method for the use of multigraphs enabled a more accurate mathematical description of real traffic in communication networks.

LED-based compressive spectral-temporal imaging.

A compressive spectral-temporal imaging system is reported. A multi-spectral light-emitting diode array is used for target illumination and spectral modulation, while a digital micro-mirror device (DMD) encodes the spatial and temporal frames. Several encoded video frames are captured in a snapshot of an integrating focal plane array (FPA). A high-frame-rate spectral video is reconstructed from the sequence of compressed measurements captured by the grayscale low-frame-rate camera. The imaging system is optimized through the design of the DMD patterns based on the forward model. Laboratory implementation is conducted to validate the performance of the proposed imaging system. We experimentally demonstrate the video acquisition with eight spectral bands and six temporal frames per FPA snapshot, and thus a 256 × 256 × 8 × 6 4D cube is reconstructed from a single 2D measurement.

StaticCodeCT: single coded aperture tensorial X-ray CT.

Coded aperture X-ray CT (CAXCT) is a new low-dose imaging technology that promises far-reaching benefits in industrial and clinical applications. It places various coded apertures (CA) at a time in front of the X-ray source to partially block the radiation. The ill-posed inverse reconstruction problem is then solved using l1-norm-based iterative reconstruction methods. Unfortunately, to attain high-quality reconstructions, the CA patterns must change in concert with the view-angles making the implementation impractical. This paper proposes a simple yet radically different approach to CAXCT, which is coined StaticCodeCT, that uses a single-static CA in the CT gantry, thus making the imaging system amenable for practical implementations. Rather than using conventional compressed sensing algorithms for recovery, we introduce a new reconstruction framework for StaticCodeCT. Namely, we synthesize the missing measurements using low-rank tensor completion principles that exploit the multi-dimensional data correlation and low-rank nature of a 3-way tensor formed by stacking the 2D coded CT projections. Then, we use the FDK algorithm to recover the 3D object. Computational experiments using experimental projection measurements exhibit up to 10% gains in the normalized root mean square distance of the reconstruction using the proposed method compared with those attained by alternative low-dose systems.

Compressive X-ray tomosynthesis using model-driven deep learning.

Compressive X-ray tomosynthesis uses a few two-dimensional projection measurements modulated by coding masks to reconstruct the three-dimensional object that can be sparsely represented on a predefined basis. However, the coding mask optimization and object reconstruction require significant computing resources. In addition, existing methods fall short to exploits the synergy between the encoding and reconstruction stages to approach the global optimum. This paper proposes a model-driven deep learning (MDL) approach to significantly improve the computational efficiency and accuracy of tomosynthesis reconstruction. A unified framework is developed to jointly optimize the coding masks and the neural network parameters, which effectively increase the degrees of optimization freedom. It shows that the computational efficiency of coding mask optimization and image reconstruction can be improved by more than one order of magnitude. Furthermore, the performance of reconstruction results is significantly improved.

Compressive hyperspectral image classification using a 3D coded convolutional neural network.

Hyperspectral image classification (HIC) is an active research topic in remote sensing. Hyperspectral images typically generate large data cubes posing big challenges in data acquisition, storage, transmission and processing. To overcome these limitations, this paper develops a novel deep learning HIC approach based on compressive measurements of coded-aperture snapshot spectral imagers (CASSI), without reconstructing the complete hyperspectral data cube. A new kind of deep learning strategy, namely 3D coded convolutional neural network (3D-CCNN), is proposed to efficiently solve for the classification problem, where the hardware-based coded aperture is regarded as a pixel-wise connected network layer. An end-to-end training method is developed to jointly optimize the network parameters and the coded apertures with periodic structures. The accuracy of classification is effectively improved by exploiting the synergy between the deep learning network and coded apertures. The superiority of the proposed method is assessed over the state-of-the-art HIC methods on several hyperspectral datasets.

Nonlinear reconstruction of coded spectral X-ray CT based on material decomposition.

Coded spectral X-ray computed tomography (CT) based on K-edge filtered illumination is a cost-effective approach to acquire both 3-dimensional structure of objects and their material composition. This approach allows sets of incomplete rays from sparse views or sparse rays with both spatial and spectral encoding to effectively reduce the inspection duration or radiation dose, which is of significance in biological imaging and medical diagnostics. However, reconstruction of spectral CT images from compressed measurements is a nonlinear and ill-posed problem. This paper proposes a material-decomposition-based approach to directly solve the reconstruction problem, without estimating the energy-binned sinograms. This approach assumes that the linear attenuation coefficient map of objects can be decomposed into a few basis materials that are separable in the spectral and space domains. The nonlinear problem is then converted to the reconstruction of the mass density maps of the basis materials. The dimensionality of the optimization variables is thus effectively reduced to overcome the ill-posedness. An alternating minimization scheme is used to solve the reconstruction with regularizations of weighted nuclear norm and total variation. Compared to the state-of-the-art reconstruction method for coded spectral CT, the proposed method can significantly improve the reconstruction quality. It is also capable of reconstructing the spectral CT images at two additional energy bins from the same set of measurements, thus providing more spectral information of the object.

Blue noise coding for a coherent x-ray diffraction imaging system.

Coded x-ray diffraction imaging (CXRDI) is an emerging computational imaging approach that aims to solve the phase retrieval problem in x-ray crystallography based on the intensity measurements of encoded diffraction patterns. Boolean coding masks (BCMs) with complementary structures have been used to modulate the diffraction pattern in CXRDI. However, the optimal spatial distribution of BCMs still remains an open problem to be studied in depth. Based on the spectral initialization criterion, we provide a theoretical proof for the premise that the optimal complementary BCMs should obey the blue noise distribution in the sense of mathematical expectation. In addition, the benefits of the blue noise coding strategy are assessed by a set of simulations, where better reconstruction quality is observed compared to the random BCMs and other complementary BCMs.

Efficient informatics-based source and mask optimization for optical lithography.

Source and mask optimization (SMO) is a widely used computational lithography technology that greatly improves the image fidelity of lithography systems. This paper develops an efficient informatics-based SMO (EISMO) method to improve the image fidelity of lithography systems. First, a communication channel model is established to depict the mechanism of information transmission in the SMO framework, where the source is obtained from the gradient-based SMO algorithm. The manufacturing-aware mask distribution is then optimized to achieve the best mutual information, and the theoretical lower bound of lithography patterning error is obtained. Subsequently, an efficient informatics-based method is proposed to refine the mask optimization result in SMO, further reducing the lithography patterning error. It is shown that the proposed EISMO method is computationally efficient and can achieve superior imaging performance over the conventional SMO method.

Multi-objective optimization for structured illumination in dynamic x-ray tomosynthesis.

Dynamic coded x-ray tomosynthesis (CXT) uses a set of encoded x-ray sources to interrogate objects lying on a moving conveyor mechanism. The object is reconstructed from the encoded measurements received by the uniform linear array detectors. We propose a multi-objective optimization (MO) method for structured illuminations to balance the reconstruction quality and radiation dose in a dynamic CXT system. The MO framework is established based on a dynamic sensing geometry with binary coding masks. The Strength Pareto Evolutionary Algorithm 2 is used to solve the MO problem by jointly optimizing the coding masks, locations of x-ray sources, and exposure moments. Computational experiments are implemented to assess the proposed MO method. They show that the proposed strategy can obtain a set of Pareto optimal solutions with different levels of radiation dose and better reconstruction quality than the initial setting.

Experimental demonstration and optimization of X-ray StaticCodeCT.

As the use of X-ray computed tomography (CT) grows in medical diagnosis, so does the concern for the harm a radiation dose can cause and the biological risks it represents. StaticCodeCT is a new low-dose imaging architecture that uses a single-static coded aperture (CA) in a CT gantry. It exploits the highly correlated data in the projection domain to estimate the unobserved measurements on the detector. We previously analyzed the StaticCodeCT system by emulating the effect of the coded mask on experimental CT data. In contrast, this manuscript presents test-bed reconstructions using an experimental cone-beam X-ray CT system with a CA holder. We analyzed the reconstruction quality using three different techniques to manufacture the CAs: metal additive manufacturing, cold casting, and ceramic additive manufacturing. Furthermore, we propose an optimization method to design the CA pattern based on the algorithm developed for the measurement estimation. The obtained results point to the possibility of the real deployment of StaticCodeCT systems in practice.

Fast inverse lithography based on dual-channel model-driven deep learning.

Inverse lithography technology (ILT) is extensively used to compensate image distortion in optical lithography systems by pre-warping the photomask at the pixel scale. However, computational complexity is always a central challenge of ILT due to the big throughput of data volume. This paper proposes a dual-channel model-driven deep learning (DMDL) method to overcome the computational burden, while break through the limit of image fidelity over traditional ILT algorithms. The architecture of DMDL network is not inherited from conventional deep learning, but derived from the inverse optimization model under a gradient-based ILT framework. A dual-channel structure is introduced to extend the capacity of the DMDL network, which allows to simultaneously modify the mask contour and insert sub-resolution assist features to further improve the lithography image fidelity. An unsupervised training strategy based on auto-decoder is developed to avoid the time-consuming labelling process. The superiority of DMDL over the state-of-the-art ILT method is verified in both of the computational efficiency and image fidelity obtained on the semiconductor wafer.

Model-informed deep learning for computational lithography with partially coherent illumination.

Computational lithography is a key technique to optimize the imaging performance of optical lithography systems. However, the large amount of calculation involved in computational lithography significantly increases the computational complexity. This paper proposes a model-informed deep learning (MIDL) approach to improve its computational efficiency and to enhance the image fidelity of lithography system with partially coherent illumination (PCI). Different from conventional deep learning approaches, the network structure of MIDL is derived from an approximate compact imaging model of PCI lithography system. MIDL has a dual-channel structure, which overcomes the vanishing gradient problem and improves its prediction capacity. In addition, an unsupervised training method is developed based on an accurate lithography imaging model to avoid the computational cost of labelling process. It is shown that the MIDL provides significant gains in terms of computational efficiency and imaging performance of PCI lithography system.

Single-snapshot X-ray imaging for nonlinear compressive tomosynthesis.

Traditional compressive X-ray tomosynthesis uses sequential illumination to interrogate the object, leading to long scanning time and image distortion due to the object variation. This paper proposes a single-snapshot compressive tomosynthesis imaging approach, where the object is simultaneously illuminated by multiple X-ray emitters equipped with coded apertures. Based on rank, intensity and sparsity prior models, a nonlinear image reconstruction framework is established. The coded aperture patterns are optimized based on uniform sensing criteria. Then, a modified split Bregman algorithm is developed to reconstruct the object from the set of nonlinear compressive measurements. It is shown that the proposed method can be used to reduce the inspection time and achieve robust reconstruction with respect to shape variation or motion of objects.

Compressive spectral imaging approach using adaptive coded apertures.

The coded aperture snapshot spectral imager (CASSI) acquires three-dimensional spectral images with two-dimensional coded projection measurements. This paper proposes an adaptive design method of the coded apertures, according to a priori knowledge of the target scene, to improve sensing efficiency and imaging performance of the super-resolution CASSI system. The adaptive coded apertures are constructed from the nonlinear thresholding of the grayscale map of the scene. Theoretical proof is provided to demonstrate the superiority of the adaptive coded apertures over traditional random coded apertures. Improvement in reconstruction performance is also verified by a set of simulations based on different spectral data.

Compressive spectral X-ray tomography based on spatial and spectral coded illumination.

Spectral computed tomography (CT) relies on the spectral dependence of X-ray attenuation coefficients to separate projection measurements into more than two energy bins. Such data can be used to unveil tomographic material characterization - key in national security and medical imaging. This paper explores a radical departure from conventional methods used in spectral imaging. It relies on K-edge coded apertures to create spatially and spectrally coded, lower-dose, X-ray bundles that interrogate specific voxels of the object. The new approach referred to as compressive spectral X-ray imaging (CSXI) uses low-cost standard X-ray integrating detectors and acquires compressive measurements, which enable the reconstruction of energy binned images from fewer measurements. Various spectral and spatial coding strategies for structured illumination are explored. Subsampling in CSXI is accomplished by either view angle spectral subsampling, spatial subsampling enabled by block-unblock coded apertures placed at the source or detector side, or both. The careful design of subsampling strategies, spectral filters, coded apertures, and their placement, are shown to be critical for the quality of tomographic image reconstruction. The forward imaging model of CSXI, which is a non-linear ill-posed problem, is analyzed and a multi-stage algorithm is developed to address the estimation of the energy binned sinograms from the integrating detector measurements. Then, an Alternating Direction Method of Multipliers (ADMM) is used to solve a joint sparse and low-rank optimization problem for reconstruction that exploits the structure of the spectral X-ray data cube.

Channeled compressive imaging spectropolarimeter.

A compressive imaging spectropolarimeter is proposed in this paper, capable of simultaneously acquiring full polarization, spatial and spectral information of the object scene. The spectral and polarization information is modulated through a combination of high-order retarders, a dispersion prism and a polarizer filter wheel. Using a random coded aperture, compressive sensing is applied to eliminate the channel crosstalk and resolution limitation of traditional channeled spectropolarimeters. The forward sensing model and inverse problem are developed. Computer simulation results are reported, followed by experimental demonstrations.