Tomographic Image Reconstruction Using an Advanced Score Function (ADSF).

Computed tomography (CT) reconstructs volumetric images using X-ray projection data acquired from multiple angles around an object. For low-dose or sparse-view CT scans, the classic image reconstruction algorithms often produce severe noise and artifacts. To address this issue, we develop a novel iterative image reconstruction method based on maximum a posteriori (MAP) estimation. In the MAP framework, the score function, i.e., the gradient of the logarithmic probability density distribution, plays a crucial role as an image prior in the iterative image reconstruction process. By leveraging the Gaussian mixture model, we derive a novel score matching formula to establish an advanced score function (ADSF) through deep learning. Integrating the new ADSF into the image reconstruction process, a new ADSF iterative reconstruction method is developed to improve image reconstruction quality. The convergence of the ADSF iterative reconstruction algorithm is proven through mathematical analysis. The performance of the ADSF reconstruction method is also evaluated on both public medical image datasets and clinical raw CT datasets. Our results show that the ADSF reconstruction method can achieve better denoising and deblurring effects than the state-of-the-art reconstruction methods, showing excellent generalizability and stability.

Diffusion Prior Regularized Iterative Reconstruction for Low-dose CT.

Computed tomography (CT) involves a patient's exposure to ionizing radiation. To reduce the radiation dose, we can either lower the X-ray photon count or down-sample projection views. However, either of the ways often compromises image quality. To address this challenge, here we introduce an iterative reconstruction algorithm regularized by a diffusion prior. Drawing on the exceptional imaging prowess of the denoising diffusion probabilistic model (DDPM), we merge it with a reconstruction procedure that prioritizes data fidelity. This fusion capitalizes on the merits of both techniques, delivering exceptional reconstruction results in an unsupervised framework. To further enhance the efficiency of the reconstruction process, we incorporate the Nesterov momentum acceleration technique. This enhancement facilitates superior diffusion sampling in fewer steps. As demonstrated in our experiments, our method offers a potential pathway to high-definition CT image reconstruction with minimized radiation.

Fluorescence molecular tomography for quantum yield and lifetime.

Fluorescence molecular tomography (FMT) is a promising modality for noninvasive imaging of internal fluorescence agents in biological tissues, especially in small animal models, with applications in diagnosis, therapy, and drug design. In this paper, we present a fluorescent reconstruction algorithm that combines time-resolved fluorescence imaging data with photon-counting microcomputed tomography (PCMCT) images to estimate the quantum yield and lifetime of fluorescent markers in a mouse model. By incorporating PCMCT images, a permissible region of interest of fluorescence yield and lifetime can be roughly estimated as prior knowledge, reducing the number of unknown variables in the inverse problem and improving the image reconstruction stability. Our numerical experiments demonstrate the accuracy and stability of the proposed reconstruction method in the presence of data noise, achieving a reconstruction error of 0.02 ns for the fluorescence lifetime and an average relative error of 18% for quantum yield reconstruction.

Fluorescence Molecular Tomography for Quantum Yield and Lifetime.

Fluorescence molecular tomography (FMT) is a promising modality for non-invasive imaging of internal fluorescence agents in biological tissues especially in small animal models, with applications in diagnosis, therapy, and drug design. In this paper, we present a new fluorescent reconstruction algorithm that combines time-resolved fluorescence imaging data with photon-counting micro-CT (PCMCT) images to estimate the quantum yield and lifetime of fluorescent markers in a mouse model. By incorporating PCMCT images, a permissible region of interest of fluorescence yield and lifetime can be roughly estimated as prior knowledge, reducing the number of unknown variables in the inverse problem and improving image reconstruction stability. Our numerical simulation results demonstrate the accuracy and stability of this method in the presence of data noise, with an average relative error of 18% in fluorescent yield and lifetime reconstruction.

Preliminary landscape analysis of deep tomographic imaging patents.

Over recent years, the importance of the patent literature has become increasingly more recognized in the academic setting. In the context of artificial intelligence, deep learning, and data sciences, patents are relevant to not only industry but also academe and other communities. In this article, we focus on deep tomographic imaging and perform a preliminary landscape analysis of the related patent literature. Our search tool is PatSeer. Our patent bibliometric data is summarized in various figures and tables. In particular, we qualitatively analyze key deep tomographic patent literature.

Estimating optical parameters of biological tissues with photon-counting micro-CT.

Wavelength-dependent absorption and scattering properties determine the fluorescence photon transport in biological tissues and image resolution of optical molecular tomography. Currently, these parameters are computed from optically measured data. For small animal imaging, estimation of optical parameters is a large-scale optimization problem, which is highly ill-posed. In this paper, we propose a new, to the best of our knowledge, approach to estimate optical parameters of biological tissues with photon-counting micro-computed tomography (micro-CT). From photon-counting x-ray data, multi-energy micro-CT images can be reconstructed to perform multi-organ segmentation and material decomposition in terms of tissue constituents. The concentration and characteristics of major tissue constituents can be utilized to calculate the optical absorption and scattering coefficients of the involved tissues. In our study, we perform numerical simulation, phantom experiments, and in vivo animal studies to calculate the optical parameters using our proposed approach. The results show that our approach can estimate optical parameters of tissues with a relative error of <10%, accurately mapping the optical parameter distributions in a small animal.

Low-dimensional Manifold Constrained Disentanglement Network for Metal Artifact Reduction.

Deep neural network based methods have achieved promising results for CT metal artifact reduction (MAR), most of which use many synthesized paired images for supervised learning. As synthesized metal artifacts in CT images may not accurately reflect the clinical counterparts, an artifact disentanglement network (ADN) was proposed with unpaired clinical images directly, producing promising results on clinical datasets. However, as the discriminator can only judge if large regions semantically look artifact-free or artifact-affected, it is difficult for ADN to recover small structural details of artifact-affected CT images based on adversarial losses only without sufficient constraints. To overcome the illposedness of this problem, here we propose a low-dimensional manifold (LDM) constrained disentanglement network (DN), leveraging the image characteristics that the patch manifold of CT images is generally low-dimensional. Specifically, we design an LDM-DN learning algorithm to empower the disentanglement network through optimizing the synergistic loss functions used in ADN while constraining the recovered images to be on a low-dimensional patch manifold. Moreover, learning from both paired and unpaired data, an efficient hybrid optimization scheme is proposed to further improve the MAR performance on clinical datasets. Extensive experiments demonstrate that the proposed LDM-DN approach can consistently improve the MAR performance in paired and/or unpaired learning settings, outperforming competing methods on synthesized and clinical datasets.

Projection decomposition via univariate optimization for dual-energy CT.

Dual-energy computed tomography (DECT) acquires two x-ray projection datasets with different x-ray energy spectra, performs material-specific image reconstruction based on the energy-dependent non-linear integral model, and provides more accurate quantification of attenuation coefficients than single energy spectrum CT. In the diagnostic energy range, x-ray energy-dependent attenuation is mainly caused by photoelectric absorption and Compton scattering. Theoretically, these two physical components of the x-ray attenuation mechanism can be determined from two projection datasets with distinct energy spectra. Practically, the solution of the non-linear integral equation is complicated due to spectral uncertainty, detector sensitivity, and data noise. Conventional multivariable optimization methods are prone to local minima. In this paper, we develop a new method for DECT image reconstruction in the projection domain. This method combines an analytic solution of a polynomial equation and a univariate optimization to solve the polychromatic non-linear integral equation. The polynomial equation of an odd order has a unique real solution with sufficient accuracy for image reconstruction, and the univariate optimization can achieve the global optimal solution, allowing accurate and stable projection decomposition for DECT. Numerical and physical phantom experiments are performed to demonstrate the effectiveness of the method in comparison with the state-of-the-art projection decomposition methods. As a result, the univariate optimization method yields a quality improvement of 15% for image reconstruction and substantial reduction of the computational time, as compared to the multivariable optimization methods.

Stabilizing deep tomographic reconstruction: Part B. Convergence analysis and adversarial attacks.

Due to lack of the kernel awareness, some popular deep image reconstruction networks are unstable. To address this problem, here we introduce the bounded relative error norm (BREN) property, which is a special case of the Lipschitz continuity. Then, we perform a convergence study consisting of two parts: (1) a heuristic analysis on the convergence of the analytic compressed iterative deep (ACID) scheme (with the simplification that the CS module achieves a perfect sparsification), and (2) a mathematically denser analysis (with the two approximations: [1] AT is viewed as an inverse A- 1 in the perspective of an iterative reconstruction procedure and [2] a pseudo-inverse is used for a total variation operator H). Also, we present adversarial attack algorithms to perturb the selected reconstruction networks respectively and, more importantly, to attack the ACID workflow as a whole. Finally, we show the numerical convergence of the ACID iteration in terms of the Lipschitz constant and the local stability against noise.

Stabilizing deep tomographic reconstruction: Part A. Hybrid framework and experimental results.

A recent PNAS paper reveals that several popular deep reconstruction networks are unstable. Specifically, three kinds of instabilities were reported: (1) strong image artefacts from tiny perturbations, (2) small features missed in a deeply reconstructed image, and (3) decreased imaging performance with increased input data. Here, we propose an analytic compressed iterative deep (ACID) framework to address this challenge. ACID synergizes a deep network trained on big data, kernel awareness from compressed sensing (CS)-inspired processing, and iterative refinement to minimize the data residual relative to real measurement. Our study demonstrates that the ACID reconstruction is accurate, is stable, and sheds light on the converging mechanism of the ACID iteration under a bounded relative error norm assumption. ACID not only stabilizes an unstable deep reconstruction network but also is resilient against adversarial attacks to the whole ACID workflow, being superior to classic sparsity-regularized reconstruction and eliminating the three kinds of instabilities.

Phase function estimation from a diffuse optical image via deep learning.

Objective.The phase function is a key element of a light propagation model for Monte Carlo (MC) simulation, which is usually fitted with an analytic function with associated parameters. In recent years, machine learning methods were reported to estimate the parameters of the phase function of a particular form such as the Henyey-Greenstein phase function but, to our knowledge, no studies have been performed to determine the form of the phase function.Approach.Here we design a convolutional neural network (CNN) to estimate the phase function from a diffuse optical image without any explicit assumption on the form of the phase function. Specifically, we use a Gaussian mixture model (GMM) as an example to represent the phase function generally and learn the model parameters accurately. The GMM is selected because it provides the analytic expression of phase function to facilitate deflection angle sampling in MC simulation, and does not significantly increase the number of free parameters.Main Results.Our proposed method is validated on MC-simulated reflectance images of typical biological tissues using the Henyey-Greenstein phase function with different anisotropy factors. The mean squared error of the phase function is 0.01 and the relative error of the anisotropy factor is 3.28%.Significance.We propose the first data-driven CNN-based inverse MC model to estimate the form of scattering phase function. The effects of field of view and spatial resolution are analyzed and the findings provide guidelines for optimizing the experimental protocol in practical applications.

Feasibility analysis on simultaneous electron density and attenuation coefficient reconstruction.

PURPOSE: Creating a viable reconstruction method for Compton scatter tomography remains challenging. Accounting for scatter attenuation when the underlying attenuation map is not known is particularly challenging, and current mathematical approaches to this vary widely. This work explores a novel approach to joint scatter and attenuation image reconstruction, which leverages the underlying structural similarity between the two images and incorporates a deep learning model in an alternating iterative reconstruction scheme. METHODS: A single-view computed tomography (CT) imaging procedure for recording Compton scatter is first described. A joint reconstruction model, which iterates between algebraically reconstructing scatter images and estimating the attenuation via deep learning, is then proposed. This model is tested on both a generated dataset of 2D phantom images designed to mimic human tissues as well as a realistically simulated dataset based on real CT images. RESULTS: Testing results yield convergence of the model and decent reconstruction quality to distinguish crucial features such as tumors and lesions, demonstrating the potential principled utilities of this configuration and deep learning approach. The model achieved a structural similarity index measure of at least 0.82 for scatter and 0.88 for attenuation reconstructions with the realistically simulated dataset. CONCLUSION: The iterative, deep learning approach outlined in this work shows potential for future efficient medical imaging procedures, reconstructing images with limited scatter information.

Deep Interactive Denoiser (DID) for X-Ray Computed Tomography.

Low-dose computed tomography (LDCT) is desirable for both diagnostic imaging and image-guided interventions. Denoisers are widely used to improve the quality of LDCT. Deep learning (DL)-based denoisers have shown state-of-the-art performance and are becoming mainstream methods. However, there are two challenges to using DL-based denoisers: 1) a trained model typically does not generate different image candidates with different noise-resolution tradeoffs, which are sometimes needed for different clinical tasks; and 2) the model's generalizability might be an issue when the noise level in the testing images differs from that in the training dataset. To address these two challenges, in this work, we introduce a lightweight optimization process that can run on top of any existing DL-based denoiser during the testing phase to generate multiple image candidates with different noise-resolution tradeoffs suitable for different clinical tasks in real time. Consequently, our method allows users to interact with the denoiser to efficiently review various image candidates and quickly pick the desired one; thus, we termed this method deep interactive denoiser (DID). Experimental results demonstrated that DID can deliver multiple image candidates with different noise-resolution tradeoffs and shows great generalizability across various network architectures, as well as training and testing datasets with various noise levels.

Focused x-ray luminescence imaging system for small animals based on a rotary gantry.

SIGNIFICANCE: The ability to detect and localize specific molecules through tissue is important for elucidating the molecular basis of disease and treatment. Unfortunately, most current molecular imaging tools in tissue either lack high spatial resolution (e.g., diffuse optical fluorescence tomography or positron emission tomography) or lack molecular sensitivity (e.g., micro-computed tomography, µCT). X-ray luminescence imaging emerged about 10 years ago to address this issue by combining the molecular sensitivity of optical probes with the high spatial resolution of x-ray imaging through tissue. In particular, x-ray luminescence computed tomography (XLCT) has been demonstrated as a powerful technique for the high-resolution imaging of deeply embedded contrast agents in three dimensions (3D) for small-animal imaging. AIM: To facilitate the translation of XLCT for small-animal imaging, we have designed and built a small-animal dedicated focused x-ray luminescence tomography (FXLT) scanner with a µCT scanner, synthesized bright and biocompatible nanophosphors as contrast agents, and have developed a deep-learning-based reconstruction algorithm. APPROACH: The proposed FXLT imaging system was designed using computer-aided design software and built according to specifications. NaGdF4 nanophosphors doped with europium or terbium were synthesized with a silica shell for increased biocompatibility and functionalized with biotin. A deep-learning-based XLCT image reconstruction was also developed based on the residual neural network as a data synthesis method of projection views from few-view data to enhance the reconstructed image quality. RESULTS: We have built the FXLT scanner for small-animal imaging based on a rotational gantry. With all major imaging components mounted, the motor controlling the gantry can be used to rotate the system with a high accuracy. The synthesized nanophosphors displayed distinct x-ray luminescence emission, which enables multi-color imaging, and has successfully been bound to streptavidin-coated substrates. Lastly, numerical simulations using the proposed deep-learning-based reconstruction algorithm has demonstrated a clear enhancement in the reconstructed image quality. CONCLUSIONS: The designed FXLT scanner, synthesized nanophosphors, and deep-learning-based reconstruction algorithm show great potential for the high-resolution molecular imaging of small animals.

Monochromatic image reconstruction via machine learning.

X-ray computed tomography (CT) is a nondestructive imaging technique to reconstruct cross-sectional images of an object using x-ray measurements taken from different view angles for medical diagnosis, therapeutic planning, security screening, and other applications. In clinical practice, the x-ray tube emits polychromatic x-rays, and the x-ray detector array operates in the energy-integrating mode to acquire energy intensity. This physical process of x-ray imaging is accurately described by an energy-dependent non-linear integral equation on the basis of the Beer-Lambert law. However, the non-linear model is not invertible using a computationally efficient solution and is often approximated as a linear integral model in the form of the Radon transform, which basically loses energy-dependent information. This approximate model produces an inaccurate quantification of attenuation images, suffering from beam-hardening effects. In this paper, a machine learning-based approach is proposed to correct the model mismatch to achieve quantitative CT imaging. Specifically, a one-dimensional network model is proposed to learn a non-linear transform from a training dataset to map a polychromatic CT image to its monochromatic sinogram at a pre-specified energy level, realizing virtual monochromatic (VM) imaging effectively and efficiently. Our results show that the proposed method recovers high-quality monochromatic projections with an average relative error of less than 2%. The resultant x-ray VM imaging can be applied for beam-hardening correction, material differentiation and tissue characterization, and proton therapy treatment planning.

Virtual Monoenergetic CT Imaging via Deep Learning.

Conventional single-spectrum computed tomography (CT) reconstructs a spectrally integrated attenuation image and reveals tissues morphology without any information about the elemental composition of the tissues. Dual-energy CT (DECT) acquires two spectrally distinct datasets and reconstructs energy-selective (virtual monoenergetic [VM]) and material-selective (material decomposition) images. However, DECT increases system complexity and radiation dose compared with single-spectrum CT. In this paper, a deep learning approach is presented to produce VM images from single-spectrum CT images. Specifically, a modified residual neural network (ResNet) model is developed to map single-spectrum CT images to VM images at pre-specified energy levels. This network is trained on clinical DECT data and shows excellent convergence behavior and image accuracy compared with VM images produced by DECT. The trained model produces high-quality approximations of VM images with a relative error of less than 2%. This method enables multi-material decomposition into three tissue classes, with accuracy comparable with DECT.

Deep Efficient End-to-end Reconstruction (DEER) Network for Few-view Breast CT Image Reconstruction.

Breast CT provides image volumes with isotropic resolution in high contrast, enabling detection of small calcification (down to a few hundred microns in size) and subtle density differences. Since breast is sensitive to x-ray radiation, dose reduction of breast CT is an important topic, and for this purpose, few-view scanning is a main approach. In this article, we propose a Deep Efficient End-to-end Reconstruction (DEER) network for few-view breast CT image reconstruction. The major merits of our network include high dose efficiency, excellent image quality, and low model complexity. By the design, the proposed network can learn the reconstruction process with as few as O ( N ) parameters, where N is the side length of an image to be reconstructed, which represents orders of magnitude improvements relative to the state-of-the-art deep-learning-based reconstruction methods that map raw data to tomographic images directly. Also, validated on a cone-beam breast CT dataset prepared by Koning Corporation on a commercial scanner, our method demonstrates a competitive performance over the state-of-the-art reconstruction networks in terms of image quality. The source code of this paper is available at: https://github.com/HuidongXie/DEER.

Clinical Micro-CT Empowered by Interior Tomography, Robotic Scanning, and Deep Learning.

While micro-CT systems are instrumental in preclinical research, clinical micro-CT imaging has long been desired with cochlear implantation as a primary application. The structural details of the cochlear implant and the temporal bone require a significantly higher image resolution than that (about 0.2 mm) provided by current medical CT scanners. In this paper, we propose a clinical micro-CT (CMCT) system design integrating conventional spiral cone-beam CT, contemporary interior tomography, deep learning techniques, and the technologies of a micro-focus X-ray source, a photon-counting detector (PCD), and robotic arms for ultrahigh-resolution localized tomography of a freely-selected volume of interest (VOI) at a minimized radiation dose level. The whole system consists of a standard CT scanner for a clinical CT exam and VOI specification, and a robotic micro-CT scanner for a local scan of high spatial and spectral resolution at minimized radiation dose. The prior information from the global scan is also fully utilized for background compensation of the local scan data for accurate and stable VOI reconstruction. Our results and analysis show that the proposed hybrid reconstruction algorithm delivers accurate high-resolution local reconstruction, and is insensitive to the misalignment of the isocenter position, initial view angle and scale mismatch in the data/image registration. These findings demonstrate the feasibility of our system design. We envision that deep learning techniques can be leveraged for optimized imaging performance. With high-resolution imaging, high dose efficiency and low system cost synergistically, our proposed CMCT system has great promise in temporal bone imaging as well as various other clinical applications.

X-ray luminescence imaging for small animals.

X-ray luminescence imaging emerged for about a decade and combines both the high spatial resolution of x-ray imaging with the high measurement sensitivity of optical imaging, which could result in a great molecular imaging tool for small animals. So far, there are two types of x-ray luminescence computed tomography (XLCT) imaging. One uses a pencil beam x-ray for high spatial resolution at a cost of longer measurement time. The other uses cone beam x-ray to cover the whole mouse to obtain XLCT images at a very short time but with a compromised spatial resolution. Here we review these two methods in this paper and highlight the synthesized nanophosphors by different research groups. We are building a focused x-ray luminescence tomography (FXLT) imaging system, developing a machine-learning based FXLT reconstruction algorithm, and synthesizing nanophosphors with different emission wavelengths. In this paper, we will report our current progress from these three aspects. Briefly, we mount all main components, including the focused x-ray tube, the fiber detector, and the x-ray tube and x-ray detector for a microCT system, on a rotary which is a heavy-duty ring track. A microCT scan will be performed before FXLT scan. For a FXLT scan, we will have four PMTs to measure four fiber detectors at two different wavelengths simultaneously for each linear scan position. We expect the spatial resolution of the FXLT imaging will be around 100 micrometers and a limit of detection of approximately 2 µg/mL (for Gd2O2S:Eu).

CT Super-Resolution GAN Constrained by the Identical, Residual, and Cycle Learning Ensemble (GAN-CIRCLE).

In this paper, we present a semi-supervised deep learning approach to accurately recover high-resolution (HR) CT images from low-resolution (LR) counterparts. Specifically, with the generative adversarial network (GAN) as the building block, we enforce the cycle-consistency in terms of the Wasserstein distance to establish a nonlinear end-to-end mapping from noisy LR input images to denoised and deblurred HR outputs. We also include the joint constraints in the loss function to facilitate structural preservation. In this process, we incorporate deep convolutional neural network (CNN), residual learning, and network in network techniques for feature extraction and restoration. In contrast to the current trend of increasing network depth and complexity to boost the imaging performance, we apply a parallel 1×1 CNN to compress the output of the hidden layer and optimize the number of layers and the number of filters for each convolutional layer. The quantitative and qualitative evaluative results demonstrate that our proposed model is accurate, efficient and robust for super-resolution (SR) image restoration from noisy LR input images. In particular, we validate our composite SR networks on three large-scale CT datasets, and obtain promising results as compared to the other state-of-the-art methods.