Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning.

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

Dual-domain mean-reverting diffusion model-enhanced temporal compressive coherent diffraction imaging.

Temporal compressive coherent diffraction imaging is a lensless imaging technique with the capability to capture fast-moving small objects. However, the accuracy of imaging reconstruction is often hindered by the loss of frequency domain information, a critical factor limiting the quality of the reconstructed images. To improve the quality of these reconstructed images, a method dual-domain mean-reverting diffusion model-enhanced temporal compressive coherent diffraction imaging (DMDTC) has been introduced. DMDTC leverages the mean-reverting diffusion model to acquire prior information in both frequency and spatial domain through sample learning. The frequency domain mean-reverting diffusion model is employed to recover missing information, while hybrid input-output algorithm is carried out to reconstruct the spatial domain image. The spatial domain mean-reverting diffusion model is utilized for denoising and image restoration. DMDTC has demonstrated a significant enhancement in the quality of the reconstructed images. The results indicate that the structural similarity and peak signal-to-noise ratio of images reconstructed by DMDTC surpass those obtained through conventional methods. DMDTC enables high temporal frame rates and high spatial resolution in coherent diffraction imaging.

High-resolution iterative reconstruction at extremely low sampling rate for Fourier single-pixel imaging via diffusion model.

The trade-off between imaging efficiency and imaging quality has always been encountered by Fourier single-pixel imaging (FSPI). To achieve high-resolution imaging, the increase in the number of measurements is necessitated, resulting in a reduction of imaging efficiency. Here, a novel high-quality reconstruction method for FSPI imaging via diffusion model was proposed. A score-based diffusion model is designed to learn prior information of the data distribution. The real-sampled low-frequency Fourier spectrum of the target is employed as a consistency term to iteratively constrain the model in conjunction with the learned prior information, achieving high-resolution reconstruction at extremely low sampling rates. The performance of the proposed method is evaluated by simulations and experiments. The results show that the proposed method has achieved superior quality compared with the traditional FSPI method and the U-Net method. Especially at the extremely low sampling rate (e.g., 1%), an approximately 241% improvement in edge intensity-based score was achieved by the proposed method for the coin experiment, compared with the traditional FSPI method. The method has the potential to achieve high-resolution imaging without compromising imaging speed, which will further expanding the application scope of FSPI in practical scenarios.

Multi-phase FZA lensless imaging via diffusion model.

Lensless imaging shifts the burden of imaging from bulky and expensive hardware to computing, which enables new architectures for portable cameras. However, the twin image effect caused by the missing phase information in the light wave is a key factor limiting the quality of lensless imaging. Conventional single-phase encoding methods and independent reconstruction of separate channels pose challenges in removing twin images and preserving the color fidelity of the reconstructed image. In order to achieve high-quality lensless imaging, the multiphase lensless imaging via diffusion model (MLDM) is proposed. A multi-phase FZA encoder integrated on a single mask plate is used to expand the data channel of a single-shot image. The information association between the color image pixel channel and the encoded phase channel is established by extracting prior information of the data distribution based on multi-channel encoding. Finally, the reconstruction quality is improved through the use of the iterative reconstruction method. The results show that the proposed MLDM method effectively removes the influence of twin images and produces high-quality reconstructed images compared with traditional methods, and the results reconstructed using MLDM have higher structural similarity and peak signal-to-noise ratio.

High-speed real 3D scene acquisition and 3D holographic reconstruction system based on ultrafast optical axial scanning.

The lack of three-dimensional (3D) content is one of the challenges that have been faced by holographic 3D display. Here, we proposed a real 3D scene acquisition and 3D holographic reconstruction system based on ultrafast optical axial scanning. An electrically tunable lens (ETL) was used for high-speed focus shift (up to 2.5 ms). A CCD camera was synchronized with the ETL to acquire multi-focused image sequence of real scene. Then, the focusing area of each multi-focused image was extracted by using Tenengrad operator, and the 3D image were obtained. Finally, 3D holographic reconstruction visible to the naked eye can be achieved by the layer-based diffraction algorithm. The feasibility and effectiveness of the proposed method have been demonstrated by simulation and experiment, and the experimental results agree well with the simulation results. This method will further expand the application of holographic 3D display in the field of education, advertising, entertainment, and other fields.

Universal generative modeling in dual domains for dynamic MRI.

Dynamic magnetic resonance image reconstruction from incomplete k-space data has generated great research interest due to its ability to reduce scan time. Nevertheless, the reconstruction problem remains a thorny issue due to its ill posed nature. Recently, diffusion models, especially score-based generative models, have demonstrated great potential in terms of algorithmic robustness and flexibility of utilization. Moreover, a unified framework through the variance exploding stochastic differential equation is proposed to enable new sampling methods and further extend the capabilities of score-based generative models. Therefore, by taking advantage of the unified framework, we propose a k-space and image dual-domain collaborative universal generative model (DD-UGM), which combines the score-based prior with a low-rank regularization penalty to reconstruct highly under-sampled measurements. More precisely, we extract prior components from both image and k-space domains via a universal generative model and adaptively handle these prior components for faster processing while maintaining good generation quality. Experimental comparisons demonstrate the noise reduction and detail preservation abilities of the proposed method. Moreover, DD-UGM can reconstruct data of different frames by only training a single frame image, which reflects the flexibility of the proposed model.

Magnetic resonance imaging reconstruction using a deep energy-based model.

Although recent deep energy-based generative models (EBMs) have shown encouraging results in many image-generation tasks, how to take advantage of self-adversarial cogitation in deep EBMs to boost the performance of magnetic resonance imaging (MRI) reconstruction is still desired. With the successful application of deep learning in a wide range of MRI reconstructions, a line of emerging research involves formulating an optimization-based reconstruction method in the space of a generative model. Leveraging this, a novel regularization strategy is introduced in this article that takes advantage of self-adversarial cogitation of the deep energy-based model. More precisely, we advocate alternating learning by a more powerful energy-based model with maximum likelihood estimation to obtain the deep energy-based information, represented as a prior image. Simultaneously, implicit inference with Langevin dynamics is a unique property of reconstruction. In contrast to other generative models for reconstruction, the proposed method utilizes deep energy-based information as the image prior in reconstruction to improve the quality of image. Experimental results imply the proposed technique can obtain remarkable performance in terms of high reconstruction accuracy that is competitive with state-of-the-art methods, and which does not suffer from mode collapse. Algorithmically, an iterative approach is presented to strengthen EBM training with the gradient of energy network. The robustness and reproducibility of the algorithm were also experimentally validated. More importantly, the proposed reconstruction framework can be generalized for most MRI reconstruction scenarios.

WKGM: weighted k-space generative model for parallel imaging reconstruction.

Deep learning based parallel imaging (PI) has made great progress in recent years to accelerate MRI. Nevertheless, it still has some limitations: for example, the robustness and flexibility of existing methods are greatly deficient. In this work, we propose a method to explore the k-space domain learning via robust generative modeling for flexible calibrationless PI reconstruction, coined the weighted k-space generative model (WKGM). Specifically, WKGM is a generalized k-space domain model, where the k-space weighting technology and high-dimensional space augmentation design are efficiently incorporated for score-based generative model training, resulting in good and robust reconstructions. In addition, WKGM is flexible and thus can be synergistically combined with various traditional k-space PI models, which can make full use of the correlation between multi-coil data and realize calibrationless PI. Even though our model was trained on only 500 images, experimental results with varying sampling patterns and acceleration factors demonstrate that WKGM can attain state-of-the-art reconstruction results with the well learned k-space generative prior.

A 4-minute solution for submillimeter whole-brain T1ρ quantification.

PURPOSE: To develop a robust, accurate, and accelerated T1ρ quantification solution for submillimeter in vivo whole-brain imaging. METHODS: A multislice T1ρ mapping solution (MS-T1ρ ) was developed based on a two-acquisition scheme using turbo spin echo with RF cycling to allow for whole-brain coverage with 0.8-mm in-plane resolution. A compressed sensing-based fast imaging method, SCOPE, was used to accelerate the MS-T1ρ acquisition time to a total scan time of 3 minutes 31 seconds. A phantom experiment was conducted to assess the accuracy of MS-T1ρ by comparing the T1ρ value obtained using MS-T1ρ with the reference value obtained using the standard single-slice T1ρ mapping method. In vivo scans of 13 volunteers were acquired prospectively to validate the robustness of MS-T1ρ . RESULTS: In the phantom study, the T1ρ values obtained with MS-T1ρ were in good agreement with the reference T1ρ values (R2 = 0.9991) and showed high consistency throughout all slices (coefficient of variation = 2.2 ± 2.43%). In the in vivo experiments, T1ρ maps were successfully acquired for all volunteers with no visually noticeable artifacts. There was no significant difference in T1ρ values between MS-T1ρ acquisitions and fully sampled acquisitions for all brain tissues (p-value > .05). In the intraclass correlation coefficient and Bland-Altman analyses, the accelerated T1ρ measurements show moderate to good agreement to the fully sampled reference values. CONCLUSION: The proposed MS-T1ρ solution allows for high-resolution whole-brain T1ρ mapping within 4 minutes and may provide a potential tool for investigating neural diseases.

Highly undersampled magnetic resonance imaging reconstruction using autoencoding priors.

PURPOSE: Although recent deep learning methodologies have shown promising results in fast MR imaging, how to explore it to learn an explicit prior and leverage it into the observation constraint is still desired. METHODS: A denoising autoencoder (DAE) network is leveraged as an explicit prior to address the highly undersampling MR image reconstruction problem. First, inspired by the observation that the prior information learned from high-dimension signals is more effective than that from the low-dimension counterpart in image restoration tasks, we train the network in a multichannel scenario and apply the learned network to single-channel image reconstruction by a variables augmentation technique. Second, because of the fact that multiple implementations of artificial noise generation in DAE favors a better underlying result, we introduce a 2-sigma rule to complement each other for improving the final reconstruction. The whole algorithm is tackled by proximal gradient descent. RESULTS: Experimental results under varying sampling trajectories and acceleration factors consistently demonstrate the superiority of the enhanced autoencoding priors, in terms of peak signal-to-noise ratio, structural similarity, and high-frequency error norm. CONCLUSION: A simple and effective way to incorporate the DAE prior into highly undersampling MR reconstruction is proposed. Once the DAE prior is obtained, it can be applied to the reconstruction tasks with different sampling trajectories and acceleration factors, and achieves superior performance in comparison with state-of-the-art methods.

Accurate vessel extraction via tensor completion of background layer in X-ray coronary angiograms.

This paper proposes an effective method for accurately recovering vessel structures and intensity information from the X-ray coronary angiography (XCA) images of moving organs or tissues. Specifically, a global logarithm transformation of XCA images is implemented to fit the X-ray attenuation sum model of vessel/background layers into a low-rank, sparse decomposition model for vessel/background separation. The contrast-filled vessel structures are extracted by distinguishing the vessels from the low-rank backgrounds by using a robust principal component analysis and by constructing a vessel mask via Radon-like feature filtering plus spatially adaptive thresholding. Subsequently, the low-rankness and inter-frame spatio-temporal connectivity in the complex and noisy backgrounds are used to recover the vessel-masked background regions using tensor completion of all other background regions, while the twist tensor nuclear norm is minimized to complete the background layers. Finally, the method is able to accurately extract vessels' intensities from the noisy XCA data by subtracting the completed background layers from the overall XCA images. We evaluated the vessel visibility of resulting images on real X-ray angiography data and evaluated the accuracy of vessel intensity recovery on synthetic data. Experiment results show the superiority of the proposed method over the state-of-the-art methods.

Positron emission tomography image reconstruction using feature extraction.

PURPOSE: To reduce the cost of positron emission tomography (PET) scanning systems, image reconstruction algorithms for low-sampled data have been extensively studied. However, the current method based on total variation (TV) minimization regularization nested in the maximum likelihood-expectation maximization (MLEM) algorithm cannot distinguish true structures from noise resulting losing some fine features in the images. Thus, this work aims to recover fine features lost in the MLEM-TV algorithm from low-sampled data. METHOD: A feature refinement (FR) approach previously developed for statistical interior computed tomography (CT) reconstruction is applied to PET imaging to recover fine features in this study. The proposed method starts with a constant initial image and the FR step is performed after each MLEM-TV iteration to extract the desired structural information lost during TV minimization. A feature descriptor is specifically designed to distinguish structure from noise and artifacts. A modified steepest descent method is adopted to minimize the objective function. After evaluating the impacts of different patch sizes on the outcome of the presented method, an optimal patch size of 7×7 is selected in this study to balance structure-detection ability and computational efficiency. RESULTS: Applying MLEM-TV-FR algorithm to the simulated brain PET imaging using an emission activity phantom, a standard Shepp-Logan phantom, and mouse results in the increased peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) as comparing to using the conventional MLEM-TV algorithm, as well as the substantial reduction of the used sampling numbers, which improves the computational efficiency. CONCLUSIONS: The presented algorithm can achieve image quality superior to that of the MLEM and MLEM-TV approaches in terms of the preservation of fine structure and the suppression of undesired artifacts and noise, indicating its useful potential for low-sampled data in PET imaging.

Image reconstruction from few-view CT data by gradient-domain dictionary learning.

BACKGROUND: Decreasing the number of projections is an effective way to reduce the radiation dose exposed to patients in medical computed tomography (CT) imaging. However, incomplete projection data for CT reconstruction will result in artifacts and distortions. OBJECTIVE: In this paper, a novel dictionary learning algorithm operating in the gradient-domain (Grad-DL) is proposed for few-view CT reconstruction. Specifically, the dictionaries are trained from the horizontal and vertical gradient images, respectively and the desired image is reconstructed subsequently from the sparse representations of both gradients by solving the least-square method. METHODS: Since the gradient images are sparser than the image itself, the proposed approach could lead to sparser representations than conventional DL methods in the image-domain, and thus a better reconstruction quality is achieved. RESULTS: To evaluate the proposed Grad-DL algorithm, both qualitative and quantitative studies were employed through computer simulations as well as real data experiments on fan-beam and cone-beam geometry. CONCLUSIONS: The results show that the proposed algorithm can yield better images than the existing algorithms.

PANDA-T1ρ: Integrating principal component analysis and dictionary learning for fast T1ρ mapping.

PURPOSE: Long scanning time greatly hinders the widespread application of spin-lattice relaxation in rotating frame (T1ρ) in clinics. In this study, a novel method is proposed to reconstruct the T1ρ-weighted images from undersampled k-space data and hence accelerate the acquisition of T1ρ imaging. METHODS: The proposed approach (PANDA-T1ρ) combined the benefit of PCA and dictionary learning when reconstructing image from undersampled data. Specifically, the PCA transform was first used to sparsify the image series along the parameter direction and then the sparsified images were reconstructed by means of dictionary learning and finally solved the images. A variation of PANDA-T1ρ was also developed for the heavy noise case. Numerical simulation and in vivo experiments were carried out with the accelerating factor from 2 to 4 to verify the performance of PANDA-T1ρ. RESULTS: The reconstructed T1ρ maps using the PANDA-T1ρ method were found to be comparable to the reference at all verified acceleration factors. Moreover, the variation exhibited better performance than the original version when the k-space data were contaminated by heavy noise. CONCLUSION: PANDA-T1ρ can significantly reduce the scanning time of T1ρ by integrating PCA and dictionary learning and provides better parameter estimation than the state-of-art methods for a fixed acceleration factor.

Incorporating reference in parallel imaging and compressed sensing.

PURPOSE: To develop a new compressed sensing parallel imaging technique called READ-PICS that can effectively incorporate prior information from a reference scan for MR image reconstruction from highly undersampled multichannel measurements. METHODS: READ-PICS incorporates information from a high-spatial-resolution reference prior using the generalized series model, to achieve increased image sparsity and mitigated noise amplification simultaneously. To further improve the ill-conditioning of the parallel imaging system, an annular area in the central residual k-space is used for calibration. Additionally, the mixed L1-L2 norm of the coefficients from the prior component and residual component is used to enforce joint sparsity. RESULTS: The evaluations on parametric imaging and multiscan experiment demonstrate superior performance of READ-PICS in terms of detail preservation and noise suppression compared to state-of-the-art technique, L1-Iterative self-consistent parallel imaging reconstruction, and prescan required method, correlation imaging. CONCLUSIONS: The proposed method can significantly increase signal sparsity and improve the ill-conditioning of the parallel imaging system using reference adaptive regularization. This technique can be easily adapted to other imaging applications where multiple images need to be acquired sequentially and a reference prior is also available.

Reconstruction of magnetic resonance imaging by three-dimensional dual-dictionary learning.

PURPOSE: To improve the magnetic resonance imaging (MRI) data acquisition speed while maintaining the reconstruction quality, a novel method is proposed for multislice MRI reconstruction from undersampled k-space data based on compressed-sensing theory using dictionary learning. THEORY AND METHODS: There are two aspects to improve the reconstruction quality. One is that spatial correlation among slices is used by extending the atoms in dictionary learning from patches to blocks. The other is that the dictionary-learning scheme is used at two resolution levels; i.e., a low-resolution dictionary is used for sparse coding and a high-resolution dictionary is used for image updating. Numerical experiments are carried out on in vivo 3D MR images of brains and abdomens with a variety of undersampling schemes and ratios. RESULTS: The proposed method (dual-DLMRI) achieves better reconstruction quality than conventional reconstruction methods, with the peak signal-to-noise ratio being 7 dB higher. The advantages of the dual dictionaries are obvious compared with the single dictionary. Parameter variations ranging from 50% to 200% only bias the image quality within 15% in terms of the peak signal-to-noise ratio. CONCLUSION: Dual-DLMRI effectively uses the a priori information in the dual-dictionary scheme and provides dramatically improved reconstruction quality.

Wavelet-Improved Score-Based Generative Model for Medical Imaging.

The score-based generative model (SGM) has demonstrated remarkable performance in addressing challenging under-determined inverse problems in medical imaging. However, acquiring high-quality training datasets for these models remains a formidable task, especially in medical image reconstructions. Prevalent noise perturbations or artifacts in low-dose Computed Tomography (CT) or under-sampled Magnetic Resonance Imaging (MRI) hinder the accurate estimation of data distribution gradients, thereby compromising the overall performance of SGMs when trained with these data. To alleviate this issue, we propose a wavelet-improved denoising technique to cooperate with the SGMs, ensuring effective and stable training. Specifically, the proposed method integrates a wavelet sub-network and the standard SGM sub-network into a unified framework, effectively alleviating inaccurate distribution of the data distribution gradient and enhancing the overall stability. The mutual feedback mechanism between the wavelet sub-network and the SGM sub-network empowers the neural network to learn accurate scores even when handling noisy samples. This combination results in a framework that exhibits superior stability during the learning process, leading to the generation of more precise and reliable reconstructed images. During the reconstruction process, we further enhance the robustness and quality of the reconstructed images by incorporating regularization constraint. Our experiments, which encompass various scenarios of low-dose and sparse-view CT, as well as MRI with varying under-sampling rates and masks, demonstrate the effectiveness of the proposed method by significantly enhanced the quality of the reconstructed images. Especially, our method with noisy training samples achieves comparable results to those obtained using clean data. Our code at https://zenodo.org/record/8266123.

Correlated and Multi-frequency Diffusion Modeling for Highly Under-sampled MRI Reconstruction.

Given the obstacle in accentuating the reconstruction accuracy for diagnostically significant tissues, most existing MRI reconstruction methods perform targeted reconstruction of the entire MR image without considering fine details, especially when dealing with highly under-sampled images. Therefore, a considerable volume of efforts has been directed towards surmounting this challenge, as evidenced by the emergence of numerous methods dedicated to preserving high-frequency content as well as fine textural details in the reconstructed image. In this case, exploring the merits associated with each method of mining high-frequency information and formulating a reasonable principle to maximize the joint utilization of these approaches will be a more effective solution to achieve accurate reconstruction. Specifically, this work constructs an innovative principle named Correlated and Multi-frequency Diffusion Model (CM-DM) for highly under-sampled MRI reconstruction. In essence, the rationale underlying the establishment of such principle lies not in assembling arbitrary models, but in pursuing the effective combinations and replacement of components. It also means that the novel principle focuses on forming a correlated and multi-frequency prior through different high-frequency operators in the diffusion process. Moreover, multi-frequency prior further constraints the noise term closer to the target distribution in the frequency domain, thereby making the diffusion process converge faster. Experimental results verify that the proposed method achieved superior reconstruction accuracy, with a notable enhancement of approximately 2dB in PSNR compared to state-of-the-art methods.

Variable augmentation network for invertible MR coil compression.

To improve the efficiency of multi-coil data compression and recover the compressed image reversibly, increasing the possibility of applying the proposed method to medical scenarios. A deep learning algorithm is employed for MR coil compression in the presented work. The approach introduces a variable augmentation network for invertible coil compression (VAN-ICC). This network utilizes the inherent reversibility of normalizing flow-based models. The aim is to enhance the readability of the sentence and clearly convey the key components of the algorithm. By applying the variable augmentation technology to image/k-space variables from multi-coils, VAN-ICC trains the invertible network by finding an invertible and bijective function, which can map the original data to the compressed counterpart and vice versa. Experiments conducted on both fully-sampled and under-sampled data verified the effectiveness and flexibility of VAN-ICC. Quantitative and qualitative comparisons with traditional non-deep learning-based approaches demonstrated that VAN-ICC carries much higher compression effects. The proposed method trains the invertible network by finding an invertible and bijective function, which improves the defects of traditional coil compression method by utilizing inherent reversibility of normalizing flow-based models. In addition, the application of variable augmentation technology ensures the implementation of reversible networks. In short, VAN-ICC offered a competitive advantage over other traditional coil compression algorithms.

Stage-by-stage Wavelet Optimization Refinement Diffusion Model for Sparse-View CT Reconstruction.

Diffusion model has emerged as a potential tool to tackle the challenge of sparse-view CT reconstruction, displaying superior performance compared to conventional methods. Nevertheless, these prevailing diffusion models predominantly focus on the sinogram or image domains, which can lead to instability during model training, potentially culminating in convergence towards local minimal solutions. The wavelet transform serves to disentangle image contents and features into distinct frequency-component bands at varying scales, adeptly capturing diverse directional structures. Employing the wavelet transform as a guiding sparsity prior significantly enhances the robustness of diffusion models. In this study, we present an innovative approach named the Stage-by-stage Wavelet Optimization Refinement Diffusion (SWORD) model for sparse-view CT reconstruction. Specifically, we establish a unified mathematical model integrating low-frequency and high-frequency generative models, achieving the solution with an optimization procedure. Furthermore, we perform the low-frequency and high-frequency generative models on wavelet's decomposed components rather than the original sinogram, ensuring the stability of model training. Our method is rooted in established optimization theory, comprising three distinct stages, including low-frequency generation, high-frequency refinement and domain transform. The experimental results demonstrated that the proposed method outperformed existing state-of-the-art methods both quantitatively and qualitatively.