Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning.

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

SuperMAP: Deep ultrafast MR relaxometry with joint spatiotemporal undersampling.

PURPOSE: To develop an ultrafast and robust MR parameter mapping network using deep learning. THEORY AND METHODS: We design a deep learning framework called SuperMAP that directly converts a series of undersampled (both in k-space and parameter-space) parameter-weighted images into several quantitative maps, bypassing the conventional exponential fitting procedure. We also present a novel technique to simultaneously reconstruct T1rho and T2 relaxation maps within a single scan. Full data were acquired and retrospectively undersampled for training and testing using traditional and state-of-the-art techniques for comparison. Prospective data were also collected to evaluate the trained network. The performance of all methods is evaluated using the parameter qualification errors and other metrics in the segmented regions of interest. RESULTS: SuperMAP achieved accurate T1rho and T2 mapping with high acceleration factors (R = 24 and R = 32). It exploited both spatial and temporal information and yielded low error (normalized mean square error of 2.7% at R = 24 and 2.8% at R = 32) and high resemblance (structural similarity of 97% at R = 24 and 96% at R = 32) to the gold standard. The network trained with retrospectively undersampled data also works well for the prospective data (with a slightly lower acceleration factor). SuperMAP is also superior to conventional methods. CONCLUSION: Our results demonstrate the feasibility of generating superfast MR parameter maps through very few undersampled parameter-weighted images. SuperMAP can simultaneously generate T1rho and T2 relaxation maps in a short scan time.

Minimizing Molecular Misidentification in Imaging Low-Abundance Protein Interactions Using Spectroscopic Single-Molecule Localization Microscopy.

Super-resolution microscopy can capture spatiotemporal organizations of protein interactions with resolution down to 10 nm; however, the analyses of more than two proteins involving low-abundance protein are challenging because spectral crosstalk and heterogeneities of individual fluorescent labels result in molecular misidentification. Here we developed a deep learning-based imaging analysis method for spectroscopic single-molecule localization microscopy to minimize molecular misidentification in three-color super-resolution imaging. We characterized the 3-fold reduction of molecular misidentification in the new imaging method using pure samples of different photoswitchable fluorophores and visualized three distinct subcellular proteins in U2-OS cell lines. We further validated the protein counts and interactions of TOMM20, DRP1, and SUMO1 in a well-studied biological process, Staurosporine-induced apoptosis, by comparing the imaging results with Western-blot analyses of different subcellular portions.

DIMENSION: Dynamic MR imaging with both k-space and spatial prior knowledge obtained via multi-supervised network training.

Dynamic MR image reconstruction from incomplete k-space data has generated great research interest due to its capability in reducing scan time. Nevertheless, the reconstruction problem is still challenging due to its ill-posed nature. Most existing methods either suffer from long iterative reconstruction time or explore limited prior knowledge. This paper proposes a dynamic MR imaging method with both k-space and spatial prior knowledge integrated via multi-supervised network training, dubbed as DIMENSION. Specifically, the DIMENSION architecture consists of a frequential prior network for updating the k-space with its network prediction and a spatial prior network for capturing image structures and details. Furthermore, a multi-supervised network training technique is developed to constrain the frequency domain information and the spatial domain information. The comparisons with classical k-t FOCUSS, k-t SLR, L+S and the state-of-the-art CNN-based method on in vivo datasets show our method can achieve improved reconstruction results in shorter time.

A 4-minute solution for submillimeter whole-brain T1ρ quantification.

PURPOSE: To develop a robust, accurate, and accelerated T1ρ quantification solution for submillimeter in vivo whole-brain imaging. METHODS: A multislice T1ρ mapping solution (MS-T1ρ ) was developed based on a two-acquisition scheme using turbo spin echo with RF cycling to allow for whole-brain coverage with 0.8-mm in-plane resolution. A compressed sensing-based fast imaging method, SCOPE, was used to accelerate the MS-T1ρ acquisition time to a total scan time of 3 minutes 31 seconds. A phantom experiment was conducted to assess the accuracy of MS-T1ρ by comparing the T1ρ value obtained using MS-T1ρ with the reference value obtained using the standard single-slice T1ρ mapping method. In vivo scans of 13 volunteers were acquired prospectively to validate the robustness of MS-T1ρ . RESULTS: In the phantom study, the T1ρ values obtained with MS-T1ρ were in good agreement with the reference T1ρ values (R2 = 0.9991) and showed high consistency throughout all slices (coefficient of variation = 2.2 ± 2.43%). In the in vivo experiments, T1ρ maps were successfully acquired for all volunteers with no visually noticeable artifacts. There was no significant difference in T1ρ values between MS-T1ρ acquisitions and fully sampled acquisitions for all brain tissues (p-value > .05). In the intraclass correlation coefficient and Bland-Altman analyses, the accelerated T1ρ measurements show moderate to good agreement to the fully sampled reference values. CONCLUSION: The proposed MS-T1ρ solution allows for high-resolution whole-brain T1ρ mapping within 4 minutes and may provide a potential tool for investigating neural diseases.

SuperDTI: Ultrafast DTI and fiber tractography with deep learning.

PURPOSE: To develop a deep learning-based reconstruction framework for ultrafast and robust diffusion tensor imaging and fiber tractography. METHODS: SuperDTI was developed to learn the nonlinear relationship between DWIs and the corresponding diffusion tensor parameter maps. It bypasses the tensor fitting procedure, which is highly susceptible to noises and motions in DWIs. The network was trained and tested using data sets from the Human Connectome Project and patients with ischemic stroke. Results from SuperDTI were compared against widely used methods for tensor parameter estimation and fiber tracking. RESULTS: Using training and testing data acquired using the same protocol and scanner, SuperDTI was shown to generate fractional anisotropy and mean diffusivity maps, as well as fiber tractography, from as few as six raw DWIs, with a quantification error of less than 5% in all white-matter and gray-matter regions of interest. It was robust to noises and motions in the testing data. Furthermore, the network trained using healthy volunteer data showed no apparent reduction in lesion detectability when directly applied to stroke patient data. CONCLUSIONS: Our results demonstrate the feasibility of superfast DTI and fiber tractography using deep learning with as few as six DWIs directly, bypassing tensor fitting. Such a significant reduction in scan time may allow the inclusion of DTI into the clinical routine for many potential applications.

Acceleration of three-dimensional diffusion magnetic resonance imaging using a kernel low-rank compressed sensing method.

Diffusion Magnetic Resonance Imaging (dMRI) has shown great potential in probing tissue microstructure and structural connectivity in the brain but is often limited by the lengthy scan time needed to sample the diffusion profile by acquiring multiple diffusion weighted images (DWIs). Although parallel imaging technique has improved the speed of dMRI acquisition, attaining high resolution three dimensional (3D) dMRI on preclinical MRI systems remained still time consuming. In this paper, kernel principal component analysis, a machine learning approach, was employed to estimate the correlation among DWIs. We demonstrated the feasibility of such correlation estimation from low-resolution training DWIs and used the correlation as a constraint to reconstruct high-resolution DWIs from highly under-sampled k-space data, which significantly reduced the scan time. Using full k-space 3D dMRI data of post-mortem mouse brains, we retrospectively compared the performance of the so-called kernel low rank (KLR) method with a conventional compressed sensing (CS) method in terms of image quality and ability to resolve complex fiber orientations and connectivity. The results demonstrated that the KLR-CS method outperformed the conventional CS method for acceleration factors up to 8 and was likely to enhance our ability to investigate brain microstructure and connectivity using high-resolution 3D dMRI.

Bio-SCOPE: fast biexponential T1ρ mapping of the brain using signal-compensated low-rank plus sparse matrix decomposition.

PURPOSE: To develop and evaluate a fast imaging method based on signal-compensated low-rank plus sparse matrix decomposition to accelerate data acquisition for biexponential brain T1ρ mapping (Bio-SCOPE). METHODS: Two novel strategies were proposed to improve reconstruction performance. A variable-rate undersampling scheme was used with a varied acceleration factor for each k-space along the spin-lock time direction, and a modified nonlinear thresholding scheme combined with a feature descriptor was used for Bio-SCOPE reconstruction. In vivo brain T1ρ mappings were acquired from 4 volunteers. The fully sampled k-space data acquired from 3 volunteers were retrospectively undersampled by net acceleration rates (R) of 4.6 and 6.1. Reference values were obtained from the fully sampled data. The agreement between the accelerated T1ρ measurements and reference values was assessed with Bland-Altman analyses. Prospectively undersampled data with R = 4.6 and R = 6.1 were acquired from 1 volunteer. RESULTS: T1ρ -weighted images were successfully reconstructed using Bio-SCOPE for R = 4.6 and 6.1 with signal-to-noise ratio variations <1 dB and normalized root mean square errors <4%. Accelerated and reference T1ρ measurements were in good agreement for R = 4.6 (T1ρs : 18.6651 ± 1.7786 ms; T1ρl : 88.9603 ± 1.7331 ms) and R = 6.1 (T1ρs : 17.8403 ± 3.3302 ms; T1ρl : 88.0275 ± 4.9606 ms) in the Bland-Altman analyses. T1ρ parameter maps from prospectively undersampled data also show reasonable image quality using the Bio-SCOPE method. CONCLUSION: Bio-SCOPE achieves a high net acceleration rate for biexponential T1ρ mapping and improves reconstruction quality by using a variable-rate undersampling data acquisition scheme and a modified soft-thresholding algorithm in image reconstruction.

Deep Magnetic Resonance Image Reconstruction: Inverse Problems Meet Neural Networks.

Image reconstruction from undersampled k-space data has been playing an important role in fast MRI. Recently, deep learning has demonstrated tremendous success in various fields and has also shown potential in significantly accelerating MRI reconstruction with fewer measurements. This article provides an overview of the deep learning-based image reconstruction methods for MRI. Two types of deep learning-based approaches are reviewed: those based on unrolled algorithms and those which are not. The main structure of both approaches are explained, respectively. Several signal processing issues for maximizing the potential of deep reconstruction in fast MRI are discussed. The discussion may facilitate further development of the networks and the analysis of performance from a theoretical point of view.

Machine-learning based spectral classification for spectroscopic single-molecule localization microscopy.

Spectroscopic single-molecule localization microscopy (sSMLM) simultaneously captures the spatial locations and emission spectra of single molecular emissions and enables simultaneous multicolor super-resolution imaging. Existing sSMLM relies on extracting spectral signatures, such as weighted spectral centroids, to distinguish different molecular labels. However, the rich information carried by the complete spectral profiles is not fully utilized; thus, the misclassification rate between molecular labels can be high at low spectral analysis photon budget. We developed a machine learning (ML)-based method to analyze the full spectral profiles of each molecular emission and reduce the misclassification rate. We experimentally validated our method by imaging immunofluorescently labeled COS-7 cells using two far-red dyes typically used in sSMLM (AF647 and CF660) to resolve mitochondria and microtubules, respectively. We showed that the ML method achieved 10-fold reduction in misclassification and two-fold improvement in spectral data utilization comparing with the existing spectral centroid method.

Improved parallel image reconstruction using feature refinement.

PURPOSE: The aim of this study was to develop a novel feature refinement MR reconstruction method from highly undersampled multichannel acquisitions for improving the image quality and preserve more detail information. THEORY AND METHODS: The feature refinement technique, which uses a feature descriptor to pick up useful features from residual image discarded by sparsity constrains, is applied to preserve the details of the image in compressed sensing and parallel imaging in MRI (CS-pMRI). The texture descriptor and structure descriptor recognizing different types of features are required for forming the feature descriptor. Feasibility of the feature refinement was validated using three different multicoil reconstruction methods on in vivo data. RESULTS: Experimental results show that reconstruction methods with feature refinement improve the quality of reconstructed image and restore the image details more accurately than the original methods, which is also verified by the lower values of the root mean square error and high frequency error norm. CONCLUSION: A simple and effective way to preserve more useful detailed information in CS-pMRI is proposed. This technique can effectively improve the reconstruction quality and has superior performance in terms of detail preservation compared with the original version without feature refinement. Magn Reson Med 80:211-223, 2018. © 2017 International Society for Magnetic Resonance in Medicine.

Accelerated exponential parameterization of T2 relaxation with model-driven low rank and sparsity priors (MORASA).

PURPOSE: This work is to develop a novel image reconstruction method from highly undersampled multichannel acquisition to reduce the scan time of exponential parameterization of T2 relaxation. THEORY AND METHODS: On top of the low-rank and joint-sparsity constraints, we propose to exploit the linear predictability of the T2 exponential decay to further improve the reconstruction of the T2-weighted images from undersampled acquisitions. Specifically, the exact rank prior (i.e., number of non-zero singular values) is adopted to enforce the spatiotemporal low rankness, while the mixed L2-L1 norm of the wavelet coefficients is used to promote joint sparsity, and the Hankel low-rank approximation is used to impose linear predictability, which integrates the exponential behavior of the temporal signal into the reconstruction process. An efficient algorithm is adopted to solve the reconstruction problem, where corresponding nonlinear filtering operations are performed to enforce corresponding priors in an iterative manner. RESULTS: Both simulated and in vivo datasets with multichannel acquisition were used to demonstrate the feasibility of the proposed method. Experimental results have shown that the newly introduced linear predictability prior improves the reconstruction quality of the T2-weighted images and benefits the subsequent T2 mapping by achieving high-speed, high-quality T2 mapping compared with the existing fast T2 mapping methods. CONCLUSION: This work proposes a novel fast T2 mapping method integrating the linear predictable property of the exponential decay into the reconstruction process. The proposed technique can effectively improve the reconstruction quality of the state-of-the-art fast imaging method exploiting image sparsity and spatiotemporal low rankness. Magn Reson Med 76:1865-1878, 2016. © 2016 International Society for Magnetic Resonance in Medicine.

Accelerating T1ρ cartilage imaging using compressed sensing with iterative locally adapted support detection and JSENSE.

PURPOSE: To accelerate T1ρ quantification in cartilage imaging using combined compressed sensing with iterative locally adaptive support detection and JSENSE. METHODS: To reconstruct T1ρ images from accelerated acquisition at different time of spin-lock (TSLs), we propose an approach to combine an advanced compressed sensing (CS) based reconstruction technique, LAISD (locally adaptive iterative support detection), and an advanced parallel imaging technique, JSENSE. Specifically, the reconstruction process alternates iteratively among local support detection in the domain of principal component analysis, compressed sensing reconstruction of the image sequence, and sensitivity estimation with JSENSE. T1ρ quantification results from accelerated scans using the proposed method are evaluated using in vivo knee cartilage data from bilateral scans of three healthy volunteers. RESULTS: T1ρ maps obtained from accelerated scans (acceleration factors of 3 and 3.5) using the proposed method showed results comparable to conventional full scans. The T1ρ errors in all compartments are below 1%, which is well below the in vivo reproducibility of cartilage T1ρ reported from previous studies. CONCLUSION: The proposed method can significantly accelerate the acquisition process of T1ρ quantification on human cartilage imaging without sacrificing accuracy, which will greatly facilitate the clinical translation of quantitative cartilage MRI.

Enabling high-speed wide-field dynamic imaging in multifocal photoacoustic computed microscopy: a simulation study.

Photoacoustic-computed microscopy (PACM) is an emerging technology that employs thousands of optical foci to provide wide-field high-resolution images of tissue optical absorption. A major limitation of PACM is the slow imaging speed, limiting its usage in dynamic imaging. In this study, we improved the speed through a two-step approach. First, we employed compressed sensing with partially known support to reduce the transducer element number, which subsequently improved the imaging speed at each optical scanning step. Second, we use the high-speed low-resolution image acquired without microlens array to inform dynamic changes in the high-resolution PACM image. Combining both approaches, we achieved high-resolution dynamic imaging over a wide field.

Fast GRAPPA reconstruction with random projection.

PURPOSE: To address the issue of computational complexity in generalized autocalibrating partially parallel acquisition (GRAPPA) when several calibration data are used. METHOD: GRAPPA requires fully sampled data for accurate calibration with increasing data needed for higher reduction factors to maintain accuracy, which leads to longer computational time, especially in a three-dimensional (3D) setting and with higher channel count coils. Channel reduction methods have been developed to address this issue when massive array coils are used. In this study, the complexity problem was addressed from a different prospective. Instead of compressing to fewer channels, we propose the use of random projections to reduce the dimension of the linear equation in the calibration phase. The equivalence before and after the reduction is supported by the Johnson-Lindenstrauss lemma. The proposed random projection method can be integrated with channel reduction sequentially for even higher computational efficiency. RESULTS: Experimental results show that GRAPPA with random projection can achieve comparable image quality with much less computational time when compared with conventional GRAPPA without random projection. CONCLUSION: The proposed random projection method is able to reduce the computational time of GRAPPA, especially in a 3D setting, without compromising the image quality, or to improve the reconstruction quality by allowing more data for calibration when the computational time is a limiting factor. Magn Reson Med 74:71-80, 2015. © 2014 Wiley Periodicals, Inc.

Incorporating reference in parallel imaging and compressed sensing.

PURPOSE: To develop a new compressed sensing parallel imaging technique called READ-PICS that can effectively incorporate prior information from a reference scan for MR image reconstruction from highly undersampled multichannel measurements. METHODS: READ-PICS incorporates information from a high-spatial-resolution reference prior using the generalized series model, to achieve increased image sparsity and mitigated noise amplification simultaneously. To further improve the ill-conditioning of the parallel imaging system, an annular area in the central residual k-space is used for calibration. Additionally, the mixed L1-L2 norm of the coefficients from the prior component and residual component is used to enforce joint sparsity. RESULTS: The evaluations on parametric imaging and multiscan experiment demonstrate superior performance of READ-PICS in terms of detail preservation and noise suppression compared to state-of-the-art technique, L1-Iterative self-consistent parallel imaging reconstruction, and prescan required method, correlation imaging. CONCLUSIONS: The proposed method can significantly increase signal sparsity and improve the ill-conditioning of the parallel imaging system using reference adaptive regularization. This technique can be easily adapted to other imaging applications where multiple images need to be acquired sequentially and a reference prior is also available.

Sparse BLIP: BLind Iterative Parallel imaging reconstruction using compressed sensing.

PURPOSE: To develop a sensitivity-based parallel imaging reconstruction method to reconstruct iteratively both the coil sensitivities and MR image simultaneously based on their prior information. METHODS: Parallel magnetic resonance imaging reconstruction problem can be formulated as a multichannel sampling problem where solutions are sought analytically. However, the channel functions given by the coil sensitivities in parallel imaging are not known exactly and the estimation error usually leads to artifacts. In this study, we propose a new reconstruction algorithm, termed Sparse BLind Iterative Parallel, for blind iterative parallel imaging reconstruction using compressed sensing. The proposed algorithm reconstructs both the sensitivity functions and the image simultaneously from undersampled data. It enforces the sparseness constraint in the image as done in compressed sensing, but is different from compressed sensing in that the sensing matrix is unknown and additional constraint is enforced on the sensitivities as well. Both phantom and in vivo imaging experiments were carried out with retrospective undersampling to evaluate the performance of the proposed method. RESULTS: Experiments show improvement in Sparse BLind Iterative Parallel reconstruction when compared with Sparse SENSE, JSENSE, IRGN-TV, and L1-SPIRiT reconstructions with the same number of measurements. CONCLUSION: The proposed Sparse BLind Iterative Parallel algorithm reduces the reconstruction errors when compared to the state-of-the-art parallel imaging methods.

Accelerated MR diffusion tensor imaging using distributed compressed sensing.

PURPOSE: Diffusion tensor imaging (DTI) is known to suffer from long acquisition time in the orders of several minutes or even hours. Therefore, a feasible way to accelerate DTI data acquisition is highly desirable. In this article, the feasibility and efficacy of distributed compressed sensing to fast DTI is investigated by exploiting the joint sparsity prior in diffusion-weighted images. METHODS: Fully sampled DTI datasets were obtained from both simulated phantom and experimental heart sample, with diffusion gradient applied in six directions. The k-space data were undersampled retrospectively with acceleration factors from 2 to 6. Diffusion-weighted images were reconstructed by solving an l2-l1 norm minimization problem. Reconstruction performance with varied signal-to-noise ratio and acceleration factors were evaluated by root-mean-square error and maps of reconstructed DTI indices. RESULTS: Superiority of distributed compressed sensing over basic compressed sensing was confirmed with simulation, and the reconstruction accuracy was influenced by signal-to-noise ratio and acceleration factors. Experimental results demonstrate that DTI indices including fractional anisotropy, mean diffusivities, and orientation of primary eigenvector can be obtained with high accuracy at acceleration factors up to 4. CONCLUSION: Distributed compressed sensing is shown to be able to accelerate DTI and may be used to reduce DTI acquisition time practically.

Self-Supervised Deep Unrolled Reconstruction Using Regularization by Denoising.

Deep learning methods have been successfully used in various computer vision tasks. Inspired by that success, deep learning has been explored in magnetic resonance imaging (MRI) reconstruction. In particular, integrating deep learning and model-based optimization methods has shown considerable advantages. However, a large amount of labeled training data is typically needed for high reconstruction quality, which is challenging for some MRI applications. In this paper, we propose a novel reconstruction method, named DURED-Net, that enables interpretable self-supervised learning for MR image reconstruction by combining a self-supervised denoising network and a plug-and-play method. We aim to boost the reconstruction performance of Noise2Noise in MR reconstruction by adding an explicit prior that utilizes imaging physics. Specifically, the leverage of a denoising network for MRI reconstruction is achieved using Regularization by Denoising (RED). Experiment results demonstrate that the proposed method requires a reduced amount of training data to achieve high reconstruction quality among the state-of-the-art approaches utilizing Noise2Noise.