Noise2Recon: Enabling SNR-robust MRI reconstruction with semi-supervised and self-supervised learning.

PURPOSE: To develop a method for building MRI reconstruction neural networks robust to changes in signal-to-noise ratio (SNR) and trainable with a limited number of fully sampled scans. METHODS: We propose Noise2Recon, a consistency training method for SNR-robust accelerated MRI reconstruction that can use both fully sampled (labeled) and undersampled (unlabeled) scans. Noise2Recon uses unlabeled data by enforcing consistency between model reconstructions of undersampled scans and their noise-augmented counterparts. Noise2Recon was compared to compressed sensing and both supervised and self-supervised deep learning baselines. Experiments were conducted using retrospectively accelerated data from the mridata three-dimensional fast-spin-echo knee and two-dimensional fastMRI brain datasets. All methods were evaluated in label-limited settings and among out-of-distribution (OOD) shifts, including changes in SNR, acceleration factors, and datasets. An extensive ablation study was conducted to characterize the sensitivity of Noise2Recon to hyperparameter choices. RESULTS: In label-limited settings, Noise2Recon achieved better structural similarity, peak signal-to-noise ratio, and normalized-RMS error than all baselines and matched performance of supervised models, which were trained with 14 × $$ 14\times $$ more fully sampled scans. Noise2Recon outperformed all baselines, including state-of-the-art fine-tuning and augmentation techniques, among low-SNR scans and when generalizing to OOD acceleration factors. Augmentation extent and loss weighting hyperparameters had negligible impact on Noise2Recon compared to supervised methods, which may indicate increased training stability. CONCLUSION: Noise2Recon is a label-efficient reconstruction method that is robust to distribution shifts, such as changes in SNR, acceleration factors, and others, with limited or no fully sampled training data.

SLfRank: Shinnar-Le-Roux Pulse Design With Reduced Energy and Accurate Phase Profiles Using Rank Factorization.

The Shinnar-Le-Roux (SLR) algorithm is widely used to design frequency selective pulses with large flip angles. We improve its design process to generate pulses with lower energy (by as much as 26%) and more accurate phase profiles. Concretely, the SLR algorithm consists of two steps: (1) an invertible transform between frequency selective pulses and polynomial pairs that represent Cayley-Klein (CK) parameters and (2) the design of the CK polynomial pair to match the desired magnetization profiles. Because the CK polynomial pair is bi-linearly coupled, the original algorithm sequentially solves for each polynomial instead of jointly. This results in sub-optimal pulses. Instead, we leverage a convex relaxation technique, commonly used for low rank matrix recovery, to address the bi-linearity. Our numerical experiments show that the resulting pulses are almost always globally optimal in practice. For slice excitation, the proposed algorithm results in more accurate linear phase profiles. And in general the improved pulses have lower energy than the original SLR pulses.

Optimization in the space domain for density compensation with the nonuniform FFT.

The non-uniform Discrete Fourier Transform algorithm has shown great utility for reconstructing images from non-uniformly spaced Fourier samples in several imaging modalities. Due to the non-uniform spacing, some correction for the variable density of the samples must be made. Common methods for generating density compensation values are either sub-optimal or only consider a finite set of points in the optimization. This manuscript presents an algorithm for generating density compensation values from a set of Fourier samples that takes into account the point spread function over an entire rectangular region in the image domain. We show that the reconstructed images using the density compensation values of this method are of superior quality when compared to other standard methods. Results are shown with a numerical phantom and with magnetic resonance images of the abdomen and the knee.

Automated MRI Field of View Prescription from Region of Interest Prediction by Intra-Stack Attention Neural Network.

Manual prescription of the field of view (FOV) by MRI technologists is variable and prolongs the scanning process. Often, the FOV is too large or crops critical anatomy. We propose a deep learning framework, trained by radiologists' supervision, for automating FOV prescription. An intra-stack shared feature extraction network and an attention network are used to process a stack of 2D image inputs to generate scalars defining the location of a rectangular region of interest (ROI). The attention mechanism is used to make the model focus on a small number of informative slices in a stack. Then, the smallest FOV that makes the neural network predicted ROI free of aliasing is calculated by an algebraic operation derived from MR sampling theory. The framework's performance is examined quantitatively with intersection over union (IoU) and pixel error on position and qualitatively with a reader study. The proposed model achieves an average IoU of 0.867 and an average ROI position error of 9.06 out of 512 pixels on 80 test cases, significantly better than two baseline models and not significantly different from a radiologist. Finally, the FOV given by the proposed framework achieves an acceptance rate of 92% from an experienced radiologist.

Rapid fat-water separated T1 mapping using a single-shot radial inversion-recovery spoiled gradient recalled pulse sequence.

T1 mapping is increasingly used in clinical practice and research studies. With limited scan time, existing techniques often have limited spatial resolution, contrast resolution and slice coverage. High fat concentrations yield complex errors in Look-Locker T1 methods. In this study, a dual-echo 2D radial inversion-recovery T1 (DEradIR-T1) technique was developed for fast fat-water separated T1 mapping. The DEradIR-T1 technique was tested in phantoms, 5 volunteers and 28 patients using a 3 T clinical MRI scanner. In our study, simulations were performed to analyze the composite (fat + water) and water-only T1 under different echo times (TE). In standardized phantoms, an inversion-recovery spin echo (IR-SE) sequence with and without fat saturation pulses served as a T1 reference. Parameter mapping with DEradIR-T1 was also assessed in vivo, and values were compared with modified Look-Locker inversion recovery (MOLLI). Bland-Altman analysis and two-tailed paired t-tests were used to compare the parameter maps from DEradIR-T1 with the references. Simulations of the composite and water-only T1 under different TE values and levels of fat matched the in vivo studies. T1 maps from DEradIR-T1 on a NIST phantom (Pcomp = 0.97) and a Calimetrix fat-water phantom (Pwater = 0.56) matched with the references. In vivo T1 was compared with that of MOLLI: R comp 2 = 0.77 ; R water 2 = 0.72 . In this work, intravoxel fat is found to have a variable, echo-time-dependent effect on measured T1 values, and this effect may be mitigated using the proposed DRradIR-T1.

Artifact- and content-specific quality assessment for MRI with image rulers.

In clinical practice MR images are often first seen by radiologists long after the scan. If image quality is inadequate either patients have to return for an additional scan, or a suboptimal interpretation is rendered. An automatic image quality assessment (IQA) would enable real-time remediation. Existing IQA works for MRI give only a general quality score, agnostic to the cause of and solution to low-quality scans. Furthermore, radiologists' image quality requirements vary with the scan type and diagnostic task. Therefore, the same score may have different implications for different scans. We propose a framework with multi-task CNN model trained with calibrated labels and inferenced with image rulers. Labels calibrated by human inputs follow a well-defined and efficient labeling task. Image rulers address varying quality standards and provide a concrete way of interpreting raw scores from the CNN. The model supports assessments of two of the most common artifacts in MRI: noise and motion. It achieves accuracies of around 90%, 6% better than the best previous method examined, and 3% better than human experts on noise assessment. Our experiments show that label calibration, image rulers, and multi-task training improve the model's performance and generalizability.

Attention-guided deep learning for gestational age prediction using fetal brain MRI.

Magnetic resonance imaging offers unrivaled visualization of the fetal brain, forming the basis for establishing age-specific morphologic milestones. However, gauging age-appropriate neural development remains a difficult task due to the constantly changing appearance of the fetal brain, variable image quality, and frequent motion artifacts. Here we present an end-to-end, attention-guided deep learning model that predicts gestational age with R2 score of 0.945, mean absolute error of 6.7 days, and concordance correlation coefficient of 0.970. The convolutional neural network was trained on a heterogeneous dataset of 741 developmentally normal fetal brain images ranging from 19 to 39 weeks in gestational age. We also demonstrate model performance and generalizability using independent datasets from four academic institutions across the U.S. and Turkey with R2 scores of 0.81-0.90 after minimal fine-tuning. The proposed regression algorithm provides an automated machine-enabled tool with the potential to better characterize in utero neurodevelopment and guide real-time gestational age estimation after the first trimester.

Utilizing the Wavelet Transform's Structure in Compressed Sensing.

Compressed sensing has empowered quality image reconstruction with fewer data samples than previously thought possible. These techniques rely on a sparsifying linear transformation. The Daubechies wavelet transform is commonly used for this purpose. In this work, we take advantage of the structure of this wavelet transform and identify an affine transformation that increases the sparsity of the result. After inclusion of this affine transformation, we modify the resulting optimization problem to comply with the form of the Basis Pursuit Denoising problem. Finally, we show theoretically that this yields a lower bound on the error of the reconstruction and present results where solving this modified problem yields images of higher quality for the same sampling patterns using both magnetic resonance and optical images.

Biopsy marker localization with thermo-acoustic ultrasound for lumpectomy guidance.

PURPOSE: Almost one in four lumpectomies fails to fully remove cancerous tissue from the breast, requiring reoperation. This high failure rate suggests that existing lumpectomy guidance methods are inadequate for allowing surgeons to consistently identify the proper volume of tissue for excision. Current guidance techniques either provide little information about the tumor position or require surgeons to frequently switch between making incisions and manually probing for a marker placed at the lesion site. This article explores the feasibility of thermo-acoustic ultrasound (TAUS) to enable hands-free localization of metallic biopsy markers throughout surgery, which would allow for continuous visualization of the lesion site in the breast without the interruption of surgery. In a TAUS-based localization system, microwave excitations would be transmitted into the breast, and the amplification in microwave absorption around the metallic markers would generate acoustic signals from the marker sites through the thermo-acoustic effect. Detection and ranging of these signals by multiple acoustic receivers on the breast could then enable marker localization through acoustic multilateration. METHODS: Physics simulations were used to characterize the TAUS signals generated from different markers by microwave excitations. First, electromagnetic simulations determined the spatial pattern of the amplification in microwave absorption around the markers. Then, acoustic simulations characterized the acoustic fields generated from these markers at various acoustic frequencies. TAUS-based one-dimensional (1D) ranging of two metallic markers-including a biopsy marker that is FDA-approved for clinical use-immersed in saline was also performed using a bench-top setup. To perform TAUS acquisitions, a microwave applicator was driven by 2.66 GHz microwave signals that were amplitude-modulated by chirps at the desired acoustic excitation frequencies, and the resulting TAUS signal from the markers was detected by an ultrasonic transducer. RESULTS: The simulation results show that the geometry of the marker strongly impacts the quantity and spatial pattern of both the microwave absorption around the marker and the resulting TAUS signal generated from the marker. The simulated TAUS signal maps and acoustic frequency responses also make clear that the marker geometry plays an important role in determining the overall system response. Using the bench-top setup, TAUS detection and 1D localization of the markers were successfully demonstrated for multiple different combinations of microwave applicator and metallic marker. These initial results indicate that TAUS-based localization of biopsy markers is feasible. CONCLUSIONS: Through microwave excitations and acoustic detection, TAUS can be used to localize metallic biopsy markers. With further development, TAUS opens new avenues to enable a more intuitive lumpectomy guidance system that could help to achieve better lumpectomy outcomes.

Wasserstein GANs for MR Imaging: From Paired to Unpaired Training.

Lack of ground-truth MR images impedes the common supervised training of neural networks for image reconstruction. To cope with this challenge, this article leverages unpaired adversarial training for reconstruction networks, where the inputs are undersampled k-space and naively reconstructed images from one dataset, and the labels are high-quality images from another dataset. The reconstruction networks consist of a generator which suppresses the input image artifacts, and a discriminator using a pool of (unpaired) labels to adjust the reconstruction quality. The generator is an unrolled neural network - a cascade of convolutional and data consistency layers. The discriminator is also a multilayer CNN that plays the role of a critic scoring the quality of reconstructed images based on the Wasserstein distance. Our experiments with knee MRI datasets demonstrate that the proposed unpaired training enables diagnostic-quality reconstruction when high-quality image labels are not available for the input types of interest, or when the amount of labels is small. In addition, our adversarial training scheme can achieve better image quality (as rated by expert radiologists) compared with the paired training schemes with pixel-wise loss.

Multi-Domain Image Completion for Random Missing Input Data.

Multi-domain data are widely leveraged in vision applications taking advantage of complementary information from different modalities, e.g., brain tumor segmentation from multi-parametric magnetic resonance imaging (MRI). However, due to possible data corruption and different imaging protocols, the availability of images for each domain could vary amongst multiple data sources in practice, which makes it challenging to build a universal model with a varied set of input data. To tackle this problem, we propose a general approach to complete the random missing domain(s) data in real applications. Specifically, we develop a novel multi-domain image completion method that utilizes a generative adversarial network (GAN) with a representational disentanglement scheme to extract shared content encoding and separate style encoding across multiple domains. We further illustrate that the learned representation in multi-domain image completion could be leveraged for high-level tasks, e.g., segmentation, by introducing a unified framework consisting of image completion and segmentation with a shared content encoder. The experiments demonstrate consistent performance improvement on three datasets for brain tumor segmentation, prostate segmentation, and facial expression image completion respectively.

Fast variable density Poisson-disc sample generation with directional variation for compressed sensing in MRI.

We present a fast method for generating random samples according to a variable density poisson-disc distribution. A minimum parameter value is used to create a background grid array for keeping track of those points that might affect any new candidate point; this reduces the number of conflicts that must be checked before acceptance of a new point, thus reducing the number of computations required. We demonstrate the algorithm's ability to generate variable density poisson-disc sampling patterns according to a parameterized function, including patterns where the variations in density are a function of direction. We further show that these sampling patterns are appropriate for compressed sensing applications. Finally, we present a method to generate patterns with a specific acceleration rate.

Multi-scale Unrolled Deep Learning Framework for Accelerated Magnetic Resonance Imaging.

Accelerating data acquisition in magnetic resonance imaging (MRI) has been of perennial interest due to its prohibitively slow data acquisition process. Recent trends in accelerating MRI employ data-centric deep learning frameworks due to its fast inference time and 'one-parameter-fit-all' principle unlike in traditional model-based acceleration techniques. Unrolled deep learning framework that combines the deep priors and model knowledge are robust compared to naive deep learning based framework. In this paper, we propose a novel multi-scale unrolled deep learning framework which learns deep image priors through multi-scale CNN and is combined with unrolled framework to enforce data-consistency and model knowledge. Essentially, this framework combines the best of both learning paradigms:model-based and data-centric learning paradigms. Proposed method is verified using several experiments on numerous data sets.

DIAGNOSTIC IMAGE QUALITY ASSESSMENT AND CLASSIFICATION IN MEDICAL IMAGING: OPPORTUNITIES AND CHALLENGES.

Magnetic Resonance Imaging (MRI) suffers from several artifacts, the most common of which are motion artifacts. These artifacts often yield images that are of non-diagnostic quality. To detect such artifacts, images are prospectively evaluated by experts for their diagnostic quality, which necessitates patient-revisits and rescans whenever non-diagnostic quality scans are encountered. This motivates the need to develop an automated framework capable of accessing medical image quality and detecting diagnostic and non-diagnostic images. In this paper, we explore several convolutional neural network-based frameworks for medical image quality assessment and investigate several challenges therein.

Compressed Sensing: From Research to Clinical Practice with Deep Neural Networks.

Compressed sensing (CS) reconstruction methods leverage sparse structure in underlying signals to recover high-resolution images from highly undersampled measurements. When applied to magnetic resonance imaging (MRI), CS has the potential to dramatically shorten MRI scan times, increase diagnostic value, and improve overall patient experience. However, CS has several shortcomings which limit its clinical translation such as: 1) artifacts arising from inaccurate sparse modelling assumptions, 2) extensive parameter tuning required for each clinical application, and 3) clinically infeasible reconstruction times. Recently, CS has been extended to incorporate deep neural networks as a way of learning complex image priors from historical exam data. Commonly referred to as unrolled neural networks, these techniques have proven to be a compelling and practical approach to address the challenges of sparse CS. In this tutorial, we will review the classical compressed sensing formulation and outline steps needed to transform this formulation into a deep learning-based reconstruction framework. Supplementary open source code in Python will be used to demonstrate this approach with open databases. Further, we will discuss considerations in applying unrolled neural networks in the clinical setting.

Two-dimensional UTE overview imaging for dental application.

PURPOSE: To investigate the applicability of a 2D-UTE half-pulse sequence for dental overview imaging and the detection of signal from mineralized dental tissue and caries lesions with ultra-short T2∗ as an efficient alternative to 3D sequences. METHODS: A modified 2D-UTE sequence using 240-µs half-pulses for excitation and a reduction of the coil tune delay from the manufacturer preset value allowed for the acquisition of in vivo dental images with a TE of 35 µs at 1.5T. The common occurrence of out-of-slice signal for half-pulse sequences was avoided by applying a quadratic-phase saturation pulse before each half-RF excitation. A conventional 2D-UTE sequence with a TE of 750 µs, using slice selection rephasing, was used for comparison. RESULTS: Quadratic phase saturation pulses adequately improve the slice profile of half-pulse excitations for dental imaging with a surface coil. In vivo images and SNR measurements show a distinct increase in signal in ultrashort T2∗ tissues for the proposed 2D-UTE half-pulse sequence compared with a 2D-UTE sequence using conventional slice selection, leading to an improved detection of caries lesions. CONCLUSION: The proposed pulse sequence enables the acquisition of in vivo images of a comprehensive overview of bone structures and teeth of a single side of the upper and lower jaw and signal detection from mineralized dental tissues in clinically acceptable scan times.

Rosette Trajectories Enable Ungated, Motion-Robust, Simultaneous Cardiac and Liver T2 * Iron Assessment.

BACKGROUND: Quantitative T2 * MRI is the standard of care for the assessment of iron overload. However, patient motion corrupts T2 * estimates. PURPOSE: To develop and evaluate a motion-robust, simultaneous cardiac and liver T2 * imaging approach using non-Cartesian, rosette sampling and a model-based reconstruction as compared to clinical-standard Cartesian MRI. STUDY TYPE: Prospective. PHANTOM/POPULATION: Six ferumoxytol-containing phantoms (26-288 µg/mL). Eight healthy subjects and 18 patients referred for clinically indicated iron overload assessment. FIELD STRENGTH/SEQUENCE: 1.5T, 2D Cartesian and rosette gradient echo (GRE) ASSESSMENT: GRE T2 * values were validated in ferumoxytol phantoms. In healthy subjects, test-retest and spatial coefficient of variation (CoV) analysis was performed during three breathing conditions. Cartesian and rosette T2 * were compared using correlation and Bland-Altman analysis. Images were rated by three experienced radiologists on a 5-point scale. STATISTICAL TESTS: Linear regression, analysis of variance (ANOVA), and paired Student's t-testing were used to compare reproducibility and variability metrics in Cartesian and rosette scans. The Wilcoxon rank test was used to assess reader score comparisons and reader reliability was measured using intraclass correlation analysis. RESULTS: Rosette R2* (1/T2 *) was linearly correlated with ferumoxytol concentration (r2 = 1.00) and not significantly different than Cartesian values (P = 0.16). During breath-holding, ungated rosette liver and heart T2 * had lower spatial CoV (liver: 18.4 ± 9.3% Cartesian, 8.8% ± 3.4% rosette, P = 0.02, heart: 37.7% ± 14.3% Cartesian, 13.4% ± 1.7% rosette, P = 0.001) and higher-quality scores (liver: 3.3 [3.0-3.6] Cartesian, 4.7 [4.1-4.9] rosette, P = 0.005, heart: 3.0 [2.3-3] Cartesian, 4.5 [3.8-5.0] rosette, P = 0.005) compared to Cartesian values. During free-breathing and failed breath-holding, Cartesian images had very poor to average image quality with significant artifacts, whereas rosette remained very good, with minimal artifacts (P = 0.001). DATA CONCLUSION: Rosette k-sampling with a model-based reconstruction offers a clinically useful motion-robust T2 * mapping approach for iron quantification. J. MAGN. RESON. IMAGING 2020;52:1688-1698.

Synthesize High-Quality Multi-Contrast Magnetic Resonance Imaging From Multi-Echo Acquisition Using Multi-Task Deep Generative Model.

Multi-echo saturation recovery sequence can provide redundant information to synthesize multi-contrast magnetic resonance imaging. Traditional synthesis methods, such as GE's MAGiC platform, employ a model-fitting approach to generate parameter-weighted contrasts. However, models' over-simplification, as well as imperfections in the acquisition, can lead to undesirable reconstruction artifacts, especially in T2-FLAIR contrast. To improve the image quality, in this study, a multi-task deep learning model is developed to synthesize multi-contrast neuroimaging jointly using both signal relaxation relationships and spatial information. Compared with previous deep learning-based synthesis, the correlation between different destination contrast is utilized to enhance reconstruction quality. To improve model generalizability and evaluate clinical significance, the proposed model was trained and tested on a large multi-center dataset, including healthy subjects and patients with pathology. Results from both quantitative comparison and clinical reader study demonstrate that the multi-task formulation leads to more efficient and accurate contrast synthesis than previous methods.

Data-driven self-calibration and reconstruction for non-cartesian wave-encoded single-shot fast spin echo using deep learning.

BACKGROUND: Current self-calibration and reconstruction methods for wave-encoded single-shot fast spin echo imaging (SSFSE) requires long computational time, especially when high accuracy is needed. PURPOSE: To develop and investigate the clinical feasibility of data-driven self-calibration and reconstruction of wave-encoded SSFSE imaging for computation time reduction and quality improvement. STUDY TYPE: Prospective controlled clinical trial. SUBJECTS: With Institutional Review Board approval, the proposed method was assessed on 29 consecutive adult patients (18 males, 11 females, range, 24-77 years). FIELD STRENGTH/SEQUENCE: A wave-encoded variable-density SSFSE sequence was developed for clinical 3.0T abdominal scans to enable 3.5× acceleration with full-Fourier acquisitions. Data-driven calibration of wave-encoding point-spread function (PSF) was developed using a trained deep neural network. Data-driven reconstruction was developed with another set of neural networks based on the calibrated wave-encoding PSF. Training of the calibration and reconstruction networks was performed on 15,783 2D wave-encoded SSFSE abdominal images. ASSESSMENT: Image quality of the proposed data-driven approach was compared independently and blindly with a conventional approach using iterative self-calibration and reconstruction with parallel imaging and compressed sensing by three radiologists on a scale from -2 to 2 for noise, contrast, sharpness, artifacts, and confidence. Computation time of these two approaches was also compared. STATISTICAL TESTS: Wilcoxon signed-rank tests were used to compare image quality and two-tailed t-tests were used to compare computation time with P values of under 0.05 considered statistically significant. RESULTS: An average 2.1-fold speedup in computation was achieved using the proposed method. The proposed data-driven self-calibration and reconstruction approach significantly reduced the perceived noise level (mean scores 0.82, P < 0.0001). DATA CONCLUSION: The proposed data-driven calibration and reconstruction achieved twice faster computation with reduced perceived noise, providing a fast and robust self-calibration and reconstruction for clinical abdominal SSFSE imaging. LEVEL OF EVIDENCE: 1 Technical Efficacy: Stage 1 J. Magn. Reson. Imaging 2020;51:841-853.