Accelerated MRI reconstructions via variational network and feature domain learning.

We introduce three architecture modifications to enhance the performance of the end-to-end (E2E) variational network (VarNet) for undersampled MRI reconstructions. We first implemented the Feature VarNet, which propagates information throughout the cascades of the network in an N-channel feature-space instead of a 2-channel feature-space. Then, we add an attention layer that utilizes the spatial locations of Cartesian undersampling artifacts to further improve performance. Lastly, we combined the Feature and E2E VarNets into the Feature-Image (FI) VarNet, to facilitate cross-domain learning and boost accuracy. Reconstructions were evaluated on the fastMRI dataset using standard metrics and clinical scoring by three neuroradiologists. Feature and FI VarNets outperformed the E2E VarNet for 4 × , 5 × and 8 × Cartesian undersampling in all studied metrics. FI VarNet secured second place in the public fastMRI leaderboard for 4 × Cartesian undersampling, outperforming all open-source models in the leaderboard. Radiologists rated FI VarNet brain reconstructions with higher quality and sharpness than the E2E VarNet reconstructions. FI VarNet excelled in preserving anatomical details, including blood vessels, whereas E2E VarNet discarded or blurred them in some cases. The proposed FI VarNet enhances the reconstruction quality of undersampled MRI and could enable clinically acceptable reconstructions at higher acceleration factors than currently possible.

Deep Learning Reconstruction Enables Prospectively Accelerated Clinical Knee MRI.

Background MRI is a powerful diagnostic tool with a long acquisition time. Recently, deep learning (DL) methods have provided accelerated high-quality image reconstructions from undersampled data, but it is unclear if DL image reconstruction can be reliably translated to everyday clinical practice. Purpose To determine the diagnostic equivalence of prospectively accelerated DL-reconstructed knee MRI compared with conventional accelerated MRI for evaluating internal derangement of the knee in a clinical setting. Materials and Methods A DL reconstruction model was trained with images from 298 clinical 3-T knee examinations. In a prospective analysis, patients clinically referred for knee MRI underwent a conventional accelerated knee MRI protocol at 3 T followed by an accelerated DL protocol between January 2020 and February 2021. The equivalence of the DL reconstruction of the images relative to the conventional images for the detection of an abnormality was assessed in terms of interchangeability. Each examination was reviewed by six musculoskeletal radiologists. Analyses pertaining to the detection of meniscal or ligament tears and bone marrow or cartilage abnormalities were based on four-point ordinal scores for the likelihood of an abnormality. Additionally, the protocols were compared with use of four-point ordinal scores for each aspect of image quality: overall image quality, presence of artifacts, sharpness, and signal-to-noise ratio. Results A total of 170 participants (mean age ± SD, 45 years ± 16; 76 men) were evaluated. The DL-reconstructed images were determined to be of diagnostic equivalence with the conventional images for detection of abnormalities. The overall image quality score, averaged over six readers, was significantly better (P < .001) for the DL than for the conventional images. Conclusion In a clinical setting, deep learning reconstruction enabled a nearly twofold reduction in scan time for a knee MRI and was diagnostically equivalent with the conventional protocol. © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Roemer in this issue.

Exploring the Acceleration Limits of Deep Learning Variational Network-based Two-dimensional Brain MRI.

Purpose: To explore the limits of deep learning-based brain MRI reconstruction and identify useful acceleration ranges for general-purpose imaging and potential screening. Materials and Methods: In this retrospective study conducted from 2019 through 2021, a model was trained for reconstruction on 5847 brain MR images. Performance was evaluated across a wide range of accelerations (up to 100-fold along a single phase-encoded direction for two-dimensional [2D] sections) on the fastMRI test set collected at New York University, consisting of 558 image volumes. In a sample of 69 volumes, reconstructions were classified by radiologists for identification of two clinical thresholds: (a) general-purpose diagnostic imaging and (b) potential use in a screening protocol. A Monte Carlo procedure was developed to estimate reconstruction error with only undersampled data. The model was evaluated on both in-domain and out-of-domain data. The 95% CIs were calculated using the percentile bootstrap method. Results: Radiologists rated 100% of 69 volumes as having sufficient image quality for general-purpose imaging at up to 4× acceleration and 65 of 69 volumes (94%) as having sufficient image quality for screening at up to 14× acceleration. The Monte Carlo procedure estimated ground truth peak signal-to-noise ratio and mean squared error with coefficients of determination greater than 0.5 at 2× to 20× acceleration levels. Out-of-distribution experiments demonstrated the model's ability to produce images substantially distinct from the training set, even at 100× acceleration. Conclusion: For 2D brain images using deep learning-based reconstruction, maximum acceleration for potential screening was three to four times higher than that for diagnostic general-purpose imaging.Keywords: MRI Reconstruction, High Acceleration, Deep Learning, Screening, Out of Distribution Supplemental material is available for this article. © RSNA, 2022.

Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction.

Accelerating MRI scans is one of the principal outstanding problems in the MRI research community. Towards this goal, we hosted the second fastMRI competition targeted towards reconstructing MR images with subsampled k-space data. We provided participants with data from 7,299 clinical brain scans (de-identified via a HIPAA-compliant procedure by NYU Langone Health), holding back the fully-sampled data from 894 of these scans for challenge evaluation purposes. In contrast to the 2019 challenge, we focused our radiologist evaluations on pathological assessment in brain images. We also debuted a new Transfer track that required participants to submit models evaluated on MRI scanners from outside the training set. We received 19 submissions from eight different groups. Results showed one team scoring best in both SSIM scores and qualitative radiologist evaluations. We also performed analysis on alternative metrics to mitigate the effects of background noise and collected feedback from the participants to inform future challenges. Lastly, we identify common failure modes across the submissions, highlighting areas of need for future research in the MRI reconstruction community.

COVID-19 Prognosis via Self-Supervised Representation Learning and Multi-Image Prediction.

The rapid spread of COVID-19 cases in recent months has strained hospital resources, making rapid and accurate triage of patients presenting to emergency departments a necessity. Machine learning techniques using clinical data such as chest X-rays have been used to predict which patients are most at risk of deterioration. We consider the task of predicting two types of patient deterioration based on chest X-rays: adverse event deterioration (i.e., transfer to the intensive care unit, intubation, or mortality) and increased oxygen requirements beyond 6 L per day. Due to the relative scarcity of COVID-19 patient data, existing solutions leverage supervised pretraining on related non-COVID images, but this is limited by the differences between the pretraining data and the target COVID-19 patient data. In this paper, we use self-supervised learning based on the momentum contrast (MoCo) method in the pretraining phase to learn more general image representations to use for downstream tasks. We present three results. The first is deterioration prediction from a single image, where our model achieves an area under receiver operating characteristic curve (AUC) of 0.742 for predicting an adverse event within 96 hours (compared to 0.703 with supervised pretraining) and an AUC of 0.765 for predicting oxygen requirements greater than 6 L a day at 24 hours (compared to 0.749 with supervised pretraining). We then propose a new transformer-based architecture that can process sequences of multiple images for prediction and show that this model can achieve an improved AUC of 0.786 for predicting an adverse event at 96 hours and an AUC of 0.848 for predicting mortalities at 96 hours. A small pilot clinical study suggested that the prediction accuracy of our model is comparable to that of experienced radiologists analyzing the same information.

Training a neural network for Gibbs and noise removal in diffusion MRI.

PURPOSE: To develop and evaluate a neural network-based method for Gibbs artifact and noise removal. METHODS: A convolutional neural network (CNN) was designed for artifact removal in diffusion-weighted imaging data. Two implementations were considered: one for magnitude images and one for complex images. Both models were based on the same encoder-decoder structure and were trained by simulating MRI acquisitions on synthetic non-MRI images. RESULTS: Both machine learning methods were able to mitigate artifacts in diffusion-weighted images and diffusion parameter maps. The CNN for complex images was also able to reduce artifacts in partial Fourier acquisitions. CONCLUSIONS: The proposed CNNs extend the ability of artifact correction in diffusion MRI. The machine learning method described here can be applied on each imaging slice independently, allowing it to be used flexibly in clinical applications.

Using Deep Learning to Accelerate Knee MRI at 3 T: Results of an Interchangeability Study.

OBJECTIVE. Deep learning (DL) image reconstruction has the potential to disrupt the current state of MRI by significantly decreasing the time required for MRI examinations. Our goal was to use DL to accelerate MRI to allow a 5-minute comprehensive examination of the knee without compromising image quality or diagnostic accuracy. MATERIALS AND METHODS. A DL model for image reconstruction using a variational network was optimized. The model was trained using dedicated multisequence training, in which a single reconstruction model was trained with data from multiple sequences with different contrast and orientations. After training, data from 108 patients were retrospectively undersampled in a manner that would correspond with a net 3.49-fold acceleration of fully sampled data acquisition and a 1.88-fold acceleration compared with our standard twofold accelerated parallel acquisition. An interchangeability study was performed, in which the ability of six readers to detect internal derangement of the knee was compared for clinical and DL-accelerated images. RESULTS. We found a high degree of interchangeability between standard and DL-accelerated images. In particular, results showed that interchanging the sequences would produce discordant clinical opinions no more than 4% of the time for any feature evaluated. Moreover, the accelerated sequence was judged by all six readers to have better quality than the clinical sequence. CONCLUSION. An optimized DL model allowed acceleration of knee images that performed interchangeably with standard images for detection of internal derangement of the knee. Importantly, readers preferred the quality of accelerated images to that of standard clinical images.

Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge.

PURPOSE: To advance research in the field of machine learning for MR image reconstruction with an open challenge. METHODS: We provided participants with a dataset of raw k-space data from 1,594 consecutive clinical exams of the knee. The goal of the challenge was to reconstruct images from these data. In order to strike a balance between realistic data and a shallow learning curve for those not already familiar with MR image reconstruction, we ran multiple tracks for multi-coil and single-coil data. We performed a two-stage evaluation based on quantitative image metrics followed by evaluation by a panel of radiologists. The challenge ran from June to December of 2019. RESULTS: We received a total of 33 challenge submissions. All participants chose to submit results from supervised machine learning approaches. CONCLUSIONS: The challenge led to new developments in machine learning for image reconstruction, provided insight into the current state of the art in the field, and highlighted remaining hurdles for clinical adoption.

fastMRI: A Publicly Available Raw k-Space and DICOM Dataset of Knee Images for Accelerated MR Image Reconstruction Using Machine Learning.

A publicly available dataset containing k-space data as well as Digital Imaging and Communications in Medicine image data of knee images for accelerated MR image reconstruction using machine learning is presented.

Image reconstruction for interrupted-beam x-ray CT on diagnostic clinical scanners.

Low-dose x-ray CT is a major research area with high clinical impact. Compressed sensing using view-based sparse sampling and sparsity-promoting regularization has shown promise in simulations, but these methods can be difficult to implement on diagnostic clinical CT scanners since the x-ray beam cannot be switched on and off rapidly enough. An alternative to view-based sparse sampling is interrupted-beam sparse sampling. SparseCT is a recently-proposed interrupted-beam scheme that achieves sparse sampling by blocking a portion of the beam using a multislit collimator (MSC). The use of an MSC necessitates a number of modifications to the standard compressed sensing reconstruction pipeline. In particular, we find that SparseCT reconstruction is feasible within a model-based image reconstruction framework that incorporates data fidelity weighting to consider penumbra effects and source jittering to consider the effect of partial source obstruction. Here, we present these modifications and demonstrate their application in simulations and real-world prototype scans. In simulations compared to conventional low-dose acquisitions, SparseCT is able to achieve smaller normalized root-mean square differences and higher structural similarity measures on two reduction factors. In prototype experiments, we successfully apply our reconstruction modifications and maintain image resolution at quarter-dose reduction level. The SparseCT design requires only small hardware modifications to current diagnostic clinical scanners, opening up new possibilities for CT dose reduction.

SparseCT: System concept and design of multislit collimators.

PURPOSE: SparseCT, an undersampling scheme for compressed sensing (CS) computed tomography (CT), has been proposed to reduce radiation dose by acquiring undersampled projection data from clinical CT scanners (Koesters et al. in, SparseCT: Interrupted-Beam Acquisition and Sparse Reconstruction for Radiation Dose Reduction; 2017). SparseCT partially blocks the x-ray beam with a multislit collimator (MSC) to perform a multidimensional undersampling along the view and detector row dimensions. SparseCT undersamples the projection data within each view and moves the MSC along the z-direction during gantry rotation to change the undersampling pattern. It enables reconstruction of images from undersampled data using CS algorithms. The purpose of this work is to design the spacing and width of the MSC slits and the MSC motion patterns based on beam separation, undersampling efficiency, and image quality. The development and testing of a SparseCT prototype with the designed MSC will be described in a following paper. METHODS: We chose a few initial MSC designs based on the guidance from two metrics: beam separation and undersampling efficiency. Both beam separation and undersampling efficiency were measured from numerically simulated photon distribution with MSC taken into consideration. Beam separation measures the separation between x-ray beams from consecutive slits, taking into account penumbra effects on both sides of each slit. Undersampling efficiency measures the dose-weighted similarity between penumbra undersampling and binary undersampling, in other words, the effective contribution of the incident dose to the signal to noise ratio of the projection data. We then compared the initially chosen MSC designs in terms of their reconstruction image quality. SparseCT projections were simulated from fully sampled patient projection data according to the MSC design and motion pattern, reconstructed iteratively using a sparsity-enforcing penalized weighted least squares cost function with ordered subsets/momentum algorithm, and compared visually and quantitatively. RESULTS: Simulated photon distributions indicate that the size of the penumbra is dominated by the size of the focal spot. Therefore, a wider MSC slit and a smaller focal spot lead to increased beam separation and undersampling efficiency. For fourfold undersampling with a 1.2 mm focal spot, a minimum MSC slit width of three detector rows (projected to the detector surface) is needed for beam separation; for threefold undersampling, a minimum slit width of four detector rows is needed. Simulations of SparseCT projection and reconstruction indicate that the motion pattern of the MSC does not have a visible impact on image quality. An MSC slit width of three or four detector rows yields similar image quality. CONCLUSION: The MSC is the key component of the SparseCT method. Simulations of MSC designs incorporating x-ray beam penumbra effects showed that for threefold and fourfold dose reductions, an MSC slit width of four detector rows provided reasonable beam separation, undersampling efficiency, and image quality.

Pancreas deformation in the presence of tumors using feature tracking from free-breathing XD-GRASP MRI.

BACKGROUND: Quantifying the biomechanical properties of pancreatic tumors could potentially help with assessment of tumor aggressiveness, prognosis, and prediction of therapy response. PURPOSE: To quantify respiratory-induced deformation in the pancreas and pancreatic lesions using XD-GRASP (eXtra-Dimensional Golden-angle RAdial Sparse Parallel), MRI. STUDY TYPE: Retrospective study where patients undergoing clinically indicated abdominal MRI which included free-breathing radial T1 -weighted (T1 W) imaging were studied. SUBJECTS: Thirty-two patients (12 male and 20 female) including nine with pancreatic lesions constituted our study cohort. FIELD STRENGTH/SEQUENCE: 3.0 T with T1 WI contrast-enhanced gradient echo radial free-breathing acquisition. ASSESSMENT: Using the XD-GRASP imaging technique, the acquired free-breathing radial data were sorted and binned into 10 consecutive respiratory motion states that were jointly reconstructed. 3D deformation fields along the respiratory dimension were computed using an optical flow method and were analyzed in the pancreas. STATISTICAL TESTS: The Wilcoxon signed-rank test was used to assess the difference in average displacement across pancreatic regions, while the Wilcoxon rank-sum test was used for displacement differences between patients with and without tumors. The interclass correlation coefficient (ICC) was computed to assess consistency between observers for each image quality measure. RESULTS: There was a significantly larger displacement in the pancreatic tail compared with the head (8.2 ± 3.7 mm > 5.8 ± 2.4 mm; P < 0.001) and body regions (8.2 ± 3.7 mm > 6.6 ± 2.9 mm; P < 0.001). Furthermore, there was reduced normalized average displacement in patients with pancreatic lesions compared with subjects without lesions (0.33 ± 0.1 < 0.69 ± 0.26, P < 0.001 for the head; 0.30 ± 0.1 < 0.84 ± 0.31, P < 0.001 for the body; and 0.44 ± 0.31 < 1.08 ± 0.53, P < 0.001 for the tail, respectively). DATA CONCLUSION: Free-breathing respiratory motion-sorted XD-GRASP MRI has the potential to noninvasively characterize the biomechanical properties of the pancreas by quantifying breathing-induced mechanical displacement. LEVEL OF EVIDENCE: 4 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2019;50:1633-1640.

Estrogen depletion and drug treatment alter the microstructure of type I collagen in bone.

The impact of estrogen depletion and drug treatment on type I collagen fibril nanomorphology and collagen fibril packing (microstructure) was evaluated by atomic force microscopy (AFM) using an ovariectomized (OVX) rabbit model of estrogen deficiency induced bone loss. Nine month-old New Zealand white female rabbits were treated as follows: sham-operated (Sham; n = 11), OVX + vehicle (OVX + Veh; n = 12), OVX + alendronate (ALN, 600 µg/kg/wk., s.c.; n = 12), and OVX + cathepsin-K inhibitor L-235 (CatKI, 10 mg/kg, daily, p.o.; n = 13) in prevention mode for 27 weeks. Samples from the cortical femur and trabecular lumbar vertebrae were polished, demineralized, and imaged using AFM. Auto-correlation of image patches was used to generate a vector field for each image that mathematically approximated the collagen fibril alignment. This vector field was used to compute an information-theoretic entropy that was employed as a quantitative fibril alignment parameter (FAP) to allow image-to-image and sample-to-sample comparison. For all samples, no change was observed in the average FAP values; however significant differences in the distribution of FAP values were observed. In particular, OVX + Veh lumbar vertebrae samples contained a tail of lower FAP values representing regions of greater fibril alignment. OVX + ALN treatment resulted in a FAP distribution with a tail indicating greater alignment for cortical femur and less alignment for trabecular lumbar vertebrae. OVX + CatKI treatment gave a distribution of FAP values with a tail indicating less alignment for cortical femur and no change for trabecular lumbar vertebrae. Fibril alignment was also evaluated by considering when a fibril was part of discrete bundles or sheets (classified as parallel) or not (classified as oblique). For this analysis, the percentage of parallel fibrils in cortical femur for the OVX group was 17% lower than the Sham group. OVX + ALN treatment partially prevented the proportion of parallel fibrils from decreasing and OVX + CatKI treatment completely prevented a change. In trabecular lumbar vertebrae, there was no difference in the percentage of parallel fibrils between Sham and any of the other treatment groups.

Fast parallel MR image reconstruction via B1-based, adaptive restart, iterative soft thresholding algorithms (BARISTA).

Sparsity-promoting regularization is useful for combining compressed sensing assumptions with parallel MRI for reducing scan time while preserving image quality. Variable splitting algorithms are the current state-of-the-art algorithms for SENSE-type MR image reconstruction with sparsity-promoting regularization. These methods are very general and have been observed to work with almost any regularizer; however, the tuning of associated convergence parameters is a commonly-cited hindrance in their adoption. Conversely, majorize-minimize algorithms based on a single Lipschitz constant have been observed to be slow in shift-variant applications such as SENSE-type MR image reconstruction since the associated Lipschitz constants are loose bounds for the shift-variant behavior. This paper bridges the gap between the Lipschitz constant and the shift-variant aspects of SENSE-type MR imaging by introducing majorizing matrices in the range of the regularizer matrix. The proposed majorize-minimize methods (called BARISTA) converge faster than state-of-the-art variable splitting algorithms when combined with momentum acceleration and adaptive momentum restarting. Furthermore, the tuning parameters associated with the proposed methods are unitless convergence tolerances that are easier to choose than the constraint penalty parameters required by variable splitting algorithms.

Spectroscopic visualization of nanoscale deformation in bone: interaction of light with partially disordered nanostructure.

Given that bone is an intriguing nanostructured dielectric as a partially disordered complex structure, we apply an elastic light scattering-based approach to image prefailure deformation and damage of bovine cortical bone under mechanical testing. We demonstrate that our imaging method can capture nanoscale deformation in a relatively large area. The unique structure, the high anisotropic property of bone, and the system configuration further allow us to use the transfer matrix method to study possible spectroscopic manifestations of prefailure deformation. Our sensitive yet simple imaging method could potentially be used to detect nanoscale structural and mechanical alterations of hard tissue and biomaterials in a fairly large field of view.