Deep-learning based super-resolution for 3D isotropic coronary MR angiography in less than a minute.

PURPOSE: To develop and evaluate a novel and generalizable super-resolution (SR) deep-learning framework for motion-compensated isotropic 3D coronary MR angiography (CMRA), which allows free-breathing acquisitions in less than a minute. METHODS: Undersampled motion-corrected reconstructions have enabled free-breathing isotropic 3D CMRA in ~5-10 min acquisition times. In this work, we propose a deep-learning-based SR framework, combined with non-rigid respiratory motion compensation, to shorten the acquisition time to less than 1 min. A generative adversarial network (GAN) is proposed consisting of two cascaded Enhanced Deep Residual Network generator, a trainable discriminator, and a perceptual loss network. A 16-fold increase in spatial resolution is achieved by reconstructing a high-resolution (HR) isotropic CMRA (0.9 mm3 or 1.2 mm3 ) from a low-resolution (LR) anisotropic CMRA (0.9 × 3.6 × 3.6 mm3 or 1.2 × 4.8 × 4.8 mm3 ). The impact and generalization of the proposed SRGAN approach to different input resolutions and operation on image and patch-level is investigated. SRGAN was evaluated on a retrospective downsampled cohort of 50 patients and on 16 prospective patients that were scanned with LR-CMRA in ~50 s under free-breathing. Vessel sharpness and length of the coronary arteries from the SR-CMRA is compared against the HR-CMRA. RESULTS: SR-CMRA showed statistically significant (P < .001) improved vessel sharpness 34.1% ± 12.3% and length 41.5% ± 8.1% compared with LR-CMRA. Good generalization to input resolution and image/patch-level processing was found. SR-CMRA enabled recovery of coronary stenosis similar to HR-CMRA with comparable qualitative performance. CONCLUSION: The proposed SR-CMRA provides a 16-fold increase in spatial resolution with comparable image quality to HR-CMRA while reducing the predictable scan time to <1 min.

Comparison of the clinical performance of upper abdominal PET/DCE-MRI with and without concurrent respiratory motion correction (MoCo).

PURPOSE: To compare the clinical performance of upper abdominal PET/DCE-MRI with and without concurrent respiratory motion correction (MoCo). METHODS: MoCo PET/DCE-MRI of the upper abdomen was acquired in 44 consecutive oncologic patients and compared with non-MoCo PET/MRI. SUVmax and MTV of FDG-avid upper abdominal malignant lesions were assessed on MoCo and non-MoCo PET images. Image quality was compared between MoCo DCE-MRI and non-MoCo CE-MRI, and between fused MoCo PET/MRI and fused non-MoCo PET/MRI images. RESULTS: MoCo PET resulted in higher SUVmax (10.8 ± 5.45) than non-MoCo PET (9.62 ± 5.42) and lower MTV (35.55 ± 141.95 cm3) than non-MoCo PET (38.11 ± 198.14 cm3; p < 0.005 for both). The quality of MoCo DCE-MRI images (4.73 ± 0.5) was higher than that of non-MoCo CE-MRI images (4.53±0.71; p = 0.037). The quality of fused MoCo-PET/MRI images (4.96 ± 0.16) was higher than that of fused non-MoCo PET/MRI images (4.39 ± 0.66; p < 0.005). CONCLUSION: MoCo PET/MRI provided qualitatively better images than non-MoCo PET/MRI, and upper abdominal malignant lesions demonstrated higher SUVmax and lower MTV on MoCo PET/MRI.

Accelerating 3D MTC-BOOST in patients with congenital heart disease using a joint multi-scale variational neural network reconstruction.

PURPOSE: Free-breathing Magnetization Transfer Contrast Bright blOOd phase SensiTive (MTC-BOOST) is a prototype balanced-Steady-State Free Precession sequence for 3D whole-heart imaging, that employs the endogenous magnetisation transfer contrast mechanism. This achieves reduction of flow and off-resonance artefacts, that often arise with the clinical T2prepared balanced-Steady-State Free Precession sequence, enabling high quality, contrast-agent free imaging of the thoracic cardiovascular anatomy. Fully-sampled MTC-BOOST acquisition requires long scan times (~10-24 min) and therefore acceleration is needed to permit its clinical incorporation. The aim of this study is to enable and clinically validate the 5-fold accelerated MTC-BOOST acquisition with joint Multi-Scale Variational Neural Network (jMS-VNN) reconstruction. METHODS: Thirty-six patients underwent free-breathing, 3D whole-heart imaging with the MTC-BOOST sequence, which is combined with variable density spiral-like Cartesian sampling and 2D image navigators for translational motion estimation. This sequence acquires two differently weighted bright-blood volumes in an interleaved fashion, which are then joined in a phase sensitive inversion recovery reconstruction to obtain a complementary fully co-registered black-blood volume. Data from eighteen patients were used for training, whereas data from the remaining eighteen patients were used for testing/evaluation. The proposed deep-learning based approach adopts a supervised multi-scale variational neural network for joint reconstruction of the two differently weighted bright-blood volumes acquired with the 5-fold accelerated MTC-BOOST. The two contrast images are stacked as different channels in the network to exploit the shared information. The proposed approach is compared to the fully-sampled MTC-BOOST and 5-fold undersampled MTC-BOOST acquisition with Compressed Sensing (CS) reconstruction in terms of scan/reconstruction time and bright-blood image quality. Comparison against conventional 2-fold undersampled T2-prepared 3D bright-blood whole-heart clinical sequence (T2prep-3DWH) is also included. RESULTS: Acquisition time was 3.0 ±â€¯1.0 min for the 5-fold accelerated MTC-BOOST versus 9.0 ±â€¯1.1 min for the fully-sampled MTC-BOOST and 11.1 ±â€¯2.6 min for the T2prep-3DWH (p < 0.001 and p < 0.001, respectively). Reconstruction time was significantly lower with the jMS-VNN method compared to CS (10 ±â€¯0.5 min vs 20 ±â€¯2 s, p < 0.001). Image quality was higher for the proposed 5-fold undersampled jMS-VNN versus conventional CS, comparable or higher to the corresponding T2prep-3DWH dataset and similar to the fully-sampled MTC-BOOST. CONCLUSION: The proposed 5-fold accelerated jMS-VNN MTC-BOOST framework provides efficient 3D whole-heart bright-blood imaging in fast acquisition and reconstruction time with concomitant reduction of flow and off-resonance artefacts, that are frequently encountered with the clinical sequence. Image quality of the cardiac anatomy and thoracic vasculature is comparable or superior to the clinical scan and 5-fold CS reconstruction in faster reconstruction time, promising potential clinical adoption.

Non-Rigid Respiratory Motion Estimation of Whole-Heart Coronary MR Images Using Unsupervised Deep Learning.

Non-rigid motion-corrected reconstruction has been proposed to account for the complex motion of the heart in free-breathing 3D coronary magnetic resonance angiography (CMRA). This reconstruction framework requires efficient and accurate estimation of non-rigid motion fields from undersampled images at different respiratory positions (or bins). However, state-of-the-art registration methods can be time-consuming. This article presents a novel unsupervised deep learning-based strategy for fast estimation of inter-bin 3D non-rigid respiratory motion fields for motion-corrected free-breathing CMRA. The proposed 3D respiratory motion estimation network (RespME-net) is trained as a deep encoder-decoder network, taking pairs of 3D image patches extracted from CMRA volumes as input and outputting the motion field between image patches. Using image warping by the estimated motion field, a loss function that imposes image similarity and motion smoothness is adopted to enable training without ground truth motion field. RespME-net is trained patch-wise to circumvent the challenges of training a 3D network volume-wise which requires large amounts of GPU memory and 3D datasets. We perform 5-fold cross-validation with 45 CMRA datasets and demonstrate that RespME-net can predict 3D non-rigid motion fields with subpixel accuracy (0.44 ± 0.38 mm) within ~10 seconds, being ~20 times faster than a GPU-implemented state-of-the-art non-rigid registration method. Moreover, we perform non-rigid motion-compensated CMRA reconstruction for 9 additional patients. The proposed RespME-net has achieved similar motion-corrected CMRA image quality to the conventional registration method regarding coronary artery length and sharpness.

From Compressed-Sensing to Artificial Intelligence-Based Cardiac MRI Reconstruction.

Cardiac magnetic resonance (CMR) imaging is an important tool for the non-invasive assessment of cardiovascular disease. However, CMR suffers from long acquisition times due to the need of obtaining images with high temporal and spatial resolution, different contrasts, and/or whole-heart coverage. In addition, both cardiac and respiratory-induced motion of the heart during the acquisition need to be accounted for, further increasing the scan time. Several undersampling reconstruction techniques have been proposed during the last decades to speed up CMR acquisition. These techniques rely on acquiring less data than needed and estimating the non-acquired data exploiting some sort of prior information. Parallel imaging and compressed sensing undersampling reconstruction techniques have revolutionized the field, enabling 2- to 3-fold scan time accelerations to become standard in clinical practice. Recent scientific advances in CMR reconstruction hinge on the thriving field of artificial intelligence. Machine learning reconstruction approaches have been recently proposed to learn the non-linear optimization process employed in CMR reconstruction. Unlike analytical methods for which the reconstruction problem is explicitly defined into the optimization process, machine learning techniques make use of large data sets to learn the key reconstruction parameters and priors. In particular, deep learning techniques promise to use deep neural networks (DNN) to learn the reconstruction process from existing datasets in advance, providing a fast and efficient reconstruction that can be applied to all newly acquired data. However, before machine learning and DNN can realize their full potentials and enter widespread clinical routine for CMR image reconstruction, there are several technical hurdles that need to be addressed. In this article, we provide an overview of the recent developments in the area of artificial intelligence for CMR image reconstruction. The underlying assumptions of established techniques such as compressed sensing and low-rank reconstruction are briefly summarized, while a greater focus is given to recent advances in dictionary learning and deep learning based CMR reconstruction. In particular, approaches that exploit neural networks as implicit or explicit priors are discussed for 2D dynamic cardiac imaging and 3D whole-heart CMR imaging. Current limitations, challenges, and potential future directions of these techniques are also discussed.

A multi-scale variational neural network for accelerating motion-compensated whole-heart 3D coronary MR angiography.

PURPOSE: To enable fast reconstruction of undersampled motion-compensated whole-heart 3D coronary magnetic resonance angiography (CMRA) by learning a multi-scale variational neural network (MS-VNN) which allows the acquisition of high-quality 1.2 × 1.2 × 1.2 mm isotropic volumes in a short and predictable scan time. METHODS: Eighteen healthy subjects and one patient underwent free-breathing 3D CMRA acquisition with variable density spiral-like Cartesian sampling, combined with 2D image navigators for translational motion estimation/compensation. The proposed MS-VNN learns two sets of kernels and activation functions for the magnitude and phase images of the complex-valued data. For the magnitude, a multi-scale approach is applied to better capture the small calibre of the coronaries. Ten subjects were considered for training and validation. Prospectively undersampled motion-compensated data with 5-fold and 9-fold accelerations, from the remaining 9 subjects, were used to evaluate the framework. The proposed approach was compared to Wavelet-based compressed-sensing (CS), conventional VNN, and to an additional fully-sampled (FS) scan. RESULTS: The average acquisition time (m:s) was 4:11 for 5-fold, 2:34 for 9-fold acceleration and 18:55 for fully-sampled. Reconstruction time with the proposed MS-VNN was ~14 s. The proposed MS-VNN achieves higher image quality than CS and VNN reconstructions, with quantitative right coronary artery sharpness (CS:43.0%, VNN:43.9%, MS-VNN:47.0%, FS:50.67%) and vessel length (CS:7.4 cm, VNN:7.7 cm, MS-VNN:8.8 cm, FS:9.1 cm) comparable to the FS scan. CONCLUSION: The proposed MS-VNN enables 5-fold and 9-fold undersampled CMRA acquisitions with comparable image quality that the corresponding fully-sampled scan. The proposed framework achieves extremely fast reconstruction time and does not require tuning of regularization parameters, offering easy integration into clinical workflow.

CINENet: deep learning-based 3D cardiac CINE MRI reconstruction with multi-coil complex-valued 4D spatio-temporal convolutions.

Cardiac CINE magnetic resonance imaging is the gold-standard for the assessment of cardiac function. Imaging accelerations have shown to enable 3D CINE with left ventricular (LV) coverage in a single breath-hold. However, 3D imaging remains limited to anisotropic resolution and long reconstruction times. Recently deep learning has shown promising results for computationally efficient reconstructions of highly accelerated 2D CINE imaging. In this work, we propose a novel 4D (3D + time) deep learning-based reconstruction network, termed 4D CINENet, for prospectively undersampled 3D Cartesian CINE imaging. CINENet is based on (3 + 1)D complex-valued spatio-temporal convolutions and multi-coil data processing. We trained and evaluated the proposed CINENet on in-house acquired 3D CINE data of 20 healthy subjects and 15 patients with suspected cardiovascular disease. The proposed CINENet network outperforms iterative reconstructions in visual image quality and contrast (+ 67% improvement). We found good agreement in LV function (bias ± 95% confidence) in terms of end-systolic volume (0 ± 3.3 ml), end-diastolic volume (- 0.4 ± 2.0 ml) and ejection fraction (0.1 ± 3.2%) compared to clinical gold-standard 2D CINE, enabling single breath-hold isotropic 3D CINE in less than 10 s scan and ~ 5 s reconstruction time.

Concurrent Respiratory Motion Correction of Abdominal PET and Dynamic Contrast-Enhanced-MRI Using a Compressed Sensing Approach.

We present an approach for concurrent reconstruction of respiratory motion-compensated abdominal dynamic contrast-enhanced (DCE)-MRI and PET data in an integrated PET/MR scanner. The MR and PET reconstructions share the same motion vector fields derived from radial MR data; the approach is robust to changes in respiratory pattern and does not increase the total acquisition time. Methods: PET and DCE-MRI data of 12 oncologic patients were simultaneously acquired for 6 min on an integrated PET/MR system after administration of 18F-FDG and gadoterate meglumine. Golden-angle radial MR data were continuously acquired simultaneously with PET data and sorted into multiple motion phases on the basis of a respiratory signal derived directly from the radial MR data. The resulting multidimensional dataset was reconstructed using a compressed sensing approach that exploits sparsity among respiratory phases. Motion vector fields obtained using the full 6-min (MC6-min) and only the last 1 min (MC1-min) of data were incorporated into the PET reconstruction to obtain motion-corrected PET images and in an MR iterative reconstruction algorithm to produce a series of motion-corrected DCE-MR images (moco_GRASP). The motion-correction methods (MC6-min and MC1-min) were evaluated by qualitative analysis of the MR images and quantitative analysis of SUVmax and SUVmean, contrast, signal-to-noise ratio (SNR), and lesion volume in the PET images. Results: Motion-corrected MC6-min PET images demonstrated 30%, 23%, 34%, and 18% increases in average SUVmax, SUVmean, contrast, and SNR and an average 40% reduction in lesion volume with respect to the non-motion-corrected PET images. The changes in these figures of merit were smaller but still substantial for the MC1-min protocol: 19%, 10%, 15%, and 9% increases in average SUVmax, SUVmean, contrast, and SNR; and a 28% reduction in lesion volume. Moco_GRASP images were deemed of acceptable or better diagnostic image quality with respect to conventional breath-hold Cartesian volumetric interpolated breath-hold examination acquisitions. Conclusion: We presented a method that allows the simultaneous acquisition of respiratory motion-corrected diagnostic quality DCE-MRI and quantitatively accurate PET data in an integrated PET/MR scanner with negligible prolongation in acquisition time compared with routine PET/DCE-MRI protocols.

PET/MRI in the Presence of Metal Implants: Completion of the Attenuation Map from PET Emission Data.

We present a novel technique for accurate whole-body attenuation correction in the presence of metallic endoprosthesis, on integrated non-time-of-flight (non-TOF) PET/MRI scanners. The proposed implant PET-based attenuation map completion (IPAC) method performs a joint reconstruction of radioactivity and attenuation from the emission data to determine the position, shape, and linear attenuation coefficient (LAC) of metallic implants. Methods: The initial estimate of the attenuation map was obtained using the MR Dixon method currently available on the Siemens Biograph mMR scanner. The attenuation coefficients in the area of the MR image subjected to metal susceptibility artifacts are then reconstructed from the PET emission data using the IPAC algorithm. The method was tested on 11 subjects presenting 13 different metallic implants, who underwent CT and PET/MR scans. Relative mean LACs and Dice similarity coefficients were calculated to determine the accuracy of the reconstructed attenuation values and the shape of the metal implant, respectively. The reconstructed PET images were compared with those obtained using the reference CT-based approach and the Dixon-based method. Absolute relative change (aRC) images were generated in each case, and voxel-based analyses were performed. Results: The error in implant LAC estimation, using the proposed IPAC algorithm, was 15.7% ± 7.8%, which was significantly smaller than the Dixon- (100%) and CT- (39%) derived values. A mean Dice similarity coefficient of 73% ± 9% was obtained when comparing the IPAC- with the CT-derived implant shape. The voxel-based analysis of the reconstructed PET images revealed quantification errors (aRC) of 13.2% ± 22.1% for the IPAC- with respect to CT-corrected images. The Dixon-based method performed substantially worse, with a mean aRC of 23.1% ± 38.4%. Conclusion: We have presented a non-TOF emission-based approach for estimating the attenuation map in the presence of metallic implants, to be used for whole-body attenuation correction in integrated PET/MR scanners. The Graphics Processing Unit implementation of the algorithm will be included in the open-source reconstruction toolbox Occiput.io.

Staging performance of whole-body DWI, PET/CT and PET/MRI in invasive ductal carcinoma of the breast.

The aim of the present study was to evaluate the performance of whole-body diffusion-weighted imaging (WB-DWI), whole-body positron emission tomography with computed tomography (WB-PET/CT), and whole-body positron emission tomography with magnetic resonance imaging (WB-PET/MRI) in staging patients with untreated invasive ductal carcinoma of the breast. Fifty-one women with newly diagnosed invasive ductal carcinoma of the breast underwent WB-DWI, WB-PET/CT and WB-PET/MRI before treatment. A radiologist and a nuclear medicine physician reviewed in consensus the images from the three modalities and searched for occurrence, number and location of metastases. Final staging, according to each technique, was compared. Pathology and imaging follow-up were used as the reference. WB-DWI, WB-PET/CT and WB-PET/MRI correctly and concordantly staged 33/51 patients: stage IIA in 7 patients, stage IIB in 8 patients, stage IIIC in 4 patients and stage IV in 14 patients. WB-DWI, WB-PET/CT and WB-PET/MRI incorrectly and concordantly staged 1/51 patient as stage IV instead of IIIA. Discordant staging was reported in 17/51 patients. WB-PET/MRI resulted in improved staging when compared to WB-PET/CT (50 correctly staged on WB-PET/MRI vs. 38 correctly staged on WB-PET/CT; McNemar's test; p<0.01). Comparing the performance of WB-PET/MRI and WB-DWI (43 correct) did not reveal a statistically significant difference (McNemar test, p=0.14). WB-PET/MRI is more accurate in the initial staging of breast cancer than WB-DWI and WB-PET/CT, however, the discrepancies between WB-PET/MRI and WB-DWI were not statistically significant. When available, WB-PET/MRI should be considered for staging patient with invasive ductal breast carcinoma.

Transmission imaging for integrated PET-MR systems.

Attenuation correction for PET-MR systems continues to be a challenging problem, particularly for body regions outside the head. The simultaneous acquisition of transmission scan based µ-maps and MR images on integrated PET-MR systems may significantly increase the performance of and offer validation for new MR-based µ-map algorithms. For the Biograph mMR (Siemens Healthcare), however, use of conventional transmission schemes is not practical as the patient table and relatively small diameter scanner bore significantly restrict radioactive source motion and limit source placement. We propose a method for emission-free coincidence transmission imaging on the Biograph mMR. The intended application is not for routine subject imaging, but rather to improve and validate MR-based µ-map algorithms; particularly for patient implant and scanner hardware attenuation correction. In this study we optimized source geometry and assessed the method's performance with Monte Carlo simulations and phantom scans. We utilized a Bayesian reconstruction algorithm, which directly generates µ-map estimates from multiple bed positions, combined with a robust scatter correction method. For simulations with a pelvis phantom a single torus produced peak noise equivalent count rates (34.8 kcps) dramatically larger than a full axial length ring (11.32 kcps) and conventional rotating source configurations. Bias in reconstructed µ-maps for head and pelvis simulations was ⩽4% for soft tissue and ⩽11% for bone ROIs. An implementation of the single torus source was filled with (18)F-fluorodeoxyglucose and the proposed method quantified for several test cases alone or in comparison with CT-derived µ-maps. A volume average of 0.095 cm(-1) was recorded for an experimental uniform cylinder phantom scan, while a bias of <2% was measured for the cortical bone equivalent insert of the multi-compartment phantom. Single torus µ-maps of a hip implant phantom showed significantly less artifacts and improved dynamic range, and differed greatly for highly attenuating materials in the case of the patient table, compared to CT results. Use of a fixed torus geometry, in combination with translation of the patient table to perform complete tomographic sampling, generated highly quantitative measured µ-maps and is expected to produce images with significantly higher SNR than competing fixed geometries at matched total acquisition time.

Efficient determination of the uncertainty for the optimization of SPECT system design: a subsampled fisher information matrix.

System designs in single photon emission tomography (SPECT) can be evaluated based on the fundamental trade-off between bias and variance that can be achieved in the reconstruction of emission tomograms. This trade off can be derived analytically using the Cramer-Rao type bounds, which imply the calculation and the inversion of the Fisher information matrix (FIM). The inverse of the FIM expresses the uncertainty associated to the tomogram, enabling the comparison of system designs. However, computing, storing and inverting the FIM is not practical with 3-D imaging systems. In order to tackle the problem of the computational load in calculating the inverse of the FIM, a method based on the calculation of the local impulse response and the variance, in a single point, from a single row of the FIM, has been previously proposed for system design. However this approximation (circulant approximation) does not capture the global interdependence between the variables in shift-variant systems such as SPECT, and cannot account e.g., for data truncation or missing data. Our new formulation relies on subsampling the FIM. The FIM is calculated over a subset of voxels arranged in a grid that covers the whole volume. Every element of the FIM at the grid points is calculated exactly, accounting for the acquisition geometry and for the object. This new formulation reduces the computational complexity in estimating the uncertainty, but nevertheless accounts for the global interdependence between the variables, enabling the exploration of design spaces hindered by the circulant approximation. The graphics processing unit accelerated implementation of the algorithm reduces further the computation times, making the algorithm a good candidate for real-time optimization of adaptive imaging systems. This paper describes the subsampled FIM formulation and implementation details. The advantages and limitations of the new approximation are explored, in comparison with the circulant approximation, in the context of design optimization of a parallel-hole collimator SPECT system and of an adaptive imaging system (similar to the commercially available D-SPECT).