Improving MR image quality with a multi-task model, using convolutional losses.

PURPOSE: During the acquisition of MRI data, patient-, sequence-, or hardware-related factors can introduce artefacts that degrade image quality. Four of the most significant tasks for improving MRI image quality have been bias field correction, super-resolution, motion-, and noise correction. Machine learning has achieved outstanding results in improving MR image quality for these tasks individually, yet multi-task methods are rarely explored. METHODS: In this study, we developed a model to simultaneously correct for all four aforementioned artefacts using multi-task learning. Two different datasets were collected, one consisting of brain scans while the other pelvic scans, which were used to train separate models, implementing their corresponding artefact augmentations. Additionally, we explored a novel loss function that does not only aim to reconstruct the individual pixel values, but also the image gradients, to produce sharper, more realistic results. The difference between the evaluated methods was tested for significance using a Friedman test of equivalence followed by a Nemenyi post-hoc test. RESULTS: Our proposed model generally outperformed other commonly-used correction methods for individual artefacts, consistently achieving equal or superior results in at least one of the evaluation metrics. For images with multiple simultaneous artefacts, we show that the performance of using a combination of models, trained to correct individual artefacts depends heavily on the order that they were applied. This is not an issue for our proposed multi-task model. The model trained using our novel convolutional loss function always outperformed the model trained with a mean squared error loss, when evaluated using Visual Information Fidelity, a quality metric connected to perceptual quality. CONCLUSION: We trained two models for multi-task MRI artefact correction of brain, and pelvic scans. We used a novel loss function that significantly improves the image quality of the outputs over using mean squared error. The approach performs well on real world data, and it provides insight into which artefacts it detects and corrects for. Our proposed model and source code were made publicly available.

Region of interest focused MRI to synthetic CT translation using regression and segmentation multi-task network.

Objective. In MR-only clinical workflow, replacing CT with MR image is of advantage for workflow efficiency and reduces radiation to the patient. An important step required to eliminate CT scan from the workflow is to generate the information provided by CT via an MR image. In this work, we aim to demonstrate a method to generate accurate synthetic CT (sCT) from an MR image to suit the radiation therapy (RT) treatment planning workflow. We show the feasibility of the method and make way for a broader clinical evaluation.Approach. We present a machine learning method for sCT generation from zero-echo-time (ZTE) MRI aimed at structural and quantitative accuracies of the image, with a particular focus on the accurate bone density value prediction. The misestimation of bone density in the radiation path could lead to unintended dose delivery to the target volume and results in suboptimal treatment outcome. We propose a loss function that favors a spatially sparse bone region in the image. We harness the ability of the multi-task network to produce correlated outputs as a framework to enable localization of region of interest (RoI) via segmentation, emphasize regression of values within RoI and still retain the overall accuracy via global regression. The network is optimized by a composite loss function that combines a dedicated loss from each task.Main results. We have included 54 brain patient images in this study and tested the sCT images against reference CT on a subset of 20 cases. A pilot dose evaluation was performed on 9 of the 20 test cases to demonstrate the viability of the generated sCT in RT planning. The average quantitative metrics produced by the proposed method over the test set were-(a) mean absolute error (MAE) of 70 ± 8.6 HU; (b) peak signal-to-noise ratio (PSNR) of 29.4 ± 2.8 dB; structural similarity metric (SSIM) of 0.95 ± 0.02; and (d) Dice coefficient of the body region of 0.984 ± 0.Significance. We demonstrate that the proposed method generates sCT images that resemble visual characteristics of a real CT image and has a quantitative accuracy that suits RT dose planning application. We compare the dose calculation from the proposed sCT and the real CT in a radiation therapy treatment planning setup and show that sCT based planning falls within 0.5% target dose error. The method presented here with an initial dose evaluation makes an encouraging precursor to a broader clinical evaluation of sCT based RT planning on different anatomical regions.

Towards MR contrast independent synthetic CT generation.

The use of synthetic CT (sCT) in the radiotherapy workflow would reduce costs and scan time while removing the uncertainties around working with both MR and CT modalities. The performance of deep learning (DL) solutions for sCT generation is steadily increasing, however most proposed methods were trained and validated on private datasets of a single contrast from a single scanner. Such solutions might not perform equally well on other datasets, limiting their general usability and therefore value. Additionally, functional evaluations of sCTs such as dosimetric comparisons with CT-based dose calculations better show the impact of the methods, but the evaluations are more labor intensive than pixel-wise metrics. To improve the generalization of an sCT model, we propose to incorporate a pre-trained DL model to pre-process the input MR images by generating artificial proton density, T1 and T2 maps (i.e. contrast-independent quantitative maps), which are then used for sCT generation. Using a dataset of only T2w MR images, the robustness towards input MR contrasts of this approach is compared to a model that was trained using the MR images directly. We evaluate the generated sCTs using pixel-wise metrics and calculating mean radiological depths, as an approximation of the mean delivered dose. On T2w images acquired with the same settings as the training dataset, there was no significant difference between the performance of the models. However, when evaluated on T1w images, and a wide range of other contrasts and scanners from both public and private datasets, our approach outperforms the baseline model. Using a dataset of T2w MR images, our proposed model implements synthetic quantitative maps to generate sCT images, improving the generalization towards other contrasts. Our code and trained models are publicly available.

Bayesian non-linear regression with spatial priors for noise reduction and error estimation in quantitative MRI with an application in T1 estimation.

PURPOSE: To develop a method that can reduce and estimate uncertainty in quantitative MR parameter maps without the need for hand-tuning of any hyperparameters. METHODS: We present an estimation method where uncertainties are reduced by incorporating information on spatial correlations between neighbouring voxels. The method is based on a Bayesian hierarchical non-linear regression model, where the parameters of interest are sampled, using Markov chain Monte Carlo (MCMC), from a high-dimensional posterior distribution with a spatial prior. The degree to which the prior affects the model is determined by an automatic hyperparameter search using an information criterion and is, therefore, free from manual user-dependent tuning. The samples obtained further provide a convenient means to obtain uncertainties in both voxels and regions. The developed method was evaluated on T 1 estimations based on the variable flip angle method. RESULTS: The proposed method delivers noise-reduced T 1 parameter maps with associated error estimates by combining MCMC sampling, the widely applicable information criterion, and total variation-based denoising. The proposed method results in an overall decrease in estimation error when compared to conventional voxel-wise maximum likelihood estimation. However, this comes with an increased bias in some regions, predominately at tissue interfaces, as well as an increase in computational time. CONCLUSIONS: This study provides a method that generates more precise estimates compared to the conventional method, without incorporating user subjectivity, and with the added benefit of uncertainty estimation.

Latent space manipulation for high-resolution medical image synthesis via the StyleGAN.

INTRODUCTION: This paper explores the potential of the StyleGAN model as an high-resolution image generator for synthetic medical images. The possibility to generate sample patient images of different modalities can be helpful for training deep learning algorithms as e.g. a data augmentation technique. METHODS: The StyleGAN model was trained on Computed Tomography (CT) and T2- weighted Magnetic Resonance (MR) images from 100 patients with pelvic malignancies. The resulting model was investigated with regards to three features: Image Modality, Sex, and Longitudinal Slice Position. Further, the style transfer feature of the StyleGAN was used to move images between the modalities. The root-mean-squard error (RMSE) and the Mean Absolute Error (MAE) were used to quantify errors for MR and CT, respectively. RESULTS: We demonstrate how these features can be transformed by manipulating the latent style vectors, and attempt to quantify how the errors change as we move through the latent style space. The best results were achieved by using the style transfer feature of the StyleGAN (58.7 HU MAE for MR to CT and 0.339 RMSE for CT to MR). Slices below and above an initial central slice can be predicted with an error below 75 HU MAE and 0.3 RMSE within 4cm for CT and MR, respectively. DISCUSSION: The StyleGAN is a promising model to use for generating synthetic medical images for MR and CT modalities as well as for 3D volumes.

Dosimetric Impact of MRI Distortions: A Study on Head and Neck Cancers.

PURPOSE: To evaluate the effect of magnetic resonance (MR) imaging (MRI) geometric distortions on head and neck radiation therapy treatment planning (RTP) for an MRI-only RTP. We also assessed the potential benefits of patient-specific shimming to reduce the magnitude of MR distortions for a 3-T scanner. METHODS AND MATERIALS: Using an in-house Matlab algorithm, shimming within entire imaging volumes and user-defined regions of interest were simulated. We deformed 21 patient computed tomography (CT) images with MR distortion fields (gradient nonlinearity and patient-induced susceptibility effects) to create distorted CT (dCT) images using bandwidths of 122 and 488 Hz/mm at 3 T. Field parameters from volumetric modulated arc therapy plans initially optimized on dCT data sets were transferred to CT data to compute a new plan. Both plans were compared to determine the impact of distortions on dose distributions. RESULTS: Shimming across entire patient volumes decreased the percentage of voxels with distortions of more than 2 mm from 15.4% to 2.0%. Using the user-defined region of interest (ROI) shimming strategy, (here the Planning target volume (PTV) was the chosen ROI volume) led to increased geometric for volumes outside the PTV, as such voxels within the spinal cord with geometric shifts above 2 mm increased from 11.5% to 32.3%. The worst phantom-measured residual system distortions after 3-dimensional gradient nonlinearity correction within a radial distance of 200 mm from the isocenter was 2.17 mm. For all patients, voxels with distortion shifts of more than 2 mm resulting from patient-induced susceptibility effects were 15.4% and 0.0% using bandwidths of 122 Hz/mm and 488 Hz/mm at 3 T. Dose differences between dCT and CT treatment plans in D50 at the planning target volume were 0.4% ± 0.6% and 0.3% ± 0.5% at 122 and 488 Hz/mm, respectively. CONCLUSIONS: The overall effect of MRI geometric distortions on data used for RTP was minimal. Shimming over entire imaging volumes decreased distortions, but user-defined subvolume shimming introduced significant errors in nearby organs and should probably be avoided.

Zero TE-based pseudo-CT image conversion in the head and its application in PET/MR attenuation correction and MR-guided radiation therapy planning.

PURPOSE: To describe a method for converting Zero TE (ZTE) MR images into X-ray attenuation information in the form of pseudo-CT images and demonstrate its performance for (1) attenuation correction (AC) in PET/MR and (2) dose planning in MR-guided radiation therapy planning (RTP). METHODS: Proton density-weighted ZTE images were acquired as input for MR-based pseudo-CT conversion, providing (1) efficient capture of short-lived bone signals, (2) flat soft-tissue contrast, and (3) fast and robust 3D MR imaging. After bias correction and normalization, the images were segmented into bone, soft-tissue, and air by means of thresholding and morphological refinements. Fixed Hounsfield replacement values were assigned for air (-1000 HU) and soft-tissue (+42 HU), whereas continuous linear mapping was used for bone. RESULTS: The obtained ZTE-derived pseudo-CT images accurately resembled the true CT images (i.e., Dice coefficient for bone overlap of 0.73 ± 0.08 and mean absolute error of 123 ± 25 HU evaluated over the whole head, including errors from residual registration mismatches in the neck and mouth regions). The linear bone mapping accounted for bone density variations. Averaged across five patients, ZTE-based AC demonstrated a PET error of -0.04 ± 1.68% relative to CT-based AC. Similarly, for RTP assessed in eight patients, the absolute dose difference over the target volume was found to be 0.23 ± 0.42%. CONCLUSION: The described method enables MR to pseudo-CT image conversion for the head in an accurate, robust, and fast manner without relying on anatomical prior knowledge. Potential applications include PET/MR-AC, and MR-guided RTP.

Quantifying the Effect of 3T Magnetic Resonance Imaging Residual System Distortions and Patient-Induced Susceptibility Distortions on Radiation Therapy Treatment Planning for Prostate Cancer.

PURPOSE: To investigate the effect of magnetic resonance system- and patient-induced susceptibility distortions from a 3T scanner on dose distributions for prostate cancers. METHODS AND MATERIALS: Combined displacement fields from the residual system and patient-induced susceptibility distortions were used to distort 17 prostate patient CT images. VMAT dose plans were initially optimized on distorted CT images and the plan parameters transferred to the original patient CT images to calculate a new dose distribution. RESULTS: Maximum residual mean distortions of 3.19 mm at a radial distance of 25 cm and maximum mean patient-induced susceptibility shifts of 5.8 mm were found using the lowest bandwidth of 122 Hz per pixel. There was a dose difference of <0.5% between distorted and undistorted treatment plans. The 90% confidence intervals of the mean difference between the dCT and CT treatment plans were all within an equivalence interval of (-0.5, 0.5) for all investigated plan quality measures. CONCLUSIONS: Patient-induced susceptibility distortions at high field strengths in closed bore magnetic resonance scanners are larger than residual system distortions after using vendor-supplied 3-dimensional correction for the delineated regions studied. However, errors in dose due to disturbed patient outline and shifts caused by patient-induced susceptibility effects are below 0.5%.