Zero-DeepSub: Zero-shot deep subspace reconstruction for rapid multiparametric quantitative MRI using 3D-QALAS.

PURPOSE: To develop and evaluate methods for (1) reconstructing 3D-quantification using an interleaved Look-Locker acquisition sequence with T2 preparation pulse (3D-QALAS) time-series images using a low-rank subspace method, which enables accurate and rapid T1 and T2 mapping, and (2) improving the fidelity of subspace QALAS by combining scan-specific deep-learning-based reconstruction and subspace modeling. THEORY AND METHODS: A low-rank subspace method for 3D-QALAS (i.e., subspace QALAS) and zero-shot deep-learning subspace method (i.e., Zero-DeepSub) were proposed for rapid and high fidelity T1 and T2 mapping and time-resolved imaging using 3D-QALAS. Using an ISMRM/NIST system phantom, the accuracy and reproducibility of the T1 and T2 maps estimated using the proposed methods were evaluated by comparing them with reference techniques. The reconstruction performance of the proposed subspace QALAS using Zero-DeepSub was evaluated in vivo and compared with conventional QALAS at high reduction factors of up to nine-fold. RESULTS: Phantom experiments showed that subspace QALAS had good linearity with respect to the reference methods while reducing biases and improving precision compared to conventional QALAS, especially for T2 maps. Moreover, in vivo results demonstrated that subspace QALAS had better g-factor maps and could reduce voxel blurring, noise, and artifacts compared to conventional QALAS and showed robust performance at up to nine-fold acceleration with Zero-DeepSub, which enabled whole-brain T1, T2, and PD mapping at 1 mm isotropic resolution within 2 min of scan time. CONCLUSION: The proposed subspace QALAS along with Zero-DeepSub enabled high fidelity and rapid whole-brain multiparametric quantification and time-resolved imaging.

SSL-QALAS: Self-Supervised Learning for rapid multiparameter estimation in quantitative MRI using 3D-QALAS.

PURPOSE: To develop and evaluate a method for rapid estimation of multiparametric T1 , T2 , proton density, and inversion efficiency maps from 3D-quantification using an interleaved Look-Locker acquisition sequence with T2 preparation pulse (3D-QALAS) measurements using self-supervised learning (SSL) without the need for an external dictionary. METHODS: An SSL-based QALAS mapping method (SSL-QALAS) was developed for rapid and dictionary-free estimation of multiparametric maps from 3D-QALAS measurements. The accuracy of the reconstructed quantitative maps using dictionary matching and SSL-QALAS was evaluated by comparing the estimated T1 and T2 values with those obtained from the reference methods on an International Society for Magnetic Resonance in Medicine/National Institute of Standards and Technology phantom. The SSL-QALAS and the dictionary-matching methods were also compared in vivo, and generalizability was evaluated by comparing the scan-specific, pre-trained, and transfer learning models. RESULTS: Phantom experiments showed that both the dictionary-matching and SSL-QALAS methods produced T1 and T2 estimates that had a strong linear agreement with the reference values in the International Society for Magnetic Resonance in Medicine/National Institute of Standards and Technology phantom. Further, SSL-QALAS showed similar performance with dictionary matching in reconstructing the T1 , T2 , proton density, and inversion efficiency maps on in vivo data. Rapid reconstruction of multiparametric maps was enabled by inferring the data using a pre-trained SSL-QALAS model within 10 s. Fast scan-specific tuning was also demonstrated by fine-tuning the pre-trained model with the target subject's data within 15 min. CONCLUSION: The proposed SSL-QALAS method enabled rapid reconstruction of multiparametric maps from 3D-QALAS measurements without an external dictionary or labeled ground-truth training data.

Deep learning referral suggestion and tumour discrimination using explainable artificial intelligence applied to multiparametric MRI.

OBJECTIVES: An appropriate and fast clinical referral suggestion is important for intra-axial mass-like lesions (IMLLs) in the emergency setting. We aimed to apply an interpretable deep learning (DL) system to multiparametric MRI to obtain clinical referral suggestion for IMLLs, and to validate it in the setting of nontraumatic emergency neuroradiology. METHODS: A DL system was developed in 747 patients with IMLLs ranging 30 diseases who underwent pre- and post-contrast T1-weighted (T1CE), FLAIR, and diffusion-weighted imaging (DWI). A DL system that segments IMLLs, classifies tumourous conditions, and suggests clinical referral among surgery, systematic work-up, medical treatment, and conservative treatment, was developed. The system was validated in an independent cohort of 130 emergency patients, and performance in referral suggestion and tumour discrimination was compared with that of radiologists using receiver operating characteristics curve, precision-recall curve analysis, and confusion matrices. Multiparametric interpretable visualisation of high-relevance regions from layer-wise relevance propagation overlaid on contrast-enhanced T1WI and DWI was analysed. RESULTS: The DL system provided correct referral suggestions in 94 of 130 patients (72.3%) and performed comparably to radiologists (accuracy 72.6%, McNemar test; p = .942). For distinguishing tumours from non-tumourous conditions, the DL system (AUC, 0.90 and AUPRC, 0.94) performed similarly to human readers (AUC, 0.81~0.92, and AUPRC, 0.88~0.95). Solid portions of tumours showed a high overlap of relevance, but non-tumours did not (Dice coefficient 0.77 vs. 0.33, p < .001), demonstrating the DL's decision. CONCLUSIONS: Our DL system could appropriately triage patients using multiparametric MRI and provide interpretability through multiparametric heatmaps, and may thereby aid neuroradiologic diagnoses in emergency settings. CLINICAL RELEVANCE STATEMENT: Our AI triages patients with raw MRI images to clinical referral pathways in brain intra-axial mass-like lesions. We demonstrate that the decision is based on the relative relevance between contrast-enhanced T1-weighted and diffusion-weighted images, providing explainability across multiparametric MRI data. KEY POINTS: • A deep learning (DL) system using multiparametric MRI suggested clinical referral to patients with intra-axial mass-like lesions (IMLLs) similar to radiologists (accuracy 72.3% vs. 72.6%). • In the differentiation of tumourous and non-tumourous conditions, the DL system (AUC, 0.90) performed similar with radiologists (AUC, 0.81-0.92). • The DL's decision basis for differentiating tumours from non-tumours can be quantified using multiparametric heatmaps obtained via the layer-wise relevance propagation method.

Intelligent noninvasive meningioma grading with a fully automatic segmentation using interpretable multiparametric deep learning.

OBJECTIVES: To establish a robust interpretable multiparametric deep learning (DL) model for automatic noninvasive grading of meningiomas along with segmentation. METHODS: In total, 257 patients with pathologically confirmed meningiomas (162 low-grade, 95 high-grade) who underwent a preoperative brain MRI, including T2-weighted (T2) and contrast-enhanced T1-weighted images (T1C), were included in the institutional training set. A two-stage DL grading model was constructed for segmentation and classification based on multiparametric three-dimensional U-net and ResNet. The models were validated in the external validation set consisting of 61 patients with meningiomas (46 low-grade, 15 high-grade). Relevance-weighted Class Activation Mapping (RCAM) method was used to interpret the DL features contributing to the prediction of the DL grading model. RESULTS: On external validation, the combined T1C and T2 model showed a Dice coefficient of 0.910 in segmentation and the highest performance for meningioma grading compared to the T2 or T1C only models, with an area under the curve (AUC) of 0.770 (95% confidence interval: 0.644-0.895) and accuracy, sensitivity, and specificity of 72.1%, 73.3%, and 71.7%, respectively. The AUC and accuracy of the combined DL grading model were higher than those of the human readers (AUCs of 0.675-0.690 and accuracies of 65.6-68.9%, respectively). The RCAM of the DL grading model showed activated maps at the surface regions of meningiomas indicating that the model recognized the features at the tumor margin for grading. CONCLUSIONS: An interpretable multiparametric DL model combining T1C and T2 can enable fully automatic grading of meningiomas along with segmentation. KEY POINTS: • The multiparametric DL model showed robustness in grading and segmentation on external validation. • The diagnostic performance of the combined DL grading model was higher than that of the human readers. • The RCAM interpreted that DL grading model recognized the meaningful features at the tumor margin for grading.

Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces.

A wearable silent speech interface (SSI) is a promising platform that enables verbal communication without vocalization. The most widely studied methodology for SSI focuses on surface electromyography (sEMG). However, sEMG suffers from low scalability because of signal quality-related issues, including signal-to-noise ratio and interelectrode interference. Hence, here, we present a novel SSI by utilizing crystalline-silicon-based strain sensors combined with a 3D convolutional deep learning algorithm. Two perpendicularly placed strain gauges with minimized cell dimension (<0.1 mm2) could effectively capture the biaxial strain information with high reliability. We attached four strain sensors near the subject's mouths and collected strain data of unprecedently large wordsets (100 words), which our SSI can classify at a high accuracy rate (87.53%). Several analysis methods were demonstrated to verify the system's reliability, as well as the performance comparison with another SSI using sEMG electrodes with the same dimension, which exhibited a relatively low accuracy rate (42.60%).

Results of the 2020 fastMRI Challenge for Machine Learning MR Image Reconstruction.

Accelerating MRI scans is one of the principal outstanding problems in the MRI research community. Towards this goal, we hosted the second fastMRI competition targeted towards reconstructing MR images with subsampled k-space data. We provided participants with data from 7,299 clinical brain scans (de-identified via a HIPAA-compliant procedure by NYU Langone Health), holding back the fully-sampled data from 894 of these scans for challenge evaluation purposes. In contrast to the 2019 challenge, we focused our radiologist evaluations on pathological assessment in brain images. We also debuted a new Transfer track that required participants to submit models evaluated on MRI scanners from outside the training set. We received 19 submissions from eight different groups. Results showed one team scoring best in both SSIM scores and qualitative radiologist evaluations. We also performed analysis on alternative metrics to mitigate the effects of background noise and collected feedback from the participants to inform future challenges. Lastly, we identify common failure modes across the submissions, highlighting areas of need for future research in the MRI reconstruction community.

Deep model-based magnetic resonance parameter mapping network (DOPAMINE) for fast T1 mapping using variable flip angle method.

Quantitative tissue characteristics, which provide valuable diagnostic information, can be represented by magnetic resonance (MR) parameter maps using magnetic resonance imaging (MRI); however, a long scan time is necessary to acquire them, which prevents the application of quantitative MR parameter mapping to real clinical protocols. For fast MR parameter mapping, we propose a deep model-based MR parameter mapping network called DOPAMINE that combines a deep learning network with a model-based method to reconstruct MR parameter maps from undersampled multi-channel k-space data. DOPAMINE consists of two networks: 1) an MR parameter mapping network that uses a deep convolutional neural network (CNN) that estimates initial parameter maps from undersampled k-space data (CNN-based mapping), and 2) a reconstruction network that removes aliasing artifacts in the parameter maps with a deep CNN (CNN-based reconstruction) and an interleaved data consistency layer by an embedded MR model-based optimization procedure. We demonstrated the performance of DOPAMINE in brain T1 map reconstruction with a variable flip angle (VFA) model. To evaluate the performance of DOPAMINE, we compared it with conventional parallel imaging, low-rank based reconstruction, model-based reconstruction, and state-of-the-art deep-learning-based mapping methods for three different reduction factors (R = 3, 5, and 7) and two different sampling patterns (1D Cartesian and 2D Poisson-disk). Quantitative metrics indicated that DOPAMINE outperformed other methods in reconstructing T1 maps for all sampling patterns and reduction factors. DOPAMINE exhibited quantitatively and qualitatively superior performance to that of conventional methods in reconstructing MR parameter maps from undersampled multi-channel k-space data. The proposed method can thus reduce the scan time of quantitative MR parameter mapping that uses a VFA model.

Robust performance of deep learning for automatic detection and segmentation of brain metastases using three-dimensional black-blood and three-dimensional gradient echo imaging.

OBJECTIVES: To evaluate whether a deep learning (DL) model using both three-dimensional (3D) black-blood (BB) imaging and 3D gradient echo (GRE) imaging may improve the detection and segmentation performance of brain metastases compared to that using only 3D GRE imaging. METHODS: A total of 188 patients with brain metastases (917 lesions) who underwent a brain metastasis MRI protocol including contrast-enhanced 3D BB and 3D GRE were included in the training set. DL models based on 3D U-net were constructed. The models were validated in the test set consisting of 45 patients with brain metastases (203 lesions) and 49 patients without brain metastases. RESULTS: The combined 3D BB and 3D GRE model yielded better performance than the 3D GRE model (sensitivities of 93.1% vs 76.8%, p < 0.001), and this effect was significantly stronger in subgroups with small metastases (p interaction < 0.001). For metastases < 3 mm, ≥ 3 mm and < 10 mm, and ≥ 10 mm, the sensitivities were 82.4%, 93.2%, and 100%, respectively. The combined 3D BB and 3D GRE model showed a false-positive per case of 0.59 in the test set. The combined 3D BB and 3D GRE model showed a Dice coefficient of 0.822, while 3D GRE model showed a lower Dice coefficient of 0.756. CONCLUSIONS: The combined 3D BB and 3D GRE DL model may improve the detection and segmentation performance of brain metastases, especially in detecting small metastases. KEY POINTS: • The combined 3D BB and 3D GRE model yielded better performance for the detection of brain metastases than the 3D GRE model (p < 0.001), with sensitivities of 93.1% and 76.8%, respectively. • The combined 3D BB and 3D GRE model showed a false-positive rate per case of 0.59 in the test set. • The combined 3D BB and 3D GRE model showed a Dice coefficient of 0.822, while the 3D GRE model showed a lower Dice coefficient of 0.756.

Accelerating Cartesian MRI by domain-transform manifold learning in phase-encoding direction.

This study developed a domain-transform framework comprising domain-transform manifold learning with an initial analytic transform to accelerate Cartesian magnetic resonance imaging (DOTA-MRI). The proposed method directly transforms undersampled Cartesian k-space data into a reconstructed image. In Cartesian undersampling, the k-space is fully or zero sampled in the data-acquisition direction (i.e., the frequency-encoding direction or the x-direction); one-dimensional (1D) inverse Fourier transform (IFT) along the x-direction on the undersampled k-space does not induce any aliasing. To exploit this, the algorithm first applies an analytic x-direction 1D IFT to the undersampled Cartesian k-space input, and subsequently transforms it into a reconstructed image using deep neural networks. The initial analytic transform (i.e., 1D IFT) allows the fully connected layers of the neural network to learn 1D global transform only in the phase-encoding direction (i.e., the y-direction) instead of 2D transform. This drastically reduces the number of parameters to be learned from O(N2) to O(N) compared with the existing manifold learning algorithm (i.e., automated transform by manifold approximation) (AUTOMAP). This enables DOTA-MRI to be applied to high-resolution MR datasets, which has previously proved difficult to implement in AUTOMAP because of the enormous memory requirements involved. After the initial analytic transform, the manifold learning phase uses a symmetric network architecture comprising three types of layers: front-end convolutional layers, fully connected layers for the 1D global transform, and back-end convolutional layers. The front-end convolutional layers take 1D IFT of the undersampled k-space (i.e., undersampled data in the intermediate domain or in the ky-x domain) as input and performs data-domain restoration. The following fully connected layers learn the 1D global transform between the ky-x domain and the image domain (i.e., the y-x domain). Finally, the back-end convolutional layers reconstruct the final image by denoising in the image domain. DOTA-MRI exhibited superior performance over nine other existing algorithms, including state-of-the-art deep learning-based algorithms. The generality of the algorithm was demonstrated by experiments conducted under various sampling ratios, datasets, and noise levels.

[The Latest Trends in Attention Mechanisms and Their Application in Medical Imaging]. / ì–´í ì ˜ 기법 및 의료 영상에의 ì ìš©ì— 관한 ìµœì‹ ë™í–¥.

Deep learning has recently achieved remarkable results in the field of medical imaging. However, as a deep learning network becomes deeper to improve its performance, it becomes more difficult to interpret the processes within. This can especially be a critical problem in medical fields where diagnostic decisions are directly related to a patient's survival. In order to solve this, explainable artificial intelligence techniques are being widely studied, and an attention mechanism was developed as part of this approach. In this paper, attention techniques are divided into two types: post hoc attention, which aims to analyze a network that has already been trained, and trainable attention, which further improves network performance. Detailed comparisons of each method, examples of applications in medical imaging, and future perspectives will be covered.

Megahertz-wave-transmitting conducting polymer electrode for device-to-device integration.

The ideal combination of high optical transparency and high electrical conductivity, especially at very low frequencies of less than the gigahertz (GHz) order, such as the radiofrequencies at which electronic devices operate (tens of kHz to hundreds of GHz), is fundamental incompatibility, which creates a barrier to the realization of enhanced user interfaces and 'device-to-device integration.' Herein, we present a design strategy for preparing a megahertz (MHz)-transparent conductor, based on a plasma frequency controlled by the electrical conductivity, with the ultimate goal of device-to-device integration through electromagnetic wave transmittance. This approach is verified experimentally using a conducting polymer, poly(3,4-ethylenedioxythiophene)-poly(styrenesulfonate) (PEDOT:PSS), the microstructure of which is manipulated by employing a solution process. The use of a transparent conducting polymer as an electrode enables the fabrication of a fully functional touch-controlled display device and magnetic resonance imaging (MRI)-compatible biomedical monitoring device, which would open up a new paradigm for transparent conductors.

Parallel imaging in time-of-flight magnetic resonance angiography using deep multistream convolutional neural networks.

PURPOSE: To develop and evaluate a method of parallel imaging time-of-flight (TOF) MRA using deep multistream convolutional neural networks (CNNs). METHODS: A deep parallel imaging network ("DPI-net") was developed to reconstruct 3D multichannel MRA from undersampled data. It comprises 2 deep-learning networks: a network of multistream CNNs for extracting feature maps of multichannel images and a network of reconstruction CNNs for reconstructing images from the multistream network output feature maps. The images were evaluated using normalized root mean square error (NRMSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) values, and the visibility of blood vessels was assessed by measuring the vessel sharpness of middle and posterior cerebral arteries on axial maximum intensity projection (MIP) images. Vessel sharpness was compared using paired t tests, between DPI-net, 2 conventional parallel imaging methods (SAKE and ESPIRiT), and a deep-learning method (U-net). RESULTS: DPI-net showed superior performance in reconstructing vessel signals in both axial slices and MIP images for all reduction factors. This was supported by the quantitative metrics, with DPI-net showing the lowest NRMSE, the highest PSNR and SSIM (except R = 3.8 on sagittal MIP images, and R = 5.7 on axial slices and sagittal MIP images), and significantly higher vessel sharpness values than the other methods. CONCLUSION: DPI-net was effective in reconstructing 3D TOF MRA from highly undersampled multichannel MR data, achieving superior performance, both quantitatively and qualitatively, over conventional parallel imaging and other deep-learning methods.

Deep-learned 3D black-blood imaging using automatic labelling technique and 3D convolutional neural networks for detecting metastatic brain tumors.

Black-blood (BB) imaging is used to complement contrast-enhanced 3D gradient-echo (CE 3D-GRE) imaging for detecting brain metastases, requiring additional scan time. In this study, we proposed deep-learned 3D BB imaging with an auto-labelling technique and 3D convolutional neural networks for brain metastases detection without additional BB scan. Patients were randomly selected for training (29 sets) and testing (36 sets). Two neuroradiologists independently evaluated deep-learned and original BB images, assessing the degree of blood vessel suppression and lesion conspicuity. Vessel signals were effectively suppressed in all patients. The figure of merits, which indicate the diagnostic performance of radiologists, were 0.9708 with deep-learned BB and 0.9437 with original BB imaging, suggesting that the deep-learned BB imaging is highly comparable to the original BB imaging (difference was not significant; p = 0.2142). In per patient analysis, sensitivities were 100% for both deep-learned and original BB imaging; however, the original BB imaging indicated false positive results for two patients. In per lesion analysis, sensitivities were 90.3% for deep-learned and 100% for original BB images. There were eight false positive lesions on the original BB imaging but only one on the deep-learned BB imaging. Deep-learned 3D BB imaging can be effective for brain metastases detection.

KIKI-net: cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images.

PURPOSE: To demonstrate accurate MR image reconstruction from undersampled k-space data using cross-domain convolutional neural networks (CNNs) METHODS: Cross-domain CNNs consist of 3 components: (1) a deep CNN operating on the k-space (KCNN), (2) a deep CNN operating on an image domain (ICNN), and (3) an interleaved data consistency operations. These components are alternately applied, and each CNN is trained to minimize the loss between the reconstructed and corresponding fully sampled k-spaces. The final reconstructed image is obtained by forward-propagating the undersampled k-space data through the entire network. RESULTS: Performances of K-net (KCNN with inverse Fourier transform), I-net (ICNN with interleaved data consistency), and various combinations of the 2 different networks were tested. The test results indicated that K-net and I-net have different advantages/disadvantages in terms of tissue-structure restoration. Consequently, the combination of K-net and I-net is superior to single-domain CNNs. Three MR data sets, the T2 fluid-attenuated inversion recovery (T2 FLAIR) set from the Alzheimer's Disease Neuroimaging Initiative and 2 data sets acquired at our local institute (T2 FLAIR and T1 weighted), were used to evaluate the performance of 7 conventional reconstruction algorithms and the proposed cross-domain CNNs, which hereafter is referred to as KIKI-net. KIKI-net outperforms conventional algorithms with mean improvements of 2.29 dB in peak SNR and 0.031 in structure similarity. CONCLUSION: KIKI-net exhibits superior performance over state-of-the-art conventional algorithms in terms of restoring tissue structures and removing aliasing artifacts. The results demonstrate that KIKI-net is applicable up to a reduction factor of 3 to 4 based on variable-density Cartesian undersampling.

High-SNR multiple T2 (*)-contrast magnetic resonance imaging using a robust denoising method based on tissue characteristics.

PURPOSE: To develop an effective method that can suppress noise in successive multiecho T2 (*)-weighted magnetic resonance (MR) brain images while preventing filtering artifacts. MATERIALS AND METHODS: For the simulation experiments, we used multiple T2 -weighted images of an anatomical brain phantom. For in vivo experiments, successive multiecho MR brain images were acquired from five healthy subjects using a multiecho gradient-recalled-echo (MGRE) sequence with a 3T MRI system. Our denoising method is a nonlinear filter whose filtering weights are determined by tissue characteristics among pixels. The similarity of the tissue characteristics is measured based on the l2 -difference between two temporal decay signals. Both numerical and subjective evaluations were performed in order to compare the effectiveness of our denoising method with those of conventional filters, including Gaussian low-pass filter (LPF), anisotropic diffusion filter (ADF), and bilateral filter. Root-mean-square error (RMSE), signal-to-noise ratio (SNR), and contrast-to-noise ratio (CNR) were used in the numerical evaluation. Five observers, including one radiologist, assessed the image quality and rated subjective scores in the subjective evaluation. RESULTS: Our denoising method significantly improves RMSE, SNR, and CNR of numerical phantom images, and CNR of in vivo brain images in comparison with conventional filters (P < 0.005). It also receives the highest scores for structure conspicuity (8.2 to 9.4 out of 10) and naturalness (9.2 to 9.8 out of 10) among the conventional filters in the subjective evaluation. CONCLUSION: This study demonstrates that high-SNR multiple T2 (*)-contrast MR images can be obtained using our denoising method based on tissue characteristics without noticeable artifacts. Evidence level: 2 J. MAGN. RESON. IMAGING 2017;45:1835-1845.