Medical Transformer: Universal Encoder for 3-D Brain MRI Analysis.

Transfer learning has attracted considerable attention in medical image analysis because of the limited number of annotated 3-D medical datasets available for training data-driven deep learning models in the real world. We propose Medical Transformer, a novel transfer learning framework that effectively models 3-D volumetric images as a sequence of 2-D image slices. To improve the high-level representation in 3-D-form empowering spatial relations, we use a multiview approach that leverages information from three planes of the 3-D volume, while providing parameter-efficient training. For building a source model generally applicable to various tasks, we pretrain the model using self-supervised learning (SSL) for masked encoding vector prediction as a proxy task, using a large-scale normal, healthy brain magnetic resonance imaging (MRI) dataset. Our pretrained model is evaluated on three downstream tasks: 1) brain disease diagnosis; 2) brain age prediction; and 3) brain tumor segmentation, which are widely studied in brain MRI research. Experimental results demonstrate that our Medical Transformer outperforms the state-of-the-art (SOTA) transfer learning methods, efficiently reducing the number of parameters by up to approximately 92% for classification and regression tasks and 97% for segmentation task, and it also achieves good performance in scenarios where only partial training samples are used.

Prediction of Decompensation and Death in Advanced Chronic Liver Disease Using Deep Learning Analysis of Gadoxetic Acid-Enhanced MRI.

OBJECTIVE: This study aimed to evaluate the usefulness of quantitative indices obtained from deep learning analysis of gadoxetic acid-enhanced hepatobiliary phase (HBP) MRI and their longitudinal changes in predicting decompensation and death in patients with advanced chronic liver disease (ACLD). MATERIALS AND METHODS: We included patients who underwent baseline and 1-year follow-up MRI from a prospective cohort that underwent gadoxetic acid-enhanced MRI for hepatocellular carcinoma surveillance between November 2011 and August 2012 at a tertiary medical center. Baseline liver condition was categorized as non-ACLD, compensated ACLD, and decompensated ACLD. The liver-to-spleen signal intensity ratio (LS-SIR) and liver-to-spleen volume ratio (LS-VR) were automatically measured on the HBP images using a deep learning algorithm, and their percentage changes at the 1-year follow-up (ΔLS-SIR and ΔLS-VR) were calculated. The associations of the MRI indices with hepatic decompensation and a composite endpoint of liver-related death or transplantation were evaluated using a competing risk analysis with multivariable Fine and Gray regression models, including baseline parameters alone and both baseline and follow-up parameters. RESULTS: Our study included 280 patients (153 male; mean age ± standard deviation, 57 ± 7.95 years) with non-ACLD, compensated ACLD, and decompensated ACLD in 32, 186, and 62 patients, respectively. Patients were followed for 11-117 months (median, 104 months). In patients with compensated ACLD, baseline LS-SIR (sub-distribution hazard ratio [sHR], 0.81; p = 0.034) and LS-VR (sHR, 0.71; p = 0.01) were independently associated with hepatic decompensation. The ΔLS-VR (sHR, 0.54; p = 0.002) was predictive of hepatic decompensation after adjusting for baseline variables. ΔLS-VR was an independent predictor of liver-related death or transplantation in patients with compensated ACLD (sHR, 0.46; p = 0.026) and decompensated ACLD (sHR, 0.61; p = 0.023). CONCLUSION: MRI indices automatically derived from the deep learning analysis of gadoxetic acid-enhanced HBP MRI can be used as prognostic markers in patients with ACLD.

Deep Learning-Based Assessment of Functional Liver Capacity Using Gadoxetic Acid-Enhanced Hepatobiliary Phase MRI.

OBJECTIVE: We aimed to develop and test a deep learning algorithm (DLA) for fully automated measurement of the volume and signal intensity (SI) of the liver and spleen using gadoxetic acid-enhanced hepatobiliary phase (HBP)-magnetic resonance imaging (MRI) and to evaluate the clinical utility of DLA-assisted assessment of functional liver capacity. MATERIALS AND METHODS: The DLA was developed using HBP-MRI data from 1014 patients. Using an independent test dataset (110 internal and 90 external MRI data), the segmentation performance of the DLA was measured using the Dice similarity score (DSS), and the agreement between the DLA and the ground truth for the volume and SI measurements was assessed with a Bland-Altman 95% limit of agreement (LOA). In 276 separate patients (male:female, 191:85; mean age ± standard deviation, 40 ± 15 years) who underwent hepatic resection, we evaluated the correlations between various DLA-based MRI indices, including liver volume normalized by body surface area (LVBSA), liver-to-spleen SI ratio (LSSR), MRI parameter-adjusted LSSR (aLSSR), LSSR × LVBSA, and aLSSR × LVBSA, and the indocyanine green retention rate at 15 minutes (ICG-R15), and determined the diagnostic performance of the DLA-based MRI indices to detect ICG-R15 ≥ 20%. RESULTS: In the test dataset, the mean DSS was 0.977 for liver segmentation and 0.946 for spleen segmentation. The Bland-Altman 95% LOAs were 0.08% ± 3.70% for the liver volume, 0.20% ± 7.89% for the spleen volume, -0.02% ± 1.28% for the liver SI, and -0.01% ± 1.70% for the spleen SI. Among DLA-based MRI indices, aLSSR × LVBSA showed the strongest correlation with ICG-R15 (r = -0.54, p < 0.001), with area under receiver operating characteristic curve of 0.932 (95% confidence interval, 0.895-0.959) to diagnose ICG-R15 ≥ 20%. CONCLUSION: Our DLA can accurately measure the volume and SI of the liver and spleen and may be useful for assessing functional liver capacity using gadoxetic acid-enhanced HBP-MRI.

Multi-scale gradual integration CNN for false positive reduction in pulmonary nodule detection.

Lung cancer is a global and dangerous disease, and its early detection is crucial for reducing the risks of mortality. In this regard, it has been of great interest in developing a computer-aided system for pulmonary nodules detection as early as possible on thoracic CT scans. In general, a nodule detection system involves two steps: (i) candidate nodule detection at a high sensitivity, which captures many false positives and (ii) false positive reduction from candidates. However, due to the high variation of nodule morphological characteristics and the possibility of mistaking them for neighboring organs, candidate nodule detection remains a challenge. In this study, we propose a novel Multi-scale Gradual Integration Convolutional Neural Network (MGI-CNN), designed with three main strategies: (1) to use multi-scale inputs with different levels of contextual information, (2) to use abstract information inherent in different input scales with gradual integration, and (3) to learn multi-stream feature integration in an end-to-end manner. To verify the efficacy of the proposed network, we conducted exhaustive experiments on the LUNA16 challenge datasets by comparing the performance of the proposed method with state-of-the-art methods in the literature. On two candidate subsets of the LUNA16 dataset, i.e., V1 and V2, our method achieved an average CPM of 0.908 (V1) and 0.942 (V2), outperforming comparable methods by a large margin. Our MGI-CNN is implemented in Python using TensorFlow and the source code is available from https://github.com/ku-milab/MGICNN.