MoViT: Memorizing Vision Transformers for Medical Image Analysis.

The synergy of long-range dependencies from transformers and local representations of image content from convolutional neural networks (CNNs) has led to advanced architectures and increased performance for various medical image analysis tasks due to their complementary benefits. However, compared with CNNs, transformers require considerably more training data, due to a larger number of parameters and an absence of inductive bias. The need for increasingly large datasets continues to be problematic, particularly in the context of medical imaging, where both annotation efforts and data protection result in limited data availability. In this work, inspired by the human decision-making process of correlating new "evidence" with previously memorized "experience", we propose a Memorizing Vision Transformer (MoViT) to alleviate the need for large-scale datasets to successfully train and deploy transformer-based architectures. MoViT leverages an external memory structure to cache history attention snapshots during the training stage. To prevent overfitting, we incorporate an innovative memory update scheme, attention temporal moving average, to update the stored external memories with the historical moving average. For inference speedup, we design a prototypical attention learning method to distill the external memory into smaller representative subsets. We evaluate our method on a public histology image dataset and an in-house MRI dataset, demonstrating that MoViT applied to varied medical image analysis tasks, can outperform vanilla transformer models across varied data regimes, especially in cases where only a small amount of annotated data is available. More importantly, MoViT can reach a competitive performance of ViT with only 3.0% of the training data. In conclusion, MoViT provides a simple plug-in for transformer architectures which may contribute to reducing the training data needed to achieve acceptable models for a broad range of medical image analysis tasks.

CEST2022: Amide proton transfer-weighted MRI improves the diagnostic performance of multiparametric non-contrast-enhanced MRI techniques in patients with post-treatment high-grade gliomas.

New or enlarged lesions in malignant gliomas after surgery and chemoradiation can be associated with tumor recurrence or treatment effect. Due to similar radiographic characteristics, conventional-and even some advanced MRI techniques-are limited in distinguishing these two pathologies. Amide proton transfer-weighted (APTw) MRI, a protein-based molecular imaging technique that does not require the administration of any exogenous contrast agent, was recently introduced into the clinical setting. In this study, we evaluated and compared the diagnostic performances of APTw MRI with several non-contrast-enhanced MRI sequences, such as diffusion-weighted imaging, susceptibility-weighted imaging, and pseudo-continuous arterial spin labeling. Thirty-nine scans from 28 glioma patients were obtained on a 3 T MRI scanner. A histogram analysis approach was employed to extract parameters from each tumor area. Statistically significant parameters (P < 0.05) were selected to train multivariate logistic regression models to evaluate the performance of MRI sequences. Multiple histogram parameters, particularly from APTw and pseudo-continuous arterial spin labeling images, demonstrated significant differences between treatment effect and recurrent tumor. The regression model trained on the combination of all significant histogram parameters achieved the best result (area under the curve = 0.89). We found that APTw images added value to other advanced MR images for the differentiation of treatment effect and tumor recurrence.

Radiomics analysis of amide proton transfer-weighted and structural MR images for treatment response assessment in malignant gliomas.

The purpose of this study was to evaluate the value of amide proton transfer-weighted (APTw) MRI radiomic features for the differentiation of tumor recurrence from treatment effect in malignant gliomas. Eighty-six patients who had suspected tumor recurrence after completion of chemoradiation or radiotherapy, and who had APTw-MRI data acquired at 3 T, were retrospectively analyzed. Using a fluid-attenuated inversion recovery (FLAIR) image-based mask, radiomics analysis was applied to the processed APTw and structural MR images. A chi-square automatic interaction detector decision tree was used for classification analysis. Models with and without APTw features were built using the same strategy. Tenfold cross-validation was applied to obtain the overall classification performance of each model. Sixty patients were confirmed as having tumor recurrence, and the remainder were confirmed as having treatment effect, at median time points of 190 and 171 days after therapy, respectively. There were 525 radiomic features extracted from each of the processed APTw and structural MR images. Based on these, the APTw-based model yielded the highest accuracy (86.0%) for the differentiation of tumor recurrence from treatment effect, compared with 74.4%, 76.7%, 83.7%, and 76.7% for T1 w, T2 w, FLAIR, and Gd-T1 w, respectively. Model classification accuracy was 82.6% when using the combined structural MR images (T1 w, T2 w, FLAIR, Gd-T1 w), and increased to 89.5% when using these structural plus APTw images. The corresponding sensitivity and specificity were 85.0% and 76.9% for the combination of structural MR images, and 85.0% and 100% after adding APTw image features. Adding APTw-based radiomic features increased MRI accuracy in the assessment of the treatment response in post-treatment malignant gliomas.