Autoencoder and restricted Boltzmann machine for transfer learning in functional magnetic resonance imaging task classification.

Deep neural networks (DNNs) have been adopted widely as classifiers for functional magnetic resonance imaging (fMRI) data, advancing beyond traditional machine learning models. Consequently, transfer learning of the pre-trained DNN becomes crucial to enhance DNN classification performance, specifically by alleviating an overfitting issue that occurs when a substantial number of DNN parameters are fitted to a relatively small number of fMRI samples. In this study, we first systematically compared the two most popularly used, unsupervised pretraining models for resting-state fMRI (rfMRI) volume data to pre-train the DNNs, namely autoencoder (AE) and restricted Boltzmann machine (RBM). The group in-brain mask used when training AE and RBM displayed a sizable overlap ratio with Yeo's seven functional brain networks (FNs). The parcellated FNs obtained from the RBM were fine-grained compared to those from the AE. The pre-trained AE and RBM served as the weight parameters of the first of the two hidden DNN layers, and the DNN fulfilled the task classifier role for fMRI (tfMRI) data in the Human Connectome Project (HCP). We tested two transfer learning schemes: (1) fixing and (2) fine-tuning the DNN's pre-trained AE or RBM weights. The DNN with transfer learning was compared to a baseline DNN, trained using random initial weights. Overall, DNN classification performance from the transfer learning proved superior when the pre-trained RBM weights were fixed and when the pre-trained AE weights were fine-tuned (average error rates: 14.8% for fixed RBM, 15.1% fine-tuned AE, and 15.5% for the baseline model) compared to the alternative scenarios of DNN transfer learning schemes. Moreover, the optimal transfer learning scheme between the fixed RBM and fine-tuned AE varied according to seven task conditions in the HCP. Nonetheless, the computational load reduced substantially for the fixed-weight-based transfer learning compared to the fine-tuning-based transfer learning (e.g., the number of weight parameters for the fixed-weight-based DNN model reduced to 1.9% compared with a baseline/fine-tuned DNN model). Our findings suggest that weight initialization at the DNN's first layer using RBM-based pre-trained weights provides the most promising approach when the whole-brain fMRI volume supports associated task classification. We believe that our proposed scheme could be applied to a variety of task conditions to improve their classification performance and to utilize computational resources efficiently using our AE/RBM-based pre-trained weights compared to random initial weights for DNN training.

Neural representations of the perception of handwritten digits and visual objects from a convolutional neural network compared to humans.

We investigated neural representations for visual perception of 10 handwritten digits and six visual objects from a convolutional neural network (CNN) and humans using functional magnetic resonance imaging (fMRI). Once our CNN model was fine-tuned using a pre-trained VGG16 model to recognize the visual stimuli from the digit and object categories, representational similarity analysis (RSA) was conducted using neural activations from fMRI and feature representations from the CNN model across all 16 classes. The encoded neural representation of the CNN model exhibited the hierarchical topography mapping of the human visual system. The feature representations in the lower convolutional (Conv) layers showed greater similarity with the neural representations in the early visual areas and parietal cortices, including the posterior cingulate cortex. The feature representations in the higher Conv layers were encoded in the higher-order visual areas, including the ventral/medial/dorsal stream and middle temporal complex. The neural representations in the classification layers were observed mainly in the ventral stream visual cortex (including the inferior temporal cortex), superior parietal cortex, and prefrontal cortex. There was a surprising similarity between the neural representations from the CNN model and the neural representations for human visual perception in the context of the perception of digits versus objects, particularly in the primary visual and associated areas. This study also illustrates the uniqueness of human visual perception. Unlike the CNN model, the neural representation of digits and objects for humans is more widely distributed across the whole brain, including the frontal and temporal areas.

Mixed-effects multilevel analysis followed by canonical correlation analysis is an effective fMRI tool for the investigation of idiosyncrasies.

We report that regions-of-interest (ROIs) associated with idiosyncratic individual behavior can be identified from functional magnetic resonance imaging (fMRI) data using statistical approaches that explicitly model individual variability in neuronal activations, such as mixed-effects multilevel analysis (MEMA). We also show that the relationship between neuronal activation in fMRI and behavioral data can be modeled using canonical correlation analysis (CCA). A real-world dataset for the neuronal response to nicotine use was acquired using a custom-made MRI-compatible apparatus for the smoking of electronic cigarettes (e-cigarettes). Nineteen participants smoked e-cigarettes in an MRI scanner using the apparatus with two experimental conditions: e-cigarettes with nicotine (ECIG) and sham e-cigarettes without nicotine (SCIG) and subjective ratings were collected. The right insula was identified in the ECIG condition from the χ2 -test of the MEMA but not from the t-test, and the corresponding activations were significantly associated with the similarity scores (r = -.52, p = .041, confidence interval [CI] = [-0.78, -0.17]) and the urge-to-smoke scores (r = .73, p <.001, CI = [0.52, 0.88]). From the contrast between the two conditions (i.e., ECIG > SCIG), the right orbitofrontal cortex was identified from the χ2 -tests, and the corresponding neuronal activations showed a statistically meaningful association with similarity (r = -.58, p = .01, CI = [-0.84, -0.17]) and the urge to smoke (r = .34, p = .15, CI = [0.09, 0.56]). The validity of our analysis pipeline (i.e., MEMA followed by CCA) was further evaluated using the fMRI and behavioral data acquired from the working memory and gambling tasks available from the Human Connectome Project.