Evaluation of the Hierarchical Correspondence between the Human Brain and Artificial Neural Networks: A Review.

Artificial neural networks (ANNs) that are heavily inspired by the human brain now achieve human-level performance across multiple task domains. ANNs have thus drawn attention in neuroscience, raising the possibility of providing a framework for understanding the information encoded in the human brain. However, the correspondence between ANNs and the brain cannot be measured directly. They differ in outputs and substrates, neurons vastly outnumber their ANN analogs (i.e., nodes), and the key algorithm responsible for most of modern ANN training (i.e., backpropagation) is likely absent from the brain. Neuroscientists have thus taken a variety of approaches to examine the similarity between the brain and ANNs at multiple levels of their information hierarchy. This review provides an overview of the currently available approaches and their limitations for evaluating brain-ANN correspondence.

On co-activation pattern analysis and non-stationarity of resting brain activity.

The non-stationarity of resting-state brain activity has received increasing attention in recent years. Functional connectivity (FC) analysis with short sliding windows and coactivation pattern (CAP) analysis are two widely used methods for assessing the dynamic characteristics of brain activity observed with functional magnetic resonance imaging (fMRI). However, the statistical nature of the dynamics captured by these techniques needs to be verified. In this study, we found that the results of CAP analysis were similar for real fMRI data and simulated stationary data with matching covariance structures and spectral contents. We also found that, for both the real and simulated data, CAPs were clustered into spatially heterogeneous modules. Moreover, for each of the modules in the real data, a spatially similar module was found in the simulated data. The present results suggest that care needs to be taken when interpreting observations drawn from CAP analysis as it does not necessarily reflect non-stationarity or a mixture of states in resting brain activity.

Distillation of Regional Activity Reveals Hidden Content of Neural Information in Visual Processing.

Multivoxel pattern analysis (MVPA) has become a standard tool for decoding mental states from brain activity patterns. Recent studies have demonstrated that MVPA can be applied to decode activity patterns of a certain region from those of the other regions. By applying a similar region-to-region decoding technique, we examined whether the information represented in the visual areas can be explained by those represented in the other visual areas. We first predicted the brain activity patterns of an area on the visual pathway from the others, then subtracted the predicted patterns from their originals. Subsequently, the visual features were derived from these residuals. During the visual perception task, the elimination of the top-down signals enhanced the simple visual features represented in the early visual cortices. By contrast, the elimination of the bottom-up signals enhanced the complex visual features represented in the higher visual cortices. The directions of such modulation effects varied across visual perception/imagery tasks, indicating that the information flow across the visual cortices is dynamically altered, reflecting the contents of visual processing. These results demonstrated that the distillation approach is a useful tool to estimate the hidden content of information conveyed across brain regions.

Counterfactual Explanation of Brain Activity Classifiers Using Image-To-Image Transfer by Generative Adversarial Network.

Deep neural networks (DNNs) can accurately decode task-related information from brain activations. However, because of the non-linearity of DNNs, it is generally difficult to explain how and why they assign certain behavioral tasks to given brain activations, either correctly or incorrectly. One of the promising approaches for explaining such a black-box system is counterfactual explanation. In this framework, the behavior of a black-box system is explained by comparing real data and realistic synthetic data that are specifically generated such that the black-box system outputs an unreal outcome. The explanation of the system's decision can be explained by directly comparing the real and synthetic data. Recently, by taking advantage of advances in DNN-based image-to-image translation, several studies successfully applied counterfactual explanation to image domains. In principle, the same approach could be used in functional magnetic resonance imaging (fMRI) data. Because fMRI datasets often contain multiple classes (e.g., multiple behavioral tasks), the image-to-image transformation applicable to counterfactual explanation needs to learn mapping among multiple classes simultaneously. Recently, a new generative neural network (StarGAN) that enables image-to-image transformation among multiple classes has been developed. By adapting StarGAN with some modifications, here, we introduce a novel generative DNN (counterfactual activation generator, CAG) that can provide counterfactual explanations for DNN-based classifiers of brain activations. Importantly, CAG can simultaneously handle image transformation among all the seven classes in a publicly available fMRI dataset. Thus, CAG could provide a counterfactual explanation of DNN-based multiclass classifiers of brain activations. Furthermore, iterative applications of CAG were able to enhance and extract subtle spatial brain activity patterns that affected the classifier's decisions. Together, these results demonstrate that the counterfactual explanation based on image-to-image transformation would be a promising approach to understand and extend the current application of DNNs in fMRI analyses.

Two-photon microscopic observation of cell-production dynamics in the developing mammalian neocortex in utero.

Morphogenesis and organ development should be understood based on a thorough description of cellular dynamics. Recent studies have explored the dynamic behaviors of mammalian neural progenitor cells (NPCs) using slice cultures in which three-dimensional systems conserve in vivo-like environments to a considerable degree. However, live observation of NPCs existing truly in vivo, as has long been performed for zebrafish NPCs, has yet to be established in mammals. Here, we performed intravital two-photon microscopic observation of NPCs in the developing cerebral cortex of H2B-EGFP or Fucci transgenic mice in utero. Fetuses in the uterine sac were immobilized using several devices and were observed through a window made in the uterine wall and the amniotic membrane while monitoring blood circulation. Clear visibility was obtained to the level of 300 µm from the scalp surface of the fetus, which enabled us to quantitatively assess NPC behaviors, such as division and interkinetic nuclear migration, within a neuroepithelial structure called the ventricular zone at embryonic day (E) 13 and E14. In fetuses undergoing healthy monitoring in utero for 60 min, the frequency of mitoses observed at the apical surface was similar to those observed in slice cultures and in freshly fixed in vivo specimens. Although the rate and duration of successful in utero observations are still limited (33% for ≥10 min and 14% for 60 min), further improvements based on this study will facilitate future understanding of how organogenetic cellular behaviors occur or are pathologically influenced by the systemic maternal condition and/or maternal-fetal relationships.

Author Correction: Role of extrinsic mechanical force in the development of the RA-I tactile mechanoreceptor.

A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has been fixed in the paper.

Role of extrinsic mechanical force in the development of the RA-I tactile mechanoreceptor.

Rapidly adapting type I (RA-I) mechanoreceptors play an important role in sensing the low-frequency vibration aspects of touch. The structure of the RA-I mechanoreceptor is extremely complex regardless of its small size, limiting our understanding of its mechanotransduction. As a result of the emergence of bioengineering, we previously proposed an in vitro bioengineering approach for RA-I receptors to overcome this limitation. Currently, the in vitro bioengineering approach for the RA-I receptor is not realizable given the lack of knowledge of its morphogenesis. This paper demonstrates our first attempt to interpret the cellular morphogenesis of the RA-I receptor. We found indications of extrinsic mechanical force nearby the RA-I receptor in the developing fingertip. Using a mechanical compression device, the axon of dorsal root ganglion (DRG) neurons buckled in vitro into a profile that resembled the morphology of the RA-I receptor. This work encourages further implementation of this bioengineering approach in tactile receptor-related research.

Effect of 3D microstructure of dermal papillae on SED concentration at a mechanoreceptor location.

The feeling of touch is an essential human sensation. Four types of mechanoreceptors (i.e., FA-I, SA-I, FA-II, and SA-II) in human skin signalize physical properties, such as shape, size, and texture, of an object that is touched and transmit the signal to the brain. Previous studies attempted to investigate the mechanical properties of skin microstructure and their effect on mechanoreceptors by using finite element modeling. However, very few studies have focused on the three-dimensional microstructure of dermal papillae, and this is related to that of FA-I receptors. A gap exists between conventional 2D models of dermal papillae and the natural configuration, which corresponds to a complex and uneven structure with depth. In this study, the three-dimensional microstructure of dermal papillae is modeled, and the differences between two-dimensional and three-dimensional aspects of dermal papillae on the strain energy density at receptor positions are examined. The three-dimensional microstructure has a focalizing effect and a localizing effect. Results also reveal the potential usefulness of these effects for tactile sensor design, and this may improve edge discrimination.

Two-Photon Imaging of DiO-Labelled Meissner Corpuscle in Living Mouse's Fingertip.

Meissner corpuscles are the fast adapting type I (FA-I) mechanoreceptor that locates at the dermal papillae of skin. The Meissner corpuscle is well known for its complex structure, consisting of spiral axons, lamellar cells, and a collagen capsule. Fluorescent microscopy has become a convenient method for observing the Meissner corpuscle and its inner structure. This method requires preparing samples with fingertip cross-sections and performing antibody staining before observation. Various kinds of microscopy can be used for observation, such as confocal microscopy, transmission electron microscopy (TEM), or scanning electron microscopy (SEM). Although the anatomical shape, distribution, and components of Meissner corpuscle are recognized, they have been mostly determined from observations of fixed tissues. Therefore, knowledge of mechanical transduction is limited by the lack of in vivo experiments and individual differences among samples. In this study, we propose a novel less invasive imaging method that incorporates a staining technique with lipophilic carbocyanine [Formula: see text] and two-photon microscopy. This combination allows us to repetitively observe the Meissner corpuscle in a living mouse.