Analysis of convolutional neural networks reveals the computational properties essential for subcortical processing of facial expression.

Perception of facial expression is crucial for primate social interactions. This visual information is processed through the ventral cortical pathway and the subcortical pathway. However, the subcortical pathway exhibits inaccurate processing, and the responsible architectural and physiological properties remain unclear. To investigate this, we constructed and examined convolutional neural networks with three key properties of the subcortical pathway: a shallow layer architecture, concentric receptive fields at the initial processing stage, and a greater degree of spatial pooling. These neural networks achieved modest accuracy in classifying facial expressions. By replacing these properties, individually or in combination, with corresponding cortical features, performance gradually improved. Similar to amygdala neurons, some units in the final processing layer exhibited sensitivity to retina-based spatial frequencies (SFs), while others were sensitive to object-based SFs. Replacement of any of these properties affected the coordinates of the SF encoding. Therefore, all three properties limit the accuracy of facial expression information and are essential for determining the SF representation coordinate. These findings characterize the role of the subcortical computational processes in facial expression recognition.

Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries.

Cultural similarities and differences in facial expressions have been a controversial issue in the field of facial communications. A key step in addressing the debate regarding the cultural dependency of emotional expression (and perception) is to characterize the visual features of specific facial expressions in individual cultures. Here we developed an image analysis framework for this purpose using convolutional neural networks (CNNs) that through training learned visual features critical for classification. We analyzed photographs of facial expressions derived from two databases, each developed in a different country (Sweden and Japan), in which corresponding emotion labels were available. While the CNNs reached high rates of correct results that were far above chance after training with each database, they showed many misclassifications when they analyzed faces from the database that was not used for training. These results suggest that facial features useful for classifying facial expressions differed between the databases. The selectivity of computational units in the CNNs to action units (AUs) of the face varied across the facial expressions. Importantly, the AU selectivity often differed drastically between the CNNs trained with the different databases. Similarity and dissimilarity of these tuning profiles partly explained the pattern of misclassifications, suggesting that the AUs are important for characterizing the facial features and differ between the two countries. The AU tuning profiles, especially those reduced by principal component analysis, are compact summaries useful for comparisons across different databases, and thus might advance our understanding of universality vs. specificity of facial expressions across cultures.

Biologically motivated learning method for deep neural networks using hierarchical competitive learning.

This study proposes a novel biologically motivated learning method for deep convolutional neural networks (CNNs). The combination of CNNs and backpropagation learning is the most powerful method in recent machine learning regimes. However, it requires a large amount of labeled data for training, and this requirement can occasionally become a barrier for real world applications. To address this problem and use unlabeled data, we introduce unsupervised competitive learning, which only requires forward propagating signals for CNNs. The method was evaluated on image discrimination tasks using the MNIST, CIFAR-10, and ImageNet datasets, and it achieved state-of-the-art performance with respect to other biologically motivated methods in the ImageNet benchmark. The results suggest that the method enables higher-level learning representations solely based on the forward propagating signals without the need for a backward error signal for training convolutional layers. The proposed method could be useful for a variety of poorly labeled data, for example, time series or medical data.

Impaired inhibition of return during free-viewing behaviour in patients with schizophrenia.

Schizophrenia affects various aspects of cognitive and behavioural functioning. Eye movement abnormalities are commonly observed in patients with schizophrenia (SZs). Here we examined whether such abnormalities reflect an anomaly in inhibition of return (IOR), the mechanism that inhibits orienting to previously fixated or attended locations. We analyzed spatiotemporal patterns of eye movement during free-viewing of visual images including natural scenes, geometrical patterns, and pseudorandom noise in SZs and healthy control participants (HCs). SZs made saccades to previously fixated locations more frequently than HCs. The time lapse from the preceding saccade was longer for return saccades than for forward saccades in both SZs and HCs, but the difference was smaller in SZs. SZs explored a smaller area than HCs. Generalized linear mixed-effect model analysis indicated that the frequent return saccades served to confine SZs' visual exploration to localized regions. The higher probability of return saccades in SZs was related to cognitive decline after disease onset but not to the dose of prescribed antipsychotics. We conclude that SZs exhibited attenuated IOR under free-viewing conditions, which led to restricted scene scanning. IOR attenuation will be a useful clue for detecting impairment in attention/orienting control and accompanying cognitive decline in schizophrenia.

Convolutional neural network with autoencoder-assisted multiclass labelling for seizure detection based on scalp electroencephalography.

OBJECTIVE: In long-term video-monitoring, automatic seizure detection holds great promise as a means to reduce the workload of the epileptologist. A convolutional neural network (CNN) designed to process images of EEG plots demonstrated high performance for seizure detection, but still has room for reducing the false-positive alarm rate. METHODS: We combined a CNN that processed images of EEG plots with patient-specific autoencoders (AE) of EEG signals to reduce the false alarms during seizure detection. The AE automatically logged abnormalities, i.e., both seizures and artifacts. Based on seizure logs compiled by expert epileptologists and errors made by AE, we constructed a CNN with 3 output classes: seizure, non-seizure-but-abnormal, and non-seizure. The accumulative measure of number of consecutive seizure labels was used to issue a seizure alarm. RESULTS: The second-by-second classification performance of AE-CNN was comparable to that of the original CNN. False-positive seizure labels in AE-CNN were more likely interleaved with "non-seizure-but-abnormal" labels than with true-positive seizure labels. Consequently, "non-seizure-but-abnormal" labels interrupted runs of false-positive seizure labels before triggering an alarm. The median false alarm rate with the AE-CNN was reduced to 0.034 h-1, which was one-fifth of that of the original CNN (0.17 h-1). CONCLUSIONS: A label of "non-seizure-but-abnormal" offers practical benefits for seizure detection. The modification of a CNN with an AE is worth considering because AEs can automatically assign "non-seizure-but-abnormal" labels in an unsupervised manner with no additional demands on the time of the epileptologist.

Autoencoding of long-term scalp electroencephalogram to detect epileptic seizure for diagnosis support system.

INTRODUCTION: Epileptologists could benefit from a diagnosis support system that automatically detects seizures because visual inspection of long-term electroencephalograms (EEGs) is extremely time-consuming. However, the diversity of seizures among patients makes it difficult to develop universal features that are applicable for automatic seizure detection in all cases, and the rarity of seizures results in a lack of sufficient training data for classifiers. METHODS: To overcome these problems, we utilized an autoencoder (AE), which is often used for anomaly detection in the field of machine learning, to perform seizure detection. We hypothesized that multichannel EEG signals are compressible by AE owing to their spatio-temporal coupling and that the AE should be able to detect seizures as anomalous events from an interictal EEG. RESULTS: Through experiments, we found that the AE error was able to classify seizure and nonseizure states with a sensitivity of 100% in 22 out of 24 available test subjects and that the AE was better than the commercially available software BESA and Persyst for half of the test subjects. CONCLUSIONS: These results suggest that the AE error is a feasible candidate for a universal seizure detection feature.

Seizure detection by convolutional neural network-based analysis of scalp electroencephalography plot images.

We hypothesized that expert epileptologists can detect seizures directly by visually analyzing EEG plot images, unlike automated methods that analyze spectro-temporal features or complex, non-stationary features of EEG signals. If so, seizure detection could benefit from convolutional neural networks because their visual recognition ability is comparable to that of humans. We explored image-based seizure detection by applying convolutional neural networks to long-term EEG that included epileptic seizures. After filtering, EEG data were divided into short segments based on a given time window and converted into plot EEG images, each of which was classified by convolutional neural networks as 'seizure' or 'non-seizure'. These resultant labels were then used to design a clinically practical index for seizure detection. The best true positive rate was obtained using a 1-s time window. The median true positive rate of convolutional neural networks labelling by seconds was 74%, which was higher than that of commercially available seizure detection software (20% by BESA and 31% by Persyst). For practical use, the median of detected seizure rate by minutes was 100% by convolutional neural networks, which was higher than the 73.3% by BESA and 81.7% by Persyst. The false alarm of convolutional neural networks' seizure detection was issued at 0.2 per hour, which appears acceptable for clinical practice. Moreover, we demonstrated that seizure detection improved when training was performed using EEG patterns similar to those of testing data, suggesting that adding a variety of seizure patterns to the training dataset will improve our method. Thus, artificial visual recognition by convolutional neural networks allows for seizure detection, which otherwise currently relies on skillful visual inspection by expert epileptologists during clinical diagnosis.

Prediction of IDH and TERT promoter mutations in low-grade glioma from magnetic resonance images using a convolutional neural network.

Identification of genotypes is crucial for treatment of glioma. Here, we developed a method to predict tumor genotypes using a pretrained convolutional neural network (CNN) from magnetic resonance (MR) images and compared the accuracy to that of a diagnosis based on conventional radiomic features and patient age. Multisite preoperative MR images of 164 patients with grade II/III glioma were grouped by IDH and TERT promoter (pTERT) mutations as follows: (1) IDH wild type, (2) IDH and pTERT co-mutations, (3) IDH mutant and pTERT wild type. We applied a CNN (AlexNet) to four types of MR sequence and obtained the CNN texture features to classify the groups with a linear support vector machine. The classification was also performed using conventional radiomic features and/or patient age. Using all features, we succeeded in classifying patients with an accuracy of 63.1%, which was significantly higher than the accuracy obtained from using either the radiomic features or patient age alone. In particular, prediction of the pTERT mutation was significantly improved by the CNN texture features. In conclusion, the pretrained CNN texture features capture the information of IDH and TERT genotypes in grade II/III gliomas better than the conventional radiomic features.

Gap junctions facilitate propagation of synchronous firing in the cortical neural population: a numerical simulation study.

This study investigates the effect of gap junctions on firing propagation in a feedforward neural network by a numerical simulation with biologically plausible parameters. Gap junctions are electrical couplings between two cells connected by a binding protein, connexin. Recent electrophysiological studies have reported that a large number of inhibitory neurons in the mammalian cortex are mutually connected by gap junctions, and synchronization of gap junctions, spread over several hundred microns, suggests that these have a strong effect on the dynamics of the cortical network. However, the effect of gap junctions on firing propagation in cortical circuits has not been examined systematically. In this study, we perform numerical simulations using biologically plausible parameters to clarify this effect on population firing in a feedforward neural network. The results suggest that gap junctions switch the temporally uniform firing in a layer to temporally clustered firing in subsequent layers, resulting in an enhancement in the propagation of population firing in the feedforward network. Because gap junctions are often modulated in physiological conditions, we speculate that gap junctions could be related to a gating function of population firing in the brain.

[Shellfish monitoring system for paralytic shellfish toxins using enzyme-linked immunosorbent assay].

We investigated the applicability of enzyme-linked immunosorbent assay (PSP-ELISA) using a monoclonal antibody against paralytic shellfish toxins (PST) for screening oysters collected at several coastal areas in Kumamoto prefecture, Japan. Oysters collected between 2007 and 2010 were analyzed by PSP-ELISA. As an alternative calibrant, a naturally contaminated oyster extract was used to quantify toxins in the oyster samples. The toxicity of the calibrant oyster extract determined by the official testing method, mouse bioassay (MBA), was 4 MU/g. Oyster samples collected over 3 years showed a similar toxin profile to the alternative standard, resulting in good agreement between the PSP-ELISA and the MBA. The PSP-ELISA method was better than the MBA in terms of sensitivity, indicating that it may be useful for earlier warning of contamination of oysters by PST in the distinct coastal areas. To use the PSP-ELISA as a screening method prior to MBA, we finally set a screening level at 2 MU/g PSP-ELISA for oyster monitoring in Kumamoto prefecture. We confirmed that there were on samples exceeding the quarantine level (4 MU/g) in MBA among samples quantified as below the screening level by the PSP-ELISA. It was concluded that the use of PSP-ELISA could reduce the numbers of animals needed for MBA testing.

Spike suppression in a local cortical circuit induced by transcranial magnetic stimulation.

Transcranial magnetic stimulation (TMS) noninvasively interferes with human cortical function, and is widely used as an effective technique for probing causal links between neural activity and cognitive function. However, the physiological mechanisms underlying TMS-induced effects on neural activity remain unclear. We examined the mechanism by which TMS disrupts neural activity in a local circuit in early visual cortex using a computational model consisting of conductance-based spiking neurons with excitatory and inhibitory synaptic connections. We found that single-pulse TMS suppressed spiking activity in a local circuit model, disrupting the population response. Spike suppression was observed when TMS was applied to the local circuit within a limited time window after the local circuit received sensory afferent input, as observed in experiments investigating suppression of visual perception with TMS targeting early visual cortex. Quantitative analyses revealed that the magnitude of suppression was significantly larger for synaptically-connected neurons than for isolated individual neurons, suggesting that intracortical inhibitory synaptic coupling also plays an important role in TMS-induced suppression. A conventional local circuit model of early visual cortex explained only the early period of visual suppression observed in experiments. However, models either involving strong recurrent excitatory synaptic connections or sustained excitatory input were able to reproduce the late period of visual suppression. These results suggest that TMS targeting early visual cortex disrupts functionally distinct neural signals, possibly corresponding to feedforward and recurrent information processing, by imposing inhibitory effects through intracortical inhibitory synaptic connections.

Flexible traffic control of the synfire-mode transmission by inhibitory modulation: nonlinear noise reduction.

Intermingled neural connections apparent in the brain make us wonder what controls the traffic of propagating activity in the brain to secure signal transmission without harmful crosstalk. Here, we reveal that inhibitory input but not excitatory input works as a particularly useful traffic controller because it controls the degree of synchrony of population firing of neurons as well as controlling the size of the population firing bidirectionally. Our dynamical system analysis reveals that the synchrony enhancement depends crucially on the nonlinear membrane potential dynamics and a hidden slow dynamical variable. Our electrophysiological study with rodent slice preparations show that the phenomenon happens in real neurons. Furthermore, our analysis with the Fokker-Planck equations demonstrates the phenomenon in a semianalytical manner.