Neuronal Population Activity in Macaque Visual Cortices Dynamically Changes through Repeated Fixations in Active Free Viewing.

During free viewing, we move our eyes and fixate on objects to recognize the visual scene of our surroundings. To investigate the neural representation of objects in this process, we studied individual and population neuronal activity in three different visual regions of the brains of macaque monkeys (Macaca fuscata): the primary and secondary visual cortices (V1, V2) and the inferotemporal cortex (IT). We designed a task where the animal freely selected objects in a stimulus image to fixate on while we examined the relationship between spiking activity, the order of fixations, and the fixated objects. We found that activity changed across repeated fixations on the same object in all three recorded areas, with observed reductions in firing rates. Furthermore, the responses of individual neurons became sparser and more selective with individual objects. The population activity for individual objects also became distinct. These results suggest that visual neurons respond dynamically to repeated input stimuli through a smaller number of spikes, thereby allowing for discrimination between individual objects with smaller energy.

A mosaic adeno-associated virus vector as a versatile tool that exhibits high levels of transgene expression and neuron specificity in primate brain.

Recent emphasis has been placed on gene transduction mediated through recombinant adeno-associated virus (AAV) vector to manipulate activity of neurons and their circuitry in the primate brain. In the present study, we created a novel vector of which capsid was composed of capsid proteins derived from both of the AAV serotypes 1 and 2 (AAV1 and AAV2). Following the injection into the frontal cortex of macaque monkeys, this mosaic vector, termed AAV2.1 vector, was found to exhibit the excellence in transgene expression (for AAV1 vector) and neuron specificity (for AAV2 vector) simultaneously. To explore its applicability to chemogenetic manipulation and in vivo calcium imaging, the AAV2.1 vector expressing excitatory DREADDs or GCaMP was injected into the striatum or the visual cortex of macaque monkeys, respectively. Our results have defined that such vectors secure intense and stable expression of the target proteins and yield conspicuous modulation and imaging of neuronal activity.

Analysis of convolutional neural networks reveals the computational properties essential for subcortical processing of facial expression.

Perception of facial expression is crucial for primate social interactions. This visual information is processed through the ventral cortical pathway and the subcortical pathway. However, the subcortical pathway exhibits inaccurate processing, and the responsible architectural and physiological properties remain unclear. To investigate this, we constructed and examined convolutional neural networks with three key properties of the subcortical pathway: a shallow layer architecture, concentric receptive fields at the initial processing stage, and a greater degree of spatial pooling. These neural networks achieved modest accuracy in classifying facial expressions. By replacing these properties, individually or in combination, with corresponding cortical features, performance gradually improved. Similar to amygdala neurons, some units in the final processing layer exhibited sensitivity to retina-based spatial frequencies (SFs), while others were sensitive to object-based SFs. Replacement of any of these properties affected the coordinates of the SF encoding. Therefore, all three properties limit the accuracy of facial expression information and are essential for determining the SF representation coordinate. These findings characterize the role of the subcortical computational processes in facial expression recognition.

Rapid processing of threatening faces in the amygdala of nonhuman primates: subcortical inputs and dual roles.

A subcortical pathway through the superior colliculus and pulvinar has been proposed to provide the amygdala with rapid but coarse visual information about emotional faces. However, evidence for short-latency, facial expression-discriminating responses from individual amygdala neurons is lacking; even if such a response exists, how it might contribute to stimulus detection is unclear. Also, no definitive anatomical evidence is available for the assumed pathway. Here we showed that ensemble responses of amygdala neurons in monkeys carried robust information about open-mouthed, presumably threatening, faces within 50 ms after stimulus onset. This short-latency signal was not found in the visual cortex, suggesting a subcortical origin. Temporal analysis revealed that the early response contained excitatory and suppressive components. The excitatory component may be useful for sending rapid signals downstream, while the sharpening of the rising phase of later-arriving inputs (presumably from the cortex) by the suppressive component might improve the processing of facial expressions over time. Injection of a retrograde trans-synaptic tracer into the amygdala revealed presumed monosynaptic labeling in the pulvinar and disynaptic labeling in the superior colliculus, including the retinorecipient layers. We suggest that the early amygdala responses originating from the colliculo-pulvino-amygdalar pathway play dual roles in threat detection.

Effects of vorticity on solitary waves.

The vorticity effect on solitary wave profiles has not been solved experimentally; previous studies theoretically and numerically showed that when a solitary wave progressed in the positive direction, the effective wavelength of a solitary wave with positive vorticity increased. Using laboratory experiments and fully nonlinear numerical simulations, we here show that the effective wavelength is extended more when positive vorticity is given to a progressive wave in the positive direction. We further show that the total energy increases with increasing positive vorticity, demonstrating that a wave with positive vorticity propagates with less attenuation and lasts longer than a solitary wave with no vorticity. We anticipate that our outcomes will provide a starting point for more sophisticated methods to investigate the effect of vorticity on solitary waves in laboratory experiments and numerical simulations.

Convolutional neural networks reveal differences in action units of facial expressions between face image databases developed in different countries.

Cultural similarities and differences in facial expressions have been a controversial issue in the field of facial communications. A key step in addressing the debate regarding the cultural dependency of emotional expression (and perception) is to characterize the visual features of specific facial expressions in individual cultures. Here we developed an image analysis framework for this purpose using convolutional neural networks (CNNs) that through training learned visual features critical for classification. We analyzed photographs of facial expressions derived from two databases, each developed in a different country (Sweden and Japan), in which corresponding emotion labels were available. While the CNNs reached high rates of correct results that were far above chance after training with each database, they showed many misclassifications when they analyzed faces from the database that was not used for training. These results suggest that facial features useful for classifying facial expressions differed between the databases. The selectivity of computational units in the CNNs to action units (AUs) of the face varied across the facial expressions. Importantly, the AU selectivity often differed drastically between the CNNs trained with the different databases. Similarity and dissimilarity of these tuning profiles partly explained the pattern of misclassifications, suggesting that the AUs are important for characterizing the facial features and differ between the two countries. The AU tuning profiles, especially those reduced by principal component analysis, are compact summaries useful for comparisons across different databases, and thus might advance our understanding of universality vs. specificity of facial expressions across cultures.

Latency shortening with enhanced sparseness and responsiveness in V1 during active visual sensing.

In natural vision, neuronal responses to visual stimuli occur due to self-initiated eye movements. Here, we compare single-unit activity in the primary visual cortex (V1) of non-human primates to flashed natural scenes (passive vision condition) to when they freely explore the images by self-initiated eye movements (active vision condition). Active vision enhances the number of neurons responding, and the response latencies become shorter and less variable across neurons. The increased responsiveness and shortened latency during active vision were not explained by increased visual contrast. While the neuronal activities in all layers of V1 show enhanced responsiveness and shortened latency, a significant increase in lifetime sparseness during active vision is observed only in the supragranular layer. These findings demonstrate that the neuronal responses become more distinct in active vision than passive vision, interpreted as consequences of top-down predictive mechanisms.

Processing of visual statistics of naturalistic videos in macaque visual areas V1 and V4.

Natural scenes are characterized by diverse image statistics, including various parameters of the luminance histogram, outputs of Gabor-like filters, and pairwise correlations between the filter outputs of different positions, orientations, and scales (Portilla-Simoncelli statistics). Some of these statistics capture the response properties of visual neurons. However, it remains unclear to what extent such statistics can explain neural responses to natural scenes and how neurons that are tuned to these statistics are distributed across the cortex. Using two-photon calcium imaging and an encoding-model approach, we addressed these issues in macaque visual areas V1 and V4. For each imaged neuron, we constructed an encoding model to mimic its responses to naturalistic videos. By extracting Portilla-Simoncelli statistics through outputs of both filters and filter correlations, and by computing an optimally weighted sum of these outputs, the model successfully reproduced responses in a subpopulation of neurons. We evaluated the selectivities of these neurons by quantifying the contributions of each statistic to visual responses. Neurons whose responses were mainly determined by Gabor-like filter outputs (low-level statistics) were abundant at most imaging sites in V1. In V4, the relative contribution of higher order statistics, such as cross-scale correlation, was increased. Preferred image statistics varied markedly across V4 sites, and the response similarity of two neurons at individual imaging sites gradually declined with increasing cortical distance. The results indicate that natural scene analysis progresses from V1 to V4, and neurons sharing preferred image statistics are locally clustered in V4.

Spatiotemporal dynamics of responses to biological motion in the human brain.

We sought to understand the spatiotemporal characteristics of biological motion perception. We presented observers with biological motion walkers that differed in terms of form coherence or kinematics (i.e., the presence or absence of natural acceleration). Participants were asked to discriminate the facing direction of the stimuli while their magnetoencephalographic responses were concurrently imaged. We found that two univariate response components can be observed around ~200 msec and ~650 msec post-stimulus onset, each engaging lateral-occipital and parietal cortex prior to temporal and frontal cortex. Moreover, while univariate responses show biological motion form-specificity only after 300 msec, multivariate patterns specific to form can be well discriminated from those for local cues as early as 100 msec after stimulus onset. By finally examining the representational similarity of fMRI and MEG patterned responses, we show that early responses to biological motion are most likely sourced to occipital cortex while later responses likely originate from extrastriate body areas.

Specialized contributions of mid-tier stages of dorsal and ventral pathways to stereoscopic processing in macaque.

The division of labor between the dorsal and ventral visual pathways has been well studied, but not often with direct comparison at the single-neuron resolution with matched stimuli. Here we directly compared how single neurons in MT and V4, mid-tier areas of the two pathways, process binocular disparity, a powerful cue for 3D perception and actions. We found that MT neurons transmitted disparity signals more quickly and robustly, whereas V4 or its upstream neurons transformed the signals into sophisticated representations more prominently. Therefore, signaling speed and robustness were traded for transformation between the dorsal and ventral pathways. The key factor in this tradeoff was disparity-tuning shape: V4 neurons had more even-symmetric tuning than MT neurons. Moreover, the tuning symmetry predicted the degree of signal transformation across neurons similarly within each area, implying a general role of tuning symmetry in the stereoscopic processing by the two pathways.

Impaired inhibition of return during free-viewing behaviour in patients with schizophrenia.

Schizophrenia affects various aspects of cognitive and behavioural functioning. Eye movement abnormalities are commonly observed in patients with schizophrenia (SZs). Here we examined whether such abnormalities reflect an anomaly in inhibition of return (IOR), the mechanism that inhibits orienting to previously fixated or attended locations. We analyzed spatiotemporal patterns of eye movement during free-viewing of visual images including natural scenes, geometrical patterns, and pseudorandom noise in SZs and healthy control participants (HCs). SZs made saccades to previously fixated locations more frequently than HCs. The time lapse from the preceding saccade was longer for return saccades than for forward saccades in both SZs and HCs, but the difference was smaller in SZs. SZs explored a smaller area than HCs. Generalized linear mixed-effect model analysis indicated that the frequent return saccades served to confine SZs' visual exploration to localized regions. The higher probability of return saccades in SZs was related to cognitive decline after disease onset but not to the dose of prescribed antipsychotics. We conclude that SZs exhibited attenuated IOR under free-viewing conditions, which led to restricted scene scanning. IOR attenuation will be a useful clue for detecting impairment in attention/orienting control and accompanying cognitive decline in schizophrenia.

Microstructural properties of the vertical occipital fasciculus explain the variability in human stereoacuity.

Stereopsis is a fundamental visual function that has been studied extensively. However, it is not clear why depth discrimination (stereoacuity) varies more significantly among people than other modalities. Previous studies have reported the involvement of both dorsal and ventral visual areas in stereopsis, implying that not only neural computations in cortical areas but also the anatomical properties of white matter tracts connecting those areas can impact stereopsis. Here, we studied how human stereoacuity relates to white matter properties by combining psychophysics, diffusion MRI (dMRI), and quantitative MRI (qMRI). We performed a psychophysical experiment to measure stereoacuity and, in the same participants, we analyzed the microstructural properties of visual white matter tracts on the basis of two independent measurements, dMRI (fractional anisotropy, FA) and qMRI (macromolecular tissue volume; MTV). Microstructural properties along the right vertical occipital fasciculus (VOF), a major tract connecting dorsal and ventral visual areas, were highly correlated with measures of stereoacuity. This result was consistent for both FA and MTV, suggesting that the behavioral-structural relationship reflects differences in neural tissue density, rather than differences in the morphological configuration of fibers. fMRI confirmed that binocular disparity stimuli activated the dorsal and ventral visual regions near VOF endpoints. No other occipital tracts explained the variance in stereoacuity. In addition, the VOF properties were not associated with differences in performance on a different psychophysical task (contrast detection). These series of experiments suggest that stereoscopic depth discrimination performance is, at least in part, constrained by dorso-ventral communication through the VOF.

Cortical and subcortical responses to biological motion.

Using fMRI and multivariate analyses we sought to understand the neural representations of articulated body shape and local kinematics in biological motion. We show that in addition to a cortical network that includes areas identified previously for biological motion perception, including the posterior superior temporal sulcus, inferior frontal gyrus, and ventral body areas, the ventral lateral nucleus, a presumably motoric thalamic area is sensitive to both form and kinematic information in biological motion. Our findings suggest that biological motion perception is not achieved as an end-point of segregated cortical form and motion networks as often suggested, but instead involves earlier parts in the visual system including a subcortical network.

Mapping stimulus feature selectivity in macaque V1 by two-photon Ca2+ imaging: Encoding-model analysis of fluorescence responses to natural movies.

In vivo calcium (Ca2+) imaging using two-photon microscopy allows activity to be monitored simultaneously from hundreds of individual neurons within a local population. While this allows us to gain important insights into how cortical neurons represent sensory information, factors such as photo-bleaching of the Ca2+ indicator limit imaging duration (and thus the numbers of stimuli that can be tested), which in turn hampers the full characterization of neuronal response properties. Here, we demonstrate that using an encoding model combined with presentation of natural movies results in detailed characterization of receptive field (RF) properties despite the relatively short time for data collection. During presentation of natural movie clips to macaque monkeys, we recorded fluorescence signals from primary visual cortex (V1) neurons that had been loaded with a Ca2+ indicator. For each recorded neuron, we constructed an encoding model that comprised an array of motion-energy filters that tiled over the RFs. We optimized the weight of each filter's output so that the linear sum of the outputs across the filters mimicked the neuron's Ca2+-signal responses. These models were able to predict the neural responses to a different set of natural movies with a significant degree of accuracy. Moreover, the orientation tunings of neurons simulated by the model were highly correlated with those experimentally obtained when grating stimuli were presented to the monkeys. The model predictions were also consistent with what is known about spatial frequency tunings, the structure of excitatory subfields of RFs (i.e., classical RFs), and functional maps for these RF properties in V1. Further analysis revealed a new aspect of V1 functional architecture; the extent and distribution of suppressive RF subfields varied among nearby neurons, while those for excitatory subfields were shared. Thus, applying our encoding-model analysis to two-photon Ca2+ imaging of neuronal responses to natural movies provides a reliable and efficient means of analyzing a wide range of RF properties in multiple neurons imaged in a local region.

A relative frame of reference underlies reversed depth perception in anticorrelated random-dot stereograms.

Binocular disparity is represented by interocular cross-correlation of visual images in the striate and some extrastriate cortices. This correlation-based representation produces reversed depth perception in a binocularly anticorrelated random-dot stereogram (aRDS) when it is accompanied by an adjacent correlated RDS (cRDS). Removal of the cRDS or spatial separation between the aRDS and cRDS abolishes reversed depth perception. However, how an immediate plane supports reversed depth perception is unclear. One possible explanation is that the correlation-based representation generates reversed depth based on the relative disparity between the aRDS and cRDS rather than the absolute disparity of the aRDS. Here, we psychophysically tested this hypothesis. We found that participants perceived reversed depth in an aRDS with zero absolute disparity when it was surrounded by a cRDS with nonzero absolute disparity (i.e., nonzero relative disparity), suggesting a role of relative disparity on the depth reversal. In addition, manipulation of the absolute disparities of the central aRDS and surrounding cRDS caused depth perception to reverse with respect to the depth of the surround. Further, depth reversal persisted after swapping the locations of the two RDSs. A model of relative-disparity encoding explains all these results. We conclude that reversed depth perception in aRDSs occurs in a relative frame of reference and suggest that the visual system contains correlation-based representation that encodes relative disparity.

Switch from ambient to focal processing mode explains the dynamics of free viewing eye movements.

Previous studies have reported that humans employ ambient and focal modes of visual exploration while they freely view natural scenes. These two modes have been characterized based on eye movement parameters such as saccade amplitude and fixation duration, but not by any visual features of the viewed scenes. Here we propose a new characterization of eye movements during free viewing based on how eyes are moved from and to objects in a visual scene. We applied this characterization to data obtained from freely-viewing macaque monkeys. We show that the analysis based on this characterization gives a direct indication of a behavioral shift from ambient to focal processing mode along the course of free viewing exploration. We further propose a stochastic model of saccade sequence generation incorporating a switch between the two processing modes, which quantitatively reproduces the behavioral features observed in the data.

Postnatal Dendritic Growth and Spinogenesis of Layer-V Pyramidal Cells Differ between Visual, Inferotemporal, and Prefrontal Cortex of the Macaque Monkey.

Pyramidal cells in the primate cerebral cortex, particularly those in layer III, exhibit regional variation in both the time course and magnitude of postnatal growth and pruning of dendrites and spines. Less is known about the development of pyramidal cell dendrites and spines in other cortical layers. Here we studied dendritic morphology of layer-V pyramidal cells in primary visual cortex (V1, sensory), cytoarchitectonic area TE in the inferotemporal cortex (sensory association), and granular prefrontal cortex (Walker's area 12, executive) of macaque monkeys at the ages of 2 days, 3 weeks, 3.5 months, and 4.5 years. We found that changes in the basal dendritic field area of pyramidal cells were different across the three areas. In V1, field size became smaller over time (largest at 2 days, half that size at 4.5 years), in TE it did not change, and in area 12 it became larger over time (smallest at 2 days, 1.5 times greater at 4.5 years). In V1 and TE, the total number of branch points in the basal dendritic trees was similar between 2 days and 4.5 years, while in area 12 the number was greater in the adult monkeys than in the younger ones. Spine density peaked at 3 weeks and declined in all areas by adulthood, with V1 exhibiting a faster decline than area TE or area 12. Estimates of the total number of spines in the dendritic trees revealed that following the onset of visual experience, pyramidal cells in V1 lose more spines than they grow, whereas those in TE and area 12 grow more spines than they lose during the same period. These data provide further evidence that the process of synaptic refinement in cortical pyramidal cells differs not only according to time, but also location within the cortex. Furthermore, given the previous finding that layer-III pyramidal cells in all these areas exhibit the highest density and total number of spines at 3.5 months, the current results indicate that pyramidal cells in layers III and V develop spines at different rates.

Postnatal Development of Intrinsic Horizontal Axons in Macaque Inferior Temporal and Primary Visual Cortices.

Two distinct areas along the ventral visual stream of monkeys, the primary visual (V1) and inferior temporal (TE) cortices, exhibit different projection patterns of intrinsic horizontal axons with patchy terminal fields in adult animals. The differences between the patches in these 2 areas may reflect differences in cortical representation and processing of visual information. We studied the postnatal development of patches by injecting an anterograde tracer into TE and V1 in monkeys of various ages. At 1 week of age, labeled patches with distribution patterns reminiscent of those in adults were already present in both areas. The labeling intensity of patches decayed exponentially with projection distance in monkeys of all ages in both areas, but this trend was far less evident in TE. The number and extent of patches gradually decreased with age in V1, but not in TE. In V1, axonal and bouton densities increased postnatally only in patches with short projection distances, whereas in TE this density change occurred in patches with various projection distances. Thus, patches with area-specific distribution patterns are formed early in life, and area-specific postnatal developmental processes shape the connectivity of patches into adulthood.

[Visual Processing in the Ventral Pathway: Shape, Face, Color, Material Properties, and Stereoscopic Depth].

The ventral visual pathway projecting from the primary visual cortex to the visual temporal association cortex processes shape, color, and material properties of objects as well as face and facial expression, and stereoscopic depth. The last decade has witnessed great advances in our understanding of how information is transformed at each transition between the stages along this pathway. Accumulating evidence also indicates that neurons in the later part of the pathway (e.g., area V4 and inferior temporal cortex) are causally involved in generating perception of shape, color, face, and fine stereoscopic depth.