Local field potential responses associated with perceptual filling-in at blind spot in macaque primary visual cortex.

To understand the neural mechanisms of perceptual filling-in at the blind spot (BS), we analyzed neural activity in the region representing the visual field corresponding to the BS (BS region) in the primary visual cortex (V1) of the macaque monkey. We inserted a linear array electrode into the BS region or surrounding region and recorded the multiunit activities (MUAs) and local field potential (LFP). We examined the responses of MUAs and LFP to a large visual stimulus that entirely covered the BS (surface stimuli) while the monkey performed a visual fixation task in either the monocular condition without receiving direct retinal input or the binocular condition receiving retinal information. We observed clear MUA responses in the deep layers within the BS region under monocular conditions, confirming previous reports that V1 neurons in the BS region are activated when perceptual filling-in occurs. Current source density analysis using LFP showed that MUA responses were mainly observed in layer 5. Although LFP responses were generally stronger in the binocular condition than in the monocular condition, a notable exception was observed in the BS region. LFP responses in the low-beta band in the superficial layers were stronger in the monocular condition than in the binocular condition. These results suggest that low-beta activity in the superficial layer is related to the occurrence of perceptual filling-in in the BS. The origin of this activity is considered to be feedback signals from the extrastriate areas to the V1.NEW & NOTEWORTHY Two characteristic activities were induced in the blind spot (BS) region in response to the stimulus, causing perceptual filling-in: 1) beta-band LFP responses in the superficial layers and 2) neuronal responses in the deep layers, mainly in layer 5. These data suggest that the feedback signal from the extrastriate areas to the BS region in V1 is involved in perceptual filling-in.

Origin of the ease of association of color names: Comparison between humans and AI.

Rapid evolution of artificial intelligence (AI) based on deep neural networks has resulted in artificial systems such as generative pre-trained transformer 3 (GPT-3), which can generate human-like language. Such a system may provide a novel platform for studying how human perception is related to knowledge and the ability of language generation. We compared the frequency distribution of basic color terms in the answers of human subjects and GPT-3 when both were asked similar questions regarding color names associated with the letters of the alphabet. We found that GPT-3 generated basic color terms at a frequency very similar to that of human non-synaesthetes. A similar frequency was observed when color names associated with numerals were tested indicating that simple co-occurrence of alphabet and color word in the trained dataset cannot explain the results. We suggest that the proposed experimental framework using the latest AI models has the potential to explore the mechanisms of human perception.

Relationship Between the Activities of Gloss-Selective Neurons in the Macaque Inferior Temporal Cortex and the Gloss Discrimination Behavior of the Monkey.

In the macaque monkey, neurons that selectively respond to specific gloss are present in a restricted region of the central part of the inferior temporal (IT) cortex. Although the population activity of these neurons is known to represent the perceptual gloss space, the involvement of their activity in gloss perception has not been directly tested. In the present study, we examined the causal relationship between the activities of gloss-selective neurons and gloss perception by applying electrical microstimulation or injection of small amounts of muscimol (GABAA agonist) to manipulate neural activities while monkeys performed a gloss discrimination task. We found that microstimulation within or in the vicinity of the region where gloss-selective neurons were recorded induced bias toward higher gloss judgment. With muscimol injection, gloss discrimination performance was degraded in one monkey after the first injection into the region where gloss-selective neurons were recorded. These results suggest that gloss discrimination behavior is mediated by the activities of a gloss-selective network that includes the gloss-selective region in the central IT cortex examined here.

Structure and function of neural circuit related to gloss perception in the macaque inferior temporal cortex: a case report.

The inferior temporal (IT) cortex of the macaque monkey plays a pivotal role in the visual recognition of objects. In the IT cortex, a feature-selective network formed by connecting subregions specialized for common visual features seems to be a basic strategy for processing biologically important visual features. Gloss perception plays an important role in the judgment of materials and conditions of objects and is a biologically significant visual function. In the present study, we attempted to determine whether a neural circuit specialized for processing information related to gloss perception exists in the IT cortex in one monkey. We injected retrograde tracer into a gloss-selective subregion in the IT cortex where gloss-selective neurons were clustered in the neural recording experiment, and anatomically examined its neural connections. We observed that retrogradely labeled neurons were densely accumulated in multiple locations in the posterior and anterior IT cortices. Based on the results of this case study, we will discuss the possibility that, together with the injection site, the sites with a dense cluster of labeled neurons form feature-selective neural circuits for the processing of gloss information in the IT cortex.

Computational Model for Human 3D Shape Perception From a Single Specular Image.

In natural conditions the human visual system can estimate the 3D shape of specular objects even from a single image. Although previous studies suggested that the orientation field plays a key role for 3D shape perception from specular reflections, its computational plausibility, and possible mechanisms have not been investigated. In this study, to complement the orientation field information, we first add prior knowledge that objects are illuminated from above and utilize the vertical polarity of the intensity gradient. Then we construct an algorithm that incorporates these two image cues to estimate 3D shapes from a single specular image. We evaluated the algorithm with glossy and mirrored surfaces and found that 3D shapes can be recovered with a high correlation coefficient of around 0.8 with true surface shapes. Moreover, under a specific condition, the algorithm's errors resembled those made by human observers. These findings show that the combination of the orientation field and the vertical polarity of the intensity gradient is computationally sufficient and probably reproduces essential representations used in human shape perception from specular reflections.

Neural Mechanisms of Material Perception: Quest on Shitsukan.

In recent years, a growing body of research has addressed the nature and mechanism of material perception. Material perception entails perceiving and recognizing a material, surface quality or internal state of an object based on sensory stimuli such as visual, tactile, and/or auditory sensations. This process is ongoing in every aspect of daily life. We can, for example, easily distinguish whether an object is made of wood or metal, or whether a surface is rough or smooth. Judging whether the ground is wet or dry or whether a fish is fresh also involves material perception. Information obtained through material perception can be used to govern actions toward objects and to make decisions about whether to approach an object or avoid it. Because the physical processes leading to sensory signals related to material perception is complicated, it has been difficult to manipulate experimental stimuli in a rigorous manner. However, that situation is now changing thanks to advances in technology and knowledge in related fields. In this article, we will review what is currently known about the neural mechanisms responsible for material perception. We will show that cortical areas in the ventral visual pathway are strongly involved in material perception. Our main focus is on vision, but every sensory modality is involved in material perception. Information obtained through different sensory modalities is closely linked in material perception. Such cross-modal processing is another important feature of material perception, and will also be covered in this review.

Dependence of behavioral performance on material category in an object-grasping task with monkeys.

Material perception is an essential part of our cognitive function that enables us to properly interact with our complex daily environment. One important aspect of material perception is its multimodal nature. When we see an object, we generally recognize its haptic properties as well as its visual properties. Consequently, one must examine behavior using real objects that are perceived both visually and haptically to fully understand the characteristics of material perception. As a first step, we examined whether there is any difference in the behavioral responses to different materials in monkeys trained to perform an object grasping task in which they saw and grasped rod-shaped real objects made of various materials. We found that the monkeys' behavior in the grasping task, which was measured based on the success rate and the pulling force, differed depending on the material category. Monkeys easily and correctly grasped objects of some materials, such as metal and glass, but failed to grasp objects of other materials. In particular, monkeys avoided grasping fur-covered objects. The differences in the behavioral responses to the material categories cannot be explained solely based on the degree of familiarity with the different materials. These results shed light on the organization of multimodal representation of materials, where their biological significance is an important factor. In addition, a monkey that avoided touching real fur-covered objects readily touched images of the same objects presented on a CRT display. This suggests that employing real objects is important when studying behaviors related to material perception. NEW & NOTEWORTHY We tested monkeys using an object-grasping task in which monkeys saw and grasped rod-shaped real objects made of various materials. We found that the monkeys' behavior differed dramatically across the material categories and that the behavioral differences could not be explained solely based on the degree of familiarity with the different materials. These results shed light on the organization of multimodal representation of materials, where the biological significance of materials is an important factor.

Task-dependent recurrent dynamics in visual cortex.

The capacity for flexible sensory-action association in animals has been related to context-dependent attractor dynamics outside the sensory cortices. Here, we report a line of evidence that flexibly modulated attractor dynamics during task switching are already present in the higher visual cortex in macaque monkeys. With a nonlinear decoding approach, we can extract the particular aspect of the neural population response that reflects the task-induced emergence of bistable attractor dynamics in a neural population, which could be obscured by standard unsupervised dimensionality reductions such as PCA. The dynamical modulation selectively increases the information relevant to task demands, indicating that such modulation is beneficial for perceptual decisions. A computational model that features nonlinear recurrent interaction among neurons with a task-dependent background input replicates the key properties observed in the experimental data. These results suggest that the context-dependent attractor dynamics involving the sensory cortex can underlie flexible perceptual abilities.

Gradual Development of Visual Texture-Selective Properties Between Macaque Areas V2 and V4.

Complex shape and texture representations are known to be constructed from V1 along the ventral visual pathway through areas V2 and V4, but the underlying mechanism remains elusive. Recent study suggests that, for processing of textures, a collection of higher-order image statistics computed by combining V1-like filter responses serves as possible representations of textures both in V2 and V4. Here, to gain a clue for how these image statistics are processed in the extrastriate visual areas, we compared neuronal responses to textures in V2 and V4 of macaque monkeys. For individual neurons, we adaptively explored their preferred textures from among thousands of naturalistic textures and fitted the obtained responses using a combination of V1-like filter responses and higher-order statistics. We found that, while the selectivity for image statistics was largely comparable between V2 and V4, V4 showed slightly stronger sensitivity to the higher-order statistics than V2. Consistent with that finding, V4 responses were reduced to a greater extent than V2 responses when the monkeys were shown spectrally matched noise images that lacked higher-order statistics. We therefore suggest that there is a gradual development in representation of higher-order features along the ventral visual hierarchy.

Comparison of the color selectivity of macaque V4 neurons in different color spaces.

Chromatic selectivity has been studied extensively in various visual areas at different stages of visual processing in the macaque brain. In these studies, color stimuli defined in the Derrington-Krauskopf-Lennie (DKL) color space with a limited range of cone contrast were typically used in early stages, whereas those defined in the Commission Internationale de l'Eclairage (CIE) color space, based on human psychophysical measurements across the gamut of the display, were often used in higher visual areas. To understand how the color information is processed along the visual pathway, it is necessary to compare color selectivity obtained in different areas on a common color space. In the present study, we tested whether the neural color selectivity obtained in DKL space can be predicted from responses obtained in CIE space and whether stimuli with limited cone contrast are sufficient to characterize neural color selectivity. We found that for most V4 neurons, there was a strong correlation between responses measured using the two chromatic coordinate systems, and the color selectivities obtained with the two stimulus sets were comparable. However, for some neurons preferring high- or low-saturation colors, stimuli defined in DKL color space did not adequately capture the neural color selectivity. This is mainly due to the use of stimuli within a limited range of cone contrast. We conclude that regardless of the choice of color space, the sampling of colors across the entire gamut is important to characterize neural color selectivity fully or to compare color selectivities in different areas so as to understand color representation in the visual system.

Similarity in Neuronal Firing Regimes across Mammalian Species.

UNLABELLED: The architectonic subdivisions of the brain are believed to be functional modules, each processing parts of global functions. Previously, we showed that neurons in different regions operate in different firing regimes in monkeys. It is possible that firing regimes reflect differences in underlying information processing, and consequently the firing regimes in homologous regions across animal species might be similar. We analyzed neuronal spike trains recorded from behaving mice, rats, cats, and monkeys. The firing regularity differed systematically, with differences across regions in one species being greater than the differences in similar areas across species. Neuronal firing was consistently most regular in motor areas, nearly random in visual and prefrontal/medial prefrontal cortical areas, and bursting in the hippocampus in all animals examined. This suggests that firing regularity (or irregularity) plays a key role in neural computation in each functional subdivision, depending on the types of information being carried. SIGNIFICANCE STATEMENT: By analyzing neuronal spike trains recorded from mice, rats, cats, and monkeys, we found that different brain regions have intrinsically different firing regimes that are more similar in homologous areas across species than across areas in one species. Because different regions in the brain are specialized for different functions, the present finding suggests that the different activity regimes of neurons are important for supporting different functions, so that appropriate neuronal codes can be used for different modalities.

Crossmodal Association of Visual and Haptic Material Properties of Objects in the Monkey Ventral Visual Cortex.

Just by looking at an object, we can recognize its non-visual properties, such as hardness. The visual recognition of non-visual object properties is generally accurate [1], and influences actions toward the object [2]. Recent studies suggest that, in the primate brain, this may involve the ventral visual cortex, which represents objects in a way that reflects not only visual but also non-visual object properties, such as haptic roughness, hardness, and weight [3-7]. This new insight raises a fundamental question: how does the visual cortex come to represent non-visual properties--knowledge that cannot be acquired directly through vision? Here we addressed this unresolved question using fMRI in macaque monkeys. Specifically, we explored whether and how simple visuo-haptic experience--just seeing and touching objects made of various materials--can shape representational content in the visual cortex. We measured brain activity evoked by viewing images of objects before and after the monkeys acquired the visuo-haptic experience and decoded the representational space from the activity patterns [8]. We show that simple long-term visuo-haptic experience greatly impacts representation in the posterior inferior temporal cortex, the higher ventral visual cortex. After the experience, but not before, the activity pattern in this region well reflected the haptic material properties of the experienced objects. Our results suggest that neural representation of non-visual object properties in the visual cortex emerges through long-term crossmodal exposure to objects. This highlights the importance of unsupervised learning of crossmodal associations through everyday experience [9-12] for shaping representation in the visual cortex.

Population Code Dynamics in Categorical Perception.

Categorical perception is a ubiquitous function in sensory information processing, and is reported to have important influences on the recognition of presented and/or memorized stimuli. However, such complex interactions among categorical perception and other aspects of sensory processing have not been explained well in a unified manner. Here, we propose a recurrent neural network model to process categorical information of stimuli, which approximately realizes a hierarchical Bayesian estimation on stimuli. The model accounts for a wide variety of neurophysiological and cognitive phenomena in a consistent framework. In particular, the reported complexity of categorical effects, including (i) task-dependent modulation of neural response, (ii) clustering of neural population representation, (iii) temporal evolution of perceptual color memory, and (iv) a non-uniform discrimination threshold, are explained as different aspects of a single model. Moreover, we directly examine key model behaviors in the monkey visual cortex by analyzing neural population dynamics during categorization and discrimination of color stimuli. We find that the categorical task causes temporally-evolving biases in the neuronal population representations toward the focal colors, which supports the proposed model. These results suggest that categorical perception can be achieved by recurrent neural dynamics that approximates optimal probabilistic inference in the changing environment.

[Recent Developments in the Field of Material Perception: Why is Shitsukan Important?].

Shitsukan (material perception) is a cognitive function to infer the material and surface conditions of objects based on sensory information. We can obtain biologically important information on objects through Shitsukan. Shitsukan is also closely associated with preference and emotion toward objects. To understand the mechanisms of Shitsukan, understanding the physical properties of the objects, sensory features generated from the objects, and neural processing in the brain are all important, and interdisciplinary collaborations across engineering science, psychophysics, and brain science is critically important.

Image statistics underlying natural texture selectivity of neurons in macaque V4.

Our daily visual experiences are inevitably linked to recognizing the rich variety of textures. However, how the brain encodes and differentiates a plethora of natural textures remains poorly understood. Here, we show that many neurons in macaque V4 selectively encode sparse combinations of higher-order image statistics to represent natural textures. We systematically explored neural selectivity in a high-dimensional texture space by combining texture synthesis and efficient-sampling techniques. This yielded parameterized models for individual texture-selective neurons. The models provided parsimonious but powerful predictors for each neuron's preferred textures using a sparse combination of image statistics. As a whole population, the neuronal tuning was distributed in a way suitable for categorizing textures and quantitatively predicts human ability to discriminate textures. Together, we suggest that the collective representation of visual image statistics in V4 plays a key role in organizing the natural texture perception.

Effects of luminance contrast on the color selectivity of neurons in the macaque area v4 and inferior temporal cortex.

Appearance of a color stimulus is significantly affected by the contrast between its luminance and the luminance of the background. In the present study, we used stimuli evenly distributed on the CIE-xy chromaticity diagram to examine how luminance contrast affects neural representation of color in V4 and the anterior inferior temporal (AITC) and posterior inferior temporal (PITC) color areas (Banno et al., 2011). The activities of single neurons were recorded from monkeys performing a visual fixation task, and the effects of luminance contrast on the color selectivity of individual neurons and their population responses were systematically examined by comparing responses to color stimuli that were brighter or darker than the background. We found that the effects of luminance contrast differed considerably across V4 and the PITC and AITC. In both V4 and the PITC, the effects of luminance contrast on the population responses of color-selective neurons depended on color. In V4, the size of the effect was largest for blue and cyan, whereas in the PITC, the effect gradually increased as the saturation of the color stimulus was reduced, and was especially large with neutral colors (white, gray, black). The pattern observed in the PITC resembles the effect of luminance contrast on color appearance, suggesting PITC neurons are closely involved in the formation of the perceived appearance of color. By contrast, the color selectivities of AITC neurons were little affected by luminance contrast, indicating that hue and saturation of color stimuli are represented independently of luminance contrast in the AITC.

Perceptual gloss parameters are encoded by population responses in the monkey inferior temporal cortex.

There are neurons localized in the lower bank of the superior temporal sulcus (STS) in the inferior temporal (IT) cortex of the monkey that selectively respond to specific ranges of gloss characterized by combinations of three physical reflectance parameters: specular reflectance (ρs), diffuse reflectance (ρd), and spread of specular reflection (α; Nishio et al., 2012). In the present study, we examined how the activities of these gloss-selective IT neurons are related to perceived gloss. In an earlier psychophysical study, Ferwerda et al. (2001) identified a perceptually uniform gloss space defined by two axes where the c-axis corresponds to a nonlinear combination of ρs and ρd and the d-axis corresponds to 1 - α. In the present study, we tested the responses of gloss-selective neurons to stimuli in the perceptual gloss space defined by the c- and d-axes. We found that gloss-selective neurons systematically changed their responses in the perceptual gloss space, and the distribution of the tuning directions of the population of gloss-selective neurons is biased toward directions in which perceived gloss increases. We also found that a set of perceptual gloss parameters as well as surface albedo can be well explained by the population activities of gloss-selective neurons, and that these parameters are likely encoded by the gloss-selective neurons in this area of the STS to represent various glosses. These results thus provide evidence that the IT cortex represents perceptual gloss space.

Audiovisual integration in the human perception of materials.

Interest in the perception of the material of objects has been growing. While material perception is a critical ability for animals to properly regulate behavioral interactions with surrounding objects (e.g., eating), little is known about its underlying processing. Vision and audition provide useful information for material perception; using only its visual appearance or impact sound, we can infer what an object is made from. However, what material is perceived when the visual appearance of one material is combined with the impact sound of another, and what are the rules that govern cross-modal integration of material information? We addressed these questions by asking 16 human participants to rate how likely it was that audiovisual stimuli (48 combinations of visual appearances of six materials and impact sounds of eight materials) along with visual-only stimuli and auditory-only stimuli fell into each of 13 material categories. The results indicated strong interactions between audiovisual material perceptions; for example, the appearance of glass paired with a pepper sound is perceived as transparent plastic. Rating material-category likelihoods follow a multiplicative integration rule in that the categories judged to be likely are consistent with both visual and auditory stimuli. On the other hand, rating-material properties, such as roughness and hardness, follow a weighted average rule. Despite a difference in their integration calculations, both rules can be interpreted as optimal Bayesian integration of independent audiovisual estimations for the two types of material judgment, respectively.

Representation of the material properties of objects in the visual cortex of nonhuman primates.

Information about the material from which objects are made provide rich and useful clues that enable us to categorize and identify those objects, know their state (e.g., ripeness of fruits), and properly act on them. However, despite its importance, little is known about the neural processes that underlie material perception in nonhuman primates. Here we conducted an fMRI experiment in awake macaque monkeys to explore how information about various real-world materials is represented in the visual areas of monkeys, how these neural representations correlate with perceptual material properties, and how they correspond to those in human visual areas that have been studied previously. Using a machine-learning technique, the representation in each visual area was read out from multivoxel patterns of regional activity elicited in response to images of nine real-world material categories (metal, wood, fur, etc.). The congruence of the neural representations with either a measure of low-level image properties, such as spatial frequency content, or with the visuotactile properties of materials, such as roughness, hardness, and warmness, were tested. We show that monkey V1 shares a common representation with human early visual areas reflecting low-level image properties. By contrast, monkey V4 and the posterior inferior temporal cortex represent the visuotactile properties of material, as in human ventral higher visual areas, although there were some interspecies differences in the representational structures. We suggest that, in monkeys, V4 and the posterior inferior temporal cortex are important stages for constructing information about the material properties of objects from their low-level image features.