Local field potential responses associated with perceptual filling-in at blind spot in macaque primary visual cortex.

To understand the neural mechanisms of perceptual filling-in at the blind spot (BS), we analyzed neural activity in the region representing the visual field corresponding to the BS (BS region) in the primary visual cortex (V1) of the macaque monkey. We inserted a linear array electrode into the BS region or surrounding region and recorded the multiunit activities (MUAs) and local field potential (LFP). We examined the responses of MUAs and LFP to a large visual stimulus that entirely covered the BS (surface stimuli) while the monkey performed a visual fixation task in either the monocular condition without receiving direct retinal input or the binocular condition receiving retinal information. We observed clear MUA responses in the deep layers within the BS region under monocular conditions, confirming previous reports that V1 neurons in the BS region are activated when perceptual filling-in occurs. Current source density analysis using LFP showed that MUA responses were mainly observed in layer 5. Although LFP responses were generally stronger in the binocular condition than in the monocular condition, a notable exception was observed in the BS region. LFP responses in the low-beta band in the superficial layers were stronger in the monocular condition than in the binocular condition. These results suggest that low-beta activity in the superficial layer is related to the occurrence of perceptual filling-in in the BS. The origin of this activity is considered to be feedback signals from the extrastriate areas to the V1.NEW & NOTEWORTHY Two characteristic activities were induced in the blind spot (BS) region in response to the stimulus, causing perceptual filling-in: 1) beta-band LFP responses in the superficial layers and 2) neuronal responses in the deep layers, mainly in layer 5. These data suggest that the feedback signal from the extrastriate areas to the BS region in V1 is involved in perceptual filling-in.

Dependence of behavioral performance on material category in an object-grasping task with monkeys.

Material perception is an essential part of our cognitive function that enables us to properly interact with our complex daily environment. One important aspect of material perception is its multimodal nature. When we see an object, we generally recognize its haptic properties as well as its visual properties. Consequently, one must examine behavior using real objects that are perceived both visually and haptically to fully understand the characteristics of material perception. As a first step, we examined whether there is any difference in the behavioral responses to different materials in monkeys trained to perform an object grasping task in which they saw and grasped rod-shaped real objects made of various materials. We found that the monkeys' behavior in the grasping task, which was measured based on the success rate and the pulling force, differed depending on the material category. Monkeys easily and correctly grasped objects of some materials, such as metal and glass, but failed to grasp objects of other materials. In particular, monkeys avoided grasping fur-covered objects. The differences in the behavioral responses to the material categories cannot be explained solely based on the degree of familiarity with the different materials. These results shed light on the organization of multimodal representation of materials, where their biological significance is an important factor. In addition, a monkey that avoided touching real fur-covered objects readily touched images of the same objects presented on a CRT display. This suggests that employing real objects is important when studying behaviors related to material perception. NEW & NOTEWORTHY We tested monkeys using an object-grasping task in which monkeys saw and grasped rod-shaped real objects made of various materials. We found that the monkeys' behavior differed dramatically across the material categories and that the behavioral differences could not be explained solely based on the degree of familiarity with the different materials. These results shed light on the organization of multimodal representation of materials, where the biological significance of materials is an important factor.

Gradual Development of Visual Texture-Selective Properties Between Macaque Areas V2 and V4.

Complex shape and texture representations are known to be constructed from V1 along the ventral visual pathway through areas V2 and V4, but the underlying mechanism remains elusive. Recent study suggests that, for processing of textures, a collection of higher-order image statistics computed by combining V1-like filter responses serves as possible representations of textures both in V2 and V4. Here, to gain a clue for how these image statistics are processed in the extrastriate visual areas, we compared neuronal responses to textures in V2 and V4 of macaque monkeys. For individual neurons, we adaptively explored their preferred textures from among thousands of naturalistic textures and fitted the obtained responses using a combination of V1-like filter responses and higher-order statistics. We found that, while the selectivity for image statistics was largely comparable between V2 and V4, V4 showed slightly stronger sensitivity to the higher-order statistics than V2. Consistent with that finding, V4 responses were reduced to a greater extent than V2 responses when the monkeys were shown spectrally matched noise images that lacked higher-order statistics. We therefore suggest that there is a gradual development in representation of higher-order features along the ventral visual hierarchy.

Image statistics underlying natural texture selectivity of neurons in macaque V4.

Our daily visual experiences are inevitably linked to recognizing the rich variety of textures. However, how the brain encodes and differentiates a plethora of natural textures remains poorly understood. Here, we show that many neurons in macaque V4 selectively encode sparse combinations of higher-order image statistics to represent natural textures. We systematically explored neural selectivity in a high-dimensional texture space by combining texture synthesis and efficient-sampling techniques. This yielded parameterized models for individual texture-selective neurons. The models provided parsimonious but powerful predictors for each neuron's preferred textures using a sparse combination of image statistics. As a whole population, the neuronal tuning was distributed in a way suitable for categorizing textures and quantitatively predicts human ability to discriminate textures. Together, we suggest that the collective representation of visual image statistics in V4 plays a key role in organizing the natural texture perception.

Similarity in Neuronal Firing Regimes across Mammalian Species.

UNLABELLED: The architectonic subdivisions of the brain are believed to be functional modules, each processing parts of global functions. Previously, we showed that neurons in different regions operate in different firing regimes in monkeys. It is possible that firing regimes reflect differences in underlying information processing, and consequently the firing regimes in homologous regions across animal species might be similar. We analyzed neuronal spike trains recorded from behaving mice, rats, cats, and monkeys. The firing regularity differed systematically, with differences across regions in one species being greater than the differences in similar areas across species. Neuronal firing was consistently most regular in motor areas, nearly random in visual and prefrontal/medial prefrontal cortical areas, and bursting in the hippocampus in all animals examined. This suggests that firing regularity (or irregularity) plays a key role in neural computation in each functional subdivision, depending on the types of information being carried. SIGNIFICANCE STATEMENT: By analyzing neuronal spike trains recorded from mice, rats, cats, and monkeys, we found that different brain regions have intrinsically different firing regimes that are more similar in homologous areas across species than across areas in one species. Because different regions in the brain are specialized for different functions, the present finding suggests that the different activity regimes of neurons are important for supporting different functions, so that appropriate neuronal codes can be used for different modalities.

Comparison of the color selectivity of macaque V4 neurons in different color spaces.

Chromatic selectivity has been studied extensively in various visual areas at different stages of visual processing in the macaque brain. In these studies, color stimuli defined in the Derrington-Krauskopf-Lennie (DKL) color space with a limited range of cone contrast were typically used in early stages, whereas those defined in the Commission Internationale de l'Eclairage (CIE) color space, based on human psychophysical measurements across the gamut of the display, were often used in higher visual areas. To understand how the color information is processed along the visual pathway, it is necessary to compare color selectivity obtained in different areas on a common color space. In the present study, we tested whether the neural color selectivity obtained in DKL space can be predicted from responses obtained in CIE space and whether stimuli with limited cone contrast are sufficient to characterize neural color selectivity. We found that for most V4 neurons, there was a strong correlation between responses measured using the two chromatic coordinate systems, and the color selectivities obtained with the two stimulus sets were comparable. However, for some neurons preferring high- or low-saturation colors, stimuli defined in DKL color space did not adequately capture the neural color selectivity. This is mainly due to the use of stimuli within a limited range of cone contrast. We conclude that regardless of the choice of color space, the sampling of colors across the entire gamut is important to characterize neural color selectivity fully or to compare color selectivities in different areas so as to understand color representation in the visual system.

Effects of luminance contrast on the color selectivity of neurons in the macaque area v4 and inferior temporal cortex.

Appearance of a color stimulus is significantly affected by the contrast between its luminance and the luminance of the background. In the present study, we used stimuli evenly distributed on the CIE-xy chromaticity diagram to examine how luminance contrast affects neural representation of color in V4 and the anterior inferior temporal (AITC) and posterior inferior temporal (PITC) color areas (Banno et al., 2011). The activities of single neurons were recorded from monkeys performing a visual fixation task, and the effects of luminance contrast on the color selectivity of individual neurons and their population responses were systematically examined by comparing responses to color stimuli that were brighter or darker than the background. We found that the effects of luminance contrast differed considerably across V4 and the PITC and AITC. In both V4 and the PITC, the effects of luminance contrast on the population responses of color-selective neurons depended on color. In V4, the size of the effect was largest for blue and cyan, whereas in the PITC, the effect gradually increased as the saturation of the color stimulus was reduced, and was especially large with neutral colors (white, gray, black). The pattern observed in the PITC resembles the effect of luminance contrast on color appearance, suggesting PITC neurons are closely involved in the formation of the perceived appearance of color. By contrast, the color selectivities of AITC neurons were little affected by luminance contrast, indicating that hue and saturation of color stimuli are represented independently of luminance contrast in the AITC.

Representation of the material properties of objects in the visual cortex of nonhuman primates.

Information about the material from which objects are made provide rich and useful clues that enable us to categorize and identify those objects, know their state (e.g., ripeness of fruits), and properly act on them. However, despite its importance, little is known about the neural processes that underlie material perception in nonhuman primates. Here we conducted an fMRI experiment in awake macaque monkeys to explore how information about various real-world materials is represented in the visual areas of monkeys, how these neural representations correlate with perceptual material properties, and how they correspond to those in human visual areas that have been studied previously. Using a machine-learning technique, the representation in each visual area was read out from multivoxel patterns of regional activity elicited in response to images of nine real-world material categories (metal, wood, fur, etc.). The congruence of the neural representations with either a measure of low-level image properties, such as spatial frequency content, or with the visuotactile properties of materials, such as roughness, hardness, and warmness, were tested. We show that monkey V1 shares a common representation with human early visual areas reflecting low-level image properties. By contrast, monkey V4 and the posterior inferior temporal cortex represent the visuotactile properties of material, as in human ventral higher visual areas, although there were some interspecies differences in the representational structures. We suggest that, in monkeys, V4 and the posterior inferior temporal cortex are important stages for constructing information about the material properties of objects from their low-level image features.

Perceptual gloss parameters are encoded by population responses in the monkey inferior temporal cortex.

There are neurons localized in the lower bank of the superior temporal sulcus (STS) in the inferior temporal (IT) cortex of the monkey that selectively respond to specific ranges of gloss characterized by combinations of three physical reflectance parameters: specular reflectance (ρs), diffuse reflectance (ρd), and spread of specular reflection (α; Nishio et al., 2012). In the present study, we examined how the activities of these gloss-selective IT neurons are related to perceived gloss. In an earlier psychophysical study, Ferwerda et al. (2001) identified a perceptually uniform gloss space defined by two axes where the c-axis corresponds to a nonlinear combination of ρs and ρd and the d-axis corresponds to 1 - α. In the present study, we tested the responses of gloss-selective neurons to stimuli in the perceptual gloss space defined by the c- and d-axes. We found that gloss-selective neurons systematically changed their responses in the perceptual gloss space, and the distribution of the tuning directions of the population of gloss-selective neurons is biased toward directions in which perceived gloss increases. We also found that a set of perceptual gloss parameters as well as surface albedo can be well explained by the population activities of gloss-selective neurons, and that these parameters are likely encoded by the gloss-selective neurons in this area of the STS to represent various glosses. These results thus provide evidence that the IT cortex represents perceptual gloss space.

Audiovisual integration in the human perception of materials.

Interest in the perception of the material of objects has been growing. While material perception is a critical ability for animals to properly regulate behavioral interactions with surrounding objects (e.g., eating), little is known about its underlying processing. Vision and audition provide useful information for material perception; using only its visual appearance or impact sound, we can infer what an object is made from. However, what material is perceived when the visual appearance of one material is combined with the impact sound of another, and what are the rules that govern cross-modal integration of material information? We addressed these questions by asking 16 human participants to rate how likely it was that audiovisual stimuli (48 combinations of visual appearances of six materials and impact sounds of eight materials) along with visual-only stimuli and auditory-only stimuli fell into each of 13 material categories. The results indicated strong interactions between audiovisual material perceptions; for example, the appearance of glass paired with a pepper sound is perceived as transparent plastic. Rating material-category likelihoods follow a multiplicative integration rule in that the categories judged to be likely are consistent with both visual and auditory stimuli. On the other hand, rating-material properties, such as roughness and hardness, follow a weighted average rule. Despite a difference in their integration calculations, both rules can be interpreted as optimal Bayesian integration of independent audiovisual estimations for the two types of material judgment, respectively.

Neural selectivity and representation of gloss in the monkey inferior temporal cortex.

When we view an object, its appearance depends in large part on specific surface reflectance properties; among these is surface gloss, which provides important information about the material composition of the object and the fine structure of its surface. To study how gloss is represented in the visual cortical areas related to object recognition, we examined the responses of neurons in the inferior temporal (IT) cortex of the macaque monkey to a set of object images exhibiting various combinations of specular reflection, diffuse reflection, and roughness, which are important physical parameters of surface gloss. We found that there are neurons in the lower bank of the superior temporal sulcus that selectively respond to specific gloss. This neuronal selectivity was largely maintained when the shape or illumination of the object was modified and perceived glossiness was unchanged. By contrast, neural responses were significantly altered when the pixels of the images were randomly rearranged, and perceived glossiness was dramatically changed. The stimulus preference of these neurons differed from cell to cell, and, as a population, they systematically represented a variety of surface glosses. We conclude that, within the visual cortex, there are mechanisms operating to integrate local image features and extract information about surface gloss and that this information is systematically represented in the IT cortex, an area playing an important role in object recognition.

Color vision test for dichromatic and trichromatic macaque monkeys.

Dichromacy is a color vision defect in which one of the three cone photoreceptors is absent. Individuals with dichromacy are called dichromats (or sometimes "color-blind"), and their color discrimination performance has contributed significantly to our understanding of color vision. Macaque monkeys, which normally have trichromatic color vision that is nearly identical to humans, have been used extensively in neurophysiological studies of color vision. In the present study we employed two tests, a pseudoisochromatic color discrimination test and a monochromatic light detection test, to compare the color vision of genetically identified dichromatic macaques (Macaca fascicularis) with that of normal trichromatic macaques. In the color discrimination test, dichromats could not discriminate colors along the protanopic confusion line, though trichromats could. In the light detection test, the relative thresholds for longer wavelength light were higher in the dichromats than the trichromats, indicating dichromats to be less sensitive to longer wavelength light. Because the dichromatic macaque is very rare, the present study provides valuable new information on the color vision behavior of dichromatic macaques, which may be a useful animal model of human dichromacy. The behavioral tests used in the present study have been previously used to characterize the color behaviors of trichromatic as well as dichromatic new world monkeys. The present results show that comparative studies of color vision employing similar tests may be feasible to examine the difference in color behaviors between trichromatic and dichromatic individuals, although the genetic mechanisms of trichromacy/dichromacy is quite different between new world monkeys and macaques.

Selective responses to specular surfaces in the macaque visual cortex revealed by fMRI.

The surface properties of objects, such as gloss, transparency and texture, provide important information about the material characteristics of objects in our visual environment. However, because there have been few reports on the neuronal responses to surface properties in primates, we still lack information about where and how surface properties are processed in the primate visual cortex. In this study, we used functional magnetic resonance imaging (fMRI) to examine the cortical responses to specular surfaces in the macaque visual cortex. Using computer graphics, we generated images of specular and matte objects and prepared scrambled images by locally randomizing the luminance phases of the images with specular and matte objects. In experiment 1, we contrasted the responses to specular images with those to matte and scrambled images. Activation was observed along the ventral visual pathway, including V1, V2, V3, V4 and the posterior inferior temporal (IT) cortex. In experiment 2, we manipulated the contrasts of images and found that the activation observed in these regions could not be explained solely by the global or local contrasts. These results suggest that image features related to specular surface are processed along the ventral visual pathway from V1 to specific regions in the IT cortex. This is consistent with previous human fMRI experiments that showed surface properties are processed in the ventral visual pathway.

Reciprocal connectivity of identified color-processing modules in the monkey inferior temporal cortex.

The inferior temporal (IT) cortex is the last unimodal visual area in the ventral visual pathway and is essential for color discrimination. Recent imaging and electrophysiological studies have revealed the presence of several distinct patches of color-selective cells in the anterior IT cortex (AIT) and posterior IT cortex (PIT). To understand the neural machinery for color processing in the IT cortex, in the present study, we combined anatomical tracing methods with electrophysiological unit recordings to investigate the anatomical connections of identified clusters of color-selective cells in monkey IT cortex. We found that a color cluster in AIT received projections from a color cluster in PIT as well as from discrete clusters of cells in other occipitotemporal areas, in the superior temporal sulcus, and in prefrontal and parietal cortices. The distribution of the labeled cells in PIT closely corresponded with that of the physiologically identified color-selective cells in this region. Furthermore, retrograde tracer injections in the posterior color cluster resulted in labeled cells in the anterior cluster. Thus, temporal lobe color-processing modules form a reciprocally interconnected loop within a distributed network.

Origin of the ease of association of color names: Comparison between humans and AI.

Rapid evolution of artificial intelligence (AI) based on deep neural networks has resulted in artificial systems such as generative pre-trained transformer 3 (GPT-3), which can generate human-like language. Such a system may provide a novel platform for studying how human perception is related to knowledge and the ability of language generation. We compared the frequency distribution of basic color terms in the answers of human subjects and GPT-3 when both were asked similar questions regarding color names associated with the letters of the alphabet. We found that GPT-3 generated basic color terms at a frequency very similar to that of human non-synaesthetes. A similar frequency was observed when color names associated with numerals were tested indicating that simple co-occurrence of alphabet and color word in the trained dataset cannot explain the results. We suggest that the proposed experimental framework using the latest AI models has the potential to explore the mechanisms of human perception.

Transformation from image-based to perceptual representation of materials along the human ventral visual pathway.

Every object in the world has its own surface quality that is a reflection of the material from which the object is made. We can easily identify and categorize materials (wood, metal, fabric etc.) at a glance, and this ability enables us to decide how to interact appropriately with these objects. Little is known, however, about how materials are represented in the brain, or how that representation is related to material perception or the physical properties of material surface. By combining multivoxel pattern analysis of functional magnetic resonance imaging data with perceptual and image-based physical measures of material properties, we found that the way visual information about materials is coded gradually changes from an image-based representation in early visual areas to a perceptual representation in the ventral higher-order visual areas. We suggest that meaningful information about multimodal aspects of real-world materials reside in the ventral cortex around the fusiform gyrus, where it can be utilized for categorization of materials.

Color selectivity of neurons in the posterior inferior temporal cortex of the macaque monkey.

We recorded the activities of neurons in the lateral surface of the posterior inferior temporal cortex (PIT) of 3 hemispheres of 3 monkeys performing a visual fixation task. We characterized the color and shape selectivities of each neuron, mapped its receptive field (RF), and studied the distributions of these response properties. Using a set of color stimuli that were systematically distributed in Commission Internationale de l'Eclairage-xy chromaticity diagram, we found numerous color-selective neurons distributed throughout the area examined. Neurons in the ventral region tended to have sharper color tuning than those in the dorsal region. We also found a crude retinotopic organization in the ventral region. Within the ventral region of PIT, neurons in the dorsal part had RFs that overlapped the foveal center; the eccentricity of RFs increased in the more ventral part, and neurons in the anterior and posterior parts had RFs that represented the lower and upper visual fields, respectively. In all 3 hemispheres, the region where sharply tuned color-selective neurons were concentrated was confined within this retinotopic map. These findings suggest that PIT is a heterogeneous area and that there is a circumscribed region within it that has crude retinotopic organization and is involved in the processing of color.

Categorical properties of the color term "GOLD".

Humans are able to categorize an infinite variety of surface colors into a small number of color terms. Previous studies have shown that 11 basic color terms are commonly used in fully developed languages. These studies usually used flat matte color plates as stimuli, but we can also perceive the colors of glossy surfaces by discounting the effect of the gloss. However, color terms such as GOLD and SILVER are specifically associated with glossy surfaces. In this study, we conducted a categorical color-naming task to examine whether the color terms GOLD and SILVER could be located in a stimulus space defined by combining CIE xy chromaticity coordinates and surface reflectance and whether they had categorical properties like ordinary basic color terms. We found that GOLD and SILVER were used for specific ranges of chromaticities with stimuli having large specular reflectances. Moreover, the strengths of the categorical properties, as assessed using measures of consistency, consensus, and reaction time, were comparable to those of the basic color terms, indicating that GOLD and SILVER are categorical color terms specifically associated with glossy surfaces. This also indicates that humans do not always discount surface gloss to identify colors but can utilize this information to categorize colors.

Effects of task demands on the responses of color-selective neurons in the inferior temporal cortex.

Categorization and fine discrimination are two different functions in visual perception, and we can switch between these two functions depending on the situation or task demands. To explore how visual cortical neurons behave in such situations, we recorded the activities of color-selective neurons in the inferior temporal (IT) cortex of two monkeys trained to perform a color categorization task, a color discrimination task and a simple fixation task. Many IT neurons changed their activity depending upon the task, although color selectivity was well conserved. A majority of neurons showed stronger responses during the categorization task. Moreover, for the population of IT neurons as a whole, signals contributing to performing the categorization task were enhanced. These results imply that judgment of color category by color-selective IT neurons is facilitated during the categorization task and suppressed during the discrimination task as a consequence of task-dependent modulation of their activities.

Structure and function of neural circuit related to gloss perception in the macaque inferior temporal cortex: a case report.

The inferior temporal (IT) cortex of the macaque monkey plays a pivotal role in the visual recognition of objects. In the IT cortex, a feature-selective network formed by connecting subregions specialized for common visual features seems to be a basic strategy for processing biologically important visual features. Gloss perception plays an important role in the judgment of materials and conditions of objects and is a biologically significant visual function. In the present study, we attempted to determine whether a neural circuit specialized for processing information related to gloss perception exists in the IT cortex in one monkey. We injected retrograde tracer into a gloss-selective subregion in the IT cortex where gloss-selective neurons were clustered in the neural recording experiment, and anatomically examined its neural connections. We observed that retrogradely labeled neurons were densely accumulated in multiple locations in the posterior and anterior IT cortices. Based on the results of this case study, we will discuss the possibility that, together with the injection site, the sites with a dense cluster of labeled neurons form feature-selective neural circuits for the processing of gloss information in the IT cortex.