Decision uncertainty as a context for motor memory.

The current view of perceptual decision-making suggests that once a decision is made, only a single motor programme associated with the decision is carried out, irrespective of the uncertainty involved in decision making. In contrast, we show that multiple motor programmes can be acquired on the basis of the preceding uncertainty of the decision, indicating that decision uncertainty functions as a contextual cue for motor memory. The actions learned after making certain (uncertain) decisions are only partially transferred to uncertain (certain) decisions. Participants were able to form distinct motor memories for the same movement on the basis of the preceding decision uncertainty. Crucially, this contextual effect generalizes to novel stimuli with matched uncertainty levels, demonstrating that decision uncertainty is itself a contextual cue. These findings broaden the understanding of contextual inference in motor memory, emphasizing that it extends beyond direct motor control cues to encompass the decision-making process.

Neural Mechanisms That Make Perceptual Decisions Flexible.

Neural mechanisms of perceptual decision making have been extensively studied in experimental settings that mimic stable environments with repeating stimuli, fixed rules, and payoffs. In contrast, we live in an ever-changing environment and have varying goals and behavioral demands. To accommodate variability, our brain flexibly adjusts decision-making processes depending on context. Here, we review a growing body of research that explores the neural mechanisms underlying this flexibility. We highlight diverse forms of context dependency in decision making implemented through a variety of neural computations. Context-dependent neural activity is observed in a distributed network of brain structures, including posterior parietal, sensory, motor, and subcortical regions, as well as the prefrontal areas classically implicated in cognitive control. We propose that investigating the distributed network underlying flexible decisions is key to advancing our understanding and discuss a path forward for experimental and theoretical investigations.

Linear Integration of Sensory Evidence over Space and Time Underlies Face Categorization.

Visual object recognition relies on elaborate sensory processes that transform retinal inputs to object representations, but it also requires decision-making processes that read out object representations and function over prolonged time scales. The computational properties of these decision-making processes remain underexplored for object recognition. Here, we study these computations by developing a stochastic multifeature face categorization task. Using quantitative models and tight control of spatiotemporal visual information, we demonstrate that human subjects (five males, eight females) categorize faces through an integration process that first linearly adds the evidence conferred by task-relevant features over space to create aggregated momentary evidence and then linearly integrates it over time with minimum information loss. Discrimination of stimuli along different category boundaries (e.g., identity or expression of a face) is implemented by adjusting feature weights of spatial integration. This linear but flexible integration process over space and time bridges past studies on simple perceptual decisions to complex object recognition behavior.SIGNIFICANCE STATEMENT Although simple perceptual decision-making such as discrimination of random dot motion has been successfully explained as accumulation of sensory evidence, we lack rigorous experimental paradigms to study the mechanisms underlying complex perceptual decision-making such as discrimination of naturalistic faces. We develop a stochastic multifeature face categorization task as a systematic approach to quantify the properties and potential limitations of the decision-making processes during object recognition. We show that human face categorization could be modeled as a linear integration of sensory evidence over space and time. Our framework to study object recognition as a spatiotemporal integration process is broadly applicable to other object categories and bridges past studies of object recognition and perceptual decision-making.

Representational geometry of perceptual decisions in the monkey parietal cortex.

Lateral intraparietal (LIP) neurons represent formation of perceptual decisions involving eye movements. In circuit models for these decisions, neural ensembles that encode actions compete to form decisions. Consequently, representation and readout of the decision variables (DVs) are implemented similarly for decisions with identical competing actions, irrespective of input and task context differences. Further, DVs are encoded as partially potentiated action plans through balance of activity of action-selective ensembles. Here, we test those core principles. We show that in a novel face-discrimination task, LIP firing rates decrease with supporting evidence, contrary to conventional motion-discrimination tasks. These opposite response patterns arise from similar mechanisms in which decisions form along curved population-response manifolds misaligned with action representations. These manifolds rotate in state space based on context, indicating distinct optimal readouts for different tasks. We show similar manifolds in lateral and medial prefrontal cortices, suggesting similar representational geometry across decision-making circuits.

Designing and Interpreting Psychophysical Investigations of Cognition.

Scientific experimentation depends on the artificial control of natural phenomena. The inaccessibility of cognitive processes to direct manipulation can make such control difficult to realize. Here, we discuss approaches for overcoming this challenge. We advocate the incorporation of experimental techniques from sensory psychophysics into the study of cognitive processes such as decision making and executive control. These techniques include the use of simple parameterized stimuli to precisely manipulate available information and computational models to jointly quantify behavior and neural responses. We illustrate the potential for such techniques to drive theoretical development, and we examine important practical details of how to conduct controlled experiments when using them. Finally, we highlight principles guiding the use of computational models in studying the neural basis of cognition.

Psychophysical reverse correlation reflects both sensory and decision-making processes.

Goal-directed behavior depends on both sensory mechanisms that gather information from the outside world and decision-making mechanisms that select appropriate behavior based on that sensory information. Psychophysical reverse correlation is commonly used to quantify how fluctuations of sensory stimuli influence behavior and is generally believed to uncover the spatiotemporal weighting functions of sensory processes. Here we show that reverse correlations also reflect decision-making processes and can deviate significantly from the true sensory filters. Specifically, changes of decision bound and mechanisms of evidence integration systematically alter psychophysical reverse correlations. Similarly, trial-to-trial variability of sensory and motor delays and decision times causes systematic distortions in psychophysical kernels that should not be attributed to sensory mechanisms. We show that ignoring details of the decision-making process results in misinterpretation of reverse correlations, but proper use of these details turns reverse correlation into a powerful method for studying both sensory and decision-making mechanisms.

Gradual Development of Visual Texture-Selective Properties Between Macaque Areas V2 and V4.

Complex shape and texture representations are known to be constructed from V1 along the ventral visual pathway through areas V2 and V4, but the underlying mechanism remains elusive. Recent study suggests that, for processing of textures, a collection of higher-order image statistics computed by combining V1-like filter responses serves as possible representations of textures both in V2 and V4. Here, to gain a clue for how these image statistics are processed in the extrastriate visual areas, we compared neuronal responses to textures in V2 and V4 of macaque monkeys. For individual neurons, we adaptively explored their preferred textures from among thousands of naturalistic textures and fitted the obtained responses using a combination of V1-like filter responses and higher-order statistics. We found that, while the selectivity for image statistics was largely comparable between V2 and V4, V4 showed slightly stronger sensitivity to the higher-order statistics than V2. Consistent with that finding, V4 responses were reduced to a greater extent than V2 responses when the monkeys were shown spectrally matched noise images that lacked higher-order statistics. We therefore suggest that there is a gradual development in representation of higher-order features along the ventral visual hierarchy.

Image statistics underlying natural texture selectivity of neurons in macaque V4.

Our daily visual experiences are inevitably linked to recognizing the rich variety of textures. However, how the brain encodes and differentiates a plethora of natural textures remains poorly understood. Here, we show that many neurons in macaque V4 selectively encode sparse combinations of higher-order image statistics to represent natural textures. We systematically explored neural selectivity in a high-dimensional texture space by combining texture synthesis and efficient-sampling techniques. This yielded parameterized models for individual texture-selective neurons. The models provided parsimonious but powerful predictors for each neuron's preferred textures using a sparse combination of image statistics. As a whole population, the neuronal tuning was distributed in a way suitable for categorizing textures and quantitatively predicts human ability to discriminate textures. Together, we suggest that the collective representation of visual image statistics in V4 plays a key role in organizing the natural texture perception.

Effects of luminance contrast on the color selectivity of neurons in the macaque area v4 and inferior temporal cortex.

Appearance of a color stimulus is significantly affected by the contrast between its luminance and the luminance of the background. In the present study, we used stimuli evenly distributed on the CIE-xy chromaticity diagram to examine how luminance contrast affects neural representation of color in V4 and the anterior inferior temporal (AITC) and posterior inferior temporal (PITC) color areas (Banno et al., 2011). The activities of single neurons were recorded from monkeys performing a visual fixation task, and the effects of luminance contrast on the color selectivity of individual neurons and their population responses were systematically examined by comparing responses to color stimuli that were brighter or darker than the background. We found that the effects of luminance contrast differed considerably across V4 and the PITC and AITC. In both V4 and the PITC, the effects of luminance contrast on the population responses of color-selective neurons depended on color. In V4, the size of the effect was largest for blue and cyan, whereas in the PITC, the effect gradually increased as the saturation of the color stimulus was reduced, and was especially large with neutral colors (white, gray, black). The pattern observed in the PITC resembles the effect of luminance contrast on color appearance, suggesting PITC neurons are closely involved in the formation of the perceived appearance of color. By contrast, the color selectivities of AITC neurons were little affected by luminance contrast, indicating that hue and saturation of color stimuli are represented independently of luminance contrast in the AITC.

Representation of the material properties of objects in the visual cortex of nonhuman primates.

Information about the material from which objects are made provide rich and useful clues that enable us to categorize and identify those objects, know their state (e.g., ripeness of fruits), and properly act on them. However, despite its importance, little is known about the neural processes that underlie material perception in nonhuman primates. Here we conducted an fMRI experiment in awake macaque monkeys to explore how information about various real-world materials is represented in the visual areas of monkeys, how these neural representations correlate with perceptual material properties, and how they correspond to those in human visual areas that have been studied previously. Using a machine-learning technique, the representation in each visual area was read out from multivoxel patterns of regional activity elicited in response to images of nine real-world material categories (metal, wood, fur, etc.). The congruence of the neural representations with either a measure of low-level image properties, such as spatial frequency content, or with the visuotactile properties of materials, such as roughness, hardness, and warmness, were tested. We show that monkey V1 shares a common representation with human early visual areas reflecting low-level image properties. By contrast, monkey V4 and the posterior inferior temporal cortex represent the visuotactile properties of material, as in human ventral higher visual areas, although there were some interspecies differences in the representational structures. We suggest that, in monkeys, V4 and the posterior inferior temporal cortex are important stages for constructing information about the material properties of objects from their low-level image features.

Color vision test for dichromatic and trichromatic macaque monkeys.

Dichromacy is a color vision defect in which one of the three cone photoreceptors is absent. Individuals with dichromacy are called dichromats (or sometimes "color-blind"), and their color discrimination performance has contributed significantly to our understanding of color vision. Macaque monkeys, which normally have trichromatic color vision that is nearly identical to humans, have been used extensively in neurophysiological studies of color vision. In the present study we employed two tests, a pseudoisochromatic color discrimination test and a monochromatic light detection test, to compare the color vision of genetically identified dichromatic macaques (Macaca fascicularis) with that of normal trichromatic macaques. In the color discrimination test, dichromats could not discriminate colors along the protanopic confusion line, though trichromats could. In the light detection test, the relative thresholds for longer wavelength light were higher in the dichromats than the trichromats, indicating dichromats to be less sensitive to longer wavelength light. Because the dichromatic macaque is very rare, the present study provides valuable new information on the color vision behavior of dichromatic macaques, which may be a useful animal model of human dichromacy. The behavioral tests used in the present study have been previously used to characterize the color behaviors of trichromatic as well as dichromatic new world monkeys. The present results show that comparative studies of color vision employing similar tests may be feasible to examine the difference in color behaviors between trichromatic and dichromatic individuals, although the genetic mechanisms of trichromacy/dichromacy is quite different between new world monkeys and macaques.

Selective responses to specular surfaces in the macaque visual cortex revealed by fMRI.

The surface properties of objects, such as gloss, transparency and texture, provide important information about the material characteristics of objects in our visual environment. However, because there have been few reports on the neuronal responses to surface properties in primates, we still lack information about where and how surface properties are processed in the primate visual cortex. In this study, we used functional magnetic resonance imaging (fMRI) to examine the cortical responses to specular surfaces in the macaque visual cortex. Using computer graphics, we generated images of specular and matte objects and prepared scrambled images by locally randomizing the luminance phases of the images with specular and matte objects. In experiment 1, we contrasted the responses to specular images with those to matte and scrambled images. Activation was observed along the ventral visual pathway, including V1, V2, V3, V4 and the posterior inferior temporal (IT) cortex. In experiment 2, we manipulated the contrasts of images and found that the activation observed in these regions could not be explained solely by the global or local contrasts. These results suggest that image features related to specular surface are processed along the ventral visual pathway from V1 to specific regions in the IT cortex. This is consistent with previous human fMRI experiments that showed surface properties are processed in the ventral visual pathway.

Categorical properties of the color term "GOLD".

Humans are able to categorize an infinite variety of surface colors into a small number of color terms. Previous studies have shown that 11 basic color terms are commonly used in fully developed languages. These studies usually used flat matte color plates as stimuli, but we can also perceive the colors of glossy surfaces by discounting the effect of the gloss. However, color terms such as GOLD and SILVER are specifically associated with glossy surfaces. In this study, we conducted a categorical color-naming task to examine whether the color terms GOLD and SILVER could be located in a stimulus space defined by combining CIE xy chromaticity coordinates and surface reflectance and whether they had categorical properties like ordinary basic color terms. We found that GOLD and SILVER were used for specific ranges of chromaticities with stimuli having large specular reflectances. Moreover, the strengths of the categorical properties, as assessed using measures of consistency, consensus, and reaction time, were comparable to those of the basic color terms, indicating that GOLD and SILVER are categorical color terms specifically associated with glossy surfaces. This also indicates that humans do not always discount surface gloss to identify colors but can utilize this information to categorize colors.