Task Context Modulates Feature-Selective Responses in Area V4.

Feature selectivity of visual cortical responses measured during passive fixation provides only a partial view of selectivity because it does not account for the influence of cognitive factors. Here we focus on primate area V4 and ask how neuronal tuning is modulated by task engagement. We investigated whether responses to colored shapes during active shape discrimination are simple, stimulus-agnostic, scaled versions of responses during passive fixation, akin to results from attentional studies. Alternatively, responses could be subject to stimulus-specific scaling, that is, responses to different stimuli are modulated differently, resulting in changes in underlying shape/color selectivity. Among 83 well-isolated V4 neurons in two male macaques, only a minority (16 of 83), which were weakly tuned to both shape and color, displayed responses during fixation and discrimination tasks that could be related by stimulus-agnostic scaling. The majority (67 of 83), which were strongly tuned to shape, color, or both, displayed stimulus-dependent response changes during discrimination. For some of these neurons (39 of 83), the shape or color of the stimulus dictated the magnitude of the change, and for others (28 of 83) it was the combination of stimulus shape and color. Importantly, for neurons with one strong and one weak tuning dimension, stimulus-dependent response changes during discrimination were associated with a relative increase in selectivity along the stronger tuning dimension, without changes in tuning peak. These results reveal that more strongly tuned V4 neurons may also be more flexible in their selectivity, and imbalances in selectivity are amplified during active task contexts.SIGNIFICANCE STATEMENT Tuning for stimulus features is typically characterized by recording responses during passive fixation, but cognitive factors, including attention, influence responses in visual cortex. To determine how behavioral engagement influences neuronal responses in area V4, we compared responses to colored shapes during passive fixation and active behavior. For a large fraction of neurons, differences in responses between passive fixation and active behavior depended on the identity of the visual stimulus. For a subgroup of strongly feature-selective neurons, this response modulation was associated with enhanced selectivity for one feature at the expense of selectivity for the other. Such flexibility in tuning strength could improve performance in tasks requiring active judgment of stimuli.

Testing hemifield independence for divided attention in visual object tasks.

In this study, we asked to what degree hemifields contribute to divided attention effects observed in tasks with object-based judgments. If object recognition processes in the two hemifields were fully independent, then placing stimuli in separate hemifields would eliminate divided attention effects; in the alternative extreme, if object recognition processes in the two hemifields were fully integrated, then placing stimuli in separate hemifields would not modulate divided attention effects. Using a dual-task paradigm, we compared performance in a semantic categorization task for relevant stimuli arranged in the same hemifield to performance for relevant stimuli arranged in separate left and right hemifields. In two experiments, there was a reliable decrease in divided attention effects when stimuli were shown in separate hemifields compared to the same hemifield. However, the effect of divided attention was not eliminated. These results reject both the independent and integrated hypotheses, and instead support a third alternative - that object recognition processes in the two hemifields are partially dependent. More specifically, the magnitude of modulation by hemifields was closer to the prediction of the integrated hypothesis, suggesting that for dual tasks with objects, dependent processing is mostly shared across the visual field.

Davida's deficits: weak encoding of impoverished stimuli or faulty egocentric representation?

Vannuscorps and colleagues present the fascinating case of Davida, a young person who makes systematic errors in judgments related to orientations of sharp or high-contrast visual stimuli. In this commentary, we discuss the findings in the context of observations from mid-level ventral visual stream physiology. We propose two additional interpretations for the specificity of the behavioural deficits: the observed impairments in orientation judgments may be consistent with a system that is not able to unambiguously represent certain impoverished stimuli, or with a system that is not able to translate visual input into head- or body-centered coordinates. Davida's case offers a unique glimpse into the complex cascade of transformations that enable accurate orientation judgments, and sparks curiosity about which mechanistic disruptions can produce such specific unstable percepts.

Is there a serial bottleneck in visual object recognition?

Divided attention has little effect for simple tasks, such as luminance detection, but it has large effects for complex tasks, such as semantic categorization of masked words. Here, we asked whether the semantic categorization of visual objects shows divided attention effects as large as those observed for words, or as small as those observed for simple feature judgments. Using a dual-task paradigm with nameable object stimuli, performance was compared with the predictions of serial and parallel models. At the extreme, parallel processes with unlimited capacity predict no effect of divided attention; alternatively, an all-or-none serial process makes two predictions: a large divided attention effect (lower accuracy for dual-task trials, compared to single-task trials) and a negative response correlation in dual-task trials (a given response is more likely to be incorrect when the response about the other stimulus is correct). These predictions were tested in two experiments examining object judgments. In both experiments, there was a large divided attention effect and a small negative correlation in responses. The magnitude of these effects was larger than for simple features, but smaller than for words. These effects were consistent with serial models, and rule out some but not all parallel models. More broadly, the results help establish one of the first examples of likely serial processing in perception.

Timing of response onset and offset in macaque V4: stimulus and task dependence.

In the primate visual cortex, both the magnitude of the neuronal response and its timing can carry important information about the visual world, but studies typically focus only on response magnitude. Here, we examine the onset and offset latency of the responses of neurons in area V4 of awake, behaving macaques across several experiments in the context of a variety of stimuli and task paradigms. Our results highlight distinct contributions of stimuli and tasks to V4 response latency. We found that response onset latencies are shorter than typically cited (median = 75.5 ms), supporting a role for V4 neurons in rapid object and scene recognition functions. Moreover, onset latencies are longer for smaller stimuli and stimulus outlines, consistent with the hypothesis that longer latencies are associated with higher spatial frequency content. Strikingly, we found that onset latencies showed no significant dependence on stimulus occlusion, unlike in inferotemporal cortex, nor on task demands. Across the V4 population, onset latencies had a broad distribution, reflecting the diversity of feedforward, recurrent, and feedback connections that inform the responses of individual neurons. Response offset latencies, on the other hand, displayed the opposite tendency in their relationship to stimulus and task attributes: they are less influenced by stimulus appearance but are shorter in guided saccade tasks compared with fixation tasks. The observation that response latency is influenced by stimulus- and task-associated factors emphasizes a need to examine response timing alongside firing rate in determining the functional role of area V4.NEW & NOTEWORTHY Onset and offset timing of neuronal responses can provide information about visual environment and neuron's role in visual processing and its anatomical connectivity. In the first comprehensive examination of onset and offset latencies in the intermediate visual cortical area V4, we find neurons respond faster than previously reported, making them ideally suited to contribute to rapid object and scene recognition. While response onset reflects stimulus characteristics, timing of response offset is influenced more by behavioral task.

Modeling diverse responses to filled and outline shapes in macaque V4.

Visual area V4 is an important midlevel cortical processing stage that subserves object recognition in primates. Studies investigating shape coding in V4 have largely probed neuronal responses with filled shapes, i.e., shapes defined by both a boundary and an interior fill. As a result, we do not know whether form-selective V4 responses are dictated by boundary features alone or if interior fill is also important. We studied 43 V4 neurons in two male macaque monkeys ( Macaca mulatta) with a set of 362 filled shapes and their corresponding outlines to determine how interior fill modulates neuronal responses in shape-selective neurons. Only a minority of neurons exhibited similar response strength and shape preferences for filled and outline stimuli. A majority responded preferentially to one stimulus category (either filled or outline shapes) and poorly to the other. Our findings are inconsistent with predictions of the hierarchical-max (HMax) V4 model that builds form selectivity from oriented boundary features and takes little account of attributes related to object surface, such as the phase of the boundary edge. We modified the V4 HMax model to include sensitivity to interior fill by either removing phase-pooling or introducing unoriented units at the V1 level; both modifications better explained our data without increasing the number of free parameters. Overall, our results suggest that boundary orientation and interior surface information are both maintained until at least the midlevel visual representation, consistent with the idea that object fill is important for recognition and perception in natural vision. NEW & NOTEWORTHY The shape of an object's boundary is critical for identification; consistent with this idea, models of object recognition predict that filled and outline versions of a shape are encoded similarly. We report that many neurons in a midlevel visual cortical area respond differently to filled and outline shapes and modify a biologically plausible model to account for our data. Our results suggest that representations of boundary shape and surface fill are interrelated in visual cortex.

Visual Functions of Primate Area V4.

Area V4-the focus of this review-is a mid-level processing stage along the ventral visual pathway of the macaque monkey. V4 is extensively interconnected with other visual cortical areas along the ventral and dorsal visual streams, with frontal cortical areas, and with several subcortical structures. Thus, it is well poised to play a broad and integrative role in visual perception and recognition-the functional domain of the ventral pathway. Neurophysiological studies in monkeys engaged in passive fixation and behavioral tasks suggest that V4 responses are dictated by tuning in a high-dimensional stimulus space defined by form, texture, color, depth, and other attributes of visual stimuli. This high-dimensional tuning may underlie the development of object-based representations in the visual cortex that are critical for tracking, recognizing, and interacting with objects. Neurophysiological and lesion studies also suggest that V4 responses are important for guiding perceptual decisions and higher-order behavior.

Object shape and surface properties are jointly encoded in mid-level ventral visual cortex.

Recognizing a myriad visual objects rapidly is a hallmark of the primate visual system. Traditional theories of object recognition have focused on how crucial form features, for example, the orientation of edges, may be extracted in early visual cortex and utilized to recognize objects. An alternative view argues that much of early and mid-level visual processing focuses on encoding surface characteristics, for example, texture. Neurophysiological evidence from primate area V4 supports a third alternative - the joint, but independent, encoding of form and texture - that would be advantageous for segmenting objects from the background in natural scenes and for object recognition that is independent of surface texture. Future studies that leverage deep convolutional network models, especially focusing on network failures to match biology and behavior, can advance our insights into how such a joint representation of form and surface properties might emerge in visual cortex.