Using High-Density Electroencephalography to Explore Spatiotemporal Representations of Object Categories in Visual Cortex.

Visual object perception involves neural processes that unfold over time and recruit multiple regions of the brain. Here, we use high-density EEG to investigate the spatiotemporal representations of object categories across the dorsal and ventral pathways. In , human participants were presented with images from two animate object categories (birds and insects) and two inanimate categories (tools and graspable objects). In , participants viewed images of tools and graspable objects from a different stimulus set, one in which a shape confound that often exists between these categories (elongation) was controlled for. To explore the temporal dynamics of object representations, we employed time-resolved multivariate pattern analysis on the EEG time series data. This was performed at the electrode level as well as in source space of two regions of interest: one encompassing the ventral pathway and another encompassing the dorsal pathway. Our results demonstrate shape, exemplar, and category information can be decoded from the EEG signal. Multivariate pattern analysis within source space revealed that both dorsal and ventral pathways contain information pertaining to shape, inanimate object categories, and animate object categories. Of particular interest, we note striking similarities obtained in both ventral stream and dorsal stream regions of interest. These findings provide insight into the spatio-temporal dynamics of object representation and contribute to a growing literature that has begun to redefine the traditional role of the dorsal pathway.

Perceived group size is determined by the centroids of the component elements.

To accomplish the deceptively simple task of perceiving the size of objects in the visual scene, the visual system combines information about the retinal size of the object with several other cues, including perceived distance, relative size, and prior knowledge. When local component elements are perceptually grouped to form objects, the task is further complicated because a grouped object does not have a continuous contour from which retinal size can be estimated. Here, we investigate how the visual system solves this problem and makes it possible for observers to judge the size of perceptually grouped objects. We systematically vary the shape and orientation of the component elements in a two-alternative forced-choice task and find that the perceived size of the array of component objects can be almost perfectly predicted from the distance between the centroids of the component elements and the center of the array. This is true whether the global contour forms a circle or a square. When elements were positioned such that the centroids along the global contour were at different distances from the center, perceived size was based on the average distance. These results indicate that perceived size does not depend on the size of individual elements, and that smooth contours formed by the outer edges of the component elements are not used to estimate size. The current study adds to a growing literature highlighting the importance of centroids in visual perception and may have implications for how size is estimated for ensembles of different objects.

The motion-induced contour revisited: Observations on 3-D structure and illusory contour formation in moving stimuli.

The motion-induced contour (MIC) was first described by Victor Klymenko and Naomi Weisstein in a series of papers in the 1980s. The effect is created by rotating the outline of a tilted cube in depth. When one of the vertical edges is removed, an illusory contour can be seen in its place. In four experiments, we explored which stimulus features influence perceived illusory contour strength. Participants provided subjective ratings of illusory contour strength as a function of orientation of the stimulus, separation between inducing edges, and the length of inducing edges. We found that the angle of tilt of the object in depth had the largest impact on perceived illusory contour strength with tilt angles of 20° and 30° producing the strongest percepts. Tilt angle is an unexplored feature of structure-from-motion displays. In addition, we found that once the depth structure of the object was extracted, other features of the display, such as the distance spanned by the illusory contour, could also influence its strength, similar to the notion of support ratio for 2-D illusory contours. Illusory contour strength was better predicted by the length of the contour in 3-D rather than in 2-D, suggesting that MICs are constructed by a 3-D process that takes as input initially recovered contour orientation and position information in depth and only then forms interpolations between them.

Electrophysiological correlates of encoding processes in a full-report visual working memory paradigm.

Why are some visual stimuli remembered, whereas others are forgotten? A limitation of recognition paradigms is that they measure aggregate behavioral performance and/or neural responses to all stimuli presented in a visual working memory (VWM) array. To address this limitation, we paired an electroencephalography (EEG) frequency-tagging technique with two full-report VWM paradigms. This permitted the tracking of individual stimuli as well as the aggregate response. We recorded high-density EEG (256 channel) while participants viewed four shape stimuli, each flickering at a different frequency. At retrieval, participants either recalled the location of all stimuli in any order (simultaneous full report) or were cued to report the item in a particular location over multiple screen displays (sequential full report). The individual frequency tag amplitudes evoked for correctly recalled items were significantly larger than the amplitudes of subsequently forgotten stimuli, regardless of retrieval task. An induced-power analysis examined the aggregate neural correlates of VWM encoding as a function of items correctly recalled. We found increased induced power across a large number of electrodes in the theta, alpha, and beta frequency bands when more items were successfully recalled. This effect was more robust for sequential full report, suggesting that retrieval demands can influence encoding processes. These data are consistent with a model in which encoding-related resources are directed to a subset of items, rather than a model in which resources are allocated evenly across the array. These data extend previous work using recognition paradigms and stress the importance of encoding in determining later VWM retrieval success.

Towards a unified perspective of object shape and motion processing in human dorsal cortex.

Although object-related areas were discovered in human parietal cortex a decade ago, surprisingly little is known about the nature and purpose of these representations, and how they differ from those in the ventral processing stream. In this article, we review evidence for the unique contribution of object areas of dorsal cortex to three-dimensional (3-D) shape representation, the localization of objects in space, and in guiding reaching and grasping actions. We also highlight the role of dorsal cortex in form-motion interaction and spatiotemporal integration, possible functional relationships between 3-D shape and motion processing, and how these processes operate together in the service of supporting goal-directed actions with objects. Fundamental differences between the nature of object representations in the dorsal versus ventral processing streams are considered, with an emphasis on how and why dorsal cortex supports veridical (rather than invariant) representations of objects to guide goal-directed hand actions in dynamic visual environments.

Decoding information about dynamically occluded objects in visual cortex.

During dynamic occlusion, an object passes behind an occluding surface and then later reappears. Even when completely occluded from view, such objects are experienced as continuing to exist or persist behind the occluder even though they are no longer visible. The contents and neural basis of this persistent representation remain poorly understood. Questions remain as to whether there is information maintained about the object itself (i.e. its shape or identity) or non-object-specific information such as its position or velocity as it is tracked behind an occluder, as well as which areas of visual cortex represent such information. Recent studies have found that early visual cortex is activated by "invisible" objects during visual imagery and by unstimulated regions along the path of apparent motion, suggesting that some properties of dynamically occluded objects may also be neurally represented in early visual cortex. We applied functional magnetic resonance imaging in human subjects to examine representations within visual cortex during dynamic occlusion. For gradually occluded, but not for instantly disappearing objects, there was an increase in activity in early visual cortex (V1, V2, and V3). This activity was spatially-specific, corresponding to the occluded location in the visual field. However, the activity did not encode enough information about object identity to discriminate between different kinds of occluded objects (circles vs. stars) using MVPA. In contrast, object identity could be decoded in spatially-specific subregions of higher-order, topographically organized areas such as ventral, lateral, and temporal occipital areas (VO, LO, and TO) as well as the functionally defined LOC and hMT+. These results suggest that early visual cortex may only represent the dynamically occluded object's position or motion path, while later visual areas represent object-specific information.

The neural representation of objects formed through the spatiotemporal integration of visual transients.

Oftentimes, objects are only partially and transiently visible as parts of them become occluded during observer or object motion. The visual system can integrate such object fragments across space and time into perceptual wholes or spatiotemporal objects. This integrative and dynamic process may involve both ventral and dorsal visual processing pathways, along which shape and spatial representations are thought to arise. We measured fMRI BOLD response to spatiotemporal objects and used multi-voxel pattern analysis (MVPA) to decode shape information across 20 topographic regions of visual cortex. Object identity could be decoded throughout visual cortex, including intermediate (V3A, V3B, hV4, LO1-2,) and dorsal (TO1-2, and IPS0-1) visual areas. Shape-specific information, therefore, may not be limited to early and ventral visual areas, particularly when it is dynamic and must be integrated. Contrary to the classic view that the representation of objects is the purview of the ventral stream, intermediate and dorsal areas may play a distinct and critical role in the construction of object representations across space and time.

Dynamic illusory size contrast: a relative-size illusion modulated by stimulus motion and eye movements.

We present a novel size-contrast illusion that depends on the dynamic nature of the stimulus. In the dynamic illusory size-contrast (DISC) effect, the viewer perceives the size of a target bar to be shrinking when it is surrounded by an expanding box and when there are additional dynamic cues such as eye movements, changes in retinal eccentricity of the bar, or changes in the spatial position of the bar. Importantly, the expanding box was necessary but not sufficient to obtain an illusory percept, distinguishing the DISC effect from other size-contrast illusions. We propose that the visual system is weighting the different sources of information that contribute to size perception based on the level of uncertainty in the retinal image size of the object. Whereas the growing box normally has a weak influence on the perceived size of the target bar, this influence is enhanced when other dynamic changes in the environment (e.g., eye movements, changes in retinal eccentricity, and target motion) lead to uncertainty in the retinal size of the target bar. Given the compelling nature of the DISC effect and the inherently dynamic nature of our environment, these factors are likely to play an important role in everyday size judgments.

The Architecture of Object-Based Attention.

The allocation of attention to objects raises several intriguing questions: What are objects, how does attention access them, what anatomical regions are involved? Here, we review recent progress in the field to determine the mechanisms underlying object-based attention. First, findings from unconscious priming and cueing suggest that the preattentive targets of object-based attention can be fully developed object representations that have reached the level of identity. Next, the control of object-based attention appears to come from ventral visual areas specialized in object analysis that project downward to early visual areas. How feedback from object areas can accurately target the object's specific locations and features is unknown but recent work in autoencoding has made this plausible. Finally, we suggest that the three classic modes of attention may not be as independent as is commonly considered, and instead could all rely on object-based attention. Specifically, studies show that attention can be allocated to the separated members of a group-without affecting the space between them-matching the defining property of feature-based attention. At the same time, object-based attention directed to a single small item has the properties of space-based attention. We outline the architecture of object-based attention, the novel predictions it brings, and discuss how it works in parallel with other attention pathways.

The combination of target motion and dynamic changes in context greatly enhance visual size illusions.

Perceived size is a function of viewing distance, retinal images size, and various contextual cues such as linear perspective and the size and location of neighboring objects. Recently, we demonstrated that illusion magnitudes of classic visual size illusions may be greatly enhanced or reduced by adding dynamic elements. Specifically, a dynamic version of the Ebbinghaus illusion (classically considered a "size contrast" illusion) led to a greatly enhanced illusory effect, whereas a dynamic version of the Corridor illusion (a "size constancy" illusion) led to a greatly diminished illusory effect. Although these differences may arise from the different processes underlying these illusions (size contrast vs. size constancy), the dynamic variants we tested in our previous work also differed in the nature of the dynamic elements; specifically, whereas the Dynamic Ebbinghaus included a moving target and inducers that changed size and position, the Dynamic Corridor only included a moving target on a static background. Here, we explore further dynamic versions of the Ebbinghaus illusion and the Corridor and Ponzo illusions by separately manipulating three types of dynamic elements: target motion, context translation, and dynamic changes in context. Across five experiments examining 21 dynamic illusory configurations, adding target motion or a dynamically changing context separately resulted in little-to-no illusory effect. In contrast, the combination of target motion and a dynamically changing context led to a robust size illusion, consistent with an interactive effect. However, illusory effects that exceeded the matched classic, static illusory configuration were only observed for the dynamic versions of the Ebbinghaus illusion and the Revealed Ponzo illusions, in which the contextual elements changed size. We conclude that the combination of target motion and a dynamically changing context are necessary to produce dynamic size illusions, but that enhancement above and beyond static illusions may be largely specific to size contrast effects. Our results have important implications for the integration of motion signals, a ubiquitous environmental stimulus, in the perception of object size.

The maintenance and disambiguation of object representations depend upon feature contrast within and between objects.

The brain processes many aspects of the visual world separately and in parallel, yet we perceive a unified world populated by objects. In order to create such a "bound" percept, the visual system must construct object-centered representations out of separate features and then maintain the representations across changes in space and time. Here, we examine the role of features themselves in maintaining and disambiguating the representations of the objects to which they belong. In three experiments, we measure how the perceived motion of two objects traversing ambiguous trajectories is affected by the contrast between the features and surrounding fields, by the contrast between features, and by changes to orientation of texture within objects. We report that the maintenance and disambiguation of object representations depend on the contrast of the features relative to their surrounds and on the extent of feature differences between the two objects. These feature dependencies indicate that object representation relies on relative response to many stimulus dimensions.

Object-based attention generalizes to multisurface objects.

When a part of an object is cued, targets presented in other locations on the same object are detected more rapidly and accurately than are targets on other objects. Often in object-based attention experiments, cues and targets appear not only on the same object but also on the same surface. In four psychophysical experiments, we examined whether the "object" of attentional selection was the entire object or one of its surfaces. In Experiment 1, facilitation effects were found for targets on uncued, adjacent surfaces on the same object, even when the cued and uncued surfaces were oriented differently in depth. This suggests that the "object-based" benefits of attention are not restricted to individual surfaces. Experiments 2a and 2b examined the interaction of perceptual grouping and object-based attention. In both experiments, cuing benefits extended across objects when the surfaces of those objects could be grouped, but the effects were not as strong as in Experiment 1, where the surfaces belonged to the same object. The cuing effect was strengthened in Experiment 3 by connecting the cued and target surfaces with an intermediate surface, making them appear to all belong to the same object. Together, the experiments suggest that the objects of attention do not necessarily map onto discrete physical objects defined by bounded surfaces. Instead, attentional selection can be allocated to perceptual groups of surfaces and objects in the same way as it can to a location or to groups of features that define a single object.

Opposite effects of motion dynamics on the Ebbinghaus and corridor illusions.

We recently showed that motion dynamics greatly enhance the magnitude of certain size contrast illusions, such as the Ebbinghaus and Delboeuf illusions. Here, we extend our study of the effect of motion dynamics on size illusions through a novel dynamic corridor illusion, in which a single target translates along a corridor background. Across three psychophysical experiments, we quantify the effects of stimulus dynamics on the Ebbinghaus and corridor illusions across different viewing conditions. The results revealed that stimulus dynamics had opposite effects on these different classes of size illusions. Whereas dynamic motion enhanced the magnitude of the Ebbinghaus illusion, it attenuated the magnitude the corridor illusion. Our results highlight precision-driven weighting of visual cues by neural circuits computing perceived object size. This hypothesis is consistent with observations beyond size perception and may represent a more general principle of cue integration in the visual system.

Interactions of flicker and motion.

We present a series of novel observations about interactions between flicker and motion that lead to three distinct perceptual effects. We use the term flicker to describe alternating changes in a stimulus' luminance or color (i.e. a circle that flickers from black to white and visa-versa). When objects flicker, three distinct phenomena can be observed: (1) Flicker Induced Motion (FLIM) in which a single, stationary object, appears to move when it flickers at certain rates; (2) Flicker Induced Motion Suppression (FLIMS) in which a moving object appears to be stationary when it flickers at certain rates, and (3) Flicker-Induced Induced-Motion (FLIIM) in which moving objects that are flickering induce another flickering stationary object to appear to move. Across four psychophysical experiments, we characterize key stimulus parameters underlying these flicker-motion interactions. Interactions were strongest in the periphery and at flicker frequencies above 10 Hz. Induced motion occurred not just for luminance flicker, but for isoluminant color changes as well. We also found that the more physically moving objects there were, the more motion induction to stationary objects occurred. We present demonstrations that the effects reported here cannot be fully accounted for by eye movements: we show that the perceived motion of multiple stationary objects that are induced to move via flicker can appear to move independently and in random directions, whereas eye movements would have caused all of the objects to appear to move coherently. These effects highlight the fundamental role of spatiotemporal dynamics in the representation of motion and the intimate relationship between flicker and motion.

The Wandering Circles: A Flicker Rate and Contour-Dependent Motion Illusion.

Understanding of the visual system can be informed by examining errors in perception. We present a novel illusion-Wandering Circles-in which stationary circles undergoing contrast-polarity reversals (i.e., flicker), when viewed peripherally, appear to move about in a random fashion. In two psychophysical experiments, participants rated the strength of perceived illusory motion under varying stimulus conditions. The illusory motion percept was strongest when the circle's edge was defined by a light/dark alternation and when the edge faded smoothly to the background gray (i.e., a circular arrangement of the Craik-O'Brien-Cornsweet illusion). In addition, the percept of illusory motion is flicker rate dependent, appearing strongest when the circles reversed polarity 9.44 times per second and weakest at 1.98 times per second. The Wandering Circles differ from many other classic motion illusions as the light/dark alternation is perfectly balanced in time and position around the edges of the circle, and thus, there is no net directional local or global motion energy in the stimulus. The perceived motion may instead rely on factors internal to the viewer such as top-down influences, asymmetries in luminance and motion perception across the retina, adaptation combined with positional uncertainty due to peripheral viewing, eye movements, or low contrast edges.

fMRI reveals that non-local processing in ventral retinotopic cortex underlies perceptual grouping by temporal synchrony.

UNLABELLED: When spatially separated objects appear and disappear in a synchronous manner, they perceptually group into a single global object that itself appears and disappears. We employed functional magnetic resonance imaging (fMRI) to identify brain regions involved in this type of perceptual grouping. Subjects viewed four chromatically-defined disks (one per visual quadrant) that flashed on and off. We contrasted %BOLD signal changes between blocks of synchronously flashing disks (Grouping) with blocks of asynchronously flashing disks (no-Grouping). RESULTS: A region of interest analysis revealed %BOLD signal change in the Grouping condition was significantly greater than in the no-Grouping condition within retinotopic areas V2, V3, and V4v. Within a single quadrant of the visual field, the spatio-temporal information present in the image was identical across the two stimulus conditions. As such, the two conditions could not be distinguished from each other on the basis of the rate or pattern of flashing within a single visual quadrant. The observed results must therefore arise through nonlocal interactions between or within these retinotopic areas, or arise from outside these retinotopic areas. Furthermore, when V2 and V3 were split into ventral and dorsal sub-ROIs, ventral retinotopic areas V2v and V3v preferentially differentiated between the two conditions whereas the corresponding dorsal areas V2d and V3d did not. In contrast, within hMT+, %BOLD signal was significantly greater in the no-Grouping condition. CONCLUSION: Nonlocal processing within, between, or to ventral retinotopic cortex at least as early as V2v, and including V3v, and V4v, underlies perceptual grouping via temporal synchrony.

Failures to see: attentive blank stares revealed by change blindness.

Change blindness illustrates a remarkable limitation in visual processing by demonstrating that substantial changes in a visual scene can go undetected. Because these changes can ultimately be detected using top-down driven search processes, many theories assign a central role to spatial attention in overcoming change blindness. Surprisingly, it has been reported that change blindness can occur during blink-contingent changes even when observers fixate the changing location [O'Regan, J. K., Deubel, H., Clark, J. J., & Rensink, R. A. (2000). Picture changes during blinks: Looking without seeing and seeing without looking. Visual Cognition, 7, 191-212]. However, eye blinks produce a transient disruption of vision that is independent of any associated changes in the retinal image. We determined whether these 'attentive blank stares' could occur in the absence of blink-mediated visual suppression. Using a flicker change-blindness paradigm we confirm that despite direct attentive fixations, obvious scene changes often remain undetected. We conclude that change detection involves object or feature based attentional mechanisms, which can be 'misdirected' despite the allocation of spatial attention to the position of the change.

Correction: Induced and Evoked Human Electrophysiological Correlates of Visual Working Memory Set-Size Effects at Encoding.

[This corrects the article DOI: 10.1371/journal.pone.0167022.].

Contour discontinuities subserve two types of form analysis that underlie motion processing.

Form analysis subserves motion processing in at least two ways: first, in terms of figural segmentation dedicated to solving the problem of figure-to-figure matching over time, and second, in terms of defining trackable features whose unambiguous motion signals can be generalized to ambiguously moving portions of an object. The former is a primarily ventral process involving the lateral occipital complex and also retinotopic areas such as V2 and V4, and the latter is a dorsal process involving V3A. Contour discontinuities, such as corners, deep concavities, maxima of positive curvature, junctions, and terminators, play a central role in both types of form analysis. Transformational apparent motion will be discussed in the context of figural segmentation and matching, and rotational motion in the context of trackable features. In both cases the analysis of form must proceed in parallel with the analysis of motion, in order to constrain the ongoing analysis of motion.

Induced and Evoked Human Electrophysiological Correlates of Visual Working Memory Set-Size Effects at Encoding.

The ability to encode, store, and retrieve visually presented objects is referred to as visual working memory (VWM). Although crucial for many cognitive processes, previous research reveals that VWM strictly capacity limited. This capacity limitation is behaviorally observable in the set size effect: the ability to successfully report items in VWM asymptotes at a small number of items. Research into the neural correlates of set size effects and VWM capacity limits in general largely focus on the maintenance period of VWM. However, we previously reported that neural resources allocated to individual items during VWM encoding correspond to successful VWM performance. Here we expand on those findings by investigating neural correlates of set size during VWM encoding. We hypothesized that neural signatures of encoding-related VWM capacity limitations should be differentiable as a function of set size. We tested our hypothesis using High Density Electroencephalography (HD-EEG) to analyze frequency components evoked by flickering target items in VWM displays of set size 2 or 4. We found that set size modulated the amplitude of the 1st and 2nd harmonic frequencies evoked during successful VWM encoding across frontal and occipital-parietal electrodes. Frontal sites exhibited the most robust effects for the 2nd harmonic (set size 2 > set size 4). Additionally, we found a set-size effect on the induced power of delta-band (1-4 Hz) activity (set size 2 > set size 4). These results are consistent with a capacity limited VWM resource at encoding that is distributed across to-be-remembered items in a VWM display. This resource may work in conjunction with a task-specific selection process that determines which items are to be encoded and which are to be ignored. These neural set size effects support the view that VWM capacity limitations begin with encoding related processes.