Effect of attentional selection on working memory for depth in a retro-cueing paradigm.

Recent studies have shown that the temporary storage and manipulation of depth information (working memory for depth; WMd) is largely different from that of visual information in a 2D context (visual working memory; VWM). Although there has been abundant evidence on VWM showing that cueing a memory item during retention could bias attention to its internal representation and thus improves its memory performance (a retro-cue effect), it is unknown whether such an effect differs for WMd that is nested in a 3D context compared with that in a conventional 2D context. Here, we used a change detection task to investigate the effect of attentional selection on WMd by testing several types of retro-cue. The memory array consisted of items positioned at various stereoscopic depth planes, and a cue was presented during retention. Participants needed to make judgments on whether the depth position of target (one memory item) had changed. Our study showed reliable valid retro-cue benefits but no invalid retro-cue cost, indicating that the relational information may be registered in WMd to prevent a strategical removal of the unattended item. There was also a slight improvement in memory performance for cueing depth order compared with that for cueing other feature dimensions or 2D locations. The attentional effect on memory representation in a 3D context is different from that in a 2D context, and the divergence may suggest the distinctive nature of working memory for depth.

Effects of disparity on visual discomfort caused by short-term stereoscopic viewing based on electroencephalograph analysis.

BACKGROUND: Discomfort evoked by stereoscopic depth has been widely concerned. Previous studies have proposed a comfortable disparity range and considered that disparities exceed this range would cause visual discomfort. Brain activity recordings including Electroencephalograph (EEG) monitoring enable better understanding of perceptual and cognitive processes related to stereo depth-induced visual comfort. METHODS: EEG data was collected using a stereo-visual evoked potential (VEP) test system by providing visual stimulus to subjects aged from 21 to 25 with normal stereoscopic vision. For each type of visual stimulus, data were processed using directed transfer function (DTF) and adaptive directed transfer function (ADTF) in combination with subjective feedbacks (comfort or discomfort). The topographies of information flow were constructed to compare responses stimulated by different stereoscopic depth, and to determine the difference in comfort and discomfort situations upon stimulation with same stereoscopic depth. RESULTS: Based on EEG analysis results, we found that the occipital P270 was moderately related to the disparity. Moreover, the ADTF of P270 showed that the information flows at frontal lobe and central-parietal lobe changed when stimulation with different stereoscopic depth applied. As to the stereo images with same stereoscopic depth, the DTF outflows at the temporal and temporal-parietal lobes in δ band, central and central-parietal lobes in α and θ bands, and the comparison of inflows in these three bands could be considered as discriminated indexes for matching the stereoscopic effect with viewers' comfort or discomfort state impacted by disparity. The subjective feedbacks indicated that the comfort judgments remained as a result of cumulative effect. CONCLUSIONS: This study proposed a short-term stereo-VEP experiment that shorted the duration of each stimulus in the experimental scheme to minimize the interference from other factors except the disparity. The occipital P270 had a mid-relevance to the disparity and its ADTF showed the affected areas when viewers are receiving stimulations with different disparities. DTF could be considered as discriminated indexes for matching the stereoscopic effect with viewers' comfort or discomfort state induced by disparity. This study proposed a preferable experiment to observe the single effect of disparity and provided an intuitive and easy-to-read result in a more convenient manner.

Interdigitated Color- and Disparity-Selective Columns within Human Visual Cortical Areas V2 and V3.

In nonhuman primates (NHPs), secondary visual cortex (V2) is composed of repeating columnar stripes, which are evident in histological variations of cytochrome oxidase (CO) levels. Distinctive "thin" and "thick" stripes of dark CO staining reportedly respond selectively to stimulus variations in color and binocular disparity, respectively. Here, we first tested whether similar color-selective or disparity-selective stripes exist in human V2. If so, available evidence predicts that such stripes should (1) radiate "outward" from the V1-V2 border, (2) interdigitate, (3) differ from each other in both thickness and length, (4) be spaced ∼3.5-4 mm apart (center-to-center), and, perhaps, (5) have segregated functional connections. Second, we tested whether analogous segregated columns exist in a "next-higher" tier area, V3. To answer these questions, we used high-resolution fMRI (1 × 1 × 1 mm(3)) at high field (7 T), presenting color-selective or disparity-selective stimuli, plus extensive signal averaging across multiple scan sessions and cortical surface-based analysis. All hypotheses were confirmed. V2 stripes and V3 columns were reliably localized in all subjects. The two stripe/column types were largely interdigitated (e.g., nonoverlapping) in both V2 and V3. Color-selective stripes differed from disparity-selective stripes in both width (thickness) and length. Analysis of resting-state functional connections (eyes closed) showed a stronger correlation between functionally alike (compared with functionally unlike) stripes/columns in V2 and V3. These results revealed a fine-scale segregation of color-selective or disparity-selective streams within human areas V2 and V3. Together with prior evidence from NHPs, this suggests that two parallel processing streams extend from visual subcortical regions through V1, V2, and V3. SIGNIFICANCE STATEMENT: In current textbooks and reviews, diagrams of cortical visual processing highlight two distinct neural-processing streams within the first and second cortical areas in monkeys. Two major streams consist of segregated cortical columns that are selectively activated by either color or ocular interactions. Because such cortical columns are so small, they were not revealed previously by conventional imaging techniques in humans. Here we demonstrate that such segregated columnar systems exist in humans. We find that, in humans, color versus binocular disparity columns extend one full area further, into the third visual area. Our approach can be extended to reveal and study additional types of columns in human cortex, perhaps including columns underlying more cognitive functions.

Integration of motion energy from overlapping random background noise increases perceived speed of coherently moving stimuli.

The perception of visual motion can be profoundly influenced by visual context. To gain insight into how the visual system represents motion speed, we investigated how a background stimulus that did not move in a net direction influenced the perceived speed of a center stimulus. Visual stimuli were two overlapping random-dot patterns. The center stimulus moved coherently in a fixed direction, whereas the background stimulus moved randomly. We found that human subjects perceived the speed of the center stimulus to be significantly faster than its veridical speed when the background contained motion noise. Interestingly, the perceived speed was tuned to the noise level of the background. When the speed of the center stimulus was low, the highest perceived speed was reached when the background had a low level of motion noise. As the center speed increased, the peak perceived speed was reached at a progressively higher background noise level. The effect of speed overestimation required the center stimulus to overlap with the background. Increasing the background size within a certain range enhanced the effect, suggesting spatial integration. The speed overestimation was significantly reduced or abolished when the center stimulus and the background stimulus had different colors, or when they were placed at different depths. When the center- and background-stimuli were perceptually separable, speed overestimation was correlated with perceptual similarity between the center- and background-stimuli. These results suggest that integration of motion energy from random motion noise has a significant impact on speed perception. Our findings put new constraints on models regarding the neural basis of speed perception.

Cortical Responses and Shape Complexity of Stereoscopic Image - A Simultaneous EEG/MEG Study.

BACKGROUND/AIMS: In exploring human factors, stereoscopic 3D images have been used to investigate the neural responses associated with excessive depth, texture complexity, and other factors. However, the cortical oscillation associated with the complexity of stereoscopic images has been studied rarely. Here, we demonstrated that the oscillatory responses to three differently shaped 3D images (circle, star, and bat) increase as the complexity of the image increases. METHODS: We recorded simultaneous EEG/MEG for three different stimuli. Spatio-temporal and spatio-spectro-temporal features were investigated by non-parametric permutation test. RESULTS: The results showed that N300 and alpha inhibition increased in the ventral area as the shape complexity of the stereoscopic image increased. CONCLUSION: It seems that the relative disparity in complex stereoscopic images may increase cognitive processing (N300) and cortical load (alpha inhibition) in the ventral area.

Three-dimensional endoscopic visualization in functional endoscopic sinus surgery.

Three-dimensional (3D) stereoscopic vision in sinus surgery has been achieved with the microscope so far. The introduction of two-dimensional (2D) endoscopes set a milestone in the visualization of the surgical field and paved the way to functional endoscopic sinus surgery (FESS), although the 2D endoscopes cannot provide a stereoscopic visualization. The latest technology of 3D endoscopes allows stereoscopic vision. We provide a clinical investigation of all commercially available 3D endoscopes in FESS to compare their clinical value and efficacy to routinely used conventional 2D HD endoscopes. In this prospective, randomized, controlled clinical study, 46 patients with polypoid chronic rhinosinusitis underwent FESS with one of the following three endoscopes: 2D 0° high definition (HD), 3D 0° standard definition (SD) and 3D 0° HD. Four surgeons qualitatively assessed endoscopes on stereoscopic depth perception (SDP) of the surgeon, sharpness and brightness of the image, as well as their comfort in use during surgery. Surgeons assessed the brightness of the control (2D HD) significantly better than 3D SD (p = 0.009) and brightness of 3D HD was rated significantly better than 3D SD (p = 0.038). Stereoscopic depth perception (SDP) of 3D SD was assessed highly significantly better than the control (2D HD) (p = 0.021), whereas 3D HD displayed best SDP (p = 0.0001). The comfort in use was rated significantly higher in the 3D HD group compared to the control group (p = 0.025). No significant differences in sharpness could be seen among all endoscopes. 3D HD endoscopy provides an improvement in SDP and brightness of the surgical field. It enhances the intraoperative visualization and is therefore an important and efficient development in endoscopic sinus surgery.

Segmentation by depth does not always facilitate visual search.

In visual search, target detection times are relatively insensitive to set size when targets and distractors differ on a single feature dimension. Search can be confined to only those elements sharing a single feature, such as color (Egeth, Virzi, & Garbart, 1984). These findings have been taken as evidence that elementary feature dimensions support a parallel segmentation of a scene into discrete sets of items. Here we explored if relative depth (signaled by binocular disparity) could support a similar parallel segmentation by examining the effects of distributing distracting elements across two depth planes. Three important empirical findings emerged. First, when the target was a feature singleton on the target depth plane, but a conjunction search among distractors on the nontarget plane, search efficiency increased compared to a single depth plane. Second, benefits of segmentation in depth were only observed when the target depth plane was known in advance. Third, no benefit of segmentation in depth was observed when both planes required a conjunction search, even with prior knowledge of the target depth plane. Overall, the benefit of distributing the elements of a search set across two depth planes was observed only when the two planes differed both in binocular disparity and in the elementary feature composition of individual elements. We conclude that segmentation of the search array into two depth planes can facilitate visual search, but unlike color or other elementary properties, does not provide an automatic, preattentive segmentation.

Spatial attention based on 2D location and relative depth order modulates visual working memory in a 3D environment.

The attentional effect on visual working memory (VWM) has been a heated research topic in the past two decades. Studies show that VWM performance for an attended memory item can be improved by cueing its two-dimensional (2D) spatial location during retention. However, few studies have investigated the effect of attentional selection on VWM in a three-dimensional setting, and it remains unknown whether depth information can produce beneficial attentional effects on 2D visual representations similar to 2D spatial information. Here we conducted four experiments, displaying memory items at various stereoscopic depth planes, and examined the retro-cue effects of four types of cues - a cue would either indicate the 2D or depth location of a memory item, and either in the form of physical (directly pointing to a location) or symbolic (numerically mapping onto a location) cues. We found that retro-cue benefits were only observed for cues directly pointing to a 2D location, whereas a null effect was observed for cues directly pointing to a depth location. However, there was a retro-cue effect when cueing the relative depth order, though the effect was weaker than that for cueing the 2D location. The selective effect on VWM based on 2D spatial attention is different from depth-based attention, and the divergence suggests that an object representation is primarily bound with its 2D spatial location, weakly bound with its depth order but not with its metric depth location. This indicates that attentional selection based on memory for depth, particularly metric depth, is ineffective.

Visual Search in 3D: Effects of Monoscopic and Stereoscopic Cues to Depth on the Validity of Feature Integration Theory and Perceptual Load Theory.

Previous research has successfully used feature integration theory to operationalise the predictions of Perceptual Load Theory, while simultaneously testing the predictions of both models. Building on this work, we test the extent to which these models hold up in a 3D world. In two experiments, participants responded to a target stimulus within an array of shapes whose apparent depth was manipulated using a combination of monoscopic and stereoscopic cues. The search task was designed to test the predictions of (a) feature integration theory, as the target was identified by a single feature or a conjunction of features and embedded in search arrays of varying size, and (b) perceptual load theory, as the task included congruent and incongruent distractors presented alongside search tasks imposing high or low perceptual load. Findings from both experiments upheld the predictions of feature integration theory, regardless of 2D/3D condition. Longer search times in conditions with a combination of monoscopic and stereoscopic depth cues suggests that binding features into three-dimensional objects requires greater attentional effort. This additional effort should have implications for perceptual load theory, yet our findings did not uphold its predictions; the effect of incongruent distractors did not differ between conjunction search trials (conceptualised as high perceptual load) and feature search trials (low perceptual load). Individual differences in susceptibility to the effects of perceptual load were evident and likely explain the absence of load effects. Overall, our findings suggest that feature integration theory may be useful for predicting attentional performance in a 3D world.

Vision in our three-dimensional world.

Many aspects of our perceptual experience are dominated by the fact that our two eyes point forward. Whilst the location of our eyes leaves the environment behind our head inaccessible to vision, co-ordinated use of our two eyes gives us direct access to the three-dimensional structure of the scene in front of us, through the mechanism of stereoscopic vision. Scientific understanding of the different brain regions involved in stereoscopic vision and three-dimensional spatial cognition is changing rapidly, with consequent influences on fields as diverse as clinical practice in ophthalmology and the technology of virtual reality devices.This article is part of the themed issue 'Vision in our three-dimensional world'.

A critical role of holistic processing in face gender perception.

Whether face gender perception is processed by encoding holistic (whole) or featural (parts) information is a controversial issue. Although neuroimaging studies have identified brain regions related to face gender perception, the temporal dynamics of this process remain under debate. Here, we identified the mechanism and temporal dynamics of face gender perception. We used stereoscopic depth manipulation to create two conditions: the front and behind condition. In the front condition, facial patches were presented stereoscopically in front of the occluder and participants perceived them as disjoint parts (featural cues). In the behind condition, facial patches were presented stereoscopically behind the occluder and were amodally completed and unified in a coherent face (holistic cues). We performed three behavioral experiments and one electroencephalography experiment, and compared the results of the front and behind conditions. We found faster reaction times (RTs) in the behind condition compared with the front, and observed priming effects and aftereffects only in the behind condition. Moreover, the EEG experiment revealed that face gender perception is processed in the relatively late phase of visual recognition (200-285 ms). Our results indicate that holistic information is critical for face gender perception, and that this process occurs with a relatively late latency.

The binocular advantage in visuomotor tasks involving tools.

We compared performance on three manual-dexterity tasks under monocular and binocular viewing. The tasks were the standard Morrisby Fine Dexterity Test, using forceps to manipulate the items, a modified version of the Morrisby test using fingers, and a "buzz-wire" task in which subjects had to guide a wire hoop around a 3D track without bringing the hoop into contact with the track. In all three tasks, performance was better for binocular viewing. The extent of the binocular advantage in individuals did not correlate significantly with their stereoacuity measured on the Randot test. However, the extent of the binocular advantage depended strongly on the task. It was weak when fingers were used on the Morrisby task, stronger with forceps, and extremely strong on the buzz-wire task (fivefold increase in error rate with monocular viewing). We suggest that the 3D buzz-wire game is particularly suitable for assessing binocularly based dexterity.