Category systems for real-world scenes.

Categorization performance is a popular metric of scene recognition and understanding in behavioral and computational research. However, categorical constructs and their labels can be somewhat arbitrary. Derived from exhaustive vocabularies of place names (e.g., Deng et al., 2009), or the judgements of small groups of researchers (e.g., Fei-Fei, Iyer, Koch, & Perona, 2007), these categories may not correspond with human-preferred taxonomies. Here, we propose clustering by increasing the rand index via coordinate ascent (CIRCA): an unsupervised, data-driven clustering method for deriving ground-truth scene categories. In Experiment 1, human participants organized 80 stereoscopic images of outdoor scenes from the Southampton-York Natural Scenes (SYNS) dataset (Adams et al., 2016) into discrete categories. In separate tasks, images were grouped according to i) semantic content, ii) three-dimensional spatial structure, or iii) two-dimensional image appearance. Participants provided text labels for each group. Using the CIRCA method, we determined the most representative category structure and then derived category labels for each task/dimension. In Experiment 2, we found that these categories generalized well to a larger set of SYNS images, and new observers. In Experiment 3, we tested the relationship between our category systems and the spatial envelope model (Oliva & Torralba, 2001). Finally, in Experiment 4, we validated CIRCA on a larger, independent dataset of same-different category judgements. The derived category systems outperformed the SUN taxonomy (Xiao, Hays, Ehinger, Oliva, & Torralba, 2010) and an alternative clustering method (Greene, 2019). In summary, we believe this novel categorization method can be applied to a wide range of datasets to derive optimal categorical groupings and labels from psychophysical judgements of stimulus similarity.

Disruptive coloration and binocular disparity: breaking camouflage.

Many species employ camouflage to disguise their true shape and avoid detection or recognition. Disruptive coloration is a form of camouflage in which high-contrast patterns obscure internal features or break up an animal's outline. In particular, edge enhancement creates illusory, or 'fake' depth edges within the animal's body. Disruptive coloration often co-occurs with background matching, and together, these strategies make it difficult for an observer to visually segment an animal from its background. However, stereoscopic vision could provide a critical advantage in the arms race between perception and camouflage: the depth information provided by binocular disparities reveals the true three-dimensional layout of a scene, and might, therefore, help an observer to overcome the effects of disruptive coloration. Human observers located snake targets embedded in leafy backgrounds. We analysed performance (response time) as a function of edge enhancement, illumination conditions and the availability of binocular depth cues. We confirm that edge enhancement contributes to effective camouflage: observers were slower to find snakes whose patterning contains 'fake' depth edges. Importantly, however, this effect disappeared when binocular depth cues were available. Illumination also affected detection: under directional illumination, where both the leaves and snake produced strong cast shadows, snake targets were localized more quickly than in scenes rendered under ambient illumination. In summary, we show that illusory depth edges, created via disruptive coloration, help to conceal targets from human observers. However, cast shadows and binocular depth information improve detection by providing information about the true three-dimensional structure of a scene. Importantly, the strong interaction between disparity and edge enhancement suggests that stereoscopic vision has a critical role in breaking camouflage, enabling the observer to overcome the disruptive effects of edge enhancement.

Visual extrapolation under risk: human observers estimate and compensate for exogenous uncertainty.

Humans commonly face choices between multiple options with uncertain outcomes. Such situations occur in many contexts, from purely financial decisions (which shares should I buy?) to perceptuo-motor decisions between different actions (where should I aim my shot at goal?). Regardless of context, successful decision-making requires that the uncertainty at the heart of the decision-making problem is taken into account. Here, we ask whether humans can recover an estimate of exogenous uncertainty and then use it to make good decisions. Observers viewed a small dot that moved erratically until it disappeared behind an occluder. We varied the size of the occluder and the unpredictability of the dot's path. The observer attempted to capture the dot as it emerged from behind the occluded region by setting the location and extent of a 'catcher' along the edge of the occluder. The reward for successfully catching the dot was reduced as the size of the catcher increased. We compared human performance with that of an agent maximizing expected gain and found that observers consistently selected catcher size close to this theoretical solution. These results suggest that humans are finely tuned to exogenous uncertainty information and can exploit it to guide action.

Natural images dominate in binocular rivalry.

Ecological approaches to perception have demonstrated that information encoding by the visual system is informed by the natural environment, both in terms of simple image attributes like luminance and contrast, and more complex relationships corresponding to Gestalt principles of perceptual organization. Here, we ask if this optimization biases perception of visual inputs that are perceptually bistable. Using the binocular rivalry paradigm, we designed stimuli that varied in either their spatiotemporal amplitude spectra or their phase spectra. We found that noise stimuli with "natural" amplitude spectra (i.e., amplitude content proportional to 1/f, where f is spatial or temporal frequency) dominate over those with any other systematic spectral slope, along both spatial and temporal dimensions. This could not be explained by perceived contrast measurements, and occurred even though all stimuli had equal energy. Calculating the effective contrast following attenuation by a model contrast sensitivity function suggested that the strong contrast dependency of rivalry provides the mechanism by which binocular vision is optimized for viewing natural images. We also compared rivalry between natural and phase-scrambled images and found a strong preference for natural phase spectra that could not be accounted for by observer biases in a control task. We propose that this phase specificity relates to contour information, and arises either from the activity of V1 complex cells, or from later visual areas, consistent with recent neuroimaging and single-cell work. Our findings demonstrate that human vision integrates information across space, time, and phase to select the input most likely to hold behavioral relevance.

The time-course of real-world scene perception: Spatial and semantic processing.

Real-world scene perception unfolds remarkably quickly, yet the underlying visual processes are poorly understood. Space-centered theory maintains that a scene's spatial structure (e.g., openness, mean depth) can be rapidly recovered from low-level image statistics. In turn, the statistical relationship between a scene's spatial properties and semantic content allows for semantic identity to be inferred from its layout. We tested this theory by investigating (1) the temporal dynamics of spatial and semantic perception in real-world scenes, and (2) dependencies between spatial and semantic judgments. Participants viewed backward-masked images for 13.3 to 106.7 ms, and identified the semantic (e.g., beach, road) or spatial structure (e.g., open, closed-off) category. We found no temporal precedence of spatial discrimination relative to semantic discrimination. Computational analyses further suggest that, instead of using spatial layout to infer semantic categories, humans exploit semantic information to discriminate spatial structure categories. These findings challenge traditional 'bottom-up' views of scene perception.

Efficient visual recalibration from either visual or haptic feedback: the importance of being wrong.

The human visual system adapts to the changing statistics of its environment. For example, the light-from-above prior, an assumption that aids the interpretation of ambiguous shading information, can be modified by haptic (touch) feedback. Here we investigate the mechanisms that drive this adaptive learning. In particular, we ask whether visual information can be as effective as haptics in driving visual recalibration and whether increased information (feedback from multiple modalities) induces faster learning. During several hours' training, feedback encouraged observers to modify their existing light-from-above assumption. Feedback was one of the following: (1) haptic only, (2) haptic and stereoscopic (providing binocular shape information), or (3) stereoscopic only. Haptic-only feedback resulted in substantial learning; the perceived shape of shaded objects was modified in accordance with observers' new light priors. However, the addition of continuous visual feedback (condition 2) substantially reduced learning. When visual-only feedback was provided intermittently (condition 3), mimicking the time course of the haptic feedback of conditions 1 and 2, substantial learning returned. The intermittent nature of conflict information, or feedback, appears critical for learning. It causes an initial, erroneous percept to be corrected. Contrary to previous proposals, we found no particular advantage for cross-modal feedback. Instead, we suggest that an "oops" factor drives efficient learning; recalibration is prioritized when a mismatch exists between sequential representations of an object property. This "oops" factor appears important both across and within sensory modalities, suggesting a general principle for perceptual learning and recalibration.

Investigating Emotional Body Posture Recognition in Adolescents with Conduct Disorder Using Eye-Tracking Methods.

Adolescents with Conduct Disorder (CD) show deficits in recognizing facial expressions of emotion, but it is not known whether these difficulties extend to other social cues, such as emotional body postures. Moreover, in the absence of eye-tracking data, it is not known whether such deficits, if present, are due to a failure to attend to emotionally informative regions of the body. Male and female adolescents with CD and varying levels of callous-unemotional (CU) traits (n = 45) and age- and sex-matched typically-developing controls (n = 51) categorized static and dynamic emotional body postures. The emotion categorization task was paired with eye-tracking methods to investigate relationships between fixation behavior and recognition performance. Having CD was associated with impaired recognition of static and dynamic body postures and atypical fixation behavior. Furthermore, males were less likely to fixate emotionally-informative regions of the body than females. While we found no effects of CU traits on body posture recognition, the effects of CU traits on fixation behavior varied according to CD status and sex, with CD males with lower levels of CU traits showing the most atypical fixation behavior. Critically, atypical fixation behavior did not explain the body posture recognition deficits observed in CD. Our findings suggest that CD-related impairments in recognition of body postures of emotion are not due to attentional issues. Training programmes designed to ameliorate the emotion recognition difficulties associated with CD may need to incorporate a body posture component.

High-level face adaptation without awareness.

When a visual stimulus is suppressed from awareness, processing of the suppressed image is necessarily reduced. Although adaptation to simple image properties such as orientation still occurs, adaptation to high-level properties such as face identity is eliminated. Here we show that emotional facial expression continues to be processed even under complete suppression, as indexed by substantial facial expression aftereffects.

Motion-aftereffect-induced blindness.

Motion-induced blindness (MIB) describes the occasional disappearance of salient visual objects in the presence of moving features (Y. S. Bonneh, A. Cooperman, & D. Sagi, 2001). Here we test whether motion adaptation and the ensuing motion aftereffect (MAE) are sufficient to trigger disappearance of salient targets. In three experiments, observers adapted to either rotating or static stimuli. Immediately afterwards, a static test pattern was presented consisting of a mask with texture elements and three superimposed target dots in a triangular arrangement. Observers reported dot disappearance and reappearance. The results clearly show that illusory motion in a static test pattern, following motion adaptation, promotes the disappearance of target dots. Furthermore, disappearance is modulated by the depth relationship between test pattern and targets, increasing for targets placed stereoscopically behind the test pattern. We conclude that MIB is influenced by perceived relative motion between depth-segregated features.

The fate of task-irrelevant visual motion: perceptual load versus feature-based attention.

We tested contrasting predictions derived from perceptual load theory and from recent feature-based selection accounts. Observers viewed moving, colored stimuli and performed low or high load tasks associated with one stimulus feature, either color or motion. The resultant motion aftereffect (MAE) was used to evaluate attentional allocation. We found that task-irrelevant visual features received less attention than co-localized task-relevant features of the same objects. Moreover, when color and motion features were co-localized yet perceived to belong to two distinct surfaces, feature-based selection was further increased at the expense of object-based co-selection. Load theory predicts that the MAE for task-irrelevant motion would be reduced with a higher load color task. However, this was not seen for co-localized features; perceptual load only modulated the MAE for task-irrelevant motion when this was spatially separated from the attended color location. Our results suggest that perceptual load effects are mediated by spatial selection and do not generalize to the feature domain. Feature-based selection operates to suppress processing of task-irrelevant, co-localized features, irrespective of perceptual load.

The spatial scale of perceptual memory in ambiguous figure perception.

Ambiguous visual stimuli highlight the constructive nature of vision: perception alternates between two plausible interpretations of unchanging input. However, when a previously viewed ambiguous stimulus reappears, its earlier perception almost entirely determines the new interpretation; memory disambiguates the input. Here, we investigate the spatial properties of this perceptual memory, taking into account strong anisotropies in percept preference across the visual field. Countering previous findings, we show that perceptual memory is not confined to the location in which it was instilled. Rather, it spreads to noncontiguous regions of the visual field, falling off at larger distances. Furthermore, this spread of perceptual memory takes place in a frame of reference that is tied to the surface of the retina. These results place the neural locus of perceptual memory in retinotopically organized sensory cortical areas, with implications for the wider function of perceptual memory in facilitating stable vision in natural, dynamic environments.

Motion adaptation and attention: A critical review and meta-analysis.

The motion aftereffect (MAE) provides a behavioural probe into the mechanisms underlying motion perception, and has been used to study the effects of attention on motion processing. Visual attention can enhance detection and discrimination of selected visual signals. However, the relationship between attention and motion processing remains contentious: not all studies find that attention increases MAEs. Our meta-analysis reveals several factors that explain superficially discrepant findings. Across studies (37 independent samples, 76 effects) motion adaptation was significantly and substantially enhanced by attention (Cohen's d = 1.12, p < .0001). The effect more than doubled when adapting to translating (vs. expanding or rotating) motion. Other factors affecting the attention-MAE relationship included stimulus size, eccentricity and speed. By considering these behavioural analyses alongside neurophysiological work, we conclude that feature-based (rather than spatial, or object-based) attention is the biggest driver of sensory adaptation. Comparisons between naïve and non-naïve observers, different response paradigms, and assessment of 'file-drawer effects' indicate that neither response bias nor publication bias are likely to have significantly inflated the estimated effect of attention.

Surface organization influences bistable vision.

A priority for the visual system is to construct 3-dimensional surfaces from visual primitives. Information is combined across individual cues to form a robust representation of the external world. Here, it is shown that surface completion relying on multiple visual cues influences relative dominance during binocular rivalry. The shape of a surface determined by 1 nonrivalrous visual cue (disparity, structure-from-motion) alters the dominance patterns of a rivalrous region of the image whose shape is defined by another cue (perspective, texture). The findings indicate that contextual information promotes the perception of the member of the rivalrous pair consistent with a smooth surface representation. The results extend the current description of binocular rivalry as a hierarchical process by implying that cue combination affects image selection during bistable viewing.

Equivalence of physical and perceived speed in binocular rivalry.

The relative dominance of gratings engaged in binocular rivalry can be influenced by their surroundings. One striking example occurs when surrounding motion is congruent with one but not the other grating (C. L. Paffen, S. F. te Pas, R. Kanai, M. J. van der Smagt, & F. A. Verstraten, 2004). However, such center-surround stimulus configurations can also modulate perceived speed, via a directionally tuned process (H. P. Norman, J. F. Norman, J. T. Todd, & D. T. Lindsey, 1996). We recorded rivalry for Gabor patches embedded in a drifting noise texture. Gratings whose directions opposed the background motion tended to dominate more, and vice versa, consistent with previous findings. Observers then matched the speed of a drifting noise-embedded Gabor to that of a Gabor surrounded by mean luminance. Surround motion produced substantial changes in perceived speed, by at least a factor of two for all observers. We then asked whether perceived speed could account for the contextual effects on dominance. We measured the effects of speed on rivalry dominance by changing the physical speeds of rivaling gratings, as determined by the matching data. We found the same pattern of dominance as for the context experiment, indicating that perceived and true speed influence rivalry in the same manner. We propose a Bayesian interpretation of the perceived speed illusion.

Experience can change the 'light-from-above' prior.

To interpret complex and ambiguous input, the human visual system uses prior knowledge or assumptions about the world. We show that the 'light-from-above' prior, used to extract information about shape from shading is modified in response to active experience with the scene. The resultant adaptation is not specific to the learned scene but generalizes to a different task, demonstrating that priors are constantly adapted by interactive experience with the environment.

The effects of attention and adaptation duration on the motion aftereffect.

The motion aftereffect (MAE) is the perception of illusory motion following extended exposure to a moving stimulus. The MAE has been used to probe the role of attention in motion processing. Many studies have reported that MAEs are reduced if attention is diverted from the adaptation stimulus, but others have argued that motion adaptation is independent of attention. We explored several factors that might modulate the attention-adaptation relationship and therefore explain apparent inconsistencies, namely (a) adaptation duration, (b) motion type: translating versus complex, and (c) response bias. Participants viewed translating (Experiments 1a and 2) or rotating (Experiment 1b) random dot patterns while fixating a central letter stream. During adaptation, participants reported brief changes in the adaptor speed (attention-focused) or the presence of white vowels within the letter stream (attention-diverted). Trials consisted of multiple adaptation-test cycles, and the MAE was measured after each adaptation period. Across experiments, focused attention produced significantly larger MAEs than did diverted attention (15% change, Cohen's d = .41). Attention affected the MAE asymptote, rather than its accumulation rate, and had larger effects for translational than for complex motion. The effect of attention remained evident after controlling for response bias. Our results suggest that attention affects multiple levels of the motion-processing hierarchy: not only higher level motion processing, as seen with apparent motion, but also low-level motion processing, as evidenced by the MAE. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Dichoptic motion perception limited to depth of fixation?

When counterphase spatio-temporal flicker is presented to the left and right eye continuous directional motion can be perceived. Here, we investigate whether this type of dichoptic motion can be observed at different depth planes. Four observers indicated direction of motion for dichoptic motion stimuli, presented in a context containing crossed and uncrossed disparity information in different conditions. Our results show that despite the presence of disparity cues in the stimulus, discrimination of motion direction remained maximal at interocular phase offsets that correspond to binocular motion perception at zero disparity. This constraint brings into question perception of dichoptic motion as the result of an early binocular motion system. We compared our results with predictions of a computational stereo-motion model [Qian, N. (1994). Computing stereo disparity and motion with known binocular cell properties. Neural Computations, 6, 390-404; Qian, N., & Andersen, R. A. (1997). A physiological model for motion-stereo integration and a unified explanation of Pulfrich-like phenomena. Vision Research, 37, 1683-1698]. In contrast to our empirical results, simulations of cell activation in this hybrid energy model predict maximal activation at non-zero disparities. It is concluded that perception of dichoptic motion is a by-product of early interocular combination at low contrasts rather than the result of a dedicated stereo-motion system.

Touch influences perceived gloss.

Identifying an object's material properties supports recognition and action planning: we grasp objects according to how heavy, hard or slippery we expect them to be. Visual cues to material qualities such as gloss have recently received attention, but how they interact with haptic (touch) information has been largely overlooked. Here, we show that touch modulates gloss perception: objects that feel slippery are perceived as glossier (more shiny).Participants explored virtual objects that varied in look and feel. A discrimination paradigm (Experiment 1) revealed that observers integrate visual gloss with haptic information. Observers could easily detect an increase in glossiness when it was paired with a decrease in friction. In contrast, increased glossiness coupled with decreased slipperiness produced a small perceptual change: the visual and haptic changes counteracted each other. Subjective ratings (Experiment 2) reflected a similar interaction - slippery objects were rated as glossier and vice versa. The sensory system treats visual gloss and haptic friction as correlated cues to surface material. Although friction is not a perfect predictor of gloss, the visual system appears to know and use a probabilistic relationship between these variables to bias perception - a sensible strategy given the ambiguity of visual clues to gloss.

The Southampton-York Natural Scenes (SYNS) dataset: Statistics of surface attitude.

Recovering 3D scenes from 2D images is an under-constrained task; optimal estimation depends upon knowledge of the underlying scene statistics. Here we introduce the Southampton-York Natural Scenes dataset (SYNS: https://syns.soton.ac.uk), which provides comprehensive scene statistics useful for understanding biological vision and for improving machine vision systems. In order to capture the diversity of environments that humans encounter, scenes were surveyed at random locations within 25 indoor and outdoor categories. Each survey includes (i) spherical LiDAR range data (ii) high-dynamic range spherical imagery and (iii) a panorama of stereo image pairs. We envisage many uses for the dataset and present one example: an analysis of surface attitude statistics, conditioned on scene category and viewing elevation. Surface normals were estimated using a novel adaptive scale selection algorithm. Across categories, surface attitude below the horizon is dominated by the ground plane (0° tilt). Near the horizon, probability density is elevated at 90°/270° tilt due to vertical surfaces (trees, walls). Above the horizon, probability density is elevated near 0° slant due to overhead structure such as ceilings and leaf canopies. These structural regularities represent potentially useful prior assumptions for human and machine observers, and may predict human biases in perceived surface attitude.