Category systems for real-world scenes.

Categorization performance is a popular metric of scene recognition and understanding in behavioral and computational research. However, categorical constructs and their labels can be somewhat arbitrary. Derived from exhaustive vocabularies of place names (e.g., Deng et al., 2009), or the judgements of small groups of researchers (e.g., Fei-Fei, Iyer, Koch, & Perona, 2007), these categories may not correspond with human-preferred taxonomies. Here, we propose clustering by increasing the rand index via coordinate ascent (CIRCA): an unsupervised, data-driven clustering method for deriving ground-truth scene categories. In Experiment 1, human participants organized 80 stereoscopic images of outdoor scenes from the Southampton-York Natural Scenes (SYNS) dataset (Adams et al., 2016) into discrete categories. In separate tasks, images were grouped according to i) semantic content, ii) three-dimensional spatial structure, or iii) two-dimensional image appearance. Participants provided text labels for each group. Using the CIRCA method, we determined the most representative category structure and then derived category labels for each task/dimension. In Experiment 2, we found that these categories generalized well to a larger set of SYNS images, and new observers. In Experiment 3, we tested the relationship between our category systems and the spatial envelope model (Oliva & Torralba, 2001). Finally, in Experiment 4, we validated CIRCA on a larger, independent dataset of same-different category judgements. The derived category systems outperformed the SUN taxonomy (Xiao, Hays, Ehinger, Oliva, & Torralba, 2010) and an alternative clustering method (Greene, 2019). In summary, we believe this novel categorization method can be applied to a wide range of datasets to derive optimal categorical groupings and labels from psychophysical judgements of stimulus similarity.

Disruptive coloration and binocular disparity: breaking camouflage.

Many species employ camouflage to disguise their true shape and avoid detection or recognition. Disruptive coloration is a form of camouflage in which high-contrast patterns obscure internal features or break up an animal's outline. In particular, edge enhancement creates illusory, or 'fake' depth edges within the animal's body. Disruptive coloration often co-occurs with background matching, and together, these strategies make it difficult for an observer to visually segment an animal from its background. However, stereoscopic vision could provide a critical advantage in the arms race between perception and camouflage: the depth information provided by binocular disparities reveals the true three-dimensional layout of a scene, and might, therefore, help an observer to overcome the effects of disruptive coloration. Human observers located snake targets embedded in leafy backgrounds. We analysed performance (response time) as a function of edge enhancement, illumination conditions and the availability of binocular depth cues. We confirm that edge enhancement contributes to effective camouflage: observers were slower to find snakes whose patterning contains 'fake' depth edges. Importantly, however, this effect disappeared when binocular depth cues were available. Illumination also affected detection: under directional illumination, where both the leaves and snake produced strong cast shadows, snake targets were localized more quickly than in scenes rendered under ambient illumination. In summary, we show that illusory depth edges, created via disruptive coloration, help to conceal targets from human observers. However, cast shadows and binocular depth information improve detection by providing information about the true three-dimensional structure of a scene. Importantly, the strong interaction between disparity and edge enhancement suggests that stereoscopic vision has a critical role in breaking camouflage, enabling the observer to overcome the disruptive effects of edge enhancement.

Shape from shading under inconsistent illumination.

People are able to perceive the 3D shape of illuminated surfaces using image shading cues. Theories about how we accomplish this often assume that the human visual system estimates a single lighting direction and interprets shading cues in accord with that estimate. In natural scenes, however, lighting can be much more complex than this, with multiple nearby light sources. Here we show that the human visual system can successfully judge 3D surface shape even when the lighting direction varies from place to place over a surface, provided the scale at which these lighting changes occur is similar to, or larger than, the size of the shape features being judged. Furthermore, we show that despite being able to accommodate rapid changes in lighting direction when judging shape, observers are generally unable to detect these changes. We conclude that, rather than relying on a single estimated illumination direction, the human visual system can accommodate illumination that varies substantially and rapidly across a surface.

Naturally glossy: Gloss perception, illumination statistics, and tone mapping.

Recognizing materials and understanding their properties is very useful-perhaps critical-in daily life as we encounter objects and plan our interactions with them. Visually derived estimates of material properties guide where and with what force we grasp an object. However, the estimation of material properties, such as glossiness, is a classic ill-posed problem. Image cues that we rely on to estimate gloss are also affected by shape, illumination and, in visual displays, tone-mapping. Here, we focus on the latter two. We define some commonalities present in the structure of natural illumination, and determine whether manipulation of these natural "signatures" impedes gloss constancy. We manipulate the illumination field to violate statistical regularities of natural illumination, such that light comes from below, or the luminance distribution is no longer skewed. These manipulations result in errors in perceived gloss. Similarly, tone mapping has a dramatic effect on perceived gloss. However, when objects are viewed against an informative (rather than plain gray) background that reflects these manipulations, there are some improvements to gloss constancy: in particular, observers are far less susceptible to the effects of tone mapping when judging gloss. We suggest that observers are sensitive to some very simple statistics of the environment when judging gloss.

The Development of Audio-Visual Integration for Temporal Judgements.

Adults combine information from different sensory modalities to estimate object properties such as size or location. This process is optimal in that (i) sensory information is weighted according to relative reliability: more reliable estimates have more influence on the combined estimate and (ii) the combined estimate is more reliable than the component uni-modal estimates. Previous studies suggest that optimal sensory integration does not emerge until around 10 years of age. Younger children rely on a single modality or combine information using inappropriate sensory weights. Children aged 4-11 and adults completed a simple audio-visual task in which they reported either the number of beeps or the number of flashes in uni-modal and bi-modal conditions. In bi-modal trials, beeps and flashes differed in number by 0, 1 or 2. Mutual interactions between the sensory signals were evident at all ages: the reported number of flashes was influenced by the number of simultaneously presented beeps and vice versa. Furthermore, for all ages, the relative strength of these interactions was predicted by the relative reliabilities of the two modalities, in other words, all observers weighted the signals appropriately. The degree of cross-modal interaction decreased with age: the youngest observers could not ignore the task-irrelevant modality-they fully combined vision and audition such that they perceived equal numbers of flashes and beeps for bi-modal stimuli. Older observers showed much smaller effects of the task-irrelevant modality. Do these interactions reflect optimal integration? Full or partial cross-modal integration predicts improved reliability in bi-modal conditions. In contrast, switching between modalities reduces reliability. Model comparison suggests that older observers employed partial integration, whereas younger observers (up to around 8 years) did not integrate, but followed a sub-optimal switching strategy, responding according to either visual or auditory information on each trial.

Does comorbid anxiety counteract emotion recognition deficits in conduct disorder?

BACKGROUND: Previous research has reported altered emotion recognition in both conduct disorder (CD) and anxiety disorders (ADs) - but these effects appear to be of different kinds. Adolescents with CD often show a generalised pattern of deficits, while those with ADs show hypersensitivity to specific negative emotions. Although these conditions often cooccur, little is known regarding emotion recognition performance in comorbid CD+ADs. Here, we test the hypothesis that in the comorbid case, anxiety-related emotion hypersensitivity counteracts the emotion recognition deficits typically observed in CD. METHOD: We compared facial emotion recognition across four groups of adolescents aged 12-18 years: those with CD alone (n = 28), ADs alone (n = 23), cooccurring CD+ADs (n = 20) and typically developing controls (n = 28). The emotion recognition task we used systematically manipulated the emotional intensity of facial expressions as well as fixation location (eye, nose or mouth region). RESULTS: Conduct disorder was associated with a generalised impairment in emotion recognition; however, this may have been modulated by group differences in IQ. AD was associated with increased sensitivity to low-intensity happiness, disgust and sadness. In general, the comorbid CD+ADs group performed similarly to typically developing controls. CONCLUSIONS: Although CD alone was associated with emotion recognition impairments, ADs and comorbid CD+ADs were associated with normal or enhanced emotion recognition performance. The presence of comorbid ADs appeared to counteract the effects of CD, suggesting a potentially protective role, although future research should examine the contribution of IQ and gender to these effects.

Effects of specular highlights on perceived surface convexity.

Shading is known to produce vivid perceptions of depth. However, the influence of specular highlights on perceived shape is unclear: some studies have shown that highlights improve quantitative shape perception while others have shown no effect. Here we ask how specular highlights combine with Lambertian shading cues to determine perceived surface curvature, and to what degree this is based upon a coherent model of the scene geometry. Observers viewed ambiguous convex/concave shaded surfaces, with or without highlights. We show that the presence/absence of specular highlights has an effect on qualitative shape, their presence biasing perception toward convex interpretations of ambiguous shaded objects. We also find that the alignment of a highlight with the Lambertian shading modulates its effect on perceived shape; misaligned highlights are less likely to be perceived as specularities, and thus have less effect on shape perception. Increasing the depth of the surface or the slant of the illuminant also modulated the effect of the highlight, increasing the bias toward convexity. The effect of highlights on perceived shape can be understood probabilistically in terms of scene geometry: for deeper objects and/or highly slanted illuminants, highlights will occur on convex but not concave surfaces, due to occlusion of the illuminant. Given uncertainty about the exact object depth and illuminant direction, the presence of a highlight increases the probability that the surface is convex.

Highlights, disparity, and perceived gloss with convex and concave surfaces.

Glossy and matte objects can be differentiated using specular highlights: bright patches in the retinal image produced when light rays are reflected regularly from smooth surfaces. However, bright patches also occur on matte objects, due to local illumination or reflectance changes. Binocular vision provides information that could distinguish specular highlights from other luminance discontinuities; unlike surface markings, specular highlights lie not at the surface depth, but "float" in front of concave surfaces and behind convex ones. We ask whether observers implicitly understand and exploit the peculiarities of specular geometry for gloss and shape perception. Our participants judged the glossiness and shape of curved surfaces that included specular highlights at various depths. Observers demonstrated substantial deviations from a full geometric model of specular reflection. Concave surfaces appeared glossy both when highlights lay in front of and (incorrectly) behind the surface. Failings in the interpretation of monocular highlights were also apparent. Highlight disparity had no effect on shape perception. However, the perceived gloss of convex surfaces did follow geometric constraints: only highlights at appropriate depths produced high gloss ratings. We suggest, in contrast with previous work, that the visual system invokes simple heuristics as gloss indicators to accommodate complex reflections and inter-reflections that occur particularly inside concavities.

Multisensory causal inference is feature-specific, not object-based.

Multisensory integration depends on causal inference about the sensory signals. We tested whether implicit causal-inference judgements pertain to entire objects or focus on task-relevant object features. Participants in our study judged virtual visual, haptic and visual-haptic surfaces with respect to two features-slant and roughness-against an internal standard in a two-alternative forced-choice task. Modelling of participants' responses revealed that the degree to which their perceptual judgements were based on integrated visual-haptic information varied unsystematically across features. For example, a perceived mismatch between visual and haptic roughness would not deter the observer from integrating visual and haptic slant. These results indicate that participants based their perceptual judgements on a feature-specific selection of information, suggesting that multisensory causal inference proceeds not at the object level but at the level of single object features. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

The time-course of real-world scene perception: Spatial and semantic processing.

Real-world scene perception unfolds remarkably quickly, yet the underlying visual processes are poorly understood. Space-centered theory maintains that a scene's spatial structure (e.g., openness, mean depth) can be rapidly recovered from low-level image statistics. In turn, the statistical relationship between a scene's spatial properties and semantic content allows for semantic identity to be inferred from its layout. We tested this theory by investigating (1) the temporal dynamics of spatial and semantic perception in real-world scenes, and (2) dependencies between spatial and semantic judgments. Participants viewed backward-masked images for 13.3 to 106.7 ms, and identified the semantic (e.g., beach, road) or spatial structure (e.g., open, closed-off) category. We found no temporal precedence of spatial discrimination relative to semantic discrimination. Computational analyses further suggest that, instead of using spatial layout to infer semantic categories, humans exploit semantic information to discriminate spatial structure categories. These findings challenge traditional 'bottom-up' views of scene perception.

Efficient visual recalibration from either visual or haptic feedback: the importance of being wrong.

The human visual system adapts to the changing statistics of its environment. For example, the light-from-above prior, an assumption that aids the interpretation of ambiguous shading information, can be modified by haptic (touch) feedback. Here we investigate the mechanisms that drive this adaptive learning. In particular, we ask whether visual information can be as effective as haptics in driving visual recalibration and whether increased information (feedback from multiple modalities) induces faster learning. During several hours' training, feedback encouraged observers to modify their existing light-from-above assumption. Feedback was one of the following: (1) haptic only, (2) haptic and stereoscopic (providing binocular shape information), or (3) stereoscopic only. Haptic-only feedback resulted in substantial learning; the perceived shape of shaded objects was modified in accordance with observers' new light priors. However, the addition of continuous visual feedback (condition 2) substantially reduced learning. When visual-only feedback was provided intermittently (condition 3), mimicking the time course of the haptic feedback of conditions 1 and 2, substantial learning returned. The intermittent nature of conflict information, or feedback, appears critical for learning. It causes an initial, erroneous percept to be corrected. Contrary to previous proposals, we found no particular advantage for cross-modal feedback. Instead, we suggest that an "oops" factor drives efficient learning; recalibration is prioritized when a mismatch exists between sequential representations of an object property. This "oops" factor appears important both across and within sensory modalities, suggesting a general principle for perceptual learning and recalibration.

Investigating Emotional Body Posture Recognition in Adolescents with Conduct Disorder Using Eye-Tracking Methods.

Adolescents with Conduct Disorder (CD) show deficits in recognizing facial expressions of emotion, but it is not known whether these difficulties extend to other social cues, such as emotional body postures. Moreover, in the absence of eye-tracking data, it is not known whether such deficits, if present, are due to a failure to attend to emotionally informative regions of the body. Male and female adolescents with CD and varying levels of callous-unemotional (CU) traits (n = 45) and age- and sex-matched typically-developing controls (n = 51) categorized static and dynamic emotional body postures. The emotion categorization task was paired with eye-tracking methods to investigate relationships between fixation behavior and recognition performance. Having CD was associated with impaired recognition of static and dynamic body postures and atypical fixation behavior. Furthermore, males were less likely to fixate emotionally-informative regions of the body than females. While we found no effects of CU traits on body posture recognition, the effects of CU traits on fixation behavior varied according to CD status and sex, with CD males with lower levels of CU traits showing the most atypical fixation behavior. Critically, atypical fixation behavior did not explain the body posture recognition deficits observed in CD. Our findings suggest that CD-related impairments in recognition of body postures of emotion are not due to attentional issues. Training programmes designed to ameliorate the emotion recognition difficulties associated with CD may need to incorporate a body posture component.

High-level face adaptation without awareness.

When a visual stimulus is suppressed from awareness, processing of the suppressed image is necessarily reduced. Although adaptation to simple image properties such as orientation still occurs, adaptation to high-level properties such as face identity is eliminated. Here we show that emotional facial expression continues to be processed even under complete suppression, as indexed by substantial facial expression aftereffects.

Motion-aftereffect-induced blindness.

Motion-induced blindness (MIB) describes the occasional disappearance of salient visual objects in the presence of moving features (Y. S. Bonneh, A. Cooperman, & D. Sagi, 2001). Here we test whether motion adaptation and the ensuing motion aftereffect (MAE) are sufficient to trigger disappearance of salient targets. In three experiments, observers adapted to either rotating or static stimuli. Immediately afterwards, a static test pattern was presented consisting of a mask with texture elements and three superimposed target dots in a triangular arrangement. Observers reported dot disappearance and reappearance. The results clearly show that illusory motion in a static test pattern, following motion adaptation, promotes the disappearance of target dots. Furthermore, disappearance is modulated by the depth relationship between test pattern and targets, increasing for targets placed stereoscopically behind the test pattern. We conclude that MIB is influenced by perceived relative motion between depth-segregated features.

The fate of task-irrelevant visual motion: perceptual load versus feature-based attention.

We tested contrasting predictions derived from perceptual load theory and from recent feature-based selection accounts. Observers viewed moving, colored stimuli and performed low or high load tasks associated with one stimulus feature, either color or motion. The resultant motion aftereffect (MAE) was used to evaluate attentional allocation. We found that task-irrelevant visual features received less attention than co-localized task-relevant features of the same objects. Moreover, when color and motion features were co-localized yet perceived to belong to two distinct surfaces, feature-based selection was further increased at the expense of object-based co-selection. Load theory predicts that the MAE for task-irrelevant motion would be reduced with a higher load color task. However, this was not seen for co-localized features; perceptual load only modulated the MAE for task-irrelevant motion when this was spatially separated from the attended color location. Our results suggest that perceptual load effects are mediated by spatial selection and do not generalize to the feature domain. Feature-based selection operates to suppress processing of task-irrelevant, co-localized features, irrespective of perceptual load.

The spatial scale of perceptual memory in ambiguous figure perception.

Ambiguous visual stimuli highlight the constructive nature of vision: perception alternates between two plausible interpretations of unchanging input. However, when a previously viewed ambiguous stimulus reappears, its earlier perception almost entirely determines the new interpretation; memory disambiguates the input. Here, we investigate the spatial properties of this perceptual memory, taking into account strong anisotropies in percept preference across the visual field. Countering previous findings, we show that perceptual memory is not confined to the location in which it was instilled. Rather, it spreads to noncontiguous regions of the visual field, falling off at larger distances. Furthermore, this spread of perceptual memory takes place in a frame of reference that is tied to the surface of the retina. These results place the neural locus of perceptual memory in retinotopically organized sensory cortical areas, with implications for the wider function of perceptual memory in facilitating stable vision in natural, dynamic environments.

Do emotional faces capture attention, and does this depend on awareness? Evidence from the visual probe paradigm.

The visual probe (VP) paradigm provides evidence that emotional stimuli attract attention. Such effects have been reported even when stimuli are presented outside of awareness. These findings have shaped the idea that humans possess a processing pathway that detects evolutionarily significant signals independently of awareness. Here, we addressed 2 unresolved questions: First, if emotional stimuli attract attention, is this driven by their affective content, or by low-level image properties (e.g., luminance contrast)? Second, does attentional capture occur under conditions of genuine unawareness? We found that observers preferentially allocated attention to emotional faces under aware viewing conditions. However, this effect was best explained by low-level stimulus properties, rather than emotional content. When stimuli were presented outside of awareness (via continuous flash suppression or masking), we found no evidence that attention was directed toward emotional face stimuli. Finally, observer's awareness of the stimuli (assessed by d') predicted attentional cuing. Our data challenge existing literature: First, we cast doubt on the notion of preferential attention to emotional stimuli in the absence of awareness. Second, we question whether effects revealed by the VP paradigm genuinely reflect emotion-sensitive processes, instead suggesting they can be more parsimoniously explained by low-level variability between stimuli. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Motion adaptation and attention: A critical review and meta-analysis.

The motion aftereffect (MAE) provides a behavioural probe into the mechanisms underlying motion perception, and has been used to study the effects of attention on motion processing. Visual attention can enhance detection and discrimination of selected visual signals. However, the relationship between attention and motion processing remains contentious: not all studies find that attention increases MAEs. Our meta-analysis reveals several factors that explain superficially discrepant findings. Across studies (37 independent samples, 76 effects) motion adaptation was significantly and substantially enhanced by attention (Cohen's d = 1.12, p < .0001). The effect more than doubled when adapting to translating (vs. expanding or rotating) motion. Other factors affecting the attention-MAE relationship included stimulus size, eccentricity and speed. By considering these behavioural analyses alongside neurophysiological work, we conclude that feature-based (rather than spatial, or object-based) attention is the biggest driver of sensory adaptation. Comparisons between naïve and non-naïve observers, different response paradigms, and assessment of 'file-drawer effects' indicate that neither response bias nor publication bias are likely to have significantly inflated the estimated effect of attention.

Frames of reference for the light-from-above prior in visual search and shape judgements.

Faced with highly complex and ambiguous visual input, human observers must rely on prior knowledge and assumptions to efficiently determine the structure of their surroundings. One of these assumptions is the 'light-from-above' prior. In the absence of explicit light-source information, the visual system assumes that the light-source is roughly overhead. A simple, low-cost strategy would place this 'light-from-above' prior in a retinal frame of reference. A more complex, but optimal strategy would be to assume that the light-source is gravitationally up, and compensate for observer orientation. Evidence to support one or other strategy from psychophysics and neurophysiology has been mixed. This paper pits the gravitational and retinal frames against each other in two different visual tasks that relate to the light-from-above prior. In the first task, observers had to report the presence or absence of a target where distractors and target were defined purely by shading. In the second task, observers made explicit shape judgements of similar stimuli. The orientation of the stimuli varied across trials and the observer's head was fixed at 0, +/-45 or +/-60 degrees . In both tasks the retinal frame of reference dominated. Visual search behaviour with shape-from-shading stimuli (SFS) was modulated purely by stimulus orientation relative to the retina. However, the gravitational frame of reference had a significant effect on shape judgements, with a 30% correction for observer orientation. In other words, shading information is processed quite differently depending on the demands of the current task. When a 'quick and dirty' representation is required to drive fast, efficient search, that is what the visual system provides. In contrast, when the task is to explicitly estimate shape, extra processing to compensate for head orientation precedes the perceptual judgment. These results are consistent with current neurophysiological data on SFS if we re-frame compensation for observer orientation as a cue-combination problem.

Surface organization influences bistable vision.

A priority for the visual system is to construct 3-dimensional surfaces from visual primitives. Information is combined across individual cues to form a robust representation of the external world. Here, it is shown that surface completion relying on multiple visual cues influences relative dominance during binocular rivalry. The shape of a surface determined by 1 nonrivalrous visual cue (disparity, structure-from-motion) alters the dominance patterns of a rivalrous region of the image whose shape is defined by another cue (perspective, texture). The findings indicate that contextual information promotes the perception of the member of the rivalrous pair consistent with a smooth surface representation. The results extend the current description of binocular rivalry as a hierarchical process by implying that cue combination affects image selection during bistable viewing.