Multisensory causal inference is feature-specific, not object-based.

Multisensory integration depends on causal inference about the sensory signals. We tested whether implicit causal-inference judgements pertain to entire objects or focus on task-relevant object features. Participants in our study judged virtual visual, haptic and visual-haptic surfaces with respect to two features-slant and roughness-against an internal standard in a two-alternative forced-choice task. Modelling of participants' responses revealed that the degree to which their perceptual judgements were based on integrated visual-haptic information varied unsystematically across features. For example, a perceived mismatch between visual and haptic roughness would not deter the observer from integrating visual and haptic slant. These results indicate that participants based their perceptual judgements on a feature-specific selection of information, suggesting that multisensory causal inference proceeds not at the object level but at the level of single object features. This article is part of the theme issue 'Decision and control processes in multisensory perception'.

The time-course of real-world scene perception: Spatial and semantic processing.

Real-world scene perception unfolds remarkably quickly, yet the underlying visual processes are poorly understood. Space-centered theory maintains that a scene's spatial structure (e.g., openness, mean depth) can be rapidly recovered from low-level image statistics. In turn, the statistical relationship between a scene's spatial properties and semantic content allows for semantic identity to be inferred from its layout. We tested this theory by investigating (1) the temporal dynamics of spatial and semantic perception in real-world scenes, and (2) dependencies between spatial and semantic judgments. Participants viewed backward-masked images for 13.3 to 106.7 ms, and identified the semantic (e.g., beach, road) or spatial structure (e.g., open, closed-off) category. We found no temporal precedence of spatial discrimination relative to semantic discrimination. Computational analyses further suggest that, instead of using spatial layout to infer semantic categories, humans exploit semantic information to discriminate spatial structure categories. These findings challenge traditional 'bottom-up' views of scene perception.

Investigating Emotional Body Posture Recognition in Adolescents with Conduct Disorder Using Eye-Tracking Methods.

Adolescents with Conduct Disorder (CD) show deficits in recognizing facial expressions of emotion, but it is not known whether these difficulties extend to other social cues, such as emotional body postures. Moreover, in the absence of eye-tracking data, it is not known whether such deficits, if present, are due to a failure to attend to emotionally informative regions of the body. Male and female adolescents with CD and varying levels of callous-unemotional (CU) traits (n = 45) and age- and sex-matched typically-developing controls (n = 51) categorized static and dynamic emotional body postures. The emotion categorization task was paired with eye-tracking methods to investigate relationships between fixation behavior and recognition performance. Having CD was associated with impaired recognition of static and dynamic body postures and atypical fixation behavior. Furthermore, males were less likely to fixate emotionally-informative regions of the body than females. While we found no effects of CU traits on body posture recognition, the effects of CU traits on fixation behavior varied according to CD status and sex, with CD males with lower levels of CU traits showing the most atypical fixation behavior. Critically, atypical fixation behavior did not explain the body posture recognition deficits observed in CD. Our findings suggest that CD-related impairments in recognition of body postures of emotion are not due to attentional issues. Training programmes designed to ameliorate the emotion recognition difficulties associated with CD may need to incorporate a body posture component.

Category systems for real-world scenes.

Categorization performance is a popular metric of scene recognition and understanding in behavioral and computational research. However, categorical constructs and their labels can be somewhat arbitrary. Derived from exhaustive vocabularies of place names (e.g., Deng et al., 2009), or the judgements of small groups of researchers (e.g., Fei-Fei, Iyer, Koch, & Perona, 2007), these categories may not correspond with human-preferred taxonomies. Here, we propose clustering by increasing the rand index via coordinate ascent (CIRCA): an unsupervised, data-driven clustering method for deriving ground-truth scene categories. In Experiment 1, human participants organized 80 stereoscopic images of outdoor scenes from the Southampton-York Natural Scenes (SYNS) dataset (Adams et al., 2016) into discrete categories. In separate tasks, images were grouped according to i) semantic content, ii) three-dimensional spatial structure, or iii) two-dimensional image appearance. Participants provided text labels for each group. Using the CIRCA method, we determined the most representative category structure and then derived category labels for each task/dimension. In Experiment 2, we found that these categories generalized well to a larger set of SYNS images, and new observers. In Experiment 3, we tested the relationship between our category systems and the spatial envelope model (Oliva & Torralba, 2001). Finally, in Experiment 4, we validated CIRCA on a larger, independent dataset of same-different category judgements. The derived category systems outperformed the SUN taxonomy (Xiao, Hays, Ehinger, Oliva, & Torralba, 2010) and an alternative clustering method (Greene, 2019). In summary, we believe this novel categorization method can be applied to a wide range of datasets to derive optimal categorical groupings and labels from psychophysical judgements of stimulus similarity.

Shape from shading under inconsistent illumination.

People are able to perceive the 3D shape of illuminated surfaces using image shading cues. Theories about how we accomplish this often assume that the human visual system estimates a single lighting direction and interprets shading cues in accord with that estimate. In natural scenes, however, lighting can be much more complex than this, with multiple nearby light sources. Here we show that the human visual system can successfully judge 3D surface shape even when the lighting direction varies from place to place over a surface, provided the scale at which these lighting changes occur is similar to, or larger than, the size of the shape features being judged. Furthermore, we show that despite being able to accommodate rapid changes in lighting direction when judging shape, observers are generally unable to detect these changes. We conclude that, rather than relying on a single estimated illumination direction, the human visual system can accommodate illumination that varies substantially and rapidly across a surface.

Disruptive coloration and binocular disparity: breaking camouflage.

Many species employ camouflage to disguise their true shape and avoid detection or recognition. Disruptive coloration is a form of camouflage in which high-contrast patterns obscure internal features or break up an animal's outline. In particular, edge enhancement creates illusory, or 'fake' depth edges within the animal's body. Disruptive coloration often co-occurs with background matching, and together, these strategies make it difficult for an observer to visually segment an animal from its background. However, stereoscopic vision could provide a critical advantage in the arms race between perception and camouflage: the depth information provided by binocular disparities reveals the true three-dimensional layout of a scene, and might, therefore, help an observer to overcome the effects of disruptive coloration. Human observers located snake targets embedded in leafy backgrounds. We analysed performance (response time) as a function of edge enhancement, illumination conditions and the availability of binocular depth cues. We confirm that edge enhancement contributes to effective camouflage: observers were slower to find snakes whose patterning contains 'fake' depth edges. Importantly, however, this effect disappeared when binocular depth cues were available. Illumination also affected detection: under directional illumination, where both the leaves and snake produced strong cast shadows, snake targets were localized more quickly than in scenes rendered under ambient illumination. In summary, we show that illusory depth edges, created via disruptive coloration, help to conceal targets from human observers. However, cast shadows and binocular depth information improve detection by providing information about the true three-dimensional structure of a scene. Importantly, the strong interaction between disparity and edge enhancement suggests that stereoscopic vision has a critical role in breaking camouflage, enabling the observer to overcome the disruptive effects of edge enhancement.

Do emotional faces capture attention, and does this depend on awareness? Evidence from the visual probe paradigm.

The visual probe (VP) paradigm provides evidence that emotional stimuli attract attention. Such effects have been reported even when stimuli are presented outside of awareness. These findings have shaped the idea that humans possess a processing pathway that detects evolutionarily significant signals independently of awareness. Here, we addressed 2 unresolved questions: First, if emotional stimuli attract attention, is this driven by their affective content, or by low-level image properties (e.g., luminance contrast)? Second, does attentional capture occur under conditions of genuine unawareness? We found that observers preferentially allocated attention to emotional faces under aware viewing conditions. However, this effect was best explained by low-level stimulus properties, rather than emotional content. When stimuli were presented outside of awareness (via continuous flash suppression or masking), we found no evidence that attention was directed toward emotional face stimuli. Finally, observer's awareness of the stimuli (assessed by d') predicted attentional cuing. Our data challenge existing literature: First, we cast doubt on the notion of preferential attention to emotional stimuli in the absence of awareness. Second, we question whether effects revealed by the VP paradigm genuinely reflect emotion-sensitive processes, instead suggesting they can be more parsimoniously explained by low-level variability between stimuli. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Motion adaptation and attention: A critical review and meta-analysis.

The motion aftereffect (MAE) provides a behavioural probe into the mechanisms underlying motion perception, and has been used to study the effects of attention on motion processing. Visual attention can enhance detection and discrimination of selected visual signals. However, the relationship between attention and motion processing remains contentious: not all studies find that attention increases MAEs. Our meta-analysis reveals several factors that explain superficially discrepant findings. Across studies (37 independent samples, 76 effects) motion adaptation was significantly and substantially enhanced by attention (Cohen's d = 1.12, p < .0001). The effect more than doubled when adapting to translating (vs. expanding or rotating) motion. Other factors affecting the attention-MAE relationship included stimulus size, eccentricity and speed. By considering these behavioural analyses alongside neurophysiological work, we conclude that feature-based (rather than spatial, or object-based) attention is the biggest driver of sensory adaptation. Comparisons between naïve and non-naïve observers, different response paradigms, and assessment of 'file-drawer effects' indicate that neither response bias nor publication bias are likely to have significantly inflated the estimated effect of attention.

Naturally glossy: Gloss perception, illumination statistics, and tone mapping.

Recognizing materials and understanding their properties is very useful-perhaps critical-in daily life as we encounter objects and plan our interactions with them. Visually derived estimates of material properties guide where and with what force we grasp an object. However, the estimation of material properties, such as glossiness, is a classic ill-posed problem. Image cues that we rely on to estimate gloss are also affected by shape, illumination and, in visual displays, tone-mapping. Here, we focus on the latter two. We define some commonalities present in the structure of natural illumination, and determine whether manipulation of these natural "signatures" impedes gloss constancy. We manipulate the illumination field to violate statistical regularities of natural illumination, such that light comes from below, or the luminance distribution is no longer skewed. These manipulations result in errors in perceived gloss. Similarly, tone mapping has a dramatic effect on perceived gloss. However, when objects are viewed against an informative (rather than plain gray) background that reflects these manipulations, there are some improvements to gloss constancy: in particular, observers are far less susceptible to the effects of tone mapping when judging gloss. We suggest that observers are sensitive to some very simple statistics of the environment when judging gloss.

The effects of attention and adaptation duration on the motion aftereffect.

The motion aftereffect (MAE) is the perception of illusory motion following extended exposure to a moving stimulus. The MAE has been used to probe the role of attention in motion processing. Many studies have reported that MAEs are reduced if attention is diverted from the adaptation stimulus, but others have argued that motion adaptation is independent of attention. We explored several factors that might modulate the attention-adaptation relationship and therefore explain apparent inconsistencies, namely (a) adaptation duration, (b) motion type: translating versus complex, and (c) response bias. Participants viewed translating (Experiments 1a and 2) or rotating (Experiment 1b) random dot patterns while fixating a central letter stream. During adaptation, participants reported brief changes in the adaptor speed (attention-focused) or the presence of white vowels within the letter stream (attention-diverted). Trials consisted of multiple adaptation-test cycles, and the MAE was measured after each adaptation period. Across experiments, focused attention produced significantly larger MAEs than did diverted attention (15% change, Cohen's d = .41). Attention affected the MAE asymptote, rather than its accumulation rate, and had larger effects for translational than for complex motion. The effect of attention remained evident after controlling for response bias. Our results suggest that attention affects multiple levels of the motion-processing hierarchy: not only higher level motion processing, as seen with apparent motion, but also low-level motion processing, as evidenced by the MAE. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

The Southampton-York Natural Scenes (SYNS) dataset: Statistics of surface attitude.

Recovering 3D scenes from 2D images is an under-constrained task; optimal estimation depends upon knowledge of the underlying scene statistics. Here we introduce the Southampton-York Natural Scenes dataset (SYNS: https://syns.soton.ac.uk), which provides comprehensive scene statistics useful for understanding biological vision and for improving machine vision systems. In order to capture the diversity of environments that humans encounter, scenes were surveyed at random locations within 25 indoor and outdoor categories. Each survey includes (i) spherical LiDAR range data (ii) high-dynamic range spherical imagery and (iii) a panorama of stereo image pairs. We envisage many uses for the dataset and present one example: an analysis of surface attitude statistics, conditioned on scene category and viewing elevation. Surface normals were estimated using a novel adaptive scale selection algorithm. Across categories, surface attitude below the horizon is dominated by the ground plane (0° tilt). Near the horizon, probability density is elevated at 90°/270° tilt due to vertical surfaces (trees, walls). Above the horizon, probability density is elevated near 0° slant due to overhead structure such as ceilings and leaf canopies. These structural regularities represent potentially useful prior assumptions for human and machine observers, and may predict human biases in perceived surface attitude.

The Development of Audio-Visual Integration for Temporal Judgements.

Adults combine information from different sensory modalities to estimate object properties such as size or location. This process is optimal in that (i) sensory information is weighted according to relative reliability: more reliable estimates have more influence on the combined estimate and (ii) the combined estimate is more reliable than the component uni-modal estimates. Previous studies suggest that optimal sensory integration does not emerge until around 10 years of age. Younger children rely on a single modality or combine information using inappropriate sensory weights. Children aged 4-11 and adults completed a simple audio-visual task in which they reported either the number of beeps or the number of flashes in uni-modal and bi-modal conditions. In bi-modal trials, beeps and flashes differed in number by 0, 1 or 2. Mutual interactions between the sensory signals were evident at all ages: the reported number of flashes was influenced by the number of simultaneously presented beeps and vice versa. Furthermore, for all ages, the relative strength of these interactions was predicted by the relative reliabilities of the two modalities, in other words, all observers weighted the signals appropriately. The degree of cross-modal interaction decreased with age: the youngest observers could not ignore the task-irrelevant modality-they fully combined vision and audition such that they perceived equal numbers of flashes and beeps for bi-modal stimuli. Older observers showed much smaller effects of the task-irrelevant modality. Do these interactions reflect optimal integration? Full or partial cross-modal integration predicts improved reliability in bi-modal conditions. In contrast, switching between modalities reduces reliability. Model comparison suggests that older observers employed partial integration, whereas younger observers (up to around 8 years) did not integrate, but followed a sub-optimal switching strategy, responding according to either visual or auditory information on each trial.

Are visual threats prioritized without awareness? A critical review and meta-analysis involving 3 behavioral paradigms and 2696 observers.

Given capacity limits, only a subset of stimuli give rise to a conscious percept. Neurocognitive models suggest that humans have evolved mechanisms that operate without awareness and prioritize threatening stimuli over neutral stimuli in subsequent perception. In this meta-analysis, we review evidence for this 'standard hypothesis' emanating from 3 widely used, but rather different experimental paradigms that have been used to manipulate awareness. We found a small pooled threat-bias effect in the masked visual probe paradigm, a medium effect in the binocular rivalry paradigm and highly inconsistent effects in the breaking continuous flash suppression paradigm. Substantial heterogeneity was explained by the stimulus type: the only threat stimuli that were robustly prioritized across all 3 paradigms were fearful faces. Meta regression revealed that anxiety may modulate threat-biases, but only under specific presentation conditions. We also found that insufficiently rigorous awareness measures, inadequate control of response biases and low level confounds may undermine claims of genuine unconscious threat processing. Considering the data together, we suggest that uncritical acceptance of the standard hypothesis is premature: current behavioral evidence for threat-sensitive visual processing that operates without awareness is weak. (PsycINFO Database Record

Does comorbid anxiety counteract emotion recognition deficits in conduct disorder?

BACKGROUND: Previous research has reported altered emotion recognition in both conduct disorder (CD) and anxiety disorders (ADs) - but these effects appear to be of different kinds. Adolescents with CD often show a generalised pattern of deficits, while those with ADs show hypersensitivity to specific negative emotions. Although these conditions often cooccur, little is known regarding emotion recognition performance in comorbid CD+ADs. Here, we test the hypothesis that in the comorbid case, anxiety-related emotion hypersensitivity counteracts the emotion recognition deficits typically observed in CD. METHOD: We compared facial emotion recognition across four groups of adolescents aged 12-18 years: those with CD alone (n = 28), ADs alone (n = 23), cooccurring CD+ADs (n = 20) and typically developing controls (n = 28). The emotion recognition task we used systematically manipulated the emotional intensity of facial expressions as well as fixation location (eye, nose or mouth region). RESULTS: Conduct disorder was associated with a generalised impairment in emotion recognition; however, this may have been modulated by group differences in IQ. AD was associated with increased sensitivity to low-intensity happiness, disgust and sadness. In general, the comorbid CD+ADs group performed similarly to typically developing controls. CONCLUSIONS: Although CD alone was associated with emotion recognition impairments, ADs and comorbid CD+ADs were associated with normal or enhanced emotion recognition performance. The presence of comorbid ADs appeared to counteract the effects of CD, suggesting a potentially protective role, although future research should examine the contribution of IQ and gender to these effects.

Touch influences perceived gloss.

Identifying an object's material properties supports recognition and action planning: we grasp objects according to how heavy, hard or slippery we expect them to be. Visual cues to material qualities such as gloss have recently received attention, but how they interact with haptic (touch) information has been largely overlooked. Here, we show that touch modulates gloss perception: objects that feel slippery are perceived as glossier (more shiny).Participants explored virtual objects that varied in look and feel. A discrimination paradigm (Experiment 1) revealed that observers integrate visual gloss with haptic information. Observers could easily detect an increase in glossiness when it was paired with a decrease in friction. In contrast, increased glossiness coupled with decreased slipperiness produced a small perceptual change: the visual and haptic changes counteracted each other. Subjective ratings (Experiment 2) reflected a similar interaction - slippery objects were rated as glossier and vice versa. The sensory system treats visual gloss and haptic friction as correlated cues to surface material. Although friction is not a perfect predictor of gloss, the visual system appears to know and use a probabilistic relationship between these variables to bias perception - a sensible strategy given the ambiguity of visual clues to gloss.

Fearful faces have a sensory advantage in the competition for awareness.

Only a subset of visual signals give rise to a conscious percept. Threat signals, such as fearful faces, are particularly salient to human vision. Research suggests that fearful faces are evaluated without awareness and preferentially promoted to conscious perception. This agrees with evolutionary theories that posit a dedicated pathway specialized in processing threat-relevant signals. We propose an alternative explanation for this "fear advantage." Using psychophysical data from continuous flash suppression (CFS) and masking experiments, we demonstrate that awareness of facial expressions is predicted by effective contrast: the relationship between their Fourier spectrum and the contrast sensitivity function. Fearful faces have higher effective contrast than neutral expressions and this, not threat content, predicts their enhanced access to awareness. Importantly, our findings do not support the existence of a specialized mechanism that promotes threatening stimuli to awareness. Rather, our data suggest that evolutionary or learned adaptations have molded the fearful expression to exploit our general-purpose sensory mechanisms.

Autonomic arousal and attentional orienting to visual threat are predicted by awareness.

The rapid detection and evaluation of threat is of fundamental importance for survival. Theories suggest that this evolutionary pressure has driven functional adaptations in a specialized visual pathway that evaluates threat independently of conscious awareness. This is supported by evidence that threat-relevant stimuli rendered invisible by backward masking can induce physiological fear responses and modulate spatial attention. The validity of these findings has since been questioned by research using stringent, objective measures of awareness. Here, we use a modified continuous flash suppression paradigm to ask whether threatening images induce adaptive changes in autonomic arousal, attention, or perception when presented outside of awareness. In trials where stimuli broke suppression to become visible, threatening stimuli induced a significantly larger skin conductance response than nonthreatening stimuli and attracted spatial attention over scrambled images. However, these effects were eliminated in trials where observers were unaware of the stimuli. In addition, concurrent behavioral data provided no evidence that threatening images gained prioritized access to awareness. Taken together, our data suggest that the evaluation and spatial detection of visual threat are predicted by awareness.

Effects of specular highlights on perceived surface convexity.

Shading is known to produce vivid perceptions of depth. However, the influence of specular highlights on perceived shape is unclear: some studies have shown that highlights improve quantitative shape perception while others have shown no effect. Here we ask how specular highlights combine with Lambertian shading cues to determine perceived surface curvature, and to what degree this is based upon a coherent model of the scene geometry. Observers viewed ambiguous convex/concave shaded surfaces, with or without highlights. We show that the presence/absence of specular highlights has an effect on qualitative shape, their presence biasing perception toward convex interpretations of ambiguous shaded objects. We also find that the alignment of a highlight with the Lambertian shading modulates its effect on perceived shape; misaligned highlights are less likely to be perceived as specularities, and thus have less effect on shape perception. Increasing the depth of the surface or the slant of the illuminant also modulated the effect of the highlight, increasing the bias toward convexity. The effect of highlights on perceived shape can be understood probabilistically in terms of scene geometry: for deeper objects and/or highly slanted illuminants, highlights will occur on convex but not concave surfaces, due to occlusion of the illuminant. Given uncertainty about the exact object depth and illuminant direction, the presence of a highlight increases the probability that the surface is convex.

Learning different light prior distributions for different contexts.

The pattern of shading across an image can provide a rich sense of object shape. Our ability to use shading information is remarkable given the infinite possible combinations of illumination, shape and reflectance that could have produced any given image. Illumination can change dramatically across environments (e.g., indoor vs. outdoor) and times of day (e.g., mid-day vs. sunset). Here we show that people can learn to associate particular illumination conditions with particular contexts, to aid shape-from-shading. Following a few hours of visual-haptic training, observers modified their shape estimates according to the illumination expected in the prevailing context. Our observers learned that red lighting was roughly overhead (consistent with their previous assumption of lighting direction), whereas green lighting was shifted by 10°. Greater learning occurred when training for the two contexts (red or green light) was intermingled rather than when it was sequentially blocked.