Searching for serial dependencies in the brain.

Identifying the neural correlates of visual serial dependence has lagged behind the behavioral understanding. A new study in PLOS Biology provides a model of interpreting the complex relationship between physiology and behavior in studies of serial dependence.

The functional role of spatial anisotropies in ensemble perception.

BACKGROUND: The human brain can rapidly represent sets of similar stimuli by their ensemble summary statistics, like the average orientation or size. Classic models assume that ensemble statistics are computed by integrating all elements with equal weight. Challenging this view, here, we show that ensemble statistics are estimated by combining parafoveal and foveal statistics in proportion to their reliability. In a series of experiments, observers reproduced the average orientation of an ensemble of stimuli under varying levels of visual uncertainty. RESULTS: Ensemble statistics were affected by multiple spatial biases, in particular, a strong and persistent bias towards the center of the visual field. This bias, evident in the majority of subjects and in all experiments, scaled with uncertainty: the higher the uncertainty in the ensemble statistics, the larger the bias towards the element shown at the fovea. CONCLUSION: Our findings indicate that ensemble perception cannot be explained by simple uniform pooling. The visual system weights information anisotropically from both the parafovea and the fovea, taking the intrinsic spatial anisotropies of vision into account to compensate for visual uncertainty.

Serial dependence in emotion perception mirrors the autocorrelations in natural emotion statistics.

A critical function of the human visual system is to track emotion accurately and continuously. However, visual information about emotion fluctuates over time. Ideally, the visual system should track these temporal fluctuations-these "natural emotion statistics" of the world-over time. This would balance the need to detect changes in emotion with the need to maintain the stability of visual scene representations. The visual system could promote this goal through serial dependence, which biases our perception of facial expressions toward those seen in the recent past and thus smooths our perception of the world. Here, we quantified the natural emotion statistics in videos by measuring the autocorrelations in emotional content present in films and movies. The results showed that observers' perception of emotion was smoothed over ∼12 seconds or more, and this time-course closely followed the temporal fluctuations in visual information about emotion found in natural scenes. Moreover, the temporal and feature tuning of the perceptual smoothing was consistent with known properties of serial dependence. Our findings suggest that serial dependence is introduced in the perception of emotion to match the natural autocorrelations that are observed in the real world, an operation that could improve the efficiency, sensitivity, and stability of emotion perception.

Serial dependence in visual perception: A meta-analysis and review.

Positive sequential dependencies are phenomena in which actions, perception, decisions, and memory of features or objects are systematically biased toward visual experiences from the recent past. Among many labels, serial dependencies have been referred to as priming, sequential dependencies, sequential effects, or serial effects. Despite extensive research on the topic, the field still lacks an operational definition of what counts as serial dependence. In this meta-analysis, we review the vast literature on serial dependence and quantitatively assess its key diagnostic characteristics across several different domains of visual perception. The meta-analyses fully characterize serial dependence in orientation, face, and numerosity perception. They show that serial dependence is defined by four main kinds of tuning: serial dependence decays with time (temporal-tuning), it depends on relative spatial location (spatial-tuning), it occurs only between similar features and objects (feature-tuning), and it is modulated by attention (attentional-tuning). We also review studies of serial dependence that report single observer data, highlighting the importance of individual differences in serial dependence. Finally, we discuss a range of outstanding questions and novel research avenues that are prompted by the meta-analyses. Together, the meta-analyses provide a full characterization of serial dependence as an operationally defined family of visual phenomena, and they outline several of the key diagnostic criteria for serial dependence that should serve as guideposts for future research.

Tracking the affective state of unseen persons.

Emotion recognition is an essential human ability critical for social functioning. It is widely assumed that identifying facial expression is the key to this, and models of emotion recognition have mainly focused on facial and bodily features in static, unnatural conditions. We developed a method called affective tracking to reveal and quantify the enormous contribution of visual context to affect (valence and arousal) perception. When characters' faces and bodies were masked in silent videos, viewers inferred the affect of the invisible characters successfully and in high agreement based solely on visual context. We further show that the context is not only sufficient but also necessary to accurately perceive human affect over time, as it provides a substantial and unique contribution beyond the information available from face and body. Our method (which we have made publicly available) reveals that emotion recognition is, at its heart, an issue of context as much as it is about faces.

Individual differences in classification images of Mooney faces.

Human face recognition is robust even under conditions of extreme lighting and in situations where there is high noise and uncertainty. Mooney faces are a canonical example of this: Mooney faces are two-tone shadow-defined images that are readily and holistically recognized despite lacking easily segmented face features. Face perception in such impoverished situations-and Mooney face perception in particular-is often thought to be supported by comparing encountered faces to stored templates. Here, we used a classification image approach to measure the templates that observers use to recognize Mooney faces. Visualizing these templates reveals the regions and structures of the image that best predict individual observer recognition, and they reflect the underlying internal representation of faces. Using this approach, we tested whether there are classification images that are consistent from session to session, whether the classification images are observer-specific, and whether they allow for pattern completion of holistic representations even in the absence of an underlying signal. We found that classification images of Mooney faces were indeed non-random (i.e., consistent session from session) within each observer, but they were different between observers. This result is in line with previously proposed existence of face templates that support face recognition, and further suggests that these templates may be unique to each observer and could drive idiosyncratic individual differences in holistic face recognition. Moreover, we found classification images that reflected information within the blank regions of the original Mooney faces, suggesting that observers may fill in missing information using idiosyncratic internal information about faces.

The test-retest reliability and spatial tuning of serial dependence in orientation perception.

Humans perceive objects and scenes consistently, even in situations where visual input is noisy and unstable. One of the mechanisms that underlies this perceptual stability is serial dependence, whereby the perception of objects or features at any given moment is pulled toward what was previously seen. Although recent findings from several studies have reported large individual differences in serial dependence, it is not clear how stable the serial dependence is within an individual. Here, we investigated the stability of serial dependence in orientation perception over two different days within the same observers. In addition, we also examined the visual field location specificity of perceptual serial dependence. On each trial, observers viewed a Gabor patch and then reported its apparent orientation by adjusting the orientation of a bar. For each observer, the Gabor was located in the foveal or peripheral (10° right or left eccentricity) visual field on both days or changed location from day to day. The results showed a very high degree of test-retest reliability in serial dependence measured across days within individual observers. Interestingly, this high within-subject consistency was only found when serial dependence was measured at the same visual field location. These results suggest that individual differences in serial dependence are stable across days, and that the spatiotemporal range in which the previous stimulus assimilates the perception of the current stimulus (the continuity field) may vary across different visual field locations in an observer-specific manner.

Spatio-temporal associations of air pollutant concentrations, GP respiratory consultations and respiratory inhaler prescriptions: a 5-year study of primary care in the borough of Lambeth, South London.

BACKGROUND: Although the associations of outdoor air pollution exposure with mortality and hospital admissions are well established, few previous studies have reported on primary care clinical and prescribing data. We assessed the associations of short and long-term pollutant exposures with General Practitioner respiratory consultations and inhaler prescriptions. METHODS: Daily primary care data, for 2009-2013, were obtained from Lambeth DataNet (LDN), an anonymised dataset containing coded data from all patients (1.2 million) registered at general practices in Lambeth, an inner-city south London borough. Counts of respiratory consultations and inhaler prescriptions by day and Lower Super Output Area (LSOA) of residence were constructed. We developed models for predicting daily PM2.5, PM10, NO2 and O3 per LSOA. We used spatio-temporal mixed effects zero inflated negative binomial models to investigate the simultaneous short- and long-term effects of exposure to pollutants on the number of events. RESULTS: The mean concentrations of NO2, PM10, PM2.5 and O3 over the study period were 50.7, 21.2, 15.6, and 49.9 µg/m3 respectively, with all pollutants except NO2 having much larger temporal rather than spatial variability. Following short-term exposure increases to PM10, NO2 and PM2.5 the number of consultations and inhaler prescriptions were found to increase, especially for PM10 exposure in children which was associated with increases in daily respiratory consultations of 3.4% and inhaler prescriptions of 0.8%, per PM10 interquartile range (IQR) increase. Associations further increased after adjustment for weekly average exposures, rising to 6.1 and 1.2%, respectively, for weekly average PM10 exposure. In contrast, a short-term increase in O3 exposure was associated with decreased number of respiratory consultations. No association was found between long-term exposures to PM10, PM2.5 and NO2 and number of respiratory consultations. Long-term exposure to NO2 was associated with an increase (8%) in preventer inhaler prescriptions only. CONCLUSIONS: We found increases in the daily number of GP respiratory consultations and inhaler prescriptions following short-term increases in exposure to NO2, PM10 and PM2.5. These associations are more pronounced in children and persist for at least a week. The association with long term exposure to NO2 and preventer inhaler prescriptions indicates likely increased chronic respiratory morbidity.

Global and high-level effects in crowding cannot be predicted by either high-dimensional pooling or target cueing.

In visual crowding, the perception of a target deteriorates in the presence of nearby flankers. Traditionally, target-flanker interactions have been considered as local, mostly deleterious, low-level, and feature specific, occurring when information is pooled along the visual processing hierarchy. Recently, a vast literature of high-level effects in crowding (grouping effects and face-holistic crowding in particular) led to a different understanding of crowding, as a global, complex, and multilevel phenomenon that cannot be captured or explained by simple pooling models. It was recently argued that these high-level effects may still be captured by more sophisticated pooling models, such as the Texture Tiling model (TTM). Unlike simple pooling models, the high-dimensional pooling stage of the TTM preserves rich information about a crowded stimulus and, in principle, this information may be sufficient to drive high-level and global aspects of crowding. In addition, it was proposed that grouping effects in crowding may be explained by post-perceptual target cueing. Here, we extensively tested the predictions of the TTM on the results of six different studies that highlighted high-level effects in crowding. Our results show that the TTM cannot explain any of these high-level effects, and that the behavior of the model is equivalent to a simple pooling model. In addition, we show that grouping effects in crowding cannot be predicted by post-perceptual factors, such as target cueing. Taken together, these results reinforce once more the idea that complex target-flanker interactions determine crowding and that crowding occurs at multiple levels of the visual hierarchy.

Idiosyncratic perception: a link between acuity, perceived position and apparent size.

Perceiving the positions of objects is a prerequisite for most other visual and visuomotor functions, but human perception of object position varies from one individual to the next. The source of these individual differences in perceived position and their perceptual consequences are unknown. Here, we tested whether idiosyncratic biases in the underlying representation of visual space propagate across different levels of visual processing. In Experiment 1, using a position matching task, we found stable, observer-specific compressions and expansions within local regions throughout the visual field. We then measured Vernier acuity (Experiment 2) and perceived size of objects (Experiment 3) across the visual field and found that individualized spatial distortions were closely associated with variations in both visual acuity and apparent object size. Our results reveal idiosyncratic biases in perceived position and size, originating from a heterogeneous spatial resolution that carries across the visual hierarchy.

Demographic variation in fit note receipt and long-term conditions in south London.

OBJECTIVES: Introduced in the UK in 2010, the fit note was designed to address the problem of long-term sickness absence. We explored (1) associations between demographic variables and fit note receipt, 'maybe fit' use and long-term conditions, (2) whether individuals with long-term conditions receive more fit notes and are more likely to have the 'maybe fit' option selected and (3) whether long-term conditions explained associations between demographic variables and fit note receipt. METHODS: Data were extracted from Lambeth DataNet, a database containing electronic medical records of all 45 general practitioner (GP) practices within the borough of Lambeth. Individual-level anonymised data on GP consultations, prescriptions, Quality and Outcomes Framework diagnostic data and demographic information were analysed using survival analysis. RESULTS: In a sample of 326 415 people, 41 502 (12.7%) received a fit note. We found substantial differences in fit note receipt by gender, age, ethnicity and area-level deprivation. Chronic pain (HR 3.7 (95% CI 3.3 to 4.0)) and depression (HR 3.4 (95% CI 3.3 to 3.6)) had the highest rates for first fit note receipt. 'Maybe fit' recommendations were used least often in patients with epilepsy and serious mental illness. The presence of long-term conditions did not explain associations between demographic variables and fit note use. CONCLUSIONS: For the first time, we show the relationships between fit note use and long-term conditions using individual-level primary care data from south London. Further research is required in order to evaluate this relatively new policy and to understand the needs of the population it was designed to support.

Visual crowding in driving.

Visual crowding-the deleterious influence of nearby objects on object recognition-is considered to be a major bottleneck for object recognition in cluttered environments. Although crowding has been studied for decades with static and artificial stimuli, it is still unclear how crowding operates when viewing natural dynamic scenes in real-life situations. For example, driving is a frequent and potentially fatal real-life situation where crowding may play a critical role. In order to investigate the role of crowding in this kind of situation, we presented observers with naturalistic driving videos and recorded their eye movements while they performed a simulated driving task. We found that the saccade localization on pedestrians was impacted by visual clutter, in a manner consistent with the diagnostic criteria of crowding (Bouma's rule of thumb, flanker similarity tuning, and the radial-tangential anisotropy). In order to further confirm that altered saccadic localization is a behavioral consequence of crowding, we also showed that crowding occurs in the recognition of cluttered pedestrians in a more conventional crowding paradigm. We asked participants to discriminate the gender of pedestrians in static video frames and found that the altered saccadic localization correlated with the degree of crowding of the saccade targets. Taken together, our results provide strong evidence that crowding impacts both recognition and goal-directed actions in natural driving situations.

Ensemble Perception.

To understand visual consciousness, we must understand how the brain represents ensembles of objects at many levels of perceptual analysis. Ensemble perception refers to the visual system's ability to extract summary statistical information from groups of similar objects-often in a brief glance. It defines foundational limits on cognition, memory, and behavior. In this review, we provide an operational definition of ensemble perception and demonstrate that ensemble perception spans across multiple levels of visual analysis, incorporating both low-level visual features and high-level social information. Further, we investigate the functional usefulness of ensemble perception and its efficiency, and we consider possible physiological and cognitive mechanisms that underlie an individual's ability to make accurate and rapid assessments of crowds of objects.

Exaggerated groups: amplification in ensemble coding of temporal and spatial features.

The human visual system represents summary statistical information (e.g. average) along many visual dimensions efficiently. While studies have indicated that approximately the square root of the number of items in a set are effectively integrated through this ensemble coding, how those samples are determined is still unknown. Here, we report that salient items are preferentially weighted over the other less salient items, by demonstrating that the perceived means of spatial (i.e. size) and temporal (i.e. flickering temporal frequency (TF)) features of the group of items are positively biased as the number of items in the group increases. This illusory 'amplification effect' was not the product of decision bias but of perceptual bias. Moreover, our visual search experiments with similar stimuli suggested that this amplification effect was due to attraction of visual attention to the salient items (i.e. large or high TF items). These results support the idea that summary statistical information is extracted from sets with an implicit preferential weighting towards salient items. Our study suggests that this saliency-based weighting may reflect a more optimal and efficient integration strategy for the extraction of spatio-temporal statistical information from the environment, and may thus be a basic principle of ensemble coding.

Unifying Visual Space Across the Left and Right Hemifields.

Visual space is perceived as continuous and stable even though visual inputs from the left and right visual fields are initially processed separately within the two cortical hemispheres. In the research reported here, we examined whether the visual system utilizes a dynamic recalibration mechanism to integrate these representations and to maintain alignment across the visual fields. Subjects adapted to randomly oriented moving lines that straddled the vertical meridian; these lines were vertically offset between the left and right hemifields. Subsequent vernier alignment judgments revealed a negative aftereffect: An offset in the same direction as the adaptation was required to correct the perceived misalignment. This aftereffect was specific to adaptation to vertical, but not horizontal, misalignments and also occurred following adaptation to movie clips and patterns without coherent motion. Our results demonstrate that the visual system unifies the left and right halves of visual space by continuously recalibrating the alignment of elements across the visual fields.

Filling-in rivalry: Perceptual alternations in the absence of retinal image conflict.

During perceptual rivalry, an observer's perceptual experience alternates over time despite constant sensory stimulation. Perceptual alternations are thought to be driven by conflicting or ambiguous retinal image features at a particular spatial location and modulated by global context from surrounding locations. However, rivalry can also occur between two illusory stimuli-such as two filled-in stimuli within the retinal blind spot. In this "filling-in rivalry," what observers perceive in the blind spot changes in the absence of local stimulation. It remains unclear if filling-in rivalry shares common mechanisms with other types of rivalry. We measured the dynamics of rivalry between filled-in percepts in the blind spot, finding a high degree of exclusivity (perceptual dominance of one filled-in percept, rather than a perception of transparency), alternation rates that were highly consistent for individual observers, and dynamics that closely resembled other forms of perceptual rivalry. The results suggest that mechanisms common to a wide range of rivalry situations need not rely on conflicting retinal image signals.

Serial dependence promotes object stability during occlusion.

Object identities somehow appear stable and continuous over time despite eye movements, disruptions in visibility, and constantly changing visual input. Recent results have demonstrated that the perception of orientation, numerosity, and facial identity is systematically biased (i.e., pulled) toward visual input from the recent past. The spatial region over which current orientations or face identities are pulled by previous orientations or identities, respectively, is known as the continuity field, which is temporally tuned over the past several seconds (Fischer & Whitney, 2014). This perceptual pull could contribute to the visual stability of objects over short time periods, but does it also address how perceptual stability occurs during visual discontinuities? Here, we tested whether the continuity field helps maintain perceived object identity during occlusion. Specifically, we found that the perception of an oriented Gabor that emerged from behind an occluder was significantly pulled toward the random (and unrelated) orientation of the Gabor that was seen entering the occluder. Importantly, this serial dependence was stronger for predictable, continuously moving trajectories, compared to unpredictable ones or static displacements. This result suggests that our visual system takes advantage of expectations about a stable world, helping to maintain perceived object continuity despite interrupted visibility.

Serial dependence in the perception of attractiveness.

The perception of attractiveness is essential for choices of food, object, and mate preference. Like perception of other visual features, perception of attractiveness is stable despite constant changes of image properties due to factors like occlusion, visual noise, and eye movements. Recent results demonstrate that perception of low-level stimulus features and even more complex attributes like human identity are biased towards recent percepts. This effect is often called serial dependence. Some recent studies have suggested that serial dependence also exists for perceived facial attractiveness, though there is also concern that the reported effects are due to response bias. Here we used an attractiveness-rating task to test the existence of serial dependence in perceived facial attractiveness. Our results demonstrate that perceived face attractiveness was pulled by the attractiveness level of facial images encountered up to 6 s prior. This effect was not due to response bias and did not rely on the previous motor response. This perceptual pull increased as the difference in attractiveness between previous and current stimuli increased. Our results reconcile previously conflicting findings and extend previous work, demonstrating that sequential dependence in perception operates across different levels of visual analysis, even at the highest levels of perceptual interpretation.

Ensemble perception of size in 4-5-year-old children.

Groups of objects are nearly everywhere we look. Adults can perceive and understand the 'gist' of multiple objects at once, engaging ensemble-coding mechanisms that summarize a group's overall appearance. Are these group-perception mechanisms in place early in childhood? Here, we provide the first evidence that 4-5-year-old children use ensemble coding to perceive the average size of a group of objects. Children viewed a pair of trees, with each containing a group of differently sized oranges. We found that, in order to determine which tree had the larger oranges overall, children integrated the sizes of multiple oranges into ensemble representations. This pooling occurred rapidly, and it occurred despite conflicting information from numerosity, continuous extent, density, and contrast. An ideal observer analysis showed that although children's integration mechanisms are sensitive, they are not yet as efficient as adults'. Overall, our results provide a new insight into the way children see and understand the environment, and they illustrate the fundamental nature of ensemble coding in visual perception.

Mixed emotions: Sensitivity to facial variance in a crowd of faces.

The visual system automatically represents summary information from crowds of faces, such as the average expression. This is a useful heuristic insofar as it provides critical information about the state of the world, not simply information about the state of one individual. However, the average alone is not sufficient for making decisions about how to respond to a crowd. The variance or heterogeneity of the crowd--the mixture of emotions--conveys information about the reliability of the average, essential for determining whether the average can be trusted. Despite its importance, the representation of variance within a crowd of faces has yet to be examined. This is addressed here in three experiments. In the first experiment, observers viewed a sample set of faces that varied in emotion, and then adjusted a subsequent set to match the variance of the sample set. To isolate variance as the summary statistic of interest, the average emotion of both sets was random. Results suggested that observers had information regarding crowd variance. The second experiment verified that this was indeed a uniquely high-level phenomenon, as observers were unable to derive the variance of an inverted set of faces as precisely as an upright set of faces. The third experiment replicated and extended the first two experiments using method-of-constant-stimuli. Together, these results show that the visual system is sensitive to emergent information about the emotional heterogeneity, or ambivalence, in crowds of faces.