The time course of Temporal Binding in social and nonsocial interactions.

Temporal Binding (TB) is the subjective compression of action-effect intervals. While the effects of nonsocial actions are highly predictable, it is not the case when interacting with conspecifics, who often act under their own volition, at a time of their choosing. Given the relative differences in action-effect predictability in non-social and social interactions, it is plausible that TB and its properties differ across these situations. To examine this, in two experiments, we compared the time course of TB in social and nonsocial interactions, systematically varying action-effect intervals (200-2,100 ms). Participants were told they were (a) interacting with another person via a live webcam, who was in fact a confederate (social condition), (b) interacting with pre-recorded videos (nonsocial condition), or (c) observing two pre-recorded videos (control condition; Experiment 2). Results across experiments showed greater TB for social compared to nonsocial conditions, and the difference was proportional to the action-effect intervals. Further, in Experiment 1, TB was consistently observed throughout the experiment for social interactions, whereas nonsocial TB decreased from the first to the second half of the experiment. In Experiment 2, the nonsocial condition did not differ from control, whereas the social condition did, exhibiting enhanced binding. We argue these results suggest that the sociality of an interaction modulates the 'internal clock' of time perception.

Neural and behavioral signatures of the multidimensionality of manipulable object processing.

Understanding how we recognize objects requires unravelling the variables that govern the way we think about objects and the neural organization of object representations. A tenable hypothesis is that the organization of object knowledge follows key object-related dimensions. Here, we explored, behaviorally and neurally, the multidimensionality of object processing. We focused on within-domain object information as a proxy for the decisions we typically engage in our daily lives - e.g., identifying a hammer in the context of other tools. We extracted object-related dimensions from subjective human judgments on a set of manipulable objects. We show that the extracted dimensions are cognitively interpretable and relevant - i.e., participants are able to consistently label them, and these dimensions can guide object categorization; and are important for the neural organization of knowledge - i.e., they predict neural signals elicited by manipulable objects. This shows that multidimensionality is a hallmark of the organization of manipulable object knowledge.

Fundamental units of numerosity estimation.

Humans can approximately enumerate a large number of objects at a single glance. While several mechanisms have been proposed to account for this ability, the fundamental units over which they operate remain unclear. Previous studies have argued that estimation mechanisms act only on topologically distinct units or on units formed by spatial grouping cues such as proximity and connectivity, but not on units grouped by similarity. Over four experiments, we tested this claim by systematically assessing and demonstrating that similarity grouping leads to underestimation, just as spatial grouping does. Ungrouped objects with the same low-level properties as grouped objects did not cause underestimation. Further, the underestimation caused by spatial and similarity grouping was additive, suggesting that these grouping processes operate independently. These findings argue against the proposal that estimation mechanisms operate solely on topological units. Instead, we conclude that estimation processes act on representations constructed after Gestalt grouping principles, whether similarity based or spatial, have organised incoming visual input.

Visual field asymmetries in numerosity processing.

A small number of objects can be rapidly and accurately enumerated, whereas a larger number of objects can only be approximately enumerated. These subitizing and estimation abilities, respectively, are both spatial processes relying on extracting information across spatial locations. Nevertheless, whether and how these processes vary across visual field locations remains unknown. Here, we examined if enumeration displays asymmetries around the visual field. Experiment 1 tested small number (1-6) enumeration at cardinal and non-cardinal peripheral locations while manipulating the spacing among the objects. Experiment 2 examined enumeration at cardinal locations in more detail while minimising crowding. Both experiments demonstrated a Horizontal-Vertical Asymmetry (HVA) where performance was better along the horizontal axis relative to the vertical. Experiment 1 found that this effect was modulated by spacing with stronger asymmetry at closer spacing. Experiment 2 revealed further asymmetries: a Vertical Meridian Asymmetry (VMA) with better enumeration on the lower vertical meridian than on the upper and a Horizontal Meridian Asymmetry (HMA) with better enumeration along the left horizontal meridian than along the right. All three asymmetries were evident for both subitizing and estimation. HVA and VMA have been observed in a range of visual tasks, indicating that they might be inherited from early visual constraints. However, HMA is observed primarily in mid-level tasks, often involving attention. These results suggest that while enumeration processes can be argued to inherit low-level visual constraints, the findings are, parsimoniously, consistent with visual attention playing a role in both subitizing and estimation.

Masking, crowding, and grouping: Connecting low and mid-level vision.

An important task for vision science is to build a unitary framework of low- and mid-level vision. As a step on this way, our study examined differences and commonalities between masking, crowding and grouping-three processes that occur through spatial interactions between neighbouring elements. We measured contrast thresholds as functions of inter-element spacing and eccentricity for Gabor detection, discrimination and contour integration, using a common stimulus grid consisting of nine Gabor elements. From these thresholds, we derived a) the baseline contrast necessary to perform each task and b) the spatial extent over which task performance was stable. This spatial window can be taken as an indicator of field size, where elements that fall within a putative field are readily combined. We found that contrast thresholds were universally modulated by inter-element distance, with a shallower and inverted effect for grouping compared with masking and crowding. Baseline contrasts for detecting stimuli and discriminating their properties were positively linked across the tested retinal locations (parafovea and near periphery), whereas those for integrating elements and discriminating their properties were negatively linked. Meanwhile, masking and crowding spatial windows remained uncorrelated across eccentricity, although they were correlated across participants. This suggests that the computation performed by each type of visual field operates over different distances that co-varies across observers, but not across retinal locations. Contrast-processing units may thus lie at the core of the shared idiosyncrasies across tasks reported in many previous studies, despite the fundamental differences in the extent of their spatial windows.

Emergence of crowding: The role of contrast and orientation salience.

Crowding causes difficulties in judging attributes of an object surrounded by other objects. We investigated crowding for stimuli that isolated either S-cone or luminance mechanisms or combined them. By targeting different retinogeniculate mechanisms with contrast-matched stimuli, we aim to determine the earliest site at which crowding emerges. Discrimination was measured in an orientation judgment task where Gabor targets were presented parafoveally among flankers. In the first experiment, we assessed flanked and unflanked orientation discrimination thresholds for pure S-cone and achromatic stimuli and their combinations. In the second experiment, to capture individual differences, we measured unflanked detection and orientation sensitivity, along with performance under flanker interference for stimuli containing luminance only or combined with S-cone contrast. We confirmed that orientation sensitivity was lower for unflanked S-cone stimuli. When flanked, the pattern of results for S-cone stimuli was the same as for achromatic stimuli with comparable (i.e. low) contrast levels. We also found that flanker interference exhibited a genuine signature of crowding only when orientation discrimination threshold was reliably surpassed. Crowding, therefore, emerges at a stage that operates on signals representing task-relevant featural (here, orientation) information. Because luminance and S-cone mechanisms have very different spatial tuning properties, it is most parsimonious to conclude that crowding takes place at a neural processing stage after they have been combined.

Characterizing the in-out asymmetry in visual crowding.

An object's processing is impaired by the presence of nearby clutter. Several distinct mechanisms, such as masking and visual crowding, are thought to contribute to such flanker-induced interference. It is therefore important to determine which mechanism is operational in any given situation. Previous studies have proposed that the in-out asymmetry (IOA), where a peripheral flanker interferes with the target more than a foveal flanker, is diagnostic of crowding. However, several studies have documented inconsistencies in the occurrence of this asymmetry, particularly at locations beyond the horizontal meridian, casting doubt on its ability to delineate crowding. In this study, to determine if IOA is diagnostic of crowding, we extensively charted its properties. We asked a relatively large set of participants (n = 38) to identify a briefly presented peripheral letter flanked by a single inward or outward letter at one of four locations. We also manipulated target location uncertainty and attentional allocation by blocking, randomizing or pre-cueing the target location. Using multilevel Bayesian regression analysis, we found robust IOA at all locations, although its strength was modulated by target location, location uncertainty, and attentional allocation. Our findings suggest that IOA can be an excellent marker of crowding, to the extent that it is not observed in other flanker-interference mechanisms, such as masking.

Subitizing object parts reveals a second stage of individuation.

Humans can efficiently individuate a small number of objects. This subitizing ability is thought to be a consequence of limited attentional resources. However, how and what is selected during the individuation process remain outstanding questions. We investigated these in four experiments by examining if parts of objects are enumerated as efficiently as distinct objects in the presence and absence of distractor objects. We found that distractor presence reduced subitizing efficiency. Crucially, parts connected to multiple objects were enumerated less efficiently than independent objects or parts connected to a single object. These results argue against direct individuation of parts and show that objecthood plays a fundamental role in individuation. Objects are selected first and their components are selected in subsequent steps. This reveals that individuation operates sequentially over multiple levels.

Social Agency as a continuum.

Sense of Agency, the phenomenology associated with causing one's own actions and corresponding effects, is a cornerstone of human experience. Social Agency can be defined as the Sense of Agency experienced in any situation in which the effects of our actions are related to a conspecific. This can be implemented as the other's reactions being caused by our action, joint action modulating our Sense of Agency, or the other's mere social presence influencing our Sense of Agency. It is currently an open question how such Social Agency can be conceptualized and how it relates to its nonsocial variant. This is because, compared with nonsocial Sense of Agency, the concept of Social Agency has remained oversimplified and underresearched, with disparate empirical paradigms yielding divergent results. Reviewing the empirical evidence and the commonalities and differences between different instantiations of Social Agency, we propose that Social Agency can be conceptualized as a continuum, in which the degree of cooperation is the key dimension that determines our Sense of Agency, and how it relates to nonsocial Sense of Agency. Taking this perspective, we review how the different factors that typically influence Sense of Agency affect Social Agency, and in the process highlight outstanding empirical questions within the field. Finally, concepts from wider research areas are discussed in relation to the ecological validity of Social Agency paradigms, and we provide recommendations for future methodology.

A simple rule to describe interactions between visual categories.

Humans can rapidly categorise visual objects when presented in isolation. However, in everyday life we encounter multiple objects at the same time. Far less is known about how simultaneously active object representations interact. We examined such interactions by asking participants to categorise a target object at the basic (Experiment 1) or the superordinate (Experiment 2) level while the representation of another object was still active. We found that the "prime" object strongly modulated the response to the target implying that the prime's category was rapidly and automatically accessed, influencing subsequent categorical processing. Using drift diffusion modelling, we show that a prime, whose category is different from that of the target, interferes with target processing primarily during the evidence accumulation stage. This suggests that the state of category-processing neurons is altered by an active representation and this modifies the processing of other categories. Interestingly, the strength of interference increases with the similarity between the distractor and the target category. Considering these results and previous studies, we propose a general principle that category interactions are determined by the distance from a distractor's representation to the target's task-relevant categorical boundary. We argue that this principle arises from the specific architectural organisation of categories in the brain.

Too little, too late, and in the wrong place: Alpha band activity does not reflect an active mechanism of selective attention.

Selective attention focuses visual processing on relevant stimuli in order to allow for adaptive behaviour despite an abundance of distracting information. It has been proposed that increases in alpha band (8-12 â€‹Hz) amplitude reflect an active mechanism for distractor suppression. If this were the case, increases in alpha band amplitude should be succeeded by a decrease in distractor processing. Surprisingly, this connection has not been tested directly; specifically, studies that have investigated changes in alpha band after attention-directing cues have not directly assessed the neuronal processing of distractors. We concurrently recorded alpha activity and steady-state visual evoked potentials (SSVEPs) to assess the processing of target and distractor stimuli. In two experiments, participants covertly shifted attention to one of two letter streams (left or right) to detect infrequent target letters 'X' while ignoring the other stream. In line with previous findings, alpha band amplitudes contralateral to the unattended location increased compared to a pre-cue baseline. However, there was no suppression of SSVEP amplitudes elicited by unattended stimuli, while there was a pronounced enhancement of SSVEPs elicited by attended stimuli. Furthermore, and crucially, changes in alpha band amplitude during attention shifts did not precede those in SSVEPs and hit rates in both experiments, indicating that changes in alpha band amplitudes are likely to be a consequence of attention shifts rather than the other way around. We conclude that these findings contradict the notion that alpha band activity reflects mechanisms that have a causal role in the allocation of selective attention.

Crowding in humans is unlike that in convolutional neural networks.

Object recognition is a primary function of the human visual system. It has recently been claimed that the highly successful ability to recognise objects in a set of emergent computer vision systems-Deep Convolutional Neural Networks (DCNNs)-can form a useful guide to recognition in humans. To test this assertion, we systematically evaluated visual crowding, a dramatic breakdown of recognition in clutter, in DCNNs and compared their performance to extant research in humans. We examined crowding in three architectures of DCNNs with the same methodology as that used among humans. We manipulated multiple stimulus factors including inter-letter spacing, letter colour, size, and flanker location to assess the extent and shape of crowding in DCNNs. We found that crowding followed a predictable pattern across architectures that was different from that in humans. Some characteristic hallmarks of human crowding, such as invariance to size, the effect of target-flanker similarity, and confusions between target and flanker identities, were completely missing, minimised or even reversed. These data show that DCNNs, while proficient in object recognition, likely achieve this competence through a set of mechanisms that are distinct from those in humans. They are not necessarily equivalent models of human or primate object recognition and caution must be exercised when inferring mechanisms derived from their operation.

Clustering leads to underestimation of numerosity, but crowding is not the cause.

Humans have the remarkable ability to rapidly estimate the number of objects in a visual scene without relying on counting, something referred to as the number sense. It has been well documented that the more clustered the elements are, the lower their perceived numerosity is. A recent account of this observation is the crowdinghypothesis, which posits that the perceived underestimation is driven by visual crowding: the inability to recognise objects in clutter. Crowding can impair individuation of the elements, which would explain the underestimation. Here, we tested the crowding hypothesis by assessing numerosity estimation and crowding for the same stimulus configurations in the same participants. Experiment 1 compared the two tasks when numerosity can be considered to be estimated directly by the visual system (reference patch density = 0.12 items/deg2), while Experiment 2 used high density stimuli (density = 0.88 items/deg2), where numerosity may be estimated indirectly. In both cases, we found that spacing and similarity between elements affected estimation and crowding tasks in markedly different ways. These results are incompatible with a crowding account of numerosity underestimation and point to separate mechanisms for object identification and number estimation, although grouping may play a moderating role in both cases.

The eye that binds: Feature integration is not disrupted by saccadic eye movements.

Feature integration theory proposes that visual features, such as shape and color, can only be combined into a unified object when spatial attention is directed to their location in retinotopic maps. Eye movements cause dramatic changes on our retinae, and are associated with obligatory shifts in spatial attention. In two experiments, we measured the prevalence of conjunction errors (that is, reporting an object as having an attribute that belonged to another object), for brief stimulus presentation before, during, and after a saccade. Planning and executing a saccade did not itself disrupt feature integration. Motion did disrupt feature integration, leading to an increase in conjunction errors. However, retinal motion of an equal extent but caused by saccadic eye movements is spared this disruption, and showed similar rates of conjunction errors as a condition with static stimuli presented to a static eye. The results suggest that extra-retinal signals are able to compensate for the motion caused by saccadic eye movements, thereby preserving the integrity of objects across saccades and preventing their features from mixing or mis-binding.

Response selection modulates crowding: a cautionary tale for invoking top-down explanations.

Object recognition in the periphery is limited by clutter. This phenomenon of visual crowding is ameliorated when the objects are dissimilar. This effect of inter-object similarity has been extensively studied for low-level features and is thought to reflect bottom-up processes. Recently, crowding was also found to be reduced when objects belonged to explicitly distinct groups; that is, crowding was weak when they had low group membership similarity. It has been claimed that top-down knowledge is necessary to explain this effect of group membership, implying that the effect of similarity on crowding cannot be a purely bottom-up process. We tested the claim that the effect of group membership relies on knowledge in two experiments and found that neither explicit knowledge about differences in group membership nor the possibility of acquiring knowledge about target identities is necessary to produce the effects. These results suggest that top-down processes need not be invoked to explain the effect of group membership. Instead, we suggest that differences in flanker reportability that emerge from the differences in group membership are the source of the effect. That is, when targets and flankers are sampled from distinct groups, flankers cannot be inadvertently reported, leading to fewer errors and hence weaker crowding. Further, we argue that this effect arises at the stage of response selection. This conclusion is well supported by an analytical model based on these principles. We conclude that previously observed effects in crowding attributed to top-down or higher level processes might instead be due to post-perceptual response selection strategies.

Two's company, three's a crowd: Individuation is necessary for object recognition.

Object recognition is essential for navigating the real world. Despite decades of research on this topic, the processing steps necessary for recognition remain unclear. In this study, we examined the necessity and role of individuation, the ability to select a small number of spatially distinct objects irrespective of their identity, in the recognition process. More specifically, we tested if the ability to rapidly individuate and enumerate a small number of objects (subitizing) can be impaired by crowding. Crowding is flanker-induced interference that specifically impedes the recognition process. We found that subitizing is impaired when objects are close to each other (Experiment 1), and if the target objects are surrounded by irrelevant but perceptually similar flankers (Experiments 2-4). This impairment cannot be attributed to confusion between targets and flankers, wherein flankers are inadvertently included in or targets are excluded from enumeration (Experiments 3-4). Importantly, the flanker induced interference was comparable in both subitizing and crowding tasks (Experiment 4), suggesting that individuation and identification share a common processing pathway. We conclude that individuation is an essential stage in the object recognition pipeline and argue for a cohesive proposal that both crowding and subitizing are due to limitations of selective attention.

Critical resolution: A superior measure of crowding.

Visual object recognition is essential for adaptive interactions with the environment. It is fundamentally limited by crowding, a breakdown of object recognition in clutter. The spatial extent over which crowding occurs is proportional to the eccentricity of the target object, but nevertheless varies substantially depending on various stimulus factors (e.g. viewing time, contrast). However, a lack of studies jointly manipulating such factors precludes predictions of crowding in more heterogeneous scenes, such as the majority of real life situations. To establish how such co-occurring variations affect crowding, we manipulated combinations of 1) flanker contrast and backward masking, 2) flanker contrast and presentation duration, and 3) flanker preview and pop-out while measuring participants' ability to correctly report the orientation of a target stimulus. In all three experiments, combining two manipulations consistently modulated the spatial extent of crowding in a way that could not be predicted from an additive combination. However, a simple transformation of the measurement scale completely abolished these interactions and all effects became additive. Precise quantitative predictions of the magnitude of crowding when combining multiple manipulations are thus possible when it is expressed in terms of what we label the 'critical resolution'. Critical resolution is proportional to the inverse of the smallest flanker free area surrounding the target object necessary for its unimpaired identification. It offers a more parsimonious description of crowding than the traditionally used critical spacing and may thus constitute a measure of fundamental importance for understanding object recognition.

Combining S-cone and luminance signals adversely affects discrimination of objects within backgrounds.

The visual system processes objects embedded in complex scenes that vary in both luminance and colour. In such scenes, colour contributes to the segmentation of objects from backgrounds, but does it also affect perceptual organisation of object contours which are already defined by luminance signals, or are these processes unaffected by colour's presence? We investigated if luminance and chromatic signals comparably sustain processing of objects embedded in backgrounds, by varying contrast along the luminance dimension and along the two cone-opponent colour directions. In the first experiment thresholds for object/non-object discrimination of Gaborised shapes were obtained in the presence and absence of background clutter. Contrast of the component Gabors was modulated along single colour/luminance dimensions or co-modulated along multiple dimensions simultaneously. Background clutter elevated discrimination thresholds only for combined S-(L + M) and L + M signals. The second experiment replicated and extended this finding by demonstrating that the effect was dependent on the presence of relatively high S-(L + M) contrast. These results indicate that S-(L + M) signals impair spatial vision when combined with luminance. Since S-(L + M) signals are characterised by relatively large receptive fields, this is likely to be due to an increase in the size of the integration field over which contour-defining information is summed.

The Temporal Dynamics of Visual Processing in Multiple Sclerosis.

Although the integrity of the visual system is often affected in multiple sclerosis (MS), the potential relationship between the temporal dynamics of visual processing and performance on neuropsychological tests assessing processing speed (PS) remains relatively unexplored. Here, we test if a PS deficit is related to abnormalities within the visual system, rather than impaired higher-level cognitive function. Two groups of participants with MS (1 group with PS deficits and another without) and a healthy control group, matched for age and education, were included. To explore the temporal dynamics of visual processing, we used 2 psychophysical paradigms: attention enhancement/prioritization and rapid serial visual presentation. Visual PS deficits were associated with a decreased capability to detect visual stimuli and a higher limitation in visual temporal-processing capacity. These results suggest that a latent sensorial temporal limitation of the visual system is significantly associated to PS deficits in MS.

The Bouma law of crowding, revised: critical spacing is equal across parts, not objects.

Crowding is the inability to identify an object among flankers in the periphery. It is due to inappropriate incorporation of features from flanking objects in perception of the target. Crowding is characterized by measuring critical spacing, the minimum distance needed between a target and flankers to allow recognition. The existing Bouma law states that, at a given point and direction in the visual field, critical spacing, measured from the center of a target object to the center of a similar flanking object, is the same for all objects (Pelli & Tillman, 2008). Because flipping an object about its center preserves its center-to-center spacing to other objects, according to the Bouma law, crowding should be unaffected. However, because crowding is a result of feature combination, the location of features within an object might matter. In a series of experiments, we find that critical spacing is affected by the location of features within the flanker. For some flankers, a flip greatly reduces crowding even though it maintains target-flanker spacing and similarity. Our results suggest that the existing Bouma law applies to simple one-part objects, such as a single roman letter or a Gabor patch. Many objects consist of multiple parts; for example, a word is composed of multiple letters that crowd each other. To cope with such complex objects, we revise the Bouma law to say that critical spacing is equal across parts, rather than objects. This accounts for old and new findings.