Examining the role of action-driven attention in ensemble processing.

Ensemble processing allows the visual system to condense visual information into useful summary statistics (e.g., average size), thereby overcoming capacity limitations to visual working memory and attention. To examine the role of attention in ensemble processing, we conducted three experiments using a novel paradigm that merged the action effect (a manipulation of attention) and ensemble processing. Participants were instructed to make a simple action if the feature of a cue word corresponded to a subsequent shape. Immediately after, they were shown an ensemble display of eight ovals of varying sizes and were asked to report either the average size of all ovals or the size of a single oval from the set. In Experiments 1 and 2, participants were cued with a task-relevant feature, and in Experiment 3, participants were cued with a task-irrelevant feature. Overall, the task-relevant cues that elicited an action influenced reports of average size in the ensemble phase more than the cues that were passively viewed, whereas task-irrelevant cues did not bias the reports of average size. The results of this study suggest that attention influences ensemble processing only when it is directed toward a task-relevant feature.

Exploring virtual reality object perception following sensory-motor interactions with different visuo-haptic collider properties.

Interacting with the environment often requires the integration of visual and haptic information. Notably, perceiving external objects depends on how our brain binds sensory inputs into a unitary experience. The feedback provided by objects when we interact (through our movements) with them might then influence our perception. In VR, the interaction with an object can be dissociated by the size of the object itself by means of 'colliders' (interactive spaces surrounding the objects). The present study investigates possible after-effects in size discrimination for virtual objects after exposure to a prolonged interaction characterized by visual and haptic incongruencies. A total of 96 participants participated in this virtual reality study. Participants were distributed into four groups, in which they were required to perform a size discrimination task between two cubes before and after 15 min of a visuomotor task involving the interaction with the same virtual cubes. Each group interacted with a different cube where the visual (normal vs. small collider) and the virtual cube's haptic (vibration vs. no vibration) features were manipulated. The quality of interaction (number of touches and trials performed) was used as a dependent variable to investigate the performance in the visuomotor task. To measure bias in size perception, we compared changes in point of subjective equality (PSE) before and after the task in the four groups. The results showed that a small visual collider decreased manipulation performance, regardless of the presence or not of the haptic signal. However, change in PSE was found only in the group exposed to the small visual collider with haptic feedback, leading to increased perception of the cube size. This after-effect was absent in the only visual incongruency condition, suggesting that haptic information and multisensory integration played a crucial role in inducing perceptual changes. The results are discussed considering the recent findings in visual-haptic integration during multisensory information processing in real and virtual environments.

Encoding contact size using static and dynamic electrotactile finger stimulation: natural decoding vs. trained cues.

Electrotactile stimulation through matrix electrodes is a promising technology to restore high-resolution tactile feedback in extended reality applications. One of the fundamental tactile effects that should be simulated is the change in the size of the contact between the finger and a virtual object. The present study investigated how participants perceive the increase of stimulation area when stimulating the index finger using static or dynamic (moving) stimuli produced by activating 1 to 6 electrode pads. To assess the ability to interpret the stimulation from the natural cues (natural decoding), without any prior training, the participants were instructed to draw the size of the stimulated area and identify the size difference when comparing two consecutive stimulations. To investigate if other "non-natural" cues can improve the size estimation, the participants were asked to enumerate the number of active pads following a training protocol. The results demonstrated that participants could perceive the change in size without prior training (e.g., the estimated area correlated with the stimulated area, p < 0.001; ≥ two-pad difference recognized with > 80% success rate). However, natural decoding was also challenging, as the response area changed gradually and sometimes in complex patterns when increasing the number of active pads (e.g., four extra pads needed for the statistically significant difference). Nevertheless, by training the participants to utilize additional cues the limitations of natural perception could be compensated. After the training, the mismatch in the activated and estimated number of pads was less than one pad regardless of the stimulus size. Finally, introducing the movement of the stimulus substantially improved discrimination (e.g., 100% median success rate to recognize ≥ one-pad difference). The present study, therefore, provides insights into stimulation size perception, and practical guidelines on how to modulate pad activation to change the perceived size in static and dynamic scenarios.

Involvement of the superior colliculi in crossmodal correspondences.

There is an increasing body of evidence suggesting that there are low-level perceptual processes involved in crossmodal correspondences. In this study, we investigate the involvement of the superior colliculi in three basic crossmodal correspondences: elevation/pitch, lightness/pitch, and size/pitch. Using a psychophysical design, we modulate visual input to the superior colliculus to test whether the superior colliculus is required for behavioural crossmodal congruency effects to manifest in an unspeeded multisensory discrimination task. In the elevation/pitch task, superior colliculus involvement is required for a behavioural elevation/pitch congruency effect to manifest in the task. In the lightness/pitch and size/pitch task, we observed a behavioural elevation/pitch congruency effect regardless of superior colliculus involvement. These results suggest that the elevation/pitch correspondence may be processed differently to other low-level crossmodal correspondences. The implications of a distributed model of crossmodal correspondence processing in the brain are discussed.

Temporal features of size constancy for perception and action in a real-world setting: A combined EEG-kinematics study.

A stable representation of object size, in spite of continuous variations in retinal input due to changes in viewing distance, is critical for perceiving and acting in a real 3D world. In fact, our perceptual and visuo-motor systems exhibit size and grip constancies in order to compensate for the natural shrinkage of the retinal image with increased distance. The neural basis of this size-distance scaling remains largely unknown, although multiple lines of evidence suggest that size-constancy operations might take place remarkably early, already at the level of the primary visual cortex. In this study, we examined for the first time the temporal dynamics of size constancy during perception and action by using a combined measurement of event-related potentials (ERPs) and kinematics. Participants were asked to maintain their gaze steadily on a fixation point and perform either a manual estimation or a grasping task towards disks of different sizes placed at different distances. Importantly, the physical size of the target was scaled with distance to yield a constant retinal angle. Meanwhile, we recorded EEG data from 64 scalp electrodes and hand movements with a motion capture system. We focused on the first positive-going visual evoked component peaking at approximately 90 ms after stimulus onset. We found earlier latencies and greater amplitudes in response to bigger than smaller disks of matched retinal size, regardless of the task. In line with the ERP results, manual estimates and peak grip apertures were larger for the bigger targets. We also found task-related differences at later stages of processing from a cluster of central electrodes, whereby the mean amplitude of the P2 component was greater for manual estimation than grasping. Taken together, these findings provide novel evidence that size constancy for real objects at real distances occurs at the earliest cortical stages and that early visual processing does not change as a function of task demands.

Object Image Size Is a Fundamental Coding Dimension in Human Vision: New Insights and Model.

In previous psychophysical work we found that luminance contrast is integrated over retinal area subject to contrast gain control. If different mechanisms perform this operation for a range of superimposed retinal regions of different sizes, this could provide the basis for size-coding. To test this idea we included two novel features in a standard adaptation paradigm to discount more pedestrian accounts of repulsive size-aftereffects. First, we used spatially jittering luminance-contrast adaptors to avoid simple contour displacement aftereffects. Second, we decoupled adaptor and target spatial frequency to avoid the well-known spatial frequency shift aftereffect. Empirical results indicated strong evidence of a bidirectional size adaptation aftereffect. We show that the textbook population model is inappropriate for our results, and develop our existing model of contrast perception to include multiple size mechanisms with divisive surround-suppression from the largest mechanism. For a given stimulus patch, this delivers a blurred step-function of responses across the population, with contrast and size encoded by the height and lateral position of the step. Unlike for textbook population coding schemes, our human results (N = 4 male, N = 4 female) displayed two asymmetries: (i) size aftereffects were greatest for targets smaller than the adaptor, and (ii) on that side of the function, results did not return to baseline, even when targets were 25% of adaptor diameter. Our results and emergent model properties provide evidence for a novel dimension of visual coding (size) and a novel strategy for that coding, consistent with previous results on contrast detection and discrimination for various stimulus sizes.

Ensemble size perception: Its neural signature and the role of global interaction over individual items.

To efficiently process complex visual scenes, the visual system often summarizes statistical information across individual items and represents them as an ensemble. However, due to the lack of techniques to disentangle the representation of the ensemble from that of the individual items constituting the ensemble, whether there exists a specialized neural mechanism for ensemble processing and how ensemble perception is computed in the brain remain unknown. To address these issues, we used a frequency-tagging EEG approach to track brain responses to periodically updated ensemble sizes. Neural responses tracking the ensemble size were detected in parieto-occipital electrodes, revealing a global and specialized neural mechanism of ensemble size perception. We then used the temporal response function to isolate neural responses to the individual sizes and their interactions. Notably, while the individual sizes and their local and global interactions were encoded in the EEG signals, only the global interaction contributed directly to the ensemble size perception. Finally, distributed attention to the global stimulus pattern enhanced the neural signature of the ensemble size, mainly by modulating the neural representation of the global interaction between all individual sizes. These findings advocate a specialized, global neural mechanism of ensemble size perception and suggest that global interaction between individual items contributes to ensemble perception.

Horizontal target size perturbations during grasping movements are described by subsequent size perception and saccade amplitude.

Perception and action are essential in our day-to-day interactions with the environment. Despite the dual-stream theory of action and perception, it is now accepted that action and perception processes interact with each other. However, little is known about the impact of unpredicted changes of target size during grasping actions on perception. We assessed whether size perception and saccade amplitude were affected before and after grasping a target that changed its horizontal size during the action execution under the presence or absence of tactile feedback. We have tested twenty-one participants in 4 blocks of 30 trials. Blocks were divided into two experimental tactile feedback paradigms: tactile and non-tactile. Trials consisted of 3 sequential phases: pre-grasping size perception, grasping, and post-grasping size perception. During pre- and post-phases, participants executed a saccade towards a horizontal bar and performed a manual size estimation of the bar size. During grasping phase, participants were asked to execute a saccade towards the bar and to make a grasping action towards the screen. While grasping, 3 horizontal size perturbation conditions were applied: non-perturbation, shortening, and lengthening. 30% of the trials presented perturbation, meaning a symmetrically shortened or lengthened by 33% of the original size. Participants' hand and eye positions were assessed by a motion capture system and a mobile eye-tracker, respectively. After grasping, in both tactile and non-tactile feedback paradigms, size estimation was significantly reduced in lengthening (p = 0.002) and non-perturbation (p<0.001), whereas shortening did not induce significant adjustments (p = 0.86). After grasping, saccade amplitude became significantly longer in shortening (p<0.001) and significantly shorter in lengthening (p<0.001). Non-perturbation condition did not display adjustments (p = 0.95). Tactile feedback did not generate changes in the collected perceptual responses, but horizontal size perturbations did so, suggesting that all relevant target information used in the movement can be extracted from the post-action target perception.

The effect of textured background and perceived distance on perceived size.

The perceived size of an object depends on its spatial context, in addition to its projected image on the retina and perceived distance. However, how these factors interact with each other to affect perceived object size is still not clear. In this study, we manipulated the binocular disparity of images to assess the effect of perceived distance on perceived object size, as well as background element size to assess the effect of context. The perceived target size under different combinations of perceived distance and context was measured with a two-interval forced-choice paradigm, in which one interval contained a standard disk with a textured background while the other contained a comparison disk on a blank background in each trial. The observers were instructed to indicate which interval contained a larger disk. A staircase procedure was used to measure the point of subjective equality for the perceived target size. Our results showed that the perceived target size increased with the perceived distance while decreased with background element size. In addition, context modulated the relationship between the perceived target size and perceived distance. The data can be explained by a computational model that incorporates several size selective channels whose size sensitivity to a stimulus can be modulated by its disparity. The target response of each channel is subjected to the divisive inhibition signal from the size information in the context. The perceived size is determined by the weighted average of the responses of these size channels. This model can explain more than 91% of variability in the averaged data. Thus, while both perceived distance and context can affect the perceived size of an object, they exert the effect through different mechanisms.

Distinct Contributions of Genes and Environment to Visual Size Illusion and the Underlying Neural Mechanism.

As exemplified by the Ebbinghaus illusion, the perceived size of an object can be significantly biased by its surrounding context. The phenomenon is experienced by humans as well as other species, hence likely evolutionarily adaptive. Here, we examined the heritability of the Ebbinghaus illusion using a combination of the classic twin method and multichannel functional near-infrared spectroscopy. Results show that genes account for over 50% of the variance in the strength of the experienced illusion. Interestingly, activations evoked by the Ebbinghaus stimuli in the early visual cortex are explained by genetic factors whereas those in the posterior temporal cortex are explained by environmental factors. In parallel, the feedforward functional connectivity between the occipital cortex and the temporal cortex is modulated by genetic effects whereas the feedback functional connectivity is entirely shaped by environment, despite both being significantly correlated with the strength of the experienced illusion. These findings demonstrate that genetic and environmental factors work in tandem to shape the context-dependent visual size illusion, and shed new light on the links among genes, environment, brain, and subjective experience.

Neonatal brachial plexus palsy and hand representation in children and young adults.

AIM: To assess the impact of neonatal brachial plexus palsy (NBPP) on higher-order hand representation. METHOD: Eighty-two left-handed children and adolescents with and without right-sided NBPP were recruited. Thirty-one participants with NBPP (mean age [SD] 11y 4mo [4y 4mo]; age range 6y 2mo-21y 0mo; 15 females; C5-6, n=4, C5-7, n=12, C5-T1, n=11, C5-T1 with Horner sign, n=4) were assessed along with 30 controls (mean age 11y 5mo [4y 4mo]; age range 6y 7mo-21y 7mo; 14 females). Participants' estimated hand size and shape on measure of implicit and explicit hand representation was assessed. A linear mixed model (LMM) was used to investigate the effect of condition, sensorimotor impairment, and age. RESULTS: Individuals with NBPP showed a significant difference in implicit hand representation between affected and non-affected hands. LMM confirmed a significant influence of the severity of sensorimotor injury. Only the estimated implicit hand representation was associated with age, with a significant difference between 6- to 8-year-olds and 9- to 10-year-olds. INTERPRETATION: The effect of sensorimotor impairment on central hand representation in individuals with NBPP is specific due to its implicit component and is characterized by finger length underestimation in the affected hand compared to the characteristic underestimation in the unaffected hand. Neither NBPP nor age impacted the explicit hand estimate. This study confirms the importance of sensorimotor contribution to the development of implicit hand representation.

Dogs (canis familiaris) underestimate the quantity of connected items: first demonstration of susceptibility to the connectedness illusion in non-human animals.

In humans, numerical estimation is affected by perceptual biases, such as those originating from the spatial arrangement of elements. Different animal species can also make relative quantity judgements. This includes dogs, who have been proposed as a good model for comparative neuroscience. However, dogs do not show the same perceptual biases observed in humans. Thus, the exact perceptual/cognitive mechanisms underlying quantity estimations in dogs and their degree of similarity with humans are still a matter of debate. Here we explored whether dogs are susceptible to the connectedness illusion, an illusion based on the tendency to underestimate the quantity of interconnected items. Dogs were first trained to choose the larger of two food arrays. Then, they were presented with two arrays containing the same quantity of food, of which one had items interconnected by lines. Dogs significantly selected the array with unconnected items, suggesting that, like in humans, connectedness determines underestimation biases, possibly disrupting the perceptual system's ability to segment the display into discrete objects. The similarity in dogs' and humans' susceptibility to the connectedness, but not to other numerical illusions, suggests that different mechanisms are involved in the estimation of quantity of stimuli with different characteristics.

The contribution of metamemory beliefs to the font size effect on judgments of learning: Is word frequency a moderating factor?

Previous studies found that metamemory beliefs dominate the font size effect on judgments of learning (JOLs). However, few studies have investigated whether beliefs about font size contribute to the font size effect in circumstances of multiple cues. The current study aims to fill this gap. Experiment 1 adopted a 2 (font size: 70 pt vs. 9 pt) * 2 (word frequency (WF): high vs. low) within-subjects design. The results showed that beliefs about font size did not mediate the font size effect on JOLs when multiple cues (font size and WF) were simultaneously provided. Experiment 2 further explored whether WF moderates the contribution of beliefs about font size to the font size effect, in which a 2 (font size: 70 pt vs. 9 pt, as a within-subjects factor) * 2 (WF: high vs. low, as a between-subjects factor) mixed design was used. The results showed that the contribution of beliefs about font size to the font size effect was present in a pure list of low-frequency words, but absent in a pure list of high-frequency words. Lastly, a meta-analysis showed evidence supporting the proposal that the contribution of beliefs about font size to the font size effect on JOLs is moderated by WF. Even though numerous studies suggested beliefs about font size play a dominant role in the font size effect on JOLs, the current study provides new evidence suggesting that such contribution is conditional. Theoretical implications are discussed.

Multimodal learning of pheromone locations.

Memorizing pheromonal locations is critical for many mammalian species as it involves finding mates and avoiding competitors. In rodents, pheromonal information is perceived by the main and accessory olfactory systems. However, the role of somatosensation in context-dependent learning and memorizing of pheromone locations remains unexplored. We addressed this problem by training female mice on a multimodal task to locate pheromones by sampling volatiles emanating from male urine through the orifices of varying dimensions or shapes that are sensed by their vibrissae. In this novel pheromone location assay, female mice' preference toward male urine scent decayed over time when they were permitted to explore pheromones vs neutral stimuli, water. On training them for the associations involving olfactory and whisker systems, it was established that they were able to memorize the location of opposite sex pheromones, when tested 15 days later. This memory was not formed either when the somatosensory inputs through whisker pad were blocked or when the pheromonal cues were replaced with that of same sex. The association between olfactory and somatosensory systems was further confirmed by the enhanced expression of the activity-regulated cytoskeleton protein. Furthermore, the activation of main olfactory bulb circuitry by pheromone volatiles did not cause any modulation in learning and memorizing non-pheromonal volatiles. Our study thus provides the evidence for associations formed between different sensory modalities facilitating the long-term memory formation relevant to social and reproductive behaviors.

Tuned neural responses to haptic numerosity in the putamen.

The ability to perceive the numerosity of items in the environment is critical for behavior of species across the evolutionary tree. Though the focus of studies of numerosity perception lays on the parietal and frontal cortices, the ability to perceive numerosity by a range of species suggests that subcortical nuclei may be implicated in the process. Recently, we have uncovered tuned neural responses to haptic numerosity in the human cortex. Here, we questioned whether subcortical nuclei are also engaged in perception of haptic numerosity. To that end, we utilized a task of haptic numerosity exploration, together with population receptive field model of numerosity selective responses measured at ultra-high field MRI (7T). We found tuned neural responses to haptic numerosity in the bilateral putamen. Similar to the cortex, the population receptive fields tuning width increased with numerosity. The tuned responses to numerosity in the putamen extend its role in cognition and propose that the motor-sensory loops of the putamen and basal ganglia might take an active part in numerosity perception and preparation for future action.

The contribution of object size, manipulability, and stability on neural responses to inanimate objects.

In human occipitotemporal cortex, brain responses to depicted inanimate objects have a large-scale organization by real-world object size. Critically, the size of objects in the world is systematically related to behaviorally-relevant properties: small objects are often grasped and manipulated (e.g., forks), while large objects tend to be less motor-relevant (e.g., tables), though this relationship does not always have to be true (e.g., picture frames and wheelbarrows). To determine how these two dimensions interact, we measured brain activity with functional magnetic resonance imaging while participants viewed a stimulus set of small and large objects with either low or high motor-relevance. The results revealed that the size organization was evident for objects with both low and high motor-relevance; further, a motor-relevance map was also evident across both large and small objects. Targeted contrasts revealed that typical combinations (small motor-relevant vs. large non-motor-relevant) yielded more robust topographies than the atypical covariance contrast (small non-motor-relevant vs. large motor-relevant). In subsequent exploratory analyses, a factor analysis revealed that the construct of motor-relevance was better explained by two underlying factors: one more related to manipulability, and the other to whether an object moves or is stable. The factor related to manipulability better explained responses in lateral small-object preferring regions, while the factor related to object stability (lack of movement) better explained responses in ventromedial large-object preferring regions. Taken together, these results reveal that the structure of neural responses to objects of different sizes further reflect behavior-relevant properties of manipulability and stability, and contribute to a deeper understanding of some of the factors that help the large-scale organization of object representation in high-level visual cortex.

OFF-transient alpha RGCs mediate looming triggered innate defensive response.

Animals respond to visual threats, such as a looming object, with innate defensive behaviors. Here, we report that a specific type of retinal ganglion cell (RGC), the OFF-transient alpha RGC, is critical for the detection of looming objects. We identified Kcnip2 as its molecular marker. The activity of the Kcnip2-expressing RGCs encodes the size of the looming object. Ablation or suppression of these RGCs abolished or severely impaired the escape and freezing behaviors of mice in response to a looming object, while activation of their somas in the retina, or their axon terminals in the superior colliculus, triggered immediate escape behavior. Our results link the activity of a single type of RGC to visually triggered innate defensive behaviors and underscore that ethologically significant visual information is encoded by a labeled line strategy as early as in the retina.

Size constancy affects the perception and parietal neural representation of object size.

Humans and animals rely on accurate object size perception to guide behavior. Object size is judged from visual input, but the relationship between an object's retinal size and its real-world size varies with distance. Humans perceive object sizes to be relatively constant when retinal size changes. Such size constancy compensates for the variable relationship between retinal size and real-world size, using the context of recent retinal sizes of the same object to bias perception towards its likely real-world size. We therefore hypothesized that object size perception may be affected by the range of recently viewed object sizes, attracting perceived object sizes towards recently viewed sizes. We demonstrate two systematic biases: a central tendency attracting perceived size towards the average size across all trials, and a serial dependence attracting perceived size towards the size presented on the previous trial. We recently described topographic object size maps in the human parietal cortex. We therefore hypothesized that neural representations of object size here would be attracted towards recently viewed sizes. We used ultra-high-field (7T) functional MRI and population receptive field modeling to compare object size representations measured with small (0.05-1.4°diameter) and large objects sizes (0.1-2.8°). We found that parietal object size preferences and tuning widths follow this presented range, but change less than presented object sizes. Therefore, perception and neural representation of object size are attracted towards recently viewed sizes. This context-dependent object size representation reveals effects on neural response preferences that may underlie context dependence of object size perception.

Lift observation conveys object weight distribution but partly enhances predictive lift planning.

Observation of object lifting allows updating of internal object representations for object weight, in turn enabling accurate scaling of fingertip forces when lifting the same object. Here, we investigated whether lift observation also enables updating of internal representations for an object's weight distribution. We asked participants to lift an inverted T-shaped manipulandum, of which the weight distribution could be changed, in turns with an actor. Participants were required to minimize object roll (i.e., "lift performance") during lifting and were allowed to place their fingertips at self-chosen locations. The center of mass changed unpredictably every third to sixth trial performed by the actor, and participants were informed that they would always lift the same weight distribution as the actor. Participants observed either erroneous (i.e., object rolling toward its heavy side) or skilled (i.e., minimized object roll) lifts. Lifting performance after observation was compared with lifts without prior observation and with lifts after active lifting, which provided haptic feedback about the weight distribution. Our results show that observing both skilled and erroneous lifts convey an object's weight distribution similar to active lifting, resulting in altered digit positioning strategies. However, minimizing object roll on novel weight distributions was only improved after observing error lifts and not after observing skilled lifts. In sum, these findings suggest that although observing motor errors and skilled motor performance enables updating of digit positioning strategy, only observing error lifts enables changes in predictive motor control when lifting objects with unexpected weight distributions.NEW & NOTEWORTHY Individuals are able to extract an object's size and weight by observing interactions with objects and subsequently integrate this information in their own motor repertoire. Here, we show that this ability extrapolates to weight distributions. Specifically, we highlighted that individuals can perceive an object's weight distribution during lift observation but can only partially embody this information when planning their own actions.

Body size overestimation in anorexia nervosa: Contributions of cognitive, affective, tactile and visual information.

Body image disturbance (BID) in anorexia nervosa (AN) is poorly understood and the individual contribution of perceptual, cognitive, and affective components remains unclear. This study compared females with AN and matched healthy controls (HC) on a perceptual size estimation task. Participants (AN n=19 M[SD] age=16.97[2.24], HC n=19, age=15.77[2.17]) were blindfolded and estimated the size of neutral objects, safe foods, unsafe foods, and parts of their bodies (hips, waist, knees, ankle) over three blocks using: 1) no sensory information (baseline), 2) tactile information, and 3) added visual information. There were no significant differences between AN and HC on neutral and safe or unsafe food objects. Participants with AN were significantly more likely to overestimate their body size across blocks compared to HC. Both groups made fewer errors on unsafe foods and body parts when using tactile or visual information compared to baseline. Exploratory analyses revealed significant correlations between body size overestimation and drive for thinness and body dissatisfaction in the AN group, with body dissatisfaction being the most robust. Results suggest that both deficits in tactile and visual perception and affective factors play a role in BID for young women with AN.