High-Pitched Sound is Open and Low-Pitched Sound is Closed: Representing the Spatial Meaning of Pitch Height.

Research shows that high- and low-pitch sounds can be associated with various meanings. For example, high-pitch sounds are associated with small concepts, whereas low-pitch sounds are associated with large concepts. This study presents three experiments revealing that high-pitch sounds are also associated with open concepts and opening hand actions, while low-pitch sounds are associated with closed concepts and closing hand actions. In Experiment 1, this sound-meaning correspondence effect was shown using the two-alternative forced-choice task, while Experiments 2 and 3 used reaction time tasks to show this interaction. In Experiment 2, high-pitch vocalizations were found to facilitate opening hand gestures, and low-pitch vocalizations were found to facilitate closing hand gestures, when performed simultaneously. In Experiment 3, high-pitched vocalizations were produced particularly rapidly when the visual target stimulus presented an open object, and low-pitched vocalizations were produced particularly rapidly when the target presented a closed object. These findings are discussed concerning the meaning of intonational cues. They are suggested to be based on cross-modally representing conceptual spatial knowledge in sensory, motor, and affective systems. Additionally, this pitch-opening effect might share cognitive processes with other pitch-meaning effects.

Peripersonal space: why so last-second?

A vast range of neurophysiological, neuropsychological and behavioural results in monkeys and humans have shown that the immediate surroundings of the body, also known as peripersonal space (PPS), are processed in a unique way. Three roles have been ascribed to PPS mechanisms: to react to threats, to avoid obstacles and to act on objects. However, in many circumstances, one does not wait for objects or agents to enter PPS to plan these behaviours. Typically, one has more chances to survive if one starts running away from the lion when one sees it in the distance than if it is a few steps away. PPS makes sense in shortsighted creatures but we are not such creatures. The crucial question is thus twofold: (i) why are these adaptive processes triggered only at the last second or even milliseconds? And (ii) what is their exact contribution, especially for defensive and navigational behaviours? Here, we propose that PPS mechanisms correspond to a plan B, useful in unpredictable situations or when other anticipatory mechanisms have failed. Furthermore, we argue that there are energetic, cognitive and behavioural costs to PPS mechanisms, which explain why this plan B is triggered only at the last second. This article is part of the theme issue 'Minds in movement: embodied cognition in the age of artificial intelligence'.

Inter-individual variability (but intra-individual stability) of overt versus covert rehearsal strategies in a digital Corsi task.

The Corsi (block-tapping) paradigm is a classic and well-established visuospatial working memory task in humans involving internal computations (memorizing of item sequences, organizing and updating the memorandum, and recall processes), as well as both overt and covert shifts of attention to facilitate rehearsal, serving to maintain the Corsi sequences during the retention phase. Here, we introduce a novel digital version of a Corsi task in which i) the difficulty of the memorandum (using sequence lengths ranging from 3 to 8) was controlled, ii) the execution of overt and/or covert attention as well as the visuospatial working memory load during the retention phase was manipulated, and iii) shifts of attention were quantified in all experimental phases. With this, we present behavioral data that demonstrate, characterize, and classify the individual effects of overt and covert strategies used as a means of encoding and rehearsal. In a full within-subject design, we tested 28 participants who had to solve three different Corsi conditions. While in condition A neither of the two strategies were restricted, in condition B the overt and in condition C the overt as well as the covert strategies were suppressed. Analyzing Corsi span, (eye) exploration index, and pupil size (change), data clearly show a continuum between overt and covert strategies over all participants (indicating inter-individual variability). Further, all participants showed stable strategy choice (indicating intra-individual stability), meaning that the preferred strategy was maintained in all three conditions, phases, and sequence lengths of the experiment.

The proactive and reactive mechanisms of learned spatial suppression.

Selection history refers to the notion that previous allocations of attention or suppression have the potential to elicit lingering and enduring selection biases that are isolated from goal-driven or stimulus-driven attention. However, in the singleton detection mode task, manipulating the selection history of distractors cannot give rise to pure proactive inhibition. Therefore, we employed a combination of a working memory task and a feature search mode task, simultaneously recording cortical activity using EEG, to investigate the mechanisms of suppression guided by selection history. The results from event-related potential and reaction times showed an enhanced inhibitory performance when the distractor was presented at the high-probability location, along with instances where the target appeared at the high-probability location of distractors. These findings demonstrate that a generalized proactive inhibition bias is learned and processed independent of cognitive resources, which is supported by selection history. In contrast, reactive rejection toward the low-probability location was evident through the Pd component under varying cognitive resource conditions. Taken together, our findings indicated that participants learned proactive inhibition when the distractor was at the high-probability location, whereas reactive rejection was involved at low-probability location.

Egocentric neural representation of geometric vertex in the retrosplenial cortex.

Egocentric neural representations of environmental features, such as edges and vertices, are important for constructing a geometrically detailed egocentric cognitive map for goal-directed navigation and episodic memory. While egocentric neural representations of edges like egocentric boundary/border cells exist, those that selectively represent vertices egocentrically are yet unknown. Here we report that granular retrosplenial cortex (RSC) neurons in male mice generate spatial receptive fields exclusively near the vertices of environmental geometries during free exploration, termed vertex cells. Their spatial receptive fields occurred at a specific orientation and distance relative to the heading direction of mice, indicating egocentric vector coding of vertex. Removing physical boundaries defining the environmental geometry abolished the egocentric vector coding of vertex, and goal-directed navigation strengthened the egocentric vector coding at the goal-located vertex. Our findings suggest that egocentric vector coding of vertex by granular RSC neurons helps construct an egocentric cognitive map that guides goal-directed navigation.

The right posterior parietal cortex mediates spatial reorienting of attentional choice bias.

Attention facilitates behavior by enhancing perceptual sensitivity (sensory processing) and choice bias (decisional weighting) for attended information. Whether distinct neural substrates mediate these distinct components of attention remains unknown. We investigate the causal role of key nodes of the right posterior parietal cortex (rPPC) in the forebrain attention network in sensitivity versus bias control. Two groups of participants performed a cued attention task while we applied either inhibitory, repetitive transcranial magnetic stimulation (n = 28) or 40 Hz transcranial alternating current stimulation (n = 26) to the dorsal rPPC. We show that rPPC stimulation - with either modality - impairs task performance by selectively altering attentional modulation of bias but not sensitivity. Specifically, participants' bias toward the uncued, but not the cued, location reduced significantly following rPPC stimulation - an effect that was consistent across both neurostimulation cohorts. In sum, the dorsal rPPC causally mediates the reorienting of choice bias, one particular component of visual spatial attention.

Electrophysiological correlation of auditory selective spatial attention in the "cocktail party" situation.

The auditory system can selectively attend to the target source in complex environments, the phenomenon known as the "cocktail party" effect. However, the spatiotemporal dynamics of electrophysiological activity associated with auditory selective spatial attention (ASSA) remain largely unexplored. In this study, single-source and multiple-source paradigms were designed to simulate different auditory environments, and microstate analysis was introduced to reveal the electrophysiological correlates of ASSA. Furthermore, cortical source analysis was employed to reveal the neural activity regions of these microstates. The results showed that five microstates could explain the spatiotemporal dynamics of ASSA, ranging from MS1 to MS5. Notably, MS2 and MS3 showed significantly lower partial properties in multiple-source situations than in single-source situations, whereas MS4 had shorter durations and MS5 longer durations in multiple-source situations than in single-source situations. MS1 had insignificant differences between the two situations. Cortical source analysis showed that the activation regions of these microstates initially transferred from the right temporal cortex to the temporal-parietal cortex, and subsequently to the dorsofrontal cortex. Moreover, the neural activity of the single-source situations was greater than that of the multiple-source situations in MS2 and MS3, correlating with the N1 and P2 components, with the greatest differences observed in the superior temporal gyrus and inferior parietal lobule. These findings suggest that these specific microstates and their associated activation regions may serve as promising substrates for decoding ASSA in complex environments.

Neural correlates of perisaccadic visual mislocalization in extrastriate cortex.

When interacting with the visual world using saccadic eye movements (saccades), the perceived location of visual stimuli becomes biased, a phenomenon called perisaccadic mislocalization. However, the neural mechanism underlying this altered visuospatial perception and its potential link to other perisaccadic perceptual phenomena have not been established. Using the electrophysiological recording of extrastriate areas in four male macaque monkeys, combined with a computational model, we were able to quantify spatial bias around the saccade target (ST) based on the perisaccadic dynamics of extrastriate spatiotemporal sensitivity captured by a statistical model. This approach could predict the perisaccadic spatial bias around the ST, consistent with behavioral data, and revealed the precise neuronal response components underlying representational bias. These findings also establish the crucial role of increased sensitivity near the ST for neurons with receptive fields far from the ST in driving the ST spatial bias. Moreover, we showed that, by allocating more resources for visual target representation, visual areas enhance their representation of the ST location, even at the expense of transient distortions in spatial representation. This potential neural basis for perisaccadic ST representation also supports a general role for extrastriate neurons in creating the perception of stimulus location.

Distal but not local auditory information supports spatial representations by place cells.

Sound is an important navigational cue for mammals. During spatial navigation, hippocampal place cells encode spatial representations of the environment based on visual information, but to what extent audiospatial information can enable reliable place cell mapping is largely unknown. We assessed this by recording from CA1 place cells in the dark, under circumstances where reliable visual, tactile, or olfactory information was unavailable. Male rats were exposed to auditory cues of different frequencies that were delivered from local or distal spatial locations. We observed that distal, but not local cue presentation, enables and supports stable place fields, regardless of the sound frequency used. Our data suggest that a context dependency exists regarding the relevance of auditory information for place field mapping: whereas locally available auditory cues do not serve as a salient spatial basis for the anchoring of place fields, auditory cue localization supports spatial representations by place cells when available in the form of distal information. Furthermore, our results demonstrate that CA1 neurons can effectively use auditory stimuli to generate place fields, and that hippocampal pyramidal neurons are not solely dependent on visual cues for the generation of place field representations based on allocentric reference frames.

Influence of scene aspect ratio and depth cues on verticality perception bias.

Perceiving verticality is crucial for accurate spatial orientation. Previous research has revealed that tilted scenes can bias verticality perception. Verticality perception bias can be represented as the sum of multiple periodic functions that play a role in the perception of visual orientation, where the specific factors affecting each periodicity remain uncertain. This study investigated the influence of the width and depth of an indoor scene on each periodic component of the bias. The participants were presented with an indoor scene showing a rectangular checkerboard room (Experiment 1), a rectangular aperture on the wall (Experiment 2), or a rectangular dotted room (Experiment 3), with various aspect ratios. The stimuli were presented with roll orientations ranging from 90° clockwise to 90° counterclockwise. The participants were asked to report their subjective visual vertical (SVV) perceptions. The contributions of 45°, 90°, and 180° periodicities to the SVV error were assessed by the weighted vector sum model. In Experiment 1, the periodic components of the SVV error increased with the aspect ratio. In Experiments 2 and 3, only the 90° component increased with the aspect ratio. These findings suggest that extended transverse surfaces may modulate the periodic components of verticality perception.

Space is a latent sequence: A theory of the hippocampus.

Fascinating phenomena such as landmark vector cells and splitter cells are frequently discovered in the hippocampus. Without a unifying principle, each experiment seemingly uncovers new anomalies or coding types. Here, we provide a unifying principle that the mental representation of space is an emergent property of latent higher-order sequence learning. Treating space as a sequence resolves numerous phenomena and suggests that the place field mapping methodology that interprets sequential neuronal responses in Euclidean terms might itself be a source of anomalies. Our model, clone-structured causal graph (CSCG), employs higher-order graph scaffolding to learn latent representations by mapping aliased egocentric sensory inputs to unique contexts. Learning to compress sequential and episodic experiences using CSCGs yields allocentric cognitive maps that are suitable for planning, introspection, consolidation, and abstraction. By explicating the role of Euclidean place field mapping and demonstrating how latent sequential representations unify myriad observed phenomena, our work positions the hippocampus in a sequence-centric paradigm, challenging the prevailing space-centric view.

Multisensory conflict affects body schema and reaching space.

Multisensory integration plays a crucial role in building the sense of body ownership, i.e., the perceptual status of one's body for which the body is perceived as belonging to oneself. Temporal and spatial mismatching of visual and tactile signals coming from one's body can reduce ownership feelings towards the body and its parts, i.e., produce disownership feelings. Here, we investigated whether visuo-tactile conflict also affects the sensorimotor representation of the body in space (i.e., body schema) and the perception of the space around the body in terms of action potentiality (i.e., reaching space). In two experiments, body schema (Experiment 1) and reaching space (Experiment 2) were assessed before and after either synchronous or asynchronous visuo-tactile stimulation. Results showed that the asynchronous condition, provoking multisensory conflict, caused disownership over one's hand and concurrently affected the body schema and the reaching space. These findings indicate that body schema and reaching space could be dynamically shaped by the multisensory regularities that build up the sense of body ownership.

Dynamic spatial representation of self and others' actions in the macaque frontal cortex.

Modulation of neuronal firing rates by the spatial locations of physical objects is a widespread phenomenon in the brain. However, little is known about how neuronal responses to the actions of biological entities are spatially tuned and whether such spatially tuned responses are affected by social contexts. These issues are of key importance for understanding the neural basis of embodied social cognition, such as imitation and perspective-taking. Here, we show that spatial representation of actions can be dynamically changed depending on others' social relevance and agents of action. Monkeys performed a turn-taking choice task with a real monkey partner sitting face-to-face or a filmed partner in prerecorded videos. Three rectangular buttons (left, center, and right) were positioned in front of the subject and partner as their choice targets. We recorded from single neurons in two frontal nodes in the social brain, the ventral premotor cortex (PMv) and the medial prefrontal cortex (MPFC). When the partner was filmed rather than real, spatial preference for partner-actions was markedly diminished in MPFC, but not PMv, neurons. This social context-dependent modulation in the MPFC was also evident for self-actions. Strikingly, a subset of neurons in both areas switched their spatial preference between self-actions and partner-actions in a diametrically opposite manner. This observation suggests that these cortical areas are associated with coordinate transformation in ways consistent with an actor-centered perspective-taking coding scheme. The PMv may subserve such functions in context-independent manners, whereas the MPFC may do so primarily in social contexts.

The influence of travel time on perceived traveled distance varies by spatiotemporal scale.

The influence of travel time on perceived traveled distance has often been studied, but the results are inconsistent regarding the relationship between the two magnitudes. We argue that this is due to differences in the lengths of investigated travel distances and hypothesize that the influence of travel time differs for rather short compared to rather long traveled distances. We tested this hypothesis in a virtual environment presented on a desktop as well as through a head-mounted display. Our results show that, for longer distances, more travel time leads to longer perceived distance, while we do not find an influence of travel time on shorter distances. The presentation through an HMD vs. desktop only influenced distance judgments in the short distance condition. These results are in line with the idea that the influence of travel time varies by the length of the traveled distance, and provide insights on the question of how distance perception in path integration studies is affected by travel time, thereby resolving inconsistencies reported in previous studies.

Consequences of eye movements for spatial selectivity.

What determines spatial tuning in the visual system? Standard views rely on the assumption that spatial information is directly inherited from the relative position of photoreceptors and shaped by neuronal connectivity.1,2 However, human eyes are always in motion during fixation,3,4,5,6 so retinal neurons receive temporal modulations that depend on the interaction of the spatial structure of the stimulus with eye movements. It has long been hypothesized that these modulations might contribute to spatial encoding,7,8,9,10,11,12 a proposal supported by several recent observations.13,14,15,16 A fundamental, yet untested, consequence of this encoding strategy is that spatial tuning is not hard-wired in the visual system but critically depends on how the fixational motion of the eye shapes the temporal structure of the signals impinging onto the retina. Here we used high-resolution techniques for eye-tracking17 and gaze-contingent display control18 to quantitatively test this distinctive prediction. We examined how contrast sensitivity, a hallmark of spatial vision, is influenced by fixational motion, both during normal active fixation and when the spatiotemporal stimulus on the retina is altered to mimic changes in fixational control. We showed that visual sensitivity closely follows the strength of the luminance modulations delivered within a narrow temporal bandwidth, so changes in fixational motion have opposite visual effects at low and high spatial frequencies. By identifying a key role for oculomotor activity in spatial selectivity, these findings have important implications for the perceptual consequences of abnormal eye movements, the sources of perceptual variability, and the function of oculomotor control.

Distinct neural mechanisms for heading retrieval and context recognition in the hippocampus during spatial reorientation.

Reorientation, the process of regaining one's bearings after becoming lost, requires identification of a spatial context (context recognition) and recovery of facing direction within that context (heading retrieval). We previously showed that these processes rely on the use of features and geometry, respectively. Here, we examine reorientation behavior in a task that creates contextual ambiguity over a long timescale to demonstrate that male mice learn to combine both featural and geometric cues to recover heading. At the neural level, most CA1 neurons persistently align to geometry, and this alignment predicts heading behavior. However, a small subset of cells remaps coherently in a context-sensitive manner, which serves to predict context. Efficient heading retrieval and context recognition correlate with rate changes reflecting integration of featural and geometric information in the active ensemble. These data illustrate how context recognition and heading retrieval are coded in CA1 and how these processes change with experience.

An allocentric human odometer for perceiving distances on the ground plane.

We reliably judge locations of static objects when we walk despite the retinal images of these objects moving with every step we take. Here, we showed our brains solve this optical illusion by adopting an allocentric spatial reference frame. We measured perceived target location after the observer walked a short distance from the home base. Supporting the allocentric coding scheme, we found the intrinsic bias , which acts as a spatial reference frame for perceiving location of a dimly lit target in the dark, remained grounded at the home base rather than traveled along with the observer. The path-integration mechanism responsible for this can utilize both active and passive (vestibular) translational motion signals, but only along the horizontal direction. This asymmetric path-integration finding in human visual space perception is reminiscent of the asymmetric spatial memory finding in desert ants, pointing to nature's wondrous and logically simple design for terrestrial creatures.

Instruction alters the influence of allocentric landmarks in a reach task.

Allocentric landmarks have an implicit influence on aiming movements, but it is not clear how an explicit instruction (to aim relative to a landmark) influences reach accuracy and precision. Here, 12 participants performed a task with two instruction conditions (egocentric vs. allocentric) but with similar sensory and motor conditions. Participants fixated gaze near the center of a display aligned with their right shoulder while a target stimulus briefly appeared alongside a visual landmark in one visual field. After a brief mask/memory delay the landmark then reappeared at a different location (same or opposite visual field), creating an ego/allocentric conflict. In the egocentric condition, participants were instructed to ignore the landmark and point toward the remembered location of the target. In the allocentric condition, participants were instructed to remember the initial target location relative to the landmark and then reach relative to the shifted landmark (same or opposite visual field). To equalize motor execution between tasks, participants were instructed to anti-point (point to the visual field opposite to the remembered target) on 50% of the egocentric trials. Participants were more accurate and precise and quicker to react in the allocentric condition, especially when pointing to the opposite field. We also observed a visual field effect, where performance was worse overall in the right visual field. These results suggest that, when egocentric and allocentric cues conflict, explicit use of the visual landmark provides better reach performance than reliance on noisy egocentric signals. Such instructions might aid rehabilitation when the egocentric system is compromised by disease or injury.

The Production Effect Becomes Spatial.

In the verbal domain, it is well established that words read aloud are better remembered than their silently read counterparts. It has been hypothesized that this production effect stems from the addition of distinctive features, with the caveat that the processing that generates added features interferes with rehearsal. Here, we tested the idea that a similar trade-off is found in the visuospatial domain. In all experiments, a short series of single dots sequentially appeared at various locations on a screen. Participants produced the items by clicking on them at presentation, watched the items appear quietly, or produced an irrelevant click after each item to better even out rehearsal opportunities between produced and control conditions. In Experiment 1, the dots appeared within a visible grid and an order reconstruction task was used. Experiment 2 also called upon reconstruction, but with the grid removed. In Experiments 3, a recall task was used. The results show that producing items hindered performance compared to the control condition. Conversely, production improved performance compared to the control condition where rehearsal was hindered. This is the first demonstration of a visuospatial production effect. The key findings were successfully modeled by the Revised Feature Model (RFM).

Human navigation strategies and their errors result from dynamic interactions of spatial uncertainties.

Goal-directed navigation requires continuously integrating uncertain self-motion and landmark cues into an internal sense of location and direction, concurrently planning future paths, and sequentially executing motor actions. Here, we provide a unified account of these processes with a computational model of probabilistic path planning in the framework of optimal feedback control under uncertainty. This model gives rise to diverse human navigational strategies previously believed to be distinct behaviors and predicts quantitatively both the errors and the variability of navigation across numerous experiments. This furthermore explains how sequential egocentric landmark observations form an uncertain allocentric cognitive map, how this internal map is used both in route planning and during execution of movements, and reconciles seemingly contradictory results about cue-integration behavior in navigation. Taken together, the present work provides a parsimonious explanation of how patterns of human goal-directed navigation behavior arise from the continuous and dynamic interactions of spatial uncertainties in perception, cognition, and action.