Rapid Recalibration of Peri-Personal Space: Psychophysical, Electrophysiological, and Neural Network Modeling Evidence.

Interactions between individuals and the environment occur within the peri-personal space (PPS). The encoding of this space plastically adapts to bodily constraints and stimuli features. However, these remapping effects have not been demonstrated on an adaptive time-scale, trial-to-trial. Here, we test this idea first via a visuo-tactile reaction time (RT) paradigm in augmented reality where participants are asked to respond as fast as possible to touch, as visual objects approach them. Results demonstrate that RTs to touch are facilitated as a function of visual proximity, and the sigmoidal function describing this facilitation shifts closer to the body if the immediately precedent trial had indexed a smaller visuo-tactile disparity. Next, we derive the electroencephalographic correlates of PPS and demonstrate that this multisensory measure is equally shaped by recent sensory history. Finally, we demonstrate that a validated neural network model of PPS is able to account for the present results via a simple Hebbian plasticity rule. The present findings suggest that PPS encoding remaps on a very rapid time-scale and, more generally, that it is sensitive to sensory history, a key feature for any process contextualizing subsequent incoming sensory information (e.g., a Bayesian prior).

The perspectives of mapping and monitoring of the sense of self in neurosurgical patients.

Surgical treatment of tumors, epileptic foci or of vascular origin, requires a detailed individual pre-surgical workup and intra-operative surveillance of brain functions to minimize the risk of post-surgical neurological deficits and decline of quality of life. Most attention is attributed to language, motor functions, and perception. However, higher cognitive functions such as social cognition, personality, and the sense of self may be affected by brain surgery. To date, the precise localization and the network patterns of brain regions involved in such functions are not yet fully understood, making the assessment of risks of related post-surgical deficits difficult. It is in the interest of neurosurgeons to understand with which neural systems related to selfhood and personality they are interfering during surgery. Recent neuroscience research using virtual reality and clinical observations suggest that the insular cortex, medial prefrontal cortex, and temporo-parietal junction are important components of a neural system dedicated to self-consciousness based on multisensory bodily processing, including exteroceptive and interoceptive cues (bodily self-consciousness (BSC)). Here, we argue that combined extra- and intra-operative approaches using targeted cognitive testing, functional imaging and EEG, virtual reality, combined with multisensory stimulations, may contribute to the assessment of the BSC and related cognitive aspects. Although the usefulness of particular biomarkers, such as cardiac and respiratory signals linked to virtual reality, and of heartbeat evoked potentials as a surrogate marker for intactness of multisensory integration for intra-operative monitoring has to be proved, systemic and automatized testing of BSC in neurosurgical patients will improve future surgical outcome.

First-person body view modulates the neural substrates of episodic memory and autonoetic consciousness: A functional connectivity study.

Episodic memory (EM) is classically conceived as a memory for events, localized in space and time, and characterized by autonoetic consciousness (ANC) allowing to mentally travel back in time and subjectively relive an event. Building on recent evidence that the first-person visual co-perception of one's own body during encoding impacts EM, we used a scene recognition task in immersive virtual reality (VR) and measured how first-person body view would modulate peri-encoding resting-state fMRI, EM performance, and ANC. Specifically, we investigated the impact of body view on post-encoding functional connectivity in an a priori network of regions related either to EM or multisensory bodily processing and used these regions in a seed-to-whole brain analysis. Post-encoding connectivity between right hippocampus (rHC) and right parahippocampus (rPHC) was enhanced when participants encoded scenes while seeing their body. Moreover, the strength of connectivity between the rHC, rPHC and the neocortex displayed two main patterns with respect to body view. The connectivity with a sensorimotor fronto-parietal network, comprising primary somatosensory and primary motor cortices, correlated with ANC after - but not before - encoding, depending on body view. The opposite change of connectivity was found between rHC, rPHC and the medial parietal cortex (from being correlated with ANC before encoding to an absence of correlation after encoding), but irrespective of body view. Linking immersive VR and fMRI for the study of EM and ANC, these findings suggest that seeing one's own body during encoding impacts the brain activity related to EM formation by modulating the connectivity between the right hippocampal formation and the neocortical regions involved in the processing of multisensory bodily signals and self-consciousness.

The Insula Mediates Access to Awareness of Visual Stimuli Presented Synchronously to the Heartbeat.

UNLABELLED: The processing of interoceptive signals in the insular cortex is thought to underlie self-awareness. However, the influence of interoception on visual awareness and the role of the insular cortex in this process remain unclear. Here, we show in a series of experiments that the relative timing of visual stimuli with respect to the heartbeat modulates visual awareness. We used two masking techniques and show that conscious access for visual stimuli synchronous to participants' heartbeat is suppressed compared with the same stimuli presented asynchronously to their heartbeat. Two independent brain imaging experiments using high-resolution fMRI revealed that the insular cortex was sensitive to both visible and invisible cardio-visual stimulation, showing reduced activation for visual stimuli presented synchronously to the heartbeat. Our results show that interoceptive insular processing affects visual awareness, demonstrating the role of the insula in integrating interoceptive and exteroceptive signals and in the processing of conscious signals beyond self-awareness. SIGNIFICANCE STATEMENT: There is growing evidence that interoceptive signals conveying information regarding the internal state of the body influence perception and self-awareness. The insular cortex, which receives sensory inputs from both interoceptive and exteroceptive sources, is thought to integrate these multimodal signals. This study shows that cardiac interoceptive signals modulate awareness for visual stimuli such that visual stimuli occurring at the cardiac frequency take longer to access visual awareness and are more difficult to discriminate. Two fMRI experiments show that the insular region is sensitive to this cardio-visual synchrony even when the visual stimuli are rendered invisible through interocular masking. The results indicate a perceptual and neural suppression for visual events coinciding with cardiac interoceptive signals.

Bilateral Rolandic operculum processing underlying heartbeat awareness reflects changes in bodily self-consciousness.

Exteroceptive bodily signals (including tactile, proprioceptive and visual signals) are important information contributing to self-consciousness. Moreover, prominent theories proposed that visceral signals about internal bodily states are equally or even more important for self-consciousness. Neuroimaging studies have described several brain regions which process signals related to bodily self-consciousness (BSC) based on the integration of exteroceptive signals (e.g. premotor cortex, angular gyrus, supramarginal gyrus and extrastriate body area), and that another brain region, the insula/operculum which is involved in interoception and interoceptive awareness, processes signals critical for self-awareness. Providing evidence for the integration of exteroceptive and interoceptive bodily signals, recent behavioral experiments have demonstrated that the manipulation of interoceptive (e.g. cardiac) signals, coupled with exteroceptive (e.g. visual) signals, also modulates BSC. Does this integration occur within or outside the structures described above? To this end, we adapted a recently designed protocol that uses cardio-visual stimulation to induce altered states of BSC to fMRI. Additionally, we measured neural activity in a classical interoceptive task. We found six brain regions (bilateral Rolandic operculum, bilateral supramarginal gyrus, right frontal inferior operculum and left temporal superior gyrus) that were activated differently during the interoception task as opposed to a control task. The brain regions which showed the highest selectivity for BSC based on our cardio-visual manipulation were found in the bilateral Rolandic operculum. Given our findings, we propose that the Rolandic operculum processes integrated exteroceptive-interoceptive signals that are necessary for interoceptive awareness as well as BSC.

Oscillatory neural responses evoked by natural vestibular stimuli in humans.

While there have been numerous studies of the vestibular system in mammals, less is known about the brain mechanisms of vestibular processing in humans. In particular, of the studies that have been carried out in humans over the last 30 years, none has investigated how vestibular stimulation (VS) affects cortical oscillations. Here we recorded high-density electroencephalography (EEG) in healthy human subjects and a group of bilateral vestibular loss patients (BVPs) undergoing transient and constant-velocity passive whole body yaw rotations, focusing our analyses on the modulation of cortical oscillations in response to natural VS. The present approach overcame significant technical challenges associated with combining natural VS with human electrophysiology and reveals that both transient and constant-velocity VS are associated with a prominent suppression of alpha power (8-13 Hz). Alpha band suppression was localized over bilateral temporo-parietal scalp regions, and these alpha modulations were significantly smaller in BVPs. We propose that suppression of oscillations in the alpha band over temporo-parietal scalp regions reflects cortical vestibular processing, potentially comparable with alpha and mu oscillations in the visual and sensorimotor systems, respectively, opening the door to the investigation of human cortical processing under various experimental conditions during natural VS.

Tuning of temporo-occipital activity by frontal oscillations during virtual mirror exposure causes erroneous self-recognition.

Self-face recognition, a hallmark of self-awareness, depends on 'off-line' stored information about one's face and 'on-line' multisensory-motor face-related cues. The brain mechanisms of how on-line sensory-motor processes affect off-line neural self-face representations are unknown. This study used 3D virtual reality to create a 'virtual mirror' in which participants saw an avatar's face moving synchronously with their own face movements. Electroencephalographic (EEG) analysis during virtual mirror exposure revealed mu oscillations in sensory-motor cortex signalling on-line congruency between the avatar's and participants' movements. After such exposure and compatible with a change in their off-line self-face representation, participants were more prone to recognize the avatar's face as their own, and this was also reflected in the activation of face-specific regions in the inferotemporal cortex. Further EEG analysis showed that the on-line sensory-motor effects during virtual mirror exposure caused these off-line visual effects, revealing the brain mechanisms that maintain a coherent self-representation, despite our continuously changing appearance.

Balancing awareness: Vestibular signals modulate visual consciousness in the absence of awareness.

The processing of visual and vestibular information is crucial for perceiving self-motion. Visual cues, such as optic flow, have been shown to induce and alter vestibular percepts, yet the role of vestibular information in shaping visual awareness remains unclear. Here we investigated if vestibular signals influence the access to awareness of invisible visual signals. Using natural vestibular stimulation (passive yaw rotations) on a vestibular self-motion platform, and optic flow masked through continuous flash suppression (CFS) we tested if congruent visual-vestibular information would break interocular suppression more rapidly than incongruent information. We found that when the unseen optic flow was congruent with the vestibular signals perceptual suppression as quantified with the CFS paradigm was broken more rapidly than when it was incongruent. We argue that vestibular signals impact the formation of visual awareness through enhanced access to awareness for congruent multisensory stimulation.

In Case of Doubt, One Follows One's Self: The Implicit Guidance of the Embodied Self-Avatar.

The sense of embodiment in virtual reality (VR) is commonly understood as the subjective experience that one's physical body is substituted by a virtual counterpart, and is typically achieved when the avatar's body, seen from a first-person view, moves like one's physical body. Embodiment can also be experienced in other circumstances (e.g., in third-person view) or with imprecise or distorted visuo-motor coupling. It was moreover observed, in various cases of small or progressive temporal and spatial manipulations of avatars' movements, that participants may spontaneously follow the movement shown by the avatar. The present work investigates whether, in some specific contexts, participants would follow what their avatar does even when large movement discrepancies occur, thereby extending the scope of understanding of the self-avatar follower effect beyond subtle changes of motion or speed manipulations. We conducted an experimental study in which we introduced uncertainty about which movement to perform at specific times and analyzed participants' movements and subjective feedback after their avatar showed them an incorrect movement. Results show that, when in doubt, participants were influenced by their avatar's movements, leading them to perform that particular error twice more often than normal. Importantly, results of the embodiment score indicate that participants experienced a dissociation with their avatar at those times. Overall, these observations not only demonstrate the possibility of provoking situations in which participants follow the guidance of their avatar for large motor distortions, despite their awareness about the avatar movement disruption and on the possible influence it had on their choice, and, importantly, exemplify how the cognitive mechanism of embodiment is deeply rooted in the necessity of having a body.

Numerosity estimation of virtual humans as a digital-robotic marker for hallucinations in Parkinson's disease.

Hallucinations are frequent non-motor symptoms in Parkinson's disease (PD) associated with dementia and higher mortality. Despite their high clinical relevance, current assessments of hallucinations are based on verbal self-reports and interviews that are limited by important biases. Here, we used virtual reality (VR), robotics, and digital online technology to quantify presence hallucination (vivid sensations that another person is nearby when no one is actually present and can neither be seen nor heard) in laboratory and home-based settings. We establish that elevated numerosity estimation of virtual human agents in VR is a digital marker for experimentally induced presence hallucinations in healthy participants, as confirmed across several control conditions and analyses. We translated the digital marker (numerosity estimation) to an online procedure that 170 PD patients carried out remotely at their homes, revealing that PD patients with disease-related presence hallucinations (but not control PD patients) showed higher numerosity estimation. Numerosity estimation enables quantitative monitoring of hallucinations, is an easy-to-use unobtrusive online method, reaching people far away from medical centers, translating neuroscientific findings using robotics and VR, to patients' homes without specific equipment or trained staff.

Customizing the human-avatar mapping based on EEG error related potentials.

Objective.A key challenge of virtual reality (VR) applications is to maintain a reliable human-avatar mapping. Users may lose the sense of controlling (sense of agency), owning (sense of body ownership), or being located (sense of self-location) inside the virtual body when they perceive erroneous interaction, i.e. a break-in-embodiment (BiE). However, the way to detect such an inadequate event is currently limited to questionnaires or spontaneous reports from users. The ability to implicitly detect BiE in real-time enables us to adjust human-avatar mapping without interruption.Approach.We propose and empirically demonstrate a novel brain computer interface (BCI) approach that monitors the occurrence of BiE based on the users' brain oscillatory activity in real-time to adjust the human-avatar mapping in VR. We collected EEG activity of 37 participants while they performed reaching movements with their avatar with different magnitude of distortion.Main results.Our BCI approach seamlessly predicts occurrence of BiE in varying magnitude of erroneous interaction. The mapping has been customized by BCI-reinforcement learning (RL) closed-loop system to prevent BiE from occurring. Furthermore, a non-personalized BCI decoder generalizes to new users, enabling 'Plug-and-Play' ErrP-based non-invasive BCI. The proposed VR system allows customization of human-avatar mapping without personalized BCI decoders or spontaneous reports.Significance.We anticipate that our newly developed VR-BCI can be useful to maintain an engaging avatar-based interaction and a compelling immersive experience while detecting when users notice a problem and seamlessly correcting it.

Out-of-body illusion induced by visual-vestibular stimulation.

Out-of-body experiences (OBEs) are characterized by the subjective feeling of being located outside one's physical body and perceiving one's own body from an elevated perspective looking downwards. OBEs have been correlated with abnormal integration of bodily signals, including visual and vestibular information. In two studies, we used mixed reality combined with a motion platform to manipulate visual and vestibular integration in healthy participants. Behavioral data and questionnaires show that congruent visual-vestibular stimulation in a self-centered reference frame induced an OBE-like illusion characterized by elevated self-location and feelings of disembodiment and lightness. The OBE-like illusion was also modulated by individuals' visual field dependency assessed by the Rod and Frame Test. These results show that the manipulation of visual-vestibular stimulation in the present study induces various aspects of OBEs and further link OBE to congruency mechanisms between visual and vestibular gravitational and self-motion cues.

Turning body and self inside out: visualized heartbeats alter bodily self-consciousness and tactile perception.

Prominent theories highlight the importance of bodily perception for self-consciousness, but it is currently not known whether bodily perception is based on interoceptive or exteroceptive signals or on integrated signals from these anatomically distinct systems. In the research reported here, we combined both types of signals by surreptitiously providing participants with visual exteroceptive information about their heartbeat: A real-time video image of a periodically illuminated silhouette outlined participants' (projected, "virtual") bodies and flashed in synchrony with their heartbeats. We investigated whether these "cardio-visual" signals could modulate bodily self-consciousness and tactile perception. We report two main findings. First, synchronous cardio-visual signals increased self-identification with and self-location toward the virtual body, and second, they altered the perception of tactile stimuli applied to participants' backs so that touch was mislocalized toward the virtual body. We argue that the integration of signals from the inside and the outside of the human body is a fundamental neurobiological process underlying self-consciousness.

Posing for awareness: proprioception modulates access to visual consciousness in a continuous flash suppression task.

The rules governing the selection of which sensory information reaches consciousness are yet unknown. Of our senses, vision is often considered to be the dominant sense, and the effects of bodily senses, such as proprioception, on visual consciousness are frequently overlooked. Here, we demonstrate that the position of the body influences visual consciousness. We induced perceptual suppression by using continuous flash suppression. Participants had to judge the orientation a target stimulus embedded in a task-irrelevant picture of a hand. The picture of the hand could either be congruent or incongruent with the participants' actual hand position. When the viewed and the real hand positions were congruent, perceptual suppression was broken more rapidly than during incongruent trials. Our findings provide the first evidence of a proprioceptive bias in visual consciousness, suggesting that proprioception not only influences the perception of one's own body and self-consciousness, but also visual consciousness.

Avatar error in your favor: Embodied avatars can fix users' mistakes without them noticing.

In immersive Virtual Reality (VR), users can experience the subjective feeling of embodiment for the avatar representing them in a virtual world. This is known to be strongly supported by a high Sense of Agency (SoA) for the movements of the avatar that follows the user. In general, users do not self-attribute actions of their avatar that are different from the one they actually performed. The situation is less clear when actions of the avatar satisfies the intention of the user despite distortions and noticeable differences between user and avatar movements. Here, a within-subject experiment was condutected to determine wether a finger swap helping users to achieve a task would be more tolerated than one penalizing them. In particular, in a context of fast-paced finger movements and with clear correct or incorrect responses, we swapped the finger animation of the avatar (e.g. user moves the index finger, the avatar moves the middle one) to either automatically correct for spontaneous mistakes or to introduce incorrect responses. Subjects playing a VR game were asked to report when they noticed the introduction of a finger swap. Results based on 3256 trials (∼24% of swaps noticed) show that swaps helping users have significantly fewer odds of being noticed (and with higher confidence) than the ones penalizing users. This demonstrates how the context and the intention for motor action are important factors for the SoA and for embodiment, opening new perspectives on how to design and study interactions in immersive VR.

The full-body illusion changes visual depth perception.

Knowing where objects are relative to us implies knowing where we are relative to the external world. Here, we investigated whether space perception can be influenced by an experimentally induced change in perceived self-location. To dissociate real and apparent body positions, we used the full-body illusion. In this illusion, participants see a distant avatar being stroked in virtual reality while their own physical back is simultaneously stroked. After experiencing the discrepancy between the seen and the felt location of the stroking, participants report a forward drift in self-location toward the avatar. We wondered whether this illusion-induced forward drift in self-location would affect where we perceive objects in depth. We applied a psychometric measurement in which participants compared the position of a probe against a reference sphere in a two-alternative forced choice task. We found a significant improvement in task performance for the right visual field, indicated by lower just-noticeable differences, i.e., participants were better at judging the differences of the two spheres in depth. Our results suggest that the full-body illusion is able to facilitate depth perception at least unilaterally, implying that depth perception is influenced by perceived self-location.

EEG signature of breaks in embodiment in VR.

The brain mechanism of embodiment in a virtual body has grown a scientific interest recently, with a particular focus on providing optimal virtual reality (VR) experiences. Disruptions from an embodied state to a less- or non-embodied state, denominated Breaks in Embodiment (BiE), are however rarely studied despite their importance for designing interactions in VR. Here we use electroencephalography (EEG) to monitor the brain's reaction to a BiE, and investigate how this reaction depends on previous embodiment conditions. The experimental protocol consisted of two sequential steps; an induction step where participants were either embodied or non-embodied in an avatar, and a monitoring step where, in some cases, participants saw the avatar's hand move while their hand remained still. Our results show the occurrence of error-related potentials linked to observation of the BiE event in the monitoring step. Importantly, this EEG signature shows amplified potentials following the non-embodied condition, which is indicative of an accumulation of errors across steps. These results provide neurophysiological indications on how progressive disruptions impact the expectation of embodiment for a virtual body.

Virtual reality intervention alleviates dyspnoea in patients recovering from COVID-19 pneumonia.

Background: Immersive virtual reality (iVR)-based digital therapeutics are gaining clinical attention in the field of pain management. Based on known analogies between pain and dyspnoea, we investigated the effects of visual respiratory feedback on persistent dyspnoea in patients recovering from coronavirus disease 2019 (COVID-19) pneumonia. Methods: We performed a controlled, randomised, single-blind, crossover proof-of-concept study (feasibility and initial clinical efficacy) to evaluate an iVR-based intervention to alleviate dyspnoea in patients recovering from COVID-19 pneumonia. Included patients reported persistent dyspnoea (≥5 on a 10-point scale) and preserved cognitive function (Montreal Cognitive Assessment score >24). Assignment was random and concealed. Patients received synchronous (intervention) or asynchronous (control) feedback of their breathing, embodied via a gender-matched virtual body. The virtual body flashed in a waxing and waning visual effect that could be synchronous or asynchronous to the patient's respiratory movements. Outcomes were assessed using questionnaires and breathing recordings. Results: Study enrolment was open between November 2020 and April 2021. 26 patients were enrolled (27% women; median age 55 years, interquartile range (IQR) 18 years). Data were available for 24 of 26 patients. The median rating on a 7-point Likert scale of breathing comfort improved from 1 (IQR 2) at baseline to 2 (IQR 1) for synchronous feedback, but remained unchanged at 1 (IQR 1.5) for asynchronous feedback (p<0.05 between iVR conditions). Moreover, 91.2% of all patients were satisfied with the intervention (p<0.0001) and 66.7% perceived it as beneficial for their breathing (p<0.05). Conclusion: Our iVR-based digital therapy presents a feasible and safe respiratory rehabilitation tool that improves breathing comfort in patients recovering from COVID-19 infection presenting with persistent dyspnoea. Future research should investigate the intervention's generalisability to persistent dyspnoea with other aetiologies and its potential for preventing chronification.

Reaching articular limits can negatively impact embodiment in virtual reality.

Providing Virtual Reality(VR) users with a 3D representation of their body complements the experience of immersion and presence in the virtual world with the experience of being physically located and more personally involved. A full-body avatar representation is known to induce a Sense of Embodiment (SoE) for this virtual body, which is associated with improvements in task performance, motivation and motor learning. Recent experimental research on embodiment provides useful guidelines, indicating the extent of discrepancy tolerated by users and, conversely, the limits and disruptive events that lead to a break in embodiment (BiE). Based on previous works on the limit of agency under movement distortion, this paper describes, studies and analyses the impact of a very common yet overlooked embodiment limitation linked to articular limits when performing a reaching movement. We demonstrate that perceiving the articular limit when fully extending the arm provides users with an additional internal proprioceptive feedback which, if not matched in the avatar's movement, leads to the disruptive realization of an incorrect posture mapping. This study complements previous works on self-contact and visuo-haptic conflicts and emphasizes the risk of disrupting the SoE when distorting users' movements or using a poorly-calibrated avatar.