Short communication: Binocular rivalry dynamics during locomotion.

Locomotion has been shown to impact aspects of visual processing in both humans and animal models. In the current study, we assess the impact of locomotion on the dynamics of binocular rivalry. We presented orthogonal gratings, one contrast-modulating at 0.8 Hz (matching average step frequency) and the other at 3.2 Hz, to participants using a virtual reality headset. We compared two conditions: stationary and walking. We continuously monitored participants' foot position using tracking devices to measure the step cycle. During the walking condition, participants viewed the rivaling gratings for 60-second trials while walking on a circular path in a virtual reality environment. During the stationary condition, observers viewed the same stimuli and environment while standing still. The task was to continuously indicate the dominant percept via button press using handheld controllers. We found no significant differences between walking and standing for normalized dominance duration distributions, mean normalized dominance distributions, mean alternation rates, or mean fitted frequencies. Although our findings do not align with prior research highlighting distinctions in normalized dominance distributions between walking and standing, our study contributes unique evidence indicating that alternation rates vary across the step cycle. Specifically, we observed that the number of alternations is at its lowest during toe-off phases and reaches its peak at heel strike. This novel insight enhances our understanding of the dynamic nature of alternation patterns throughout the step cycle.

Impaired stationarity perception is associated with increased virtual reality sickness.

Stationarity perception refers to the ability to accurately perceive the surrounding visual environment as world-fixed during self-motion. Perception of stationarity depends on mechanisms that evaluate the congruence between retinal/oculomotor signals and head movement signals. In a series of psychophysical experiments, we systematically varied the congruence between retinal/oculomotor and head movement signals to find the range of visual gains that is compatible with perception of a stationary environment. On each trial, human subjects wearing a head-mounted display execute a yaw head movement and report whether the visual gain was perceived to be too slow or fast. A psychometric fit to the data across trials reveals the visual gain most compatible with stationarity (a measure of accuracy) and the sensitivity to visual gain manipulation (a measure of precision). Across experiments, we varied 1) the spatial frequency of the visual stimulus, 2) the retinal location of the visual stimulus (central vs. peripheral), and 3) fixation behavior (scene-fixed vs. head-fixed). Stationarity perception is most precise and accurate during scene-fixed fixation. Effects of spatial frequency and retinal stimulus location become evident during head-fixed fixation, when retinal image motion is increased. Virtual Reality sickness assessed using the Simulator Sickness Questionnaire covaries with perceptual performance. Decreased accuracy is associated with an increase in the nausea subscore, while decreased precision is associated with an increase in the oculomotor and disorientation subscores.

Motor Signals Mediate Stationarity Perception.

Head movement relative to the stationary environment gives rise to congruent vestibular and visual optic-flow signals. The resulting perception of a stationary visual environment, referred to herein as stationarity perception, depends on mechanisms that compare visual and vestibular signals to evaluate their congruence. Here we investigate the functioning of these mechanisms and their dependence on fixation behavior as well as on the active versus passive nature of the head movement. Stationarity perception was measured by modifying the gain on visual motion relative to head movement on individual trials and asking subjects to report whether the gain was too low or too high. Fitting a psychometric function to the data yields two key parameters of performance. The mean is a measure of accuracy, and the standard deviation is a measure of precision. Experiments were conducted using a head-mounted display with fixation behavior monitored by an embedded eye tracker. During active conditions, subjects rotated their heads in yaw ∼15 deg/s over ∼1 s. Each subject's movements were recorded and played back via rotating chair during the passive condition. During head-fixed and scene-fixed fixation the fixation target moved with the head or scene, respectively. Both precision and accuracy were better during active than passive head movement, likely due to increased precision on the head movement estimate arising from motor prediction and neck proprioception. Performance was also better during scene-fixed than head-fixed fixation, perhaps due to decreased velocity of retinal image motion and increased precision on the retinal image motion estimate. These results reveal how the nature of head and eye movements mediate encoding, processing, and comparison of relevant sensory and motor signals.

Continuous peripersonal tracking accuracy is limited by the speed and phase of locomotion.

Recent evidence suggests that perceptual and cognitive functions are codetermined by rhythmic bodily states. Prior investigations have focused on the cardiac and respiratory rhythms, both of which are also known to synchronise with locomotion-arguably our most common and natural of voluntary behaviours. Compared to the cardiorespiratory rhythms, walking is easier to voluntarily control, enabling a test of how natural and voluntary rhythmic action may affect sensory function. Here we show that the speed and phase of human locomotion constrains sensorimotor performance. We used a continuous visuo-motor tracking task in a wireless, body-tracking virtual environment, and found that the accuracy and reaction time of continuous reaching movements were decreased at slower walking speeds, and rhythmically modulated according to the phases of the step-cycle. Decreased accuracy when walking at slow speeds suggests an advantage for interlimb coordination at normal walking speeds, in contrast to previous research on dual-task walking and reach-to-grasp movements. Phasic modulations of reach precision within the step-cycle also suggest that the upper limbs are affected by the ballistic demands of motor-preparation during natural locomotion. Together these results show that the natural phases of human locomotion impose constraints on sensorimotor function and demonstrate the value of examining dynamic and natural behaviour in contrast to the traditional and static methods of psychological science.

Eye Tracking in Virtual Reality: a Broad Review of Applications and Challenges.

Eye tracking is becoming increasingly available in head-mounted virtual reality displays with various headsets with integrated eye trackers already commercially available. The applications of eye tracking in virtual reality are highly diversified and span multiple disciplines. As a result, the number of peer-reviewed publications that study eye tracking applications has surged in recent years. We performed a broad review to comprehensively search academic literature databases with the aim of assessing the extent of published research dealing with applications of eye tracking in virtual reality, and highlighting challenges, limitations and areas for future research.

Natural statistics of human head orientation constrain models of vestibular processing.

Head orientation relative to gravity determines how gravity-dependent environmental structure is sampled by the visual system, as well as how gravity itself is sampled by the vestibular system. Therefore, both visual and vestibular sensory processing should be shaped by the statistics of head orientation relative to gravity. Here we report the statistics of human head orientation during unconstrained natural activities in humans for the first time, and we explore implications for models of vestibular processing. We find that the distribution of head pitch is more variable than head roll and that the head pitch distribution is asymmetrical with an over-representation of downward head pitch, consistent with ground-looking behavior. We further suggest that pitch and roll distributions can be used as empirical priors in a Bayesian framework to explain previously measured biases in perception of both roll and pitch. Gravitational and inertial acceleration stimulate the otoliths in an equivalent manner, so we also analyze the dynamics of human head orientation to better understand how knowledge of these dynamics can constrain solutions to the problem of gravitoinertial ambiguity. Gravitational acceleration dominates at low frequencies and inertial acceleration dominates at higher frequencies. The change in relative power of gravitational and inertial components as a function of frequency places empirical constraints on dynamic models of vestibular processing, including both frequency segregation and probabilistic internal model accounts. We conclude with a discussion of methodological considerations and scientific and applied domains that will benefit from continued measurement and analysis of natural head movements moving forward.

Natural statistics of human head orientation constrain models of vestibular processing.

Head orientation relative to gravity determines how gravity-dependent environmental structure is sampled by the visual system, as well as how gravity itself is sampled by the vestibular system. Therefore, both visual and vestibular sensory processing should be shaped by the statistics of head orientation relative to gravity. Here we report the statistics of human head orientation during unconstrained natural activities in humans for the first time, and we explore implications for models of vestibular processing. We find that the distribution of head pitch is more variable than head roll and that the head pitch distribution is asymmetrical with an over-representation of downward head pitch, consistent with ground-looking behavior. We further show that pitch and roll distributions can be used as empirical priors in a Bayesian framework to explain previously measured biases in perception of both roll and pitch. We also analyze the dynamics of human head orientation to better understand how gravitational and inertial acceleration are processed by the vestibular system. Gravitational acceleration dominates at low frequencies and inertial acceleration dominates at higher frequencies. The change in relative power of gravitational and inertial components as a function of frequency places empirical constraints on dynamic models of vestibular processing. We conclude with a discussion of methodological considerations and scientific and applied domains that will benefit from continued measurement and analysis of natural head movements moving forward.

VR Sickness Adaptation With Ramped Optic Flow Transfers From Abstract To Realistic Environments.

VR sickness is a major concern for many users as VR continues its expansion towards widespread everyday use. VR sickness is thought to arise, at least in part, due to the user's intolerance of conflict between the visually simulated self-motion and actual physical movement. Many mitigation strategies involve consistently modifying the visual stimulus to reduce its impact on the user, but this individualized approach can have drawbacks in terms of complexity of implementation and non-uniformity of user experience. This study presents a novel alternative approach that involves training the user to better tolerate the adverse stimulus by tapping into natural adaptive perceptual mechanisms. In this study, we recruited users with limited VR experience that reported susceptibility to VR sickness. Baseline sickness was measured as participants navigated a rich and naturalistic visual environment. Then, on successive days, participants were exposed to optic flow in a more abstract visual environment, and strength of the optic flow was successively increased by increasing the visual contrast of the scene, because strength of optic flow and the resulting vection are thought to be major causes of VR sickness. Sickness measures decreased on successive days, indicating that adaptation was successful. On the final day, participants were again exposed to the rich and naturalistic visual environment, and the adaptation was maintained, demonstrating that it is possible for adaptation to transfer from more abstract to richer and more naturalistic environments. These results demonstrate that gradual adaptation to increasing optic flow strength in well-controlled, abstract environments allows users to gradually reduce their susceptibility to sickness, thereby increasing VR accessibility for those prone to sickness.

Evaluation of the Intel RealSense T265 for tracking natural human head motion.

Accurate and robust tracking of natural human head motion in natural environments is important for a number of applications including virtual and augmented reality, clinical diagnostics, as well as basic scientific research. IMU provide a versatile solution for recording inertial data including linear acceleration and angular velocity, but reconstructing head position is difficult or impossible. This problem can be solved by incorporating visual data using a technique known as visual-inertial simultaneous localization and mapping (VI-SLAM). A recently released commercial solution, the Intel RealSense T265, uses a proprietary VI-SLAM algorithm to estimate linear and angular position and velocity, but the performance of this device for tracking of natural human head motion in natural environments has not yet been comprehensively evaluated against gold-standard methods. In this study, we used a wide range of metrics to evaluate the performance of the T265 with different walking speeds in different environments, both indoor and outdoor, against two gold-standard methods, an optical tracking system and a so-called perambulator. Overall, we find that performance of the T265 relative to these gold-standard methods is most accurate for slow to normal walking speeds in small- to medium-sized environments. The suitability of this device for future scientific studies depends on the application; data presented here can be useful in making that determination.

Perception of postural verticality in roll and pitch while sitting and standing in healthy subjects.

Assessment of verticality perception is increasingly used in patients with disorders of upright body orientation, e.g. in pusher behavior after hemispheric stroke or in retropulsion in neurovascular or degenerative diseases. The subjective postural vertical (SPV) assesses the individual's perceived upright body orientation in space and can be determined in a sitting or a standing position. As somatosensory input differs between sitting and standing, we suppose a different representation of verticality. Thus, the aim of this study was to compare the SPV while sitting and while standing, and to determine the influence of proprioceptive and visual stimulations on the SPV in both positions. Twenty healthy subjects (18-40 years, 9 female) participated in two experiments. In experiment 1 the SPV was assessed in the pitch and roll planes while sitting and standing. In experiment 2 we additionally applied somatosensory Achilles tendon vibration and visual optokinetic stimulation. Results of experiment 1 revealed no difference of the SPV error between sitting and standing. Though, we found an increased SPV variability while sitting. Experiment 2 showed no effect of sensory manipulation in the pitch plane, but an effect of optokinetic stimulation in the roll plane. The tilt in direction of the visual stimuli was more pronounced while sitting. Optokinetic stimulation also resulted in a larger SPV variability in roll, indicating larger insecurity in verticality estimation during stimulation. In conclusion, even though the SPV assessment while sitting and standing resulted in similar mean values, the larger variability and the higher sensitivity to visual stimuli while sitting suggest different strategies for verticality estimation while sitting and while standing.

Positional head-eye tracking outside the lab: an open-source solution.

Simultaneous head and eye tracking has traditionally been confined to a laboratory setting and real-world motion tracking limited to measuring linear acceleration and angular velocity. Recently available mobile devices such as the Pupil Core eye tracker and the Intel RealSense T265 motion tracker promise to deliver accurate measurements outside the lab. Here, the researchers propose a hard- and software framework that combines both devices into a robust, usable, low-cost head and eye tracking system. The developed software is open source and the required hardware modifications can be 3D printed. The researchers demonstrate the system's ability to measure head and eye movements in two tasks: an eyes-fixed head rotation task eliciting the vestibulo-ocular reflex inside the laboratory, and a natural locomotion task where a subject walks around a building outside of the laboratory. The resultant head and eye movements are discussed, as well as future implementations of this system.

GazeMetrics: An Open-Source Tool for Measuring the Data Quality of HMD-based Eye Trackers.

As virtual reality (VR) garners more attention for eye tracking research, knowledge of accuracy and precision of head-mounted display (HMD) based eye trackers becomes increasingly necessary. It is tempting to rely on manufacturer-provided information about the accuracy and precision of an eye tracker. However, unless data is collected under ideal conditions, these values seldom align with on-site metrics. Therefore, best practices dictate that accuracy and precision should be measured and reported for each study. To address this issue, we provide a novel open-source suite for rigorously measuring accuracy and precision for use with a variety of HMD-based eye trackers. This tool is customizable without having to alter the source code, but changes to the code allow for further alteration. The outputs are available in real time and easy to interpret, making eye tracking with VR more approachable for all users.

Virtual Locomotion: A Survey.

Virtual reality (VR) has enjoyed significant popularity in recent years. Where navigation has been a fundamental appeal of 3D applications for decades, facilitating this in VR has been quite a challenge. Over the past decades, various virtual locomotion techniques (VLTs) have been developed that aim to offer natural, usable and efficient ways of navigating VR without inducing VR sickness. Several studies of these techniques have been conducted in order to evaluate their performance in various study conditions and virtual contexts. Taxonomies have also been proposed to either place similar techniques in meaningful categories or decompose them to their underlying design components. In this survey, we aim to aggregate and understand the current state of the art of VR locomotion research and discuss the design implications of VLTs in terms of strengths, weaknesses and applicability.

Ecological Momentary Assessment of Head Motion: Toward Normative Data of Head Stabilization.

Head stabilization is fundamental for balance during locomotion but can be impaired in elderly or diseased populations. Previous studies have identified several parameters of head stability with possible diagnostic value in a laboratory setting. Recently, the ecological validity of measures obtained in such controlled contexts has been called into question. The aim of this study was to investigate the ecological validity of previously described parameters of head stabilization in a real-world setting. Ten healthy subjects participated in the study. Head and trunk movements of each subject were recorded with inertial measurement units (IMUs) for a period of at least 10 h. Periods of locomotion were extracted from the measurements and predominant frequencies, root mean squares (RMSs) and bout lengths were estimated. As parameters of head stabilization, attenuation coefficients (ACs), harmonic ratios (HRs), coherences, and phase differences were computed. Predominant frequencies were distributed tightly around 2 Hz and ACs, HRs, and coherences exhibited the highest values in this frequency range. All head stability parameters exhibited characteristics consistent with previous reports, although higher variances were observed. These results suggest that head stabilization is tuned to the 2 Hz fundamental frequency of locomotion and that previously described measures of head stability could generalize to a real-world setting. This is the first study to address the ecological validity of these measures, highlighting the potential use of head stability parameters as diagnostic tools or outcome measures for clinical trials. The low cost and ease of use of the IMU technology used in this study could additionally be of benefit for a clinical application.

Towards dynamic modeling of visual-vestibular conflict detection.

Visual-vestibular mismatch is a common occurrence, with causes ranging from vehicular travel, to vestibular dysfunction, to virtual reality displays. Behavioral and physiological consequences of this mismatch include adaptation of reflexive eye movements, oscillopsia, vertigo, and nausea. Despite this significance, we still do not have a good understanding of how the nervous system evaluates visual-vestibular conflict. Here we review research that quantifies perceptual sensitivity to visual-vestibular conflict and factors that mediate this sensitivity, such as noise on visual and vestibular sensory estimates. We emphasize that dynamic modeling methods are necessary to investigate how the nervous system monitors conflict between time-varying visual and vestibular signals, and we present a simple example of a drift-diffusion model for visual-vestibular conflict detection. The model makes predictions for detection of conflict arising from changes in both visual gain and latency. We conclude with discussion of topics for future research.

Vestibulo-Ocular Responses and Dynamic Visual Acuity During Horizontal Rotation and Translation.

Dynamic visual acuity (DVA) provides an overall functional measure of visual stabilization performance that depends on the vestibulo-ocular reflex (VOR), but also on other processes, including catch-up saccades and likely visual motion processing. Capturing the efficiency of gaze stabilization against head movement as a whole, it is potentially valuable in the clinical context where assessment of overall patient performance provides an important indication of factors impacting patient participation and quality of life. DVA during head rotation (rDVA) has been assessed previously, but to our knowledge, DVA during horizontal translation (tDVA) has not been measured. tDVA can provide a valuable measure of how otolith, rather than canal, function impacts visual acuity. In addition, comparison of DVA during rotation and translation can shed light on whether common factors are limiting DVA performance in both cases. We therefore measured and compared DVA during both passive head rotations (head impulse test) and translations in the same set of healthy subjects (n = 7). In addition to DVA, we computed average VOR gain and retinal slip within and across subjects. We observed that during translation, VOR gain was reduced (VOR during rotation, mean ± SD: position gain = 1.05 ± 0.04, velocity gain = 0.97 ± 0.07; VOR during translation, mean ± SD: position gain = 0.21 ± 0.08, velocity gain = 0.51 ± 0.16), retinal slip was increased, and tDVA was worse than during rotation (average rDVA = 0.32 ± 0.15 logMAR; average tDVA = 0.56 ± 0.09 logMAR, p = 0.02). This suggests that reduced VOR gain leads to worse tDVA, as expected. We conclude with speculation about non-oculomotor factors that could vary across individuals and affect performance similarly during both rotation and translation.

Sensitivity to Visual Speed Modulation in Head-Mounted Displays Depends on Fixation.

A primary cause of simulator sickness in head-mounted displays (HMDs) is conflict between the visual scene displayed to the user and the visual scene expected by the brain when the user's head is in motion. It is useful to measure perceptual sensitivity to visual speed modulation in HMDs because conditions that minimize this sensitivity may prove less likely to elicit simulator sickness. In prior research, we measured sensitivity to visual gain modulation during slow, passive, full-body yaw rotations and observed that sensitivity was reduced when subjects fixated a head-fixed target compared with when they fixated a scene-fixed target. In the current study, we investigated whether this pattern of results persists when (1) movements are faster, active head turns, and (2) visual stimuli are presented on an HMD rather than on a monitor. Subjects wore an Oculus Rift CV1 HMD and viewed a 3D scene of white points on a black background. On each trial, subjects moved their head from a central position to face a 15° eccentric target. During the head movement they fixated a point that was either head-fixed or scene-fixed, depending on condition. They then reported if the visual scene motion was too fast or too slow. Visual speed on subsequent trials was modulated according to a staircase procedure to find the speed increment that was just noticeable. Sensitivity to speed modulation during active head movement was reduced during head-fixed fixation, similar to what we observed during passive whole-body rotation. We conclude that fixation of a head-fixed target is an effective way to reduce sensitivity to visual speed modulation in HMDs, and may also be an effective strategy to reduce susceptibility to simulator sickness.

Insufficient compensation for self-motion during perception of object speed: The vestibular Aubert-Fleischl phenomenon.

To estimate object speed with respect to the self, retinal signals must be summed with extraretinal signals that encode the speed of eye and head movement. Prior work has shown that differences in perceptual estimates of object speed based on retinal and oculomotor signals lead to biased percepts such as the Aubert-Fleischl phenomenon (AF), in which moving targets appear slower when pursued. During whole-body movement, additional extraretinal signals, such as those from the vestibular system, may be used to transform object speed estimates from a head-centered to a world-centered reference frame. Here we demonstrate that whole-body pursuit in the form of passive yaw rotation, which stimulates the semicircular canals of the vestibular system, leads to a slowing of perceived object speed similar to the classic oculomotor AF. We find that the magnitude of the vestibular and oculomotor AF is comparable across a range of speeds, despite the different types of input signal involved. This covariation might hint at a common modality-independent mechanism underlying the AF in both cases.

Gravity Perception: The Role of the Cerebellum.

The cerebellum is known to support motor behaviors, including postural stability, but new research supports the view that cerebellar function is also critical for perception of spatial orientation, particularly because of its role in vestibular processing.

Psychophysical evidence for auditory motion parallax.

Distance is important: From an ecological perspective, knowledge about the distance to either prey or predator is vital. However, the distance of an unknown sound source is particularly difficult to assess, especially in anechoic environments. In vision, changes in perspective resulting from observer motion produce a reliable, consistent, and unambiguous impression of depth known as motion parallax. Here we demonstrate with formal psychophysics that humans can exploit auditory motion parallax, i.e., the change in the dynamic binaural cues elicited by self-motion, to assess the relative depths of two sound sources. Our data show that sensitivity to relative depth is best when subjects move actively; performance deteriorates when subjects are moved by a motion platform or when the sound sources themselves move. This is true even though the dynamic binaural cues elicited by these three types of motion are identical. Our data demonstrate a perceptual strategy to segregate intermittent sound sources in depth and highlight the tight interaction between self-motion and binaural processing that allows assessment of the spatial layout of complex acoustic scenes.