Impaired stationarity perception is associated with increased virtual reality sickness.

Stationarity perception refers to the ability to accurately perceive the surrounding visual environment as world-fixed during self-motion. Perception of stationarity depends on mechanisms that evaluate the congruence between retinal/oculomotor signals and head movement signals. In a series of psychophysical experiments, we systematically varied the congruence between retinal/oculomotor and head movement signals to find the range of visual gains that is compatible with perception of a stationary environment. On each trial, human subjects wearing a head-mounted display execute a yaw head movement and report whether the visual gain was perceived to be too slow or fast. A psychometric fit to the data across trials reveals the visual gain most compatible with stationarity (a measure of accuracy) and the sensitivity to visual gain manipulation (a measure of precision). Across experiments, we varied 1) the spatial frequency of the visual stimulus, 2) the retinal location of the visual stimulus (central vs. peripheral), and 3) fixation behavior (scene-fixed vs. head-fixed). Stationarity perception is most precise and accurate during scene-fixed fixation. Effects of spatial frequency and retinal stimulus location become evident during head-fixed fixation, when retinal image motion is increased. Virtual Reality sickness assessed using the Simulator Sickness Questionnaire covaries with perceptual performance. Decreased accuracy is associated with an increase in the nausea subscore, while decreased precision is associated with an increase in the oculomotor and disorientation subscores.

Psychophysical evidence for auditory motion parallax.

Distance is important: From an ecological perspective, knowledge about the distance to either prey or predator is vital. However, the distance of an unknown sound source is particularly difficult to assess, especially in anechoic environments. In vision, changes in perspective resulting from observer motion produce a reliable, consistent, and unambiguous impression of depth known as motion parallax. Here we demonstrate with formal psychophysics that humans can exploit auditory motion parallax, i.e., the change in the dynamic binaural cues elicited by self-motion, to assess the relative depths of two sound sources. Our data show that sensitivity to relative depth is best when subjects move actively; performance deteriorates when subjects are moved by a motion platform or when the sound sources themselves move. This is true even though the dynamic binaural cues elicited by these three types of motion are identical. Our data demonstrate a perceptual strategy to segregate intermittent sound sources in depth and highlight the tight interaction between self-motion and binaural processing that allows assessment of the spatial layout of complex acoustic scenes.

Insufficient compensation for self-motion during perception of object speed: The vestibular Aubert-Fleischl phenomenon.

To estimate object speed with respect to the self, retinal signals must be summed with extraretinal signals that encode the speed of eye and head movement. Prior work has shown that differences in perceptual estimates of object speed based on retinal and oculomotor signals lead to biased percepts such as the Aubert-Fleischl phenomenon (AF), in which moving targets appear slower when pursued. During whole-body movement, additional extraretinal signals, such as those from the vestibular system, may be used to transform object speed estimates from a head-centered to a world-centered reference frame. Here we demonstrate that whole-body pursuit in the form of passive yaw rotation, which stimulates the semicircular canals of the vestibular system, leads to a slowing of perceived object speed similar to the classic oculomotor AF. We find that the magnitude of the vestibular and oculomotor AF is comparable across a range of speeds, despite the different types of input signal involved. This covariation might hint at a common modality-independent mechanism underlying the AF in both cases.

Dependence of auditory spatial updating on vestibular, proprioceptive, and efference copy signals.

Humans localize sounds by comparing inputs across the two ears, resulting in a head-centered representation of sound-source position. When the head moves, information about head movement must be combined with the head-centered estimate to correctly update the world-centered sound-source position. Spatial updating has been extensively studied in the visual system, but less is known about how head movement signals interact with binaural information during auditory spatial updating. In the current experiments, listeners compared the world-centered azimuthal position of two sound sources presented before and after a head rotation that depended on condition. In the active condition, subjects rotated their head by ∼35° to the left or right, following a pretrained trajectory. In the passive condition, subjects were rotated along the same trajectory in a rotating chair. In the cancellation condition, subjects rotated their head as in the active condition, but the chair was counter-rotated on the basis of head-tracking data such that the head effectively remained fixed in space while the body rotated beneath it. Subjects updated most accurately in the passive condition but erred in the active and cancellation conditions. Performance is interpreted as reflecting the accuracy of perceived head rotation across conditions, which is modeled as a linear combination of proprioceptive/efference copy signals and vestibular signals. Resulting weights suggest that auditory updating is dominated by vestibular signals but with significant contributions from proprioception/efference copy. Overall, results shed light on the interplay of sensory and motor signals that determine the accuracy of auditory spatial updating.

Multisensory self-motion compensation during object trajectory judgments.

Judging object trajectory during self-motion is a fundamental ability for mobile organisms interacting with their environment. This fundamental ability requires the nervous system to compensate for the visual consequences of self-motion in order to make accurate judgments, but the mechanisms of this compensation are poorly understood. We comprehensively examined both the accuracy and precision of observers' ability to judge object trajectory in the world when self-motion was defined by vestibular, visual, or combined visual-vestibular cues. Without decision feedback, subjects demonstrated no compensation for self-motion that was defined solely by vestibular cues, partial compensation (47%) for visually defined self-motion, and significantly greater compensation (58%) during combined visual-vestibular self-motion. With decision feedback, subjects learned to accurately judge object trajectory in the world, and this generalized to novel self-motion speeds. Across conditions, greater compensation for self-motion was associated with decreased precision of object trajectory judgments, indicating that self-motion compensation comes at the cost of reduced discriminability. Our findings suggest that the brain can flexibly represent object trajectory relative to either the observer or the world, but a world-centered representation comes at the cost of decreased precision due to the inclusion of noisy self-motion signals.

The effect of supine body position on human heading perception.

The use of virtual environments in functional imaging experiments is a promising method to investigate and understand the neural basis of human navigation and self-motion perception. However, the supine position in the fMRI scanner is unnatural for everyday motion. In particular, the head-horizontal self-motion plane is parallel rather than perpendicular to gravity. Earlier studies have shown that perception of heading from visual self-motion stimuli, such as optic flow, can be modified due to visuo-vestibular interactions. With this study, we aimed to identify the effects of the supine body position on visual heading estimation, which is a basic component of human navigation. Visual and vestibular heading judgments were measured separately in 11 healthy subjects in upright and supine body positions. We measured two planes of self-motion, the transverse and the coronal plane, and found that, although vestibular heading perception was strongly modified in a supine position, visual performance, in particular for the preferred head-horizontal (i.e., transverse) plane, did not change. This provides behavioral evidence in humans that direction estimation from self-motion consistent optic flow is not modified by supine body orientation, demonstrating that visual heading estimation is one component of human navigation that is not influenced by the supine body position required for functional brain imaging experiments.

Comparison of linear motion perception thresholds in vestibular migraine and Menière's disease.

Linear motion perceptual thresholds (PTs) were compared between patients with Menière's disease (MD) and vestibular migraine (VM). Twenty patients with VM, 27 patients with MD and 34 healthy controls (HC) were examined. PTs for linear motion along the inter-aural (IA), naso-occipital axes (NO), and head-vertical (HV) axis were measured using a multi-axis motion platform. Ocular and cervical vestibular evoked myogenic potentials (o/c VEMP) were performed and the dizziness handicap inventory (DHI) administered. In order to discriminate between VM and MD, we also evaluated the diagnostic accuracy of applied methods. PTs depended significantly on the group tested (VM, MD and HC), as revealed by ANCOVA with group as the factor and age as the covariate. This was true for all motion axes (IA, HV and NO). Thresholds were highest for MD patients, significantly higher than for all other groups for all motion axes, except for the IA axis when compared with HC group suggesting decreased otolith sensitivity in MD patients. VM patients had thresholds that were not different from those of HC, but were significantly lower than those of the MD group for all motion axes. The cVEMP p13 latencies differed significantly across groups being lowest in VM. There was a statistically significant association between HV and NO thresholds and cVEMP PP amplitudes. Diagnostic accuracy was highest for the IA axis, followed by cVEMP PP amplitudes, NO and HV axes. To conclude, patients with MD had significantly higher linear motion perception thresholds compared to patients with VM and controls. Except for reduced cVEMP latency, there were no differences in c/oVEMP between MD, VM and controls.

Human sensitivity to vertical self-motion.

Perceiving vertical self-motion is crucial for maintaining balance as well as for controlling an aircraft. Whereas heave absolute thresholds have been exhaustively studied, little work has been done in investigating how vertical sensitivity depends on motion intensity (i.e., differential thresholds). Here we measure human sensitivity for 1-Hz sinusoidal accelerations for 10 participants in darkness. Absolute and differential thresholds are measured for upward and downward translations independently at 5 different peak amplitudes ranging from 0 to 2 m/s(2). Overall vertical differential thresholds are higher than horizontal differential thresholds found in the literature. Psychometric functions are fit in linear and logarithmic space, with goodness of fit being similar in both cases. Differential thresholds are higher for upward as compared to downward motion and increase with stimulus intensity following a trend best described by two power laws. The power laws' exponents of 0.60 and 0.42 for upward and downward motion, respectively, deviate from Weber's Law in that thresholds increase less than expected at high stimulus intensity. We speculate that increased sensitivity at high accelerations and greater sensitivity to downward than upward self-motion may reflect adaptations to avoid falling.

Vestibular perception and the vestibulo-ocular reflex in young and older adults.

OBJECTIVES: Quantification of the perceptual thresholds to vestibular stimuli may offer valuable complementary information to that provided by measures of the vestibulo-ocular reflex (VOR). Perceptual thresholds could be particularly important in evaluating some subjects, such as the elderly, who might have a greater potential of central as well as peripheral vestibular dysfunction. The authors hypothesized that perceptual detection and discrimination thresholds would worsen with aging, and that there would be a poor relation between thresholds and traditional measures of the angular VOR represented by gain and phase on rotational chair testing. DESIGN: The authors compared the detection and discrimination thresholds of 19 younger and 16 older adults in response to earth-vertical, 0.5 Hz rotations. Perceptual results of the older subjects were then compared with the gain and phase of their VOR in response to earth-vertical rotations over the frequency range from 0.025 to 0.5 Hz. RESULTS: Detection thresholds were found to be 0.69 ± 0.29 degree/sec (mean ± standard deviation) for the younger participants and 0.81 ± 0.42 degree/sec for older participants. Discrimination thresholds in younger and older adults were 4.83 ± 1.80 degree/sec and 4.33 ± 1.57 degree/sec, respectively. There was no difference in either measure between age groups. Perceptual thresholds were independent of the gain and phase of the VOR. CONCLUSIONS: These results indicate that there is no inevitable loss of vestibular perception with aging. Elevated thresholds among the elderly are therefore suggestive of pathology rather than normal consequences of aging. Furthermore, perceptual thresholds offer additional insight, beyond that supplied by the VOR alone, into vestibular function.

Natural statistics of human head orientation constrain models of vestibular processing.

Head orientation relative to gravity determines how gravity-dependent environmental structure is sampled by the visual system, as well as how gravity itself is sampled by the vestibular system. Therefore, both visual and vestibular sensory processing should be shaped by the statistics of head orientation relative to gravity. Here we report the statistics of human head orientation during unconstrained natural activities in humans for the first time, and we explore implications for models of vestibular processing. We find that the distribution of head pitch is more variable than head roll and that the head pitch distribution is asymmetrical with an over-representation of downward head pitch, consistent with ground-looking behavior. We further show that pitch and roll distributions can be used as empirical priors in a Bayesian framework to explain previously measured biases in perception of both roll and pitch. We also analyze the dynamics of human head orientation to better understand how gravitational and inertial acceleration are processed by the vestibular system. Gravitational acceleration dominates at low frequencies and inertial acceleration dominates at higher frequencies. The change in relative power of gravitational and inertial components as a function of frequency places empirical constraints on dynamic models of vestibular processing. We conclude with a discussion of methodological considerations and scientific and applied domains that will benefit from continued measurement and analysis of natural head movements moving forward.

Natural statistics of human head orientation constrain models of vestibular processing.

Head orientation relative to gravity determines how gravity-dependent environmental structure is sampled by the visual system, as well as how gravity itself is sampled by the vestibular system. Therefore, both visual and vestibular sensory processing should be shaped by the statistics of head orientation relative to gravity. Here we report the statistics of human head orientation during unconstrained natural activities in humans for the first time, and we explore implications for models of vestibular processing. We find that the distribution of head pitch is more variable than head roll and that the head pitch distribution is asymmetrical with an over-representation of downward head pitch, consistent with ground-looking behavior. We further suggest that pitch and roll distributions can be used as empirical priors in a Bayesian framework to explain previously measured biases in perception of both roll and pitch. Gravitational and inertial acceleration stimulate the otoliths in an equivalent manner, so we also analyze the dynamics of human head orientation to better understand how knowledge of these dynamics can constrain solutions to the problem of gravitoinertial ambiguity. Gravitational acceleration dominates at low frequencies and inertial acceleration dominates at higher frequencies. The change in relative power of gravitational and inertial components as a function of frequency places empirical constraints on dynamic models of vestibular processing, including both frequency segregation and probabilistic internal model accounts. We conclude with a discussion of methodological considerations and scientific and applied domains that will benefit from continued measurement and analysis of natural head movements moving forward.

Motor Signals Mediate Stationarity Perception.

Head movement relative to the stationary environment gives rise to congruent vestibular and visual optic-flow signals. The resulting perception of a stationary visual environment, referred to herein as stationarity perception, depends on mechanisms that compare visual and vestibular signals to evaluate their congruence. Here we investigate the functioning of these mechanisms and their dependence on fixation behavior as well as on the active versus passive nature of the head movement. Stationarity perception was measured by modifying the gain on visual motion relative to head movement on individual trials and asking subjects to report whether the gain was too low or too high. Fitting a psychometric function to the data yields two key parameters of performance. The mean is a measure of accuracy, and the standard deviation is a measure of precision. Experiments were conducted using a head-mounted display with fixation behavior monitored by an embedded eye tracker. During active conditions, subjects rotated their heads in yaw ∼15 deg/s over ∼1 s. Each subject's movements were recorded and played back via rotating chair during the passive condition. During head-fixed and scene-fixed fixation the fixation target moved with the head or scene, respectively. Both precision and accuracy were better during active than passive head movement, likely due to increased precision on the head movement estimate arising from motor prediction and neck proprioception. Performance was also better during scene-fixed than head-fixed fixation, perhaps due to decreased velocity of retinal image motion and increased precision on the retinal image motion estimate. These results reveal how the nature of head and eye movements mediate encoding, processing, and comparison of relevant sensory and motor signals.

Vestibular heading discrimination and sensitivity to linear acceleration in head and world coordinates.

Effective navigation and locomotion depend critically on an observer's ability to judge direction of linear self-motion, i.e., heading. The vestibular cue to heading is the direction of inertial acceleration that accompanies transient linear movements. This cue is transduced by the otolith organs. The otoliths also respond to gravitational acceleration, so vestibular heading discrimination could depend on (1) the direction of movement in head coordinates (i.e., relative to the otoliths), (2) the direction of movement in world coordinates (i.e., relative to gravity), or (3) body orientation (i.e., the direction of gravity relative to the otoliths). To quantify these effects, we measured vestibular and visual discrimination of heading along azimuth and elevation dimensions with observers oriented both upright and side-down relative to gravity. We compared vestibular heading thresholds with corresponding measurements of sensitivity to linear motion along lateral and vertical axes of the head (coarse direction discrimination and amplitude discrimination). Neither heading nor coarse direction thresholds depended on movement direction in world coordinates, demonstrating that the nervous system compensates for gravity. Instead, they depended similarly on movement direction in head coordinates (better performance in the horizontal plane) and on body orientation (better performance in the upright orientation). Heading thresholds were correlated with, but significantly larger than, predictions based on sensitivity in the coarse discrimination task. Simulations of a neuron/anti-neuron pair with idealized cosine-tuning properties show that heading thresholds larger than those predicted from coarse direction discrimination could be accounted for by an amplitude-response nonlinearity in the neural representation of inertial motion.

Estimating distance during self-motion: a role for visual-vestibular interactions.

A fundamental challenge for the visual system is to extract the 3D spatial structure of the environment. When an observer translates without moving the eyes, the retinal speed of a stationary object is related to its distance by a scale factor that depends on the velocity of the observer's self-motion. Here, we aim to test whether the brain uses vestibular cues to self-motion to estimate distance to stationary surfaces in the environment. This relationship was systematically probed using a two-alternative forced-choice task in which distance perceived from monocular image motion during passive body translation was compared to distance perceived from binocular disparity while subjects were stationary. We show that perceived distance from motion depended on both observer velocity and retinal speed. For a given head speed, slower retinal speeds led to the perception of farther distances. Likewise, for a given retinal speed, slower head speeds led to the perception of nearer distances. However, these relationships were weak in some subjects and absent in others, and distance estimated from self-motion and retinal image motion was substantially compressed relative to distance estimated from binocular disparity. Overall, our findings suggest that the combination of retinal image motion and vestibular signals related to head velocity can provide a rudimentary capacity for distance estimation.

Evaluation of the Intel RealSense T265 for tracking natural human head motion.

Accurate and robust tracking of natural human head motion in natural environments is important for a number of applications including virtual and augmented reality, clinical diagnostics, as well as basic scientific research. IMU provide a versatile solution for recording inertial data including linear acceleration and angular velocity, but reconstructing head position is difficult or impossible. This problem can be solved by incorporating visual data using a technique known as visual-inertial simultaneous localization and mapping (VI-SLAM). A recently released commercial solution, the Intel RealSense T265, uses a proprietary VI-SLAM algorithm to estimate linear and angular position and velocity, but the performance of this device for tracking of natural human head motion in natural environments has not yet been comprehensively evaluated against gold-standard methods. In this study, we used a wide range of metrics to evaluate the performance of the T265 with different walking speeds in different environments, both indoor and outdoor, against two gold-standard methods, an optical tracking system and a so-called perambulator. Overall, we find that performance of the T265 relative to these gold-standard methods is most accurate for slow to normal walking speeds in small- to medium-sized environments. The suitability of this device for future scientific studies depends on the application; data presented here can be useful in making that determination.

Canal-otolith interactions and detection thresholds of linear and angular components during curved-path self-motion.

Gravitational signals arising from the otolith organs and vertical plane rotational signals arising from the semicircular canals interact extensively for accurate estimation of tilt and inertial acceleration. Here we used a classical signal detection paradigm to examine perceptual interactions between otolith and horizontal semicircular canal signals during simultaneous rotation and translation on a curved path. In a rotation detection experiment, blindfolded subjects were asked to detect the presence of angular motion in blocks where half of the trials were pure nasooccipital translation and half were simultaneous translation and yaw rotation (curved-path motion). In separate, translation detection experiments, subjects were also asked to detect either the presence or the absence of nasooccipital linear motion in blocks, in which half of the trials were pure yaw rotation and half were curved path. Rotation thresholds increased slightly, but not significantly, with concurrent linear velocity magnitude. Yaw rotation detection threshold, averaged across all conditions, was 1.45 +/- 0.81 degrees/s (3.49 +/- 1.95 degrees/s(2)). Translation thresholds, on the other hand, increased significantly with increasing magnitude of concurrent angular velocity. Absolute nasooccipital translation detection threshold, averaged across all conditions, was 2.93 +/- 2.10 cm/s (7.07 +/- 5.05 cm/s(2)). These findings suggest that conscious perception might not have independent access to separate estimates of linear and angular movement parameters during curved-path motion. Estimates of linear (and perhaps angular) components might instead rely on integrated information from canals and otoliths. Such interaction may underlie previously reported perceptual errors during curved-path motion and may originate from mechanisms that are specialized for tilt-translation processing during vertical plane rotation.

Positional head-eye tracking outside the lab: an open-source solution.

Simultaneous head and eye tracking has traditionally been confined to a laboratory setting and real-world motion tracking limited to measuring linear acceleration and angular velocity. Recently available mobile devices such as the Pupil Core eye tracker and the Intel RealSense T265 motion tracker promise to deliver accurate measurements outside the lab. Here, the researchers propose a hard- and software framework that combines both devices into a robust, usable, low-cost head and eye tracking system. The developed software is open source and the required hardware modifications can be 3D printed. The researchers demonstrate the system's ability to measure head and eye movements in two tasks: an eyes-fixed head rotation task eliciting the vestibulo-ocular reflex inside the laboratory, and a natural locomotion task where a subject walks around a building outside of the laboratory. The resultant head and eye movements are discussed, as well as future implementations of this system.

Towards dynamic modeling of visual-vestibular conflict detection.

Visual-vestibular mismatch is a common occurrence, with causes ranging from vehicular travel, to vestibular dysfunction, to virtual reality displays. Behavioral and physiological consequences of this mismatch include adaptation of reflexive eye movements, oscillopsia, vertigo, and nausea. Despite this significance, we still do not have a good understanding of how the nervous system evaluates visual-vestibular conflict. Here we review research that quantifies perceptual sensitivity to visual-vestibular conflict and factors that mediate this sensitivity, such as noise on visual and vestibular sensory estimates. We emphasize that dynamic modeling methods are necessary to investigate how the nervous system monitors conflict between time-varying visual and vestibular signals, and we present a simple example of a drift-diffusion model for visual-vestibular conflict detection. The model makes predictions for detection of conflict arising from changes in both visual gain and latency. We conclude with discussion of topics for future research.

Ecological Momentary Assessment of Head Motion: Toward Normative Data of Head Stabilization.

Head stabilization is fundamental for balance during locomotion but can be impaired in elderly or diseased populations. Previous studies have identified several parameters of head stability with possible diagnostic value in a laboratory setting. Recently, the ecological validity of measures obtained in such controlled contexts has been called into question. The aim of this study was to investigate the ecological validity of previously described parameters of head stabilization in a real-world setting. Ten healthy subjects participated in the study. Head and trunk movements of each subject were recorded with inertial measurement units (IMUs) for a period of at least 10 h. Periods of locomotion were extracted from the measurements and predominant frequencies, root mean squares (RMSs) and bout lengths were estimated. As parameters of head stabilization, attenuation coefficients (ACs), harmonic ratios (HRs), coherences, and phase differences were computed. Predominant frequencies were distributed tightly around 2 Hz and ACs, HRs, and coherences exhibited the highest values in this frequency range. All head stability parameters exhibited characteristics consistent with previous reports, although higher variances were observed. These results suggest that head stabilization is tuned to the 2 Hz fundamental frequency of locomotion and that previously described measures of head stability could generalize to a real-world setting. This is the first study to address the ecological validity of these measures, highlighting the potential use of head stability parameters as diagnostic tools or outcome measures for clinical trials. The low cost and ease of use of the IMU technology used in this study could additionally be of benefit for a clinical application.

Vestibulo-Ocular Responses and Dynamic Visual Acuity During Horizontal Rotation and Translation.

Dynamic visual acuity (DVA) provides an overall functional measure of visual stabilization performance that depends on the vestibulo-ocular reflex (VOR), but also on other processes, including catch-up saccades and likely visual motion processing. Capturing the efficiency of gaze stabilization against head movement as a whole, it is potentially valuable in the clinical context where assessment of overall patient performance provides an important indication of factors impacting patient participation and quality of life. DVA during head rotation (rDVA) has been assessed previously, but to our knowledge, DVA during horizontal translation (tDVA) has not been measured. tDVA can provide a valuable measure of how otolith, rather than canal, function impacts visual acuity. In addition, comparison of DVA during rotation and translation can shed light on whether common factors are limiting DVA performance in both cases. We therefore measured and compared DVA during both passive head rotations (head impulse test) and translations in the same set of healthy subjects (n = 7). In addition to DVA, we computed average VOR gain and retinal slip within and across subjects. We observed that during translation, VOR gain was reduced (VOR during rotation, mean ± SD: position gain = 1.05 ± 0.04, velocity gain = 0.97 ± 0.07; VOR during translation, mean ± SD: position gain = 0.21 ± 0.08, velocity gain = 0.51 ± 0.16), retinal slip was increased, and tDVA was worse than during rotation (average rDVA = 0.32 ± 0.15 logMAR; average tDVA = 0.56 ± 0.09 logMAR, p = 0.02). This suggests that reduced VOR gain leads to worse tDVA, as expected. We conclude with speculation about non-oculomotor factors that could vary across individuals and affect performance similarly during both rotation and translation.