Mechanically Induced Motor Tremors Disrupt the Perception of Time.

Contemporary research has begun to show a strong relationship between movements and the perception of time. More specifically, concurrent movements serve to both bias and enhance time estimates. To explain these effects, we recently proposed a mechanism by which movements provide a secondary channel for estimating duration that is combined optimally with sensory estimates. However, a critical test of this framework is that by introducing "noise" into movements, sensory estimates of time should similarly become noisier. To accomplish this, we had human participants move a robotic arm while estimating intervals of time in either auditory or visual modalities (n = 24, ea.). Crucially, we introduced an artificial "tremor" in the arm while subjects were moving, that varied across three levels of amplitude (1-3 N) or frequency (4-12 Hz). The results of both experiments revealed that increasing the frequency of the tremor led to noisier estimates of duration. Further, the effect of noise varied with the base precision of the interval, such that a naturally less precise timing (i.e., visual) was more influenced by the tremor than a naturally more precise modality (i.e., auditory). To explain these findings, we fit the data with a recently developed drift-diffusion model of perceptual decision-making, in which the momentary, within-trial variance was allowed to vary across conditions. Here, we found that the model could recapitulate the observed findings, further supporting the theory that movements influence perception directly. Overall, our findings support the proposed framework, and demonstrate the utility of inducing motor noise via artificial tremors.

Perceptual error based on Bayesian cue combination drives implicit motor adaptation.

The sensorimotor system can recalibrate itself without our conscious awareness, a type of procedural learning whose computational mechanism remains undefined. Recent findings on implicit motor adaptation, such as over-learning from small perturbations and fast saturation for increasing perturbation size, challenge existing theories based on sensory errors. We argue that perceptual error, arising from the optimal combination of movement-related cues, is the primary driver of implicit adaptation. Central to our theory is the increasing sensory uncertainty of visual cues with increasing perturbations, which was validated through perceptual psychophysics (Experiment 1). Our theory predicts the learning dynamics of implicit adaptation across a spectrum of perturbation sizes on a trial-by-trial basis (Experiment 2). It explains proprioception changes and their relation to visual perturbation (Experiment 3). By modulating visual uncertainty in perturbation, we induced unique adaptation responses in line with our model predictions (Experiment 4). Overall, our perceptual error framework outperforms existing models based on sensory errors, suggesting that perceptual error in locating one's effector, supported by Bayesian cue integration, underpins the sensorimotor system's implicit adaptation.

How small changes to one eye's retinal image can transform the perceived shape of a very familiar object.

Vision can provide useful cues about the geometric properties of an object, like its size, distance, pose, and shape. But how the brain merges these properties into a complete sensory representation of a three-dimensional object is poorly understood. To address this gap, we investigated a visual illusion in which humans misperceive the shape of an object due to a small change in one eye's retinal image. We first show that this illusion affects percepts of a highly familiar object under completely natural viewing conditions. Specifically, people perceived their own rectangular mobile phone to have a trapezoidal shape. We then investigate the perceptual underpinnings of this illusion by asking people to report both the perceived shape and pose of controlled stimuli. Our results suggest that the shape illusion results from distorted cues to object pose. In addition to yielding insights into object perception, this work informs our understanding of how the brain combines information from multiple visual cues in natural settings. The shape illusion can occur when people wear everyday prescription spectacles; thus, these findings also provide insight into the cue combination challenges that some spectacle wearers experience on a regular basis.

The Suite for the Assessment of Low-Level cues on Orientation (SALLO): The psychophysics of spatial orientation in virtual reality.

Spatial orientation is a complex ability that emerges from the interaction of several systems in a way that is still unclear. One of the reasons limiting the research on the topic is the lack of methodologies aimed at studying multimodal psychophysics in an ecological manner and with affordable settings. Virtual reality can provide a workaround to this impasse by using virtual stimuli rather than real ones. However, the available virtual reality development platforms are not meant for psychophysical testing; therefore, using them as such can be very difficult for newcomers, especially the ones new to coding. For this reason, we developed SALLO, the Suite for the Assessment of Low-Level cues on Orientation, which is a suite of utilities that simplifies assessing the psychophysics of multimodal spatial orientation in virtual reality. The tools in it cover all the fundamental steps to design a psychophysical experiment. Plus, dedicated tracks guide the users in extending the suite components to simplify developing new experiments. An experimental use-case used SALLO and virtual reality to show that the head posture affects both the egocentric and the allocentric mental representations of spatial orientation. Such a use-case demonstrated how SALLO and virtual reality can be used to accelerate hypothesis testing concerning the psychophysics of spatial orientation and, more broadly, how the community of researchers in the field may benefit from such a tool to carry out their investigations.

Correctly establishing evidence for cue combination via gains in sensory precision: Why the choice of comparator matters.

Studying how sensory signals from different sources (sensory cues) are integrated within or across multiple senses allows us to better understand the perceptual computations that lie at the foundation of adaptive behaviour. As such, determining the presence of precision gains - the classic hallmark of cue combination - is important for characterising perceptual systems, their development and functioning in clinical conditions. However, empirically measuring precision gains to distinguish cue combination from alternative perceptual strategies requires careful methodological considerations. Here, we note that the majority of existing studies that tested for cue combination either omitted this important contrast, or used an analysis approach that, unknowingly, strongly inflated false positives. Using simulations, we demonstrate that this approach enhances the chances of finding significant cue combination effects in up to 100% of cases, even when cues are not combined. We establish how this error arises when the wrong cue comparator is chosen and recommend an alternative analysis that is easy to implement but has only been adopted by relatively few studies. By comparing combined-cue perceptual precision with the best single-cue precision, determined for each observer individually rather than at the group level, researchers can enhance the credibility of their reported effects. We also note that testing for deviations from optimal predictions alone is not sufficient to ascertain whether cues are combined. Taken together, to correctly test for perceptual precision gains, we advocate for a careful comparator selection and task design to ensure that cue combination is tested with maximum power, while reducing the inflation of false positives.

The role of consciously timed movements in shaping and improving auditory timing.

Our subjective sense of time is intertwined with a plethora of perceptual, cognitive and motor functions, and likewise, the brain is equipped to expertly filter, weight and combine these signals for seamless interactions with a dynamic world. Until relatively recently, the literature on time perception has excluded the influence of simultaneous motor activity, yet it has been found that motor circuits in the brain are at the core of most timing functions. Several studies have now identified that concurrent movements exert robust effects on perceptual timing estimates, but critically have not assessed how humans consciously judge the duration of their own movements. This creates a gap in our understanding of the mechanisms driving movement-related effects on sensory timing. We sought to address this gap by administering a sensorimotor timing task in which we explicitly compared the timing of isolated auditory tones and arm movements, or both simultaneously. We contextualized our findings within a Bayesian cue combination framework, in which separate sources of temporal information are weighted by their reliability and integrated into a unitary time estimate that is more precise than either unisensory estimate. Our results revealed differences in accuracy between auditory, movement and combined trials, and (crucially) that combined trials were the most accurately timed. Under the Bayesian framework, we found that participants' combined estimates were more precise than isolated estimates, yet were sub-optimal when compared with the model's prediction, on average. These findings elucidate previously unknown qualities of conscious motor timing and propose computational mechanisms that can describe how movements combine with perceptual signals to create unified, multimodal experiences of time.

When is a disparity not a disparity? Toward an old theory of three-dimensional vision.

The aims of this paper are twofold: first, to discuss and analyze the concept of binocular disparity and second, to contrast the traditional "air theory" of three-dimensional vision with the much older "ground theory," first suggested by Ibn al-Haytham more than a thousand years ago. The origins of an "air theory" of perception can be traced back to Descartes and subsequently to the philosopher George Berkeley, who claimed that distance "could not be seen" because points lying along the same line of sight (in an empty space) would all project to the same location on the retina. However, Descartes was also aware that the angle of convergence of the two eyes could solve the problem of the "missing" information for the monocular observer and, since then, most visual scientists have assumed that eye vergence plays an important role both in judging absolute distance and for scaling retinal size and binocular disparities. In contrast, al-Haytham's and Gibson's "ground theories," which are based on the geometry of the textured ground plane surface that has surrounded us throughout evolution and during our lifetimes, are not just more ecologically based but they also obviate the need for disparity scaling.

The role of viscosity in flavor preference: plasticity and interactions with taste.

The brain combines gustatory, olfactory, and somatosensory information to create our perception of flavor. Within the somatosensory modality, texture attributes such as viscosity appear to play an important role in flavor preference. However, research into the role of texture in flavor perception is relatively sparse, and the contribution of texture cues to hedonic evaluation of flavor remains largely unknown. Here, we used a rat model to investigate whether viscosity preferences can be manipulated through association with nutrient value, and how viscosity interacts with taste to inform preferences for taste + viscosity mixtures. To address these questions, we measured preferences for moderately viscous solutions prepared with xanthan gum using 2-bottle consumption tests. By experimentally exposing animals to viscous solutions with and without nutrient value, we demonstrate that viscosity preferences are susceptible to appetitive conditioning. By independently varying viscosity and taste content of solutions, we further show that taste and viscosity cues both contribute to preferences for taste + viscosity mixtures. How these 2 modalities are combined depended on relative palatability, with mixture preferences falling in between component preferences, suggesting that hedonic aspects of taste and texture inputs are centrally integrated. Together, these findings provide new insight into how texture aspects of flavor inform hedonic perception and impact food choice behavior.

Is source elevation an auditory distance cue? A preliminary study.

The aim of this work was to evaluate whether the angular elevation of a sound source could generate auditory cues which improve the auditory distance perception in a similar way to that previously reported by visual modality. For this purpose, we compared ADP curves obtained with sources located both at the listeners' ears and at ground level. Our hypothesis was that the participants can interpret the relation between elevation and distance of ground-level sources (which are linked geometrically) so we expected them to perceive their distances more accurately than those at ear level. However, the responses obtained with sources located at ground level were almost identical to those obtained at the height of the listeners' ears, showing that, under the conditions of our experiment, auditory elevation cues do not influence auditory distance perception.

Familiarity with an Object's Size Influences the Perceived Size of Its Image.

It is known that judgments about objects' distances are influenced by familiar size: a soccer ball looks farther away than a tennis ball if their images are equally large on the retina. We here investigate whether familiar size also influences judgments about the size of images of objects that are presented side-by-side on a computer screen. Sixty-three participants indicated which of two images appeared larger on the screen in a 2-alternative forced-choice discrimination task. The objects were either two different types of balls, two different types of coins, or a ball and a grey disk. We found that the type of ball biased the comparison between their image sizes: the size of the image of the soccer ball was over-estimated by about 5% (assimilation). The bias in the comparison between the two balls was equal to the sum of the biases in the comparisons with the grey disk. The bias for the coins was smaller and in the opposite direction (contrast). The average precision of the size comparison was 3.5%, irrespective of the type of object. We conclude that knowing a depicted object's real size can influence the perceived size of its image, but the perceived size is not always attracted towards the familiar size.

Bayesian decision theory and navigation.

Spatial navigation is a complex cognitive activity that depends on perception, action, memory, reasoning, and problem-solving. Effective navigation depends on the ability to combine information from multiple spatial cues to estimate one's position and the locations of goals. Spatial cues include landmarks, and other visible features of the environment, and body-based cues generated by self-motion (vestibular, proprioceptive, and efferent information). A number of projects have investigated the extent to which visual cues and body-based cues are combined optimally according to statistical principles. Possible limitations of these investigations are that they have not accounted for navigators' prior experiences with or assumptions about the task environment and have not tested complete decision models. We examine cue combination in spatial navigation from a Bayesian perspective and present the fundamental principles of Bayesian decision theory. We show that a complete Bayesian decision model with an explicit loss function can explain a discrepancy between optimal cue weights and empirical cues weights observed by (Chen et al. Cognitive Psychology, 95, 105-144, 2017) and that the use of informative priors to represent cue bias can explain the incongruity between heading variability and heading direction observed by (Zhao and Warren 2015b, Psychological Science, 26[6], 915-924). We also discuss (Petzschner and Glasauer's , Journal of Neuroscience, 31(47), 17220-17229, 2011) use of priors to explain biases in estimates of linear displacements during visual path integration. We conclude that Bayesian decision theory offers a productive theoretical framework for investigating human spatial navigation and believe that it will lead to a deeper understanding of navigational behaviors.

Binocular Information Improves the Reliability and Consistency of Pictorial Relief.

Binocular disparity is an important cue to three-dimensional shape. We assessed the contribution of this cue to the reliability and consistency of depth in stereoscopic photographs of natural scenes. Observers viewed photographs of cluttered scenes while adjusting a gauge figure to indicate the apparent three-dimensional orientation of the surfaces of objects. The gauge figure was positioned on the surfaces of objects at multiple points in the scene, and settings were made under monocular and binocular, stereoscopic viewing. Settings were used to create a depth relief map, indicating the apparent three-dimensional structure of the scene. We found that binocular cues increased the magnitude of apparent depth, the reliability of settings across repeated measures, and the consistency of perceived depth across participants. These results show that binocular cues make an important contribution to the precise and accurate perception of depth in natural scenes that contain multiple pictorial cues.

Explicit and implicit depth-cue integration: Evidence of systematic biases with real objects.

In previous studies using VR, we found evidence that 3D shape estimation agrees to a superadditivity rule of depth-cue combination, by which adding depth cues leads to greater perceived depth and, in principle, to depth overestimation. Superadditivity can be quantitatively accounted for by a normative theory of cue integration, via adapting a model termed Intrinsic Constraint (IC). As for its qualitative nature, it remains unclear whether superadditivity represents the genuine readout of depth-cue integration, as predicted by IC, or alternatively a byproduct of artificial virtual displays, because they carry flatness cues that can bias depth estimates in a Bayesian fashion, or even just a way for observers to express that a scene "looks deeper" with more depth cues by explicitly inflating their depth judgments. In the present study, we addressed this question by testing whether the IC model's prediction of superadditivity generalizes to real world settings. We asked participants to judge the perceived 3D shape of cardboard prisms through a matching task. To control for the potential interference of explicit reasoning, we also asked participants to reach-to-grasp the same objects and we analyzed the in-flight grip size throughout the reaching. We designed a novel technique to carefully control binocular and monocular 3D cues independently, allowing to add or remove depth information seamlessly. Even with real objects, participants exhibited a clear superadditivity effect in both tasks. Furthermore, the magnitude of this effect was accurately predicted by the IC model. These results confirm that superadditivity is an inherent feature of depth estimation.

Bayesian causal inference in visuotactile integration in children and adults.

If cues from different sensory modalities share the same cause, their information can be integrated to improve perceptual precision. While it is well established that adults exploit sensory redundancy by integrating cues in a Bayes optimal fashion, whether children under 8 years of age combine sensory information in a similar fashion is still under debate. If children differ from adults in the way they infer causality between cues, this may explain mixed findings on the development of cue integration in earlier studies. Here we investigated the role of causal inference in the development of cue integration, by means of a visuotactile localization task. Young children (6-8 years), older children (9.5-12.5 years) and adults had to localize a tactile stimulus, which was presented to the forearm simultaneously with a visual stimulus at either the same or a different location. In all age groups, responses were systematically biased toward the position of the visual stimulus, but relatively more so when the distance between the visual and tactile stimulus was small rather than large. This pattern of results was better captured by a Bayesian causal inference model than by alternative models of forced fusion or full segregation of the two stimuli. Our results suggest that already from a young age the brain implicitly infers the probability that a tactile and a visual cue share the same cause and uses this probability as a weighting factor in visuotactile localization.

Central tendency biases must be accounted for to consistently capture Bayesian cue combination in continuous response data.

Observers in perceptual tasks are often reported to combine multiple sensory cues in a weighted average that improves precision-in some studies, approaching statistically optimal (Bayesian) weighting, but in others departing from optimality, or not benefitting from combined cues at all. To correctly conclude which combination rules observers use, it is crucial to have accurate measures of their sensory precision and cue weighting. Here, we present a new approach for accurately recovering these parameters in perceptual tasks with continuous responses. Continuous responses have many advantages, but are susceptible to a central tendency bias, where responses are biased towards the central stimulus value. We show that such biases lead to inaccuracies in estimating both precision gains and cue weightings, two key measures used to assess sensory cue combination. We introduce a method that estimates sensory precision by regressing continuous responses on targets and dividing the variance of the residuals by the squared slope of the regression line, "correcting-out" the error introduced by the central bias and increasing statistical power. We also suggest a complementary analysis that recovers the sensory cue weights. Using both simulations and empirical data, we show that the proposed methods can accurately estimate sensory precision and cue weightings in the presence of central tendency biases. We conclude that central tendency biases should be (and can easily be) accounted for to consistently capture Bayesian cue combination in continuous response data.

Echoes of L1 Syllable Structure in L2 Phoneme Recognition.

Learning to move from auditory signals to phonemic categories is a crucial component of first, second, and multilingual language acquisition. In L1 and simultaneous multilingual acquisition, learners build up phonological knowledge to structure their perception within a language. For sequential multilinguals, this knowledge may support or interfere with acquiring language-specific representations for a new phonemic categorization system. Syllable structure is a part of this phonological knowledge, and language-specific syllabification preferences influence language acquisition, including early word segmentation. As a result, we expect to see language-specific syllable structure influencing speech perception as well. Initial evidence of an effect appears in Ali et al. (2011), who argued that cross-linguistic differences in McGurk fusion within a syllable reflected listeners' language-specific syllabification preferences. Building on a framework from Cho and McQueen (2006), we argue that this could reflect the Phonological-Superiority Hypothesis (differences in L1 syllabification preferences make some syllabic positions harder to classify than others) or the Phonetic-Superiority Hypothesis (the acoustic qualities of speech sounds in some positions make it difficult to perceive unfamiliar sounds). However, their design does not distinguish between these two hypotheses. The current research study extends the work of Ali et al. (2011) by testing Japanese, and adding audio-only and congruent audio-visual stimuli to test the effects of syllabification preferences beyond just McGurk fusion. Eighteen native English speakers and 18 native Japanese speakers were asked to transcribe nonsense words in an artificial language. English allows stop consonants in syllable codas while Japanese heavily restricts them, but both groups showed similar patterns of McGurk fusion in stop codas. This is inconsistent with the Phonological-Superiority Hypothesis. However, when visual information was added, the phonetic influences on transcription accuracy largely disappeared. This is inconsistent with the Phonetic-Superiority Hypothesis. We argue from these results that neither acoustic informativity nor interference of a listener's phonological knowledge is superior, and sketch a cognitively inspired rational cue integration framework as a third hypothesis to explain how L1 phonological knowledge affects L2 perception.

J. J. Gibson's "Ground Theory of Space Perception".

J. J. Gibson's ground theory of space perception is contrasted with Descartes' theory, which reduces all of space perception to the perception of distance and angular direction, relative to an abstract viewpoint. Instead, Gibson posits an embodied perceiver, grounded by gravity, in a stable layout of realistically textured, extended surfaces and more delimited objects supported by these surfaces. Gibson's concept of optical contact ties together this spatial layout, locating each surface relative to the others and specifying the position of each object by its location relative to its surface of support. His concept of surface texture-augmented by perspective structures such as the horizon-specifies the scale of objects and extents within this layout. And his concept of geographical slant provides surfaces with environment-centered orientations that remain stable as the perceiver moves around. Contact-specified locations on extended environmental surfaces may be the unattended primitives of the visual world, rather than egocentric or allocentric distances. The perception of such distances may best be understood using Gibson's concept of affordances. Distances may be perceived only as needed, bound through affordances to the particular actions that require them.

Cue vetoing in depth estimation: Physical and virtual stimuli.

Motion parallax and binocular disparity contribute to the perceived depth of three-dimensional (3D) objects. However, depth is often misperceived, even when both cues are available. This may be due in part to conflicts with unmodelled cues endemic to computerized displays. Here we evaluated the impact of display-based cue conflicts on depth cue integration by comparing perceived depth for physical and virtual objects. Truncated square pyramids were rendered using Blender and 3D printed. We assessed perceived depth using a discrimination task with motion parallax, binocular disparity, and their combination. Physical stimuli were presented with precise control over position and lighting. Virtual stimuli were viewed using a head-mounted display. To generate motion parallax, observers made lateral head movements using a chin rest on a motion platform. Observers indicated if the width of the front face appeared greater or less than the distance between this surface and the base. We found that accuracy was similar for virtual and physical pyramids. All estimates were more precise when depth was defined by binocular disparity than motion parallax. Our probabilistic model shows that a linear combination model does not adequately describe performance in either physical or virtual conditions. While there was inter-observer variability in weights, performance in all conditions was best predicted by a veto model that excludes the less reliable depth cue, in this case motion parallax.

Cue combination in goal-oriented navigation.

This study examined cue combination of self-motion and landmark cues in goal-localisation. In an immersive virtual environment, before walking a two-leg path, participants learned the locations of three goal objects (one at the path origin, that is, home) and landmarks. After walking the path without seeing landmarks or goals, participants indicated the locations of the home and non-home goals in four conditions: (1) path integration only, (2) landmarks only, (3) both path integration and the landmarks, and (4) path integration and rotated landmarks. The ratio of the length between the testing position (P) and the turning point (T) over the length between the T and the three goals (G) (i.e., PT/TG) was manipulated. The results showed the cue combination consistently for participants' heading estimates but not for goal-localisation. In Experiments 1 and 2 (using distal landmarks), the cue combination for goal estimates appeared in a small length ratio (PT/TG = 0.5) but disappeared in a large length ratio (PT/TG = 2). In Experiments 3 and 4 (using proximal landmarks), while the cue combination disappeared for the home with a medium length ratio (PT/TG = 1), it appeared for the non-home goal with a large length ratio (PT/TG = 2) and only disappeared with a very large length ratio (PT/TG = 3). These findings are explained by a model stipulating that cue combination occurs in self-localisation (e.g., heading estimates), which leads to one estimate of the goal location; proximal landmarks produce another goal location estimate; these two goal estimates are then combined, which may only occur for non-home goals.

Optic Flow: Perceiving and Acting in a 3-D World.

In 1979, James Gibson completed his third and final book "The Ecological Approach to Visual Perception". That book can be seen as the synthesis of the many radical ideas he proposed over the previous 30 years - the concept of information and its sufficiency, the necessary link between perception and action, the need to see perception in relation to an animal's particular ecological niche and the meanings (affordances) offered by the visual world. One of the fundamental concepts that lies beyond all of Gibson's thinking is that of optic flow: the constantly changing patterns of light that reach our eyes and the information it provides. My purpose in writing this paper has been to evaluate the legacy of Gibson's conceptual ideas and to consider how his ideas have influenced and changed the way we study perception.