Prediction of time to contact under perceptual and contextual uncertainties.

Accurately estimating time to contact (TTC) is crucial for successful interactions with moving objects, yet it is challenging under conditions of sensory and contextual uncertainty, such as occlusion. In this study, participants engaged in a prediction motion task, monitoring a target that moved rightward and an occluder. The participants' task was to press a key when they predicted the target would be aligned with the occluder's right edge. We manipulated sensory uncertainty by varying the visible and occluded periods of the target, thereby modulating the time available to integrate sensory information and the duration over which motion must be extrapolated. Additionally, contextual uncertainty was manipulated by having a predictable and unpredictable condition, meaning the occluder either reliably indicated where the moving target would disappear or provided no such indication. Results showed differences in accuracy between the predictable and unpredictable occluder conditions, with different eye movement patterns in each case. Importantly, the ratio of the time the target was visible, which allows for the integration of sensory information, to the occlusion time, which determines perceptual uncertainty, was a key factor in determining performance. This ratio is central to our proposed model, which provides a robust framework for understanding and predicting human performance in dynamic environments with varying degrees of uncertainty.

Contrasting contributions of movement onset and duration to self-evaluation of sensorimotor timing performance.

Movement execution is not always optimal. Understanding how humans evaluate their own motor decisions can give us insights into their suboptimality. Here, we investigated how humans time the action of synchronizing an arm movement with a predictable visual event and how well they can evaluate the outcome of this action. On each trial, participants had to decide when to start (reaction time) and for how long to move (movement duration) to reach a target on time. After each trial, participants judged the confidence they had that their performance on that trial was better than average. We found that participants mostly varied their reaction time, keeping the average movement duration short and relatively constant across conditions. Interestingly, confidence judgements reflected deviations from the planned reaction time and were not related to planned movement duration. In two other experiments, we replicated these results in conditions where the contribution of sensory uncertainty was reduced. In contrast to confidence judgements, when asked to make an explicit estimation of their temporal error, participants' estimates were related in a similar manner to both reaction time and movement duration. In summary, humans control the timing of their actions primarily by adjusting the delay to initiate the action, and they estimate their confidence in their action from the difference between the planned and executed movement onset. Our results highlight the critical role of the internal model for the self-evaluation of one's motor performance.

Flexible viewing time when estimating time-to-contact in 3D parabolic trajectories.

Obtaining reliable estimates of the time-to-contact (TTC) in a three-dimensional (3D) parabolic trajectory is still an open issue. A direct analysis of the optic flow cannot make accurate predictions for gravitationally accelerated objects. Alternatively, resorting to prior knowledge of gravity and size can provide accurate estimates of TTC in parabolic head-on trajectories, but its generalization depends on the specific geometry of the trajectory and particular moments. The aim of this work is to explore the preferred viewing windows to estimate TTC and how the available visual information affects these estimations. We designed a task in which participants, wearing an head-mounted display (HMD), had to time the moment a ball in a parabolic path returned at eye level. We used five trajectories for which accurate temporal predictions were available at different points of flight time. Our results show that our observers can predict both the trajectory of the ball and TTC based on the available visual information and previous experience with the task. However, the times at which our observers chose to gather the visual evidence did not match those in which visual information provided accurate TTC. Instead, they looked at the ball at relatively fixed temporal windows depending on the trajectory but not of TTC.

Looking away from a moving target does not disrupt the way in which the movement toward the target is guided.

People usually follow a moving object with their gaze if they intend to interact with it. What would happen if they did not? We recorded eye and finger movements while participants moved a cursor toward a moving target. An unpredictable delay in updating the position of the cursor on the basis of that of the invisible finger made it essential to use visual information to guide the finger's ongoing movement. Decreasing the contrast between the cursor and the background from trial to trial made it difficult to see the cursor without looking at it. In separate experiments, either participants were free to hit the target anywhere along its trajectory or they had to move along a specified path. In the two experiments, participants tracked the cursor rather than the target with their gaze on 13% and 32% of the trials, respectively. They hit fewer targets when the contrast was low or a path was imposed. Not looking at the target did not disrupt the visual guidance that was required to deal with the delays that we imposed. Our results suggest that peripheral vision can be used to guide one item to another, irrespective of which item one is looking at.

Eye movements in interception with delayed visual feedback.

The increased reliance on electronic devices such as smartphones in our everyday life exposes us to various delays between our actions and their consequences. Whereas it is known that people can adapt to such delays, the mechanisms underlying such adaptation remain unclear. To better understand these mechanisms, the current study explored the role of eye movements in interception with delayed visual feedback. In two experiments, eye movements were recorded as participants tried to intercept a moving target with their unseen finger while receiving delayed visual feedback about their own movement. In Experiment 1, the target randomly moved in one of two different directions at one of two different velocities. The delay between the participant's finger movement and movement of the cursor that provided feedback about the finger movements was gradually increased. Despite the delay, participants followed the target with their gaze. They were quite successful at hitting the target with the cursor. Thus, they moved their finger to a position that was ahead of where they were looking. Removing the feedback showed that participants had adapted to the delay. In Experiment 2, the target always moved in the same direction and at the same velocity, while the cursor's delay varied across trials. Participants still always directed their gaze at the target. They adjusted their movement to the delay on each trial, often succeeding to intercept the target with the cursor. Since their gaze was always directed at the target, and they could not know the delay until the cursor started moving, participants must have been using peripheral vision of the delayed cursor to guide it to the target. Thus, people deal with delays by directing their gaze at the target and using both experience from previous trials (Experiment 1) and peripheral visual information (Experiment 2) to guide their finger in a way that will make the cursor hit the target.

An object-tracking model that combines position and speed explains spatial and temporal responses in a timing task.

Many tasks require synchronizing our actions with particular moments along the path of moving targets. However, it is controversial whether we base these actions on spatial or temporal information, and whether using either can enhance our performance. We addressed these questions with a coincidence timing task. A target varying in speed and motion duration approached a goal. Participants stopped the target and were rewarded according to its proximity to the goal. Results showed larger reward for responses temporally (rather than spatially) equidistant to the goal across speeds, and this pattern was promoted by longer motion durations. We used a Kalman filter to simulate time and space-based responses, where modeled speed uncertainty depended on motion duration and positional uncertainty on target speed. The comparison between simulated and observed responses revealed that a single position-tracking mechanism could account for both spatial and temporal patterns, providing a unified computational explanation.

Flexible timing of eye movements when catching a ball.

In ball games, one cannot direct ones gaze at the ball all the time because one must also judge other aspects of the game, such as other players' positions. We wanted to know whether there are times at which obtaining information about the ball is particularly beneficial for catching it. We recently found that people could catch successfully if they saw any part of the ball's flight except the very end, when sensory-motor delays make it impossible to use new information. Nevertheless, there may be a preferred time to see the ball. We examined when six catchers would choose to look at the ball if they had to both catch the ball and find out what to do with it while the ball was approaching. A catcher and a thrower continuously threw a ball back and forth. We recorded their hand movements, the catcher's eye movements, and the ball's path. While the ball was approaching the catcher, information was provided on a screen about how the catcher should throw the ball back to the thrower (its peak height). This information disappeared just before the catcher caught the ball. Initially there was a slight tendency to look at the ball before looking at the screen but, later, most catchers tended to look at the screen before looking at the ball. Rather than being particularly eager to see the ball at a certain time, people appear to adjust their eye movements to the combined requirements of the task.

Microstructure of the superior longitudinal fasciculus predicts stimulation-induced interference with on-line motor control.

A cortical visuomotor network, comprising the medial intraparietal sulcus (mIPS) and the dorsal premotor area (PMd), encodes the sensorimotor transformations required for the on-line control of reaching movements. How information is transmitted between these two regions and which pathways are involved, are less clear. Here, we use a multimodal approach combining repetitive transcranial magnetic stimulation (rTMS) and diffusion tensor imaging (DTI) to investigate whether structural connectivity in the 'reaching' circuit is associated to variations in the ability to control and update a movement. We induced a transient disruption of the neural processes underlying on-line motor adjustments by applying 1Hz rTMS over the mIPS. After the stimulation protocol, participants globally showed a reduction of the number of corrective trajectories during a reaching task that included unexpected visual perturbations. A voxel-based analysis revealed that participants exhibiting higher fractional anisotropy (FA) in the second branch of the superior longitudinal fasciculus (SLF II) suffered less rTMS-induced behavioral impact. These results indicate that the microstructural features of the white matter bundles within the parieto-frontal 'reaching' circuit play a prominent role when action reprogramming is interfered. Moreover, our study suggests that the structural alignment and cohesion of the white matter tracts might be used as a predictor to characterize the extent of motor impairments.

Why do movements drift in the dark? Passive versus active mechanisms of error accumulation.

When vision of the hand is unavailable, movements drift systematically away from their targets. It is unclear, however, why this drift occurs. We investigated whether drift is an active process, in which people deliberately modify their movements based on biased position estimates, causing the real hand to move away from the real target location, or a passive process, in which execution error accumulates because people have diminished sensory feedback and fail to adequately compensate for the execution error. In our study participants reached back and forth between two targets when vision of the hand, targets, or both the hand and targets was occluded. We observed the most drift when hand vision and target vision were occluded and equivalent amounts of drift when either hand vision or target vision was occluded. In a second experiment, we observed movement drift even when no visual target was ever present, providing evidence that drift is not driven by a visual-proprioceptive discrepancy. The observed drift in both experiments was consistent with a model of passive error accumulation in which the amount of drift is determined by the precision of the sensory estimate of movement error.

Deconstructing multisensory enhancement in detection.

The mechanisms responsible for the integration of sensory information from different modalities have become a topic of intense interest in psychophysics and neuroscience. Many authors now claim that early, sensory-based cross-modal convergence improves performance in detection tasks. An important strand of supporting evidence for this claim is based on statistical models such as the Pythagorean model or the probabilistic summation model. These models establish statistical benchmarks representing the best predicted performance under the assumption that there are no interactions between the two sensory paths. Following this logic, when observed detection performances surpass the predictions of these models, it is often inferred that such improvement indicates cross-modal convergence. We present a theoretical analyses scrutinizing some of these models and the statistical criteria most frequently used to infer early cross-modal interactions during detection tasks. Our current analysis shows how some common misinterpretations of these models lead to their inadequate use and, in turn, to contradictory results and misleading conclusions. To further illustrate the latter point, we introduce a model that accounts for detection performances in multimodal detection tasks but for which surpassing of the Pythagorean or probabilistic summation benchmark can be explained without resorting to early cross-modal interactions. Finally, we report three experiments that put our theoretical interpretation to the test and further propose how to adequately measure multimodal interactions in audiotactile detection tasks.

Hitting moving targets with a continuously changing temporal window.

Hitting a moving target requires that we do not miss the target when it is around the aimed position. The time available for us not to miss the target when it is at the position of interest is usually called the time window and depends on target's speed and size. These variables, among others, have been manipulated in previous studies but kept constant within the same trial or session. Here, we present results of a hitting task in which targets underwent simple harmonic motion, which is defined by a sinusoidal function. Target velocity changes continuously in this motion and so does the time window which is shorter in the centre (peak velocity) and longer at the turning points (lowest velocity) within a single trial. We studied two different conditions in which the target moved with a reliable (across trials) amplitude displacement or reliable peak velocity, respectively, and subjects were free to decide where and when to hit it. Results show that subjects made a compromise between maximum and minimum target's speed, so that they did hit the target at intermediate speed values. Interestingly, the reliability of target peak velocity (or displacement) modulated the point of interception. When target's peak velocity was more reliable, subjects intercepted the target at positions with smaller temporal windows and the reverse was true when displacement was reliable. Subjects adapted the interceptive behaviour to the underlying statistical structure of the targets. Finally, in a control condition in which the temporal window also depended on the instant size and not only on speed, subjects intercepted the target when it moved at similar speeds than when the size was constant. This finding suggests that velocity rather than the temporal window contributed more to controlling the interceptive movements.

Temporal and spatial constraints of action effect on sensory binding.

Previous studies have shown that asynchrony in perceived changes in the visual attributes of an object is attenuated when the object is the target of a manual reaching action (e.g. Corveleyn et al. in J Vis, 2012. doi: 10.1167/12.11.20 ). In the present study, we examined the temporal and spatial constraints associated with the effect of action on sensory binding. Participants performed a temporal order judgment task which required them to judge which changed first, the position or the colour of a visual stimulus, either while performing a concurrent motor task (manual acquisition of a visual target) or not (perceptual task). In Experiment 1, the fixed-attribute change (colour or position) occurred 0, 250, 500 or 1000 ms following the end of the motor action or the presentation of an auditory cue, while the variable-attribute change (position or colour) occurred randomly within an interval of ±200 ms from the fixed-attribute change. In Experiment 2, the visual stimulus was presented at a distance of 0, 2, 4 or 8 cm from a central fixation cross which was the target in the motor task. The fixed attribute (colour or position) changed 700 ms after an auditory cue (perceptual task) or when the hand reached the visual target (motor task). The variable-attribute change (position or colour) again occurred within an interval of ±200 ms from the fixed-attribute change. Statistical analysis of the point of subjective simultaneity revealed that performing a motor action reduced the perceived temporal asynchrony in the perceptual task, but only when the visual changes occurred less than 500 ms (for the fixed attribute) following movement execution (Exp. 1) and at a distance of less than 4 cm from the movement endpoint (Exp. 2). These results indicate that action-induced sensory binding requires temporal contiguity and spatial congruency between the endpoint of the action and its visual consequences.

Visual limitations shape audio-visual integration.

Recent studies have proposed that some cross-modal illusions might be expressed in what were previously thought of as sensory-specific brain areas. Therefore, one interesting question is whether auditory-driven visual illusory percepts respond to manipulations of low-level visual attributes (such as luminance or chromatic contrast) in the same way as their nonillusory analogs. Here, we addressed this question using the double flash illusion (DFI), whereby one brief flash can be perceived as two when combined with two beeps presented in rapid succession. Our results showed that the perception of two illusory flashes depended on luminance contrast, just as the temporal resolution for two real flashes did. Specifically we found that the higher the luminance contrast, the stronger the DFI. Such a pattern seems to contradict what would be predicted from a maximum likelihood estimation perspective, and can be explained by considering that low-level visual stimulus attributes similarly modulate the perception of sound-induced visual phenomena and "real" visual percepts. This finding provides psychophysical support for the involvement of sensory-specific brain areas in the expression of the DFI. On the other hand, the addition of chromatic contrast failed to produce a change in the strength of the DFI despite it improved visual sensitivity to real flashes. The null impact of chromaticity on the cross-modal illusion might suggest a weaker interaction of the parvocellular visual pathway with the auditory system for cross-modal illusions.

Dealing with delays does not transfer across sensorimotor tasks.

It is known that people can learn to deal with delays between their actions and the consequences of such actions. We wondered whether they do so by adjusting their anticipations about the sensory consequences of their actions or whether they simply learn to move in certain ways when performing specific tasks. To find out, we examined details of how people learn to intercept a moving target with a cursor that follows the hand with a delay and examined the transfer of learning between this task and various other tasks that require temporal precision. Subjects readily learned to intercept the moving target with the delayed cursor. The compensation for the delay generalized across modifications of the task, so subjects did not simply learn to move in a certain way in specific circumstances. The compensation did not generalize to completely different timing tasks, so subjects did not generally expect the consequences of their motor commands to be delayed. We conclude that people specifically learn to control the delayed visual consequences of their actions to perform certain tasks.

Sound-driven enhancement of vision: disentangling detection-level from decision-level contributions.

Cross-modal enhancement can be mediated both by higher-order effects due to attention and decision making and by detection-level stimulus-driven interactions. However, the contribution of each of these sources to behavioral improvements has not been conclusively determined and quantified separately. Here, we apply psychophysical analysis based on Piéron functions in order to separate stimulus-dependent changes from those accounted by decision-level contributions. Participants performed a simple visual speeded detection task on Gabor patches of different spatial frequencies and contrast values, presented with and without accompanying sounds. On one hand, we identified an additive cross-modal improvement in mean reaction times across all types of visual stimuli that would be well explained by interactions not strictly based on stimulus-driven modulations (e.g., due to reduction of temporal uncertainty and motor times). On the other hand, we singled out an audio-visual benefit that strongly depended on stimulus features such as frequency and contrast. This particular enhancement was selective to low-visual spatial frequency stimuli, optimized for magnocellular sensitivity. We therefore conclude that interactions at detection stages and at decisional processes in response selection that contribute to audio-visual enhancement can be separated online and express on partly different aspects of visual processing.

Unifying time to contact estimation and collision avoidance across species.

The τ-function and the η-function are phenomenological models that are widely used in the context of timing interceptive actions and collision avoidance, respectively. Both models were previously considered to be unrelated to each other: τ is a decreasing function that provides an estimation of time-to-contact (ttc) in the early phase of an object approach; in contrast, g has a maximum before ttc. Furthermore, it is not clear how both functions could be implemented at the neuronal level in a biophysically plausible fashion. Here we propose a new framework--the corrected modified Tau function--capable of predicting both τ-type ("τ(cm)") and g-type ("t(mod)") responses. The outstanding property of our new framework is its resilience to noise. We show that t(mod) can be derived from a firing rate equation, and, as g, serves to describe the response curves of collision sensitive neurons. Furthermore, we show that tcm predicts the psychophysical performance of subjects determining ttc. Our new framework is thus validated successfully against published and novel experimental data. Within the framework, links between τ-type and η-type neurons are established. Therefore, it could possibly serve as a model for explaining the co-occurrence of such neurons in the brain.

Motor action reduces temporal asynchrony between perceived visual changes.

Perceiving a visual object requires binding sensory estimates of its various physical attributes. This process can be facilitated if changes of different attributes are perceived with little asynchronies when they are physically aligned, which is not always the case as revealed by temporal order judgment or perceptual synchronization tasks of visual attributes changes. In this study, we analyzed the effect of performing a motor action on the perceived relative timing between changes of position and color of a visual target by using a temporal order judgment (TOJ) task. Results showed that in the perceptual condition, the change of color must precede (-37.9 ms) the change of position in order to perceive a synchronous change of both target's visual attributes. This physical asynchrony vanished when the same changes took place near the end of a manual reaching action executed towards the visual target (-3.3 ms). The reduction of asynchrony was, however, not observed when participants performed TOJ of visual attributes change in the presence of concomitant tactile information (-36 ms) but with no action. The perceptual relative timing between visual changes was also unaffected when the timing was obtained by comparing each visual change to tactile information resulting from motor action (-33.5 ms) or external stimulation (-27.8 ms). Altogether, these results suggest that signals associated with the organization of a motor action, but not sensory information itself, contribute to reduce the differential delays when processing visual attributes of a single object. Furthermore, the effect of action was not observed when judging relative timing of object-related (visual) versus object-unrelated (tactile) sensory information.

Seeing the last part of a hitting movement is enough to adapt to a temporal delay.

Being able to see the object that you are aiming for is evidently useful for guiding the hand to a moving object. We examined to what extent seeing the moving hand also influences performance. Subjects tried to intercept moving targets while either instantaneous or delayed feedback about the moving hand was provided at certain times. After each attempt, subjects had to indicate whether they thought they had hit the target, had passed ahead of it, or had passed behind it. Providing visual feedback early in the movement enabled subjects to use visual information about the moving hand to correct their movements. Providing visual feedback when the moving hand passed the target helped them judge how they had performed. Performance was almost as good when visual feedback about the moving hand was provided only when the hand was passing the target as when it was provided throughout the movement. We conclude that seeing the temporal relationship between the hand and the target as the hand crosses the target's path is instrumental for adapting to a temporal delay.

The time to passage of biological and complex motion.

A significant part of human interactions occur with other human beings and not only with inanimate objects. It is important in everyday tasks to estimate the time it takes other people to reach (time to contact) or pass us (time to passage). Surprisingly, little is known about judging time to contact or time to passage of biological or other complex motions. In two experiments, rigid and non-rigid (biological, inverted, scrambled, and complex non-biological) motion conditions were compared in a time-to-passage judgment task. Subjects could judge time to passage of point-light-walker displays. However, due to relative and opponent movements of body parts, all articulated patterns conveyed a noisier looming pattern. Non-rigid stimuli were judged as passing sooner than rigid stimuli but reflected more uncertainty in the judgments as revealed by precision judgments and required longer reaction times. Our findings suggested that perceptual judgments for complex motion, including biological patterns, are built on top of the same processing channels that are involved on rigid motion perception. The complexity of the motion pattern (rigid vs. non-rigid) plays a more determinant role than the "biologicity" of the stimulus (biological vs. non-biological), at least concerning time-to-passage judgments.

Duration judgments are mediated by the similarity with the temporal context.

When we try to assess the duration of an event, we are often affected by external information. Studies on multiple timing have found that simultaneous timing information can produce an averaging or central tendency effect, where the perceived duration of the elements tends to be biased towards a general average. We wanted to assess how this effect induced by simultaneous distractors could depend on the temporal similarity between stimuli. We used a duration judgment task in which participants (n = 22) had to compare the duration of two identical targets (1 s) accompanied by simultaneous distractors of different durations (0.3, 0.7, 1.5 or 3 s). We found a central tendency effect, where duration judgments of the target were systematically biased towards the duration of the distractors that accompanied them. We put forward a model based on the concept of duration-channels that can explain the central tendency effect with only one estimated parameter. This parameter modulates the rate of decay of this effect as distractors duration become more different than the duration of the target.