Integrating Gaze, image analysis, and body tracking: Foothold selection during locomotion.

Relatively little is known about the way vision is use to guide locomo-tion in the natural world. What visual features are used to choose paths in natural complex terrain? How do walkers trade off different costs such as getting to the goal, minimizing energy, and satisfying stability constraints? To answer these questions, it is necessary to monitor not only the eyes and the body, but also to represent the three dimensional structure of the terrain. We used photogrammetry techniques to do this, and found substantial regularities in the choice of paths. Walkers avoid paths that involve changes in height and choose more circuitous and flatter paths. This stable tradeoff is related to the walker's leg length and reflects both energetic and stability constraints. Gaze data and path choices suggest that subjects take into account the terrain approximately 5 steps ahead, and so are planning routes as well as particular footplants. Such planning ahead allows the minimization of energetic costs. Thus locomotor behavior in natural environments is controlled by decision mechanisms that attempt to optimize for multiple factors in the context of well-calibrated sensory and motor internal models.

Binocular vision and the control of foot placement during walking in natural terrain.

Coordination between visual and motor processes is critical for the selection of stable footholds when walking in uneven terrains. While recent work (Matthis et al. in Curr Biol 8(28):1224-1233, 2018) demonstrates a tight link between gaze (visual) and gait (motor), it remains unclear which aspects of visual information play a role in this visuomotor control loop, and how the loss of this information affects that relationship. Here we examine the role of binocular information in the visuomotor control of walking over complex terrain. We recorded eye and body movements while normally-sighted participants walked over terrains of varying difficulty, with intact vision or with vision in one eye blurred to disrupt binocular vision. Gaze strategy was highly sensitive to the complexity of the terrain, with more fixations dedicated to foothold selection as the terrain became more difficult. The primary effect of increased sensory uncertainty due to disrupted binocular vision was a small bias in gaze towards closer footholds, indicating greater pressure on the visuomotor control process. Participants with binocular vision losses due to developmental disorders (i.e., amblyopia, strabismus), who have had the opportunity to develop alternative strategies, also biased their gaze towards closer footholds. Across all participants, we observed a relationship between an individual's typical level of binocular visual function and the degree to which gaze is shifted toward the body. Thus the gaze-gait relationship is sensitive to the level of sensory uncertainty, and deficits in binocular visual function (whether transient or long-standing) have systematic effects on gaze strategy in complex terrains. We conclude that binocular vision provides useful information for locating footholds during locomotion. Furthermore, we have demonstrated that combined eye/body tracking in natural environments can be used to provide a more detailed understanding of the impact of a type of vision loss on the visuomotor control process of walking, a vital everyday task.

Control strategies for rapid, visually guided adjustments of the foot during continuous walking.

When walking over stable, complex terrain, visual information about an upcoming foothold is primarily utilized during the preceding step to organize a nearly ballistic forward movement of the body. However, it is often necessary to respond to changes in the position of an intended foothold that occur around step initiation. Although humans are capable of rapidly adjusting foot trajectory mid-swing in response to a perturbation of target position, such movements may disrupt the efficiency and stability of the gait cycle. In the present study, we consider whether walkers sometimes adopt alternative strategies for responding to perturbations that interfere less with ongoing forward locomotion. Subjects walked along a path of irregularly spaced stepping targets projected onto the ground, while their movements were recorded by a full-body motion-capture system. On a subset of trials, the location of one target was perturbed in either a medial-lateral or anterior-posterior direction. We found that subjects were best able to respond to perturbations that occurred during the latter half of the preceding step and that responses to perturbations that occurred during a step were less successful than previously reported in studies using a single-step paradigm. We also found that, when possible, subjects adjusted the ballistic movement of their center of mass in response to perturbations. We conclude that, during continuous walking, strategies for responding to perturbations that rely on reach-like movements of the foot may be less effective than previously assumed. For perturbations that are detected around step initiation, walkers prefer to adapt by tailoring the global, pendular mechanics of the body.

Visual regulation of gait: Zeroing in on a solution to the complex terrain problem.

We examine the theoretical understanding of visual gait regulation that has emerged from decades of research since the publication of Lee, Lishman, and Thompson's (1982) classic study of elite long jumpers. The first round of research identified specific informational variables, parameters of the action system, and laws of control that capture the coupling of perception and action in this context, but left unanswered important questions about why visual information is sampled in an intermittent manner and how the strategies that actors adopt ensure stability and energetic efficiency. More recent developments lead to a refined view according to which visual information is used at a specific phase of the gait cycle to modify the parameters that govern the passive dynamics of the body. We then present the results of a new experiment designed to test the prediction that when the terrain offers multiple foothold options for a given step, walkers' choices will be constrained by a strong preference for not interfering with the natural, ballistic movement of the body throughout the single support phase of that step. The findings are consistent with this prediction and support a view of visual gait regulation that is concordant with contemporary accounts of how actors use both active and passive modes of control. (PsycINFO Database Record

Visual control of foot placement when walking over complex terrain.

The aim of this study was to investigate the role of visual information in the control of walking over complex terrain with irregularly spaced obstacles. We developed an experimental paradigm to measure how far along the future path people need to see in order to maintain forward progress and avoid stepping on obstacles. Participants walked over an array of randomly distributed virtual obstacles that were projected onto the floor by an LCD projector while their movements were tracked by a full-body motion capture system. Walking behavior in a full-vision control condition was compared with behavior in a number of other visibility conditions in which obstacles did not appear until they fell within a window of visibility centered on the moving observer. Collisions with obstacles were more frequent and, for some participants, walking speed was slower when the visibility window constrained vision to less than two step lengths ahead. When window sizes were greater than two step lengths, the frequency of collisions and walking speed were weakly affected or unaffected. We conclude that visual information from at least two step lengths ahead is needed to guide foot placement when walking over complex terrain. When placed in the context of recent research on the biomechanics of walking, the findings suggest that two step lengths of visual information may be needed because it allows walkers to exploit the passive mechanical forces inherent to bipedal locomotion, thereby avoiding obstacles while maximizing energetic efficiency.

Humans perceive object motion in world coordinates during obstacle avoidance.

A fundamental question about locomotion in the presence of moving objects is whether movements are guided based upon perceived object motion in an observer-centered or world-centered reference frame. The former captures object motion relative to the moving observer and depends on both observer and object motion. The latter captures object motion relative to the stationary environment and is independent of observer motion. Subjects walked through a virtual environment (VE) viewed through a head-mounted display and indicated whether they would pass in front of or behind a moving obstacle that was on course to cross their future path. Subjects' movement through the VE was manipulated such that object motion in observer coordinates was affected while object motion in world coordinates was the same. We found that when moving observers choose routes around moving obstacles, they rely on object motion perceived in world coordinates. This entails a process, which has been called flow parsing (Rushton & Warren, 2005; Warren & Rushton, 2009a), that recovers the component of optic flow due to object motion independent of self-motion. We found that when self-motion is real and actively generated, the process by which object motion is recovered relies on both visual and nonvisual information to factor out the influence of self-motion. The remaining component contains information about object motion in world coordinates that is needed to guide locomotion.

Visual and non-visual contributions to the perception of object motion during self-motion.

Many locomotor tasks involve interactions with moving objects. When observer (i.e., self-)motion is accompanied by object motion, the optic flow field includes a component due to self-motion and a component due to object motion. For moving observers to perceive the movement of other objects relative to the stationary environment, the visual system could recover the object-motion component - that is, it could factor out the influence of self-motion. In principle, this could be achieved using visual self-motion information, non-visual self-motion information, or a combination of both. In this study, we report evidence that visual information about the speed (experiment 1) and direction (experiment 2) of self-motion plays a role in recovering the object-motion component even when non-visual self-motion information is also available. However, the magnitude of the effect was less than one would expect if subjects relied entirely on visual self-motion information. Taken together with previous studies, we conclude that when self-motion is real and actively generated, both visual and non-visual self-motion information contribute to the perception of object motion. We also consider the possible role of this process in visually guided interception and avoidance of moving objects.

Direct perception of action-scaled affordances: the shrinking gap problem.

The aim of this study was to investigate the perception of possibilities for action (i.e., affordances) that depend on one's movement capabilities, and more specifically, the passability of a shrinking gap between converging obstacles. We introduce a new optical invariant that specifies in intrinsic units the minimum locomotor speed needed to safely pass through a shrinking gap. Detecting this information during self-motion requires recovering the component of the obstacles' local optical expansion attributable to obstacle motion, independent of self-motion. In principle, recovering the obstacle motion component could involve either visual or non-visual self-motion information. We investigated the visual and non-visual contributions in two experiments in which subjects walked through a virtual environment and made judgments about whether it was possible to pass through a shrinking gap. On a small percentage of trials, visual and non-visual self-motion information were independently manipulated by varying the speed with which subjects moved through the virtual environment. Comparisons of judgments on such catch trials with judgments on normal trials revealed both visual and non-visual contributions to the detection of information about minimum walking speed.