Perceptual-Cognitive Integration for Goal-Directed Action in Naturalistic Environments.

Real-world actions require one to simultaneously perceive, think, and act on the surrounding world, requiring the integration of (bottom-up) sensory information and (top-down) cognitive and motor signals. Studying these processes involves the intellectual challenge of cutting across traditional neuroscience silos, and the technical challenge of recording data in uncontrolled natural environments. However, recent advances in techniques, such as neuroimaging, virtual reality, and motion tracking, allow one to address these issues in naturalistic environments for both healthy participants and clinical populations. In this review, we survey six topics in which naturalistic approaches have advanced both our fundamental understanding of brain function and how neurologic deficits influence goal-directed, coordinated action in naturalistic environments. The first part conveys fundamental neuroscience mechanisms related to visuospatial coding for action, adaptive eye-hand coordination, and visuomotor integration for manual interception. The second part discusses applications of such knowledge to neurologic deficits, specifically, steering in the presence of cortical blindness, impact of stroke on visual-proprioceptive integration, and impact of visual search and working memory deficits. This translational approach-extending knowledge from lab to rehab-provides new insights into the complex interplay between perceptual, motor, and cognitive control in naturalistic tasks that are relevant for both basic and clinical research.

Predictive posture stabilization before contact with moving objects: equivalence of smooth pursuit tracking and peripheral vision.

Postural stabilization is essential to effectively interact with our environment. Humans preemptively adjust their posture to counteract impending disturbances, such as those encountered during interactions with moving objects, a phenomenon known as anticipatory postural adjustments (APAs). APAs are thought to be influenced by predictive models that incorporate object motion via retinal motion and extraretinal signals. Building on our previous work that examined APAs in relation to the perceived momentum of moving objects, here we explored the impact of object motion within different visual field sectors on the human capacity to anticipate motion and prepare APAs for contact between virtual moving objects and the limb. Participants interacted with objects moving toward them under different gaze conditions. In one condition, participants fixated on either a central point (central fixation) or left-right of the moving object (peripheral fixation), whereas in another, they followed the moving object with smooth pursuit eye movements (SPEMs). We found that APAs had the smallest magnitude in the central fixation condition and that no notable differences in APAs were apparent between the SPEM and peripheral fixation conditions. This suggests that the visual system can accurately perceive motion of objects in peripheral vision for posture stabilization. Using Bayesian model averaging, we also evaluated the contribution of different gaze variables, such as eye velocity and gain (ratio of eye and object velocity) and showed that both eye velocity and gain signals were significant predictors of APAs. Taken together, our study underscores the roles of oculomotor signals in the modulation of APAs.NEW & NOTEWORTHY We show that the human visuomotor system can detect motion in peripheral vision and make anticipatory adjustments to posture before contact with moving objects, just as effectively as when the eye movement system tracks those objects with smooth pursuit eye movements. These findings pave the way for research into how age-induced changes in spatial vision, eye movements, and motion perception could affect the control of limb movements and postural stability during motion-mediated interactions with objects.

Prefrontal cortex hemodynamic activity during a test of lower extremity functional muscle strength in children with cerebral palsy: A functional near-infrared spectroscopy study.

Children with cerebral palsy (CP) exhibit impaired motor control and significant muscle weakness due to a brain lesion. However, studies that assess the relationship between brain activity and performance on dynamic functional muscle strength assessments in CP are needed. The aim of this study was to determine the effect of a progressive lateral step-up test on prefrontal cortex (PFC) hemodynamic activity in children with CP. Fourteen ambulatory children with spastic CP (Gross Motor Function Classification System level I; 5-11 y) and 14 age- and sex-matched typically developing control children completed a progressive lateral step-up test at incremental step heights (0, 10, 15 and 20 cm) using their non-dominant lower limb. Hemodynamic activity in the PFC was assessed using non-invasive, portable functional neuroimaging (functional near-infrared spectroscopy). Children with CP completed fewer repetitions at each step height and exhibited lower PFC hemodynamic activity across step heights compared to controls. Lower PFC activation in CP was maintained after statistically controlling for the number of repetitions completed at each step height. PFC hemodynamic activity was not associated with LSUT task performance in children with CP, but a positive relationship was observed in controls at the most challenging 20 cm step height. The results suggest there is an altered PFC recruitment pattern in children with CP during a highly dynamic test of functional strength. Further studies are needed to explore the mechanisms underlying the suppressed PFC activation observed in children with CP compared to typically developing children.

Smooth pursuit eye movements contribute to anticipatory force control during mechanical stopping of moving objects.

When stopping a closing door or catching an object, humans process the motion of inertial objects and apply reactive limb force over short period to interact with them. One way in which the visual system processes motion is through extraretinal signals associated with smooth pursuit eye movements (SPEMs). We conducted three experiments to investigate how SPEMs contribute to anticipatory and reactive hand force modulation when interacting with a virtual object moving in the horizontal plane. We hypothesized that SPEM signals are critical for timing motor responses, anticipatory control of hand force, and task performance. Participants held a robotic manipulandum and attempted to stop an approaching simulated object by applying a force impulse (area under force-time curve) that matched the object's virtual momentum upon contact. We manipulated the object's momentum by varying either its virtual mass or its speed under free gaze or constrained gaze conditions. We examined gaze variables, the timing of hand motor responses, anticipatory force control, and overall task performance. Our results show that when participants were fixated at a designated location instead of following objects with SPEM, anticipatory modulation of hand force before contact decreased. However, constraining gaze by asking participants to fixate did not seem to affect the timing of the motor response or the task performance. Together, these results suggest that SPEMs may be important for anticipatory control of hand force before contact and may also play a critical role in anticipatory stabilization of limb posture when humans interact with moving objects.NEW & NOTEWORTHY We show for the first time that smooth pursuit eye movements (SPEMs) play a role in the modulation of anticipatory control of hand force to stabilize posture against contact forces. SPEMs are critical for tracking moving objects, facilitate processing motion of moving objects, and are impacted during aging and in many neurological disorders, such as Alzheimer's disease and multiple sclerosis. These results provide a novel basis to probe how changes in SPEMs could contribute to deficient limb motor control in older adults and patients with neurological disorders.

Benefits associated with the standing position during visual search tasks.

The literature on postural control highlights that task performance should be worse in challenging dual tasks than in a single task, because the brain has limited attentional resources. Instead, in the context of visual tasks, we assumed that (i) performance in a visual search task should be better when standing than when sitting and (ii) when standing, postural control should be better when searching than performing the control task. 32 and 16 young adults participated in studies 1 and 2, respectively. They performed three visual tasks (searching to locate targets, free-viewing and fixating a stationary cross) displayed in small images (visual angle: 22°) either when standing or when sitting. Task performance, eye, head, upper back, lower back and center of pressure displacements were recorded. In both studies, task performance in searching was as good (and clearly not worse) when standing as when sitting. Sway magnitude was smaller during the search task (vs. other tasks) when standing but not when sitting. Hence, only when standing, postural control was adapted to perform the challenging search task. When exploring images, and especially so in the search task, participants rotated their head instead of their eyes as if they used an eye-centered strategy. Remarkably in Study 2, head rotation was greater when sitting than when standing. Overall, we consider that variability in postural control was not detrimental but instead useful to facilitate visual task performance. When sitting, this variability may be lacking, thus requiring compensatory movements.

Object motion influences feedforward motor responses during mechanical stopping of virtual projectiles: a preliminary study.

An important window into sensorimotor function is how humans interact and stop moving projectiles, such as stopping a door from closing shut or catching a ball. Previous studies have suggested that humans time the initiation and modulate the amplitude of their muscle activity based on the momentum of the approaching object. However, real-world experiments are constrained by laws of mechanics, which cannot be manipulated experimentally to probe the mechanisms of sensorimotor control and learning. An augmented-reality variant of such tasks allows for experimental manipulation of the relationship between motion and force to obtain novel insights into how the nervous system prepares motor responses to interact with moving stimuli. Existing paradigms for studying interactions with moving projectiles use massless objects and are primarily focused on quantifying gaze and hand kinematics. Here, we developed a novel collision paradigm using a robotic manipulandum where participants mechanically stopped a virtual object moving in the horizontal plane. On each block of trials, we varied the virtual object's momentum by increasing either its velocity or mass. Participants stopped the object by applying a force impulse that matched the object momentum. We observed that hand force increased as a function of object momentum linked to changes in virtual mass or velocity, similar to results from studies involving catching free-falling objects. In addition, increasing object velocity resulted in later onset of hand force relative to the impending time-to-contact. These findings show that the present paradigm can be used to determine how humans process projectile motion for hand motor control.

Enhanced cognitive interference during visuomotor tasks may cause eye-hand dyscoordination.

In complex visuomotor tasks, such as cooking, people make many saccades to continuously search for items before and during reaching movements. These tasks require cognitive resources, such as short-term memory and task-switching. Cognitive load may impact limb motor performance by increasing demands on mental processes, but mechanisms remain unclear. The Trail-Making Tests, in which participants sequentially search for and make reaching movements to 25 targets, consist of a simple numeric variant (Trails-A) and a cognitively challenging variant that requires alphanumeric switching (Trails-B). We have previously shown that stroke survivors and age-matched controls make many more saccades in Trails-B, and those increases in saccades are associated with decreases in speed and smoothness of reaching movements. However, it remains unclear how patients with neurological injuries, e.g., stroke, manage progressive increases in cognitive load during visuomotor tasks, such as the Trail-Making Tests. As Trails-B trial progresses, switching between numbers and letters leads to progressive increases in cognitive load. Here, we show that stroke survivors with damage to frontoparietal areas and age-matched controls made more saccades and had longer fixations as they progressed through the 25 alphanumeric targets in Trails-B. Furthermore, when stroke survivors made saccades during reaching movements in Trails-B, their movement speed slowed down significantly. Thus, damage to frontoparietal areas serving cognitive motor functions may cause interference between oculomotor, visual, and limb motor functions, which could lead to significant disruptions in activities of daily living. These findings augment our understanding of the mechanisms that underpin cognitive-motor interference during complex visuomotor tasks.

Minimal reporting guideline for research involving eye tracking (2023 edition).

A guideline is proposed that comprises the minimum items to be reported in research studies involving an eye tracker and human or non-human primate participant(s). This guideline was developed over a 3-year period using a consensus-based process via an open invitation to the international eye tracking community. This guideline will be reviewed at maximum intervals of 4 years.

Behavioral synergic relations between eye and postural movements in young adults searching to locate objects in room inside houses.

During precise gaze shifts, eye, head, and body movements exhibit synergic relations. In the present study, we tested the existence of behavioural synergic relations between eye and postural movements in a goal-directed, precise, visual search task (locate target objects in large images). More precisely, we tested if postural control could be adjusted specifically to facilitate precise gaze shifts. Participants also performed a free-viewing task (gaze images with no goal) and a fixation task. In both search and free-viewing tasks, young participants (n = 20; mean age = 22 years) were free to move their eyes, head, and body segments as they pleased to self-explore the images with no external perturbation. We measured eye and postural kinematic movements. The results showed significant negative correlations between eye and postural (head and upper back) movements in the precise task, but not in the free-viewing task. The negative correlations were considered to be stabilizing and synergic. Indeed, the further the eyes moved, the more postural variables were adjusted to reduce postural sway. These results suggest that postural control was adjusted to succeed in subtle and active self-induced precise gaze shifts. Furthermore, partial correlations showed significant relations between (1) task performance to find target objects and (2) synergic relations between eye and postural movements. These later results tend to show that synergic eye-postural relations were performed to improve the task performance in the precise visual task.

Age-related deficits in rapid visuomotor decision-making.

Many goal-directed actions that require rapid visuomotor planning and perceptual decision-making are affected in older adults, causing difficulties in execution of many functional activities of daily living. Visuomotor planning and perceptual identification are mediated by the dorsal and ventral visual streams, respectively, but it is unclear how age-induced changes in sensory processing in these streams contribute to declines in visuomotor decision-making performance. Previously, we showed that in young adults, task demands influenced movement strategies during visuomotor decision-making, reflecting differential integration of sensory information between the two streams. Here, we asked the question if older adults would exhibit deficits in interactions between the two streams during demanding motor tasks. Older adults (n = 15) and young controls (n = 26) performed reaching or interception movements toward virtual objects. In some blocks of trials, participants also had to select an appropriate movement goal based on the shape of the object. Our results showed that older adults corrected fewer initial decision errors during both reaching and interception movements. During the interception decision task, older adults made more decision- and execution-related errors than young adults, which were related to early initiation of their movements. Together, these results suggest that older adults have a reduced ability to integrate new perceptual information to guide online action, which may reflect impaired ventral-dorsal stream interactions.NEW & NOTEWORTHY Older adults show declines in vision, decision-making, and motor control, which can lead to functional limitations. We used a rapid visuomotor decision task to examine how these deficits may interact to affect task performance. Compared with healthy young adults, older adults made more errors in both decision-making and motor execution, especially when the task required intercepting moving targets. This suggests that age-related declines in integrating perceptual and motor information may contribute to functional deficits.

Parkinson's disease-related changes in the behavioural synergy between eye movements and postural movements.

Patients with Parkinson's disease (PD patients) have been shown to exhibit abnormally low levels of synergy in their posture control. The goal of this study was to determine how synergic interactions between vision and posture are affected in PD patients. These synergic interactions were expected to be impaired because PD affects the basal ganglia, which are involved in the modulation of both types of movement. Twenty patients (mean age: 60) on levodopa and 20 age-matched-controls (mean age: 61) performed a precise visual task (searching for targets in an image) and an unprecise control task (randomly looking at an image) in which images were projected onto a large panoramic display. Lower back, upper back, head and eye movements were recorded simultaneously. To test behavioural synergies, Pearson correlations between eye and postural movements were analysed. The relationships between eye movements and upper and lower back movements were impaired in the patients. The age-matched controls did not show any significant correlations between eye and postural movements. Overall, our results showed that the PD patients failed to adjust and control their postural stability for success in the visual task. The impaired synergy between eye and postural movements was not related to clinical variables-probably because our patients had early-stage PD. Our results showed that impairments in synergy can occur very early in PD. Hence, the analysis of this synergy might provide a better understanding of postural instability, visual task performance in the upright stance, and perhaps the risk of falls in PD patients.

In the upright stance, posture is better controlled to perform precise visual tasks than laser pointing tasks.

PURPOSE: In the upright stance, young adults better stabilize their posture when they perform precise visual or pointing movements than when they stand quietly. We tested if postural stability could be improved further if precise and pointing tasks were combined. METHOD: Twenty-four healthy young adults (22 ± 12 years) performed six tasks combining three visual tasks (precise search, unprecise free-viewing and fixation tasks) and two pointing tasks (pointing-on and pointing-off tasks with laser beam on and off, respectively). In the visual tasks, participants either searched to locate targets within an image (precise task), looked at the image with no goal (unprecise task) or fixated on a cross (fixation task). In the pointing-on tasks, participants pointed a laser beam onto a small circle (2°) located in the middle of a larger circle (21°) containing the image. RESULT: As expected, postural sway was reduced in the precise tasks in contrast to the fixation tasks. Contrary to expectations, both precise and pointing-on tasks did not add their stabilizing effects. Furthermore, the pointing-on task almost did not influence body movements. The participants rotated their eyes and head more and their upper back less in the precise visual tasks than in the unprecise visual tasks. CONCLUSION: The participants used a stabilizing coordination to fully explore images with eye and head rotations while stabilizing their body to perform precise gaze shifts. Our findings suggest that posture stabilization is performed to facilitate success in precise visual tasks more so than to perform pointing-on tasks.

Reaching movements are automatically redirected to nearby options during target split.

Motor behavior often occurs in environments with multiple goal options that can vary during the ongoing action. We explored this situation by requiring subjects to select between different target options during an ongoing reach. During split trials the original target was replaced with a left and a right flanking target, and participants had to select between them. This contrasted with the standard jump trials, where the original target would be replaced with a single flanking target, left or right. When participants were instructed to follow their natural tendency, they all tended to select the split target nearest the original. The near-target preference was more prominent with increased spatial disparity between the options and when participants could preview the potential options. Moreover, explicit instruction to obtain the "far" target during split trials resulted many errors compared with a "near" instruction, ~50% vs. ~15%. Online reaction times to target change were delayed in split trials compared with jump trials, ~200 ms vs. ~150 ms, but also highly automatic. Trials in which the instructed far target was correctly obtained were delayed by a further ~50 ms, unlike those in which the near target was incorrectly obtained. We also observed nonspecific responses from arm muscles at the jump trial latency during split trials. Taken together, our results indicate that online selection of reach targets is automatically linked to the spatial distribution of the options, though at greater delays than redirecting to a single target.NEW & NOTEWORTHY This work demonstrates that target selection during an ongoing reach is automatically linked to the option nearest a voided target. Online reaction times for two options are longer than redirection to a single option. Attempts to override the near-target tendency result in a high number of errors at the normal delay and further delays when the attempt is successful.

Perceptual decisions about object shape bias visuomotor coordination during rapid interception movements.

Visual processing in parietal areas of the dorsal stream facilitates sensorimotor transformations for rapid movement. This action-related visual processing is hypothesized to play a distinct functional role from perception-related processing in the ventral stream. However, it is unclear how the two streams interact when perceptual identification is a prerequisite to executing an accurate movement. In the current study, we investigated how perceptual decision-making involving the ventral stream influences arm and eye movement strategies. Participants (n = 26) moved a robotic manipulandum using right whole arm movements to rapidly reach a stationary object or intercept a moving object on an augmented-reality display. On some blocks of trials, participants needed to identify the shape of the object (circle or ellipse) as a cue to either hit the object (circle) or move to a predefined location away from the object (ellipse). We found that during perceptual decision-making, there was an increased urgency to act during interception movements relative to reaching, which was associated with more decision errors. Faster hand reaction times were associated with a strategy to adjust the movement postinitiation, and this strategy was more prominent during interception. Saccadic reaction times were faster and initial saccadic peak velocity, initial gaze lags, and gains greater during decisions, suggesting that eye movements adapt to perceptual decision-making requirements. Together, our findings suggest that the integration of ventral stream information with visuomotor planning depends on imposed (or perceived) task demands.NEW & NOTEWORTHY Visual processing for perception and for action is thought to be mediated by two specialized neural pathways. Using a visuomotor decision-making task, we show that participants differentially utilized online perceptual decision-making in reaching and interception and that eye movements necessary for perception influenced motor decision strategies. These results provide evidence that task complexity modulates how pathways processing perception versus action information interact during the visual control of movement.

New insight into Parkinson's disease-related impairment of the automatic control of upright stance.

Parkinson's disease (PD) affects the automatic control of body movements. In our study, we tested PD-related impairments in automatic postural control in quiet upright stance. Twenty PD patients (mean age: 60 ± 8 years; Hoehn and Yahr: 2.00 ± 0.32, on-drug) and twenty age-matched controls (61 ± 7 years) were recruited. We studied interrelations between center-of-pressure movements, body movements (head, neck, and lower back), eye movements and variability of pupil size. Participants performed two fixation tasks while standing, during which they looked at: (a) a cross surrounded by a white background; and (b) a cross surrounded by a structured visual background (images used: rooms in houses). PD patients exhibited stronger and weaker correlations between eye and center-of-pressure/body movement variables than age-matched controls in the white and structured fixation tasks, respectively. Partial correlations, controlling for variability of pupil size showed that PD patients used lower and greater attentional resources than age-matched controls to control their eye and center-of-pressure/body movements simultaneously in the white fixation and structured fixation tasks, respectively. In the white fixation task, PD patients used attentional resources to optimize visuomotor coupling between eye and body movements to control their posture. In the structured fixation task, the salient visual stimuli distracted PD patients' attention and that possibly affected postural control by deteriorating the automatic visuomotor coupling. In contrast, age-matched controls were able to use surrounding visual background to improve the automatic coupling between eye and center-of-pressure movements to control their posture. These results suggest that cluttered environments may distract PD patients and deteriorate their postural control.

Multiple processes independently predict motor learning.

BACKGROUND: Our ability to acquire, refine and adapt skilled limb movements is a hallmark of human motor learning that allows us to successfully perform many daily activities. The capacity to acquire, refine and adapt other features of motor performance, such as visual search, eye-hand coordination and visuomotor decisions, may also contribute to motor learning. However, the extent to which refinements of multiple behavioral features and their underlying neural processes independently contribute to motor learning remains unknown. In the current study, we used an ethological approach to test the hypothesis that practice-related refinements of multiple behavioral features would be independently predictive of motor learning. METHODS: Eighteen healthy, young adults used an upper-limb robot with eye-tracking to practice six trials of a continuous, visuomotor task once a week for six consecutive weeks. Participants used virtual paddles to hit away 200 "Targets" and avoid hitting 100 "Distractors" that continuously moved towards them from the back of the workspace. Motor learning was inferred from trial-by-trial acquisition and week-by-week retention of improvements on two measures of task performance related to motor execution and motor inhibition. Adaptations involving underlying neural processes were inferred from trial-by-trial acquisition and week-by-week retention of refinements on measures of skilled limb movement, visual search, eye-hand coordination and visuomotor decisions. We tested our hypothesis by quantifying the extent to which refinements on measures of multiple behavioral features (predictors) were independently predictive of improvements on our two measures of task performance (outcomes) after removing all shared variance between predictors. RESULTS: We found that refinements on measures of skilled limb movement, visual search and eye-hand coordination were independently predictive of improvements on our measure of task performance related to motor execution. In contrast, only refinements of eye-hand coordination were independently predictive of improvements on our measure of task performance related to motor inhibition. CONCLUSION: Our results provide indirect evidence that refinements involving multiple, neural processes may independently contribute to motor learning, and distinct neural processes may underlie improvements in task performance related to motor execution and motor inhibition. This also suggests that refinements involving multiple, neural processes may contribute to motor recovery after stroke, and rehabilitation interventions should be designed to produce refinements of all behavioral features that may contribute to motor recovery.

Muscular effort differentially mediates perception of heaviness and length via dynamic touch.

Our ability to perceive properties of handheld objects (e.g., heaviness, orientation, length, width, and shape) by wielding via dynamic touch is crucial for tooling and other forms of object manipulation-activities that are the basis of much human experience. Here, we investigated how muscular effort mediates perception of heaviness and length via dynamic touch. Twelve participants wielded nine occluded elongated objects of distinct moments of inertia and reported their perceptual judgments of heaviness and length. We measured the electromyography (EMG) activity of the participants' biceps brachii, flexor carpi radialis, and flexor carpi ulnaris muscles during wielding. Distinct single-valued functions of the eigenvalues I1 and I3 of the inertial tensor, I, closely predicted perceived heaviness and perceived length of the wielded objects. Perceived heaviness showed a direct and linear relationship with EMG activity of biceps brachii, flexor carpi radialis, and flexor carpi ulnaris. However, while perceived length showed a very weak relationship with EMG activity of biceps brachii, we found no association between perceived length and EMG activity of flexor carpi radialis and flexor carpi ulnaris. Our findings indicate that muscular effort contributes directly to perception of heaviness, but likely only serves as a medium for perception of length. While the same physical variable-i.e., the moment of inertia-provides the informational support for perception of heaviness and length, distinct psychophysiological processes underlie perception of heaviness and length via dynamic touch.

Fractal fluctuations in muscular activity contribute to judgments of length but not heaviness via dynamic touch.

The applied muscular effort to wield, hold, or balance an object shapes the medium by which action-relevant perceptual judgments (e.g., heaviness, length, width, and shape) are derived. Strikingly, the integrity of these judgments is retained over a range of exploratory conditions, a phenomenon known as perceptual invariance. For instance, judgments of length do not vary with the speed of rotation, despite the greater muscular effort required to wield objects at higher speeds. If not the amount of muscular effort alone, then what features of the neuromuscular activity implicated while wielding objects contribute to perception via dynamic touch? In the present study, we investigated how muscular activity mediates perception of heaviness and length of objects via dynamic touch. We measured EMG activity in biceps brachii and flexor carpi radialis as participants wielded objects of different moments of inertia. We found that variation in the amount of muscular effort (literally, root-mean-square values of EMG activity) predicted variations in judgments of heaviness but not length. In contrast, fluctuations in the activity of biceps brachii and flexor carpi radialis were fractal, and variation in the degree of fractality in the two muscles predicted variation in judgments of length. These findings reflect the distinct implications of dynamic touch for perception of heaviness and length. Perceptions of length can be derived from minimal effort, and muscular effort only shapes the medium from which judgments of length are derived. We discuss our findings in the context of the body as a multifractal tensegrity system, wherein perceptual judgments of length by wielding implicate, at least in part, rapidly diffusing mechanotransduction perturbations cascading across the whole body.

A novel computational model to probe visual search deficits during motor performance.

Successful execution of many motor skills relies on well-organized visual search (voluntary eye movements that actively scan the environment for task-relevant information). Although impairments of visual search that result from brain injuries are linked to diminished motor performance, the neural processes that guide visual search within this context remain largely unknown. The first objective of this study was to examine how visual search in healthy adults and stroke survivors is used to guide hand movements during the Trail Making Test (TMT), a neuropsychological task that is a strong predictor of visuomotor and cognitive deficits. Our second objective was to develop a novel computational model to investigate combinatorial interactions between three underlying processes of visual search (spatial planning, working memory, and peripheral visual processing). We predicted that stroke survivors would exhibit deficits in integrating the three underlying processes, resulting in deteriorated overall task performance. We found that normal TMT performance is associated with patterns of visual search that primarily rely on spatial planning and/or working memory (but not peripheral visual processing). Our computational model suggested that abnormal TMT performance following stroke is associated with impairments of visual search that are characterized by deficits integrating spatial planning and working memory. This innovative methodology provides a novel framework for studying how the neural processes underlying visual search interact combinatorially to guide motor performance. NEW & NOTEWORTHY: Visual search has traditionally been studied in cognitive and perceptual paradigms, but little is known about how it contributes to visuomotor performance. We have developed a novel computational model to examine how three underlying processes of visual search (spatial planning, working memory, and peripheral visual processing) contribute to visual search during a visuomotor task. We show that deficits integrating spatial planning and working memory underlie abnormal performance in stroke survivors with frontoparietal damage.