How workload and availability of spatial reference shape eye movement coupling in visuospatial working memory.

Eyes are active in memory recall and visual imagination, yet our grasp of the underlying qualities and factors of these internally coupled eye movements is limited. To explore this, we studied 50 participants, examining how workload, spatial reference availability, and imagined movement direction influence internal coupling of eye movements. We designed a visuospatial working memory task in which participants mentally moved a black patch along a path within a matrix and each trial involved one step along this path (presented via speakers: up, down, left, or right). We varied workload by adjusting matrix size (3 × 3 vs. 5 × 5), manipulated availability of a spatial frame of reference by presenting either a blank screen (requiring participants to rely solely on their mental representation of the matrix) or spatial reference in the form of an empty matrix, and contrasted active task performance to two control conditions involving only active or passive listening. Our findings show that eye movements consistently matched the imagined movement of the patch in the matrix, not driven solely by auditory or semantic cues. While workload influenced pupil diameter, perceived demand, and performance, it had no observable impact on internal coupling. The availability of spatial reference enhanced coupling of eye movements, leading more frequent, precise, and resilient saccades against noise and bias. The absence of workload effects on coupled saccades in our study, in combination with the relatively high degree of coupling observed even in the invisible matrix condition, indicates that eye movements align with shifts in attention across both visually and internally represented information. This suggests that coupled eye movements are not merely strategic efforts to reduce workload, but rather a natural response to where attention is directed.

Decoupling of the pupillary light response during internal attention: The modulating effect of luminance intensity.

In a world full of sensory stimuli, attention guides us between the external environment and our internal thoughts. While external attention involves processing sensory stimuli, internal attention is devoted to self-generated representations such as planning or spontaneous mind wandering. They both draw from common cognitive resources, thus simultaneous engagement in both often leads to interference between processes. In order to maintain internal focus, an attentional mechanism known as perceptual decoupling takes effect. This mechanism supports internal cognition by decoupling attention from the perception of sensory information. Two previous studies of our lab investigated to what extent perceptual decoupling is evident in voluntary eye movements. Findings showed that the effect is mediated by the internal task modality and workload (visuospatial > arithmetic and high > low, respectively). However, it remains unclear whether it extends to involuntary eye behavior, which may not share cognitive resources with internal activities. Therefore, the present experiment aimed to further elucidate attentional dynamics by examining whether internal attention affects the pupillary light response (PLR). Specifically, we consistently observed that workload and task modality of the internal task reduced the PLR to luminance changes of medium intensity. However, the PLR to strong luminance changes was less or not at all affected by the internal task. These results suggest that perceptual decoupling effects may be less consistent in involuntary eye behavior, particularly in the context of a salient visual stimulus.

The effects of type and workload of internal tasks on voluntary saccades in a target-distractor saccade task.

When we engage in internally directed cognition, like doing mental arithmetic or mind wandering, fewer cognitive resources are assigned for other activities like reacting to perceptual input-an effect termed perceptual decoupling. However, the exact conditions under which perceptual decoupling occurs and its underlying cognitive mechanisms are still unclear. Hence, the present study systematically manipulated the task type (arithmetic, visuospatial) and workload (control, low, high) of the internal task in a within-subject design and tested its effects on voluntary saccades in a target-distractor saccade task. As expected, engagement in internal tasks delayed saccades to the target. This effect was moderated by time, task, and workload: The delay was largest right after internal task onset and then decreased, potentially reflecting the intensity of internal task demands. Saccades were also more delayed for the high compared to the low workload condition in the arithmetic task, whereas workload conditions had similarly high effects in the visuospatial task. Findings suggests that perceptual decoupling of eye behavior gradually increases with internal demands on general resources and that perceptual decoupling is specifically sensitive to internal demands on visuospatial resources. The latter may be mediated by interference due to eye behavior elicited by the internal task itself. Internal tasks did not affect the saccade latency-deviation trade-off, indicating that while the internal tasks delayed the execution of the saccade, the perception of the saccade stimuli and spatial planning of the saccade continued unaffected in parallel to the internal tasks. Together, these findings shed further light on the specific mechanisms underlying perceptual decoupling by suggesting that perceptual decoupling of eye behavior increases as internal demands on cognitive resources overlap more strongly with demands of the external task.

Effects of internally directed cognition on smooth pursuit eye movements: A systematic examination of perceptual decoupling.

Eye behavior differs between internally and externally directed cognition and thus is indicative of an internal versus external attention focus. Recent work implicated perceptual decoupling (i.e., eye behavior becoming less determined by the sensory environment) as one of the key mechanisms involved in these attention-related eye movement differences. However, it is not yet understood how perceptual decoupling depends on the characteristics of the internal task. Therefore, we systematically examined effects of varying internal task demands on smooth pursuit eye movements. Specifically, we evaluated effects of the internal workload (control vs. low vs. high) and of internal task (arithmetic vs. visuospatial). The results of multilevel modelling showed that effects of perceptual decoupling were stronger for higher workload, and more pronounced for the visuospatial modality. Effects also followed a characteristic time-course relative to internal operations. The findings provide further support of the perceptual decoupling mechanism by showing that it is sensitive to the degree of interference between external and internal information.

Neurophysiological indicators of internal attention: An fMRI-eye-tracking coregistration study.

Many goal-directed, as well as spontaneous everyday activities (e.g., planning, mind-wandering), rely on an internal focus of attention. This fMRI-eye-tracking coregistration study investigated brain mechanisms and eye behavior related to internally versus externally directed cognition. Building on an established paradigm, we manipulated internal attention demands within tasks utilizing conditional stimulus masking. Internally directed cognition involved bilateral activation of the lingual gyrus and inferior parietal lobe areas as well as wide-spread deactivation of visual networks. Moreover, internally directed cognition was related to greater pupil diameter, pupil diameter variance, blink duration, fixation disparity variance, and smaller amounts of microsaccades. FMRI-eye-tracking covariation analyses further revealed that larger pupil diameter was related to increased activation of basal ganglia and lingual gyrus. It can be concluded that internally and externally directed cognition are characterized by distinct neurophysiological signatures. The observed neurophysiological differences indicate that internally directed cognition is associated with reduced processing of task-irrelevant information and increased mental load. These findings shed further light on the interplay between neural and perceptual mechanisms contributing to an internal focus of attention.

Imaging Time Series of Eye Tracking Data to Classify Attentional States.

It has been shown that conclusions about the human mental state can be drawn from eye gaze behavior by several previous studies. For this reason, eye tracking recordings are suitable as input data for attentional state classifiers. In current state-of-the-art studies, the extracted eye tracking feature set usually consists of descriptive statistics about specific eye movement characteristics (i.e., fixations, saccades, blinks, vergence, and pupil dilation). We suggest an Imaging Time Series approach for eye tracking data followed by classification using a convolutional neural net to improve the classification accuracy. We compared multiple algorithms that used the one-dimensional statistical summary feature set as input with two different implementations of the newly suggested method for three different data sets that target different aspects of attention. The results show that our two-dimensional image features with the convolutional neural net outperform the classical classifiers for most analyses, especially regarding generalization over participants and tasks. We conclude that current attentional state classifiers that are based on eye tracking can be optimized by adjusting the feature set while requiring less feature engineering and our future work will focus on a more detailed and suited investigation of this approach for other scenarios and data sets.

How Reliably Do Eye Parameters Indicate Internal Versus External Attentional Focus?

Eye behavior is increasingly used as an indicator of internal versus external focus of attention both in research and application. However, available findings are partly inconsistent, which might be attributed to the different nature of the employed types of internal and external cognition tasks. The present study, therefore, investigated how consistently different eye parameters respond to internal versus external attentional focus across three task modalities: numerical, verbal, and visuo-spatial. Three eye parameters robustly differentiated between internal and external attentional focus across all tasks. Blinks, pupil diameter variance, and fixation disparity variance were consistently increased during internally directed attention. We also observed substantial attentional focus effects on other parameters (pupil diameter, fixation disparity, saccades, and microsaccades), but they were moderated by task type. Single-trial analysis of our data using machine learning techniques further confirmed our results: Classifying the focus of attention by means of eye tracking works well across participants, but generalizing across tasks proves to be challenging. Based on the effects of task type on eye parameters, we discuss what eye parameters are best suited as indicators of internal versus external attentional focus in different settings.

Neurophysiological indicators of internal attention: An electroencephalography-eye-tracking coregistration study.

INTRODUCTION: Many goal-directed and spontaneous everyday activities (e.g., planning, mind wandering) rely on an internal focus of attention. Internally directed cognition (IDC) was shown to differ from externally directed cognition in a range of neurophysiological indicators such as electroencephalogram (EEG) alpha activity and eye behavior. METHODS: In this EEG-eye-tracking coregistration study, we investigated effects of attention direction on EEG alpha activity and various relevant eye parameters. We used an established paradigm to manipulate internal attention demands in the visual domain within tasks by means of conditional stimulus masking. RESULTS: Consistent with previous research, IDC involved relatively higher EEG alpha activity (lower alpha desynchronization) at posterior cortical sites. Moreover, IDC was characterized by greater pupil diameter (PD), fewer microsaccades, fixations, and saccades. These findings show that internal versus external cognition is associated with robust differences in several indicators at the neural and perceptual level. In a second line of analysis, we explored the intrinsic temporal covariation between EEG alpha activity and eye parameters during rest. This analysis revealed a positive correlation of EEG alpha power with PD especially in bilateral parieto-occipital regions. CONCLUSION: Together, these findings suggest that EEG alpha activity and PD represent time-sensitive indicators of internal attention demands, which may be involved in a neurophysiological gating mechanism serving to shield internal cognition from irrelevant sensory information.

Eye behavior predicts susceptibility to visual distraction during internally directed cognition.

When we engage in internally directed cognition (e.g., planning or imagination), our eye behavior decouples from external stimuli and couples to internal representations (e.g., internal visualizations of ideas). Here, we investigated whether eye behavior predicts the susceptibility to visual distraction during internally directed cognition. To this end, participants performed a divergent thinking task, which required internally directed attention, and we measured distraction in terms of attention capture by unrelated images. We used multilevel mixed models to predict visual distraction by eye behavior right before distractor onset. In Study 1 (N = 38), visual distraction was predicted by increased saccade and blink rate, and higher pupil dilation. We replicated these findings in Study 2 using the same task, but with less predictable distractor onsets and a larger sample (N = 144). We also explored whether individual differences in susceptibility to visual distraction were related to cognitive ability and task performance. Taken together, variation in eye behavior was found to be a consistent predictor of visual distraction during internally directed cognition. This highlights the relevance of eye parameters as objective indicators of internal versus external attentional focus and distractibility during complex mental tasks.

Are you with me? Probing the human capacity to recognize external/internal attention in others' faces.

Facial cues provide information about affective states and the direction of attention that is important for human social interaction. The present study examined how this capacity extends to judging whether attention is internally or externally directed. Participants evaluated a set of videos and images showing the face of people focused externally on a task, or internally while they performed a task in imagination. We found that participants could identify the focus of attention above chance in videos, and to a lesser degree in static images, but only when the eye region was visible. Self-reports further indicated that participants relied particularly on the eye region in their judgements. Interestingly, people engaged in demanding cognitive tasks were more likely judged to be externally focused independent of the actual focus of attention. These findings demonstrate that humans use information from the face and especially from the eyes of others not only to infer external goals or actions, but also to detect when others focus internally on their own thoughts and feelings.

Eye behavior does not adapt to expected visual distraction during internally directed cognition.

When focused on a specific internal task like calculating a multiplication in mind we are able to ignore sensory distraction. This may be achieved by effective perceptual decoupling during internally directed cognition. The present study investigated whether decoupling from external events during internally directed cognition represents an active shielding mechanism that adapts to expected external distraction or a passive/automatic shielding mechanism that is independent of external distraction. Participants performed multiplications in mind (e.g. 26 x 7), a task that required to turn attention inward as soon as the problem was encoded. At the beginning of a block of trials, participants were informed whether or not distractors could appear during the calculation period, thereby potentially allowing them to prepare for the distractors. We tracked their eye behavior as markers of perceptual decoupling and workload. Turning attention inward to calculate the multiplication elicited evidence of perceptual decoupling for five of six eye parameters: blink rate, saccade and microsaccade rate increased, gaze was less constricted to the center, and pupils dilated. Although participants perceived blocks with distractors as more challenging, performance and eye behavior markers of both perceptual decoupling and workload were unaffected. This result supports the notion of perceptual decoupling as an automatic mechanism: focusing inward induces desensitization to external events independent of external distraction.

Data on eye behavior during idea generation and letter-by-letter reading.

This article includes the description of data information from an idea generation task (alternate uses task, (Guilford, 1967) [1]) and a letter-by-letter reading task under two background brightness conditions with healthy adults as well as a baseline measurement and questionnaire data (SIPI (Huba et al., 1981) [2]; DDFS (Singer and Antrobus, 1972) [3], 1963; RIBS (Runco et al., 2001) [4]). Data are hosted at the Open Science Framework (OSF): https://osf.io/fh66g/ (Walcher et al., 2017) [5]. There you will find eye tracking data, task performance data, questionnaires data, analyses scripts (in R, R Core Team, 2017 [6]), eye tracking paradigms (in the Experiment Builder (SR Research Ltd., [7]) and graphs on pupil and angle of eye vergence dynamics. Data are interpreted and discussed in the article 'Looking for ideas: Eye behavior during goal-directed internally focused cognition' (Walcher et al., 2017) [8].

Eye Behavior Associated with Internally versus Externally Directed Cognition.

What do our eyes do when we are focused on internal representations such as during imagination or planning? Evidence from mind wandering research suggests that spontaneous shifts from externally directed cognition (EDC) to internally directed cognition (IDC) involves oculomotor changes indicative of visual disengagement. In the present study, we investigated potential differences in eye behavior between goal-directed forms of IDC and EDC. To this end, we manipulated the focus of attention (internal versus external) in two demanding cognitive tasks (anagram and sentence generation). IDC was associated with fewer and longer fixations and higher variability in pupil diameter and eye vergence compared to EDC, suggesting reduced visual scanning and higher spontaneous eye activity. IDC was further related to longer blinks, lower microsaccade frequency, and a lower angle of eye vergence. These latter changes appear conducive to attenuate visual input and thereby shield ongoing internal processes from external distraction. Together, these findings suggest that IDC is accompanied by characteristic eye behavior that reflects a decoupling of attention from external events and serves gating out visual input.

Looking for ideas: Eye behavior during goal-directed internally focused cognition.

Humans have a highly developed visual system, yet we spend a high proportion of our time awake ignoring the visual world and attending to our own thoughts. The present study examined eye movement characteristics of goal-directed internally focused cognition. Deliberate internally focused cognition was induced by an idea generation task. A letter-by-letter reading task served as external task. Idea generation (vs. reading) was associated with more and longer blinks and fewer microsaccades indicating an attenuation of visual input. Idea generation was further associated with more and shorter fixations, more saccades and saccades with higher amplitudes as well as heightened stimulus-independent variation of eye vergence. The latter results suggest a coupling of eye behavior to internally generated information and associated cognitive processes, i.e. searching for ideas. Our results support eye behavior patterns as indicators of goal-directed internally focused cognition through mechanisms of attenuation of visual input and coupling of eye behavior to internally generated information.