Ambient and focal attention during complex problem-solving: preliminary evidence from real-world eye movement data.

Time course analysis of eye movements during free exploration of real-world scenes often reveals an increase in fixation durations together with a decrease in saccade amplitudes, which has been explained within the two visual systems approach, i.e., a transition from ambient to focal. Short fixations and long saccades during early viewing periods are classified as ambient mode of vision, which is concerned with spatial orientation and is related to simple visual properties such as motion, contrast, and location. Longer fixations and shorter saccades during later viewing periods are classified as focal mode of vision, which is concentrated in the foveal projection and is capable of object identification and its semantic categorization. While these findings are mainly obtained in the context of image exploration, the present study endeavors to investigate whether the same pattern of interplay between ambient and focal visual attention is deployed when people work on complex real-world tasks-and if so, when? Based on a re-analysis of existing data that integrates concurrent think aloud and eye tracking protocols, the present study correlated participants' internal thinking models to the parameters of their eye movements when they planned solutions to an open-ended design problem in a real-world setting. We hypothesize that switching between ambient and focal attentional processing is useful when solvers encounter difficulty compelling them to shift their conceptual direction to adjust the solution path. Individuals may prefer different attentional strategies for information-seeking behavior, such as ambient-to-focal or focal-to-ambient. The observed increase in fixation durations and decrease in saccade amplitudes during the periods around shifts in conceptual direction lends support to the postulation of the ambient-to-focal processing; however, focal-to-ambient processing is not evident. Furthermore, our data demonstrate that the beginning of a shift in conceptual direction is observable in eye movement behavior with a significant prolongation of fixation. Our findings add to the conclusions drawn from laboratory settings by providing preliminary evidence for ambient and focal processing characteristics in real-world problem-solving.

Impairment of oculomotor functions in patients with early to advanced amyotrophic lateral sclerosis.

Amyotrophic lateral sclerosis (ALS) can result into an incomplete locked in state (iLIS), in which communication depends on eye tracking computer devices. Oculomotor function impairments in ALS have been reported, but there is little research, particularly with respect to patients in iLIS. In the present study, we compared reflexive and executive oculomotor function by means of an eye tracking test battery between three groups: advanced ALS patients in iLIS (n = 22), patients in early to middle ALS stages (n = 44) and healthy subjects (n = 32). Patients with ALS showed significant deteriorations in oculomotor functions, with stronger impairments in iLIS. More specifically, ALS patients produced visually guided prosaccades with longer latencies and more frequent hypometria compared to healthy subjects. Longest latencies were obtained in iLIS patients, with a stronger prolongation for vertical than for horizontal prosaccades. ALS patients made more antisaccade errors and generated antisaccades with longer latencies. Smooth pursuit was also impaired in ALS. In the earlier ALS stages, bulbar onset patients presented stronger antisaccade and smooth pursuit deficits than spinal onset patients. Our findings reveal a relevant deterioration of important oculomotor functions in ALS, which increases in iLIS. It includes impairments of reflexive eye movements to loss of executive inhibitory control, indicating a progressing pathological involvement of prefrontal, midbrain and brainstem areas. The assessment of oculomotor functions may therefore provide clinically relevant bio- and progression marker, particularly in advanced ALS.

Assessment of the human response to acute mental stress-An overview and a multimodal study.

Numerous vital signs are reported in association with stress response assessment, but their application varies widely. This work provides an overview over methods for stress induction and strain assessment, and presents a multimodal experimental study to identify the most important vital signs for effective assessment of the response to acute mental stress. We induced acute mental stress in 65 healthy participants with the Mannheim Multicomponent Stress Test and acquired self-assessment measures (Likert scale, Self-Assessment Manikin), salivary α-amylase and cortisol concentrations as well as 60 vital signs from biosignals, such as heart rate variability parameters, QT variability parameters, skin conductance level, and breath rate. By means of statistical testing and a self-optimizing logistic regression, we identified the most important biosignal vital signs. Fifteen biosignal vital signs related to ventricular repolarization variability, blood pressure, skin conductance, and respiration showed significant results. The logistic regression converged with QT variability index, left ventricular work index, earlobe pulse arrival time, skin conductance level, rise time and number of skin conductance responses, breath rate, and breath rate variability (F1 = 0.82). Self-assessment measures indicated successful stress induction. α-amylase and cortisol showed effect sizes of -0.78 and 0.55, respectively. In summary, the hypothalamic-pituitary-adrenocortical axis and sympathetic nervous system were successfully activated. Our findings facilitate a coherent and integrative understanding of the assessment of the stress response and help to align applications and future research concerning acute mental stress.

Eye movement patterns in complex tasks: Characteristics of ambient and focal processing.

Analyzing the time course of eye movements during scene viewing often indicates that people progress through two distinct modes of visual processing: an ambient mode, which is associated with overall spatial orientation in a scene, followed by a focal mode, which requires central vision of an object. However, the shifts between ambient and focal processing modes have mainly been identified relative to changes in the environment, such as relative to the onset of various visual stimuli but also following scene cuts or subjective event boundaries in dynamic stimuli. The results so far do not allow conclusions about the nature of the two processing mechanisms beyond the influence of externally triggered events. It remains unclear whether people shift back and forth from ambient to focal processing also based on internal triggers, such as switching between different tasks while no external event is given. The present study therefore investigated ambient to focal processing shifts in an active task solving paradigm. The Rubik's Cube task introduced here is a multi-step task, which can be broken down into smaller sub-tasks that are performed serially. The time course of eye movements was analyzed at multiple levels of this Rubik's Cube task, including when there were no external changes to the stimuli but when internal representations of the task were hypothesized to change (i.e., switching between different sub-tasks). Results suggest that initial ambient exploration is followed by a switch to more focal viewing across various levels of task processing with and without external changes to the stimuli. More importantly, the present findings suggest that ambient and focal eye movement characteristics might serve as a probe for the attentional state in task processing, which does not seem to be influenced by changes in task performance.

Quality of life and mental health in the locked-in-state-differences between patients with amyotrophic lateral sclerosis and their next of kin.

For both patients with amyotrophic lateral sclerosis (ALS) and their next of kin (NOK), the maintenance of quality of life (QoL) and mental health is particularly important. First studies suggest significant discrepancies between QoL reports by patients and NOK, but little is known for advanced ALS stages. To address this issue, we screened 52 ALS patients in incomplete locked-in state (iLIS). Final results were obtained for 15 couples of iLIS patients and NOK. We assessed patients' and NOK's subjective QoL, depression and anxiety and NOK's caregiver burden. Gaze controlled questionnaires allowed direct assessment of patients. Patients and NOK self-reported comparable, mostly moderate to high levels of QoL. Of note, NOK indicated stronger anxiety symptoms. Higher anxiety levels in NOK were associated with stronger caregiver burden and reduced QoL. No significant misjudgment of patient's QoL by the NOK was evident, while patients overestimated NOK's global QoL. However, NOK with severe caregiver burden and depression symptoms gave poorer estimations of patients' QoL. This relationship is relevant, considering NOK's impact on life critical treatment decisions. While the daily time NOK and patient spend together was positively correlated with NOK's QoL and mental health, this was not reversely found for the patients. Our results suggest that NOK adapt less successfully to the disease and concomitant experience of loss and point to an urgent need for specialized psychosocial support. The findings emphasize the importance of direct psychological wellbeing assessment of both patients and NOK in clinical practice, enabled by eye-tracking technology for patients in iLIS.

Investigating the utility of VR for spatial understanding in surgical planning: evaluation of head-mounted to desktop display.

Recent technological advances have made Virtual Reality (VR) attractive in both research and real world applications such as training, rehabilitation, and gaming. Although these other fields benefited from VR technology, it remains unclear whether VR contributes to better spatial understanding and training in the context of surgical planning. In this study, we evaluated the use of VR by comparing the recall of spatial information in two learning conditions: a head-mounted display (HMD) and a desktop screen (DT). Specifically, we explored (a) a scene understanding and then (b) a direction estimation task using two 3D models (i.e., a liver and a pyramid). In the scene understanding task, participants had to navigate the rendered the 3D models by means of rotation, zoom and transparency in order to substantially identify the spatial relationships among its internal objects. In the subsequent direction estimation task, participants had to point at a previously identified target object, i.e., internal sphere, on a materialized 3D-printed version of the model using a tracked pointing tool. Results showed that the learning condition (HMD or DT) did not influence participants' memory and confidence ratings of the models. In contrast, the model type, that is, whether the model to be recalled was a liver or a pyramid significantly affected participants' memory about the internal structure of the model. Furthermore, localizing the internal position of the target sphere was also unaffected by participants' previous experience of the model via HMD or DT. Overall, results provide novel insights on the use of VR in a surgical planning scenario and have paramount implications in medical learning by shedding light on the mental model we make to recall spatial structures.

Dispositional individual differences in cognitive effort investment: establishing the core construct.

BACKGROUND: Individuals tend to avoid effortful tasks, regardless of whether they are physical or mental in nature. Recent experimental evidence is suggestive of individual differences in the dispositional willingness to invest cognitive effort in goal-directed behavior. The traits need for cognition (NFC) and self-control are related to behavioral measures of cognitive effort discounting and demand avoidance, respectively. Given that these traits are only moderately related, the question arises whether they reflect a common core factor underlying cognitive effort investment. If so, the common core of both traits might be related to behavioral measures of effort discounting in a more systematic fashion. To address this question, we aimed at specifying a core construct of cognitive effort investment that reflects dispositional differences in the willingness and tendency to exert effortful control. METHODS: We conducted two studies (N = 613 and N = 244) with questionnaires related to cognitive motivation and effort investment including assessment of NFC, intellect, self-control and effortful control. We first calculated Pearson correlations followed by two mediation models regarding intellect and its separate aspects, seek and conquer, as mediators. Next, we performed a confirmatory factor analysis of a hierarchical model of cognitive effort investment as second-order latent variable. First-order latent variables were cognitive motivation reflecting NFC and intellect, and effortful self-control reflecting self-control and effortful control. Finally, we calculated Pearson correlations between factor scores of the latent variables and general self-efficacy as well as traits of the Five Factor Model of Personality for validation purposes. RESULTS: Our findings support the hypothesized correlations between the assessed traits, where the relationship of NFC and self-control is specifically mediated via goal-directedness. We established and replicated a hierarchical factor model of cognitive motivation and effortful self-control that explains the shared variance of the first-order factors by a second-order factor of cognitive effort investment. CONCLUSIONS: Taken together, our results integrate disparate literatures on cognitive motivation and self-control and provide a basis for further experimental research on the role of dispositional individual differences in goal-directed behavior and cost-benefit-models.

Stochastic Motion Stimuli Influence Perceptual Choices in Human Participants.

In the study of perceptual decision making, it has been widely assumed that random fluctuations of motion stimuli are irrelevant for a participant's choice. Recently, evidence was presented that these random fluctuations have a measurable effect on the relationship between neuronal and behavioral variability, the so-called choice probability. Here, we test, in a behavioral experiment, whether stochastic motion stimuli influence the choices of human participants. Our results show that for specific stochastic motion stimuli, participants indeed make biased choices, where the bias is consistent over participants. Using a computational model, we show that this consistent choice bias is caused by subtle motion information contained in the motion noise. We discuss the implications of this finding for future studies of perceptual decision making. Specifically, we suggest that future experiments should be complemented with a stimulus-informed modeling approach to control for the effects of apparent decision evidence in random stimuli.

Dispositional cognitive effort investment and behavioral demand avoidance: Are they related?

Individuals tend to avoid cognitive demand, yet, individual differences appear to exist. Recent evidence from two studies suggests that individuals high in the personality traits Self-Control and Need for Cognition that are related to the broader construct Cognitive Effort Investment are less prone to avoid cognitive demand and show less effort discounting. These findings suggest that cost-benefit models of decision-making that integrate the costs due to effort should consider individual differences in the willingness to exert mental effort. However, to date, there are almost no replication attempts of the above findings. For the present conceptual replication, we concentrated on the avoidance of cognitive demand and used a longitudinal design and latent state-trait modeling. This approach enabled us to separate the trait-specific variance in our measures of Cognitive Effort Investment and Demand Avoidance that is due to stable, individual differences from the variance that is due to the measurement occasion, the methods used, and measurement error. Doing so allowed us to test the assumption that self-reported Cognitive Effort Investment is related to behavioral Demand Avoidance more directly by relating their trait-like features to each other. In a sample of N = 217 participants, we observed both self-reported Cognitive Effort Investment and behavioral Demand Avoidance to exhibit considerable portions of trait variance. However, these trait variances were not significantly related to each other. Thus, our results call into question previous findings of a relationship between self-reported effort investment and demand avoidance. We suggest that novel paradigms are needed to emulate real-world effortful situations and enable better mapping between self-reported measures and behavioral markers of the willingness to exert cognitive effort.

Congenital Deafness Leads to Altered Overt Oculomotor Behaviors.

The human brain is highly cross-modal, and sensory information may affect a wide range of behaviors. In particular, there is evidence that auditory functions are implicated in oculomotor behaviors. Considering this apparent auditory-oculomotor link, one might wonder how the loss of auditory input from birth might have an influence on these motor behaviors. Eye movement tracking enables to extract several components, including saccades and smooth pursuit. One study suggested that deafness can alter saccades processing. Oculomotor behaviors have not been examined further in the deaf. The main goal of this study was to examine smooth pursuit following deafness. A pursuit task paradigm was used in this experiment. Participants were instructed to move their eyes to follow a target as it moved. The target movements have a possibility of four different trajectories (horizontal, vertical, elliptic clockwise, and elliptic counter-clockwise). Results indicate a significant reduction in the ability to track a target in both elliptical conditions showing that more complex motion processing differs in deaf individuals. The data also revealed significantly more saccades per trial in the vertical, anti-clockwise, and, to a lesser extent, the clockwise elliptic condition. This suggests that auditory deprivation from birth leads to altered overt oculomotor behaviors.

Size matters: How reaching and vergence movements are influenced by the familiar size of stereoscopically presented objects.

The knowledge about the usual size of objects-familiar size-is known to be a taken into account for distance perception. The influence of familiar size on action programming is less clear and has not yet been tested with regard to vergence eye movements. In two experiments, we stereoscopically presented everyday objects, such as a credit card or a package of paper tissues, and varied the distance as specified by binocular disparity and the distance as specified by familiar size. Participants had to fixate the shown object and subsequently reach towards it either with open or with closed eyes. When binocular disparity and familiar size were in conflict, reaching movements revealed a combination of the two depth cues with individually different weights. The influence of familiar size was larger when no visual feedback was available during the reaching movement. Vergence movements closely followed binocular disparity and were largely unaffected by familiar size. In sum, the results suggest that in this experimental setting familiar size is taken into account for programming and executing reaching movements while vergence movements are primarily based on binocular disparity.

VisME: Visual Microsaccades Explorer.

This work presents a visual analytics approach to explore microsaccade distributions in high-frequency eye tracking data. Research studies often apply filter algorithms and parameter values for microsaccade detection. Even when the same algorithms are employed, different parameter values might be adopted across different studies. In this paper, we present a visual analytics system (VisME) to promote reproducibility in the data analysis of microsaccades. It allows users to interactively vary the parametric values for microsaccade filters and evaluate the resulting influence on microsaccade behavior across individuals and on a group level. In particular, we exploit brushing-and-linking techniques that allow the microsaccadic properties of space, time, and movement direction to be extracted, visualized, and compared across multiple views. We demonstrate in a case study the use of our visual analytics system on data sets collected from natural scene viewing and show in a qualitative usability study the usefulness of this approach for eye tracking researchers. We believe that interactive tools such as VisME will promote greater transparency in eye movement research by providing researchers with the ability to easily understand complex eye tracking data sets; such tools can also serve as teaching systems. VisME is provided as open source software.

Gaze3DFix: Detecting 3D fixations with an ellipsoidal bounding volume.

Nowadays, the use of eyetracking to determine 2-D gaze positions is common practice, and several approaches to the detection of 2-D fixations exist, but ready-to-use algorithms to determine eye movements in three dimensions are still missing. Here we present a dispersion-based algorithm with an ellipsoidal bounding volume that estimates 3D fixations. Therefore, 3D gaze points are obtained using a vector-based approach and are further processed with our algorithm. To evaluate the accuracy of our method, we performed experimental studies with real and virtual stimuli. We obtained good congruence between stimulus position and both the 3D gaze points and the 3D fixation locations within the tested range of 200-600 mm. The mean deviation of the 3D fixations from the stimulus positions was 17 mm for the real as well as for the virtual stimuli, with larger variances at increasing stimulus distances. The described algorithms are implemented in two dynamic linked libraries (Gaze3D.dll and Fixation3D.dll), and we provide a graphical user interface (Gaze3DFixGUI.exe) that is designed for importing 2-D binocular eyetracking data and calculating both 3D gaze points and 3D fixations using the libraries. The Gaze3DFix toolkit, including both libraries and the graphical user interface, is available as open-source software at https://github.com/applied-cognition-research/Gaze3DFix .

Influence of semantic consistency and perceptual features on visual attention during scene viewing in toddlers.

Conceptual representations of everyday scenes are built in interaction with visual environment and these representations guide our visual attention. Perceptual features and object-scene semantic consistency have been found to attract our attention during scene exploration. The present study examined how visual attention in 24-month-old toddlers is attracted by semantic violations and how perceptual features (i. e. saliency, centre distance, clutter and object size) and linguistic properties (i. e. object label frequency and label length) affect gaze distribution. We compared eye movements of 24-month-old toddlers and adults while exploring everyday scenes which either contained an inconsistent (e.g., soap on a breakfast table) or consistent (e.g., soap in a bathroom) object. Perceptual features such as saliency, centre distance and clutter of the scene affected looking times in the toddler group during the whole viewing time whereas looking times in adults were affected only by centre distance during the early viewing time. Adults looked longer to inconsistent than consistent objects either if the objects had a high or a low saliency. In contrast, toddlers presented semantic consistency effect only when objects were highly salient. Additionally, toddlers with lower vocabulary skills looked longer to inconsistent objects while toddlers with higher vocabulary skills look equally long to both consistent and inconsistent objects. Our results indicate that 24-month-old children use scene context to guide visual attention when exploring the visual environment. However, perceptual features have a stronger influence in eye movement guidance in toddlers than in adults. Our results also indicate that language skills influence cognitive but not perceptual guidance of eye movements during scene perception in toddlers.

Visual information representation and rapid-scene categorization are simultaneous across cortex: An MEG study.

Perceiving the visual world around us requires the brain to represent the features of stimuli and to categorize the stimulus based on these features. Incorrect categorization can result either from errors in visual representation or from errors in processes that lead to categorical choice. To understand the temporal relationship between the neural signatures of such systematic errors, we recorded whole-scalp magnetoencephalography (MEG) data from human subjects performing a rapid-scene categorization task. We built scene category decoders based on (1) spatiotemporally resolved neural activity, (2) spatial envelope (SpEn) image features, and (3) behavioral responses. Using confusion matrices, we tracked how well the pattern of errors from neural decoders could be explained by SpEn decoders and behavioral errors, over time and across cortical areas. Across the visual cortex and the medial temporal lobe, we found that both SpEn and behavioral errors explained unique variance in the errors of neural decoders. Critically, these effects were nearly simultaneous, and most prominent between 100 and 250ms after stimulus onset. Thus, during rapid-scene categorization, neural processes that ultimately result in behavioral categorization are simultaneous and co-localized with neural processes underlying visual information representation.

Eye movement patterns and visual attention during scene viewing in 3- to 12-month-olds.

Recently, two attentional modes have been associated with specific eye movement patterns during scene processing. Ambient mode, characterized by short fixations and long saccades during early scene inspection, is associated with localization of objects. Focal mode, characterized by longer fixations, is associated with more detailed object feature processing during later inspection phase. The aim of the present study was to investigate the development of these attentional modes. More specifically, we examined whether indications of ambient and focal attention modes are similar in infants and adults. Therefore, we measured eye movements in 3- to 12-months-old infants while exploring visual scenes. Our results show that both adults and 12-month-olds had shorter fixation durations within the first 1.5 s of scene viewing compared with later time phases (>2.5 s); indicating that there was a transition from ambient to focal processing during image inspection. In younger infants, fixation durations between two viewing phases did not differ. Our results suggest that at the end of the first year of life, infants have developed an adult-like scene viewing behavior. The evidence for the existence of distinct attentional processing mechanisms during early infancy furthermore underlines the importance of the concept of the two modes.

Social communication with virtual agents: The effects of body and gaze direction on attention and emotional responding in human observers.

In social communication, the gaze direction of other persons provides important information to perceive and interpret their emotional response. Previous research investigated the influence of gaze by manipulating mutual eye contact. Therefore, gaze and body direction have been changed as a whole, resulting in only congruent gaze and body directions (averted or directed) of another person. Here, we aimed to disentangle these effects by using short animated sequences of virtual agents posing with either direct or averted body or gaze. Attention allocation by means of eye movements, facial muscle response, and emotional experience to agents of different gender and facial expressions were investigated. Eye movement data revealed longer fixation durations, i.e., a stronger allocation of attention, when gaze and body direction were not congruent with each other or when both were directed towards the observer. This suggests that direct interaction as well as incongruous signals increase the demands of attentional resources in the observer. For the facial muscle response, only the reaction of muscle zygomaticus major revealed an effect of body direction, expressed by stronger activity in response to happy expressions for direct compared to averted gaze when the virtual character's body was directed towards the observer. Finally, body direction also influenced the emotional experience ratings towards happy expressions. While earlier findings suggested that mutual eye contact is the main source for increased emotional responding and attentional allocation, the present results indicate that direction of the virtual agent's body and head also plays a minor but significant role.

Effects of driving time on microsaccadic dynamics.

Driver fatigue is a common cause of car accidents. Thus, the objective detection of driver fatigue is a first step toward the effective management of fatigue-related traffic accidents. Here, we investigated the effects of driving time, a common inducer of driver fatigue, on the dynamics of fixational eye movements. Participants drove for 2 h in a virtual driving environment while we recorded their eye movements. Microsaccade velocities decreased with driving time, suggesting a potential effect of fatigue on microsaccades during driving.

The maturation of eye movement behavior: scene viewing characteristics in children and adults.

While the close link between eye movements and visual attention has often been demonstrated, recently distinct attentional modes have been associated with specific eye movement patterns. The ambient mode-serving the localization of objects and dominating early scene inspection-is expressed by short fixations and large saccade amplitudes. The focal mode-associated with the identification of object details and dominating later stages of scene exploration-is indicated by longer fixations embedded in short saccades. The relationship between these processing modes and eye movement characteristics has so far only been examined in adults. While studies in children revealed a maturation of oculomotor behavior up to adolescence, developmental aspects of the processing modes are still unknown. Here we explored these mechanisms by comparing eye movements during the inspection of naturalistic scenes. Therefore, gaze behavior from adults and children in four different age groups (2, 4-6, 6-8, 8-10years old) was examined. We found a general effect of age, revealing that with age fixation durations decrease and saccade amplitudes increase. However, in all age groups fixations were shorter and saccades were longer at the beginning of scene inspection but fixations became longer and saccades became shorter over time. While saliency influenced eye guidance in the two youngest groups over the full inspection period, for the older groups this influence was found only at the beginning of scene inspection. The results reveal indications for ambient and focal processing strategies for as early as 2 years of age.

Limitations of gaze transfer: without visual context, eye movements do not to help to coordinate joint action, whereas mouse movements do.

Remote cooperation can be improved by transferring the gaze of one participant to the other. However, based on a partner's gaze, an interpretation of his communicative intention can be difficult. Thus, gaze transfer has been inferior to mouse transfer in remote spatial referencing tasks where locations had to be pointed out explicitly. Given that eye movements serve as an indicator of visual attention, it remains to be investigated whether gaze and mouse transfer differentially affect the coordination of joint action when the situation demands an understanding of the partner's search strategies. In the present study, a gaze or mouse cursor was transferred from a searcher to an assistant in a hierarchical decision task. The assistant could use this cursor to guide his movement of a window which continuously opened up the display parts the searcher needed to find the right solution. In this context, we investigated how the ease of using gaze transfer depended on whether a link could be established between the partner's eye movements and the objects he was looking at. Therefore, in addition to the searcher's cursor, the assistant either saw the positions of these objects or only a grey background. When the objects were visible, performance and the number of spoken words were similar for gaze and mouse transfer. However, without them, gaze transfer resulted in longer solution times and more verbal effort as participants relied more strongly on speech to coordinate the window movement. Moreover, an analysis of the spatio-temporal coupling of the transmitted cursor and the window indicated that when no visual object information was available, assistants confidently followed the searcher's mouse but not his gaze cursor. Once again, the results highlight the importance of carefully considering task characteristics when applying gaze transfer in remote cooperation.