Novelty preference assessed by eye tracking: A sensitive measure of impaired recognition memory in epilepsy.

OBJECTIVE: Epilepsy patients often report memory deficits despite normal objective testing, suggesting that available measures are insensitive or that non-mnemonic factors are involved. The Visual Paired Comparison Task (VPCT) assesses novelty preference, the tendency to fixate on novel images rather than previously viewed items, requiring recognition memory for the "old" images. As novelty preference is a sensitive measure of hippocampal-dependent memory function, we predicted impaired VPCT performance in epilepsy patients compared to healthy controls. METHODS: We assessed 26 healthy adult controls and 31 epilepsy patients (16 focal-onset, 13 generalized-onset, 2 unknown-onset) with the VPCT using delays of 2 or 30 s between encoding and recognition. Fifteen healthy controls and 17 epilepsy patients (10 focal-onset, 5 generalized-onset, 2 unknown-onset) completed the task at 2-, 5-, and 30-minute delays. Subjects also performed standard memory measures, including the Medical College of Georgia (MCG) Paragraph Test, California Verbal Learning Test-Second Edition (CVLT-II), and Brief Visual Memory Test-Revised (BVMT-R). RESULTS: The epilepsy group was high functioning, with greater estimated IQ (p = 0.041), greater years of education (p = 0.034), and higher BVMT-R scores (p = 0.024) compared to controls. Both the control group and epilepsy cohort, as well as focal- and generalized-onset subgroups, had intact novelty preference at the 2- and 30-second delays (p-values ≤ 0.001) and declined at 30 min (p-values > 0.05). Only the epilepsy patients had early declines at 2- and 5-minute delays (controls with intact novelty preference at p = 0.003 and p ≤ 0.001, respectively; epilepsy groups' p-values > 0.05). CONCLUSIONS: Memory for the "old" items decayed more rapidly in overall, focal-onset, and generalized-onset epilepsy groups. The VPCT detected deficits while standard memory measures were largely intact, suggesting that the VPCT may be a more sensitive measure of temporal lobe memory function than standard neuropsychological batteries.

Do Children with High Callous-Unemotional Traits Have Attentional Deficits to Emotional Stimuli? Evidence from a Multi-Method and Multi-Informant Study.

Callous-unemotional (CU) traits in children and adolescents are linked to severe and persistent antisocial behavior. Based on past empirical research, several theoretical models have suggested that CU traits may be partly explained by difficulties in correctly identifying others' emotional states as well as their reduced attention to others' eyes, which could be important for both causal theory and treatment. This study tested the relationships among CU traits, emotion recognition of facial expressions and visual behavior in a sample of 52 boys referred to a clinic for conduct problems (Mage = 10.29 years; SD = 2.06). We conducted a multi-method and multi-informant assessment of CU traits through the Child Problematic Traits Inventory (CPTI), the Inventory of Callous-Unemotional (ICU), and the Clinical Assessment of Prosocial Emotions-Version 1.1 (CAPE). The primary goal of the study was to compare the utility of these methods for forming subgroups of youth that differ in their emotional processing abilities. An emotion recognition task assessed recognition accuracy (percentage of mistakes) and absolute dwell time on the eyes or mouth region for each emotion. Results from repeated measures ANOVAs revealed that low and high CU groups did not differ in emotion recognition accuracy, irrespective of the method of assessing CU traits. However, the high CU group showed reduced attention to the eyes of fearful and sad facial expressions (using the CPTI) or to all emotions (using the CAPE). The high CU group also showed a general increase in attention to the mouth area, but only when assessed by the CAPE. These findings provide evidence to support abnormalities in how those elevated on CU traits process emotional stimuli, especially when assessed by a clinical interview, which could guide appropriate assessment and more successful interventions for this group of youth.

Evaluating the Link between Visual Attention Bias and Emotion Dysregulation of Young Children.

The ability to regulate emotions is vital to successful social interactions. This study explores whether visual attention bias is associated with emotion dysregulation (ED) in early childhood. Parental reports of child ED (Child Behaviour Checklist (CBCL) and Temper Tantrum Scale) were examined in relation to child visual attention bias whilst viewing emotional faces. Results indicated that the level of eye gaze fixation towards emotional images and faces was associated with ED when social function (measured with the Social Responsiveness Scale), gender, age, and attention problems (measured from the CBCL subscale), were adjusted. The modifying effect on visual attention bias was evaluated using interaction analysis in the generalized linear model. The level of visual attention bias, indicated by the proportion of eye gaze fixation time on areas of interest (AOIs) in images displaying unpleasant emotions (such as anger), was inversely associated with the level of externalising problem behaviours (p = .014). Additionally, the association of eye gaze fixation time for AOIs displaying negative emotional cues with the level of externalising problem behaviours varied by age (p = .04), with younger children (aged < 70 months) demonstrating a stronger association than older children (aged ≥ 70 months). Findings suggest that young children with greater ED symptoms look less at unpleasant emotional cues. However, this relationship is attenuated as children become older. Further research to identify objective biomarkers that incorporate eye-tracking tasks may support prediction of ED-related mental health issues in the early years.

Maximizing valid eye-tracking data in human and macaque infants by optimizing calibration and adjusting areas of interest.

Remote eye tracking with automated corneal reflection provides insights into the emergence and development of cognitive, social, and emotional functions in human infants and non-human primates. However, because most eye-tracking systems were designed for use in human adults, the accuracy of eye-tracking data collected in other populations is unclear, as are potential approaches to minimize measurement error. For instance, data quality may differ across species or ages, which are necessary considerations for comparative and developmental studies. Here we examined how the calibration method and adjustments to areas of interest (AOIs) of the Tobii TX300 changed the mapping of fixations to AOIs in a cross-species longitudinal study. We tested humans (N = 119) at 2, 4, 6, 8, and 14 months of age and macaques (Macaca mulatta; N = 21) at 2 weeks, 3 weeks, and 6 months of age. In all groups, we found improvement in the proportion of AOI hits detected as the number of successful calibration points increased, suggesting calibration approaches with more points may be advantageous. Spatially enlarging and temporally prolonging AOIs increased the number of fixation-AOI mappings, suggesting improvements in capturing infants' gaze behaviors; however, these benefits varied across age groups and species, suggesting different parameters may be ideal, depending on the population studied. In sum, to maximize usable sessions and minimize measurement error, eye-tracking data collection and extraction approaches may need adjustments for the age groups and species studied. Doing so may make it easier to standardize and replicate eye-tracking research findings.

Gaze-Triggered Communicative Intention Compresses Perceived Temporal Duration.

Eye gaze communicates a person's attentional state and intentions toward objects. Here we demonstrate that this important social signal has the potential to distort time perception of gazed-at objects (N = 70 adults). By using a novel gaze-associated learning paradigm combined with the time-discrimination task, we showed that objects previously associated with others' eye gaze were perceived as significantly shorter in duration than the nonassociated counterparts. The time-compression effect cannot be attributed to general attention allocation because it disappeared when objects were associated with nonsocial attention cues (i.e., arrows). Critically, this effect correlated with observers' autistic traits and vanished when the gazing agent's line of sight was blocked by barriers, reflecting the key role of intention processing triggered by gaze in modulating time perception. Our findings support the existence of a special mechanism tuned to social cues, which can shape our perception of the outer world in time domains.

Associations between emotion recognition and autistic and callous-unemotional traits: differential effects of cueing to the eyes.

BACKGROUND: Although autism and callous-unemotional (CU) traits are distinct conditions, both are associated with difficulties in emotion recognition. However, it is unknown whether the emotion recognition difficulties characteristic of autism and CU traits are driven by comparable underpinning mechanisms. METHODS: We tested whether cueing to the eyes improved emotion recognition in relation to autistic and CU traits in a heterogeneous sample of children enhanced for social, emotional and behavioural difficulties. Participants were 171 (n = 75 male) children aged 10-16 years with and without a diagnosis of autism (n = 99 autistic), who completed assessments of emotion recognition with and without cueing to the eyes. Parents completed the assessment of autistic and CU traits. RESULTS: Associations between autistic and CU traits and emotion recognition accuracy were dependent upon gaze cueing. CU traits were associated with an overall decrease in emotion recognition in the uncued condition, but better fear recognition when cued to the eyes. Conversely, autistic traits were associated with decreased emotion recognition in the cued condition only, and no interactions between autistic traits and emotion were found. CONCLUSIONS: The differential effect of cueing to the eyes in autistic and CU traits suggests different mechanisms underpin emotion recognition abilities. Results suggest interventions designed to promote looking to the eyes may be beneficial for children with CU traits, but not for children with autistic characteristics. Future developmental studies of autism and CU characteristics are required to better understand how different pathways lead to overlapping socio-cognitive profiles.

Social gaze behavior and hyperarousal in young females with fragile X syndrome: A within-person approach.

Children with fragile X syndrome (FXS) often avoid eye contact, a behavior that is potentially related to hyperarousal. Prior studies, however, have focused on between-person associations rather than coupling of within-person changes in gaze behaviors and arousal. In addition, there is debate about whether prompts to maintain eye contact are beneficial for individuals with FXS. In a study of young females (ages 6-16), we used eye tracking to assess gaze behavior and pupil dilation during social interactions in a group with FXS (n = 32) and a developmentally similar comparison group (n = 23). Participants engaged in semi-structured conversations with a female examiner during blocks with and without verbal prompts to maintain eye contact. We identified a social-behavioral and psychophysiological profile that is specific to females with FXS; this group exhibited lower mean levels of eye contact, significantly increased mean pupil dilation during conversations that included prompts to maintain eye contact, and showed stronger positive coupling between eye contact and pupil dilation. Our findings strengthen support for the perspective that gaze aversion in FXS reflects negative reinforcement of social avoidance behavior. We also found that behavioral skills training may improve eye contact, but maintaining eye contact appears to be physiologically taxing for females with FXS.

Attentional differences in audiovisual face perception between full- and preterm very low birthweight toddlers.

AIM: To investigate whether rightward attention to the mouth during audiovisual speech perception may be a behavioural marker for early brain development, we studied very preterm and low birthweight (VLBW) and typically developing (TD) toddlers. METHODS: We tested the distribution of gaze points in Japanese-learning TD and VLBW toddlers when exposed to talking, silent and mouth moving faces at 12, 18 and 24 months (corrected age). Each participant was categorised based upon the area they gazed at most (Eye-Right, Eye-Left, Mouth-Right, Mouth-Left) per stimulus per age. A log-linear model was applied to three-dimensional contingency tables (region, side and group). RESULTS: VLBW toddlers showed fewer gaze points than TD toddlers. At 12 months, more VLBW toddlers than TD toddlers showed left attentional bias toward any one face; however, this difference in attention asymmetry receded somewhat by 24 months. In talking condition, TD toddlers showed right attentional bias from 12 to 24 months, whereas VLBW toddlers showed such bias upon reaching 24 months. Additionally, more TD toddlers than VLBW toddlers attended to the mouth. CONCLUSION: Delays in exhibiting the attentional bias for an audiovisual face or general faces displayed by typically developing children might suggest differential developmental timing for hemispheric specialisation or dominance.

Association between attachment anxiety and the gaze direction-related N170.

Attachment theory suggests that interindividual differences in attachment security versus insecurity (anxiety and avoidance) contribute to the ways in which people perceive social emotional signals, particularly from the human face. Among different facial features, eye gaze conveys crucial information for social interaction, with a straight gaze triggering different cognitive and emotional processes as compared to an averted gaze. It remains unknown, however, how interindividual differences in attachment associate with early face encoding in the context of a straight versus averted gaze. Using electroencephalography (EEG) and recording event-related potentials (ERPs), specifically the N170 component, the present study (N = 50 healthy adults) measured how the characteristics of attachment anxiety and avoidance relate to the encoding of faces with respect to gaze direction and head orientation. Our findings reveal a significant relationship between gaze direction (irrespective of head orientation) and attachment anxiety on the interhemispheric (i.e. right) asymmetry of the N170 and thus provide evidence for an association between attachment anxiety and eye gaze processing during early visual face encoding.

Deep learning models for webcam eye tracking in online experiments.

Eye tracking is prevalent in scientific and commercial applications. Recent computer vision and deep learning methods enable eye tracking with off-the-shelf webcams and reduce dependence on expensive, restrictive hardware. However, such deep learning methods have not yet been applied and evaluated for remote, online psychological experiments. In this study, we tackle critical challenges faced in remote eye tracking setups and systematically evaluate appearance-based deep learning methods of gaze tracking and blink detection. From their own homes and laptops, 65 participants performed a battery of eye tracking tasks including (i) fixation, (ii) zone classification, (iii) free viewing, (iv) smooth pursuit, and (v) blink detection. Webcam recordings of the participants performing these tasks were processed offline through appearance-based models of gaze and blink detection. The task battery required different eye movements that characterized gaze and blink prediction accuracy over a comprehensive list of measures. We find the best gaze accuracy to be 2.4° and precision of 0.47°, which outperforms previous online eye tracking studies and reduces the gap between laboratory-based and online eye tracking performance. We release the experiment template, recorded data, and analysis code with the motivation to escalate affordable, accessible, and scalable eye tracking that has the potential to accelerate research in the fields of psychological science, cognitive neuroscience, user experience design, and human-computer interfaces.

Internal Social Attention: Gaze Cues Stored in Working Memory Trigger Involuntary Attentional Orienting.

Previous research has shown that social cues, including eye gaze, can readily guide our focus of attention-a phenomenon referred to as social attention. Here, we demonstrated that internally maintained social cues in working memory (WM) can produce an analogous attentional effect (N = 57). Using the delayed-match-to-sample paradigm combined with the dot-probe task, we found that holding irrelevant gaze cues in WM can induce attentional orienting in college-age adults. Importantly, this WM-induced attention effect could not be explained simply by the perceptual-attentional process, because the identical gaze cues that were only passively viewed and not memorized in WM could not trigger attentional orienting beyond the typical time window of social attention. Furthermore, nonsocial cues (i.e., arrows) held in WM failed to elicit the attentional-orienting effect. These findings provide new evidence for the conceptualization of WM as internally directed attention and highlight the uniqueness of social attention compared with nonsocial attention.

Can facial emotion recognition be rapidly improved in children with disruptive behavior? A targeted and preventative early intervention study.

An impairment in recognizing distress is implicated in the development and severity of antisocial behavior. It has been hypothesized that a lack of attention to the eyes plays a role, but supporting evidence is limited. We developed a computerized training to improve emotion recognition in children and examined the role of eye gaze before and after training. Children referred into an intervention program to prevent antisocial outcomes completed an emotion recognition task with concurrent eye tracking. Those with emotion recognition impairments (n = 54, mean age: 8.72 years, 78% male) completed the training, while others (n = 38, mean age: 8.95 years, 84% male) continued with their usual interventions. Emotion recognition and eye gaze were reassessed in all children 8 weeks later. Impaired negative emotion recognition was significantly related to severity of behavioral problems at pretest. Children who completed the training significantly improved in emotion recognition; eye gaze did not contribute to impairment or improvement in emotion recognition. This study confirms the role of emotion recognition in severity of disruptive behavior and shows that a targeted intervention can quickly improve emotion impairments. The training works by improving children's ability to appraise emotional stimuli rather than by influencing their visual attention.

Transition in eye gaze as a predictor of emergence from general anesthesia in children and adults: a prospective observational study.

BACKGROUND: It is useful to monitor eye movements during general anesthesia, but few studies have examined neurological finding of the eyes during emergence from general anesthesia maintained with short-acting opioids and volatile anesthetics. METHODS: Thirty children aged 1-6 years and 30 adults aged 20-79 years were enrolled. Patients received general anesthesia maintained with sevoflurane and remifentanil. The timing of three physical-behavioral responses-eye-gaze transition (the cycle from conjugate to disconjugate and back to conjugate), resumption of somatic movement (limbs or body), and resumption of respiration-were recorded until spontaneous awakening. The primary outcome measure was the timing of the physical-behavioral responses. Secondary outcome measures were the incidence of eye-gaze transition, and the bispectral index, concentration of end-tidal sevoflurane, and heart rate at the timing of eye-gaze transition. RESULTS: Eye-gaze transition was evident in 29 children (96.7%; 95% confidence interval, 82.8-99.9). After the end of surgery, eye-gaze transition was observed significantly earlier than resumption of somatic movement or respiration (472 [standard deviation 219] s, 723 [235] s, and 754 [232] s, respectively; p < 0.001). In adults, 3 cases (10%; 95% CI, 0.2-26.5) showed eye-gaze transition during emergence from anesthesia. The incidence of eye-gaze transition was significantly lower in adults than in children (p < 0.001). CONCLUSION: In children, eye-gaze transition was observed significantly earlier than other physical-behavioral responses during emergence from general anesthesia and seemed to reflect emergence from anesthesia. In contrast, observation of eye gaze was not a useful indicator of emergence from anesthesia in adults.

Enhancing the Sense of Attention from an Assistance Mobile Robot by Improving Eye-Gaze Contact from Its Iconic Face Displayed on a Flat Screen.

One direct way to express the sense of attention in a human interaction is through the gaze. This paper presents the enhancement of the sense of attention from the face of a human-sized mobile robot during an interaction. This mobile robot was designed as an assistance mobile robot and uses a flat screen at the top of the robot to display an iconic (simplified) face with big round eyes and a single line as a mouth. The implementation of eye-gaze contact from this iconic face is a problem because of the difficulty of simulating real 3D spherical eyes in a 2D image considering the perspective of the person interacting with the mobile robot. The perception of eye-gaze contact has been improved by manually calibrating the gaze of the robot relative to the location of the face of the person interacting with the robot. The sense of attention has been further enhanced by implementing cyclic face explorations with saccades in the gaze and by performing blinking and small movements of the mouth.

Improving Haptic Response for Contextual Human Robot Interaction.

For haptic interaction, a user in a virtual environment needs to interact with proxies attached to a robot. The device must be at the exact location defined in the virtual environment in time. However, due to device limitations, delays are always unavoidable. One of the solutions to improve the device response is to infer human intended motion and move the robot at the earliest time possible to the desired goal. This paper presents an experimental study to improve the prediction time and reduce the robot time taken to reach the desired position. We developed motion strategies based on the hand motion and eye-gaze direction to determine the point of user interaction in a virtual environment. To assess the performance of the strategies, we conducted a subject-based experiment using an exergame for reach and grab tasks designed for upper limb rehabilitation training. The experimental results in this study revealed that eye-gaze-based prediction significantly improved the detection time by 37% and the robot time taken to reach the target by 27%. Further analysis provided more insight on the effect of the eye-gaze window and the hand threshold on the device response for the experimental task.

Improving Performance of the Human Pupil Orbit Model (HPOM) Estimation Method for Eye-Gaze Tracking.

Eye-gaze direction-tracking technology is used in fields such as medicine, education, engineering, and gaming. Stability, accuracy, and precision of eye-gaze direction-tracking are demanded with simultaneous upgrades in response speed. In this study, a method is proposed to improve the speed with decreases in the system load and precision in the human pupil orbit model (HPOM) estimation method. The new method was proposed based on the phenomenon that the minor axis of the elliptical-deformed pupil always pointed toward the rotational center presented in various eye-gaze direction detection studies and HPOM estimation methods. Simulation experimental results confirmed that the speed was improved by at least 74 times by consuming less than 7 ms compared to the HPOM estimation. The accuracy of the eye's ocular rotational center point showed a maximum error of approximately 0.2 pixels on the x-axis and approximately 8 pixels on the y-axis. The precision of the proposed method was 0.0 pixels when the number of estimation samples (ES) was 7 or less, which showed results consistent with those of the HPOM estimation studies. However, the proposed method was judged to work conservatively against the allowable angle error (AAE), considering that the experiment was conducted under the worst conditions and the cost used to estimate the final model. Therefore, the proposed method could estimate HPOM with high accuracy and precision through AAE adjustment according to system performance and the usage environment.

Investigating Methods for Cognitive Workload Estimation for Assistive Robots.

Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing modality might allow robots to derive the best evidence of human workload. In this work, we analyzed and modeled data from a multi-modal simulated driving study specifically designed to evaluate different levels of cognitive workload induced by various secondary tasks such as dialogue interactions and braking events in addition to the primary driving task. Specifically, we performed statistical analyses of various physiological signals including eye gaze, electroencephalography, and arterial blood pressure from the healthy volunteers and utilized several machine learning methodologies including k-nearest neighbor, naive Bayes, random forest, support-vector machines, and neural network-based models to infer human cognitive workload levels. Our analyses provide evidence for eye gaze being the best physiological indicator of human cognitive workload, even when multiple signals are combined. Specifically, the highest accuracy (in %) of binary workload classification based on eye gaze signals is 80.45 ∓ 3.15 achieved by using support-vector machines, while the highest accuracy combining eye gaze and electroencephalography is only 77.08 ∓ 3.22 achieved by a neural network-based model. Our findings are important for future efforts of real-time workload estimation in the multimodal human-robot interactive systems given that eye gaze is easy to collect and process and less susceptible to noise artifacts compared to other physiological signal modalities.

Are Children High on Callous-Unemotional Traits Emotionally Blind? Testing Eye-Gaze Differences.

Callous-unemotional traits have been associated with difficulties in identifying and responding to others' emotions. To inform this line of research, the current study investigated the eye gaze behavior of children (n = 59; mean-age = 6.35) with varying levels of callous-unemotional (CU) traits with the use of eye-tracker methodology, as well as their ability to accurately identify emotional expressions. Participating children were selected from a large screening sample (N = 1283). Main findings supported a reduced fixation rate to the eye-region and an increased fixation in the mouth area of emotional faces among children high on callous-unemotional traits (HCU), irrespective of emotion expressed (i.e., fear, sad, angry and happy) and age of individuals portrayed in images (adult versus child faces). Further, findings suggested that HCU children were less likely to accurately identify facial emotional expressions, which might be due to the identified attentional neglect to the eye region of emotional faces. Current findings support the importance of early prevention and intervention programs that can enhance the emotional development and social adjustment of HCU children.

Eyes on you: Ensuring empathic accuracy or signalling empathy?

The eye region is thought to play an important role in the ability to accurately infer others' feelings, or empathic accuracy (EA), which is an important skill for social interaction. However, most past studies used static pictures, including only visual information, and knowledge about the contribution of the eye region to EA when visual information is presented together with verbal content is lacking. We therefore examined whether eye gazing contributes to EA during videos of emotional autobiographical stories including both visual and verbal content. One hundred seven perceivers watched videos of targets talking about positive and negative life events and continuously rated the targets' feelings during the videos. Simultaneously, perceivers' eyes were tracked. After each video, perceivers reported on their feelings and the extent to which they empathized with and took the perspective of the targets. In contrast to studies using static pictures, we found that gazing to the eyes of targets during the videos did not significantly contribute to EA. At the same time, results on the association between the amount of gaze towards the eye region of targets and perceivers' state and trait empathy ratings suggest that eye gazing might signal empathy and social engagement to others.

Facial and neural mechanisms during interactive disclosure of biographical information.

Pairs of participants mutually communicated (or not) biographical information to each other. By combining simultaneous eye-tracking, face-tracking and functional near-infrared spectroscopy, we examined how this mutual sharing of information modulates social signalling and brain activity. When biographical information was disclosed, participants directed more eye gaze to the face of the partner and presented more facial displays. We also found that spontaneous production and observation of facial displays was associated with activity in the left SMG and right dlPFC/IFG, respectively. Moreover, mutual information-sharing increased activity in bilateral TPJ and left dlPFC, as well as cross-brain synchrony between right TPJ and left dlPFC. This suggests that a complex long-range mechanism is recruited during information-sharing. These multimodal findings support the second-person neuroscience hypothesis, which postulates that communicative interactions activate additional neurocognitive mechanisms to those engaged in non-interactive situations. They further advance our understanding of which neurocognitive mechanisms underlie communicative interactions.