Gaze-speech coordination during social interaction in Parkinson's disease.

BACKGROUND: Parkinson's disease (PD) can affect social interaction and communication as well as motor and cognitive processes. Speech is affected in PD, as is the control of voluntary eye movements which are thought to play an important role in 'turn taking' in conversation. AIMS: This study aimed to measure eye movements during spoken conversation in PD to assess whether differences in patterns of eye gaze are linked to disrupted turn taking and impaired communication efficiency. METHODS & PROCEDURE: Eleven participants with mild PD and 14 controls completed a two-player guessing game. During each 3 min game turn, one of the players had to guess the names of as many objects as possible based only on the other player's description. Eye movements were recorded simultaneously in both participants using mobile eye trackers along with speech onset and offset times. OUTCOMES & RESULTS: When people with PD played the role of describer, the other player guessed fewer objects compared to when controls described objects. When guessing objects, people with PD performed just as well as controls. Analysis of eye fixations showed that people with PD made longer periods of fixation on the other player's face relative to controls and a lower number of such 'gaze on face' periods. CONCLUSIONS & IMPLICATIONS: A combination of oculomotor, cognitive and speech abnormalities may disrupt communication in PD. Better public awareness of oculomotor, speech and other deficits in the condition could improve social connectedness in people with Parkinson's. WHAT THIS PAPER ADDS: What is already known on this subject? Parkinson's disease is known to affect the control of voluntary eye movements. Direction of eye gaze is important in spoken conversation as a cue to turn-taking, but no studies have examined whether eye movements are different during communication in people with Parkinson's. What this paper adds to existing knowledge? People with Parkinson's showed longer periods of eye fixation during conversations compared to controls. Delays and overlaps between speech turns were also affected in patients. What are the clinical implications of this work? Better knowledge of the effect of the disease on eye gaze control amongst clinicians may help improve communication and social connectedness for patients in the future.

The construct validity of attitudes toward sex offenders (ATS) scale: ATS is more strongly linked to the acceptance of sex offenders than other offenders or non-offenders.

Negative attitudes toward offenders may hinder the rehabilitation process. The present study examines the relationship between attitudes toward sex offenders and stated acceptance of offenders and non-offenders into various aspects of daily life. Sixty female members of the public (18-50 years old, UK residents, recruited by word of mouth and via social media) completed an attitudes towards sex offenders (ATS) scale and indicated for each of eight vignettes describing ex-offenders and non-offenders whether they would accept them in various situations (housing, employment, day-to-day activities). Results indicate that in this group of female participants, harsher attitudes toward sex offenders are associated with lower acceptance of sex offenders (around 50% less acceptance) and other offenders (around 25% less acceptance), but not non-offenders, suggesting a tight coupling between attitudes and acceptance. The observed coupling between attitudes toward sex offenders and acceptance of offenders suggests that it will be difficult to change one without changing the other.

Automatic object detection for behavioural research using YOLOv8.

Observational studies of human behaviour often require the annotation of objects in video recordings. Automatic object detection has been facilitated strongly by the development of YOLO ('you only look once') and particularly by YOLOv8 from Ultralytics, which is easy to use. The present study examines the conditions required for accurate object detection with YOLOv8. The results show almost perfect object detection even when the model was trained on a small dataset (100 to 350 images). The detector, however, does not extrapolate well to the same object in other backgrounds. By training the detector on images from a variety of backgrounds, excellent object detection can be restored. YOLOv8 could be a game changer for behavioural research that requires object annotation in video recordings.

Visual attention in change blindness for objects and shadows.

Studies have found that observers pay less attention to cast shadows in images than to better illuminated regions. In line with such observations, a recent study has suggested stronger change blindness for shadows than for objects (Ehinger et al., 2016). We here examine the role of (overt) visual attention in these findings by recording participants' eye movements. Participants first viewed all original images (without changes). They then performed a change detection task on a subset of the images with changes in objects or shadows. During both tasks, their eye movements were recorded. In line with the original study, objects (subject to change in the change detection task) were fixated more often than shadows. In contrast to the previous study, better change detection was found for shadows than for objects. The improved change detection for shadows may be explained by the balancing of trials with object and shadow changes in the present study. Eye movements during change detection indicated that participants searched the bottom half of the images. Shadows were more often present in this region, which may explain why they were easier to find.

The effect of directional social cues on saccadic eye movements in Parkinson's disease.

There is growing interest in how social processes and behaviour might be affected in Parkinson's disease. A task which has been widely used to assess how people orient attention in response to social cues is the spatial cueing task. Socially relevant directional cues, such as a picture of someone gazing or pointing to the left or the right have been shown to cause orienting of visual attention in the cued direction. The basal ganglia may play a role in responding to such directional cues, but no studies to date have examined whether similar social cueing effects are seen in people with Parkinson's disease. In this study, patients and healthy controls completed a prosaccade (Experiment 1) and an antisaccade task (Experiment 2) in which the target was preceded by arrow, eye gaze or pointing finger cues. Patients showed increased errors and response times for antisaccades but not prosaccades. Healthy participants made most anticipatory errors on pointing finger cue trials, but Parkinson's patients were equally affected by arrow, eye gaze and pointing cues. It is concluded that Parkinson's patients have a reduced ability to suppress responding to directional cues, but this effect is not specific to social cues.

Does agreement mean accuracy? Evaluating glance annotation in naturalistic driving data.

Naturalistic driving studies often make use of cameras to monitor driver behavior. To analyze the resulting video images, human annotation is often adopted. These annotations then serve as the 'gold standard' to train and evaluate automated computer vision algorithms, even though it is uncertain how accurate human annotation is. In this study, we provide a first evaluation of glance direction annotation by comparing instructed, actual glance direction of truck drivers with annotated direction. Findings indicate that while for some locations high annotation accuracy is achieved, for most locations accuracy is well below 50%. Higher accuracy can be obtained by clustering these locations, but this also leads to reduced detail of the annotation, suggesting that decisions to use clustering should take into account the purpose of the annotation. The data also show that high agreement between annotators does not guarantee high accuracy. We argue that the accuracy of annotation needs to be verified experimentally more often.

Eye Movements in the "Morris Maze" Spatial Working Memory Task Reveal Deficits in Strategic Planning.

Analysis of eye movements can provide insights into processes underlying performance of cognitive tasks. We recorded eye movements in healthy participants and people with idiopathic Parkinson disease during a token foraging task based on the spatial working memory component of the widely used Cambridge Neuropsychological Test Automated Battery. Participants selected boxes (using a mouse click) to reveal hidden tokens. Tokens were never hidden under a box where one had been found before, such that memory had to be used to guide box selections. A key measure of performance in the task is between search errors (BSEs) in which a box where a token has been found is selected again. Eye movements were found to be most commonly directed toward the next box to be clicked on, but fixations also occurred at rates higher than expected by chance on boxes farther ahead or back along the search path. Looking ahead and looking back in this way was found to correlate negatively with BSEs and was significantly reduced in patients with Parkinson disease. Refixating boxes where tokens had already been found correlated with BSEs and the severity of Parkinson disease symptoms. It is concluded that eye movements can provide an index of cognitive planning in the task. Refixations on locations where a token has been found may also provide a sensitive indicator of visuospatial memory integrity. Eye movement measures derived from the spatial working memory task may prove useful in the assessment of executive functions as well as neurological and psychiatric diseases in the future.

The eyes have it: Using eye tracking to inform information processing strategies in multi-attributes choices.

Although choice experiments (CEs) are widely applied in economics to study choice behaviour, understanding of how individuals process attribute information remains limited. We show how eye-tracking methods can provide insight into how decisions are made. Participants completed a CE, while their eye movements were recorded. Results show that although the information presented guided participants' decisions, there were also several processing biases at work. Evidence was found of (a) top-to-bottom, (b) left-to-right, and (c) first-to-last order biases. Experimental factors-whether attributes are defined as "best" or "worst," choice task complexity, and attribute ordering-also influence information processing. How individuals visually process attribute information was shown to be related to their choices. Implications for the design and analysis of CEs and future research are discussed.

Responding to social and symbolic extrafoveal cues: cue shape trumps biological relevance.

Social cues presented at visual fixation have been shown to strongly influence an observer's attention and response selection. Here we ask whether the same holds for cues (initially) presented away from fixation, as cues are commonly perceived in natural vision. In six experiments, we show that extrafoveally presented cues with a distinct outline, such as pointing hands, rotated heads, and arrow cues result in strong cueing of responses (either to the cue itself, or a cued object). In contrast, cues without a clear outline, such as gazing eyes and direction words exert much weaker effects on participants' responses to a target cue. We also show that distraction effects on response times are relatively weak, but that strong interference effects can be obtained by measuring mouse trajectories. Eye tracking suggests that gaze cues are slower to respond to because their direction cannot easily be perceived in extrafoveal vision. Together, these data suggest that the strength of an extrafoveal cue is determined by the shape of the cue outline, rather than its biological relevance (i.e., whether the cue is provided by another human being), and that this shape effect is due to how easily the direction of a cue can be perceived in extrafoveal vision.

The developmental trajectory of attentional orienting to socio-biological cues.

It has been proposed that the orienting of attention in the same direction as another's point of gaze relies on innate brain mechanisms which are present from birth, but direct evidence relating to the influence of eye gaze cues on attentional orienting in young children is limited. In two experiments, 137 children aged 3-10 years old performed an adapted pro-saccade task with centrally presented uninformative eye gaze, finger pointing and arrow pre-cues which were either congruent or incongruent with the direction of target presentations. When the central cue overlapped with presentation of the peripheral target (Experiment 1), children up to 5 years old had difficulty disengaging fixation from central fixation in order to saccade to the target. This effect was found to be particularly marked for eye gaze cues. When central cues were extinguished simultaneously with peripheral target onset (Experiment 2), this effect was greatly reduced. In both experiments finger pointing cues (image of pointing index finger presented at fixation) exerted a strong influence on saccade reaction time to the peripheral stimulus for the youngest group of children (<5 years). Overall the results suggest that although young children are strongly engaged by centrally presented eye gaze cues, the directional influence of such cues on overt attentional orienting is only present in older children, meaning that the effect is unlikely to be dependent upon an innate brain module. Instead, the results are consistent with the existence of stimulus-response associations which develop with age and environmental experience.

Is it really search or just matching? The influence of Goodness, number of stimuli and presentation sequence in same-different tasks.

The Goodness of Garner dot patterns has been shown to influence same-different response times in a specific way, which has led to the formulation of a memory search model of pattern comparison. In this model, the space of possible variations of each pattern is searched separately for each pattern in the comparison, resulting in faster response times for patterns that have fewer alternatives. Compared to an alternative explanation based on stimulus encoding plus mental rotation, however, the existing data strongly favor this explanation. To obtain a more constraining set of data to distinguish between the two possible accounts, we extended the original paradigm to a situation in which participants needed to compare three, rather than two patterns and varied the way the stimuli were presented (simultaneously or sequentially). Our findings suggest that neither the memory search nor the encoding plus mental rotation model provides a complete description of the data, and that the effects of Goodness must be understood in a combination of both mechanisms, or in terms of cascades processing.

Release of crowding by pattern completion.

In crowding, target perception deteriorates in the presence of flanking elements. Crowding is classically explained by low-level mechanisms such as pooling or feature substitution. However, we have previously shown that perceptual grouping between the target and flankers, rather than low-level mechanisms, determines crowding. There are many grouping cues that can determine crowding, such as low- and high-level feature similarity, low- and high-level pattern regularity, and good Gestalt. Here we show that pattern completion, another grouping cue that is important for crowding in foveal vision, is also important in peripheral vision. We also describe computer simulations that show how pattern completion, and crowding in general, can be partly explained by recurrent processing.

Speeded classification in simultaneous masking.

In simultaneous masking, flanking elements impair performance on a visual target. Previous studies on simultaneous masking have predominantly used discrimination threshold estimation to quantify performance on the target. Results based on thresholds suggest that an important factor in simultaneous masking is the strength of grouping between the target and the flankers. Recently Panis and Hermens (2014) showed that error rates in a speeded discrimination task, using the same simultaneous masking paradigm, provide a very similar pattern of results compared to thresholds from previous studies. In contrast, response times were found to deviate from the pattern in the error rates in some of the conditions, possibly providing a method to tap into low-level visual processes within the same paradigm. However, only a small number of masks was used, and it is therefore unclear to what extent differences can be found between thresholds, error rates, and response times. Here, we address this issue by examining speeded classification performance on a vernier target for a broad range of simultaneous mask layouts. Results suggest that thresholds and error rates are strongly associated, and that response times generally show the same pattern of results, although to a slightly weaker extent. We suggest that masking strength, defined by either measure, appears to be linked to target-flanker grouping rather than to low-level interactions.

The Influence of Competing Social and Symbolic Cues on Observers' Gaze Behaviour.

The effects of social (eye gaze, pointing gestures) and symbolic (arrows) cues on observers' attention are often studied by presenting such cues in isolation and at fixation. Here, we extend this work by embedding cues in natural scenes. Participants were presented with a single cue (Experiment 1) or a combination of cues (Experiment 2) embedded in natural scenes and were asked to 'simply look at the images' while their eye movements were recorded to assess the effects of the cues on (overt) attention. Single-gaze and pointing cues were fixated for longer than arrows but at the cost of shorter dwell times on the cued object. When presented together, gaze and pointing cues were fixated faster and for longer than simultaneously presented arrows. Attention to the cued object depended on the combination of cues and whether both cues were directed towards or away from the target object. Together, the findings confirm earlier observations that people attract attention more strongly than arrows but that arrows more strongly direct attention.

Advance information modulates the global effect even without instruction on where to look.

When observers are asked to make an eye movement to a visual target in the presence of a near distractor, their eyes tend to land on a position in between the target and the distractor, an effect known as the global effect. While it was initially believed that the global effect is a mandatory eye movement strategy, recent studies have shown that explicit instructions to make an eye movement to a certain part of the scene can overrule the effect. We here investigate whether such top-down influences are also found when people are not actively involved in an explicit eye movement task, but instead, make eye movements in the service of another task. Participants were presented with arrays of yellow and green discs, each containing a letter, and were asked to identify a target letter. Because the discs were presented away from fixation, participants made an eye movement to the array of discs on most of the trials. An analysis of the landing sites of these eye movements revealed that, even without an explicit instruction, observers take the advance information about the colour of the disc containing the target into account before moving their eyes. Moreover, when asking participants to maintain fixation for intervals of different durations, it was found that the implicit top-down influences operated on a very similar time-scale as previously observed for explicit eye movement instructions.

The role of eye movements in a contour detection task.

Vision combines local feature integration with active viewing processes, such as eye movements, to perceive complex visual scenes. However, it is still unclear how these processes interact and support each other. Here, we investigated how the dynamics of saccadic eye movements interact with contour integration, focusing on situations in which contours are difficult to find or even absent. We recorded observers' eye movements while they searched for a contour embedded in a background of randomly oriented elements. Task difficulty was manipulated by varying the contour's path angle. An association field model of contour integration was employed to predict potential saccade targets by identifying stimulus locations with high contour salience. We found that the number and duration of fixations increased with the increasing path angle of the contour. In addition, fixation duration increased over the course of a trial, and the time course of saccade amplitude depended on the percept of observers. Model fitting revealed that saccades fully compensate for the reduced saliency of peripheral contour targets. Importantly, our model predicted fixation locations to a considerable degree, indicating that observers fixated collinear elements. These results show that contour integration actively guides eye movements and determines their spatial and temporal parameters.

Thinking on your feet: Anticipatory foot placements in repeated bimanual object displacements.

Effective handling of objects requires proper use of the hands. If the object handling is done while standing or walking, it also requires proper use of the feet. We asked how people position their feet to meet future and ongoing object-handling demands. In previous research on this topic, participants walked to a table and picked up an object for a single displacement from one place to another. These studies shed light on sensitivity to kinematics but, strictly speaking, may not have revealed anything about sensitivity to dynamics. In the present study, we asked participants to walk to a table to move an object back and forth over different distances and at different rates. Prior to walking to the table, participants had full knowledge of what the task would be. By using a rhythmic rather than discrete object placement task, we could analyze participants' sensitivity to dynamics as well as kinematics. Consistent with our expectation that participants would tune their foot separations to demands related to dynamics, we found that stance width was wider for long than for short object displacements and was more pronounced for high displacement rates than for low displacement rates. Also consistent with our expectations about planning, these effects were evident as soon as participants reached the table. Our results add to the limited research on coordinated action of the hands and feet in purposeful object manipulation.

How the global layout of the mask influences masking strength.

In visual backward masking, the perception of a target is influenced by a trailing mask. Masking is usually explained by local interactions between the target and the mask representations. However, recently it has been shown that the global spatial layout of the mask rather than its local structure determines masking strength (Hermens & Herzog, 2007). Here, we varied the mask layout by spatial, luminance, and temporal cues. We presented a vernier target followed by a mask with 25 elements. Performance deteriorated when the length of the two mask elements neighboring the target vernier was doubled. However, when the length of every second mask element was doubled, performance improved. When the luminance of the neighboring elements was doubled, performance also deteriorated but no improvement in performance was observed when every second element had a double luminance. For temporal manipulations, a complex nonmonotonic masking function was observed. Hence, changes in the mask layout by spatial, luminance, and temporal cues lead to highly different results.

Data-driven group comparisons of eye fixations to dynamic stimuli.

Recent advances in software and hardware have allowed eye tracking to move away from static images to more ecologically relevant video streams. The analysis of eye tracking data for such dynamic stimuli, however, is not without challenges. The frame-by-frame coding of regions of interest (ROIs) is labour-intensive and computer vision techniques to automatically code such ROIs are not yet mainstream, restricting the use of such stimuli. Combined with the more general problem of defining relevant ROIs for video frames, methods are needed that facilitate data analysis. Here, we present a first evaluation of an easy-to-implement data-driven method with the potential to address these issues. To test the new method, we examined the differences in eye movements of self-reported politically left- or right-wing leaning participants to video clips of left- and right-wing politicians. The results show that our method can accurately predict group membership on the basis of eye movement patterns, isolate video clips that best distinguish people on the political left-right spectrum, and reveal the section of each video clip with the largest group differences. Our methodology thereby aids the understanding of group differences in gaze behaviour, and the identification of critical stimuli for follow-up studies or for use in saccade diagnosis.

Face and word composite effects are similarly affected by priming of local and global processing.

Holistic processing has been shown with both faces and words, but it is unclear how similar their underlying mechanisms are. In this study attention to global and local features was manipulated and the consequences for holistic word and face processing were examined. On each trial participants were presented two Navon figures and told to focus on either the global or the local level. Then they performed a composite task in which they indicated whether the target halves of two sequentially presented faces or words were the same or different, ignoring the irrelevant halves. Similar stronger global priming effects were found for faces and words, indicating that holistic processing for the two types of stimuli were susceptible to attention manipulations to similar degrees, which was confirmed with Bayesian analyses. The findings add to the investigation of the similarity and differences between holistic processing and help reveal those aspects of holistic processing that are domain general and those specific to individual categories.