Anaphoric distance dependencies in visual narrative structure and processing.

Linguistic syntax has often been claimed as uniquely complex due to features like anaphoric relations and distance dependencies. However, visual narratives of sequential images, like those in comics, have been argued to use sequencing mechanisms analogous to those in language. These narrative structures include "refiner" panels that "zoom in" on the contents of another panel. Similar to anaphora in language, refiners indexically connect inexplicit referential information in one unit (refiner, pronoun) to a more informative "antecedent" elsewhere in the discourse. Also like in language, refiners can follow their antecedents (anaphoric) or precede them (cataphoric), along with having either proximal or distant connections. We here explore the constraints on visual narrative refiners created by modulating these features of order and distance. Experiment 1 examined participants' preferences for where refiners are placed in a sequence using a force-choice test, which revealed that refiners are preferred to follow their antecedents and have proximal distances from them. Experiment 2 then showed that distance dependencies lead to slower self-paced viewing times. Finally, measurements of event-related brain potentials (ERPs) in Experiment 3 revealed that these patterns evoke similar brain responses as referential dependencies in language (i.e., N400, LAN, Nref). Across all three studies, the constraints and (neuro)cognitive responses to refiners parallel those shown to anaphora in language, suggesting domain-general constraints on the sequencing of referential dependencies.

Eye tracking: empirical foundations for a minimal reporting guideline.

In this paper, we present a review of how the various aspects of any study using an eye tracker (such as the instrument, methodology, environment, participant, etc.) affect the quality of the recorded eye-tracking data and the obtained eye-movement and gaze measures. We take this review to represent the empirical foundation for reporting guidelines of any study involving an eye tracker. We compare this empirical foundation to five existing reporting guidelines and to a database of 207 published eye-tracking studies. We find that reporting guidelines vary substantially and do not match with actual reporting practices. We end by deriving a minimal, flexible reporting guideline based on empirical research (Section "An empirically based minimal reporting guideline").

Meaning above (and in) the head: Combinatorial visual morphology from comics and emoji.

Compositionality is a primary feature of language, but graphics can also create combinatorial meaning, like with items above faces (e.g., lightbulbs to mean inspiration). We posit that these "upfixes" (i.e., upwards affixes) involve a productive schema enabling both stored and novel face-upfix dyads. In two experiments, participants viewed either conventional (e.g., lightbulb) or unconventional (e.g., clover-leaves) upfixes with faces which either matched (e.g., lightbulb/smile) or mismatched (e.g., lightbulb/frown). In Experiment 1, matching dyads sponsored higher comprehensibility ratings and faster response times, modulated by conventionality. In Experiment 2, event-related brain potentials (ERPs) revealed conventional upfixes, regardless of matching, evoked larger N250s, indicating perceptual expertise, but mismatching and unconventional dyads elicited larger semantic processing costs (N400) than conventional-matching dyads. Yet mismatches evoked a late negativity, suggesting congruent novel dyads remained construable compared with violations. These results support that combinatorial graphics involve a constrained productive schema, similar to the lexicon of language.

Zooming in on visual narrative comprehension.

The comprehension of visual narratives requires paying attention to certain elements and integrating them across a sequence of images. To study this process, we developed a new approach that modified comic strips according to where observers looked while viewing each sequence. Across three self-paced experiments, we presented sequences of six panels that were sometimes automatically "zoomed-in" or re-framed in order to highlight parts of the image that had been fixated by another group of observers. Fixation zoom panels were rated as easier to understand and produced viewing times more similar to the original comic than panels modified to contain non-fixated or incongruous regions. When a single panel depicting the start of an action was cropped to show only the most fixated region, viewing times were similar to the original narrative despite the reduced information. Modifying such panels also had an impact on the viewing time on subsequent panels, both when zoomed in and when regions were highlighted through an "inset" panel. These findings demonstrate that fixations in a visual narrative are guided to informative elements, and that these elements influence both the current panel and the processing of the sequence.

Zooming in on the cognitive neuroscience of visual narrative.

Visual narratives like comics and films often shift between showing full scenes and close, zoomed-in viewpoints. These zooms are similar to the "spotlight of attention" cast across a visual scene in perception. We here measured ERPs to visual narratives (comic strips) that used zoomed-in and full-scene panels either throughout the whole sequence context or at specific critical panels. Zoomed-in panels were automatically generated on the basis of fixations from prior participants' eye movements to the crucial content of panels (Foulsham & Cohn, 2020). We found that these fixation panels evoked a smaller N300 than full-scenes, indicative of reduced cost for object identification, but that they also evoked a slightly larger amplitude N400 response, suggesting a greater cost for accessing semantic memory with constrained content. Panels in sequences where fixation panels persisted across all positions of the sequence also evoked larger posterior P600s, implying that constrained views required more updating or revision processes throughout the sequence. Altogether, these findings suggest that constraining a visual scene to its crucial parts triggers various processes related not only to the density of its information but also to its integration into a sequential context.

Turning the (virtual) world around: Patterns in saccade direction vary with picture orientation and shape in virtual reality.

Research investigating gaze in natural scenes has identified a number of spatial biases in where people look, but it is unclear whether these are partly due to constrained testing environments (e.g., a participant with their head restrained and looking at a landscape image framed within a computer monitor). We examined the extent to which image shape (square vs. circle), image rotation, and image content (landscapes vs. fractal images) influence eye and head movements in virtual reality (VR). Both the eyes and head were tracked while observers looked at natural scenes in a virtual environment. In line with previous work, we found a bias for saccade directions parallel to the image horizon, regardless of image shape or content. We found that, when allowed to do so, observers move both their eyes and head to explore images. Head rotation, however, was idiosyncratic; some observers rotated a lot, whereas others did not. Interestingly, the head rotated in line with the rotation of landscape but not fractal images. That head rotation and gaze direction respond differently to image content suggests that they may be under different control systems. We discuss our findings in relation to current theories on head and eye movement control and how insights from VR might inform more traditional eye-tracking studies.

Is the frequency of adult strabismus surgery increasing?

PURPOSE: In recent years there has been an increase in evidence for the functional and psychosocial benefits of correcting strabismus/heterotropia in adults. This study aimed to establish whether there has been an associated change in the frequency of strabismus surgery performed on adults in England since 2000. METHODS: Data on strabismus surgery performed in England between 2000 and 2014 were obtained from Hospital Episode Statistics, Health and Social Care Information Centre, England. The frequency of strabismus surgery was analysed for different age groups. Data were considered in the context of total population data for England, obtained from the Office for National Statistics. RESULTS: There was little change in the total number of strabismus operations performed in 2000-2014 (1% reduction). In the same period the number of operations performed on children aged 0-15 years decreased by 17%. In contrast, there was a 24% increase in the number of strabismus operations performed on patients aged 15 years or older. CONCLUSIONS: Although strabismus surgery is still most commonly performed on children, the data show there has been a significant increase in the number of strabismus operations performed on adults. We speculate that this increase is connected to the growing weight of evidence detailing the functional and psychosocial consequences of strabismus and the benefits of correction. These results have potential implications for the delivery of future care.

Potential social interactions are important to social attention.

Social attention, or how spatial attention is allocated to biologically relevant stimuli, has typically been studied using simplistic paradigms that do not provide any opportunity for social interaction. To study social attention in a complex setting that affords social interaction, we measured participants' looking behavior as they were sitting in a waiting room, either in the presence of a confederate posing as another research participant, or in the presence of a videotape of the same confederate. Thus, the potential for social interaction existed only when the confederate was physically present. Although participants frequently looked at the videotaped confederate, they seldom turned toward or looked at the live confederate. Ratings of participants' social skills correlated with head turns to the live, but not videotaped, confederate. Our results demonstrate the importance of studying social attention within a social context, and suggest that the mere opportunity for social interaction can alter social attention.

Effects of lorazepam on saccadic eye movements - evidence from prosaccade and free viewing tasks.

RATIONALE: Peak velocities of saccadic eye movements are reduced after benzodiazepine administration. Even though this is an established effect, past research has only examined it in horizontal prosaccade tasks. OBJECTIVES: The spectrum of saccadic eye movements, however, is much larger. Therefore, we aimed to make a first attempt at filling this research gap by testing benzodiazepine effects on saccades under different experimental task conditions. METHODS: 1 mg lorazepam or placebo was administered (within-subjects, double-blind, in randomised order) to n = 30 healthy adults. Participants performed an extended version of the prosaccade task, including vertical saccade directions and different stimulus eccentricities, as well as a free viewing task. RESULTS: Results from the prosaccade task confirmed established effects of benzodiazepines as well as saccade direction on saccadic parameters but additionally showed that the drug effect on peak velocity was independent of saccade direction. Remarkably, in the free viewing task peak velocities as well as other saccade parameters were unaffected by lorazepam. Furthermore, exploration patterns during free viewing did not change under lorazepam. CONCLUSIONS: Overall, our findings further consolidate the peak velocity of prosaccades as a biomarker of sedation. Additionally, we suggest that sedative effects of low doses of benzodiazepines may be compensated in tasks that more closely resemble natural eye movement behaviour, possibly due to the lack of time constraints or via neurophysiological processes related to volition.

Eye movements in visual impairment.

This Special Issue describes the impact of visual impairment on visuomotor function. It includes contributions that examine gaze control in conditions associated with abnormal visual development such as amblyopia, dyslexia and neurofibromatosis as well as disorders associated with field loss later in life, such as macular degeneration and stroke. Specifically, the papers address both gaze holding (fixation), and gaze-following behavior (single saccades, sequences of saccades and smooth-pursuit) that characterize active vision in daily life and evaluate the influence of both pathological and simulated field loss. Several papers address the challenges to reading and visual search; describing how the patterns of eye movements in these real-world tasks adapt to visual impairment and highlighting how they could serve as diagnostic markers of visuomotor function.

Dancing out for a voice; a narrative review of the literature exploring autism, physical activity, and dance.

Autism Spectrum Disorder is characterised by profound challenges with social communication and social interaction. Currently, there are few therapeutic interventions that successfully target some of the functionally impairing traits associated with autism. Furthermore, many of these interventions comprise a variety of limitations; including, limited accessibility, extensive durations, or the requirement of a trained professional to deliver the intervention. New research suggests that instead of targeting all traits associated with Autism Spectrum Disorder with a single solution, scientific research should focus on providing therapeutic tools that alleviate functionally impairing facets specific to the individual. Owing to the nature of physical activity, sports, and dance (coordinated movement) these activities could provide opportunities to enhance communication skills and social development in autistic children. Therefore, this paper gives a narrative overview of the literature surrounding communication and coordinated movement; outlining what is meant by communication challenges, exploring the benefits of coordinated movement for traits associated with Autism Spectrum Disorder, and delineating how coordinated movement elicits positive outcomes for autistic children.

Social prioritisation in scene viewing and the effects of a spatial memory load.

When free-viewing scenes, participants tend to preferentially fixate social elements (e.g., people). In the present study, we tested whether this bias would be disrupted by increasing the demands of a secondary dual-task: holding a set of (one or six) spatial locations in memory, presented either simultaneously or sequentially. Following a retention interval, participants judged whether a test location was present in the to-be-remembered stimuli. During the retention interval participants free-viewed scenes containing a social element (a person) and a non-social element (an object) that served as regions of interest. In order to assess the impact of physical salience, the non-social element was presented in both an unaltered baseline version, and in a version where its salience was artificially increased. The results showed that the preference to look at social elements decreased when the demands of the spatial memory task were increased from one to six locations, regardless of presentation mode (simultaneous or sequential). The high-load condition also resulted in more central fixations and reduced exploration of the scene. The results indicate that the social prioritisation effect, and scene viewing more generally, can be affected by a concurrent memory load.

Parent-reported social-communication changes in children diagnosed with autism spectrum disorder during the COVID-19 pandemic in the UK.

Introduction: The coronavirus pandemic has swept across the United Kingdom (UK). Given the ever-evolving situation, little is known about the repercussions of coronavirus and the subsequent lockdowns for children diagnosed with Autism Spectrum Disorder (ASD). Therefore, this study explores the social-communicative impact of the first lockdown (March 2020 - July 2020) in the UK and the return to school period (September 2020 - October 2020), following prolonged disruption to routine, in children diagnosed with ASD. Methods : Parents of autistic children completed 2 separate online surveys following the first lockdown in the UK (n = 176) and also when children returned to school following the summer break (n = 54). Results : The results suggested that self-regulation skills (p < .05) and co-operation skills (p < .05) were most affected over the course of the lockdown. Children's physical activity levels were perceived to significantly increase during the return to school (p < .0001), which was associated with better social-communication outcomes (p < .05). Conclusion : Future work is needed to confirm and explore the findings. Such work could be implemented to protect and improve the social-communicative outcomes of autistic children.

Reading the room: Autistic traits, gaze behaviour, and the ability to infer social relationships.

Individuals high in autistic traits can have difficulty understanding verbal and non-verbal cues, and may display atypical gaze behaviour during social interactions. The aim of this study was to examine differences among neurotypical individuals with high and low levels of autistic traits with regard to their gaze behaviour and their ability to assess peers' social status accurately. Fifty-four university students who completed the 10-item Autism Quotient (AQ-10) were eye-tracked as they watched six 20-second video clips of people ("targets") involved in a group decision-making task. Simulating natural, everyday social interactions, the video clips included moments of debate, humour, interruptions, and cross talk. Results showed that high-scorers on the AQ-10 (i.e., those with more autistic traits) did not differ from the low-scorers in either gaze behaviour or assessing the targets' relative social status. The results based on this neurotypical group of participants suggest that the ability of individuals high in autistic traits to read social cues may be preserved in certain tasks crucial to navigating day-to-day social relationships. These findings are discussed in terms of their implications for theory of mind, weak central coherence, and social motivation theories of autism.

It depends on how you look at it: scanpath comparison in multiple dimensions with MultiMatch, a vector-based approach.

Eye movement sequences-or scanpaths-vary depending on the stimulus characteristics and the task (Foulsham & Underwood Journal of Vision, 8(2), 6:1-17, 2008; Land, Mennie, & Rusted, Perception, 28, 1311-1328, 1999). Common methods for comparing scanpaths, however, are limited in their ability to capture both the spatial and temporal properties of which a scanpath consists. Here, we validated a new method for scanpath comparison based on geometric vectors, which compares scanpaths over multiple dimensions while retaining positional and sequential information (Jarodzka, Holmqvist, & Nyström, Symposium on Eye-Tracking Research and Applications (pp. 211-218), 2010). "MultiMatch" was tested in two experiments and pitted against ScanMatch (Cristino, Mathôt, Theeuwes, & Gilchrist, Behavior Research Methods, 42, 692-700, 2010), the most comprehensive adaptation of the popular Levenshtein method. In Experiment 1, we used synthetic data, demonstrating the greater sensitivity of MultiMatch to variations in spatial position. In Experiment 2, real eye movement recordings were taken from participants viewing sequences of dots, designed to elicit scanpath pairs with commonalities known to be problematic for algorithms (e.g., when one scanpath is shifted in locus or when fixations fall on either side of an AOI boundary). The results illustrate the advantages of a multidimensional approach, revealing how two scanpaths differ. For instance, if one scanpath is the reverse copy of another, the difference is in the direction but not the positions of fixations; or if a scanpath is scaled down, the difference is in the length of the saccadic vectors but not in the overall shape. As well as having enormous potential for any task in which consistency in eye movements is important (e.g., learning), MultiMatch is particularly relevant for "eye movements to nothing" in mental imagery and embodiment-of-cognition research, where satisfactory scanpath comparison algorithms are lacking.

Do cognitive load and ADHD traits affect the tendency to prioritise social information in scenes?

We report two experiments investigating the effect of working memory (WM) load on selective attention. Experiment 1 was a modified version of Lavie et al. and confirmed that increasing memory load disrupted performance in the classic flanker task. Experiment 2 used the same manipulation of WM load to probe attention during the viewing of complex scenes while also investigating individual differences in attention deficit hyperactivity disorder (ADHD) traits. In the image-viewing task, we measured the degree to which fixations targeted each of two crucial objects: (1) a social object (a person in the scene) and (2) a non-social object of higher or lower physical salience. We compared the extent to which increasing WM load would change the pattern of viewing of the physically salient and socially salient objects. If attending to the social item requires greater default voluntary top-down resources, then the viewing of social objects should show stronger modulation by WM load compared with viewing of physically salient objects. The results showed that the social object was fixated to a greater degree than the other object (regardless of physical salience). Increased salience drew fixations away from the background leading to slightly increased fixations on the non-social object, without changing fixations on the social object. Increased levels of ADHD-like traits were associated with fewer fixations on the social object, but only in the high-salient, low-load condition. Importantly, WM load did not affect the number of fixations on the social object. Such findings suggest rather surprisingly that attending to a social area in complex stimuli is not dependent on the availability of voluntary top-down resources.

Theory of mind affects the interpretation of another person's focus of attention.

People are drawn to social, animate things more than inanimate objects. Previous research has also shown gaze following in humans, a process that has been linked to theory of mind (ToM). In three experiments, we investigated whether animacy and ToM are involved when making judgements about the location of a cursor in a scene. In Experiment 1, participants were told that this cursor represented the gaze of an observer and were asked to decide whether the observer was looking at a target object. This task is similar to that carried out by researchers manually coding eye-tracking data. The results showed that participants were biased to perceive the gaze cursor as directed towards animate objects (faces) compared to inanimate objects. In Experiments 2 and 3 we tested the role of ToM, by presenting the same scenes to new participants but now with the statement that the cursor was generated by a 'random' computer system or by a computer system designed to seek targets. The bias to report that the cursor was directed toward faces was abolished in Experiment 2, and minimised in Experiment 3. Together, the results indicate that people attach minds to the mere representation of an individual's gaze, and this attribution of mind influences what people believe an individual is looking at.

Athlete-Opponent Interdependency Alters Pacing and Information-Seeking Behavior.

PURPOSE: The influence of interdependency between competitors on pacing decision-making and information-seeking behavior has been explored. This has been done by only altering instructions, and thereby action possibilities, while controlling environment (i.e., competitor behavior) and exercise task. METHODS: Twelve participants performed a 4-km time trial on a Velotron cycle ergometer in a randomized, counterbalanced order alone with no virtual opponent (NO), against a virtual opponent with no restrictions (low athlete-opponent interdependency [OP-IND]), or against a virtual opponent who the participant was permitted to overtake only once during the trial (high athlete-opponent interdependency [OP-DEP]). Information-seeking behavior was evaluated using an SMI eye tracker. Differences in pacing, performance, and information-seeking behavior were examined using repeated-measures ANOVA (P < 0.05). RESULTS: Neither mean power output (NO, 298 ± 35 W; OP-IND, 297 ± 38 W; OP-DEP, 296 ± 37 W) nor finishing time (NO, 377.7 ± 17.4 s; OP-IND, 379.3 ± 19.5 s; OP-DEP, 378.5 ± 17.7 s) differed between experimental conditions. However, power output was lower in the first kilometer of OP-DEP compared with the other experimental conditions (NO, 332 ± 59 W; OP-IND, 325 ± 62 W; OP-DEP, 316 ± 58 W; both P < 0.05), and participants decided to wait longer before they overtook their opponent (OP-IND, 137 ± 130 s; OP-DEP, 255 ± 107 s; P = 0.040). Moreover, total fixation time spent on the avatar of the virtual opponent increased when participants were only allowed to overtake once (OP-IND, 23.3 ± 16.6 s; OP-DEP, 55.8 ± 32.7 s; P = 0.002). CONCLUSION: A higher interdependency between athlete and opponent altered pacing behavior in terms of in-race adaptations based on opponent's behavior, and it induced an increased attentional focus on the virtual opponent. Thus, in the context of exercise regulation, attentional cues are likely to be used in an adaptive way according to their availability and situational relevance, consistent with a decision-making framework based on the interdependence of perception and action.

Reading and Misleading: Changes in Head and Eye Movements Reveal Attentional Orienting in a Social Context.

Social attention describes how observers orient to social information and exhibit behaviors such as gaze following. These behaviors are examples of how attentional orienting may differ when in the presence of other people, although they have typically been studied without actual social presence. In the present study we ask whether orienting, as measured by head and eye movements, will change when participants are trying to mislead or hide their attention from a bystander. In two experiments, observers performed a preference task while being video-recorded, and subsequent participants were asked to guess the response of the participant based on a video of the head and upper body. In a second condition, observers were told to try to mislead the "guesser". The results showed that participants' preference responses could be guessed from videos of the head and, critically, that participants spontaneously changed their orienting behavior in order to mislead by reducing the rate at which they made large head movements. Masking the eyes with sunglasses suggested that head movements were most important in our setup. This indicates that head and eye movements can be used flexibly according to the socio-communicative context.