Dual Input Stream Transformer for Vertical Drift Correction in Eye-tracking Reading Data.

We introduce a novel Dual Input Stream Transformer (DIST) for the challenging problem of assigning fixation points from eye-tracking data collected during passage reading to the line of text that the reader was actually focused on. This post-processing step is crucial for analysis of the reading data due to the presence of noise in the form of vertical drift. We evaluate DIST against eleven classical approaches on a comprehensive suite of nine diverse datasets. We demonstrate that combining multiple instances of the DIST model in an ensemble achieves high accuracy across all datasets. Further combining the DIST ensemble with the best classical approach yields an average accuracy of 98.17 %. Our approach presents a significant step towards addressing the bottleneck of manual line assignment in reading research. Through extensive analysis and ablation studies, we identify key factors that contribute to DIST's success, including the incorporation of line overlap features and the use of a second input stream. Via rigorous evaluation, we demonstrate that DIST is robust to various experimental setups, making it a safe first choice for practitioners in the field.

Unexpected sounds inhibit the movement of the eyes during reading and letter scanning.

Novel sounds that unexpectedly deviate from a repetitive sound sequence are well known to cause distraction. Such unexpected sounds have also been shown to cause global motor inhibition, suggesting that they trigger a neurophysiological response aimed at stopping ongoing actions. Recently, evidence from eye movements has suggested that unexpected sounds also temporarily pause the movements of the eyes during reading, though it is unclear if this effect is due to inhibition of oculomotor planning or inhibition of language processes. Here, we sought to distinguish between these two possibilities by comparing a natural reading task to a letter scanning task that involves similar oculomotor demands to reading, but no higher level lexical processing. Participants either read sentences for comprehension or scanned letter strings of these sentences for the letter 'o' in three auditory conditions: silence, standard, and novel sounds. The results showed that novel sounds were equally distracting in both tasks, suggesting that they generally inhibit ongoing oculomotor processes independent of lexical processing. These results suggest that novel sounds may have a global suppressive effect on eye-movement control.

Parafoveal processing and transposed-letter effects in dyslexic reading.

During parafoveal processing, skilled readers encode letter identity independently of letter position (Johnson et al., 2007). In the current experiment, we examined orthographic parafoveal processing in readers with dyslexia. Specifically, the eye movements of skilled readers and adult readers with dyslexia were recorded during a boundary paradigm experiment (Rayner, 1975). Parafoveal previews were either identical to the target word (e.g., nearly), a transposed-letter preview (e.g., enarly), or a substituted-letter preview (e.g., acarly). Dyslexic and non-dyslexic readers demonstrated orthographic parafoveal preview benefits during silent sentence reading and both reading groups encoded letter identity and letter position information parafoveally. However, dyslexic adults showed, that very early in lexical processing, during parafoveal preview, the positional information of a word's initial letters were encoded less flexibly compared to during skilled adult reading. We suggest that dyslexic readers are less able to benefit from correct letter identity information (i.e., in the letter transposition previews) due to the lack of direct mapping of orthography to phonology. The current findings demonstrate that dyslexic readers show consistent and dyslexic-specific reading difficulties in foveal and parafoveal processing during silent sentence reading.

Return-sweep saccades in oral reading.

Recent research on return-sweep saccades has improved our understanding of eye movements when reading paragraphs. However, these saccades, which take our gaze from the end of one line to the start of the next line, have been studied only within the context of silent reading. Articulatory demands and the coordination of the eye-voice span (EVS) at line boundaries suggest that the execution of this saccade may be different in oral reading. We compared launch and landing positions of return-sweeps, corrective saccade probability and fixations adjacent to return-sweeps in skilled adult readers while reading paragraphs aloud and silently. Compared to silent reading, return-sweeps were launched from closer to the end of the line and landed closer to the start of the next line when reading aloud. The probability of making a corrective saccade was higher for oral reading than silent reading. These indicate that oral reading may compel readers to rely more on foveal processing at the expense of parafoveal processing. We found an interaction between reading modality and fixation type on fixation durations. The reading modality effect (i.e., increased fixation durations in oral compared to silent reading) was greater for accurate line-initial fixations and marginally greater for line-final fixations compared to intra-line fixations. This suggests that readers may use the fixations adjacent to return-sweeps as natural pause locations to modulate the EVS.

Distraction by auditory novelty during reading: Evidence for disruption in saccade planning, but not saccade execution.

Novel or unexpected sounds that deviate from an otherwise repetitive sequence of the same sound cause behavioural distraction. Recent work has suggested that distraction also occurs during reading as fixation durations increased when a deviant sound was presented at the fixation onset of words. The present study tested the hypothesis that this increase in fixation durations occurs due to saccadic inhibition. This was done by manipulating the temporal onset of sounds relative to the fixation onset of words in the text. If novel sounds cause saccadic inhibition, they should be more distracting when presented during the second half of fixations when saccade programming usually takes place. Participants read single sentences and heard a 120 ms sound when they fixated five target words in the sentence. On most occasions (p = .9), the same sine wave tone was presented ("standard"), while on the remaining occasions (p = .1) a new sound was presented ("novel"). Critically, sounds were played, on average, either during the first half of the fixation (0 ms delay) or during the second half of the fixation (120 ms delay). Consistent with the saccadic inhibition hypothesis (SIH), novel sounds led to longer fixation durations in the 120 ms compared to the 0 ms delay condition. However, novel sounds did not generally influence the execution of the subsequent saccade. These results suggest that unexpected sounds have a rapid influence on saccade planning, but not saccade execution.

Undersweep fixations during reading in adults and children.

Return sweeps take a reader's fixation from the end of one line to the start of the next. Return sweeps frequently undershoot their target and are followed by a corrective saccade toward the left margin. The pauses prior to corrective saccades are typically considered to be uninvolved in linguistic processing. However, recent findings indicate that these undersweep fixations influence skilled adult readers' subsequent reading pass across the line and provide preview of line-initial words. The current research examined these effects in children. First, a children's reading corpus analysis revealed that words receiving an undersweep fixation were more likely skipped and received shorter gaze durations during a subsequent pass. Second, a novel eye movement experiment that directly compared adults' and children's eye movements indicated that, during an undersweep fixation, readers very briefly allocate their attention to the fixated word-as indicated by inhibition of return effects during a subsequent pass-prior to deploying attention toward the line-initial word. We argue that prior to the redeployment of attention, readers extract information at the point of fixation that facilitates later encoding and saccade targeting. Given similar patterns of results for adults and children, we conclude that the mechanisms controlling for oculomotor coordination and attention necessary for reading across line boundaries are established from a very early point in reading development.

Reading is disrupted by intelligible background speech: Evidence from eye-tracking.

It is not well understood whether background speech affects the initial processing of words during reading or only the later processes of sentence integration. Additionally, it is not clear how eye movements support text comprehension in the face of distraction by background speech and noise. In the present research, participants read single sentences (Experiment 1) and short paragraphs (Experiments 2-3) in 4 sound conditions: silence, speech-spectrum Gaussian noise, English speech (intelligible to participants), and Mandarin speech (unintelligible to participants). Intelligible speech did not affect the lexical access of words and had a limited effect on the first-pass fixations of words. However, it led to more regressions and more rereading fixations compared with both unintelligible speech and silence. The results suggested that the distraction is mostly semantic in nature, and there was only limited evidence for a contribution of phonology. Finally, intelligible speech disrupted comprehension only when participants were prevented from rereading previous words. These findings suggest that the semantic properties of irrelevant speech can disrupt the ongoing reading process, but that this disruption occurs in the postlexical stages of reading when participants need to integrate words to form the sentence context and to construct a coherent discourse of the text. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Binocular coordination and return-sweep saccades among skilled adult readers.

During reading, binocular coordination ensures that a unified perceptual representation of the text is maintained across eye movements. However, slight vergence errors exist. The magnitude of disparity at fixation onset is related to the length of the preceding saccade. Return-sweeps are saccadic eye movements that span a line of text and direct gaze from the end of one line to the start of the next. As these eye movements travel much farther than intraline saccades, increased binocular disparity following a return-sweep is likely. Indeed, increased disparity has been a proposed explanation for longer line-initial fixations. Thus, we sought to address the following questions: Is binocular disparity larger following a return-sweep saccade than it is following an intraline saccade, and is the duration of a line-initial fixation related to binocular disparity and coordination processes? To examine these questions, we recorded binocular eye movements as participants read multiline texts. We report that, following return-sweeps, the magnitude of disparity at fixation onset is increased. However, this increased magnitude of disparity is unrelated to the duration of line-initial fixations. We argue that longer line-initial fixations result from a lack of parafoveal preview for words at the start of the line.

Return-sweep saccades during reading in adults and children.

During reading, eye movement patterns differ between children and adults. Children make more fixations that are longer in duration and make shorter saccades. Return-sweeps are saccadic eye movements that move a reader's fixation to a new line of text. Return-sweeps move fixation further than intra-line saccades and often undershoot their target. This necessitates a corrective saccade to bring fixation closer to the start of the line. There have been few empirical investigations of return-sweep saccades in adults, and even fewer in children. In the present study, we examined return-sweeps of 47 adults and 48 children who read identical multiline texts. We found that children launch their return-sweeps closer to the end of the line and target a position closer to the left margin. Therefore, children fixate more extreme positions on the screen when reading for comprehension. Furthermore, children required a corrective saccade following a return-sweep more often than adults. Analysis of the duration of the fixation preceding the corrective saccade indicated that children are as efficient as adults at responding to retinal feedback following a saccade. Rather than consider differences in adult's and children's return-sweep behaviour an artefact of oculomotor control, we believe that these differences represent adult's ability to utilise parafoveal processing to encode text at extreme positions.

Distraction by deviant sounds during reading: An eye-movement study.

Oddball studies have shown that sounds unexpectedly deviating from an otherwise repeated sequence capture attention away from the task at hand. While such distraction is typically regarded as potentially important in everyday life, previous work has so far not examined how deviant sounds affect performance on more complex daily tasks. In this study, we developed a new method to examine whether deviant sounds can disrupt reading performance by recording participants' eye movements. Participants read single sentences in silence and while listening to task-irrelevant sounds. In the latter condition, a 50-ms sound was played contingent on the fixation of five target words in the sentence. On most occasions, the same tone was presented (standard sound), whereas on rare and unexpected occasions it was replaced by white noise (deviant sound). The deviant sound resulted in significantly longer fixation durations on the target words relative to the standard sound. A time-course analysis showed that the deviant sound began to affect fixation durations around 180 ms after fixation onset. Furthermore, deviance distraction was not modulated by the lexical frequency of target words. In summary, fixation durations on the target words were longer immediately after the presentation of the deviant sound, but there was no evidence that it interfered with the lexical processing of these words. The present results are in line with the recent proposition that deviant sounds yield a temporary motor suppression and suggest that deviant sounds likely inhibit the programming of the next saccade.

Auditory Distraction During Reading: A Bayesian Meta-Analysis of a Continuing Controversy.

Everyday reading occurs in different settings, such as on the train to work, in a busy cafeteria, or at home while listening to music. In these situations, readers are exposed to external auditory stimulation from nearby noise, speech, or music that may distract them from their task and reduce their comprehension. Although many studies have investigated auditory-distraction effects during reading, the results have proved to be inconsistent and sometimes even contradictory. In addition, the broader theoretical implications of the findings have not always been explicitly considered. We report a Bayesian meta-analysis of 65 studies on auditory-distraction effects during reading and use metaregression models to test predictions derived from existing theories. The results showed that background noise, speech, and music all have a small but reliably detrimental effect on reading performance. The degree of disruption in reading comprehension did not generally differ between adults and children. Intelligible speech and lyrical music resulted in the biggest distraction. Although this last result is consistent with theories of semantic distraction, there was also reliable distraction by noise. It is argued that new theoretical models are needed that can account for distraction by both background speech and noise.

Comments on: "What Is Developmental Dyslexia?" Brain Sci. 2018, 8, 26. The Relationship between Eye Movements and Reading Difficulties.

We are writing in response to the review article: Stein. J. (2018). What is Developmental Dyslexia? Brain Sciences, 8, 26, doi:10.3390/brainsci8020026. We consider that the section entitled, "Eye Movement Control", presents a misleading characterisation of current empirical and theoretical understanding. We outline five specific points relating to Stein's views on eye movement control and developmental dyslexia with which we disagree and conclude that disruption to oculomotor behaviour occurs as a consequence of processing difficulty that individuals with dyslexia experience as they engage in reading.

What are the costs of degraded parafoveal previews during silent reading?

It has been suggested that the preview benefit effect is actually a combination of preview benefit and preview costs. Marx et al. (2015) proposed that visually degrading the parafoveal preview reduces the costs associated with traditional parafoveal letter masks used in the boundary paradigm (Rayner, 1975), thus leading to a more neutral baseline. We report 2 experiments of skilled adults reading silently. In Experiment 1, we found no compelling evidence that degraded previews reduced processing costs associated with traditional letter masks. Moreover, participants were highly sensitive to detecting degraded display changes. Experiment 2 used the boundary detection paradigm (Slattery, Angele, & Rayner, 2011) to explore whether participants were capable of detecting actual letter changes or if they were responding purely to changes in degradation. Half of the participants were instructed to respond to any noticed display changes; the other half were instructed to respond only to changes in letter identities. Participants were highly sensitive to degraded changes. In fact, these changes were so apparent that they reduced the sensitivity to letter masks. In the context of the model proposed by Angele, Slattery, and Rayner (2016), we suggest that degraded previews interfere with the attentional stage, as evidenced by the general lack of foveal load effects. In summary, we found that increasingly degrading parafoveal letter masks does not reduce their processing costs in adults, but that both degraded valid and invalid previews introduce additional costs in terms of greater display change awareness. (PsycINFO Database Record

The influence of word shading and word length on eye movements during reading.

An interesting issue in reading is how parafoveal information affects saccadic targeting and fixation durations. We investigated the influence of shading selected regions of text on eye movements during reading of long and short words within sentences. A target word, either four- or eight-letters long, was presented in one of four shading conditions: the whole target word shaded; the first half shaded; second half shaded; no shading. There was no evidence of a visually mediated parafoveal-on-foveal effect. Saccadic targeting was modulated by the shading on the first half of the word, such that fixations landed closer to the beginning of the word than in the other three shading conditions. Furthermore, partial word shading, resulting in visual non-uniformity of the target word, produced longer gaze durations than the other conditions. Finally, readers spent more time re-reading target words when they were partially shaded than in the other two conditions. We suggest that our effects are due to targeting of the optimal viewing location and revisits to check words that appear visually unusual. Together, the results indicate robust effects of low-level visual characteristics of the word on oculomotor decisions of where and when to move the eyes during reading.

Binocular coordination: reading stereoscopic sentences in depth.

The present study employs a stereoscopic manipulation to present sentences in three dimensions to subjects as they read for comprehension. Subjects read sentences with (a) no depth cues, (b) a monocular depth cue that implied the sentence loomed out of the screen (i.e., increasing retinal size), (c) congruent monocular and binocular (retinal disparity) depth cues (i.e., both implied the sentence loomed out of the screen) and (d) incongruent monocular and binocular depth cues (i.e., the monocular cue implied the sentence loomed out of the screen and the binocular cue implied it receded behind the screen). Reading efficiency was mostly unaffected, suggesting that reading in three dimensions is similar to reading in two dimensions. Importantly, fixation disparity was driven by retinal disparity; fixations were significantly more crossed as readers progressed through the sentence in the congruent condition and significantly more uncrossed in the incongruent condition. We conclude that disparity depth cues are used on-line to drive binocular coordination during reading.

Reading text increases binocular disparity in dyslexic children.

Children with developmental dyslexia show reading impairment compared to their peers, despite being matched on IQ, socio-economic background, and educational opportunities. The neurological and cognitive basis of dyslexia remains a highly debated topic. Proponents of the magnocellular theory, which postulates abnormalities in the M-stream of the visual pathway cause developmental dyslexia, claim that children with dyslexia have deficient binocular coordination, and this is the underlying cause of developmental dyslexia. We measured binocular coordination during reading and a non-linguistic scanning task in three participant groups: adults, typically developing children, and children with dyslexia. A significant increase in fixation disparity was observed for dyslexic children solely when reading. Our study casts serious doubts on the claims of the magnocellular theory. The exclusivity of increased fixation disparity in dyslexics during reading might be a result of the allocation of inadequate attentional and/or cognitive resources to the reading process, or suboptimal linguistic processing per se.

Binocular coordination during scanning of simple dot stimuli.

We examined the influence of a variety of visual factors on binocular coordination during saccadic orienting. Some experimental conditions placed similar demands on the oculomotor system as those that occur during reading, but in the absence of linguistic processing. We examined whether saccade target extent, preceding saccade magnitude, preceding saccade direction, and parafoveal availability of saccade target influenced fixation disparity. Disparities similar in magnitude and frequency to those obtained in previous binocular reading experiments occurred. Saccade magnitude had a robust influence upon fixation disparities. The results are very similar to those obtained in investigations of binocular coordination during reading, and indicate that similar patterns occur during reading-like eye scanning behaviour, in the absence of linguistic processing.

Binocular coordination during reading and non-reading tasks.

The goal of this review is to evaluate the literature on binocular coordination during reading and non-reading tasks in adult, child, and dyslexic populations. The review begins with a description of the basic characteristics of eye movements during reading. Then, reading and non-reading studies investigating binocular coordination are evaluated. Areas of future research in the field are identified and discussed. Finally, some general conclusions are made regarding binocular coordination. The review demonstrates that findings from traditionally independent areas of research are largely consistent and complementary. Throughout the review, theoretical and methodological commonalities are identified and clarified in order to advance current understanding of this fundamental aspect of human visual processing.