Mouse cursor trajectories capture the flexible adaptivity of predictive sentence processing.

Recent psycholinguistic findings raise fundamental questions about comprehenders' ability to rationally adapt their predictions during sentence processing. Two mouse cursor tracking experiments (each N = 85) assessed this adaptivity by manipulating the reliability of verb-based semantic cues. In Experiment 1, predictive mouse cursor movements to targets (e.g., bike) versus distractors (e.g., kite) were measured while participants heard equal proportions of nonpredictive (e.g., "spot … the bike"), predictive (e.g., "ride … the bike"), and antipredictive (e.g., "fly … the bike") sentences. In Experiment 2, participants heard equal proportions of nonpredictive and antipredictive sentences. Participants were observed to flexibly adapt their predictions, such that they disengaged prediction in Experiment 1 when verb-based cues were unreliable and as likely to be disconfirmed as confirmed, while they generated adapted predictions in Experiment 2 when verb-based cues were reliably disconfirmed. However, links to individual differences in cognitive control were not observed. These results are interpreted as supporting rational theoretical approaches. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Predictive Sentence Processing at Speed: Evidence from Online Mouse Cursor Tracking.

Three online mouse cursor-tracking experiments investigated predictive sentence processing at speed. Participants viewed visual arrays with objects like a bike and kite while hearing predictive sentences like, "What the man will ride, which is shown on this page, is the bike," or non-predictive sentences like, "What the man will spot, which is shown on this page, is the bike." Based on the selectional restrictions of "ride" (i.e., vs. "spot"), participants made mouse cursor movements to the bike before hearing the noun "bike." Compellingly, this effect was observed at speech rates of ∼3 (Experiment 1), ∼6 (Experiment 2), and ∼9 (Experiment 3) syllables/s. While prior research suggests striking limits on prediction, these results highlight temporal dynamics that may impact comprehenders' ability to preactivate information when hearing impressively rapid speech. Implications for theories of sentence processing are discussed.

Continuous cursor-captured conceptual competition: Investigating the spatiotemporal dynamics of spoken word comprehension.

Semantically related concepts are coactivated during spoken word comprehension. Two internet-mediated cursor-tracking experiments examined the spatiotemporal dynamics of this coactivation. Participants viewed visual arrays containing images of a target (e.g., accordion) and a semantically related (e.g., banjo) or unrelated (e.g., plum) distractor whilst hearing the target word (e.g., "accordion"). Participants were tasked with moving their cursor from the bottom of the visual array to the target in one of the upper corners. In contrast to Experiment 1, the onset of stimulus presentation was triggered by cursor movement in Experiment 2. Across both experiments, temporal (e.g., RT) and spatial (e.g., AUC) measures revealed significantly greater attraction to images of semantically related compared with unrelated distractors. These results reveal that online cursor-tracking methods are sensitive to semantic competition and suitable for studying the activation of semantic knowledge during language comprehension.

Online mouse cursor trajectories distinguish phonological activation by linguistic and nonlinguistic sounds.

Four online mouse cursor tracking experiments (total N = 208) examined the activation of phonological representations by linguistic and nonlinguistic auditory stimuli. Participants hearing spoken words (e.g., "bell") produced less direct mouse cursor trajectories toward corresponding pictures or text when visual arrays also included phonologically related competitors (e.g., belt) as compared with unrelated distractors (e.g., hose), but no such phonological competition was observed during environmental sounds (e.g., the ring of a bell). While important similarities have been observed between spoken words and environmental sounds, these experiments provide novel mouse cursor evidence that environmental sounds directly activate conceptual knowledge without needing to engage linguistic knowledge, contrasting with spoken words. Implications for theories of conceptual knowledge are discussed.

Contextual constraints on the activation of lexical forms by nonlinguistic sounds.

Three visual world experiments investigated the activation of linguistic knowledge during the processing of nonlinguistic auditory stimuli. In Experiment 1, participants heard spoken words such as "car" or environmental sounds such as a sound produced by a car while viewing visual arrays with objects such as a car (target), card (phonologically related competitor), and box (unrelated distractor). Interleaved throughout, participants heard both spoken words and environmental sounds. In Experiments 2 and 3, to assess contextual constraints on processing, participants only heard environmental sounds while viewing similar visual arrays (targets were included in the former but not the latter). When participants heard environmental sounds interleaved among spoken words (Experiment 1), they fixated competitors significantly more than distractors during both types of auditory stimuli, suggesting that both engage linguistic systems and representations; however, when participants only heard environmental sounds (Experiments 2 and 3), phonological competition was not observed. These results suggest that the activation of linguistic knowledge by environmental sounds is context dependent rather than automatic, differing from spoken words. Implications for theories addressing the mapping of auditory stimuli onto conceptual knowledge are discussed. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Improving young children's handwashing behaviour and understanding of germs: The impact of A Germ's Journey educational resources in schools and public spaces.

CONTEXT: Effective handwashing can prevent the spread of germs, including Covid-19. However, young children can lack a fundamental understanding of germ transfer. A Germ's Journey educational resources were designed to support young children in learning about germs and handwashing. These resources include a book, website, song, online games and glo-gel activities that are informed by a behaviour change model. RESEARCH GAP: Prior research has not evaluated the impacts of these resources on behavioural outcomes. PURPOSE OF THE STUDY: Two intervention studies evaluated the impacts of these resources on both knowledge and behavioural outcomes. METHOD: In Study 1, children (n = 225) were recruited from four schools and randomly assigned by classrooms to participate in a multicomponent intervention (vs. control). In Study 2, children (n = 104) were recruited from a museum and randomly assigned to participate in a song intervention (vs. control). Trained observers recorded participants' engagement in six handwashing behaviours and their understanding of germs. These behavioural and knowledge outcomes were analysed using regression and related analyses. RESULTS: In Study 1, significant improvements were observed between baseline and follow up in the intervention group for both behavioural scores (Est = 0.48, SE = 0.14, t = 3.30, p = 0.001) and knowledge scores (Est = 2.14, SE = 0.52, z = 4.11, p < 0.001), whereas these improvements were not observed in the control group (ts < 1). In Study 2, the intervention group had significantly higher behavioural scores compared to the control group (Est. = -0.71, SE = 0.34, t = -2.07, p = 0.04). CONCLUSION: This research demonstrates that specifically designed hand hygiene educational resources can improve handwashing practice and understanding in young children, and could lead to the reduction of the transmission of disease within this group.

Lexical constraints on the prediction of form: Insights from the visual world paradigm.

Two visual world experiments investigated the priming of form (e.g., phonology) during language processing. In Experiment 1, participants heard high cloze probability sentences like "In order to have a closer look, the dentist asked the man to open his . . ." while viewing visual arrays with objects like a predictable target mouth, phonological competitor mouse, and unrelated distractors. In Experiment 2, participants heard target-associated nouns like "dentist" that were isolated from the sentences in Experiment 1 while viewing the same visual arrays. In both experiments, participants fixated the target (e.g., mouth) most, but also fixated the phonological competitor (e.g., mouse) more than unrelated distractors. Taken together, these results are interpreted as supporting association-based mechanisms in prediction, such that activation spreads across both semantics and form within the mental lexicon (e.g., dentist-mouth-mouse) and likewise primes (i.e., preactivates) the form of upcoming words during sentence processing. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Activating Semantic Knowledge During Spoken Words and Environmental Sounds: Evidence From the Visual World Paradigm.

Two visual world experiments investigated the activation of semantically related concepts during the processing of environmental sounds and spoken words. Participants heard environmental sounds such as barking or spoken words such as "puppy" while viewing visual arrays with objects such as a bone (semantically related competitor) and candle (unrelated distractor). In Experiment 1, a puppy (target) was also included in the visual array; in Experiment 2, it was not. During both types of auditory stimuli, competitors were fixated significantly more than distractors, supporting the coactivation of semantically related concepts in both cases; comparisons of the two types of auditory stimuli also revealed significantly larger effects with environmental sounds than spoken words. We discuss implications of these results for theories of semantic knowledge.

Individual differences in subphonemic sensitivity and phonological skills.

Many studies have established a link between phonological abilities (indexed by phonological awareness and phonological memory tasks) and typical and atypical reading development. Individuals who perform poorly on phonological assessments have been mostly assumed to have underspecified (or "fuzzy") phonological representations, with typical phonemic categories, but with greater category overlap due to imprecise encoding. An alternative posits that poor readers have overspecified phonological representations, with speech sounds perceived allophonically (phonetically distinct variants of a single phonemic category). On both accounts, mismatch between phonological categories and orthography leads to reading difficulty. Here, we consider the implications of these accounts for online speech processing. We used eye tracking and an individual differences approach to assess sensitivity to subphonemic detail in a community sample of young adults with a wide range of reading-related skills. Subphonemic sensitivity inversely correlated with meta-phonological task performance, consistent with overspecification.

The Influence of Globally Ungrammatical Local Syntactic Constraints on Real-Time Sentence Comprehension: Evidence From the Visual World Paradigm and Reading.

We investigated the influence of globally ungrammatical local syntactic constraints on sentence comprehension, as well as the corresponding activation of global and local representations. In Experiment 1, participants viewed visual scenes with objects like a carousel and motorbike while hearing sentences with noun phrase (NP) or verb phrase (VP) modifiers like "The girl who likes the man (from London/very much) will ride the carousel." In both cases, "girl" and "ride" predicted carousel as the direct object; however, the locally coherent combination "the man from London will ride…" in NP cases alternatively predicted motorbike. During "ride," local constraints, although ruled out by the global constraints, influenced prediction as strongly as global constraints: While motorbike was fixated less than carousel in VP cases, it was fixated as much as carousel in NP cases. In Experiment 2, these local constraints likewise slowed reading times. We discuss implications for theories of sentence processing.

The real-time prediction and inhibition of linguistic outcomes: Effects of language and literacy skill.

Recent studies have found considerable individual variation in language comprehenders' predictive behaviors, as revealed by their anticipatory eye movements during language comprehension. The current study investigated the relationship between these predictive behaviors and the language and literacy skills of a diverse, community-based sample of young adults. We found that rapid automatized naming (RAN) was a key determinant of comprehenders' prediction ability (e.g., as reflected in predictive eye movements to a white cake on hearing "The boy will eat the white…"). Simultaneously, comprehension-based measures predicted participants' ability to inhibit eye movements to objects that shared features with predictable referents but were implausible completions (e.g., as reflected in eye movements to a white but inedible white car). These findings suggest that the excitatory and inhibitory mechanisms that support prediction during language processing are closely linked with specific cognitive abilities that support literacy. We show that a self-organizing cognitive architecture captures this pattern of results.

Event processing in the visual world: Projected motion paths during spoken sentence comprehension.

Motion events in language describe the movement of an entity to another location along a path. In 2 eye-tracking experiments, we found that comprehension of motion events involves the online construction of a spatial mental model that integrates language with the visual world. In Experiment 1, participants listened to sentences describing the movement of an agent to a goal while viewing visual scenes depicting the agent, goal, and empty space in between. Crucially, verbs suggested either upward (e.g., jump) or downward (e.g., crawl) paths. We found that in the rare event of fixating the empty space between the agent and goal, visual attention was biased upward or downward in line with the verb. In Experiment 2, visual scenes depicted a central obstruction, which imposed further constraints on the paths and increased the likelihood of fixating the empty space between the agent and goal. The results from this experiment corroborated and refined the previous findings. Specifically, eye-movement effects started immediately after hearing the verb and were in line with data from an additional mouse-tracking task that encouraged a more explicit spatial reenactment of the motion event. In revealing how event comprehension operates in the visual world, these findings suggest a mental simulation process whereby spatial details of motion events are mapped onto the world through visual attention. The strength and detectability of such effects in overt eye-movements is constrained by the visual world and the fact that perceivers rarely fixate regions of empty space. (PsycINFO Database Record

Knowing what, where, and when: event comprehension in language processing.

We investigated the retrieval of location information, and the deployment of attention to these locations, following (described) event-related location changes. In two visual world experiments, listeners viewed arrays with containers like a bowl, jar, pan, and jug, while hearing sentences like "The boy will pour the sweetcorn from the bowl into the jar, and he will pour the gravy from the pan into the jug. And then, he will taste the sweetcorn". At the discourse-final "sweetcorn", listeners fixated context-relevant "Target" containers most (jar). Crucially, we also observed two forms of competition: listeners fixated containers that were not directly referred to but associated with "sweetcorn" (bowl), and containers that played the same role as Targets (goals of moving events; jug), more than distractors (pan). These results suggest that event-related location changes are encoded across representations that compete for comprehenders' attention, such that listeners retrieve, and fixate, locations that are not referred to in the unfolding language, but related to them via object or role information.

Low working memory capacity is only spuriously related to poor reading comprehension.

Accounts of comprehension failure, whether in the case of readers with poor skill or when syntactic complexity is high, have overwhelmingly implicated working memory capacity as the key causal factor. However, extant research suggests that this position is not well supported by evidence on the span of active memory during online sentence processing, nor is it well motivated by models that make explicit claims about the memory mechanisms that support language processing. The current study suggests that sensitivity to interference from similar items in memory may provide a better explanation of comprehension failure. Through administration of a comprehensive skill battery, we found that the previously observed association of working memory with comprehension is likely due to the collinearity of working memory with many other reading-related skills, especially IQ. In analyses which removed variance shared with IQ, we found that receptive vocabulary knowledge was the only significant predictor of comprehension performance in our task out of a battery of 24 skill measures. In addition, receptive vocabulary and non-verbal memory for serial order-but not simple verbal memory or working memory-were the only predictors of reading times in the region where interference had its primary affect. We interpret these results in light of a model that emphasizes retrieval interference and the quality of lexical representations as key determinants of successful comprehension.

Lexical interference effects in sentence processing: evidence from the visual world paradigm and self-organizing models.

Psycholinguistic research spanning a number of decades has produced diverging results with regard to the nature of constraint integration in online sentence processing. For example, evidence that language users anticipatorily fixate likely upcoming referents in advance of evidence in the speech signal supports rapid context integration. By contrast, evidence that language users activate representations that conflict with contextual constraints, or only indirectly satisfy them, supports nonintegration or late integration. Here we report on a self-organizing neural network framework that addresses 1 aspect of constraint integration: the integration of incoming lexical information (i.e., an incoming word) with sentence context information (i.e., from preceding words in an unfolding utterance). In 2 simulations, we show that the framework predicts both classic results concerned with lexical ambiguity resolution (Swinney, 1979; Tanenhaus, Leiman, & Seidenberg, 1979), which suggest late context integration, and results demonstrating anticipatory eye movements (e.g., Altmann & Kamide, 1999), which support rapid context integration. We also report 2 experiments using the visual world paradigm that confirm a new prediction of the framework. Listeners heard sentences like "The boy will eat the white …" while viewing visual displays with objects like a white cake (i.e., a predictable direct object of "eat"), white car (i.e., an object not predicted by "eat," but consistent with "white"), and distractors. In line with our simulation predictions, we found that while listeners fixated white cake most, they also fixated white car more than unrelated distractors in this highly constraining sentence (and visual) context.

Impulse processing: a dynamical systems model of incremental eye movements in the visual world paradigm.

The Visual World Paradigm (VWP) presents listeners with a challenging problem: They must integrate two disparate signals, the spoken language and the visual context, in support of action (e.g., complex movements of the eyes across a scene). We present Impulse Processing, a dynamical systems approach to incremental eye movements in the visual world that suggests a framework for integrating language, vision, and action generally. Our approach assumes that impulses driven by the language and the visual context impinge minutely on a dynamical landscape of attractors corresponding to the potential eye-movement behaviors of the system. We test three unique predictions of our approach in an empirical study in the VWP, and describe an implementation in an artificial neural network. We discuss the Impulse Processing framework in relation to other models of the VWP.

The time course of anticipatory constraint integration.

Several studies have demonstrated that as listeners hear sentences describing events in a scene, their eye movements anticipate upcoming linguistic items predicted by the unfolding relationship between scene and sentence. While this may reflect active prediction based on structural or contextual expectations, the influence of local thematic priming between words has not been fully examined. In Experiment 1, we presented verbs (e.g., arrest) in active (Subject-Verb-Object) sentences with displays containing verb-related patients (e.g., crook) and agents (e.g., policeman). We examined patient and agent fixations following the verb, after the agent role had been filled by another entity, but prior to bottom-up specification of the object. Participants were nearly as likely to fixate agents "anticipatorily" as patients, even though the agent role was already filled. However, the patient advantage suggested simultaneous influences of both local priming and active prediction. In Experiment 2, using passive sentences (Object-Verb-Subject), we found stronger, but still graded influences of role prediction when more time elapsed between verb and target, and more syntactic cues were available. We interpret anticipatory fixations as emerging from constraint-based processes that involve both non-predictive thematic priming and active prediction.