Prediction as a basis for skilled reading: insights from modern language models.

Reading is not an inborn human capability, and yet, English-speaking adults read with impressive speed. This study considered how predictions of upcoming words impact on this skilled behaviour. We used a powerful language model (GPT-2) to derive predictions of upcoming words in text passages. These predictions were highly accurate and showed a tight relationship to fine-grained aspects of eye-movement behaviour when adults read those same passages, including whether to skip the next word and how long to spend on it. Strong predictions that were incorrect resulted in a prediction error cost on fixation durations. Our findings suggest that predictions for upcoming words can be made based on the analysis of text statistics and that these predictions guide how our eyes interrogate text at very short timescales. These findings open new perspectives on reading and language comprehension and illustrate the capability of modern language models to inform understanding of human language processing.

What is semantic diversity and why does it facilitate visual word recognition?

Previous research has speculated that semantic diversity and lexical ambiguity may be closely related constructs. Our research sought to test this claim in respect of the semantic diversity measure proposed by Hoffman et al. (2013). To this end, we replicated the procedure described by Hoffman et al., Behavior Research Methods, 45(3), 718-730 (2013) for computing multidimensional representations of contextual information using Latent Semantic Analysis, and from these we derived semantic diversity values for 28,555 words. We then replicated the facilitatory effect of semantic diversity on word recognition using existing data resources and observed this effect to be greater for low-frequency words. Yet, we found no relationship between this measure and lexical ambiguity effects in word recognition. Further analysis of the LSA-based contextual representations used to compute Hoffman et al. (2013) measure of semantic diversity revealed that they do not capture the distinct meanings of ambiguous words. Instead, these contextual representations appear to capture general information about the topics and types of written material in which words occur. These analyses suggest that the semantic diversity metric previously proposed by Hoffman et al. (2013) facilitates word recognition because high-diversity words are likely to have been encountered no matter what one has read, whereas many participants may not have encountered lower-diversity words simply because the topics and types of written material in which they occur are more restricted.

Generalisation in language learning can withstand total sleep deprivation.

Research suggests that sleep plays a vital role in memory. We tested the impact of total sleep deprivation on adults' memory for a newly learned writing system and on their ability to generalise this knowledge to read untrained novel words. We trained participants to read fictitious words printed in a novel artificial orthography, while depriving them of sleep the night after learning (Experiment 1) or the night before learning (Experiment 2). Following two nights of recovery sleep, and again 10 days later, participants were tested on trained words and untrained words, and performance was compared to control groups who had not undergone sleep deprivation. Participants showed a high degree of accuracy in learning the trained words and in generalising their knowledge to untrained words. There was little evidence of impact of sleep deprivation on memory or generalisation. These data support emerging theories which suggest sleep-associated memory consolidation can be accelerated or entirely bypassed under certain conditions, and that such conditions also facilitate generalisation.