Mechanism of semantic processing of lexicalized and novel compound words: An eye movement study.

The integration of semantic information of compound words with context is a crucial aspect of reading comprehension. In two eye-tracking experiments, we used two-character and four-character Chinese lexicalized and novel compound words to investigate how Chinese readers integrate semantic information of compound words with contexts in the present study. By manipulating the temporary plausibility of the first constituent through varying the preceding verb, we aimed to investigate how readers process semantic information of compound words during normal reading. A significant plausibility effect pattern in the first constituent region was observed for the four-character novel words, but not for the lexicalized compound words and two-character novel compound words. However, for both two-character and four-character novel compound words, a reverse plausibility effect was found in the second constituent region. This was not the case for lexicalized compound words. These results indicate that novel compound words are integrated with the context in a decompositional manner, while lexicalized compound words are integrated holistically. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Language Models Explain Word Reading Times Better Than Empirical Predictability.

Though there is a strong consensus that word length and frequency are the most important single-word features determining visual-orthographic access to the mental lexicon, there is less agreement as how to best capture syntactic and semantic factors. The traditional approach in cognitive reading research assumes that word predictability from sentence context is best captured by cloze completion probability (CCP) derived from human performance data. We review recent research suggesting that probabilistic language models provide deeper explanations for syntactic and semantic effects than CCP. Then we compare CCP with three probabilistic language models for predicting word viewing times in an English and a German eye tracking sample: (1) Symbolic n-gram models consolidate syntactic and semantic short-range relations by computing the probability of a word to occur, given two preceding words. (2) Topic models rely on subsymbolic representations to capture long-range semantic similarity by word co-occurrence counts in documents. (3) In recurrent neural networks (RNNs), the subsymbolic units are trained to predict the next word, given all preceding words in the sentences. To examine lexical retrieval, these models were used to predict single fixation durations and gaze durations to capture rapidly successful and standard lexical access, and total viewing time to capture late semantic integration. The linear item-level analyses showed greater correlations of all language models with all eye-movement measures than CCP. Then we examined non-linear relations between the different types of predictability and the reading times using generalized additive models. N-gram and RNN probabilities of the present word more consistently predicted reading performance compared with topic models or CCP. For the effects of last-word probability on current-word viewing times, we obtained the best results with n-gram models. Such count-based models seem to best capture short-range access that is still underway when the eyes move on to the subsequent word. The prediction-trained RNN models, in contrast, better predicted early preprocessing of the next word. In sum, our results demonstrate that the different language models account for differential cognitive processes during reading. We discuss these algorithmically concrete blueprints of lexical consolidation as theoretically deep explanations for human reading.

Simple Co-Occurrence Statistics Reproducibly Predict Association Ratings.

What determines human ratings of association? We planned this paper as a test for association strength (AS) that is derived from the log likelihood that two words co-occur significantly more often together in sentences than is expected from their single word frequencies. We also investigated the moderately correlated interactions of word frequency, emotional valence, arousal, and imageability of both words (r's ≤ .3). In three studies, linear mixed effects models revealed that AS and valence reproducibly account for variance in the human ratings. To understand further correlated predictors, we conducted a hierarchical cluster analysis and examined the predictors of four clusters in competitive analyses: Only AS and word2vec skip-gram cosine distances reproducibly accounted for variance in all three studies. The other predictors of the first cluster (number of common associates, (positive) point-wise mutual information, and word2vec CBOW cosine) did not reproducibly explain further variance. The same was true for the second cluster (word frequency and arousal); the third cluster (emotional valence and imageability); and the fourth cluster (consisting of joint frequency only). Finally, we discuss emotional valence as an important dimension of semantic space. Our results suggest that a simple definition of syntagmatic word contiguity (AS) and a paradigmatic measure of semantic similarity (skip-gram cosine) provide the most general performance-independent explanation of association ratings.

A novel co-occurrence-based approach to predict pure associative and semantic priming.

The theoretical "difficulty in separating association strength from [semantic] feature overlap" has resulted in inconsistent findings of either the presence or absence of "pure" associative priming in recent literature (Hutchison, 2003, Psychonomic Bulletin & Review, 10(4), p. 787). The present study used co-occurrence statistics of words in sentences to provide a full factorial manipulation of direct association (strong/no) and the number of common associates (many/no) of the prime and target words. These common associates were proposed to serve as semantic features for a recent interactive activation model of semantic processing (i.e., the associative read-out model; Hofmann & Jacobs, 2014). With stimulus onset asynchrony (SOA) as an additional factor, our findings indicate that associative and semantic priming are indeed dissociable. Moreover, the effect of direct association was strongest at a long SOA (1,000 ms), while many common associates facilitated lexical decisions primarily at a short SOA (200 ms). This response pattern is consistent with previous performance-based accounts and suggests that associative and semantic priming can be evoked by computationally determined direct and common associations.

An adaptive annotation approach for biomedical entity and relation recognition.

In this article, we demonstrate the impact of interactive machine learning: we develop biomedical entity recognition dataset using a human-into-the-loop approach. In contrary to classical machine learning, human-in-the-loop approaches do not operate on predefined training or test sets, but assume that human input regarding system improvement is supplied iteratively. Here, during annotation, a machine learning model is built on previous annotations and used to propose labels for subsequent annotation. To demonstrate that such interactive and iterative annotation speeds up the development of quality dataset annotation, we conduct three experiments. In the first experiment, we carry out an iterative annotation experimental simulation and show that only a handful of medical abstracts need to be annotated to produce suggestions that increase annotation speed. In the second experiment, clinical doctors have conducted a case study in annotating medical terms documents relevant for their research. The third experiment explores the annotation of semantic relations with relation instance learning across documents. The experiments validate our method qualitatively and quantitatively, and give rise to a more personalized, responsive information extraction technology.

Remembering words in context as predicted by an associative read-out model.

Interactive activation models (IAMs) simulate orthographic and phonological processes in implicit memory tasks, but they neither account for associative relations between words nor explicit memory performance. To overcome both limitations, we introduce the associative read-out model (AROM), an IAM extended by an associative layer implementing long-term associations between words. According to Hebbian learning, two words were defined as "associated" if they co-occurred significantly often in the sentences of a large corpus. In a study-test task, a greater amount of associated items in the stimulus set increased the "yes" response rates of non-learned and learned words. To model test-phase performance, the associative layer is initialized with greater activation for learned than for non-learned items. Because IAMs scale inhibitory activation changes by the initial activation, learned items gain a greater signal variability than non-learned items, irrespective of the choice of the free parameters. This explains why the slope of the z-transformed receiver-operating characteristics (z-ROCs) is lower one during recognition memory. When fitting the model to the empirical z-ROCs, it likewise predicted which word is recognized with which probability at the item-level. Since many of the strongest associates reflect semantic relations to the presented word (e.g., synonymy), the AROM merges form-based aspects of meaning representation with meaning relations between words.