Language knowledge and event knowledge in language use.

This paper examines how semantic knowledge is used in language comprehension and in making judgments about events in the world. We contrast knowledge gleaned from prior language experience ("language knowledge") and knowledge coming from prior experience with the world ("world knowledge"). In two corpus analyses, we show that previous research linking verb aspect and event representations have confounded language and world knowledge. Then, using carefully chosen stimuli that remove this confound, we performed four experiments that manipulated the degree to which language knowledge or world knowledge should be salient and relevant to performing a task, finding in each case that participants use the type of knowledge most appropriate to the task. These results provide evidence for a highly context-sensitive and interactionist perspective on how semantic knowledge is represented and used during language processing.

Spatial versus graphical representation of distributional semantic knowledge.

Spatial distributional semantic models represent word meanings in a vector space. While able to model many basic semantic tasks, they are limited in many ways, such as their inability to represent multiple kinds of relations in a single semantic space and to directly leverage indirect relations between two lexical representations. To address these limitations, we propose a distributional graphical model that encodes lexical distributional data in a graphical structure and uses spreading activation for determining the plausibility of word sequences. We compare our model to existing spatial and graphical models by systematically varying parameters that contributing to dimensions of theoretical interest in semantic modeling. In order to be certain about what the models should be able to learn, we trained each model on an artificial corpus describing events in an artificial world simulation containing experimentally controlled verb-noun selectional preferences. The task used for model evaluation requires recovering observed selectional preferences and inferring semantically plausible but never observed verb-noun pairs. We show that the distributional graphical model performed better than all other models. Further, we argue that the relative success of this model comes from its improved ability to access the different orders of spatial representations with the spreading activation on the graph, enabling the model to infer the plausibility of noun-verb pairs unobserved in the training data. The model integrates classical ideas of representing semantic knowledge in a graph with spreading activation and more recent trends focused on the extraction of lexical distributional data from large natural language corpora. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Structured Semantic Knowledge Can Emerge Automatically from Predicting Word Sequences in Child-Directed Speech.

Previous research has suggested that distributional learning mechanisms may contribute to the acquisition of semantic knowledge. However, distributional learning mechanisms, statistical learning, and contemporary "deep learning" approaches have been criticized for being incapable of learning the kind of abstract and structured knowledge that many think is required for acquisition of semantic knowledge. In this paper, we show that recurrent neural networks, trained on noisy naturalistic speech to children, do in fact learn what appears to be abstract and structured knowledge. We trained two types of recurrent neural networks (Simple Recurrent Network, and Long Short-Term Memory) to predict word sequences in a 5-million-word corpus of speech directed to children ages 0-3 years old, and assessed what semantic knowledge they acquired. We found that learned internal representations are encoding various abstract grammatical and semantic features that are useful for predicting word sequences. Assessing the organization of semantic knowledge in terms of the similarity structure, we found evidence of emergent categorical and hierarchical structure in both models. We found that the Long Short-term Memory (LSTM) and SRN are both learning very similar kinds of representations, but the LSTM achieved higher levels of performance on a quantitative evaluation. We also trained a non-recurrent neural network, Skip-gram, on the same input to compare our results to the state-of-the-art in machine learning. We found that Skip-gram achieves relatively similar performance to the LSTM, but is representing words more in terms of thematic compared to taxonomic relations, and we provide reasons why this might be the case. Our findings show that a learning system that derives abstract, distributed representations for the purpose of predicting sequential dependencies in naturalistic language may provide insight into emergence of many properties of the developing semantic system.

Evidence of disturbances of deep levels of semantic cohesion within personal narratives in schizophrenia.

Since initial conceptualizations, schizophrenia has been thought to involve core disturbances in the ability to form complex, integrated ideas. Although this has been studied in terms of formal thought disorder, the level of involvement of altered latent semantic structure is less clear. To explore this question, we compared the personal narratives of adults with schizophrenia (n=200) to those produced by an HIV+ sample (n=55) using selected indices from Coh-Metrix. Coh-Metrix is a software system designed to compute various language usage statistics from transcribed written and spoken language documents. It differs from many other frequency-based systems in that Coh-Metrix measures a wide range of language processes, ranging from basic descriptors (e.g., total words) to indices assessing more sophisticated processes within sentences, between sentences, and across paragraphs (e.g., deep cohesion). Consistent with predictions, the narratives in schizophrenia exhibited less cohesion even after controlling for age and education. Specifically, the schizophrenia group spoke fewer words, demonstrated less connection between ideas and clauses, provided fewer causal/intentional markers, and displayed lower levels of deep cohesion. A classification model using only Coh-Metrix indices found language markers correctly classified participants in nearly three-fourths of cases. These findings suggest a particular pattern of difficulties cohesively connecting thoughts about oneself and the world results in a perceived lack of coherence in schizophrenia. These results are consistent with Bleuler's model of schizophrenia and offer a novel way to understand and measure alterations in thought and speech over time.

Distributional structure in language: contributions to noun-verb difficulty differences in infant word recognition.

What makes some words easy for infants to recognize, and other words difficult? We addressed this issue in the context of prior results suggesting that infants have difficulty recognizing verbs relative to nouns. In this work, we highlight the role played by the distributional contexts in which nouns and verbs occur. Distributional statistics predict that English nouns should generally be easier to recognize than verbs in fluent speech. However, there are situations in which distributional statistics provide similar support for verbs. The statistics for verbs that occur with the English morpheme -ing, for example, should facilitate verb recognition. In two experiments with 7.5- and 9.5-month-old infants, we tested the importance of distributional statistics for word recognition by varying the frequency of the contextual frames in which verbs occur. The results support the conclusion that distributional statistics are utilized by infant language learners and contribute to noun-verb differences in word recognition.

Organizing the space and behavior of semantic models.

Semantic models play an important role in cognitive science. These models use statistical learning to model word meanings from co-occurrences in text corpora. A wide variety of semantic models have been proposed, and the literature has typically emphasized situations in which one model outperforms another. However, because these models often vary with respect to multiple sub-processes (e.g., their normalization or dimensionality-reduction methods), it can be difficult to delineate which of these processes are responsible for observed performance differences. Furthermore, the fact that any two models may vary along multiple dimensions makes it difficult to understand where these models fall within the space of possible psychological theories. In this paper, we propose a general framework for organizing the space of semantic models. We then illustrate how this framework can be used to understand model comparisons in terms of individual manipulations along sub-processes. Using several artificial datasets we show how both representational structure and dimensionality-reduction influence a model's ability to pick up on different types of word relationships.

Toddlers Activate Lexical Semantic Knowledge in the Absence of Visual Referents: Evidence from Auditory Priming.

Language learners rapidly acquire extensive semantic knowledge, but the development of this knowledge is difficult to study, in part because it is difficult to assess young children's lexical semantic representations. In our studies, we solved this problem by investigating lexical semantic knowledge in 24-month-olds using the Head-turn Preference Procedure. In Experiment 1, looking times to a repeating spoken word stimulus (e.g., kitty-kitty-kitty) were shorter for trials preceded by a semantically related word (e.g., dog-dog-dog) than trials preceded by an unrelated word (e.g., juice-juice-juice). Experiment 2 yielded similar results using a method in which pairs of words were presented on the same trial. The studies provide evidence that young children activate of lexical semantic knowledge, and critically, that they do so in the absence of visual referents or sentence contexts. Auditory lexical priming is a promising technique for studying the development and structure of semantic knowledge in young children.