Unraveling lexical semantics in the brain: Comparing internal, external, and hybrid language models.

To explain how the human brain represents and organizes meaning, many theoretical and computational language models have been proposed over the years, varying in their underlying computational principles and in the language samples based on which they are built. However, how well they capture the neural encoding of lexical semantics remains elusive. We used representational similarity analysis (RSA) to evaluate to what extent three models of different types explained neural responses elicited by word stimuli: an External corpus-based word2vec model, an Internal free word association model, and a Hybrid ConceptNet model. Semantic networks were constructed using word relations computed in the three models and experimental stimuli were selected through a community detection procedure. The similarity patterns between language models and neural responses were compared at the community, exemplar, and word node levels to probe the potential hierarchical semantic structure. We found that semantic relations computed with the Internal model provided the closest approximation to the patterns of neural activation, whereas the External model did not capture neural responses as well. Compared with the exemplar and the node levels, community-level RSA demonstrated the broadest involvement of brain regions, engaging areas critical for semantic processing, including the angular gyrus, superior frontal gyrus and a large portion of the anterior temporal lobe. The findings highlight the multidimensional semantic organization in the brain which is better captured by Internal models sensitive to multiple modalities such as word association compared with External models trained on text corpora.

Semantic representations in inferior frontal and lateral temporal cortex during picture naming, reading, and repetition.

Reading, naming, and repetition are classical neuropsychological tasks widely used in the clinic and psycholinguistic research. While reading and repetition can be accomplished by following a direct or an indirect route, pictures can be named only by means of semantic mediation. By means of fMRI multivariate pattern analysis, we evaluated whether this well-established fundamental difference at the cognitive level is associated at the brain level with a difference in the degree to which semantic representations are activated during these tasks. Semantic similarity between words was estimated based on a word association model. Twenty subjects participated in an event-related fMRI study where the three tasks were presented in pseudo-random order. Linear discriminant analysis of fMRI patterns identified a set of regions that allow to discriminate between words at a high level of word-specificity across tasks. Representational similarity analysis was used to determine whether semantic similarity was represented in these regions and whether this depended on the task performed. The similarity between neural patterns of the left Brodmann area 45 (BA45) and of the superior portion of the left supramarginal gyrus correlated with the similarity in meaning between entities during picture naming. In both regions, no significant effects were seen for repetition or reading. The semantic similarity effect during picture naming was significantly larger than the similarity effect during the two other tasks. In contrast, several regions including left anterior superior temporal gyrus and left ventral BA44/frontal operculum, among others, coded for semantic similarity in a task-independent manner. These findings provide new evidence for the dynamic, task-dependent nature of semantic representations in the left BA45 and a more task-independent nature of the representational activation in the lateral temporal cortex and ventral BA44/frontal operculum.

The "Small World of Words" free association norms for Rioplatense Spanish.

Large-scale word association datasets are both important tools used in psycholinguistics and used as models that capture meaning when considered as semantic networks. Here, we present word association norms for Rioplatense Spanish, a variant spoken in Argentina and Uruguay. The norms were derived through a large-scale crowd-sourced continued word association task in which participants give three associations to a list of cue words. Covering over 13,000 words and +3.6 M responses, it is currently the most extensive dataset available for Spanish. We compare the obtained dataset with previous studies in Dutch and English to investigate the role of grammatical gender and studies that used Iberian Spanish to test generalizability to other Spanish variants. Finally, we evaluated the validity of our data in word processing (lexical decision reaction times) and semantic (similarity judgment) tasks. Our results demonstrate that network measures such as in-degree provide a good prediction of lexical decision response times. Analyzing semantic similarity judgments showed that results replicate and extend previous findings demonstrating that semantic similarity derived using spreading activation or spectral methods outperform word embeddings trained on text corpora.

Orienting to different dimensions of word meaning alters the representation of word meaning in early processing regions.

Conscious processing of word meaning can be guided by attention. In this event-related functional magnetic resonance imaging study in 22 healthy young volunteers, we examined in which regions orienting attention to two fundamental and generic dimensions of word meaning, concreteness versus valence, alters the semantic representations coded in activity patterns. The stimuli consisted of 120 nouns in written or spoken modality which varied factorially along the concreteness and valence axis. Participants performed a forced-choice judgement of either concreteness or valence. Rostral and subgenual anterior cingulate were strongly activated during valence judgement, and precuneus and the dorsal attention network during concreteness judgement. Task and stimulus type interacted in right posterior fusiform gyrus, left lingual gyrus, precuneus, and insula. In the right posterior fusiform gyrus and the left lingual gyrus, the correlation between the pairwise similarity in activity patterns evoked by words and the pairwise distance in valence and concreteness was modulated by the direction of attention, word valence or concreteness. The data indicate that orienting attention to basic dimensions of word meaning exerts effects on the representation of word meaning in more peripheral nodes, such as the ventral occipital cortex, rather than the core perisylvian language regions.

Semantic gender: Norms for 24,000 Dutch words and its role in word meaning.

Semantic gender norms are presented for 24,037 Dutch words. Eighty participants rated 6017 words each on a five-point Likert scale ranging from feminine to masculine. Each word was rated by ten male and ten female participants. The collected norms show high reliability and correlate well with similar norms in English. We show that semantic gender is distinct from other lexical dimensions such as valence, arousal, dominance, concreteness, and age of acquisition. Semantic gender is not the same as the grammatical gender of words, either. The collected norms can be predicted accurately using a semantic space based on word association data. A dimension explaining a good amount of variance is present in this space, indicating that semantic gender is an important component of the human meaning system.

Representation of associative and affective semantic similarity of abstract words in the lateral temporal perisylvian language regions.

The examination of semantic cognition has traditionally identified word concreteness as well as valence as two of the principal dimensions in the representation of conceptual knowledge. More recently, corpus-based vector space models as well as graph-theoretical analysis of large-scale task-related behavioural responses have revolutionized our insight into how the meaning of words is structured. In this fMRI study, we apply representational similarity analysis to investigate the conceptual representation of abstract words. Brain activity patterns were related to a cued-association based graph as well as to a vector-based co-occurrence model of word meaning. Twenty-six subjects (19 females and 7 males) performed an overt repetition task during fMRI. First, we performed a searchlight classification procedure to identify regions where activity is discriminable between abstract and concrete words. These regions were left inferior frontal gyrus, the upper and lower bank of the superior temporal sulcus bilaterally, posterior middle temporal gyrus and left fusiform gyrus. Representational Similarity Analysis demonstrated that for abstract words, the similarity of activity patterns in the cortex surrounding the superior temporal sulcus bilaterally and in the left anterior superior temporal gyrus reflects the similarity in word meaning. These effects were strongest for semantic similarity derived from the cued association-based graph and for affective similarity derived from either of the two models. The latter effect was mainly driven by positive valence words. This research highlights the close neurobiological link between the information structure of abstract and affective word content and the similarity in activity pattern in the lateral and anterior temporal language system.

A practical primer on processing semantic property norm data.

Semantic property listing tasks require participants to generate short propositions (e.g., [Formula: see text], [Formula: see text]) for a specific concept (e.g., DOG). This task is the cornerstone of the creation of semantic property norms which are essential for modeling, stimuli creation, and understanding similarity between concepts. Despite the wide applicability of semantic property norms for a large variety of concepts across different groups of people, the methodological aspects of the property listing task have received less attention, even though the procedure and processing of the data can substantially affect the nature and quality of the measures derived from them. The goal of this paper is to provide a practical primer on how to collect and process semantic property norms. We will discuss the key methods to elicit semantic properties and compare different methods to derive meaningful representations from them. This will cover the role of instructions and test context, property preprocessing (e.g., lemmatization), property weighting, and relationship encoding using ontologies. With these choices in mind, we propose and demonstrate a processing pipeline that transparently documents these steps, resulting in improved comparability across different studies. The impact of these choices will be demonstrated using intrinsic (e.g., reliability, number of properties) and extrinsic measures (e.g., categorization, semantic similarity, lexical processing). This practical primer will offer potential solutions to several long-standing problems and allow researchers to develop new property listing norms overcoming the constraints of previous studies.

Lexicosemantic, affective, and distributional norms for 1,000 Dutch adjectives.

The research of the word is still very much the research of the noun. Adjectives have been largely overlooked, despite being the second-largest word class in many languages and serving an important communicative function, because of the rich, nuanced qualifications they afford. Adjectives are also ideally suited to study the interface between cognition and emotion, as they naturally cover the entire range of lexicosemantic variables such as imageability (infinite-green), and affective variables such as valence (sad-happy). We illustrate this by showing how the centrality of words in the mental lexicon varies as a function of the words' affective dimensions, using newly collected norms for 1,000 Dutch adjectives. The norms include the lexicosemantic variables age of acquisition, familiarity, concreteness, and imageability; the affective variables valence, arousal, and dominance; and a variety of distributional variables, including network statistics resulting from a large-scale word association study. The norms are freely available from https://osf.io/nyg8v/, for researchers studying adjectives specifically or for whom adjectives constitute convenient stimuli to study other topics, such as vagueness, inference, spatial cognition, or affective word processing.

Left perirhinal cortex codes for semantic similarity between written words defined from cued word association.

Knowledge of visual and nonvisual attributes of concrete entities is distributed over neocortical uni- and polymodal association cortex. Here we investigated the role of left perirhinal cortex in explicit knowledge retrieval from written words. We examined whether it extended across visual and nonvisual properties, animate and inanimate entities, how this differed from picture input and how specific it was for perirhinal cortex compared to surrounding structures. The semantic similarity between stimuli was determined on the basis of a word association-based model. Eighteen participants participated in this event-related fMRI experiment. During property verification, the left perirhinal cortex coded for the similarity in meaning between written words. No differences were found between visual and nonvisual properties or between animate and inanimate entities. Among the surrounding regions, a semantic similarity effect for written words was also present in the left parahippocampal gyrus, but not in the hippocampus nor in the right perirhinal cortex. Univariate analysis revealed higher activity for visual property verification in visual processing regions and for nonvisual property verification in an extended system encompassing the superior temporal sulcus along its anterior-posterior axis, the inferior and the superior frontal gyrus. The association strength between the concept and the property correlated positively with fMRI response amplitude in visual processing regions, and negatively with response amplitude in left inferior and superior frontal gyrus. The current findings establish that input-modality determines the semantic similarity effect in left perirhinal cortex more than the content of the knowledge retrieved or the semantic control demand do. We propose that left perirhinal cortex codes for the association between a concrete written word and the object it refers to and operates as a connector hub linking written word input to the distributed cortical representation of word meaning.

The "Small World of Words" English word association norms for over 12,000 cue words.

Word associations have been used widely in psychology, but the validity of their application strongly depends on the number of cues included in the study and the extent to which they probe all associations known by an individual. In this work, we address both issues by introducing a new English word association dataset. We describe the collection of word associations for over 12,000 cue words, currently the largest such English-language resource in the world. Our procedure allowed subjects to provide multiple responses for each cue, which permits us to measure weak associations. We evaluate the utility of the dataset in several different contexts, including lexical decision and semantic categorization. We also show that measures based on a mechanism of spreading activation derived from this new resource are highly predictive of direct judgments of similarity. Finally, a comparison with existing English word association sets further highlights systematic improvements provided through these new norms.

Cross-modal representation of spoken and written word meaning in left pars triangularis.

The correspondence in meaning extracted from written versus spoken input remains to be fully understood neurobiologically. Here, in a total of 38 subjects, the functional anatomy of cross-modal semantic similarity for concrete words was determined based on a dual criterion: First, a voxelwise univariate analysis had to show significant activation during a semantic task (property verification) performed with written and spoken concrete words compared to the perceptually matched control condition. Second, in an independent dataset, in these clusters, the similarity in fMRI response pattern to two distinct entities, one presented as a written and the other as a spoken word, had to correlate with the similarity in meaning between these entities. The left ventral occipitotemporal transition zone and ventromedial temporal cortex, retrosplenial cortex, pars orbitalis bilaterally, and the left pars triangularis were all activated in the univariate contrast. Only the left pars triangularis showed a cross-modal semantic similarity effect. There was no effect of phonological nor orthographic similarity in this region. The cross-modal semantic similarity effect was confirmed by a secondary analysis in the cytoarchitectonically defined BA45. A semantic similarity effect was also present in the ventral occipital regions but only within the visual modality, and in the anterior superior temporal cortex only within the auditory modality. This study provides direct evidence for the coding of word meaning in BA45 and positions its contribution to semantic processing at the confluence of input-modality specific pathways that code for meaning within the respective input modalities.

Estimating affective word covariates using word association data.

Word ratings on affective dimensions are an important tool in psycholinguistic research. Traditionally, they are obtained by asking participants to rate words on each dimension, a time-consuming procedure. As such, there has been some interest in computationally generating norms, by extrapolating words' affective ratings using their semantic similarity to words for which these values are already known. So far, most attempts have derived similarity from word co-occurrence in text corpora. In the current paper, we obtain similarity from word association data. We use these similarity ratings to predict the valence, arousal, and dominance of 14,000 Dutch words with the help of two extrapolation methods: Orientation towards Paradigm Words and k-Nearest Neighbors. The resulting estimates show very high correlations with human ratings when using Orientation towards Paradigm Words, and even higher correlations when using k-Nearest Neighbors. We discuss possible theoretical accounts of our results and compare our findings with previous attempts at computationally generating affective norms.

Using the speeded word fragment completion task to examine semantic priming.

The present research investigates semantic priming with an adapted version of the word fragment completion task. In this task, which we refer to as the speeded word fragment completion task, participants need to complete words such as lett_ce (lettuce), from which one letter was omitted, as quickly as possible. This paradigm has some interesting qualities in comparison with the traditionally used lexical decision task. That is, it requires no pseudowords, it is more engaging for participants, and most importantly, it allows for a more fine-grained investigation of semantic activation. In two studies, we found that words were completed faster when the preceding trial comprised a semantically related fragment such as tom_to (tomato) than when it comprised an unrelated fragment such as guit_r (guitar). A third experiment involved a lexical decision task, to compare both paradigms. The results showed that the magnitude of the priming effect was similar, but item-level priming effects were inconsistent over tasks. Crucially, the speeded word fragment completion task obtained strong priming effects for highly frequent, central words, such as work, money, and warm, whereas the lexical decision task did not. In a final experiment featuring only short, highly frequent words, the lexical decision task failed to find a priming effect, whereas the fragment completion task did obtain a robust effect. Taken together, these results suggest that the speeded word fragment completion task may prove a viable alternative for examining semantic priming.

Similarity of fMRI activity patterns in left perirhinal cortex reflects semantic similarity between words.

How verbal and nonverbal visuoperceptual input connects to semantic knowledge is a core question in visual and cognitive neuroscience, with significant clinical ramifications. In an event-related functional magnetic resonance imaging (fMRI) experiment we determined how cosine similarity between fMRI response patterns to concrete words and pictures reflects semantic clustering and semantic distances between the represented entities within a single category. Semantic clustering and semantic distances between 24 animate entities were derived from a concept-feature matrix based on feature generation by >1000 subjects. In the main fMRI study, 19 human subjects performed a property verification task with written words and pictures and a low-level control task. The univariate contrast between the semantic and the control task yielded extensive bilateral occipitotemporal activation from posterior cingulate to anteromedial temporal cortex. Entities belonging to a same semantic cluster elicited more similar fMRI activity patterns in left occipitotemporal cortex. When words and pictures were analyzed separately, the effect reached significance only for words. The semantic similarity effect for words was localized to left perirhinal cortex. According to a representational similarity analysis of left perirhinal responses, semantic distances between entities correlated inversely with cosine similarities between fMRI response patterns to written words. An independent replication study in 16 novel subjects confirmed these novel findings. Semantic similarity is reflected by similarity of functional topography at a fine-grained level in left perirhinal cortex. The word specificity excludes perceptually driven confounds as an explanation and is likely to be task dependent.

A Computational Approach to Identifying Cultural Keywords Across Languages.

Distinctive aspects of a culture are often reflected in the meaning and usage of words in the language spoken by bearers of that culture. Keywords such as душа (soul) in Russian, hati (heart) in Indonesian and Malay, and gezellig (convivial/cosy/fun) in Dutch are held to be especially culturally revealing, and scholars have identified a number of such keywords using careful linguistic analyses (Peeters, 2020b; Wierzbicka, 1990). Because keywords are expected to have different statistical properties than related words in other languages, we argue that a quantitative comparison of word usage across languages can help to identify cultural keywords. To support this claim, we describe a computational method that compares word frequencies across languages, and apply it to both linguistic corpora and word association data. The method identifies culturally specific words that range from "obvious" examples, such as Amsterdam in Dutch, to non-obvious yet independently proposed examples, such as hati (heart) in Indonesian. We show in addition that linguistic corpora and word association data provide converging evidence about culturally specific words. Our results therefore show how computational analyses and behavioral experiments can supplement the methods previously used by linguists to identify culturally salient words across languages.

Right fusiform response patterns reflect visual object identity rather than semantic similarity.

We previously reported the neuropsychological consequences of a lesion confined to the middle and posterior part of the right fusiform gyrus (case JA) causing a partial loss of knowledge of visual attributes of concrete entities in the absence of category-selectivity (animate versus inanimate). We interpreted this in the context of a two-step model that distinguishes structural description knowledge from associative-semantic processing and implicated the lesioned area in the former process. To test this hypothesis in the intact brain, multi-voxel pattern analysis was used in a series of event-related fMRI studies in a total of 46 healthy subjects. We predicted that activity patterns in this region would be determined by the identity of rather than the conceptual similarity between concrete entities. In a prior behavioral experiment features were generated for each entity by more than 1000 subjects. Based on a hierarchical clustering analysis the entities were organised into 3 semantic clusters (musical instruments, vehicles, tools). Entities were presented as words or pictures. With foveal presentation of pictures, cosine similarity between fMRI response patterns in right fusiform cortex appeared to reflect both the identity of and the semantic similarity between the entities. No such effects were found for words in this region. The effect of object identity was invariant for location, scaling, orientation axis and color (grayscale versus color). It also persisted for different exemplars referring to a same concrete entity. The apparent semantic similarity effect however was not invariant. This study provides further support for a neurobiological distinction between structural description knowledge and processing of semantic relationships and confirms the role of right mid-posterior fusiform cortex in the former process, in accordance with previous lesion evidence.

Better explanations of lexical and semantic cognition using networks derived from continued rather than single-word associations.

In this article, we describe the most extensive set of word associations collected to date. The database contains over 12,000 cue words for which more than 70,000 participants generated three responses in a multiple-response free association task. The goal of this study was (1) to create a semantic network that covers a large part of the human lexicon, (2) to investigate the implications of a multiple-response procedure by deriving a weighted directed network, and (3) to show how measures of centrality and relatedness derived from this network predict both lexical access in a lexical decision task and semantic relatedness in similarity judgment tasks. First, our results show that the multiple-response procedure results in a more heterogeneous set of responses, which lead to better predictions of lexical access and semantic relatedness than do single-response procedures. Second, the directed nature of the network leads to a decomposition of centrality that primarily depends on the number of incoming links or in-degree of each node, rather than its set size or number of outgoing links. Both studies indicate that adequate representation formats and sufficiently rich data derived from word associations represent a valuable type of information in both lexical and semantic processing.

The Pandemic in Words: Tracking Fast Semantic Changes via a Large-Scale Word Association Task.

Most words have a variety of senses that can be added, removed, or altered over time. Understanding how they change across different contexts and time periods is crucial for revealing the role of language in social and cultural evolution. In this study we aimed to explore the collective changes in the mental lexicon as a consequence of the COVID-19 pandemic. We performed a large-scale word association experiment in Rioplatense Spanish. The data were obtained in December 2020, and compared with responses previously obtained from the Small World of Words database (SWOW-RP, Cabana et al., 2023). Three different word-association measures detected changes in a word's mental representation from Precovid to Covid. First, significantly more new associations appeared for a set of pandemic-related words. These new associations can be interpreted as incorporating new senses. For example, the word 'isolated' incorporated direct associations with 'coronavirus' and 'quarantine'. Second, when analyzing the distribution of responses, we observed a greater Kullback-Leibler divergence (i.e., relative entropy) between the Precovid and Covid periods for pandemic words. Thus, some words (e.g., 'protocol', or 'virtual') changed their overall association patterns due to the COVID-19 pandemic. Finally, using semantic similarity analysis, we evaluated the changes between the Precovid and Covid periods for each cue word's nearest neighbors and the changes in their similarity to certain word senses. We found a larger diachronic difference for pandemic cues where polysemic words like 'immunity' or 'trial' increased their similarity to sanitary/health words during the Covid period. We propose that this novel methodology can be expanded to other scenarios of fast diachronic semantic changes.

Independency of Coding for Affective Similarities and for Word Co-occurrences in Temporal Perisylvian Neocortex.

Word valence is one of the principal dimensions in the organization of word meaning. Co-occurrence-based similarities calculated by predictive natural language processing models are relatively poor at representing affective content, but very powerful in their own way. Here, we determined how these two canonical but distinct ways of representing word meaning relate to each other in the human brain both functionally and neuroanatomically. We re-analysed an fMRI study of word valence. A co-occurrence-based model was used and the correlation with the similarity of brain activity patterns was compared to that of affective similarities. The correlation between affective and co-occurrence-based similarities was low (r = 0.065), confirming that affect was captured poorly by co-occurrence modelling. In a whole-brain representational similarity analysis, word embedding similarities correlated significantly with the similarity between activity patterns in a region confined to the superior temporal sulcus to the left, and to a lesser degree to the right. Affective word similarities correlated with the similarity in activity patterns in this same region, confirming previous findings. The affective similarity effect extended more widely beyond the superior temporal cortex than the effect of co-occurrence-based similarities did. The effect of co-occurrence-based similarities remained unaltered after partialling out the effect of affective similarities (and vice versa). To conclude, different aspects of word meaning, derived from affective judgements or from word co-occurrences, are represented in superior temporal language cortex in a neuroanatomically overlapping but functionally independent manner.

Using Network Science to Understand the Aging Lexicon: Linking Individuals' Experience, Semantic Networks, and Cognitive Performance.

People undergo many idiosyncratic experiences throughout their lives that may contribute to individual differences in the size and structure of their knowledge representations. Ultimately, these can have important implications for individuals' cognitive performance. We review evidence that suggests a relationship between individual experiences, the size and structure of semantic representations, as well as individual and age differences in cognitive performance. We conclude that the extent to which experience-dependent changes in semantic representations contribute to individual differences in cognitive aging remains unclear. To help fill this gap, we outline an empirical agenda that utilizes network analysis and involves the concurrent assessment of large-scale semantic networks and cognitive performance in younger and older adults. We present preliminary data to establish the feasibility and limitations of such empirical, network-analytical approaches.