Spatiotemporal dynamics of abstract and concrete semantic representations.

Dual Coding Theories (DCT) suggest that meaning is represented in the brain by a double code: a language-derived code in the Anterior Temporal Lobe (ATL) and a sensory-derived code in perceptual and motor regions. Concrete concepts should activate both codes, while abstract ones rely solely on the linguistic code. To test these hypotheses, the present magnetoencephalography (MEG) experiment had participants judge whether visually presented words relate to the senses while we recorded brain responses to abstract and concrete semantic components obtained from 65 independently rated semantic features. Results evidenced early involvement of anterior-temporal and inferior-frontal brain areas in both abstract and concrete semantic information encoding. At later stages, occipital and occipito-temporal regions showed greater responses to concrete compared to abstract features. The present findings suggest that the concreteness of words is processed first with a transmodal/linguistic code, housed in frontotemporal brain systems, and only after with an imagistic/sensorimotor code in perceptual regions.

Access to meaning from visual input: Object and word frequency effects in categorization behavior.

Object and word recognition are both cognitive processes that transform visual input into meaning. When reading words, the frequency of their occurrence ("word frequency," WF) strongly modulates access to their meaning, as seen in recognition performance. Does the frequency of objects in our world also affect access to their meaning? With object labels available in real-world image datasets, one can now estimate the frequency of occurrence of objects in scenes ("object frequency," OF). We explored frequency effects in word and object recognition behavior by employing a natural versus man-made categorization task (Experiment 1) and a matching-mismatching priming task (Experiments 2-3). In Experiment 1, we found a WF effect for both words and objects but no OF effect. In Experiment 2, we replicated the WF effect for both stimulus types during cross-modal priming but not uni-modal priming. Moreover, in cross-modal priming, we found an OF effect for both objects and words, but with faster responses when objects occur less frequently in image datasets. We replicated this counterintuitive OF effect in Experiment 3 and suggest that better recognition of rare objects might interact with the structure of object categories: while access to the meaning of objects and words is faster when their meaning often occurs in our language, the homogeneity of object categories seems to also impact recognition, mainly when semantic processing happens in the context of previously presented information. These findings have major implications for studies attempting to include frequency measures in investigations of access to meaning from visual inputs. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Hierarchical organization of objects in scenes is reflected in mental representations of objects.

The arrangement of objects in scenes follows certain rules ("Scene Grammar"), which we exploit to perceive and interact efficiently with our environment. We have proposed that Scene Grammar is hierarchically organized: scenes are divided into clusters of objects ("phrases", e.g., the sink phrase); within every phrase, one object ("anchor", e.g., the sink) holds strong predictions about identity and position of other objects ("local objects", e.g., a toothbrush). To investigate if this hierarchy is reflected in the mental representations of objects, we collected pairwise similarity judgments for everyday object pictures and for the corresponding words. Similarity judgments were stronger not only for object pairs appearing in the same scene, but also object pairs appearing within the same phrase of the same scene as opposed to appearing in different phrases of the same scene. Besides, object pairs with the same status in the scenes (i.e., being both anchors or both local objects) were judged as more similar than pairs of different status. Comparing effects between pictures and words, we found similar, significant impact of scene hierarchy on the organization of mental representation of objects, independent of stimulus modality. We conclude that the hierarchical structure of visual environment is incorporated into abstract, domain general mental representations of the world.

A neural mechanism for contextualizing fragmented inputs during naturalistic vision.

With every glimpse of our eyes, we sample only a small and incomplete fragment of the visual world, which needs to be contextualized and integrated into a coherent scene representation. Here we show that the visual system achieves this contextualization by exploiting spatial schemata, that is our knowledge about the composition of natural scenes. We measured fMRI and EEG responses to incomplete scene fragments and used representational similarity analysis to reconstruct their cortical representations in space and time. We observed a sorting of representations according to the fragments' place within the scene schema, which occurred during perceptual analysis in the occipital place area and within the first 200 ms of vision. This schema-based coding operates flexibly across visual features (as measured by a deep neural network model) and different types of environments (indoor and outdoor scenes). This flexibility highlights the mechanism's ability to efficiently organize incoming information under dynamic real-world conditions.