Recruitment of Control and Representational Components of the Semantic System during Successful and Unsuccessful Access to Complex Factual Knowledge.

Our ability to effectively retrieve complex semantic knowledge meaningfully impacts our daily lives, yet the neural processes that underly successful access and transient failures in access remain only partially understood. In this fMRI study, we contrast activation during successful semantic access, unsuccessful semantic access because of transient access failures (i.e., "tip-of-the-tongue," "feeling-of-knowing"), and trials where the semantic knowledge was not possessed. Twenty-four human participants (14 female) were presented 240 trivia-based questions relating to person, place, object, or scholastic knowledge domains. Analyses of the recall event indicated a relatively greater role of a dorsomedial section of the prefrontal cortex in unsuccessful semantic access and relatively greater recruitment of the pars orbitalis of the inferior frontal gyrus in successful access. Successful access was also associated with increased activation in knowledge domain-selective areas. Generally, knowledge domain-selective areas showed increased responses for both preferred and nonpreferred stimulus classes. The exception was place-selective regions (parahippocampal gyrus, transverse occipital sulcus, retrosplenial complex), which were recruited during unsuccessful access attempts for all stimulus domains. Collectively, these results suggest that prefrontal semantic control systems and classical spatial knowledge-selective regions work together to locate relevant information and that access to complex knowledge results in a broad activation of semantic representation extending to regions selective for other knowledge domains.SIGNIFICANCE STATEMENT The ability to access the deep factual knowledge we possess has a meaningful influence on our scholastic, professional, and social lives. In this fMRI study, we investigate the neural processes associated with successful access to this knowledge as well as transient failures in semantic access (tip-of-the-tongue/feeling-of-knowing). Participants attempted to answer trivia-style general knowledge questions drawn from four different knowledge domains. Results suggest that prefrontal semantic control systems and classical spatial knowledge-selective regions work to locate relevant information and that access to complex knowledge results in a broad activation of semantic representation extending to regions selective for other knowledge domains.

Similar representation of names and faces in the network for person perception.

Person-knowledge encompasses the diverse types of knowledge we have about other people. This knowledge spans the social, physical, episodic, semantic & nominal information we possess about others and is served by a distributed cortical network including core (perceptual) and extended (non-perceptual) subsystems. Our understanding of this cortical system is tightly linked to the perception of faces and the extent to which cortical knowledge-access processes are independent of perception is unclear. In this study, participants were presented with the written names of famous people and performed ten different semantic access tasks drawn from five cognitive domains (biographic, episodic, nominal, social and physical). We used representational similarity analysis, adapted to investigate network-level representations (NetRSA) to characterise the inter-regional functional coordination within the non-perceptual extended subsystem across access to varied forms of person-knowledge. Results indicate a hierarchical cognitive taxonomy consistent with that seen during face-processing and forming the same three macro-domains: socio-perceptual judgements, episodic-semantic memory and nominal knowledge. The coordination across regions was largely preserved within elements of the extended system associated with internalised cognition but differed in prefrontal regions. Results suggest the elements of the extended system work together in a consistent way to access knowledge when viewing faces and names but that coordination patterns also change as a function of input-processing demands.

Similarity-based fMRI-MEG fusion reveals hierarchical organisation within the brain's semantic system.

Our ability to understand and interact with our environment relies upon conceptual knowledge of the meaning of objects. This process is supported by a distributed network of frontal, parietal, and temporal brain regions. Insight into the differential roles of various elements of this system can be inferred from the timing of activation, and here we use similarity-based fMRI-MEG fusion to understand when the representational spaces in different elements of the semantic system converge with representational spaces in the evolving MEG signal. Participants performed a semantic-typicality judgement of written words drawn from nine different semantic categories in separate fMRI and MEG sessions. Results indicate an initial period of congruence between MEG and fMRI informational spaces dominated by the posterior inferior temporal gyrus and the ventral temporal cortex between 350 and 450 msec. This is followed by a second period of convergence between 450 and 795 msec where MEG and fMRI representational spaces conform in left angular gyrus and precuneus in addition to ventral temporal cortex. Results are consistent with the multistage recruitment of the semantic system, initially involving automatic aspects of the representational system and later extending to broader elements of the semantic system more strongly associated with internalised cognition.

Deep neural networks reveal topic-level representations of sentences in medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus.

When reading a sentence, individual words can be combined to create more complex meaning. In this study, we sought to uncover brain regions that reflect the representation of the meaning of sentences at the topic level, as opposed to the meaning of their individual constituent words when considered irrespective of their context. Using fMRI, we recorded the neural activity of participants while reading sentences. We constructed a topic-level sentence representations using the final layer of a convolutional neural network (CNN) trained to classify Wikipedia sentences into broad semantic categories. This model was contrasted with word-level sentence representations constructed using the average of the word embeddings constituting the sentence. Using representational similarity analysis, we found that the medial prefrontal cortex, lateral anterior temporal lobe, precuneus, and angular gyrus more strongly represent sentence topic-level, compared to word-level, meaning, uncovering the important role of these semantic system regions in the representation of topic-level meaning. Results were comparable when sentence meaning was modelled with a multilayer perceptron that was not sensitive to word order within a sentence, suggesting that the learning objective, in the terms of the topic being modelled, is the critical factor in capturing these neural representational spaces.

Cross Recruitment of Domain-Selective Cortical Representations Enables Flexible Semantic Knowledge.

Knowledge about objects encompasses not only their prototypical features but also complex, atypical, semantic knowledge (e.g., "Pizza was invented in Naples"). This fMRI study of male and female human participants combines univariate and multivariate analyses to consider the cortical representation of this more complex semantic knowledge. Using the categories of food, people, and places, this study investigates whether access to spatially related geographic semantic knowledge (1) involves the same domain-selective neural representations involved in access to prototypical taste knowledge about food; and (2) elicits activation of neural representations classically linked to places when this geographic knowledge is accessed about food and people. In three experiments using word stimuli, domain-relevant and atypical conceptual access for the categories food, people, and places were assessed. Results uncover two principles of semantic representation: food-selective representations in the left insula continue to be recruited when prototypical taste knowledge is task-irrelevant and under conditions of high cognitive demand; access to geographic knowledge for food and people categories involves the additional recruitment of classically place-selective parahippocampal gyrus, retrosplenial complex, and transverse occipital sulcus. These findings underscore the importance of object category in the representation of a broad range of knowledge, while showing how the cross recruitment of specialized representations may endow the considerable flexibility of our complex semantic knowledge.SIGNIFICANCE STATEMENT We know not only stereotypical things about objects (an apple is round, graspable, edible) but can also flexibly combine typical and atypical features to form complex concepts (the metaphorical role an apple plays in Judeo-Christian belief). In this fMRI study, we observe that, when atypical geographic knowledge is accessed about food dishes, domain-selective sensorimotor-related cortical representations continue to be recruited, but that regions classically associated with place perception are additionally engaged. This interplay between categorically driven representations, linked to the object being accessed, and the flexible recruitment of semantic stores linked to the content being accessed, provides a potential mechanism for the broad representational repertoire of our semantic system.

fMRI-Indexed neural temporal tuning reveals the hierarchical organsiation of the face and person selective network.

Recognising and knowing about conspecifics is vital to human interaction and is served in the brain by a well-characterised cortical network. Understanding the temporal dynamics of this network is critical to gaining insight into both hierarchical organisation and regional coordination. Here, we combine the high spatial resolution of fMRI with a paradigm that permits investigation of differential temporal tuning across cortical regions. We cognitively under- and overload the system using the rapid presentation (100-1200msec) of famous faces and buildings. We observed an increase in activity as presentation rates slowed and a negative deflection when inter-stimulus intervals (ISIs) were extended to longer periods. The primary distinction in tuning patterns was between core (perceptual) and extended (non-perceptual) systems but there was also evidence for nested hierarchies within systems, as well as indications of widespread parallel processing. Extended regions demonstrated common temporal tuning across regions which may indicate coordinated activity as they cooperate to manifest the diverse cognitive representation accomplished by this network. With the support of an additional psychophysical study, we demonstrated that ISIs necessary for different levels of semantic access are consistent with temporal tuning patterns. Collectively, these results show that regions of the person-knowledge network operate over different temporal timescales consistent with hierarchical organisation.

fMRI response to automatic and purposeful familiar-face processing in perceptual and nonperceptual cortical regions.

Viewing the faces of familiar people selectively activates a distributed network of brain regions implicated in both the perceptual and nonperceptual processing of conspecifics. In this functional magnetic resonance imaging (fMRI) study, we investigate the influence of depth of famous-face processing on this network, comparing a passive incidental face processing to a task that required the extraction of identity and biographic information. We observed that the precuneus, ventromedial prefrontal cortex (vmPFC), anterior temporal face patch (ATFP), and the amygdala exhibit a selective response even during incidental face processing. At the same time, face selectivity was enhanced in the lateral anterior temporal lobe (latATL) and the posterior superior temporal sulcus (pSTS) when identity and information extraction was required. In addition, goal-directed identity and information extraction was associated with a recruitment of inferior frontal gyrus (IFG), whereas this region was deactivated during passive viewing. Collectively, these results show that: 1) in addition to active information extraction, the extended system is recruited by the passive retrieval of person-related knowledge and 2) active access to such knowledge modulates activity in latATL and pSTS, potentially mediated via control circuits in the IFG.NEW & NOTEWORTHY Information is extracted from familiar faces in both automatic and active modes. Using functional MRI, we show: 1) that automatic access results in the selective activation of nonperceptual brain regions, the precuneus, ventromedial prefrontal cortex, and the anterior face patch and amygdala, demonstrating the automaticity of access to information in these regions; 2) selective increases in the activation of the lateral anterior temporal lobe and posterior superior temporal gyrus when biographic information is actively extracted.

Regional Specialization and Coordination Within the Network for Perceiving and Knowing About Others.

Seeing familiar faces prompts the recall of diverse kinds of person-related knowledge. How this information is encoded within the well-characterized face-/person-selective network remains an outstanding question. In this functional magnetic resonance imaging study, participants rated famous faces in 10 tasks covering 5 domains of person knowledge (social, episodic, semantic, physical, and nominal). Comparing different cognitive domains enabled us to 1) test the relative roles of brain regions in specific cognitive processes and 2) apply a multivariate network-level representational similarity analysis (NetRSA) to gain insight into underlying system-level organization. Comparing across cognitive domains revealed the importance of multiple domains in most regions, the importance of social over nominal knowledge in the anterior temporal lobe, and the functional subdivision of the temporoparietal junction into perceptual superior temporal sulcus and knowledge-related angular gyrus. NetRSA revealed a strong divide between regions implicated in "default-mode" cognition and the fronto-lateral elements that coordinated more with "core" perceptual components (fusiform/occipital face areas and posterior superior temporal sulcus). NetRSA also revealed a taxonomy of cognitive processes, with semantic retrieval being more similar to episodic than nominal knowledge. Collectively, these results illustrate the importance of coordinated activity of the person knowledge network in the instantiation of the diverse cognitive capacities of this system.

Spatiotemporal properties of the neural representation of conceptual content for words and pictures - an MEG study.

The entwined nature of perceptual and conceptual processes renders an understanding of the interplay between perceptual recognition and conceptual access a continuing challenge. Here, to disentangle perceptual and conceptual processing in the brain, we combine magnetoencephalography (MEG), picture and word presentation and representational similarity analysis (RSA). We replicate previous findings of early and robust sensitivity to semantic distances between objects presented as pictures and show earlier (~105 â€‹msec), but not later, representations can be accounted for by contemporary computer models of visual similarity (AlexNet). Conceptual content for word stimuli is reliably present in two temporal clusters, the first ranging from 230 to 335 â€‹msec, the second from 360 to â€‹585 msec. The time-course of picture induced semantic content and the spatial location of conceptual representation were highly convergent, and the spatial distribution of both differed from that of words. While this may reflect differences in picture and word induced conceptual access, this underscores potential confounds in visual perceptual and conceptual processing. On the other hand, using the stringent criterion that neural and conceptual spaces must align, the robust representation of semantic content by 230-240 msec for visually unconfounded word stimuli significantly advances estimates of the timeline of semantic access and its orthographic and lexical precursors.

Spatiotopic updating across saccades revealed by spatially-specific fMRI adaptation.

Brain representations of visual space are predominantly eye-centred (retinotopic) yet our experience of the world is largely world-centred (spatiotopic). A long-standing question is how the brain creates continuity between these reference frames across successive eye movements (saccades). Here we use functional magnetic resonance imaging (fMRI) to address whether spatially specific repetition suppression (RS) is evident during trans-saccadic perception. We presented two successive Gabor patches (S1 and S2) in either the upper or lower visual field, left or right of fixation. Spatial congruency was manipulated by having S1 and S2 occur in the same or different upper/lower visual field. On half the trials, a saccade was cued between S1 and S2, placing spatiotopic and retinotopic reference frames in opposition. Equivalent RS was observed in the posterior parietal cortex and frontal eye fields when S1-S2 were spatiotopically congruent, irrespective of whether retinotopic and spatiotopic coordinates were in accord or were placed in opposition by a saccade. Additionally the post-saccadic response to S2 demonstrated spatially-specific RS in retinotopic visual regions, with stronger RS in extrastriate than striate cortex. Collectively, these results are consistent with a robust trans-saccadic spatial updating mechanism for object position that directly influences even the earliest levels of visual processing.

Person- and place-selective neural substrates for entity-specific semantic access.

Object-category has a pronounced effect on the representation of objects in higher level visual cortex. However, the influence of category on semantic/conceptual processes is less well characterized. In the present study, we conduct 2 fMRI experiments to investigate the semantic processing of information specific to individual people and places (entities). First, during picture presentation, we determined which brain regions show category-selective increases during access to entity-specific semantic information (i.e., nationality) in comparison to general-category discrimination (person vs. place). In the second experiment, we presented either words or pictures to assess the independence of entity-specific category-selective semantic representations from the processes used to access those representations. Convergent results from these 2 experiments show that brain regions exhibiting a category-selective increase during entity-specific semantic access are the same as those that show a supramodal (word/picture) category-selective response during the same task. These responses were different from classical "perceptual" category-selective responses and were evident in the medial precuneus for people and in the retrosplenial complex as well as anterior/superior sections of the transverse occipital sulcus and parahippocampal gyrus for places. These results reveal the pervasive influence of object-category in cortical organization, which extends to aspects of semantic knowledge arbitrarily related to physical/perceptual properties.

Decoding representations of face identity that are tolerant to rotation.

In order to recognize the identity of a face we need to distinguish very similar images (specificity) while also generalizing identity information across image transformations such as changes in orientation (tolerance). Recent studies investigated the representation of individual faces in the brain, but it remains unclear whether the human brain regions that were found encode representations of individual images (specificity) or face identity (specificity plus tolerance). In the present article, we use multivoxel pattern analysis in the human ventral stream to investigate the representation of face identity across rotations in depth, a kind of transformation in which no point in the face image remains unchanged. The results reveal representations of face identity that are tolerant to rotations in depth in occipitotemporal cortex and in anterior temporal cortex, even when the similarity between mirror symmetrical views cannot be used to achieve tolerance. Converging evidence from different analysis techniques shows that the right anterior temporal lobe encodes a comparable amount of identity information to occipitotemporal regions, but this information is encoded over a smaller extent of cortex.

Brain regions that represent amodal conceptual knowledge.

To what extent do the brain regions implicated in semantic processing contribute to the representation of amodal conceptual content rather than modality-specific mechanisms or mechanisms of semantic access and manipulation? Here, we propose that a brain region can be considered to represent amodal conceptual object knowledge if it is supramodal and plays a role in distinguishing among the conceptual representations of different objects. In an fMRI study, human participants made category typicality judgments about pictured objects or their names drawn from five different categories. Crossmodal multivariate pattern analysis revealed a network of six left-lateralized regions largely outside of category-selective visual cortex that showed a supramodal representation of object categories. These were located in the posterior middle/inferior temporal gyrus (pMTG/ITG), angular gyrus, ventral temporal cortex, posterior cingulate/precuneus (PC), and lateral and dorsomedial prefrontal cortex. Representational similarity analysis within these regions determined that the similarity between category-specific patterns of neural activity in the pMTG/ITG and the PC was consistent with the semantic similarity between these categories. This finding supports the PC and pMTG/ITG as candidate regions for the amodal representation of the conceptual properties of objects.

Predication drives verb cortical signatures.

Verbs and nouns are fundamental units of language, but their neural instantiation remains poorly understood. Neuropsychological research has shown that nouns and verbs can be damaged independently of each other, and neuroimaging research has found that several brain regions respond differentially to the two word classes. However, the semantic-lexical properties of verbs and nouns that drive these effects remain unknown. Here we show that the most likely candidate is predication: a core lexical feature involved in binding constituent arguments (boy, candies) into a unified syntactic-semantic structure expressing a proposition (the boy likes the candies). We used functional neuroimaging to test whether the intrinsic "predication-building" function of verbs is what drives the verb-noun distinction in the brain. We first identified verb-preferring regions with a localizer experiment including verbs and nouns. Then, we examined whether these regions are sensitive to transitivity--an index measuring its tendency to select for a direct object. Transitivity is a verb-specific property lying at the core of its predication function. Neural activity in the left posterior middle temporal and inferior frontal gyri correlates with transitivity, indicating sensitivity to predication. This represents the first evidence that grammatical class preference in the brain is driven by a word's function to build predication structures.

Sentence-level embeddings reveal dissociable word- and sentence-level cortical representation across coarse- and fine-grained levels of meaning.

In this large-sample (N = 64) fMRI study, sentence embeddings (text-embedding-ada-002, OpenAI) and representational similarity analysis were used to contrast sentence-level and word-level semantic representation. Overall, sentence-level information resulted in a 20-25 % increase in the model's ability to captures neural representation when compared to word-level only information (word-order scrambled embeddings). This increase was relatively undifferentiated across the cortex. However, when coarse-grained (across thematic category) and fine-grained (within thematic category) combinatorial meaning were separately assessed, word- and sentence-level representations were seen to strongly dissociate across the cortex and to do so differently as a function of grain. Coarse-grained sentence-level representations were evident in occipitotemporal, ventral temporal and medial prefrontal cortex, while fine-grained differences were seen in lateral prefrontal and parietal cortex, middle temporal gyrus, the precuneus, and medial prefrontal cortex. This result indicates dissociable cortical substrates underly single concept versus combinatorial meaning and that different cortical regions specialise for fine- and coarse-grained meaning.

Distinct neural substrates for semantic knowledge and naming in the temporoparietal network.

Patients with anterior temporal lobe (ATL) lesions show semantic and lexical retrieval deficits, and the differential role of this area in the 2 processes is debated. Functional neuroimaging in healthy individuals has not clarified the matter because semantic and lexical processes usually occur simultaneously and automatically. Furthermore, the ATL is a region challenging for functional magnetic resonance imaging (fMRI) due to susceptibility artifacts, especially at high fields. In this study, we established an optimized ATL-sensitive fMRI acquisition protocol at 4 T and applied an event-related paradigm to study the identification (i.e., association of semantic biographical information) of celebrities, with and without the ability to retrieve their proper names. While semantic processing reliably activated the ATL, only more posterior areas in the left temporal and temporal-parietal junction were significantly modulated by covert lexical retrieval. These results suggest that within a temporoparietal network, the ATL is relatively more important for semantic processing, and posterior language regions are relatively more important for lexical retrieval.

Task-based activation and resting-state connectivity predict individual differences in semantic capacity for complex semantic knowledge.

Our ability to know and access complex factual information has far reaching effects, influencing our scholastic, professional and social lives. Here we employ functional MRI to assess the relationship between individual differences in semantic aptitude in the task-based activation and resting-state functional connectivity. Using psychometric and behavioural measures, we quantified the semantic and executive aptitude of individuals and had them perform a general-knowledge semantic-retrieval task (N = 41) and recorded resting-state data (N = 43). During the semantic-retrieval task, participants accessed general-knowledge facts drawn from four different knowledge-domains (people, places, objects and 'scholastic'). Individuals with greater executive capacity more strongly recruit anterior sections of prefrontal cortex (PFC) and the precuneus, and individuals with lower semantic capacity more strongly activate a posterior section of the dorsomedial PFC (dmPFC). The role of these regions in semantic processing was validated by analysis of independent resting-state data, where increased connectivity between a left anterior PFC and the precuneus predict higher semantic aptitude, and increased connectivity between left anterior PFC and posterior dmPFC predict lower semantic aptitude. Results suggest that coordination between core semantic regions in the precuneus and anterior prefrontal regions associated with executive processes support greater semantic aptitude.

Concordance between perceptual and categorical repetition effects in the ventral visual stream.

The process of object categorization is an integral part of human cognition. In the present study, we have used a repetition suppression paradigm to determine the degree to which the ventral visual cortex is sensitive to categorical relationships. By using images of animals and tools, suppression across perceptual (stimulus level) and categorical repetitions (basic level and domain level) was compared and contrasted across the domain-selective and hierarchical organization of the ventral visual stream. Both perceptual and categorical repetition effects were insensitive to domain-selective tuning, with suppression most prominent in regions responding maximally to images, irrespective of stimulus domain. Likewise, both perceptual and categorical repetition produced overlapping suppression across multiple regions of the visual hierarchy. Some divergent patterns were observed. The right superior temporal sulcus demonstrated repetition suppression only at the basic level (different examples of the same basic object), and the right anterior fusiform gyrus was sensitive to direct stimulus repetition but not basic-level categorical repetition. Because of the high concordance between the response profiles of perceptual and categorical repetition effects, we conclude they arise from a common cognitive mechanism.

Combining concepts across categorical domains: a linking role of the precuneus.

The human capacity for semantic knowledge entails not only the representation of single concepts but the capacity to combine these concepts into the increasingly complex ideas that underlie human thought. This process involves not only the combination of concepts from within the same semantic category but frequently the conceptual combination across semantic domains. In this fMRI study (N=24) we investigate the cortical mechanisms underlying our ability to combine concepts across different semantic domains. Using five different semantic domains (People, Places, Food, Objects and Animals), we present sentences depicting concepts drawn from a single semantic domain as well as sentences that combine concepts from two of these domains. Contrasting single-category and combined-category sentences reveals that the precuneus is more active when concepts from different domains have to be combined. At the same time, we observe that distributed category selectivity representations persist when higher-order meaning involves the combination of categories and that this category-selective response is captured by the combination of the single categories composing the sentence. Collectively, these results suggest that the combination of concepts across different semantic domains is mediated by the precuneus, which functions to link together category-selective representations distributed across the cortex.

Impaired sensorimotor integration in focal hand dystonia patients in the absence of symptoms.

BACKGROUND: Functional imaging studies of people with focal hand dystonia (FHD) have indicated abnormal activity in sensorimotor brain regions. Few studies however, have examined FHD during movements that do not provoke symptoms of the disorder. It is possible, therefore, that any differences between FHD and controls are confounded by activity due to the occurrence of symptoms. Thus, in order to characterise impairments in patients with FHD during movements that do not induce dystonic symptoms, we investigated the neural correlates of externally paced finger tapping movements. METHODS: Functional MRI (fMRI) was used to compare patients with FHD to controls with respect to activation in networks modulated by task complexity and hand used to perform simple and complex tapping movements. RESULTS: In the 'complexity network,' patients with FHD showed significantly less activity relative to controls in posterior parietal cortex, medial supplementary motor area (SMA), anterior putamen and cerebellum. In the 'hand network,' patients with FHD showed less activation than controls in primary motor (M1) and somatosensory (S1) cortices, SMA and cerebellum. Conjunction analysis revealed that patients with FHD demonstrated reduced activation in the majority of combined network regions (M1, S1 and cerebellum). CONCLUSION: Dysfunction in FHD is widespread in both complexity and hand networks, and impairments are demonstrated even when performing tasks that do not evoke dystonic symptoms. These results suggest that such impairments are inherent to, rather than symptomatic of, the disorder.