A multimodal interface for speech perception: the role of the left superior temporal sulcus in social cognition and autism.

Multimodal integration is crucial for human interaction, in particular for social communication, which relies on integrating information from various sensory modalities. Recently a third visual pathway specialized in social perception was proposed, which includes the right superior temporal sulcus (STS) playing a key role in processing socially relevant cues and high-level social perception. Importantly, it has also recently been proposed that the left STS contributes to audiovisual integration of speech processing. In this article, we propose that brain areas along the right STS that support multimodal integration for social perception and cognition can be considered homologs to those in the left, language-dominant hemisphere, sustaining multimodal integration of speech and semantic concepts fundamental for social communication. Emphasizing the significance of the left STS in multimodal integration and associated processes such as multimodal attention to socially relevant stimuli, we underscore its potential relevance in comprehending neurodevelopmental conditions characterized by challenges in social communication such as autism spectrum disorder (ASD). Further research into this left lateral processing stream holds the promise of enhancing our understanding of social communication in both typical development and ASD, which may lead to more effective interventions that could improve the quality of life for individuals with atypical neurodevelopment.

Switching off: disruptive TMS reveals distinct contributions of the posterior middle temporal gyrus and angular gyrus to bilingual speech production.

The role of the left temporoparietal cortex in speech production has been extensively studied during native language processing, proving crucial in controlled lexico-semantic retrieval under varying cognitive demands. Yet, its role in bilinguals, fluent in both native and second languages, remains poorly understood. Here, we employed continuous theta burst stimulation to disrupt neural activity in the left posterior middle-temporal gyrus (pMTG) and angular gyrus (AG) while Italian-Friulian bilinguals performed a cued picture-naming task. The task involved between-language (naming objects in Italian or Friulian) and within-language blocks (naming objects ["knife"] or associated actions ["cut"] in a single language) in which participants could either maintain (non-switch) or change (switch) instructions based on cues. During within-language blocks, cTBS over the pMTG entailed faster naming for high-demanding switch trials, while cTBS to the AG elicited slower latencies in low-demanding non-switch trials. No cTBS effects were observed in the between-language block. Our findings suggest a causal involvement of the left pMTG and AG in lexico-semantic processing across languages, with distinct contributions to controlled vs. "automatic" retrieval, respectively. However, they do not support the existence of shared control mechanisms within and between language(s) production. Altogether, these results inform neurobiological models of semantic control in bilinguals.

Role of the left posterior middle temporal gyrus in shape recognition and its reconstruction during drawing: A study combining transcranial magnetic stimulation and functional near infrared spectroscopy.

There are numerous reports of enhanced or emerged visual arts abilities in patients with semantic impairment. These reports led to the theory that a loss of function on the language side of the brain can result in changes of ability to draw and/or to paint. Further, the left posterior middle temporal gyrus (l-pMTG) has been revealed to contribute to the higher control semantic mechanisms with objects recognition and integration of visual information, within a widely distributed network of the left hemisphere. Nevertheless, the theory has not been fully studied in neural bases. The aim of this study is to examine role of the l-pMTG on shape recognition and its reconstruction within drawing behavior, by using a combining method of the repetitive transcranial magnetic stimulation (rTMS) and functional near-infrared spectroscopy (fNIRS). Eighteen healthy participants received a low frequency inhibitory rTMS to their l-pMTG during the drawing task of the Benton Visual Retention Test (BVRT). There was a significant decrease of the mean accuracy of reproductions in the Complex designs of the BVRT, compared to the Simple and Medium designs. The fNIRS data showed strong negative correlations with the results of the BVRT. Though our hypothesis had a contradiction that rTMS would have inhibited the brain activity in the stimulated site, the results suggest that shape recognition and its reconstruction such as the BVRT require neural activations of the l-TL as well as that of the l-pMTG.

Cerebellum function: The chronometry of social perception.

The posterior cerebellum is emerging as a key structure for social cognition. A new study causally demonstrates its early involvement during emotion perception and functional connectivity with the posterior superior temporal sulcus, a cortical hub of the social brain.

Visual perception of highly memorable images is mediated by a distributed network of ventral visual regions that enable a late memorability response.

Behavioral and neuroscience studies in humans and primates have shown that memorability is an intrinsic property of an image that predicts its strength of encoding into and retrieval from memory. While previous work has independently probed when or where this memorability effect may occur in the human brain, a description of its spatiotemporal dynamics is missing. Here, we used representational similarity analysis (RSA) to combine functional magnetic resonance imaging (fMRI) with source-estimated magnetoencephalography (MEG) to simultaneously measure when and where the human cortex is sensitive to differences in image memorability. Results reveal that visual perception of High Memorable images, compared to Low Memorable images, recruits a set of regions of interest (ROIs) distributed throughout the ventral visual cortex: a late memorability response (from around 300 ms) in early visual cortex (EVC), inferior temporal cortex, lateral occipital cortex, fusiform gyrus, and banks of the superior temporal sulcus. Image memorability magnitude results are represented after high-level feature processing in visual regions and reflected in classical memory regions in the medial temporal lobe (MTL). Our results present, to our knowledge, the first unified spatiotemporal account of visual memorability effect across the human cortex, further supporting the levels-of-processing theory of perception and memory.

Experience transforms crossmodal object representations in the anterior temporal lobes.

Combining information from multiple senses is essential to object recognition, core to the ability to learn concepts, make new inferences, and generalize across distinct entities. Yet how the mind combines sensory input into coherent crossmodal representations - the crossmodal binding problem - remains poorly understood. Here, we applied multi-echo fMRI across a 4-day paradigm, in which participants learned three-dimensional crossmodal representations created from well-characterized unimodal visual shape and sound features. Our novel paradigm decoupled the learned crossmodal object representations from their baseline unimodal shapes and sounds, thus allowing us to track the emergence of crossmodal object representations as they were learned by healthy adults. Critically, we found that two anterior temporal lobe structures - temporal pole and perirhinal cortex - differentiated learned from non-learned crossmodal objects, even when controlling for the unimodal features that composed those objects. These results provide evidence for integrated crossmodal object representations in the anterior temporal lobes that were different from the representations for the unimodal features. Furthermore, we found that perirhinal cortex representations were by default biased toward visual shape, but this initial visual bias was attenuated by crossmodal learning. Thus, crossmodal learning transformed perirhinal representations such that they were no longer predominantly grounded in the visual modality, which may be a mechanism by which object concepts gain their abstraction.

Control of working memory by phase-amplitude coupling of human hippocampal neurons.

Retaining information in working memory is a demanding process that relies on cognitive control to protect memoranda-specific persistent activity from interference1,2. However, how cognitive control regulates working memory storage is unclear. Here we show that interactions of frontal control and hippocampal persistent activity are coordinated by theta-gamma phase-amplitude coupling (TG-PAC). We recorded single neurons in the human medial temporal and frontal lobe while patients maintained multiple items in their working memory. In the hippocampus, TG-PAC was indicative of working memory load and quality. We identified cells that selectively spiked during nonlinear interactions of theta phase and gamma amplitude. The spike timing of these PAC neurons was coordinated with frontal theta activity when cognitive control demand was high. By introducing noise correlations with persistently active neurons in the hippocampus, PAC neurons shaped the geometry of the population code. This led to higher-fidelity representations of working memory content that were associated with improved behaviour. Our results support a multicomponent architecture of working memory1,2, with frontal control managing maintenance of working memory content in storage-related areas3-5. Within this framework, hippocampal TG-PAC integrates cognitive control and working memory storage across brain areas, thereby suggesting a potential mechanism for top-down control over sensory-driven processes.

Neuronal population representation of human emotional memory.

Understanding how emotional processing modulates learning and memory is crucial for the treatment of neuropsychiatric disorders characterized by emotional memory dysfunction. We investigate how human medial temporal lobe (MTL) neurons support emotional memory by recording spiking activity from the hippocampus, amygdala, and entorhinal cortex during encoding and recognition sessions of an emotional memory task in patients with pharmaco-resistant epilepsy. Our findings reveal distinct representations for both remembered compared to forgotten and emotional compared to neutral scenes in single units and MTL population spiking activity. Additionally, we demonstrate that a distributed network of human MTL neurons exhibiting mixed selectivity on a single-unit level collectively processes emotion and memory as a network, with a small percentage of neurons responding conjointly to emotion and memory. Analyzing spiking activity enables a detailed understanding of the neurophysiological mechanisms underlying emotional memory and could provide insights into how emotion alters memory during healthy and maladaptive learning.

Prediction error processing and sharpening of expected information across the face-processing hierarchy.

The perception and neural processing of sensory information are strongly influenced by prior expectations. The integration of prior and sensory information can manifest through distinct underlying mechanisms: focusing on unexpected input, denoted as prediction error (PE) processing, or amplifying anticipated information via sharpened representation. In this study, we employed computational modeling using deep neural networks combined with representational similarity analyses of fMRI data to investigate these two processes during face perception. Participants were cued to see face images, some generated by morphing two faces, leading to ambiguity in face identity. We show that expected faces were identified faster and perception of ambiguous faces was shifted towards priors. Multivariate analyses uncovered evidence for PE processing across and beyond the face-processing hierarchy from the occipital face area (OFA), via the fusiform face area, to the anterior temporal lobe, and suggest sharpened representations in the OFA. Our findings support the proposition that the brain represents faces grounded in prior expectations.

A chronometric study of the posterior cerebellum's function in emotional processing.

The posterior cerebellum is a recently discovered hub of the affective and social brain, with different subsectors contributing to different social functions. However, very little is known about when the posterior cerebellum plays a critical role in social processing. Due to its location and anatomy, it has been difficult to use traditional approaches to directly study the chronometry of the cerebellum. To address this gap in cerebellar knowledge, here we investigated the causal contribution of the posterior cerebellum to social processing using a chronometric transcranial magnetic stimulation (TMS) approach. We show that the posterior cerebellum is recruited at an early stage of emotional processing (starting from 100 ms after stimulus onset), simultaneously with the posterior superior temporal sulcus (pSTS), a key node of the social brain. Moreover, using a condition-and-perturb TMS approach, we found that the recruitment of the pSTS in emotional processing is dependent on cerebellar activation. Our results are the first to shed light on chronometric aspects of cerebellar function and its causal functional connectivity with other nodes of the social brain.

Perceptography unveils the causal contribution of inferior temporal cortex to visual perception.

Neurons in the inferotemporal (IT) cortex respond selectively to complex visual features, implying their role in object perception. However, perception is subjective and cannot be read out from neural responses; thus, bridging the causal gap between neural activity and perception demands independent characterization of perception. Historically, though, the complexity of the perceptual alterations induced by artificial stimulation of IT cortex has rendered them impossible to quantify. To address this old problem, we tasked male macaque monkeys to detect and report optical impulses delivered to their IT cortex. Combining machine learning with high-throughput behavioral optogenetics, we generated complex and highly specific images that were hard for the animal to distinguish from the state of being cortically stimulated. These images, named "perceptograms" for the first time, reveal and depict the contents of the complex hallucinatory percepts induced by local neural perturbation in IT cortex. Furthermore, we found that the nature and magnitude of these hallucinations highly depend on concurrent visual input, stimulation location, and intensity. Objective characterization of stimulation-induced perceptual events opens the door to developing a mechanistic theory of visual perception. Further, it enables us to make better visual prosthetic devices and gain a greater understanding of visual hallucinations in mental disorders.

Childhood Book Availability Helps to Preserve Cognitive Function in Older Adults With Low Education: Results From the NEIGE Study.

OBJECTIVES: It is well known that low educational attainment is associated with cognitive function decline in older age. Childhood book availability may help to preserve cognitive function in older adults with low education. The study objective was to examine the association between childhood book availability and cognitive function among older adults with low educational attainment, and to investigate the mediating effect of the volume of reading-related brain regions (e.g., superior temporal cortex). METHODS: A cross-sectional study of community-dwelling older Japanese adults aged 65-84 years was conducted (n = 474). Cognitive function was assessed using the Mini-Mental State Examination (MMSE). Childhood book availability was assessed using a retrospective questionnaire. Brain region volume was measured using magnetic resonance imaging. Multivariate regression modeling and structural equation modeling were used for analysis. RESULTS: Both high educational attainment and childhood book availability were independently associated with high MMSE score. Stratification of educational level showed that childhood book availability was positively associated with MMSE score among participants with low educational attainment (coefficient = 1.48, 95% confidence interval [CI]: 0.31 to 2.66), but not among those with moderate or high educational attainment (coefficient = -0.01, 95% CI: -1.44 to 1.42 and -1.21, 95% CI: -3.85 to 1.42, respectively). Among participants with low educational attainment, left superior temporal cortex volume mediated the association between childhood book availability and MMSE score. DISCUSSION: The availability of books in childhood helps to preserve cognitive function in older adults with low education via left superior temporal cortex volume. Further research is needed to replicate these findings.

Neural signatures of natural behaviour in socializing macaques.

Our understanding of the neurobiology of primate behaviour largely derives from artificial tasks in highly controlled laboratory settings, overlooking most natural behaviours that primate brains evolved to produce1-3. How primates navigate the multidimensional social relationships that structure daily life4 and shape survival and reproductive success5 remains largely unclear at the single-neuron level. Here we combine ethological analysis, computer vision and wireless recording technologies to identify neural signatures of natural behaviour in unrestrained, socially interacting pairs of rhesus macaques. Single-neuron and population activity in the prefrontal and temporal cortex robustly encoded 24 species-typical behaviours, as well as social context. Male-female partners demonstrated near-perfect reciprocity in grooming, a key behavioural mechanism supporting friendships and alliances6, and neural activity maintained a running account of these social investments. Confronted with an aggressive intruder, behavioural and neural population responses reflected empathy and were buffered by the presence of a partner. Our findings reveal a highly distributed neurophysiological ledger of social dynamics, a potential computational foundation supporting communal life in primate societies, including our own.

Ultrahigh Field fMRI Reveals Different Roles of the Temporal and Frontoparietal Cortices in Subjective Awareness.

A central question in consciousness theories is whether one is dealing with a dichotomous ("all-or-none") or a gradual phenomenon. In this 7T fMRI study, we investigated whether dichotomy or gradualness in fact depends on the brain region associated with perceptual awareness reports. Both male and female human subjects performed an emotion discrimination task (fear vs neutral bodies) presented under continuous flash suppression with trial-based perceptual awareness measures. Behaviorally, recognition sensitivity increased linearly with increased stimuli awareness and was at chance level during perceptual unawareness. Physiologically, threat stimuli triggered a slower heart rate than neutral ones during "almost clear" stimulus experience, indicating freezing behavior. Brain results showed that activity in the occipitotemporal, parietal, and frontal regions as well as in the amygdala increased with increased stimulus awareness while early visual areas showed the opposite pattern. The relationship between temporal area activity and perceptual awareness best fitted a gradual model while the activity in frontoparietal areas fitted a dichotomous model. Furthermore, our findings illustrate that specific experimental decisions, such as stimulus type or the approach used to evaluate awareness, play pivotal roles in consciousness studies and warrant careful consideration.

Graded and sharp transitions in semantic function in left temporal lobe.

Recent work has focussed on how patterns of functional change within the temporal lobe relate to whole-brain dimensions of intrinsic connectivity variation (Margulies et al., 2016). We examined two such 'connectivity gradients' reflecting the separation of (i) unimodal versus heteromodal and (ii) visual versus auditory-motor cortex, examining visually presented verbal associative and feature judgments, plus picture-based context and emotion generation. Functional responses along the first dimension sometimes showed graded change between modality-tuned and heteromodal cortex (in the verbal matching task), and other times showed sharp functional transitions, with deactivation at the extremes and activation in the middle of this gradient (internal generation). The second gradient revealed more visual than auditory-motor activation, regardless of content (associative, feature, context, emotion) or task process (matching/generation). We also uncovered subtle differences across each gradient for content type, which predominantly manifested as differences in relative magnitude of activation or deactivation.

The transition from vision to language: Distinct patterns of functional connectivity for subregions of the visual word form area.

Reading entails transforming visual symbols to sound and meaning. This process depends on specialized circuitry in the visual cortex, the visual word form area (VWFA). Recent findings suggest that this text-selective cortex comprises at least two distinct subregions: the more posterior VWFA-1 is sensitive to visual features, while the more anterior VWFA-2 processes higher level language information. Here, we explore whether these two subregions also exhibit different patterns of functional connectivity. To this end, we capitalize on two complementary datasets: Using the Natural Scenes Dataset (NSD), we identify text-selective responses in high-quality 7T adult data (N = 8), and investigate functional connectivity patterns of VWFA-1 and VWFA-2 at the individual level. We then turn to the Healthy Brain Network (HBN) database to assess whether these patterns replicate in a large developmental sample (N = 224; age 6-20 years), and whether they relate to reading development. In both datasets, we find that VWFA-1 is primarily correlated with bilateral visual regions. In contrast, VWFA-2 is more strongly correlated with language regions in the frontal and lateral parietal lobes, particularly the bilateral inferior frontal gyrus. Critically, these patterns do not generalize to adjacent face-selective regions, suggesting a specific relationship between VWFA-2 and the frontal language network. No correlations were observed between functional connectivity and reading ability. Together, our findings support the distinction between subregions of the VWFA, and suggest that functional connectivity patterns in the ventral temporal cortex are consistent over a wide range of reading skills.

Reciprocal interactions among parietal and occipito-temporal representations support everyday object-directed actions.

Everyday interactions with common manipulable objects require the integration of conceptual knowledge about objects and actions with real-time sensory information about the position, orientation and volumetric structure of the grasp target. The ability to successfully interact with everyday objects involves analysis of visual form and shape, surface texture, material properties, conceptual attributes such as identity, function and typical context, and visuomotor processing supporting hand transport, grasp form, and object manipulation. Functionally separable brain regions across the dorsal and ventral visual pathways support the processing of these different object properties and, in cohort, are necessary for functional object use. Object-directed grasps display end-state-comfort: they anticipate in form and force the shape and material properties of the grasp target, and how the object will be manipulated after it is grasped. End-state-comfort is the default for everyday interactions with manipulable objects and implies integration of information across the ventral and dorsal visual pathways. We propose a model of how visuomotor and action representations in parietal cortex interact with object representations in ventral and lateral occipito-temporal cortex. One pathway, from the supramarginal gyrus to the middle and inferior temporal gyrus, supports the integration of action-related information, including hand and limb position (supramarginal gyrus) with conceptual attributes and an appreciation of the action goal (middle temporal gyrus). A second pathway, from posterior IPS to the fusiform gyrus and collateral sulcus supports the integration of grasp parameters (IPS) with the surface texture and material properties (e.g., weight distribution) of the grasp target. Reciprocal interactions among these regions are part of a broader network of regions that support everyday functional object interactions.

The role of preSMA and STS in face recognition: A transcranial magnetic stimulation (TMS) study.

Current models propose that facial recognition is mediated by two independent yet interacting anatomo-functional systems: one processing facial features mainly mediated by the Fusiform Face Area and the other involved in the extraction of dynamic information from faces, subserved by Superior Temporal Sulcus (STS). Also, the pre-Supplementary Motor Area (pre-SMA) is implicated in facial expression processing as it is involved in its motor mimicry. However, the literature only shows evidence of the implication of STS and preSMA for facial expression recognition, without relating it to face recognition. In addition, the literature shows a facilitatory role of facial motion in the recognition of unfamiliar faces, particularly for poor recognizers. The present study aimed at studying the role of STS and preSMA in unfamiliar face recognition in people with different face recognition skills. 34 healthy participants received repetitive transcranial magnetic stimulation over the right posterior STS, pre-SMA and as sham during a task of matching of faces encoded through: facial expression, rigid head movement or as static (i.e., absence of any facial or head motion). All faces were represented without emotional content. Results indicate that STS has a direct role in recognizing identities through rigid head movement and an indirect role in facial expression processing. This dissociation represents a step forward with respect to current face processing models suggesting that different types of motion involve separate brain and cognitive processes. PreSMA interacts with face recognition skills, increasing the performance of poor recognizers and decreasing that of good recognizers in all presentation conditions. Together, the results suggest the use of at least partially different mechanisms for face recognition in poor and good recognizers and a different role of STS and preSMA in face recognition.

Nonfrontal Control of Working Memory.

Items held in visual working memory can be quickly updated, replaced, removed, and even manipulated in accordance with current behavioral goals. Here, we use multivariate pattern analyses to identify the patterns of neuronal activity that realize the executive control processes supervising these flexible stores. We find that portions of the middle temporal gyrus and the intraparietal sulcus represent what item is cued for continued memorization independently of representations of the item itself. Importantly, this selection-specific activity could not be explained by sensory representations of the cue and is only present when control is exerted. Our results suggest that the selection of memorized items might be controlled in a distributed and decentralized fashion. This evidence provides an alternative perspective to the notion of "domain general" central executive control over memory function.