Modelling brain representations of abstract concepts.

conceptual representations are critical for human cognition. Despite their importance, key properties of these representations remain poorly understood. Here, we used computational models of distributional semantics to predict multivariate fMRI activity patterns during the activation and contextualization of abstract concepts. We devised a task in which participants had to embed abstract nouns into a story that they developed around a given background context. We found that representations in inferior parietal cortex were predicted by concept similarities emerging in models of distributional semantics. By constructing different model families, we reveal the models' learning trajectories and delineate how abstract and concrete training materials contribute to the formation of brain-like representations. These results inform theories about the format and emergence of abstract conceptual representations in the human brain.

Neural processing of vision and language in kindergarten is associated with prereading skills and predicts future literacy.

The main objective of this longitudinal study was to investigate the neural predictors of reading acquisition. For this purpose, we followed a sample of 54 children from the end of kindergarten to the end of second grade. Preliterate children were tested for visual symbol (checkerboards, houses, faces, written words) and auditory language processing (spoken words) using a passive functional magnetic resonance imaging paradigm. To examine brain-behavior relationships, we also tested cognitive-linguistic prereading skills at kindergarten age and reading performance of 48 of the same children 2 years later. Face-selective response in the bilateral fusiform gyrus was positively associated with rapid automatized naming (RAN). Response to both spoken and written words at preliterate age was negatively associated with RAN in the dorsal temporo-parietal language system. Longitudinally, neural response to faces in the ventral stream predicted future reading fluency. Here, stronger neural activity in inferior and middle temporal gyri at kindergarten age was associated with higher reading performance. Our results suggest that interindividual differences in the neural system of language and reading affect literacy acquisition and thus might serve as a marker for successful reading acquisition in preliterate children.

Affective iconic words benefit from additional sound-meaning integration in the left amygdala.

Recent studies have shown that a similarity between sound and meaning of a word (i.e., iconicity) can help more readily access the meaning of that word, but the neural mechanisms underlying this beneficial role of iconicity in semantic processing remain largely unknown. In an fMRI study, we focused on the affective domain and examined whether affective iconic words (e.g., high arousal in both sound and meaning) activate additional brain regions that integrate emotional information from different domains (i.e., sound and meaning). In line with our hypothesis, affective iconic words, compared to their non-iconic counterparts, elicited additional BOLD responses in the left amygdala known for its role in multimodal representation of emotions. Functional connectivity analyses revealed that the observed amygdalar activity was modulated by an interaction of iconic condition and activations in two hubs representative for processing sound (left superior temporal gyrus) and meaning (left inferior frontal gyrus) of words. These results provide a neural explanation for the facilitative role of iconicity in language processing and indicate that language users are sensitive to the interaction between sound and meaning aspect of words, suggesting the existence of iconicity as a general property of human language.

Do Words Stink? Neural Reuse as a Principle for Understanding Emotions in Reading.

How do we understand the emotional content of written words? Here, we investigate the hypothesis that written words that carry emotions are processed through phylogenetically ancient neural circuits that are involved in the processing of the very same emotions in nonlanguage contexts. This hypothesis was tested with respect to disgust. In an fMRI experiment, it was found that the same region of the left anterior insula responded whether people observed facial expressions of disgust or whether they read words with disgusting content. In a follow-up experiment, it was found that repetitive TMS over the left insula in comparison with a control site interfered with the processing of disgust words to a greater extent than with the processing of neutral words. Together, the results support the hypothesis that the affective processes we experience when reading rely on the reuse of phylogenetically ancient brain structures that process basic emotions in other domains and species.

The Temporal Pole Top-Down Modulates the Ventral Visual Stream During Social Cognition.

The temporal pole (TP) has been associated with diverse functions of social cognition and emotion processing. Although the underlying mechanism remains elusive, one possibility is that TP acts as domain-general hub integrating socioemotional information. To test this, 26 participants were presented with 60 empathy-evoking film clips during fMRI scanning. The film clips were preceded by a linguistic sad or neutral context and half of the clips were accompanied by sad music. In line with its hypothesized role, TP was involved in the processing of sad context and furthermore tracked participants' empathic concern. To examine the neuromodulatory impact of TP, we applied nonlinear dynamic causal modeling to a multisensory integration network from previous work consisting of superior temporal gyrus (STG), fusiform gyrus (FG), and amygdala, which was extended by an additional node in the TP. Bayesian model comparison revealed a gating of STG and TP on fusiform-amygdalar coupling and an increase of TP to FG connectivity during the integration of contextual information. Moreover, these backward projections were strengthened by emotional music. The findings indicate that during social cognition, TP integrates information from different modalities and top-down modulates lower-level perceptual areas in the ventral visual stream as a function of integration demands.

Neural representation of emotion regulation goals.

The use of top-down cognitive control mechanisms to regulate emotional responses as circumstances change is critical for mental and physical health. Several theoretical models of emotion regulation have been postulated; it remains unclear, however, in which brain regions emotion regulation goals (e.g., the downregulation of fear) are represented. Here, we examined the neural mechanisms of regulating emotion using fMRI and identified brain regions representing reappraisal goals. Using a multimethodological analysis approach, combining standard activation-based and pattern-information analyses, we identified a distributed network of lateral frontal, temporal, and parietal regions implicated in reappraisal and within it, a core system that represents reappraisal goals in an abstract, stimulus-independent fashion. Within this core system, the neural pattern-separability in a subset of regions including the left inferior frontal gyrus, middle temporal gyrus, and inferior parietal lobe was related to the success in emotion regulation. Those brain regions might link the prefrontal control regions with the subcortical affective regions. Given the strong association of this subsystem with inner speech functions and semantic memory, we conclude that those cognitive mechanisms may be used for orchestrating emotion regulation. Hum Brain Mapp 37:600-620, 2016. © 2015 Wiley Periodicals, Inc.

Effects of empathic social responses on the emotions of the recipient.

Empathy is highly relevant for social behavior and can be verbally expressed by voicing sympathy and concern (emotional empathy) as well as by paraphrasing or stating that one can mentally reconstruct and understand another person's thoughts and feelings (cognitive empathy). In this study, we investigated the emotional effects and neural correlates of receiving empathic social responses after negative performance feedback and compared the effects of emotionally vs. cognitively empathic comments. 20 participants (10 male) underwent functional magnetic resonance imaging while receiving negative performance feedback for a cognitive task. Performance feedback was followed by verbal comments either expressing cognitive and emotional empathy or demonstrating a lack of empathy. Empathic comments in general led to less negative self-reported feelings and calmer breathing. At the neural level, empathic comments induced activity in regions associated with social cognition and emotion processing, specifically in right postcentral gyrus and left cerebellum (cognitively empathic comments), right precentral gyrus, the opercular part of left inferior frontal gyrus, and left middle temporal gyrus (emotionally empathic comments), as well as the orbital part of the left middle frontal gyrus and left superior parietal gyrus (emotionally empathic vs. unempathic comments). The study shows that cognitively and emotionally empathic comments appear to be processed in partially separable neural systems. Furthermore, confirming and expanding on another study on the same subject, the present results demonstrate that the social display of cognitive empathy exerts almost as positive effects on the recipient's feelings and emotions in states of distress as emotionally empathic response does. This can be relevant for professional settings in which strong negative emotions need to be de-escalated while maintaining professional impartiality, which may allow the display of cognitive but not emotional empathy.

When emotions are expressed figuratively: Psycholinguistic and Affective Norms of 619 Idioms for German (PANIG).

Despite flourishing research on the relationship between emotion and literal language, and despite the pervasiveness of figurative expressions in communication, the role of figurative language in conveying affect has been underinvestigated. This study provides affective and psycholinguistic norms for 619 German idiomatic expressions and explores the relationships between affective and psycholinguistic idiom properties. German native speakers rated each idiom for emotional valence, arousal, familiarity, semantic transparency, figurativeness, and concreteness. They also described the figurative meaning of each idiom and rated how confident they were about the attributed meaning. The results showed that idioms rated high in valence were also rated high in arousal. Negative idioms were rated as more arousing than positive ones, in line with results from single words. Furthermore, arousal correlated positively with figurativeness (supporting the idea that figurative expressions are more emotionally engaging than literal expressions) and with concreteness and semantic transparency. This suggests that idioms may convey a more direct reference to sensory representations, mediated by the meanings of their constituting words. Arousal correlated positively with familiarity. In addition, positive idioms were rated as more familiar than negative idioms. Finally, idioms without a literal counterpart were rated as more emotionally valenced and arousing than idioms with a literal counterpart. Although the meanings of ambiguous idioms were less correctly defined than those of unambiguous idioms, ambiguous idioms were rated as more concrete than unambiguous ones. We also discuss the relationships between the various psycholinguistic variables characterizing idioms, with reference to the literature on idiom structure and processing.

Emotions in reading: Dissociation of happiness and positivity.

The hierarchical emotion model proposed by Panksepp (1998) predicts that affective processing will rely on three functionally and neuroanatomically distinct levels, engaging subcortical networks (primary level), the limbic system (secondary level), and the neocortex (tertiary level). In the present fMRI study, we manipulated happiness and positivity, which are assumed to rely on secondary- and tertiary-level processes, respectively, to test these assumptions in a word recognition task. In accordance with the model predictions, evidence for a double dissociation was found in the brain activation patterns: Secondary-level processes engaged parts of the limbic system-specifically, the right hemispheric amygdala. Tertiary-level processes, in contrast, relied predominantly on frontal neocortical structures such as the left inferior frontal and medial frontal gyri. These results are interpreted as support for Panksepp's (1998) model and as an indicator of a semantic foundation of affective dimensions.

Occipital and orbitofrontal hemodynamics during naturally paced reading: an fNIRS study.

Humans typically read at incredibly fast rates, because they predict likely occurring words from a given context. Here, we used functional near-infrared spectroscopy (fNIRS) to track the ultra-rapid hemodynamic responses of words presented every 280 ms in a naturally paced sentence context. We found a lower occipital deoxygenation to unpredictable than to predictable words. The greater hemodynamic responses to unexpected words suggest that the visual features of expected words have been pre-activated previous to stimulus presentation. Second, we tested opposing theoretical proposals about the role of the medial orbitofrontal cortex (OFC): Either OFC may respond to the breach of expectation; or OFC is activated when the present stimulus matches the prediction. A significant interaction between word frequency and predictability indicated OFC responses to breaches of expectation for low- but not for high-frequency words: OFC is sensitive to both, bottom-up processing as mediated by word frequency, as well as top-down predictions. Particularly, when a rare word is unpredictable, OFC becomes active. Finally, we discuss how a high temporal resolution can help future studies to disentangle the hemodynamic responses of single trials in such an ultra-rapid event succession as naturally paced reading.

Talking about social conflict in the MRI scanner: neural correlates of being empathized with.

This study investigated the emotional effects and neural correlates of being empathized with while speaking about a currently experienced real-life social conflict during fMRI. Specifically, we focused on the effects of cognitive empathy in the form of paraphrasing, a technique regularly used in conflict resolution. 22 participants underwent fMRI while being interviewed on their social conflict and receiving empathic or unempathic responses from the interviewer. Skin conductance response (SCR) and self-report ratings of feeling understood and emotional valence were used to assess emotional responses. Results confirm previous findings indicating that cognitive empathy exerts a positive short-term effect on emotions in social conflict, while at the same time increasing autonomic arousal reflected by SCR. Effects of paraphrasing and unempathic interventions as indicated by self-report ratings varied depending on self-esteem, pre-interview negative affect, and participants' empathy quotient. Empathic responses engaged a fronto-parietal network with activity in the right precentral gyrus (PrG), left middle frontal gyrus (MFG), left inferior parietal gyrus (IPG), and right postcentral gyrus (PoG). Processing unempathic responses involved a fronto-temporal network with clusters peaking in the left inferior frontal gyrus, pars triangularis (IFGTr), and right temporal pole (TP). A specific modeling of feeling misunderstood activated a network consisting of the IFG, left TP, left Heschl gyrus, IFGTr, and right precuneus, extending to several limbic regions, such as the insula, amygdala, putamen, and anterior cingulate cortex/right middle cingulum (ACC/MCC). The results support the effectiveness of a widely used conflict resolution technique, which may also be useful for professionals who regularly deal with and have to de-escalate situations highly charged with negative emotion, e.g. physicians or judges.

ANGST: affective norms for German sentiment terms, derived from the affective norms for English words.

We present the German adaptation of the Affective Norms for English Words (ANEW; Bradley & Lang in Technical Report No. C-1. Gainsville: University of Florida, Center for Research in Psychophysiology). A total of 1,003 Words-German translations of the ANEW material-were rated on a total of six dimensions: The classic ratings of valence, arousal, and dominance (as in the ANEW corpus) were extended with additional arousal ratings using a slightly different scale (see BAWL: Võ et al. in Behavior Research Methods 41: 531-538, 2009; Võ, Jacobs, & Conrad in Behavior Research Methods 38: 606-609, 2006), along with ratings of imageability and potency. Measures of several objective psycholinguistic variables (different types of word frequency counts, grammatical class, number of letters, number of syllables, and number of orthographic neighbors) for the words were also added, so as to further facilitate the use of this new database in psycholinguistic research. These norms can be downloaded as supplemental materials with this article.

The roles of superficial amygdala and auditory cortex in music-evoked fear and joy.

This study investigates neural correlates of music-evoked fear and joy with fMRI. Studies on neural correlates of music-evoked fear are scant, and there are only a few studies on neural correlates of joy in general. Eighteen individuals listened to excerpts of fear-evoking, joy-evoking, as well as neutral music and rated their own emotional state in terms of valence, arousal, fear, and joy. Results show that BOLD signal intensity increased during joy, and decreased during fear (compared to the neutral condition) in bilateral auditory cortex (AC) and bilateral superficial amygdala (SF). In the right primary somatosensory cortex (area 3b) BOLD signals increased during exposure to fear-evoking music. While emotion-specific activity in AC increased with increasing duration of each trial, SF responded phasically in the beginning of the stimulus, and then SF activity declined. Psychophysiological Interaction (PPI) analysis revealed extensive emotion-specific functional connectivity of AC with insula, cingulate cortex, as well as with visual, and parietal attentional structures. These findings show that the auditory cortex functions as a central hub of an affective-attentional network that is more extensive than previously believed. PPI analyses also showed functional connectivity of SF with AC during the joy condition, taken to reflect that SF is sensitive to social signals with positive valence. During fear music, SF showed functional connectivity with visual cortex and area 7 of the superior parietal lobule, taken to reflect increased visual alertness and an involuntary shift of attention during the perception of auditory signals of danger.

Neural mechanisms underlying the integration of emotion and working memory.

The present study aimed at investigating the behavioral effects and neuronal correlates of emotional content and emotional components, i.e. valence and arousal, in the context of a verbal working memory task. Our findings in twenty healthy male subjects demonstrate that (1) word valence has no impact on performance in the verbal working memory task, and (2) that emotion leads to an increase of activation in cognition-related lateral prefrontal regions, whereas cognitive effort yields enhanced deactivation in emotion-related cortical midline regions. The stronger dorsolateral prefrontal recruitment during emotional stimuli may reflect an arousal effect or higher cognitive effort due to interference with emotion.

Neural correlates of combinatorial semantic processing of literal and figurative noun noun compound words.

The right hemisphere's role in language comprehension is supported by results from several neuropsychology and neuroimaging studies. Special interest surrounds right temporoparietal structures, which are thought to be involved in processing novel metaphorical expressions, primarily due to the coarse semantic coding of concepts. In this event related fMRI experiment we aimed at assessing the extent of semantic distance processing in the comprehension of figurative meaning to clarify the role of the right hemisphere. Four categories of German noun noun compound words were presented in a semantic decision task: a) conventional metaphors; b) novel metaphors; c) conventional literal, and; d) novel literal expressions, controlled for length, frequency, imageability, arousal, and emotional valence. Conventional literal and metaphorical compounds increased BOLD signal change in right temporoparietal regions, suggesting combinatorial semantic processing, in line with the coarse semantic coding theory, but at odds with the graded salience hypothesis. Both novel literal and novel metaphorical expressions increased activity in left inferior frontal areas, presumably as a result of phonetic, morphosyntactic, and semantic unification processes, challenging predictions regarding right hemispheric involvement in processing unusual meanings. Meanwhile, both conventional and novel metaphorical expressions induced BOLD signal change in left hemispherical regions, suggesting that even novel metaphor processing involves more than linking semantically distant concepts.

The physiological origin of task-evoked systemic artefacts in functional near infrared spectroscopy.

A major methodological challenge of functional near-infrared spectroscopy (fNIRS) is its high sensitivity to haemodynamic fluctuations in the scalp. Superficial fluctuations contribute on the one hand to the physiological noise of fNIRS, impairing the signal-to-noise ratio, and may on the other hand be erroneously attributed to cerebral changes, leading to false positives in fNIRS experiments. Here we explore the localisation, time course and physiological origin of task-evoked superficial signals in fNIRS and present a method to separate them from cortical signals. We used complementary fNIRS, fMRI, MR-angiography and peripheral physiological measurements (blood pressure, heart rate, skin conductance and skin blood flow) to study activation in the frontal lobe during a continuous performance task. The General Linear Model (GLM) was applied to analyse the fNIRS data, which included an additional predictor to account for systemic changes in the skin. We found that skin blood volume strongly depends on the cognitive state and that sources of task-evoked systemic signals in fNIRS are co-localized with veins draining the scalp. Task-evoked superficial artefacts were mainly observed in concentration changes of oxygenated haemoglobin and could be effectively separated from cerebral signals by GLM analysis. Based on temporal correlation of fNIRS and fMRI signals with peripheral physiological measurements we conclude that the physiological origin of the systemic artefact is a task-evoked sympathetic arterial vasoconstriction followed by a decrease in venous volume. Since changes in sympathetic outflow accompany almost any cognitive and emotional process, we expect scalp vessel artefacts to be present in a wide range of fNIRS settings used in neurocognitive research. Therefore a careful separation of fNIRS signals originating from activated brain and from scalp is a necessary precondition for unbiased fNIRS brain activation maps.

Eye movements and brain electric potentials during reading.

The development of theories and computational models of reading requires an understanding of processing constraints, in particular of timelines related to word recognition and oculomotor control. Timelines of word recognition are usually determined with event-related potentials (ERPs) recorded under conditions of serial visual presentation (SVP) of words; timelines of oculomotor control are derived from parameters of eye movements (EMs) during natural reading. We describe two strategies to integrate these approaches. One is to collect ERPs and EMs in separate SVP and natural reading experiments for the same experimental material (but different subjects). The other strategy is to co-register EMs and ERPs during natural reading from the same subjects. Both strategies yield data that allow us to determine how lexical properties influence ERPs (e.g., the N400 component) and EMs (e.g., fixation durations) across neighboring words. We review our recent research on the effects of frequency and predictability of words on both EM and ERP measures with reference to current models of eye-movement control during reading. Results are in support of the proposition that lexical access is distributed across several fixations and across brain-electric potentials measured on neighboring words.

A dual-route cascaded model of reading by deaf adults: evidence for grapheme to viseme conversion.

There is an ongoing debate whether deaf individuals access phonology when reading, and if so, what impact the ability to access phonology might have on reading achievement. However, the debate so far has been theoretically unspecific on two accounts: (a) the phonological units deaf individuals may have of oral language have not been specified and (b) there seem to be no explicit cognitive models specifying how phonology and other factors operate in reading by deaf individuals. We propose that deaf individuals have representations of the sublexical structure of oral-aural language which are based on mouth shapes and that these sublexical units are activated during reading by deaf individuals. We specify the sublexical units of deaf German readers as 11 "visemes" and incorporate the viseme set into a working model of single-word reading by deaf adults based on the dual-route cascaded model of reading aloud by Coltheart, Rastle, Perry, Langdon, and Ziegler (2001. DRC: A dual route cascaded model of visual word recognition and reading aloud. Psychological Review, 108, 204-256. doi: 10.1037//0033-295x.108.1.204). We assessed the indirect route of this model by investigating the "pseudo-homoviseme" effect using a lexical decision task in deaf German reading adults. We found a main effect of pseudo-homovisemy, suggesting that at least some deaf individuals do automatically access sublexical structure during single-word reading.

Computational Models of Readers' Apperceptive Mass.

Recent progress in machine-learning-based distributed semantic models (DSMs) offers new ways to simulate the apperceptive mass (AM; Kintsch, 1980) of reader groups or individual readers and to predict their performance in reading-related tasks. The AM integrates the mental lexicon with world knowledge, as for example, acquired via reading books. Following pioneering work by Denhière and Lemaire (2004), here, we computed DSMs based on a representative corpus of German children and youth literature (Jacobs et al., 2020) as null models of the part of the AM that represents distributional semantic input, for readers of different reading ages (grades 1-2, 3-4, and 5-6). After a series of DSM quality tests, we evaluated the performance of these models quantitatively in various tasks to simulate the different reader groups' hypothetical semantic and syntactic skills. In a final study, we compared the models' performance with that of human adult and children readers in two rating tasks. Overall, the results show that with increasing reading age performance in practically all tasks becomes better. The approach taken in these studies reveals the limits of DSMs for simulating human AM and their potential for applications in scientific studies of literature, research in education, or developmental science.

Structural gray matter features and behavioral preliterate skills predict future literacy - A machine learning approach.

When children learn to read, their neural system undergoes major changes to become responsive to print. There seem to be nuanced interindividual differences in the neurostructural anatomy of regions that later become integral parts of the reading network. These differences might affect literacy acquisition and, in some cases, might result in developmental disorders like dyslexia. Consequently, the main objective of this longitudinal study was to investigate those interindividual differences in gray matter morphology that might facilitate or hamper future reading acquisition. We used a machine learning approach to examine to what extent gray matter macrostructural features and cognitive-linguistic skills measured before formal literacy teaching could predict literacy 2 years later. Forty-two native German-speaking children underwent T1-weighted magnetic resonance imaging and psychometric testing at the end of kindergarten. They were tested again 2 years later to assess their literacy skills. A leave-one-out cross-validated machine-learning regression approach was applied to identify the best predictors of future literacy based on cognitive-linguistic preliterate behavioral skills and cortical measures in a priori selected areas of the future reading network. With surprisingly high accuracy, future literacy was predicted, predominantly based on gray matter volume in the left occipito-temporal cortex and local gyrification in the left insular, inferior frontal, and supramarginal gyri. Furthermore, phonological awareness significantly predicted future literacy. In sum, the results indicate that the brain morphology of the large-scale reading network at a preliterate age can predict how well children learn to read.