Predicting the brain activation pattern associated with the propositional content of a sentence: Modeling neural representations of events and states.

Even though much has recently been learned about the neural representation of individual concepts and categories, neuroimaging research is only beginning to reveal how more complex thoughts, such as event and state descriptions, are neurally represented. We present a predictive computational theory of the neural representations of individual events and states as they are described in 240 sentences. Regression models were trained to determine the mapping between 42 neurally plausible semantic features (NPSFs) and thematic roles of the concepts of a proposition and the fMRI activation patterns of various cortical regions that process different types of information. Given a semantic characterization of the content of a sentence that is new to the model, the model can reliably predict the resulting neural signature, or, given an observed neural signature of a new sentence, the model can predict its semantic content. The models were also reliably generalizable across participants. This computational model provides an account of the brain representation of a complex yet fundamental unit of thought, namely, the conceptual content of a proposition. In addition to characterizing a sentence representation at the level of the semantic and thematic features of its component concepts, factor analysis was used to develop a higher level characterization of a sentence, specifying the general type of event representation that the sentence evokes (e.g., a social interaction versus a change of physical state) and the voxel locations most strongly associated with each of the factors. Hum Brain Mapp 38:4865-4881, 2017. © 2017 Wiley Periodicals, Inc.

Neural representations of the concepts in simple sentences: Concept activation prediction and context effects.

Although it has been possible to identify individual concepts from a concept's brain activation pattern, there have been significant obstacles to identifying a proposition from its fMRI signature. Here we demonstrate the ability to decode individual prototype sentences from readers' brain activation patterns, by using theory-driven regions of interest and semantic properties. It is possible to predict the fMRI brain activation patterns evoked by propositions and words which are entirely new to the model with reliably above-chance rank accuracy. The two core components implemented in the model that reflect the theory were the choice of intermediate semantic features and the brain regions associated with the neurosemantic dimensions. This approach also predicts the neural representation of object nouns across participants, studies, and sentence contexts. Moreover, we find that the neural representation of an agent-verb-object proto-sentence is more accurately characterized by the neural signatures of its components as they occur in a similar context than by the neural signatures of these components as they occur in isolation.

Alterations of functional connectivity and intrinsic activity within the cingulate cortex of suicidal ideators.

The 'default mode network' (DMN), a collection of brain regions including the posterior cingulate cortex (PCC), shows reliable inter-regional functional connectivity at rest. It has been implicated in rumination and other negative affective states, but its role in suicidal ideation is not well understood. We employed seed based functional connectivity methods to analyze resting state fMRI data in 34 suicidal ideators and 40 healthy control participants. Whole-brain connectivity with dorsal PCC or ventral PCC was broadly intact between the two groups, but while the control participants showed greater coupling between the dorsal anterior cingulate cortex (dACC) and dorsal PCC, compared to the dACC and ventral PCC, this difference was reversed in the ideators. Furthermore, ongoing low frequency BOLD signal in these three regions (dorsal, ventral PCC, dACC) was reduced in the ideators. The structural integrity of the cingulum bundle, as measured using diffusion tensor imaging (DTI), also explained variation in the functional connectivity measures but did not abolish the group differences. Together, these findings provide evidence of abnormalities in the DMN underlying the tendency towards suicidal ideation.

Machine learning of neural representations of suicide and emotion concepts identifies suicidal youth.

The clinical assessment of suicidal risk would be significantly complemented by a biologically-based measure that assesses alterations in the neural representations of concepts related to death and life in people who engage in suicidal ideation. This study used machine-learning algorithms (Gaussian Naïve Bayes) to identify such individuals (17 suicidal ideators vs 17 controls) with high (91%) accuracy, based on their altered fMRI neural signatures of death and life-related concepts. The most discriminating concepts were death, cruelty, trouble, carefree, good, and praise. A similar classification accurately (94%) discriminated 9 suicidal ideators who had made a suicide attempt from 8 who had not. Moreover, a major facet of the concept alterations was the evoked emotion, whose neural signature served as an alternative basis for accurate (85%) group classification. The study establishes a biological, neurocognitive basis for altered concept representations in participants with suicidal ideation, which enables highly accurate group membership classification.

Identifying thematic roles from neural representations measured by functional magnetic resonance imaging.

The generativity and complexity of human thought stem in large part from the ability to represent relations among concepts and form propositions. The current study reveals how a given object such as rabbit is neurally encoded differently and identifiably depending on whether it is an agent ("the rabbit punches the monkey") or a patient ("the monkey punches the rabbit"). Machine-learning classifiers were trained on functional magnetic resonance imaging (fMRI) data evoked by a set of short videos that conveyed agent-verb-patient propositions. When tested on a held-out video, the classifiers were able to reliably identify the thematic role of an object from its associated fMRI activation pattern. Moreover, when trained on one subset of the study participants, classifiers reliably identified the thematic roles in the data of a left-out participant (mean accuracy = .66), indicating that the neural representations of thematic roles were common across individuals.

Modality-independent representations of small quantities based on brain activation patterns.

Machine learning or MVPA (Multi Voxel Pattern Analysis) studies have shown that the neural representation of quantities of objects can be decoded from fMRI patterns, in cases where the quantities were visually displayed. Here we apply these techniques to investigate whether neural representations of quantities depicted in one modality (say, visual) can be decoded from brain activation patterns evoked by quantities depicted in the other modality (say, auditory). The main finding demonstrated, for the first time, that quantities of dots were decodable by a classifier that was trained on the neural patterns evoked by quantities of auditory tones, and vice-versa. The representations that were common across modalities were mainly right-lateralized in frontal and parietal regions. A second finding was that the neural patterns in parietal cortex that represent quantities were common across participants. These findings demonstrate a common neuronal foundation for the representation of quantities across sensory modalities and participants and provide insight into the role of parietal cortex in the representation of quantity information.

Aberrant functioning of the theory-of-mind network in children and adolescents with autism.

BACKGROUND: Theory-of-mind (ToM), the ability to infer people's thoughts and feelings, is a pivotal skill in effective social interactions. Individuals with autism spectrum disorders (ASD) have been found to have altered ToM skills, which significantly impacts the quality of their social interactions. Neuroimaging studies have reported altered activation of the ToM cortical network, especially in adults with autism, yet little is known about the brain responses underlying ToM in younger individuals with ASD. This functional magnetic resonance imaging (fMRI) study investigated the neural mechanisms underlying ToM in high-functioning children and adolescents with ASD and matched typically developing (TD) peers. METHODS: fMRI data were acquired from 13 participants with ASD and 13 TD control participants while they watched animations involving two "interacting" geometrical shapes. RESULTS: Participants with ASD showed significantly reduced activation, relative to TD controls, in regions considered part of the ToM network, the mirror network, and the cerebellum. Functional connectivity analyses revealed underconnectivity between frontal and posterior regions during task performance in the ASD participants. CONCLUSIONS: Overall, the findings of this study reveal disruptions in the brain circuitry underlying ToM in ASD at multiple levels, including decreased activation and decreased functional connectivity.

Identifying autism from neural representations of social interactions: neurocognitive markers of autism.

Autism is a psychiatric/neurological condition in which alterations in social interaction (among other symptoms) are diagnosed by behavioral psychiatric methods. The main goal of this study was to determine how the neural representations and meanings of social concepts (such as to insult) are altered in autism. A second goal was to determine whether these alterations can serve as neurocognitive markers of autism. The approach is based on previous advances in fMRI analysis methods that permit (a) the identification of a concept, such as the thought of a physical object, from its fMRI pattern, and (b) the ability to assess the semantic content of a concept from its fMRI pattern. These factor analysis and machine learning methods were applied to the fMRI activation patterns of 17 adults with high-functioning autism and matched controls, scanned while thinking about 16 social interactions. One prominent neural representation factor that emerged (manifested mainly in posterior midline regions) was related to self-representation, but this factor was present only for the control participants, and was near-absent in the autism group. Moreover, machine learning algorithms classified individuals as autistic or control with 97% accuracy from their fMRI neurocognitive markers. The findings suggest that psychiatric alterations of thought can begin to be biologically understood by assessing the form and content of the altered thought's underlying brain activation patterns.

Identifying Emotions on the Basis of Neural Activation.

We attempt to determine the discriminability and organization of neural activation corresponding to the experience of specific emotions. Method actors were asked to self-induce nine emotional states (anger, disgust, envy, fear, happiness, lust, pride, sadness, and shame) while in an fMRI scanner. Using a Gaussian Naïve Bayes pooled variance classifier, we demonstrate the ability to identify specific emotions experienced by an individual at well over chance accuracy on the basis of: 1) neural activation of the same individual in other trials, 2) neural activation of other individuals who experienced similar trials, and 3) neural activation of the same individual to a qualitatively different type of emotion induction. Factor analysis identified valence, arousal, sociality, and lust as dimensions underlying the activation patterns. These results suggest a structure for neural representations of emotion and inform theories of emotional processing.

Brain function differences in language processing in children and adults with autism.

Comparison of brain function between children and adults with autism provides an understanding of the effects of the disorder and associated maturational differences on language processing. Functional imaging (functional magnetic resonance imaging) was used to examine brain activation and cortical synchronization during the processing of literal and ironic texts in 15 children with autism, 14 children with typical development, 13 adults with autism, and 12 adult controls. Both the children and adults with autism had lower functional connectivity (synchronization of brain activity among activated areas) than their age and ability comparison group in the left hemisphere language network during irony processing, and neither autism group had an increase in functional connectivity in response to increased task demands. Activation differences for the literal and irony conditions occurred in key language-processing regions (left middle temporal, left pars triangularis, left pars opercularis, left medial frontal, and right middle temporal). The children and adults with autism differed from each other in the use of some brain regions during the irony task, with the adults with autism having activation levels similar to those of the control groups. Overall, the children and adults with autism differed from the adult and child controls in (a) the degree of network coordination, (b) the distribution of the workload among member nodes, and (3) the dynamic recruitment of regions in response to text content. Moreover, the differences between the two autism age groups may be indicative of positive changes in the neural function related to language processing associated with maturation and/or educational experience.

Autonomy of lower-level perception from global processing in autism: evidence from brain activation and functional connectivity.

Previous behavioral studies have shown that individuals with autism are less hindered by interference from global processing during the performance of lower-level perceptual tasks, such as finding embedded figures. The primary goal of this study was to examine the brain manifestation of such atypicality in high-functioning autism using fMRI. Fifteen participants with high-functioning autism and fifteen age- and IQ-matched typical controls were asked to perform a lower-level perceptual line-counting task in the presence of a distracting depiction of a 3-D object, in which participants counted whether there were more red or more green contours (In a contrasting 3-D task, participants judged whether the same 3-D stimulus objects (but without color in any contours) depicted a possible or impossible 3-D object). We hypothesized that individuals with autism would be less likely than controls to process the global 3-D information (and would hence show fewer neural signs of such interfering 3-D processing) during the lower-level line-counting task. The fMRI results revealed that in the line-counting task, the autism group did not show the increased medial frontal activity (relative to the possibility task), or the increased functional connectivity between the medial frontal region and posterior visual-spatial regions, demonstrated by the control group. Both findings are indices of lesser effort and difficulty in the line-counting task for the autism group than for the control group, attributed to less interference from the 3-D processing in the autism group. In addition, in the line-counting task, the control group showed a positive correlation between a measure of spatial ability (Vandenberg scores) and activation in the medial frontal region, suggesting that more spatially able control participants did more suppression of the irrelevant 3-D background information in order to focus on the line-counting task. The findings collectively indicate that the global 3-D structure of the figure had a smaller effect, if any, on local processing in the group with autism compared to the control group. The results from this study provide the first direct neural evidence of reduced global-to-local interference in autism.

Cortical underconnectivity coupled with preserved visuospatial cognition in autism: Evidence from an fMRI study of an embedded figures task.

Individuals with high-functioning autism sometimes exhibit intact or superior performance on visuospatial tasks, in contrast to impaired functioning in other domains such as language comprehension, executive tasks, and social functions. The goal of the current study was to investigate the neural bases of preserved visuospatial processing in high-functioning autism from the perspective of the cortical underconnectivity theory. We used a combination of behavioral, functional magnetic resonance imaging, functional connectivity, and corpus callosum morphometric methodological tools. Thirteen participants with high-functioning autism and 13 controls (age-, IQ-, and gender-matched) were scanned while performing an Embedded Figures Task. Despite the ability of the autism group to attain behavioral performance comparable to the control group, the brain imaging results revealed several group differences consistent with the cortical underconnectivity account of autism. First, relative to controls, the autism group showed less activation in the left dorsolateral prefrontal and inferior parietal areas and more activation in visuospatial (bilateral superior parietal extending to inferior parietal and right occipital) areas. Second, the autism group demonstrated lower functional connectivity between higher-order working memory/executive areas and visuospatial regions (between frontal and parietal-occipital). Third, the size of the corpus callosum (an index of anatomical connectivity) was positively correlated with frontal-posterior (parietal and occipital) functional connectivity in the autism group. Thus, even in the visuospatial domain, where preserved performance among people with autism is observed, the neuroimaging signatures of cortical underconnectivity persist.

A neurosemantic theory of concrete noun representation based on the underlying brain codes.

This article describes the discovery of a set of biologically-driven semantic dimensions underlying the neural representation of concrete nouns, and then demonstrates how a resulting theory of noun representation can be used to identify simple thoughts through their fMRI patterns. We use factor analysis of fMRI brain imaging data to reveal the biological representation of individual concrete nouns like apple, in the absence of any pictorial stimuli. From this analysis emerge three main semantic factors underpinning the neural representation of nouns naming physical objects, which we label manipulation, shelter, and eating. Each factor is neurally represented in 3-4 different brain locations that correspond to a cortical network that co-activates in non-linguistic tasks, such as tool use pantomime for the manipulation factor. Several converging methods, such as the use of behavioral ratings of word meaning and text corpus characteristics, provide independent evidence of the centrality of these factors to the representations. The factors are then used with machine learning classifier techniques to show that the fMRI-measured brain representation of an individual concrete noun like apple can be identified with good accuracy from among 60 candidate words, using only the fMRI activity in the 16 locations associated with these factors. To further demonstrate the generativity of the proposed account, a theory-based model is developed to predict the brain activation patterns for words to which the algorithm has not been previously exposed. The methods, findings, and theory constitute a new approach of using brain activity for understanding how object concepts are represented in the mind.

Atypical frontal-posterior synchronization of Theory of Mind regions in autism during mental state attribution.

This study used fMRI to investigate the functioning of the Theory of Mind (ToM) cortical network in autism during the viewing of animations that in some conditions entailed the attribution of a mental state to animated geometric figures. At the cortical level, mentalizing (attribution of metal states) is underpinned by the coordination and integration of the components of the ToM network, which include the medial frontal gyrus, the anterior paracingulate, and the right temporoparietal junction. The pivotal new finding was a functional underconnectivity (a lower degree of synchronization) in autism, especially in the connections between frontal and posterior areas during the attribution of mental states. In addition, the frontal ToM regions activated less in participants with autism relative to control participants. In the autism group, an independent psychometric assessment of ToM ability and the activation in the right temporoparietal junction were reliably correlated. The results together provide new evidence for the biological basis of atypical processing of ToM in autism, implicating the underconnectivity between frontal regions and more posterior areas.

Modifying the brain activation of poor readers during sentence comprehension with extended remedial instruction: a longitudinal study of neuroplasticity.

This study used fMRI to longitudinally assess the impact of intensive remedial instruction on cortical activation among 5th grade poor readers during a sentence comprehension task. The children were tested at three time points: prior to remediation, after 100 h of intensive instruction, and 1 year after the instruction had ended. Changes in brain activation were also measured among 5th grade good readers at the same time points for comparison. The central finding was that prior to instruction, the poor readers had significantly less activation than good readers bilaterally in the parietal cortex. Immediately after instruction, poor readers made substantial gains in reading ability, and demonstrated significantly increased activation in the left angular gyrus and the left superior parietal lobule. Activation in these regions continued to increase among poor readers 1 year post-remediation, resulting in a normalization of the activation. These results are interpreted as reflecting changes in the processes involved in word-level and sentence-level assembly. Areas of overactivation were also found among poor readers in the medial frontal cortex, possibly indicating a more effortful and attentive guided reading strategy.

fMRI investigation of working memory for faces in autism: visual coding and underconnectivity with frontal areas.

Brain activation and functional connectivity were investigated in high functioning autism using functional magnetic resonance imaging in an n-back working memory task involving photographic face stimuli. The autism group showed reliably lower activation compared with controls in the inferior left prefrontal area (involved in verbal processing and working memory maintenance) and the right posterior temporal area (associated with theory of mind processing). The participants with autism also showed activation in a somewhat different location in the fusiform area than the control participants. These results suggest that the neural circuitry of the brain for face processing in autism may be analyzing the features of the face more as objects and less in terms of their human significance. The functional connectivity results revealed that the abnormal fusiform activation was embedded in a larger context of smaller and less synchronized networks, particularly indicating lower functional connectivity with frontal areas. In contrast to the underconnectivity with frontal areas, the autism group showed no underconnectivity among posterior cortical regions. These results extend previous findings of abnormal face perception in autism by demonstrating that the abnormalities are embedded in an abnormal cortical network that manages to perform the working memory task proficiently, using a visually oriented, asocial processing style that minimizes reliance on prefrontal areas.

Brain activation during sentence comprehension among good and poor readers.

This study sought to increase current understanding of the neuropsychological basis of poor reading ability by using fMRI to examine brain activation during a visual sentence comprehension task among good and poor readers in the third (n = 32) and fifth (n = 35) grades. Reading ability, age, and the combination of both factors made unique contributions to cortical activation. The main finding was of parietotemporal underactivation (less activation than controls) among poor readers at the 2 grade levels. A positive linear relationship (spanning both the poor and good readers) was found between reading ability and activation in the left posterior middle temporal and postcentral gyri and in the right inferior parietal lobule such that activation increased with reading ability. Different developmental trajectories characterized good and poor readers in the left angular gyrus: activation increased with age among good readers, a change that failed to occur among poor readers. The parietotemporal cortex is discussed in terms of its role in reading acquisition, with the left angular gyrus playing a key role. It is proposed that the functioning of the cortical network underlying reading is dependent on a combination of interacting factors, including physiological maturation, neural integrity, skill level, and the nature of the task.