Hierarchical Brain Network for Face and Voice Integration of Emotion Expression.

The brain has separate specialized computational units to process faces and voices located in occipital and temporal cortices. However, humans seamlessly integrate signals from the faces and voices of others for optimal social interaction. How are emotional expressions, when delivered by different sensory modalities (faces and voices), integrated in the brain? In this study, we characterized the brains' response to faces, voices, and combined face-voice information (congruent, incongruent), which varied in expression (neutral, fearful). Using a whole-brain approach, we found that only the right posterior superior temporal sulcus (rpSTS) responded more to bimodal stimuli than to face or voice alone but only when the stimuli contained emotional expression. Face- and voice-selective regions of interest, extracted from independent functional localizers, similarly revealed multisensory integration in the face-selective rpSTS only; further, this was the only face-selective region that also responded significantly to voices. Dynamic causal modeling revealed that the rpSTS receives unidirectional information from the face-selective fusiform face area, and voice-selective temporal voice area, with emotional expression affecting the connection strength. Our study promotes a hierarchical model of face and voice integration, with convergence in the rpSTS, and that such integration depends on the (emotional) salience of the stimuli.

Perceptual Learning of Faces: A Rehabilitative Study of Acquired Prosopagnosia.

Despite many studies of acquired prosopagnosia, there have been only a few attempts at its rehabilitation, all in single cases, with a variety of mnemonic or perceptual approaches, and of variable efficacy. In a cohort with acquired prosopagnosia, we evaluated a perceptual learning program that incorporated variations in view and expression, which was aimed at training perceptual stages of face processing with an emphasis on ecological validity. Ten patients undertook an 11-week face training program and an 11-week control task. Training required shape discrimination between morphed facial images, whose similarity was manipulated by a staircase procedure to keep training near a perceptual threshold. Training progressed from blocks of neutral faces in frontal view through increasing variations in view and expression. Whereas the control task did not change perception, training improved perceptual sensitivity for the trained faces and generalized to new untrained expressions and views of those faces. There was also a significant transfer to new faces. Benefits were maintained over a 3-month period. Training efficacy was greater for those with more perceptual deficits at baseline. We conclude that perceptual learning can lead to persistent improvements in face discrimination in acquired prosopagnosia. This reflects both acquisition of new skills that can be applied to new faces as well as a degree of overlearning of the stimulus set at the level of 3-D expression-invariant representations.

Image-invariant responses in face-selective regions do not explain the perceptual advantage for familiar face recognition.

The ability to recognize familiar faces across different viewing conditions contrasts with the inherent difficulty in the perception of unfamiliar faces across similar image manipulations. It is widely believed that this difference in perception and recognition is based on the neural representation for familiar faces being less sensitive to changes in the image than it is for unfamiliar faces. Here, we used an functional magnetic resonance-adaptation paradigm to investigate image invariance in face-selective regions of the human brain. We found clear evidence for a degree of image-invariant adaptation to facial identity in face-selective regions, such as the fusiform face area. However, contrary to the predictions of models of face processing, comparable levels of image invariance were evident for both familiar and unfamiliar faces. This suggests that the marked differences in the perception of familiar and unfamiliar faces may not depend on differences in the way multiple images are represented in core face-selective regions of the human brain.

Impaired Face Feature-to-Location Statistical Learning and Single-Feature Discrimination in Developmental Prosopagnosia.

Individuals with developmental prosopagnosia (DP) experience severe face memory deficits that are often accompanied by impairments in face perception. Images of human facial features are better discriminated between when they are presented in the locations on the visual field that they typically appear in while viewing human faces in daily life, than in locations which they do not typically appear (i.e., better performance for eyes in the upper visual field, and better performance for mouths in the lower visual field). These feature-to-location tuning effects (FLEs) can be explained by a retinotopically organised visual statistical learning mechanism. We had a large group of DP participants (N = 64), a control group (N = 74) and a group of individuals with a mild form of DP (N = 58) complete a single-feature discrimination task to determine whether face perception deficits in DP can be accounted for by an impairment in face feature-to-location tuning. The results showed that individuals with DP did not have significant FLEs, suggesting a marked impairment in the underlying visual statistical learning mechanism. In contrast, the mild DP group showed normal FLE effects which did not differ from the control group. Both DP groups had impaired single-feature processing (SFP) as compared to the control group. We also examined the effects of age on FLEs and SFP.

Face Feature Change Detection Ability in Developmental Prosopagnosia and Super-Recognisers.

In non-clinical populations, facial features (eyes, nose, mouth) may vary in their contribution to face identity perception. Changes to whole faces are easier to detect than changes to individual features, and eye changes are typically easier to detect than mouth changes, which in turn are easier to detect than nose changes. However, how this differs for people with face recognition difficulties (developmental prosopagnosia; DP) and for individuals with superior face recognition abilities (super-recognisers; SR) is not clear; although findings from previous studies have suggested differences, the nature of this difference is not understood. The aim of this study was to examine whether differences in the ability to detect feature changes in DPs and SRs were (a) quantitative, meaning that the pattern across feature changes remained the same but there was an overall upwards or downwards shift in performance, or (b) qualitative, meaning that the pattern across feature changes was different. Using a change detection task in which individual face features (eyes, nose, mouth) changed between sequentially presented faces, we found that while prosopagnosics showed a quantitative difference in performance with a downwards shift across all conditions, super-recognisers only showed qualitative differences: they were better able to detect when the face was the same and were marginally (but not non-significantly) worse at detecting when the eyes changed. Further, the only condition which distinguished between the three groups was the ability to identify when the same face was presented, with SRs being better than controls, and controls being better than DPs. Our findings suggest that, in feature-matching tasks, differences for DPs are due to them being overall worse at the task, while SRs use a qualitatively different strategy.

Intra- and interhemispheric connectivity between face-selective regions in the human brain.

Neuroimaging studies have revealed a number of regions in the human brain that respond to faces. However, the way these regions interact is a matter of current debate. The aim of this study was to use functional MRI to define face-selective regions in the human brain and then determine how these regions interact in a large population of subjects (n = 72). We found consistent face selectivity in the core face regions of the occipital and temporal lobes: the fusiform face area (FFA), occipital face area (OFA), and superior temporal sulcus (STS). Face selectivity extended into the intraparietal sulcus (IPS), precuneus (PCu), superior colliculus (SC), amygdala (AMG), and inferior frontal gyrus (IFG). We found evidence for significant functional connectivity between the core face-selective regions, particularly between the OFA and FFA. However, we found that the covariation in activity between corresponding face regions in different hemispheres (e.g., right and left FFA) was higher than between different face regions in the same hemisphere (e.g., right OFA and right FFA). Although functional connectivity was evident between regions in the core and extended network, there were significant differences in the magnitude of the connectivity between regions. Activity in the OFA and FFA were most correlated with the IPS, PCu, and SC. In contrast, activity in the STS was most correlated with the AMG and IFG. Correlations between the extended regions suggest strong functional connectivity between the IPS, PCu, and SC. In contrast, the IFG was only correlated with the AMG. This study reveals that interhemispheric as well as intrahemispheric connections play an important role in face perception.

Right hemi-alexia.

While pure alexia was long considered a disconnection syndrome, it may also be a selective visual word agnosia due to damage to the visual word form area. Disconnection is still the likely explanation of hemi-alexias, though, particularly when splenial lesions damage inter-hemispheric projections and cause left hemi-alexia. An intra-hemispheric disconnection causing right hemi-alexia is theoretically possible but seems very rare, with only a single report that has been challenged on the grounds of inadequate perimetry. We describe the case of PH, who had a severe reading deficit in her right hemifield. Detailed perimetry showed only a small relative hemi-scotoma along the horizontal meridian, while word reading was impaired over a much larger expanse of her right hemifield, in which object recognition was spared. Reading, lexical decisions, and perceptual discrimination of words were impaired in the right hemifield, and this extended to letters and numbers, with a trend to an effect on the perception of an unfamiliar script, namely Korean. On magnetic resonance imaging she had a large left lateral occipital meningioma with vasogenic edema of occipital white matter tracts. Functional magnetic resonance imaging showed that the visual word form area was located just anterior to the mass. Her perceptual abnormalities resolved after resection of the tumor. We conclude that right hemi-alexia exists and is most likely due to intra-hemispheric disconnection of occipital input to the visual word form area.

Internal and external features of the face are represented holistically in face-selective regions of visual cortex.

The perception and recognition of familiar faces depends critically on an analysis of the internal features of the face (eyes, nose, mouth). We therefore contrasted how information about the internal and external (hair, chin, face outline) features of familiar and unfamiliar faces is represented in face-selective regions. There was a significant response to both the internal and external features of the face when presented in isolation. However, the response to the internal features was greater than the response to the external features. There was significant adaptation to repeated images of either the internal or external features of the face in the fusiform face area (FFA). However, the magnitude of this adaptation was greater for the internal features of familiar faces. Next, we asked whether the internal features of the face are represented independently from the external features. There was a release from adaptation in the FFA to composite images in which the internal features were varied but the external features were unchanged, or when the internal features were unchanged but the external features varied, demonstrating a holistic response. Finally, we asked whether the holistic response to faces could be influenced by the context in which the face was presented. We found that adaptation was still evident to composite images in which the face was unchanged but body features were varied. Together, these findings show that although internal features are important in the neural representation of familiar faces, the face's internal and external features are represented holistically in face-selective regions of the human brain.

Can Machines Find the Bilingual Advantage? Machine Learning Algorithms Find No Evidence to Differentiate Between Lifelong Bilingual and Monolingual Cognitive Profiles.

Bilingualism has been identified as a potential cognitive factor linked to delayed onset of dementia as well as boosting executive functions in healthy individuals. However, more recently, this claim has been called into question following several failed replications. It remains unclear whether these contradictory findings reflect how bilingualism is defined between studies, or methodological limitations when measuring the bilingual effect. One key issue is that despite the claims that bilingualism yields general protection to cognitive processes (i.e., the cognitive reserve hypothesis), studies reporting putative bilingual differences are often focused on domain specific experimental paradigms. This study chose a broader approach, by considering the consequences of bilingualism on a wide range of cognitive functions within individuals. We utilised 19 measures of different cognitive functions commonly associated with bilingual effects, to form a "cognitive profile" for 215 non-clinical participants. We recruited Welsh speakers, who as a group of bilinguals were highly homogeneous, as means of isolating the bilingualism criterion. We sought to determine if such analyses would independently classify bilingual/monolingual participant groups based on emergent patterns driven by collected cognitive profiles, such that population differences would emerge. Multiple predictive models were trained to independently recognise the cognitive profiles of bilinguals, older adults (60-90 years of age) and higher education attainment. Despite managing to successfully classify cognitive profiles based on age and education, the model failed to differentiate between bilingual and monolingual cognitive ability at a rate greater than that of chance. Repeated modelling using alternative definitions of bilingualism, and just the older adults, yielded similar results. In all cases then, using our "bottom-up" analytical approach, there was no evidence that bilingualism as a variable indicated differential cognitive performance - as a consequence, we conclude that bilinguals are not cognitively different from their monolingual counterparts, even in older demographics. We suggest that studies that have reported a bilingual advantage (typically recruiting immigrant populations) could well have confounded other key variables that may be driving reported advantages. We recommend that future research refine the machine learning methods used in this study to further investigate the complex relationship between bilingualism and cognition.

Prosopagnosia and disorders of face processing.

Face recognition is a form of expert visual processing. Acquired prosopagnosia is the loss of familiarity for facial identity and has several functional variants, namely apperceptive, amnestic, and associative forms. Acquired forms are usually caused by either occipitotemporal or anterior temporal lesions, right or bilateral in most cases. In addition, there is a developmental form, whose functional and structural origins are still being elucidated. Despite their difficulties with recognizing faces, some of these subjects still show signs of covert recognition, which may have a number of explanations. Other aspects of face perception can be spared in prosopagnosic subjects. Patients with other types of face processing difficulties have been described, including impaired expression processing, impaired lip-reading, false familiarity for faces, and a people-specific amnesia. Recent rehabilitative studies have shown some modest ability to improve face perception in prosopagnosic subjects through perceptual training protocols.

Training face perception in developmental prosopagnosia through perceptual learning.

BACKGROUND: Recent work has shown that perceptual learning can improve face discrimination in subjects with acquired prosopagnosia. OBJECTIVE: In this study, we administered the same program to determine if such training would improve face perception in developmental prosopagnosia. METHOD: We trained ten subjects with developmental prosopagnosia for several months with a program that required shape discrimination between morphed facial images, using a staircase procedure to keep training near each subject's perceptual threshold. To promote ecological validity, training progressed from blocks of neutral faces in frontal view through increasing variations in view and expression. Five subjects did 11 weeks of a control television task before training, and the other five were re-assessed for maintenance of benefit 3 months after training. RESULTS: Perceptual sensitivity for faces improved after training but did not improve after the control task. Improvement generalized to untrained expressions and views of these faces, and there was some evidence of transfer to new faces. Benefits were maintained over three months. Training also led to improvements on standard neuropsychological tests of short-term familiarity, and some subjects reported positive effects in daily life. CONCLUSION: We conclude that perceptual learning can lead to persistent improvements in face discrimination in developmental prosopagnosia. The strong generalization suggests that learning is occurring at the level of three-dimensional representations with some invariance for the dynamic effects of expression.

Cortical correlates of speech intelligibility measured using functional near-infrared spectroscopy (fNIRS).

Functional neuroimaging has identified that the temporal, frontal and parietal cortex support core aspects of speech processing. An objective measure of speech intelligibility based on cortical activation in these brain regions would be extremely useful to speech communication and hearing device applications. In the current study, we used noise-vocoded speech to examine cortical correlates of speech intelligibility in normally-hearing listeners using functional near-infrared spectroscopy (fNIRS), a non-invasive, neuroimaging technique that is fully-compatible with hearing devices, including cochlear implants. In twenty-three normally-hearing adults we measured (1) activation in superior temporal, inferior frontal and inferior parietal cortex bilaterally and (2) behavioural speech intelligibility. Listeners heard noise-vocoded sentences targeting five equally spaced levels of intelligibility between 0 and 100% correct. Activation in superior temporal regions increased linearly with intelligibility. This relationship appears to have been driven in part by changing acoustic properties across stimulation conditions, rather than solely by intelligibility per se. Superior temporal activation was also predictive of individual differences in intelligibility in a challenging listening condition. Beyond superior temporal cortex, we identified regions in which activation varied non-linearly with intelligibility. For example, in left inferior frontal cortex, activation peaked in response to heavily degraded, yet still somewhat intelligible, speech. Activation in this region was linearly related to response time on a simultaneous behavioural task, suggesting it may contribute to decision making. Our results indicate that fNIRS has the potential to provide an objective measure of speech intelligibility in normally-hearing listeners. Should these results be found to apply similarly in the case of individuals listening through a cochlear implant, fNIRS would demonstrate potential for a clinically useful measure not only of speech intelligibility, but also of listening effort.

The relationship between visual word and face processing lateralization in the fusiform gyri: A cross-sectional study.

Visual words and faces activate similar networks but with complementary hemispheric asymmetries, faces being lateralized to the right and words to the left. A recent theory proposes that this reflects developmental competition between visual word and face processing. We investigated whether this results in an inverse correlation between the degree of lateralization of visual word and face activation in the fusiform gyri. 26 literate right-handed healthy adults underwent functional MRI with face and word localizers. We derived lateralization indices for cluster size and peak responses for word and face activity in left and right fusiform gyri, and correlated these across subjects. A secondary analysis examined all face- and word-selective voxels in the inferior occipitotemporal cortex. No negative correlations were found. There were positive correlations for the peak MR response between word and face activity within the left hemisphere, and between word activity in the left visual word form area and face activity in the right fusiform face area. The face lateralization index was positively rather than negatively correlated with the word index. In summary, we do not find a complementary relationship between visual word and face lateralization across subjects. The significance of the positive correlations is unclear: some may reflect the influences of general factors such as attention, but others may point to other factors that influence lateralization of function.

Representation of visual symbols in the visual word processing network.

BACKGROUND: Previous studies have shown that word processing involves a predominantly left-sided occipitotemporal network. Words are a form of symbolic representation, in that they are arbitrary perceptual stimuli that represent other objects, actions or concepts. Lesions of parts of the visual word processing network can cause alexia, which can be associated with difficulty processing other types of symbols such as musical notation or road signs. OBJECTIVE: We investigated whether components of the visual word processing network were also activated by other types of symbols. METHOD: In 16 music-literate subjects, we defined the visual word network using fMRI and examined responses to four symbolic categories: visual words, musical notation, instructive symbols (e.g. traffic signs), and flags and logos. For each category we compared responses not only to scrambled stimuli, but also to similar stimuli that lacked symbolic meaning. RESULTS: The left visual word form area and a homologous right fusiform region responded similarly to all four categories, but equally to both symbolic and non-symbolic equivalents. Greater response to symbolic than non-symbolic stimuli occurred only in the left inferior frontal and middle temporal gyri, but only for words, and in the case of the left inferior frontal gyri, also for musical notation. A whole-brain analysis comparing symbolic versus non-symbolic stimuli revealed a distributed network of inferior temporooccipital and parietal regions that differed for different symbols. CONCLUSION: The fusiform gyri are involved in processing the form of many symbolic stimuli, but not specifically for stimuli with symbolic content. Selectivity for stimuli with symbolic content only emerges in the visual word network at the level of the middle temporal and inferior frontal gyri, but is specific for words and musical notation.

Changes in cerebral vascular reactivity and structure following prolonged exposure to high altitude in humans.

Although high-altitude exposure can lead to neurocognitive impairment, even upon return to sea level, it remains unclear the extent to which brain volume and regional cerebral vascular reactivity (CVR) are altered following high-altitude exposure. The purpose of this study was to simultaneously determine the effect of 3 weeks at 5050 m on: (1) structural brain alterations; and (2) regional CVR after returning to sea level for 1 week. Healthy human volunteers (n = 6) underwent baseline and follow-up structural and functional magnetic resonance imaging (MRI) at rest and during a CVR protocol (end-tidal PCO2 reduced by -10, -5 and increased by +5, +10, and +15 mmHg from baseline). CVR maps (% mmHg(-1)) were generated using BOLD MRI and brain volumes were estimated. Following return to sea level, whole-brain volume and gray matter volume was reduced by 0.4 ± 0.3% (P < 0.01) and 2.6 ± 1.0% (P < 0.001), respectively; white matter was unchanged. Global gray matter CVR and white matter CVR were unchanged following return to sea level, but CVR was selectively increased (P < 0.05) in the brainstem (+30 ± 12%), hippocampus (+12 ± 3%), and thalamus (+10 ± 3%). These changes were the result of improvement and/or reversal of negative CVR to positive CVR in these regions. Three weeks of high-altitude exposure is reflected in loss of gray matter volume and improvements in negative CVR.

Acquired prosopagnosia: structural basis and processing impairments.

Cognitive models propose a hierarchy of parallel processing stages in face perception, and functional neuroimaging shows a network of regions involved in face processing. Reflecting this, acquired prosopagnosia is not a single entity but a family of disorders with different anatomic lesions and different functional deficits. One classic distinction is between an apperceptive variant, in which there is impaired perception of facial structure, and an associative/amnestic variant, in which perception is relatively intact, with subsequent problems matching perception to facial memories, because of either disconnection or loss of those memories. These disorders also have to be distinguished from people-specific amnesia, a multimodal impairment, and prosop-anomia, in which familiarity with faces is preserved but access to names is disrupted. These different disorders can be conceived as specific deficits at different processing stages in cognitive models, and suggests that these functional stages may have distinct neuroanatomic substrates. It remains to be seen whether a similar anatomic and functional variability is present in developmental prosopagnosia.

An adaptation study of internal and external features in facial representations.

Prior work suggests that internal features contribute more than external features to face processing. Whether this asymmetry is also true of the mental representations of faces is not known. We used face adaptation to determine whether the internal and external features of faces contribute differently to the representation of facial identity, whether this was affected by familiarity, and whether the results differed if the features were presented in isolation or as part of a whole face. In a first experiment, subjects performed a study of identity adaptation for famous and novel faces, in which the adapting stimuli were whole faces, the internal features alone, or the external features alone. In a second experiment, the same faces were used, but the adapting internal and external features were superimposed on whole faces that were ambiguous to identity. The first experiment showed larger aftereffects for unfamiliar faces, and greater aftereffects from internal than from external features, and the latter was true for both familiar and unfamiliar faces. When internal and external features were presented in a whole-face context in the second experiment, aftereffects from either internal or external features was less than that from the whole face, and did not differ from each other. While we reproduce the greater importance of internal features when presented in isolation, we find this is equally true for familiar and unfamiliar faces. The dominant influence of internal features is reduced when integrated into a whole-face context, suggesting another facet of expert face processing.

Erasing the face after-effect.

Perceptual after-effects decay over time at a rate that depends on several factors, such as the duration of adaptation and the duration of the test stimuli. Whether this decay is accelerated by exposure to other faces after adaptation is not known. Our goal was to determine if the appearance of other faces during a delay period after adaptation affected the face identity after-effect. In the first experiment we investigated whether, in the perception of ambiguous stimuli created by morphing between two faces, the repulsive after-effects from adaptation to one face were reduced by brief presentation of the second face in a delay period. We found no effect; however, this may have been confounded by a small attractive after-effect from the interference face. In the second experiment, the interference stimuli were faces unrelated to those used as adaptation stimuli, and we examined after-effects at three different delay periods. This showed a decline in after-effects as the time since adaptation increased, and an enhancement of this decline by the presentation of intervening faces. An exponential model estimated that the intervening faces caused an 85% reduction in the time constant of the after-effect decay. In conclusion, we confirm that face after-effects decline rapidly after adaptation and that exposure to other faces hastens the re-setting of the system.

Neuroanatomic correlates of the feature-salience hierarchy in face processing: an fMRI -adaptation study.

Previous fMRI studies suggest that faces are represented holistically in human face processing regions. On the other hand, behavioral studies have also shown that some facial features are more salient than others for face recognition: the neural basis of this feature-salience hierarchy is not known. We used fMRI-adaptation together with a behavioral discrimination task and an ideal observer analysis to ask (1) whether different face parts contribute different amounts to the neural signal in face responsive regions, and (2) whether this response correlates more with the behavioral performance of human subjects or with the physical properties of the face stimuli. Twenty-three subjects performed a same/different discrimination experiment to characterize their ability to detect changes to different face parts. The same subjects underwent an fMRI-adaptation study, in which limited portions of the faces were repeated or changed between alternating stimuli. The behavioral study showed high efficiency in identity discrimination when the whole face, top half, or eyes changed, and low efficiency when the bottom half, nose, or mouth changed. During fMRI, there was a release of adaptation in the right and left fusiform face area (FFA) with changes to the whole face, top face-half, or the eyes. Changes to the bottom half, nose or mouth did not result in a significant release of adaptation in the right FFA, although bottom-half changes resulted in a release of adaptation in the left FFA. Adaptation in the right and left FFA and the right pSTS was correlated with human perceptual efficiency but not with ideal observer measures of the physical image differences between face parts. The feature-salience hierarchy of human face perception is therefore reflected in the activity in the right and left FFA and right pSTS, further supporting the key role of these structures in our perceptual experience of faces.

Spontaneous perceptual facial distortions correlate with ventral occipitotemporal activity.

Prosopometamorphopsia is a disorder of face perception in which faces appear distorted to the perceiver. The neural basis of prosopometamorphopsia is unclear, but may involve abnormal activity in face-selective areas in the ventral occipito-temporal pathway. Here we present the case of AS, a 44-year-old woman who reports persistent perceptual distortions of faces with no known cause. AS was presented with facial images and rated the magnitude of her distortions while activity in her core face areas and other areas in the ventral visual pathway was measured using functional magnetic resonance imaging. The magnitude of her distortions was positively correlated with signal changes in the right occipital face area (OFA) and right fusiform face area (FFA), as well as right V1-V3, and right lateral occipital cortex (LOC). There was also a trend for a significant correlation with signal in the left OFA and right inferior frontal gyrus (IFG), but not in the right or left superior temporal sulcus (STS). These results suggest that AS' prosopometamorphopsia reflects anomalous activity in face-processing network, particularly in the ventral occipitotemporal cortex.