A connectivity-constrained computational account of topographic organization in primate high-level visual cortex.

Inferotemporal (IT) cortex in humans and other primates is topographically organized, containing multiple hierarchically organized areas selective for particular domains, such as faces and scenes. This organization is commonly viewed in terms of evolved domain-specific visual mechanisms. Here, we develop an alternative, domain-general and developmental account of IT cortical organization. The account is instantiated in interactive topographic networks (ITNs), a class of computational models in which a hierarchy of model IT areas, subject to biologically plausible connectivity-based constraints, learns high-level visual representations optimized for multiple domains. We find that minimizing a wiring cost on spatially organized feedforward and lateral connections, alongside realistic constraints on the sign of neuronal connectivity within model IT, results in a hierarchical, topographic organization. This organization replicates a number of key properties of primate IT cortex, including the presence of domain-selective spatial clusters preferentially involved in the representation of faces, objects, and scenes; columnar responses across separate excitatory and inhibitory units; and generic spatial organization whereby the response correlation of pairs of units falls off with their distance. We thus argue that topographic domain selectivity is an emergent property of a visual system optimized to maximize behavioral performance under generic connectivity-based constraints.

Spatiotemporal dynamics of similarity-based neural representations of facial identity.

Humans' remarkable ability to quickly and accurately discriminate among thousands of highly similar complex objects demands rapid and precise neural computations. To elucidate the process by which this is achieved, we used magnetoencephalography to measure spatiotemporal patterns of neural activity with high temporal resolution during visual discrimination among a large and carefully controlled set of faces. We also compared these neural data to lower level "image-based" and higher level "identity-based" model-based representations of our stimuli and to behavioral similarity judgments of our stimuli. Between ∼50 and 400 ms after stimulus onset, face-selective sources in right lateral occipital cortex and right fusiform gyrus and sources in a control region (left V1) yielded successful classification of facial identity. In all regions, early responses were more similar to the image-based representation than to the identity-based representation. In the face-selective regions only, responses were more similar to the identity-based representation at several time points after 200 ms. Behavioral responses were more similar to the identity-based representation than to the image-based representation, and their structure was predicted by responses in the face-selective regions. These results provide a temporally precise description of the transformation from low- to high-level representations of facial identity in human face-selective cortex and demonstrate that face-selective cortical regions represent multiple distinct types of information about face identity at different times over the first 500 ms after stimulus onset. These results have important implications for understanding the rapid emergence of fine-grained, high-level representations of object identity, a computation essential to human visual expertise.

Protracted Developmental Trajectory of Shape Processing along the Two Visual Pathways.

Studies of the emergence of shape representations in childhood have focused primarily on the ventral visual pathway. Importantly, however, there is increasing evidence that, in adults, the dorsal pathway also represents shape-based information. These dorsal representations follow a gradient with more posterior regions being more shape-sensitive than anterior regions and with representational similarity in some posterior regions that is equivalent to that observed in some ventral regions. To explore the emergence and nature of dorsal shape representations in development, we acquired both fMRI BOLD signals and behavioral data in children (aged 8-10 years) using a parametric image scrambling paradigm. Children exhibited adult-like large-scale organization of shape processing along both ventral and dorsal pathways. Also, as in adults, the activation profiles of children's posterior dorsal and ventral regions were correlated with recognition performance, reflecting a possible contribution of these signals to perception. There were age-related changes, however, with children being more affected by the distortion of shape information than adults, both behaviorally and neurally. These findings reveal that shape-processing mechanisms along both dorsal and ventral pathways are subject to a protracted developmental trajectory.

Feature-based face representations and image reconstruction from behavioral and neural data.

The reconstruction of images from neural data can provide a unique window into the content of human perceptual representations. Although recent efforts have established the viability of this enterprise using functional magnetic resonance imaging (MRI) patterns, these efforts have relied on a variety of prespecified image features. Here, we take on the twofold task of deriving features directly from empirical data and of using these features for facial image reconstruction. First, we use a method akin to reverse correlation to derive visual features from functional MRI patterns elicited by a large set of homogeneous face exemplars. Then, we combine these features to reconstruct novel face images from the corresponding neural patterns. This approach allows us to estimate collections of features associated with different cortical areas as well as to successfully match image reconstructions to corresponding face exemplars. Furthermore, we establish the robustness and the utility of this approach by reconstructing images from patterns of behavioral data. From a theoretical perspective, the current results provide key insights into the nature of high-level visual representations, and from a practical perspective, these findings make possible a broad range of image-reconstruction applications via a straightforward methodological approach.

Relative meaning frequencies for 578 homonyms in two Spanish dialects: A cross-linguistic extension of the English eDom norms.

Relative meaning frequency is a critical factor to consider in studies of semantic ambiguity. In this work, we examined how this measure may change across the European and Rioplatense dialects of Spanish, as well as how the overall distributional properties differ between Spanish and English, using a computer-assisted norming approach based on dictionary definitions (Armstrong, Tokowicz, & Plaut, 2012). The results showed that the two dialects differ considerably in terms of the relative meaning frequencies of their constituent homonyms, and that the overall distributions of relative frequencies vary considerably across languages, as well. These results highlight the need for localized norms to design powerful studies of semantic ambiguity and suggest that dialectal differences may be responsible for some discrepant effects related to homonymy. In quantifying the reliability of the norms, we also established that as few as seven ratings are needed to converge on a highly stable set of ratings. This approach is therefore a very practical means of acquiring essential data in studies of semantic ambiguity, relative to past approaches, such as those based on the classification of free associates. The norms also present new possibilities for studying semantic ambiguity effects within and between populations who speak one or more languages. The norms and associated software are available for download at http://edom.cnbc.cmu.edu/ or http://www.bcbl.eu/databases/edom/ .

Variable left-hemisphere language and orthographic lateralization reduces right-hemisphere face lateralization.

It is commonly believed that, in right-handed individuals, words and faces are processed by distinct neural systems: one in the left hemisphere (LH) for words and the other in the right hemisphere (RH) for faces. Emerging evidence suggests, however, that hemispheric selectivity for words and for faces may not be independent of each other. One recent account suggests that words become lateralized to the LH to interact more effectively with language regions, and subsequently, as a result of competition with words for representational space, faces become lateralized to the RH. On this interactive account, left-handed individuals, who as a group show greater variability with respect to hemispheric language dominance, might be expected to show greater variability in their degree of RH lateralization of faces as well. The current study uses behavioral measures and ERPs to compare the hemispheric specialization for both words and faces in right- and left-handed adult individuals. Although both right- and left-handed groups demonstrated LH over RH superiority in discrimination accuracy for words, only the right-handed group demonstrated RH over LH advantage in discrimination accuracy for faces. Consistent with this, increased right-handedness was related to an increase in RH superiority for face processing, as measured by the strength of the N170 ERP component. Interestingly, the degree of RH behavioral superiority for face processing and the amplitude of the RH N170 for faces could be predicted by the magnitude of the N170 ERP response to words in the LH. These results are discussed in terms of a theoretical account in which the typical RH face lateralization fails to emerge in individuals with atypical language lateralization because of weakened competition from the LH representation of words.

Bilateral hemispheric processing of words and faces: evidence from word impairments in prosopagnosia and face impairments in pure alexia.

Considerable research has supported the view that faces and words are subserved by independent neural mechanisms located in the ventral visual cortex in opposite hemispheres. On this view, right hemisphere ventral lesions that impair face recognition (prosopagnosia) should leave word recognition unaffected, and left hemisphere ventral lesions that impair word recognition (pure alexia) should leave face recognition unaffected. The current study shows that neither of these predictions was upheld. A series of experiments characterizing speed and accuracy of word and face recognition were conducted in 7 patients (4 pure alexic, 3 prosopagnosic) and matched controls. Prosopagnosic patients revealed mild but reliable word recognition deficits, and pure alexic patients demonstrated mild but reliable face recognition deficits. The apparent comingling of face and word mechanisms is unexpected from a domain-specific perspective, but follows naturally as a consequence of an interactive, learning-based account in which neural processes for both faces and words are the result of an optimization procedure embodying specific computational principles and constraints.

The neural basis of visual word form processing: a multivariate investigation.

Current research on the neurobiological bases of reading points to the privileged role of a ventral cortical network in visual word processing. However, the properties of this network and, in particular, its selectivity for orthographic stimuli such as words and pseudowords remain topics of significant debate. Here, we approached this issue from a novel perspective by applying pattern-based analyses to functional magnetic resonance imaging data. Specifically, we examined whether, where and how, orthographic stimuli elicit distinct patterns of activation in the human cortex. First, at the category level, multivariate mapping found extensive sensitivity throughout the ventral cortex for words relative to false-font strings. Secondly, at the identity level, the multi-voxel pattern classification provided direct evidence that different pseudowords are encoded by distinct neural patterns. Thirdly, a comparison of pseudoword and face identification revealed that both stimulus types exploit common neural resources within the ventral cortical network. These results provide novel evidence regarding the involvement of the left ventral cortex in orthographic stimulus processing and shed light on its selectivity and discriminability profile. In particular, our findings support the existence of sublexical orthographic representations within the left ventral cortex while arguing for the continuity of reading with other visual recognition skills.

Unraveling the distributed neural code of facial identity through spatiotemporal pattern analysis.

Face individuation is one of the most impressive achievements of our visual system, and yet uncovering the neural mechanisms subserving this feat appears to elude traditional approaches to functional brain data analysis. The present study investigates the neural code of facial identity perception with the aim of ascertaining its distributed nature and informational basis. To this end, we use a sequence of multivariate pattern analyses applied to functional magnetic resonance imaging (fMRI) data. First, we combine information-based brain mapping and dynamic discrimination analysis to locate spatiotemporal patterns that support face classification at the individual level. This analysis reveals a network of fusiform and anterior temporal areas that carry information about facial identity and provides evidence that the fusiform face area responds with distinct patterns of activation to different face identities. Second, we assess the information structure of the network using recursive feature elimination. We find that diagnostic information is distributed evenly among anterior regions of the mapped network and that a right anterior region of the fusiform gyrus plays a central role within the information network mediating face individuation. These findings serve to map out and characterize a cortical system responsible for individuation. More generally, in the context of functionally defined networks, they provide an account of distributed processing grounded in information-based architectures.

Visual word processing engages a hierarchical, distributed, and bilateral cortical network.

Although the Visual Word Form Area (VWFA) in left temporal cortex is considered the pre-eminent region in visual word processing, other regions are also implicated. We examined the entire text-selective circuit, using functional MRI. Ten regions of interest (ROIs) per hemisphere were defined, which, based on clustering, grouped into early vision, high-level vision, and language clusters. We analyzed the responses of the ROIs and clusters to words, inverted words, and consonant strings using univariate, multivariate, and functional connectivity measures. Bilateral modulation by stimulus condition was evident, with a stronger effect in left hemisphere regions. Last, using graph theory, we observed that the VWFA was equivalently connected with early visual and language clusters in both hemispheres, reflecting its role as a mediator in the circuit. Although the individual ROIs and clusters bilaterally were flexibly altered by the nature of the input, stability held at the level of global circuit connectivity, reflecting the complex hierarchical distributed system serving visual text perception.

Face-space architectures: evidence for the use of independent color-based features.

The concept of psychological face space lies at the core of many theories of face recognition and representation. To date, much of the understanding of face space has been based on principal component analysis (PCA); the structure of the psychological space is thought to reflect some important aspects of a physical face space characterized by PCA applications to face images. In the present experiments, we investigated alternative accounts of face space and found that independent component analysis provided the best fit to human judgments of face similarity and identification. Thus, our results challenge an influential approach to the study of human face space and provide evidence for the role of statistically independent features in face encoding. In addition, our findings support the use of color information in the representation of facial identity, and we thus argue for the inclusion of such information in theoretical and computational constructs of face space.

Giving theories of reading a sporting chance.

The search for a universal theory of reading is misguided. Instead, theories should articulate general principles of neural computation that interact with language-specific learning environments to explain the full diversity of observed reading-related phenomena across the world's languages.

SOS! An algorithm and software for the stochastic optimization of stimuli.

The characteristics of the stimuli used in an experiment critically determine the theoretical questions the experiment can address. Yet there is relatively little methodological support for selecting optimal sets of items, and most researchers still carry out this process by hand. In this research, we present SOS, an algorithm and software package for the stochastic optimization of stimuli. SOS takes its inspiration from a simple manual stimulus selection heuristic that has been formalized and refined as a stochastic relaxation search. The algorithm rapidly and reliably selects a subset of possible stimuli that optimally satisfy the constraints imposed by an experimenter. This allows the experimenter to focus on selecting an optimization problem that suits his or her theoretical question and to avoid the tedious task of manually selecting stimuli. We detail how this optimization algorithm, combined with a vocabulary of constraints that define optimal sets, allows for the quick and rigorous assessment and maximization of the internal and external validity of experimental items. In doing so, the algorithm facilitates research using factorial, multiple/mixed-effects regression, and other experimental designs. We demonstrate the use of SOS with a case study and discuss other research situations that could benefit from this tool. Support for the generality of the algorithm is demonstrated through Monte Carlo simulations on a range of optimization problems faced by psychologists. The software implementation of SOS and a user manual are provided free of charge for academic purposes as precompiled binaries and MATLAB source files at http://sos.cnbc.cmu.edu.

eDom: norming software and relative meaning frequencies for 544 English homonyms.

Words that are homonyms-that is, for which a single written and spoken form is associated with multiple, unrelated interpretations, such as COMPOUND, which can denote an < enclosure > or a < composite > meaning-are an invaluable class of items for studying word and discourse comprehension. When using homonyms as stimuli, it is critical to control for the relative frequencies of each interpretation, because this variable can drastically alter the empirical effects of homonymy. Currently, the standard method for estimating these frequencies is based on the classification of free associates generated for a homonym, but this approach is both assumption-laden and resource-demanding. Here, we outline an alternative norming methodology based on explicit ratings of the relative meaning frequencies of dictionary definitions. To evaluate this method, we collected and analyzed data in a norming study involving 544 English homonyms, using the eDom norming software that we developed for this purpose. Dictionary definitions were generally sufficient to exhaustively cover word meanings, and the methods converged on stable norms with fewer data and less effort on the part of the experimenter. The predictive validity of the norms was demonstrated in analyses of lexical decision data from the English Lexicon Project (Balota et al., Behavior Research Methods, 39, 445-459, 2007), and from Armstrong and Plaut (Proceedings of the 33rd Annual Meeting of the Cognitive Science Society, 2223-2228, 2011). On the basis of these results, our norming method obviates relying on the unsubstantiated assumptions involved in estimating relative meaning frequencies on the basis of classification of free associates. Additional details of the norming procedure, the meaning frequency norms, and the source code, standalone binaries, and user manual for the software are available at http://edom.cnbc.cmu.edu .

From decomposition to distributed theories of morphological processing in reading.

The morphological structure of complex words impacts how they are processed during visual word recognition. This impact varies over the course of reading acquisition and for different languages and writing systems. Many theories of morphological processing rely on a decomposition mechanism, in which words are decomposed into explicit representations of their constituent morphemes. In distributed accounts, in contrast, morphological sensitivity arises from the tuning of finer-grained representations to useful statistical regularities in the form-to-meaning mapping, without the need for explicit morpheme representations. In this theoretically guided review, we summarize research into the mechanisms of morphological processing, and discuss findings within the context of decomposition and distributed accounts. Although many findings fit within a decomposition model of morphological processing, we suggest that the full range of results is more naturally explained by a distributed approach, and discuss additional benefits of adopting this perspective.

A neural network model of the effect of prior experience with regularities on subsequent category learning.

Categories are often structured by the similarities of instances within the category defined across dimensions or features. Researchers typically assume that there is a direct, linear relationship between the physical input dimensions across which category exemplars are defined and the psychological representation of these dimensions. However, this assumption is not always warranted. Through a set of simulations, we demonstrate that the psychological representations of input dimensions developed through long-term prior experience can place very strong constraints on category learning. We compare the model's behavior to auditory, visual, and cross-modal human category learning and make conclusions regarding the nature of the psychological representations of the dimensions in those studies. These simulations support the conclusion that the nature of psychological representations of input dimensions is a critical aspect to understanding the mechanisms underlying category learning.

Complementary neural representations for faces and words: a computational exploration.

A key issue that continues to generate controversy concerns the nature of the psychological, computational, and neural mechanisms that support the visual recognition of objects such as faces and words. While some researchers claim that visual recognition is accomplished by category-specific modules dedicated to processing distinct object classes, other researchers have argued for a more distributed system with only partially specialized cortical regions. Considerable evidence from both functional neuroimaging and neuropsychology would seem to favour the modular view, and yet close examination of those data reveals rather graded patterns of specialization that support a more distributed account. This paper explores a theoretical middle ground in which the functional specialization of brain regions arises from general principles and constraints on neural representation and learning that operate throughout cortex but that nonetheless have distinct implications for different classes of stimuli. The account is supported by a computational simulation, in the form of an artificial neural network, that illustrates how cooperative and competitive interactions in the formation of neural representations for faces and words account for both their shared and distinctive properties. We set out a series of empirical predictions, which are also examined, and consider the further implications of this account.

Finite case series or infinite single-case studies? Comments on "Case series investigations in cognitive neuropsychology" by Schwartz and Dell (2010).

In this commentary, though acknowledging that a case-series approach in neuropsychology is not always possible, we set out a series of considerations that in our view make this approach generally superior to single-case study. We argue that case-series designs are crucial for theory-testing, assessment of computational models, evaluation of inter-patient variation (including selection criteria, patient homogeneity/heterogeneity, premorbid individual differences, etc.) and to establish solid foundations for the interpretation of behavioural dissociations and associations. We conclude by suggesting that, alongside other neuroscience techniques, case-series cognitive neuropsychology provides a crucial contribution to the future of clinical and cognitive neuroscience.

Deep learning of shared perceptual representations for familiar and unfamiliar faces: Reply to commentaries.

We recently argued that human unfamiliar face identity perception reflects substantial perceptual expertise, and that the advantage for familiar over unfamiliar face identity matching reflects a learned mapping between generic high-level perceptual features and a unique identity representation of each individual (Blauch, Behrmann and Plaut, 2020). Here we respond to two commentaries by Young and Burton (2020) and Yovel and Abudarham (2020), clarifying and elaborating our stance on various theoretical issues, and discussing topics for future research in human face recognition and the learning of perceptual representations.