Neural computations in prosopagnosia.

We report an investigation of the neural processes involved in the processing of faces and objects of brain-lesioned patient PS, a well-documented case of pure acquired prosopagnosia. We gathered a substantial dataset of high-density electrophysiological recordings from both PS and neurotypicals. Using representational similarity analysis, we produced time-resolved brain representations in a format that facilitates direct comparisons across time points, different individuals, and computational models. To understand how the lesions in PS's ventral stream affect the temporal evolution of her brain representations, we computed the temporal generalization of her brain representations. We uncovered that PS's early brain representations exhibit an unusual similarity to later representations, implying an excessive generalization of early visual patterns. To reveal the underlying computational deficits, we correlated PS' brain representations with those of deep neural networks (DNN). We found that the computations underlying PS' brain activity bore a closer resemblance to early layers of a visual DNN than those of controls. However, the brain representations in neurotypicals became more akin to those of the later layers of the model compared to PS. We confirmed PS's deficits in high-level brain representations by demonstrating that her brain representations exhibited less similarity with those of a DNN of semantics.

Decoding face recognition abilities in the human brain.

Why are some individuals better at recognizing faces? Uncovering the neural mechanisms supporting face recognition ability has proven elusive. To tackle this challenge, we used a multimodal data-driven approach combining neuroimaging, computational modeling, and behavioral tests. We recorded the high-density electroencephalographic brain activity of individuals with extraordinary face recognition abilities-super-recognizers-and typical recognizers in response to diverse visual stimuli. Using multivariate pattern analyses, we decoded face recognition abilities from 1 s of brain activity with up to 80% accuracy. To better understand the mechanisms subtending this decoding, we compared representations in the brains of our participants with those in artificial neural network models of vision and semantics, as well as with those involved in human judgments of shape and meaning similarity. Compared to typical recognizers, we found stronger associations between early brain representations of super-recognizers and midlevel representations of vision models as well as shape similarity judgments. Moreover, we found stronger associations between late brain representations of super-recognizers and representations of the artificial semantic model as well as meaning similarity judgments. Overall, these results indicate that important individual variations in brain processing, including neural computations extending beyond purely visual processes, support differences in face recognition abilities. They provide the first empirical evidence for an association between semantic computations and face recognition abilities. We believe that such multimodal data-driven approaches will likely play a critical role in further revealing the complex nature of idiosyncratic face recognition in the human brain.

Re-expression of CA1 and entorhinal activity patterns preserves temporal context memory at long timescales.

Converging, cross-species evidence indicates that memory for time is supported by hippocampal area CA1 and entorhinal cortex. However, limited evidence characterizes how these regions preserve temporal memories over long timescales (e.g., months). At long timescales, memoranda may be encountered in multiple temporal contexts, potentially creating interference. Here, using 7T fMRI, we measured CA1 and entorhinal activity patterns as human participants viewed thousands of natural scene images distributed, and repeated, across many months. We show that memory for an image's original temporal context was predicted by the degree to which CA1/entorhinal activity patterns from the first encounter with an image were re-expressed during re-encounters occurring minutes to months later. Critically, temporal memory signals were dissociable from predictors of recognition confidence, which were carried by distinct medial temporal lobe expressions. These findings suggest that CA1 and entorhinal cortex preserve temporal memories across long timescales by coding for and reinstating temporal context information.

Disentangling five dimensions of animacy in human brain and behaviour.

Distinguishing animate from inanimate things is of great behavioural importance. Despite distinct brain and behavioural responses to animate and inanimate things, it remains unclear which object properties drive these responses. Here, we investigate the importance of five object dimensions related to animacy ("being alive", "looking like an animal", "having agency", "having mobility", and "being unpredictable") in brain (fMRI, EEG) and behaviour (property and similarity judgements) of 19 participants. We used a stimulus set of 128 images, optimized by a genetic algorithm to disentangle these five dimensions. The five dimensions explained much variance in the similarity judgments. Each dimension explained significant variance in the brain representations (except, surprisingly, "being alive"), however, to a lesser extent than in behaviour. Different brain regions sensitive to animacy may represent distinct dimensions, either as accessible perceptual stepping stones toward detecting whether something is alive or because they are of behavioural importance in their own right.

Improving the accuracy of single-trial fMRI response estimates using GLMsingle.

Advances in artificial intelligence have inspired a paradigm shift in human neuroscience, yielding large-scale functional magnetic resonance imaging (fMRI) datasets that provide high-resolution brain responses to thousands of naturalistic visual stimuli. Because such experiments necessarily involve brief stimulus durations and few repetitions of each stimulus, achieving sufficient signal-to-noise ratio can be a major challenge. We address this challenge by introducing GLMsingle, a scalable, user-friendly toolbox available in MATLAB and Python that enables accurate estimation of single-trial fMRI responses (glmsingle.org). Requiring only fMRI time-series data and a design matrix as inputs, GLMsingle integrates three techniques for improving the accuracy of trial-wise general linear model (GLM) beta estimates. First, for each voxel, a custom hemodynamic response function (HRF) is identified from a library of candidate functions. Second, cross-validation is used to derive a set of noise regressors from voxels unrelated to the experiment. Third, to improve the stability of beta estimates for closely spaced trials, betas are regularized on a voxel-wise basis using ridge regression. Applying GLMsingle to the Natural Scenes Dataset and BOLD5000, we find that GLMsingle substantially improves the reliability of beta estimates across visually-responsive cortex in all subjects. Comparable improvements in reliability are also observed in a smaller-scale auditory dataset from the StudyForrest experiment. These improvements translate into tangible benefits for higher-level analyses relevant to systems and cognitive neuroscience. We demonstrate that GLMsingle: (i) helps decorrelate response estimates between trials nearby in time; (ii) enhances representational similarity between subjects within and across datasets; and (iii) boosts one-versus-many decoding of visual stimuli. GLMsingle is a publicly available tool that can significantly improve the quality of past, present, and future neuroimaging datasets sampling brain activity across many experimental conditions.

Researcher perspectives on ethics considerations in epigenetics: an international survey.

Over the past decade, bioethicists, legal scholars and social scientists have started to investigate the potential implications of epigenetic research and technologies on medicine and society. There is growing literature discussing the most promising opportunities, as well as arising ethical, legal and social issues (ELSI). This paper explores the views of epigenetic researchers about some of these discussions. From January to March 2020, we conducted an online survey of 189 epigenetic researchers working in 31 countries. We questioned them about the scope of their field, opportunities in different areas of specialization, and ELSI in the conduct of research and knowledge translation. We also assessed their level of concern regarding four emerging non-medical applications of epigenetic testing-i.e., in life insurance, forensics, immigration and direct-to-consumer testing. Although there was strong agreement on DNA methylation, histone modifications, 3D structure of chromatin and nucleosomes being integral elements of the field, there was considerable disagreement on transcription factors, RNA interference, RNA splicing and prions. The most prevalent ELSI experienced or witnessed by respondents were in obtaining timely access to epigenetic data in existing databases, and in the communication of epigenetic findings by the media. They expressed high levels of concern regarding non-medical applications of epigenetics, echoing cautionary appraisals in the social sciences and humanities literature.

Sleep spindles track cortical learning patterns for memory consolidation.

Memory consolidation-the transformation of labile memory traces into stable long-term representations-is facilitated by post-learning sleep. Computational and biophysical models suggest that sleep spindles may play a key mechanistic role for consolidation, igniting structural changes at cortical sites involved in prior learning. Here, we tested the resulting prediction that spindles are most pronounced over learning-related cortical areas and that the extent of this learning-spindle overlap predicts behavioral measures of memory consolidation. Using high-density scalp electroencephalography (EEG) and polysomnography (PSG) in healthy volunteers, we first identified cortical areas engaged during a temporospatial associative memory task (power decreases in the alpha/beta frequency range, 6-20 Hz). Critically, we found that participant-specific topographies (i.e., spatial distributions) of post-learning sleep spindle amplitude correlated with participant-specific learning topographies. Importantly, the extent to which spindles tracked learning patterns further predicted memory consolidation across participants. Our results provide empirical evidence for a role of post-learning sleep spindles in tracking learning networks, thereby facilitating memory consolidation.

A massive 7T fMRI dataset to bridge cognitive neuroscience and artificial intelligence.

Extensive sampling of neural activity during rich cognitive phenomena is critical for robust understanding of brain function. Here we present the Natural Scenes Dataset (NSD), in which high-resolution functional magnetic resonance imaging responses to tens of thousands of richly annotated natural scenes were measured while participants performed a continuous recognition task. To optimize data quality, we developed and applied novel estimation and denoising techniques. Simple visual inspections of the NSD data reveal clear representational transformations along the ventral visual pathway. Further exemplifying the inferential power of the dataset, we used NSD to build and train deep neural network models that predict brain activity more accurately than state-of-the-art models from computer vision. NSD also includes substantial resting-state and diffusion data, enabling network neuroscience perspectives to constrain and enhance models of perception and memory. Given its unprecedented scale, quality and breadth, NSD opens new avenues of inquiry in cognitive neuroscience and artificial intelligence.

Diagnostic Features for Human Categorisation of Adult and Child Faces.

Knowing how humans differentiate children from adults has useful implications in many areas of both forensic and cognitive psychology. Yet, how we extract age from faces has been surprisingly underexplored in both disciplines. Here, we used a novel data-driven experimental technique to objectively measure the facial features human observers use to categorise child and adult faces. Relying on more than 35,000 trials, we used a reverse correlation technique that enabled us to reveal how specific features which are known to be important in face-perception - position, spatial-frequency (SF), and orientation - are associated with accurate child and adult discrimination. This showed that human observers relied on evidence in the nasal bone and eyebrow area for accurate adult categorisation, while they relied on the eye and jawline area to accurately categorise child faces. For orientation structure, only facial information of vertical orientation was linked to face-adult categorisation, while features of horizontal and, to a lesser extent oblique orientations, were more diagnostic of a child face. Finally, we found that SF diagnosticity showed a U-shaped pattern for face-age categorisation, with information in low and high SFs being diagnostic of child faces, and mid SFs being diagnostic of adult faces. Through this first characterisation of the facial features of face-age categorisation, we show that important information found in psychophysical studies of face-perception in general (i.e., the eye area, horizontals, and mid-level SFs) is crucial to the practical context of face-age categorisation, and present data-driven procedures through which face-age classification training could be implemented for real-world challenges.

The hippocampus as the switchboard between perception and memory.

Adaptive memory recall requires a rapid and flexible switch from external perceptual reminders to internal mnemonic representations. However, owing to the limited temporal or spatial resolution of brain imaging modalities used in isolation, the hippocampal-cortical dynamics supporting this process remain unknown. We thus employed an object-scene cued recall paradigm across two studies, including intracranial electroencephalography (iEEG) and high-density scalp EEG. First, a sustained increase in hippocampal high gamma power (55 to 110 Hz) emerged 500 ms after cue onset and distinguished successful vs. unsuccessful recall. This increase in gamma power for successful recall was followed by a decrease in hippocampal alpha power (8 to 12 Hz). Intriguingly, the hippocampal gamma power increase marked the moment at which extrahippocampal activation patterns shifted from perceptual cue toward mnemonic target representations. In parallel, source-localized EEG alpha power revealed that the recall signal progresses from hippocampus to posterior parietal cortex and then to medial prefrontal cortex. Together, these results identify the hippocampus as the switchboard between perception and memory and elucidate the ensuing hippocampal-cortical dynamics supporting the recall process.

Does sleep-dependent consolidation favour weak memories?

Sleep stabilizes newly acquired memories, a process referred to as memory consolidation. According to recent studies, sleep-dependent consolidation processes might be deployed to different extents for different types of memories. In particular, weaker memories might benefit greater from post-learning sleep than stronger memories. However, under standard testing conditions, sleep-dependent consolidation effects for stronger memories might be obscured by ceiling effects. To test this possibility, we devised a new memory paradigm (Memory Arena) in which participants learned temporospatial arrangements of objects. Prior to a delay period spent either awake or asleep, training thresholds were controlled to yield relatively weak or relatively strong memories. After the delay period, retrieval difficulty was controlled via the presence or absence of a retroactive interference task. Under standard testing conditions (no interference), a sleep-dependent consolidation effect was indeed observed for weaker memories only. Critically though, with increased retrieval demands, sleep-dependent consolidation effects were seen for both weaker and stronger memories. These results suggest that all memories are consolidated during sleep, but that memories of different strengths require different testing conditions to unveil their benefit from post-learning sleep.

Cochlea to categories: The spatiotemporal dynamics of semantic auditory representations.

How does the auditory system categorize natural sounds? Here we apply multimodal neuroimaging to illustrate the progression from acoustic to semantically dominated representations. Combining magnetoencephalographic (MEG) and functional magnetic resonance imaging (fMRI) scans of observers listening to naturalistic sounds, we found superior temporal responses beginning ∼55 ms post-stimulus onset, spreading to extratemporal cortices by ∼100 ms. Early regions were distinguished less by onset/peak latency than by functional properties and overall temporal response profiles. Early acoustically-dominated representations trended systematically toward category dominance over time (after ∼200 ms) and space (beyond primary cortex). Semantic category representation was spatially specific: Vocalizations were preferentially distinguished in frontotemporal voice-selective regions and the fusiform; scenes and objects were distinguished in parahippocampal and medial place areas. Our results are consistent with real-world events coded via an extended auditory processing hierarchy, in which acoustic representations rapidly enter multiple streams specialized by category, including areas typically considered visual cortex.

Recurrent neural networks can explain flexible trading of speed and accuracy in biological vision.

Deep feedforward neural network models of vision dominate in both computational neuroscience and engineering. The primate visual system, by contrast, contains abundant recurrent connections. Recurrent signal flow enables recycling of limited computational resources over time, and so might boost the performance of a physically finite brain or model. Here we show: (1) Recurrent convolutional neural network models outperform feedforward convolutional models matched in their number of parameters in large-scale visual recognition tasks on natural images. (2) Setting a confidence threshold, at which recurrent computations terminate and a decision is made, enables flexible trading of speed for accuracy. At a given confidence threshold, the model expends more time and energy on images that are harder to recognise, without requiring additional parameters for deeper computations. (3) The recurrent model's reaction time for an image predicts the human reaction time for the same image better than several parameter-matched and state-of-the-art feedforward models. (4) Across confidence thresholds, the recurrent model emulates the behaviour of feedforward control models in that it achieves the same accuracy at approximately the same computational cost (mean number of floating-point operations). However, the recurrent model can be run longer (higher confidence threshold) and then outperforms parameter-matched feedforward comparison models. These results suggest that recurrent connectivity, a hallmark of biological visual systems, may be essential for understanding the accuracy, flexibility, and dynamics of human visual recognition.

Clinically relevant autistic traits predict greater reliance on detail for image recognition.

Individuals with an autism spectrum disorder (ASD) diagnosis are often described as having an eye for detail. But it remains to be shown that a detail-focused processing bias is a ubiquitous property of vision in individuals with ASD. To address this question, we investigated whether a greater number of autistic traits in neurotypical subjects is associated with an increased reliance on image details during a natural image recognition task. To this end, we use a novel reverse correlation-based method (feature diagnosticity mapping) for measuring the relative importance of low-level image features for object recognition. The main finding of this study is that image recognition in participants with an above-median number of autistic traits benefited more from the presence of high-spatial frequency image features. Furthermore, we found that this reliance-on-detail effect was best predicted by the presence of the most clinically relevant autistic traits. Therefore, our findings suggest that a greater number of autistic traits in neurotypical individuals is associated with a more detail-oriented visual information processing strategy and that this effect might generalize to a clinical ASD population.

Alpha/beta power decreases track the fidelity of stimulus-specific information.

Massed synchronised neuronal firing is detrimental to information processing. When networks of task-irrelevant neurons fire in unison, they mask the signal generated by task-critical neurons. On a macroscopic level, such synchronisation can contribute to alpha/beta (8-30 Hz) oscillations. Reducing the amplitude of these oscillations, therefore, may enhance information processing. Here, we test this hypothesis. Twenty-one participants completed an associative memory task while undergoing simultaneous EEG-fMRI recordings. Using representational similarity analysis, we quantified the amount of stimulus-specific information represented within the BOLD signal on every trial. When correlating this metric with concurrently-recorded alpha/beta power, we found a significant negative correlation which indicated that as post-stimulus alpha/beta power decreased, stimulus-specific information increased. Critically, we found this effect in three unique tasks: visual perception, auditory perception, and visual memory retrieval, indicating that this phenomenon transcends both stimulus modality and cognitive task. These results indicate that alpha/beta power decreases parametrically track the fidelity of both externally-presented and internally-generated stimulus-specific information represented within the cortex.

Conscious perception of natural images is constrained by category-related visual features.

Conscious perception is crucial for adaptive behaviour yet access to consciousness varies for different types of objects. The visual system comprises regions with widely distributed category information and exemplar-level representations that cluster according to category. Does this categorical organisation in the brain provide insight into object-specific access to consciousness? We address this question using the Attentional Blink approach with visual objects as targets. We find large differences across categories in the attentional blink. We then employ activation patterns extracted from a deep convolutional neural network to reveal that these differences depend on mid- to high-level, rather than low-level, visual features. We further show that these visual features can be used to explain variance in performance across trials. Taken together, our results suggest that the specific organisation of the higher-tier visual system underlies important functions relevant for conscious perception of differing natural images.

The spatiotemporal neural dynamics underlying perceived similarity for real-world objects.

The degree to which we perceive real-world objects as similar or dissimilar structures our perception and guides categorization behavior. Here, we investigated the neural representations enabling perceived similarity using behavioral judgments, fMRI and MEG. As different object dimensions co-occur and partly correlate, to understand the relationship between perceived similarity and brain activity it is necessary to assess the unique role of multiple object dimensions. We thus behaviorally assessed perceived object similarity in relation to shape, function, color and background. We then used representational similarity analyses to relate these behavioral judgments to brain activity. We observed a link between each object dimension and representations in visual cortex. These representations emerged rapidly within 200 ms of stimulus onset. Assessing the unique role of each object dimension revealed partly overlapping and distributed representations: while color-related representations distinctly preceded shape-related representations both in the processing hierarchy of the ventral visual pathway and in time, several dimensions were linked to high-level ventral visual cortex. Further analysis singled out the shape dimension as neither fully accounted for by supra-category membership, nor a deep neural network trained on object categorization. Together our results comprehensively characterize the relationship between perceived similarity of key object dimensions and neural activity.

GLMdenoise improves multivariate pattern analysis of fMRI data.

GLMdenoise is a denoising technique for task-based fMRI. In GLMdenoise, estimates of spatially correlated noise (which may be physiological, instrumental, motion-related, or neural in origin) are derived from the data and incorporated as nuisance regressors in a general linear model (GLM) analysis. We previously showed that GLMdenoise outperforms a variety of other denoising techniques in terms of cross-validation accuracy of GLM estimates (Kay et al., 2013a). However, the practical impact of denoising for experimental studies remains unclear. Here we examine whether and to what extent GLMdenoise improves sensitivity in the context of multivariate pattern analysis of fMRI data. On a large number of participants (31 participants across 4 experiments; 3 T, gradient-echo, spatial resolution 2-3.75 mm, temporal resolution 1.3-2 s, number of conditions 32-75), we perform representational similarity analysis (Kriegeskorte et al., 2008a) as well as pattern classification (Haxby et al., 2001). We find that GLMdenoise substantially improves replicability of representational dissimilarity matrices (RDMs) across independent splits of each participant's dataset (average RDM replicability increases from r = 0.46 to r = 0.61). Additionally, we find that GLMdenoise substantially improves pairwise classification accuracy (average classification accuracy increases from 79% correct to 84% correct). We show that GLMdenoise often improves and never degrades performance for individual participants and that GLMdenoise also improves across-participant consistency. We conclude that GLMdenoise is a useful tool that can be routinely used to maximize the amount of information extracted from fMRI activity patterns.

Author Correction: Retrieval induces adaptive forgetting of competing memories via cortical pattern suppression.

In the published version of this article, a detail is missing from the Methods section "Experimental procedure." The following sentence is to be inserted at the end of its fourth paragraph: "If participants failed to respond within 3.5 s, we assumed that they were unable to successfully recognize the item and coded the corresponding trial as an error." The critical behavioral forgetting effect is significant irrespective of whether these timeouts are coded as errors (t23 = 4.91, P < 0.001) or as missing data (t23 = 3.31, P < 0.01). The original article has not been corrected.

The human voice areas: Spatial organization and inter-individual variability in temporal and extra-temporal cortices.

fMRI studies increasingly examine functions and properties of non-primary areas of human auditory cortex. However there is currently no standardized localization procedure to reliably identify specific areas across individuals such as the standard 'localizers' available in the visual domain. Here we present an fMRI 'voice localizer' scan allowing rapid and reliable localization of the voice-sensitive 'temporal voice areas' (TVA) of human auditory cortex. We describe results obtained using this standardized localizer scan in a large cohort of normal adult subjects. Most participants (94%) showed bilateral patches of significantly greater response to vocal than non-vocal sounds along the superior temporal sulcus/gyrus (STS/STG). Individual activation patterns, although reproducible, showed high inter-individual variability in precise anatomical location. Cluster analysis of individual peaks from the large cohort highlighted three bilateral clusters of voice-sensitivity, or "voice patches" along posterior (TVAp), mid (TVAm) and anterior (TVAa) STS/STG, respectively. A series of extra-temporal areas including bilateral inferior prefrontal cortex and amygdalae showed small, but reliable voice-sensitivity as part of a large-scale cerebral voice network. Stimuli for the voice localizer scan and probabilistic maps in MNI space are available for download.