Specialized Networks for Social Cognition in the Primate Brain.

Primates have evolved diverse cognitive capabilities to navigate their complex social world. To understand how the brain implements critical social cognitive abilities, we describe functional specialization in the domains of face processing, social interaction understanding, and mental state attribution. Systems for face processing are specialized from the level of single cells to populations of neurons within brain regions to hierarchically organized networks that extract and represent abstract social information. Such functional specialization is not confined to the sensorimotor periphery but appears to be a pervasive theme of primate brain organization all the way to the apex regions of cortical hierarchies. Circuits processing social information are juxtaposed with parallel systems involved in processing nonsocial information, suggesting common computations applied to different domains. The emerging picture of the neural basis of social cognition is a set of distinct but interacting subnetworks involved in component processes such as face perception and social reasoning, traversing large parts of the primate brain.

Advances in human intracranial electroencephalography research, guidelines and good practices.

Since the second-half of the twentieth century, intracranial electroencephalography (iEEG), including both electrocorticography (ECoG) and stereo-electroencephalography (sEEG), has provided an intimate view into the human brain. At the interface between fundamental research and the clinic, iEEG provides both high temporal resolution and high spatial specificity but comes with constraints, such as the individual's tailored sparsity of electrode sampling. Over the years, researchers in neuroscience developed their practices to make the most of the iEEG approach. Here we offer a critical review of iEEG research practices in a didactic framework for newcomers, as well addressing issues encountered by proficient researchers. The scope is threefold: (i) review common practices in iEEG research, (ii) suggest potential guidelines for working with iEEG data and answer frequently asked questions based on the most widespread practices, and (iii) based on current neurophysiological knowledge and methodologies, pave the way to good practice standards in iEEG research. The organization of this paper follows the steps of iEEG data processing. The first section contextualizes iEEG data collection. The second section focuses on localization of intracranial electrodes. The third section highlights the main pre-processing steps. The fourth section presents iEEG signal analysis methods. The fifth section discusses statistical approaches. The sixth section draws some unique perspectives on iEEG research. Finally, to ensure a consistent nomenclature throughout the manuscript and to align with other guidelines, e.g., Brain Imaging Data Structure (BIDS) and the OHBM Committee on Best Practices in Data Analysis and Sharing (COBIDAS), we provide a glossary to disambiguate terms related to iEEG research.

Pupil Diameter Tracks Statistical Structure in the Environment to Increase Visual Sensitivity.

Pupil diameter determines how much light hits the retina and, thus, how much information is available for visual processing. This is regulated by a brainstem reflex pathway. Here, we investigate whether this pathway is under the control of internal models about the environment. This would allow adjusting pupil dynamics to environmental statistics to augment information transmission. We present image sequences containing internal temporal structure to humans of either sex and male macaque monkeys. We then measure whether the pupil tracks this temporal structure not only at the rate of luminance variations, but also at the rate of statistics not available from luminance information alone. We find entrainment to environmental statistics in both species. This entrainment directly affects visual processing by increasing sensitivity at the environmentally relevant temporal frequency. Thus, pupil dynamics are matched to the temporal structure of the environment to optimize perception, in line with an active sensing account.SIGNIFICANCE STATEMENT When light hits the retina, the pupil reflexively constricts. This determines how much light and thus how much information is available for visual processing. We show that the rate at which the pupil constricts and dilates is matched to the temporal structure of our visual environment, although this information is not directly contained in the light variations that usually trigger reflexive pupil constrictions. Adjusting pupil diameter in accordance with environmental regularities optimizes information transmission at ecologically relevant temporal frequencies. We show that this is the case in humans and macaque monkeys, suggesting that the reflex pathways that regulate pupil diameter are under some degree of cognitive control across primate species.

Minimal specifications for non-human primate MRI: Challenges in standardizing and harmonizing data collection.

Recent methodological advances in MRI have enabled substantial growth in neuroimaging studies of non-human primates (NHPs), while open data-sharing through the PRIME-DE initiative has increased the availability of NHP MRI data and the need for robust multi-subject multi-center analyses. Streamlined acquisition and analysis protocols would accelerate and improve these efforts. However, consensus on minimal standards for data acquisition protocols and analysis pipelines for NHP imaging remains to be established, particularly for multi-center studies. Here, we draw parallels between NHP and human neuroimaging and provide minimal guidelines for harmonizing and standardizing data acquisition. We advocate robust translation of widely used open-access toolkits that are well established for analyzing human data. We also encourage the use of validated, automated pre-processing tools for analyzing NHP data sets. These guidelines aim to refine methodological and analytical strategies for small and large-scale NHP neuroimaging data. This will improve reproducibility of results, and accelerate the convergence between NHP and human neuroimaging strategies which will ultimately benefit fundamental and translational brain science.

Not All Predictions Are Equal: "What" and "When" Predictions Modulate Activity in Auditory Cortex through Different Mechanisms.

Using predictions based on environmental regularities is fundamental for adaptive behavior. While it is widely accepted that predictions across different stimulus attributes (e.g., time and content) facilitate sensory processing, it is unknown whether predictions across these attributes rely on the same neural mechanism. Here, to elucidate the neural mechanisms of predictions, we combine invasive electrophysiological recordings (human electrocorticography in 4 females and 2 males) with computational modeling while manipulating predictions about content ("what") and time ("when"). We found that "when" predictions increased evoked activity over motor and prefrontal regions both at early (∼180 ms) and late (430-450 ms) latencies. "What" predictability, however, increased evoked activity only over prefrontal areas late in time (420-460 ms). Beyond these dissociable influences, we found that "what" and "when" predictability interactively modulated the amplitude of early (165 ms) evoked responses in the superior temporal gyrus. We modeled the observed neural responses using biophysically realistic neural mass models, to better understand whether "what" and "when" predictions tap into similar or different neurophysiological mechanisms. Our modeling results suggest that "what" and "when" predictability rely on complementary neural processes: "what" predictions increased short-term plasticity in auditory areas, whereas "when" predictability increased synaptic gain in motor areas. Thus, content and temporal predictions engage complementary neural mechanisms in different regions, suggesting domain-specific prediction signaling along the cortical hierarchy. Encoding predictions through different mechanisms may endow the brain with the flexibility to efficiently signal different sources of predictions, weight them by their reliability, and allow for their encoding without mutual interference.SIGNIFICANCE STATEMENT Predictions of different stimulus features facilitate sensory processing. However, it is unclear whether predictions of different attributes rely on similar or different neural mechanisms. By combining invasive electrophysiological recordings of cortical activity with experimental manipulations of participants' predictions about content and time of acoustic events, we found that the two types of predictions had dissociable influences on cortical activity, both in terms of the regions involved and the timing of the observed effects. Further, our biophysical modeling analysis suggests that predictability of content and time rely on complementary neural processes: short-term plasticity in auditory areas and synaptic gain in motor areas, respectively. This suggests that predictions of different features are encoded with complementary neural mechanisms in different brain regions.

Face Patch Resting State Networks Link Face Processing to Social Cognition.

Faces transmit a wealth of social information. How this information is exchanged between face-processing centers and brain areas supporting social cognition remains largely unclear. Here we identify these routes using resting state functional magnetic resonance imaging in macaque monkeys. We find that face areas functionally connect to specific regions within frontal, temporal, and parietal cortices, as well as subcortical structures supporting emotive, mnemonic, and cognitive functions. This establishes the existence of an extended face-recognition system in the macaque. Furthermore, the face patch resting state networks and the default mode network in monkeys show a pattern of overlap akin to that between the social brain and the default mode network in humans: this overlap specifically includes the posterior superior temporal sulcus, medial parietal, and dorsomedial prefrontal cortex, areas supporting high-level social cognition in humans. Together, these results reveal the embedding of face areas into larger brain networks and suggest that the resting state networks of the face patch system offer a new, easily accessible venue into the functional organization of the social brain and into the evolution of possibly uniquely human social skills.

Expecting to See a Letter: Alpha Oscillations as Carriers of Top-Down Sensory Predictions.

Predictions strongly influence perception. However, the neurophysiological processes that implement predictions remain underexplored. It has been proposed that high- and low-frequency neuronal oscillations act as carriers of sensory evidence and top-down predictions, respectively (von Stein and Sarnthein 2000; Bastos et al. 2012). However, evidence for the latter hypothesis remains scarce. In particular, it remains to be shown whether slow prestimulus alpha oscillations in task-relevant brain regions are stronger in the presence of predictions, whether they influence early categorization processes, and whether this interplay indeed boosts perception. Here, we directly address these questions by manipulating subjects' prior expectations about the identity of visually presented letters while collecting magnetoencephalographic recordings. We find that predictions lead to increased prestimulus alpha oscillations in a multisensory network representing grapheme/phoneme associations. Furthermore, alpha power interacts with stimulus degradation and top-down expectations to predict visibility ratings, and correlates with the amplitude of early sensory components (P1/N1m complex), suggesting a role in the selective amplification of predicted information. Our results thus indicate that low-frequency alpha oscillations can serve as a mechanism to carry and test sensory predictions about letters.

Untangling perceptual memory: hysteresis and adaptation map into separate cortical networks.

Perception is an active inferential process in which prior knowledge is combined with sensory input, the result of which determines the contents of awareness. Accordingly, previous experience is known to help the brain "decide" what to perceive. However, a critical aspect that has not been addressed is that previous experience can exert 2 opposing effects on perception: An attractive effect, sensitizing the brain to perceive the same again (hysteresis), or a repulsive effect, making it more likely to perceive something else (adaptation). We used functional magnetic resonance imaging and modeling to elucidate how the brain entertains these 2 opposing processes, and what determines the direction of such experience-dependent perceptual effects. We found that although affecting our perception concurrently, hysteresis and adaptation map into distinct cortical networks: a widespread network of higher-order visual and fronto-parietal areas was involved in perceptual stabilization, while adaptation was confined to early visual areas. This areal and hierarchical segregation may explain how the brain maintains the balance between exploiting redundancies and staying sensitive to new information. We provide a Bayesian model that accounts for the coexistence of hysteresis and adaptation by separating their causes into 2 distinct terms: Hysteresis alters the prior, whereas adaptation changes the sensory evidence (the likelihood function).

Subjective and objective learning effects dissociate in space and in time.

Perceptual learning not only improves sensitivity, but it also changes our subjective experience. However, the question of how these two learning effects relate is largely unexplored. Here we investigate how subjects learn to see initially indiscriminable metacontrast-masked shapes. We find that sensitivity and subjective awareness increase with training. However, sensitivity and subjective awareness dissociate in space: Learning effects on performance are lost when the task is performed at an untrained location in another quadrant, whereas learning effects on subjective awareness are maintained. This finding indicates that improvements in shape sensitivity involve visual areas up to V4, whereas changes in subjective awareness involve other brain regions. Furthermore, subjective awareness dissociates from sensitivity in time: In an early phase of perceptual learning, subjects perform above chance on trials that they rate as subjectively invisible. Later, this phenomenon disappears. Subjective awareness is thus neither necessary nor sufficient for achieving above-chance objective performance.

Visual perceptual learning of feature conjunctions leverages non-linear mixed selectivity.

Visual objects are often defined by multiple features. Therefore, learning novel objects entails learning feature conjunctions. Visual cortex is organized into distinct anatomical compartments, each of which is devoted to processing a single feature. A prime example are neurons purely selective to color and orientation, respectively. However, neurons that jointly encode multiple features (mixed selectivity) also exist across the brain and play critical roles in a multitude of tasks. Here, we sought to uncover the optimal policy that our brain adapts to achieve conjunction learning using these available resources. 59 human subjects practiced orientation-color conjunction learning in four psychophysical experiments designed to nudge the visual system towards using one or the other resource. We find that conjunction learning is possible by linear mixing of pure color and orientation information, but that more and faster learning takes place when both pure and mixed selectivity representations are involved. We also find that learning with mixed selectivity confers advantages in performing an untrained "exclusive or" (XOR) task several months after learning the original conjunction task. This study sheds light on possible mechanisms underlying conjunction learning and highlights the importance of learning by mixed selectivity.

Decision-making processes in perceptual learning depend on effectors.

Visual perceptual learning is traditionally thought to arise in visual cortex. However, typical perceptual learning tasks also involve systematic mapping of visual information onto motor actions. Because the motor system contains both effector-specific and effector-unspecific representations, the question arises whether visual perceptual learning is effector-specific itself, or not. Here, we study this question in an orientation discrimination task. Subjects learn to indicate their choices either with joystick movements or with manual reaches. After training, we challenge them to perform the same task with eye movements. We dissect the decision-making process using the drift diffusion model. We find that learning effects on the rate of evidence accumulation depend on effectors, albeit not fully. This suggests that during perceptual learning, visual information is mapped onto effector-specific integrators. Overlap of the populations of neurons encoding motor plans for these effectors may explain partial generalization. Taken together, visual perceptual learning is not limited to visual cortex, but also affects sensorimotor mapping at the interface of visual processing and decision making.

Variability in training unlocks generalization in visual perceptual learning through invariant representations.

Stimulus and location specificity are long considered hallmarks of visual perceptual learning. This renders visual perceptual learning distinct from other forms of learning, where generalization can be more easily attained, and therefore unsuitable for practical applications, where generalization is key. Based on the hypotheses derived from the structure of the visual system, we test here whether stimulus variability can unlock generalization in perceptual learning. We train subjects in orientation discrimination, while we vary the amount of variability in a task-irrelevant feature, spatial frequency. We find that, independently of task difficulty, this manipulation enables generalization of learning to new stimuli and locations, while not negatively affecting the overall amount of learning on the task. We then use deep neural networks to investigate how variability unlocks generalization. We find that networks develop invariance to the task-irrelevant feature when trained with variable inputs. The degree of learned invariance strongly predicts generalization. A reliance on invariant representations can explain variability-induced generalization in visual perceptual learning. This suggests new targets for understanding the neural basis of perceptual learning in the higher-order visual cortex and presents an easy-to-implement modification of common training paradigms that may benefit practical applications.

Expectations change the signatures and timing of electrophysiological correlates of perceptual awareness.

Previous experience allows the brain to predict what comes next. How these expectations affect conscious experience is poorly understood. In particular, it is unknown whether and when expectations interact with sensory evidence in granting access to conscious perception, and how this is reflected electrophysiologically. Here, we parametrically manipulate sensory evidence and expectations while measuring event-related potentials in human subjects to assess the time course of evoked responses that correlate with subjective visibility, the properties of the stimuli, and/or perceptual expectations. We found that expectations lower the threshold of conscious perception and reduce the latency of neuronal signatures differentiating seen and unseen stimuli. Without expectations, this differentiation occurs ∼300 ms and with expectations ∼200 ms after stimulus in occipitoparietal sensors. The amplitude of this differentiating response component (P2) decreases as visibility increases, regardless of whether this increase is attributable to enhanced sensory evidence and/or the gradual buildup of perceptual expectations. Importantly, at matched performance levels, responses to seen and unseen stimuli differed regardless of the physical stimulus properties. These findings indicate that the latency of the neuronal correlates of access to consciousness depend on whether access is driven by stimulus saliency or by a combination of expectations and sensory evidence.

Stimulus predictability reduces responses in primary visual cortex.

In this functional magnetic resonance imaging study we tested whether the predictability of stimuli affects responses in primary visual cortex (V1). The results of this study indicate that visual stimuli evoke smaller responses in V1 when their onset or motion direction can be predicted from the dynamics of surrounding illusory motion. We conclude from this finding that the human brain anticipates forthcoming sensory input that allows predictable visual stimuli to be processed with less neural activation at early stages of cortical processing.

Previous Motor Actions Outweigh Sensory Information in Sensorimotor Statistical Learning.

Humans can use their previous experience in form of statistical priors to improve decisions. It is, however, unclear how such priors are learned and represented. Importantly, it has remained elusive whether prior learning is independent of the sensorimotor system involved in the learning process or not, as both modality-specific and modality-general learning have been reported in the past. Here, we used a saccadic eye movement task to probe the learning and representation of a spatial prior across a few trials. In this task, learning occurs in an unsupervised manner and through encountering trial-by-trial visual hints drawn from a distribution centered on the target location. Using a model-comparison approach, we found that participants' prior knowledge is largely represented in the form of their previous motor actions, with minimal influence from the previously seen visual hints. By using two different motor contexts for response (looking either at the estimated target location, or exactly opposite to it), we could further compare whether prior experience obtained in one motor context can be transferred to the other. Although learning curves were highly similar, and participants seemed to use the same strategy for both response types, they could not fully transfer their knowledge between contexts, as performance and confidence ratings dropped after a switch of the required response. Together, our results suggest that humans preferably use the internal representations of their previous motor actions, rather than past incoming sensory information, to form statistical sensorimotor priors on the timescale of a few trials.

(Micro)Saccades, corollary activity and cortical oscillations.

In natural vision, attention and eye movements are linked. Furthermore, eye movements structure the inflow of information into the visual system. Saccades, where little vision occurs, alternate with fixations, when most vision occurs. A mechanism must be in place to maximize information intake during fixations. Oscillatory synchrony has been proposed as a mechanism for rapid and reliable communication of signals, subserving cognitive functions such as attention and object identification. We propose that saccade-related corollary activity has a crucial role in anticipatory preparation of visual centers, which interacts with ongoing oscillation, favoring the processing of postfixational signals. During prolonged fixations, microsaccades could be generated to exploit this mechanism. Studying this interplay between the sensory and the motor system will provide novel insight into the dynamics of natural vision.

Memory guidance of value-based decision making at an abstract level of representation.

Value-based decisions about alternatives we have never experienced can be guided by associations between current choice options and memories of prior reward. A critical question is how similar memories need to be to the current situation to effectively guide decisions. We address this question in the context of associative learning of faces using a sensory preconditioning paradigm. We find that memories of reward spread along established associations between faces to guide decision making. While memory guidance is specific for associated facial identities, it does not only occur for the specific images that were originally encountered. Instead, memory guidance generalizes across different images of the associated identities. This suggests that memory guidance does not rely on a pictorial format of representation but on a higher, view-invariant level of abstraction. Thus, memory guidance operates on a level of representation that neither over- nor underspecifies associative relationships in the context of obtaining reward.

Sensitivity and perceptual awareness increase with practice in metacontrast masking.

Can practice effects on unconscious stimuli lead to awareness? Can we "learn to see"? Recent evidence suggests that blindsight patients trained for an extensive period of time can learn to discriminate and consciously perceive stimuli that they were previously unaware of. So far, it is unknown whether these effects generalize to normal observers. Here we investigated practice effects in metacontrast masking. Subjects were trained for five consecutive days on the stimulus onset asynchrony (SOA) that resulted in chance performance. Our results show a linear increase in sensitivity (d') but no change in bias (c) for the trained SOA. This practice effect on sensitivity spreads to all tested SOAs. Additionally, we show that subjects rate their perceptual awareness of the target stimuli differently before and after training, exhibiting not only an increase in sensitivity, but also in the subjective awareness of the percept. Thus, subjects can indeed "learn to see."

Mooney face stimuli for visual perception research.

In 1957, Craig Mooney published a set of human face stimuli to study perceptual closure: the formation of a coherent percept on the basis of minimal visual information. Images of this type, now known as "Mooney faces", are widely used in cognitive psychology and neuroscience because they offer a means of inducing variable perception with constant visuo-spatial characteristics (they are often not perceived as faces if viewed upside down). Mooney's original set of 40 stimuli has been employed in several studies. However, it is often necessary to use a much larger stimulus set. We created a new set of over 500 Mooney faces and tested them on a cohort of human observers. We present the results of our tests here, and make the stimuli freely available via the internet. Our test results can be used to select subsets of the stimuli that are most suited for a given experimental purpose.

Medial prefrontal cortex supports perceptual memory.

Our visual environment constantly changes, yet we experience the world as a stable, unified whole. How is this stability achieved? It has been proposed that the brain preserves an implicit perceptual memory in sensory cortices [1] which stabilizes perception towards previously experienced states [2,3]. The role of higher-order areas, especially prefrontal cortex (PFC), in perceptual memory is less explored. Because PFC exhibits long neural time constants, invariance properties, and large receptive fields which may stabilize perception against time-varying inputs, it seems particularly suited to implement perceptual memory [4]. Support for this idea comes from a neuroimaging study reporting that dorsomedial PFC (dmPFC) correlates with perceptual memory [5]. But dmPFC also participates in decision making [6], so its contribution to perceptual memory could arise on a post-perceptual, decisional level [7]. To determine which role, if any, PFC plays in perceptual memory, we obtained direct intracranial recordings in six epilepsy patients while they performed sequential orientation judgements on ambiguous stimuli known to elicit perceptual memory [8]. We found that dmPFC activity in the high gamma frequency band (HGB, 70-150 Hz) correlates with perceptual memory. This effect is anatomically specific to dmPFC and functionally specific for memories of preceding percepts. Further, dmPFC appears to play a causal role, as a patient with a lesion in this area showed impaired perceptual memory. Thus, dmPFC integrates current sensory information with prior percepts, stabilizing visual experience against the perpetual variability of our surroundings.