Under the Mind's Hood: What We Have Learned by Watching the Brain at Work.

Imagine you were asked to investigate the workings of an engine, but to do so without ever opening the hood. Now imagine the engine fueled the human mind. This is the challenge faced by cognitive neuroscientists worldwide aiming to understand the neural bases of our psychological functions. Luckily, human ingenuity comes to the rescue. Around the same time as the Society for Neuroscience was being established in the 1960s, the first tools for measuring the human brain at work were becoming available. Noninvasive human brain imaging and neurophysiology have continued developing at a relentless pace ever since. In this 50 year anniversary, we reflect on how these methods have been changing our understanding of how brain supports mind.

Temporal orienting in Parkinson's disease.

Temporal orienting of attention can affect multiple stages of processing to guide adaptive behaviour. We tested whether temporal expectation in different task contexts is compromised in individuals with Parkinson's disease (PD). In Experiment 1 two temporal-orienting tasks were used: a speeded task emphasizing motor preparation and a non-speeded task emphasizing perceptual discrimination using rapid serial visual presentation. In both tasks, auditory cues indicated the likelihood of a target appearing after a short or long interval. In the speeded-response task, participants used the cues to anticipate an easily detectable target stimulus. In the non-speeded perceptual-discrimination task, participants used the cues to help discriminate a target letter embedded in a stream of letters. Relative to healthy participants, participants with PD did not show altered temporal orienting effects in the speeded-response task. However, they were impaired in using temporal cues to improve perceptual discrimination. In Experiment 2, we tested whether the temporal-orienting deficits in the perceptual-discrimination task depended on the requirement to ignore temporally distracting stimuli. We replicated the impaired temporal orienting for perceptual discrimination in an independent group of individuals with PD, and showed the impairment was abolished when individuals were on their dopaminergic medication. In a task without any distracting letters, however, patients off or on medication benefited normally from temporal orienting cues. Our findings suggest that deficits in temporal orienting in individuals with PD interact with specific task demands, such as the requirement to select target from temporally competing distractors.

The Functional Consequences of Social Attention for Memory-guided Attention Orienting and Anticipatory Neural Dynamics.

Social attention when viewing natural social (compared with nonsocial) images has functional consequences on contextual memory in healthy human adults. In addition to attention affecting memory performance, strong evidence suggests that memory, in turn, affects attentional orienting. Here, we ask whether the effects of social processing on memory alter subsequent memory-guided attention orienting and corresponding anticipatory dynamics of 8-12 Hz alpha-band oscillations as measured with EEG. Eighteen young adults searched for targets in scenes that contained either social or nonsocial distracters and their memory precision tested. Subsequently, RT was measured as participants oriented to targets appearing in those scenes at either valid (previously learned) locations or invalid (different) locations. Memory precision was poorer for target locations in social scenes. In addition, distractor type moderated the validity effect during memory-guided attentional orienting, with a larger cost in RT when targets appeared at invalid (different) locations within scenes with social distractors. The poorer memory performance was also marked by reduced anticipatory dynamics of spatially lateralized 8-12 Hz alpha-band oscillations for scenes with social distractors. The functional consequences of a social attention bias therefore extend from memory to memory-guided attention orienting, a bidirectional chain that may further reinforce attentional biases.

The Cumulative Effects of Predictability on Synaptic Gain in the Auditory Processing Stream.

Stimulus predictability can lead to substantial modulations of brain activity, such as shifts in sustained magnetic field amplitude, measured with magnetoencephalography (MEG). Here, we provide a mechanistic explanation of these effects using MEG data acquired from healthy human volunteers (N = 13, 7 female). In a source-level analysis of induced responses, we established the effects of orthogonal predictability manipulations of rapid tone-pip sequences (namely, sequence regularity and alphabet size) along the auditory processing stream. In auditory cortex, regular sequences with smaller alphabets induced greater gamma activity. Furthermore, sequence regularity shifted induced activity in frontal regions toward higher frequencies. To model these effects in terms of the underlying neurophysiology, we used dynamic causal modeling for cross-spectral density and estimated slow fluctuations in neural (postsynaptic) gain. Using the model-based parameters, we accurately explain the sensor-level sustained field amplitude, demonstrating that slow changes in synaptic efficacy, combined with sustained sensory input, can result in profound and sustained effects on neural responses to predictable sensory streams.SIGNIFICANCE STATEMENT Brain activity can be strongly modulated by the predictability of stimuli it is currently processing. An example of such a modulation is a shift in sustained magnetic field amplitude, measured with magnetoencephalography. Here, we provide a mechanistic explanation of these effects. First, we establish the oscillatory neural correlates of independent predictability manipulations in hierarchically distinct areas of the auditory processing stream. Next, we use a biophysically realistic computational model to explain these effects in terms of the underlying neurophysiology. Finally, using the model-based parameters describing neural gain modulation, we can explain the previously unexplained effects observed at the sensor level. This demonstrates that slow modulations of synaptic gain can result in profound and sustained effects on neural activity.

Training Working Memory in Childhood Enhances Coupling between Frontoparietal Control Network and Task-Related Regions.

UNLABELLED: Working memory is a capacity upon which many everyday tasks depend and which constrains a child's educational progress. We show that a child's working memory can be significantly enhanced by intensive computer-based training, relative to a placebo control intervention, in terms of both standardized assessments of working memory and performance on a working memory task performed in a magnetoencephalography scanner. Neurophysiologically, we identified significantly increased cross-frequency phase amplitude coupling in children who completed training. Following training, the coupling between the upper alpha rhythm (at 16 Hz), recorded in superior frontal and parietal cortex, became significantly coupled with high gamma activity (at ∼90 Hz) in inferior temporal cortex. This altered neural network activity associated with cognitive skill enhancement is consistent with a framework in which slower cortical rhythms enable the dynamic regulation of higher-frequency oscillatory activity related to task-related cognitive processes. SIGNIFICANCE STATEMENT: Whether we can enhance cognitive abilities through intensive training is one of the most controversial topics of cognitive psychology in recent years. This is particularly controversial in childhood, where aspects of cognition, such as working memory, are closely related to school success and are implicated in numerous developmental disorders. We provide the first neurophysiological account of how working memory training may enhance ability in childhood, using a brain recording technique called magnetoencephalography. We borrowed an analysis approach previously used with intracranial recordings in adults, or more typically in other animal models, called "phase amplitude coupling."

Cognitive Training in the Elderly: Bottlenecks and New Avenues.

Development of measures to preserve cognitive function or even reverse cognitive decline in the ever-growing elderly population is the focus of many research and commercial efforts. One such measure gaining in popularity is the development of computer-based interventions that "exercise" cognitive functions. Computer-based cognitive training has the potential to be specific and flexible, accommodates feedback, and is highly accessible. As in most budding fields, there are still considerable inconsistencies across methodologies and results, as well as a lack of consensus on a comprehensive assessment protocol. We propose that the success of training-based therapeutics will rely on targeting specific cognitive functions, informed by comprehensive and sensitive batteries that can provide a "fingerprint" of an individual's abilities. Instead of expecting a panacea from training regimens, focused and personalized training interventions that accommodate individual differences should be developed to redress specific patterns of deficits in cognitive rehabilitation, both in healthy aging and in disease.

Behavioral and Neural Markers of Flexible Attention over Working Memory in Aging.

Working memory (WM) declines as we age and, because of its fundamental role in higher order cognition, this can have highly deleterious effects in daily life. We investigated whether older individuals benefit from flexible orienting of attention within WM to mitigate cognitive decline. We measured magnetoencephalography (MEG) in older adults performing a WM precision task with cues during the maintenance period that retroactively predicted the location of the relevant items for performance (retro-cues). WM performance of older adults significantly benefitted from retro-cues. Whereas WM maintenance declined with age, retro-cues conferred strong attentional benefits. A model-based analysis revealed an increase in the probability of recalling the target, a lowered probability of retrieving incorrect items or guessing, and an improvement in memory precision. MEG recordings showed that retro-cues induced a transient lateralization of alpha (8-14 Hz) and beta (15-30 Hz) oscillatory power. Interestingly, shorter durations of alpha/beta lateralization following retro-cues predicted larger cueing benefits, reinforcing recent ideas about the dynamic nature of access to WM representations. Our results suggest that older adults retain flexible control over WM, but individual differences in control correspond to differences in neural dynamics, possibly reflecting the degree of preservation of control in healthy aging.

Top-Down Activation of Spatiotopic Sensory Codes in Perceptual and Working Memory Search.

A critical requirement of an efficient cognitive system is the selection and prioritization of relevant information. This occurs when selecting specific items from our sensory inputs, which then receive preferential status at subsequent levels of processing. Many everyday tasks also require us to select internal representations, such as a relevant item from memory. We show that both of these types of search are underpinned by the spatiotopic activation of sensory codes, using both fMRI and MEG data. When individuals searched for perceived and remembered targets, the MEG data highlighted a sensor level electrophysiological effect that reflects the contralateral organization of the visual system-namely, the N2pc. The fMRI data were used to identify a network of frontoparietal areas common to both types of search, as well as the early visual areas activated by the search display. We then combined fMRI and MEG data to explore the temporal dynamics of functional connections between the frontoparietal network and the early visual areas. Searching for a target item resulted in significantly enhanced phase-phase coupling between the frontoparietal network and the visual areas contralateral to the perceived or remembered location of that target. This enhancement of spatially specific phase-phase coupling occurred before the N2pc effect and was significantly associated with it on a trial-by-trial basis. The combination of these two imaging modalities suggests that perceptual and working memory search are underpinned by the synchronization of a frontoparietal network and the relevant sensory cortices.

Frontoparietal and Cingulo-opercular Networks Play Dissociable Roles in Control of Working Memory.

We used magnetoencephalography to characterize the spatiotemporal dynamics of cortical activity during top-down control of working memory (WM). fMRI studies have previously implicated both the frontoparietal and cingulo-opercular networks in control over WM, but their respective contributions are unclear. In our task, spatial cues indicating the relevant item in a WM array occurred either before the memory array or during the maintenance period, providing a direct comparison between prospective and retrospective control of WM. We found that in both cases a frontoparietal network activated following the cue, but following retrocues this activation was transient and was succeeded by a cingulo-opercular network activation. We also characterized the time course of top-down modulation of alpha activity in visual/parietal cortex. This modulation was transient following retrocues, occurring in parallel with the frontoparietal network activation. We suggest that the frontoparietal network is responsible for top-down modulation of activity in sensory cortex during both preparatory attention and orienting within memory. In contrast, the cingulo-opercular network plays a more downstream role in cognitive control, perhaps associated with output gating of memory.

Dissociable prior influences of signal probability and relevance on visual contrast sensitivity.

According to signal detection theoretical analyses, visual signals occurring at a cued location are detected more accurately, whereas frequently occurring ones are reported more often but are not better distinguished from noise. However, conventional analyses that estimate sensitivity and bias by comparing true- and false-positive rates offer limited insights into the mechanisms responsible for these effects. Here, we reassessed the prior influences of signal probability and relevance on visual contrast detection using a reverse-correlation technique that quantifies how signal-like fluctuations in noise predict trial-to-trial variability in choice discarded by conventional analyses. This approach allowed us to estimate separately the sensitivity of true and false positives to parametric changes in signal energy. We found that signal probability and relevance both increased energy sensitivity, but in dissociable ways. Cues predicting the relevant location increased primarily the sensitivity of true positives by suppressing internal noise during signal processing, whereas cues predicting greater signal probability increased both the frequency and the sensitivity of false positives by biasing the baseline activity of signal-selective units. We interpret these findings in light of "predictive-coding" models of perception, which propose separable top-down influences of expectation (probability driven) and attention (relevance driven) on bottom-up sensory processing.

Long-term memory prepares neural activity for perception.

Past experience provides a rich source of predictive information about the world that could be used to guide and optimize ongoing perception. However, the neural mechanisms that integrate information coded in long-term memory (LTM) with ongoing perceptual processing remain unknown. Here, we explore how the contents of LTM optimize perception by modulating anticipatory brain states. By using a paradigm that integrates LTM and attentional orienting, we first demonstrate that the contents of LTM sharpen perceptual sensitivity for targets presented at memory-predicted spatial locations. Next, we examine oscillations in EEG to show that memory-guided attention is associated with spatially specific desynchronization of alpha-band activity over visual cortex. Additionally, we use functional MRI to confirm that target-predictive spatial information stored in LTM triggers spatiotopic modulation of preparatory activity in extrastriate visual cortex. Finally, functional MRI results also implicate an integrated cortical network, including the hippocampus and a dorsal frontoparietal circuit, as a likely candidate for organizing preparatory states in visual cortex according to the contents of LTM.

Attention biases visual activity in visual short-term memory.

In the current study, we tested whether representations in visual STM (VSTM) can be biased via top-down attentional modulation of visual activity in retinotopically specific locations. We manipulated attention using retrospective cues presented during the retention interval of a VSTM task. Retrospective cues triggered activity in a large-scale network implicated in attentional control and led to retinotopically specific modulation of activity in early visual areas V1-V4. Importantly, shifts of attention during VSTM maintenance were associated with changes in functional connectivity between pFC and retinotopic regions within V4. Our findings provide new insights into top-down control mechanisms that modulate VSTM representations for flexible and goal-directed maintenance of the most relevant memoranda.

Reward associations magnify memory-based biases on perception.

Long-term spatial contextual memories are a rich source of predictions about the likely locations of relevant objects in the environment and should enable tuning of neural processing of unfolding events to optimize perception and action. Of particular importance is whether and how the reward outcome of past events can impact perception. We combined behavioral measures with recordings of brain activity with high temporal resolution to test whether the previous reward outcome associated with a memory could modulate the impact of memory-based biases on perception, and if so, the level(s) at which visual neural processing is biased by reward-associated memory-guided attention. Data showed that past rewards potentiate the effects of spatial memories upon the discrimination of target objects embedded within complex scenes starting from early perceptual stages. We show that a single reward outcome of learning impacts on how we perceive events in our complex environments.

Long-term memories bias sensitivity and target selection in complex scenes.

In everyday situations, we often rely on our memories to find what we are looking for in our cluttered environment. Recently, we developed a new experimental paradigm to investigate how long-term memory (LTM) can guide attention and showed how the pre-exposure to a complex scene in which a target location had been learned facilitated the detection of the transient appearance of the target at the remembered location [Summerfield, J. J., Rao, A., Garside, N., & Nobre, A. C. Biasing perception by spatial long-term memory. The Journal of Neuroscience, 31, 14952-14960, 2011; Summerfield, J. J., Lepsien, J., Gitelman, D. R., Mesulam, M. M., & Nobre, A. C. Orienting attention based on long-term memory experience. Neuron, 49, 905-916, 2006]. This study extends these findings by investigating whether and how LTM can enhance perceptual sensitivity to identify targets occurring within their complex scene context. Behavioral measures showed superior perceptual sensitivity (d') for targets located in remembered spatial contexts. We used the N2pc ERP to test whether LTM modulated the process of selecting the target from its scene context. Surprisingly, in contrast to effects of visual spatial cues or implicit contextual cueing, LTM for target locations significantly attenuated the N2pc potential. We propose that the mechanism by which these explicitly available LTMs facilitate perceptual identification of targets may differ from mechanisms triggered by other types of top-down sources of information.

Attention modulates maintenance of representations in visual short-term memory.

Recent studies have shown that selective attention is of considerable importance for encoding task-relevant items into visual short-term memory (VSTM) according to our behavioral goals. However, it is not known whether top-down attentional biases can continue to operate during the maintenance period of VSTM. We used ERPs to investigate this question across two experiments. Specifically, we tested whether orienting attention to a given spatial location within a VSTM representation resulted in modulation of the contralateral delay activity (CDA), a lateralized ERP marker of VSTM maintenance generated when participants selectively encode memory items from one hemifield. In both experiments, retrospective cues during the maintenance period could predict a specific item (spatial retrocue) or multiple items (neutral retrocue) that would be probed at the end of the memory delay. Our results revealed that VSTM performance is significantly improved by orienting attention to the location of a task-relevant item. The behavioral benefit was accompanied by modulation of neural activity involved in VSTM maintenance. Spatial retrocues reduced the magnitude of the CDA, consistent with a reduction in memory load. Our results provide direct evidence that top-down control modulates neural activity associated with maintenance in VSTM, biasing competition in favor of the task-relevant information.

EMD: Empirical Mode Decomposition and Hilbert-Huang Spectral Analyses in Python.

The Empirical Mode Decomposition (EMD) package contains Python (>=3.5) functions for analysis of non-linear and non-stationary oscillatory time series. EMD implements a family of sifting algorithms, instantaneous frequency transformations, power spectrum construction and single-cycle feature analysis. These implementations are supported by online documentation containing a range of practical tutorials.

Superior short-term memory in APOE ε2 carriers across the age range.

The Apolipoprotein-E (APOE) gene is now known to be associated with individual differences in cognitive health in ageing. However, while the APOE ε4 allele confers significantly increased risk of developing Alzheimer's disease (AD), the APOE ε2 allele is hypothesized to be protective against the development of AD. This is in line with neuroimaging and pathological findings associated with ε2 APOE allele, which go in the opposite direction to those observed in AD-related pathology. However, the precise impact of this allele on cognition remains inconclusive, with some small-cohort studies raising the possibility of an advantageous memory performance in these individuals. Here, we tested short-term memory (STM) performance in a large cohort of individuals, 300 of which were ε2/ε3 carriers. Their performance was compared to 554 ε3/ε3 carriers. We included participants from a wide age range spanning young, middle-aged and elderly adults. All of them performed a STM task that has previously been shown to be sensitive to subtle changes in memory in various patient and at-risk cohorts. Individuals carrying the APOE-ε2 allele exhibited a significant memory advantage, regardless of STM task difficulty and across all ages. The observed memory advantage was present across the age range, suggestive of a phenotypical effect of this allele on cognition, possibly independent of any effects of this genetic allele that occur later life in these individuals.

Reduced cortico-muscular beta coupling in Parkinson's disease predicts motor impairment.

Long-range communication through the motor system is thought to be facilitated by phase coupling between neural activity in the 15-30 Hz beta range. During periods of sustained muscle contraction (grip), such coupling is manifest between motor cortex and the contralateral forearm muscles-measured as the cortico-muscular coherence. We examined alterations in cortico-muscular coherence in individuals with Parkinson's disease, while equating grip strength between individuals with Parkinson's disease (off their medication) and healthy control participants. We show a marked reduction in beta cortico-muscular coherence in the Parkinson's disease group, even though the grip strength was comparable between the two groups. Moreover, the reduced cortico-muscular coherence was related to motor symptoms, so that individuals with lower cortico-muscular coherence also displayed worse motor symptoms. These findings highlight the cortico-muscular coherence as a simple, effective and clinically relevant neural marker of Parkinson's disease pathology, with the potential to aid monitoring of disease progression and the efficacy of novel treatments for Parkinson's disease.

Searching for targets within the spatial layout of visual short-term memory.

Recent studies have revealed that the internal representations that we construct from the environment and maintain in visual short-term memory (VSTM) to guide behavior are highly flexible and can be selectively modulated according to our task goals and expectations. In the current study, we conducted two experiments to compare and contrast neural mechanisms of selective attention related to searching for target items within perceptual versus VSTM representations. We used event-related potentials to investigate whether searching for relevant target items from within VSTM representations involves spatially specific biasing of neural activity in a manner analogous to that which occurs during visual search for target items in perceptual arrays. The results, replicated across the two experiments, revealed that selection of a target object within a search array maintained in VSTM proceeds through a similar mechanism as that in the perceptual domain. In line with previous results, N2pc potentials were obtained when targets were identified within a perceptual visual-search array. Interestingly, equivalent N2pcs, with similar time courses and scalp distributions, were also elicited when target items were identified within a VSTM representation. The findings reinforce the notion of highly flexible VSTM representations that can be modulated according to task goals and suggest a large degree of overlap in the spatially specific neural mechanisms of target selection across the perceptual and VSTM domains.

Short-term memory advantage for brief durations in human APOE ε4 carriers.

The Apolipoprotein-E (APOE) ε4 gene allele, the highest known genetic risk factor for Alzheimer's disease, has paradoxically been well preserved in the human population. One possible explanation offered by evolutionary biology for survival of deleterious genes is antagonistic pleiotropy. This theory proposes that such genetic variants might confer an advantage, even earlier in life when humans are also reproductively fit. The results of some small-cohort studies have raised the possibility of such a pleiotropic effect for the ε4 allele in short-term memory (STM) but the findings have been inconsistent. Here, we tested STM performance in a large cohort of individuals (N = 1277); nine hundred and fifty-nine of which included carrier and non-carriers of the APOE ε4 gene, those at highest risk of developing Alzheimer's disease. We first confirm that this task is sensitive to subtle deterioration in memory performance across ageing. Importantly, individuals carrying the APOE ε4 gene actually exhibited a significant memory advantage across all ages, specifically for brief retention periods but crucially not for longer durations. Together, these findings present the strongest evidence to date for a gene having an antagonistic pleiotropy effect on human cognitive function across a wide age range, and hence provide an explanation for the survival of the APOE ε4 allele in the gene pool.