Altered hierarchical auditory predictive processing after lesions to the orbitofrontal cortex.

Orbitofrontal cortex (OFC) is classically linked to inhibitory control, emotion regulation, and reward processing. Recent perspectives propose that the OFC also generates predictions about perceptual events, actions, and their outcomes. We tested the role of the OFC in detecting violations of prediction at two levels of abstraction (i.e., hierarchical predictive processing) by studying the event-related potentials (ERPs) of patients with focal OFC lesions (n = 12) and healthy controls (n = 14) while they detected deviant sequences of tones in a local-global paradigm. The structural regularities of the tones were controlled at two hierarchical levels by rules defined at a local (i.e., between tones within sequences) and at a global (i.e., between sequences) level. In OFC patients, ERPs elicited by standard tones were unaffected at both local and global levels compared to controls. However, patients showed an attenuated mismatch negativity (MMN) and P3a to local prediction violation, as well as a diminished MMN followed by a delayed P3a to the combined local and global level prediction violation. The subsequent P3b component to conditions involving violations of prediction at the level of global rules was preserved in the OFC group. Comparable effects were absent in patients with lesions restricted to the lateral PFC, which lends a degree of anatomical specificity to the altered predictive processing resulting from OFC lesion. Overall, the altered magnitudes and time courses of MMN/P3a responses after lesions to the OFC indicate that the neural correlates of detection of auditory regularity violation are impacted at two hierarchical levels of rule abstraction.

Decoding reveals the neural representation of held and manipulated musical thoughts.

Imagine a song you know by heart. With little effort you could sing it or play it vividly in your mind. However, we are only beginning to understand how the brain represents, holds, and manipulates these musical "thoughts". Here, we decoded listened and imagined melodies from MEG brain data (N = 71) to show that auditory regions represent the sensory properties of individual sounds, whereas cognitive control (prefrontal cortex, basal nuclei, thalamus) and episodic memory areas (inferior and medial temporal lobe, posterior cingulate, precuneus) hold and manipulate the melody as an abstract unit. Furthermore, the mental manipulation of a melody systematically changes its neural representation, reflecting the volitional control of auditory images. Our work sheds light on the nature and dynamics of auditory representations and paves the way for future work on neural decoding of auditory imagination.

Music can be reconstructed from human auditory cortex activity using nonlinear decoding models.

Music is core to human experience, yet the precise neural dynamics underlying music perception remain unknown. We analyzed a unique intracranial electroencephalography (iEEG) dataset of 29 patients who listened to a Pink Floyd song and applied a stimulus reconstruction approach previously used in the speech domain. We successfully reconstructed a recognizable song from direct neural recordings and quantified the impact of different factors on decoding accuracy. Combining encoding and decoding analyses, we found a right-hemisphere dominance for music perception with a primary role of the superior temporal gyrus (STG), evidenced a new STG subregion tuned to musical rhythm, and defined an anterior-posterior STG organization exhibiting sustained and onset responses to musical elements. Our findings show the feasibility of applying predictive modeling on short datasets acquired in single patients, paving the way for adding musical elements to brain-computer interface (BCI) applications.

Unexpected sound omissions are signaled in human posterior superior temporal gyrus: an intracranial study.

Context modulates sensory neural activations enhancing perceptual and behavioral performance and reducing prediction errors. However, the mechanism of when and where these high-level expectations act on sensory processing is unclear. Here, we isolate the effect of expectation absent of any auditory evoked activity by assessing the response to omitted expected sounds. Electrocorticographic signals were recorded directly from subdural electrode grids placed over the superior temporal gyrus (STG). Subjects listened to a predictable sequence of syllables, with some infrequently omitted. We found high-frequency band activity (HFA, 70-170 Hz) in response to omissions, which overlapped with a posterior subset of auditory-active electrodes in STG. Heard syllables could be distinguishable reliably from STG, but not the identity of the omitted stimulus. Both omission- and target-detection responses were also observed in the prefrontal cortex. We propose that the posterior STG is central for implementing predictions in the auditory environment. HFA omission responses in this region appear to index mismatch-signaling or salience detection processes.

Deep brain stimulation of the ventrointermediate nucleus of the thalamus to treat essential tremor improves motor sequence learning.

The network of brain structures engaged in motor sequence learning comprises the same structures as those involved in tremor, including basal ganglia, cerebellum, thalamus, and motor cortex. Deep brain stimulation (DBS) of the ventrointermediate nucleus of the thalamus (VIM) reduces tremor, but the effects on motor sequence learning are unknown. We investigated whether VIM stimulation has an impact on motor sequence learning and hypothesized that stimulation effects depend on the laterality of electrode location. Twenty patients (age: 38-81 years; 12 female) with VIM electrodes implanted to treat essential tremor (ET) successfully performed a serial reaction time task, varying whether the stimuli followed a repeating pattern or were selected at random, during which VIM-DBS was either on or off. Analyses of variance were applied to evaluate motor sequence learning performance according to reaction times (RTs) and accuracy. An interaction was observed between whether the sequence was repeated or random and whether VIM-DBS was on or off (F[1,18] = 7.89, p = .012). Motor sequence learning, reflected by reduced RTs for repeated sequences, was greater with DBS on than off (T[19] = 2.34, p = .031). Stimulation location correlated with the degree of motor learning, with greater motor learning when stimulation targeted the lateral VIM (n = 23, ρ = 0.46; p = .027). These results demonstrate the beneficial effects of VIM-DBS on motor sequence learning in ET patients, particularly with lateral VIM electrode location, and provide evidence for a role for the VIM in motor sequence learning.

Detection of mind wandering using EEG: Within and across individuals.

Mind wandering is often characterized by attention oriented away from an external task towards our internal, self-generated thoughts. This universal phenomenon has been linked to numerous disruptive functional outcomes, including performance errors and negative affect. Despite its prevalence and impact, studies to date have yet to identify robust behavioral signatures, making unobtrusive, yet reliable detection of mind wandering a difficult but important task for future applications. Here we examined whether electrophysiological measures can be used in machine learning models to accurately predict mind wandering states. We recorded scalp EEG from participants as they performed an auditory target detection task and self-reported whether they were on task or mind wandering. We successfully classified attention states both within (person-dependent) and across (person-independent) individuals using event-related potential (ERP) measures. Non-linear and linear machine learning models detected mind wandering above-chance within subjects: support vector machine (AUC = 0.715) and logistic regression (AUC = 0.635). Importantly, these models also generalized across subjects: support vector machine (AUC = 0.613) and logistic regression (AUC = 0.609), suggesting we can reliably predict a given individual's attention state based on ERP patterns observed in the group. This study is the first to demonstrate that machine learning models can generalize to "never-seen-before" individuals using electrophysiological measures, highlighting their potential for real-time prediction of covert attention states.

Top-Down Attentional Modulation in Human Frontal Cortex: Differential Engagement during External and Internal Attention.

Decades of electrophysiological research on top-down control converge on the role of the lateral frontal cortex in facilitating attention to behaviorally relevant external inputs. However, the involvement of frontal cortex in the top-down control of attention directed to the external versus internal environment remains poorly understood. To address this, we recorded intracranial electrocorticography while subjects directed their attention externally to tones and responded to infrequent target tones, or internally to their own thoughts while ignoring the tones. Our analyses focused on frontal and temporal cortices. We first computed the target effect, as indexed by the difference in high frequency activity (70-150 Hz) between target and standard tones. Importantly, we then compared the target effect between external and internal attention, reflecting a top-down attentional effect elicited by task demands, in each region of interest. Both frontal and temporal cortices showed target effects during external and internal attention, suggesting this effect is present irrespective of attention states. However, only the frontal cortex showed an enhanced target effect during external relative to internal attention. These findings provide electrophysiological evidence for top-down attentional modulation in the lateral frontal cortex, revealing preferential engagement with external attention.

The brain tracks auditory rhythm predictability independent of selective attention.

The brain responds to violations of expected rhythms, due to extraction- and prediction of the temporal structure in auditory input. Yet, it is unknown how probability of rhythm violations affects the overall rhythm predictability. Another unresolved question is whether predictive processes are independent of attention processes. In this study, EEG was recorded while subjects listened to rhythmic sequences. Predictability was manipulated by changing the stimulus-onset-asynchrony (SOA deviants) for given tones in the rhythm. When SOA deviants were inserted rarely, predictability remained high, whereas predictability was lower with more frequent SOA deviants. Dichotic tone-presentation allowed for independent manipulation of attention, as specific tones of the rhythm were presented to separate ears. Attention was manipulated by instructing subjects to attend to tones in one ear only, while keeping the rhythmic structure of tones constant. The analyses of event-related potentials revealed an attenuated N1 for tones when rhythm predictability was high, while the N1 was enhanced by attention to tones. Bayesian statistics revealed no interaction between predictability and attention. A right-lateralization of attention effects, but not predictability effects, suggested potentially different cortical processes. This is the first study to show that probability of rhythm violation influences rhythm predictability, independent of attention.

Neural Encoding of Auditory Features during Music Perception and Imagery.

Despite many behavioral and neuroimaging investigations, it remains unclear how the human cortex represents spectrotemporal sound features during auditory imagery, and how this representation compares to auditory perception. To assess this, we recorded electrocorticographic signals from an epileptic patient with proficient music ability in 2 conditions. First, the participant played 2 piano pieces on an electronic piano with the sound volume of the digital keyboard on. Second, the participant replayed the same piano pieces, but without auditory feedback, and the participant was asked to imagine hearing the music in his mind. In both conditions, the sound output of the keyboard was recorded, thus allowing precise time-locking between the neural activity and the spectrotemporal content of the music imagery. This novel task design provided a unique opportunity to apply receptive field modeling techniques to quantitatively study neural encoding during auditory mental imagery. In both conditions, we built encoding models to predict high gamma neural activity (70-150 Hz) from the spectrogram representation of the recorded sound. We found robust spectrotemporal receptive fields during auditory imagery with substantial, but not complete overlap in frequency tuning and cortical location compared to receptive fields measured during auditory perception.

Rapid tuning shifts in human auditory cortex enhance speech intelligibility.

Experience shapes our perception of the world on a moment-to-moment basis. This robust perceptual effect of experience parallels a change in the neural representation of stimulus features, though the nature of this representation and its plasticity are not well-understood. Spectrotemporal receptive field (STRF) mapping describes the neural response to acoustic features, and has been used to study contextual effects on auditory receptive fields in animal models. We performed a STRF plasticity analysis on electrophysiological data from recordings obtained directly from the human auditory cortex. Here, we report rapid, automatic plasticity of the spectrotemporal response of recorded neural ensembles, driven by previous experience with acoustic and linguistic information, and with a neurophysiological effect in the sub-second range. This plasticity reflects increased sensitivity to spectrotemporal features, enhancing the extraction of more speech-like features from a degraded stimulus and providing the physiological basis for the observed 'perceptual enhancement' in understanding speech.

Word pair classification during imagined speech using direct brain recordings.

People that cannot communicate due to neurological disorders would benefit from an internal speech decoder. Here, we showed the ability to classify individual words during imagined speech from electrocorticographic signals. In a word imagery task, we used high gamma (70-150 Hz) time features with a support vector machine model to classify individual words from a pair of words. To account for temporal irregularities during speech production, we introduced a non-linear time alignment into the SVM kernel. Classification accuracy reached 88% in a two-class classification framework (50% chance level), and average classification accuracy across fifteen word-pairs was significant across five subjects (mean = 58%; p < 0.05). We also compared classification accuracy between imagined speech, overt speech and listening. As predicted, higher classification accuracy was obtained in the listening and overt speech conditions (mean = 89% and 86%, respectively; p < 0.0001), where speech stimuli were directly presented. The results provide evidence for a neural representation for imagined words in the temporal lobe, frontal lobe and sensorimotor cortex, consistent with previous findings in speech perception and production. These data represent a proof of concept study for basic decoding of speech imagery, and delineate a number of key challenges to usage of speech imagery neural representations for clinical applications.

Sensory Deviancy Detection Measured Directly Within the Human Nucleus Accumbens.

Rapid changes in the environment evoke a comparison between expectancy and actual outcome to inform optimal subsequent behavior. The nucleus accumbens (NAcc), a key interface between the hippocampus and neocortical regions, is a candidate region for mediating this comparison. Here, we report event-related potentials obtained from the NAcc using direct intracranial recordings in 5 human participants while they listened to trains of auditory stimuli differing in their degree of deviation from repetitive background stimuli. NAcc recordings revealed an early mismatch signal (50-220 ms) in response to all deviants. NAcc activity in this time window was also sensitive to the statistics of stimulus deviancy, with larger amplitudes as a function of the level of deviancy. Importantly, this NAcc mismatch signal also predicted generation of longer latency scalp potentials (300-400 ms). The results provide direct human evidence that the NAcc is a key component of a network engaged in encoding statistics of the sensory environmental.

Age-Related Changes in 1/f Neural Electrophysiological Noise.

Aging is associated with performance decrements across multiple cognitive domains. The neural noise hypothesis, a dominant view of the basis of this decline, posits that aging is accompanied by an increase in spontaneous, noisy baseline neural activity. Here we analyze data from two different groups of human subjects: intracranial electrocorticography from 15 participants over a 38 year age range (15-53 years) and scalp EEG data from healthy younger (20-30 years) and older (60-70 years) adults to test the neural noise hypothesis from a 1/f noise perspective. Many natural phenomena, including electrophysiology, are characterized by 1/f noise. The defining characteristic of 1/f is that the power of the signal frequency content decreases rapidly as a function of the frequency (f) itself. The slope of this decay, the noise exponent (χ), is often <-1 for electrophysiological data and has been shown to approach white noise (defined as χ = 0) with increasing task difficulty. We observed, in both electrophysiological datasets, that aging is associated with a flatter (more noisy) 1/f power spectral density, even at rest, and that visual cortical 1/f noise statistically mediates age-related impairments in visual working memory. These results provide electrophysiological support for the neural noise hypothesis of aging. Significance statement: Understanding the neurobiological origins of age-related cognitive decline is of critical scientific, medical, and public health importance, especially considering the rapid aging of the world's population. We find, in two separate human studies, that 1/f electrophysiological noise increases with aging. In addition, we observe that this age-related 1/f noise statistically mediates age-related working memory decline. These results significantly add to this understanding and contextualize a long-standing problem in cognition by encapsulating age-related cognitive decline within a neurocomputational model of 1/f noise-induced deficits in neural communication.

Thalamic theta phase alignment predicts human memory formation and anterior thalamic cross-frequency coupling.

Previously we reported electrophysiological evidence for a role for the anterior thalamic nucleus (ATN) in human memory formation (Sweeney-Reed et al., 2014). Theta-gamma cross-frequency coupling (CFC) predicted successful memory formation, with the involvement of gamma oscillations suggesting memory-relevant local processing in the ATN. The importance of the theta frequency range in memory processing is well-established, and phase alignment of oscillations is considered to be necessary for synaptic plasticity. We hypothesized that theta phase alignment in the ATN would be necessary for memory encoding. Further analysis of the electrophysiological data reveal that phase alignment in the theta rhythm was greater during successful compared with unsuccessful encoding, and that this alignment was correlated with the CFC. These findings support an active processing role for the ATN during memory formation.

Impaired Facilitatory Mechanisms of Auditory Attention After Damage of the Lateral Prefrontal Cortex.

There is growing evidence that auditory selective attention operates via distinct facilitatory and inhibitory mechanisms enabling selective enhancement and suppression of sound processing, respectively. The lateral prefrontal cortex (LPFC) plays a crucial role in the top-down control of selective attention. However, whether the LPFC controls facilitatory, inhibitory, or both attentional mechanisms is unclear. Facilitatory and inhibitory mechanisms were assessed, in patients with LPFC damage, by comparing event-related potentials (ERPs) to attended and ignored sounds with ERPs to these same sounds when attention was equally distributed to all sounds. In control subjects, we observed 2 late frontally distributed ERP components: a transient facilitatory component occurring from 150 to 250 ms after sound onset; and an inhibitory component onsetting at 250 ms. Only the facilitatory component was affected in patients with LPFC damage: this component was absent when attending to sounds delivered in the ear contralateral to the lesion, with the most prominent decreases observed over the damaged brain regions. These findings have 2 important implications: (i) they provide evidence for functionally distinct facilitatory and inhibitory mechanisms supporting late auditory selective attention; (ii) they show that the LPFC is involved in the control of the facilitatory mechanisms of auditory attention.

Corticothalamic phase synchrony and cross-frequency coupling predict human memory formation.

The anterior thalamic nucleus (ATN) is thought to play an important role in a brain network involving the hippocampus and neocortex, which enables human memories to be formed. However, its small size and location deep within the brain have impeded direct investigation in humans with non-invasive techniques. Here we provide direct evidence for a functional role for the ATN in memory formation from rare simultaneous human intrathalamic and scalp electroencephalogram (EEG) recordings from eight volunteering patients receiving intrathalamic electrodes implanted for the treatment of epilepsy, demonstrating real-time communication between neocortex and ATN during successful memory encoding. Neocortical-ATN theta oscillatory phase synchrony of local field potentials and neocortical-theta-to-ATN-gamma cross-frequency coupling during presentation of complex photographic scenes predicted later memory for the scenes, demonstrating a key role for the ATN in human memory encoding.

Spatial and temporal relationships of electrocorticographic alpha and gamma activity during auditory processing.

Neuroimaging approaches have implicated multiple brain sites in musical perception, including the posterior part of the superior temporal gyrus and adjacent perisylvian areas. However, the detailed spatial and temporal relationship of neural signals that support auditory processing is largely unknown. In this study, we applied a novel inter-subject analysis approach to electrophysiological signals recorded from the surface of the brain (electrocorticography (ECoG)) in ten human subjects. This approach allowed us to reliably identify those ECoG features that were related to the processing of a complex auditory stimulus (i.e., continuous piece of music) and to investigate their spatial, temporal, and causal relationships. Our results identified stimulus-related modulations in the alpha (8-12 Hz) and high gamma (70-110 Hz) bands at neuroanatomical locations implicated in auditory processing. Specifically, we identified stimulus-related ECoG modulations in the alpha band in areas adjacent to primary auditory cortex, which are known to receive afferent auditory projections from the thalamus (80 of a total of 15,107 tested sites). In contrast, we identified stimulus-related ECoG modulations in the high gamma band not only in areas close to primary auditory cortex but also in other perisylvian areas known to be involved in higher-order auditory processing, and in superior premotor cortex (412/15,107 sites). Across all implicated areas, modulations in the high gamma band preceded those in the alpha band by 280 ms, and activity in the high gamma band causally predicted alpha activity, but not vice versa (Granger causality, p<1e(-8)). Additionally, detailed analyses using Granger causality identified causal relationships of high gamma activity between distinct locations in early auditory pathways within superior temporal gyrus (STG) and posterior STG, between posterior STG and inferior frontal cortex, and between STG and premotor cortex. Evidence suggests that these relationships reflect direct cortico-cortical connections rather than common driving input from subcortical structures such as the thalamus. In summary, our inter-subject analyses defined the spatial and temporal relationships between music-related brain activity in the alpha and high gamma bands. They provide experimental evidence supporting current theories about the putative mechanisms of alpha and gamma activity, i.e., reflections of thalamo-cortical interactions and local cortical neural activity, respectively, and the results are also in agreement with existing functional models of auditory processing.

Contribution of subregions of human frontal cortex to novelty processing.

Novelty processing was studied in patients with lesions centered in either OFC or lateral pFC (LPFC). An auditory novelty oddball ERP paradigm was applied with environmental sounds serving as task irrelevant novel stimuli. Lesions to the LPFC as well as the OFC resulted in a reduction of the frontal Novelty P3 response, supporting a key role of both frontal subdivisions in novelty processing. The posterior P3b to target sounds was unaffected in patients with frontal lobe lesions in either location, indicating intact posterior cortical target detection mechanisms. LPFC patients displayed an enhanced sustained negative slow wave (NSW) to novel sounds not observed in OFC patients, indicating prolonged resource allocation to task-irrelevant stimuli after LPFC damage. Both patient groups displayed an enhanced NSW to targets relative to controls. However, there was no difference in behavior between patients and controls suggesting that the enhanced NSW to targets may index an increased resource allocation to response requirements enabling comparable performance in the frontal lesioned patients. The current findings indicate that the LPFC and OFC have partly shared and partly differential contributions to the cognitive subcomponents of novelty processing.

Cortical spatio-temporal dynamics underlying phonological target detection in humans.

Selective processing of task-relevant stimuli is critical for goal-directed behavior. We used electrocorticography to assess the spatio-temporal dynamics of cortical activation during a simple phonological target detection task, in which subjects press a button when a prespecified target syllable sound is heard. Simultaneous surface potential recordings during this task revealed a highly ordered temporal progression of high gamma (HG, 70-200 Hz) activity across the lateral hemisphere in less than 1 sec. The sequence demonstrated concurrent regional sensory processing of speech syllables in the posterior superior temporal gyrus (STG) and speech motor cortex, and then transitioned to sequential task-dependent processing from prefrontal cortex (PFC), to the final motor response in the hand sensorimotor cortex. STG activation was modestly enhanced for target over nontarget sounds, supporting a selective gain mechanism in early sensory processing, whereas PFC was entirely selective to targets, supporting its role in guiding response behavior. These results reveal that target detection is not a single cognitive event, but rather a process of progressive target selectivity that involves large-scale rapid parallel and serial processing in sensory, cognitive, and motor structures to support goal-directed human behavior.

Single-trial speech suppression of auditory cortex activity in humans.

The human auditory cortex is engaged in monitoring the speech of interlocutors as well as self-generated speech. During vocalization, auditory cortex activity is reported to be suppressed, an effect often attributed to the influence of an efference copy from motor cortex. Single-unit studies in non-human primates have demonstrated a rich dynamic range of single-trial auditory responses to self-speech consisting of suppressed, nonsuppressed and excited auditory neurons. However, human research using noninvasive methods has only reported suppression of averaged auditory cortex responses to self-generated speech. We addressed this discrepancy by recording electrocorticographic activity from neurosurgical subjects performing auditory repetition tasks. We observed that the degree of suppression varied across different regions of auditory cortex, revealing a variety of suppressed and nonsuppressed responses during vocalization. Importantly, single-trial high-gamma power (γ(High), 70-150 Hz) robustly tracked individual auditory events and exhibited stable responses across trials for suppressed and nonsuppressed regions.