Natural Language Processing Applied to Spontaneous Recall of Famous Faces Reveals Memory Dysfunction in Temporal Lobe Epilepsy Patients.

Objective and Background: Epilepsy patients rank memory problems as their most significant cognitive comorbidity. Current clinical assessments are laborious to administer and score and may not always detect subtle memory decline. The Famous Faces Task (FF) has robustly demonstrated that left temporal lobe epilepsy (LTLE) patients remember fewer names and biographical details compared to right TLE (RTLE) patients and healthy controls (HCs). We adapted the FF task to capture subjects' entire spontaneous spoken recall, then scored responses using manual and natural language processing (NLP) methods. We expected to replicate previous group level differences using spontaneous speech and semi-automated analysis. Methods: Seventy-three (N=73) adults (28 LTLE, 18 RTLE, and 27 HCs) were included in a case-control prospective study design. Twenty FF in politics, sports, and entertainment (active 2008-2017) were shown to subjects, who were asked if they could recognize and spontaneously recall as much biographical detail as possible. We created human-generated and automatically-generated keyword dictionaries for each celebrity, based on a randomly selected training set of half of the HC transcripts. To control for speech output, we measured the speech duration, total word count and content word count for the FF task and a Cookie Theft Control Task (CTT), in which subjects were merely asked to describe a visual scene. Subjects' responses to FF and CTT tasks were recorded, transcribed, and analyzed in a blinded manner with a combination of manual and automated NLP approaches. Results: Famous face recognition accuracy was similar between groups. LTLE patients recalled fewer biographical details compared to HCs and RTLEs using both the gold-standard human-generated dictionary (24%±12% vs. 31%±12% and 30%±12%, p=0.007) and the automated dictionary (24%±12% vs. 31%±12% and 32%±13%, p=0.007). There were no group level differences in speech duration, total word count, or content word count for either the FF and CTT to explain difference in recall performance. There was a positive, statistically significant relationship between MOCA score and FF recall performance as scored by the human-generated (ρ= .327, p= .029) and automatically-generated dictionaries (ρ= .422, p= .004) for TLE subjects, but not HCs, an effect that was driven by LTLE subjects. Discussion: LTLE patients remember fewer details of famous people than HCs or RTLE patients, as discovered by NLP analysis of spontaneous recall. Decreased biographical memory was not due to decreased speech output and correlated with lower MOCA scores. NLP analysis of spontaneous recall can detect memory dysfunction in clinical populations in a semi-automated, objective, and sensitive manner.

A consensus statement on detection of hippocampal sharp wave ripples and differentiation from other fast oscillations.

Decades of rodent research have established the role of hippocampal sharp wave ripples (SPW-Rs) in consolidating and guiding experience. More recently, intracranial recordings in humans have suggested their role in episodic and semantic memory. Yet, common standards for recording, detection, and reporting do not exist. Here, we outline the methodological challenges involved in detecting ripple events and offer practical recommendations to improve separation from other high-frequency oscillations. We argue that shared experimental, detection, and reporting standards will provide a solid foundation for future translational discovery.

Time-dependent transformations of memory representations differ along the long axis of the hippocampus.

Research has shown that sleep is beneficial for the long-term retention of memories. According to theories of memory consolidation, memories are gradually reorganized, becoming supported by widespread, distributed cortical networks, particularly during postencoding periods of sleep. However, the effects of sleep on the organization of memories in the hippocampus itself remains less clear. In a 3-d study, participants encoded separate lists of word-image pairs differing in their opportunity for sleep-dependent consolidation. Pairs were initially studied either before or after an overnight sleep period, and were then restudied in a functional magnetic resonance imaging (fMRI) scan session. We used multivariate pattern similarity analyses to examine fine-grained effects of consolidation on memory representations in the hippocampus. We provide evidence for a dissociation along the long axis of the hippocampus that emerges with consolidation, such that representational patterns for object-word memories initially formed prior to sleep become differentiated in anterior hippocampus and more similar, or overlapping, in posterior hippocampus. Differentiation in anterior hippocampal representations correlated with subsequent behavioral performance. Furthermore, representational overlap in posterior hippocampus correlated with the duration of intervening slow wave sleep. Together, these results demonstrate that sleep-dependent consolidation promotes the reorganization of memory traces along the long axis of the hippocampus.

Spatiotemporal dynamics between interictal epileptiform discharges and ripples during associative memory processing.

We describe the spatiotemporal course of cortical high-gamma activity, hippocampal ripple activity and interictal epileptiform discharges during an associative memory task in 15 epilepsy patients undergoing invasive EEG. Successful encoding trials manifested significantly greater high-gamma activity in hippocampus and frontal regions. Successful cued recall trials manifested sustained high-gamma activity in hippocampus compared to failed responses. Hippocampal ripple rates were greater during successful encoding and retrieval trials. Interictal epileptiform discharges during encoding were associated with 15% decreased odds of remembering in hippocampus (95% confidence interval 6-23%). Hippocampal interictal epileptiform discharges during retrieval predicted 25% decreased odds of remembering (15-33%). Odds of remembering were reduced by 25-52% if interictal epileptiform discharges occurred during the 500-2000 ms window of encoding or by 41% during retrieval. During encoding and retrieval, hippocampal interictal epileptiform discharges were followed by a transient decrease in ripple rate. We hypothesize that interictal epileptiform discharges impair associative memory in a regionally and temporally specific manner by decreasing physiological hippocampal ripples necessary for effective encoding and recall. Because dynamic memory impairment arises from pathological interictal epileptiform discharge events competing with physiological ripples, interictal epileptiform discharges represent a promising therapeutic target for memory remediation in patients with epilepsy.

Learning hierarchical sequence representations across human cortex and hippocampus.

Sensory input arrives in continuous sequences that humans experience as segmented units, e.g., words and events. The brain's ability to discover regularities is called statistical learning. Structure can be represented at multiple levels, including transitional probabilities, ordinal position, and identity of units. To investigate sequence encoding in cortex and hippocampus, we recorded from intracranial electrodes in human subjects as they were exposed to auditory and visual sequences containing temporal regularities. We find neural tracking of regularities within minutes, with characteristic profiles across brain areas. Early processing tracked lower-level features (e.g., syllables) and learned units (e.g., words), while later processing tracked only learned units. Learning rapidly shaped neural representations, with a gradient of complexity from early brain areas encoding transitional probability, to associative regions and hippocampus encoding ordinal position and identity of units. These findings indicate the existence of multiple, parallel computational systems for sequence learning across hierarchically organized cortico-hippocampal circuits.

Sleep Spindles Promote the Restructuring of Memory Representations in Ventromedial Prefrontal Cortex through Enhanced Hippocampal-Cortical Functional Connectivity.

Memory consolidation is hypothesized to involve the distribution and restructuring of memory representations across hippocampal and cortical regions. Theories suggest that, through extended hippocampal-cortical interactions, cortical ensembles come to represent more integrated, or overlapping, memory traces that prioritize commonalities across related memories. Sleep processes, particularly fast sleep spindles, are thought to support consolidation, but evidence for this relationship has been mostly limited to memory retention benefits. Whether fast spindles provide a mechanism for neural changes hypothesized to support consolidation, including the strengthening of hippocampal-cortical networks and integration across memory representations, remains unclear, as does the specificity of regions involved. Using functional connectivity analyses of human fMRI data (both sexes), we show that fast spindle density during overnight sleep is related to enhanced hippocampal-cortical functional connectivity the next day, when restudying information learned before sleep. Spindle density modulated connectivity in distinct hippocampal-cortical networks depending on the category of the consolidated stimuli. Specifically, spindle density correlated with functional connectivity between anterior hippocampus and ventromedial prefrontal cortex (vmPFC) for object-word pairs, and posterior hippocampus and posteromedial cortex for scene-word pairs. Using multivariate pattern analyses, we also show that fast spindle density during postlearning sleep is associated with greater pattern similarity, or representational overlap, across individual object-word memories in vmPFC the next day. Further, the relationship between fast spindle density and representational overlap in vmPFC was mediated by the degree of anterior hippocampal-vmPFC functional connectivity. Together, these results suggest that fast spindles support the network distribution of memory traces, potentially restructuring memory representations in vmPFC.SIGNIFICANCE STATEMENT How new experiences are transformed into long-term memories remains a fundamental question for neuroscience research. Theories suggest that memories are stabilized as they are reorganized in the brain, a process thought to be supported by sleep oscillations, particularly sleep spindles. Although sleep spindles have been associated with benefits in memory retention, it is not well understood how spindles modify neural memory traces. This study found that spindles during overnight sleep correlate with changes in neural memory traces, including enhanced functional connectivity in distinct hippocampal-cortical networks and increased pattern similarity among memories in the cortex. The results provide critical evidence that spindles during overnight sleep may act as a physiological mechanism for the restructuring of neural memory traces.

Closed-Loop Acoustic Stimulation Enhances Sleep Oscillations But Not Memory Performance.

Slow oscillations and spindle activity during non-rapid eye movement sleep have been implicated in memory consolidation. Closed-loop acoustic stimulation has previously been shown to enhance slow oscillations and spindle activity during sleep and improve verbal associative memory. We assessed the effect of closed-loop acoustic stimulation during a daytime nap on a virtual reality spatial navigation task in 12 healthy human subjects in a randomized within-subject crossover design. We show robust enhancement of slow oscillation and spindle activity during sleep. However, no effects on behavioral performance were observed when comparing real versus sham stimulation. To explore whether memory enhancement effects were task specific and dependent on nocturnal sleep, in a second experiment with 19 healthy subjects, we aimed to replicate a previous study that used closed-loop acoustic stimulation to enhance memory for word pairs. The methods used were as close as possible to those used in the original study, except that we used a double-blind protocol, in which both subject and experimenter were unaware of the test condition. Again, we successfully enhanced slow oscillation and spindle power, but again did not strengthen associative memory performance with stimulation. We conclude that enhancement of sleep oscillations may be insufficient to enhance memory performance in spatial navigation or verbal association tasks, and provide possible explanations for lack of behavioral replication.

A deep learning approach for real-time detection of sleep spindles.

OBJECTIVE: Sleep spindles have been implicated in memory consolidation and synaptic plasticity during NREM sleep. Detection accuracy and latency in automatic spindle detection are critical for real-time applications. APPROACH: Here we propose a novel deep learning strategy (SpindleNet) to detect sleep spindles based on a single EEG channel. While the majority of spindle detection methods are used for off-line applications, our method is well suited for online applications. MAIN RESULTS: Compared with other spindle detection methods, SpindleNet achieves superior detection accuracy and speed, as demonstrated in two publicly available expert-validated EEG sleep spindle datasets. Our real-time detection of spindle onset achieves detection latencies of 150-350 ms (~two-three spindle cycles) and retains excellent performance under low EEG sampling frequencies and low signal-to-noise ratios. SpindleNet has good generalization across different sleep datasets from various subject groups of different ages and species. SIGNIFICANCE: SpindleNet is ultra-fast and scalable to multichannel EEG recordings, with an accuracy level comparable to human experts, making it appealing for long-term sleep monitoring and closed-loop neuroscience experiments.

Hippocampal gamma predicts associative memory performance as measured by acute and chronic intracranial EEG.

Direct recordings from the human brain have historically involved epilepsy patients undergoing invasive electroencephalography (iEEG) for surgery. However, these measurements are temporally limited and affected by clinical variables. The RNS System (NeuroPace, Inc.) is a chronic, closed-loop electrographic seizure detection and stimulation system. When adapted by investigators for research, it facilitates cognitive testing in a controlled ambulatory setting, with measurements collected over months to years. We utilized an associative learning paradigm in 5 patients with traditional iEEG and 3 patients with chronic iEEG, and found increased hippocampal gamma (60-100 Hz) sustained at 1.3-1.5 seconds during encoding in successful versus failed trials in surgical patients, with similar results in our RNS System patients (1.4-1.6 seconds). Our findings replicate other studies demonstrating that sustained hippocampal gamma supports encoding. Importantly, we have validated the RNS System to make sensitive measurements of hippocampal dynamics during cognitive tasks in a chronic ambulatory research setting.

Tracking Changes in Brain Network Connectivity under Transcranial Current Stimulation.

Noninvasive transcranial brain stimulation has been widely used in experimental and clinical applications to perturb the brain activity, aiming at promoting synaptic plasticity or enhancing functional connectivity within targeted brain regions. However, there are different types of neurostimulations and various choices of stimulation parameters; how these choices influence the intermediate neurophysiological effects and brain connectivity remain incompletely understood. We propose several quantitative methods to investigate the brain connectivity of an epileptic patient before and after transcranial alternating/direct current stimulation (tACS/tDCS). The neuro-feedback derived from our analyses may provide useful cues for the effectiveness of neurostimulation.

Immediate neurophysiological effects of transcranial electrical stimulation.

Noninvasive brain stimulation techniques are used in experimental and clinical fields for their potential effects on brain network dynamics and behavior. Transcranial electrical stimulation (TES), including transcranial direct current stimulation (tDCS) and transcranial alternating current stimulation (tACS), has gained popularity because of its convenience and potential as a chronic therapy. However, a mechanistic understanding of TES has lagged behind its widespread adoption. Here, we review data and modelling on the immediate neurophysiological effects of TES in vitro as well as in vivo in both humans and other animals. While it remains unclear how typical TES protocols affect neural activity, we propose that validated models of current flow should inform study design and artifacts should be carefully excluded during signal recording and analysis. Potential indirect effects of TES (e.g., peripheral stimulation) should be investigated in more detail and further explored in experimental designs. We also consider how novel technologies may stimulate the next generation of TES experiments and devices, thus enhancing validity, specificity, and reproducibility.

Neural engagement with online educational videos predicts learning performance for individual students.

Online educational materials are largely disseminated through videos, and yet there is little understanding of how these videos engage students and fuel academic success. We hypothesized that components of the electroencephalogram (EEG), previously shown to reflect video engagement, would be predictive of academic performance in the context of educational videos. Two groups of subjects watched educational videos in either an intentional learning paradigm, in which they were aware of an upcoming test, or in an incidental learning paradigm, in which they were unaware that they would be tested. "Neural engagement" was quantified by the inter-subject correlation (ISC) of the EEG that was evoked by the videos. In both groups, students with higher neural engagement retained more information. Neural engagement also discriminated between attentive and inattentive video viewing. These results suggest that this EEG metric is a marker of the stimulus-related attentional mechanisms necessary to retain information. In the future, EEG may be used as a tool to design and assess online educational content.

Elucidating relations between fMRI, ECoG, and EEG through a common natural stimulus.

Human brain mapping relies heavily on fMRI, ECoG and EEG, which capture different physiological signals. Relationships between these signals have been established in the context of specific tasks or during resting state, often using spatially confined concurrent recordings in animals. But it is not certain whether these correlations generalize to other contexts relevant for human cognitive neuroscience. Here, we address the case of complex naturalistic stimuli and ask two basic questions. First, how reliable are the responses evoked by a naturalistic audio-visual stimulus in each of these imaging methods, and second, how similar are stimulus-related responses across methods? To this end, we investigated a wide range of brain regions and frequency bands. We presented the same movie clip twice to three different cohorts of subjects (NEEG = 45, NfMRI = 11, NECoG = 5) and assessed stimulus-driven correlations across viewings and between imaging methods, thereby ruling out task-irrelevant confounds. All three imaging methods had similar repeat-reliability across viewings when fMRI and EEG data were averaged across subjects, highlighting the potential to achieve large signal-to-noise ratio by leveraging large sample sizes. The fMRI signal correlated positively with high-frequency ECoG power across multiple task-related cortical structures but positively with low-frequency EEG and ECoG power. In contrast to previous studies, these correlations were as strong for low-frequency as for high frequency ECoG. We also observed links between fMRI and infra-slow EEG voltage fluctuations. These results extend previous findings to the case of natural stimulus processing.

Author Correction: Low frequency transcranial electrical stimulation does not entrain sleep rhythms measured by human intracranial recordings.

It has come to our attention that we did not specify whether the stimulation magnitudes we report in this Article are peak amplitudes or peak-to-peak. All references to intensity given in mA in the manuscript refer to peak-to-peak amplitudes, except in Fig. 2, where the model is calibrated to 1 mA peak amplitude, as stated. In the original version of the paper we incorrectly calibrated the computational models to 1 mA peak-to-peak, rather than 1 mA peak amplitude. This means that we divided by a value twice as large as we should have. The correct estimated fields are therefore twice as large as shown in the original Fig. 2 and Supplementary Figure 11. The corrected figures are now properly calibrated to 1 mA peak amplitude. Furthermore, the sentence in the first paragraph of the Results section 'Intensity ranged from 0.5 to 2.5 mA (current density 0.125-0.625 mA mA/cm2), which is stronger than in previous reports', should have read 'Intensity ranged from 0.5 to 2.5 mA peak to peak (peak current density 0.0625-0.3125 mA/cm2), which is stronger than in previous reports.' These errors do not affect any of the Article's conclusions.

The Variability of Neural Responses to Naturalistic Videos Change with Age and Sex.

Neural development is generally marked by an increase in the efficiency and diversity of neural processes. In a large sample (n = 114) of human children and adults with ages ranging from 5 to 44 yr, we investigated the neural responses to naturalistic video stimuli. Videos from both real-life classroom settings and Hollywood feature films were used to probe different aspects of attention and engagement. For all stimuli, older ages were marked by more variable neural responses. Variability was assessed by the intersubject correlation of evoked electroencephalographic responses. Young males also had less-variable responses than young females. These results were replicated in an independent cohort (n = 303). When interpreted in the context of neural maturation, we conclude that neural function becomes more variable with maturity, at least during the passive viewing of real-world stimuli.

Low frequency transcranial electrical stimulation does not entrain sleep rhythms measured by human intracranial recordings.

Transcranial electrical stimulation has widespread clinical and research applications, yet its effect on ongoing neural activity in humans is not well established. Previous reports argue that transcranial alternating current stimulation (tACS) can entrain and enhance neural rhythms related to memory, but the evidence from non-invasive recordings has remained inconclusive. Here, we measure endogenous spindle and theta activity intracranially in humans during low-frequency tACS and find no stable entrainment of spindle power during non-REM sleep, nor of theta power during resting wakefulness. As positive controls, we find robust entrainment of spindle activity to endogenous slow-wave activity in 66% of electrodes as well as entrainment to rhythmic noise-burst acoustic stimulation in 14% of electrodes. We conclude that low-frequency tACS at common stimulation intensities neither acutely modulates spindle activity during sleep nor theta activity during waking rest, likely because of the attenuated electrical fields reaching the cortical surface.

Listening Into 2030 Workshop: An Experiment in Envisioning the Future of Hearing and Communication Science.

Here we report the methods and output of a workshop examining possible futures of speech and hearing science out to 2030. Using a design thinking approach, a range of human-centered problems in communication were identified that could provide the motivation for a wide range of research. Nine main research programs were distilled and are summarized: (a) measuring brain and other physiological parameters, (b) auditory and multimodal displays of information, (c) auditory scene analysis, (d) enabling and understanding shared auditory virtual spaces, (e) holistic approaches to health management and hearing impairment, (f) universal access to evolving and individualized technologies, (g) biological intervention for hearing dysfunction, (h) understanding the psychosocial interactions with technology and other humans as mediated by technology, and (i) the impact of changing models of security and privacy. The design thinking approach attempted to link the judged level of importance of different research areas to the "end in mind" through empathy for the real-life problems embodied in the personas created during the workshop.

Engaging narratives evoke similar neural activity and lead to similar time perception.

It is said that we lose track of time - that "time flies" - when we are engrossed in a story. How does engagement with the story cause this distorted perception of time, and what are its neural correlates? People commit both time and attentional resources to an engaging stimulus. For narrative videos, attentional engagement can be represented as the level of similarity between the electroencephalographic responses of different viewers. Here we show that this measure of neural engagement predicted the duration of time that viewers were willing to commit to narrative videos. Contrary to popular wisdom, engagement did not distort the average perception of time duration. Rather, more similar brain responses resulted in a more uniform perception of time across viewers. These findings suggest that by capturing the attention of an audience, narrative videos bring both neural processing and the subjective perception of time into synchrony.

A resource for assessing information processing in the developing brain using EEG and eye tracking.

We present a dataset combining electrophysiology and eye tracking intended as a resource for the investigation of information processing in the developing brain. The dataset includes high-density task-based and task-free EEG, eye tracking, and cognitive and behavioral data collected from 126 individuals (ages: 6-44). The task battery spans both the simple/complex and passive/active dimensions to cover a range of approaches prevalent in modern cognitive neuroscience. The active task paradigms facilitate principled deconstruction of core components of task performance in the developing brain, whereas the passive paradigms permit the examination of intrinsic functional network activity during varying amounts of external stimulation. Alongside these neurophysiological data, we include an abbreviated cognitive test battery and questionnaire-based measures of psychiatric functioning. We hope that this dataset will lead to the development of novel assays of neural processes fundamental to information processing, which can be used to index healthy brain development as well as detect pathologic processes.

The Effects of Compensatory Auditory Stimulation and High-Definition Transcranial Direct Current Stimulation (HD-tDCS) on Tinnitus Perception - A Randomized Pilot Study.

BACKGROUND: Tinnitus correlates with elevated hearing thresholds and reduced cochlear compression. We hypothesized that reduced peripheral input leads to elevated neuronal gain resulting in the perception of a phantom sound. OBJECTIVE: The purpose of this pilot study was to test whether compensating for this peripheral deficit could reduce the tinnitus percept acutely using customized auditory stimulation. To further enhance the effects of auditory stimulation, this intervention was paired with high-definition transcranial direct current stimulation (HD-tDCS). METHODS: A randomized sham-controlled, single blind study was conducted in a clinical setting on adult participants with chronic tinnitus (n = 14). Compensatory auditory stimulation (CAS) and HD-tDCS were administered either individually or in combination in order to access the effects of both interventions on tinnitus perception. CAS consisted of sound exposure typical to daily living (20-minute sound-track of a TV show), which was adapted with compressive gain to compensate for deficits in each subject's individual audiograms. Minimum masking levels and the visual analog scale were used to assess the strength of the tinnitus percept immediately before and after the treatment intervention. RESULTS: CAS reduced minimum masking levels, and visual analog scale trended towards improvement. Effects of HD-tDCS could not be resolved with the current sample size. CONCLUSIONS: The results of this pilot study suggest that providing tailored auditory stimulation with frequency-specific gain and compression may alleviate tinnitus in a clinical population. Further experimentation with longer interventions is warranted in order to optimize effect sizes.