Speech Comprehension by Cochlear Implant Users Assessed with Evoked Potentials and Response Times.

INTRODUCTION: Cochlear implant (CI) users differ greatly in their rehabilitation outcomes, including speech understanding in noise. This variability may be related to brain changes associated with intact senses recruiting cortical areas from stimulation-deprived senses. Numerous studies have demonstrated such cross-modal reorganization in individuals with untreated hearing loss. How it is affected by regular use of hearing devices remains unclear, however. To shed light on this, the current study measured cortical responses reflecting comprehension abilities in experienced CI users and normal-hearing controls. METHODS: Using multichannel electroencephalography, we tested CI users who had used their devices for at least 12 months and closely matched controls (N = 2 × 13). Cortical responses reflecting comprehension abilities - the N400 and late positive complex (LPC) components - were evoked using congruent and incongruent digit-triplet stimuli. The participants' task was to assess digit-triplet congruency by means of timed button presses. All measurements were performed in speech-shaped noise 15 dB above individually measured speech recognition thresholds. Three stimulus presentation modes were used: auditory-only, visual-only, and visual-then-auditory. RESULTS: The analyses revealed no group differences in the N400 and LPC responses. In terms of response times, the CI users were slower and differentially affected by the three stimulus presentation modes relative to the controls. CONCLUSION: Compared to normal-hearing controls, experienced CI users may need more time to comprehend speech in noise. Response times can serve as a proxy for speech comprehension by CI users.

Does experience with hearing aid amplification influence electrophysiological measures of speech comprehension?

OBJECTIVE: To explore if experience with hearing aid (HA) amplification affects speech-evoked cortical potentials reflecting comprehension abilities. DESIGN: N400 and late positive complex (LPC) responses as well as behavioural response times to congruent and incongruent digit triplets were measured. The digits were presented against stationary speech-shaped noise 10 dB above individually measured speech recognition thresholds. Stimulus presentation was either acoustic (digits 1-3) or first visual (digits 1-2) and then acoustic (digit 3). STUDY SAMPLE: Three groups of older participants (N = 3 × 15) with (1) pure-tone average hearing thresholds <25 dB HL from 500-4000 Hz, (2) mild-to-moderate sensorineural hearing loss (SNHL) but no prior HA experience, and (3) mild-to-moderate SNHL and >2 years of HA experience. Groups 2-3 were fitted with test devices in accordance with clinical gain targets. RESULTS: No group differences were found in the electrophysiological data. N400 amplitudes were larger and LPC latencies shorter with acoustic presentation. For group 1, behavioural response times were shorter with visual-then-acoustic presentation. CONCLUSION: When speech audibility is ensured, comprehension-related electrophysiological responses appear intact in individuals with mild-to-moderate SNHL, regardless of prior experience with amplified sound. Further research into the effects of audibility versus acclimatisation-related neurophysiological changes is warranted.

A walk in the park? Characterizing gait-related artifacts in mobile EEG recordings.

Brain activity during natural walking outdoors can be captured using mobile electroencephalography (EEG). However, EEG recorded during gait is confounded with artifacts from various sources, possibly obstructing the interpretation of brain activity patterns. Currently, there is no consensus on how the amount of artifact present in these recordings should be quantified, or is there a systematic description of gait artifact properties. In the current study, we expand several features into a seven-dimensional footprint of gait-related artifacts, combining features of time, time-frequency, spatial, and source domains. EEG of N = 26 participants was recorded while standing and walking outdoors. Footprints of gait-related artifacts before and after two different artifact attenuation strategies (after artifact subspace reconstruction (ASR) and after subsequent independent component analysis [ICA]) were systematically different. We also evaluated topographies, morphologies, and signal-to-noise ratios (SNR) of button-press event-related potentials (ERP) before and after artifact handling, to confirm gait-artifact reduction specificity. Morphologies and SNR remained unchanged after artifact attenuation, whereas topographies improved in quality. Our results show that the footprint can provide a detailed assessment of gait-related artifacts and can be used to estimate the sensitivity of different artifact reduction strategies. Moreover, the analysis of button-press ERPs demonstrated its specificity, as processing did not only reduce gait-related artifacts but ERPs of interest remained largely unchanged. We conclude that the proposed footprint is well suited to characterize individual differences in gait-related artifact extent. In the future, it could be used to compare and optimize recording setups and processing pipelines comprehensively.

Does the electrode amplification style matter? A comparison of active and passive EEG system configurations during standing and walking.

It has been stated that active-transmission electrodes should improve signal quality in mobile EEG recordings. However, few studies have directly compared active- and passive-transmission electrodes during a mobile task. In this repeated measurement study, we investigated the performance of active and passive signal transmission electrodes with the same amplifier system in their respective typical configurations, during a mobile auditory task. The task was an auditory discrimination (1,000 vs. 800 Hz; counterbalanced) oddball task using approximately 560 trials (15% targets) for each condition. Eighteen participants performed the auditory oddball task both while standing and walking in an outdoor environment. While walking, there was a significant decrease in P3 amplitude, post-trial rejection trial numbers, and signal-to-noise ratio (SNR). No significant differences were found in signal quality between the two electrode configurations. SNR and P3 amplitude were test-retest reliable between recordings. We conclude that adequate use of a passive EEG electrode system achieves signal quality equivalent to that of an active system during a mobile task.

Pocketable Labs for Everyone: Synchronized Multi-Sensor Data Streaming and Recording on Smartphones with the Lab Streaming Layer.

The streaming and recording of smartphone sensor signals is desirable for mHealth, telemedicine, environmental monitoring and other applications. Time series data gathered in these fields typically benefit from the time-synchronized integration of different sensor signals. However, solutions required for this synchronization are mostly available for stationary setups. We hope to contribute to the important emerging field of portable data acquisition by presenting open-source Android applications both for the synchronized streaming (Send-a) and recording (Record-a) of multiple sensor data streams. We validate the applications in terms of functionality, flexibility and precision in fully mobile setups and in hybrid setups combining mobile and desktop hardware. Our results show that the fully mobile solution is equivalent to well-established desktop versions. With the streaming application Send-a and the recording application Record-a, purely smartphone-based setups for mobile research and personal health settings can be realized on off-the-shelf Android devices.

Influence of auditory attention on sentence recognition captured by the neural phase.

The aim of this study was to investigate whether attentional influences on speech recognition are reflected in the neural phase entrained by an external modulator. Sentences were presented in 7 Hz sinusoidally modulated noise while the neural response to that modulation frequency was monitored by electroencephalogram (EEG) recordings in 21 participants. We implemented a selective attention paradigm including three different attention conditions while keeping physical stimulus parameters constant. The participants' task was either to repeat the sentence as accurately as possible (speech recognition task), to count the number of decrements implemented in modulated noise (decrement detection task), or to do both (dual task), while the EEG was recorded. Behavioural analysis revealed reduced performance in the dual task condition for decrement detection, possibly reflecting limited cognitive resources. EEG analysis revealed no significant differences in power for the 7 Hz modulation frequency, but an attention-dependent phase difference between tasks. Further phase analysis revealed a significant difference 500 ms after sentence onset between trials with correct and incorrect responses for speech recognition, indicating that speech recognition performance and the neural phase are linked via selective attention mechanisms, at least shortly after sentence onset. However, the neural phase effects identified were small and await further investigation.

The Sensitivity of Ear-EEG: Evaluating the Source-Sensor Relationship Using Forward Modeling.

Ear-EEG allows to record brain activity in every-day life, for example to study natural behaviour or unhindered social interactions. Compared to conventional scalp-EEG, ear-EEG uses fewer electrodes and covers only a small part of the head. Consequently, ear-EEG will be less sensitive to some cortical sources. Here, we perform realistic electromagnetic simulations to compare cEEGrid ear-EEG with 128-channel cap-EEG. We compute the sensitivity of ear-EEG for different cortical sources, and quantify the expected signal loss of ear-EEG relative to cap-EEG. Our results show that ear-EEG is most sensitive to sources in the temporal cortex. Furthermore, we show how ear-EEG benefits from a multi-channel configuration (i.e. cEEGrid). The pipelines presented here can be adapted to any arrangement of electrodes and can therefore provide an estimate of sensitivity to cortical regions, thereby increasing the chance of successful experiments using ear-EEG.

Challenge Accepted? Individual Performance Gains for Motor Imagery Practice with Humanoid Robotic EEG Neurofeedback.

Optimizing neurofeedback (NF) and brain-computer interface (BCI) implementations constitutes a challenge across many fields and has so far been addressed by, among others, advancing signal processing methods or predicting the user's control ability from neurophysiological or psychological measures. In comparison, how context factors influence NF/BCI performance is largely unexplored. We here investigate whether a competitive multi-user condition leads to better NF/BCI performance than a single-user condition. We implemented a foot motor imagery (MI) NF with mobile electroencephalography (EEG). Twenty-five healthy, young participants steered a humanoid robot in a single-user condition and in a competitive multi-user race condition using a second humanoid robot and a pseudo competitor. NF was based on 8-30 Hz relative event-related desynchronization (ERD) over sensorimotor areas. There was no significant difference between the ERD during the competitive multi-user condition and the single-user condition but considerable inter-individual differences regarding which condition yielded a stronger ERD. Notably, the stronger condition could be predicted from the participants' MI-induced ERD obtained before the NF blocks. Our findings may contribute to enhance the performance of NF/BCI implementations and highlight the necessity of individualizing context factors.

Hearing-impaired listeners show increased audiovisual benefit when listening to speech in noise.

Recent studies provide evidence for changes in audiovisual perception as well as for adaptive cross-modal auditory cortex plasticity in older individuals with high-frequency hearing impairments (presbycusis). We here investigated whether these changes facilitate the use of visual information, leading to an increased audiovisual benefit of hearing-impaired individuals when listening to speech in noise. We used a naturalistic design in which older participants with a varying degree of high-frequency hearing loss attended to running auditory or audiovisual speech in noise and detected rare target words. Passages containing only visual speech served as a control condition. Simultaneously acquired scalp electroencephalography (EEG) data were used to study cortical speech tracking. Target word detection accuracy was significantly increased in the audiovisual as compared to the auditory listening condition. The degree of this audiovisual enhancement was positively related to individual high-frequency hearing loss and subjectively reported listening effort in challenging daily life situations, which served as a subjective marker of hearing problems. On the neural level, the early cortical tracking of the speech envelope was enhanced in the audiovisual condition. Similar to the behavioral findings, individual differences in the magnitude of the enhancement were positively associated with listening effort ratings. Our results therefore suggest that hearing-impaired older individuals make increased use of congruent visual information to compensate for the degraded auditory input.

Machine-learning-derived sleep-wake staging from around-the-ear electroencephalogram outperforms manual scoring and actigraphy.

Quantification of sleep is important for the diagnosis of sleep disorders and sleep research. However, the only widely accepted method to obtain sleep staging is by visual analysis of polysomnography (PSG), which is expensive and time consuming. Here, we investigate automated sleep scoring based on a low-cost, mobile electroencephalogram (EEG) platform consisting of a lightweight EEG amplifier combined with flex-printed cEEGrid electrodes placed around the ear, which can be implemented as a fully self-applicable sleep system. However, cEEGrid signals have different amplitude characteristics to normal scalp PSG signals, which might be challenging for visual scoring. Therefore, this study evaluates the potential of automatic scoring of cEEGrid signals using a machine learning classifier ("random forests") and compares its performance with manual scoring of standard PSG. In addition, the automatic scoring of cEEGrid signals is compared with manual annotation of the cEEGrid recording and with simultaneous actigraphy. Acceptable recordings were obtained in 15 healthy volunteers (aged 35 ± 14.3 years) during an extended nocturnal sleep opportunity, which induced disrupted sleep with a large inter-individual variation in sleep parameters. The results demonstrate that machine-learning-based scoring of around-the-ear EEG outperforms actigraphy with respect to sleep onset and total sleep time assessments. The automated scoring outperforms human scoring of cEEGrid by standard criteria. The accuracy of machine-learning-based automated scoring of cEEGrid sleep recordings compared with manual scoring of standard PSG was satisfactory. The findings show that cEEGrid recordings combined with machine-learning-based scoring holds promise for large-scale sleep studies.

The Right Temporoparietal Junction Supports Speech Tracking During Selective Listening: Evidence from Concurrent EEG-fMRI.

Listening selectively to one out of several competing speakers in a "cocktail party" situation is a highly demanding task. It relies on a widespread cortical network, including auditory sensory, but also frontal and parietal brain regions involved in controlling auditory attention. Previous work has shown that, during selective listening, ongoing neural activity in auditory sensory areas is dominated by the attended speech stream, whereas competing input is suppressed. The relationship between these attentional modulations in the sensory tracking of the attended speech stream and frontoparietal activity during selective listening is, however, not understood. We studied this question in young, healthy human participants (both sexes) using concurrent EEG-fMRI and a sustained selective listening task, in which one out of two competing speech streams had to be attended selectively. An EEG-based speech envelope reconstruction method was applied to assess the strength of the cortical tracking of the to-be-attended and the to-be-ignored stream during selective listening. Our results show that individual speech envelope reconstruction accuracies obtained for the to-be-attended speech stream were positively correlated with the amplitude of sustained BOLD responses in the right temporoparietal junction, a core region of the ventral attention network. This brain region further showed task-related functional connectivity to secondary auditory cortex and regions of the frontoparietal attention network, including the intraparietal sulcus and the inferior frontal gyrus. This suggests that the right temporoparietal junction is involved in controlling attention during selective listening, allowing for a better cortical tracking of the attended speech stream.SIGNIFICANCE STATEMENT Listening selectively to one out of several simultaneously talking speakers in a "cocktail party" situation is a highly demanding task. It activates a widespread network of auditory sensory and hierarchically higher frontoparietal brain regions. However, how these different processing levels interact during selective listening is not understood. Here, we investigated this question using fMRI and concurrently acquired scalp EEG. We found that activation levels in the right temporoparietal junction correlate with the sensory representation of a selectively attended speech stream. In addition, this region showed significant functional connectivity to both auditory sensory and other frontoparietal brain areas during selective listening. This suggests that the right temporoparietal junction contributes to controlling selective auditory attention in "cocktail party" situations.

Dynamic phase alignment of ongoing auditory cortex oscillations.

Neural oscillations can synchronize to external rhythmic stimuli, as for example in speech and music. While previous studies have mainly focused on elucidating the fundamental concept of neural entrainment, less is known about the time course of entrainment. In this human electroencephalography (EEG) study, we unravel the temporal evolution of neural entrainment by contrasting short and long periods of rhythmic stimulation. Listeners had to detect short silent gaps that were systematically distributed with respect to the phase of a 3 Hz frequency-modulated tone. We found that gap detection performance was modulated by the stimulus stream with a consistent stimulus phase across participants for short and long stimulation. Electrophysiological analysis confirmed neural entrainment effects at 3 Hz and the 6 Hz harmonic for both short and long stimulation lengths. 3 Hz source level analysis revealed that longer stimulation resulted in a phase shift of a participant's neural phase relative to the stimulus phase. Phase coupling increased over the first second of stimulation, but no effects for phase coupling strength were observed over time. The dynamic evolution of phase alignment suggests that the brain attunes to external rhythmic stimulation by adapting the brain's internal representation of incoming environmental stimuli.

Did You Listen to the Beat? Auditory Steady-State Responses in the Human Electroencephalogram at 4 and 7 Hz Modulation Rates Reflect Selective Attention.

The acoustic envelope of human speech correlates with the syllabic rate (4-8 Hz) and carries important information for intelligibility, which is typically compromised in multi-talker, noisy environments. In order to better understand the dynamics of selective auditory attention to low frequency modulated sound sources, we conducted a two-stream auditory steady-state response (ASSR) selective attention electroencephalogram (EEG) study. The two streams consisted of 4 and 7 Hz amplitude and frequency modulated sounds presented from the left and right side. One of two streams had to be attended while the other had to be ignored. The attended stream always contained a target, allowing for the behavioral confirmation of the attention manipulation. EEG ASSR power analysis revealed a significant increase in 7 Hz power for the attend compared to the ignore conditions. There was no significant difference in 4 Hz power when the 4 Hz stream had to be attended compared to when it had to be ignored. This lack of 4 Hz attention modulation could be explained by a distracting effect of a third frequency at 3 Hz (beat frequency) perceivable when the 4 and 7 Hz streams are presented simultaneously. Taken together our results show that low frequency modulations at syllabic rate are modulated by selective spatial attention. Whether attention effects act as enhancement of the attended stream or suppression of to be ignored stream may depend on how well auditory streams can be segregated.

Enhanced visual adaptation in cochlear implant users revealed by concurrent EEG-fNIRS.

Previous studies have observed lower visual cortex activation for visual processing in cochlear implant (CI) users compared to normal hearing controls, while others reported enhanced visual speechreading abilities in CI users. The present work investigated whether lower visual cortical activation for visual processing can be explained by a more efficient visual sensory encoding in CI users. Specifically, we investigated whether CI users show enhanced stimulus-specific adaptation for visual stimuli compared to controls. Auditory sensory adaptation was also investigated to explore the sensory specificity of the predicted effect. Twenty post-lingually deafened adult CI users and twenty age-matched controls were presented with repeated visual and auditory stimuli during simultaneous acquisition of electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS). By integrating EEG and fNIRS signals we found significantly enhanced visual adaptation and lower visual cortex activation in CI users compared to controls. That is, responses to repeated visual stimuli decreased more prominently in CI users than in controls. The results suggest that CI users process visual stimuli more efficiently than controls.

Motor Imagery Impairment in Postacute Stroke Patients.

Not much is known about how well stroke patients are able to perform motor imagery (MI) and which MI abilities are preserved after stroke. We therefore applied three different MI tasks (one mental chronometry task, one mental rotation task, and one EEG-based neurofeedback task) to a sample of postacute stroke patients (n = 20) and age-matched healthy controls (n = 20) for addressing the following questions: First, which of the MI tasks indicate impairment in stroke patients and are impairments restricted to the paretic side? Second, is there a relationship between MI impairment and sensory loss or paresis severity? And third, do the results of the different MI tasks converge? Significant differences between the stroke and control groups were found in all three MI tasks. However, only the mental chronometry task and EEG analysis revealed paresis side-specific effects. Moreover, sensitivity loss contributed to a performance drop in the mental rotation task. The findings indicate that although MI abilities may be impaired after stroke, most patients retain their ability for MI EEG-based neurofeedback. Interestingly, performance in the different MI measures did not strongly correlate, neither in stroke patients nor in healthy controls. We conclude that one MI measure is not sufficient to fully assess an individual's MI abilities.

Brain oscillations track the formation of episodic memories in the real world.

Despite the well-known influence of environmental context on episodic memory, little has been done to increase contextual richness within the lab. This leaves a blind spot lingering over the neuronal correlates of episodic memory formation in day-to-day life. To address this, we presented participants with a series of words to memorise along a pre-designated route across campus while a mobile EEG system acquired ongoing neural activity. Replicating lab-based subsequent memory effects (SMEs), we identified significant low to mid frequency power decreases (<30Hz), including beta power decreases over the left inferior frontal gyrus. When investigating the oscillatory correlates of temporal and spatial context binding, we found that items strongly bound to spatial context exhibited significantly greater theta power decreases than items strongly bound to temporal context. These findings expand upon lab-based studies by demonstrating the influence of real world contextual factors that underpin memory formation.

Cross-Modal Functional Reorganization of Visual and Auditory Cortex in Adult Cochlear Implant Users Identified with fNIRS.

Cochlear implant (CI) users show higher auditory-evoked activations in visual cortex and higher visual-evoked activation in auditory cortex compared to normal hearing (NH) controls, reflecting functional reorganization of both visual and auditory modalities. Visual-evoked activation in auditory cortex is a maladaptive functional reorganization whereas auditory-evoked activation in visual cortex is beneficial for speech recognition in CI users. We investigated their joint influence on CI users' speech recognition, by testing 20 postlingually deafened CI users and 20 NH controls with functional near-infrared spectroscopy (fNIRS). Optodes were placed over occipital and temporal areas to measure visual and auditory responses when presenting visual checkerboard and auditory word stimuli. Higher cross-modal activations were confirmed in both auditory and visual cortex for CI users compared to NH controls, demonstrating that functional reorganization of both auditory and visual cortex can be identified with fNIRS. Additionally, the combined reorganization of auditory and visual cortex was found to be associated with speech recognition performance. Speech performance was good as long as the beneficial auditory-evoked activation in visual cortex was higher than the visual-evoked activation in the auditory cortex. These results indicate the importance of considering cross-modal activations in both visual and auditory cortex for potential clinical outcome estimation.

Lateralization patterns of covert but not overt movements change with age: An EEG neurofeedback study.

The mental practice of movements has been suggested as a promising add-on therapy to facilitate motor recovery after stroke. In the case of mentally practised movements, electroencephalogram (EEG) can be utilized to provide feedback about an otherwise covert act. The main target group for such an intervention are elderly patients, though research so far is largely focused on young populations (<30 years). The present study therefore aimed to examine the influence of age on the neural correlates of covert movements (CMs) in a real-time EEG neurofeedback framework. CM-induced event-related desynchronization (ERD) was studied in young (mean age: 23.6 years) and elderly (mean age: 62.7 years) healthy adults. Participants performed covert and overt hand movements. CMs were based on kinesthetic motor imagery (MI) or quasi-movements (QM). Based on previous studies investigating QM in the mu frequency range (8-13Hz) QM were expected to result in more lateralized ERD% patterns and accordingly higher classification accuracies. Independent of CM strategy the elderly were characterized by a significantly reduced lateralization of ERD%, due to stronger ipsilateral ERD%, and in consequence, reduced classification accuracies. QM were generally perceived as more vivid, but no differences were evident between MI and QM in ERD% or classification accuracies. EEG feedback enhanced task-related activity independently of strategy and age. ERD% measures of overt and covert movements were strongly related in young adults, whereas in the elderly ERD% lateralization is dissociated. In summary, we did not find evidence in support of more pronounced ERD% lateralization patterns in QM. Our finding of a less lateralized activation pattern in the elderly is in accordance to previous research and with the idea that compensatory processes help to overcome neurodegenerative changes related to normal ageing. Importantly, it indicates that EEG neurofeedback studies should place more emphasis on the age of the potential end-users.

Cross-modal reorganization in cochlear implant users: Auditory cortex contributes to visual face processing.

There is converging evidence that the auditory cortex takes over visual functions during a period of auditory deprivation. A residual pattern of cross-modal take-over may prevent the auditory cortex to adapt to restored sensory input as delivered by a cochlear implant (CI) and limit speech intelligibility with a CI. The aim of the present study was to investigate whether visual face processing in CI users activates auditory cortex and whether this has adaptive or maladaptive consequences. High-density electroencephalogram data were recorded from CI users (n=21) and age-matched normal hearing controls (n=21) performing a face versus house discrimination task. Lip reading and face recognition abilities were measured as well as speech intelligibility. Evaluation of event-related potential (ERP) topographies revealed significant group differences over occipito-temporal scalp regions. Distributed source analysis identified significantly higher activation in the right auditory cortex for CI users compared to NH controls, confirming visual take-over. Lip reading skills were significantly enhanced in the CI group and appeared to be particularly better after a longer duration of deafness, while face recognition was not significantly different between groups. However, auditory cortex activation in CI users was positively related to face recognition abilities. Our results confirm a cross-modal reorganization for ecologically valid visual stimuli in CI users. Furthermore, they suggest that residual takeover, which can persist even after adaptation to a CI is not necessarily maladaptive.

Real-time EEG feedback during simultaneous EEG-fMRI identifies the cortical signature of motor imagery.

Motor imagery (MI) combined with real-time electroencephalogram (EEG) feedback is a popular approach for steering brain-computer interfaces (BCI). MI BCI has been considered promising as add-on therapy to support motor recovery after stroke. Yet whether EEG neurofeedback indeed targets specific sensorimotor activation patterns cannot be unambiguously inferred from EEG alone. We combined MI EEG neurofeedback with concurrent and continuous functional magnetic resonance imaging (fMRI) to characterize the relationship between MI EEG neurofeedback and activation in cortical sensorimotor areas. EEG signals were corrected online from interfering MRI gradient and ballistocardiogram artifacts, enabling the delivery of real-time EEG feedback. Significantly enhanced task-specific brain activity during feedback compared to no feedback blocks was present in EEG and fMRI. Moreover, the contralateral MI related decrease in EEG sensorimotor rhythm amplitude correlated inversely with fMRI activation in the contralateral sensorimotor areas, whereas a lateralized fMRI pattern did not necessarily go along with a lateralized EEG pattern. Together, the findings indicate a complex relationship between MI EEG signals and sensorimotor cortical activity, whereby both are similarly modulated by EEG neurofeedback. This finding supports the potential of MI EEG neurofeedback for motor rehabilitation and helps to better understand individual differences in MI BCI performance.