Impact of Reduced Spectral Resolution on Temporal-Coherence-Based Source Segregation.

Hearing-impaired listeners struggle to understand speech in noise, even when using cochlear implants (CIs) or hearing aids. Successful listening in noisy environments depends on the brain's ability to organize a mixture of sound sources into distinct perceptual streams (i.e., source segregation). In normal-hearing listeners, temporal coherence of sound fluctuations across frequency channels supports this process by promoting grouping of elements belonging to a single acoustic source. We hypothesized that reduced spectral resolution-a hallmark of both electric/CI (from current spread) and acoustic (from broadened tuning) hearing with sensorineural hearing loss-degrades segregation based on temporal coherence. This is because reduced frequency resolution decreases the likelihood that a single sound source dominates the activity driving any specific channel; concomitantly, it increases the correlation in activity across channels. Consistent with our hypothesis, predictions from a physiologically plausible model of temporal-coherence-based segregation suggest that CI current spread reduces comodulation masking release (CMR; a correlate of temporal-coherence processing) and speech intelligibility in noise. These predictions are consistent with our behavioral data with simulated CI listening. Our model also predicts smaller CMR with increasing levels of outer-hair-cell damage. These results suggest that reduced spectral resolution relative to normal hearing impairs temporal-coherence-based segregation and speech-in-noise outcomes.

Perceptual organization and task demands jointly shape auditory working memory capacity.

Listeners performed two different tasks in which they remembered short sequences comprising either complex tones (generally heard as one melody) or everyday sounds (generally heard as separate objects). In one, listeners judged whether a probe item had been present in the preceding sequence. In the other, they judged whether a second sequence of the same items was identical in order to the preceding sequence. Performance on the first task was higher for everyday sounds; performance on the second was higher for complex tones. Perceptual organization strongly shapes listeners' memory for sounds, with implications for real-world communication.

Auditory evoked potentials in adolescents with autism: An investigation of brain development, intellectual impairment, and neural encoding.

Limited research has evaluated neural encoding of sounds from a developmental perspective in individuals with autism (ASD), especially among those with intellectual disability. We compared auditory evoked potentials (AEPs) in autistic adolescents with a wide range of intellectual abilities (n = 40, NVIQ 30-160) to both age-matched cognitively able neurotypical adolescent controls (NT-A, n = 37) and younger neurotypical children (NT-C, n = 27) to assess potential developmental delays. In addition to a classic measure of peak amplitude, we calculated a continuous measure of intra-class correlation (ICC) between each adolescent participant's AEP and the age-normative, average AEP waveforms calculated from NT-C and NT-A to study differences in signal morphology. We found that peak amplitudes of neural responses were significantly smaller in autistic adolescents compared to NT-A. We also found that the AEP morphology of autistic adolescents looked more like NT-A peers than NT-C but was still significantly different from NT-A AEP waveforms. Results suggest that AEPs of autistic adolescents present differently from NTs, regardless of age, and differences cannot be accounted for by developmental delay. Nonverbal intelligence significantly predicted how closely each adolescent's AEP resembled the age-normed waveform. These results support an evolving theory that the degree of disruption in early neural responses to low-level inputs is reflected in the severity of intellectual impairments in autism.

Evaluating feasibility of functional near-infrared spectroscopy in dolphins.

Significance: Using functional near-infrared spectroscopy (fNIRS) in bottlenose dolphins (Tursiops truncatus) could help to understand how echolocating animals perceive their environment and how they focus on specific auditory objects, such as fish, in noisy marine settings. Aim: To test the feasibility of near-infrared spectroscopy (NIRS) in medium-sized marine mammals, such as dolphins, we modeled the light propagation with computational tools to determine the wavelengths, optode locations, and separation distances that maximize sensitivity to brain tissue. Approach: Using frequency-domain NIRS, we measured the absorption and reduced scattering coefficient of dolphin sculp. We assigned muscle, bone, and brain optical properties from the literature and modeled light propagation in a spatially accurate and biologically relevant model of a dolphin head, using finite-element modeling. We assessed tissue sensitivities for a range of wavelengths (600 to 1700 nm), source-detector distances (50 to 120 mm), and animal sizes (juvenile model 25% smaller than adult). Results: We found that the wavelengths most suitable for imaging the brain fell into two ranges: 700 to 900 nm and 1100 to 1150 nm. The optimal location for brain sensing positioned the center point between source and detector 30 to 50 mm caudal of the blowhole and at an angle 45 deg to 90 deg lateral off the midsagittal plane. Brain tissue sensitivity comparable to human measurements appears achievable only for smaller animals, such as juvenile bottlenose dolphins or smaller species of cetaceans, such as porpoises, or with source-detector separations ≫100 mm in adult dolphins. Conclusions: Brain measurements in juvenile or subadult dolphins, or smaller dolphin species, may be possible using specialized fNIRS devices that support optode separations of >100 mm. We speculate that many measurement repetitions will be required to overcome hemodynamic signals originating predominantly from the muscle layer above the skull. NIRS measurements of muscle tissue are feasible today with source-detector separations of 50 mm, or even less.

Diffuse optical tomography spatial prior for EEG source localization in human visual cortex.

Electroencephalography (EEG) and diffuse optical tomography (DOT) are imaging methods which are widely used for neuroimaging. While the temporal resolution of EEG is high, the spatial resolution is typically limited. DOT, on the other hand, has high spatial resolution, but the temporal resolution is inherently limited by the slow hemodynamics it measures. In our previous work, we showed using computer simulations that when using the results of DOT reconstruction as the spatial prior for EEG source reconstruction, high spatio-temporal resolution could be achieved. In this work, we experimentally validate the algorithm by alternatingly flashing two visual stimuli at a speed that is faster than the temporal resolution of DOT. We show that the joint reconstruction using both EEG and DOT clearly resolves the two stimuli temporally, and the spatial confinement is drastically improved in comparison to reconstruction using EEG alone.

Induced alpha and beta electroencephalographic rhythms covary with single-trial speech intelligibility in competition.

Neurophysiological studies suggest that intrinsic brain oscillations influence sensory processing, especially of rhythmic stimuli like speech. Prior work suggests that brain rhythms may mediate perceptual grouping and selective attention to speech amidst competing sound, as well as more linguistic aspects of speech processing like predictive coding. However, we know of no prior studies that have directly tested, at the single-trial level, whether brain oscillations relate to speech-in-noise outcomes. Here, we combined electroencephalography while simultaneously measuring intelligibility of spoken sentences amidst two different interfering sounds: multi-talker babble or speech-shaped noise. We find that induced parieto-occipital alpha (7-15 Hz; thought to modulate attentional focus) and frontal beta (13-30 Hz; associated with maintenance of the current sensorimotor state and predictive coding) oscillations covary with trial-wise percent-correct scores; importantly, alpha and beta power provide significant independent contributions to predicting single-trial behavioral outcomes. These results can inform models of speech processing and guide noninvasive measures to index different neural processes that together support complex listening.

Statistical learning across passive listening adjusts perceptual weights of speech input dimensions.

Statistical learning across passive exposure has been theoretically situated with unsupervised learning. However, when input statistics accumulate over established representations - like speech syllables, for example - there is the possibility that prediction derived from activation of rich, existing representations may support error-driven learning. Here, across five experiments, we present evidence for error-driven learning across passive speech listening. Young adults passively listened to a string of eight beer - pier speech tokens with distributional regularities following either a canonical American-English acoustic dimension correlation or a correlation reversed to create an accent. A sequence-final test stimulus assayed the perceptual weight - the effectiveness - of the secondary dimension in signaling category membership as a function of preceding sequence regularities. Perceptual weight flexibly adjusted according to the passively experienced regularities even when the preceding regularities shifted on a trial-by-trial basis. The findings align with a theoretical view that activation of established internal representations can support learning across statistical regularities via error-driven learning. At the broadest level, this suggests that not all statistical learning need be unsupervised. Moreover, these findings help to account for how cognitive systems may accommodate competing demands for flexibility and stability: instead of overwriting existing representations when short-term input distributions depart from the norms, the mapping from input to category representations may be dynamically - and rapidly - adjusted via error-driven learning from predictions derived from internal representations.

Brainstem auditory physiology in children with listening difficulties.

Children who have listening difficulties (LiD) despite having normal audiometry are often diagnosed as having an auditory processing disorder. A lack of evidence regarding involvement of specific auditory mechanisms has limited development of effective treatments for these children. Here, we examined electrophysiologic evidence for brainstem pathway mechanisms in children with and without defined LiD. We undertook a prospective controlled study of 132 children aged 6-14 years with normal pure tone audiometry, grouped into LiD (n = 63) or Typically Developing (TD; n = 69) based on scores on the Evaluation of Children's Listening and Processing Skills (ECLiPS), a validated caregiver report. The groups were matched on age at test, sex, race, and ethnicity. Neither group had diagnoses of major neurologic disorder, intellectual disability, or brain injuries. Both groups received a test battery including a measure of receptive speech perception against distractor speech, Listening in Spatialized Noise - Sentences (LiSN-S), along with multiple neurophysiologic measures that tap afferent and efferent auditory subcortical pathways. Group analysis showed that participants with LiD performed significantly poorer on all subtests of the LiSN-S. The LiD group had significantly greater wideband middle ear muscle reflex (MEMR) growth functions in the left ear, and shorter Wave III and Wave V latencies in auditory brainstem responses (ABR). Across individual participants, shorter latency ABR Wave V correlated significantly with poorer parent report of LiD (ECLiPS composite). Greater MEMR growth functions also correlated with poorer ECLiPS scores and reduced LiSN-S talker advantage. The LiD and TD groups had equivalent summating potentials, compound action potentials, envelope-following responses, and binaurally activated medial olivocochlear reflexes. In conclusion, there was no evidence for auditory synaptopathy for LiD. Evidence for brainstem differences in the LiD group was interpreted as increased central gain, with shorter ABR Wave III and V latencies and steeper MEMR growth curves. These differences were related to poorer parent report and speech perception in competing speech ability.

Induced Alpha And Beta Electroencephalographic Rhythms Covary With Single-Trial Speech Intelligibility In Competition.

Neurophysiological studies suggest that intrinsic brain oscillations influence sensory processing, especially of rhythmic stimuli like speech. Prior work suggests that brain rhythms may mediate perceptual grouping and selective attention to speech amidst competing sound, as well as more linguistic aspects of speech processing like predictive coding. However, we know of no prior studies that have directly tested, at the single-trial level, whether brain oscillations relate to speech-in-noise outcomes. Here, we combined electroencephalography while simultaneously measuring intelligibility of spoken sentences amidst two different interfering sounds: multi-talker babble or speech-shaped noise. We find that induced parieto-occipital alpha (7-15 Hz; thought to modulate attentional focus) and frontal beta (13-30 Hz; associated with maintenance of the current sensorimotor state and predictive coding) oscillations covary with trial-wise percent-correct scores; importantly, alpha and beta power provide significant independent contributions to predicting single-trial behavioral outcomes. These results can inform models of speech processing and guide noninvasive measures to index different neural processes that together support complex listening.

Defining attention from an auditory perspective.

Attention prioritizes certain information at the expense of other information in ways that are similar across vision, audition, and other sensory modalities. It influences how-and even what-information is represented and processed, affecting brain activity at every level. Much of the core research into cognitive and neural mechanisms of attention has used visual tasks. However, the same top-down, object-based, and bottom-up attentional processes shape auditory perception, largely through the same underlying, cognitive networks. This article is categorized under: Psychology > Attention.

Top-down auditory attention modulates neural responses more strongly in neurotypical than ADHD young adults.

Human cognitive abilities naturally vary along a spectrum, even among those we call "neurotypical". Individuals differ in their ability to selectively attend to goal-relevant auditory stimuli. We sought to characterize this variability in a cohort of people with diverse attentional functioning. We recruited both neurotypical (N = 20) and ADHD (N = 25) young adults, all with normal hearing. Participants listened to one of three concurrent, spatially separated speech streams and reported the order of the syllables in that stream while we recorded electroencephalography (EEG). We tested both the ability to sustain attentional focus on a single "Target" stream and the ability to monitor the Target but flexibly either ignore or switch attention to an unpredictable "Interrupter" stream from another direction that sometimes appeared. Although differences in both stimulus structure and task demands affected behavioral performance, ADHD status did not. In both groups, the Interrupter evoked larger neural responses when it was to be attended compared to when it was irrelevant, including for the P3a "reorienting" response previously described as involuntary. This attentional modulation was weaker in ADHD listeners, even though their behavioral performance was the same. Across the entire cohort, individual performance correlated with the degree of top-down modulation of neural responses. These results demonstrate that listeners differ in their ability to modulate neural representations of sound based on task goals, while suggesting that adults with ADHD may have weaker volitional control of attentional processes than their neurotypical counterparts.

Alternating current stimulation entrains and connects cortical regions in a neural mass model.

Transcranial alternating current stimulation (tACS) is a neuromodulatory technique that is widely used to investigate the functions of oscillations in the brain. Despite increasing usage in both research and clinical settings, the mechanisms of tACS are still not completely understood. To shed light on these mechanisms, we injected alternating current into a Jansen and Rit neural mass model. Two cortical columns were linked with long-range connections to examine how alternating current impacted cortical connectivity. Alternating current injected to both columns increased power and coherence at the stimulation frequency; however this effect was greatest at the model's resonant frequency. Varying the phase of stimulation impacted the time it took for entrainment to stabilize, an effect we believe is due to constructive and destructive inteference with endogenous membrane currents. The power output the model also depended on the phase of the stimulation between cortical columns. These results provide insight on the mechanisms of neurostimulation, by demonstrating that tACS increases both power and coherence at a neural network's resonant frequency, in a phase-dependent manner.

Cat-astrophic effects of sudden interruptions on spatial auditory attention.

Salient interruptions draw attention involuntarily. Here, we explored whether this effect depends on the spatial and temporal relationships between a target stream and interrupter. In a series of online experiments, listeners focused spatial attention on a target stream of spoken syllables in the presence of an otherwise identical distractor stream from the opposite hemifield. On some random trials, an interrupter (a cat "MEOW") occurred. Experiment 1 established that the interrupter, which occurred randomly in 25% of the trials in the hemifield opposite the target, degraded target recall. Moreover, a majority of participants exhibited this degradation for the first target syllable, which finished before the interrupter began. Experiment 2 showed that the effect of an interrupter was similar whether it occurred in the opposite or the same hemifield as the target. Experiment 3 found that the interrupter degraded performance slightly if it occurred before the target stream began but had no effect if it began after the target stream ended. Experiment 4 showed decreased interruption effects when the interruption frequency increased (50% of the trials). These results demonstrate that a salient interrupter disrupts recall of a target stream, regardless of its direction, especially if it occurs during a target stream.

Extended Frontal Networks for Visual and Auditory Working Memory.

Working memory (WM) supports the persistent representation of transient sensory information. Visual and auditory stimuli place different demands on WM and recruit different brain networks. Separate auditory- and visual-biased WM networks extend into the frontal lobes, but several challenges confront attempts to parcellate human frontal cortex, including fine-grained organization and between-subject variability. Here, we use differential intrinsic functional connectivity from 2 visual-biased and 2 auditory-biased frontal structures to identify additional candidate sensory-biased regions in frontal cortex. We then examine direct contrasts of task functional magnetic resonance imaging during visual versus auditory 2-back WM to validate those candidate regions. Three visual-biased and 5 auditory-biased regions are robustly activated bilaterally in the frontal lobes of individual subjects (N = 14, 7 women). These regions exhibit a sensory preference during passive exposure to task stimuli, and that preference is stronger during WM. Hierarchical clustering analysis of intrinsic connectivity among novel and previously identified bilateral sensory-biased regions confirms that they functionally segregate into visual and auditory networks, even though the networks are anatomically interdigitated. We also observe that the frontotemporal auditory WM network is highly selective and exhibits strong functional connectivity to structures serving non-WM functions, while the frontoparietal visual WM network hierarchically merges into the multiple-demand cognitive system.

Cutting Through the Noise: Noise-Induced Cochlear Synaptopathy and Individual Differences in Speech Understanding Among Listeners With Normal Audiograms.

Following a conversation in a crowded restaurant or at a lively party poses immense perceptual challenges for some individuals with normal hearing thresholds. A number of studies have investigated whether noise-induced cochlear synaptopathy (CS; damage to the synapses between cochlear hair cells and the auditory nerve following noise exposure that does not permanently elevate hearing thresholds) contributes to this difficulty. A few studies have observed correlations between proxies of noise-induced CS and speech perception in difficult listening conditions, but many have found no evidence of a relationship. To understand these mixed results, we reviewed previous studies that have examined noise-induced CS and performance on speech perception tasks in adverse listening conditions in adults with normal or near-normal hearing thresholds. Our review suggests that superficially similar speech perception paradigms used in previous investigations actually placed very different demands on sensory, perceptual, and cognitive processing. Speech perception tests that use low signal-to-noise ratios and maximize the importance of fine sensory details- specifically by using test stimuli for which lexical, syntactic, and semantic cues do not contribute to performance-are more likely to show a relationship to estimated CS levels. Thus, the current controversy as to whether or not noise-induced CS contributes to individual differences in speech perception under challenging listening conditions may be due in part to the fact that many of the speech perception tasks used in past studies are relatively insensitive to CS-induced deficits.

Speech Categorization Reveals the Role of Early-Stage Temporal-Coherence Processing in Auditory Scene Analysis.

Temporal coherence of sound fluctuations across spectral channels is thought to aid auditory grouping and scene segregation. Although prior studies on the neural bases of temporal-coherence processing focused mostly on cortical contributions, neurophysiological evidence suggests that temporal-coherence-based scene analysis may start as early as the cochlear nucleus (i.e., the first auditory region supporting cross-channel processing over a wide frequency range). Accordingly, we hypothesized that aspects of temporal-coherence processing that could be realized in early auditory areas may shape speech understanding in noise. We then explored whether physiologically plausible computational models could account for results from a behavioral experiment that measured consonant categorization in different masking conditions. We tested whether within-channel masking of target-speech modulations predicted consonant confusions across the different conditions and whether predictions were improved by adding across-channel temporal-coherence processing mirroring the computations known to exist in the cochlear nucleus. Consonant confusions provide a rich characterization of error patterns in speech categorization, and are thus crucial for rigorously testing models of speech perception; however, to the best of our knowledge, they have not been used in prior studies of scene analysis. We find that within-channel modulation masking can reasonably account for category confusions, but that it fails when temporal fine structure cues are unavailable. However, the addition of across-channel temporal-coherence processing significantly improves confusion predictions across all tested conditions. Our results suggest that temporal-coherence processing strongly shapes speech understanding in noise and that physiological computations that exist early along the auditory pathway may contribute to this process.SIGNIFICANCE STATEMENT Temporal coherence of sound fluctuations across distinct frequency channels is thought to be important for auditory scene analysis. Prior studies on the neural bases of temporal-coherence processing focused mostly on cortical contributions, and it was unknown whether speech understanding in noise may be shaped by across-channel processing that exists in earlier auditory areas. Using physiologically plausible computational modeling to predict consonant confusions across different listening conditions, we find that across-channel temporal coherence contributes significantly to scene analysis and speech perception and that such processing may arise in the auditory pathway as early as the brainstem. By virtue of providing a richer characterization of error patterns not obtainable with just intelligibility scores, consonant confusions yield unique insight into scene analysis mechanisms.

Decoding auditory attention from EEG using a convolutional neural network.

Brain-computer interface (BCI) systems allow users to communicate directly with a device using their brain. BCI devices leveraging electroencephalography (EEG) signals as a means of communication typically use manual feature engineering on the data to perform decoding. This approach is time intensive, requires substantial domain knowledge, and does not translate well, even to similar tasks. To combat this issue, we designed a convolutional neural network (CNN) model to perform decoding on EEG data collected from an auditory attention paradigm. Our CNN model not only bypasses the need for manual feature engineering, but additionally improves decoding accuracy (∼77%) and efficiency (∼11 bits/min) compared to a support vector machine (SVM) baseline. The results demonstrate the potential for the use of CNN in auditory BCI designs.

Spatial alignment between faces and voices improves selective attention to audio-visual speech.

The ability to see a talker's face improves speech intelligibility in noise, provided that the auditory and visual speech signals are approximately aligned in time. However, the importance of spatial alignment between corresponding faces and voices remains unresolved, particularly in multi-talker environments. In a series of online experiments, we investigated this using a task that required participants to selectively attend a target talker in noise while ignoring a distractor talker. In experiment 1, we found improved task performance when the talkers' faces were visible, but only when corresponding faces and voices were presented in the same hemifield (spatially aligned). In experiment 2, we tested for possible influences of eye position on this result. In auditory-only conditions, directing gaze toward the distractor voice reduced performance, but this effect could not fully explain the cost of audio-visual (AV) spatial misalignment. Lowering the signal-to-noise ratio (SNR) of the speech from +4 to -4 dB increased the magnitude of the AV spatial alignment effect (experiment 3), but accurate closed-set lipreading caused a floor effect that influenced results at lower SNRs (experiment 4). Taken together, these results demonstrate that spatial alignment between faces and voices contributes to the ability to selectively attend AV speech.

Temporal fine structure influences voicing confusions for consonant identification in multi-talker babble.

To understand the mechanisms of speech perception in everyday listening environments, it is important to elucidate the relative contributions of different acoustic cues in transmitting phonetic content. Previous studies suggest that the envelope of speech in different frequency bands conveys most speech content, while the temporal fine structure (TFS) can aid in segregating target speech from background noise. However, the role of TFS in conveying phonetic content beyond what envelopes convey for intact speech in complex acoustic scenes is poorly understood. The present study addressed this question using online psychophysical experiments to measure the identification of consonants in multi-talker babble for intelligibility-matched intact and 64-channel envelope-vocoded stimuli. Consonant confusion patterns revealed that listeners had a greater tendency in the vocoded (versus intact) condition to be biased toward reporting that they heard an unvoiced consonant, despite envelope and place cues being largely preserved. This result was replicated when babble instances were varied across independent experiments, suggesting that TFS conveys voicing information beyond what is conveyed by envelopes for intact speech in babble. Given that multi-talker babble is a masker that is ubiquitous in everyday environments, this finding has implications for the design of assistive listening devices such as cochlear implants.

Modulation masking and fine structure shape neural envelope coding to predict speech intelligibility across diverse listening conditions.

A fundamental question in the neuroscience of everyday communication is how scene acoustics shape the neural processing of attended speech sounds and in turn impact speech intelligibility. While it is well known that the temporal envelopes in target speech are important for intelligibility, how the neural encoding of target-speech envelopes is influenced by background sounds or other acoustic features of the scene is unknown. Here, we combine human electroencephalography with simultaneous intelligibility measurements to address this key gap. We find that the neural envelope-domain signal-to-noise ratio in target-speech encoding, which is shaped by masker modulations, predicts intelligibility over a range of strategically chosen realistic listening conditions unseen by the predictive model. This provides neurophysiological evidence for modulation masking. Moreover, using high-resolution vocoding to carefully control peripheral envelopes, we show that target-envelope coding fidelity in the brain depends not only on envelopes conveyed by the cochlea, but also on the temporal fine structure (TFS), which supports scene segregation. Our results are consistent with the notion that temporal coherence of sound elements across envelopes and/or TFS influences scene analysis and attentive selection of a target sound. Our findings also inform speech-intelligibility models and technologies attempting to improve real-world speech communication.