EEG alpha and pupil diameter reflect endogenous auditory attention switching and listening effort.

Everyday environments often contain distracting competing talkers and background noise, requiring listeners to focus their attention on one acoustic source and reject others. During this auditory attention task, listeners may naturally interrupt their sustained attention and switch attended sources. The effort required to perform this attention switch has not been well studied in the context of competing continuous speech. In this work, we developed two variants of endogenous attention switching and a sustained attention control. We characterized these three experimental conditions under the context of decoding auditory attention, while simultaneously evaluating listening effort and neural markers of spatial-audio cues. A least-squares, electroencephalography (EEG)-based, attention decoding algorithm was implemented across all conditions. It achieved an accuracy of 69.4% and 64.0% when computed over nonoverlapping 10 and 5-s correlation windows, respectively. Both decoders illustrated smooth transitions in the attended talker prediction through switches at approximately half of the analysis window size (e.g., the mean lag taken across the two switch conditions was 2.2 s when the 5-s correlation window was used). Expended listening effort, as measured by simultaneous EEG and pupillometry, was also a strong indicator of whether the listeners sustained attention or performed an endogenous attention switch (peak pupil diameter measure [ p=0.034 ] and minimum parietal alpha power measure [ p=0.016 ]). We additionally found evidence of talker spatial cues in the form of centrotemporal alpha power lateralization ( p=0.0428 ). These results suggest that listener effort and spatial cues may be promising features to pursue in a decoding context, in addition to speech-based features.

Prospective measurements of hearing threshold during military rifle training with in-ear, protected, noise exposure monitoring.

Although a causal relationship exists between military occupational noise exposure and hearing loss, researchers have struggled to identify and/or characterize specific operational noise exposures that produce measurable changes in hearing function shortly following an exposure. Growing evidence suggests that current standards for noise-exposure limits are not good predictors of true hearing damage. In this study, the aim was to capture the dose-response relationship during military rifle training exercises for noise exposure and hearing threshold. To capture exposure, a wearable system capable of measuring impulse noise simultaneously on-body and in-ear, behind hearing protection was used. To characterize hearing threshold changes, portable audiometry was employed within 2 h before and after exposure. The median 8-h time-weighted, protected, free-field equivalent in-ear exposure was 87.5 dBA at one site and 80.7 dBA at a second site. A significant dose-response correlation between in-ear noise exposure and postexposure hearing threshold changes across our population ( R = 0.40 , p = 0.0281) was observed. The results demonstrate an approach for establishing damage risk criteria (DRC) for in-ear, protected measurements based on hearing threshold changes. While an in-ear DRC does not currently exist, it may be critical for predicting the risk of injury for noise environments where protection is mandatory and fit status can vary.

The Effect of Hearing-Protection Devices on Auditory Situational Awareness and Listening Effort.

OBJECTIVES: Hearing-protection devices (HPDs) are made available, and often are required, for industrial use as well as military training exercises and operational duties. However, these devices often are disliked, and consequently not worn, in part because they compromise situational awareness through reduced sound detection and localization performance as well as degraded speech intelligibility. In this study, we carried out a series of tests, involving normal-hearing subjects and multiple background-noise conditions, designed to evaluate the performance of four HPDs in terms of their modifications of auditory-detection thresholds, sound-localization accuracy, and speech intelligibility. In addition, we assessed their impact on listening effort to understand how the additional effort required to perceive and process auditory signals while wearing an HPD reduces available cognitive resources for other tasks. DESIGN: Thirteen normal-hearing subjects participated in a protocol, which included auditory tasks designed to measure detection and localization performance, speech intelligibility, and cognitive load. Each participant repeated the battery of tests with unoccluded ears and four hearing protectors, two active (electronic) and two passive. The tasks were performed both in quiet and in background noise. RESULTS: Our findings indicate that, in variable degrees, all of the tested HPDs induce performance degradation on most of the conducted tasks as compared to the open ear. Of particular note in this study is the finding of increased cognitive load or listening effort, as measured by visual reaction time, for some hearing protectors during a dual-task, which added working-memory demands to the speech-intelligibility task. CONCLUSIONS: These results indicate that situational awareness can vary greatly across the spectrum of HPDs, and that listening effort is another aspect of performance that should be considered in future studies. The increased listening effort induced by hearing protectors may lead to earlier cognitive fatigue in noisy environments. Further study is required to characterize how auditory performance is limited by the combination of hearing impairment and the use of HPDs, and how the effects of such limitations can be linked to safe and effective use of hearing protection to maximize job performance.

Characterization of acute hearing changes in United States military populations.

Until recently, most hearing conservation programs, including those in the military, have used permanent shifts in the pure-tone audiometric threshold as the gold standard for measuring hearing impairment in noise-exposed populations. However, recent results from animal studies suggest that high-level noise exposures can cause the permanent destruction of synapses between the inner hair cells and auditory nerve fibers, even in cases where pure-tone audiometric thresholds eventually return to their normal pre-exposure baselines. This has created a dilemma for researchers, who are now increasingly interested in studying the long-term effects that temporary hearing shifts might have on hearing function, but are also concerned about the ethical considerations of exposing human listeners to high levels of noise for research purposes. One method that remains viable to study the effects of high noise exposures on human listeners, or to evaluate the efficacy of interventions designed to prevent noise-related inner ear damage, is to identify individuals in occupations with unavoidable noise exposures and measure hearing before and as soon as possible after exposure. This paper discusses some of the important factors to be considered in studies that attempt to measure acute hearing changes in noise-exposed military populations.

In-ear and on-body measurements of impulse-noise exposure.

Accurate quantification of noise exposure in military environments is challenging due to movement of listeners and noise sources, spectral and temporal noise characteristics, and varied use of hearing protection. This study evaluates a wearable recording device designed to measure on-body and in-ear noise exposure, specifically in an environment with significant impulse noise resulting from firearms. A commercial audio recorder was augmented to obtain simultaneous measurements inside the ear canal behind an integrated hearing protector, and near the outer ear. Validation measurements, conducted with an acoustic test fixture and shock tube, indicated high impulse peak insertion loss with a proper fit of the integrated hearing protector. The recording devices were worn by five subjects during a live-fire data collection at Marine Corps Base Quantico where Marines fired semi-automatic rifles. The field test demonstrated the successful measurement of high-level impulse waveforms with the on-body and in-ear recording system. Dual channels allowed for instantaneous fit estimates for the hearing protection component, and the device worked as intended in terms of hearing protection and noise dosimetry. Accurate measurements of noise exposure and hearing protector fit should improve the ability to model and assess the risks of noise-induced hearing loss.

Neural correlates of adaptation in freely-moving normal hearing subjects under cochlear implant acoustic simulations.

Neurobiological correlates of adaptation to spectrally degraded speech were investigated with fMRI before and after exposure to a portable real-time speech processor that implements an acoustic simulation model of a cochlear implant (CI). The speech processor, in conjunction with isolating insert earphones and a microphone to capture environment sounds, was worn by participants over a two week chronic exposure period. fMRI and behavioral speech comprehension testing were conducted before and after this two week period. After using the simulator each day for 2h, participants significantly improved in word and sentence recognition scores. fMRI shows that these improvements came accompanied by changes in patterns of neuronal activation. In particular, we found additional recruitment of visual, motor, and working memory areas after the perceptual training period. These findings suggest that the human brain is able to adapt in a short period of time to a degraded auditory signal under a natural learning environment, and gives insight on how a CI might interact with the central nervous system. This paradigm can be furthered to investigate neural correlates of new rehabilitation, training, and signal processing strategies non-invasively in normal hearing listeners to improve CI patient outcomes.

Changes in Eye Tracking Features Across Periods of Overpressure Exposure.

INTRODUCTION: Repetitive exposure to blast overpressure waves can be a part of routine military and law enforcement training. However, our understanding of the effects of that repetitive exposure on human neurophysiology remains limited. To link an individual's cumulative exposure with their neurophysiological effects, overpressure dosimetry needs to be concurrently collected with relevant physiological signals. Eye tracking has shown promise for providing insight into neurophysiological change because of neural injury, but video-based technology limits usage to a laboratory or clinic. In the present work, we show capability for using electrooculography-based eye tracking to enable physiological assessment in the field during activities involved repetitive blast exposures. MATERIALS AND METHODS: Overpressure dosimetry was accomplished by using a body-worn measurement system that captures continuous sound pressure levels as well as pressure waveforms of blast event in the range of 135-185 dB peak (0.1-36 kPa). Electrooculography eye tracking was performed using a commercial Shimmer Sensing system, which captured horizontal eye movements of both the left and right eyes, as well as vertical eye movements of the right eye, from which blinks can also be extracted. Data were collected during breaching activities that included repetitive use of explosives. Participants in the study were U.S. Army Special Operators and Federal Bureau of Investigations special agents. Approval for research was received by the Massachucetts Institute of Technology Committee on the Use of Humans as Experimental Subjects, the Air Force Human Research Protections Office, and the Federal Bureau of Investigations Institutional Review Board. RESULTS: The energy from overpressure events was accumulated and summarized into an 8-hour equivalent of sound pressure level (i.e., LZeq8hr). The total exposure in a single day, i.e., the LZeq8hr, ranged from 110 to 160 dB. Oculomotor features, such as blink and saccade rate, as well as variance in blink waveforms, show changes across the period of overpressure exposure. However, the features that showed significant change across the population were not necessarily the ones that showed significant correlation with the levels of overpressure exposure. A regression model built to predict overpressure levels from oculomotor features alone showed a significant association (R = 0.51, P < .01). Investigation of the model indicates that changes in the saccade rate and blink waveforms are driving the relationship. CONCLUSIONS: This study successfully demonstrated that eye tracking can be performed during training activities, such as explosive breaching, and that the modality may provide insight into neurophysiological change across periods of overpressure exposure. The results presented herein show that electrooculography-based eye tracking may be a useful method of assessing individualized physiological effects of overpressure exposure in the field. Future work is focused on time-dependent modeling to assess continuous changes in eye movements as this will enable building dose-response curves.

Using Body-worn Accelerometers to Detect Physiological Changes During Periods of Blast Overpressure Exposure.

Repetitive exposure to non-concussive blast expo-sure may result in sub-clinical neurological symptoms. These changes may be reflected in the neural control gait and balance. In this study, we collected body-worn accelerometry data on individuals who were exposed to repetitive blast overpressures as part of their occupation. Accelerometry features were gener-ated within periods of low-movement and gait. These features were the eigenvalues of high-dimensional correlation matrices, which were constructed with time-delay embedding at multiple delay scales. When focusing on the gait windows, there were significant correlations of the changes in features with the cumulative dose of blast exposure. When focusing on the low-movement frames, the correlation with exposure were lower than that of the gait frames and statistically insignificant. In a cross-validated model, the overpressure exposure was predicted from gait features alone. The model was statistically significant and yielded an RMSE of 1.27 dB. With continued development, the model may be used to assess the physiological effects of repetitive blast exposure and guide training procedures to minimize impact on the individual.

Automatic identification of tinnitus malingering based on overt and covert behavioral responses during psychoacoustic testing.

Tinnitus, or ringing in the ears, is a prevalent condition that imposes a substantial health and financial burden on the patient and to society. The diagnosis of tinnitus, like pain, relies on patient self-report, which can complicate the distinction between actual and fraudulent claims. Here, we combined tablet-based self-directed hearing assessments with neural network classifiers to automatically differentiate participants with tinnitus (N = 24) from a malingering cohort, who were instructed to feign an imagined tinnitus percept (N = 28). We identified clear differences between the groups, both in their overt reporting of tinnitus features, but also covert differences in their fingertip movement trajectories on the tablet surface as they performed the reporting assay. Using only 10 min of data, we achieved 81% accuracy classifying patients and malingerers (ROC AUC = 0.88) with leave-one-out cross validation. Quantitative, automated measurements of tinnitus salience could improve clinical outcome assays and more accurately determine tinnitus incidence.

Development and Evaluation of a Body-Worn Dosimeter for Continuous and Impulsive Noise.

Noise exposure is encountered nearly everyday in both recreational and occupational settings, and can lead to a number of health concerns including hearing-loss, tinnitus, social-isolation and possibly dementia. Although guidelines exist to protect workers from noise, it remains a challenge to accurately quantify the noise exposure experienced by an individual due to the complexity and non-stationarity of noise sources. This is especially true for impulsive noise sources, such as weapons fire and industrial impact noise which are difficult to quantify due to technical challenges relating to sensor design and size, weight and power requirements. Because of this, personal noise dosimeters are often limited to a maximum 140 dB SPL and are not sufficient to measure impulse noise. This work details the design of a body-worn noise dosimeter (mNOISE) that processes both impulse and continuous noise ranging in level from 40 dBA-185 dBP (i.e. a quiet whisper to a shoulder fired rocket). Also detailed is the capability of the device to log the kurtosis of the sound pressure waveform in real-time, which is thought to be useful in characterizing complex noise exposures. Finally, we demonstrate the use of mNOISE in a military-flight noise environment. Clinical Relevance- On-body noise exposure monitoring can be used by audiologists industrial hygiene personnel and others to determine threshold of injury adequate hearing protection requirements and ultimately reduce permanent noise-induced hearing loss.

Speaker separation in realistic noise environments with applications to a cognitively-controlled hearing aid.

Future wearable technology may provide for enhanced communication in noisy environments and for the ability to pick out a single talker of interest in a crowded room simply by the listener shifting their attentional focus. Such a system relies on two components, speaker separation and decoding the listener's attention to acoustic streams in the environment. To address the former, we present a system for joint speaker separation and noise suppression, referred to as the Binaural Enhancement via Attention Masking Network (BEAMNET). The BEAMNET system is an end-to-end neural network architecture based on self-attention. Binaural input waveforms are mapped to a joint embedding space via a learned encoder, and separate multiplicative masking mechanisms are included for noise suppression and speaker separation. Pairs of output binaural waveforms are then synthesized using learned decoders, each capturing a separated speaker while maintaining spatial cues. A key contribution of BEAMNET is that the architecture contains a separation path, an enhancement path, and an autoencoder path. This paper proposes a novel loss function which simultaneously trains these paths, so that disabling the masking mechanisms during inference causes BEAMNET to reconstruct the input speech signals. This allows dynamic control of the level of suppression applied by BEAMNET via a minimum gain level, which is not possible in other state-of-the-art approaches to end-to-end speaker separation. This paper also proposes a perceptually-motivated waveform distance measure. Using objective speech quality metrics, the proposed system is demonstrated to perform well at separating two equal-energy talkers, even in high levels of background noise. Subjective testing shows an improvement in speech intelligibility across a range of noise levels, for signals with artificially added head-related transfer functions and background noise. Finally, when used as part of an auditory attention decoder (AAD) system using existing electroencephalogram (EEG) data, BEAMNET is found to maintain the decoding accuracy achieved with ideal speaker separation, even in severe acoustic conditions. These results suggest that this enhancement system is highly effective at decoding auditory attention in realistic noise environments, and could possibly lead to improved speech perception in a cognitively controlled hearing aid.

Deep Neural Network Model of Hearing-Impaired Speech-in-Noise Perception.

Many individuals struggle to understand speech in listening scenarios that include reverberation and background noise. An individual's ability to understand speech arises from a combination of peripheral auditory function, central auditory function, and general cognitive abilities. The interaction of these factors complicates the prescription of treatment or therapy to improve hearing function. Damage to the auditory periphery can be studied in animals; however, this method alone is not enough to understand the impact of hearing loss on speech perception. Computational auditory models bridge the gap between animal studies and human speech perception. Perturbations to the modeled auditory systems can permit mechanism-based investigations into observed human behavior. In this study, we propose a computational model that accounts for the complex interactions between different hearing damage mechanisms and simulates human speech-in-noise perception. The model performs a digit classification task as a human would, with only acoustic sound pressure as input. Thus, we can use the model's performance as a proxy for human performance. This two-stage model consists of a biophysical cochlear-nerve spike generator followed by a deep neural network (DNN) classifier. We hypothesize that sudden damage to the periphery affects speech perception and that central nervous system adaptation over time may compensate for peripheral hearing damage. Our model achieved human-like performance across signal-to-noise ratios (SNRs) under normal-hearing (NH) cochlear settings, achieving 50% digit recognition accuracy at -20.7 dB SNR. Results were comparable to eight NH participants on the same task who achieved 50% behavioral performance at -22 dB SNR. We also simulated medial olivocochlear reflex (MOCR) and auditory nerve fiber (ANF) loss, which worsened digit-recognition accuracy at lower SNRs compared to higher SNRs. Our simulated performance following ANF loss is consistent with the hypothesis that cochlear synaptopathy impacts communication in background noise more so than in quiet. Following the insult of various cochlear degradations, we implemented extreme and conservative adaptation through the DNN. At the lowest SNRs (<0 dB), both adapted models were unable to fully recover NH performance, even with hundreds of thousands of training samples. This implies a limit on performance recovery following peripheral damage in our human-inspired DNN architecture.

Predicting Cognitive Load and Operational Performance in a Simulated Marksmanship Task.

Modern operational environments can place significant demands on a service member's cognitive resources, increasing the risk of errors or mishaps due to overburden. The ability to monitor cognitive burden and associated performance within operational environments is critical to improving mission readiness. As a key step toward a field-ready system, we developed a simulated marksmanship scenario with an embedded working memory task in an immersive virtual reality environment. As participants performed the marksmanship task, they were instructed to remember numbered targets and recall the sequence of those targets at the end of the trial. Low and high cognitive load conditions were defined as the recall of three- and six-digit strings, respectively. Physiological and behavioral signals recorded included speech, heart rate, breathing rate, and body movement. These features were input into a random forest classifier that significantly discriminated between the low- and high-cognitive load conditions (AUC = 0.94). Behavioral features of gait were the most informative, followed by features of speech. We also showed the capability to predict performance on the digit recall (AUC = 0.71) and marksmanship (AUC = 0.58) tasks. The experimental framework can be leveraged in future studies to quantify the interaction of other types of stressors and their impact on operational cognitive and physical performance.

Predicting changes in performance due to cognitive fatigue: A multimodal approach based on speech motor coordination and electrodermal activity.

OBJECTIVE: Military job and training activities place significant demands on service members' (SMs') cognitive resources, increasing risk of injury and degrading performance. Early detection of cognitive fatigue is essential to reduce risk and support optimal function. This paper describes a multimodal approach, based on changes in measures of speech motor coordination and electrodermal activity (EDA), for predicting changes in performance following sustained cognitive effort. METHODS: Twenty-nine active duty SMs completed computer-based cognitive tasks for 2 h (load period). Measures of speech derived from audio were acquired, along with concurrent measures of EDA, before and after the load period. Cognitive performance was assessed before and during the load period using the Automated Neuropsychological Assessment Metrics Military Battery (ANAM MIL). Subjective assessments of cognitive effort and alertness were obtained intermittently. RESULTS: Across the load period, participants' ratings of cognitive workload increased, while alertness ratings declined. Cognitive performance declined significantly during the first half of the load period. Three speech and arousal features predicted cognitive performance changes during this period with statistically significant accuracy: EDA (r = 0.43, p = 0.01), articulator velocity coordination (r = 0.50, p = 0.00), and vocal creak (r = 0.35, p = 0.03). Fusing predictions from these features predicted performance changes with r = 0.68 (p = 0.00). CONCLUSIONS: Results suggest that speech and arousal measures may be used to predict changes in performance associated with cognitive fatigue. This work supports ongoing efforts to develop reliable, unobtrusive measures for cognitive state assessment aimed at reducing injury risk, informing return to work decisions, and supporting diverse mobile healthcare applications in civilian and military settings.

Comparison of Two-Talker Attention Decoding from EEG with Nonlinear Neural Networks and Linear Methods.

Auditory attention decoding (AAD) through a brain-computer interface has had a flowering of developments since it was first introduced by Mesgarani and Chang (2012) using electrocorticograph recordings. AAD has been pursued for its potential application to hearing-aid design in which an attention-guided algorithm selects, from multiple competing acoustic sources, which should be enhanced for the listener and which should be suppressed. Traditionally, researchers have separated the AAD problem into two stages: reconstruction of a representation of the attended audio from neural signals, followed by determining the similarity between the candidate audio streams and the reconstruction. Here, we compare the traditional two-stage approach with a novel neural-network architecture that subsumes the explicit similarity step. We compare this new architecture against linear and non-linear (neural-network) baselines using both wet and dry electroencephalogram (EEG) systems. Our results indicate that the new architecture outperforms the baseline linear stimulus-reconstruction method, improving decoding accuracy from 66% to 81% using wet EEG and from 59% to 87% for dry EEG. Also of note was the finding that the dry EEG system can deliver comparable or even better results than the wet, despite the latter having one third as many EEG channels as the former. The 11-subject, wet-electrode AAD dataset for two competing, co-located talkers, the 11-subject, dry-electrode AAD dataset, and our software are available for further validation, experimentation, and modification.

Noise dosimetry for tactical environments.

Noise exposure and the subsequent hearing loss are well documented aspects of military life. Numerous studies have indicated high rates of noise-induced hearing injury (NIHI) in active-duty service men and women, and recent statistics from the U.S. Department of Veterans Affairs indicate a population of veterans with hearing loss that is growing at an increasing rate. In an effort to minimize hearing loss, the U.S. Department of Defense (DoD) updated its Hearing Conservation Program in 2010, and also has recently revised the DoD Design Criteria Standard Noise Limits (MIL-STD-1474E) which defines allowable noise levels in the design of all military acquisitions including weapons and vehicles. Even with such mandates, it remains a challenge to accurately quantify the noise exposure experienced by a Warfighter over the course of a mission or training exercise, or even in a standard work day. Noise dosimeters are intended for exactly this purpose, but variations in device placement (e.g., free-field, on-body, in/near-ear), hardware (e.g., microphone, analog-to-digital converter), measurement time (e.g., work day, 24-h), and dose metric calculations (e.g., time-weighted energy, peak levels, Auditory Risk Units), as well as noise types (e.g., continuous, intermittent, impulsive) can cause exposure measurements to be incomplete, inaccurate, or inappropriate for a given situation. This paper describes the design of a noise dosimeter capable of acquiring exposure data across tactical environments. Two generations of prototypes have been built at MIT Lincoln Laboratory with funding from the U.S. Army, Navy, and Marine Corps. Details related to hardware, signal processing, and testing efforts are provided, along with example tactical military noise data and lessons learned from early fieldings. Finally, we discuss the continued need to prioritize personalized dosimetry in order to improve models that predict or characterize the risk of auditory damage, to integrate dosimeters with hearing-protection devices, and to inform strategies and metrics for reducing NIHI.

Modeling the time-varying and level-dependent effects of the medial olivocochlear reflex in auditory nerve responses.

The medial olivocochlear reflex (MOCR) has been hypothesized to provide benefit for listening in noisy environments. This advantage can be attributed to a feedback mechanism that suppresses auditory nerve (AN) firing in continuous background noise, resulting in increased sensitivity to a tone or speech. MOC neurons synapse on outer hair cells (OHCs), and their activity effectively reduces cochlear gain. The computational model developed in this study implements the time-varying, characteristic frequency (CF) and level-dependent effects of the MOCR within the framework of a well-established model for normal and hearing-impaired AN responses. A second-order linear system was used to model the time-course of the MOCR using physiological data in humans. The stimulus-level-dependent parameters of the efferent pathway were estimated by fitting AN sensitivity derived from responses in decerebrate cats using a tone-in-noise paradigm. The resulting model uses a binaural, time-varying, CF-dependent, level-dependent OHC gain reduction for both ipsilateral and contralateral stimuli that improves detection of a tone in noise, similarly to recorded AN responses. The MOCR may be important for speech recognition in continuous background noise as well as for protection from acoustic trauma. Further study of this model and its efferent feedback loop may improve our understanding of the effects of sensorineural hearing loss in noisy situations, a condition in which hearing aids currently struggle to restore normal speech perception.

Relationship between brainstem, cortical and behavioral measures relevant to pitch salience in humans.

Neural representation of pitch-relevant information at both the brainstem and cortical levels of processing is influenced by language or music experience. However, the functional roles of brainstem and cortical neural mechanisms in the hierarchical network for language processing, and how they drive and maintain experience-dependent reorganization are not known. In an effort to evaluate the possible interplay between these two levels of pitch processing, we introduce a novel electrophysiological approach to evaluate pitch-relevant neural activity at the brainstem and auditory cortex concurrently. Brainstem frequency-following responses and cortical pitch responses were recorded from participants in response to iterated rippled noise stimuli that varied in stimulus periodicity (pitch salience). A control condition using iterated rippled noise devoid of pitch was employed to ensure pitch specificity of the cortical pitch response. Neural data were compared with behavioral pitch discrimination thresholds. Results showed that magnitudes of neural responses increase systematically and that behavioral pitch discrimination improves with increasing stimulus periodicity, indicating more robust encoding for salient pitch. Absence of cortical pitch response in the control condition confirms that the cortical pitch response is specific to pitch. Behavioral pitch discrimination was better predicted by brainstem and cortical responses together as compared to each separately. The close correspondence between neural and behavioral data suggest that neural correlates of pitch salience that emerge in early, preattentive stages of processing in the brainstem may drive and maintain with high fidelity the early cortical representations of pitch. These neural representations together contain adequate information for the development of perceptual pitch salience.

Distortion products and their influence on representation of pitch-relevant information in the human brainstem for unresolved harmonic complex tones.

Pitch experiments aimed at evaluating temporal pitch mechanism(s) often utilize complex sounds with only unresolved harmonic components, and a low-pass noise masker to eliminate the potential contribution of audible distortion products to the pitch percept. Herein we examine how: (i) masker induced reduction of neural distortion products (difference tone: DT; and cubic difference tone: CDT) alters the representation of pitch relevant information in the brainstem; and (ii) the pitch salience is altered when distortion products are reduced and/or eliminated. Scalp recorded brainstem frequency following responses (FFR) were recorded in normal hearing individuals using a complex tone with only unresolved harmonics presented in quiet, and in the presence of a low-pass masker at SNRs of +15, +5, and -5 dB. Difference limen for F0 discrimination (F0 DL) was obtained in quiet and in the presence of low-pass noise. Magnitude of DT components (with the exception of components at F0 and 2F0), and the CDT components decreased with increasing masker level. Neural pitch strength decreased with increasing masker level for both the envelope-related (FFR(ENV)) and spectral-related (FFR(SPEC)) phase-locked activity. Finally, F0 DLs increased with decreasing SNRs suggesting poorer F0 discrimination with reduction of the distortion products. Collectively, these findings support the notion that both DT and CDT, as reflected in the FFR(ENV) and FFR(SPEC), respectively, influence both the brainstem representation of pitch relevant information and the pitch salience of the complex sounds.

Neural adaptation and perceptual learning using a portable real-time cochlear implant simulator in natural environments.

A portable real-time speech processor that implements an acoustic simulation model of a cochlear implant (CI) has been developed on the Apple iPhone / iPod Touch to permit testing and experimentation under extended exposure in real-world environments. This simulator allows for both a variable number of noise band channels and electrode insertion depth. Utilizing this portable CI simulator, we tested perceptual learning in normal hearing listeners by measuring word and sentence comprehension behaviorally before and after 2 weeks of exposure. To evaluate changes in neural activation related to adaptation to transformed speech, fMRI was also conducted. Differences in brain activation after training occurred in the inferior frontal gyrus and areas related to language processing. A 15-20% improvement in word and sentence comprehension of cochlear implant simulated speech was also observed. These results demonstrate the effectiveness of a portable CI simulator as a research tool and provide new information about the physiological changes that accompany perceptual learning of degraded auditory input.