Midbrain adaptation may set the stage for the perception of musical beat.

The ability to spontaneously feel a beat in music is a phenomenon widely believed to be unique to humans. Though beat perception involves the coordinated engagement of sensory, motor and cognitive processes in humans, the contribution of low-level auditory processing to the activation of these networks in a beat-specific manner is poorly understood. Here, we present evidence from a rodent model that midbrain preprocessing of sounds may already be shaping where the beat is ultimately felt. For the tested set of musical rhythms, on-beat sounds on average evoked higher firing rates than off-beat sounds, and this difference was a defining feature of the set of beat interpretations most commonly perceived by human listeners over others. Basic firing rate adaptation provided a sufficient explanation for these results. Our findings suggest that midbrain adaptation, by encoding the temporal context of sounds, creates points of neural emphasis that may influence the perceptual emergence of a beat.

State-dependent population coding in primary auditory cortex.

Sensory function is mediated by interactions between external stimuli and intrinsic cortical dynamics that are evident in the modulation of evoked responses by cortical state. A number of recent studies across different modalities have demonstrated that the patterns of activity in neuronal populations can vary strongly between synchronized and desynchronized cortical states, i.e., in the presence or absence of intrinsically generated up and down states. Here we investigated the impact of cortical state on the population coding of tones and speech in the primary auditory cortex (A1) of gerbils, and found that responses were qualitatively different in synchronized and desynchronized cortical states. Activity in synchronized A1 was only weakly modulated by sensory input, and the spike patterns evoked by tones and speech were unreliable and constrained to a small range of patterns. In contrast, responses to tones and speech in desynchronized A1 were temporally precise and reliable across trials, and different speech tokens evoked diverse spike patterns with extremely weak noise correlations, allowing responses to be decoded with nearly perfect accuracy. Restricting the analysis of synchronized A1 to activity within up states yielded similar results, suggesting that up states are not equivalent to brief periods of desynchronization. These findings demonstrate that the representational capacity of A1 depends strongly on cortical state, and suggest that cortical state should be considered as an explicit variable in all studies of sensory processing.

Nonlinear transfer of signal and noise correlations in cortical networks.

Signal and noise correlations, a prominent feature of cortical activity, reflect the structure and function of networks during sensory processing. However, in addition to reflecting network properties, correlations are also shaped by intrinsic neuronal mechanisms. Here we show that spike threshold transforms correlations by creating nonlinear interactions between signal and noise inputs; even when input noise correlation is constant, spiking noise correlation varies with both the strength and correlation of signal inputs. We characterize these effects systematically in vitro in mice and demonstrate their impact on sensory processing in vivo in gerbils. We also find that the effects of nonlinear correlation transfer on cortical responses are stronger in the synchronized state than in the desynchronized state, and show that they can be reproduced and understood in a model with a simple threshold nonlinearity. Since these effects arise from an intrinsic neuronal property, they are likely to be present across sensory systems and, thus, our results are a critical step toward a general understanding of how correlated spiking relates to the structure and function of cortical networks.

The neural representation of interaural time differences in gerbils is transformed from midbrain to cortex.

Interaural time differences (ITDs) are the dominant cue for the localization of low-frequency sounds. While much is known about the processing of ITDs in the auditory brainstem and midbrain, there have been relatively few studies of ITD processing in auditory cortex. In this study, we compared the neural representation of ITDs in the inferior colliculus (IC) and primary auditory cortex (A1) of gerbils. Our IC results were largely consistent with previous studies, with most cells responding maximally to ITDs that correspond to the contralateral edge of the physiological range. In A1, however, we found that preferred ITDs were distributed evenly throughout the physiological range without any contralateral bias. This difference in the distribution of preferred ITDs in IC and A1 had a major impact on the coding of ITDs at the population level: while a labeled-line decoder that considered the tuning of individual cells performed well on both IC and A1 responses, a two-channel decoder based on the overall activity in each hemisphere performed poorly on A1 responses relative to either labeled-line decoding of A1 responses or two-channel decoding of IC responses. These results suggest that the neural representation of ITDs in gerbils is transformed from IC to A1 and have important implications for how spatial location may be combined with other acoustic features for the analysis of complex auditory scenes.

Independent population coding of speech with sub-millisecond precision.

To understand the strategies used by the brain to analyze complex environments, we must first characterize how the features of sensory stimuli are encoded in the spiking of neuronal populations. Characterizing a population code requires identifying the temporal precision of spiking and the extent to which spiking is correlated, both between cells and over time. In this study, we characterize the population code for speech in the gerbil inferior colliculus (IC), the hub of the auditory system where inputs from parallel brainstem pathways are integrated for transmission to the cortex. We find that IC spike trains can carry information about speech with sub-millisecond precision, and, consequently, that the temporal correlations imposed by refractoriness can play a significant role in shaping spike patterns. We also find that, in contrast to most other brain areas, the noise correlations between IC cells are extremely weak, indicating that spiking in the population is conditionally independent. These results demonstrate that the problem of understanding the population coding of speech can be reduced to the problem of understanding the stimulus-driven spiking of individual cells, suggesting that a comprehensive model of the subcortical processing of speech may be attainable in the near future.

Temporal precision in the neural code and the timescales of natural vision.

The timing of action potentials relative to sensory stimuli can be precise down to milliseconds in the visual system, even though the relevant timescales of natural vision are much slower. The existence of such precision contributes to a fundamental debate over the basis of the neural code and, specifically, what timescales are important for neural computation. Using recordings in the lateral geniculate nucleus, here we demonstrate that the relevant timescale of neuronal spike trains depends on the frequency content of the visual stimulus, and that 'relative', not absolute, precision is maintained both during spatially uniform white-noise visual stimuli and naturalistic movies. Using information-theoretic techniques, we demonstrate a clear role of relative precision, and show that the experimentally observed temporal structure in the neuronal response is necessary to represent accurately the more slowly changing visual world. By establishing a functional role of precision, we link visual neuron function on slow timescales to temporal structure in the response at faster timescales, and uncover a straightforward purpose of fine-timescale features of neuronal spike trains.

Large-scale electrophysiology and deep learning reveal distorted neural signal dynamics after hearing loss.

Listeners with hearing loss often struggle to understand speech in noise, even with a hearing aid. To better understand the auditory processing deficits that underlie this problem, we made large-scale brain recordings from gerbils, a common animal model for human hearing, while presenting a large database of speech and noise sounds. We first used manifold learning to identify the neural subspace in which speech is encoded and found that it is low-dimensional and that the dynamics within it are profoundly distorted by hearing loss. We then trained a deep neural network (DNN) to replicate the neural coding of speech with and without hearing loss and analyzed the underlying network dynamics. We found that hearing loss primarily impacts spectral processing, creating nonlinear distortions in cross-frequency interactions that result in a hypersensitivity to background noise that persists even after amplification with a hearing aid. Our results identify a new focus for efforts to design improved hearing aids and demonstrate the power of DNNs as a tool for the study of central brain structures.

The effects of interaural time difference and intensity on the coding of low-frequency sounds in the mammalian midbrain.

We examined how changes in intensity and interaural time difference (ITD) influenced the coding of low-frequency sounds in the inferior colliculus of male gerbils at both the single neuron and population levels. We found that changes in intensity along the positive slope of the rate-level function (RLF) evoked changes in spectrotemporal filtering that influenced the overall timing of spike events but preserved their precision across trials such that the decoding of single neuron responses was not affected. In contrast, changes in ITD did not trigger changes in spectrotemporal filtering, but did have strong effects on the precision of spike events and, consequently, on decoder performance. However, changes in ITD had opposing effects in the two brain hemispheres and, thus, canceled out at the population level. These results were similar with and without the addition of background noise. We also found that the effects of changes in intensity along the negative slope of the RLF were different from the effects of changes in intensity along the positive slope in that they evoked changes in both spectrotemporal filtering and in the precision of spike events across trials, as well as in decoder performance. These results demonstrate that, at least at moderate intensities, the auditory system employs different strategies at the single neuron and population levels simultaneously to ensure that the coding of sounds is robust to changes in other stimulus features.

Impaired auditory temporal selectivity in the inferior colliculus of aged Mongolian gerbils.

Aged humans show severe difficulties in temporal auditory processing tasks (e.g., speech recognition in noise, low-frequency sound localization, gap detection). A degradation of auditory function with age is also evident in experimental animals. To investigate age-related changes in temporal processing, we compared extracellular responses to temporally variable pulse trains and human speech in the inferior colliculus of young adult (3 month) and aged (3 years) Mongolian gerbils. We observed a significant decrease of selectivity to the pulse trains in neuronal responses from aged animals. This decrease in selectivity led, on the population level, to an increase in signal correlations and therefore a decrease in heterogeneity of temporal receptive fields and a decreased efficiency in encoding of speech signals. A decrease in selectivity to temporal modulations is consistent with a downregulation of the inhibitory transmitter system in aged animals. These alterations in temporal processing could underlie declines in the aging auditory system, which are unrelated to peripheral hearing loss. These declines cannot be compensated by traditional hearing aids (that rely on amplification of sound) but may rather require pharmacological treatment.

Analysis and modelling of variability and covariability of population spike trains across multiple time scales.

As multi-electrode and imaging technology begin to provide us with simultaneous recordings of large neuronal populations, new methods for modelling such data must also be developed. We present a model of responses to repeated trials of a sensory stimulus based on thresholded Gaussian processes that allows for analysis and modelling of variability and covariability of population spike trains across multiple time scales. The model framework can be used to specify the values of many different variability measures including spike timing precision across trials, coefficient of variation of the interspike interval distribution, and Fano factor of spike counts for individual neurons, as well as signal and noise correlations and correlations of spike counts across multiple neurons. Using both simulated data and data from different stages of the mammalian auditory pathway, we demonstrate the range of possible independent manipulations of different variability measures, and explore how this range depends on the sensory stimulus. The model provides a powerful framework for the study of experimental and surrogate data and for analyzing dependencies between different statistical properties of neuronal populations.

Compression and amplification algorithms in hearing aids impair the selectivity of neural responses to speech.

In quiet environments, hearing aids improve the perception of low-intensity sounds. However, for high-intensity sounds in background noise, the aids often fail to provide a benefit to the wearer. Here, using large-scale single-neuron recordings from hearing-impaired gerbils-an established animal model of human hearing-we show that hearing aids restore the sensitivity of neural responses to speech, but not their selectivity. Rather than reflecting a deficit in supra-threshold auditory processing, the low selectivity is a consequence of hearing-aid compression (which decreases the spectral and temporal contrasts of incoming sound) and amplification (which distorts neural responses, regardless of whether hearing is impaired). Processing strategies that avoid the trade-off between neural sensitivity and selectivity should improve the performance of hearing aids.

Harnessing the Power of Artificial Intelligence in Otolaryngology and the Communication Sciences.

Use of artificial intelligence (AI) is a burgeoning field in otolaryngology and the communication sciences. A virtual symposium on the topic was convened from Duke University on October 26, 2020, and was attended by more than 170 participants worldwide. This review presents summaries of all but one of the talks presented during the symposium; recordings of all the talks, along with the discussions for the talks, are available at https://www.youtube.com/watch?v=ktfewrXvEFg and https://www.youtube.com/watch?v=-gQ5qX2v3rg . Each of the summaries is about 2500 words in length and each summary includes two figures. This level of detail far exceeds the brief summaries presented in traditional reviews and thus provides a more-informed glimpse into the power and diversity of current AI applications in otolaryngology and the communication sciences and how to harness that power for future applications.

Adaptation to stimulus contrast and correlations during natural visual stimulation.

In this study, we characterize the adaptation of neurons in the cat lateral geniculate nucleus to changes in stimulus contrast and correlations. By comparing responses to high- and low-contrast natural scene movie and white noise stimuli, we show that an increase in contrast or correlations results in receptive fields with faster temporal dynamics and stronger antagonistic surrounds, as well as decreases in gain and selectivity. We also observe contrast- and correlation-induced changes in the reliability and sparseness of neural responses. We find that reliability is determined primarily by processing in the receptive field (the effective contrast of the stimulus), while sparseness is determined by the interactions between several functional properties. These results reveal a number of adaptive phenomena and suggest that adaptation to stimulus contrast and correlations may play an important role in visual coding in a dynamic natural environment.

Population coding of interaural time differences in gerbils and barn owls.

Interaural time differences (ITDs) are the primary cue for the localization of low-frequency sound sources in the azimuthal plane. For decades, it was assumed that the coding of ITDs in the mammalian brain was similar to that in the avian brain, where information is sparsely distributed across individual neurons, but recent studies have suggested otherwise. In this study, we characterized the representation of ITDs in adult male and female gerbils. First, we performed behavioral experiments to determine the acuity with which gerbils can use ITDs to localize sounds. Next, we used different decoders to infer ITDs from the activity of a population of neurons in central nucleus of the inferior colliculus. These results show that ITDs are not represented in a distributed manner, but rather in the summed activity of the entire population. To contrast these results with those from a population where the representation of ITDs is known to be sparsely distributed, we performed the same analysis on activity from the external nucleus of the inferior colliculus of adult male and female barn owls. Together, our results support the idea that, unlike the avian brain, the mammalian brain represents ITDs in the overall activity of a homogenous population of neurons within each hemisphere.

Timing precision in population coding of natural scenes in the early visual system.

The timing of spiking activity across neurons is a fundamental aspect of the neural population code. Individual neurons in the retina, thalamus, and cortex can have very precise and repeatable responses but exhibit degraded temporal precision in response to suboptimal stimuli. To investigate the functional implications for neural populations in natural conditions, we recorded in vivo the simultaneous responses, to movies of natural scenes, of multiple thalamic neurons likely converging to a common neuronal target in primary visual cortex. We show that the response of individual neurons is less precise at lower contrast, but that spike timing precision across neurons is relatively insensitive to global changes in visual contrast. Overall, spike timing precision within and across cells is on the order of 10 ms. Since closely timed spikes are more efficient in inducing a spike in downstream cortical neurons, and since fine temporal precision is necessary to represent the more slowly varying natural environment, we argue that preserving relative spike timing at a approximately 10-ms resolution is a crucial property of the neural code entering cortex.

Incremental mutual information: a new method for characterizing the strength and dynamics of connections in neuronal circuits.

Understanding the computations performed by neuronal circuits requires characterizing the strength and dynamics of the connections between individual neurons. This characterization is typically achieved by measuring the correlation in the activity of two neurons. We have developed a new measure for studying connectivity in neuronal circuits based on information theory, the incremental mutual information (IMI). By conditioning out the temporal dependencies in the responses of individual neurons before measuring the dependency between them, IMI improves on standard correlation-based measures in several important ways: 1) it has the potential to disambiguate statistical dependencies that reflect the connection between neurons from those caused by other sources (e.g. shared inputs or intrinsic cellular or network mechanisms) provided that the dependencies have appropriate timescales, 2) for the study of early sensory systems, it does not require responses to repeated trials of identical stimulation, and 3) it does not assume that the connection between neurons is linear. We describe the theory and implementation of IMI in detail and demonstrate its utility on experimental recordings from the primate visual system.

Enhancement of ITD coding within the initial stages of the auditory pathway.

Sensory systems use a variety of strategies to increase the signal-to-noise ratio in their inputs at the receptor level. However, important cues for sound localization are not present at the individual ears but are computed after inputs from the two ears converge within the brain, and we hypothesized that additional strategies to enhance the representation of these cues might be employed in the initial stages after binaural convergence. Specifically, we investigated the transformation that takes place between the first two stages of the gerbil auditory pathway that are sensitive to differences in the arrival time of a sound at the two ears (interaural time differences; ITDs): the medial superior olive (MSO), where ITD tuning originates, and the dorsal nucleus of the lateral lemniscus (DNLL), to which the MSO sends direct projections. We use a combined experimental and computational approach to demonstrate that the coding of ITDs is dramatically enhanced between these two stages, with the mutual information in the responses of single neurons increasing by a factor of 2. We also show that this enhancement is related to an increase in dynamic range for neurons with high preferred frequencies and a decrease in variability for neurons with low preferred frequencies. These results suggest that a major role of the initial stages of the ITD pathway may be to enhance the representation created at the site of coincidence detection and illustrate the potential of this pathway as a model system for the study of strategies for enhancing sensory representations in the mammalian brain.

Dynamic spectrotemporal feature selectivity in the auditory midbrain.

The transformation of auditory information from the cochlea to the cortex is a highly nonlinear process. Studies using tone stimuli have revealed that changes in even the most basic parameters of the auditory stimulus can alter neural response properties; for example, a change in stimulus intensity can cause a shift in a neuron's preferred frequency. However, it is not yet clear how such nonlinearities contribute to the processing of spectrotemporal features in complex sounds. Here, we use spectrotemporal receptive fields (STRFs) to characterize the effects of stimulus intensity on feature selectivity in the mammalian inferior colliculus (IC). At low intensities, we find that STRFs are relatively simple, typically consisting of a single excitatory region, indicating that the neural response is simply a reflection of the stimulus amplitude at the preferred frequency. In contrast, we find that STRFs at high intensities typically consist of a combination of an excitatory region and one or more inhibitory regions, often in a spectrotemporally inseparable arrangement, indicating selectivity for complex auditory features. We show that a linear-nonlinear model with the appropriate STRF can predict neural responses to stimuli with a fixed intensity, and we demonstrate that a simple extension of the model with an intensity-dependent STRF can predict responses to stimuli with varying intensity. These results illustrate the complexity of auditory feature selectivity in the IC, but also provide encouraging evidence that the prediction of nonlinear responses to complex stimuli is a tractable problem.

Dynamic encoding of natural luminance sequences by LGN bursts.

In the lateral geniculate nucleus (LGN) of the thalamus, visual stimulation produces two distinct types of responses known as tonic and burst. Due to the dynamics of the T-type Ca(2+) channels involved in burst generation, the type of response evoked by a particular stimulus depends on the resting membrane potential, which is controlled by a network of modulatory connections from other brain areas. In this study, we use simulated responses to natural scene movies to describe how modulatory and stimulus-driven changes in LGN membrane potential interact to determine the luminance sequences that trigger burst responses. We find that at low resting potentials, when the T channels are de-inactivated and bursts are relatively frequent, an excitatory stimulus transient alone is sufficient to evoke a burst. However, to evoke a burst at high resting potentials, when the T channels are inactivated and bursts are relatively rare, prolonged inhibitory stimulation followed by an excitatory transient is required. We also observe evidence of these effects in vivo, where analysis of experimental recordings demonstrates that the luminance sequences that trigger bursts can vary dramatically with the overall burst percentage of the response. To characterize the functional consequences of the effects of resting potential on burst generation, we simulate LGN responses to different luminance sequences at a range of resting potentials with and without a mechanism for generating bursts. Using analysis based on signal detection theory, we show that bursts enhance detection of specific luminance sequences, ranging from the onset of excitatory sequences at low resting potentials to the offset of inhibitory sequences at high resting potentials. These results suggest a dynamic role for burst responses during visual processing that may change according to behavioral state.

Why Do Hearing Aids Fail to Restore Normal Auditory Perception?

Hearing loss is a widespread condition that is linked to declines in quality of life and mental health. Hearing aids remain the treatment of choice, but, unfortunately, even state-of-the-art devices provide only limited benefit for the perception of speech in noisy environments. While traditionally viewed primarily as a loss of sensitivity, hearing loss is also known to cause complex distortions of sound-evoked neural activity that cannot be corrected by amplification alone. This Opinion article describes the effects of hearing loss on neural activity to illustrate the reasons why current hearing aids are insufficient and to motivate the use of new technologies to explore directions for improving the next generation of devices.