Rule-based and stimulus-based cues bias auditory decisions via different computational and physiological mechanisms.

Expectations, such as those arising from either learned rules or recent stimulus regularities, can bias subsequent auditory perception in diverse ways. However, it is not well understood if and how these diverse effects depend on the source of the expectations. Further, it is unknown whether different sources of bias use the same or different computational and physiological mechanisms. We examined how rule-based and stimulus-based expectations influenced behavior and pupil-linked arousal, a marker of certain forms of expectation-based processing, of human subjects performing an auditory frequency-discrimination task. Rule-based cues consistently biased choices and response times (RTs) toward the more-probable stimulus. In contrast, stimulus-based cues had a complex combination of effects, including choice and RT biases toward and away from the frequency of recently presented stimuli. These different behavioral patterns also had: 1) distinct computational signatures, including different modulations of key components of a novel form of a drift-diffusion decision model and 2) distinct physiological signatures, including substantial bias-dependent modulations of pupil size in response to rule-based but not stimulus-based cues. These results imply that different sources of expectations can modulate auditory processing via distinct mechanisms: one that uses arousal-linked, rule-based information and another that uses arousal-independent, stimulus-based information to bias the speed and accuracy of auditory perceptual decisions.

Neuronal phase consistency tracks dynamic changes in acoustic spectral regularity.

The brain parses the auditory environment into distinct sounds by identifying those acoustic features in the environment that have common relationships (e.g., spectral regularities) with one another and then grouping together the neuronal representations of these features. Although there is a large literature that tests how the brain tracks spectral regularities that are predictable, it is not known how the auditory system tracks spectral regularities that are not predictable and that change dynamically over time. Furthermore, the contribution of brain regions downstream of the auditory cortex to the coding of spectral regularity is unknown. Here, we addressed these two issues by recording electrocorticographic activity, while human patients listened to tone-burst sequences with dynamically varying spectral regularities, and identified potential neuronal mechanisms of the analysis of spectral regularities throughout the brain. We found that the degree of oscillatory stimulus phase consistency (PC) in multiple neuronal-frequency bands tracked spectral regularity. In particular, PC in the delta-frequency band seemed to be the best indicator of spectral regularity. We also found that these regularity representations existed in multiple regions throughout cortex. This widespread reliable modulation in PC - both in neuronal-frequency space and in cortical space - suggests that phase-based modulations may be a general mechanism for tracking regularity in the auditory system specifically and other sensory systems more generally. Our findings also support a general role for the delta-frequency band in processing the regularity of auditory stimuli.

The what, where and how of auditory-object perception.

The fundamental perceptual unit in hearing is the 'auditory object'. Similar to visual objects, auditory objects are the computational result of the auditory system's capacity to detect, extract, segregate and group spectrotemporal regularities in the acoustic environment; the multitude of acoustic stimuli around us together form the auditory scene. However, unlike the visual scene, resolving the component objects within the auditory scene crucially depends on their temporal structure. Neural correlates of auditory objects are found throughout the auditory system. However, neural responses do not become correlated with a listener's perceptual reports until the level of the cortex. The roles of different neural structures and the contribution of different cognitive states to the perception of auditory objects are not yet fully understood.

Contribution of spiking activity in the primary auditory cortex to detection in noise.

A fundamental problem in hearing is detecting a "target" stimulus (e.g., a friend's voice) that is presented with a noisy background (e.g., the din of a crowded restaurant). Despite its importance to hearing, a relationship between spiking activity and behavioral performance during such a "detection-in-noise" task has yet to be fully elucidated. In this study, we recorded spiking activity in primary auditory cortex (A1) while rhesus monkeys detected a target stimulus that was presented with a noise background. Although some neurons were modulated, the response of the typical A1 neuron was not modulated by the stimulus- and task-related parameters of our task. In contrast, we found more robust representations of these parameters in population-level activity: small populations of neurons matched the monkeys' behavioral sensitivity. Overall, these findings are consistent with the hypothesis that the sensory evidence, which is needed to solve such detection-in-noise tasks, is represented in population-level A1 activity and may be available to be read out by downstream neurons that are involved in mediating this task.NEW & NOTEWORTHY This study examines the contribution of A1 to detecting a sound that is presented with a noisy background. We found that population-level A1 activity, but not single neurons, could provide the evidence needed to make this perceptual decision.

Functional Organization of the Ventral Auditory Pathway.

The fundamental problem in audition is determining the mechanisms required by the brain to transform an unlabelled mixture of auditory stimuli into coherent perceptual representations. This process is called auditory-scene analysis. The perceptual representations that result from auditory-scene analysis are formed through a complex interaction of perceptual grouping, attention, categorization and decision-making. Despite a great deal of scientific energy devoted to understanding these aspects of hearing, we still do not understand (1) how sound perception arises from neural activity and (2) the causal relationship between neural activity and sound perception. Here, we review the role of the "ventral" auditory pathway in sound perception. We hypothesize that, in the early parts of the auditory cortex, neural activity reflects the auditory properties of a stimulus. However, in latter parts of the auditory cortex, neurons encode the sensory evidence that forms an auditory decision and are causally involved in the decision process. Finally, in the prefrontal cortex, which receives input from the auditory cortex, neural activity reflects the actual perceptual decision. Together, these studies indicate that the ventral pathway contains hierarchical circuits that are specialized for auditory perception and scene analysis.

Characterizing the impact of category uncertainty on human auditory categorization behavior.

Categorization is an important cognitive process. However, the correct categorization of a stimulus is often challenging because categories can have overlapping boundaries. Whereas perceptual categorization has been extensively studied in vision, the analogous phenomenon in audition has yet to be systematically explored. Here, we test whether and how human subjects learn to use category distributions and prior probabilities, as well as whether subjects employ an optimal decision strategy when making auditory-category decisions. We asked subjects to classify the frequency of a tone burst into one of two overlapping, uniform categories according to the perceived tone frequency. We systematically varied the prior probability of presenting a tone burst with a frequency originating from one versus the other category. Most subjects learned these changes in prior probabilities early in testing and used this information to influence categorization. We also measured each subject's frequency-discrimination thresholds (i.e., their sensory uncertainty levels). We tested each subject's average behavior against variations of a Bayesian model that either led to optimal or sub-optimal decision behavior (i.e. probability matching). In both predicting and fitting each subject's average behavior, we found that probability matching provided a better account of human decision behavior. The model fits confirmed that subjects were able to learn category prior probabilities and approximate forms of the category distributions. Finally, we systematically explored the potential ways that additional noise sources could influence categorization behavior. We found that an optimal decision strategy can produce probability-matching behavior if it utilized non-stationary category distributions and prior probabilities formed over a short stimulus history. Our work extends previous findings into the auditory domain and reformulates the issue of categorization in a manner that can help to interpret the results of previous research within a generative framework.

A high-density, high-channel count, multiplexed µECoG array for auditory-cortex recordings.

Our understanding of the large-scale population dynamics of neural activity is limited, in part, by our inability to record simultaneously from large regions of the cortex. Here, we validated the use of a large-scale active microelectrode array that simultaneously records 196 multiplexed micro-electrocortigraphical (µECoG) signals from the cortical surface at a very high density (1,600 electrodes/cm(2)). We compared µECoG measurements in auditory cortex using a custom "active" electrode array to those recorded using a conventional "passive" µECoG array. Both of these array responses were also compared with data recorded via intrinsic optical imaging, which is a standard methodology for recording sound-evoked cortical activity. Custom active µECoG arrays generated more veridical representations of the tonotopic organization of the auditory cortex than current commercially available passive µECoG arrays. Furthermore, the cortical representation could be measured efficiently with the active arrays, requiring as little as 13.5 s of neural data acquisition. Next, we generated spectrotemporal receptive fields from the recorded neural activity on the active µECoG array and identified functional organizational principles comparable to those observed using intrinsic metabolic imaging and single-neuron recordings. This new electrode array technology has the potential for large-scale, temporally precise monitoring and mapping of the cortex, without the use of invasive penetrating electrodes.

A multi-institutional machine learning algorithm for prognosticating facial nerve injury following microsurgical resection of vestibular schwannoma.

Vestibular schwannomas (VS) are the most common tumor of the skull base with available treatment options that carry a risk of iatrogenic injury to the facial nerve, which can significantly impact patients' quality of life. As facial nerve outcomes remain challenging to prognosticate, we endeavored to utilize machine learning to decipher predictive factors relevant to facial nerve outcomes following microsurgical resection of VS. A database of patient-, tumor- and surgery-specific features was constructed via retrospective chart review of 242 consecutive patients who underwent microsurgical resection of VS over a 7-year study period. This database was then used to train non-linear supervised machine learning classifiers to predict facial nerve preservation, defined as House-Brackmann (HB) I vs. facial nerve injury, defined as HB II-VI, as determined at 6-month outpatient follow-up. A random forest algorithm demonstrated 90.5% accuracy, 90% sensitivity and 90% specificity in facial nerve injury prognostication. A random variable (rv) was generated by randomly sampling a Gaussian distribution and used as a benchmark to compare the predictiveness of other features. This analysis revealed age, body mass index (BMI), case length and the tumor dimension representing tumor growth towards the brainstem as prognosticators of facial nerve injury. When validated via prospective assessment of facial nerve injury risk, this model demonstrated 84% accuracy. Here, we describe the development of a machine learning algorithm to predict the likelihood of facial nerve injury following microsurgical resection of VS. In addition to serving as a clinically applicable tool, this highlights the potential of machine learning to reveal non-linear relationships between variables which may have clinical value in prognostication of outcomes for high-risk surgical procedures.

A model of the differential representation of signal novelty in the local field potentials and spiking activity of the ventrolateral prefrontal cortex.

Local field potentials (LFPs) and spiking activity reflect different types of information procssing. For example, neurophysiological studies indicate that signal novelty in the ventrolateral prefrontal cortex is differentially represented by LFPs and spiking activity: LFPs habituate to repeated stimulus presentations, whereas spiking activity does not. The neural mechanisms that allow for this differential representation between LFPs and spiking activity are not clear. Here, we model and simulate LFPs and spiking activity of neurons in the ventrolateral prefrontal cortex in order to elucidate potential mechanisms underlying this differential representation. We demonstrate that dynamic negative-feedback loops cause LFPs to habituate in response to repeated presentations of the same stimulus while spiking activity is maintained. This disassociation between LFPs and spiking activity may be a mechanism by which LFPs code stimulus novelty, whereas spiking activity carries abstract information, such as category membership and decision-related activity.

Rhesus macaques recognize unique multimodal face-voice relations of familiar individuals and not of unfamiliar ones.

Communication signals in nonhuman primates are inherently multimodal. However, for laboratory-housed monkeys, there is relatively little evidence in support of the use of multimodal communication signals in individual recognition. Here, we used a preferential-looking paradigm to test whether laboratory-housed rhesus monkeys could 'spontaneously' (i.e., in the absence of operant training) use multimodal communication stimuli to discriminate between known conspecifics. The multimodal stimulus was a silent movie of 2 monkeys vocalizing and an audio file of the vocalization from one of the monkeys in the movie. We found that the gaze patterns of the monkeys that knew the individuals in the movie were reliably biased toward the individual that did not produce the vocalization. In contrast, there was not a systematic gaze pattern for the monkeys that did not know the individuals in the movie. These data are consistent with the hypothesis that laboratory-housed rhesus macaques can recognize and distinguish between conspecifics based on auditory and visual communication signals.

Functional network properties of the auditory cortex.

The auditory system transforms auditory stimuli from the external environment into perceptual auditory objects. Recent studies have focused on the contribution of the auditory cortex to this transformation. Other studies have yielded important insights into the contributions of neural activity in the auditory cortex to cognition and decision-making. However, despite this important work, the relationship between auditory-cortex activity and behavior/perception has not been fully elucidated. Two of the more important gaps in our understanding are (1) the specific and differential contributions of different fields of the auditory cortex to auditory perception and behavior and (2) the way networks of auditory neurons impact and facilitate auditory information processing. Here, we focus on recent work from non-human-primate models of hearing and review work related to these gaps and put forth challenges to further our understanding of how single-unit activity and network activity in different cortical fields contribution to behavior and perception.

Changes in neural readout of response magnitude during auditory streaming do not correlate with behavioral choice in the auditory cortex.

A fundamental goal of the auditory system is to group stimuli from the auditory environment into a perceptual unit (i.e., "stream") or segregate the stimuli into multiple different streams. Although previous studies have clarified the psychophysical and neural mechanisms that may underlie this ability, the relationship between these mechanisms remains elusive. Here, we recorded multiunit activity (MUA) from the auditory cortex of monkeys while they participated in an auditory-streaming task consisting of interleaved low- and high-frequency tone bursts. As the streaming stimulus unfolded over time, MUA amplitude habituated; the magnitude of this habituation was correlated with the frequency difference between the tone bursts. An ideal-observer model could classify these time- and frequency-dependent changes into reports of "one stream" or "two streams" in a manner consistent with the behavioral literature. However, because classification was not modulated by the monkeys' behavioral choices, this MUA habituation may not directly reflect perceptual reports.

Time as a supervisor: temporal regularity and auditory object learning.

Sensory systems appear to learn to transform incoming sensory information into perceptual representations, or "objects," that can inform and guide behavior with minimal explicit supervision. Here, we propose that the auditory system can achieve this goal by using time as a supervisor, i.e., by learning features of a stimulus that are temporally regular. We will show that this procedure generates a feature space sufficient to support fundamental computations of auditory perception. In detail, we consider the problem of discriminating between instances of a prototypical class of natural auditory objects, i.e., rhesus macaque vocalizations. We test discrimination in two ethologically relevant tasks: discrimination in a cluttered acoustic background and generalization to discriminate between novel exemplars. We show that an algorithm that learns these temporally regular features affords better or equivalent discrimination and generalization than conventional feature-selection algorithms, i.e., principal component analysis and independent component analysis. Our findings suggest that the slow temporal features of auditory stimuli may be sufficient for parsing auditory scenes and that the auditory brain could utilize these slowly changing temporal features.

Differential representation of auditory categories between cell classes in primate auditory cortex.

A comprehensive understanding of the neural mechanisms of cognitive function requires an understanding of how neural representations are transformed across different scales of neural organization: from within local microcircuits to across different brain areas. However, the neural transformations within the local microcircuits are poorly understood. Particularly, the role that two main cell classes of neurons in cortical microcircuits (i.e. pyramidal neurons and interneurons) have in auditory behaviour and cognition remains unknown. In this study, we tested the hypothesis that pyramidal cells and interneurons in the auditory cortex play a differential role in auditory categorization. To test this hypothesis, we recorded single-unit activity from the auditory cortex of rhesus monkeys while they categorized speech sounds. Based on the spike-waveform shape, a neuron was classified as either a narrow-spiking putative interneuron or a broad-spiking putative pyramidal neuron. We found that putative interneurons and pyramidal neurons in the auditory cortex differentially coded category information: interneurons were more selective for auditory categories than pyramidal neurons. These differences between cell classes may be an essential property of the neural computations underlying auditory categorization within the microcircuitry of the auditory cortex.

Cortical circuit-based lossless neural integrator for perceptual decision-making: A computational modeling study.

The intrinsic uncertainty of sensory information (i.e., evidence) does not necessarily deter an observer from making a reliable decision. Indeed, uncertainty can be reduced by integrating (accumulating) incoming sensory evidence. It is widely thought that this accumulation is instantiated via recurrent rate-code neural networks. Yet, these networks do not fully explain important aspects of perceptual decision-making, such as a subject's ability to retain accumulated evidence during temporal gaps in the sensory evidence. Here, we utilized computational models to show that cortical circuits can switch flexibly between "retention" and "integration" modes during perceptual decision-making. Further, we found that, depending on how the sensory evidence was readout, we could simulate "stepping" and "ramping" activity patterns, which may be analogous to those seen in different studies of decision-making in the primate parietal cortex. This finding may reconcile these previous empirical studies because it suggests these two activity patterns emerge from the same mechanism.

Representation of speech categories in the primate auditory cortex.

A "ventral" auditory pathway in nonhuman primates that originates in the core auditory cortex and ends in the prefrontal cortex is thought to be involved in components of nonspatial auditory processing. Previous work from our laboratory has indicated that neurons in the prefrontal cortex reflect monkeys' decisions during categorical judgments. Here, we tested the role of the superior temporal gyrus (STG), a region of the secondary auditory cortex that is part of this ventral pathway, during similar categorical judgments. While monkeys participated in a match-to-category task and reported whether two consecutive auditory stimuli belonged to the same category or to different categories, we recorded spiking activity from STG neurons. The auditory stimuli were morphs of two human-speech sounds (bad and dad). We found that STG neurons represented auditory categories. However, unlike activity in the prefrontal cortex, STG activity was not modulated by the monkeys' behavioral reports (choices). This finding is consistent with the anterolateral STG's role as a part of functional circuit involved in the coding, representation, and perception of the nonspatial features of an auditory stimulus.

Prefrontal neurons predict choices during an auditory same-different task.

The detection of stimuli is critical for an animal's survival [1]. However, it is not adaptive for an animal to respond automatically to every stimulus that is present in the environment [2-5]. Given that the prefrontal cortex (PFC) plays a key role in executive function [6-8], we hypothesized that PFC activity should be involved in context-dependent responses to uncommon stimuli. As a test of this hypothesis, monkeys participated in a same-different task, a variant of an oddball task [2]. During this task, a monkey heard multiple presentations of a "reference" stimulus that were followed by a "test" stimulus and reported whether these stimuli were the same or different. While they participated in this task, we recorded from neurons in the ventrolateral prefrontal cortex (vPFC; a cortical area involved in aspects of nonspatial auditory processing [9, 10]). We found that vPFC activity was correlated with the monkeys' choices. This finding demonstrates a direct link between single neurons and behavioral choices in the PFC on a nonspatial auditory task.

Motor-related signals in the intraparietal cortex encode locations in a hybrid, rather than eye-centered reference frame.

The reference frame used by intraparietal cortex neurons to encode locations is controversial. Many previous studies have suggested eye-centered coding, whereas we have reported that visual and auditory signals employ a hybrid reference frame (i.e., a combination of head- and eye-centered information) (Mullette-Gillman et al. 2005). One possible explanation for this discrepancy is that sensory-related activity, which we studied previously, is hybrid, whereas motor-related activity might be eye centered. Here, we examined the reference frame of visual and auditory saccade-related activity in the lateral and medial banks of the intraparietal sulcus (areas lateral intraparietal area [LIP] and medial intraparietal area [MIP]) of 2 rhesus monkeys. We recorded from 275 single neurons as monkeys performed visual and auditory saccades from different initial eye positions. We found that both visual and auditory signals reflected a hybrid of head- and eye-centered coordinates during both target and perisaccadic task periods rather than shifting to an eye-centered format as the saccade approached. This account differs from numerous previous recording studies. We suggest that the geometry of the receptive field sampling in prior studies was biased in favor of an eye-centered reference frame. Consequently, the overall hybrid nature of the reference frame was overlooked because the non-eye-centered response patterns were not fully characterized.

Computational and neurophysiological principles underlying auditory perceptual decisions.

A fundamental scientific goal in auditory neuroscience is identifying what mechanisms allow the brain to transform an unlabeled mixture of auditory stimuli into distinct perceptual representations. This transformation is accomplished by a complex interaction of multiple neurocomputational processes, including Gestalt grouping mechanisms, categorization, attention, and perceptual decision-making. Despite a great deal of scientific energy devoted to understanding these principles of hearing, we still do not understand either how auditory perception arises from neural activity or the causal relationship between neural activity and auditory perception. Here, we review the contributions of cortical and subcortical regions to auditory perceptual decisions with an emphasis on those studies that simultaneously measure behavior and neural activity. We also put forth challenges to the field that must be faced if we are to further our understanding of the relationship between neural activity and auditory perception.

A modular high-density µECoG system on macaque vlPFC for auditory cognitive decoding.

OBJECTIVE: A fundamental goal of the auditory system is to parse the auditory environment into distinct perceptual representations. Auditory perception is mediated by the ventral auditory pathway, which includes the ventrolateral prefrontal cortex (vlPFC). Because large-scale recordings of auditory signals are quite rare, the spatiotemporal resolution of the neuronal code that underlies vlPFC's contribution to auditory perception has not been fully elucidated. Therefore, we developed a modular, chronic, high-resolution, multi-electrode array system with long-term viability in order to identify the information that could be decoded from µECoG vlPFC signals. APPROACH: We molded three separate µECoG arrays into one and implanted this system in a non-human primate. A custom 3D-printed titanium chamber was mounted on the left hemisphere. The molded 294-contact µECoG array was implanted subdurally over the vlPFC. µECoG activity was recorded while the monkey participated in a 'hearing-in-noise' task in which they reported hearing a 'target' vocalization from a background 'chorus' of vocalizations. We titrated task difficulty by varying the sound level of the target vocalization, relative to the chorus (target-to-chorus ratio, TCr). MAIN RESULTS: We decoded the TCr and the monkey's behavioral choices from the µECoG signal. We analyzed decoding accuracy as a function of number of electrodes, spatial resolution, and time from implantation. Over a one-year period, we found significant decoding with individual electrodes that increased significantly as we decoded simultaneously more electrodes. Further, we found that the decoding for behavioral choice was better than the decoding of TCr. Finally, because the decoding accuracy of individual electrodes varied on a day-by-day basis, electrode arrays with high channel counts ensure robust decoding in the long term. SIGNIFICANCE: Our results demonstrate the utility of high-resolution and high-channel-count, chronic µECoG recording. We developed a surface electrode array that can be scaled to cover larger cortical areas without increasing the chamber footprint.