Acoustic Pattern Recognition and Courtship Songs: Insights from Insects.

Across the animal kingdom, social interactions rely on sound production and perception. From simple cricket chirps to more elaborate bird songs, animals go to great lengths to communicate information critical for reproduction and survival via acoustic signals. Insects produce a wide array of songs to attract a mate, and the intended receivers must differentiate these calls from competing sounds, analyze the quality of the sender from spectrotemporal signal properties, and then determine how to react. Insects use numerically simple nervous systems to analyze and respond to courtship songs, making them ideal model systems for uncovering the neural mechanisms underlying acoustic pattern recognition. We highlight here how the combination of behavioral studies and neural recordings in three groups of insects-crickets, grasshoppers, and fruit flies-reveals common strategies for extracting ethologically relevant information from acoustic patterns and how these findings might translate to other systems.

Intersecting kinematic encoding and readout of intention in autism.

Observers with autism spectrum disorders (ASDs) find it difficult to read intentions from movements. However, the computational bases of these difficulties are unknown. Do these difficulties reflect an intention readout deficit, or are they more likely rooted in kinematic (dis-)similarities between typical and ASD kinematics? We combined motion tracking, psychophysics, and computational analyses to uncover single-trial intention readout computations in typically developing (TD) children (n = 35) and children with ASD (n = 35) who observed actions performed by TD children and children with ASD. Average intention discrimination performance was above chance for TD observers but not for ASD observers. However, single-trial analysis showed that both TD and ASD observers read single-trial variations in movement kinematics. TD readers were better able to identify intention-informative kinematic features during observation of TD actions; conversely, ASD readers were better able to identify intention-informative features during observation of ASD actions. Crucially, while TD observers were generally able to extract the intention information encoded in movement kinematics, those with autism were unable to do so. These results extend existing conceptions of mind reading in ASD by suggesting that intention reading difficulties reflect both an interaction failure, rooted in kinematic dissimilarity between TD and ASD kinematics (at the level of feature identification), and an individual readout deficit (at the level of information extraction), accompanied by an overall reduced sensitivity of intention readout to single-trial variations in movement kinematics.

Language statistical learning responds to reinforcement learning principles rooted in the striatum.

Statistical learning (SL) is the ability to extract regularities from the environment. In the domain of language, this ability is fundamental in the learning of words and structural rules. In lack of reliable online measures, statistical word and rule learning have been primarily investigated using offline (post-familiarization) tests, which gives limited insights into the dynamics of SL and its neural basis. Here, we capitalize on a novel task that tracks the online SL of simple syntactic structures combined with computational modeling to show that online SL responds to reinforcement learning principles rooted in striatal function. Specifically, we demonstrate-on 2 different cohorts-that a temporal difference model, which relies on prediction errors, accounts for participants' online learning behavior. We then show that the trial-by-trial development of predictions through learning strongly correlates with activity in both ventral and dorsal striatum. Our results thus provide a detailed mechanistic account of language-related SL and an explanation for the oft-cited implication of the striatum in SL tasks. This work, therefore, bridges the long-standing gap between language learning and reinforcement learning phenomena.

Neurocognitive processing efficiency for discriminating human non-alarm rather than alarm scream calls.

Across many species, scream calls signal the affective significance of events to other agents. Scream calls were often thought to be of generic alarming and fearful nature, to signal potential threats, with instantaneous, involuntary, and accurate recognition by perceivers. However, scream calls are more diverse in their affective signaling nature than being limited to fearfully alarming a threat, and thus the broader sociobiological relevance of various scream types is unclear. Here we used 4 different psychoacoustic, perceptual decision-making, and neuroimaging experiments in humans to demonstrate the existence of at least 6 psychoacoustically distinctive types of scream calls of both alarming and non-alarming nature, rather than there being only screams caused by fear or aggression. Second, based on perceptual and processing sensitivity measures for decision-making during scream recognition, we found that alarm screams (with some exceptions) were overall discriminated the worst, were responded to the slowest, and were associated with a lower perceptual sensitivity for their recognition compared with non-alarm screams. Third, the neural processing of alarm compared with non-alarm screams during an implicit processing task elicited only minimal neural signal and connectivity in perceivers, contrary to the frequent assumption of a threat processing bias of the primate neural system. These findings show that scream calls are more diverse in their signaling and communicative nature in humans than previously assumed, and, in contrast to a commonly observed threat processing bias in perceptual discriminations and neural processes, we found that especially non-alarm screams, and positive screams in particular, seem to have higher efficiency in speeded discriminations and the implicit neural processing of various scream types in humans.

Vocal learning and flexible rhythm pattern perception are linked: Evidence from songbirds.

Rhythm perception is fundamental to speech and music. Humans readily recognize a rhythmic pattern, such as that of a familiar song, independently of the tempo at which it occurs. This shows that our perception of auditory rhythms is flexible, relying on global relational patterns more than on the absolute durations of specific time intervals. Given that auditory rhythm perception in humans engages a complex auditory-motor cortical network even in the absence of movement and that the evolution of vocal learning is accompanied by strengthening of forebrain auditory-motor pathways, we hypothesize that vocal learning species share our perceptual facility for relational rhythm processing. We test this by asking whether the best-studied animal model for vocal learning, the zebra finch, can recognize a fundamental rhythmic pattern-equal timing between event onsets (isochrony)-based on temporal relations between intervals rather than on absolute durations. Prior work suggests that vocal nonlearners (pigeons and rats) are quite limited in this regard and are biased to attend to absolute durations when listening to rhythmic sequences. In contrast, using naturalistic sounds at multiple stimulus rates, we show that male zebra finches robustly recognize isochrony independent of absolute time intervals, even at rates distant from those used in training. Our findings highlight the importance of comparative studies of rhythmic processing and suggest that vocal learning species are promising animal models for key aspects of human rhythm perception. Such models are needed to understand the neural mechanisms behind the positive effect of rhythm on certain speech and movement disorders.

Limitations in human auditory spectral analysis at high frequencies.

Humans are adept at identifying spectral patterns, such as vowels, in different rooms, at different sound levels, or produced by different talkers. How this feat is achieved remains poorly understood. Two psychoacoustic analogs of spectral pattern recognition are spectral profile analysis and spectrotemporal ripple direction discrimination. This study tested whether pattern-recognition abilities observed previously at low frequencies are also observed at extended high frequencies. At low frequencies (center frequency ∼500 Hz), listeners were able to achieve accurate profile-analysis thresholds, consistent with prior literature. However, at extended high frequencies (center frequency ∼10 kHz), listeners' profile-analysis thresholds were either unmeasurable or could not be distinguished from performance based on overall loudness cues. A similar pattern of results was observed with spectral ripple discrimination, where performance was again considerably better at low than at high frequencies. Collectively, these results suggest a severe deficit in listeners' ability to analyze patterns of intensity across frequency in the extended high-frequency region that cannot be accounted for by cochlear frequency selectivity. One interpretation is that the auditory system is not optimized to analyze such fine-grained across-frequency profiles at extended high frequencies, as they are not typically informative for everyday sounds.

Biased 5-HT1A receptor agonists F13714 and NLX-101 differentially affect pattern separation and neuronal plasticity in rats after acute and chronic treatment.

Pattern separation is a hippocampal process in which highly similar stimuli are recognized as separate representations, and deficits could lead to memory impairments in neuropsychiatric disorders such as schizophrenia. The 5-HT1A receptor (5-HT1AR) is believed to be involved in these hippocampal pattern separation processes. However, in the dorsal raphe nucleus (DRN), the 5-HT1AR is expressed as a somatodendritic autoreceptor, negatively regulates serotonergic signaling, and could thereby counteract the effects of hippocampal postsynaptic 5-HT1A receptors. Therefore, this study aims to identify how pre- and post-synaptic 5-HT1AR activity affects pattern separation. Object pattern separation (OPS) performance was measured in male Wistar rats after both acute and chronic treatment (i.p.) with 5-HT1AR biased agonists F13714 (0.0025 mg/kg acutely, 0.02 mg/kg/day chronically) or NLX-101 (0.08 mg/kg acutely, 0.32 mg/kg/day chronically), which preferentially activate autoreceptors or postsynaptic receptors respectively, for 14 days. Body temperature - a functional correlate of hypothalamic 5-HT1AR stimulation - was measured daily. Additionally, 5-HT1AR density (DRN) and plasticity markers (hippocampus) were assessed. Acute treatment with F13714 impaired OPS performance, whereas chronic treatment normalized this, and a drop in body temperature was found from day 4 onwards. NLX-101 enhanced OPS performance acutely and chronically, and caused an acute drop in body temperature. Chronic NLX-101 treatment increased doublecortin positive neurons in the dorsal hippocampus, while chronic treatment with F13714 resulted in a downregulation of 5-HT1A autoreceptors, which likely reversed the acute impairment in OPS performance. Chronic treatment with NLX-101 appears to have therapeutic potential to improve brain plasticity and OPS performance.

Controversial stimuli: Pitting neural networks against each other as models of human cognition.

Distinct scientific theories can make similar predictions. To adjudicate between theories, we must design experiments for which the theories make distinct predictions. Here we consider the problem of comparing deep neural networks as models of human visual recognition. To efficiently compare models' ability to predict human responses, we synthesize controversial stimuli: images for which different models produce distinct responses. We applied this approach to two visual recognition tasks, handwritten digits (MNIST) and objects in small natural images (CIFAR-10). For each task, we synthesized controversial stimuli to maximize the disagreement among models which employed different architectures and recognition algorithms. Human subjects viewed hundreds of these stimuli, as well as natural examples, and judged the probability of presence of each digit/object category in each image. We quantified how accurately each model predicted the human judgments. The best-performing models were a generative analysis-by-synthesis model (based on variational autoencoders) for MNIST and a hybrid discriminative-generative joint energy model for CIFAR-10. These deep neural networks (DNNs), which model the distribution of images, performed better than purely discriminative DNNs, which learn only to map images to labels. None of the candidate models fully explained the human responses. Controversial stimuli generalize the concept of adversarial examples, obviating the need to assume a ground-truth model. Unlike natural images, controversial stimuli are not constrained to the stimulus distribution models are trained on, thus providing severe out-of-distribution tests that reveal the models' inductive biases. Controversial stimuli therefore provide powerful probes of discrepancies between models and human perception.

A neural ensemble correlation code for sound category identification.

Humans and other animals effortlessly identify natural sounds and categorize them into behaviorally relevant categories. Yet, the acoustic features and neural transformations that enable sound recognition and the formation of perceptual categories are largely unknown. Here, using multichannel neural recordings in the auditory midbrain of unanesthetized female rabbits, we first demonstrate that neural ensemble activity in the auditory midbrain displays highly structured correlations that vary with distinct natural sound stimuli. These stimulus-driven correlations can be used to accurately identify individual sounds using single-response trials, even when the sounds do not differ in their spectral content. Combining neural recordings and an auditory model, we then show how correlations between frequency-organized auditory channels can contribute to discrimination of not just individual sounds but sound categories. For both the model and neural data, spectral and temporal correlations achieved similar categorization performance and appear to contribute equally. Moreover, both the neural and model classifiers achieve their best task performance when they accumulate evidence over a time frame of approximately 1-2 seconds, mirroring human perceptual trends. These results together suggest that time-frequency correlations in sounds may be reflected in the correlations between auditory midbrain ensembles and that these correlations may play an important role in the identification and categorization of natural sounds.

Prepontine non-giant neurons drive flexible escape behavior in zebrafish.

Many species execute ballistic escape reactions to avoid imminent danger. Despite fast reaction times, responses are often highly regulated, reflecting a trade-off between costly motor actions and perceived threat level. However, how sensory cues are integrated within premotor escape circuits remains poorly understood. Here, we show that in zebrafish, less precipitous threats elicit a delayed escape, characterized by flexible trajectories, which are driven by a cluster of 38 prepontine neurons that are completely separate from the fast escape pathway. Whereas neurons that initiate rapid escapes receive direct auditory input and drive motor neurons, input and output pathways for delayed escapes are indirect, facilitating integration of cross-modal sensory information. These results show that rapid decision-making in the escape system is enabled by parallel pathways for ballistic responses and flexible delayed actions and defines a neuronal substrate for hierarchical choice in the vertebrate nervous system.

Creating a neuroprosthesis for active tactile exploration of textures.

Intracortical microstimulation (ICMS) of the primary somatosensory cortex (S1) can produce percepts that mimic somatic sensation and, thus, has potential as an approach to sensorize prosthetic limbs. However, it is not known whether ICMS could recreate active texture exploration-the ability to infer information about object texture by using one's fingertips to scan a surface. Here, we show that ICMS of S1 can convey information about the spatial frequencies of invisible virtual gratings through a process of active tactile exploration. Two rhesus monkeys scanned pairs of visually identical screen objects with the fingertip of a hand avatar-controlled first via a joystick and later via a brain-machine interface-to find the object with denser virtual gratings. The gratings consisted of evenly spaced ridges that were signaled through individual ICMS pulses generated whenever the avatar's fingertip crossed a ridge. The monkeys learned to interpret these ICMS patterns, evoked by the interplay of their voluntary movements and the virtual textures of each object, to perform a sensory discrimination task. Discrimination accuracy followed Weber's law of just-noticeable differences (JND) across a range of grating densities; a finding that matches normal cutaneous sensation. Moreover, 1 monkey developed an active scanning strategy where avatar velocity was integrated with the ICMS pulses to interpret the texture information. We propose that this approach could equip upper-limb neuroprostheses with direct access to texture features acquired during active exploration of natural objects.

Cortical Overlap and Cortical-Hippocampal Interactions Predict Subsequent True and False Memory.

The declarative memory system allows us to accurately recognize a countless number of items and events, particularly those strengthened by repeated exposure. However, increased familiarity due to repetition can also lead to false recognition of related but new items, particularly when mechanisms supporting fine-grain mnemonic discrimination fail. The hippocampus is thought to be particularly important in separating overlapping cortical inputs during encoding so that similar experiences can be differentiated. In the current study of male and female human subjects, we examine how neural pattern similarity between repeated exemplars of a given concept (e.g., apple) influences true and false memory for target or lure images. Consistent with past work, we found that subsequent true recognition was related to pattern similarity between concept exemplars and the entire encoding set (global encoding similarity), particularly in ventral visual stream. In addition, memory for an individual target exemplar (a specific apple) could be predicted solely by the degree of pattern overlap between the other exemplars (different apple pictures) of that concept (concept-specific encoding similarity). Critically, subsequent false memory for lures was mitigated when high concept-specific similarity in cortical areas was accompanied by differentiated hippocampal representations of the corresponding exemplars. Furthermore, both true and false memory entailed the reinstatement of concept-related information at varying levels of specificity. These results link both true and false memory to a measure of concept strength expressed in the overlap of cortical representations, and importantly, illustrate how the hippocampus serves to separate concurrent cortical overlap in the service of detailed memory.SIGNIFICANCE STATEMENT In some instances, the same processes that help promote memory for a general idea or concept can also hinder more detailed memory judgments, which may involve differentiating between closely related items. The current study shows that increased overlap in cortical representations for conceptually-related pictures is associated with increased recognition of repeated concept pictures. Whether similar lure items were falsely remembered as old further depended on the hippocampus, where the presence of more distinct representations protected against later false memory. This work suggests that the differentiability of brain patterns during perception is related to the differentiability of items in memory, but that fine-grain discrimination depends on the interaction between cortex and hippocampus.

Biased Neural Representation of Feature-Based Attention in the Human Frontoparietal Network.

Selective attention is a core cognitive function for efficient processing of information. Although it is well known that attention can modulate neural responses in many brain areas, the computational principles underlying attentional modulation remain unclear. Contrary to the prevailing view of a high-dimensional, distributed neural representation, here we show a surprisingly simple, biased neural representation for feature-based attention in a large dataset including five human fMRI studies. We found that when human participants (both sexes) selected one feature from a compound stimulus, voxels in many cortical areas responded consistently higher to one attended feature over the other. This univariate bias was consistent across brain areas within individual subjects. Importantly, this univariate bias showed a progressively stronger magnitude along the cortical hierarchy. In frontoparietal areas, the bias was strongest and contributed largely to pattern-based decoding, whereas early visual areas lacked such a bias. These findings suggest a gradual transition from a more analog to a more abstract representation of attentional priority along the cortical hierarchy. Biased neural responses in high-level areas likely reflect a low-dimensional neural code that can facilitate a robust representation and simple readout of cognitive variables.SIGNIFICANCE STATEMENT It is typically assumed that cognitive variables are represented by distributed population activities. Although this view is rooted in decades of work in the sensory system, it has not been rigorously tested at different levels of cortical hierarchy. Here we show a novel, low-dimensional coding scheme that dominated the representation of feature-based attention in frontoparietal areas. The simplicity of such a biased code may confer a robust representation of cognitive variables, such as attentional selection, working memory, and decision-making.

Sustained neural activity correlates with rapid perceptual learning of auditory patterns.

Repeating structures forming regular patterns are common in sounds. Learning such patterns may enable accurate perceptual organization. In five experiments, we investigated the behavioral and neural signatures of rapid perceptual learning of regular sound patterns. We show that recurring (compared to novel) patterns are detected more quickly and increase sensitivity to pattern deviations and to the temporal order of pattern onset relative to a visual stimulus. Sustained neural activity reflected perceptual learning in two ways. Firstly, sustained activity increased earlier for recurring than novel patterns when participants attended to sounds, but not when they ignored them; this earlier increase mirrored the rapid perceptual learning we observed behaviorally. Secondly, the magnitude of sustained activity was generally lower for recurring than novel patterns, but only for trials later in the experiment, and independent of whether participants attended to or ignored sounds. The late manifestation of sustained activity reduction suggests that it is not directly related to rapid perceptual learning, but to a mechanism that does not require attention to sound. In sum, we demonstrate that the latency of sustained activity reflects rapid perceptual learning of auditory patterns, while the magnitude may reflect a result of learning, such as better prediction of learned auditory patterns.

Neurophysiology goes wild: from exploring sensory coding in sound proof rooms to natural environments.

To perform adaptive behaviours, animals have to establish a representation of the physical "outside" world. How these representations are created by sensory systems is a central issue in sensory physiology. This review addresses the history of experimental approaches toward ideas about sensory coding, using the relatively simple auditory system of acoustic insects. I will discuss the empirical evidence in support of Barlow's "efficient coding hypothesis", which argues that the coding properties of neurons undergo specific adaptations that allow insects to detect biologically important acoustic stimuli. This hypothesis opposes the view that the sensory systems of receivers are biased as a result of their phylogeny, which finally determine whether a sound stimulus elicits a behavioural response. Acoustic signals are often transmitted over considerable distances in complex physical environments with high noise levels, resulting in degradation of the temporal pattern of stimuli, unpredictable attenuation, reduced signal-to-noise levels, and degradation of cues used for sound localisation. Thus, a more naturalistic view of sensory coding must be taken, since the signals as broadcast by signallers are rarely equivalent to the effective stimuli encoded by the sensory system of receivers. The consequences of the environmental conditions for sensory coding are discussed.

Auditory midbrain coding of statistical learning that results from discontinuous sensory stimulation.

Detecting regular patterns in the environment, a process known as statistical learning, is essential for survival. Neuronal adaptation is a key mechanism in the detection of patterns that are continuously repeated across short (seconds to minutes) temporal windows. Here, we found in mice that a subcortical structure in the auditory midbrain was sensitive to patterns that were repeated discontinuously, in a temporally sparse manner, across windows of minutes to hours. Using a combination of behavioral, electrophysiological, and molecular approaches, we found changes in neuronal response gain that varied in mechanism with the degree of sound predictability and resulted in changes in frequency coding. Analysis of population activity (structural tuning) revealed an increase in frequency classification accuracy in the context of increased overlap in responses across frequencies. The increase in accuracy and overlap was paralleled at the behavioral level in an increase in generalization in the absence of diminished discrimination. Gain modulation was accompanied by changes in gene and protein expression, indicative of long-term plasticity. Physiological changes were largely independent of corticofugal feedback, and no changes were seen in upstream cochlear nucleus responses, suggesting a key role of the auditory midbrain in sensory gating. Subsequent behavior demonstrated learning of predictable and random patterns and their importance in auditory conditioning. Using longer timescales than previously explored, the combined data show that the auditory midbrain codes statistical learning of temporally sparse patterns, a process that is critical for the detection of relevant stimuli in the constant soundscape that the animal navigates through.

Spiking network optimized for word recognition in noise predicts auditory system hierarchy.

The auditory neural code is resilient to acoustic variability and capable of recognizing sounds amongst competing sound sources, yet, the transformations enabling noise robust abilities are largely unknown. We report that a hierarchical spiking neural network (HSNN) optimized to maximize word recognition accuracy in noise and multiple talkers predicts organizational hierarchy of the ascending auditory pathway. Comparisons with data from auditory nerve, midbrain, thalamus and cortex reveals that the optimal HSNN predicts several transformations of the ascending auditory pathway including a sequential loss of temporal resolution and synchronization ability, increasing sparseness, and selectivity. The optimal organizational scheme enhances performance by selectively filtering out noise and fast temporal cues such as voicing periodicity, that are not directly relevant to the word recognition task. An identical network arranged to enable high information transfer fails to predict auditory pathway organization and has substantially poorer performance. Furthermore, conventional single-layer linear and nonlinear receptive field networks that capture the overall feature extraction of the HSNN fail to achieve similar performance. The findings suggest that the auditory pathway hierarchy and its sequential nonlinear feature extraction computations enhance relevant cues while removing non-informative sources of noise, thus enhancing the representation of sounds in noise impoverished conditions.

Olfactory Influences on Visual Categorization: Behavioral and ERP Evidence.

Visual stimuli often dominate nonvisual stimuli during multisensory perception. Evidence suggests higher cognitive processes prioritize visual over nonvisual stimuli during divided attention. Visual stimuli should thus be disproportionally distracting when processing incongruent cross-sensory stimulus pairs. We tested this assumption by comparing visual processing with olfaction, a "primitive" sensory channel that detects potentially hazardous chemicals by alerting attention. Behavioral and event-related brain potentials (ERPs) were assessed in a bimodal object categorization task with congruent or incongruent odor-picture pairings and a delayed auditory target that indicated whether olfactory or visual cues should be categorized. For congruent pairings, accuracy was higher for visual compared to olfactory decisions. However, for incongruent pairings, reaction times (RTs) were faster for olfactory decisions. Behavioral results suggested that incongruent odors interfered more with visual decisions, thereby providing evidence for an "olfactory dominance" effect. Categorization of incongruent pairings engendered a late "slow wave" ERP effect. Importantly, this effect had a later amplitude peak and longer latency during visual decisions, likely reflecting additional categorization effort for visual stimuli in the presence of incongruent odors. In sum, contrary to what might be inferred from theories of "visual dominance," incongruent odors may in fact uniquely attract mental processing resources during perceptual incongruence.

Multivariate resting-state functional connectivity predicts response to cognitive behavioral therapy in obsessive-compulsive disorder.

Cognitive behavioral therapy (CBT) is an effective treatment for many with obsessive-compulsive disorder (OCD). However, response varies considerably among individuals. Attaining a means to predict an individual's potential response would permit clinicians to more prudently allocate resources for this often stressful and time-consuming treatment. We collected resting-state functional magnetic resonance imaging from adults with OCD before and after 4 weeks of intensive daily CBT. We leveraged machine learning with cross-validation to assess the power of functional connectivity (FC) patterns to predict individual posttreatment OCD symptom severity. Pretreatment FC patterns within the default mode network and visual network significantly predicted posttreatment OCD severity, explaining up to 67% of the variance. These networks were stronger predictors than pretreatment clinical scores. Results have clinical implications for developing personalized medicine approaches to identifying individual OCD patients who will maximally benefit from intensive CBT.