Nearest neighbours reveal fast and slow components of motor learning.

Changes in behaviour resulting from environmental influences, development and learning1-5 are commonly quantified on the basis of a few hand-picked features2-4,6,7 (for example, the average pitch of acoustic vocalizations3), assuming discrete classes of behaviours (such as distinct vocal syllables)2,3,8-10. However, such methods generalize poorly across different behaviours and model systems and may miss important components of change. Here we present a more-general account of behavioural change that is based on nearest-neighbour statistics11-13, and apply it to song development in a songbird, the zebra finch3. First, we introduce the concept of 'repertoire dating', whereby each rendition of a behaviour (for example, each vocalization) is assigned a repertoire time, reflecting when similar renditions were typical in the behavioural repertoire. Repertoire time isolates the components of vocal variability that are congruent with long-term changes due to vocal learning and development, and stratifies the behavioural repertoire into 'regressions', 'anticipations' and 'typical renditions'. Second, we obtain a holistic, yet low-dimensional, description of vocal change in terms of a stratified 'behavioural trajectory', revealing numerous previously unrecognized components of behavioural change on fast and slow timescales, as well as distinct patterns of overnight consolidation1,2,4,14,15 across the behavioral repertoire. We find that diurnal changes in regressions undergo only weak consolidation, whereas anticipations and typical renditions consolidate fully. Because of its generality, our nonparametric description of how behaviour evolves relative to itself-rather than to a potentially arbitrary, experimenter-defined goal2,3,14,16-appears well suited for comparing learning and change across behaviours and species17,18, as well as biological and artificial systems5.

A Neural Code That Is Isometric to Vocal Output and Correlates with Its Sensory Consequences.

What cortical inputs are provided to motor control areas while they drive complex learned behaviors? We study this question in the nucleus interface of the nidopallium (NIf), which is required for normal birdsong production and provides the main source of auditory input to HVC, the driver of adult song. In juvenile and adult zebra finches, we find that spikes in NIf projection neurons precede vocalizations by several tens of milliseconds and are insensitive to distortions of auditory feedback. We identify a local isometry between NIf output and vocalizations: quasi-identical notes produced in different syllables are preceded by highly similar NIf spike patterns. NIf multiunit firing during song precedes responses in auditory cortical neurons by about 50 ms, revealing delayed congruence between NIf spiking and a neural representation of auditory feedback. Our findings suggest that NIf codes for imminent acoustic events within vocal performance.

Reconstruction of vocal interactions in a group of small songbirds.

The main obstacle for investigating vocal interactions in vertebrates is the difficulty of discriminating individual vocalizations of rapidly moving, sometimes simultaneously vocalizing individuals. We developed a method of recording and analyzing individual vocalizations in free-ranging animals using ultraminiature back-attached sound and acceleration recorders. Our method allows the separation of zebra finch vocalizations irrespective of background noise and the number of vocalizing animals nearby.

Evidence for a causal inverse model in an avian cortico-basal ganglia circuit.

Learning by imitation is fundamental to both communication and social behavior and requires the conversion of complex, nonlinear sensory codes for perception into similarly complex motor codes for generating action. To understand the neural substrates underlying this conversion, we study sensorimotor transformations in songbird cortical output neurons of a basal-ganglia pathway involved in song learning. Despite the complexity of sensory and motor codes, we find a simple, temporally specific, causal correspondence between them. Sensory neural responses to song playback mirror motor-related activity recorded during singing, with a temporal offset of roughly 40 ms, in agreement with short feedback loop delays estimated using electrical and auditory stimulation. Such matching of mirroring offsets and loop delays is consistent with a recent Hebbian theory of motor learning and suggests that cortico-basal ganglia pathways could support motor control via causal inverse models that can invert the rich correspondence between motor exploration and sensory feedback.

A higher sensory brain region is involved in reversing reinforcement-induced vocal changes in a songbird.

Many animals exhibit flexible behaviors that they can adjust to increase reward or avoid harm (learning by positive or aversive reinforcement). But what neural mechanisms allow them to restore their original behavior (motor program) after reinforcement is withdrawn? One possibility is that motor restoration relies on brain areas that have a role in memorization but no role in either motor production or in sensory processing relevant for expressing the behavior and its refinement. We investigated the role of a higher auditory brain area in the songbird for modifying and restoring the stereotyped adult song. We exposed zebra finches to aversively reinforcing white noise stimuli contingent on the pitch of one of their stereotyped song syllables. In response, birds significantly changed the pitch of that syllable to avoid the aversive reinforcer. After we withdrew reinforcement, birds recovered their original song within a few days. However, we found that large bilateral lesions in the caudal medial nidopallium (NCM, a high auditory area) impaired recovery of the original pitch even several weeks after withdrawal of the reinforcing stimuli. Because NCM lesions spared both successful noise-avoidance behavior and birds' auditory discrimination ability, our results show that NCM is not needed for directed motor changes or for auditory discriminative processing, but is implied in memorizing or recalling the memory of the recent song target.

Linear Methods for Efficient and Fast Separation of Two Sources Recorded with a Single Microphone.

This letter addresses the problem of separating two speakers from a single microphone recording. Three linear methods are tested for source separation, all of which operate directly on sound spectrograms: (1) eigenmode analysis of covariance difference to identify spectro-temporal features associated with large variance for one source and small variance for the other source; (2) maximum likelihood demixing in which the mixture is modeled as the sum of two gaussian signals and maximum likelihood is used to identify the most likely sources; and (3) suppression-regression, in which autoregressive models are trained to reproduce one source and suppress the other. These linear approaches are tested on the problem of separating a known male from a known female speaker. The performance of these algorithms is assessed in terms of the residual error of estimated source spectrograms, waveform signal-to-noise ratio, and perceptual evaluation of speech quality scores. This work shows that the algorithms compare favorably to nonlinear approaches such as nonnegative sparse coding in terms of simplicity, performance, and suitability for real-time implementations, and they provide benchmark solutions for monaural source separation tasks.

Dynamic alignment models for neural coding.

Recently, there have been remarkable advances in modeling the relationships between the sensory environment, neuronal responses, and behavior. However, most models cannot encompass variable stimulus-response relationships such as varying response latencies and state or context dependence of the neural code. Here, we consider response modeling as a dynamic alignment problem and model stimulus and response jointly by a mixed pair hidden Markov model (MPH). In MPHs, multiple stimulus-response relationships (e.g., receptive fields) are represented by different states or groups of states in a Markov chain. Each stimulus-response relationship features temporal flexibility, allowing modeling of variable response latencies, including noisy ones. We derive algorithms for learning of MPH parameters and for inference of spike response probabilities. We show that some linear-nonlinear Poisson cascade (LNP) models are a special case of MPHs. We demonstrate the efficiency and usefulness of MPHs in simulations of both jittered and switching spike responses to white noise and natural stimuli. Furthermore, we apply MPHs to extracellular single and multi-unit data recorded in cortical brain areas of singing birds to showcase a novel method for estimating response lag distributions. MPHs allow simultaneous estimation of receptive fields, latency statistics, and hidden state dynamics and so can help to uncover complex stimulus response relationships that are subject to variable timing and involve diverse neural codes.

Neural processing of auditory feedback during vocal practice in a songbird.

Songbirds are capable of vocal learning and communication and are ideally suited to the study of neural mechanisms of complex sensory and motor processing. Vocal communication in a noisy bird colony and vocal learning of a specific song template both require the ability to monitor auditory feedback to distinguish self-generated vocalizations from external sounds and to identify mismatches between the developing song and a memorized template acquired from a tutor. However, neurons that respond to auditory feedback from vocal output have not been found in song-control areas despite intensive searching. Here we investigate feedback processing outside the traditional song system, in single auditory forebrain neurons of juvenile zebra finches that were in a late developmental stage of song learning. Overall, we found similarity of spike responses during singing and during playback of the bird's own song, with song responses commonly leading by a few milliseconds. However, brief time-locked acoustic perturbations of auditory feedback revealed complex sensitivity that could not be predicted from passive playback responses. Some neurons that responded to playback perturbations did not respond to song perturbations, which is reminiscent of sensory-motor mirror neurons. By contrast, some neurons were highly feedback sensitive in that they responded vigorously to song perturbations, but not to unperturbed songs or perturbed playback. These findings suggest that a computational function of forebrain auditory areas may be to detect errors between actual feedback and mirrored feedback deriving from an internal model of the bird's own song or that of its tutor. Such feedback-sensitive spikes could constitute the key signals that trigger adaptive motor responses to song disruptions or reinforce exploratory motor gestures for vocal learning.

Goal-directed vocal planning in a songbird.

Songbirds' vocal mastery is impressive, but to what extent is it a result of practice? Can they, based on experienced mismatch with a known target, plan the necessary changes to recover the target in a practice-free manner without intermittently singing? In adult zebra finches, we drive the pitch of a song syllable away from its stable (baseline) variant acquired from a tutor, then we withdraw reinforcement and subsequently deprive them of singing experience by muting or deafening. In this deprived state, birds do not recover their baseline song. However, they revert their songs toward the target by about 1 standard deviation of their recent practice, provided the sensory feedback during the latter signaled a pitch mismatch with the target. Thus, targeted vocal plasticity does not require immediate sensory experience, showing that zebra finches are capable of goal-directed vocal planning.

Learning the sound inventory of a complex vocal skill via an intrinsic reward.

Reinforcement learning (RL) is thought to underlie the acquisition of vocal skills like birdsong and speech, where sounding like one's "tutor" is rewarding. However, what RL strategy generates the rich sound inventories for song or speech? We find that the standard actor-critic model of birdsong learning fails to explain juvenile zebra finches' efficient learning of multiple syllables. However, when we replace a single actor with multiple independent actors that jointly maximize a common intrinsic reward, then birds' empirical learning trajectories are accurately reproduced. The influence of each actor (syllable) on the magnitude of global reward is competitively determined by its acoustic similarity to target syllables. This leads to each actor matching the target it is closest to and, occasionally, to the competitive exclusion of an actor from the learning process (i.e., the learned song). We propose that a competitive-cooperative multi-actor RL (MARL) algorithm is key for the efficient learning of the action inventory of a complex skill.

Perceptron learning rule derived from spike-frequency adaptation and spike-time-dependent plasticity.

It is widely believed that sensory and motor processing in the brain is based on simple computational primitives rooted in cellular and synaptic physiology. However, many gaps remain in our understanding of the connections between neural computations and biophysical properties of neurons. Here, we show that synaptic spike-time-dependent plasticity (STDP) combined with spike-frequency adaptation (SFA) in a single neuron together approximate the well-known perceptron learning rule. Our calculations and integrate-and-fire simulations reveal that delayed inputs to a neuron endowed with STDP and SFA precisely instruct neural responses to earlier arriving inputs. We demonstrate this mechanism on a developmental example of auditory map formation guided by visual inputs, as observed in the external nucleus of the inferior colliculus (ICX) of barn owls. The interplay of SFA and STDP in model ICX neurons precisely transfers the tuning curve from the visual modality onto the auditory modality, demonstrating a useful computation for multimodal and sensory-guided processing.

Interactive extraction of diverse vocal units from a planar embedding without the need for prior sound segmentation.

Annotating and proofreading data sets of complex natural behaviors such as vocalizations are tedious tasks because instances of a given behavior need to be correctly segmented from background noise and must be classified with minimal false positive error rate. Low-dimensional embeddings have proven very useful for this task because they can provide a visual overview of a data set in which distinct behaviors appear in different clusters. However, low-dimensional embeddings introduce errors because they fail to preserve distances; and embeddings represent only objects of fixed dimensionality, which conflicts with vocalizations that have variable dimensions stemming from their variable durations. To mitigate these issues, we introduce a semi-supervised, analytical method for simultaneous segmentation and clustering of vocalizations. We define a given vocalization type by specifying pairs of high-density regions in the embedding plane of sound spectrograms, one region associated with vocalization onsets and the other with offsets. We demonstrate our two-neighborhood (2N) extraction method on the task of clustering adult zebra finch vocalizations embedded with UMAP. We show that 2N extraction allows the identification of short and long vocal renditions from continuous data streams without initially committing to a particular segmentation of the data. Also, 2N extraction achieves much lower false positive error rate than comparable approaches based on a single defining region. Along with our method, we present a graphical user interface (GUI) for visualizing and annotating data.

Regulation of learned vocal behavior by an auditory motor cortical nucleus in juvenile zebra finches.

In the process of song learning, songbirds such as the zebra finch shape their initial soft and poorly formed vocalizations (subsong) first into variable plastic songs with a discernable recurring motif and then into highly stereotyped adult songs. A premotor brain area critically involved in plastic and adult song production is the cortical nucleus HVC. One of HVC's primary afferents, the nucleus interface of the nidopallium (NIf), provides a significant source of auditory input to HVC. However, the premotor involvement of NIf has not been extensively studied yet. Here we report that brief and reversible pharmacological inactivation of NIf in juvenile birds leads to transient degradation of plastic song toward subsong, as revealed by spectral and temporal song features. No such song degradation is seen following NIf inactivation in adults. However, in both juveniles and adults NIf inactivation leads to a transient decrease in song stereotypy. Our findings reveal a contribution of NIf to song production in juveniles that agrees with its known role in adults in mediating thalamic drive to downstream vocal motor areas during sleep.

Rapid interhemispheric switching during vocal production in a songbird.

To generate complex bilateral motor patterns such as those underlying birdsong, neural activity must be highly coordinated across the two cerebral hemispheres. However, it remains largely elusive how this coordination is achieved given that interhemispheric communication between song-control areas in the avian cerebrum is restricted to projections received from bilaterally connecting areas in the mid- and hindbrain. By electrically stimulating cerebral premotor areas in zebra finches, we find that behavioral effectiveness of stimulation rapidly switches between hemispheres. In time intervals in which stimulation in one hemisphere tends to distort songs, stimulation in the other hemisphere is mostly ineffective, revealing an idiosyncratic form of motor dominance that bounces back and forth between hemispheres like a virtual ping-pong ball. The intervals of lateralized effectiveness are broadly distributed and are unrelated to simple spectral and temporal song features. Such interhemispheric switching could be an important dynamical aspect of neural coordination that may have evolved from simpler pattern generator circuits.

Sensory substitution reveals a manipulation bias.

Sensory substitution is a promising therapeutic approach for replacing a missing or diseased sensory organ by translating inaccessible information into another sensory modality. However, many substitution systems are not well accepted by subjects. To explore the effect of sensory substitution on voluntary action repertoires and their associated affective valence, we study deaf songbirds to which we provide visual feedback as a substitute of auditory feedback. Surprisingly, deaf birds respond appetitively to song-contingent binary visual stimuli. They skillfully adapt their songs to increase the rate of visual stimuli, showing that auditory feedback is not required for making targeted changes to vocal repertoires. We find that visually instructed song learning is basal-ganglia dependent. Because hearing birds respond aversively to the same visual stimuli, sensory substitution reveals a preference for actions that elicit sensory feedback over actions that do not, suggesting that substitution systems should be designed to exploit the drive to manipulate.

Undirected singing rate as a non-invasive tool for welfare monitoring in isolated male zebra finches.

Research on the songbird zebra finch (Taeniopygia guttata) has advanced our behavioral, hormonal, neuronal, and genetic understanding of vocal learning. However, little is known about the impact of typical experimental manipulations on the welfare of these birds. Here we explore whether the undirected singing rate can be used as an indicator of welfare. We tested this idea by performing a post hoc analysis of singing behavior in isolated male zebra finches subjected to interactive white noise, to surgery, or to tethering. We find that the latter two experimental manipulations transiently but reliably decreased singing rates. By contraposition, we infer that a high-sustained singing rate is suggestive of successful coping or improved welfare in these experiments. Our analysis across more than 300 days of song data suggests that a singing rate above a threshold of several hundred song motifs per day implies an absence of an acute stressor or a successful coping with stress. Because singing rate can be measured in a completely automatic fashion, its observation can help to reduce experimenter bias in welfare monitoring. Because singing rate measurements are non-invasive, we expect this study to contribute to the refinement of the current welfare monitoring tools in zebra finches.

Fast Retrograde Access to Projection Neuron Circuits Underlying Vocal Learning in Songbirds.

Understanding the structure and function of neural circuits underlying speech and language is a vital step toward better treatments for diseases of these systems. Songbirds, among the few animal orders that share with humans the ability to learn vocalizations from a conspecific, have provided many insights into the neural mechanisms of vocal development. However, research into vocal learning circuits has been hindered by a lack of tools for rapid genetic targeting of specific neuron populations to meet the quick pace of developmental learning. Here, we present a viral tool that enables fast and efficient retrograde access to projection neuron populations. In zebra finches, Bengalese finches, canaries, and mice, we demonstrate fast retrograde labeling of cortical or dopaminergic neurons. We further demonstrate the suitability of our construct for detailed morphological analysis, for in vivo imaging of calcium activity, and for multi-color brainbow labeling.

Spikes and bursts in two types of thalamic projection neurons differentially shape sleep patterns and auditory responses in a songbird.

In mammals, the thalamus plays important roles for cortical processing, such as relay of sensory information and induction of rhythmical firing during sleep. In neurons of the avian cerebrum, in analogy with cortical up and down states, complex patterns of regular-spiking and dense-bursting modes are frequently observed during sleep. However, the roles of thalamic inputs for shaping these firing modes are largely unknown. A suspected key player is the avian thalamic nucleus uvaeformis (Uva). Uva is innervated by polysensory input, receives indirect cerebral feedback via the midbrain, and projects to the cerebrum via two distinct pathways. Using pharmacological manipulation, electrical stimulation, and extracellular recordings of Uva projection neurons, we study the involvement of Uva in zebra finches for the generation of spontaneous activity and auditory responses in premotor area HVC (used as a proper name) and the downstream robust nucleus of the arcopallium (RA). In awake and sleeping birds, we find that single Uva spikes suppress and spike bursts enhance spontaneous and auditory-evoked bursts in HVC and RA neurons. Strong burst suppression is mediated mainly via tonically firing HVC-projecting Uva neurons, whereas a fast burst drive is mediated indirectly via Uva neurons projecting to the nucleus interface of the nidopallium. Our results reveal that cerebral sleep-burst epochs and arousal-related burst suppression are both shaped by sophisticated polysynaptic thalamic mechanisms.

Spike sorting with hidden Markov models.

The ability to detect and sort overlapping spike waveforms in extracellular recordings is key to studies of neural coding at high spatial and temporal resolution. Most spike-sorting algorithms are based on initial spike detection (e.g. by a voltage threshold) and subsequent waveform classification. Much effort has been devoted to the clustering step, despite the fact that conservative spike detection is notoriously difficult in low signal-to-noise conditions and often entails many spike misses. Hidden Markov models (HMMs) can serve as generative models for continuous extracellular data records. These models naturally combine the spike detection and classification steps into a single computational procedure. They unify the advantages of independent component analysis (ICA) and overlap-search algorithms because they blindly perform source separation even in cases where several neurons are recorded on a single electrode. We apply HMMs to artificially generated data and to extracellular signals recorded with glass electrodes. We show that in comparison with state-of-art spike-sorting algorithms, HMM-based spike sorting exhibits a comparable number of false positive spike classifications but many fewer spike misses.

Spike correlations in a songbird agree with a simple markov population model.

The relationships between neural activity at the single-cell and the population levels are of central importance for understanding neural codes. In many sensory systems, collective behaviors in large cell groups can be described by pairwise spike correlations. Here, we test whether in a highly specialized premotor system of songbirds, pairwise spike correlations themselves can be seen as a simple corollary of an underlying random process. We test hypotheses on connectivity and network dynamics in the motor pathway of zebra finches using a high-level population model that is independent of detailed single-neuron properties. We assume that neural population activity evolves along a finite set of states during singing, and that during sleep population activity randomly switches back and forth between song states and a single resting state. Individual spike trains are generated by associating with each of the population states a particular firing mode, such as bursting or tonic firing. With an overall modification of one or two simple control parameters, the Markov model is able to reproduce observed firing statistics and spike correlations in different neuron types and behavioral states. Our results suggest that song- and sleep-related firing patterns are identical on short time scales and result from random sampling of a unique underlying theme. The efficiency of our population model may apply also to other neural systems in which population hypotheses can be tested on recordings from small neuron groups.