Learning the sound inventory of a complex vocal skill via an intrinsic reward.

Reinforcement learning (RL) is thought to underlie the acquisition of vocal skills like birdsong and speech, where sounding like one's "tutor" is rewarding. However, what RL strategy generates the rich sound inventories for song or speech? We find that the standard actor-critic model of birdsong learning fails to explain juvenile zebra finches' efficient learning of multiple syllables. However, when we replace a single actor with multiple independent actors that jointly maximize a common intrinsic reward, then birds' empirical learning trajectories are accurately reproduced. The influence of each actor (syllable) on the magnitude of global reward is competitively determined by its acoustic similarity to target syllables. This leads to each actor matching the target it is closest to and, occasionally, to the competitive exclusion of an actor from the learning process (i.e., the learned song). We propose that a competitive-cooperative multi-actor RL (MARL) algorithm is key for the efficient learning of the action inventory of a complex skill.

The Development of Structured Vocalizations in Songbirds and Humans: A Comparative Analysis.

Humans and songbirds face a common challenge: acquiring the complex vocal repertoire of their social group. Although humans are thought to be unique in their ability to convey symbolic meaning through speech, speech and birdsong are comparable in their acoustic complexity and the mastery with which the vocalizations of adults are acquired by young individuals. In this review, we focus on recent advances in the study of vocal development in humans and songbirds that shed new light on the emergence of distinct structural levels of vocal behavior and point to new possible parallels between both groups.

Animal Communication: Origins of Sequential Structure in Birdsong.

Culturally transmitted behaviors have an innate foundation, but the detailed sequential structure of such complex, acquired behaviors is often an outcome of historical accidents. New research has identified innate predispositions for structuring vocal sequences in culturally acquired birdsong.

Songbirds work around computational complexity by learning song vocabulary independently of sequence.

While acquiring motor skills, animals transform their plastic motor sequences to match desired targets. However, because both the structure and temporal position of individual gestures are adjustable, the number of possible motor transformations increases exponentially with sequence length. Identifying the optimal transformation towards a given target is therefore a computationally intractable problem. Here we show an evolutionary workaround for reducing the computational complexity of song learning in zebra finches. We prompt juveniles to modify syllable phonology and sequence in a learned song to match a newly introduced target song. Surprisingly, juveniles match each syllable to the most spectrally similar sound in the target, regardless of its temporal position, resulting in unnecessary sequence errors, that they later try to correct. Thus, zebra finches prioritize efficient learning of syllable vocabulary, at the cost of inefficient syntax learning. This strategy provides a non-optimal but computationally manageable solution to the task of vocal sequence learning.

Neural circuits. Inhibition protects acquired song segments during vocal learning in zebra finches.

Vocal imitation involves incorporating instructive auditory information into relevant motor circuits through processes that are poorly understood. In zebra finches, we found that exposure to a tutor's song drives spiking activity within premotor neurons in the juvenile, whereas inhibition suppresses such responses upon learning in adulthood. We measured inhibitory currents evoked by the tutor song throughout development while simultaneously quantifying each bird's learning trajectory. Surprisingly, we found that the maturation of synaptic inhibition onto premotor neurons is correlated with learning but not age. We used synthetic tutoring to demonstrate that inhibition is selective for specific song elements that have already been learned and not those still in refinement. Our results suggest that structured inhibition plays a crucial role during song acquisition, enabling a piece-by-piece mastery of complex tasks.

Stepwise acquisition of vocal combinatorial capacity in songbirds and human infants.

Human language, as well as birdsong, relies on the ability to arrange vocal elements in new sequences. However, little is known about the ontogenetic origin of this capacity. Here we track the development of vocal combinatorial capacity in three species of vocal learners, combining an experimental approach in zebra finches (Taeniopygia guttata) with an analysis of natural development of vocal transitions in Bengalese finches (Lonchura striata domestica) and pre-lingual human infants. We find a common, stepwise pattern of acquiring vocal transitions across species. In our first study, juvenile zebra finches were trained to perform one song and then the training target was altered, prompting the birds to swap syllable order, or insert a new syllable into a string. All birds solved these permutation tasks in a series of steps, gradually approximating the target sequence by acquiring new pairwise syllable transitions, sometimes too slowly to accomplish the task fully. Similarly, in the more complex songs of Bengalese finches, branching points and bidirectional transitions in song syntax were acquired in a stepwise fashion, starting from a more restrictive set of vocal transitions. The babbling of pre-lingual human infants showed a similar pattern: instead of a single developmental shift from reduplicated to variegated babbling (that is, from repetitive to diverse sequences), we observed multiple shifts, where each new syllable type slowly acquired a diversity of pairwise transitions, asynchronously over development. Collectively, these results point to a common generative process that is conserved across species, suggesting that the long-noted gap between perceptual versus motor combinatorial capabilities in human infants may arise partly from the challenges in constructing new pairwise vocal transitions.

Vocal exploration is locally regulated during song learning.

Exploratory variability is essential for sensorimotor learning, but it is not known how and at what timescales it is regulated. We manipulated song learning in zebra finches to experimentally control the requirements for vocal exploration in different parts of their song. We first trained birds to perform a one-syllable song, and once they mastered it, we added a new syllable to the song model. Remarkably, when practicing the modified song, birds rapidly alternated between high and low acoustic variability to confine vocal exploration to the newly added syllable. Furthermore, even within syllables, acoustic variability changed independently across song elements that were only milliseconds apart. Analysis of the entire vocal output during learning revealed that the variability of each song element decreased as it approached the target, correlating with momentary local distance from the target and less so with the overall distance within a syllable. We conclude that vocal error is computed locally in subsyllabic timescales and that song elements can be learned and crystallized independently. Songbirds have dedicated brain circuitry for vocal babbling in the anterior forebrain pathway (AFP), which generates exploratory song patterns that drive premotor neurons at the song nucleus RA. We hypothesize that either AFP adjusts the gain of vocal exploration in fine timescales or that the sensitivity of RA premotor neurons to AFP/HVC inputs varies across song elements.

Quantification of developmental birdsong learning from the subsyllabic scale to cultural evolution.

Quantitative analysis of behavior plays an important role in birdsong neuroethology, serving as a common denominator in studies spanning molecular to system-level investigation of sensory-motor conversion, developmental learning, and pattern generation in the brain. In this review, we describe the role of behavioral analysis in facilitating cross-level integration. Modern sound analysis approaches allow investigation of developmental song learning across multiple time scales. Combined with novel methods that allow experimental control of vocal changes, it is now possible to test hypotheses about mechanisms of vocal learning. Further, song analysis can be done at the population level across generations to track cultural evolution and multigenerational behavioral processes. Complementing the investigation of song development with noninvasive brain imaging technology makes it now possible to study behavioral dynamics at multiple levels side by side with developmental changes in brain connectivity and in auditory responses.

Ten ways to improve the quality of descriptions of whole-animal movement.

The demand for replicability of behavioral results across laboratories is viewed as a burden in behavior genetics. We demonstrate how it can become an asset offering a quantitative criterion that guides the design of better ways to describe behavior. Passing the high benchmark dictated by the replicability demand requires less stressful and less restraining experimental setups, less noisy data, individually customized cutoff points between the building blocks of movement, and less variable yet discriminative dynamic representations that would capture more faithfully the nature of the behavior, unmasking similarities and differences and revealing novel animal-centered measures. Here we review ten tools that enhance replicability without compromising discrimination. While we demonstrate the usefulness of these tools in the context of inbred mouse exploratory behavior they can readily be used in any study involving a high-resolution analysis of spatial behavior. Viewing replicability as a design concept and using the ten methodological improvements may prove useful in many fields not necessarily related to spatial behavior.

New replicable anxiety-related measures of wall vs center behavior of mice in the open field.

Anxiety is a widely studied psychiatric disorder and is thought to be a complex and multidimensional phenomenon. Sensitive behavioral discrimination of animal models of anxiety is crucial for the elucidation of the behavioral components of anxiety and the physiological processes that mediate them. Commonly used behavior paradigms of anxiety usually include only a few automatically collected measures; these do not exhaust the behavioral richness exhibited by animals, thus perhaps missing important differences between preparations. The aim of the present study was to expand the repertoire of automatically collected measures in a classical test of anxiety: behavior in relation to the wall in the open field. We present an algorithm, based on the Software for the Exploration of Exploration strategy, which automatically partitions the mouse path into intrinsically defined patterns of movement near the wall and in the center. These patterns are used to design new end points, which provide an articulated description of various aspects of behavior near the wall and in the center. Sixteen new end points were designed with data from C57BL/6J and DBA/2J mice tested in three laboratories. The strain differences in all end points were evaluated on another data set to assess their validity and were found to remain stable. Ten of the sixteen end points were found to discriminate between the two strains in a replicable manner. The entire set of end points can be used on various genetic and pharmacological models of anxiety with good prospects of providing fine discrimination in a replicable manner.

SEE locomotor behavior test discriminates C57BL/6J and DBA/2J mouse inbred strains across laboratories and protocol conditions.

Conventional tests of behavioral phenotyping frequently have difficulties differentiating certain genotypes and replicating these differences across laboratories and protocol conditions. This study explores the hypothesis that automated tests can be designed to quantify ethologically relevant behavior patterns that more readily characterize heritable and replicable phenotypes. It used SEE (Strategy for the Exploration of Exploration) to phenotype the locomotor behavior of the C57BL/6 and DBA/2 mouse inbred strains across 3 laboratories. The 2 genotypes differed in 15 different measures of behavior, none of which had a significant genotype-laboratory interaction. Within the same laboratory, most of these differences were replicated in additional experiments despite the test photoperiod phase being changed and saline being injected. Results suggest that well-designed tests may considerably enhance replicability across laboratories.

Darting behavior: a quantitative movement pattern designed for discrimination and replicability in mouse locomotor behavior.

In the open-field behavior of rodents, Software for Exploring Exploration (SEE) can be used for an explicit design of behavioral endpoints with high genotype discrimination and replicability across laboratories. This ability is demonstrated here in the development of a measure for darting behavior. The behavior of two common mouse inbred strains, C57BL/6J (B6) and DBA/2J (D2), was analyzed across three different laboratories, and under the effect of cocaine or amphetamine. "Darting" was defined as having higher acceleration during progression segments while moving less during stops. D2 mice darted significantly more than B6 mice in each laboratory, despite being significantly less active. These differences were maintained following cocaine administration (up to 20mg/kg) and only slightly altered by amphetamine (up to 5mg/kg) despite a several fold increase in activity. The replicability of darting behavior was confirmed in additional experiments distinct from those used for its design. The strategy leading to the darting measure may be used to develop additional discriminative and replicable endpoints of open-field behavior.