Acoustic Pattern Recognition and Courtship Songs: Insights from Insects.

Across the animal kingdom, social interactions rely on sound production and perception. From simple cricket chirps to more elaborate bird songs, animals go to great lengths to communicate information critical for reproduction and survival via acoustic signals. Insects produce a wide array of songs to attract a mate, and the intended receivers must differentiate these calls from competing sounds, analyze the quality of the sender from spectrotemporal signal properties, and then determine how to react. Insects use numerically simple nervous systems to analyze and respond to courtship songs, making them ideal model systems for uncovering the neural mechanisms underlying acoustic pattern recognition. We highlight here how the combination of behavioral studies and neural recordings in three groups of insects-crickets, grasshoppers, and fruit flies-reveals common strategies for extracting ethologically relevant information from acoustic patterns and how these findings might translate to other systems.

Drosophila females have an acoustic preference for symmetric males.

SignificanceTheoretically, symmetry in bilateral animals is subject to sexual selection, since it can serve as a proxy for genetic quality of competing mates during mate choice. Here, we report female preference for symmetric males in Drosophila, using a mate-choice paradigm where males with environmentally or genetically induced wing asymmetry were competed. Analysis of courtship songs revealed that males with asymmetric wings produced songs with asymmetric features that served as acoustic cues, facilitating this female preference. Females experimentally evolved in the absence of mate choice lost this preference for symmetry, suggesting that it is maintained by sexual selection.

Sex-specific speed-accuracy trade-offs shape neural processing of acoustic signals in a grasshopper.

Speed-accuracy trade-offs-being fast at the risk of being wrong-are fundamental to many decisions and natural selection is expected to resolve these trade-offs according to the costs and benefits of behaviour. We here test the prediction that females and males should integrate information from courtship signals differently because they experience different pay-offs along the speed-accuracy continuum. We fitted a neural model of decision making (a drift-diffusion model of integration to threshold) to behavioural data from the grasshopper Chorthippus biguttulus to determine the parameters of temporal integration of acoustic directional information used by male grasshoppers to locate receptive females. The model revealed that males had a low threshold for initiating a turning response, yet a large integration time constant enabled them to continue to gather information when cues were weak. This contrasts with parameters estimated for females of the same species when evaluating potential mates, in which response thresholds were much higher and behaviour was strongly influenced by unattractive stimuli. Our results reveal differences in neural integration consistent with the sex-specific costs of mate search: males often face competition and need to be fast, while females often pay high error costs and need to be deliberate.

Dynamic sensory cues shape song structure in Drosophila.

The generation of acoustic communication signals is widespread across the animal kingdom, and males of many species, including Drosophilidae, produce patterned courtship songs to increase their chance of success with a female. For some animals, song structure can vary considerably from one rendition to the next; neural noise within pattern generating circuits is widely assumed to be the primary source of such variability, and statistical models that incorporate neural noise are successful at reproducing the full variation present in natural songs. In direct contrast, here we demonstrate that much of the pattern variability in Drosophila courtship song can be explained by taking into account the dynamic sensory experience of the male. In particular, using a quantitative behavioural assay combined with computational modelling, we find that males use fast modulations in visual and self-motion signals to pattern their songs, a relationship that we show is evolutionarily conserved. Using neural circuit manipulations, we also identify the pathways involved in song patterning choices and show that females are sensitive to song features. Our data not only demonstrate that Drosophila song production is not a fixed action pattern, but establish Drosophila as a valuable new model for studies of rapid decision-making under both social and naturalistic conditions.

Experimental and statistical reevaluation provides no evidence for Drosophila courtship song rhythms.

From 1980 to 1992, a series of influential papers reported on the discovery, genetics, and evolution of a periodic cycling of the interval between Drosophila male courtship song pulses. The molecular mechanisms underlying this periodicity were never described. To reinitiate investigation of this phenomenon, we previously performed automated segmentation of songs but failed to detect the proposed rhythm [Arthur BJ, et al. (2013) BMC Biol 11:11; Stern DL (2014) BMC Biol 12:38]. Kyriacou et al. [Kyriacou CP, et al. (2017) Proc Natl Acad Sci USA 114:1970-1975] report that we failed to detect song rhythms because (i) our flies did not sing enough and (ii) our segmenter did not identify many of the song pulses. Kyriacou et al. manually annotated a subset of our recordings and reported that two strains displayed rhythms with genotype-specific periodicity, in agreement with their original reports. We cannot replicate this finding and show that the manually annotated data, the original automatically segmented data, and a new dataset provide no evidence for either the existence of song rhythms or song periodicity differences between genotypes. Furthermore, we have reexamined our methods and analysis and find that our automated segmentation method was not biased to prevent detection of putative song periodicity. We conclude that there is no evidence for the existence of Drosophila courtship song rhythms.

Relative weighting of acoustic information during mating decisions in grasshoppers indicates signatures of sexual selection.

The decision with whom to mate is crucial in determining an individual's fitness and is often based on the evaluation of visual or acoustic displays produced during courtship. Accordingly, the algorithms for evaluating such courtship signals are shaped by sexual selection and should reflect the expected benefits and costs of mating: signals bearing heterospecific features should be rapidly rejected, since mating would produce no fertile offspring, while signals resembling conspecific ones should be weighted proportional to mate quality. We test these hypotheses in females of the grasshopper Chorthippus biguttulus who assess males by their song, which is produced as a sequence of subunits with species and individual specific signatures. We present mixed sequences of subunits with conflicting cues and use a computational model of decision-making to infer how sensory information is weighted and integrated over the song. Consistent with our hypothesis, females do weight sensory cues according to the expected fitness benefits/costs: heterospecific subunits are weighted particularly negatively and lead to a rejection of the male early in the song. Conspecific subunits are weighted moderately, permitting a more complete evaluation of the full song. However, there exists an overall negative bias against mating, possible causes of which are discussed.

Asymmetrical integration of sensory information during mating decisions in grasshoppers.

Decision-making processes, like all traits of an organism, are shaped by evolution; they thus carry a signature of the selection pressures associated with choice behaviors. The way sexual communication signals are integrated during courtship likely reflects the costs and benefits associated with mate choice. Here, we study the evaluation of male song by females during acoustic courtship in grasshoppers. Using playback experiments and computational modeling we find that information of different valence (attractive vs. nonattractive) is weighted asymmetrically: while information associated with nonattractive features has large weight, attractive features add little to the decision to mate. Accordingly, nonattractive features effectively veto female responses. Because attractive features have so little weight, the model suggests that female responses are frequently driven by integration noise. Asymmetrical weighting of negative and positive information may reflect the fitness costs associated with mating with a nonattractive over an attractive singer, which are also highly asymmetrical. In addition, nonattractive cues tend to be more salient and therefore more reliable. Hence, information provided by them should be weighted more heavily. Our findings suggest that characterizing the integration of sensory information during a natural behavior has the potential to provide valuable insights into the selective pressures shaping decision-making during evolution.

Context-dependent coding and gain control in the auditory system of crickets.

Sensory systems process stimuli that greatly vary in intensity and complexity. To maintain efficient information transmission, neural systems need to adjust their properties to these different sensory contexts, yielding adaptive or stimulus-dependent codes. Here, we demonstrated adaptive spectrotemporal tuning in a small neural network, i.e. the peripheral auditory system of the cricket. We found that tuning of cricket auditory neurons was sharper for complex multi-band than for simple single-band stimuli. Information theoretical considerations revealed that this sharpening improved information transmission by separating the neural representations of individual stimulus components. A network model inspired by the structure of the cricket auditory system suggested two putative mechanisms underlying this adaptive tuning: a saturating peripheral nonlinearity could change the spectral tuning, whereas broad feed-forward inhibition was able to reproduce the observed adaptive sharpening of temporal tuning. Our study revealed a surprisingly dynamic code usually found in more complex nervous systems and suggested that stimulus-dependent codes could be implemented using common neural computations.

Computational principles underlying recognition of acoustic signals in grasshoppers and crickets.

Grasshoppers and crickets independently evolved hearing organs and acoustic communication. They differ considerably in the organization of their auditory pathways, and the complexity of their songs, which are essential for mate attraction. Recent approaches aimed at describing the behavioral preference functions of females in both taxa by a simple modeling framework. The basic structure of the model consists of three processing steps: (1) feature extraction with a bank of 'LN models'-each containing a linear filter followed by a nonlinearity, (2) temporal integration, and (3) linear combination. The specific properties of the filters and nonlinearities were determined using a genetic learning algorithm trained on a large set of different song features and the corresponding behavioral response scores. The model showed an excellent prediction of the behavioral responses to the tested songs. Most remarkably, in both taxa the genetic algorithm found Gabor-like functions as the optimal filter shapes. By slight modifications of Gabor filters several types of preference functions could be modeled, which are observed in different cricket species. Furthermore, this model was able to explain several so far enigmatic results in grasshoppers. The computational approach offered a remarkably simple framework that can account for phenotypically rather different preference functions across several taxa.

Firing-rate resonances in the peripheral auditory system of the cricket, Gryllus bimaculatus.

In many communication systems, information is encoded in the temporal pattern of signals. For rhythmic signals that carry information in specific frequency bands, a neuronal system may profit from tuning its inherent filtering properties towards a peak sensitivity in the respective frequency range. The cricket Gryllus bimaculatus evaluates acoustic communication signals of both conspecifics and predators. The song signals of conspecifics exhibit a characteristic pulse pattern that contains only a narrow range of modulation frequencies. We examined individual neurons (AN1, AN2, ON1) in the peripheral auditory system of the cricket for tuning towards specific modulation frequencies by assessing their firing-rate resonance. Acoustic stimuli with a swept-frequency envelope allowed an efficient characterization of the cells' modulation transfer functions. Some of the examined cells exhibited tuned band-pass properties. Using simple computational models, we demonstrate how different, cell-intrinsic or network-based mechanisms such as subthreshold resonances, spike-triggered adaptation, as well as an interplay of excitation and inhibition can account for the experimentally observed firing-rate resonances. Therefore, basic neuronal mechanisms that share negative feedback as a common theme may contribute to selectivity in the peripheral auditory pathway of crickets that is designed towards mate recognition and predator avoidance.

Feature extraction and integration underlying perceptual decision making during courtship behavior.

Traditionally, perceptual decision making is studied in trained animals and carefully controlled tasks. Here, we sought to elucidate the stimulus features and their combination underlying a naturalistic behavior--female decision making during acoustic courtship in grasshoppers. Using behavioral data, we developed a model in which stimulus features were extracted by physiologically plausible models of sensory neurons from the time-varying stimulus. This sensory evidence was integrated over the stimulus duration and combined to predict the behavior. We show that decisions were determined by the interaction of an excitatory and a suppressive stimulus feature. The observed increase of behavioral response with stimulus intensity was the result of an increase of the excitatory feature's gain that was not controlled by an equivalent increase of the suppressive feature. Differences in how these two features were combined could explain interindividual variability. In addition, the mapping between the two stimulus features and different parameters of the song led us to re-evaluate the cues underlying acoustic communication. Our framework provided a rich and plausible explanation of behavior in terms of two stimulus cues that were extracted by models of sensory neurons and combined through excitatory-inhibitory interactions. We thus were able to link single neuron's feature selectivity and network computations with decision making in a natural task. This data-driven approach has the potential to advance our understanding of decision making in other systems and can inform the search for the neural correlates of behavior.

Efficient transformation of an auditory population code in a small sensory system.

Optimal coding principles are implemented in many large sensory systems. They include the systematic transformation of external stimuli into a sparse and decorrelated neuronal representation, enabling a flexible readout of stimulus properties. Are these principles also applicable to size-constrained systems, which have to rely on a limited number of neurons and may only have to fulfill specific and restricted tasks? We studied this question in an insect system--the early auditory pathway of grasshoppers. Grasshoppers use genetically fixed songs to recognize mates. The first steps of neural processing of songs take place in a small three-layer feed-forward network comprising only a few dozen neurons. We analyzed the transformation of the neural code within this network. Indeed, grasshoppers create a decorrelated and sparse representation, in accordance with optimal coding theory. Whereas the neuronal input layer is best read out as a summed population, a labeled-line population code for temporal features of the song is established after only two processing steps. At this stage, information about song identity is maximal for a population decoder that preserves neuronal identity. We conclude that optimal coding principles do apply to the early auditory system of the grasshopper, despite its size constraints. The inputs, however, are not encoded in a systematic, map-like fashion as in many larger sensory systems. Already at its periphery, part of the grasshopper auditory system seems to focus on behaviorally relevant features, and is in this property more reminiscent of higher sensory areas in vertebrates.

Nonlinear computations underlying temporal and population sparseness in the auditory system of the grasshopper.

Sparse coding schemes are employed by many sensory systems and implement efficient coding principles. Yet, the computations yielding sparse representations are often only partly understood. The early auditory system of the grasshopper produces a temporally and population-sparse representation of natural communication signals. To reveal the computations generating such a code, we estimated 1D and 2D linear-nonlinear models. We then used these models to examine the contribution of different model components to response sparseness. 2D models were better able to reproduce the sparseness measured in the system: while 1D models only captured 55% of the population sparseness at the network's output, 2D models accounted for 88% of it. Looking at the model structure, we could identify two types of computation, which increase sparseness. First, a sensitivity to the derivative of the stimulus and, second, the combination of a fast, excitatory and a slow, suppressive feature. Both were implemented in different classes of cells and increased the specificity and diversity of responses. The two types produced more transient responses and thereby amplified temporal sparseness. Additionally, the second type of computation contributed to population sparseness by increasing the diversity of feature selectivity through a wide range of delays between an excitatory and a suppressive feature. Both kinds of computation can be implemented through spike-frequency adaptation or slow inhibition-mechanisms found in many systems. Our results from the auditory system of the grasshopper are thus likely to reflect general principles underlying the emergence of sparse representations.

Computational principles underlying the recognition of acoustic signals in insects.

Many animals produce pulse-like signals during acoustic communication. These signals exhibit structure on two time scales: they consist of trains of pulses that are often broadcast in packets-so called chirps. Temporal parameters of the pulse and of the chirp are decisive for female preference. Despite these signals being produced by animals from many different taxa (e.g. frogs, grasshoppers, crickets, bushcrickets, flies), a general framework for their evaluation is still lacking. We propose such a framework, based on a simple and physiologically plausible model. The model consists of feature detectors, whose time-varying output is averaged over the signal and then linearly combined to yield the behavioral preference. We fitted this model to large data sets collected in two species of crickets and found that Gabor filters--known from visual and auditory physiology--explain the preference functions in these two species very well. We further explored the properties of Gabor filters and found a systematic relationship between parameters of the filters and the shape of preference functions. Although these Gabor filters were relatively short, they were also able to explain aspects of the preference for signal parameters on the longer time scale due to the integration step in our model. Our framework explains a wide range of phenomena associated with female preference for a widespread class of signals in an intuitive and physiologically plausible fashion. This approach thus constitutes a valuable tool to understand the functioning and evolution of communication systems in many species.

Flexible control of vocal timing in Carollia perspicillata bats enables escape from acoustic interference.

In natural environments, background noise can degrade the integrity of acoustic signals, posing a problem for animals that rely on their vocalizations for communication and navigation. A simple behavioral strategy to combat acoustic interference would be to restrict call emissions to periods of low-amplitude or no noise. Using audio playback and computational tools for the automated detection of over 2.5 million vocalizations from groups of freely vocalizing bats, we show that bats (Carollia perspicillata) can dynamically adapt the timing of their calls to avoid acoustic jamming in both predictably and unpredictably patterned noise. This study demonstrates that bats spontaneously seek out temporal windows of opportunity for vocalizing in acoustically crowded environments, providing a mechanism for efficient echolocation and communication in cluttered acoustic landscapes.

A small, computationally flexible network produces the phenotypic diversity of song recognition in crickets.

How neural networks evolved to generate the diversity of species-specific communication signals is unknown. For receivers of the signals, one hypothesis is that novel recognition phenotypes arise from parameter variation in computationally flexible feature detection networks. We test this hypothesis in crickets, where males generate and females recognize the mating songs with a species-specific pulse pattern, by investigating whether the song recognition network in the cricket brain has the computational flexibility to recognize different temporal features. Using electrophysiological recordings from the network that recognizes crucial properties of the pulse pattern on the short timescale in the cricket Gryllus bimaculatus, we built a computational model that reproduces the neuronal and behavioral tuning of that species. An analysis of the model's parameter space reveals that the network can provide all recognition phenotypes for pulse duration and pause known in crickets and even other insects. Phenotypic diversity in the model is consistent with known preference types in crickets and other insects, and arises from computations that likely evolved to increase energy efficiency and robustness of pattern recognition. The model's parameter to phenotype mapping is degenerate - different network parameters can create similar changes in the phenotype - which likely supports evolutionary plasticity. Our study suggests that computationally flexible networks underlie the diverse pattern recognition phenotypes, and we reveal network properties that constrain and support behavioral diversity.

Fast and accurate annotation of acoustic signals with deep neural networks.

Acoustic signals serve communication within and across species throughout the animal kingdom. Studying the genetics, evolution, and neurobiology of acoustic communication requires annotating acoustic signals: segmenting and identifying individual acoustic elements like syllables or sound pulses. To be useful, annotations need to be accurate, robust to noise, and fast.We here introduce DeepAudioSegmenter (DAS), a method that annotates acoustic signals across species based on a deep-learning derived hierarchical presentation of sound. We demonstrate the accuracy, robustness, and speed of DAS using acoustic signals with diverse characteristics from insects, birds, and mammals. DAS comes with a graphical user interface for annotating song, training the network, and for generating and proofreading annotations. The method can be trained to annotate signals from new species with little manual annotation and can be combined with unsupervised methods to discover novel signal types. DAS annotates song with high throughput and low latency for experimental interventions in realtime. Overall, DAS is a universal, versatile, and accessible tool for annotating acoustic communication signals.

Intensity invariance properties of auditory neurons compared to the statistics of relevant natural signals in grasshoppers.

The temporal pattern of amplitude modulations (AM) is often used to recognize acoustic objects. To identify objects reliably, intensity invariant representations have to be formed. We approached this problem within the auditory pathway of grasshoppers. We presented AM patterns modulated at different time scales and intensities. Metric space analysis of neuronal responses allowed us to determine how well, how invariantly, and at which time scales AM frequency is encoded. We find that in some neurons spike-count cues contribute substantially (20-60%) to the decoding of AM frequency at a single intensity. However, such cues are not robust when intensity varies. The general intensity invariance of the system is poor. However, there exists a range of AM frequencies around 83 Hz where intensity invariance of local interneurons is relatively high. In this range, natural communication signals exhibit much variation between species, suggesting an important behavioral role for this frequency band. We hypothesize, just as has been proposed for human speech, that the communication signals might have evolved to match the processing properties of the receivers. This contrasts with optimal coding theory, which postulates that neuronal systems are adapted to the statistics of the relevant signals.

The neural basis for a persistent internal state in Drosophila females.

Sustained changes in mood or action require persistent changes in neural activity, but it has been difficult to identify the neural circuit mechanisms that underlie persistent activity and contribute to long-lasting changes in behavior. Here, we show that a subset of Doublesex+ pC1 neurons in the Drosophila female brain, called pC1d/e, can drive minutes-long changes in female behavior in the presence of males. Using automated reconstruction of a volume electron microscopic (EM) image of the female brain, we map all inputs and outputs to both pC1d and pC1e. This reveals strong recurrent connectivity between, in particular, pC1d/e neurons and a specific subset of Fruitless+ neurons called aIPg. We additionally find that pC1d/e activation drives long-lasting persistent neural activity in brain areas and cells overlapping with the pC1d/e neural network, including both Doublesex+ and Fruitless+ neurons. Our work thus links minutes-long persistent changes in behavior with persistent neural activity and recurrent circuit architecture in the female brain.

Long-term mental states such as arousal and mood variations rely on persistent changes in the activity of certain neural circuits which have been difficult to identify. For instance, in male fruit flies, the activation of a particular circuit containing 'P1 neurons' can escalate aggressive and mating behaviors. However, less is known about the neural networks that underlie arousal in female flies. A group of female-specific, 'pC1 neurons' similar to P1 neurons could play this role, but it was unclear whether it could drive lasting changes in female fly behavior. To investigate this question, Deutsch et al. stimulated or shut down pC1 circuits in female flies, and then recorded the insects' interactions with male flies. Stimulation was accomplished using optogenetics, a technique which allows researchers to precisely control the activity of specially modified light-sensitive neurons. Silencing pC1 neurons in female flies diminished their interest in male partners and their suitor's courtship songs. Activating these neural circuits made the females more receptive to males; it also triggered long-lasting aggressive behaviors not typically observed in virgin females, such as shoving and chasing. Deutsch et al. then identified the brain cells that pC1 neurons connect to, discovering that these neurons are part of an interconnected circuit also formed of aIPg neurons ­ a population of fly brain cells that shows sex differences and is linked to female aggression. The brains of females were then imaged as pC1 neurons were switched on, revealing a persistent activity which outlasted the activation in circuits containing both pC1 and aIPg neurons. Thus, these results link neural circuit architecture to long lasting changes in neural activity, and ultimately, in behavior. Future experiments can build on these results to determine how this circuit is activated during natural social interactions.

Shared Song Detector Neurons in Drosophila Male and Female Brains Drive Sex-Specific Behaviors.

Males and females often produce distinct responses to the same sensory stimuli. How such differences arise-at the level of sensory processing or in the circuits that generate behavior-remains largely unresolved across sensory modalities. We address this issue in the acoustic communication system of Drosophila. During courtship, males generate time-varying songs, and each sex responds with specific behaviors. We characterize male and female behavioral tuning for all aspects of song and show that feature tuning is similar between sexes, suggesting sex-shared song detectors drive divergent behaviors. We then identify higher-order neurons in the Drosophila brain, called pC2, that are tuned for multiple temporal aspects of one mode of the male's song and drive sex-specific behaviors. We thus uncover neurons that are specifically tuned to an acoustic communication signal and that reside at the sensory-motor interface, flexibly linking auditory perception with sex-specific behavioral responses.