Automatic classification of dog barking using deep learning.

Barking and other dog vocalizations have acoustic properties related to emotions, physiological reactions, attitudes, or some particular internal states. In the field of intelligent audio analysis, researchers use methods based on signal processing and machine learning to analyze the digitized acoustic signals' properties and obtain relevant information. The present work describes a method to classify the identity, breed, age, sex, and context associated with each bark. This information can support the decisions of people who regularly interact with animals, such as dog trainers, veterinarians, rescuers, police, people with visual impairment. Our approach uses deep neural networks to generate trained models for each classification task. We worked with 19,643 barks recorded from 113 dogs of different breeds, ages and sexes. Our methodology consists of three stages. First, the pre-processing stage prepares the data and transforms it into the appropriate format for each classification model. Second, the characterization stage evaluates different representation models to identify the most suitable for each task. Third, the classification stage trains each classification model and selects the best hyperparameters. After tuning and training each model, we evaluated its performance. We analyzed the most relevant features extracted from the audio and the most appropriate deep neural network architecture for that feature type. Even if the application of our method is not ready for being used in ethological practice, our evaluation showed an outstanding performance of the proposed method, surpassing previous research results on this topic, providing the basis for further technological development.

Update on frequency decline of Northeast Pacific blue whale (Balaenoptera musculus) calls.

Worldwide, the frequency (pitch) of blue whale (Balaenoptera musculus) calls has been decreasing since first recorded in the 1960s. This frequency decline occurs over annual and inter-annual timescales and has recently been documented in other baleen whale species, yet it remains unexplained. In the Northeast Pacific, blue whales produce two calls, or units, that, when regularly repeated, are referred to as song: A and B calls. In this population, frequency decline has thus far only been examined in B calls. In this work, passive acoustic data collected in the Southern California Bight from 2006 to 2019 were examined to determine if A calls are also declining in frequency and whether the call pulse rate was similarly impacted. Additionally, frequency measurements were made for B calls to determine whether the rate of frequency decline is the same as was calculated when this phenomenon was first reported in 2009. We found that A calls decreased at a rate of 0.32 Hz yr-1 during this period and that B calls were still decreasing, albeit at a slower rate (0.27 Hz yr-1) than reported previously. The A call pulse rate also declined over the course of the study, at a rate of 0.006 pulses/s yr-1. With this updated information, we consider the various theories that have been proposed to explain frequency decline in blue whales. We conclude that no current theory adequately accounts for all aspects of this phenomenon and consider the role that individual perception of song frequency may play. To understand the cause behind call frequency decline, future studies might want to explore the function of these songs and the mechanism for their synchronization. The ubiquitous nature of the frequency shift phenomenon may indicate a consistent level of vocal plasticity and fine auditory processing abilities across baleen whale species.

Acoustic approach as an alternative to human-based survey in bird biodiversity monitoring in agricultural meadows.

Acoustic monitoring has been tested as an alternative to the traditional, human-based approach of surveying birds, however studies examining the effectiveness of different acoustic methods sometimes yield inconsistent results. In this study we examined whether bird biodiversity estimated by traditional surveys of birds differs to that obtained through soundscape surveys in meadow habitats that are of special agricultural importance, and whether acoustic monitoring can deliver reliable indicators of meadows and farmland bird biodiversity. We recorded soundscape and simultaneously surveyed birds by highly skilled human-observers within a fixed (50 m and 100 m) and unlimited radius using the point-count method twice in the breeding season at 74 recording sites located in meadows, in order to compare differences in (1) bird biodiversity estimation of meadow, farmland, songbird, and all bird species and (2) the detection rate of single bird species by these two methods. We found that recorders detected more species in comparison to the human-observers who surveyed birds within a fixed radius (50 and 100 m) and fewer when detection distance for human-observers was unlimited. We did not find significant differences in the number of meadow and farmland bird species detected by recorders and observers within a 100 m radius-the most often used fixed radius in traditional human based point-counts. We also showed how detection rate of 48 the most common bird species in our study differ between these two methods. Our study showed that an acoustic survey is equally effective as human observers surveying birds within a 100 m radius in estimation of farmland and meadow bird biodiversity. These groups of species are important for agricultural landscape and commonly used as indicators of habitat quality and its changes. Even though recorders rarely detect species that remain mostly silent during the observation periods, or species that are further distant than 100 m away, we recommend using acoustic soundscape recording methods as an equally effective and more easily standardised alternative for monitoring of farmland and meadow bird biodiversity. We propose adaptation of acoustic approach to long-term, large-scale monitoring by collecting acoustic data by non-specialists, including landowners and volunteers, and analysing them in a standardised way by units supervising monitoring of agriculture landscape.

Measuring context dependency in birdsong using artificial neural networks.

Context dependency is a key feature in sequential structures of human language, which requires reference between words far apart in the produced sequence. Assessing how long the past context has an effect on the current status provides crucial information to understand the mechanism for complex sequential behaviors. Birdsongs serve as a representative model for studying the context dependency in sequential signals produced by non-human animals, while previous reports were upper-bounded by methodological limitations. Here, we newly estimated the context dependency in birdsongs in a more scalable way using a modern neural-network-based language model whose accessible context length is sufficiently long. The detected context dependency was beyond the order of traditional Markovian models of birdsong, but was consistent with previous experimental investigations. We also studied the relation between the assumed/auto-detected vocabulary size of birdsong (i.e., fine- vs. coarse-grained syllable classifications) and the context dependency. It turned out that the larger vocabulary (or the more fine-grained classification) is assumed, the shorter context dependency is detected.

Bird population declines and species turnover are changing the acoustic properties of spring soundscapes.

Natural sounds, and bird song in particular, play a key role in building and maintaining our connection with nature, but widespread declines in bird populations mean that the acoustic properties of natural soundscapes may be changing. Using data-driven reconstructions of soundscapes in lieu of historical recordings, here we quantify changes in soundscape characteristics at more than 200,000 sites across North America and Europe. We integrate citizen science bird monitoring data with recordings of individual species to reveal a pervasive loss of acoustic diversity and intensity of soundscapes across both continents over the past 25 years, driven by changes in species richness and abundance. These results suggest that one of the fundamental pathways through which humans engage with nature is in chronic decline, with potentially widespread implications for human health and well-being.

Comparing recurrent convolutional neural networks for large scale bird species classification.

We present a deep learning approach towards the large-scale prediction and analysis of bird acoustics from 100 different bird species. We use spectrograms constructed on bird audio recordings from the Cornell Bird Challenge (CBC)2020 dataset, which includes recordings of multiple and potentially overlapping bird vocalizations with background noise. Our experiments show that a hybrid modeling approach that involves a Convolutional Neural Network (CNN) for learning the representation for a slice of the spectrogram, and a Recurrent Neural Network (RNN) for the temporal component to combine across time-points leads to the most accurate model on this dataset. We show results on a spectrum of models ranging from stand-alone CNNs to hybrid models of various types obtained by combining CNNs with other CNNs or RNNs of the following types: Long Short-Term Memory (LSTM) networks, Gated Recurrent Units (GRU), and Legendre Memory Units (LMU). The best performing model achieves an average accuracy of 67% over the 100 different bird species, with the highest accuracy of 90% for the bird species, Red crossbill. We further analyze the learned representations visually and find them to be intuitive, where we find that related bird species are clustered close together. We present a novel way to empirically interpret the representations learned by the LMU-based hybrid model which shows how memory channel patterns change over time with the changes seen in the spectrograms.

Balanced imitation sustains song culture in zebra finches.

Songbirds acquire songs by imitation, as humans do speech. Although imitation should drive convergence within a group and divergence through drift between groups, zebra finch songs sustain high diversity within a colony, but mild variation across colonies. We investigated this phenomenon by analyzing vocal learning statistics in 160 tutor-pupil pairs from a large breeding colony. Song imitation is persistently accurate in some families, but poor in others. This is not attributed to genetic differences, as fostered pupils copied their tutors' songs as accurately or poorly as biological pupils. Rather, pupils of tutors with low song diversity make more improvisations compared to pupils of tutors with high song diversity. We suggest that a frequency dependent balanced imitation prevents extinction of rare song elements and overabundance of common ones, promoting repertoire diversity within groups, while constraining drift across groups, which together prevents the collapse of vocal culture into either complete uniformity or chaos.

Finding, visualizing, and quantifying latent structure across diverse animal vocal repertoires.

Animals produce vocalizations that range in complexity from a single repeated call to hundreds of unique vocal elements patterned in sequences unfolding over hours. Characterizing complex vocalizations can require considerable effort and a deep intuition about each species' vocal behavior. Even with a great deal of experience, human characterizations of animal communication can be affected by human perceptual biases. We present a set of computational methods for projecting animal vocalizations into low dimensional latent representational spaces that are directly learned from the spectrograms of vocal signals. We apply these methods to diverse datasets from over 20 species, including humans, bats, songbirds, mice, cetaceans, and nonhuman primates. Latent projections uncover complex features of data in visually intuitive and quantifiable ways, enabling high-powered comparative analyses of vocal acoustics. We introduce methods for analyzing vocalizations as both discrete sequences and as continuous latent variables. Each method can be used to disentangle complex spectro-temporal structure and observe long-timescale organization in communication.

Temporal structure of mouse courtship vocalizations facilitates syllable labeling.

Mice emit sequences of ultrasonic vocalizations (USVs) but little is known about the rules governing their temporal order and no consensus exists on the classification of USVs into syllables. To address these questions, we recorded USVs during male-female courtship and found a significant temporal structure. We labeled USVs using three popular algorithms and found that there was no one-to-one relationships between their labels. As label assignment affects the high order temporal structure, we developed the Syntax Information Score (based on information theory) to rank labeling algorithms based on how well they predict the next syllable in a sequence. Finally, we derived a novel algorithm (Syntax Information Maximization) that utilizes sequence statistics to improve the clustering of individual USVs with respect to the underlying sequence structure. Improvement in USV classification is crucial for understanding neural control of vocalization. We demonstrate that USV syntax holds valuable information towards achieving this goal.

Comparison of methods for rhythm analysis of complex animals' acoustic signals.

Analyzing the rhythm of animals' acoustic signals is of interest to a growing number of researchers: evolutionary biologists want to disentangle how these structures evolved and what patterns can be found, and ecologists and conservation biologists aim to discriminate cryptic species on the basis of parameters of acoustic signals such as temporal structures. Temporal structures are also relevant for research on vocal production learning, a part of which is for the animal to learn a temporal structure. These structures, in other words, these rhythms, are the topic of this paper. How can they be investigated in a meaningful, comparable and universal way? Several approaches exist. Here we used five methods to compare their suitability and interpretability for different questions and datasets and test how they support the reproducibility of results and bypass biases. Three very different datasets with regards to recording situation, length and context were analyzed: two social vocalizations of Neotropical bats (multisyllabic, medium long isolation calls of Saccopteryx bilineata, and monosyllabic, very short isolation calls of Carollia perspicillata) and click trains of sperm whales, Physeter macrocephalus. Techniques to be compared included Fourier analysis with a newly developed goodness-of-fit value, a generate-and-test approach where data was overlaid with varying artificial beats, and the analysis of inter-onset-intervals and calculations of a normalized Pairwise Variability Index (nPVI). We discuss the advantages and disadvantages of the methods and we also show suggestions on how to best visualize rhythm analysis results. Furthermore, we developed a decision tree that will enable researchers to select a suitable and comparable method on the basis of their data.

Rapid development of mature vocal patterns of ultrasonic calls in a fast-growing rodent, the yellow steppe lemming (Eolagurus luteus).

Ultrasonic vocalizations (USV) of laboratory rodents may serve as age-dependent indicators of emotional arousal and anxiety. Fast-growing Arvicolinae rodent species might be advantageous wild-type animal models for behavioural and medical research related to USV ontogeny. For the yellow steppe lemming Eolagurus luteus, only audible calls of adults were previously described. This study provides categorization and spectrographic analyses of 1176 USV calls emitted by 120 individual yellow steppe lemmings at 12 age classes, from birth to breeding adults over 90 days (d) of age, 10 individuals per age class, up to 10 USV calls per individual. The USV calls emerged since 1st day of pup life and occurred at all 12 age classes and in both sexes. The unified 2-min isolation procedure on an unfamiliar territory was equally applicable for inducing USV calls at all age classes. Rapid physical growth (1 g body weight gain per day from birth to 40 d of age) and the early (9-12 d) eyes opening correlated with the early (9-12 d) emergence of mature vocal patterns of USV calls. The mature vocal patterns included a prominent shift in percentages of chevron and upward contours of fundamental frequency (f0) and the changes in the acoustic variables of USV calls. Call duration was the longest at 1-4 d, significantly shorter at 9-12 d and did not between 9-12-d and older age classes. The maximum fundamental frequency (f0max) decreased with increase of age class, from about 50 kHz in neonates to about 40 kHz in adults. These ontogenetic pathways of USV duration and f0max (towards shorter and lower-frequency USV calls) were reminiscent of those in laboratory mice Mus musculus.

DetEdit: A graphical user interface for annotating and editing events detected in long-term acoustic monitoring data.

Passive acoustic monitoring has become an important data collection method, yielding massive datasets replete with biological, environmental and anthropogenic information. Automated signal detectors and classifiers are needed to identify events within these datasets, such as the presence of species-specific sounds or anthropogenic noise. These automated methods, however, are rarely a complete substitute for expert analyst review. The ability to visualize and annotate acoustic events efficiently can enhance scientific insights from large, previously intractable datasets. A MATLAB-based graphical user interface, called DetEdit, was developed to accelerate the editing and annotating of automated detections from extensive acoustic datasets. This tool is highly-configurable and multipurpose, with uses ranging from annotation and classification of individual signals or signal-clusters and evaluation of signal properties, to identification of false detections and false positive rate estimation. DetEdit allows users to step through acoustic events, displaying a range of signal features, including time series of received levels, long-term spectral averages, time intervals between detections, and scatter plots of peak frequency, RMS, and peak-to-peak received levels. Additionally, it displays either individual, or averaged sound pressure waveforms, and power spectra within each acoustic event. These views simultaneously provide analysts with signal-level detail and encounter-level context. DetEdit creates datasets of signal labels for further analyses, such as training classifiers and quantifying occurrence, abundances, or trends. Although designed for evaluating underwater-recorded odontocete echolocation click detections, DetEdit can be adapted to almost any stereotyped impulsive signal. Our software package complements available tools for the bioacoustic community and is provided open source at https://github.com/MarineBioAcousticsRC/DetEdit.

High-throughput analysis of vocalizations reveals sex-specific changes in Fmr1 mutant pups.

There have been several reports that individuals with Fragile X syndrome (FXS) and animal models of FXS have communication deficits. The present study utilized two different call classification taxonomies to examine the sex-specificity of ultrasonic vocalization (USV) production on postnatal day (PD8) in the FVB strain of Fmr1 knockout (KO) mice. One classification protocol requires the investigator to score each call by hand, while the other protocol uses an automated algorithm. Results using the hand-scoring protocol indicated that male Fmr1 KO mice exhibited longer calls (P = .03) than wild types on PD8. Male KOs also produced fewer complex, composite, downward, short and two-syllable call-types, as well as more frequency steps and chevron call-types. Female heterozygotes exhibited no significant changes in acoustic or temporal aspects of calls, yet showed significant changes in call-type production proportions across two different classification taxonomies (P < .001). They exhibited increased production of harmonic and frequency steps calls, as well as fewer chevron, downward and short calls. According to the second high-throughput analysis, female heterozygotes produced significantly fewer single-type and more multiple-type syllables, unlike male KOs that showed no changes in these aspects of syllable production. Finally, we correlated both scoring methods and found a high level of correlation between the two methods. These results contribute further knowledge of sex differences in USV calling behavior for Fmr1 heterozygote and KO mice and provide a foundation for the use of high-throughput analysis of neonatal USVs.

Towards a new taxonomy of primate vocal production learning.

The extent to which vocal learning can be found in nonhuman primates is key to reconstructing the evolution of speech. Regarding the adjustment of vocal output in relation to auditory experience (vocal production learning in the narrow sense), effects on the ontogenetic trajectory of vocal development as well as adjustment to group-specific call features have been found. Yet, a comparison of the vocalizations of different primate genera revealed striking similarities in the structure of calls and repertoires in different species of the same genus, indicating that the structure of nonhuman primate vocalizations is highly conserved. Thus, modifications in relation to experience only appear to be possible within relatively tight species-specific constraints. By contrast, comprehension learning may be extremely rapid and open-ended. In conjunction, these findings corroborate the idea of an ancestral independence of vocal production and auditory comprehension learning. To overcome the futile debate about whether or not vocal production learning can be found in nonhuman primates, we suggest putting the focus on the different mechanisms that may mediate the adjustment of vocal output in response to experience; these mechanisms may include auditory facilitation and learning from success. This article is part of the theme issue 'What can animal communication teach us about human language?'

A taxonomy for vocal learning.

Humans and songbirds learn to sing or speak by listening to acoustic models, forming auditory templates, and then learning to produce vocalizations that match the templates. These taxa have evolved specialized telencephalic pathways to accomplish this complex form of vocal learning, which has been reported for very few other taxa. By contrast, the acoustic structure of most animal vocalizations is produced by species-specific vocal motor programmes in the brainstem that do not require auditory feedback. However, many mammals and birds can learn to fine-tune the acoustic features of inherited vocal motor patterns based upon listening to conspecifics or noise. These limited forms of vocal learning range from rapid alteration based on real-time auditory feedback to long-term changes of vocal repertoire and they may involve different mechanisms than complex vocal learning. Limited vocal learning can involve the brainstem, mid-brain and/or telencephalic networks. Understanding complex vocal learning, which underpins human speech, requires careful analysis of which species are capable of which forms of vocal learning. Selecting multiple animal models for comparing the neural pathways that generate these different forms of learning will provide a richer view of the evolution of complex vocal learning and the neural mechanisms that make it possible. This article is part of the theme issue 'What can animal communication teach us about human language?'

Species Boundaries within Morphologically Cryptic Galagos: Evidence from Acoustic and Genetic Data.

Describing primate biodiversity is one of the main goals in primatology. Species are the fundamental unit of study in phylogeny, behaviour, ecology and conservation. Identifying species boundaries is particularly challenging for nocturnal taxa where only subtle morphological variation is present. Traditionally, vocal signals have been used to identify species within nocturnal primates: species-specific signals often play a critical role in mate recognition, and they can restrict gene flow with other species. However, little research has been conducted to test whether different "acoustic forms" also represent genetically distinct species. Here, we investigate species boundaries between two putative highly cryptic species of Eastern dwarf galagos (Paragalago cocosand P. zanzibaricus). We combined vocal and genetic data: molecular data included the complete mitochondrial cytochrome b gene (1,140 bp) for 50 samples across 11 localities in Kenya and Tanzania, while vocal data comprised 221 vocalisations recorded across 8 localities. Acoustic analyses showed a high level of correct assignation to the putative species (approx. 90%), while genetic analyses identified two separate clades at the mitochondrial level. We conclude that P. cocos and P. zanzibaricus represent two valid cryptic species that probably underwent speciation in the Late Pliocene while fragmented in isolated populations in the eastern forests.

Automatic acoustic identification of individuals in multiple species: improving identification across recording conditions.

Many animals emit vocal sounds which, independently from the sounds' function, contain some individually distinctive signature. Thus the automatic recognition of individuals by sound is a potentially powerful tool for zoology and ecology research and practical monitoring. Here, we present a general automatic identification method that can work across multiple animal species with various levels of complexity in their communication systems. We further introduce new analysis techniques based on dataset manipulations that can evaluate the robustness and generality of a classifier. By using these techniques, we confirmed the presence of experimental confounds in situations resembling those from past studies. We introduce data manipulations that can reduce the impact of these confounds, compatible with any classifier. We suggest that assessment of confounds should become a standard part of future studies to ensure they do not report over-optimistic results. We provide annotated recordings used for analyses along with this study and we call for dataset sharing to be a common practice to enhance the development of methods and comparisons of results.

Deep convolutional network for animal sound classification and source attribution using dual audio recordings.

This paper introduces an end-to-end feedforward convolutional neural network that is able to reliably classify the source and type of animal calls in a noisy environment using two streams of audio data after being trained on a dataset of modest size and imperfect labels. The data consists of audio recordings from captive marmoset monkeys housed in pairs, with several other cages nearby. The network in this paper can classify both the call type and which animal made it with a single pass through a single network using raw spectrogram images as input. The network vastly increases data analysis capacity for researchers interested in studying marmoset vocalizations, and allows data collection in the home cage, in group housed animals.

Characteristics of vocalisations recorded from free-ranging Shepherd's beaked whales, Tasmacetus shepherdi.

Beaked whales (family Ziphiidae) are among the least studied of all the large mammals. This is especially true of Shepherd's beaked whale (Tasmacetus shepherdi), which until recently had been very rarely sighted alive, with nothing known about the species' acoustic behaviour. Vocalisations of Shepherd's beaked whales were recorded using a hydrophone array on two separate days during marine mammal surveys of the Otago submarine canyons in New Zealand. After carefully screening the recordings, two distinct call types were found; broadband echolocation clicks, and burst pulses. Broadband echolocation clicks (n = 476) had a median inter-click-interval (ICI) of 0.46 s and median peak frequency of 19.2 kHz. The burst pulses (n = 33) had a median peak frequency of constituent clicks (n = 1741) of 14.7 kHz, and median ICI of 11 ms. These results should be interpreted with caution due to the limited bandwidth used to record the signals. To the authors' knowledge, this study presents the first analysis of the characteristics of Shepherd's beaked whale sounds. It will help with identification of the species in passive acoustic monitoring records, and future efforts to further analyse this species' vocalisations.

Resource-Efficient Pet Dog Sound Events Classification Using LSTM-FCN Based on Time-Series Data.

The use of IoT (Internet of Things) technology for the management of pet dogs left alone at home is increasing. This includes tasks such as automatic feeding, operation of play equipment, and location detection. Classification of the vocalizations of pet dogs using information from a sound sensor is an important method to analyze the behavior or emotions of dogs that are left alone. These sounds should be acquired by attaching the IoT sound sensor to the dog, and then classifying the sound events (e.g., barking, growling, howling, and whining). However, sound sensors tend to transmit large amounts of data and consume considerable amounts of power, which presents issues in the case of resource-constrained IoT sensor devices. In this paper, we propose a way to classify pet dog sound events and improve resource efficiency without significant degradation of accuracy. To achieve this, we only acquire the intensity data of sounds by using a relatively resource-efficient noise sensor. This presents issues as well, since it is difficult to achieve sufficient classification accuracy using only intensity data due to the loss of information from the sound events. To address this problem and avoid significant degradation of classification accuracy, we apply long short-term memory-fully convolutional network (LSTM-FCN), which is a deep learning method, to analyze time-series data, and exploit bicubic interpolation. Based on experimental results, the proposed method based on noise sensors (i.e., Shapelet and LSTM-FCN for time-series) was found to improve energy efficiency by 10 times without significant degradation of accuracy compared to typical methods based on sound sensors (i.e., mel-frequency cepstrum coefficient (MFCC), spectrogram, and mel-spectrum for feature extraction, and support vector machine (SVM) and k-nearest neighbor (K-NN) for classification).