Applying machine learning to primate bioacoustics: Review and perspectives.

This paper provides a comprehensive review of the use of computational bioacoustics as well as signal and speech processing techniques in the analysis of primate vocal communication. We explore the potential implications of machine learning and deep learning methods, from the use of simple supervised algorithms to more recent self-supervised models, for processing and analyzing large data sets obtained within the emergence of passive acoustic monitoring approaches. In addition, we discuss the importance of automated primate vocalization analysis in tackling essential questions on animal communication and highlighting the role of comparative linguistics in bioacoustic research. We also examine the challenges associated with data collection and annotation and provide insights into potential solutions. Overall, this review paper runs through a set of common or innovative perspectives and applications of machine learning for primate vocal communication analysis and outlines opportunities for future research in this rapidly developing field.

A large-scale assessment of eastern whip-poor-will (Antrostomus vociferus) occupancy across a gradient of forest management intensity using autonomous recording units.

Conservationists spend considerable resources to create and enhance wildlife habitat. Monitoring how species respond to these efforts helps managers allocate limited resources. However, monitoring efforts often encounter logistical challenges that are exacerbated as geographic extent increases. We used autonomous recording units (ARUs) and automated acoustic classification to mitigate the challenges of assessing Eastern Whip-poor-will (Antrostomus vociferus) response to forest management across the eastern USA. We deployed 1263 ARUs in forests with varying degrees of management intensity. Recordings were processed using an automated classifier and the resulting detection data were used to assess occupancy. Whip-poor-wills were detected at 401 survey locations. Across our study region, whip-poor-will occupancy decreased with latitude and elevation. At the landscape scale, occupancy decreased with the amount of impervious cover, increased with herbaceous cover and oak and evergreen forests, and exhibited a quadratic relationship with the amount of shrub-scrub cover. At the site-level, occupancy was negatively associated with basal area and brambles (Rubus spp.) and exhibited a quadratic relationship with woody stem density. Implementation of practices that create and sustain a mosaic of forest age classes and a diverse range of canopy closure within oak (Quercus spp.) dominated landscapes will have the highest probability of hosting whip-poor-wills. The use of ARUs and a machine learning classifier helped overcome challenges associated with monitoring a nocturnal species with a short survey window across a large spatial extent. Future monitoring efforts that combine ARU-based protocols and mappable fine-resolution structural vegetation data would likely further advance our understanding of whip-poor-will response to forest management.

Windy events detection in big bioacoustics datasets using a pre-trained Convolutional Neural Network.

Passive Acoustic Monitoring (PAM), which involves using autonomous record units for studying wildlife behaviour and distribution, often requires handling big acoustic datasets collected over extended periods. While these data offer invaluable insights about wildlife, their analysis can present challenges in dealing with geophonic sources. A major issue in the process of detection of target sounds is represented by wind-induced noise. This can lead to false positive detections, i.e., energy peaks due to wind gusts misclassified as biological sounds, or false negative, i.e., the wind noise masks the presence of biological sounds. Acoustic data dominated by wind noise makes the analysis of vocal activity unreliable, thus compromising the detection of target sounds and, subsequently, the interpretation of the results. Our work introduces a straightforward approach for detecting recordings affected by windy events using a pre-trained convolutional neural network. This process facilitates identifying wind-compromised data. We consider this dataset pre-processing crucial for ensuring the reliable use of PAM data. We implemented this preprocessing by leveraging YAMNet, a deep learning model for sound classification tasks. We evaluated YAMNet as-is ability to detect wind-induced noise and tested its performance in a Transfer Learning scenario by using our annotated data from the Stony Point Penguin Colony in South Africa. While the classification of YAMNet as-is achieved a precision of 0.71, and recall of 0.66, those metrics strongly improved after the training on our annotated dataset, reaching a precision of 0.91, and recall of 0.92, corresponding to a relative increment of >28 %. Our study demonstrates the promising application of YAMNet in the bioacoustics and ecoacoustics fields, addressing the need for wind-noise-free acoustic data. We released an open-access code that, combined with the efficiency and peak performance of YAMNet, can be used on standard laptops for a broad user base.

Ziphius cavirostris presence relative to the vertical and temporal variability of oceanographic conditions in the Southern California Bight.

The oceanographic conditions of the Southern California Bight (SCB) dictate the distribution and abundance of prey resources and therefore the presence of mobile predators, such as goose-beaked whales (Ziphius cavirostris). Goose-beaked whales are deep-diving odontocetes that spend a majority of their time foraging at depth. Due to their cryptic behavior, little is known about how they respond to seasonal and interannual changes in their environment. This study utilizes passive acoustic data recorded from two sites within the SCB to explore the oceanographic conditions that goose-beaked whales appear to favor. Utilizing optimum multiparameter analysis, modeled temperature and salinity data are used to identify and quantify these source waters: Pacific Subarctic Upper Water (PSUW), Pacific Equatorial Water (PEW), and Eastern North Pacific Central Water (ENPCW). The interannual and seasonal variability in goose-beaked whale presence was related to the variability in El Niño Southern Oscillation events and the fraction and vertical distribution of the three source waters. Goose-beaked whale acoustic presence was highest during the winter and spring and decreased during the late summer and early fall. These seasonal increases occurred at times of increased fractions of PEW in the California Undercurrent and decreased fractions of ENPCW in surface waters. Interannual increases in goose-beaked whale presence occurred during El Niño events. These results establish a baseline understanding of the oceanographic characteristics that correlate with goose-beaked whale presence in the SCB. Furthering our knowledge of this elusive species is key to understanding how anthropogenic activities impact goose-beaked whales.

Listening to animal behavior to understand changing ecosystems.

Interpreting sound gives powerful insight into the health of ecosystems. Beyond detecting the presence of wildlife, bioacoustic signals can reveal their behavior. However, behavioral bioacoustic information is underused because identifying the function and context of animals' sounds remains challenging. A growing acoustic toolbox is allowing researchers to begin decoding bioacoustic signals by linking individual and population-level sensing. Yet, studies integrating acoustic tools for behavioral insight across levels of biological organization remain scarce. We aim to catalyze the emerging field of behavioral bioacoustics by synthesizing recent successes and rising analytical, logistical, and ethical challenges. Because behavior typically represents animals' first response to environmental change, we posit that behavioral bioacoustics will provide theoretical and applied insights into animals' adaptations to global change.

Underwater soundscape in Seaview Bay, Antarctica, and triple ascending trill of the leopard seal (Hydrurga leptonyx) underwater vocalizations.

The underwater soundscape was recorded in Seaview Bay off Inexpressible Island, Ross Sea region Marine protected area, for 3 days in December 2021. Leopard seal Hydrurga leptonyx vocalizations were a prominent sound source that led to variations in ambient sound pressure levels in a frequency range of approximately 150-4500 Hz. Among the 14 call types previously identified, except ultrasound vocalizations, six types of broadcast calls were classified, and their acoustic characteristics were analyzed. We focused on the acoustic characteristics of four low-frequency calls, clustered in a relatively narrow bandwidth, which have been relatively less studied. We identified a new call type of a triple ascending trill consisting of three trill parts, expanding upon the findings of previous studies. The audio data extracted from leopard seal vocalization videos, recorded by a monitoring camera on sea ice, enhanced the reliability of identifications of the underwater triple ascending trill. We present the unique results of underwater passive acoustic monitoring conducted at Seaview Bay, designated as Antarctic Specially Protected Area No 178. Our results could contribute to the development of detection and localization algorithms for leopard seal vocalizations and can be used as fundamental data for studies related to the vocalization and behavior of this species.

Individual identification in acoustic recordings.

Recent advances in bioacoustics combined with acoustic individual identification (AIID) could open frontiers for ecological and evolutionary research because traditional methods of identifying individuals are invasive, expensive, labor-intensive, and potentially biased. Despite overwhelming evidence that most taxa have individual acoustic signatures, the application of AIID remains challenging and uncommon. Furthermore, the methods most commonly used for AIID are not compatible with many potential AIID applications. Deep learning in adjacent disciplines suggests opportunities to advance AIID, but such progress is limited by training data. We suggest that broadscale implementation of AIID is achievable, but researchers should prioritize methods that maximize the potential applications of AIID, and develop case studies with easy taxa at smaller spatiotemporal scales before progressing to more difficult scenarios.

Acoustic fish community in the Madeira Archipelago (North Atlantic Ocean): Characterization of sound diversity and daily patterns.

Marine ecosystems are increasingly subjected to anthropogenic pressures, which demands urgent monitoring plans. Understanding soundscapes can offer unique insights into the ocean status providing important information and revealing different sounds and their sources. Fishes can be prominent soundscape contributors, making passive acoustic monitoring (PAM) a potential tool to detect the presence of vocal fish species and to monitor changes in biodiversity. The major goal of this research was to provide a first reference of the marine soundscapes of the Madeira Archipelago focusing on fish sounds, as a basis for a long-term PAM program. Based on the literature, 102 potentially vocal and 35 vocal fish species were identified. Additionally 43 putative fish sound types were detected in audio recordings from two marine protected areas (MPAs) in the Archipelago: the Garajau MPA and the Desertas MPA. The Garajau MPA exhibited higher fish vocal activity, a greater variety of putative fish sound types and higher fish sound diversity. Lower abundance of sounds was found at night at both MPAs. Acoustic activity revealed a clear distinction between diurnal and nocturnal fish groups and demonstrated daily patterns of fish sound activity, suggesting temporal and spectral partitioning of the acoustic space. Pomacentridae species were proposed as candidates for some of the dominant sound types detected during the day, while scorpionfishes (Scorpaena spp.) were proposed as sources for some of the dominant nocturnal fish sounds. This study provides an important baseline about this community acoustic behaviour and is a valuable steppingstone for future non-invasive and cost-effective monitoring programs in Madeira.

Vessels and aircraft are chronic sources of anthropogenic noise in coastal marine and terrestrial soundscapes on Long Island, New York.

Passive acoustic data collected during 2020 and 2021 were used to monitor changes in both terrestrial and underwater soundscapes, as well as human activity from aircraft and vessels. Passive acoustic data were collected at two artificial reefs south of Long Island, as well as along ocean beaches in Southampton, NY. At the artificial reefs, vessel noise was recorded more frequently during 2020 than in 2021. Commercial vessels and multi-user charter fishing vessels were more abundant during 2020. Peaks in power spectral density occurred at 60, 90 and 120 Hz in 2020 and 2021, which are frequencies consistent with noise generated by commercial vessels, suggesting that vessels are a significant contributor to the soundscape of the artificial reefs. In the terrestrial environment, noise generated by aircraft was more common during 2021. Peaks in power spectral density were measured around 160 and 290 Hz at one of the ocean beach sites. These frequencies are consistent with noise generated by aircraft. This study documents the chronic extent of anthropogenic noise in both the underwater and terrestrial environments of Long Island, NY, as well as quantifies the occurrence of various noise sources in these habitats.

Extensive data engineering to the rescue: building a multi-species katydid detector from unbalanced, atypical training datasets.

Passive acoustic monitoring (PAM) is a powerful tool for studying ecosystems. However, its effective application in tropical environments, particularly for insects, poses distinct challenges. Neotropical katydids produce complex species-specific calls, spanning mere milliseconds to seconds and spread across broad audible and ultrasonic frequencies. However, subtle differences in inter-pulse intervals or central frequencies are often the only discriminatory traits. These extremities, coupled with low source levels and susceptibility to masking by ambient noise, challenge species identification in PAM recordings. This study aimed to develop a deep learning-based solution to automate the recognition of 31 katydid species of interest in a biodiverse Panamanian forest with over 80 katydid species. Besides the innate challenges, our efforts were also encumbered by a limited and imbalanced initial training dataset comprising domain-mismatched recordings. To overcome these, we applied rigorous data engineering, improving input variance through controlled playback re-recordings and by employing physics-based data augmentation techniques, and tuning signal-processing, model and training parameters to produce a custom well-fit solution. Methods developed here are incorporated into Koogu, an open-source Python-based toolbox for developing deep learning-based bioacoustic analysis solutions. The parametric implementations offer a valuable resource, enhancing the capabilities of PAM for studying insects in tropical ecosystems. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.

Time and habitat structure shape insect acoustic activity in the Amazon.

Insects are the most diverse animal taxon on Earth and play a key role in ecosystem functioning. However, they are often neglected by ecological surveys owing to the difficulties involved in monitoring this small and hyper-diverse taxon. With technological advances in biomonitoring and analytical methods, these shortcomings may finally be addressed. Here, we performed passive acoustic monitoring at 141 sites (eight habitats) to investigate insect acoustic activity in the Viruá National Park, Brazil. We first describe the frequency range occupied by three soniferous insect groups (cicadas, crickets and katydids) to calculate the acoustic evenness index (AEI). Then, we assess how AEI varies spatially and temporally among habitat types, and finally we investigate the relationship between vegetation structure variables and AEI for each insect category. Overall, crickets occupied lower and narrower frequency bands than cicadas and katydids. AEI values varied among insect categories and across space and time. The highest acoustic activity occurred before sunrise and the lowest acoustic activity was recorded in pastures. Canopy cover was positively associated with cricket acoustic activity but not with katydids. Our findings contribute to a better understanding of the role of time, habitat and vegetation structure in shaping insect activity within diverse Amazonian ecosystems. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.

The potential of acoustic monitoring of aquatic insects for freshwater assessment.

Aquatic insects are a major indicator used to assess ecological condition in freshwater environments. However, current methods to collect and identify aquatic insects require advanced taxonomic expertise and rely on invasive techniques that lack spatio-temporal replication. Passive acoustic monitoring (PAM) is emerging as a non-invasive complementary sampling method allowing broad spatio-temporal and taxonomic coverage. The application of PAM in freshwater ecosystems has already proved useful, revealing unexpected acoustic diversity produced by fishes, amphibians, submerged aquatic plants, and aquatic insects. However, the identity of species producing sounds remains largely unknown. Among them, aquatic insects appear to be the major contributor to freshwater soundscapes. Here, we estimate the potential number of soniferous aquatic insects worldwide using data from the Global Biodiversity Information Facility. We found that four aquatic insect orders produce sounds totalling over 7000 species. This number is probably underestimated owing to poor knowledge of aquatic insects bioacoustics. We then assess the value of sound producing aquatic insects to evaluate ecological condition and find that they might be useful despite having similar responses in pristine and degraded environments in some cases. Both expert and automated identifications will be necessary to build international reference libraries and to conduct acoustic bioassessment in freshwaters. This article is part of the theme issue 'Towards a toolkit for global insect biodiversity monitoring'.

Increased Yangtze finless porpoise presence in urban Wuhan waters of the Yangtze River during fishing closures.

Wuhan, a highly urbanized and rapidly growing region within China's Yangtze Economic Zone, has historically been identified as a gap area for the critically endangered Yangtze finless porpoise (Neophocaena asiaeorientalis asiaeorientalis) based on daytime visual surveys. However, there has been a noticeable increase in porpoise sightings since 2020. This study employed passive acoustic monitoring to investigate porpoise distribution in Wuhan between 2020 and 2022. Generalized linear models were used to explore the relationship between shipping, hydrological patterns, light intensity, and porpoise biosonar activity. Over 603 days of effective monitoring, the daily positive rate for porpoise biosonar detection reached 43%, with feeding-related buzz signals accounting for 55% of all porpoise biosonar signals. However, the proportion of minutes during which porpoise presence was detected was 0.18%, suggesting that while porpoises may frequent the area, their visits were brief and mainly focused on feeding. A significant temporal trend emerged, showing higher porpoise biosonar detection during winter (especially in February) and 2022. Additionally, periods without boat traffic correlated with increased porpoise activity. Hydrological conditions and light levels exhibited significant negative correlations with porpoise activity. Specifically, porpoise sonar detections were notably higher during the night, twilight, and new moon phases. It is highly conceivable that both fishing bans and COVID-19 pandemic-related lockdowns contributed to the heightened presence of porpoises in Wuhan. The rapid development of municipal transportation and shipping in Wuhan and resulting underwater noise pollution have emerged as a significant threat to the local porpoise population. Accordingly, it is imperative for regulatory bodies to effectively address this environmental stressor and formulate targeted protection measures to ensure the conservation of the finless porpoise.

Efficient Speech Detection in Environmental Audio Using Acoustic Recognition and Knowledge Distillation.

The ongoing biodiversity crisis, driven by factors such as land-use change and global warming, emphasizes the need for effective ecological monitoring methods. Acoustic monitoring of biodiversity has emerged as an important monitoring tool. Detecting human voices in soundscape monitoring projects is useful both for analyzing human disturbance and for privacy filtering. Despite significant strides in deep learning in recent years, the deployment of large neural networks on compact devices poses challenges due to memory and latency constraints. Our approach focuses on leveraging knowledge distillation techniques to design efficient, lightweight student models for speech detection in bioacoustics. In particular, we employed the MobileNetV3-Small-Pi model to create compact yet effective student architectures to compare against the larger EcoVAD teacher model, a well-regarded voice detection architecture in eco-acoustic monitoring. The comparative analysis included examining various configurations of the MobileNetV3-Small-Pi-derived student models to identify optimal performance. Additionally, a thorough evaluation of different distillation techniques was conducted to ascertain the most effective method for model selection. Our findings revealed that the distilled models exhibited comparable performance to the EcoVAD teacher model, indicating a promising approach to overcoming computational barriers for real-time ecological monitoring.

Propagating variational model uncertainty for bioacoustic call label smoothing.

Along with propagating the input toward making a prediction, Bayesian neural networks also propagate uncertainty. This has the potential to guide the training process by rejecting predictions of low confidence, and recent variational Bayesian methods can do so without Monte Carlo sampling of weights. Here, we apply sample-free methods for wildlife call detection on recordings made via passive acoustic monitoring equipment in the animals' natural habitats. We further propose uncertainty-aware label smoothing, where the smoothing probability is dependent on sample-free predictive uncertainty, in order to downweigh data samples that should contribute less to the loss value. We introduce a bioacoustic dataset recorded in Malaysian Borneo, containing overlapping calls from 30 species. On that dataset, our proposed method achieves an absolute percentage improvement of around 1.5 points on area under the receiver operating characteristic (AU-ROC), 13 points in F1, and 19.5 points in expected calibration error (ECE) compared to the point-estimate network baseline averaged across all target classes.

Rerouting of a major shipping lane through important harbour porpoise habitat caused no detectable change in annual occurrence or foraging patterns.

Shipping is one of the largest industries globally, with well-known negative impacts on the marine environment. Despite the known negative short-term (minutes to hours) impact of shipping on individual animal behavioural responses, very little is understood about the long-term (months to years) impact on marine species presence and area use. This study took advantage of a planned rerouting of a major shipping lane leading into the Baltic Sea, to investigate the impact on the presence and foraging behaviour of a marine species known to be sensitive to underwater noise, the harbour porpoise (Phocoena phocoena). Passive acoustic monitoring data were collected from 15 stations over two years. Against predictions, no clear change occurred in monthly presence or foraging behaviour of the porpoises, despite the observed changes in noise and vessel traffic. However, long-term heightened noise levels may still impact communication, echolocation, or stress levels of individuals, and needs further investigation.

Acoustic features as a tool to visualize and explore marine soundscapes: Applications illustrated using marine mammal passive acoustic monitoring datasets.

Passive Acoustic Monitoring (PAM) is emerging as a solution for monitoring species and environmental change over large spatial and temporal scales. However, drawing rigorous conclusions based on acoustic recordings is challenging, as there is no consensus over which approaches are best suited for characterizing marine acoustic environments. Here, we describe the application of multiple machine-learning techniques to the analysis of two PAM datasets. We combine pre-trained acoustic classification models (VGGish, NOAA and Google Humpback Whale Detector), dimensionality reduction (UMAP), and balanced random forest algorithms to demonstrate how machine-learned acoustic features capture different aspects of the marine acoustic environment. The UMAP dimensions derived from VGGish acoustic features exhibited good performance in separating marine mammal vocalizations according to species and locations. RF models trained on the acoustic features performed well for labeled sounds in the 8 kHz range; however, low- and high-frequency sounds could not be classified using this approach. The workflow presented here shows how acoustic feature extraction, visualization, and analysis allow establishing a link between ecologically relevant information and PAM recordings at multiple scales, ranging from large-scale changes in the environment (i.e., changes in wind speed) to the identification of marine mammal species.

Divergent ecological responses to typhoon disturbance revealed via landscape-scale acoustic monitoring.

Climate change is increasing the frequency, intensity, and duration of extreme weather events across the globe. Understanding the capacity for ecological communities to withstand and recover from such events is critical. Typhoons are extreme weather events that are expected to broadly homogenize ecological dynamics through structural damage to vegetation and longer-term effects of salinization. Given their unpredictable nature, monitoring ecological responses to typhoons is challenging, particularly for mobile animals such as birds. Here, we report spatially variable ecological responses to typhoons across terrestrial landscapes. Using a high temporal resolution passive acoustic monitoring network across 24 sites on the subtropical island of Okinawa, Japan, we found that typhoons elicit divergent ecological responses among Okinawa's diverse terrestrial habitats, as indicated by increased spatial variability of biological sound production (biophony) and individual species detections. This suggests that soniferous communities are capable of a diversity of different responses to typhoons. That is, spatial insurance effects among local ecological communities provide resilience to typhoons at the landscape scale. Even though site-level typhoon impacts on soundscapes and bird detections were not particularly strong, monitoring at scale with high temporal resolution across a broad spatial extent nevertheless enabled detection of spatial heterogeneity in typhoon responses. Further, species-level responses mirrored those of acoustic indices, underscoring the utility of such indices for revealing insight into fundamental questions concerning disturbance and stability. Our findings demonstrate the significant potential of landscape-scale acoustic sensor networks to capture the understudied ecological impacts of unpredictable extreme weather events.

Seasonal acoustic presence of marine mammals at the South Orkney Islands, Scotia Sea.

Increased knowledge about marine mammal seasonal distribution and species assemblage from the South Orkney Islands waters is needed for the development of management regulations of the commercial fishery for Antarctic krill (Euphausia superba) in this region. Passive acoustic monitoring (PAM) data were collected during the autumn and winter seasons in two consecutive years (2016, 2017), which represented highly contrasting environmental conditions due to the 2016 El Niño event. We explored differences in seasonal patterns in marine mammal acoustic presence between the two years in context of environmental cues and climate variability. Acoustic signals from five baleen whale species, two pinniped species and odontocete species were detected and separated into guilds. Although species diversity remained stable over time, the ice-avoiding and ice-affiliated species dominated before and after the onset of winter, respectively, and thus demonstrating a shift in guild composition related to season. Herein, we provide novel information about local marine mammal species diversity, community structure and residency times in a krill hotspot. Our study also demonstrates the utility of PAM data and its usefulness in providing new insights into the marine mammal habitat use and responses to environmental conditions, which are essential knowledge for the future development of a sustainable fishery management in a changing ecosystem.

Novel community data in ecology-properties and prospects.

New technologies for monitoring biodiversity such as environmental (e)DNA, passive acoustic monitoring, and optical sensors promise to generate automated spatiotemporal community observations at unprecedented scales and resolutions. Here, we introduce 'novel community data' as an umbrella term for these data. We review the emerging field around novel community data, focusing on new ecological questions that could be addressed; the analytical tools available or needed to make best use of these data; and the potential implications of these developments for policy and conservation. We conclude that novel community data offer many opportunities to advance our understanding of fundamental ecological processes, including community assembly, biotic interactions, micro- and macroevolution, and overall ecosystem functioning.