Leveraging social media and deep learning to detect rare megafauna in video surveys.

Deep learning has become a key tool for the automated monitoring of animal populations with video surveys. However, obtaining large numbers of images to train such models is a major challenge for rare and elusive species because field video surveys provide few sightings. We designed a method that takes advantage of videos accumulated on social media for training deep-learning models to detect rare megafauna species in the field. We trained convolutional neural networks (CNNs) with social media images and tested them on images collected from field surveys. We applied our method to aerial video surveys of dugongs (Dugong dugon) in New Caledonia (southwestern Pacific). CNNs trained with 1303 social media images yielded 25% false positives and 38% false negatives when tested on independent field video surveys. Incorporating a small number of images from New Caledonia (equivalent to 12% of social media images) in the training data set resulted in a nearly 50% decrease in false negatives. Our results highlight how and the extent to which images collected on social media can offer a solid basis for training deep-learning models for rare megafauna detection and that the incorporation of a few images from the study site further boosts detection accuracy. Our method provides a new generation of deep-learning models that can be used to rapidly and accurately process field video surveys for the monitoring of rare megafauna.

El aprendizaje profundo se ha convertido en una importante herramienta para el monitoreo automatizado de las poblaciones animales con video-censos. Sin embargo, la obtención de cantidades abundantes de imágenes para preparar dichos modelos es un reto primordial para las especies elusivas e infrecuentes porque los video-censos de campo proporcionan pocos avistamientos. Diseñamos un método que aprovecha los videos acumulados en las redes sociales para preparar a los modelos de aprendizaje profundo para detectar especies infrecuentes de megafauna en el campo. Preparamos algunas redes neurales convolucionales con imágenes tomadas de las redes sociales y las pusimos a prueba con imágenes tomadas en los censos de campo. Aplicamos nuestro método a los censos aéreos en video de dugongos (Dugong dugon) en Nueva Caledonia (Pacífico sudoccidental). Las redes neurales convolucionales preparadas con 1,303 imágenes de las redes sociales produjeron 25% de falsos positivos y 38% de falsos negativos cuando las probamos en video-censos de campo independientes. La incorporación de un número pequeño de imágenes tomadas en Nueva Caledonia (equivalente al 12% de las imágenes de las redes sociales) dentro del conjunto de datos usados en la preparación dio como resultado una disminución de casi el 50% en los falsos negativos. Nuestros resultados destacan cómo y a qué grado las imágenes recolectadas en las redes sociales pueden ofrecer una base sólida para la preparación de modelos de aprendizaje profundo para la detección de megafauna infrecuente. También resaltan que la incorporación de unas cuantas imágenes del sitio de estudio aumenta mucho más la certeza de detección. Nuestro método proporciona una nueva generación de modelos de aprendizaje profundo que pueden usarse para procesar rápida y acertadamente los video-censos de campo para el monitoreo de megafauna infrecuente.

Herbivorous fish feeding dynamics and energy expenditure on a coral reef: Insights from stereo-video and AI-driven 3D tracking.

Unveiling the intricate relationships between animal movement ecology, feeding behavior, and internal energy budgeting is crucial for a comprehensive understanding of ecosystem functioning, especially on coral reefs under significant anthropogenic stress. Here, herbivorous fishes play a vital role as mediators between algae growth and coral recruitment. Our research examines the feeding preferences, bite rates, inter-bite distances, and foraging energy expenditure of the Brown surgeonfish (Acanthurus nigrofuscus) and the Yellowtail tang (Zebrasoma xanthurum) within the fish community on a Red Sea coral reef. To this end, we used advanced methods such as remote underwater stereo-video, AI-driven object recognition, species classification, and 3D tracking. Despite their comparatively low biomass, the two surgeonfish species significantly influence grazing pressure on the studied coral reef. A. nigrofuscus exhibits specialized feeding preferences and Z. xanthurum a more generalist approach, highlighting niche differentiation and their importance in maintaining reef ecosystem balance. Despite these differences in their foraging strategies, on a population level, both species achieve a similar level of energy efficiency. This study highlights the transformative potential of cutting-edge technologies in revealing the functional feeding traits and energy utilization of keystone species. It facilitates the detailed mapping of energy seascapes, guiding targeted conservation efforts to enhance ecosystem health and biodiversity.

Author Correction: A new method to control error rates in automated species identification with deep learning algorithms.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

A new method to control error rates in automated species identification with deep learning algorithms.

Processing data from surveys using photos or videos remains a major bottleneck in ecology. Deep Learning Algorithms (DLAs) have been increasingly used to automatically identify organisms on images. However, despite recent advances, it remains difficult to control the error rate of such methods. Here, we proposed a new framework to control the error rate of DLAs. More precisely, for each species, a confidence threshold was automatically computed using a training dataset independent from the one used to train the DLAs. These species-specific thresholds were then used to post-process the outputs of the DLAs, assigning classification scores to each class for a given image including a new class called "unsure". We applied this framework to a study case identifying 20 fish species from 13,232 underwater images on coral reefs. The overall rate of species misclassification decreased from 22% with the raw DLAs to 2.98% after post-processing using the thresholds defined to minimize the risk of misclassification. This new framework has the potential to unclog the bottleneck of information extraction from massive digital data while ensuring a high level of accuracy in biodiversity assessment.