SonoNERFs: Neural Radiance Fields Applied to Biological Echolocation Systems Allow 3D Scene Reconstruction through Perceptual Prediction.

In this paper, we introduce SonoNERFs, a novel approach that adapts Neural Radiance Fields (NeRFs) to model and understand the echolocation process in bats, focusing on the challenges posed by acoustic data interpretation without phase information. Leveraging insights from the field of optical NeRFs, our model, termed SonoNERF, represents the acoustic environment through Neural Reflectivity Fields. This model allows us to reconstruct three-dimensional scenes from echolocation data, obtained by simulating how bats perceive their surroundings through sound. By integrating concepts from biological echolocation and modern computational models, we demonstrate the SonoNERF's ability to predict echo spectrograms for unseen echolocation poses and effectively reconstruct a mesh-based and energy-based representation of complex scenes. Our work bridges a gap in understanding biological echolocation and proposes a methodological framework that provides a first-order model of how scene understanding might arise in echolocating animals. We demonstrate the efficacy of the SonoNERF model on three scenes of increasing complexity, including some biologically relevant prey-predator interactions.

Bats integrate multiple echolocation and flight tactics to track prey.

The ability of "target tracking," such as keeping a target object in sight, is crucial for various activities. However, most sensing systems experience a certain degree of delay due to information processing, which challenges accurate target tracking. The long history of studies on animal behavior has revealed several tactics for it, although a systematic understanding of how individual tactics are combined into a strategy has not been reached. This study demonstrates a multifaceted tracking strategy in animals, which mitigates the adverse delay effects with small implementation costs. Using an active-sensing bat to measure their sensing state while chasing natural prey, we found that bats use a tracking strategy by combining multiple echolocation and flight tactics. The three echolocation tactics, namely the predictive control of sensing direction accompanied by adjusting the sensing rate and angular range, produce a direct compensation effect. Simultaneously, the flight tactic, the counter maneuver, assists echolocation by stabilizing the target direction. Our simulation results demonstrate that these combined tactics improve tracking accuracy over a wide range of delay constraints. In addition, a concise rule based on the angular velocity between bats and targets explains how bats control these tactics, suggesting that bats successfully reduce the burden of multitasking management. Our findings reveal the sophisticated strategy in animals' tracking systems and provide insights into understanding and developing efficiently integrated strategies in target tracking across various disciplines.

Clutter resilience via auditory stream segregation in echolocating greater mouse-eared bats.

Bats use echolocation to navigate and hunt in darkness, and must in that process segregate target echoes from unwanted clutter echoes. Bats may do this by approaching a target at steep angles relative to the plane of the background, utilizing their directional transmission and receiving systems to minimize clutter from background objects, but it remains unknown how bats negotiate clutter that cannot be spatially avoided. Here, we tested the hypothesis that when movement no longer offers spatial release, echolocating bats mitigate clutter by calling at lower source levels and longer call intervals to ease auditory streaming. We trained five greater mouse-eared bats (Myotis myotis) to land on a spherical loudspeaker with two microphones attached. We used a phantom-echo setup, where the loudspeaker/target transmitted phantom clutter echoes by playing back the bats' own calls at time delays of 1, 3 and 5 ms with a virtual target strength 7 dB higher than the physical target. We show that the bats successfully landed on the target, irrespective of the clutter echo delays. Rather than decreasing their source levels, the bats used similar source level distributions in clutter and control trials. Similarly, the bats did not increase their call intervals, but instead used the same distribution of call intervals across control and clutter trials. These observations reject our hypothesis, leading us to conclude that bats display great resilience to clutter via short auditory integration times and acute auditory stream segregation rather than via biosonar adjustments.

Daubenton's bats maintain stereotypical echolocation behaviour and a lombard response during target interception in light.

Most bats hunt insects on the wing at night using echolocation as their primary sensory modality, but nevertheless maintain complex eye anatomy and functional vision. This raises the question of how and when insectivorous bats use vision during their largely nocturnal lifestyle. Here, we test the hypothesis that the small insectivorous bat, Myotis daubentonii, relies less on echolocation, or dispenses with it entirely, as visual cues become available during challenging acoustic noise conditions. We trained five wild-caught bats to land on a spherical target in both silence and when exposed to broad-band noise to decrease echo detectability, while light conditions were manipulated in both spectrum and intensity. We show that during noise exposure, the bats were almost three times more likely to use multiple attempts to solve the task compared to in silent controls. Furthermore, the bats exhibited a Lombard response of 0.18 dB/dBnoise and decreased call intervals earlier in their flight during masking noise exposures compared to in silent controls. Importantly, however, these adjustments in movement and echolocation behaviour did not differ between light and dark control treatments showing that small insectivorous bats maintain the same echolocation behaviour when provided with visual cues under challenging conditions for echolocation. We therefore conclude that bat echolocation is a hard-wired sensory system with stereotyped compensation strategies to both target range and masking noise (i.e. Lombard response) irrespective of light conditions. In contrast, the adjustments of call intervals and movement strategies during noise exposure varied substantially between individuals indicating a degree of flexibility that likely requires higher order processing and perhaps vocal learning.

Bioinspiration from bats and new paradigms for autonomy in natural environments.

Achieving autonomous operation in complex natural environment remains an unsolved challenge. Conventional engineering approaches to this problem have focused on collecting large amounts of sensory data that are used to create detailed digital models of the environment. However, this only postpones solving the challenge of identifying the relevant sensory information and linking it to action control to the domain of the digital world model. Furthermore, it imposes high demands in terms of computing power and introduces large processing latencies that hamper autonomous real-time performance. Certain species of bats that are able to navigate and hunt their prey in dense vegetation could be a biological model system for an alternative approach to addressing the fundamental issues associated with autonomy in complex natural environments. Bats navigating in dense vegetation rely on clutter echoes, i.e. signals that consist of unresolved contributions from many scatters. Yet, the animals are able to extract the relevant information from these input signals with brains that are often less than 1 g in mass. Pilot results indicate that information relevant to location identification and passageway finding can be directly obtained from clutter echoes, opening up the possibility that the bats' skill can be replicated in man-made autonomous systems.

Bionic study of distance-azimuth discrimination of multi-scattered point objects in bat bio-sonar.

This paper presents a novel approach to enhance the discrimination capacity of multi-scattered point objects in bat bio-sonar. A broadband interferometer mathematical model is developed, incorporating both distance and azimuth information, to simulate the transmitted and received signals of bats. The Fourier transform is employed to simulate the preprocessing step of bat information for feature extraction. Furthermore, the bat bio-sonar model based on convolutional neural network (BS-CNN) is constructed to compensate for the limitations of conventional machine learning and CNN networks, including three strategies: Mix-up data enhancement, joint feature and hybrid atrous convolution module. The proposed BS-CNN model emulates the perceptual nerves of the bat brain for distance-azimuth discrimination and compares with four conventional classifiers to assess its discrimination efficacy. Experimental results demonstrate that the overall discrimination accuracy of the BS-CNN model is 93.4%, surpassing conventional CNN networks and machine learning methods by at least 5.9%. This improvement validates the efficacy of the BS-CNN bionic model in enhancing the discrimination accuracy in bat bio-sonar and offers valuable references for radar and sonar target classification.

Does rotation increase the acoustic field of view? Comparative models based on CT data of a live dolphin versus a dead dolphin.

Rotational behaviour has been observed when dolphins track or detect targets, however, its role in echolocation is unknown. We used computed tomography data of one live and one recently deceased bottlenose dolphin, together with measurements of the acoustic properties of head tissues, to perform acoustic property reconstruction. The anatomical configuration and acoustic properties of the main forehead structures between the live and deceased dolphins were compared. Finite element analysis (FEA) was applied to simulate the generation and propagation of echolocation clicks, to compute their waveforms and spectra in both near- and far-fields, and to derive echolocation beam patterns. Modelling results from both the live and deceased dolphins were in good agreement with click recordings from other, live, echolocating individuals. FEA was also used to estimate the acoustic scene experienced by a dolphin rotating 180° about its longitudinal axis to detect fish in the far-field at elevation angles of -20° to 20°. The results suggest that the rotational behaviour provides a wider insonification area and a wider receiving area. Thus, it may provide compensation for the dolphin's relatively narrow biosonar beam, asymmetries in sound reception, and constraints on the pointing direction that are limited by head movement. The results also have implications for examining the accuracy of FEA in acoustic simulations using recently deceased specimens.

Small-scale location identification in natural environments with deep learning based on biomimetic sonar echoes.

Many bat species navigate in complex, heavily vegetated habitats. To achieve this, the animal relies on a sensory basis that is very different from what is typically done in engineered systems that are designed for outdoor navigation. Whereas the engineered systems rely on data-heavy senses such as lidar, bats make do with echoes triggered by short, ultrasonic pulses. Prior work has shown that 'clutter echoes' originating from vegetation can convey information on the environment they were recorded in-despite their unpredictable nature. The current work has investigated the spatial granularity that these clutter echoes can convey by applying deep-learning location identification to an echo data set that resulted from the dense spatial sampling of a forest environment. The Global Positioning System (GPS) location corresponding to the echo collection events was clustered to break the survey area into the number of spatial patches ranging from two to 100. A convolutional neural network (Resnet 152) was used to identify the patch associated with echo sets ranging from one to ten echoes. The results demonstrate a spatial resolution that is comparable to the accuracy of recreation-grade GPS operating under foliage cover. This demonstrates that fine-grained location identification can be accomplished at very low data rates.

The Distinctive Forehead Cleft of the Risso's Dolphin (Grampus griseus) Hardly Affects Biosonar Beam Formation.

The Risso's dolphin (Grampus griseus) has a distinctive vertical crease (or cleft) along the anterior surface of the forehead. Previous studies have speculated that the cleft may contribute to biosonar beam formation. To explore this, we constructed 2D finite element models based on computer tomography data of the head of a naturally deceased Risso's dolphin. The simulated acoustic near-field signals, far-field signals, and transmission beam patterns were compared to corresponding measurements from a live, echolocating Risso's dolphin. To investigate the effect of the cleft, we filled the cleft with neighboring soft tissues in our model, creating a hypothetical "cleftless" forehead, as found in other odontocetes. We compared the acoustic pressure field and the beam pattern between the clefted and cleftless cases. Our results suggest that the cleft plays an insignificant role in forehead biosonar sound propagation and far-field beam formation. Furthermore, the cleft was not responsible for the bimodal click spectrum recorded and reported from this species.

High duty cycle moth sounds jam bat echolocation: bats counter with compensatory changes in buzz duration.

Tiger moth species vary greatly in the number of clicks they produce and the resultant duty cycle. Signals with higher duty cycles are expected to more effectively interfere with bat sonar. However, little is known about the minimum duty cycle of tiger moth signals for sonar jamming. Is there a threshold that allows us to classify moths as acoustically aposematic versus sonar jammers based on their duty cycles? We performed playback experiments with three wild-caught adult male bats, Eptesicus fuscus. Bat attacks on tethered moths were challenged using acoustic signals of Bertholdia trigona with modified duty cycles ranging from 0 to 46%. We did not find evidence for a duty cycle threshold; rather, the ability to jam the bat's sonar was a continuous function of duty cycle consistent with a steady increase in the number of clicks arriving during a critical signal processing time window just prior to the arrival of an echo. The proportion of successful captures significantly decreased as the moth duty cycle increased. Our findings suggest that moths cannot be unambiguously classified as acoustically aposematic or sonar jammers based solely on duty cycle. Bats appear to compensate for sonar jamming by lengthening the duration of their terminal buzz and they are more successful in capturing moths when they do so. In contrast to previous findings for bats performing difficult spatial tasks, the number of sonar sound groups decreased in response to high duty cycles and did not affect capture success.

Object-specific adaptation in the auditory cortex of bats.

Identifying behaviorally relevant sounds is a vital function of the auditory system. Echolocating bats must negotiate a wealth of sounds during navigation and foraging. They must detect relatively rare but behaviorally relevant echoes and segregate them from many unimportant background echoes. For this, the bat's auditory system might rely on neural deviance detection-a process influencing the excitability of a neuron depending on the frequency of occurrence of a stimulus. To investigate neural deviance detection in the auditory cortex (AC) of anesthetized bats (Phyllostomus discolor), we designed sequences of repetitive virtual echoes differing in the spectrotemporal envelope, resembling those that bats might perceive in their natural environment. A standard echo was repeatedly played 10 times in these sequences, followed by a deviant echo at the end. Time intervals between echoes within the sequences varied. Our results show that neurons in the AC of the bat P. discolor are sensitive to novel virtual echoes presented at the end of these repetitive sequences: In 49% (62/126) of cortical neurons, extracellularly recorded responses adapted to the standard echo but showed a strong response to the deviant echo presented at the end. This effect depended strongly on the time intervals between echoes, with stronger adaptation at shorter intervals. This type of response might indicate a form of neuronal deviance detection mechanism in the AC that could help the bats to detect echoes of novel and potentially important objects within a stream of homogeneous background echoes.NEW & NOTEWORTHY In this study, we show that neurons in the auditory cortex of the bat Phyllostomus discolor are sensitive to novel acoustic stimuli in the context of repetitive virtual echo sequences differing in spectrotemporal envelope. This represents a form of neuronal deviance detection that might help the bats to detect echoes of rare but relevant objects among the clutter.

Detection of passageways in natural foliage using biomimetic sonar.

The ability of certain bat species to navigate in dense vegetation based on trains of short biosonar echoes could provide for an alternative parsimonious approach to obtaining the sensory information that is needed to achieve autonomy in complex natural environments. Although bat biosonar has much lower data rates and spatial (angular) resolution than commonly used human-made sensing systems such as LiDAR or stereo cameras, bat species that live in dense habitats have the ability to reliably detect narrow passageways in foliage. To study the sensory information that the animals may have available to accomplish this, we have used a biomimetic sonar system that was combined with a camera to record echoes and synchronized images from 10 different field sites that featured narrow passageways in foliage. The synchronized camera and sonar data allowed us to create a large data set (130 000 samples) of labeled echoes using a teacher-student approach that used class labels derived from the images to provide training data for echo-based classifiers. The performance achieved in detecting passageways based on the field data closely matched previous results obtained for gaps in an artificial foliage setup in the laboratory. With a deep feature extraction neural network (VGG16) a foliage-versus-passageway classification accuracy of 96.64% was obtained. A transparent artificial intelligence approach (class-activation mapping) indicated that the classifier network relied heavily on the initial rising flank of the echoes. This finding could be exploited with a neuromorphic echo representation that consisted of times where the echo envelope crossed a certain amplitude threshold in a given frequency channel. Whereas a single amplitude threshold was sufficient for this in the previous laboratory study, multiple thresholds were needed to achieve an accuracy of 92.23%. These findings indicate that despite many sources of variability that shape clutter echoes from natural environments, these signals contain sufficient sensory information to enable the detection of passageways in foliage.

Neural Processing of Naturalistic Echolocation Signals in Bats.

Echolocation behavior, a navigation strategy based on acoustic signals, allows scientists to explore neural processing of behaviorally relevant stimuli. For the purpose of orientation, bats broadcast echolocation calls and extract spatial information from the echoes. Because bats control call emission and thus the availability of spatial information, the behavioral relevance of these signals is undiscussable. While most neurophysiological studies, conducted in the past, used synthesized acoustic stimuli that mimic portions of the echolocation signals, recent progress has been made to understand how naturalistic echolocation signals are encoded in the bat brain. Here, we review how does stimulus history affect neural processing, how spatial information from multiple objects and how echolocation signals embedded in a naturalistic, noisy environment are processed in the bat brain. We end our review by discussing the huge potential that state-of-the-art recording techniques provide to gain a more complete picture on the neuroethology of echolocation behavior.

Echolocating Daubenton's bats call louder, but show no spectral jamming avoidance in response to bands of masking noise during a landing task.

Echolocating bats listen for weak echoes to navigate and hunt, which makes them prone to masking from background noise and jamming from other bats and prey. As for electrical fish that display clear spectral jamming avoidance responses (JAR), bats have been reported to mitigate the effects of jamming by shifting the spectral contents of their calls, thereby reducing acoustic interference to improve echo-to-noise ratio (ENR). Here, we tested the hypothesis that frequency-modulating bats (FM bats) employ a spectral JAR in response to six masking noise bands ranging from 15 to 90 kHz, by measuring the -3 dB endpoints and peak frequency of echolocation calls from five male Daubenton's bats (Myotis daubentonii) during a landing task. The bats were trained to land on a noise-generating spherical transducer surrounded by a star-shaped microphone array, allowing for acoustic localization and source parameter quantification of on-axis calls. We show that the bats did not employ spectral JAR as the peak frequency during jamming remained unaltered compared with that of silent controls (all P>0.05, 60.73±0.96 kHz, mean±s.e.m.), and -3 dB endpoints decreased in noise irrespective of treatment type. Instead, Daubenton's bats responded to acoustic jamming by increasing call amplitude via a Lombard response that was bandwidth dependent, ranging from a mean of 0.05 dB/dB (95% confidence interval 0.04-0.06 dB/dB) noise for the most narrowband noise (15-30 kHz) to 0.17 dB/dB (0.16-0.18 dB/dB) noise for the most broadband noise (30-90 kHz). We conclude that Daubenton's bats, despite having the vocal flexibility to do so, do not employ a spectral JAR, but defend ENRs via a bandwidth-dependent Lombard response.

Echolocating Daubenton's bats are resilient to broadband, ultrasonic masking noise during active target approaches.

Echolocating bats hunt prey on the wing under conditions of poor lighting by emission of loud calls and subsequent auditory processing of weak returning echoes. To do so, they need adequate echo-to-noise ratios (ENRs) to detect and distinguish target echoes from masking noise. Early obstacle avoidance experiments report high resilience to masking in free-flying bats, but whether this is due to spectral or spatiotemporal release from masking, advanced auditory signal detection or an increase in call amplitude (Lombard effect) remains unresolved. We hypothesized that bats with no spectral, spatial or temporal release from masking noise defend a certain ENR via a Lombard effect. We trained four bats (Myotis daubentonii) to approach and land on a target that broadcasted broadband noise at four different levels. An array of seven microphones enabled acoustic localization of the bats and source level estimation of their approach calls. Call duration and peak frequency did not change, but average call source levels (SLRMS, at 0.1 m as dB re. 20 µPa) increased, from 112 dB in the no-noise treatment, to 118 dB (maximum 129 dB) at the maximum noise level of 94 dB re. 20 µPa root mean square. The magnitude of the Lombard effect was small (0.13 dB SLRMS dB-1 of noise), resulting in mean broadband and narrowband ENRs of -11 and 8 dB, respectively, at the highest noise level. Despite these poor ENRs, the bats still performed echo-guided landings, making us conclude that they are very resilient to masking even when they cannot avoid it spectrally, spatially or temporally.

Large-scale recognition of natural landmarks with deep learning based on biomimetic sonar echoes.

The ability to identify natural landmarks on a regional scale could contribute to the navigation skills of echolocating bats and also advance the quest for autonomy in natural environments with man-made systems. However, recognizing natural landmarks based on biosonar echoes has to deal with the unpredictable nature of echoes that are typically superpositions of contributions from many different reflectors with unknown properties. The results presented here show that a deep neural network (ResNet50) was able to classify ten different field sites and 20 different tracks (two at each site) distributed over an area about 40 km in diameter. Based on spectrogram representations of single echoes, classification accuracies up to 99.6% for different sites and 94.7% for different tracks have been achieved. Classification performance was found to depend on the used pulse component (constant-frequency-CF vs frequency-modulated-FM) and the trade-off between time and frequency resolution in the spectrogram representations of the echoes. For the former, classification performance increased monotonically with better time resolution. For the latter, classification performance peaked at an intermediate trade-off point between time and frequency resolution indicating that both dimensions contained relevant information. Future work will be needed to further characterize the quality of the spatial information contained in the echoes, e.g. in terms of spatial resolution and potential ambiguities.

The brain limit.

How fast the brain and muscles can respond to information about prey location constrains visual and echolocating predators in similar ways.

Echolocating toothed whales use ultra-fast echo-kinetic responses to track evasive prey.

Visual predators rely on fast-acting optokinetic responses to track and capture agile prey. Most toothed whales, however, rely on echolocation for hunting and have converged on biosonar clicking rates reaching 500/s during prey pursuits. If echoes are processed on a click-by-click basis, as assumed, neural responses 100× faster than those in vision are required to keep pace with this information flow. Using high-resolution biologging of wild predator-prey interactions, we show that toothed whales adjust clicking rates to track prey movement within 50-200 ms of prey escape responses. Hypothesising that these stereotyped biosonar adjustments are elicited by sudden prey accelerations, we measured echo-kinetic responses from trained harbour porpoises to a moving target and found similar latencies. High biosonar sampling rates are, therefore, not supported by extreme speeds of neural processing and muscular responses. Instead, the neurokinetic response times in echolocation are similar to those of tracking responses in vision, suggesting a common neural underpinning.

In the animal world, split-second decisions determine whether a predator eats, or its prey survives. There is a strong evolutionary advantage to fast reacting brains and bodies. For example, the eye muscles of hunting cheetahs must lock on to a gazelle and keep track of it, no matter how quickly or unpredictably it moves. In fact, in monkeys and primates, these muscles can react to sudden movements in as little as 50 milliseconds ­ faster than the blink of an eye. But what about animals that do not rely on vision to hunt? To find food at night or in the deep ocean, whales and porpoises make short ultrasonic sounds, or 'clicks', and then listen for returning echoes. As they close in on a prey, they need to click faster to get quicker updates on its location. What is unclear is how fast they react to the echoes. Just before a kill, a harbour porpoise can click over 500 times a second: if they wait for the echo from one click before making the next one, they would need responses 100 times faster than human eyes. Exploring this topic is difficult, as it requires tracking predator and prey at the same time. Vance et al. took up the challenge by building sound and movement recorders that attach to whales with suction cups. These were used on two different hunters: deep-diving beaked whales and shallow-hunting harbour porpoises. Both species adapted their click rate depending on how far they were from their prey, but their response times were similar to visual responses in monkeys and humans. This means that whales and porpoises do not act on each echo before clicking again: instead, they respond to groups of tens of clicks at a time. This suggests that their brains may be wired in much the same way as the ones of visual animals. In the ocean, increased human activity creates a dangerous noise pollution that disrupts the delicate hunting mechanism of whales and porpoises. Better understanding how these animals find their food may therefore help conservation efforts.

Bioinspired solution to finding passageways in foliage with sonar.

Finding narrow gaps in foliage is an important component skill for autonomous navigation in densely vegetated environments. Traditional approaches are based on collecting large amounts of data with high spatial resolution. However, the biosonar systems of bats that live in dense habitats demonstrate that finding gaps is possible based on sensors with angular resolutions that are poor compared to technologies such as man-made sonar and lidar. To investigate these capabilities, we have used a biomimetic sonar head to ensonify artificial hedges in the laboratory. We found that a conventional approach based on echo energy performed poorly on detecting gaps with the area under the receiver operating characteristic (ROC) curve ranging from 0.69 to 0.75 depending on the distance to the hedge and gap width. A deep-learning approach based on a convolutional neural network (CNN) operating on the echo spectrograms achieved area under the ROC curve (AUC) values between 0.94 and 0.97. Class activation mapping indicated that the rising flank of the echoes was critical for detecting the gaps. As a consequence, a simple code consisting of first threshold-crossing times was able to almost reproduce the performance of the CNN classifier (AUC 0.9 to 0.95). This demonstrates that the echo waveforms contained patterns that were indicative of a gap in the foliage but did not suffer from the comparatively large beamwidth used.

Directional biosonar beams allow echolocating harbour porpoises to actively discriminate and intercept closely spaced targets.

Echolocating toothed whales face the problem that high sound speeds in water mean that echoes from closely spaced targets will arrive at time delays within their reported auditory integration time of some 264 µs. Here, we test the hypothesis that echolocating harbour porpoises cannot resolve and discriminate targets within a clutter interference zone given by their integration time. To do this, we trained two harbour porpoises (Phocoena phocoena) to actively approach and choose between two spherical targets at four varying inter-target distances (13.5, 27, 56 and 108 cm) in a two-alternative forced-choice task. The free-swimming, blindfolded porpoises were tagged with a sound and movement tag (DTAG4) to record their echoic scene and acoustic outputs. The known ranges between targets and the porpoise, combined with the sound levels received on target-mounted hydrophones revealed how the porpoises controlled their acoustic gaze. When targets were close together, the discrimination task was more difficult because of smaller echo time delays and lower echo level ratios between the targets. Under these conditions, buzzes were longer and started from farther away, source levels were reduced at short ranges, and the porpoises clicked faster, scanned across the targets more, and delayed making their discrimination decision until closer to the target. We conclude that harbour porpoises can resolve and discriminate closely spaced targets, suggesting a clutter rejection zone much shorter than their auditory integration time, and that such clutter rejection is greatly aided by spatial filtering with their directional biosonar beam.