Monitoring the courtship flight trajectory of Latham's snipe (Gallinago hardwickii) using microphone arrays.

This study is the first to quantitatively measure the courtship display flights of Latham's snipe (Gallinago hardwickii), which is a "near threatened" species as of 2022 (IUCN red list of threatened species). By using a 16-channel microphone array and 8-channel microphone arrays, we localized the fine-scale movements of courtship flights of one male performing at high altitude and high speed, and we estimated the direction from which each sound arrived using robot audition. Preliminary analyses of the azimuthal and elevation angles of the courtship flights partially revealed a fine-scale flight trajectory. First, a male Latham's snipe gradually gained altitude while vocalizing sharp and harsh repeating calls, until it reached the flight peak altitude, then dove down while producing winnowing sound to the ground along the wetland zones without tall vegetation. This observation method is methodologically useful to establish a better understanding of Latham's snipe courtship flight site selection. Furthermore, this method can be extended to investigate other rare nocturnal or crepuscular birds that are too timid to risk ringing or tagging.

Auditory Survey of Endangered Eurasian Bittern Using Microphone Arrays and Robot Audition.

Bioacoustics monitoring has become increasingly popular for studying the behavior and ecology of vocalizing birds. This study aims to verify the practical effectiveness of localization technology for auditory monitoring of endangered Eurasian bittern (Botaurus stellaris) which inhabits wetlands in remote areas with thick vegetation. Their crepuscular and highly secretive nature, except during the breeding season when they vocalize advertisement calls, make them difficult to monitor. Because of the increasing rates of habitat loss, surveying accurate numbers and their habitat needs are both important conservation tasks. We investigated the feasibility of localizing their booming calls, at a low frequency range between 100-200 Hz, using microphone arrays and robot audition HARK (Honda Research Institute, Audition for Robots with Kyoto University). We first simulated sound source localization of actual bittern calls for microphone arrays of radii 10 cm, 50 cm, 1 m, and 10 m, under different noise levels. Second, we monitored bitterns in an actual field environment using small microphone arrays (height = 12 cm; width = 8 cm), in the Sarobetsu Mire, Hokkaido Island, Japan. The simulation results showed that the spectral detectability was higher for larger microphone arrays, whereas the temporal detectability was higher for smaller microphone arrays. We identified that false detection in smaller microphone arrays, which was coincidentally generated in the calculation proximate to the transfer function for the opposite side. Despite technical limitations, we successfully localized booming calls of at least two males in a reverberant wetland, surrounded by thick vegetation and riparian trees. This study is the first case of localizing such rare birds using small-sized microphone arrays in the field, thereby presenting how this technology could contribute to auditory surveys of population numbers, behaviors, and microhabitat selection, all of which are difficult to investigate using other observation methods. This methodology is not only useful for the better understanding of bitterns, but it can also be extended to investigate other rare nocturnal birds with low-frequency vocalizations, without direct ringing or tagging. Our results also suggest a future necessity for a robust localization system to avoid reverberation and echoing in the field, resulting in the false detection of the target birds.

Visualization of a chorus structure in multiple frog species by a sound discrimination device.

We developed a sound discrimination device to identify and localize the species of nocturnal animals in their natural habitat. The sound discrimination device is equipped with a microphone, a light-emitting diode, and a band-pass filter. By tuning the center frequency of the filter to include a dominant frequency of the calls of a focal species, we enable the device to be illuminated only when detecting the calls of the focal species. In experiments in a laboratory room, we tuned the sound discrimination devices to detect the calls of Hyla japonica or Rhacophorus schlegelii and broadcast the frog calls from loudspeakers. By analyzing the illumination pattern of the devices, we successfully identified and localized the two kinds of sound sources. Next, we placed the sound discrimination devices in a field site where actual male frogs (H. japonica and R. schlegelii) produced sounds. The analysis of the illumination pattern demonstrates the efficacy of the developed devices in a natural environment and also enables us to extract pairs of male frogs that significantly overlapped or alternated their calls.

A spatiotemporal analysis of acoustic interactions between great reed warblers (Acrocephalus arundinaceus) using microphone arrays and robot audition software HARK.

Acoustic interactions are important for understanding intra- and interspecific communication in songbird communities from the viewpoint of soundscape ecology. It has been suggested that birds may divide up sound space to increase communication efficiency in such a manner that they tend to avoid overlap with other birds when they sing. We are interested in clarifying the dynamics underlying the process as an example of complex systems based on short-term behavioral plasticity. However, it is very problematic to manually collect spatiotemporal patterns of acoustic events in natural habitats using data derived from a standard single-channel recording of several species singing simultaneously. Our purpose here was to investigate fine-scale spatiotemporal acoustic interactions of the great reed warbler. We surveyed spatial and temporal patterns of several vocalizing color-banded great reed warblers (Acrocephalus arundinaceus) using an open-source software for robot audition HARK (Honda Research Institute Japan Audition for Robots with Kyoto University) and three new 16-channel, stand-alone, and water-resistant microphone arrays, named DACHO spread out in the bird's habitat. We first show that our system estimated the location of two color-banded individuals' song posts with mean error distance of 5.5 ± 4.5 m from the location of observed song posts. We then evaluated the temporal localization accuracy of the songs by comparing the duration of localized songs around the song posts with those annotated by human observers, with an accuracy score of average 0.89 for one bird that stayed at one song post. We further found significant temporal overlap avoidance and an asymmetric relationship between songs of the two singing individuals, using transfer entropy. We believe that our system and analytical approach contribute to a better understanding of fine-scale acoustic interactions in time and space in bird communities.

Design of UAV-Embedded Microphone Array System for Sound Source Localization in Outdoor Environments.

In search and rescue activities, unmanned aerial vehicles (UAV) should exploit sound information to compensate for poor visual information. This paper describes the design and implementation of a UAV-embedded microphone array system for sound source localization in outdoor environments. Four critical development problems included water-resistance of the microphone array, efficiency in assembling, reliability of wireless communication, and sufficiency of visualization tools for operators. To solve these problems, we developed a spherical microphone array system (SMAS) consisting of a microphone array, a stable wireless network communication system, and intuitive visualization tools. The performance of SMAS was evaluated with simulated data and a demonstration in the field. Results confirmed that the SMAS provides highly accurate localization, water resistance, prompt assembly, stable wireless communication, and intuitive information for observers and operators.

Visualizing Phonotactic Behavior of Female Frogs in Darkness.

Many animals use sounds produced by conspecifics for mate identification. Female insects and anuran amphibians, for instance, use acoustic cues to localize, orient toward and approach conspecific males prior to mating. Here we present a novel technique that utilizes multiple, distributed sound-indication devices and a miniature LED backpack to visualize and record the nocturnal phonotactic approach of females of the Australian orange-eyed tree frog (Litoria chloris) both in a laboratory arena and in the animal's natural habitat. Continuous high-definition digital recording of the LED coordinates provides automatic tracking of the female's position, and the illumination patterns of the sound-indication devices allow us to discriminate multiple sound sources including loudspeakers broadcasting calls as well as calls emitted by individual male frogs. This innovative methodology is widely applicable for the study of phonotaxis and spatial structures of acoustically communicating nocturnal animals.

Spatio-temporal dynamics in collective frog choruses examined by mathematical modeling and field observations.

This paper reports theoretical and experimental studies on spatio-temporal dynamics in the choruses of male Japanese tree frogs. First, we theoretically model their calling times and positions as a system of coupled mobile oscillators. Numerical simulation of the model as well as calculation of the order parameters show that the spatio-temporal dynamics exhibits bistability between two-cluster antisynchronization and wavy antisynchronization, by assuming that the frogs are attracted to the edge of a simple circular breeding site. Second, we change the shape of the breeding site from the circle to rectangles including a straight line, and evaluate the stability of two-cluster and wavy antisynchronization. Numerical simulation shows that two-cluster antisynchronization is more frequently observed than wavy antisynchronization. Finally, we recorded frog choruses at an actual paddy field using our sound-imaging method. Analysis of the video demonstrated a consistent result with the aforementioned simulation: namely, two-cluster antisynchronization was more frequently realized.

Efficient blind dereverberation and echo cancellation based on independent component analysis for actual acoustic signals.

This letter presents a new algorithm for blind dereverberation and echo cancellation based on independent component analysis (ICA) for actual acoustic signals. We focus on frequency domain ICA (FD-ICA) because its computational cost and speed of learning convergence are sufficiently reasonable for practical applications such as hands-free speech recognition. In applying conventional FD-ICA as a preprocessing of automatic speech recognition in noisy environments, one of the most critical problems is how to cope with reverberations. To extract a clean signal from the reverberant observation, we model the separation process in the short-time Fourier transform domain and apply the multiple input/output inverse-filtering theorem (MINT) to the FD-ICA separation model. A naive implementation of this method is computationally expensive, because its time complexity is the second order of reverberation time. Therefore, the main issue in dereverberation is to reduce the high computational cost of ICA. In this letter, we reduce the computational complexity to the linear order of the reverberation time by using two techniques: (1) a separation model based on the independence of delayed observed signals with MINT and (2) spatial sphering for preprocessing. Experiments show that the computational cost grows in proportion to the linear order of the reverberation time and that our method improves the word correctness of automatic speech recognition by 10 to 20 points in a RT20= 670 ms reverberant environment.

Sound imaging of nocturnal animal calls in their natural habitat.

We present a novel method for imaging acoustic communication between nocturnal animals. Investigating the spatio-temporal calling behavior of nocturnal animals, e.g., frogs and crickets, has been difficult because of the need to distinguish many animals' calls in noisy environments without being able to see them. Our method visualizes the spatial and temporal dynamics using dozens of sound-to-light conversion devices (called "Firefly") and an off-the-shelf video camera. The Firefly, which consists of a microphone and a light emitting diode, emits light when it captures nearby sound. Deploying dozens of Fireflies in a target area, we record calls of multiple individuals through the video camera. We conduct two experiments, one indoors and the other in the field, using Japanese tree frogs (Hyla japonica). The indoor experiment demonstrates that our method correctly visualizes Japanese tree frogs' calling behavior. It has confirmed the known behavior; two frogs call synchronously or in anti-phase synchronization. The field experiment (in a rice paddy where Japanese tree frogs live) also visualizes the same calling behavior to confirm anti-phase synchronization in the field. Experimental results confirm that our method can visualize the calling behavior of nocturnal animals in their natural habitat.

Complex and transitive synchronization in a frustrated system of calling frogs.

This letter reports synchronization phenomena and mathematical modeling on a frustrated system of living beings, or Japanese tree frogs (Hyla japonica). While an isolated male Japanese tree frog calls nearly periodically, he can hear sounds including calls of other males. Therefore, the spontaneous calling behavior of interacting males can be understood as a system of coupled oscillators. We construct a simple but biologically reasonable model based on the experimental results of two frogs, extend the model to a system of three frogs, and theoretically predict the occurrence of rich synchronization phenomena, such as triphase synchronization and 1:2 antiphase synchronization. In addition, we experimentally verify the theoretical prediction by ethological experiments on the calling behavior of three frogs and time series analysis on recorded sound data. Note that the calling behavior of three male Japanese tree frogs is frustrated because almost perfect antiphase synchronization is robustly observed in a system of two male frogs. Thus, nonlinear dynamics of the three-frogs system should be far from trivial.

Emergence of hierarchical structure mirroring linguistic composition in a recurrent neural network.

We show that a Multiple Timescale Recurrent Neural Network (MTRNN) can acquire the capabilities to recognize, generate, and correct sentences by self-organizing in a way that mirrors the hierarchical structure of sentences: characters grouped into words, and words into sentences. The model can control which sentence to generate depending on its initial states (generation phase) and the initial states can be calculated from the target sentence (recognition phase). In an experiment, we trained our model over a set of unannotated sentences from an artificial language, represented as sequences of characters. Once trained, the model could recognize and generate grammatical sentences, even if they were not learned. Moreover, we found that our model could correct a few substitution errors in a sentence, and the correction performance was improved by adding the errors to the training sentences in each training iteration with a certain probability. An analysis of the neural activations in our model revealed that the MTRNN had self-organized, reflecting the hierarchical linguistic structure by taking advantage of the differences in timescale among its neurons: in particular, neurons that change the fastest represented "characters", those that change more slowly, "words", and those that change the slowest, "sentences".

A computational model of monkey cortical grating cells.

Grating cells were discovered in the V1 and V2 areas of the monkey visual cortex by von der Heydt et al. (1992). These cells responded vigorously to grating patterns of appropriate orientation and periodicity. Computational models inspired by these findings were used as texture operator (Kruzinga and Petkov 1995, 1999; Petkov and Kruzinga 1997) and for the emergence and self-organization of grating cells (Brunner et al. 1998; Bauer et al. 1999). The aim of this paper is to create a grating cell operator that demonstrates similar responses to monkey grating cells by applying operator to the same stimuli as in the experiments carried out by von der Heydt et al. (1992). Operator will be tested on images that contain periodic patterns as suggested by De Valois (1988). In order to learn more about the role of grating cells in natural vision, operator is applied to 338 real-world images of textures obtained from three different databases. The results suggest that grating cells respond strongly to regular alternating periodic patterns of a certain orientation. Such patterns are common in images of human-made structures, like buildings, fabrics, and tiles, and to regular natural periodic patterns, which are relatively rare in nature.