Links between energy budgets, somatic condition, and life history reveal heterogeneous energy management tactics in a group-living mesocarnivore.

BACKGROUND: Optimal management of voluntary energy expenditure is crucial to the survival and reproductive success of wild animals. Nevertheless, a growing appreciation of inter-individual variation in the internal state driving movement suggests that individuals may follow different, yet equally optimal tactics under the same environmental conditions. However, few studies in wild populations have investigated the occurrence and demographic context of different contemporaneous energetic expenditure tactics. Here, we explore this neglected aspect of energy budgeting in order to determine the effect of life-history traits such as age and reproductive status on the co-occurrence of different energy-budgeting tactics in wild populations. METHODS: We investigated inter-individual heterogeneity in energy expenditure within a wild population of European badgers (Meles meles) by quantifying individual overall dynamic body acceleration (ODBA, from tri-axial accelerometry collars) and total daily energy expenditure (DEE, from doubly-labelled water) during 6-9 day deployments and dosing periods over six different seasons (spring, summer, and autumn) in 2018-2019. We obtained ODBA values for 41 deployments (24 unique badgers) and DEE measurements for 41 dosings (22 unique badgers). We then evaluated correlations between these energetic metrics and computed individual ratios of ODBA/DEE as a proxy for the proportion of total energy spent on activity. We measured the impact of alternative ODBA/DEE ratios on body condition, and use survival models constructed using 29 years of demographic data from the same population to situate body-condition changes in the context of age and reproductive status. RESULTS: Both ODBA and DEE were highly variable between individuals and exhibited season-specific relationships with individual body condition and life-history factors. DEE scaled allometrically with body weight, but only in summer and autumn; post-reproductive female badgers were lighter than other badgers during the spring but expended on average 350 kJ/day more than predicted from allometric scaling. Older badgers expended significantly less energy on movement during the summer than did younger adults. The ratio of ODBA to DEE (OD) provides a measure of proportional investment into movement. This ratio correlated more significantly with next-season body condition than either energetic metric did independently. However, the majority of individuals with high OD ratios were either younger badgers or reproductive females, for which lower body condition typically presented less of a mortality risk in previous analyses of this population. CONCLUSIONS: Within a single population under the same environmental conditions, we found wide inter-individual variation in both mechanical and total energy expenditure. The adoption of different tactics aligns with relationships between life-history parameters and mortality risk previously studied within the population. Crucially, younger badgers and reproductive females appeared able to tolerate energy expenditure tactics that depleted their body condition more than other badgers. These findings provide a mechanism by which differences in individual energetic context set by life history can maintain heterogeneity in wild populations, providing a wide range of potential energetic tactics under changing environmental conditions.

Semantic segmentation of methane plumes with hyperspectral machine learning models.

Methane is the second most important greenhouse gas contributor to climate change; at the same time its reduction has been denoted as one of the fastest pathways to preventing temperature growth due to its short atmospheric lifetime. In particular, the mitigation of active point-sources associated with the fossil fuel industry has a strong and cost-effective mitigation potential. Detection of methane plumes in remote sensing data is possible, but the existing approaches exhibit high false positive rates and need manual intervention. Machine learning research in this area is limited due to the lack of large real-world annotated datasets. In this work, we are publicly releasing a machine learning ready dataset with manually refined annotation of methane plumes. We present labelled hyperspectral data from the AVIRIS-NG sensor and provide simulated multispectral WorldView-3 views of the same data to allow for model benchmarking across hyperspectral and multispectral sensors. We propose sensor agnostic machine learning architectures, using classical methane enhancement products as input features. Our HyperSTARCOP model outperforms strong matched filter baseline by over 25% in F1 score, while reducing its false positive rate per classified tile by over 41.83%. Additionally, we demonstrate zero-shot generalisation of our trained model on data from the EMIT hyperspectral instrument, despite the differences in the spectral and spatial resolution between the two sensors: in an annotated subset of EMIT images HyperSTARCOP achieves a 40% gain in F1 score over the baseline.

Deep Learning for Visual Localization and Mapping: A Survey.

Deep-learning-based localization and mapping approaches have recently emerged as a new research direction and receive significant attention from both industry and academia. Instead of creating hand-designed algorithms based on physical models or geometric theories, deep learning solutions provide an alternative to solve the problem in a data-driven way. Benefiting from the ever-increasing volumes of data and computational power on devices, these learning methods are fast evolving into a new area that shows potential to track self-motion and estimate environmental models accurately and robustly for mobile agents. In this work, we provide a comprehensive survey and propose a taxonomy for the localization and mapping methods using deep learning. This survey aims to discuss two basic questions: whether deep learning is promising for localization and mapping, and how deep learning should be applied to solve this problem. To this end, a series of localization and mapping topics are investigated, from the learning-based visual odometry and global relocalization to mapping, and simultaneous localization and mapping (SLAM). It is our hope that this survey organically weaves together the recent works in this vein from robotics, computer vision, and machine learning communities and serves as a guideline for future researchers to apply deep learning to tackle the problem of visual localization and mapping.

You Only Train Once: Learning General and Distinctive 3D Local Descriptors.

Extracting distinctive, robust, and general 3D local features is essential to downstream tasks such as point cloud registration. However, existing methods either rely on noise-sensitive handcrafted features, or depend on rotation-variant neural architectures. It remains challenging to learn robust and general local feature descriptors for surface matching. In this paper, we propose a new, simple yet effective neural network, termed SpinNet, to extract local surface descriptors which are rotation-invariant whilst sufficiently distinctive and general. A Spatial Point Transformer is first introduced to embed the input local surface into an elaborate cylindrical representation (SO(2) rotation-equivariant), further enabling end-to-end optimization of the entire framework. A Neural Feature Extractor, composed of point-based and 3D cylindrical convolutional layers, is then presented to learn representative and general geometric patterns. An invariant layer is finally used to generate rotation-invariant feature descriptors. Extensive experiments on both indoor and outdoor datasets demonstrate that SpinNet outperforms existing state-of-the-art techniques by a large margin. More critically, it has the best generalization ability across unseen scenarios with different sensor modalities.

Learning Selective Sensor Fusion for State Estimation.

Autonomous vehicles and mobile robotic systems are typically equipped with multiple sensors to provide redundancy. By integrating the observations from different sensors, these mobile agents are able to perceive the environment and estimate system states, e.g., locations and orientations. Although deep learning (DL) approaches for multimodal odometry estimation and localization have gained traction, they rarely focus on the issue of robust sensor fusion--a necessary consideration to deal with noisy or incomplete sensor observations in the real world. Moreover, current deep odometry models suffer from a lack of interpretability. To this extent, we propose SelectFusion, an end-to-end selective sensor fusion module that can be applied to useful pairs of sensor modalities, such as monocular images and inertial measurements, depth images, and light detection and ranging (LIDAR) point clouds. Our model is a uniform framework that is not restricted to specific modality or task. During prediction, the network is able to assess the reliability of the latent features from different sensor modalities and to estimate trajectory at both scale and global pose. In particular, we propose two fusion modules--a deterministic soft fusion and a stochastic hard fusion--and offer a comprehensive study of the new strategies compared with trivial direct fusion. We extensively evaluate all fusion strategies both on public datasets and on progressively degraded datasets that present synthetic occlusions, noisy and missing data, and time misalignment between sensors, and we investigate the effectiveness of the different fusion strategies in attending the most reliable features, which in itself provides insights into the operation of the various models.

Multiscale Human Activity Recognition and Anticipation Network.

Deep convolutional neural networks have been leveraged to achieve huge improvements in video understanding and human activity recognition performance in the past decade. However, most existing methods focus on activities that have similar time scales, leaving the task of action recognition on multiscale human behaviors less explored. In this study, a two-stream multiscale human activity recognition and anticipation (MS-HARA) network is proposed, which is jointly optimized using a multitask learning method. The MS-HARA network fuses the two streams of the network using an efficient temporal-channel attention (TCA)-based fusion approach to improve the model's representational ability for both temporal and spatial features. We investigate the multiscale human activities from two basic categories, namely, midterm activities and long-term activities. The network is designed to function as part of a real-time processing framework to support interaction and mutual understanding between humans and intelligent machines. It achieves state-of-the-art results on several datasets for different tasks and different application domains. The midterm and long-term action recognition and anticipation performance, as well as the network fusion, are extensively tested to show the efficiency of the proposed network. The results show that the MS-HARA network can easily be extended to different application domains.

SelfVIO: Self-supervised deep monocular Visual-Inertial Odometry and depth estimation.

In the last decade, numerous supervised deep learning approaches have been proposed for visual-inertial odometry (VIO) and depth map estimation, which require large amounts of labelled data. To overcome the data limitation, self-supervised learning has emerged as a promising alternative that exploits constraints such as geometric and photometric consistency in the scene. In this study, we present a novel self-supervised deep learning-based VIO and depth map recovery approach (SelfVIO) using adversarial training and self-adaptive visual-inertial sensor fusion. SelfVIO learns the joint estimation of 6 degrees-of-freedom (6-DoF) ego-motion and a depth map of the scene from unlabelled monocular RGB image sequences and inertial measurement unit (IMU) readings. The proposed approach is able to perform VIO without requiring IMU intrinsic parameters and/or extrinsic calibration between IMU and the camera. We provide comprehensive quantitative and qualitative evaluations of the proposed framework and compare its performance with state-of-the-art VIO, VO, and visual simultaneous localization and mapping (VSLAM) approaches on the KITTI, EuRoC and Cityscapes datasets. Detailed comparisons prove that SelfVIO outperforms state-of-the-art VIO approaches in terms of pose estimation and depth recovery, making it a promising approach among existing methods in the literature.

Learning Semantic Segmentation of Large-Scale Point Clouds With Random Sampling.

We study the problem of efficient semantic segmentation of large-scale 3D point clouds. By relying on expensive sampling techniques or computationally heavy pre/post-processing steps, most existing approaches are only able to be trained and operate over small-scale point clouds. In this paper, we introduce RandLA-Net, an efficient and lightweight neural architecture to directly infer per-point semantics for large-scale point clouds. The key to our approach is to use random point sampling instead of more complex point selection approaches. Although remarkably computation and memory efficient, random sampling can discard key features by chance. To overcome this, we introduce a novel local feature aggregation module to progressively increase the receptive field for each 3D point, thereby effectively preserving geometric details. Comparative experiments show that our RandLA-Net can process 1 million points in a single pass up to 200× faster than existing approaches. Moreover, extensive experiments on five large-scale point cloud datasets, including Semantic3D, SemanticKITTI, Toronto3D, NPM3D and S3DIS, demonstrate the state-of-the-art semantic segmentation performance of our RandLA-Net.

Deep learning-based robust positioning for all-weather autonomous driving.

Interest in autonomous vehicles (AVs) is growing at a rapid pace due to increased convenience, safety benefits and potential environmental gains. Although several leading AV companies predicted that AVs would be on the road by 2020, they are still limited to relatively small-scale trials. The ability to know their precise location on the map is a challenging prerequisite for safe and reliable AVs due to sensor imperfections under adverse environmental and weather conditions, posing a formidable obstacle to their widespread use. Here we propose a deep learning-based self-supervised approach for ego-motion estimation that is a robust and complementary localization solution under inclement weather conditions. The proposed approach is a geometry-aware method that attentively fuses the rich representation capability of visual sensors and the weather-immune features provided by radars using an attention-based learning technique. Our method predicts reliability masks for the sensor measurements, eliminating the deficiencies in the multimodal data. In various experiments we demonstrate the robust all-weather performance and effective cross-domain generalizability under harsh weather conditions such as rain, fog and snow, as well as day and night conditions. Furthermore, we employ a game-theoretic approach to analyse the interpretability of the model predictions, illustrating the independent and uncorrelated failure modes of the multimodal system. We anticipate our work will bring AVs one step closer to safe and reliable all-weather autonomous driving.

DynaNet: Neural Kalman Dynamical Model for Motion Estimation and Prediction.

Dynamical models estimate and predict the temporal evolution of physical systems. State-space models (SSMs) in particular represent the system dynamics with many desirable properties, such as being able to model uncertainty in both the model and measurements, and optimal (in the Bayesian sense) recursive formulations, e.g., the Kalman filter. However, they require significant domain knowledge to derive the parametric form and considerable hand tuning to correctly set all the parameters. Data-driven techniques, e.g., recurrent neural networks, have emerged as compelling alternatives to SSMs with wide success across a number of challenging tasks, in part due to their impressive capability to extract relevant features from rich inputs. They, however, lack interpretability and robustness to unseen conditions. Thus, data-driven models are hard to be applied in safety-critical applications, such as self-driving vehicles. In this work, we present DynaNet, a hybrid deep learning and time-varying SSM, which can be trained end-to-end. Our neural Kalman dynamical model allows us to exploit the relative merits of both SSM and deep neural networks. We demonstrate its effectiveness in the estimation and prediction on a number of physically challenging tasks, including visual odometry, sensor fusion for visual-inertial navigation, and motion prediction. In addition, we show how DynaNet can indicate failures through investigation of properties, such as the rate of innovation (Kalman gain).

Seismic localization of elephant rumbles as a monitoring approach.

African elephants (Loxodonta africana) are sentient and intelligent animals that use a variety of vocalizations to greet, warn or communicate with each other. Their low-frequency rumbles propagate through the air as well as through the ground and the physical properties of both media cause differences in frequency filtering and propagation distances of the respective wave. However, it is not well understood how each mode contributes to the animals' abilities to detect these rumbles and extract behavioural or spatial information. In this study, we recorded seismic and co-generated acoustic rumbles in Kenya and compared their potential use to localize the vocalizing animal using the same multi-lateration algorithms. For our experimental set-up, seismic localization has higher accuracy than acoustic, and bimodal localization does not improve results. We conclude that seismic rumbles can be used to remotely monitor and even decipher elephant social interactions, presenting us with a tool for far-reaching, non-intrusive and surprisingly informative wildlife monitoring.

Learning With Stochastic Guidance for Robot Navigation.

Due to the sparse rewards and high degree of environmental variation, reinforcement learning approaches, such as deep deterministic policy gradient (DDPG), are plagued by issues of high variance when applied in complex real-world environments. We present a new framework for overcoming these issues by incorporating a stochastic switch, allowing an agent to choose between high- and low-variance policies. The stochastic switch can be jointly trained with the original DDPG in the same framework. In this article, we demonstrate the power of the framework in a navigation task, where the robot can dynamically choose to learn through exploration or to use the output of a heuristic controller as guidance. Instead of starting from completely random actions, the navigation capability of a robot can be quickly bootstrapped by several simple independent controllers. The experimental results show that with the aid of stochastic guidance, we are able to effectively and efficiently train DDPG navigation policies and achieve significantly better performance than state-of-the-art baseline models.

Dense 3D Object Reconstruction from a Single Depth View.

In this paper, we propose a novel approach, 3D-RecGAN++, which reconstructs the complete 3D structure of a given object from a single arbitrary depth view using generative adversarial networks. Unlike existing work which typically requires multiple views of the same object or class labels to recover the full 3D geometry, the proposed 3D-RecGAN++ only takes the voxel grid representation of a depth view of the object as input, and is able to generate the complete 3D occupancy grid with a high resolution of 2563 by recovering the occluded/missing regions. The key idea is to combine the generative capabilities of 3D encoder-decoder and the conditional adversarial networks framework, to infer accurate and fine-grained 3D structures of objects in high-dimensional voxel space. Extensive experiments on large synthetic datasets and real-world Kinect datasets show that the proposed 3D-RecGAN++ significantly outperforms the state of the art in single view 3D object reconstruction, and is able to reconstruct unseen types of objects.

Getting to the core: Internal body temperatures help reveal the ecological function and thermal implications of the lions' mane.

It has been proposed that there is a thermal cost of the mane to male lions, potentially leading to increased body surface temperatures (Ts), increased sperm abnormalities, and to lower food intake during hot summer months. To test whether a mane imposes thermal costs on males, we measured core body temperature (Tb) continuously for approximately 1 year in 18 free-living lions. There was no difference in the 24-hr maximum Tb of males (n = 12) and females (n = 6), and males had a 24-hr mean Tb that was 0.2 ± 0.1°C lower than females after correcting for seasonal effects. Although feeding on a particular day increased 24-hr mean and 24-hr maximum Tb, this phenomenon was true of both male and female lions, and females had higher 24-hr mean and 24-hr maximum Tb than males, on both days when lions did not feed, and on days when lions did feed. Twenty-four-hour Tb was not influenced by mane length or color, and 24-hr mean Tb was negatively correlated with mane length. These data contradict the suggestion that there exists a thermal cost to male lions in possessing a long dark mane, but do not preclude the possibility that males compensate for a mane with increased heat loss. The increased insulation caused by a mane does not necessarily have to impair heat loss by males, which in hot environments is primarily through respiratory evaporative cooling, nor does in necessarily lead to increased heat gain, as lions are nocturnal and seek shade during the day. The mane may even act as a heat shield by increasing insulation. However, dominant male lions frequent water points more than twice as often as females, raising the possibility that male lions are increasing water uptake to facilitate increased evaporative cooling. The question of whether male lions with manes compensate for a thermal cost to the mane remains unresolved, but male lions with access to water do not have higher Tb than females or males with smaller manes.

Climate and the individual: inter-annual variation in the autumnal activity of the European badger (Meles meles).

We establish intra-individual and inter-annual variability in European badger (Meles meles) autumnal nightly activity in relation to fine-scale climatic variables, using tri-axial accelerometry. This contributes further to understanding of causality in the established interaction between weather conditions and population dynamics in this species. Modelling found that measures of daylight, rain/humidity, and soil temperature were the most supported predictors of ACTIVITY, in both years studied. In 2010, the drier year, the most supported model included the SOLAR*RH interaction, RAIN, and 30cmTEMP (w = 0.557), while in 2012, a wetter year, the most supported model included the SOLAR*RH interaction, and the RAIN*10cmTEMP (w = 0.999). ACTIVITY also differed significantly between individuals. In the 2012 autumn study period, badgers with the longest per noctem activity subsequently exhibited higher Body Condition Indices (BCI) when recaptured. In contrast, under drier 2010 conditions, badgers in good BCI engaged in less per noctem activity, while badgers with poor BCI were the most active. When compared on the same calendar dates, to control for night length, duration of mean badger nightly activity was longer (9.5 hrs ±3.3 SE) in 2010 than in 2012 (8.3 hrs ±1.9 SE). In the wetter year, increasing nightly activity was associated with net-positive energetic gains (from BCI), likely due to better foraging conditions. In a drier year, with greater potential for net-negative energy returns, individual nutritional state proved crucial in modifying activity regimes; thus we emphasise how a 'one size fits all' approach should not be applied to ecological responses.

Biochemical characterization of L-DOPA 2,3-dioxygenase, a single-domain type I extradiol dioxygenase from lincomycin biosynthesis.

L-DOPA-2,3-dioxygenase from Streptomyces lincolnensis is a single-domain type I extradiol dioxygenase of the vicinal oxygen chelate superfamily and catalyzes the second step in the metabolism of tyrosine to the propylhygric acid moiety of the antibiotic, lincomycin. S. lincolnensis L-DOPA-2,3-dioxygenase was overexpressed, purified and reconstituted with Fe(II). The activity of L-DOPA-2,3-dioxygenase was kinetically characterized with L-DOPA (K(M)=38 microM, k(cat)=4.2 min(-1)) and additional catecholic substrates including dopamine, 3,4-dihydroxyhydrocinnamic acid, catechol and D-DOPA. 3,4-Dihydroxyphenylacetic acid was characterized as a competitive inhibitor of the enzyme (K(i) =2.2 mM). Site-directed mutagenesis and its effects on enzymatic activity were used to identify His14 and His70 as iron ligands.