Policy Compression for Intelligent Continuous Control on Low-Power Edge Devices.

Interest in deploying deep reinforcement learning (DRL) models on low-power edge devices, such as Autonomous Mobile Robots (AMRs) and Internet of Things (IoT) devices, has seen a significant rise due to the potential of performing real-time inference by eliminating the latency and reliability issues incurred from wireless communication and the privacy benefits of processing data locally. Deploying such energy-intensive models on power-constrained devices is not always feasible, however, which has led to the development of model compression techniques that can reduce the size and computational complexity of DRL policies. Policy distillation, the most popular of these methods, can be used to first lower the number of network parameters by transferring the behavior of a large teacher network to a smaller student model before deploying these students at the edge. This works well with deterministic policies that operate using discrete actions. However, many real-world tasks that are power constrained, such as in the field of robotics, are formulated using continuous action spaces, which are not supported. In this work, we improve the policy distillation method to support the compression of DRL models designed to solve these continuous control tasks, with an emphasis on maintaining the stochastic nature of continuous DRL algorithms. Experiments show that our methods can be used effectively to compress such policies up to 750% while maintaining or even exceeding their teacher's performance by up to 41% in solving two popular continuous control tasks.

Design and Construction of an Experimental Setup to Enhance Mineral Weathering through the Activity of Soil Organisms.

Enhanced weathering (EW) is an emerging carbon dioxide (CO2) removal technology that can contribute to climate change mitigation. This technology relies on accelerating the natural process of mineral weathering in soils by manipulating the abiotic variables that govern this process, in particular mineral grain size and exposure to acids dissolved in water. EW mainly aims at reducing atmospheric CO2 concentrations by enhancing inorganic carbon sequestration. Until now, knowledge of EW has been mainly gained through experiments that focused on the abiotic variables known for stimulating mineral weathering, thereby neglecting the potential influence of biotic components. While bacteria, fungi, and earthworms are known to increase mineral weathering rates, the use of soil organisms in the context of EW remains underexplored. This protocol describes the design and construction of an experimental setup developed to enhance mineral weathering rates through soil organisms while concurrently controlling abiotic conditions. The setup is designed to maximize weathering rates while maintaining soil organisms' activity. It consists of a large number of columns filled with rock powder and organic material, located in a climate chamber and with water applied via a downflow irrigation system. Columns are placed above a fridge containing jerrycans to collect the leachate. Representative results demonstrate that this setup is suitable to ensure the activity of soil organisms and quantify their effect on inorganic carbon sequestration. Challenges remain in minimizing leachate losses, ensuring homogeneous ventilation through the climate chamber, and avoiding flooding of the columns. With this setup, an innovative and promising approach is proposed to enhance mineral weathering rates through the activity of soil biota and disentangle the effect of biotic and abiotic factors as drivers of EW.

A Learn-to-Rank Approach for Predicting Road Cycling Race Outcomes.

Professional road cycling is a very competitive sport, and many factors influence the outcome of the race. These factors can be internal (e.g., psychological preparedness, physiological profile of the rider, and the preparedness or fitness of the rider) or external (e.g., the weather or strategy of the team) to the rider, or even completely unpredictable (e.g., crashes or mechanical failure). This variety makes perfectly predicting the outcome of a certain race an impossible task and the sport even more interesting. Nonetheless, before each race, journalists, ex-pro cyclists, websites and cycling fans try to predict the possible top 3, 5, or 10 riders. In this article, we use easily accessible data on road cycling from the past 20 years and the Machine Learning technique Learn-to-Rank (LtR) to predict the top 10 contenders for 1-day road cycling races. We accomplish this by mapping a relevancy weight to the finishing place in the first 10 positions. We assess the performance of this approach on 2018, 2019, and 2021 editions of six spring classic 1-day races. In the end, we compare the output of the framework with a mass fan prediction on the Normalized Discounted Cumulative Gain (NDCG) metric and the number of correct top 10 guesses. We found that our model, on average, has slightly higher performance on both metrics than the mass fan prediction. We also analyze which variables of our model have the most influence on the prediction of each race. This approach can give interesting insights to fans before a race but can also be helpful to sports coaches to predict how a rider might perform compared to other riders outside of the team.