Making Bipedal Robot Experiments Reproducible and Comparable: The Eurobench Software Approach.

This study describes the software methodology designed for systematic benchmarking of bipedal systems through the computation of performance indicators from data collected during an experimentation stage. Under the umbrella of the European project Eurobench, we collected approximately 30 protocols with related testbeds and scoring algorithms, aiming at characterizing the performances of humanoids, exoskeletons, and/or prosthesis under different conditions. The main challenge addressed in this study concerns the standardization of the scoring process to permit a systematic benchmark of the experiments. The complexity of this process is mainly due to the lack of consistency in how to store and organize experimental data, how to define the input and output of benchmarking algorithms, and how to implement these algorithms. We propose a simple but efficient methodology for preparing scoring algorithms, to ensure reproducibility and replicability of results. This methodology mainly constrains the interface of the software and enables the engineer to develop his/her metric in his/her favorite language. Continuous integration and deployment tools are then used to verify the replicability of the software and to generate an executable instance independent of the language through dockerization. This article presents this methodology and points at all the metrics and documentation repositories designed with this policy in Eurobench. Applying this approach to other protocols and metrics would ease the reproduction, replication, and comparison of experiments.

A novel human effort estimation method for knee assistive exoskeletons.

In this work we present a novel method to estimate online the torques at the knee joints with the goal to generate reference signals for knee assistive devices. One of the main advantages of the proposed approach is its reduced sensing requirements, which leads to an ergonomic setup with minimal instrumentation, especially above the knee and of the upper body. Indeed, only the measurement of the forces and torques exchanged between the ground and the user's feet, the posture of the shanks, and the model of the user's shank itself are needed for the estimation of the knee torque. The method does not require information of the state of the upper body and of a possible payload i.e. body pose, mass and center of mass (CoM) location. As a result, a minimalistic sensory system consisting of sensorized shoes and IMUs to track the shanks' orientation are adequate, allowing for an easily wearable and portable setup. The estimation of the knee torques is achieved by imposing an equilibrium condition to the user's shank. Several experiments were performed to test the effectiveness of the proposed estimation method under different body postures and motions (e.g. squat motion and switching foot contacts) and payloads (e.g. by holding weights at different arm postures resulting in variable upper body CoM). Finally an assistive task, conducted with the iT-Knee bipedal system is presented, where the lifted payload changed its CoM location over time.