Information-Driven Gas Distribution Mapping for Autonomous Mobile Robots.

The ability to sense airborne pollutants with mobile robots provides a valuable asset for domains such as industrial safety and environmental monitoring. Oftentimes, this involves detecting how certain gases are spread out in the environment, commonly referred to as a gas distribution map, to subsequently take actions that depend on the collected information. Since the majority of gas transducers require physical contact with the analyte to sense it, the generation of such a map usually involves slow and laborious data collection from all key locations. In this regard, this paper proposes an efficient exploration algorithm for 2D gas distribution mapping with an autonomous mobile robot. Our proposal combines a Gaussian Markov random field estimator based on gas and wind flow measurements, devised for very sparse sample sizes and indoor environments, with a partially observable Markov decision process to close the robot's control loop. The advantage of this approach is that the gas map is not only continuously updated, but can also be leveraged to choose the next location based on how much information it provides. The exploration consequently adapts to how the gas is distributed during run time, leading to an efficient sampling path and, in turn, a complete gas map with a relatively low number of measurements. Furthermore, it also accounts for wind currents in the environment, which improves the reliability of the final gas map even in the presence of obstacles or when the gas distribution diverges from an ideal gas plume. Finally, we report various simulation experiments to evaluate our proposal against a computer-generated fluid dynamics ground truth, as well as physical experiments in a wind tunnel.

Integrating Social Assistive Robots, IoT, Virtual Communities and Smart Objects to Assist at-Home Independently Living Elders: the MoveCare Project.

The integration of Ambient Assisted Living (AAL) frameworks with Socially Assistive Robots (SARs) has proven useful for monitoring and assisting older adults in their own home. However, the difficulties associated with long-term deployments in real-world complex environments are still highly under-explored. In this work, we first present the MoveCare system, an unobtrusive platform that, through the integration of a SAR into an AAL framework, aimed to monitor, assist and provide social, cognitive, and physical stimulation in the own houses of elders living alone and at risk of falling into frailty. We then focus on the evaluation and analysis of a long-term pilot campaign of more than 300 weeks of usages. We evaluated the system's acceptability and feasibility through various questionnaires and empirically assessed the impact of the presence of an assistive robot by deploying the system with and without it. Our results provide strong empirical evidence that Socially Assistive Robots integrated with monitoring and stimulation platforms can be successfully used for long-term support to older adults. We describe how the robot's presence significantly incentivised the use of the system, but slightly lowered the system's overall acceptability. Finally, we emphasise that real-world long-term deployment of SARs introduces a significant technical, organisational, and logistical overhead that should not be neglected nor underestimated in the pursuit of long-term robust systems. We hope that the findings and lessons learned from our work can bring value towards future long-term real-world and widespread use of SARs.

Efficient 3D Lidar Odometry Based on Planar Patches.

In this paper we present a new way to compute the odometry of a 3D lidar in real-time. Due to the significant relation between these sensors and the rapidly increasing sector of autonomous vehicles, 3D lidars have improved in recent years, with modern models producing data in the form of range images. We take advantage of this ordered format to efficiently estimate the trajectory of the sensor as it moves in 3D space. The proposed method creates and leverages a flatness image in order to exploit the information found in flat surfaces of the scene. This allows for an efficient selection of planar patches from a first range image. Then, from a second image, keypoints related to said patches are extracted. This way, our proposal computes the ego-motion by imposing a coplanarity constraint between pairs whose correspondences are iteratively updated. The proposed algorithm is tested and compared with state-of-the-art ICP algorithms. Experiments show that our proposal, running on a single thread, can run 5× faster than a multi-threaded implementation of GICP, while providing a more accurate localization. A second version of the algorithm is also presented, which reduces the drift even further while needing less than half of the computation time of GICP. Both configurations of the algorithm run at frame rates common for most 3D lidars, 10 and 20 Hz on a standard CPU.

LTC-Mapping, Enhancing Long-Term Consistency of Object-Oriented Semantic Maps in Robotics.

This paper proposes LTC-Mapping, a method for building object-oriented semantic maps that remain consistent in the long-term operation of mobile robots. Among the different challenges that compromise this aim, LTC-Mapping focuses on two of the more relevant ones: preventing duplicate instances of objects (instance duplication) and handling dynamic scenes. The former refers to creating multiple instances of the same physical object in the map, usually as a consequence of partial views or occlusions. The latter deals with the typical assumption made by object-oriented mapping methods that the world is static, resulting in outdated representations when the objects change their positions. To face these issues, we model the detected objects with 3D bounding boxes, and analyze the visibility of their vertices to detect occlusions and partial views. Besides this geometric modeling, the boxes are augmented with semantic information regarding the categories of the objects they represent. Both the geometric entities (bounding boxes) and their semantic content are propagated over time through data association and a fusion technique. In addition, in order to keep the map curated, the non-detection of objects in the areas where they should appear is also considered, proposing a mechanism that removes them from the map once there is evidence that they have been moved (i.e., multiple non-detections occur). To validate our proposal, a number of experiments have been carried out using the Robot@VirtualHome ecosystem, comparing its performance with a state-of-the-art alternative. The results report a superior performance of LTC-Mapping when modeling both geometric and semantic information of objects, and also support its online execution.

Human motion capture for movement limitation analysis using an RGB-D camera in spondyloarthritis: a validation study.

A human motion capture system using an RGB-D camera could be a good option to understand the trunk limitations in spondyloarthritis. The aim of this study is to validate a human motion capture system using an RGB-D camera to analyse trunk movement limitations in spondyloarthritis patients. Cross-sectional study was performed where spondyloarthritis patients were diagnosed with a rheumatologist. The RGB-D camera analysed the kinematics of each participant during seven functional tasks based on rheumatologic assessment. The OpenNI2 library collected the depth data, the NiTE2 middleware detected a virtual skeleton and the MRPT library recorded the trunk positions. The gold standard was registered using an inertial measurement unit. The outcome variables were angular displacement, angular velocity and lineal acceleration of the trunk. Criterion validity and the reliability were calculated. Seventeen subjects (54.35 (11.75) years) were measured. The Bending task obtained moderate results in validity (r = 0.55-0.62) and successful results in reliability (ICC = 0.80-0.88) and validity and reliability of angular kinematic results in Chair task were moderate and (r = 0.60-0.74, ICC = 0.61-0.72). The kinematic results in Timed Up and Go test were less consistent. The RGB-D camera was documented to be a reliable tool to assess the movement limitations in spondyloarthritis depending on the functional tasks: Bending task. Chair task needs further research and the TUG analysis was not validated. Comparation of both systems, required software for camera analysis, outcomes and final results of validity and reliability of each test.

Appearance-Based Sequential Robot Localization Using a Patchwise Approximation of a Descriptor Manifold.

This paper addresses appearance-based robot localization in 2D with a sparse, lightweight map of the environment composed of descriptor-pose image pairs. Based on previous research in the field, we assume that image descriptors are samples of a low-dimensional Descriptor Manifold that is locally articulated by the camera pose. We propose a piecewise approximation of the geometry of such Descriptor Manifold through a tessellation of so-called Patches of Smooth Appearance Change (PSACs), which defines our appearance map. Upon this map, the presented robot localization method applies both a Gaussian Process Particle Filter (GPPF) to perform camera tracking and a Place Recognition (PR) technique for relocalization within the most likely PSACs according to the observed descriptor. A specific Gaussian Process (GP) is trained for each PSAC to regress a Gaussian distribution over the descriptor for any particle pose lying within that PSAC. The evaluation of the observed descriptor in this distribution gives us a likelihood, which is used as the weight for the particle. Besides, we model the impact of appearance variations on image descriptors as a white noise distribution within the GP formulation, ensuring adequate operation under lighting and scene appearance changes with respect to the conditions in which the map was constructed. A series of experiments with both real and synthetic images show that our method outperforms state-of-the-art appearance-based localization methods in terms of robustness and accuracy, with median errors below 0.3 m and 6°.

A Simulation Framework for the Integration of Artificial Olfaction into Multi-Sensor Mobile Robots.

The simulation of how a gas disperses in a environment is a necessary asset for the development of olfaction-based autonomous agents. A variety of simulators already exist for this purpose, but none of them allows for a sufficiently convenient integration with other types of sensing (such as vision), which hinders the development of advanced, multi-sensor olfactory robotics applications. In this work, we present a framework for the simulation of gas dispersal and sensing alongside vision by integrating GADEN, a state-of-the-art Gas Dispersion Simulator, with the Unity 3D, a video game development engine that is used in many different areas of research and helps with the creation of visually realistic, complex environments. We discuss the motivation for the development of this tool, describe its characteristics, and present some potential use cases that are based on cutting-edge research in the field of olfactory robotics.

Improving the Head Pose Variation Problem in Face Recognition for Mobile Robots.

Face recognition is a technology with great potential in the field of robotics, due to its prominent role in human-robot interaction (HRI). This interaction is a keystone for the successful deployment of robots in areas requiring a customized assistance like education and healthcare, or assisting humans in everyday tasks. These unconstrained environments present additional difficulties for face recognition, extreme head pose variability being one of the most challenging. In this paper, we address this issue and make a fourfold contribution. First, it has been designed a tool for gathering an uniform distribution of head pose images from a person, which has been used to collect a new dataset of faces, both presented in this work. Then, the dataset has served as a testbed for analyzing the detrimental effects this problem has on a number of state-of-the-art methods, showing their decreased effectiveness outside a limited range of poses. Finally, we propose an optimization method to mitigate said negative effects by considering key pose samples in the recognition system's set of known faces. The conducted experiments demonstrate that this optimized set of poses significantly improves the performance of a state-of-the-art, cutting-edge system based on Multitask Cascaded Convolutional Neural Networks (MTCNNs) and ArcFace.

Differences in movement limitations in different low back pain severity in functional tests using an RGB-D camera.

Low back pain (LBP) can lead to motor control disturbance which can be one of the causes of reoccurrence of the complaint. It is important to improve our knowledge of movement related disturbances during assessment in LBP and to classify patients according to the severity. The aim of this study is to present differences in kinematic variables using a RGB-D camera in order to classify LBP patients with different severity. A cross-sectional study was carried out. Subjects with non-specific subacute and chronic LBP were screened 6 weeks following an episode. Functional tests were bending trunk test, sock test and sit to stand test. Participants performed as many repetitions as possible during 30 s for each functional test. Angular displacement, velocity and acceleration, linear acceleration, time and repetitions were analysed. Participants were divided into two groups to determine their different LBP severity with a k-means clusters according to the results obtained in Roland Morris questionnaire (RMQ). Comparing different severity groups based on RMQ score (high impact = 17.15, low impact = 7.47), bending trunk test obtained significative differences in linear acceleration (p = 0.002-0.01). The differences of total linear acceleration during the Sit to Stand test were significative (p = 0.004-0.02). Sock test showed not significative differences between groups (p > 0.05). Linear acceleration variables during Sit to Stand test and Bending trunk test were significatively different between the different severity groups. RGB-D camera system and functional tests can detect kinematic differences in different type of LBP according to the functionality. Trial registration: ClinicalTrials.gov NCT03293095 "Functional Task Kinematic in Musculoskeletal Pathology" September 26, 2017.

Validation, Reliability, and Responsiveness Outcomes Of Kinematic Assessment With An RGB-D Camera To Analyze Movement In Subacute And Chronic Low Back Pain.

BACKGROUND: The RGB-D camera is an alternative to asses kinematics in order to obtain objective measurements of functional limitations. The aim of this study is to analyze the validity, reliability, and responsiveness of the motion capture depth camera in sub-acute and chronic low back pain patients. METHODS: Thirty subjects (18-65 years) with non-specific lumbar pain were screened 6 weeks following an episode. RGB-D camera measurements were compared with an inertial measurement unit. Functional tests included climbing stairs, bending, reaching sock, lie-to-sit, sit-to-stand, and timed up-and-go. Subjects performed the maximum number of repetitions during 30 s. Validity was analyzed using Spearman's correlation, reliability of repetitions was calculated by the intraclass correlation coefficient and the standard error of measurement, and receiver operating characteristic curves were calculated to assess the responsiveness. RESULTS: The kinematic analysis obtained variable results according to the test. The time variable had good values in the validity and reliability of all tests (r = 0.93-1.00, (intraclass correlation coefficient (ICC) = 0.62-0.93). Regarding kinematics, the best results were obtained in bending test, sock test, and sit-to-stand test (r = 0.53-0.80, ICC = 0.64-0.83, area under the curve (AUC) = 0.55-84). CONCLUSION: Functional tasks, such as bending, sit-to-stand, reaching, and putting on sock, assessed with the RGB-D camera, revealed acceptable validity, reliability, and responsiveness in the assessment of patients with low back pain (LBP). TRIAL REGISTRATION: ClinicalTrials.gov NCT03293095 "Functional Task Kinematic in Musculoskeletal Pathology" September 26, 2017.

Human 3D Pose Estimation with a Tilting Camera for Social Mobile Robot Interaction.

Human-Robot interaction represents a cornerstone of mobile robotics, especially within the field of social robots. In this context, user localization becomes of crucial importance for the interaction. This work investigates the capabilities of wide field-of-view RGB cameras to estimate the 3D position and orientation (i.e., the pose) of a user in the environment. For that, we employ a social robot endowed with a fish-eye camera hosted in a tilting head and develop two complementary approaches: (1) a fast method relying on a single image that estimates the user pose from the detection of their feet and does not require either the robot or the user to remain static during the reconstruction; and (2) a method that takes some views of the scene while the camera is being tilted and does not need the feet to be visible. Due to the particular setup of the tilting camera, special equations for 3D reconstruction have been developed. In both approaches, a CNN-based skeleton detector (OpenPose) is employed to identify humans within the image. A set of experiments with real data validate our two proposed methods, yielding similar results than commercial RGB-D cameras while surpassing them in terms of coverage of the scene (wider FoV and longer range) and robustness to light conditions.

Olfaction, Vision, and Semantics for Mobile Robots. Results of the IRO Project.

Olfaction is a valuable source of information about the environment that has not been sufficiently exploited in mobile robotics yet. Certainly, odor information can contribute to other sensing modalities, e.g., vision, to accomplish high-level robot activities, such as task planning or execution in human environments. This paper organizes and puts together the developments and experiences on combining olfaction and vision into robotics applications, as the result of our five-years long project IRO: Improvement of the sensory and autonomous capability of Robots through Olfaction. Particularly, it investigates mechanisms to exploit odor information (usually coming in the form of the type of volatile and its concentration) in problems such as object recognition and scene-activity understanding. A distinctive aspect of this research is the special attention paid to the role of semantics within the robot perception and decision-making processes. The obtained results have improved the robot capabilities in terms of efficiency, autonomy, and usefulness, as reported in our publications.

Automatic Waypoint Generation to Improve Robot Navigation Through Narrow Spaces.

In domestic robotics, passing through narrow areas becomes critical for safe and effective robot navigation. Due to factors like sensor noise or miscalibration, even if the free space is sufficient for the robot to pass through, it may not see enough clearance to navigate, hence limiting its operational space. An approach to facing this is to insert waypoints strategically placed within the problematic areas in the map, which are considered by the robot planner when generating a trajectory and help to successfully traverse them. This is typically carried out by a human operator either by relying on their experience or by trial-and-error. In this paper, we present an automatic procedure to perform this task that: (i) detects problematic areas in the map and (ii) generates a set of auxiliary navigation waypoints from which more suitable trajectories can be generated by the robot planner. Our proposal, fully compatible with the robotic operating system (ROS), has been successfully applied to robots deployed in different houses within the H2020 MoveCare project. Moreover, we have performed extensive simulations with four state-of-the-art robots operating within real maps. The results reveal significant improvements in the number of successful navigations for the evaluated scenarios, demonstrating its efficacy in realistic situations.

A Semantic-Based Gas Source Localization with a Mobile Robot Combining Vision and Chemical Sensing.

This paper addresses the localization of a gas emission source within a real-world human environment with a mobile robot. Our approach is based on an efficient and coherent system that fuses different sensor modalities (i.e., vision and chemical sensing) to exploit, for the first time, the semantic relationships among the detected gases and the objects visually recognized in the environment. This novel approach allows the robot to focus the search on a finite set of potential gas source candidates (dynamically updated as the robot operates), while accounting for the non-negligible uncertainties in the object recognition and gas classification tasks involved in the process. This approach is particularly interesting for structured indoor environments containing multiple obstacles and objects, enabling the inference of the relations between objects and between objects and gases. A probabilistic Bayesian framework is proposed to handle all these uncertainties and semantic relations, providing an ordered list of candidates to be the source. This candidate list is updated dynamically upon new sensor measurements to account for objects not previously considered in the search process. The exploitation of such probabilities together with information such as the locations of the objects, or the time needed to validate whether a given candidate is truly releasing gases, is delegated to a path planning algorithm based on Markov decision processes to minimize the search time. The system was tested in an office-like scenario, both with simulated and real experiments, to enable the comparison of different path planning strategies and to validate its efficiency under real-world conditions.

GADEN: A 3D Gas Dispersion Simulator for Mobile Robot Olfaction in Realistic Environments.

This work presents a simulation framework developed under the widely used Robot Operating System (ROS) to enable the validation of robotics systems and gas sensing algorithms under realistic environments. The framework is rooted in the principles of computational fluid dynamics and filament dispersion theory, modeling wind flow and gas dispersion in 3D real-world scenarios (i.e., accounting for walls, furniture, etc.). Moreover, it integrates the simulation of different environmental sensors, such as metal oxide gas sensors, photo ionization detectors, or anemometers. We illustrate the potential and applicability of the proposed tool by presenting a simulation case in a complex and realistic office-like environment where gas leaks of different chemicals occur simultaneously. Furthermore, we accomplish quantitative and qualitative validation by comparing our simulated results against real-world data recorded inside a wind tunnel where methane was released under different wind flow profiles. Based on these results, we conclude that our simulation framework can provide a good approximation to real world measurements when advective airflows are present in the environment.

Experimental Validation of Depth Cameras for the Parameterization of Functional Balance of Patients in Clinical Tests.

In clinical practice, patients' balance can be assessed using standard scales. Two of the most validated clinical tests for measuring balance are the Timed Up and Go (TUG) test and the MultiDirectional Reach Test (MDRT). Nowadays, inertial sensors (IS) are employed for kinematic analysis of functional tests in the clinical setting, and have become an alternative to expensive, 3D optical motion capture systems. In daily clinical practice, however, IS-based setups are yet cumbersome and inconvenient to apply. Current depth cameras have the potential for such application, presenting many advantages as, for instance, being portable, low-cost and minimally-invasive. This paper aims at experimentally validating to what extent this technology can substitute IS for the parameterization and kinematic analysis of the TUG and the MDRT tests. Twenty healthy young adults were recruited as participants to perform five different balance tests while kinematic data from their movements were measured by both a depth camera and an inertial sensor placed on their trunk. The reliability of the camera's measurements is examined through the Interclass Correlation Coefficient (ICC), whilst the Pearson Correlation Coefficient (r) is computed to evaluate the correlation between both sensor's measurements, revealing excellent reliability and strong correlations in most cases.

Overcoming the slow recovery of MOX gas sensors through a system modeling approach.

Metal Oxide Semiconductor (MOX) gas transducers are one of the preferable technologies to build electronic noses because of their high sensitivity and low price. In this paper we present an approach to overcome to a certain extent one of their major disadvantages: their slow recovery time (tens of seconds), which limits their suitability to applications where the sensor is exposed to rapid changes of the gas concentration. Our proposal consists of exploiting a double first-order model of the MOX-based sensor from which a steady-state output is anticipated in real time given measurements of the transient state signal. This approach assumes that the nature of the volatile is known and requires a precalibration of the system time constants for each substance, an issue that is also described in the paper. The applicability of the proposed approach is validated with several experiments in real, uncontrolled scenarios with a mobile robot bearing an e-nose.

The Multi-Chamber Electronic Nose--an improved olfaction sensor for mobile robotics.

One of the major disadvantages of the use of Metal Oxide Semiconductor (MOS) technology as a transducer for electronic gas sensing devices (e-noses) is the long recovery period needed after each gas exposure. This severely restricts its usage in applications where the gas concentrations may change rapidly, as in mobile robotic olfaction, where allowing for sensor recovery forces the robot to move at a very low speed, almost incompatible with any practical robot operation. This paper describes the design of a new e-nose which overcomes, to a great extent, such a limitation. The proposed e-nose, called Multi-Chamber Electronic Nose (MCE-nose), comprises several identical sets of MOS sensors accommodated in separate chambers (four in our current prototype), which alternate between sensing and recovery states, providing, as a whole, a device capable of sensing changes in chemical concentrations faster. The utility and performance of the MCE-nose in mobile robotic olfaction is shown through several experiments involving rapid sensing of gas concentration and mobile robot gas mapping.