Real-Time Object Detection for Autonomous Solar Farm Inspection via UAVs.

Robotic missions for solar farm inspection demand agile and precise object detection strategies. This paper introduces an innovative keypoint-based object detection framework specifically designed for real-time solar farm inspections with UAVs. Moving away from conventional bounding box or segmentation methods, our technique focuses on detecting the vertices of solar panels, which provides a richer granularity than traditional approaches. Drawing inspiration from CenterNet, our architecture is optimized for embedded platforms like the NVIDIA AGX Jetson Orin, achieving close to 60 FPS at a resolution of 1024 ×1376 pixels, thus outperforming the camera's operational frequency. Such a real-time capability is essential for efficient robotic operations in time-critical industrial asset inspection environments. The design of our model emphasizes reduced computational demand, positioning it as a practical solution for real-world deployment. Additionally, the integration of active learning strategies promises a considerable reduction in annotation efforts and strengthens the model's operational feasibility. In summary, our research emphasizes the advantages of keypoint-based object detection, offering a practical and effective approach for real-time solar farm inspections with UAVs.

Thrust Vectoring Control for Heavy UAVs, Employing a Redundant Communication System.

Recently, various research studies have been developed to address communication sensors for Unmanned Aerial Systems (UASs). In particular, when pondering control difficulties, communication is a crucial component. To this end, strengthening a control algorithm with redundant linking sensors ensures the overall system works accurately, even if some components fail. This paper proposes a novel approach to integrate several sensors and actuators for a heavy Unmanned Aerial Vehicle (UAV). Additionally, a cutting-edge Robust Thrust Vectoring Control (RTVC) technique is designed to control various communicative modules during a flying mission and converge the attitude system to stability. The results of the study demonstrate that even though RTVC is not frequently utilized, it works as well as cascade PID controllers, particularly for multi-rotors with mounted flaps, and could be perfectly functional in UAVs powered by thermal engines to increase the autonomy since the propellers cannot be used as controller surfaces.

A Proposed System for Multi-UAVs in Remote Sensing Operations.

This paper proposes the design of the communications, control systems, and navigation algorithms of a multi-UAV system focused on remote sensing operations. A new controller based on a compensator and a nominal controller is designed to dynamically regulate the UAVs' attitude. The navigation system addresses the multi-region coverage trajectory planning task using a new approach to solve the TSP-CPP problem. The navigation algorithms were tested theoretically, and the combination of the proposed navigation techniques and control strategy was simulated through the Matlab SimScape platform to optimize the controller's parameters over several iterations. The results reveal the robustness of the controller and optimal performance of the route planner.

Performance Analysis of Localization Algorithms for Inspections in 2D and 3D Unstructured Environments Using 3D Laser Sensors and UAVs.

One of the most relevant problems related to Unmanned Aerial Vehicle's (UAV) autonomous navigation for industrial inspection is localization or pose estimation relative to significant elements of the environment. This paper analyzes two different approaches in this regard, focusing on its application to unstructured scenarios where objects of considerable size are present, such as a truck, a wind tower, an airplane, a building, etc. The presented methods require a previously developed Computer-Aided Design (CAD) model of the main object to be inspected. The first approach is based on an occupancy map built from a horizontal projection of this CAD model and the Adaptive Monte Carlo Localization (AMCL) algorithm to reach convergence by considering the likelihood field observation model between the 2D projection of 3D sensor data and the created map. The second approach uses a point cloud prior map of the 3D CAD model and a scan-matching algorithm based on the Iterative Closest Point Algorithm (ICP) and the Unscented Kalman Filter (UKF). The presented approaches have been extensively evaluated using simulated as well as previously recorded real flight data. We focus on aircraft inspection as a test example, but our results and conclusions can be directly extended to other applications. To support this assertion, a truck inspection has been performed. Our tests reflected that creating a 2D or 3D map from a standard CAD model and using a 3D laser scan on the created maps can optimize the processing time, resources and improve robustness. The techniques used to segment unexpected objects in 2D maps improved the performance of AMCL. In addition, we showed that moving around locations with relevant geometry after take-off when running AMCL enabled faster convergence and high accuracy. Hence, it could be used as an initial position estimation method for other localization algorithms. The ICP-NL method works well in environments with elements other than the object to inspect, but it can provide better results if some techniques to segment the new objects are applied. Furthermore, the proposed ICP-NL scan-matching method together with UKF performed faster, in a more robust manner, than NDT. Moreover, it is not affected by flight height. However, ICP-NL error may still be too high for applications requiring increased accuracy.

WILD HOPPER: A heavy-duty UAV for day and night firefighting operations.

Forest fires are among the most dangerous accidents, as they lead to the repercussions of climate change by reducing oxygen levels and increasing carbon dioxide levels. These risks led to the attention of many institutions worldwide, most notably the European Union and the European Parliament, which led to the emergence of many directives and regulations aimed at controlling the phenomenon of forest fires in Europe, such as the (E.U.) 2019/570. Among the proposed solutions, the usage of unmanned aerial vehicles (UAVs) is considered to operate alongside existing aircraft and helicopters through extinguishing forest fires. Scientific researches in this regard have shown the high effectiveness use of UAVs. Still, some defects and shortcomings appeared during practical experiments represented in the limited operating time and low payload. As UAVs are used for firefighting forest fires, they must be characterized by the heavy payload for the extinguishing fluids, long time for flight endurance during the mission, the ability to high maneuver, and work as a decision-making system. In this paper, a new UAV platform for forest firefighting is represented named WILD HOPPER. WILD HOPPER is a 600-liter platform designed for forest firefighting. This payload capacity overcomes typical limitations of electrically powered drones that cannot be used for anything more than fire monitoring, as they do not have sufficient lifting power. The enhanced capabilities of the WILD HOPPER allow it to complement existing aerial means and overcome their main limitations, especially the need to cover night operations. This allows reducing the duration of the wildfires heavily by allowing continuous aerial support to the extinguishing activities once the conventional aerial means (hydroplanes and helicopters) are set back to the base at night. On the other hand, WILD HOPPER has significant powerful advantages due to the accuracy of the release, derived from multirotor platform dynamic capabilities.

Fast Multi-UAV Path Planning for Optimal Area Coverage in Aerial Sensing Applications.

This paper deals with the problems and the solutions of fast coverage path planning (CPP) for multiple UAVs. Through this research, the problem is solved and analyzed with both a software framework and algorithm. The implemented algorithm generates a back-and-forth path based on the onboard sensor footprint. In addition, three methods are proposed for the individual path assignment: simple bin packing trajectory planner (SIMPLE-BINPAT); bin packing trajectory planner (BINPAT); and Powell optimized bin packing trajectory planner (POWELL-BINPAT). The three methods use heuristic algorithms, linear sum assignment, and minimization techniques to optimize the planning task. Furthermore, this approach is implemented with applicable software to be easily used by first responders such as police and firefighters. In addition, simulation and real-world experiments were performed using UAVs with RGB and thermal cameras. The results show that POWELL-BINPAT generates optimal UAV paths to complete the entire mission in minimum time. Furthermore, the computation time for the trajectory generation task decreases compared to other techniques in the literature. This research is part of a real project funded by the H2020 FASTER Project, with grant ID: 833507.

Zenithal isotropic object counting by localization using adversarial training.

Counting objects in images is a very time-consuming task for humans that yields to errors caused by repetitiveness and boredom. In this paper, we present a novel object counting method that, unlike most of the recent works that focus on the regression of a density map, performs the counting procedure by localizing each single object. This key difference allows us to provide not only an accurate count but the position of every counted object, information that can be critical in some areas such as precision agriculture. The method is designed in two steps: first, a CNN is in charge of mapping arbitrary objects to blob-like structures. Then, using a Laplacian of Gaussian (LoG) filter, we are able to gather the position of all detected objects. We also propose a semi-adversarial training procedure that, combined with the former design, improves the result by a large margin. After evaluating the method on two public benchmarks of isometric objects, we stay on par with the state of the art while being able to provide extra position information.

Altitude Measurement-Based Optimization of the Landing Process of UAVs.

The paper addresses the loop shaping problem in the altitude control of an unmanned aerial vehicle to land the flying robot with a specific landing scenario adopted. The proposed solution is optimal, in the sense of the selected performance indices, namely minimum-time, minimum-energy, and velocity-penalized related functions, achieving their minimal values, with numerous experiments conducted throughout the development and preparation to the Mohamed Bin Zayed International Robotics Challenge (MBZIRC 2020). A novel approach to generation of a reference altitude trajectory is presented, which is then tracked in a standard, though optimized, control loop. Three landing scenarios are considered, namely: minimum-time, minimum-energy, and velocity-penalized landing scenarios. The experimental results obtained with the use of the Simulink Support Package for Parrot Minidrones, and the OptiTrack motion capture system proved the effectiveness of the proposed approach.

Optimal Frontier-Based Autonomous Exploration in Unconstructed Environment Using RGB-D Sensor.

Aerial robots are widely used in search and rescue applications because of their small size and high maneuvering. However, designing an autonomous exploration algorithm is still a challenging and open task, because of the limited payload and computing resources on board UAVs. This paper presents an autonomous exploration algorithm for the aerial robots that shows several improvements for being used in the search and rescue tasks. First of all, an RGB-D sensor is used to receive information from the environment and the OctoMap divides the environment into obstacles, free and unknown spaces. Then, a clustering algorithm is used to filter the frontiers extracted from the OctoMap, and an information gain based cost function is applied to choose the optimal frontier. At last, the feasible path is given by A* path planner and a safe corridor generation algorithm. The proposed algorithm has been tested and compared with baseline algorithms in three different environments with the map resolutions of 0.2 m, and 0.3 m. The experimental results show that the proposed algorithm has a shorter exploration path and can save more exploration time when compared with the state of the art. The algorithm has also been validated in the real flight experiments.

Vision-Based Multirotor Following Using Synthetic Learning Techniques.

Deep- and reinforcement-learning techniques have increasingly required large sets of real data to achieve stable convergence and generalization, in the context of image-recognition, object-detection or motion-control strategies. On this subject, the research community lacks robust approaches to overcome unavailable real-world extensive data by means of realistic synthetic-information and domain-adaptation techniques. In this work, synthetic-learning strategies have been used for the vision-based autonomous following of a noncooperative multirotor. The complete maneuver was learned with synthetic images and high-dimensional low-level continuous robot states, with deep- and reinforcement-learning techniques for object detection and motion control, respectively. A novel motion-control strategy for object following is introduced where the camera gimbal movement is coupled with the multirotor motion during the multirotor following. Results confirm that our present framework can be used to deploy a vision-based task in real flight using synthetic data. It was extensively validated in both simulated and real-flight scenarios, providing proper results (following a multirotor up to 1.3 m/s in simulation and 0.3 m/s in real flights).

Vision-Based Steering Control, Speed Assistance and Localization for Inner-City Vehicles.

Autonomous route following with road vehicles has gained popularity in the last few decades. In order to provide highly automated driver assistance systems, different types and combinations of sensors have been presented in the literature. However, most of these approaches apply quite sophisticated and expensive sensors, and hence, the development of a cost-efficient solution still remains a challenging problem. This work proposes the use of a single monocular camera sensor for an automatic steering control, speed assistance for the driver and localization of the vehicle on a road. Herein, we assume that the vehicle is mainly traveling along a predefined path, such as in public transport. A computer vision approach is presented to detect a line painted on the road, which defines the path to follow. Visual markers with a special design painted on the road provide information to localize the vehicle and to assist in its speed control. Furthermore, a vision-based control system, which keeps the vehicle on the predefined path under inner-city speed constraints, is also presented. Real driving tests with a commercial car on a closed circuit finally prove the applicability of the derived approach. In these tests, the car reached a maximum speed of 48 km/h and successfully traveled a distance of 7 km without the intervention of a human driver and any interruption.

Towards an Autonomous Vision-Based Unmanned Aerial System against Wildlife Poachers.

Poaching is an illegal activity that remains out of control in many countries. Based on the 2014 report of the United Nations and Interpol, the illegal trade of global wildlife and natural resources amounts to nearly $ 213 billion every year, which is even helping to fund armed conflicts. Poaching activities around the world are further pushing many animal species on the brink of extinction. Unfortunately, the traditional methods to fight against poachers are not enough, hence the new demands for more efficient approaches. In this context, the use of new technologies on sensors and algorithms, as well as aerial platforms is crucial to face the high increase of poaching activities in the last few years. Our work is focused on the use of vision sensors on UAVs for the detection and tracking of animals and poachers, as well as the use of such sensors to control quadrotors during autonomous vehicle following and autonomous landing.

Automated Low-Cost Smartphone-Based Lateral Flow Saliva Test Reader for Drugs-of-Abuse Detection.

Lateral flow assay tests are nowadays becoming powerful, low-cost diagnostic tools. Obtaining a result is usually subject to visual interpretation of colored areas on the test by a human operator, introducing subjectivity and the possibility of errors in the extraction of the results. While automated test readers providing a result-consistent solution are widely available, they usually lack portability. In this paper, we present a smartphone-based automated reader for drug-of-abuse lateral flow assay tests, consisting of an inexpensive light box and a smartphone device. Test images captured with the smartphone camera are processed in the device using computer vision and machine learning techniques to perform automatic extraction of the results. A deep validation of the system has been carried out showing the high accuracy of the system. The proposed approach, applicable to any line-based or color-based lateral flow test in the market, effectively reduces the manufacturing costs of the reader and makes it portable and massively available while providing accurate, reliable results.

Real-time recognition of patient intentions from sequences of pressure maps using artificial neural networks.

OBJECTIVE: In this paper we address the problem of recognising the movement intentions of patients restricted to a medical bed. The developed recognition system will be used to implement a natural human-machine interface to move a medical bed by means of the slight movements of patients with reduced mobility. METHODS AND MATERIAL: Our proposal uses pressure map sequences as input and presents a novel system based on artificial neural networks to recognise the movement intentions. The system analyses each pressure map in real-time and classifies the raw information into output classes which represent these intentions. The complexity of the recognition problem is high because of the multiple body characteristics and distinct ways of communicating intentions. To address this problem, a complete processing chain was developed consisting of image processing algorithms, a knowledge extraction process, and a multilayer perceptron (MLP) classification model. RESULTS: Different configurations of the MLP have been investigated and quantitatively compared. The accuracy of our approach is high, obtaining an accuracy of 87%. The model was compared with five well-known classification paradigms. The performance of a reduced model, obtained by through feature selection algorithms, was found to be better and less time-consuming than the original model. The whole proposal has been validated with real patients in pre-clinical tests using the final medical bed prototype. CONCLUSIONS: The proposed approach produced very promising results, outperforming existing classification approaches. The excellent behaviour of the recognition system will enable its use in controlling the movements of the bed, in several degrees of freedom, by the patient with his/her own body.