Seamless Fusion: Multi-Modal Localization for First Responders in Challenging Environments.

In dynamic and unpredictable environments, the precise localization of first responders and rescuers is crucial for effective incident response. This paper introduces a novel approach leveraging three complementary localization modalities: visual-based, Galileo-based, and inertial-based. Each modality contributes uniquely to the final Fusion tool, facilitating seamless indoor and outdoor localization, offering a robust and accurate localization solution without reliance on pre-existing infrastructure, essential for maintaining responder safety and optimizing operational effectiveness. The visual-based localization method utilizes an RGB camera coupled with a modified implementation of the ORB-SLAM2 method, enabling operation with or without prior area scanning. The Galileo-based localization method employs a lightweight prototype equipped with a high-accuracy GNSS receiver board, tailored to meet the specific needs of first responders. The inertial-based localization method utilizes sensor fusion, primarily leveraging smartphone inertial measurement units, to predict and adjust first responders' positions incrementally, compensating for the GPS signal attenuation indoors. A comprehensive validation test involving various environmental conditions was carried out to demonstrate the efficacy of the proposed fused localization tool. Our results show that our proposed solution always provides a location regardless of the conditions (indoors, outdoors, etc.), with an overall mean error of 1.73 m.

UAV Path Optimization for Angle-Only Self-Localization and Target Tracking Based on the Bayesian Fisher Information Matrix.

In this paper, new path optimization algorithms are developed for uncrewed aerial vehicle (UAV) self-localization and target tracking, exploiting beacon (landmark) bearings and angle-of-arrival (AOA) measurements from a manoeuvring target. To account for time-varying rotations in the local UAV coordinates with respect to the global Cartesian coordinate system, the unknown orientation angle of the UAV is also estimated jointly with its location from the beacon bearings. This is critically important, as orientation errors can significantly degrade the self-localization performance. The joint self-localization and target tracking problem is formulated as a Kalman filtering problem with an augmented state vector that includes all the unknown parameters and a measurement vector of beacon bearings and target AOA measurements. This formulation encompasses applications where Global Navigation Satellite System (GNSS)-based self-localization is not available or reliable, and only beacons or landmarks can be utilized for UAV self-localization. An optimal UAV path is determined from the optimization of the Bayesian Fisher information matrix by means of A- and D-optimality criteria. The performance of this approach at different measurement noise levels is investigated. A modified closed-form projection algorithm based on a previous work is also proposed to achieve optimal UAV paths. The performance of the developed UAV path optimization algorithms is demonstrated with extensive simulation examples.

Deep Voxelized Feature Maps for Self-Localization in Autonomous Driving.

Lane-level self-localization is essential for autonomous driving. Point cloud maps are typically used for self-localization but are known to be redundant. Deep features produced by neural networks can be used as a map, but their simple utilization could lead to corruption in large environments. This paper proposes a practical map format using deep features. We propose voxelized deep feature maps for self-localization, consisting of deep features defined in small regions. The self-localization algorithm proposed in this paper considers per-voxel residual and reassignment of scan points in each optimization iteration, which could result in accurate results. Our experiments compared point cloud maps, feature maps, and the proposed map from the self-localization accuracy and efficiency perspective. As a result, more accurate and lane-level self-localization was achieved with the proposed voxelized deep feature map, even with a smaller storage requirement compared with the other map formats.

Improving Odometric Model Performance Based on LSTM Networks.

This paper presents a localization system for an autonomous wheelchair that includes several sensors, such as odometers, LIDARs, and an IMU. It focuses on improving the odometric localization accuracy using an LSTM neural network. Improved odometry will improve the result of the localization algorithm, obtaining a more accurate pose. The localization system is composed by a neural network designed to estimate the current pose using the odometric encoder information as input. The training is carried out by analyzing multiple random paths and defining the velodyne sensor data as training ground truth. During wheelchair navigation, the localization system retrains the network in real time to adjust any change or systematic error that occurs with respect to the initial conditions. Furthermore, another network manages to avoid certain random errors by using the relationship between the power consumed by the motors and the actual wheel speeds. The experimental results show several examples that demonstrate the ability to self-correct against variations over time, and to detect non-systematic errors in different situations using this relation. The final robot localization is improved with the designed odometric model compared to the classic robot localization based on sensor fusion using a static covariance.

A Minimalist Self-Localization Approach for Swarm Robots Based on Active Beacon in Indoor Environments.

When performing indoor tasks, miniature swarm robots are suffered from their small size, poor on-board computing power, and electromagnetic shielding of buildings, which means that some traditional localization methods, such as global positioning system (GPS), simultaneous localization and mapping (SLAM), and ultra-wideband (UWB), cannot be employed. In this paper, a minimalist indoor self-localization approach for swarm robots is proposed based on active optical beacons. A robotic navigator is introduced into a swarm of robots to provide locally localization services by actively projecting a customized optical beacon on the indoor ceiling, which contains the origin and the reference direction of localization coordinates. The swarm robots observe the optical beacon on the ceiling via a bottom-up-view monocular camera, and extract the beacon information on-board to localize their positions and headings. The uniqueness of this strategy is that it uses the flat, smooth, and well-reflective ceiling in the indoor environment as a ubiquitous plane for displaying the optical beacon; meanwhile, the bottom-up view of swarm robots is not easily blocked. Real robotic experiments are conducted to validate and analyze the localization performance of the proposed minimalist self-localization approach. The results show that our approach is feasible and effective, and can meet the needs of swarm robots to coordinate their motion. Specifically, for the stationary robots, the average position error and heading error are 2.41 cm and 1.44°; when the robots are moving, the average position error and heading error are less than 2.40 cm and 2.66°.

Reverberation-Robust Self-Calibration and Synchronization of Distributed Microphone Arrays by Mitigating Heteroscedasticity and Outlier Occurrence in TDoA Measurements.

The network of distributed microphone arrays is usually established in an ad hoc manner; hence, network parameters such as the mutual positioning and rotation of the arrays, positions of sources, and synchronization of their recording onset times are initially unknown. In this article, we consider the problem of passively jointly self-calibrating and synchronizing distributed arrays in reverberant rooms. We use a typical two-step approach where, initially, the relative geometry of the network is estimated using Direction of Arrival (DoA) measurements. Subsequently, the absolute scale and synchronization parameters are estimated using Time Difference of Arrival (TDoA) measurements. This article presents methods to improve the robustness and accuracy of estimation of the absolute geometric scaling and synchronization parameters in reverberant conditions, in which TDoA measurements do not follow a normal distribution; furthermore, outliers often occur. To remedy these issues, we propose a Weighted Least Squares (WLS) estimator and schema for weighting the TDoA measurements to increase the estimation accuracy from heteroscedastic TDoA measurements. In addition, we propose an iterative reweighing algorithm with a binary weight to detect and reject TDoA outliers, which exploits the residuals of the parametric model in the least absolute value minimization. A numerical evaluation shows significant improvements in the proposed method over the state of the art in terms of the relative scaling error and mean absolute value of the synchronization parameters.

A Self-Calibrating Localization Solution for Sport Applications with UWB Technology.

This study addressed the problem of localization in an ultrawide-band (UWB) network, where the positions of both the access points and the tags needed to be estimated. We considered a fully wireless UWB localization system, comprising both software and hardware, featuring easy plug-and-play usability for the consumer, primarily targeting sport and leisure applications. Anchor self-localization was addressed by two-way ranging, also embedding a Gauss-Newton algorithm for the estimation and compensation of antenna delays, and a modified isolation forest algorithm working with low-dimensional set of measurements for outlier identification and removal. This approach avoids time-consuming calibration procedures, and it enables accurate tag localization by the multilateration of time difference of arrival measurements. For the assessment of performance and the comparison of different algorithms, we considered an experimental campaign with data gathered by a proprietary UWB localization system.

Seamless Navigation, 3D Reconstruction, Thermographic and Semantic Mapping for Building Inspection.

We present a workflow for seamless real-time navigation and 3D thermal mapping in combined indoor and outdoor environments in a global reference frame. The automated workflow and partly real-time capabilities are of special interest for inspection tasks and also for other time-critical applications. We use a hand-held integrated positioning system (IPS), which is a real-time capable visual-aided inertial navigation technology, and augment it with an additional passive thermal infrared camera and global referencing capabilities. The global reference is realized through surveyed optical markers (AprilTags). Due to the sensor data's fusion of the stereo camera and the thermal images, the resulting georeferenced 3D point cloud is enriched with thermal intensity values. A challenging calibration approach is used to geometrically calibrate and pixel-co-register the trifocal camera system. By fusing the terrestrial dataset with additional geographic information from an unmanned aerial vehicle, we gain a complete building hull point cloud and automatically reconstruct a semantic 3D model. A single-family house with surroundings in the village of Morschenich near the city of Jülich (German federal state North Rhine-Westphalia) was used as a test site to demonstrate our workflow. The presented work is a step towards automated building information modeling.

Self-Localization of Mobile Robots Using a Single Catadioptric Camera with Line Feature Extraction.

This paper presents a novel self-localization technique for mobile robots using a central catadioptric camera. A unified sphere model for the image projection is derived by the catadioptric camera calibration. The geometric property of the camera projection model is utilized to obtain the intersections of the vertical lines and ground plane in the scene. Different from the conventional stereo vision techniques, the feature points are projected onto a known planar surface, and the plane equation is used for depth computation. The 3D coordinates of the base points on the ground are calculated using the consecutive image frames. The derivation of motion trajectory is then carried out based on the computation of rotation and translation between the robot positions. We develop an algorithm for feature correspondence matching based on the invariability of the structure in the 3D space. The experimental results obtained using the real scene images have demonstrated the feasibility of the proposed method for mobile robot localization applications.

Analysis of Occlusion Effects for Map-Based Self-Localization in Urban Areas.

A high-definition (HD) map provides structural information for map-based self-localization, enabling stable estimation in real environments. In urban areas, there are many obstacles, such as buses, that occlude sensor observations, resulting in self-localization errors. However, most of the existing HD map-based self-localization evaluations do not consider sudden significant errors due to obstacles. Instead, they evaluate this in terms of average error over estimated trajectories in an environment with few occlusions. This study evaluated the effects of self-localization estimation on occlusion with synthetically generated obstacles in a real environment. Various patterns of synthetic occlusion enabled the analyses of the effects of self-localization error from various angles. Our experiments showed various characteristics that locations susceptible to obstacles have. For example, we found that occlusion in intersections tends to increase self-localization errors. In addition, we analyzed the geometrical structures of a surrounding environment in high-level error cases and low-level error cases with occlusions. As a result, we suggested the concept that the real environment should have to achieve robust self-localization under occlusion conditions.

Clutter Suppression for Indoor Self-Localization Systems by Iteratively Reweighted Low-Rank Plus Sparse Recovery.

Self-localization based on passive RFID-based has many potential applications. One of the main challenges it faces is the suppression of the reflected signals from unwanted objects (i.e., clutter). Typically, the clutter echoes are much stronger than the backscattered signals of the passive tag landmarks used in such scenarios. Therefore, successful tag detection can be very challenging. We consider two types of tags, namely low-Q and high-Q tags. The high-Q tag features a sparse frequency response, whereas the low-Q tag presents a broad frequency response. Further, the clutter usually showcases a short-lived response. In this work, we propose an iterative algorithm based on a low-rank plus sparse recovery approach (RPCA) to mitigate clutter and retrieve the landmark response. In addition to that, we compare the proposed approach with the well-known time-gating technique. It turns out that RPCA outperforms significantly time-gating for low-Q tags, achieving clutter suppression and tag identification when clutter encroaches on the time-gating window span, whereas it also increases the backscattered power at resonance by approximately 12 dB at 80 cm for high-Q tags. Altogether, RPCA seems a promising approach to improve the identification of passive indoor self-localization tag landmarks.

MMW Radar-Based Technologies in Autonomous Driving: A Review.

With the rapid development of automated vehicles (AVs), more and more demands are proposed towards environmental perception. Among the commonly used sensors, MMW radar plays an important role due to its low cost, adaptability In different weather, and motion detection capability. Radar can provide different data types to satisfy requirements for various levels of autonomous driving. The objective of this study is to present an overview of the state-of-the-art radar-based technologies applied In AVs. Although several published research papers focus on MMW Radars for intelligent vehicles, no general survey on deep learning applied In radar data for autonomous vehicles exists. Therefore, we try to provide related survey In this paper. First, we introduce models and representations from millimeter-wave (MMW) radar data. Secondly, we present radar-based applications used on AVs. For low-level automated driving, radar data have been widely used In advanced driving-assistance systems (ADAS). For high-level automated driving, radar data is used In object detection, object tracking, motion prediction, and self-localization. Finally, we discuss the remaining challenges and future development direction of related studies.

Dynamic Reference Selection-Based Self-Localization Algorithm for Drifted Underwater Acoustic Networks.

Self-localization has become one of the major areas of research in drifted underwater acoustic networks (DUANs) since many applications are based on the knowledge of nodes' positions. However, self-localization for DUANs faces two main challenges: the insufficient anchors and the varying network topology. Both affect the localization performance seriously. In this paper, we focus on these two challenges and propose a dynamic reference selection-based self-localization algorithm for DUANs (DRSL) to improve the localization performance. First, an optimal reference selection scheme is presented to solve the insufficient anchors' problem. The selected optimal reference node can not only assist the insufficient anchors in accomplishing the localization procedure, but also obviously increase the localization accuracy. Based on the proposed optimal reference selection scheme, a dynamic reference selection-based self-localization algorithm is proposed to solve the topology changing problem. The proposed algorithm can improve the localization performance for DUANs significantly by selecting the reference node dynamically according to the predicted network topology, which is more suitable for DUANs with mobile sensor nodes. Simulation results show that the proposed DRSL algorithm can increase the localization accuracy greatly with insufficient anchor nodes and varying network topology. In addition, DRSL algorithm also has a lower communication cost than other anchor-free approaches, which distinctly demonstrates the advantages of the proposed DRSL algorithm.

Controlling the Flight of a Drone and Its Camera for 3D Reconstruction of Large Objects.

In this paper we present a simple stand-alone system performing the autonomous acquisition of multiple pictures all around large objects, i.e., objects that are too big to be photographed from any side just with a camera held by hand. In this approach, a camera carried by a drone (an off-the-shelf quadcopter) is employed to carry out the acquisition of an image sequence representing a valid dataset for the 3D reconstruction of the captured scene. Both the drone flight and the choice of the viewpoints for shooting a picture are automatically controlled by the developed application, which runs on a tablet wirelessly connected to the drone, and controls the entire process in real time. The system and the acquisition workflow have been conceived with the aim to keep the user intervention minimal and as simple as possible, requiring no particular skill to the user. The system has been experimentally tested on several subjects of different shapes and sizes, showing the ability to follow the requested trajectory with good robustness against any flight perturbations. The collected images are provided to a scene reconstruction software, which generates a 3D model of the acquired subject. The quality of the obtained reconstructions, in terms of accuracy and richness of details, have proved the reliability and efficacy of the proposed system.

Efficient Convex Optimization for Energy-Based Acoustic Sensor Self-Localization and Source Localization in Sensor Networks.

The energy reading has been an efficient and attractive measure for collaborative acoustic source localization in practical application due to its cost saving in both energy and computation capability. The maximum likelihood problems by fusing received acoustic energy readings transmitted from local sensors are derived. Aiming to efficiently solve the nonconvex objective of the optimization problem, we present an approximate estimator of the original problem. Then, a direct norm relaxation and semidefinite relaxation, respectively, are utilized to derive the second-order cone programming, semidefinite programming or mixture of them for both cases of sensor self-location and source localization. Furthermore, by taking the colored energy reading noise into account, several minimax optimization problems are formulated, which are also relaxed via the direct norm relaxation and semidefinite relaxation respectively into convex optimization problems. Performance comparison with the existing acoustic energy-based source localization methods is given, where the results show the validity of our proposed methods.

Design of an HF-Band RFID System with Multiple Readers and Passive Tags for Indoor Mobile Robot Self-Localization.

Radio frequency identification (RFID) technology has already been explored for efficient self-localization of indoor mobile robots. A mobile robot equipped with RFID readers detects passive RFID tags installed on the floor in order to locate itself. The Monte-Carlo localization (MCL) method enables the localization of a mobile robot equipped with an RFID system with reasonable accuracy, sufficient robustness and low computational cost. The arrangements of RFID readers and tags and the size of antennas are important design parameters for realizing accurate and robust self-localization using a low-cost RFID system. The design of a likelihood model of RFID tag detection is also crucial for the accurate self-localization. This paper presents a novel design and arrangement of RFID readers and tags for indoor mobile robot self-localization. First, by considering small-sized and large-sized antennas of an RFID reader, we show how the design of the likelihood model affects the accuracy of self-localization. We also design a novel likelihood model by taking into consideration the characteristics of the communication range of an RFID system with a large antenna. Second, we propose a novel arrangement of RFID tags with eight RFID readers, which results in the RFID system configuration requiring much fewer readers and tags while retaining reasonable accuracy of self-localization. We verify the performances of MCL-based self-localization realized using the high-frequency (HF)-band RFID system with eight RFID readers and a lower density of RFID tags installed on the floor based on MCL in simulated and real environments. The results of simulations and real environment experiments demonstrate that our proposed low-cost HF-band RFID system realizes accurate and robust self-localization of an indoor mobile robot.

Passive Sensor Integration for Vehicle Self-Localization in Urban Traffic Environment.

This research proposes an accurate vehicular positioning system which can achieve lane-level performance in urban canyons. Multiple passive sensors, which include Global Navigation Satellite System (GNSS) receivers, onboard cameras and inertial sensors, are integrated in the proposed system. As the main source for the localization, the GNSS technique suffers from Non-Line-Of-Sight (NLOS) propagation and multipath effects in urban canyons. This paper proposes to employ a novel GNSS positioning technique in the integration. The employed GNSS technique reduces the multipath and NLOS effects by using the 3D building map. In addition, the inertial sensor can describe the vehicle motion, but has a drift problem as time increases. This paper develops vision-based lane detection, which is firstly used for controlling the drift of the inertial sensor. Moreover, the lane keeping and changing behaviors are extracted from the lane detection function, and further reduce the lateral positioning error in the proposed localization system. We evaluate the integrated localization system in the challenging city urban scenario. The experiments demonstrate the proposed method has sub-meter accuracy with respect to mean positioning error.

Self-Localization of IoT Devices Using Noisy Anchor Positions and RSSI Measurements.

Location-enabled Internet of things (IoT) has attracted much attention from the scientific and industrial communities given its high relevance in application domains such as agriculture, wildlife management, and infectious disease control. The frequency and accuracy of location information plays an important role in the success of these applications. However, frequent and accurate self-localization of IoT devices is challenging due to their resource-constrained nature. In this paper, we propose a new algorithm for self-localization of IoT devices using noisy received signal strength indicator (RSSI) measurements and perturbed anchor node position estimates. In the proposed algorithm, we minimize a weighted sum-square-distance-error cost function in an iterative fashion utilizing the gradient-descent method. We calculate the weights using the statistical properties of the perturbations in the measurements. We assume log-normal distribution for the RSSI-induced distance estimates due to considering the log-distance path-loss model with normally-distributed perturbations for the RSSI measurements in the logarithmic scale. We also assume normally-distributed perturbation in the anchor position estimates. We compare the performance of the proposed algorithm with that of an existing algorithm that takes a similar approach but only accounts for the perturbations in the RSSI measurements. Our simulation results show that by taking into account the error in the anchor positions, a significant improvement in the localization accuracy can be achieved. The proposed algorithm uses only a single measurement of RSSI and one estimate of each anchor position. This makes the proposed algorithm suitable for frequent and accurate localization of IoT devices.

Fuzzy mobile-robot positioning in intelligent spaces using wireless sensor networks.

This work presents the development and experimental evaluation of a method based on fuzzy logic to locate mobile robots in an Intelligent Space using wireless sensor networks (WSNs). The problem consists of locating a mobile node using only inter-node range measurements, which are estimated by radio frequency signal strength attenuation. The sensor model of these measurements is very noisy and unreliable. The proposed method makes use of fuzzy logic for modeling and dealing with such uncertain information. Besides, the proposed approach is compared with a probabilistic technique showing that the fuzzy approach is able to handle highly uncertain situations that are difficult to manage by well-known localization methods.

Grid Cells Encode Local Positional Information.

The brain has an extraordinary ability to create an internal spatial map of the external world [1]. This map-like representation of environmental surroundings is encoded through specific types of neurons, located within the hippocampus and entorhinal cortex, which exhibit spatially tuned firing patterns [2, 3]. In addition to encoding space, these neurons are believed to be related to contextual information and memory [4-7]. One class of such cells is the grid cells, which are located within the entorhinal cortex, presubiculum, and parasubiculum [3, 8]. Grid cell firing forms a hexagonal array of firing fields, a pattern that is largely thought to reflect the operation of intrinsic self-motion-related computations [9-12]. If this is the case, then fields should be relatively uniform in size, number of spikes, and peak firing rate. However, it has been suggested that this is not in fact the case [3, 13]. The possibility exists that local spatial information also influences grid cells, which-if true-would greatly change the way in which grid cells are thought to contribute to place coding. Accordingly, we asked how discriminable the individual fields of a given grid cell are by looking at the distribution of field firing rates and reproducibility of this distribution across trials. Grid fields were less uniform in intensity than expected, and the pattern of strong and weak fields was spatially stable and recurred across trials. The distribution remained unchanged even after arena rescaling, but not after remapping. This suggests that additional local information is being overlaid onto the global hexagonal pattern of grid cells.