Real-Time Three-Dimensional Pedestrian Localization System Using Smartphones.

Robust and accurate three-dimensional localization is essential for personal navigation, emergency rescue, and worker monitoring in indoor environments. For localization technology to be employed in various applications, it is necessary to reduce infrastructure dependence and limit the maximum error bound. This study aims to accurately estimate the location of various people using smartphones in a building with a cloud platform-based localization system. The proposed technology is modularized in a hierarchical structure to sequentially estimate the floor and location. This system comprises four localization modules: course level detection, fine level detection (FLD), fine location tracking (FLT), and level change detection (LCD). Each module operates organically according to the current user status. The position estimation range is defined as a total of three phases, and an appropriate location estimation module suitable for the corresponding phase operates to estimate the user's location gradually and precisely. When the user's floor is determined by an FLD, the two-dimensional position of the user is estimated by an FLT module that tracks the user's position by comparing the received signal strength indicator vector sequence and radio map. Also, LCD recognizes the user's floor change and converts the user's phase. To verify the proposed technology, various experiments were conducted in a six-story building, and an average accuracy of less than 2 m was obtained.

Improving Indoor Pedestrian Dead Reckoning for Smartphones under Magnetic Interference Using Deep Learning.

As micro-electro-mechanical systems (MEMS) technology continues its rapid ascent, a growing array of smart devices are integrating lightweight, compact, and cost-efficient magnetometers and inertial sensors, paving the way for advanced human motion analysis. However, sensors housed within smartphones frequently grapple with the detrimental effects of magnetic interference on heading estimation, resulting in diminished accuracy. To counteract this challenge, this study introduces a method that synergistically employs convolutional neural networks (CNNs) and support vector machines (SVMs) for adept interference detection. Utilizing a CNN, we automatically extract profound features from single-step pedestrian motion data that are then channeled into an SVM for interference detection. Based on these insights, we formulate heading estimation strategies aptly suited for scenarios both devoid of and subjected to magnetic interference. Empirical assessments underscore our method's prowess, boasting an impressive interference detection accuracy of 99.38%. In indoor environments influenced by such magnetic disturbances, evaluations conducted along square and equilateral triangle trajectories revealed single-step heading absolute error averages of 2.1891° and 1.5805°, with positioning errors averaging 0.7565 m and 0.3856 m, respectively. These results lucidly attest to the robustness of our proposed approach in enhancing indoor pedestrian positioning accuracy in the face of magnetic interferences.

Step Length Is a More Reliable Measurement Than Walking Speed for Pedestrian Dead-Reckoning.

Pedestrian dead reckoning (PDR) relies on the estimation of the length of each step taken by the walker in a path from inertial data (e.g. as recorded by a smartphone). Existing algorithms either estimate step lengths directly, or predict walking speed, which can then be integrated over a step period to obtain step length. We present an analysis, using a common architecture formed by an LSTM followed by four fully connected layers, of the quality of reconstruction when predicting step length vs. walking speed. Our experiments, conducted on a data set collected by twelve participants, strongly suggest that step length can be predicted more reliably than average walking speed over each step.

3D Visual Reconstruction as Prior Information for First Responder Localization and Visualization.

In professional use cases like police or fire brigade missions, coordinated and systematic force management is crucial for achieving operational success during intervention by the emergency personnel. A real-time situation picture enhances the coordination of the team. This situation picture includes not only an overview of the environment but also the positions, i.e., localization, of the emergency forces. The overview of the environment can be obtained either from known situation pictures like floorplans or by scanning the environment with the aid of visual sensors. The self-localization problem can be solved outdoors using the Global Navigation Satellite System (GNSS), but it is not fully solved indoors, where the GNSS signal might not be received or might be degraded. In this paper, we propose a novel combination of an inertial localization technique based on simultaneous localization and mapping (SLAM) with 3D building scans, which are used as prior information, for geo-referencing the positions, obtaining a situation picture, and finally visualizing the results with an appropriate visualization tool. We developed a new method for converting point clouds into a hexagonal prism map specifically designed for our SLAM algorithm. With this combination, we could keep the equipment for first responders as lightweight as required. We showed that the positioning led to an average accuracy of less than 1m indoors, and the final visualization including the building layout obtained by the 3D building reconstruction will be advantageous for coordinating first responder operations.

Accurate Stride-Length Estimation Based on LT-StrideNet for Pedestrian Dead Reckoning Using a Shank-Mounted Sensor.

Pedestrian dead reckoning (PDR) is a self-contained positioning technology and has been a significant research topic in recent years. Pedestrian-stride-length estimation is the core part of the PDR system and directly affects the performance of the PDR. The current stride-length-estimation method is difficult to adapt to changes in pedestrian walking speed, which leads to a rapid increase in the error of the PDR. In this paper, a new deep-learning model based on long short-term memory (LSTM) and Transformer, LT-StrideNet, is proposed to estimate pedestrian-stride length. Next, a shank-mounted PDR framework is built based on the proposed stride-length-estimation method. In the PDR framework, the detection of pedestrian stride is achieved by peak detection with a dynamic threshold. An extended Kalman filter (EKF) model is adopted to fuse the gyroscope, accelerometer, and magnetometer. The experimental results show that the proposed stride-length-estimation method can effectively adapt to changes in pedestrian walking speed, and our PDR framework has excellent positioning performance.

RadarPDR: Radar-Assisted Indoor Pedestrian Dead Reckoning.

Pedestrian dead reckoning (PDR) is the critical component in indoor pedestrian tracking and navigation services. While most of the recent PDR solutions exploit in-built inertial sensors in smartphones for next step estimation, due to measurement errors and sensing drift, the accuracy of walking direction, step detection, and step length estimation cannot be guaranteed, leading to large accumulative tracking errors. In this paper, we propose a radar-assisted PDR scheme, called RadarPDR, which integrates a frequency-modulation continuous-wave (FMCW) radar to assist the inertial sensors-based PDR. We first establish a segmented wall distance calibration model to deal with the radar ranging noise caused by irregular indoor building layouts and fuse wall distance estimation with acceleration and azimuth signals measured by the inertial sensors of a smartphone. We also propose a hierarchical particle filter(PF) together with an extended Kalman filter for position and trajectory adjustment. Experiments have been conducted in practical indoor scenarios. Results demonstrate that the proposed RadarPDR is efficient and stable and outperforms the widely used inertial sensors-based PDR scheme.

Smartphone-Based Localization for Passengers Commuting in Traffic Hubs.

Passengers commute between different modes of transportation in traffic hubs, and passenger localization is a key component for the effective functioning of these spaces. The smartphone-based localization system presented in this work is based on the 3D step and heading approach, which is adapted depending on the position of the smartphone, i.e., held in the hand or in the front pocket of the trousers. We use the accelerometer, gyroscope and barometer embedded in the smartphone to detect the steps and the direction of movement of the passenger. To correct the accumulated error, we detect landmarks, particularly staircases and elevators. To test our localization algorithm, we have recorded real-world mobility data in a test station in Munich city center where we have ground truth points. We achieve a 3D position accuracy of 12 m for a smartphone held in the hand and 10 m when the phone is placed in the front pocket of the trousers.

Practical and Accurate Indoor Localization System Using Deep Learning.

Indoor localization is an important technology for providing various location-based services to smartphones. Among the various indoor localization technologies, pedestrian dead reckoning using inertial measurement units is a simple and highly practical solution for indoor localization. In this study, we propose a smartphone-based indoor localization system using pedestrian dead reckoning. To create a deep learning model for estimating the moving speed, accelerometer data and GPS values were used as input data and data labels, respectively. This is a practical solution compared with conventional indoor localization mechanisms using deep learning. We improved the positioning accuracy via data preprocessing, data augmentation, deep learning modeling, and correction of heading direction. In a horseshoe-shaped indoor building of 240 m in length, the experimental results show a distance error of approximately 3 to 5 m.

Intelligent lecture recording system based on coordination of face-detection and pedestrian dead reckoning.

Automatic lecture recording is an appealing alternative approach to manually recording lectures in the process of online course making as it can to a large extent save labor cost. The key of the automatic recording system is lecturer tracking, and the existing automatic tracking methods tend to lose the target in the case of lecturer's rapid movement. This article proposes a lecturer tracking system based on MobileNet-SSD face detection and Pedestrian Dead Reckoning (PDR) technology to solve this problem. First, the particle filter algorithm is used to fuse the PDR information with the rotation angle information of the Pan-Tilt camera, which can improve the accuracy of detection under the tracking process. In addition, to improve face detection performance on the edge side, we utilize the OpenVINO toolkit to optimize the inference speed of the Convolutional Neural Networks (CNNs) before deploying the model. Further, when the lecturer is beyond the camera's field of view, the PDR auxiliary module is enabled to capture the object automatically. We built the entire lecture recording system from scratch and performed the experiments in the real lectures. The experimental results show that our system outperforms the systems without a PDR module in terms of the accuracy and robustness.

Indoor Spatiotemporal Contact Analytics Using Landmark-Aided Pedestrian Dead Reckoning on Smartphones.

Due to the prevalence of COVID-19, providing safe environments and reducing the risks of virus exposure play pivotal roles in our daily lives. Contact tracing is a well-established and widely-used approach to track and suppress the spread of viruses. Most digital contact tracing systems can detect direct face-to-face contact based on estimated proximity, without quantifying the exposed virus concentration. In particular, they rarely allow for quantitative analysis of indirect environmental exposure due to virus survival time in the air and constant airborne transmission. In this work, we propose an indoor spatiotemporal contact awareness framework (iSTCA), which explicitly considers the self-containing quantitative contact analytics approach with spatiotemporal information to provide accurate awareness of the virus quanta concentration in different origins at various times. Smartphone-based pedestrian dead reckoning (PDR) is employed to precisely detect the locations and trajectories for distance estimation and time assessment without the need to deploy extra infrastructure. The PDR technique we employ calibrates the accumulative error by identifying spatial landmarks automatically. We utilized a custom deep learning model composed of bidirectional long short-term memory (Bi-LSTM) and multi-head convolutional neural networks (CNNs) for extracting the local correlation and long-term dependency to recognize landmarks. By considering the spatial distance and time difference in an integrated manner, we can quantify the virus quanta concentration of the entire indoor environment at any time with all contributed virus particles. We conducted an extensive experiment based on practical scenarios to evaluate the performance of the proposed system, showing that the average positioning error is reduced to less than 0.7 m with high confidence and demonstrating the validity of our system for the virus quanta concentration quantification involving virus movement in a complex indoor environment.

An Enhanced Pedestrian Visual-Inertial SLAM System Aided with Vanishing Point in Indoor Environments.

The visual-inertial simultaneous localization and mapping (SLAM) is a feasible indoor positioning system that combines the visual SLAM with inertial navigation. There are accumulated drift errors in inertial navigation due to the state propagation and the bias of the inertial measurement unit (IMU) sensor. The visual-inertial SLAM can correct the drift errors via loop detection and local pose optimization. However, if the trajectory is not a closed loop, the drift error might not be significantly reduced. This paper presents a novel pedestrian dead reckoning (PDR)-aided visual-inertial SLAM, taking advantage of the enhanced vanishing point (VP) observation. The VP is integrated into the visual-inertial SLAM as an external observation without drift error to correct the system drift error. Additionally, the estimated trajectory's scale is affected by the IMU measurement errors in visual-inertial SLAM. Pedestrian dead reckoning (PDR) velocity is employed to constrain the double integration result of acceleration measurement from the IMU. Furthermore, to enhance the proposed system's robustness and the positioning accuracy, the local optimization based on the sliding window and the global optimization based on the segmentation window are proposed. A series of experiments are conducted using the public ADVIO dataset and a self-collected dataset to compare the proposed system with the visual-inertial SLAM. Finally, the results demonstrate that the proposed optimization method can effectively correct the accumulated drift error in the proposed visual-inertial SLAM system.

Sensor-Fusion for Smartphone Location Tracking Using Hybrid Multimodal Deep Neural Networks.

Many engineered approaches have been proposed over the years for solving the hard problem of performing indoor localization using smartphone sensors. However, specialising these solutions for difficult edge cases remains challenging. Here we propose an end-to-end hybrid multimodal deep neural network localization system, MM-Loc, relying on zero hand-engineered features, but learning automatically from data instead. This is achieved by using modality-specific neural networks to extract preliminary features from each sensing modality, which are then combined by cross-modality neural structures. We show that our choice of modality-specific neural architectures can estimate the location independently. But for better accuracy, a multimodal neural network that fuses the features of early modality-specific representations is a better proposition. Our proposed MM-Loc system is tested on cross-modality samples characterised by different sampling rate and data representation (inertial sensors, magnetic and WiFi signals), outperforming traditional approaches for location estimation. MM-Loc elegantly trains directly from data unlike conventional indoor positioning systems, which rely on human intuition.

Multi-Floor Indoor Pedestrian Dead Reckoning with a Backtracking Particle Filter and Viterbi-Based Floor Number Detection.

We present a smartphone-based indoor localisation system, able to track pedestrians over multiple floors. The system uses Pedestrian Dead Reckoning (PDR), which exploits data from the smartphone's inertial measurement unit to estimate the trajectory. The PDR output is matched to a scaled floor plan and fused with model-based WiFi received signal strength fingerprinting by a Backtracking Particle Filter (BPF). We proposed a new Viterbi-based floor detection algorithm, which fuses data from the smartphone's accelerometer, barometer and WiFi RSS measurements to detect stairs and elevator usage and to estimate the correct floor number. We also proposed a clustering algorithm on top of the BPF to solve multimodality, a known problem with particle filters. The proposed system relies on only a few pre-existing access points, whereas most systems assume or require the presence of a dedicated localisation infrastructure. In most public buildings and offices, access points are often available at smaller densities than used for localisation. Our system was extensively tested in a real office environment with seven 41 m × 27 m floors, each of which had two WiFi access points. Our system was evaluated in real-time and batch mode, since the system was able to correct past states. The clustering algorithm reduced the median position error by 17% in real-time and 13% in batch mode, while the floor detection algorithm achieved a 99.1% and 99.7% floor number accuracy in real-time and batch mode, respectively.

A Pilot Study to Validate a Wearable Inertial Sensor for Gait Assessment in Older Adults with Falls.

The high prevalence of falls and the enormous impact they have on the elderly population is a cause for concern. We aimed to develop a walking-monitor gait pattern (G-STRIDE) for older adults based on a 6-axis inertial measurement (IMU) with the application of pedestrian dead reckoning algorithms and tested its structural and clinical validity. A cross-sectional case-control study was conducted with 21 participants (11 fallers and 10 non-fallers). We measured gait using an IMU attached to the foot while participants walked around different grounds (indoor flooring, outdoor floor, asphalt, etc.). The G-STRIDE consisted of a portable inertial device that monitored the gait pattern and a mobile app for telematic clinical analysis. G-STRIDE made it possible to measure gait parameters under normal living conditions when walking without assessing the patient in the outpatient clinic. Moreover, we verified concurrent validity with convectional outcome measures using intraclass correlation coefficients (ICCs) and analyzed the differences between participants. G-STRIDE showed high estimation accuracy for the walking speed of the elderly and good concurrent validity compared to conventional measures (ICC = 0.69; p < 0.000). In conclusion, the developed inertial-based G-STRIDE can accurately classify older people with risk to fall with a significance as high as using traditional but more subjective clinical methods (gait speed, Timed Up and Go Test).

Real-Time Pedestrian Tracking Terminal Based on Adaptive Zero Velocity Update.

In the field of pedestrian dead reckoning (PDR), the zero velocity update (ZUPT) method with an inertial measurement unit (IMU) is a mature technology to calibrate dead reckoning. However, due to the complex walking modes of different individuals, it is essential and challenging to determine the ZUPT conditions, which has a direct and significant influence on the tracking accuracy. In this research, we adopted an adaptive zero velocity update (AZUPT) method based on convolution neural networks to classify the ZUPT conditions. The AZUPT model was robust regardless of the different motion types of various individuals. AZUPT was then implemented on the Zynq-7000 SoC platform to work in real time to validate its computational efficiency and performance superiority. Extensive real-world experiments were conducted by 60 different individuals in three different scenarios. It was demonstrated that the proposed system could work equally well in different environments, making it portable for PDR to be widely performed in various real-world situations.

Chest-Worn Inertial Sensors: A Survey of Applications and Methods.

Inertial Measurement Units (IMUs) are frequently implemented in wearable devices. Thanks to advances in signal processing and machine learning, applications of IMUs are not limited to those explicitly addressing body movements such as Activity Recognition (AR). On the other hand, wearing IMUs on the chest offers a few advantages over other body positions. AR and posture analysis, cardiopulmonary parameters estimation, voice and swallowing activity detection and other measurements can be approached through chest-worn inertial sensors. This survey tries to introduce the applications that come with the chest-worn IMUs and summarizes the existing methods, current challenges and future directions associated with them. In this regard, this paper references a total number of 57 relevant studies from the last 10 years and categorizes them into seven application areas. We discuss the inertial sensors used as well as their placement on the body and their associated validation methods based on the application categories. Our investigations show meaningful correlations among the studies within the same application categories. Then, we investigate the data processing architectures of the studies from the hardware point of view, indicating a lack of effort on handling the main processing through on-body units. Finally, we propose combining the discussed applications in a single platform, finding robust ways for artifact cancellation, and planning optimized sensing/processing architectures for them, to be taken more seriously in future research.

Enhancing Performance of Magnetic Field Based Indoor Localization Using Magnetic Patterns from Multiple Smartphones.

Wide expansion of smartphones triggered a rapid demand for precise localization that can meet the requirements of location-based services. Although the global positioning system is widely used for outdoor positioning, it cannot provide the same accuracy for the indoor. As a result, many alternative indoor positioning technologies like Wi-Fi, Bluetooth Low Energy (BLE), and geomagnetic field localization have been investigated during the last few years. Today smartphones possess a rich variety of embedded sensors like accelerometer, gyroscope, and magnetometer that can facilitate estimating the current location of the user. Traditional geomagnetic field-based fingerprint localization, although it shows promising results, it is limited by the fact that various smartphones have embedded magnetic sensors from different manufacturers and the magnetic field strength that is measured from these smartphones vary significantly. Consequently, the localization performance from various smartphones is different even when the same localization approach is used. So devising an approach that can provide similar performance with various smartphones is a big challenge. Contrary to previous works that build the fingerprint database from the geomagnetic field data of a single smartphone, this study proposes using the geomagnetic field data collected from multiple smartphones to make the geomagnetic field pattern (MP) database. Many experiments are carried out to analyze the performance of the proposed approach with various smartphones. Additionally, a lightweight threshold technique is proposed that can detect user motion using the acceleration data. Results demonstrate that the localization performance for four different smartphones is almost identical when tested with the database made using the magnetic field data from multiple smartphones than that of which considers the magnetic field data from only one smartphone. Moreover, the performance comparison with previous research indicates that the overall performance of smartphones is improved.

Indoor Radio Map Construction Based on Position Adjustment and Equipment Calibration.

The crowdsourcing-based wireless local area network (WLAN) indoor localization system has been widely promoted for the effective reduction of the workload from the offline phase data collection while constructing radio maps. Aiming at the problem of the diverse terminal devices and the inaccurate location annotation of the crowdsourced samples, which will result in the construction of the wrong radio map, an effective indoor radio map construction scheme (RMPAEC) is proposed based on position adjustment and equipment calibration. The RMPAEC consists of three main modules: terminal equipment calibration, pedestrian dead reckoning (PDR) estimated position adjustment, and fingerprint amendment. A position adjustment algorithm based on selective particle filtering is used by RMPAEC to reduce the cumulative error in PDR tracking. Moreover, an inter-device calibration algorithm is put forward based on receiver pattern analysis to obtain a device-independent grid fingerprint. The experimental results demonstrate that the proposed solution achieves higher localization accuracy than the peer schemes, and it possesses good effectiveness at the same time.

Improving the Heading Accuracy in Indoor Pedestrian Navigation Based on a Decision Tree and Kalman Filter.

In pedestrian inertial navigation, multi-sensor fusion is often used to obtain accurate heading estimates. As a widely distributed signal source, the geomagnetic field is convenient to provide sufficiently accurate heading angles. Unfortunately, there is a broad presence of artificial magnetic perturbations in indoor environments, leading to difficulties in geomagnetic correction. In this paper, by analyzing the spatial distribution model of the magnetic interference field on the geomagnetic field, two quantitative features have been found to be crucial in distinguishing normal magnetic data from anomalies. By leveraging these two features and the classification and regression tree (CART) algorithm, we trained a decision tree that is capable of extracting magnetic data from distorted measurements. Furthermore, this well-trained decision tree can be used as a reject gate in a Kalman filter. By combining the decision tree and Kalman filter, a high-precision indoor pedestrian navigation system based on a magnetically assisted inertial system is proposed. This system is then validated in a real indoor environment, and the results show that our system delivers state-of-the-art positioning performance. Compared to other baseline algorithms, an improvement of over 70% in the positioning accuracy is achieved.

Indoor Trajectory Reconstruction of Walking, Jogging, and Running Activities Based on a Foot-Mounted Inertial Pedestrian Dead-Reckoning System.

The evaluation of trajectory reconstruction of the human body obtained by foot-mounted Inertial Pedestrian Dead-Reckoning (IPDR) methods has usually been carried out in controlled environments, with very few participants and limited to walking. In this study, a pipeline for trajectory reconstruction using a foot-mounted IPDR system is proposed and evaluated in two large datasets containing activities that involve walking, jogging, and running, as well as movements such as side and backward strides, sitting, and standing. First, stride segmentation is addressed using a multi-subsequence Dynamic Time Warping method. Then, detection of Toe-Off and Mid-Stance is performed by using two new algorithms. Finally, stride length and orientation estimation are performed using a Zero Velocity Update algorithm empowered by a complementary Kalman filter. As a result, the Toe-Off detection algorithm reached an F-score between 90% and 100% for activities that do not involve stopping, and between 71% and 78% otherwise. Resulting return position errors were in the range of 0.5% to 8.8% for non-stopping activities and 8.8% to 27.4% otherwise. The proposed pipeline is able to reconstruct indoor trajectories of people performing activities that involve walking, jogging, running, side and backward walking, sitting, and standing.