OpenSense: An open-source toolbox for inertial-measurement-unit-based measurement of lower extremity kinematics over long durations.

BACKGROUND: The ability to measure joint kinematics in natural environments over long durations using inertial measurement units (IMUs) could enable at-home monitoring and personalized treatment of neurological and musculoskeletal disorders. However, drift, or the accumulation of error over time, inhibits the accurate measurement of movement over long durations. We sought to develop an open-source workflow to estimate lower extremity joint kinematics from IMU data that was accurate and capable of assessing and mitigating drift. METHODS: We computed IMU-based estimates of kinematics using sensor fusion and an inverse kinematics approach with a constrained biomechanical model. We measured kinematics for 11 subjects as they performed two 10-min trials: walking and a repeated sequence of varied lower-extremity movements. To validate the approach, we compared the joint angles computed with IMU orientations to the joint angles computed from optical motion capture using root mean square (RMS) difference and Pearson correlations, and estimated drift using a linear regression on each subject's RMS differences over time. RESULTS: IMU-based kinematic estimates agreed with optical motion capture; median RMS differences over all subjects and all minutes were between 3 and 6 degrees for all joint angles except hip rotation and correlation coefficients were moderate to strong (r = 0.60-0.87). We observed minimal drift in the RMS differences over 10 min; the average slopes of the linear fits to these data were near zero (- 0.14-0.17 deg/min). CONCLUSIONS: Our workflow produced joint kinematics consistent with those estimated by optical motion capture, and could mitigate kinematic drift even in the trials of continuous walking without rest, which may obviate the need for explicit sensor recalibration (e.g. sitting or standing still for a few seconds or zero-velocity updates) used in current drift-mitigation approaches when studying similar activities. This could enable long-duration measurements, bringing the field one step closer to estimating kinematics in natural environments.

Measuring Roadway Lane Widths Using Connected Vehicle Sensor Data.

The United States has over three trillion vehicle miles of travel annually on over four million miles of public roadways, which require regular maintenance. To maintain and improve these facilities, agencies often temporarily close lanes, reconfigure lane geometry, or completely close the road depending on the scope of the construction project. Lane widths of less than 11 feet in construction zones can impact highway capacity and crash rates. Crash data can be used to identify locations where the road geometry could be improved. However, this is a manual process that does not scale well. This paper describes findings for using data from onboard sensors in production vehicles for measuring lane widths. Over 200 miles of roadway on US-52, US-41, and I-65 in Indiana were measured using vehicle sensor data and compared with mobile LiDAR point clouds as ground truth and had a root mean square error of approximately 0.24 feet. The novelty of these results is that vehicle sensors can identify when work zones use lane widths substantially narrower than the 11 foot standard at a network level and can be used to aid in the inspection and verification of construction specification conformity. This information would contribute to the construction inspection performed by agencies in a safer, more efficient way.

Weighted Iterative CD-Spline for Mitigating Occlusion Effects on Building Boundary Regularization Using Airborne LiDAR Data.

Building occlusions usually decreases the accuracy of boundary regularization. Thus, it is essential that modeling methods address this problem, aiming to minimize its effects. In this context, we propose a weighted iterative changeable degree spline (WICDS) approach. The idea is to use a weight function for initial building boundary points, assigning a lower weight to the points in the occlusion region. As a contribution, the proposed method allows the minimization of errors caused by the occlusions, resulting in a more accurate contour modeling. The conducted experiments are performed using both simulated and real data. In general, the results indicate the potential of the WICDS approach to model a building boundary with occlusions, including curved boundary segments. In terms of Fscore and PoLiS, the proposed approach presents values around 99% and 0.19 m, respectively. Compared with the previous iterative changeable degree spline (ICDS), the WICDS resulted in an improvement of around 6.5% for completeness, 4% for Fscore, and 0.24 m for the PoLiS metric.

Assessing Vehicle Profiling Accuracy of Handheld LiDAR Compared to Terrestrial Laser Scanning for Crash Scene Reconstruction.

Forensic crash investigation often requires developing detailed profiles showing the location and extent of vehicle damage to identify impact areas, impact direction, deformation, and estimated vehicle speeds at impact. Traditional damage profiling techniques require extended and comprehensive setups for mapping and measurement that are quite labor- and time-intensive. Due to the time involved, this damage profiling is usually done in a remote holding area after the crash scene is cleared. Light detection and ranging (LiDAR) scanning technology in consumer handheld electronic devices, such as smartphones and tablets, holds significant potential for conducting this damage profile mapping in just a few minutes, allowing the mapping to be conducted at the scene before the vehicle(s) are moved. However, there is limited research and even scarcer published literature on field procedures and/or accuracy for these emerging smartphones and tablets with LiDAR. This paper proposes a methodology and subsequent measurement accuracy comparisons for survey-grade terrestrial laser scanning (TLS) and handheld alternatives. The maximum root mean square error (RMSE) obtained for profile distance between handheld (iPad) and survey-grade TLS LiDAR scans for a damaged vehicle was observed to be 3 cm, a level of accuracy that is likely sufficient and acceptable for most forensic studies.

Processing Strategy and Comparative Performance of Different Mobile LiDAR System Grades for Bridge Monitoring: A Case Study.

Collecting precise as-built data is essential for tracking construction progress. Three-dimensional models generated from such data capture the as-is conditions of the structures, providing valuable information for monitoring existing infrastructure over time. As-built data can be acquired using a wide range of remote sensing technologies, among which mobile LiDAR is gaining increasing attention due to its ability to collect high-resolution data over a relatively large area in a short time. The quality of mobile LiDAR data depends not only on the grade of onboard LiDAR scanners but also on the accuracy of direct georeferencing information and system calibration. Consequently, millimeter-level accuracy is difficult to achieve. In this study, the performance of mapping-grade and surveying-grade mobile LiDAR systems for bridge monitoring is evaluated against static laser scanners. Field surveys were conducted over a concrete bridge where grinding was required to achieve desired smoothness. A semi-automated, feature-based fine registration strategy is proposed to compensate for the impact of georeferencing and system calibration errors on mobile LiDAR data. Bridge deck thickness is evaluated using surface segments to minimize the impact of inherent noise in the point cloud. The results show that the two grades of mobile LiDAR delivered thickness estimates that are in agreement with those derived from static laser scanning in the 1 cm range. The mobile LiDAR data acquisition took roughly five minutes without having a significant impact on traffic, while the static laser scanning required more than three hours.

OpenSim: Simulating musculoskeletal dynamics and neuromuscular control to study human and animal movement.

Movement is fundamental to human and animal life, emerging through interaction of complex neural, muscular, and skeletal systems. Study of movement draws from and contributes to diverse fields, including biology, neuroscience, mechanics, and robotics. OpenSim unites methods from these fields to create fast and accurate simulations of movement, enabling two fundamental tasks. First, the software can calculate variables that are difficult to measure experimentally, such as the forces generated by muscles and the stretch and recoil of tendons during movement. Second, OpenSim can predict novel movements from models of motor control, such as kinematic adaptations of human gait during loaded or inclined walking. Changes in musculoskeletal dynamics following surgery or due to human-device interaction can also be simulated; these simulations have played a vital role in several applications, including the design of implantable mechanical devices to improve human grasping in individuals with paralysis. OpenSim is an extensible and user-friendly software package built on decades of knowledge about computational modeling and simulation of biomechanical systems. OpenSim's design enables computational scientists to create new state-of-the-art software tools and empowers others to use these tools in research and clinical applications. OpenSim supports a large and growing community of biomechanics and rehabilitation researchers, facilitating exchange of models and simulations for reproducing and extending discoveries. Examples, tutorials, documentation, and an active user forum support this community. The OpenSim software is covered by the Apache License 2.0, which permits its use for any purpose including both nonprofit and commercial applications. The source code is freely and anonymously accessible on GitHub, where the community is welcomed to make contributions. Platform-specific installers of OpenSim include a GUI and are available on simtk.org.

Time Series UAV Image-Based Point Clouds for Landslide Progression Evaluation Applications.

Landslides are major and constantly changing threats to urban landscapes and infrastructure. It is essential to detect and capture landslide changes regularly. Traditional methods for monitoring landslides are time-consuming, costly, dangerous, and the quality and quantity of the data is sometimes unable to meet the necessary requirements of geotechnical projects. This motivates the development of more automatic and efficient remote sensing approaches for landslide progression evaluation. Automatic change detection involving low-altitude unmanned aerial vehicle image-based point clouds, although proven, is relatively unexplored, and little research has been done in terms of accounting for volumetric changes. In this study, a methodology for automatically deriving change displacement rates, in a horizontal direction based on comparisons between extracted landslide scarps from multiple time periods, has been developed. Compared with the iterative closest projected point (ICPP) registration method, the developed method takes full advantage of automated geometric measuring, leading to fast processing. The proposed approach easily processes a large number of images from different epochs and enables the creation of registered image-based point clouds without the use of extensive ground control point information or further processing such as interpretation and image correlation. The produced results are promising for use in the field of landslide research.

Segmentation of Planar Surfaces from Laser Scanning Data Using the Magnitude of Normal Position Vector for Adaptive Neighborhoods.

Diverse approaches to laser point segmentation have been proposed since the emergence of the laser scanning system. Most of these segmentation techniques, however, suffer from limitations such as sensitivity to the choice of seed points, lack of consideration of the spatial relationships among points, and inefficient performance. In an effort to overcome these drawbacks, this paper proposes a segmentation methodology that: (1) reduces the dimensions of the attribute space; (2) considers the attribute similarity and the proximity of the laser point simultaneously; and (3) works well with both airborne and terrestrial laser scanning data. A neighborhood definition based on the shape of the surface increases the homogeneity of the laser point attributes. The magnitude of the normal position vector is used as an attribute for reducing the dimension of the accumulator array. The experimental results demonstrate, through both qualitative and quantitative evaluations, the outcomes' high level of reliability. The proposed segmentation algorithm provided 96.89% overall correctness, 95.84% completeness, a 0.25 m overall mean value of centroid difference, and less than 1° of angle difference. The performance of the proposed approach was also verified with a large dataset and compared with other approaches. Additionally, the evaluation of the sensitivity of the thresholds was carried out. In summary, this paper proposes a robust and efficient segmentation methodology for abstraction of an enormous number of laser points into plane information.

New calibration method using low cost MEM IMUs to verify the performance of UAV-borne MMS payloads.

Spatial information plays a critical role in remote sensing and mapping applications such as environment surveying and disaster monitoring. An Unmanned Aerial Vehicle (UAV)-borne mobile mapping system (MMS) can accomplish rapid spatial information acquisition under limited sky conditions with better mobility and flexibility than other means. This study proposes a long endurance Direct Geo-referencing (DG)-based fixed-wing UAV photogrammetric platform and two DG modules that each use different commercial Micro-Electro Mechanical Systems' (MEMS) tactical grade Inertial Measurement Units (IMUs). Furthermore, this study develops a novel kinematic calibration method which includes lever arms, boresight angles and camera shutter delay to improve positioning accuracy. The new calibration method is then compared with the traditional calibration approach. The results show that the accuracy of the DG can be significantly improved by flying at a lower altitude using the new higher specification hardware. The new proposed method improves the accuracy of DG by about 20%. The preliminary results show that two-dimensional (2D) horizontal DG positioning accuracy is around 5.8 m at a flight height of 300 m using the newly designed tactical grade integrated Positioning and Orientation System (POS). The positioning accuracy in three-dimensions (3D) is less than 8 m.

Stability analysis for a multi-camera photogrammetric system.

Consumer-grade digital cameras suffer from geometrical instability that may cause problems when used in photogrammetric applications. This paper provides a comprehensive review of this issue of interior orientation parameter variation over time, it explains the common ways used for coping with the issue, and describes the existing methods for performing stability analysis for a single camera. The paper then points out the lack of coverage of stability analysis for multi-camera systems, suggests a modification of the collinearity model to be used for the calibration of an entire photogrammetric system, and proposes three methods for system stability analysis. The proposed methods explore the impact of the changes in interior orientation and relative orientation/mounting parameters on the reconstruction process. Rather than relying on ground truth in real datasets to check the system calibration stability, the proposed methods are simulation-based. Experiment results are shown, where a multi-camera photogrammetric system was calibrated three times, and stability analysis was performed on the system calibration parameters from the three sessions. The proposed simulation-based methods provided results that were compatible with a real-data based approach for evaluating the impact of changes in the system calibration parameters on the three-dimensional reconstruction.

Row selection in remote sensing from four-row plots of maize and sorghum based on repeatability and predictive modeling.

Remote sensing enables the rapid assessment of many traits that provide valuable information to plant breeders throughout the growing season to improve genetic gain. These traits are often extracted from remote sensing data on a row segment (rows within a plot) basis enabling the quantitative assessment of any row-wise subset of plants in a plot, rather than a few individual representative plants, as is commonly done in field-based phenotyping. Nevertheless, which rows to include in analysis is still a matter of debate. The objective of this experiment was to evaluate row selection and plot trimming in field trials conducted using four-row plots with remote sensing traits extracted from RGB (red-green-blue), LiDAR (light detection and ranging), and VNIR (visible near infrared) hyperspectral data. Uncrewed aerial vehicle flights were conducted throughout the growing seasons of 2018 to 2021 with data collected on three years of a sorghum experiment and two years of a maize experiment. Traits were extracted from each plot based on all four row segments (RS) (RS1234), inner rows (RS23), outer rows (RS14), and individual rows (RS1, RS2, RS3, and RS4). Plot end trimming of 40 cm was an additional factor tested. Repeatability and predictive modeling of end-season yield were used to evaluate performance of these methodologies. Plot trimming was never shown to result in significantly different outcomes from non-trimmed plots. Significant differences were often observed based on differences in row selection. Plots with more row segments were often favorable for increasing repeatability, and excluding outer rows improved predictive modeling. These results support long-standing principles of experimental design in agronomy and should be considered in breeding programs that incorporate remote sensing.

An Open-Source and Wearable System for Measuring 3D Human Motion in Real-Time.

OBJECTIVE: Analyzing human motion is essential for diagnosing movement disorders and guiding rehabilitation for conditions like osteoarthritis, stroke, and Parkinson's disease. Optical motion capture systems are the standard for estimating kinematics, but the equipment is expensive and requires a predefined space. While wearable sensor systems can estimate kinematics in any environment, existing systems are generally less accurate than optical motion capture. Many wearable sensor systems require a computer in close proximity and use proprietary software, limiting experimental reproducibility. METHODS: Here, we present OpenSenseRT, an open-source and wearable system that estimates upper and lower extremity kinematics in real time by using inertial measurement units and a portable microcontroller. RESULTS: We compared the OpenSenseRT system to optical motion capture and found an average RMSE of 4.4 degrees across 5 lower-limb joint angles during three minutes of walking and an average RMSE of 5.6 degrees across 8 upper extremity joint angles during a Fugl-Meyer task. The open-source software and hardware are scalable, tracking 1 to 14 body segments, with one sensor per segment. A musculoskeletal model and inverse kinematics solver estimate Kinematics in real-time. The computation frequency depends on the number of tracked segments, but is sufficient for real-time measurement for many tasks of interest; for example, the system can track 7 segments at 30 Hz in real-time. The system uses off-the-shelf parts costing approximately $100 USD plus $20 for each tracked segment. SIGNIFICANCE: The OpenSenseRT system is validated against optical motion capture, low-cost, and simple to replicate, enabling movement analysis in clinics, homes, and free-living settings.

Geometric calibration and radiometric correction of LiDAR data and their impact on the quality of derived products.

LiDAR (Light Detection And Ranging) systems are capable of providing 3D positional and spectral information (in the utilized spectrum range) of the mapped surface. Due to systematic errors in the system parameters and measurements, LiDAR systems require geometric calibration and radiometric correction of the intensity data in order to maximize the benefit from the collected positional and spectral information. This paper presents a practical approach for the geometric calibration of LiDAR systems and radiometric correction of collected intensity data while investigating their impact on the quality of the derived products. The proposed approach includes the use of a quasi-rigorous geometric calibration and the radar equation for the radiometric correction of intensity data. The proposed quasi-rigorous calibration procedure requires time-tagged point cloud and trajectory position data, which are available to most of the data users. The paper presents a methodology for evaluating the impact of the geometric calibration on the relative and absolute accuracy of the LiDAR point cloud. Furthermore, the impact of the geometric calibration and radiometric correction on land cover classification accuracy is investigated. The feasibility of the proposed methods and their impact on the derived products are demonstrated through experimental results using real data.

Direct sensor orientation of a land-based mobile mapping system.

A land-based mobile mapping system (MMS) is flexible and useful for the acquisition of road environment geospatial information. It integrates a set of imaging sensors and a position and orientation system (POS). The positioning quality of such systems is highly dependent on the accuracy of the utilized POS. This limitation is the major drawback due to the elevated cost associated with high-end GPS/INS units, particularly the inertial system. The potential accuracy of the direct sensor orientation depends on the architecture and quality of the GPS/INS integration process as well as the validity of the system calibration (i.e., calibration of the individual sensors as well as the system mounting parameters). In this paper, a novel single-step procedure using integrated sensor orientation with relative orientation constraint for the estimation of the mounting parameters is introduced. A comparative analysis between the proposed single-step and the traditional two-step procedure is carried out. Moreover, the estimated mounting parameters using the different methods are used in a direct geo-referencing procedure to evaluate their performance and the feasibility of the implemented system. Experimental results show that the proposed system using single-step system calibration method can achieve high 3D positioning accuracy.

Object-Based Integration of Photogrammetric and LiDAR Data for Automated Generation of Complex Polyhedral Building Models.

This research is concerned with a methodology for automated generation of polyhedral building models for complex structures, whose rooftops are bounded by straight lines. The process starts by utilizing LiDAR data for building hypothesis generation and derivation of individual planar patches constituting building rooftops. Initial boundaries of these patches are then refined through the integration of LiDAR and photogrammetric data and hierarchical processing of the planar patches. Building models for complex structures are finally produced using the refined boundaries. The performance of the developed methodology is evaluated through qualitative and quantitative analysis of the generated building models from real data.

OpenSim: open-source software to create and analyze dynamic simulations of movement.

Dynamic simulations of movement allow one to study neuromuscular coordination, analyze athletic performance, and estimate internal loading of the musculoskeletal system. Simulations can also be used to identify the sources of pathological movement and establish a scientific basis for treatment planning. We have developed a freely available, open-source software system (OpenSim) that lets users develop models of musculoskeletal structures and create dynamic simulations of a wide variety of movements. We are using this system to simulate the dynamics of individuals with pathological gait and to explore the biomechanical effects of treatments. OpenSim provides a platform on which the biomechanics community can build a library of simulations that can be exchanged, tested, analyzed, and improved through a multi-institutional collaboration. Developing software that enables a concerted effort from many investigators poses technical and sociological challenges. Meeting those challenges will accelerate the discovery of principles that govern movement control and improve treatments for individuals with movement pathologies.