How Can the Trust-Change Direction be Measured and Identified During Takeover Transitions in Conditionally Automated Driving? Using Physiological Responses and Takeover-Related Factors.

OBJECTIVE: This paper proposes an objective method to measure and identify trust-change directions during takeover transitions (TTs) in conditionally automated vehicles (AVs). BACKGROUND: Takeover requests (TORs) will be recurring events in conditionally automated driving that could undermine trust, and then lead to inappropriate reliance on conditionally AVs, such as misuse and disuse. METHOD: 34 drivers engaged in the non-driving-related task were involved in a sequence of takeover events in a driving simulator. The relationships and effects between drivers' physiological responses, takeover-related factors, and trust-change directions during TTs were explored by the combination of an unsupervised learning algorithm and statistical analyses. Furthermore, different typical machine learning methods were applied to establish recognition models of trust-change directions during TTs based on takeover-related factors and physiological parameters. RESULT: Combining the change values in the subjective trust rating and monitoring behavior before and after takeover can reliably measure trust-change directions during TTs. The statistical analysis results showed that physiological parameters (i.e., skin conductance and heart rate) during TTs are negatively linked with the trust-change directions. And drivers were more likely to increase trust during TTs when they were in longer TOR lead time, with more takeover frequencies, and dealing with the stationary vehicle scenario. More importantly, the F1-score of the random forest (RF) model is nearly 77.3%. CONCLUSION: The features investigated and the RF model developed can identify trust-change directions during TTs accurately. APPLICATION: Those findings can provide additional support for developing trust monitoring systems to mitigate both drivers' overtrust and undertrust in conditionally AVs.

Correction: Osman, M., et al. A Novel Online Approach for Drift Covariance Estimation of Odometries Used in Intelligent Vehicle Localization. Sensors 2019, 19, 5178.

The authors wish to make the following corrections to this paper [...].

Hybrid Path Planning Combining Potential Field with Sigmoid Curve for Autonomous Driving.

The traditional potential field-based path planning is likely to generate unexpected path by strictly following the minimum potential field, especially in the driving scenarios with multiple obstacles closely distributed. A hybrid path planning is proposed to avoid the unsatisfying path generation and to improve the performance of autonomous driving by combining the potential field with the sigmoid curve. The repulsive and attractive potential fields are redesigned by considering the safety and the feasibility. Based on the objective of the shortest path generation, the optimized trajectory is obtained to improve the vehicle stability and driving safety by considering the constraints of collision avoidance and vehicle dynamics. The effectiveness is examined by simulations in multiobstacle dynamic and static scenarios. The simulation results indicate that the proposed method shows better performance on vehicle stability and ride comfortability than that of the traditional potential field-based method in all the examined scenarios during the autonomous driving.

A Novel Online Approach for Drift Covariance Estimation of Odometries Used in Intelligent Vehicle Localization.

Localization is the fundamental problem of intelligent vehicles. For a vehicle to autonomously operate, it first needs to locate itself in the environment. A lot of different odometries (visual, inertial, wheel encoders) have been introduced through the past few years for autonomous vehicle localization. However, such odometries suffers from drift due to their reliance on integration of sensor measurements. In this paper, the drift error in an odometry is modeled and a Drift Covariance Estimation (DCE) algorithm is introduced. The DCE algorithm estimates the covariance of an odometry using the readings of another on-board sensor which does not suffer from drift. To validate the proposed algorithm, several real-world experiments in different conditions as well as sequences from Oxford RobotCar Dataset and EU long-term driving dataset are used. The effect of the covariance estimation on three different fusion-based localization algorithms (EKF, UKF and EH-infinity) is studied in comparison with the use of constant covariance, which were calculated based on the true variance of the sensors being used. The obtained results show the efficacy of the estimation algorithm compared to constant covariances in terms of improving the accuracy of localization.

A Personalized Behavior Learning System for Human-Like Longitudinal Speed Control of Autonomous Vehicles.

As the main component of an autonomous driving system, the motion planner plays an essential role for safe and efficient driving. However, traditional motion planners cannot make full use of the on-board sensing information and lack the ability to efficiently adapt to different driving scenes and behaviors of different drivers. To overcome this limitation, a personalized behavior learning system (PBLS) is proposed in this paper to improve the performance of the traditional motion planner. This system is based on the neural reinforcement learning (NRL) technique, which can learn from human drivers online based on the on-board sensing information and realize human-like longitudinal speed control (LSC) through the learning from demonstration (LFD) paradigm. Under the LFD framework, the desired speed of human drivers can be learned by PBLS and converted to the low-level control commands by a proportion integration differentiation (PID) controller. Experiments using driving simulator and real driving data show that PBLS can adapt to different drivers by reproducing their driving behaviors for LSC in different scenes. Moreover, through a comparative experiment with the traditional adaptive cruise control (ACC) system, the proposed PBLS demonstrates a superior performance in maintaining driving comfort and smoothness.

An Orientation Sensor-Based Head Tracking System for Driver Behaviour Monitoring.

Although at present legislation does not allow drivers in a Level 3 autonomous vehicle to engage in a secondary task, there may become a time when it does. Monitoring the behaviour of drivers engaging in various non-driving activities (NDAs) is crucial to decide how well the driver will be able to take over control of the vehicle. One limitation of the commonly used face-based head tracking system, using cameras, is that sufficient features of the face must be visible, which limits the detectable angle of head movement and thereby measurable NDAs, unless multiple cameras are used. This paper proposes a novel orientation sensor based head tracking system that includes twin devices, one of which measures the movement of the vehicle while the other measures the absolute movement of the head. Measurement error in the shaking and nodding axes were less than 0.4°, while error in the rolling axis was less than 2°. Comparison with a camera-based system, through in-house tests and on-road tests, showed that the main advantage of the proposed system is the ability to detect angles larger than 20° in the shaking and nodding axes. Finally, a case study demonstrated that the measurement of the shaking and nodding angles, produced from the proposed system, can effectively characterise the drivers' behaviour while engaged in the NDAs of chatting to a passenger and playing on a smartphone.

Precise Tracking Control for Articulating Crane: Prescribed Performance, Adaptation, and Fuzzy Optimality by Nash Game.

Articulating crane (AC) is used in various industrial activities. The articulated multisection arm exacerbates nonlinearities and uncertainties, making the precise tracking control challenging. This study proposes an adaptive prescribed performance tracking control (APPTC) for AC to robustly fulfill the task of precise tracking control, with adaptation to resist time-variant uncertainties, whose bounds are unknown but lie in prescribed fuzzy sets. Particularly, a state transformation is applied to simultaneously track the desired trajectory and satisfy the prescribed performance. Adopting the fuzzy set theory to describe uncertainties, APPTC does not invoke any IF-THEN fuzzy rules. There is no linearizations, or nonlinear cancelation for APPTC, thus making it approximation free. The performance of the controlled AC is twofold. First, deterministic performance in fulfilling the control task is ensured by the Lyapunov analysis using uniform boundedness and uniform ultimate boundedness. Second, fuzzy-based performance is further improved by an optimal design, which seeks the optima of control parameters by formulating a two-player Nash game. The existence of Nash equilibrium is theoretically proved, and its acquisition process is given. The simulation results are provided for validations. This is the first endeavor that explores the precise tracking control for fuzzy AC.

3D-DFM: Anchor-Free Multimodal 3-D Object Detection With Dynamic Fusion Module for Autonomous Driving.

Recent advances in cross-modal 3D object detection rely heavily on anchor-based methods, and however, intractable anchor parameter tuning and computationally expensive postprocessing severely impede an embedded system application, such as autonomous driving. In this work, we develop an anchor-free architecture for efficient camera-light detection and ranging (LiDAR) 3D object detection. To highlight the effect of foreground information from different modalities, we propose a dynamic fusion module (DFM) to adaptively interact images with point features via learnable filters. In addition, the 3D distance intersection-over-union (3D-DIoU) loss is explicitly formulated as a supervision signal for 3D-oriented box regression and optimization. We integrate these components into an end-to-end multimodal 3D detector termed 3D-DFM. Comprehensive experimental results on the widely used KITTI dataset demonstrate the superiority and universality of 3D-DFM architecture, with competitive detection accuracy and real-time inference speed. To the best of our knowledge, this is the first work that incorporates an anchor-free pipeline with multimodal 3D object detection.

A multimodal psychological, physiological and behavioural dataset for human emotions in driving tasks.

Human emotions are integral to daily tasks, and driving is now a typical daily task. Creating a multi-modal human emotion dataset in driving tasks is an essential step in human emotion studies. we conducted three experiments to collect multimodal psychological, physiological and behavioural dataset for human emotions (PPB-Emo). In Experiment I, 27 participants were recruited, the in-depth interview method was employed to explore the driver's viewpoints on driving scenarios that induce different emotions. For Experiment II, 409 participants were recruited, a questionnaire survey was conducted to obtain driving scenarios information that induces human drivers to produce specific emotions, and the results were used as the basis for selecting video-audio stimulus materials. In Experiment III, 40 participants were recruited, and the psychological data and physiological data, as well as their behavioural data were collected of all participants in 280 times driving tasks. The PPB-Emo dataset will largely support the analysis of human emotion in driving tasks. Moreover, The PPB-Emo dataset will also benefit human emotion research in other daily tasks.

Deep Learning for LiDAR Point Clouds in Autonomous Driving: A Review.

Recently, the advancement of deep learning (DL) in discriminative feature learning from 3-D LiDAR data has led to rapid development in the field of autonomous driving. However, automated processing uneven, unstructured, noisy, and massive 3-D point clouds are a challenging and tedious task. In this article, we provide a systematic review of existing compelling DL architectures applied in LiDAR point clouds, detailing for specific tasks in autonomous driving, such as segmentation, detection, and classification. Although several published research articles focus on specific topics in computer vision for autonomous vehicles, to date, no general survey on DL applied in LiDAR point clouds for autonomous vehicles exists. Thus, the goal of this article is to narrow the gap in this topic. More than 140 key contributions in the recent five years are summarized in this survey, including the milestone 3-D deep architectures, the remarkable DL applications in 3-D semantic segmentation, object detection, and classification; specific data sets, evaluation metrics, and the state-of-the-art performance. Finally, we conclude the remaining challenges and future researches.

Pattern Recognition and Characterization of Upper Limb Neuromuscular Dynamics during Driver-Vehicle Interactions.

In this work, pattern recognition and characterization of the neuromuscular dynamics of driver upper limb during naturalistic driving were studied. During the human-in-the-loop experiments, two steering tasks, namely, the passive and active steering tasks, were instructed to be completed by the subjects. Furthermore, subjects manipulated the steering wheel with two distinct postures and six different hand positions. The neuromuscular dynamics of subjects' upper limb were measured using electromyogram signals, and the behavioral data, including the steering torque and steering angle, were also collected. Based on the experimental data, patterns of muscle activities during naturalistic driving were investigated. The correlations, amplitudes, and responsiveness of the electromyogram signals, as well as the smoothness and regularity of the steering torque were discussed. The results reveal the mechanisms of neuromuscular dynamics of driver upper limb and provide a theoretical foundation for the design of the future human-machine interface for automated vehicles.

Neural Network Based Uncertainty Prediction for Autonomous Vehicle Application.

This paper proposes a framework for uncertainty prediction in complex fusion networks, where signals become available sporadically. Assuming there is no information of the sensor characteristics available, a surrogated model of the sensor uncertainty is yielded directly from data through artificial neural networks. The strategy developed is applied to autonomous vehicle localization through odometry sensors (speed and orientation), so as to determine the location uncertainty in the trajectory. The results obtained allow for fusion of autonomous vehicle location measurements, and effective correction of the accumulated odometry error in most scenarios. The neural networks applicability and generalization capacity are proven, evidencing the suitability of the presented methodology for uncertainty estimation in non-linear and intractable processes.

Parallel testing of vehicle intelligence via virtual-real interaction.

A self-driven closed-loop parallel testing system implements more challenging tests to accelerate evaluation and development of autonomous vehicles.

Simultaneous Observation of Hybrid States for Cyber-Physical Systems: A Case Study of Electric Vehicle Powertrain.

As a typical cyber-physical system (CPS), electrified vehicle becomes a hot research topic due to its high efficiency and low emissions. In order to develop advanced electric powertrains, accurate estimations of the unmeasurable hybrid states, including discrete backlash nonlinearity and continuous half-shaft torque, are of great importance. In this paper, a novel estimation algorithm for simultaneously identifying the backlash position and half-shaft torque of an electric powertrain is proposed using a hybrid system approach. System models, including the electric powertrain and vehicle dynamics models, are established considering the drivetrain backlash and flexibility, and also calibrated and validated using vehicle road testing data. Based on the developed system models, the powertrain behavior is represented using hybrid automata according to the piecewise affine property of the backlash dynamics. A hybrid-state observer, which is comprised of a discrete-state observer and a continuous-state observer, is designed for the simultaneous estimation of the backlash position and half-shaft torque. In order to guarantee the stability and reachability, the convergence property of the proposed observer is investigated. The proposed observer are validated under highly dynamical transitions of vehicle states. The validation results demonstrates the feasibility and effectiveness of the proposed hybrid-state observer.