The Novelty Appraisal of the Feeling of Risk in Vehicles.

Nowadays, vehicle assistance systems may assess the risks of the traffic situation with the help of advanced sensor technology and optimized algorithms. However, the passengers' feelings of risk in the vehicle have been mostly neglected. According to the Component Process Model of emotions, during the feeling of risk, novelty is one of the relevant event appraisals leading to particular physiological and facial responses. In order to identify whether or not indicators for novelty appraisal may be used for detecting the feeling of risk of vehicle occupants, we investigated physiological responses and facial expressions of individuals experiencing the feeling of risk with different levels of novelty. This secondary analysis of an earlier simulator study revealed that pupil diameter amplitude, skin conductance level changes, and changes in and amplitude of activity in facial expressions (the inner and outer brow raiser, brow lowerer, upper lid raiser and lid tightener) were correlated with the reduction in the novelty, suggesting that they could indicate the novelty of the feeling of risk of vehicle occupants. Hence, this research provides evidence for the novelty appraisal of the feeling of risk. Furthermore, it informs research on affect-aware systems to identify and reduce the feeling of risk of vehicle occupants in order to help to keep trust in automated vehicles high.

Toward a Holistic Communication Approach to an Automated Vehicle's Communication With Pedestrians: Combining Vehicle Kinematics With External Human-Machine Interfaces for Differently Sized Automated Vehicles.

Future automated vehicles (AVs) of different sizes will share the same space with other road users, e. g., pedestrians. For a safe interaction, successful communication needs to be ensured, in particular, with vulnerable road users, such as pedestrians. Two possible communication means exist for AVs: vehicle kinematics for implicit communication and external human-machine interfaces (eHMIs) for explicit communication. However, the exact interplay is not sufficiently studied yet for pedestrians' interactions with AVs. Additionally, very few other studies focused on the interplay of vehicle kinematics and eHMI for pedestrians' interaction with differently sized AVs, although the precise coordination is decisive to support the communication with pedestrians. Therefore, this study focused on how the interplay of vehicle kinematics and eHMI affects pedestrians' willingness to cross, trust and perceived safety for the interaction with two differently sized AVs (smaller AV vs. larger AV). In this experimental online study (N = 149), the participants interacted with the AVs in a shared space. Both AVs were equipped with a 360° LED light-band eHMI attached to the outer vehicle body. Three eHMI statuses (no eHMI, static eHMI, and dynamic eHMI) were displayed. The vehicle kinematics were varied at two levels (non-yielding vs. yielding). Moreover, "non-matching" conditions were included for both AVs in which the dynamic eHMI falsely communicated a yielding intent although the vehicle did not yield. Overall, results showed that pedestrians' willingness to cross was significantly higher for the smaller AV compared to the larger AV. Regarding the interplay of vehicle kinematics and eHMI, results indicated that a dynamic eHMI increased pedestrians' perceived safety when the vehicle yielded. When the vehicle did not yield, pedestrians' perceived safety still increased for the dynamic eHMI compared to the static eHMI and no eHMI. The findings of this study demonstrated possible negative effects of eHMIs when they did not match the vehicle kinematics. Further implications for a holistic communication strategy for differently sized AVs will be discussed.

One Solution Fits All? Evaluating Different Communication Strategies of a Light-based External Human-Machine Interface for Differently Sized Automated Vehicles from a Pedestrian's Perspective.

Differently sized automated vehicles (AVs) will enter the roads of tomorrow and will interact with other road users. Pedestrians as vulnerable road users heavily rely on the communication with other road users, especially for the interaction with larger vehicles, as miscommunication pose a high risk. Therefore, AVs need to provide communication abilities to safely interact with pedestrians. This study's focus was on the explicit communication which is highly relevant in low-speed and low-distance traffic scenarios to clarify misunderstandings before they result in accidents. External human-machine interfaces (eHMIs) placed on the outside of AVs can be used as a communication tool to explicitly inform the surrounding traffic environment. Although research manifested effects of vehicle size on pedestrians' perceived safety and crossing behavior, little research about the eHMI design for differently sized AVs exists. This experimental online study (N = 155) aimed at investigating the application of a light-based eHMI on two differently sized AVs (car, bus) by focusing on the overall goal of ensuring traffic safety in future traffic. The light-based eHMI showed different communication strategies, i.e., a static eHMI and three dynamic eHMIs. The results revealed that an automated car was perceived as safer and affectively rated as more positive compared to an automated bus. Nevertheless, no significant differences were found between the two AVs in terms of the eHMI communication. A dynamic eHMI was perceived as safer and evaluated affectively as more positive compared to a static eHMI or no eHMI for both AVs. In conclusion, the use of a light-based eHMI had a positive effect on pedestrians' interaction with an automated car and an automated bus and, therefore, could contribute to the overall traffic safety in this study. Implications for the design of eHMIs for differently sized AVs were discussed.

[Subjective certainty to increase the acceptance of automated and connected driving]. / Subjektive Sicherheit zur Steigerung der Akzeptanz des automatisierten und vernetzten Fahrens.

An important factor for the acceptance and thus the spread of automated and connected driving (ACD) is the degree of subjective uncertainty that users experience when interacting with automated vehicles. Subjective uncertainties always occur when people are not able to predict the further course of a situation or future events due to lack of experience or information. If such uncertainties occur during the use of automated vehicles, the development of trust and thus acceptance of this technology is impaired by the negative emotions accompanying subjective uncertainties. Within the AutoAkzept project (which full title translates to: Automation without uncertainty to increase the acceptance of automated and connected driving), solutions for user-focused automation have been developed that put vehicle occupants at the center of system development. User-focused systems take into account two basic human needs in human-machine interaction, the need to understand and the need to be understood. For this purpose, user-focused systems use different sensors to detect subjective uncertainties and their influencing factors in real time, integrate this information with context data and make adjustments that reduce subjective uncertainties. The systemic adaptations of user-focused systems follow a holistic approach that includes the levels of vehicle guidance, interior adaptation and information presentation as well as target guidance are included. By reducing or avoiding subjective uncertainties, the project developments contribute to a positive, comfortable user experience and help to increase the acceptance of ACD. This paper presents research results of AutoAkzept on the topics of user state and activity modelling as well as needs-based adaptation strategies, which represent key components for the implementation of user-focused automation.

Understanding the Multidimensional and Dynamic Nature of Facial Expressions Based on Indicators for Appraisal Components as Basis for Measuring Drivers' Fear.

Facial expressions are one of the commonly used implicit measurements for the in-vehicle affective computing. However, the time courses and the underlying mechanism of facial expressions so far have been barely focused on. According to the Component Process Model of emotions, facial expressions are the result of an individual's appraisals, which are supposed to happen in sequence. Therefore, a multidimensional and dynamic analysis of drivers' fear by using facial expression data could profit from a consideration of these appraisals. A driving simulator experiment with 37 participants was conducted, in which fear and relaxation were induced. It was found that the facial expression indicators of high novelty and low power appraisals were significantly activated after a fear event (high novelty: Z = 2.80, p < 0.01, r contrast = 0.46; low power: Z = 2.43, p < 0.05, r contrast = 0.50). Furthermore, after the fear event, the activation of high novelty occurred earlier than low power. These results suggest that multidimensional analysis of facial expression is suitable as an approach for the in-vehicle measurement of the drivers' emotions. Furthermore, a dynamic analysis of drivers' facial expressions considering of effects of appraisal components can add valuable information for the in-vehicle assessment of emotions.

Recognizing Frustration of Drivers From Face Video Recordings and Brain Activation Measurements With Functional Near-Infrared Spectroscopy.

Experiencing frustration while driving can harm cognitive processing, result in aggressive behavior and hence negatively influence driving performance and traffic safety. Being able to automatically detect frustration would allow adaptive driver assistance and automation systems to adequately react to a driver's frustration and mitigate potential negative consequences. To identify reliable and valid indicators of driver's frustration, we conducted two driving simulator experiments. In the first experiment, we aimed to reveal facial expressions that indicate frustration in continuous video recordings of the driver's face taken while driving highly realistic simulator scenarios in which frustrated or non-frustrated emotional states were experienced. An automated analysis of facial expressions combined with multivariate logistic regression classification revealed that frustrated time intervals can be discriminated from non-frustrated ones with accuracy of 62.0% (mean over 30 participants). A further analysis of the facial expressions revealed that frustrated drivers tend to activate muscles in the mouth region (chin raiser, lip pucker, lip pressor). In the second experiment, we measured cortical activation with almost whole-head functional near-infrared spectroscopy (fNIRS) while participants experienced frustrating and non-frustrating driving simulator scenarios. Multivariate logistic regression applied to the fNIRS measurements allowed us to discriminate between frustrated and non-frustrated driving intervals with higher accuracy of 78.1% (mean over 12 participants). Frustrated driving intervals were indicated by increased activation in the inferior frontal, putative premotor and occipito-temporal cortices. Our results show that facial and cortical markers of frustration can be informative for time resolved driver state identification in complex realistic driving situations. The markers derived here can potentially be used as an input for future adaptive driver assistance and automation systems that detect driver frustration and adaptively react to mitigate it.

Demonstrating Brain-Level Interactions Between Visuospatial Attentional Demands and Working Memory Load While Driving Using Functional Near-Infrared Spectroscopy.

Driving is a complex task concurrently drawing on multiple cognitive resources. Yet, there is a lack of studies investigating interactions at the brain-level among different driving subtasks in dual-tasking. This study investigates how visuospatial attentional demands related to increased driving difficulty interacts with different working memory load (WML) levels at the brain level. Using multichannel whole-head high density functional near-infrared spectroscopy (fNIRS) brain activation measurements, we aimed to predict driving difficulty level, both separate for each WML level and with a combined model. Participants drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. In half of the time, the course led through a construction site with reduced lane width, increasing visuospatial attentional demands. Concurrently, participants performed a modified version of the n-back task with five different WML levels (from 0-back up to 4-back), forcing them to continuously update, memorize, and recall the sequence of the previous 'n' speed signs and adjust their speed accordingly. Using multivariate logistic ridge regression, we were able to correctly predict driving difficulty in 75.0% of the signal samples (1.955 Hz sampling rate) across 15 participants in an out-of-sample cross-validation of classifiers trained on fNIRS data separately for each WML level. There was a significant effect of the WML level on the driving difficulty prediction accuracies [range 62.2-87.1%; χ2(4) = 19.9, p < 0.001, Kruskal-Wallis H test] with highest prediction rates at intermediate WML levels. On the contrary, training one classifier on fNIRS data across all WML levels severely degraded prediction performance (mean accuracy of 46.8%). Activation changes in the bilateral dorsal frontal (putative BA46), bilateral inferior parietal (putative BA39), and left superior parietal (putative BA7) areas were most predictive to increased driving difficulty. These discriminative patterns diminished at higher WML levels indicating that visuospatial attentional demands and WML involve interacting underlying brain processes. The changing pattern of driving difficulty related brain areas across WML levels could indicate potential changes in the multitasking strategy with level of WML demand, in line with the multiple resource theory.

Corrigendum: Assessing the Driver's Current Level of Working Memory Load With High Density Functional Near-infrared Spectroscopy: A Realistic Driving Simulator Study.

[This corrects the article DOI: 10.3389/fnhum.2017.00167.].

Assessing the Driver's Current Level of Working Memory Load with High Density Functional Near-infrared Spectroscopy: A Realistic Driving Simulator Study.

Cognitive overload or underload results in a decrease in human performance which may result in fatal incidents while driving. We envision that driver assistive systems which adapt their functionality to the driver's cognitive state could be a promising approach to reduce road accidents due to human errors. This research attempts to predict variations of cognitive working memory load levels in a natural driving scenario with multiple parallel tasks and to reveal predictive brain areas. We used a modified version of the n-back task to induce five different working memory load levels (from 0-back up to 4-back) forcing the participants to continuously update, memorize, and recall the previous 'n' speed sequences and adjust their speed accordingly while they drove for approximately 60 min on a highway with concurrent traffic in a virtual reality driving simulator. We measured brain activation using multichannel whole head, high density functional near-infrared spectroscopy (fNIRS) and predicted working memory load level from the fNIRS data by combining multivariate lasso regression and cross-validation. This allowed us to predict variations in working memory load in a continuous time-resolved manner with mean Pearson correlations between induced and predicted working memory load over 15 participants of 0.61 [standard error (SE) 0.04] and a maximum of 0.8. Restricting the analysis to prefrontal sensors placed over the forehead reduced the mean correlation to 0.38 (SE 0.04), indicating additional information gained through whole head coverage. Moreover, working memory load predictions derived from peripheral heart rate parameters achieved much lower correlations (mean 0.21, SE 0.1). Importantly, whole head fNIRS sampling revealed increasing brain activation in bilateral inferior frontal and bilateral temporo-occipital brain areas with increasing working memory load levels suggesting that these areas are specifically involved in workload-related processing.

Reaction Times to Consecutive Automation Failures: A Function of Working Memory and Sustained Attention.

OBJECTIVE: This study explored whether working memory and sustained attention influence cognitive lock-up, which is a delay in the response to consecutive automation failures. BACKGROUND: Previous research has demonstrated that the information that automation provides about failures and the time pressure that is associated with a task influence cognitive lock-up. Previous research has also demonstrated considerable variability in cognitive lock-up between participants. This is why individual differences might influence cognitive lock-up. The present study tested whether working memory-including flexibility in executive functioning-and sustained attention might be crucial in this regard. METHOD: Eighty-five participants were asked to monitor automated aircraft functions. The experimental manipulation consisted of whether or not an initial automation failure was followed by a consecutive failure. Reaction times to the failures were recorded. Participants' working-memory and sustained-attention abilities were assessed with standardized tests. RESULTS: As expected, participants' reactions to consecutive failures were slower than their reactions to initial failures. In addition, working-memory and sustained-attention abilities enhanced the speed with which participants reacted to failures, more so with regard to consecutive than to initial failures. CONCLUSION: The findings highlight that operators with better working memory and sustained attention have small advantages when initial failures occur, but their advantages increase across consecutive failures. APPLICATION: The results stress the need to consider personnel selection strategies to mitigate cognitive lock-up in general and training procedures to enhance the performance of low ability operators.

Does assisted driving behavior lead to safety-critical encounters with unequipped vehicles' drivers?

With Intelligent Transport Systems (e.g., traffic light assistance systems) assisted drivers are able to show driving behavior in anticipation of upcoming traffic situations. In the years to come, the penetration rate of such systems will be low. Therefore, the majority of vehicles will not be equipped with these systems. Unequipped vehicles' drivers may not expect the driving behavior of assisted drivers. However, drivers' predictions and expectations can play a significant role in their reaction times. Thus, safety issues could arise when unequipped vehicles' drivers encounter driving behavior of assisted drivers. This is why we tested how unequipped vehicles' drivers (N=60) interpreted and reacted to the driving behavior of an assisted driver. We used a multi-driver simulator with three drivers. The three drivers were driving in a line. The lead driver in the line was a confederate who was followed by two unequipped vehicles' drivers. We varied the equipment of the confederate with an Intelligent Transport System: The confederate was equipped either with or without a traffic light assistance system. The traffic light assistance system provided a start-up maneuver before a light turned green. Therefore, the assisted confederate seemed to show unusual deceleration behavior by coming to a halt at an unusual distance from the stop line at the red traffic light. The unusual distance was varied as we tested a moderate (4m distance from the stop line) and an extreme (10m distance from the stop line) parameterization of the system. Our results showed that the extreme parametrization resulted in shorter minimal time-to-collision of the unequipped vehicles' drivers. One rear-end crash was observed. These results provided initial evidence that safety issues can arise when unequipped vehicles' drivers encounter assisted driving behavior. We recommend that future research identifies counteractions to prevent these safety issues. Moreover, we recommend that system developers discuss the best parameterizations of their systems to ensure benefits but also the safety in encounters with unequipped vehicles' drivers.

Expertise Development With Different Types of Automation: A Function of Different Cognitive Abilities.

OBJECTIVE: I explored whether different cognitive abilities (information-processing ability, working-memory capacity) are needed for expertise development when different types of automation (information vs. decision automation) are employed. BACKGROUND: It is well documented that expertise development and the employment of automation lead to improved performance. Here, it is argued that a learner's ability to reason about an activity may be hindered by the employment of information automation. Additional feedback needs to be processed, thus increasing the load on working memory and decelerating expertise development. By contrast, the employment of decision automation may stimulate reasoning, increase the initial load on information-processing ability, and accelerate expertise development. Authors of past research have not investigated the interrelations between automation assistance, individual differences, and expertise development. METHOD: Sixty-one naive learners controlled simulated air traffic with two types of automation: information automation and decision automation. Their performance was captured across 16 trials. Well-established tests were used to assess information-processing ability and working-memory capacity. RESULTS: As expected, learners' performance benefited from expertise development and decision automation. Furthermore, individual differences moderated the effect of the type of automation on expertise development: The employment of only information automation increased the load on working memory during later expertise development. The employment of decision automation initially increased the need to process information. CONCLUSION: These findings highlight the importance of considering individual differences and expertise development when investigating human-automation interaction. APPLICATION: The results are relevant for selecting automation configurations for expertise development.

Individual differences and their impact on the safety and the efficiency of human-wheelchair systems.

OBJECTIVE: The extent to which individual differences in fine motor abilities affect indoor safety and efficiency of human-wheelchair systems was examined. BACKGROUND: To reduce the currently large number of indoor wheelchair accidents, assistance systems with a high level of automation were developed. It was proposed to adapt the wheelchair's level of automation to the user's ability to steer the device to avoid drawbacks of highly automated wheelchairs. The state of the art, however, lacks an empirical identification of those abilities. METHOD: A study with 23 participants is described. The participants drove through various sections of a course with a powered wheelchair. Repeatedly measured criteria were safety (numbers of collisions) and efficiency (times required for reaching goals). As covariates, the participants' fine motor abilities were assessed. RESULTS: A random coefficient modeling approach was conducted to analyze the data,which were available on two levels as course sections were nested within participants.The participants' aiming, precision, and armhand speed contributed significantly to both criteria: Participants with lower fine motor abilities had more collisions and required more time for reaching goals. CONCLUSION: Adapting the wheelchair's level of automation to these fine motor abilities can improve indoor safety and efficiency. In addition, the results highlight the need to further examine the impact of individual differences on the design of automation features for powered wheelchairs as well as other applications of automation. APPLICATION: The results facilitate the improvement of current wheelchair technology.