Subjective awareness of sleepiness while driving in younger and older adults.

Understanding whether drivers can accurately assess sleepiness is essential for educational campaigns advising drivers to stop driving when feeling sleepy. However, few studies have examined this in real-world driving environments, particularly among older drivers who comprise a large proportion of all road users. To examine the accuracy of subjective sleepiness ratings in predicting subsequent driving impairment and physiological drowsiness, 16 younger (21-33 years) and 17 older (50-65 years) adults drove an instrumented vehicle for 2 h on closed loop under two conditions: well-rested and 29 h sleep deprivation. Sleepiness ratings (Karolinska Sleepiness Scale, Likelihood of Falling Asleep scale, Sleepiness Symptoms Questionnaire) were obtained every 15min, alongside lane deviations, near crash events, and ocular indices of drowsiness. All subjective sleepiness measures increased with sleep deprivation for both age groups (p < 0.013). While most subjective sleepiness ratings significantly predicted driving impairment and drowsiness in younger adults (OR: 1.7-15.6, p < 0.02), this was only apparent for KSS, likelihood of falling asleep, and "difficulty staying in the lane for the older adults" (OR: 2.76-2.86, p = 0.02). This may be due to an altered perception of sleepiness in older adults, or due to lowered objective signs of impairment in the older group. Our data suggest that (i) younger and older drivers are aware of sleepiness; (ii) the best subjective scale may differ across age groups; and (iii) future research should expand on the best subjective measures to inform of crash risk in older adults to inform tailored educational road safety campaigns on signs of sleepiness.

Get Ready for Take-Overs: Using Head-Up Display for Drivers to Engage in Non-Driving-Related Tasks in Automated Vehicles.

OBJECTIVE: The study aims to investigate the potential of using HUD (head-up display) as an approach for drivers to engage in non-driving-related tasks (NDRTs) during automated driving, and examine the impacts on driver state and take-over performance in comparison to the traditional mobile phone. BACKGROUND: Advances in automated vehicle technology have the potential to relieve drivers from driving tasks so that they can engage in NDRTs freely. However, drivers will still need to take-over control under certain circumstances. METHOD: A driving simulation experiment was conducted using an Advanced Driving Simulator and real-world driving videos. Forty-six participants completed three drives in three display conditions, respectively (HUD, mobile phone and baseline without NDRT). The HUD was integrated with the vehicle in displaying NDRTs while the mobile phone was not. Drivers' visual (e.g. gaze, blink) and physiological (e.g. ECG, EDA) data were collected to measure driver state. Two take-over reaction times (hand and foot) were used to measure take-over performance. RESULTS: The HUD significantly shortened the take-over reaction times compared to the mobile phone condition. Compared to the baseline condition, drivers in the HUD condition also experienced lower cognitive workload and physiological arousal. Drivers' take-over reaction times were significantly correlated with their visual and electrodermal activities during automated driving prior to the take-over request. CONCLUSION: HUDs can improve driver performance and lower workload when used as an NDRT interface. APPLICATION: The study sheds light on a promising approach for drivers to engage in NDRTs in future AVs.

European NCAP Driver State Monitoring Protocols: Prevalence of Distraction in Naturalistic Driving.

OBJECTIVE: examine the prevalence of driver distraction in naturalistic driving when implementing European New Car Assessment Program (Euro NCAP)-defined distraction behaviours. BACKGROUND: The 2023 introduction of Occupant Status monitoring (OSM) into Euro NCAP will accelerate uptake of Driver State Monitoring (DSM). Euro NCAP outlines distraction behaviours that DSM must detect to earn maximum safety points. Distraction behaviour prevalence and driver alerting and intervention frequency have yet to be examined in naturalistic driving. METHOD: Twenty healthcare workers were provided with an instrumented vehicle for approximately two weeks. Data were continuously monitored with automotive grade DSM during daily work commutes, resulting in 168.8 hours of driver head, eye and gaze tracking. RESULTS: Single long distraction events were the most prevalent, with .89 events/hour. Implementing different thresholds for driving-related and driving-unrelated glance regions impacts alerting rates. Lizard glances (primarily gaze movement) occurred more frequently than owl glances (primarily head movement). Visual time-sharing events occurred at a rate of .21 events/hour. CONCLUSION: Euro NCAP-described driver distraction occurs naturalistically. Lizard glances, requiring gaze tracking, occurred in high frequency relative to owl glances, which only require head tracking, indicating that less sophisticated DSM will miss a substantial amount of distraction events. APPLICATION: This work informs OEMs, DSM manufacturers and regulators of the expected alerting rate of Euro NCAP defined distraction behaviours. Alerting rates will vary with protocol implementation, technology capability, and HMI strategies adopted by the OEMs, in turn impacting safety outcomes, user experience and acceptance of DSM technology.

Evaluating Driver Features for Cognitive Distraction Detection and Validation in Manual and Level 2 Automated Driving.

OBJECTIVE: This study aimed to investigate the impacts of feature selection on driver cognitive distraction (CD) detection and validation in real-world nonautomated and Level 2 automated driving scenarios. BACKGROUND: Real-time driver state monitoring is critical to promote road user safety. METHOD: Twenty-four participants were recruited to drive a Tesla Model S in manual and Autopilot modes on the highway while engaging in the N-back task. In each driving mode, CD was classified by the random forest algorithm built on three "hand-crafted" glance features (i.e., percent road center [PRC], the standard deviation of gaze pitch, and yaw angles), or through a large number of features that were transformed from the output of a driver monitoring system (DMS) and other sensing systems. RESULTS: In manual driving, the small set of glance features was as effective as the large set of machine-generated features in terms of classification accuracy. Whereas in Level 2 automated driving, both glance and vehicle features were less sensitive to CD. The glance features also revealed that the misclassified driver state was the result of the dynamic fluctuations and individual differences of cognitive loads under CD. CONCLUSION: Glance metrics are critical for the detection and validation of CD in on-road driving. APPLICATIONS: The paper suggests the practical value of human factors domain knowledge in feature selection and ground truth validation for the development of driver monitoring technologies.

Effects of Distraction in On-Road Level 2 Automated Driving: Impacts on Glance Behavior and Takeover Performance.

OBJECTIVE: The paper aimed to investigate glance behaviors under different levels of distraction in automated driving (AD) and understand the impact of distraction levels on driver takeover performance. BACKGROUND: Driver distraction detrimentally affects takeover performance. Glance-based distraction measurement could be a promising method to remind drivers to maintain enough attentiveness before the takeover request in partially AD. METHOD: Thirty-six participants were recruited to drive a Tesla Model S in manual and Autopilot modes on a test track while engaging in secondary tasks, including temperature-control, email-sorting, and music-selection, to impose low and high distractions. During the test drive, participants needed to quickly change the lane as if avoiding an immediate road hazard if they heard an unexpected takeover request (an auditory warning). Driver state and behavior over the test drive were recorded in real time by a driver monitoring system and several other sensors installed in the Tesla vehicle. RESULTS: The distribution of off-road glance duration was heavily skewed (with a long tail) by high distractions, with extreme glance duration more than 30 s. Moreover, being eyes-off-road before takeover could cause more delay in the urgent takeover reaction compared to being hands-off-wheel. CONCLUSION: The study measured off-road glance duration under different levels of distraction and demonstrated the impacts of being eyes-off-road and hands-off-wheel on the following takeover performance. APPLICATION: The findings provide new insights about engagement in Level 2 AD and are useful for the design of driver monitoring technologies for distraction management.

The Impacts of Temporal Variation and Individual Differences in Driver Cognitive Workload on ECG-Based Detection.

OBJECTIVE: This paper aimed to investigate the robustness of driver cognitive workload detection based on electrocardiogram (ECG) when considering temporal variation and individual differences in cognitive workload. BACKGROUND: Cognitive workload is a critical component to be monitored for error prevention in human-machine systems. It may fluctuate instantaneously over time even in the same tasks and differ across individuals. METHOD: A driving simulation study was conducted to classify driver cognitive workload underlying four experimental conditions (baseline, N-back, texting, and N-back + texting distraction) in two repeated 1-hr blocks. Heart rate (HR) and heart rate variability (HRV) were compared among the experimental conditions and between the blocks. Random forests were built on HR and HRV to classify cognitive workload in different blocks and for different individuals. RESULTS: HR and HRV were significantly different between repeated blocks in the study, demonstrating the time-induced variation in cognitive workload. The performance of cognitive workload classification across blocks and across individuals was significantly improved after normalizing HR and HRV in each block by the corresponding baseline. CONCLUSION: The temporal variation and individual differences in cognitive workload affects ECG-based cognitive workload detection. But normalization approaches relying on the choice of appropriate baselines help compensate for the effects of temporal variation and individual differences. APPLICATION: The findings provide insight into the value and limitations of ECG-based driver cognitive workload monitoring during prolonged driving for individual drivers.

Modeling Driving Performance Using In-Vehicle Speech Data From a Naturalistic Driving Study.

OBJECTIVE: We aimed to (a) describe the development and application of an automated approach for processing in-vehicle speech data from a naturalistic driving study (NDS), (b) examine the influence of child passenger presence on driving performance, and (c) model this relationship using in-vehicle speech data. BACKGROUND: Parent drivers frequently engage in child-related secondary behaviors, but the impact on driving performance is unknown. Applying automated speech-processing techniques to NDS audio data would facilitate the analysis of in-vehicle driver-child interactions and their influence on driving performance. METHOD: Speech activity detection and speaker diarization algorithms were applied to audio data from a Melbourne-based NDS involving 42 families. Multilevel models were developed to evaluate the effect of speech activity and the presence of child passengers on driving performance. RESULTS: Speech activity was significantly associated with velocity and steering angle variability. Child passenger presence alone was not associated with changes in driving performance. However, speech activity in the presence of two child passengers was associated with the most variability in driving performance. CONCLUSION: The effects of in-vehicle speech on driving performance in the presence of child passengers appear to be heterogeneous, and multiple factors may need to be considered in evaluating their impact. This goal can potentially be achieved within large-scale NDS through the automated processing of observational data, including speech. APPLICATION: Speech-processing algorithms enable new perspectives on driving performance to be gained from existing NDS data, and variables that were once labor-intensive to process can be readily utilized in future research.

Beyond gaze fixation: Modeling peripheral vision in relation to speed, Tesla Autopilot, cognitive load, and age in highway driving.

OBJECTIVE: The study aims to model driver perception across the visual field in dynamic, real-world highway driving. BACKGROUND: Peripheral vision acquires information across the visual field and guides a driver's information search. Studies in naturalistic settings are lacking however, with most research having been conducted in controlled simulation environments with limited eccentricities and driving dynamics. METHODS: We analyzed data from 24 participants who drove a Tesla Model S with Autopilot on the highway. While driving, participants completed the peripheral detection task (PDT) using LEDs and the N-back task to generate cognitive load. The I-DT (identification by dispersion threshold) algorithm sampled naturalistic gaze fixations during PDTs to cover a broader and continuous spectrum of eccentricity. A generalized Bayesian regression model predicted LED detection probability during the PDT-as a surrogate for peripheral vision-in relation to eccentricity, vehicle speed, driving mode, cognitive load, and age. RESULTS: The model predicted that LED detection probability was high and stable through near-peripheral vision but it declined rapidly beyond 20°-30° eccentricity, showing a narrower useful field over a broader visual field (maximum 70°) during highway driving. Reduced speed (while following another vehicle), cognitive load, and older age were the main factors that degraded the mid-peripheral vision (20°-50°), while using Autopilot had little effect. CONCLUSIONS: Drivers can reliably detect objects through near-peripheral vision, but their peripheral detection degrades gradually due to further eccentricity, foveal demand during low-speed vehicle following, cognitive load, and age. APPLICATIONS: The findings encourage the development of further multivariate computational models to estimate peripheral vision and assess driver situation awareness for crash prevention.

On-road driving impairment following sleep deprivation differs according to age.

Impaired driving performance due to sleep loss is a major contributor to motor-vehicle crashes, fatalities, and serious injuries. As on-road, fully-instrumented studies of drowsy driving have largely focused on young drivers, we examined the impact of sleep loss on driving performance and physiological drowsiness in both younger and older drivers of working age. Sixteen 'younger' adults (M = 24.3 ± 3.1 years [21-33 years], 9 males) and seventeen 'older' adults (M = 57.3 ± 5.2, [50-65 years], 9 males) undertook two 2 h drives on a closed-loop track in an instrumented vehicle with a qualified instructor following (i) 8 h sleep opportunity the night prior (well-rested), and (ii) after 29-h of total sleep deprivation (TSD). Following TSD, both age groups displayed increased subjective sleepiness and lane departures (p < 0.05), with younger drivers exhibiting 7.37 × more lane departures, and 11 × greater risk of near crash events following sleep loss. While older drivers exhibited a 3.5 × more lane departures following sleep loss (p = 0.008), they did not have a significant increase in near-crash events (3/34 drives). Compared to older adults, younger adults had 3.1 × more lane departures (p = < 0.001), and more near crash events (79% versus 21%, p = 0.007). Ocular measures of drowsiness, including blink duration, number of long eye closures and PERCLOS increased following sleep loss for younger adults only (p < 0.05). These results suggest that for older working-aged adults, driving impairments observed following sleep loss may not be due to falling asleep. Future work should examine whether this is attributed to other consequences of sleep loss, such as inattention or distraction from the road.

The impact of 7-hour and 11-hour rest breaks between shifts on heavy vehicle truck drivers' sleep, alertness and naturalistic driving performance.

BACKGROUND: An inadequate rest break between shifts may contribute to driver sleepiness. This study assessed whether extending the major rest break between shifts from 7-hours (Australian industry standard) to 11-hours, improved drivers' sleep, alertness and naturalistic driving performance. METHODS: 17 heavy vehicle drivers (16 male) were recruited to complete two conditions. Each condition comprised two 13-hour shifts, separated by either a 7- or 11-hour rest break. The initial 13-hour shift was the drivers' regular work. The rest break and following 13-hour shift were simulated. The simulated shift included 5-hours of naturalistic driving with measures of subjective sleepiness, physiological alertness (ocular and electroencephalogram) and performance (steering and lane departures). RESULTS: 13 drivers provided useable data. Total sleep during the rest break was greater in the 11-hour than the 7-hour condition (median hours [25th to 75th percentile] 6.59 [6.23, 7.23] vs. 5.07 [4.46, 5.38], p = 0.008). During the simulated shift subjective sleepiness was marginally better for the 11-hour condition (mean Karolinska Sleepiness Scale [95th CI] = 4.52 [3.98, 5.07] vs. 5.12 [4.56, 5.68], p = 0.009). During the drive, ocular and vehicle metrics were improved for the 11-hour condition (p<0.05). Contrary to expectations, mean lane departures p/hour were increased during the 11-hour condition (1.34 [-0.38,3.07] vs. 0.63 [-0.2,1.47], p = 0.027). CONCLUSIONS: Extending the major rest between shifts substantially increases sleep duration and has a modest positive impact on driver alertness and performance. Future work should replicate the study in a larger sample size to improve generalisability and assess the impact of consecutive 7-hour major rest breaks.

A pre-drive ocular assessment predicts alertness and driving impairment: A naturalistic driving study in shift workers.

Sleepiness is a major contributor to motor vehicle crashes and shift workers are particularly vulnerable. There is currently no validated objective field-based measure of sleep-related impairment prior to driving. Ocular parameters are promising markers of continuous driver alertness in laboratory and track studies, however their ability to determine fitness-to-drive in naturalistic driving is unknown. This study assessed the efficacy of a pre-drive ocular assessment for predicting sleep-related impairment in naturalistic driving, in rotating shift workers. Fifteen healthcare workers drove an instrumented vehicle for 2 weeks, while working a combination of day, evening and night shifts. The vehicle monitored lane departures and behavioural microsleeps (blinks >500 ms) during the drive. Immediately prior to driving, ocular parameters were assessed with a 4-min test. Lane departures and behavioural microsleeps occurred on 17.5 % and 10 % of drives that had pre-drive assessments, respectively. Pre-drive blink duration significantly predicted behavioural microsleeps and showed promise for predicting lane departures (AUC = 0.79 and 0.74). Pre-drive percentage of time with eyes closed had high accuracy for predicting lane departures and behavioural microsleeps (AUC = 0.73 and 0.96), although was not statistically significant. Pre-drive psychomotor vigilance task variables were not statistically significant predictors of lane departures. Self-reported sleep-related and hazardous driving events were significantly predicted by mean blink duration (AUC = 0.65 and 0.69). Measurement of ocular parameters pre-drive predict drowsy driving during naturalistic driving, demonstrating potential for fitness-to-drive assessment in operational environments.

Automated recognition of rear seat occupants' head position using Kinect™ 3D point cloud.

INTRODUCTION: Child occupant safety in motor-vehicle crashes is evaluated using Anthropomorphic Test Devices (ATD) seated in optimal positions. However, child occupants often assume suboptimal positions during real-world driving trips. Head impact to the seat back has been identified as one important injury causation scenario for seat belt restrained, head-injured children (Bohman et al., 2011). There is therefore a need to understand the interaction of children with the Child Restraint System to optimize protection. METHOD: Naturalistic driving studies (NDS) will improve understanding of out-of-position (OOP) trends. To quantify OOP positions, an NDS was conducted. Families used a study vehicle for two weeks during their everyday driving trips. The positions of rear-seated child occupants, representing 22 families, were evaluated. The study vehicle - instrumented with data acquisition systems, including Microsoft Kinect™ V1 - recorded rear seat occupants in 1120 driving 26 trips. Three novel analytical methods were used to analyze data. To assess skeletal tracking accuracy, analysts recorded occurrences where Kinect™ exhibited invalid head recognition among a randomly-selected subset (81 trips). Errors included incorrect target detection (e.g., vehicle headrest) or environmental interference (e.g., sunlight). When head data was present, Kinect™ was correct 41% of the time; two other algorithms - filtering for extreme motion, and background subtraction/head-based depth detection are described in this paper and preliminary results are presented. Accuracy estimates were not possible because of their experimental nature and the difficulty to use a ground truth for this large database. This NDS tested methods to quantify the frequency and magnitude of head positions for rear-seated child occupants utilizing Kinect™ motion-tracking. RESULTS: This study's results informed recent ATD sled tests that replicated observed positions (most common and most extreme), and assessed the validity of child occupant protection on these typical CRS uses. SUMMARY: Optimal protection in vehicles requires an understanding of how child occupants use the rear seat space. This study explored the feasibility of using Kinect™ to log positions of rear seated child occupants. Initial analysis used the Kinect™ system's skeleton recognition and two novel analytical algorithms to log head location. PRACTICAL APPLICATIONS: This research will lead to further analysis leveraging Kinect™ raw data - and other NDS data - to quantify the frequency/magnitude of OOP situations, ATD sled tests that replicate observed positions, and advances in the design and testing of child occupant protection technology.

Naturalistic driving study of rear seat child occupants: Quantification of head position using a Kinect™ sensor.

OBJECTIVE: Restraint performance is evaluated using anthropomorphic test devices (ATDs) positioned in prescribed, optimal seating positions. Anecdotally, humans-children in particular-assume a variety of positions that may affect restraint performance. Naturalistic driving studies (NDSs), where cameras and other data acquisition systems are placed in a vehicle used by participants during their regular transportation, offer means to collect these data. To date, these studies have used conventional video and analysis methods and, thus, analyses have largely been qualitative. This article describes a recently completed NDS of child occupants in which their position was monitored using a Kinect sensor to quantify their head position throughout normal, everyday driving trips. METHODS: A study vehicle was instrumented with a data acquisition system to measure vehicle dynamics, a set of video cameras, and a Kinect sensor providing 3D motion capture at 1 Hz of the rear seat occupants. Participant families used the vehicle for all driving trips over 2 weeks. The child occupants' head position was manually identified via custom software from each Kinect color image. The 3D head position was then extracted and its distribution summarized by seat position (left, rear, center) and restraint type (forward-facing child restraint system [FFCRS], booster seat, seat belt). RESULTS: Data from 18 families (37 child occupants) resulted in 582 trips (with children) for analysis. The average age of the child occupants was 45.6 months and 51% were male. Twenty-five child occupants were restrained in FFCRS, 9 in booster seats, and 3 in seat belts. As restraint type moved from more to less restraint (FFCRS to booster seat to seat belt), the range of fore-aft head position increased: 218, 244, and 340 mm on average, respectively. This observation was also true for left-right movement for every seat position. In general, those in the center seat position demonstrated a smaller range of head positions. CONCLUSIONS: For the first time in a naturalistic setting, the range of head positions for child occupants was quantified. More variability was observed for those restrained in booster seats and seat belts than for those in FFCRS. The role of activities, in particular interactions with electronic devices, on head position was notable; this will be the subject of further analysis in other components of the broader study. These data can lead to solutions for optimal protection for occupants who assume positions that differ from prescribed, optimal testing positions.

Evaluation of a video-based measure of driver heart rate.

INTRODUCTION: Internal driver events such as emotional arousal do not consistently elicit observable behaviors. However, heart rate (HR) offers promise as a surrogate measure for predicting these states in drivers. Imaging photoplethysmography (IPPG) can measure HR from face video recorded in static, indoor settings, but has yet to be examined in an in-vehicle driving environment. METHODS: Participants (N=10) completed an on-road driving task whilst wearing a commercial, chest-strap style heart rate monitor ("baseline"). IPPG was applied to driver face video to estimate HR and the two measures of HR were compared. RESULTS: For 4 of 10 participants, IPPG produced a valid HR signal (±5 BPM of baseline) between 48 and 75% of trip duration. For the remaining participants, IPPG accuracy was poor (<20%). CONCLUSIONS: In-vehicle IPPG is achievable, but significant challenges remain. PRACTICAL APPLICATIONS: The relationship between IPPG accuracy and various confounding factors was quantified for future refinement.

Computer vision and driver distraction: developing a behaviour-flagging protocol for naturalistic driving data.

Naturalistic driving studies (NDS) allow researchers to discreetly observe everyday, real-world driving to better understand the risk factors that contribute to hazardous situations. In particular, NDS designs provide high ecological validity in the study of driver distraction. With increasing dataset sizes, current best practice of manually reviewing videos to classify the occurrence of driving behaviours, including those that are indicative of distraction, is becoming increasingly impractical. Current statistical solutions underutilise available data and create further epistemic problems. Similarly, technical solutions such as eye-tracking often require dedicated hardware that is not readily accessible or feasible to use. A computer vision solution based on open-source software was developed and tested to improve the accuracy and speed of processing NDS video data for the purpose of quantifying the occurrence of driver distraction. Using classifier cascades, manually-reviewed video data from a previously published NDS was reanalysed and used as a benchmark of current best practice for performance comparison. Two software coding systems were developed - one based on hierarchical clustering (HC), and one based on gender differences (MF). Compared to manual video coding, HC achieved 86 percent concordance, 55 percent reduction in processing time, and classified an additional 69 percent of target behaviour not previously identified through manual review. MF achieved 67 percent concordance, a 75 percent reduction in processing time, and classified an additional 35 percent of target behaviour not identified through manual review. The findings highlight the improvements in processing speed and correctly classifying target behaviours achievable through the use of custom developed computer vision solutions. Suggestions for improved system performance and wider implementation are discussed.