Adopting Stimulus Detection Tasks for Cognitive Workload Assessment: Some Considerations.

OBJECTIVE: This article tackles the issue of correct data interpretation when using stimulus detection tasks for determining the operator's workload. BACKGROUND: Stimulus detection tasks are a relative simple and inexpensive means of measuring the operator's state. While stimulus detection tasks may be better geared to measure conditions of high workload, adopting this approach for the assessment of low workload may be more problematic. METHOD: This mini-review details the use of common stimulus detection tasks and their contributions to the Human Factors practice. It also borrows from the conceptual framework of the inverted-U shape model to discuss the issue of data interpretation. RESULTS: The evidence being discussed here highlights a clear limitation of stimulus detection task paradigms. CONCLUSION: There is an inherent risk in using a unidimensional tool like stimulus detection tasks as the primary source of information for determining the operator's psychophysiological state. APPLICATION: Two recommendations are put forward to Human Factors researchers and practitioners dealing with the interpretation conundrum of dealing with stimulus detection tasks.

A Broader Application of the Detection Response Task to Cognitive Tasks and Online Environments.

OBJECTIVE: The present research applied a well-established measure of cognitive workload in driving literature to an in-lab paradigm. We then extended this by comparing the in-lab version of the task to an online version. BACKGROUND: The accurate and objective measurement of cognitive workload is important in many aspects of psychological research. The detection response task (DRT) is a well-validated method for measuring cognitive workload that has been used extensively in applied tasks, for example, to investigate the effects of phone usage or passenger conversation on driving, but has been used sparingly outside of this field. METHOD: The study investigated whether the DRT could be used to measure cognitive workload in tasks more commonly used in experimental cognitive psychology and whether this application could be extended to online environments. We had participants perform a multiple object tracking (MOT) task while simultaneously performing a DRT. We manipulated the cognitive load of the MOT task by changing the number of dots to be tracked. RESULTS: Measurements from the DRT were sensitive to changes in the cognitive load, establishing the efficacy of the DRT for experimental cognitive tasks in lab-based situations. This sensitivity continued when applied to an online environment (our code for the online DRT implementation is freely available at https://osf.io/dc39s/), though to a reduced extent compared to the in-lab situation. CONCLUSION: The MOT task provides an effective manipulation of cognitive workload. The DRT is sensitive to changes in workload across a range of settings and is suitable to use outside of driving scenarios, as well as via online delivery. APPLICATION: Methodology shows how the DRT could be used to measure sources of cognitive workload in a range of human factors contexts.

On the Cost of Detection Response Task Performance on Cognitive Load.

OBJECTIVE: This study investigates the cost of detection response task performance on cognitive load. BACKGROUND: Measuring system operator's cognitive load is a foremost challenge in human factors and ergonomics. The detection response task is a standardized measure of cognitive load. It is hypothesized that, given its simple reaction time structure, it has no cost on cognitive load. We set out to test this hypothesis by utilizing pupil diameter as an alternative metric of cognitive load. METHOD: Twenty-eight volunteers completed one of four experimental tasks with increasing levels of cognitive demand (control, 0-back, 1-back, and 2-back) with or without concurrent DRT performance. Pupil diameter was selected as nonintrusive metric of cognitive load. Self-reported workload was also recorded. RESULTS: A significant main effect of DRT presence was found for pupil diameter and self-reported workload. Larger pupil diameter was found when the n-back task was performed concurrently with the DRT, compared to no-DRT conditions. Consistent results were found for mental workload ratings and n-back performance. CONCLUSION: Results indicate that DRT performance produced an added cost on cognitive load. The magnitude of the change in pupil diameter was comparable to that observed when transitioning from a condition of low task load to one where the 2-back was performed. The significant increase in cognitive load accompanying DRT performance was also reflected in higher self-reported workload. APPLICATION: DRT is a valuable tool to measure operator's cognitive load. However, these results advise caution when discounting it as cost-free metric with no added burden on operator's cognitive resources.

The Effects of Increased Visual Information on Cognitive Workload in a Helicopter Simulator.

OBJECTIVE: To test the effects of enhanced display information ("symbology") on cognitive workload in a simulated helicopter environment, using the detection response task (DRT). BACKGROUND: Workload in highly demanding environments can be influenced by the amount of information given to the operator and consequently it is important to limit potential overload. METHODS: Participants (highly trained military pilots) completed simulated helicopter flights, which varied in visual conditions and the amount of information given. During these flights, participants also completed a DRT as a measure of cognitive workload. RESULTS: With more visual information available, pilots' landing accuracy was improved across environmental conditions. The DRT is sensitive to changes in cognitive workload, with workload differences shown between environmental conditions. Increasing symbology appeared to have a minor effect on workload, with an interaction effect of symbology and environmental condition showing that symbology appeared to moderate workload. CONCLUSION: The DRT is a useful workload measure in simulated helicopter settings. The level of symbology-moderated pilot workload. The increased level of symbology appeared to assist pilots' flight behavior and landing ability. Results indicate that increased symbology has benefits in more difficult scenarios. APPLICATIONS: The DRT is an easily implemented and effective measure of cognitive workload in a variety of settings. In the current experiment, the DRT captures the increased workload induced by varying the environmental conditions, and provides evidence for the use of increased symbology to assist pilots.

Allocation of Driver Attention for Varying In-Vehicle System Modalities.

OBJECTIVE: This paper examines drivers' allocation of attention using response time to a tactile detection response task (TDRT) while interacting with an in-vehicle information system (IVIS) over time. BACKGROUND: Longer TDRT response time is associated with higher cognitive workload. However, it is not clear what role is assumed by the human and system in response to varying in-vehicle environments over time. METHOD: A driving simulator study with 24 participants was conducted with a restaurant selection task of two difficulty levels (easy and hard) presented in three modalities (audio only, visual only, hybrid). A linear mixed-effects model was applied to identify factors that affect TDRT response time. A nonparametric time-series model was also used to explore the visual attention allocation under the hybrid mode over time. RESULTS: The visual-only mode significantly increased participants' response time compared with the audio-only mode. Females took longer to respond to the TDRT when engaged with an IVIS. The study showed that participants tend to use the visual component more toward the end of the easy tasks, whereas the visual mode was used more at the beginning of the harder tasks. CONCLUSION: The visual-only mode of the IVIS increased drivers' cognitive workload when compared with the auditory-only mode. Drivers showed different visual attention allocation during the easy and hard restaurant selection tasks in the hybrid mode. APPLICATION: The findings can help guide the design of automotive user interfaces and help manage cognitive workload.

Cognitive Load and Situation Awareness for Soldiers: Effects of Message Presentation Rate and Sensory Modality.

OBJECTIVE: We sought to determine the influence of message presentation rate (MPR) and sensory modality on soldier cognitive load. BACKGROUND: Soldiers commonly communicate tactical information by radio. The Canadian Army is equipping soldiers with a battle management system (BMS), which also allows them to communicate by text. METHOD: We varied presentation modality (auditory vs. visual) and MPR (fast or slow) in an experiment involving a tactical scenario. Participants (soldiers) received messages and periodically provided situation reports to higher level command, and the scored reports were used to provide a measure of situation awareness (SA). The detection response task (DRT) and NASA-TLX were used to measure cognitive load. RESULTS: The fast MPR reduced DRT accuracy and increased response times relative to slow MPR. The NASA-TLX results also showed higher subjective workload ratings for several subscales with fast MPR. Messages presented visually produced greater cognitive load, with slower DRT response times for the visual than the auditory condition. SA scores were higher with slower MPR and auditory presentation. There was no statistical interaction of presentation modality and rate for any measure. CONCLUSION: Fast MPR and visual presentation increased cognitive load and degraded SA. APPLICATION: These findings show that the DRT can be used to measure workload effectively in a tactical military context and that the method of information presentation affects how soldiers process information in a BMS.

Optic Flow and Tunnel Vision in the Detection Response Task.

OBJECTIVE: In a driving simulator, a backwards counting task, a simple steering task, and a fully autonomous driving task were applied to study the independent effects of cognitive load, visual-cognitive-manual load, and optic flow on visual detection response task (vDRT) performance. The study was designed to increase the understanding of the processes underlying vDRT effects. BACKGROUND: The tunnel vision effect induced by a "steering while driving" task found in a previous study was investigated further in this experiment. METHOD: Stimulus eccentricity and conspicuity were applied as within-subjects factors. RESULTS: Cognitive load, visual-cognitive-manual load, and optic flow all resulted in increased vDRT response time (RT). Cognitive load and visual-cognitive-manual load both increased RT but revealed no interaction of task by stimulus eccentricity. However, optic flow resulted in a task by stimulus eccentricity interaction on vDRT RT that was evidence of a tunnel vision effect. CONCLUSION: The results suggested that optic flow may be a factor responsible for tunnel vision while driving, although this does not support the tunnel vision model because it is unrelated to workload. However, the results supported the general interference model for cognitive workload. APPLICATION: The results have implications for the diagnosticity of the vDRT. During driving tasks, tunnel vision effects may occur as a result of optic flow, and these effects are unrelated to workload.

Detection-Response Task-Uses and Limitations.

The Detection-Response Task is a method for assessing the attentional effects of cognitive load in a driving environment. Drivers are presented with a sensory stimulus every 3-5 s, and are asked to respond to it by pressing a button attached to their finger. Response times and hit rates are interpreted as indicators of the attentional effect of cognitive load. The stimuli can be visual, tactile and auditory, and are chosen based on the type of in-vehicle system or device that is being evaluated. Its biggest disadvantage is that the method itself also affects the driver's performance and secondary task completion times. Nevertheless, this is an easy to use and implement method, which allows relevant assessment and evaluation of in-vehicle systems. By following the recommendations and taking into account its limitations, researchers can obtain reliable and valuable results on the attentional effects of cognitive load on drivers.

The Effects of Cognitive and Visual Workload on Peripheral Detection in the Detection Response Task.

OBJECTIVE: The independent effects of cognitive and visual load on visual Detection Response Task (vDRT) reaction times were studied in a driving simulator by performing a backwards counting task and a simple driving task that required continuous focused visual attention to the forward view of the road. The study aimed to unravel the attentional processes underlying the Detection Response Task effects. BACKGROUND: The claim of previous studies that performance degradation on the vDRT is due to a general interference instead of visual tunneling was challenged in this experiment. METHOD: vDRT stimulus eccentricity and stimulus conspicuity were applied as within-subject factors. RESULTS: Increased cognitive load and visual load both resulted in increased response times (RTs) on the vDRT. Cognitive load increased RT but revealed no task by stimulus eccentricity interaction. However, effects of visual load on RT showed a strong task by stimulus eccentricity interaction under conditions of low stimulus conspicuity. Also, more experienced drivers performed better on the vDRT while driving. CONCLUSION: This was seen as evidence for a differential effect of cognitive and visual workload. The results supported the tunnel vision model for visual workload, where the sensitivity of the peripheral visual field reduced as a function of visual load. However, the results supported the general interference model for cognitive workload. APPLICATION: This has implications for the diagnosticity of the vDRT: The pattern of results differentiated between visual task load and cognitive task load. It also has implications for theory development and workload measurement for different types of tasks.

Comparison of concurrent cognitive load measures during n-back tasks.

The cognitive load experienced by humans is an important factor affecting their performance. Cognitive overload or underload may result in suboptimal human performance and may compromise safety in emerging human-in-the-loop systems. In driving, cognitive overload, due to various secondary tasks, such as texting, results in driver distraction. On the other hand, cognitive underload may result in fatigue. In automated manufacturing systems, a distracted operator may be prone to muscle injuries. Similar outcomes are possible in many other fields of human performance such as aviation, healthcare, and learning environments. The challenge with such human-centred applications is that the cognitive load is not directly measurable. Only the change in cognitive load is measured indirectly through various physiological, behavioural, performance-based and subjective means. A method to objectively assess the performance of such diverse measures of cognitive load is lacking in the literature. In this paper, a performance metric for the comparison of different measures to determine the cognitive workload is proposed in terms of the signal-to-noise ratio. Using this performance metric, several measures of cognitive load, that fall under the four broad groups were compared on the same scale for their ability to measure changes in cognitive load. Using the proposed metrics, the cognitive load measures were compared based on data collected from 28 participants while they underwent n-back tasks of varying difficulty. The results show that the proposed performance evaluation method can be useful to individually assess different measures of cognitive load.

No Difference in Arousal or Cognitive Demands Between Manual and Partially Automated Driving: A Multi-Method On-Road Study.

INTRODUCTION: Partial driving automation is not always reliable and requires that drivers maintain readiness to take over control and manually operate the vehicle. Little is known about differences in drivers' arousal and cognitive demands under partial automation and how it may make it difficult for drivers to transition from automated to manual modes. This research examined whether there are differences in drivers' arousal and cognitive demands during manual versus partial automation driving. METHOD: We compared arousal (using heart rate) and cognitive demands (using the root mean square of successive differences in normal heartbeats; RMSSD, and Detection Response Task; DRT) while 39 younger (M = 28.82 years) and 32 late-middle-aged (M = 52.72 years) participants drove four partially automated vehicles (Cadillac, Nissan Rogue, Tesla, and Volvo) on interstate highways. If compared to manual driving, drivers' arousal and cognitive demands were different under partial automation, then corresponding differences in heart rate, RMSSD, and DRT would be expected. Alternatively, if drivers' arousal and cognitive demands were similar in manual and partially automated driving, no difference in the two driving modes would be expected. RESULTS: Results suggest no significant differences in heart rate, RMSSD, or DRT reaction time performance between manual and partially automated modes of driving for either younger or late-middle-aged adults across the four test vehicles. A Bayes Factor analysis suggested that heart rate, RMSSD, and DRT data showed extreme evidence in favor of the null hypothesis. CONCLUSION: This novel study conducted on real roads with a representative sample provides important evidence of no difference in arousal and cognitive demands. Younger and late-middle-aged motorists who are new to partial automation are able to maintain arousal and cognitive demands comparable to manual driving while using the partially automated technology. Drivers who are more experienced with partially automated technology may respond differently than those with limited prior experience.

Real-time prediction of short-timescale fluctuations in cognitive workload.

Human operators often experience large fluctuations in cognitive workload over seconds timescales that can lead to sub-optimal performance, ranging from overload to neglect. Adaptive automation could potentially address this issue, but to do so it needs to be aware of real-time changes in operators' spare cognitive capacity, so it can provide help in times of peak demand and take advantage of troughs to elicit operator engagement. However, it is unclear whether rapid changes in task demands are reflected in similarly rapid fluctuations in spare capacity, and if so what aspects of responses to those demands are predictive of the current level of spare capacity. We used the ISO standard detection response task (DRT) to measure cognitive workload approximately every 4 s in a demanding task requiring monitoring and refueling of a fleet of simulated unmanned aerial vehicles (UAVs). We showed that the DRT provided a valid measure that can detect differences in workload due to changes in the number of UAVs. We used cross-validation to assess whether measures related to task performance immediately preceding the DRT could predict detection performance as a proxy for cognitive workload. Although the simple occurrence of task events had weak predictive ability, composite measures that tapped operators' situational awareness with respect to fuel levels were much more effective. We conclude that cognitive workload does vary rapidly as a function of recent task events, and that real-time predictive models of operators' cognitive workload provide a potential avenue for automation to adapt without an ongoing need for intrusive workload measurements.

Response time and eye tracking datasets for activities demanding varying cognitive load.

The dataset contains the following three measures that are widely used to determine cognitive load in humans: Detection Response Task - response time, pupil diameter, and eye gaze. These measures were recorded from 28 participants while they underwent tasks that are designed to permeate three different cognitive difficulty levels. The dataset will be useful to those researchers who seek to employ low cost, non-invasive sensors to detect cognitive load in humans and to develop algorithms for human-system automation. One such application is found in Advanced Driver Assistance Systems where eye-trackers are employed to monitor the alertness of the drivers. The dataset would also be helpful to researchers who are interested in employing machine learning algorithms to develop predictive models of humans for applications in human-machine system automation. The data is collected by the authors at the Department of Electrical & Computer Engineering in collaboration with the Faculty of Human Kinetics at the University of Windsor under the guidance of their Research Ethics Board.

The long road home from distraction: Investigating the time-course of distraction recovery in driving.

Driver distraction is a leading cause of accidents. While there has been significant research examining driver performance during a distraction, there has been less focus on how much time is required to recover performance following a distraction. To address this issue, participants in the current study completed a simulated 40-min drive while being presented with distractions. Distractions were followed by a visual Detection Response Task (DRT) to assess participants' resource availability and potential capacity to respond to hazards, as well as continuous measures of driving performance including their ability to maintain a consistent speed and lane position. We examined recovery for a 40 s period following three types of distraction: cognitive only, cognitive + visual, and cognitive + visual + manual. Since safe driving requires cognitive, visual, and manual resources, we expected recovery to take longer when the distraction involved more of these resources. Consistent with this, each additional level of distraction further slowed DRT response times and increased speed variability during 0-10 s post-distraction. However, DRT accuracy was equally impaired for all conditions during 0-20 s post-distraction, while lane position maintenance from 0 to 10 s post-distraction was only impaired when the distraction included a manual component. In addition, while participants in all three conditions exhibited some degree of post-distraction impairment, only those in the cognitive + visual + manual condition reduced their speed during the time when distracted, suggesting drivers show limited awareness of the potential persistent consequences of distraction.

The relative impact of smartwatch and smartphone use while driving on workload, attention, and driving performance.

The impact of using a smartwatch to initiate phone calls on driver workload, attention, and performance was compared to smartphone visual-manual (VM) and auditory-vocal (AV) interfaces. In a driving simulator, 36 participants placed calls using each method. While task time and number of glances were greater for AV calling on the smartwatch vs. smartphone, remote detection task (R-DRT) responsiveness, mean single glance duration, percentage of long duration off-road glances, total off-road glance time, and percent time looking off-road were similar; the later metrics were all significantly higher for the VM interface vs. AV methods. Heart rate and skin conductance were higher during phone calling tasks than "just driving", but did not consistently differentiate calling method. Participants exhibited more erratic driving behavior (lane position and major steering wheel reversals) for smartphone VM calling compared to both AV methods. Workload ratings were lower for AV calling on both devices vs. VM calling.

On the selection of stimulus for the auditory variant of the detection response task method for driving experiments.

OBJECTIVES: The detection response task (DRT) is a method for measuring attentional effects of secondary tasks on a driver's cognitive load by measuring response times and hit rates to different types of stimuli as indirect indicators of increased cognitive load. ISO 17488 (International Organization for Standardization 2016) only provides guidelines for the technical implementation and measurement methods for the visual and tactile versions (use of visual and tactile stimuli) of the DRT method. This article presents a study with the goal of finding the most appropriate auditory stimulus for the implementation of an auditory version of the DRT method. METHODS: This article presents the results of an experiment in which responses to 7 different auditory DRT stimuli-varying in frequency-were compared while inducing users' cognitive load with a modified n-back task. The experiment was conducted in a surrogate driving environment and in a within-subject design. Response times, hit rates, and secondary task performances were observed as indicators of increased cognitive load. RESULTS: Significantly shorter response times were found for the white noise signal compared to single-frequency signals. However, the largest differences in response times, for trials without and with a cognitive task, were found for 4- and 8-kHz single-frequency signals. No significant differences were found for hit rates and secondary task performances between the different stimuli. CONCLUSIONS: Consistent significant differences in response times for all tested stimuli prove that the auditory DRT variant is also sensitive to changes in cognitive load. The mean increase in response times of more than 25% for 4- and 8-kHz signals for trials with a cognitive task compared to trials without one indicates that one of these signals could be used as a potential auditory stimulus for the auditory DRT variant.

Validation of auditory detection response task method for assessing the attentional effects of cognitive load.

OBJECTIVES: There are 3 standardized versions of the Detection Response Task (DRT), 2 using visual stimuli (remote DRT and head-mounted DRT) and one using tactile stimuli. In this article, we present a study that proposes and validates a type of auditory signal to be used as DRT stimulus and evaluate the proposed auditory version of this method by comparing it with the standardized visual and tactile version. METHODS: This was a within-subject design study performed in a driving simulator with 24 participants. Each participant performed 8 2-min-long driving sessions in which they had to perform 3 different tasks: driving, answering to DRT stimuli, and performing a cognitive task (n-back task). Presence of additional cognitive load and type of DRT stimuli were defined as independent variables. DRT response times and hit rates, n-back task performance, and pupil size were observed as dependent variables. RESULTS: Significant changes in pupil size for trials with a cognitive task compared to trials without showed that cognitive load was induced properly. Each DRT version showed a significant increase in response times and a decrease in hit rates for trials with a secondary cognitive task compared to trials without. Similar and significantly better results in differences in response times and hit rates were obtained for the auditory and tactile version compared to the visual version. There were no significant differences in performance rate between the trials without DRT stimuli compared to trials with and among the trials with different DRT stimuli modalities. CONCLUSIONS: The results from this study show that the auditory DRT version, using the signal implementation suggested in this article, is sensitive to the effects of cognitive load on driver's attention and is significantly better than the remote visual and tactile version for auditory-vocal cognitive (n-back) secondary tasks.

Comparing the demands of destination entry using Google Glass and the Samsung Galaxy S4 during simulated driving.

The relative impact of using a Google Glass based voice interface to enter a destination address compared to voice and touch-entry methods using a handheld Samsung Galaxy S4 smartphone was assessed in a driving simulator. Voice entry (Google Glass and Samsung) had lower subjective workload ratings, lower standard deviation of lateral lane position, shorter task durations, faster remote Detection Response Task (DRT) reaction times, lower DRT miss rates, and resulted in less time glancing off-road than the primary visual-manual interaction with the Samsung Touch interface. Comparing voice entry methods, using Google Glass took less time, while glance metrics and reaction time to DRT events responded to were similar. In contrast, DRT miss rate was higher for Google Glass, suggesting that drivers may be under increased distraction levels but for a shorter period of time; whether one or the other equates to an overall safer driving experience is an open question.

Modeling cognitive load effects of conversation between a passenger and driver.

Cognitive load from secondary tasks is a source of distraction causing injuries and fatalities on the roadway. The Detection Response Task (DRT) is an international standard for assessing cognitive load on drivers' attention that can be performed as a secondary task with little to no measurable effect on the primary driving task. We investigated whether decrements in DRT performance were related to the rate of information processing, levels of response caution, or the non-decision processing of drivers. We had pairs of participants take part in the DRT while performing a simulated driving task, manipulated cognitive load via the conversation between driver and passenger, and observed associated slowing in DRT response time. Fits of the single-bound diffusion model indicated that slowing was mediated by an increase in response caution. We propose the novel hypothesis that, rather than the DRT's sensitivity to cognitive load being a direct result of a loss of information processing capacity to other tasks, it is an indirect result of a general tendency to be more cautious when making responses in more demanding situations.

Sensitivity evaluation of the visual, tactile, and auditory detection response task method while driving.

OBJECTIVES: In this article, we evaluate the sensitivity to cognitive load of 3 versions of the Detection Response Task method (DRT), proposed in ISO Draft Standard DIS-17488. METHODS: We present a user study with 30 participants in which we compared the sensitivity to cognitive load of visual, audio, and tactile DRT in a simulated driving environment. The amount of cognitive load was manipulated with secondary n-back tasks at 2 levels of difficulty (0-back and 1-back). We also explored whether the DRT method is least sensitive to cognitive load when the stimuli and secondary task are of the same modality. For this purpose, we used 3 forms to present the n-back task stimuli: visual, audio, and tactile. Responses to the task were always vocal. The experiment was based on a between-subject design (the DRT modalities) with 2 levels of within-subject design study (modalities and difficulty of the secondary n-back tasks). The participants' primary task in the study was to drive safely, and a second priority was to answer to DRT stimuli and perform secondary tasks. RESULTS: The results indicate that all 3 versions of the DRT tested were sensitive to detecting the difference in cognitive load between the reference driving period and driving and engaging in the secondary tasks. Only the visual DRT discriminated between the 0-back and 1-back conditions on mean response time. Contrary to expectations, no interaction was observed between DRT modality and the stimuli modality used for presentation of the secondary tasks. CONCLUSIONS: None of the 3 methods of presenting DRT stimuli showed a consistent advantage in sensitivity in differentiating multiple levels of cognitive load if all response times, hit rates, and secondary task performance are considered. If only response time is considered, the visual presentation of the DRT stimulus used in this study showed some advantages. In interpreting these data, it should be noted that the methods of DRT stimulus presentation varied somewhat from the currently proposed draft ISO standard and it is possible that the relative salience level of the visual DRT stimulus influenced the findings. It is further suggested that more than 2 levels of difficulty of the n-back task should be considered for further investigation of the relative sensitivity of different DRT stimuli modalities. Parameters that indicate change in cognitive load (response time, hit rate, task performance) should be analyzed together in assessing the overall impact on the driver and not individually, in order to obtain a fuller insight of the assessed cognitive load.