The Effect of Cognitive Load on Auditory Susceptibility During Automated Driving.

OBJECTIVE: We experimentally test the effect of cognitive load on auditory susceptibility during automated driving. BACKGROUND: In automated vehicles, auditory alerts are frequently used to request human intervention. To ensure safe operation, human drivers need to be susceptible to auditory information. Previous work found reduced susceptibility during manual driving and in a lesser amount during automated driving. However, in practice, drivers also perform nondriving tasks during automated driving, of which the associated cognitive load may further reduce susceptibility to auditory information. We therefore study the effect of cognitive load during automated driving on auditory susceptibility. METHOD: Twenty-four participants were driven in a simulated automated car. Concurrently, they performed a task with two levels of cognitive load: repeat a noun or generate a verb that expresses the use of this noun. Every noun was followed by a probe stimulus to elicit a neurophysiological response: the frontal P3 (fP3), which is a known indicator for the level of auditory susceptibility. RESULTS: The fP3 was significantly lower during automated driving with cognitive load compared with without. The difficulty level of the cognitive task (repeat or generate) showed no effect. CONCLUSION: Engaging in other tasks during automated driving decreases auditory susceptibility as indicated by a reduced fP3. APPLICATION: Nondriving task can create additional cognitive load. Our study shows that performing such tasks during automated driving reduces the susceptibility for auditory alerts. This can inform designers of semi-automated vehicles (SAE levels 3 and 4), where human intervention might be needed.

Training eye movements for visual search in individuals with macular degeneration.

We report a method to train individuals with central field loss due to macular degeneration to improve the efficiency of visual search. Our method requires participants to make a same/different judgment on two simple silhouettes. One silhouette is presented in an area that falls within the binocular scotoma while they are fixating the center of the screen with their preferred retinal locus (PRL); the other silhouette is presented diametrically opposite within the intact visual field. Over the course of 480 trials (approximately 6 hr), we gradually reduced the amount of time that participants have to make a saccade and judge the similarity of stimuli. This requires that they direct their PRL first toward the stimulus that is initially hidden behind the scotoma. Results from nine participants show that all participants could complete the task faster with training without sacrificing accuracy on the same/different judgment task. Although a majority of participants were able to direct their PRL toward the initially hidden stimulus, the ability to do so varied between participants. Specifically, six of nine participants made faster saccades with training. A smaller set (four of nine) made accurate saccades inside or close to the target area and retained this strategy 2 to 3 months after training. Subjective reports suggest that training increased awareness of the scotoma location for some individuals. However, training did not transfer to a different visual search task. Nevertheless, our study suggests that increasing scotoma awareness and training participants to look toward their scotoma may help them acquire missing information.

Stop before you saccade: Looking into an artificial peripheral scotoma.

We investigated whether adults with healthy vision can move their eyes toward an informative target area that is initially hidden by a gaze-contingent scotoma in the periphery when they are under time pressure. In the experimental task, participants had to perform an object-comparison task requiring a same-different judgment about two silhouettes. One silhouette was visible, whereas the other was hidden under the scotoma. Despite time pressure and the presence of the visible silhouette, most participants were able to move their eyes toward the informative region to reveal the hidden silhouette. Saccades to the hidden stimulus occurred when the visible stimulus was presented directly opposite in either fixed or variable locations and when the visible stimulus was presented at an adjacent location. Older participants were also able to perform this task. First saccades in the direction of the hidden stimulus had longer latencies compared with saccades toward the visible stimulus. This suggests the use of a deliberate, nonreflexive saccade strategy ("stop before you saccade"). A subset of participants occasionally made curved saccades that were aimed first toward the visible stimulus and then toward the hidden stimulus. We discuss the implications of our findings for patients who have a biological scotoma, for example, in macular degeneration.

Action Sounds Informing Own Body Perception Influence Gender Identity and Social Cognition.

Sensory information can temporarily affect mental body representations. For example, in Virtual Reality (VR), visually swapping into a body with another sex can temporarily alter perceived gender identity. Outside of VR, real-time auditory changes to walkers' footstep sounds can affect perceived body weight and masculinity/femininity. Here, we investigate whether altered footstep sounds also impact gender identity and relation to gender groups. In two experiments, cisgender participants (26 females, 26 males) walked with headphones which played altered versions of their own footstep sounds that sounded more typically male or female. Baseline and post-intervention measures quantified gender identity [Implicit Association Test (IAT)], relation to gender groups [Inclusion of the Other-in-the-Self (IOS)], and perceived masculinity/femininity. Results show that females felt more feminine and closer to the group of women (IOS) directly after walking with feminine sounding footsteps. Similarly, males felt more feminine after walking with feminine sounding footsteps and associated themselves relatively stronger with "female" (IAT). The findings suggest that gender identity is temporarily malleable through auditory-induced own body illusions. Furthermore, they provide evidence for a connection between body perception and an abstract representation of the Self, supporting the theory that bodily illusions affect social cognition through changes in the self-concept.

The influence of cognitive load on susceptibility to audio.

In this study we evaluate how cognitive load affects susceptibility to auditory signals. Previous research has used the frontal P3 (fP3) event related potential response to auditory novel stimuli as an index for susceptibility to auditory signals. This work demonstrated that tasks that induce cognitive load such as visual and manual tasks, reduced susceptibility. It is however unknown whether cognitive load without visual or manual components also reduces susceptibility. To investigate this, we induced cognitive load by means of the verb generation task, in which participants need to think about a verb that matches a noun. The susceptibility to auditory signals was measured by recording the event related potential in response to a successively presented oddball probe stimulus at 3 different inter-stimulus intervals, 0 ms, 200 ms or 400 ms after the offset of the noun from the verb generation task. An additional control baseline condition, in which oddball response was probed without a verb generation task, was also included. Results show that the cognitive load associated with the verb task reduces fP3 response (and associated auditory signal susceptibility) compared to baseline, independent of presentation interval. This suggests that not only visual and motor processing, but also cognitive load without visual or manual components, can reduce susceptibility to auditory signals and alerts.

The influence of rewards on (sub-)optimal interleaving.

We investigate how the rewards of individual tasks dictate a priori how easy it is to interleave two discrete tasks efficiently, and whether people then interleave efficiently. Previous research found that people vary in their ability to interleave efficiently. Less attention has been given to whether it was realistic to expect efficient interleaving, given the reward rate of each of the involved tasks. Using a simulation model, we demonstrate how the rewards of individual tasks lead to different dual-task interleaving scenarios. We identify three unique dual-task scenarios. In easy scenarios, many strategies for time division between tasks can achieve optimal performance. This gives great opportunity to optimize performance, but also leads to variation in the applied strategies due to a lack of pressure to settle on a small set of optimal strategies. In difficult scenarios, the optimal strategy is hard to identify, therefore giving little opportunity to optimize. Finally, constrained scenarios have a well-defined prediction of the optimal strategy. It gives a narrow prediction, which limits the options to achieve optimal scores, yet given the structure people are able to optimize their strategies. These scenarios are therefore best to test people's general capability of optimizing interleaving. We report three empirical studies that test these hypotheses. In each study, participants interleave between two identical discrete tasks, that differ only in the underlying reward functions and the combined result (easy, difficult, or constrained scenario). Empirical results match the theoretical pattern as predicted by simulation models. Implications for theory and practice are discussed.

Dividing Attention Between Tasks: Testing Whether Explicit Payoff Functions Elicit Optimal Dual-Task Performance.

We test people's ability to optimize performance across two concurrent tasks. Participants performed a number entry task while controlling a randomly moving cursor with a joystick. Participants received explicit feedback on their performance on these tasks in the form of a single combined score. This payoff function was varied between conditions to change the value of one task relative to the other. We found that participants adapted their strategy for interleaving the two tasks, by varying how long they spent on one task before switching to the other, in order to achieve the near maximum payoff available in each condition. In a second experiment, we show that this behavior is learned quickly (within 2-3 min over several discrete trials) and remained stable for as long as the payoff function did not change. The results of this work show that people are adaptive and flexible in how they prioritize and allocate attention in a dual-task setting. However, it also demonstrates some of the limits regarding people's ability to optimize payoff functions.

Susceptibility to audio signals during autonomous driving.

We investigate how susceptible human drivers are to auditory signals in three situations: when stationary, when driving, or when being driven by an autonomous vehicle. Previous research has shown that human susceptibility is reduced when driving compared to when being stationary. However, it is not known how susceptible humans are under autonomous driving conditions. At the same time, good susceptibility is crucial under autonomous driving conditions, as such systems might use auditory signals to communicate a transition of control from the automated vehicle to the human driver. We measured susceptibility using a three-stimulus auditory oddball paradigm while participants experienced three driving conditions: stationary, autonomous, or driving. We studied susceptibility through the frontal P3 (fP3) Electroencephalography Event-Related Potential response (EEG ERP response). Results show that the fP3 component is reduced in autonomous compared to stationary conditions, but not as strongly as when participants drove themselves. In addition, the fP3 component is further reduced when the oddball task does not require a response (i.e., in a passive condition, versus active). The implication is that, even in a relatively simple autonomous driving scenario, people's susceptibility of auditory signals is not as high as would be beneficial for responding to auditory stimuli.

Strategic Adaptation to Task Characteristics, Incentives, and Individual Differences in Dual-Tasking.

We investigate how good people are at multitasking by comparing behavior to a prediction of the optimal strategy for dividing attention between two concurrent tasks. In our experiment, 24 participants had to interleave entering digits on a keyboard with controlling a randomly moving cursor with a joystick. The difficulty of the tracking task was systematically varied as a within-subjects factor. Participants were also exposed to different explicit reward functions that varied the relative importance of the tracking task relative to the typing task (between-subjects). Results demonstrate that these changes in task characteristics and monetary incentives, together with individual differences in typing ability, influenced how participants choose to interleave tasks. This change in strategy then affected their performance on each task. A computational cognitive model was used to predict performance for a wide set of alternative strategies for how participants might have possibly interleaved tasks. This allowed for predictions of optimal performance to be derived, given the constraints placed on performance by the task and cognition. A comparison of human behavior with the predicted optimal strategy shows that participants behaved near optimally. Our findings have implications for the design and evaluation of technology for multitasking situations, as consideration should be given to the characteristics of the task, but also to how different users might use technology depending on their individual characteristics and their priorities.

Sharing a driver's context with a caller via continuous audio cues to increase awareness about driver state.

In an experiment using a driving simulator we investigated whether sharing information of a driver's context with a remote caller via continuous audio cues can make callers more aware of the driving situation. Increased awareness could potentially help in making the conversation less distracting. Prior research has shown that although sharing context using video can create such beneficial effects, it also has some practical disadvantages. It is an open question whether other modalities might also provide sufficient context for a caller. In particular, the effects of sharing audio, a cheaper, more salient, and perhaps more practical alternative than video, are not well understood. We investigated sharing context using direct cues in the form of realistic driving sounds (e.g., car honks, sirens) and indirect cues in the form of elevated heartbeats. Sound sharing affected the caller's perception of the driver's busyness. However, this had at most a modest effect on conversation and driving performance. An implication of these results is that although sharing sounds can increase a caller's awareness of changes in the driver's busyness, they need more training or information on how to leverage such context information to reduce disruption to driving. Limitations and implications are discussed.

When, what, and how much to reward in reinforcement learning-based models of cognition.

Reinforcement learning approaches to cognitive modeling represent task acquisition as learning to choose the sequence of steps that accomplishes the task while maximizing a reward. However, an apparently unrecognized problem for modelers is choosing when, what, and how much to reward; that is, when (the moment: end of trial, subtask, or some other interval of task performance), what (the objective function: e.g., performance time or performance accuracy), and how much (the magnitude: with binary, categorical, or continuous values). In this article, we explore the problem space of these three parameters in the context of a task whose completion entails some combination of 36 state-action pairs, where all intermediate states (i.e., after the initial state and prior to the end state) represent progressive but partial completion of the task. Different choices produce profoundly different learning paths and outcomes, with the strongest effect for moment. Unfortunately, there is little discussion in the literature of the effect of such choices. This absence is disappointing, as the choice of when, what, and how much needs to be made by a modeler for every learning model.

Identifying optimum performance trade-offs using a cognitively bounded rational analysis model of discretionary task interleaving.

We report the results of a dual-task study in which participants performed a tracking and typing task under various experimental conditions. An objective payoff function was used to provide explicit feedback on how participants should trade off performance between the tasks. Results show that participants' dual-task interleaving strategy was sensitive to changes in the difficulty of the tracking task and resulted in differences in overall task performance. To test the hypothesis that people select strategies that maximize payoff, a Cognitively Bounded Rational Analysis model was developed. This analysis evaluated a variety of dual-task interleaving strategies to identify the optimal strategy for maximizing payoff in each condition. The model predicts that the region of optimum performance is different between experimental conditions. The correspondence between human data and the prediction of the optimal strategy is found to be remarkably high across a number of performance measures. This suggests that participants were honing their behavior to maximize payoff. Limitations are discussed.

Strategic adaptation to performance objectives in a dual-task setting.

How do people interleave attention when multitasking? One dominant account is that the completion of a subtask serves as a cue to switch tasks. But what happens if switching solely at subtask boundaries led to poor performance? We report a study in which participants manually dialed a UK-style telephone number while driving a simulated vehicle. If the driver were to exclusively return his or her attention to driving after completing a subtask (i.e., using the single break in the xxxxx-xxxxxx representational structure of the number), then we would expect to see a relatively poor driving performance. In contrast, our results show that drivers choose to return attention to steering control before the natural subtask boundary. A computational modeling analysis shows that drivers had to adopt this strategy to meet the required performance objective of maintaining an acceptable lateral position in the road while dialing. Taken together these results support the idea that people can strategically control the allocation of attention in multitask settings to meet specific performance criteria.