Estimating Systemic Cognitive States from a Mixture of Physiological and Brain Signals.

As human-machine teams are being considered for a variety of mixed-initiative tasks, detecting and being responsive to human cognitive states, in particular systematic cognitive states, is among the most critical capabilities for artificial systems to ensure smooth interactions with humans and high overall team performance. Various human physiological parameters, such as heart rate, respiration rate, blood pressure, and skin conductance, as well as brain activity inferred from functional near-infrared spectroscopy or electroencephalogram, have been linked to different systemic cognitive states, such as workload, distraction, or mind-wandering among others. Whether these multimodal signals are indeed sufficient to isolate such cognitive states across individuals performing tasks or whether additional contextual information (e.g., about the task state or the task environment) is required for making appropriate inferences remains an important open problem. In this paper, we introduce an experimental and machine learning framework for investigating these questions and focus specifically on using physiological and neurophysiological measurements to learn classifiers associated with systemic cognitive states like cognitive load, distraction, sense of urgency, mind wandering, and interference. Specifically, we describe a multitasking interactive experimental setting used to obtain a comprehensive multimodal data set which provided the foundation for a first evaluation of various standard state-of-the-art machine learning techniques with respect to their effectiveness in inferring systemic cognitive states. While the classification success of these standard methods based on just the physiological and neurophysiological signals across subjects was modest, which is to be expected given the complexity of the classification problem and the possibility that higher accuracy rates might not in general be achievable, the results nevertheless can serve as a baseline for evaluating future efforts to improve classification, especially methods that take contextual aspects such as task and environmental states into account.

Ultra-brief training in cognitive reappraisal or mindfulness reduces anxiety and improves motor performance efficiency under stress.

BACKGROUND AND OBJECTIVES: We examined the effects of ultra-brief training in mindfulness and cognitive reappraisal on affective response and performance under stress. We hypothesized that one or both types of training would decrease affective responding and improve performance, and that these effects might be moderated by acute stress induction. DESIGN: We manipulated training (mindfulness, cognitive reappraisal, control) between subjects and level of stress (low, high) within subjects in a 3 × 2 mixed factorial design. Method: Participants (N = 112, ages 18-35) completed two sessions on different days. In each session, they received mindfulness or cognitive reappraisal training or listened to a control script prior to a low- or high-stress simulated hostage situation. We measured motor performance efficiency (proportion of shots that hit hostile and hostage targets), affective responding (self-reported anxiety, salivary cortisol and alpha amylase, and autonomic physiology), and physical activity. RESULTS: Compared to control instructions, ultra-brief training in cognitive reappraisal or mindfulness reduced subjective anxiety and increased performance efficiency. There were few effects of training on other measures. CONCLUSION: Ultra-brief training in cognitive reappraisal or mindfulness prior to a stressful task may be both helpful and harmful; effects are preliminary and subject to boundary conditions.

Investigating Methods for Cognitive Workload Estimation for Assistive Robots.

Robots interacting with humans in assistive contexts have to be sensitive to human cognitive states to be able to provide help when it is needed and not overburden the human when the human is busy. Yet, it is currently still unclear which sensing modality might allow robots to derive the best evidence of human workload. In this work, we analyzed and modeled data from a multi-modal simulated driving study specifically designed to evaluate different levels of cognitive workload induced by various secondary tasks such as dialogue interactions and braking events in addition to the primary driving task. Specifically, we performed statistical analyses of various physiological signals including eye gaze, electroencephalography, and arterial blood pressure from the healthy volunteers and utilized several machine learning methodologies including k-nearest neighbor, naive Bayes, random forest, support-vector machines, and neural network-based models to infer human cognitive workload levels. Our analyses provide evidence for eye gaze being the best physiological indicator of human cognitive workload, even when multiple signals are combined. Specifically, the highest accuracy (in %) of binary workload classification based on eye gaze signals is 80.45 ∓ 3.15 achieved by using support-vector machines, while the highest accuracy combining eye gaze and electroencephalography is only 77.08 ∓ 3.22 achieved by a neural network-based model. Our findings are important for future efforts of real-time workload estimation in the multimodal human-robot interactive systems given that eye gaze is easy to collect and process and less susceptible to noise artifacts compared to other physiological signal modalities.

Domain adaptation for robust workload level alignment between sessions and subjects using fNIRS.

SIGNIFICANCE: We demonstrated the potential of using domain adaptation on functional near-infrared spectroscopy (fNIRS) data to classify different levels of n-back tasks that involve working memory. AIM: Domain shift in fNIRS data is a challenge in the workload level alignment across different experiment sessions and subjects. To address this problem, two domain adaptation approaches-Gromov-Wasserstein (G-W) and fused Gromov-Wasserstein (FG-W) were used. APPROACH: Specifically, we used labeled data from one session or one subject to classify trials in another session (within the same subject) or another subject. We applied G-W for session-by-session alignment and FG-W for subject-by-subject alignment to fNIRS data acquired during different n-back task levels. We compared these approaches with three supervised methods: multiclass support vector machine (SVM), convolutional neural network (CNN), and recurrent neural network (RNN). RESULTS: In a sample of six subjects, G-W resulted in an alignment accuracy of 68 % ± 4 % (weighted mean ± standard error) for session-by-session alignment, FG-W resulted in an alignment accuracy of 55 % ± 2 % for subject-by-subject alignment. In each of these cases, 25% accuracy represents chance. Alignment accuracy results from both G-W and FG-W are significantly greater than those from SVM, CNN, and RNN. We also showed that removal of motion artifacts from the fNIRS data plays an important role in improving alignment performance. CONCLUSIONS: Domain adaptation has potential for session-by-session and subject-by-subject alignment of mental workload by using fNIRS data.

Exerting cognitive control under threat: Interactive effects of physical and emotional stress.

Individuals with stressful occupations, such as law enforcement and military personnel, are required to operate in high stakes environments that can be simultaneously physically and emotionally demanding. These individuals are tasked with maintaining peak performance under stressful and often unpredictable conditions, exerting high levels of cognitive control to sustain attention and suppress task-irrelevant actions. Previous research has shown that physical and emotional stressors differentially influence such cognitive control processes. For example, physical stress impairs while emotional stress facilitates the ability to inhibit a prepotent response, yet, interactions between the two remain poorly understood. Here we examined whether emotional stress induced by threat of unpredictable electric shock mitigates the effects of physical stress on response inhibition. Participants performed an auditory Go/NoGo task under safe versus threat conditions while cycling at high intensity (84% HRmax) for 50 min. In threat conditions, participants were told they would receive mild electric shocks that were unpredictable and unrelated to task performance. Self-reported anxiety increased under threat versus safe conditions, and perceived exertion increased with exercise duration. As predicted, we observed decrements in response inhibition (increased false alarms) as exertion increased under safe conditions, but improved response inhibition as exertion increased under threat conditions. These findings are consistent with previous work showing that anxiety induced by unpredictable threat promotes adaptive survival mechanisms, such as improved vigilance, threat detection, cautious behavior, and harm avoidance. Here, we suggest that emotional stress induced by unpredictable threat can also mitigate decrements in cognitive performance experienced under physically demanding conditions. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The impact of uncertain threat on affective bias: Individual differences in response to ambiguity.

Individuals who operate under highly stressful conditions (e.g., military personnel and first responders) are often faced with the challenge of quickly interpreting ambiguous information in uncertain and threatening environments. When faced with ambiguity, it is likely adaptive to view potentially dangerous stimuli as threatening until contextual information proves otherwise. One laboratory-based paradigm that can be used to simulate uncertain threat is known as threat of shock (TOS), in which participants are told that they might receive mild but unpredictable electric shocks while performing an unrelated task. The uncertainty associated with this potential threat induces a state of emotional arousal that is not overwhelmingly stressful, but has widespread-both adaptive and maladaptive-effects on cognitive and affective function. For example, TOS is thought to enhance aversive processing and abolish positivity bias. Importantly, in certain situations (e.g., when walking home alone at night), this anxiety can promote an adaptive state of heightened vigilance and defense mobilization. In the present study, we used TOS to examine the effects of uncertain threat on valence bias, or the tendency to interpret ambiguous social cues as positive or negative. As predicted, we found that heightened emotional arousal elicited by TOS was associated with an increased tendency to interpret ambiguous cues negatively. Such negative interpretations are likely adaptive in situations in which threat detection is critical for survival and should override an individual's tendency to interpret ambiguity positively in safe contexts. (PsycINFO Database Record