Facing successfully high mental workload and stressors: An fMRI study.

The present fMRI study aimed at highlighting patterns of brain activations and autonomic activity when confronted with high mental workload and the threat of auditory stressors. Twenty participants performed a complex cognitive task in either safe or aversive conditions. Our results showed that increased mental workload induced recruitment of the lateral frontoparietal executive control network (ECN), along with disengagement of medial prefrontal and posterior cingulate regions of the default mode network (DMN). Mental workload also elicited an increase in heart rate and pupil diameter. Task performance did not decrease under the threat of stressors, most likely due to efficient inhibition of auditory regions, as reflected by a large decrement of activity in the superior temporal gyri. The threat of stressors was also accompanied with deactivations of limbic regions of the salience network (SN), possibly reflecting emotional regulation mechanisms through control from dorsal medial prefrontal and parietal regions, as indicated by functional connectivity analyses. Meanwhile, the threat of stressors induced enhanced ECN activity, likely for improved attentional and cognitive processes toward the task, as suggested by increased lateral prefrontal and parietal activations. These fMRI results suggest that measuring the balance between ECN, SN, and DMN recruitment could be used for objective mental state assessment. In this sense, an extra recruitment of task-related regions and a high ratio of lateral versus medial prefrontal activity may represent a relevant marker of increased but efficient mental effort, while the opposite may indicate a disengagement from the task due to mental overload and/or stressors.

Human Mirror Neuron System Based Alarms in the Cockpit: A Neuroergonomic Evaluation.

Controlled Flight Into Terrain (CFIT) events still remain among the deadliest accidents in aviation. When facing the possible occurrence of such an event, pilots have to immediately react to the ground proximity alarm ("Pull Up" alarm) in order to avoid the impending collision. However, the pilots' reaction to this alarm is not always optimal. This may be at least partly due to the low visual saliency of the current alarm and the deleterious effects of stress that alleviate the pilot's reactions. In the present study, two experiments (in a laboratory and in a flight simulator) were conducted to (1) investigate whether hand gesture videos (a hand pulling back the sidestick) can trigger brainwave frequencies related to the mirror neuron system; (2) determine whether enhancing the visual characteristics of the "Pull Up" alarm could improve pilots' response times. Electrophysiological results suggest that hand gesture videos attracted more participants' attention (greater alpha desynchronization in the parieto-occipital area) and possibly triggered greater activity of the mirror neuron system (greater mu and beta desynchronizations at central electrodes). Results obtained in the flight simulator revealed that enhancing the visual characteristics of the original "Pull Up" alarm improved the pilots' reaction times. However, no significant difference in reaction times between an enlarged "Pull Up" inscription and the hand gesture video was found. Further work is needed to determine whether mirror neuron system based alarms could bring benefits for flight safety, in particular, these alarms should be assessed during a high stress context.

Testing usability and trainability of indirect touch interaction: perspective for the next generation of air traffic control systems.

This study aims to determine whether indirect touch device can be used to interact with graphical objects displayed on another screen in an air traffic control (ATC) context. The introduction of such a device likely requires an adaptation of the sensory-motor system. The operator has to simultaneously perform movements on the horizontal plane while assessing them on the vertical plane. Thirty-six right-handed participants performed movement training with either constant or variable practice and with or without visual feedback of the displacement of their actions. Participants then performed a test phase without visual feedback. Performance improved in both practice conditions, but accuracy was higher with visual feedback. During the test phase, movement time was longer for those who had practiced with feedback, suggesting an element of dependency. However, this 'cost' of feedback did not extend to movement accuracy. Finally, participants who had received variable training performed better in the test phase, but accuracy was still unsatisfactory. We conclude that continuous visual feedback on the stylus position is necessary if tablets are to be introduced in ATC.

Affective decision making under uncertainty during a plausible aviation task: an fMRI study.

In aeronautics, plan continuation error (PCE) represents failure to revise a flight plan despite emerging evidence suggesting that it is no longer safe. Assuming that PCE may be associated with a shift from cold to hot reasoning, we hypothesized that this transition may result from a large range of strong negative emotional influences linked with the decision to abort a landing and circle for a repeat attempt, referred to as a "go-around". We investigated this hypothesis by combining functional neuroimaging with an ecologically valid aviation task performed under contextual variation in incentive and situational uncertainty. Our goal was to identify regional brain activity related to the sorts of conservative or liberal decision-making strategies engaged when participants were both exposed to a financial payoff matrix constructed to bias responses in favor of landing acceptance, while they were simultaneously experiencing maximum levels of uncertainty related to high levels of stimulus ambiguity. Combined with the observed behavioral outcomes, our neuroimaging results revealed a shift from cold to hot decision making in response to high uncertainty when participants were exposed to the financial incentive. Most notably, while we observed activity increases in response to uncertainty in many frontal regions such as dorsolateral prefrontal cortex (DLPFC) and anterior cingulate cortex (ACC), less overall activity was observed when the reward was combined with uncertainty. Moreover, participants with poor decision making, quantified as a lower discriminability index d', exhibited riskier behavior coupled with lower activity in the right DLPFC. These outcomes suggest a disruptive effect of biased financial incentive and high uncertainty on the rational decision-making neural network, and consequently, on decision relevance.

Evaluation of head-free eye tracking as an input device for air traffic control.

The purpose of this study was to investigate the possibility to integrate a free head motion eye-tracking system as input device in air traffic control (ATC) activity. Sixteen participants used an eye tracker to select targets displayed on a screen as quickly and accurately as possible. We assessed the impact of the presence of visual feedback about gaze position and the method of target selection on selection performance under different difficulty levels induced by variations in target size and target-to-target separation. We tend to consider that the combined use of gaze dwell-time selection and continuous eye-gaze feedback was the best condition as it suits naturally with gaze displacement over the ATC display and free the hands of the controller, despite a small cost in terms of selection speed. In addition, target size had a greater impact on accuracy and selection time than target distance. These findings provide guidelines on possible further implementation of eye tracking in ATC everyday activity. PRACTITIONER SUMMARY: We investigated the possibility to integrate a free head motion eye-tracking system as input device in air traffic control (ATC). We found that the combined use of gaze dwell-time selection and continuous eye-gaze feedback allowed the best performance and that target size had a greater impact on performance than target distance.

Fast & scrupulous: Gesture-based alarms improve accuracy and reaction times under various mental workload levels. An ERSP study.

In high-risk environments, fast and accurate responses to warning systems are essential to efficiently handle emergency situations. The aim of the present study was twofold: 1) investigating whether hand action videos (i.e., gesture alarms) trigger faster and more accurate responses than text alarm messages (i.e., written alarms), especially when mental workload (MWL) is high; and 2) investigating the brain activity in response to both types of alarms as a function of MWL. Regardless of MWL, participants (N = 28) were found to be both faster and more accurate when responding to gesture alarms than to written alarms. Brain electrophysiological results suggest that this greater efficiency might be due to a facilitation of the action execution, reflected by the decrease in mu and beta power observed around the response time window observed at C3 and C4 electrodes. These results suggest that gesture alarms may improve operators' performances in emergency situations.

Investigating the impact of offer frame manipulations on responders playing the ultimatum game.

The present study was designed to test the impact of frame manipulations on the decision-making of responders playing the ultimatum game. Experiment 1 investigated responders' event-related potentials (ERPs) measured in response to the offers as a function of the frame (i.e., negative: "the proposer keeps" versus positive: "the proposer offers"). While no difference in acceptation rate was found as a function of the offer's frame, electrophysiological results suggest that the stronger negative affective response to the offers in the negative frame (N400) was successfully reappraised by the responders (P600), possibly explaining why the offer frame manipulation did not modulate acceptation rates. No framing effect was found when the ultimatum game was played in its one-shot version (Experiment 2), suggesting that repeated measurements did not affect responders' behavior. However, an offer framing effect was found in female (but not in male) responders, when the complexity of the game statement increased, presumably recruiting more cognitive resources and taxing the reappraisal process (Experiment 3). Taken together, these results suggest that framing manipulations are associated with complex affective and cognitive processes, supporting the cognitive-affective tradeoff model.

Busy and confused? High risk of missed alerts in the cockpit: An electrophysiological study.

The ability to react to unexpected auditory stimuli is critical in complex settings such as aircraft cockpits or air traffic control towers, characterized by high mental load and highly complex auditory environments (i.e., many different auditory alerts). Evidence shows that both factors can negatively impact auditory attention and prevent appropriate reactions. In the present study, 60 participants performed a simulated aviation task varying in terms of mental load (no, low, high) concurrently to a tone detection paradigm in which the complexity of the auditory environment (i.e., auditory load) was manipulated (1, 2 or 3 different tones). We measured both detection performance (miss, false alarm, d') and brain activity (event-related potentials) associated with the target tone. Our results showed that both mental and auditory loads affected target tone detection performance. Importantly, their combined effects had a large impact on the percentage of missed target tones. While, in the no mental load condition, miss rate was very low with 1 (0.53%) and 2 tones (1.11%), it increased drastically with 3 tones (24.44%), and this effect was accentuated as mental load increased, yielding to the higher miss rate in the 3-tone paradigm under high mental load conditions (68.64%). Increased mental and auditory loads and miss rates were associated with disrupted brain responses to the target tone, as shown by a reduced P3b amplitude. In sum, our results highlight the importance of balancing mental and auditory loads to maintain efficient reactions to alarms in complex working environment.

Reward and uncertainty favor risky decision-making in pilots: evidence from cardiovascular and oculometric measurements.

In this paper we examined plan continuation error (PCE), a well known error made by pilots consisting in continuing the flight plan despite adverse meteorological conditions. Our hypothesis is that a large range of strong negative emotional consequences, including those induced by economic pressure, are associated with the decision to revise the flight plan and favor PCE. We investigated the economic hypothesis with a simplified landing task (reproduction of a real aircraft instrument) in which uncertainty and reward were manipulated. Heart rate (HR), heart rate variability (HRV) and eye tracking measurements were performed to get objective clues both on the cognitive and emotional state of the volunteers. Results showed that volunteers made more risky decisions under the influence of the financial incentive, in particular when uncertainty was high. Psychophysiological examination showed that HR increased and total HRV decreased in response to the cognitive load generated by the task. In addition, HR also increased in response to the financially motivated condition. Eventually, fixation times increased when uncertainty was high, confirming the difficulty in obtaining/interpreting information from the instrument in this condition. These results support the assumption that risky-decision making observed in pilots can be, at least partially, explained by a shift from cold to hot (emotional) decision-making in response to economic constraints and uncertainty.

Visual scanning strategies in the cockpit are modulated by pilots' expertise: A flight simulator study.

During a flight, pilots must rigorously monitor their flight instruments since it is one of the critical activities that contribute to update their situation awareness. The monitoring is cognitively demanding, but is necessary for timely intervention in the event of a parameter deviation. Many studies have shown that a large part of commercial aviation accidents involved poor cockpit monitoring from the crew. Research in eye-tracking has developed numerous metrics to examine visual strategies in fields such as art viewing, sports, chess, reading, aviation, and space. In this article, we propose to use both basic and advanced eye metrics to study visual information acquisition, gaze dispersion, and gaze patterning among novices and pilots. The experiment involved a group of sixteen certified professional pilots and a group of sixteen novice during a manual landing task scenario performed in a flight simulator. The two groups landed three times with different levels of difficulty (manipulated via a double task paradigm). Compared to novices, professional pilots had a higher perceptual efficiency (more numerous and shorter dwells), a better distribution of attention, an ambient mode of visual attention, and more complex and elaborate visual scanning patterns. We classified pilot's profiles (novices-experts) by machine learning based on Cosine KNN (K-Nearest Neighbors) using transition matrices. Several eye metrics were also sensitive to the landing difficulty. Our results can benefit the aviation domain by helping to assess the monitoring performance of the crews, improve initial and recurrent training and ultimately reduce incidents, and accidents due to human error.

Monitoring cognitive and emotional processes through pupil and cardiac response during dynamic versus logical task.

The paper deals with the links between physiological measurements and cognitive and emotional functioning. As long as the operator is a key agent in charge of complex systems, the definition of metrics able to predict his performance is a great challenge. The measurement of the physiological state is a very promising way but a very acute comprehension is required; in particular few studies compare autonomous nervous system reactivity according to specific cognitive processes during task performance and task related psychological stress is often ignored. We compared physiological parameters recorded on 24 healthy subjects facing two neuropsychological tasks: a dynamic task that require problem solving in a world that continually evolves over time and a logical task representative of cognitive processes performed by operators facing everyday problem solving. Results showed that the mean pupil diameter change was higher during the dynamic task; conversely, the heart rate was more elevated during the logical task. Finally, the systolic blood pressure seemed to be strongly sensitive to psychological stress. A better taking into account of the precise influence of a given cognitive activity and both workload and related task-induced psychological stress during task performance is a promising way to better monitor operators in complex working situations to detect mental overload or pejorative stress factor of error.

Influences of age, mental workload, and flight experience on cognitive performance and prefrontal activity in private pilots: a fNIRS study.

The effects of aging on cognitive performance must be better understood, especially to protect older individuals who are engaged in risky activities (e.g. aviation). Current literature on executive functions suggests that brain compensatory mechanisms may counter cognitive deterioration due to aging, at least up to certain task load levels. The present study assesses this hypothesis in private pilots engaged in two executive tasks from the standardized CANTAB battery, namely Spatial Working Memory (SWM) and  One Touch Stockings of Cambridge (OTS). Sixty-one pilots from three age groups (young, middle-aged, older) performed these two tasks from low to very high difficulty levels, beyond those reported in previous aging studies. A fNIRS headband measured changes in oxyhemoglobin (HbO2) in the prefrontal cortex. Results confirmed an overall effect of the difficulty level in the three age groups, with a decline in task performance and an increase in prefrontal HbO2 signal. Performance of older relative to younger pilots was impaired in both tasks, with the greatest impairment observed for the highest-load Spatial Working Memory task. Consistent with this behavioral deficit in older pilots, a plateau of prefrontal activity was observed at this highest-load level, suggesting that a ceiling in neural resources was reached. When behavioral performance was either equivalent between age groups or only slightly impaired in the older group, there were not any age-related differences in prefrontal activity. Finally, older pilots with extensive flying experience tend to show better preserved spatial working memory performance when compared to mildly-experienced of the same age group. The present findings are discussed in the frames of HAROLD and CRUNCH theoretical models of cognitive and neural aging, evoking the possibility that piloting expertise may contribute to preserve executive functions throughout adulthood.

The diagnosticity of psychophysiological signatures: Can we disentangle mental workload from acute stress with ECG and fNIRS?

The ability to identify reliable and sensitive physiological signatures of psychological dimensions is key to developing intelligent adaptive systems that may in turn help to mitigate human error in complex operations. The challenge of this endeavor lies with diagnosticity. Despite different underlying causes, the physiological correlates of workload and acute psychological stress manifest in rather similar ways and can be easily confounded. The current work aimed to build a diagnostic model of mental state through the simultaneous classification of mental workload (varied through three levels of the n-back task) and acute stress (the presence/absence of aversive sounds) with machine learning. Using functional near infrared spectroscopy (fNIRS) and electrocardiography (ECG), the model's classifiers was above-chance to disentangle variations of mental workload from variations of acute stress. Both ECG and fNIRS could predict mental workload level, the best accuracy resulted from the two measures in combination. Stress level could not be accurately diagnosed through ECG alone, only with fNIRS or ECG and fNIRS combined. Individual calibration may be important since stress classification was more accurate for those with higher subjective state anxiety, perhaps due to a greater sensitivity to stress. Mental workload and stress were both better classified with activity in lateral prefrontal regions of the cortex than the medial areas, and the HbO2 signal generally lead to better classification than HHB. The current model represents a step forward to finely discriminate different mental states despite their rather analog physiological correlates.

Momentary lapse of control: A cognitive continuum approach to understanding and mitigating perseveration in human error.

Everyday complex and stressful real-life situations can overwhelm the human brain to an extent that the person is no longer able to accurately evaluate the situation and persists in irrational actions or strategies. Safety analyses reveal that such perseverative behavior is exhibited by operators in many critical domains, which can lead to potentially fatal incidents. There are neuroimaging evidences of changes in healthy brain functioning when engaged in non-adaptive behaviors that are akin to executive deficits such as perseveration shown in patients with brain lesion. In this respect, we suggest a cognitive continuum whereby stressors can render the healthy brain temporarily impaired. We show that the dorsolateral prefrontal cortex is a key structure for executive and attentional control whereby any transient (stressors, neurostimulation) or permanent (lesion) impairment compromises adaptive behavior. Using this neuropsychological insight, we discuss solutions involving training, neurostimulation, and the design of cognitive countermeasures for mitigating perseveration.

Author Correction: Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS.

A correction to this article has been published and is linked from the HTML and PDF versions of this paper. The error has not been fixed in the paper.

Cardiovascular correlates of emotional state, cognitive workload and time-on-task effect during a realistic flight simulation.

In aviation, emotion and cognitive workload can considerably increase the probability of human error. An accurate online physiological monitoring of pilot's mental state could prevent accidents. The heart rate (HR) and heart rate variability (HRV) of 21 private pilots were analysed during two realistic flight simulator scenarios. Emotion was manipulated by a social stressor and cognitive workload with the difficulty of a secondary task. Our results confirmed the sensitivity of the HR to cognitive demand and training effects, with increased HR when the task was more difficult and decreased HR with training (time-on-task). Training was also associated with an increased HRV, with increased values along the flight scenario time course. Finally, the social stressor seemed to provoke an emotional reaction that enhanced motivation and performance on the secondary task. However, this was not reflected by the cardiovascular activity.

Mental workload and neural efficiency quantified in the prefrontal cortex using fNIRS.

An improved understanding of how the brain allocates mental resources as a function of task difficulty is critical for enhancing human performance. Functional near infrared spectroscopy (fNIRS) is a field-deployable optical brain monitoring technology that provides a direct measure of cerebral blood flow in response to cognitive activity. We found that fNIRS was sensitive to variations in task difficulty in both real-life (flight simulator) and laboratory settings (tests measuring executive functions), showing increased concentration of oxygenated hemoglobin (HbO2) and decreased concentration of deoxygenated hemoglobin (HHb) in the prefrontal cortex as the tasks became more complex. Intensity of prefrontal activation (HbO2 concentration) was not clearly correlated to task performance. Rather, activation intensity shed insight on the level of mental effort, i.e., how hard an individual was working to accomplish a task. When combined with performance, fNIRS provided an estimate of the participants' neural efficiency, and this efficiency was consistent across levels of difficulty of the same task. Overall, our data support the suitability of fNIRS to assess the mental effort related to human operations and represents a promising tool for the measurement of neural efficiency in other contexts such as training programs or the clinical setting.

Can an aircraft be piloted via sonification with an acceptable attentional cost? A comparison of blind and sighted pilots.

In the aeronautics field, some authors have suggested that an aircraft's attitude sonification could be used by pilots to cope with spatial disorientation situations. Such a system is currently used by blind pilots to control the attitude of their aircraft. However, given the suspected higher auditory attentional capacities of blind people, the possibility for sighted individuals to use this system remains an open question. For example, its introduction may overload the auditory channel, which may in turn alter the responsiveness of pilots to infrequent but critical auditory warnings. In this study, two groups of pilots (blind versus sighted) performed a simulated flight experiment consisting of successive aircraft maneuvers, on the sole basis of an aircraft sonification. Maneuver difficulty was varied while we assessed flight performance along with subjective and electroencephalographic (EEG) measures of workload. The results showed that both groups of participants reached target-attitudes with a good accuracy. However, more complex maneuvers increased subjective workload and impaired brain responsiveness toward unexpected auditory stimuli as demonstrated by lower N1 and P3 amplitudes. Despite that the EEG signal showed a clear reorganization of the brain in the blind participants (higher alpha power), the brain responsiveness to unexpected auditory stimuli was not significantly different between the two groups. The results suggest that an auditory display might provide useful additional information to spatially disoriented pilots with normal vision. However, its use should be restricted to critical situations and simple recovery or guidance maneuvers.

High Working Memory Load Impairs Language Processing during a Simulated Piloting Task: An ERP and Pupillometry Study.

Given the important amount of visual and auditory linguistic information that pilots have to process, operating an aircraft generates a high working-memory load (WML). In this context, the ability to focus attention on relevant information and to remain responsive to concurrent stimuli might be altered. Consequently, understanding the effects of WML on the processing of both linguistic targets and distractors is of particular interest in the study of pilot performance. In the present work, participants performed a simplified piloting task in which they had to follow one of three colored aircraft, according to specific written instructions (i.e., the written word for the color corresponding to the color of one of the aircraft) and to ignore either congruent or incongruent concurrent auditory distractors (i.e., a spoken name of color). The WML was manipulated with an n-back sub-task. Participants were instructed to apply the current written instruction in the low WML condition, and the 2-back written instruction in the high WML condition. Electrophysiological results revealed a major effect of WML at behavioral (i.e., decline of piloting performance), electrophysiological, and autonomic levels (i.e., greater pupil diameter). Increased WML consumed resources that could not be allocated to the processing of the linguistic stimuli, as indexed by lower P300/P600 amplitudes. Also, significantly, lower P600 responses were measured in incongruent vs. congruent trials in the low WML condition, showing a higher difficulty reorienting attention toward the written instruction, but this effect was canceled in the high WML condition. This suppression of interference in the high load condition is in line with the engagement/distraction trade-off model. We propose that P300/P600 components could be reliable indicators of WML and that they allow an estimation of its impact on the processing of linguistic stimuli.

The Responders' Gender Stereotypes Modulate the Strategic Decision-Making of Proposers Playing the Ultimatum Game.

Despite the wealth of studies investigating factors affecting decisions, not much is known about the impact of stereotypical beliefs on strategic economic decision-making. In the present study, we used the ultimatum game paradigm to investigate how participants playing as proposer modulate their strategic economic behavior, according to their game counterparts' stereotypical identity (i.e., responders). The latter were introduced to the participants using occupational role nouns stereotypically marked with gender paired with feminine or masculine proper names (e.g., linguist-Anna; economist-David; economist-Cristina; linguist-Leonardo). When playing with male-stereotyped responders, proposers quickly applied the equity rule, behaving fairly, while they adopted a strategic behavior with responders characterized by female stereotypes. They were also longer to make their offers to female than to male responders but both kinds of responders received comparable offers, suggesting a greater cognitive effort to treat females as equally as males. The present study explicitly demonstrates that gender stereotypical information affect strategic economic decision-making and highlights a possible evolution of gender discrimination into a more insidious discrimination toward individuals with female characteristics.