Attentional Tunneling in Pilots During a Visual Tracking Task With a Head Mounted Display.

OBJECTIVE: We examined whether active head aiming with a Helmet Mounted Display (HMD) can draw the pilot's attention away from a primary flight task. Furthermore, we examined whether visual clutter increases this effect. BACKGROUND: Head up display symbology can result in attentional tunneling, and clutter makes it difficult to identify objects. METHOD: Eighteen military pilots had to simultaneously perform an attitude control task while flying in clouds and a head aiming task in a fixed-base flight simulator. The former consisted of manual compensation for roll disturbances of the aircraft, while the latter consisted of keeping a moving visual target inside a small or large head-referenced circle. A "no head aiming" condition served as a baseline. Furthermore, all conditions were performed with or without visual clutter. RESULTS: Head aiming led to deterioration of the attitude control task performance and an increase of the amount of roll-reversal errors (RREs). This was even the case when head aiming required minimal effort. Head aiming accuracy was significantly lower when the roll disturbances in the attitude control task were large compared to when they were small. Visual clutter had no effect on both tasks. CONCLUSION: We suggest that active head aiming of HMD symbology can cause attentional tunneling, as expressed by an increased number of RREs and less accuracy on a simultaneously performed attitude control task. APPLICATION: This study improves our understanding in the perceptual and cognitive effects of (military) HMDs, and has implications for operational use and possibly (re)design of HMDs.

Development and validation of an assessment index for quantifying cognitive task load in pilots under simulated flight conditions using heart rate variability and principal component analysis.

To investigate whether high cognitive task load (CTL) for aircraft pilots can be identified by analysing heart-rate variability, electrocardiograms were recorded while cadet pilots (n = 68) performed the plane tracking, anti-gravity pedalling, and reaction tasks during simulated flight missions. Data for standard electrocardiogram parameters were extracted from the R-R-interval series. In the research phase, low frequency power (LF), high frequency power (HF), normalised HF, and LF/HF differed significantly between high and low CTL conditions (p < .05 for all). A principal component analysis identified three components contributing 90.62% of cumulative heart-rate variance. These principal components were incorporated into a composite index. Validation in a separate group of cadet pilots (n = 139) under similar conditions showed that the index value significantly increased with increasing CTL (p < .05). The heart-rate variability index can be used to objectively identify high CTL flight conditions.Practitioner summary: We used principal component analysis of electrocardiogram data to construct a composite index for identifying high cognitive task load in pilots during simulated flight. We validated the index in a separate group of pilots under similar conditions. The index can be used to improve cadet training and flight safety.Abbreviations: ANOVA: a one-way analysis of variance; AP: anti-gravity pedaling task; CTL: cognitive task load; ECG: electrocardiograms; HR: heart rate; HRV: heart-rate variability; HRVI: heart-rate variability index; PT: plane-tracking task; RMSSD: root-mean square of differences between consecutive R-R intervals; RT: reaction task; SDNN: standard deviation of R-R intervals; HF: high frequency power; HFnu: normalized HF; LF: low frequency power; LFnu: normalized LF; PCA: principal component analysis.

Visual learning in tethered bees modifies flight orientation and is impaired by epinastine.

Visual-orientation learning of a tethered flying bee was investigated using a flight simulator and a novel protocol in which orientation preference toward trained visual targets was assessed in tests performed before and after appetitive conditioning. Either a blue or a green rectangle (conditioned stimulus, CS) was associated with 30% sucrose solution (unconditioned stimulus, US), whereas the other rectangle was not paired with US. Bees were tested in a closed-looped flight simulator 5 min after ten pairings of the US and CS. Conditioned bees were preferentially oriented to the CS after such training. This increase in preference for CS was maintained for 24 h, indicating the presence of long-term memory. Because the total orienting time was not altered by conditioning, conditioning did not enhance orientation activity itself but increased the relative time for orientation to CS. When 0.4 or 4 mM epinastine (an antagonist of octopamine receptors) was injected into the bee's head 30 min prior to the experiment, both short- and long-term memory formation were significantly impaired, suggesting that octopamine, which is crucial for appetitive olfactory learning in insects, is also involved in visual orientation learning.

Design and Implementation of a New Training Flight Simulator System.

Aircraft flight simulators have good cost efficiency, high reusability, and high flight safety. All airlines and aircraft manufacturing companies choose it as sophisticated training equipment for ground simulation, effectively reducing pilot training costs, ensuring personnel safety and aircraft wear and tear. The new simulator proposed in this paper combines a digital motion-cueing algorithm, flight software and motion platform to make pilots feel as if they are in the real world. By using EtherCAT technology to drive the motion-cueing platform, it can improve the data transmission speed of the simulator as well as the strong anti-interference ability of communication and the control operation efficiency. Therefore, the simulated flight subjects can perform long-distance and large-angle training. Next, a set of measurement systems was established to provide monitoring items including attitude, velocity and acceleration, which can be displayed on the screen and recorded on the computer in real time and dynamically. Finally, seven training subjects were implemented to demonstrate the feasibility and correctness of the proposed method.

Hoverflies use a time-compensated sun compass to orientate during autumn migration.

The sun is the most reliable celestial cue for orientation available to daytime migrants. It is widely assumed that diurnal migratory insects use a 'time-compensated sun compass' to adjust for the changing position of the sun throughout the day, as demonstrated in some butterfly species. The mechanisms used by other groups of diurnal insect migrants remain to be elucidated. Migratory species of hoverflies (Diptera: Syrphidae) are one of the most abundant and beneficial groups of diurnal migrants, providing multiple ecosystem services and undergoing directed seasonal movements throughout much of the temperate zone. To identify the hoverfly navigational strategy, a flight simulator was used to measure orientation responses of the hoverflies Scaeva pyrastri and Scaeva selenitica to celestial cues during their autumn migration. Hoverflies oriented southwards when they could see the sun and shifted this orientation westward following a 6 h advance of their circadian clocks. Our results demonstrate the use of a time-compensated sun compass as the primary navigational mechanism, consistent with field observations that hoverfly migration occurs predominately under clear and sunny conditions.

Exposure to hypoxia impairs helicopter pilots' awareness of environment.

The purpose of the present study was to determine how hypoxia effects awareness of environment (AoE) in helicopter pilots operating at high altitude. Eight helicopter crews flew two operational flights in a flight simulator while breathing gas mixtures of 20.9% (equivalent to 0 m altitude) and 11.4% (equivalent to 4572 m or 15,000 ft altitude) oxygen in a single blinded, counterbalanced, repeated measures study. Each flight included five missions, during which environment items were introduced that the crews needed to be aware of, and respond to. In the 4572 m simulation, the crews missed overall 28 AoE items compared to 12 in the 0 m simulation (Z = -1.992; p = .046). In contrast, the crews' technical skills were not significantly effected by hypoxia. Remarkably, the majority of pilots did not notice they were hypoxic or recognise their hypoxia symptoms during the simulation flight at 4572 m. Practitioner summary We show that hypoxia has a detrimental effect on helicopter pilot's AoE and alertness. This can lead to an increased risk for flight safety. To mitigate this risk we recommend performing hypoxia training in a flight simulator, developing wearable systems for physiological monitoring of pilots and re-evaluating current altitude regulations. Abbreviations: ANOVA: Analysis of variance; AoE: awareness of environment; CSV: comma-separated values; HDU: helmet display unit; HR: heart rate; IQR: interquartile range; Mdn: median; NTS: non-technical skills; RNLAF: Royal Netherlands Air Force; PPM: parts per million; SpO2: oxygen saturation; SSS: Stanford sleepiness scale; TS: technical Skills.

The effect of training on the perceived approach angle in visual vertical heading judgements in a virtual environment.

Past studies have found poorer performance on vertical heading judgement accuracy compared to horizontal heading judgement accuracy. In everyday life, precise vertical heading judgements are used less often than horizontal heading judgements as we cannot usually control our vertical direction. However, pilots judging a landing approach need to consistently discriminate vertical heading angles to land safely. This study addresses the impact of training on participants' ability to judge their touchdown point relative to a target in a virtual environment with a clearly defined ground plane and horizon. Thirty-one participants completed a touchdown point estimation task twice, using three angles of descent (3°, 6° and 9°). In between the two testing tasks, half of the participants completed a flight simulator landing training task which provided feedback on their vertical heading performance; while, the other half completed a two-dimensional puzzle game as a control. Overall, participants were more precise in their responses in the second testing compared to the first (from a SD of ± 0.91° to ± 0.67°), but only the experimental group showed improvement in accuracy (from a mean error of - 2.1° to - 0.6°). Our results suggest that with training, vertical heading judgments can be as accurate as horizontal heading judgments. This study is the first to show the effectiveness of training in vertical heading judgement in naïve individuals. The results are applicable in the field of aviation, informing possible strategies for pilot training.

Devising a Distributed Co-Simulator for a Multi-UAV Network.

Practical evaluation of the Unmanned Aerial Vehicle (UAV) network requires a lot of money to build experiment environments, which includes UAVs, network devices, flight controllers, and so on. To investigate the time-sensitivity of the multi-UAV network, the influence of the UAVs' mobility should be precisely evaluated in the long term. Although there are some simulators for UAVs' physical flight, there is no explicit scheme for simulating both the network environment and the flight environments simultaneously. In this paper, we propose a novel co-simulation scheme for the multiple UAVs network, which performs the flight simulation and the network simulation simultaneously. By considering the dependency between the flight status and networking situations of UAV, our work focuses on the consistency of simulation state through synchronization among simulation components. Furthermore, we extend our simulator to perform multiple scenarios by exploiting distributed manner. We verify our system with respect to the robustness of time management and propose some use cases which can be solely simulated by this.

Central complex and mushroom bodies mediate novelty choice behavior in Drosophila.

Novelty choice, a visual paired-comparison task, for the fly Drosophila melanogaster is studied with severely restrained single animals in a flight simulator. The virtual environment simulates free flight for rotation in the horizontal plane. The behavior has three functional components: visual azimuth orientation, working memory, and pattern discrimination (perception). Here we study novelty choice in relation to its neural substrate in the brain and show that it requires the central complex and, in particular, the ring neurons of the ellipsoid body. Surprisingly, it also involves the mushroom bodies which are needed specifically in the comparison of patterns of different sizes.

Cardiac autonomic responses in relation to cognitive workload during simulated military flight.

Understanding the operator's cognitive workload is crucial for efficiency and safety in human-machine systems. This study investigated how cognitive workload modulates cardiac autonomic regulation during a standardized military simulator flight. Military student pilots completed simulated flight tasks in a Hawk flight simulator. Continuous electrocardiography was recorded to analyze time and frequency domain heart rate variability (HRV). After the simulation, a flight instructor used a standardized method to evaluate student pilot's individual cognitive workload from video-recorded flight simulator data. Results indicated that HRV was able to differentiate flight phases that induced varying levels of cognitive workload; an increasing level of cognitive workload caused significant decreases in many HRV variables, mainly reflecting parasympathetic deactivation of cardiac autonomic regulation. In conclusion, autonomic physiological responses can be used to examine reactions to increased cognitive workload during simulated military flights. HRV could be beneficial in assessing individual responses to cognitive workload and pilot performance during simulator training.

Mosquitoes integrate visual and acoustic cues to mediate conspecific interactions in swarms.

Male mosquitoes form aerial aggregations, known as swarms, to attract females and maximize their chances of finding a mate. Within these swarms, individuals must be able to recognize potential mates and navigate the social environment to successfully intercept a mating partner. Prior research has almost exclusively focused on the role of acoustic cues in mediating the male mosquito's ability to recognize and pursue females. However, the role of other sensory modalities in this behavior has not been explored. Moreover, how males avoid collisions with one another in the swarm while pursuing females remains poorly understood. In this study, we combined free-flight and tethered-flight simulator experiments to demonstrate that swarming Anopheles coluzzii mosquitoes integrate visual and acoustic information to track conspecifics and avoid collisions. Our tethered experiments revealed that acoustic stimuli gated mosquito steering responses to visual objects simulating nearby mosquitoes, especially in males that exhibited a strong response toward visual objects in the presence of female flight tones. Additionally, we observed that visual cues alone could trigger changes in mosquitoes' wingbeat amplitude and frequency. These findings were corroborated by our free-flight experiments, which revealed that Anopheles coluzzii modulate their thrust-based flight responses to nearby conspecifics in a similar manner to tethered animals, potentially allowing for collision avoidance within swarms. Together, these results demonstrate that both males and females integrate multiple sensory inputs to mediate swarming behavior, and for males, the change in flight kinematics in response to multimodal cues might allow them to simultaneously track females while avoiding collisions.

Multisensory integration in Anopheles mosquito swarms: The role of visual and acoustic information in mate tracking and collision avoidance.

Male mosquitoes form aerial aggregations, known as swarms, to attract females and maximize their chances of finding a mate. Within these swarms, individuals must be able to recognize potential mates and navigate the dynamic social environment to successfully intercept a mating partner. Prior research has almost exclusively focused on the role of acoustic cues in mediating the male mosquito's ability to recognize and pursue flying females. However, the role of other sensory modalities in this behavior has not been explored. Moreover, how males avoid collisions with one another in the dense swarm while pursuing females remains poorly understood. In this study, we combined free-flight and tethered flight simulator experiments to demonstrate that swarming Anopheles coluzzii mosquitoes integrate visual and acoustic information to track conspecifics and avoid collisions. Our tethered experiments revealed that acoustic stimuli gated mosquito steering responses to visual objects simulating nearby mosquitoes, especially in males that exhibited attraction to visual objects in the presence of female flight tones. Additionally, we observed that visual cues alone could trigger changes in mosquitoes' wingbeat amplitude and frequency. These findings were corroborated by our free-flight experiments, which revealed that mosquitoes modulate their flight responses to nearby conspecifics in a similar manner to tethered animals, allowing for collision avoidance within swarms. Together, these results demonstrate that both males and females integrate multiple sensory inputs to mediate swarming behavior, and for males, the change in flight kinematics in response to multimodal cues allows them to simultaneously track females while avoiding collisions.

Augmented reality for training formation flights: An analysis of human factors.

Pilot training has been, for decades, aided by flight simulators with different characteristics and degrees of fidelity. However, many studies indicate that, despite the recognized contribution of simulator training, actual flying practice is still necessary, depending on the trained task. This work introduces the proposal of using augmented reality for in-flight training, where elements in the environment outside the aircraft are displayed through an augmented reality headset to create a simulation scenario. The training of basic formation flight is used as an example, as it requires flying with at least two aircraft, resulting in high operational costs and risk of collision between aircraft. In this case, the augmented reality system replaces the real leader aircraft with a projection. In order to evaluate the Technology Readiness Level (TRL) of this proposal, this work presents a prototype of an augmented reality system integrated into a flight simulator to conduct an evaluation campaign. We investigate how the introduction of the augmented reality system impacts on human factors, such as stress and workload, as well as performance. Although the results obtained in a simulated environment are not equivalent to those from an in-flight campaign, the experimental campaign performed in the flight simulator provides a way of evaluating the impact on the pilot of some aspects of the proposed solution, such as the performance of occlusion routines and some ergonomic aspects of the augmented reality headset.

Tilt perception is different in the pitch and roll planes in human.

Neurophysiological tests probing the vestibulo-ocular, colic and spinal pathways are the gold standard to evaluate the vestibular system in clinics. In contrast, vestibular perception is rarely tested despite its potential usefulness in professional training and for the longitudinal follow-up of professionals dealing with complex man-machine interfaces, such as aircraft pilots. This is explored here using a helicopter flight simulator to probe the vestibular perception of pilots. The vestibular perception of nine professional helicopter pilots was tested using a full flight helicopter simulator. The cabin was tilted six times in roll and six times in pitch (-15°, -10°, -5°, 5°, 10° and 15°) while the pilots had no visual cue. The velocities of the outbound displacement of the cabin were kept below the threshold of the semicircular canal perception. After the completion of each movement, the pilots were asked to put the cabin back in the horizontal plane (still without visual cues). The order of the 12 trials was randomized with two additional control trials where the cabin stayed in the horizontal plane but rotated in yaw (-10° and +10°). Pilots were significantly more precise in roll (average error in roll: 1.15 ± 0.67°) than in pitch (average error in pitch: 2.89 ± 1.06°) (Wilcoxon signed-rank test: p < 0.01). However, we did not find a significant difference either between left and right roll tilts (p = 0.51) or between forward and backward pitch tilts (p = 0.59). Furthermore, we found that the accuracies were significantly biased with respect to the initial tilt. The greater the initial tilt was, the less precise the pilots were, although maintaining the direction of the tilt, meaning that the error can be expressed as a vestibular error gain in the ability to perceive the modification in the orientation. This significant result was found in both roll (Friedman test: p < 0.01) and pitch (p < 0.001). However, the pitch trend error was more prominent (gain = 0.77 vs gain = 0.93) than roll. This study is a first step in the determination of the perceptive-motor profile of pilots, which could be of major use for their training and their longitudinal follow-up. A similar protocol may also be useful in clinics to monitor the aging process of the otolith system with a simplified testing device.

Averaged sub-gradient integral sliding mode control design for cueing end-effector acceleration of a two-link robotic arm.

Acceleration tracking is a significant problem in aeronautics, automotive, and biomedical technical areas because its solution may yield effective simulation of motion cues. In the case of aeronautics, the proper solution for the tracking problem improves the in-flight simulations for the training of plane pilots. These simulators can be set up using robotic devices that develop controlled motions with the end-effector following the required three-dimensional reference accelerations robustly. Hence, the primary goal of this study is the effective application of the integral sliding mode controller to solve the acceleration tracking problem for the end-effector of a two-link robotic arm. The control design problem is formulated as an optimization of a convex (non-strict) performance functional depending on the difference between the acceleration of the robotic arm and the desired acceleration using the averaged sub-gradient (ASG) descendant method. A novel sliding surface considers the sensitiveness threshold for acceleration dynamics, inspired by the limit of detection in the pilot vestibular apparatus. The proposed controller was analyzed in terms of the finite-time convergence of the sliding surface and the practical stability analysis for the tracking error dynamics. Our main contribution is the design of the online averaged sub-gradient optimization controller based on integral SMCs. The controller solves the end-effector acceleration tracking for a two-link robotic arm, which implements a simplified version of a flight simulator that is considered to be operated under uncertain scenarios and assumes the presence of perturbations and modeling errors. The controller considers the case of incomplete knowledge of the robotic arm model, which adds an extra degree of robustness to the control design. The numerical evaluations demonstrate the attributes of the ASG formulation compared to traditional state feedback control, using the performance functional, the norm of the acceleration tracking error, and the control input variation.

Dopaminergic neurons dynamically update sensory values during olfactory maneuver.

Dopaminergic neurons (DANs) drive associative learning to update the value of sensory cues, but their contribution to the assessment of sensory values outside the context of association remains largely unexplored. Here, we show in Drosophila that DANs in the mushroom body encode the innate value of odors and constantly update the current value by inducing plasticity during olfactory maneuver. Our connectome-based network model linking all the way from the olfactory neurons to DANs reproduces the characteristics of DAN responses, proposing a concrete circuit mechanism for computation. Downstream of DANs, odors alone induce value- and dopamine-dependent changes in the activity of mushroom body output neurons, which store the current value of odors. Consistent with this neural plasticity, specific sets of DANs bidirectionally modulate flies' steering in a virtual olfactory environment. Thus, the DAN circuit known for discrete, associative learning also continuously updates odor values in a nonassociative manner.

Aspects of yaw control design of an aircraft with distributed electric propulsion.

Distributed electric propulsion (DEP) offers new options in aircraft design. Besides the optimization of the wing, another area of optimization is the vertical tail plane (VTP) and yaw control. The large number of engines significantly relaxes the one-engine-inoperative (OEI) case during take-off, which is mostly the sizing case for the VTP. This offers the possibility to reduce the VTP and rudder size to a certain amount. Also, the dynamics of electric motors offer the possibility to use differential thrust for yaw control. This can compensate at least some of the reduced rudder effectiveness coming from the smaller VTP size. In the framework of the German nationally funded project SynergIE, different aircraft designs of a hybrid-electric regional aircraft were investigated. Three aircraft concepts with 2, 6 and 12 propellers were designed in the project, for which reasonable minimum VTP sizes were investigated. For the 12-propeller aircraft, the investigations showed that the VTP could be reduced by 50%, still allowing the compensation of OEI during take-off and the generation of sideslip angle during crosswind operations. This reduction in VTP size results in a reduction of the block fuel by about 4%. For the 12-engine aircraft, a 6-degrees-of-freedom simulation model was developed including flight control laws for yaw control using the rudder and differential thrust. Virtual flight tests were performed in a full-flight simulator. The tests generally showed a good agreement with the theoretical results from the handling quality analysis but also outlined deficiencies in aircraft handling at low speed with full flaps. The use of a flight simulator at this early stage of aircraft design has proven to be a useful tool to investigate such unconventional designs.

Postural Control and Psychophysical State Following of Flight Simulator Session in Novice Pilots.

Flight simulators can cause side effects usually called simulator sickness. Scientific research proves that postural instability can be an indicator of the occurrence of simulator sickness symptoms. This study aims to assess changes of postural control and psychophysical state in novice pilots following 2-h exposure to simulator conditions. The postural sway was quantified based on variables describing the displacement of the Center of Pressure (COP) generated in a quiet stance with eyes open (EO) and closed (EC). The psychophysical state was assessed using the Simulator Sickness Questionnaire (SSQ). The research was carried out in a group of 24 novice pilots who performed procedural and emergency flight exercises in the simulator at Instrument Meteorological Conditions. Each subject was examined twice: immediately before the simulator session (pre-exposure test), and just after the session (post-exposure test). The differences in postural stability between pre- and post-exposure to simulator conditions were assessed based on the normalized Romberg quotients, calculated for individual variables. The lower median values of all Romberg quotients confirmed the decreasing difference between the measures with eyes open and with eyes closed in the post-exposure tests. After the flight simulator session in both measurements (EO and EC) the values of the length of sway path (SP), the mean amplitude (MA), the sway area (SA) have changed. The visual contribution to postural sway control was reduced. The median values for all SSQ scores (total, nausea, oculomotor, and disorientation scales) were significantly higher in post-exposure tests. The largest increase was noted in the oculomotor SSQ scores (from 7.6 ± 7.6 to 37.9 ± 26.5). Over 50% of pilots participating in this study expressed symptoms typical of simulator sickness connected with visual induction: fatigue, eyestrain, difficulty focusing and difficulty concentrating. The severity of oculomotor and disorientation symptoms were rated as moderate (total SSQ score of more than 25 and <60). This study concludes that changes noted in the postural control and psychophysical state of the studied pilots after exposure to the flight simulator confirm the occurrence of the simulator sickness symptoms. Although, we did not find significant correlation of postural stability with SSQ scores.

Cognitive Load Estimation in VR Flight Simulator.

This paper discusses the design and development of a low-cost virtual reality (VR) based flight simulator with cognitive load estimation feature using ocular and EEG signals. Focus is on exploring methods to evaluate pilot's interactions with aircraft by means of quantifying pilot's perceived cognitive load under different task scenarios. Realistic target tracking and context of the battlefield is designed in VR. Head mounted eye gaze tracker and EEG headset are used for acquiring pupil diameter, gaze fixation, gaze direction and EEG theta, alpha, and beta band power data in real time. We developed an AI agent model in VR and created scenarios of interactions with the piloted aircraft. To estimate the pilot's cognitive load, we used low-frequency pupil diameter variations, fixation rate, gaze distribution pattern, EEG signal-based task load index and EEG task engagement index. We compared the physiological measures of workload with the standard user's inceptor control-based workload metrics. Results of the piloted simulation study indicate that the metrics discussed in the paper have strong association with pilot's perceived task difficulty.

Dual Passive Reactive Brain-Computer Interface: A Novel Approach to Human-Machine Symbiosis.

The present study proposes a novel concept of neuroadaptive technology, namely a dual passive-reactive Brain-Computer Interface (BCI), that enables bi-directional interaction between humans and machines. We have implemented such a system in a realistic flight simulator using the NextMind classification algorithms and framework to decode pilots' intention (reactive BCI) and to infer their level of attention (passive BCI). Twelve pilots used the reactive BCI to perform checklists along with an anti-collision radar monitoring task that was supervised by the passive BCI. The latter simulated an automatic avoidance maneuver when it detected that pilots missed an incoming collision. The reactive BCI reached 100% classification accuracy with a mean reaction time of 1.6 s when exclusively performing the checklist task. Accuracy was up to 98.5% with a mean reaction time of 2.5 s when pilots also had to fly the aircraft and monitor the anti-collision radar. The passive BCI achieved a F1-score of 0.94. This first demonstration shows the potential of a dual BCI to improve human-machine teaming which could be applied to a variety of applications.