Videoconference fatigue from a neurophysiological perspective: experimental evidence based on electroencephalography (EEG) and electrocardiography (ECG).

In the recent past, many organizations and people have substituted face-to-face meetings with videoconferences. Among others, tools like Zoom, Teams, and Webex have become the "new normal" of human social interaction in many domains (e.g., business, education). However, this radical adoption and extensive use of videoconferencing tools also has a dark side, referred to as videoconference fatigue (VCF). To date only self-report evidence has shown that VCF is a serious issue. However, based on self-reports alone it is hardly possible to provide a comprehensive understanding of a cognitive phenomenon like VCF. Against this background, we examined VCF also from a neurophysiological perspective. Specifically, we collected and analyzed electroencephalography (continuous and event-related) and electrocardiography (heart rate and heart rate variability) data to investigate whether VCF can also be proven on a neurophysiological level. We conducted a laboratory experiment based on a within-subjects design (N = 35). The study context was a university lecture, which was given in a face-to-face and videoconferencing format. In essence, the neurophysiological data-together with questionnaire data that we also collected-show that 50 min videoconferencing, if compared to a face-to-face condition, results in changes in the human nervous system which, based on existing literature, can undoubtedly be interpreted as fatigue. Thus, individuals and organizations must not ignore the fatigue potential of videoconferencing. A major implication of our study is that videoconferencing should be considered as a possible complement to face-to-face interaction, but not as a substitute.

Stylized faces enhance ERP features used for the detection of emotional responses.

For their ease of accessibility and low cost, current Brain-Computer Interfaces (BCI) used to detect subjective emotional and affective states rely largely on electroencephalographic (EEG) signals. Public datasets are available for researchers to design models for affect detection from EEG. However, few designs focus on optimally exploiting the nature of the stimulus elicitation to improve accuracy. The RSVP protocol is used in this experiment to present human faces of emotion to 28 participants while EEG was measured. We found that artificially enhanced human faces with exaggerated, cartoonish visual features significantly improve some commonly used neural correlates of emotion as measured by event-related potentials (ERPs). These images elicit an enhanced N170 component, well known to relate to the facial visual encoding process. Our findings suggest that the study of emotion elicitation could exploit consistent, high detail, AI generated stimuli transformations to study the characteristics of electrical brain activity related to visual affective stimuli. Furthermore, this specific result might be useful in the context of affective BCI design, where a higher accuracy in affect decoding from EEG can improve the experience of a user.

Limb Preference and Skill Level Dependence During the Imagery of a Whole-Body Movement: A Functional Near Infrared Spectroscopy Study.

In the past years motor imagery (MI) turned out to be also an innovative and effective tool for motor learning and improvement of sports performance. Whereas many studies investigating sports MI focusing on upper or lower limbs involvement, knowledge about involved neural structures during whole-body movements is still limited. In the present study we investigated brain activity of climbers during a kinesthetic motor imagery (KMI) climbing task with different difficulties by means of functional near infrared spectroscopy (fNIRS). Twenty healthy participants were split into two groups according to their climbing skill level. The aim of the current study is investigating neural correlates of a whole-body sports MI task with an additional focus on skill level dependency. Climbing experts and non-experts imagined bouldering an "easy" and "difficult" route from a first-person perspective while hemodynamic responses were recorded simultaneously. We found significant differences between the two climbing routes, easy and difficult within participants as well as between the two groups of different climbing skill levels. Overall beginners showed increased hemodynamic responses compared to experts in all defined regions of interest (ROI) supporting the claim of the neural efficiency hypothesis (NEH). Even though climbing is a complex, coordinated movement of upper and lower limbs we found a stronger activation focus of the upper limbs, especially of the dominant hand-area, while the foot area seems to be deactivated or inhibited simultaneously. Summarizing, these findings provide novel insights into brain activation during the imagery of a whole-body movement and its relation to climbing expertise.

Corrigendum: Mental State Detection Using Riemannian Geometry on Electroencephalogram Brain Signals.

[This corrects the article DOI: 10.3389/fnhum.2021.746081.].

Mental State Detection Using Riemannian Geometry on Electroencephalogram Brain Signals.

The goal of this study was to implement a Riemannian geometry (RG)-based algorithm to detect high mental workload (MWL) and mental fatigue (MF) using task-induced electroencephalogram (EEG) signals. In order to elicit high MWL and MF, the participants performed a cognitively demanding task in the form of the letter n-back task. We analyzed the time-varying characteristics of the EEG band power (BP) features in the theta and alpha frequency band at different task conditions and cortical areas by employing a RG-based framework. MWL and MF were considered as too high, when the Riemannian distances of the task-run EEG reached or surpassed the threshold of the baseline EEG. The results of this study showed a BP increase in the theta and alpha frequency bands with increasing experiment duration, indicating elevated MWL and MF that impedes/hinders the task performance of the participants. High MWL and MF was detected in 8 out of 20 participants. The Riemannian distances also showed a steady increase toward the threshold with increasing experiment duration, with the most detections occurring toward the end of the experiment. To support our findings, subjective ratings (questionnaires concerning fatigue and workload levels) and behavioral measures (performance accuracies and response times) were also considered.

Neural Suppression Elicited During Motor Imagery Following the Observation of Biological Motion From Point-Light Walker Stimuli.

Introduction: Advantageous effects of biological motion (BM) detection, a low-perceptual mechanism that allows the rapid recognition and understanding of spatiotemporal characteristics of movement via salient kinematics information, can be amplified when combined with motor imagery (MI), i.e., the mental simulation of motor acts. According to Jeannerod's neurostimulation theory, asynchronous firing and reduction of mu and beta rhythm oscillations, referred to as suppression over the sensorimotor area, are sensitive to both MI and action observation (AO) of BM. Yet, not many studies investigated the use of BM stimuli using combined AO-MI tasks. In this study, we assessed the neural response in the form of event-related synchronization and desynchronization (ERD/S) patterns following the observation of point-light-walkers and concordant MI, as compared to MI alone. Methods: Twenty right-handed healthy participants accomplished the experimental task by observing BM stimuli and subsequently performing the same movement using kinesthetic MI (walking, cycling, and jumping conditions). We recorded an electroencephalogram (EEG) with 32 channels and performed time-frequency analysis on alpha (8-13 Hz) and beta (18-24 Hz) frequency bands during the MI task. A two-way repeated-measures ANOVA was performed to test statistical significance among conditions and electrodes of interest. Results: The results revealed significant ERD/S patterns in the alpha frequency band between conditions and electrode positions. Post hoc comparisons showed significant differences between condition 1 (walking) and condition 3 (jumping) over the left primary motor cortex. For the beta band, a significantly less difference in ERD patterns (p < 0.01) was detected only between condition 3 (jumping) and condition 4 (reference). Discussion: Our results confirmed that the observation of BM combined with MI elicits a neural suppression, although just in the case of jumping. This is in line with previous findings of AO and MI (AOMI) eliciting a neural suppression for simulated whole-body movements. In the last years, increasing evidence started to support the integration of AOMI training as an adjuvant neurorehabilitation tool in Parkinson's disease (PD). Conclusion: We concluded that using BM stimuli in AOMI training could be promising, as it promotes attention to kinematic features and imitative motor learning.

Inter- and Intra-individual Variability in Brain Oscillations During Sports Motor Imagery.

The aim of this work was to re-evaluate electrophysiological data from a previous study on motor imagery (MI) with a special focus on observed inter- and intra-individual differences. More concretely, we investigated event-related desynchronization/synchronization patterns during sports MI (playing tennis) compared with simple MI (squeezing a ball) and discovered high variability across participants. Thirty healthy volunteers were divided in two groups; the experimental group (EG) performed a physical exercise between two imagery sessions, and the control group (CG) watched a landscape movie without physical activity. We computed inter-individual differences by assessing the dissimilarities among subjects for each group, condition, time period, and frequency band. In the alpha band, we observe some clustering in the ranking of the subjects, therefore showing smaller distances than others. Moreover, in our statistical evaluation, we observed a consistency in ranking across time periods both for the EG and for the CG. For the latter, we also observed similar rankings across conditions. On the contrary, in the beta band, the ranking of the subjects was more similar for the EG across conditions and time periods than for the subjects of the CG. With this study, we would like to draw attention to variability measures instead of primarily focusing on the identification of common patterns across participants, which often do not reflect the whole neurophysiological reality.

NICA: A Novel Toolbox for Near-Infrared Spectroscopy Calculations and Analyses.

Functional near-infrared spectroscopy (fNIRS) measures the functional activity of the cerebral cortex. The concentration changes of oxygenated (oxy-Hb) and deoxygenated hemoglobin (deoxy-Hb) can be detected and associated with activation of the cortex in the investigated area (neurovascular coupling). Recorded signals of hemodynamic responses may contain influences from physiological signals (systemic influences, physiological artifacts) which do not originate from the cerebral cortex activity. The physiological artifacts contain the blood pressure (BP), respiratory patterns, and the pulsation of the heart. In order to perform a comprehensive analysis of recorded fNIRS data, a proper correction of these physiological artifacts is necessary. This article introduces NICA - a novel toolbox for near-infrared spectroscopy calculations and analyses based on MATLAB. With NICA it is possible to process and visualize fNIRS data, including different signal processing methods for physiological artifact correction. The artifact correction methods used in this toolbox are common average reference (CAR), independent component analysis (ICA), and transfer function (TF) models. A practical example provides results from a study, where NICA was used for analyzing the measurement data, in order to demonstrate the signal processing steps and the physiological artifact correction. The toolbox was developed for fNIRS data recorded with the NIRScout 1624 measurement device and the corresponding recording software NIRStar.

Frequency Specific Cortical Dynamics During Motor Imagery Are Influenced by Prior Physical Activity.

Motor imagery is often used inducing changes in electroencephalographic (EEG) signals for imagery-based brain-computer interfacing (BCI). A BCI is a device translating brain signals into control signals providing severely motor-impaired persons with an additional, non-muscular channel for communication and control. In the last years, there is increasing interest using BCIs also for healthy people in terms of enhancement or gaming. Most studies focusing on improving signal processing feature extraction and classification methods, but the performance of a BCI can also be improved by optimizing the user's control strategies, e.g., using more vivid and engaging mental tasks for control. We used multichannel EEG to investigate neural correlates of a sports imagery task (playing tennis) compared to a simple motor imagery task (squeezing a ball). To enhance the vividness of both tasks participants performed a short physical exercise between two imagery sessions. EEG was recorded from 60 closely spaced electrodes placed over frontal, central, and parietal areas of 30 healthy volunteers divided in two groups. Whereas Group 1 (EG) performed a physical exercise between the two imagery sessions, Group 2 (CG) watched a landscape movie without physical activity. Spatiotemporal event-related desynchronization (ERD) and event-related synchronization (ERS) patterns during motor imagery (MI) tasks were evaluated. The results of the EG showed significant stronger ERD patterns in the alpha frequency band (8-13 Hz) during MI of tennis after training. Our results are in evidence with previous findings that MI in combination with motor execution has beneficial effects. We conclude that sports MI combined with an interactive game environment could be a future promising task in motor learning and rehabilitation improving motor functions in late therapy processes or support neuroplasticity.

Imagine squeezing a cactus: Cortical activation during affective motor imagery measured by functional near-infrared spectroscopy.

The activation of different brain areas during kinaesthetic and visual motor imagery has been extensively studied, whereas little is known about affective motor imagery, i.e. the imagery of pleasant/unpleasant movements. In the present neuroimaging study we investigated cortical activation of kinaesthetic motor imagery (KMI) based on emotional stimulus content by means of functional near infrared spectroscopy (fNIRS). Twenty healthy adult participants were instructed to imagine affective, and neutral motor tasks while multichannel fNIRS was recorded simultaneously. In the affective MI condition they had to imagine e.g. squeezing a cactus with their right hand several times, eliciting an unpleasant emotion. In the neutral condition their task was to imagine squeezing a ball. Significant differences in oxy-hemoglobin [oxy-Hb] concentration changes during KMI including affective objects in different brain regions were found. Specifically activation in left parietal and frontal regions was increased during the imagery of squeezing a cactus which induced a painful feeling. Both areas are also involved in the perception of pain and commonly labelled as parts of the "pain matrix". Our study provides novel insights in cortical activation patterns during affective motor imagery and its psychological and cognitive mechanisms underlying pain experience.

Action affordances and visuo-spatial complexity in motor imagery: An fMRI study.

Imagining a complex action requires not only motor-related processing but also visuo-spatial imagery. In the current study, we examined visuo-spatial complexity and action affordances in motor imagery (MI). Using functional magnetic resonance imaging, we investigated the neural activity in MI of reach-to-grasp movements of the right hand in five conditions. Thirty participants were scanned while imagining grasping an everyday object, grasping a geometrical shape, grasping next to an everyday object, grasping next to a geometrical shape, and grasping at nothing (no object involved). We found that MI of grasping next to an object recruited the visuo-spatial cognition network including posterior parietal and premotor regions more strongly than MI of grasping an object. This indicates that grasping next to an object requires additional processing resources rendering MI more complex. MI of a grasping movement involving a familiar everyday object compared to a geometrical shape yielded stronger activation in motor-related regions, including the bilateral supplementary motor area. This activation might be due to inhibitory processes preventing motor execution of motor scripts evoked by everyday objects (action affordances). Our results indicate that visuo-spatial cognition plays a significant role in MI.

Cortical activation patterns to spatially presented pure tone stimuli with different intensities measured by functional near-infrared spectroscopy.

Functional near-infrared spectroscopy (fNIRS) is an emerging technique for the assessment of functional activity of the cerebral cortex. Recently fNIRS was also envisaged as a novel neuroimaging approach for measuring the auditory cortex activity in the field of in auditory diagnostics. This study aimed to investigate differences in brain activity related to spatially presented sounds with different intensities in 10 subjects by means of functional near-infrared spectroscopy (fNIRS). We found pronounced cortical activation patterns in the temporal and frontal regions of both hemispheres. In contrast to these activation patterns, we found deactivation patterns in central and parietal regions of both hemispheres. Furthermore our results showed an influence of spatial presentation and intensity of the presented sounds on brain activity in related regions of interest. These findings are in line with previous fMRI studies which also reported systematic changes of activation in temporal and frontal areas with increasing sound intensity. Although clear evidence for contralaterality effects and hemispheric asymmetries were absent in the group data, these effects were partially visible on the single subject level. Concluding, fNIRS is sensitive enough to capture differences in brain responses during the spatial presentation of sounds with different intensities in several cortical regions. Our results may serve as a valuable contribution for further basic research and the future use of fNIRS in the area of central auditory diagnostics.

Composing only by thought: Novel application of the P300 brain-computer interface.

The P300 event-related potential is a well-known pattern in the electroencephalogram (EEG). This kind of brain signal is used for many different brain-computer interface (BCI) applications, e.g., spellers, environmental controllers, web browsers, or for painting. In recent times, BCI systems are mature enough to leave the laboratories to be used by the end-users, namely severely disabled people. Therefore, new challenges arise and the systems should be implemented and evaluated according to user-centered design (USD) guidelines. We developed and implemented a new system that utilizes the P300 pattern to compose music. Our Brain Composing system consists of three parts: the EEG acquisition device, the P300-based BCI, and the music composing software. Seventeen musical participants and one professional composer performed a copy-spelling, a copy-composing, and a free-composing task with the system. According to the USD guidelines, we investigated the efficiency, the effectiveness and subjective criteria in terms of satisfaction, enjoyment, frustration, and attractiveness. The musical participants group achieved high average accuracies: 88.24% (copy-spelling), 88.58% (copy-composing), and 76.51% (free-composing). The professional composer achieved also high accuracies: 100% (copy-spelling), 93.62% (copy-composing), and 98.20% (free-composing). General results regarding the subjective criteria evaluation were that the participants enjoyed the usage of the Brain Composing system and were highly satisfied with the system. Showing very positive results with healthy people in this study, this was the first step towards a music composing system for severely disabled people.

Force related hemodynamic responses during execution and imagery of a hand grip task: A functional near infrared spectroscopy study.

We examined force related hemodynamic changes during the performance of a motor execution (ME) and motor imagery (MI) task by means of multichannel functional near infrared spectroscopy (fNIRS). The hemodynamic responses of fourteen healthy participants were measured while they performed a hand grip execution or imagery task with low and high grip forces. We found an overall higher increase of [oxy-Hb] concentration changes during ME for both grip forces but with a delayed peak maximum for the lower grip force. During the MI task with lower grip force, the [oxy-Hb] level increases are stronger compared to the MI with higher grip force. The facilitation in performing MI with higher grip strength might thus indicate less inhibition of the actual motor act which could also explain the later increase onset of [oxy-Hb] in the ME task with the lower grip force. Our results suggest that execution and imagery of a hand grip task with high and low grip forces, leads to different cortical activation patterns. Since impaired control of grip forces during object manipulation in particular is one aspect of fine motor control deficits after stroke, our study will contribute to future rehabilitation programs enhancing patient's grip force control.

Evaluation of Different EEG Acquisition Systems Concerning Their Suitability for Building a Brain-Computer Interface: Case Studies.

One important aspect in non-invasive brain-computer interface (BCI) research is to acquire the electroencephalogram (EEG) in a proper way. From an end-user perspective, it means with maximum comfort and without any extra inconveniences (e.g., washing the hair), whereas from a technical perspective, the signal quality has to be optimal to make the BCI work effectively and efficiently. In this work, we evaluated three different commercially available EEG acquisition systems that differ in the type of electrodes (gel-, water-, and dry-based), the amplifier technique, and the data transmission method. Every system was tested regarding three different aspects, namely, technical, BCI effectiveness and efficiency (P300 communication and control), and user satisfaction (comfort). We found that water-based system had the lowest short circuit noise level, the hydrogel-based system had the highest P300 spelling accuracies, and the dry electrode-based system caused the least inconveniences. Therefore, building a reliable BCI is possible with all the evaluated systems, and it is on the user to decide which system meets the given requirements best.

Brain Computer Interface on Track to Home.

The novel BackHome system offers individuals with disabilities a range of useful services available via brain-computer interfaces (BCIs), to help restore their independence. This is the time such technology is ready to be deployed in the real world, that is, at the target end users' home. This has been achieved by the development of practical electrodes, easy to use software, and delivering telemonitoring and home support capabilities which have been conceived, implemented, and tested within a user-centred design approach. The final BackHome system is the result of a 3-year long process involving extensive user engagement to maximize effectiveness, reliability, robustness, and ease of use of a home based BCI system. The system is comprised of ergonomic and hassle-free BCI equipment; one-click software services for Smart Home control, cognitive stimulation, and web browsing; and remote telemonitoring and home support tools to enable independent home use for nonexpert caregivers and users. BackHome aims to successfully bring BCIs to the home of people with limited mobility to restore their independence and ultimately improve their quality of life.

Control or non-control state: that is the question! An asynchronous visual P300-based BCI approach.

OBJECTIVE: Brain-computer interfaces (BCI) based on event-related potentials (ERP) were proven to be a reliable synchronous communication method. For everyday life situations, however, this synchronous mode is impractical because the system will deliver a selection even if the user is not paying attention to the stimulation. So far, research into attention-aware visual ERP-BCIs (i.e., asynchronous ERP-BCIs) has led to variable success. In this study, we investigate new approaches for detection of user engagement. APPROACH: Classifier output and frequency-domain features of electroencephalogram signals as well as the hybridization of them were used to detect the user's state. We tested their capabilities for state detection in different control scenarios on offline data from 21 healthy volunteers. MAIN RESULTS: The hybridization of classifier output and frequency-domain features outperformed the results of the single methods, and allowed building an asynchronous P300-based BCI with an average correct state detection accuracy of more than 95%. SIGNIFICANCE: Our results show that all introduced approaches for state detection in an asynchronous P300-based BCI can effectively avoid involuntary selections, and that the hybrid method is the most effective approach.

Sheet music by mind: Towards a brain-computer interface for composing.

Providing brain-computer interface (BCI) users engaging applications should be one of the main targets in BCI research. A painting application, a web browser and other applications can already be controlled via BCI. Another engaging application would be a music composer for self-expression. In this work, we describe Brain Composing: A BCI controlled music composing software. We tested and evaluated the implemented brain composing system with five volunteers. Using a tap water-based electrode biosignal amplifier further improved the usability of the system. Three participants reached accuracies above 77% and were able to copy-compose a given melody. Results of questionnaires support that our brain composing system is an attractive and easy way to compose music via a BCI.

Brain-controlled applications using dynamic P300 speller matrices.

OBJECTIVES: Access to the world wide web and multimedia content is an important aspect of life. We present a web browser and a multimedia user interface adapted for control with a brain-computer interface (BCI) which can be used by severely motor impaired persons. METHODS: The web browser dynamically determines the most efficient P300 BCI matrix size to select the links on the current website. This enables control of the web browser with fewer commands and smaller matrices. The multimedia player was based on an existing software. Both applications were evaluated with a sample of ten healthy participants and three end-users. All participants used a visual P300 BCI with face-stimuli for control. RESULTS: The healthy participants completed the multimedia player task with 90% accuracy and the web browsing task with 85% accuracy. The end-users completed the tasks with 62% and 58% accuracy. All healthy participants and two out of three end-users reported that they felt to be in control of the system. CONCLUSIONS: In this study we presented a multimedia application and an efficient web browser implemented for control with a BCI. SIGNIFICANCE: Both applications provide access to important areas of modern information retrieval and entertainment.