Imagined speech event detection from electrocorticography and its transfer between speech modes and subjects.

Speech brain-computer interfaces aim to support communication-impaired patients by translating neural signals into speech. While impressive progress was achieved in decoding performed, perceived and attempted speech, imagined speech remains elusive, mainly due to the absence of behavioral output. Nevertheless, imagined speech is advantageous since it does not depend on any articulator movements that might become impaired or even lost throughout the stages of a neurodegenerative disease. In this study, we analyzed electrocortigraphy data recorded from 16 participants in response to 3 speech modes: performed, perceived (listening), and imagined speech. We used a linear model to detect speech events and examined the contributions of each frequency band, from delta to high gamma, given the speech mode and electrode location. For imagined speech detection, we observed a strong contribution of gamma bands in the motor cortex, whereas lower frequencies were more prominent in the temporal lobe, in particular of the left hemisphere. Based on the similarities in frequency patterns, we were able to transfer models between speech modes and participants with similar electrode locations.

Temporal-spatial cross attention network for recognizing imagined characters.

Previous research has primarily employed deep learning models such as Convolutional Neural Networks (CNNs), and Recurrent Neural Networks (RNNs) for decoding imagined character signals. These approaches have treated the temporal and spatial features of the signals in a sequential, parallel, or single-feature manner. However, there has been limited research on the cross-relationships between temporal and spatial features, despite the inherent association between channels and sampling points in Brain-Computer Interface (BCI) signal acquisition, which holds significant information about brain activity. To address the limited research on the relationships between temporal and spatial features, we proposed a Temporal-Spatial Cross-Attention Network model, named TSCA-Net. The TSCA-Net is comprised of four modules: the Temporal Feature (TF), the Spatial Feature (SF), the Temporal-Spatial Cross (TSCross), and the Classifier. The TF combines LSTM and Transformer to extract temporal features from BCI signals, while the SF captures spatial features. The TSCross is introduced to learn the correlations between the temporal and spatial features. The Classifier predicts the label of BCI data based on its characteristics. We validated the TSCA-Net model using publicly available datasets of handwritten characters, which recorded the spiking activity from two micro-electrode arrays (MEAs). The results showed that our proposed TSCA-Net outperformed other comparison models (EEG-Net, EEG-TCNet, S3T, GRU, LSTM, R-Transformer, and ViT) in terms of accuracy, precision, recall, and F1 score, achieving 92.66 % , 92.77 % , 92.70 % , and 92.58 % , respectively. The TSCA-Net model demonstrated a 3.65 % to 7.49 % improvement in accuracy over the comparison models.

Modeling the contribution of theta-gamma coupling to sequential memory, imagination, and dreaming.

Gamma oscillations nested in a theta rhythm are observed in the hippocampus, where are assumed to play a role in sequential episodic memory, i.e., memorization and retrieval of events that unfold in time. In this work, we present an original neurocomputational model based on neural masses, which simulates the encoding of sequences of events in the hippocampus and subsequent retrieval by exploiting the theta-gamma code. The model is based on a three-layer structure in which individual Units oscillate with a gamma rhythm and code for individual features of an episode. The first layer (working memory in the prefrontal cortex) maintains a cue in memory until a new signal is presented. The second layer (CA3 cells) implements an auto-associative memory, exploiting excitatory and inhibitory plastic synapses to recover an entire episode from a single feature. Units in this layer are disinhibited by a theta rhythm from an external source (septum or Papez circuit). The third layer (CA1 cells) implements a hetero-associative net with the previous layer, able to recover a sequence of episodes from the first one. During an encoding phase, simulating high-acetylcholine levels, the network is trained with Hebbian (synchronizing) and anti-Hebbian (desynchronizing) rules. During retrieval (low-acetylcholine), the network can correctly recover sequences from an initial cue using gamma oscillations nested inside the theta rhythm. Moreover, in high noise, the network isolated from the environment simulates a mind-wandering condition, randomly replicating previous sequences. Interestingly, in a state simulating sleep, with increased noise and reduced synapses, the network can "dream" by creatively combining sequences, exploiting features shared by different episodes. Finally, an irrational behavior (erroneous superimposition of features in various episodes, like "delusion") occurs after pathological-like reduction in fast inhibitory synapses. The model can represent a straightforward and innovative tool to help mechanistically understand the theta-gamma code in different mental states.

Tactile versus motor imagery: differences in corticospinal excitability assessed with single-pulse TMS.

Tactile Imagery (TI) remains a fairly understudied phenomenon despite growing attention to this topic in recent years. Here, we investigated the effects of TI on corticospinal excitability by measuring motor evoked potentials (MEPs) induced by single-pulse transcranial magnetic stimulation (TMS). The effects of TI were compared with those of tactile stimulation (TS) and kinesthetic motor imagery (kMI). Twenty-two participants performed three tasks in randomly assigned order: imagine finger tapping (kMI); experience vibratory sensations in the middle finger (TS); and mentally reproduce the sensation of vibration (TI). MEPs increased during both kMI and TI, with a stronger increase for kMI. No statistically significant change in MEP was observed during TS. The demonstrated differential effects of kMI, TI and TS on corticospinal excitability have practical implications for devising the imagery-based and TS-based brain-computer interfaces (BCIs), particularly the ones intended to improve neurorehabilitation by evoking plasticity changes in sensorimotor circuitry.

A cross-dataset adaptive domain selection transfer learning framework for motor imagery-based brain-computer interfaces.

Objective. In brain-computer interfaces (BCIs) that utilize motor imagery (MI), minimizing calibration time has become increasingly critical for real-world applications. Recently, transfer learning (TL) has been shown to effectively reduce the calibration time in MI-BCIs. However, variations in data distribution among subjects can significantly influence the performance of TL in MI-BCIs.Approach.We propose a cross-dataset adaptive domain selection transfer learning framework that integrates domain selection, data alignment, and an enhanced common spatial pattern (CSP) algorithm. Our approach uses a huge dataset of 109 subjects as the source domain. We begin by identifying non-BCI illiterate subjects from this huge dataset, then determine the source domain subjects most closely aligned with the target subjects using maximum mean discrepancy. After undergoing Euclidean alignment processing, features are extracted by multiple composite CSP. The final classification is carried out using the support vector machine.Main results.Our findings indicate that the proposed technique outperforms existing methods, achieving classification accuracies of 75.05% and 76.82% in two cross-dataset experiments, respectively.Significance.By reducing the need for extensive training data, yet maintaining high accuracy, our method optimizes the practical implementation of MI-BCIs.

Evaluating the Feasibility of Visual Imagery for an EEG-Based Brain-Computer Interface.

Visual imagery, or the mental simulation of visual information from memory, could serve as an effective control paradigm for a brain-computer interface (BCI) due to its ability to directly convey the user's intention with many natural ways of envisioning an intended action. However, multiple initial investigations into using visual imagery as a BCI control strategies have been unable to fully evaluate the capabilities of true spontaneous visual mental imagery. One major limitation in these prior works is that the target image is typically displayed immediately preceding the imagery period. This paradigm does not capture spontaneous mental imagery as would be necessary in an actual BCI application but something more akin to short-term retention in visual working memory. Results from the present study show that short-term visual imagery following the presentation of a specific target image provides a stronger, more easily classifiable neural signature in EEG than spontaneous visual imagery from long-term memory following an auditory cue for the image. We also show that short-term visual imagery and visual perception share commonalities in the most predictive electrodes and spectral features. However, visual imagery received greater influence from frontal electrodes whereas perception was mostly confined to occipital electrodes. This suggests that visual perception is primarily driven by sensory information whereas visual imagery has greater contributions from areas associated with memory and attention. This work provides the first direct comparison of short-term and long-term visual imagery tasks and provides greater insight into the feasibility of using visual imagery as a BCI control strategy.

An EEG channel selection method for motor imagery based on Fisher score and local optimization.

Objective. Multi-channel electroencephalogram (EEG) technology in brain-computer interface (BCI) research offers the advantage of enhanced spatial resolution and system performance. However, this also implies that more time is needed in the data processing stage, which is not conducive to the rapid response of BCI. Hence, it is a necessary and challenging task to reduce the number of EEG channels while maintaining decoding effectiveness.Approach. In this paper, we propose a local optimization method based on the Fisher score for within-subject EEG channel selection. Initially, we extract the common spatial pattern characteristics of EEG signals in different bands, calculate Fisher scores for each channel based on these characteristics, and rank them accordingly. Subsequently, we employ a local optimization method to finalize the channel selection.Main results. On the BCI Competition IV Dataset IIa, our method selects an average of 11 channels across four bands, achieving an average accuracy of 79.37%. This represents a 6.52% improvement compared to using the full set of 22 channels. On our self-collected dataset, our method similarly achieves a significant improvement of 24.20% with less than half of the channels, resulting in an average accuracy of 76.95%.Significance. This research explores the importance of channel combinations in channel selection tasks and reveals that appropriately combining channels can further enhance the quality of channel selection. The results indicate that the model selected a small number of channels with higher accuracy in two-class motor imagery EEG classification tasks. Additionally, it improves the portability of BCI systems through channel selection and combinations, offering the potential for the development of portable BCI systems.

Motor imagery drives the effects of combined action observation and motor imagery on corticospinal excitability for coordinative lower-limb actions.

Combined action observation and motor imagery (AOMI) facilitates corticospinal excitability (CSE) and may potentially induce plastic-like changes in the brain in a similar manner to physical practice. This study used transcranial magnetic stimulation (TMS) to explore changes in CSE for AOMI of coordinative lower-limb actions. Twenty-four healthy adults completed two baseline (BLH, BLNH) and three AOMI conditions, where they observed a knee extension while simultaneously imagining the same action (AOMICONG), plantarflexion (AOMICOOR-FUNC), or dorsiflexion (AOMICOOR-MOVE). Motor evoked potential (MEP) amplitudes were recorded as a marker of CSE for all conditions from two knee extensor, one dorsi flexor, and two plantar flexor muscles following TMS to the right leg representation of the left primary motor cortex. A main effect for experimental condition was reported for all three muscle groups. MEP amplitudes were significantly greater in the AOMICONG condition compared to the BLNH condition (p = .04) for the knee extensors, AOMICOOR-FUNC condition compared to the BLH condition (p = .03) for the plantar flexors, and AOMICOOR-MOVE condition compared to the two baseline conditions for the dorsi flexors (ps ≤ .01). The study findings support the notion that changes in CSE are driven by the imagined actions during coordinative AOMI.

Representations of imaginary scenes and their properties in cortical alpha activity.

Imagining natural scenes enables us to engage with a myriad of simulated environments. How do our brains generate such complex mental images? Recent research suggests that cortical alpha activity carries information about individual objects during visual imagery. However, it remains unclear if more complex imagined contents such as natural scenes are similarly represented in alpha activity. Here, we answer this question by decoding the contents of imagined scenes from rhythmic cortical activity patterns. In an EEG experiment, participants imagined natural scenes based on detailed written descriptions, which conveyed four complementary scene properties: openness, naturalness, clutter level and brightness. By conducting classification analyses on EEG power patterns across neural frequencies, we were able to decode both individual imagined scenes as well as their properties from the alpha band, showing that also the contents of complex visual images are represented in alpha rhythms. A cross-classification analysis between alpha power patterns during the imagery task and during a perception task, in which participants were presented images of the described scenes, showed that scene representations in the alpha band are partly shared between imagery and late stages of perception. This suggests that alpha activity mediates the top-down re-activation of scene-related visual contents during imagery.

A diagonal masking self-attention-based multi-scale network for motor imagery classification.

Objective. Electroencephalography (EEG)-based motor imagery (MI) is a promising paradigm for brain-computer interface (BCI), but the non-stationarity and low signal-to-noise ratio of EEG signals make it a challenging task.Approach. To achieve high-precision MI classification, we propose a Diagonal Masking Self-Attention-based Multi-Scale Network (DMSA-MSNet) to fully develop, extract, and emphasize features from different scales. First, for local features, a multi-scale temporal-spatial block is proposed to extract features from different receptive fields. Second, an adaptive branch fusion block is specifically designed to bridge the semantic gap between these coded features from different scales. Finally, in order to analyze global information over long ranges, a diagonal masking self-attention block is introduced, which highlights the most valuable features in the data.Main results. The proposed DMSA-MSNet outperforms state-of-the-art models on the BCI Competition IV 2a and the BCI Competition IV 2b datasets.Significance. Our study achieves rich information extraction from EEG signals and provides an effective solution for MI classification.

Large-Scale Cortical Network Analysis and Classification of MI-BCI Tasks Based on Bayesian Nonnegative Matrix Factorization.

Motor imagery (MI) is a high-level cognitive process that has been widely applied to clinical rehabilitation and brain-computer interfaces (BCIs). However, the decoding of MI tasks still faces challenges, and the neural mechanisms underlying its application are unclear, which seriously hinders the development of MI-based clinical applications and BCIs. Here, we combined EEG source reconstruction and Bayesian nonnegative matrix factorization (NMF) methods to construct large-scale cortical networks of left-hand and right-hand MI tasks. Compared to right-hand MI, the results showed that the significantly increased functional network connectivities (FNCs) mainly located among the visual network (VN), sensorimotor network (SMN), right temporal network, right central executive network, and right parietal network in the left-hand MI at the ß (13-30Hz) and all (8-30Hz) frequency bands. For the network properties analysis, we found that the clustering coefficient, global efficiency, and local efficiency were significantly increased and characteristic path length was significantly decreased in left-hand MI compared to right-hand MI at the ß and all frequency bands. These network pattern differences indicated that the left-hand MI may need more modulation of multiple large-scale networks (i.e., VN and SMN) mainly located in the right hemisphere. Finally, based on the spatial pattern network of FNC and network properties, we propose a classification model. The proposed model achieves a top classification accuracy of 78.2% in cross-subject two-class MI-BCI tasks. Overall, our findings provide new insights into the neural mechanisms of MI and a potential network biomarker to identify MI-BCI tasks.

Motor Imagery EEG Signal Classification Using Distinctive Feature Fusion with Adaptive Structural LASSO.

A motor imagery brain-computer interface connects the human brain and computers via electroencephalography (EEG). However, individual differences in the frequency ranges of brain activity during motor imagery tasks pose a challenge, limiting the manual feature extraction for motor imagery classification. To extract features that match specific subjects, we proposed a novel motor imagery classification model using distinctive feature fusion with adaptive structural LASSO. Specifically, we extracted spatial domain features from overlapping and multi-scale sub-bands of EEG signals and mined discriminative features by fusing the task relevance of features with spatial information into the adaptive LASSO-based feature selection. We evaluated the proposed model on public motor imagery EEG datasets, demonstrating that the model has excellent performance. Meanwhile, ablation studies and feature selection visualization of the proposed model further verified the great potential of EEG analysis.

[A study on the effects of transcranial direct current stimulation combined with motor imagery on brain function based on electroencephalogram and near infrared spectrum].

Motor imagery is often used in the fields of sports training and neurorehabilitation for its advantages of being highly targeted, easy to learn, and requiring no special equipment, and has become a major research paradigm in cognitive neuroscience. Transcranial direct current stimulation (tDCS), an emerging neuromodulation technique, modulates cortical excitability, which in turn affects functions such as locomotion. However, it is unclear whether tDCS has a positive effect on motor imagery task states. In this paper, 16 young healthy subjects were included, and the electroencephalogram (EEG) signals and near-infrared spectrum (NIRS) signals of the subjects were collected when they were performing motor imagery tasks before and after receiving tDCS, and the changes in multiscale sample entropy (MSE) and haemoglobin concentration were calculated and analyzed during the different tasks. The results found that MSE of task-related brain regions increased, oxygenated haemoglobin concentration increased, and total haemoglobin concentration rose after tDCS stimulation, indicating that tDCS increased the activation of task-related brain regions and had a positive effect on motor imagery. This study may provide some reference value for the clinical study of tDCS combined with motor imagery.

Neurophysiologically Meaningful Motor Imagery EEG Simulation With Applications to Data Augmentation.

Motor imagery-based Brain-Computer Interfaces (MI-BCIs) have gained a lot of attention due to their potential usability in neurorehabilitation and neuroprosthetics. However, the accurate recognition of MI patterns in electroencephalography signals (EEG) is hindered by several data-related limitations, which restrict the practical utilization of these systems. Moreover, leveraging deep learning (DL) models for MI decoding is challenged by the difficulty of accessing user-specific MI-EEG data on large scales. Simulated MI-EEG signals can be useful to address these issues, providing well-defined data for the validation of decoding models and serving as a data augmentation approach to improve the training of DL models. While substantial efforts have been dedicated to implementing effective data augmentation strategies and model-based EEG signal generation, the simulation of neurophysiologically plausible EEG-like signals has not yet been exploited in the context of data augmentation. Furthermore, none of the existing approaches have integrated user-specific neurophysiological information during the data generation process. Here, we present PySimMIBCI, a framework for generating realistic MI-EEG signals by integrating neurophysiologically meaningful activity into biophysical forward models. By means of PySimMIBCI, different user capabilities to control an MI-BCI can be simulated and fatigue effects can be included in the generated EEG. Results show that our simulated data closely resemble real data. Moreover, a proposed data augmentation strategy based on our simulated user-specific data significantly outperforms other state-of-the-art augmentation approaches, enhancing DL models performance by up to 15%.

A two-stage transformer based network for motor imagery classification.

Brain-computer interfaces (BCIs) are used to understand brain functioning and develop therapies for neurological and neurodegenerative disorders. Therefore, BCIs are crucial in rehabilitating motor dysfunction and advancing motor imagery applications. For motor imagery, electroencephalogram (EEG) signals are used to classify the subject's intention of moving a body part without actually moving it. This paper presents a two-stage transformer-based architecture that employs handcrafted features and deep learning techniques to enhance the classification performance on benchmarked EEG signals. Stage-1 is built on parallel convolution based EEGNet, multi-head attention, and separable temporal convolution networks for spatiotemporal feature extraction. Further, for enhanced classification, in stage-2, additional features and embeddings extracted from stage-1 are used to train TabNet. In addition, a novel channel cluster swapping data augmentation technique is also developed to handle the issue of limited samples for training deep learning architectures. The developed two-stage architecture offered an average classification accuracy of 88.5 % and 88.3 % on the BCI Competition IV-2a and IV-2b datasets, respectively, which is approximately 3.0 % superior over similar recent reported works.

Multi-scale self-attention approach for analysing motor imagery signals in brain-computer interfaces.

BACKGROUND: Motor imagery-based electroencephalogram (EEG) brain-computer interface (BCI) technology has seen tremendous advancements in the past several years. Deep learning has outperformed more traditional approaches, such next-gen neuro-technologies, in terms of productivity. It is still challenging to develop and train an end-to-end network that can sufficiently extract the possible characteristics from EEG data used in motor imaging. Brain-computer interface research is largely reliant on the fundamental problem of accurately classifying EEG data. There are still many challenges in the field of MI classification even after researchers have proposed a variety of methods, such as deep learning and machine learning techniques. METHODOLOGY: We provide a model for four-class categorization of motor imagery EEG signals using attention mechanisms: left hand, right hand, foot, and tongue/rest. The model is built on multi-scale spatiotemporal self-attention networks. To determine the most effective channels, self-attention networks are implemented spatially to assign greater weight to channels associated with motion and lesser weight to channels unrelated to motion. To eliminate noise in the temporal domain, parallel multi-scale Temporal Convolutional Network (TCN) layers are utilized to extract temporal domain features at various scales. RESULT: On the IV-2b dataset from the BCI Competition, the suggested model achieved an accuracy of 85.09 %; on the IV-2a and IV-2b datasets from the HGD datasets, it was 96.26 %. COMPARISON WITH EXISTING METHODS: In single-subject classification, this approach demonstrates superior accuracy when compared to existing methods. CONCLUSION: The findings suggest that this approach exhibits commendable performance, resilience, and capacity for transfer learning.

Motor imagery-based brain-computer interface rehabilitation programs enhance upper extremity performance and cortical activation in stroke patients.

BACKGROUND: The most challenging aspect of rehabilitation is the repurposing of residual functional plasticity in stroke patients. To achieve this, numerous plasticity-based clinical rehabilitation programs have been developed. This study aimed to investigate the effects of motor imagery (MI)-based brain-computer interface (BCI) rehabilitation programs on upper extremity hand function in patients with chronic hemiplegia. DESIGN: A 2010 Consolidated Standards for Test Reports (CONSORT)-compliant randomized controlled trial. METHODS: Forty-six eligible stroke patients with upper limb motor dysfunction participated in the study, six of whom dropped out. The patients were randomly divided into a BCI group and a control group. The BCI group received BCI therapy and conventional rehabilitation therapy, while the control group received conventional rehabilitation only. The Fugl-Meyer Assessment of the Upper Extremity (FMA-UE) score was used as the primary outcome to evaluate upper extremity motor function. Additionally, functional magnetic resonance imaging (fMRI) scans were performed on all patients before and after treatment, in both the resting and task states. We measured the amplitude of low-frequency fluctuation (ALFF), regional homogeneity (ReHo), z conversion of ALFF (zALFF), and z conversion of ReHo (ReHo) in the resting state. The task state was divided into four tasks: left-hand grasping, right-hand grasping, imagining left-hand grasping, and imagining right-hand grasping. Finally, meaningful differences were assessed using correlation analysis of the clinical assessments and functional measures. RESULTS: A total of 40 patients completed the study, 20 in the BCI group and 20 in the control group. Task-related blood-oxygen-level-dependent (BOLD) analysis showed that when performing the motor grasping task with the affected hand, the BCI group exhibited significant activation in the ipsilateral middle cingulate gyrus, precuneus, inferior parietal gyrus, postcentral gyrus, middle frontal gyrus, superior temporal gyrus, and contralateral middle cingulate gyrus. When imagining a grasping task with the affected hand, the BCI group exhibited greater activation in the ipsilateral superior frontal gyrus (medial) and middle frontal gyrus after treatment. However, the activation of the contralateral superior frontal gyrus decreased in the BCI group relative to the control group. Resting-state fMRI revealed increased zALFF in multiple cerebral regions, including the contralateral precentral gyrus and calcarine and the ipsilateral middle occipital gyrus and cuneus, and decreased zALFF in the ipsilateral superior temporal gyrus in the BCI group relative to the control group. Increased zReHo in the ipsilateral cuneus and contralateral calcarine and decreased zReHo in the contralateral middle temporal gyrus, temporal pole, and superior temporal gyrus were observed post-intervention. According to the subsequent correlation analysis, the increase in the FMA-UE score showed a positive correlation with the mean zALFF of the contralateral precentral gyrus (r = 0.425, P < 0.05), the mean zReHo of the right cuneus (r = 0.399, P < 0.05). CONCLUSION: In conclusion, BCI therapy is effective and safe for arm rehabilitation after severe poststroke hemiparesis. The correlation of the zALFF of the contralateral precentral gyrus and the zReHo of the ipsilateral cuneus with motor improvements suggested that these values can be used as prognostic measures for BCI-based stroke rehabilitation. We found that motor function was related to visual and spatial processing, suggesting potential avenues for refining treatment strategies for stroke patients. TRIAL REGISTRATION: The trial is registered in the Chinese Clinical Trial Registry (number ChiCTR2000034848, registered July 21, 2020).

A novel feature extraction method PSS-CSP for binary motor imagery - based brain-computer interfaces.

In order to improve the performance of binary motor imagery (MI) - based brain-computer interfaces (BCIs) using electroencephalography (EEG), a novel method (PSS-CSP) is proposed, which combines spectral subtraction with common spatial pattern. Spectral subtraction is an effective denoising method which is initially adopted to process MI-based EEG signals for binary BCIs in this work. On this basis, we proposed a novel feature extraction method called power spectral subtraction-based common spatial pattern (PSS-CSP) , which calculates the differences in power spectrum between binary classes of EEG signals and uses the differences in the feature extraction process. Additionally, support vector machine (SVM) algorithm is used for signal classification. Results show the proposed method (PSS-CSP) outperforms certain existing methods, achieving a classification accuracy of 76.8% on the BCIIV dataset 2b, and 76.25% and 77.38% on the OpenBMI dataset session 1 and session 2, respectively.

Hypoactivation of the language network during auditory imagery contributes to hallucinations in Schizophrenia.

Auditory verbal hallucinations (AVHs) involve perceptions, often voices, in the absence of external stimuli, and rank among the most common symptoms of schizophrenia. Metrical stress evaluation requires determination of the stronger syllable in words, and therefore requires auditory imagery, of interest for investigation of hallucinations in schizophrenia. The current functional magnetic resonance imaging study provides an updated whole-brain network analysis of a previously published study on metrical stress, which showed reduced directed connections between Broca's and Wernicke's regions of interest (ROIs) for hallucinations. Three functional brain networks were extracted, with the language network (LN) showing an earlier and shallower blood-oxygen-level dependent (BOLD) response for hallucinating patients, in the auditory imagery condition only (the reduced activation for hallucinations observed in the original ROI-based results were not specific to the imagery condition). This suggests that hypoactivation of the LN during internal auditory imagery may contribute to the propensity to hallucinate. This accords with cognitive accounts holding that an impaired balance between internal and external linguistic processes (underactivity in networks involved in internal auditory imagery and overactivity in networks involved in speech perception) contributes to our understanding of the biological underpinnings of hallucinations.

How workload and availability of spatial reference shape eye movement coupling in visuospatial working memory.

Eyes are active in memory recall and visual imagination, yet our grasp of the underlying qualities and factors of these internally coupled eye movements is limited. To explore this, we studied 50 participants, examining how workload, spatial reference availability, and imagined movement direction influence internal coupling of eye movements. We designed a visuospatial working memory task in which participants mentally moved a black patch along a path within a matrix and each trial involved one step along this path (presented via speakers: up, down, left, or right). We varied workload by adjusting matrix size (3 × 3 vs. 5 × 5), manipulated availability of a spatial frame of reference by presenting either a blank screen (requiring participants to rely solely on their mental representation of the matrix) or spatial reference in the form of an empty matrix, and contrasted active task performance to two control conditions involving only active or passive listening. Our findings show that eye movements consistently matched the imagined movement of the patch in the matrix, not driven solely by auditory or semantic cues. While workload influenced pupil diameter, perceived demand, and performance, it had no observable impact on internal coupling. The availability of spatial reference enhanced coupling of eye movements, leading more frequent, precise, and resilient saccades against noise and bias. The absence of workload effects on coupled saccades in our study, in combination with the relatively high degree of coupling observed even in the invisible matrix condition, indicates that eye movements align with shifts in attention across both visually and internally represented information. This suggests that coupled eye movements are not merely strategic efforts to reduce workload, but rather a natural response to where attention is directed.