Inferring Macroscale Brain Dynamics via Fusion of Simultaneous EEG-fMRI.

Advances in the instrumentation and signal processing for simultaneously acquired electroencephalography and functional magnetic resonance imaging (EEG-fMRI) have enabled new ways to observe the spatiotemporal neural dynamics of the human brain. Central to the utility of EEG-fMRI neuroimaging systems are the methods for fusing the two data streams, with machine learning playing a key role. These methods can be dichotomized into those that are symmetric and asymmetric in terms of how the two modalities inform the fusion. Studies using these methods have shown that fusion yields new insights into brain function that are not possible when each modality is acquired separately. As technology improves and methods for fusion become more sophisticated, the future of EEG-fMRI for noninvasive measurement of brain dynamics includes mesoscale mapping at ultrahigh magnetic resonance fields, targeted perturbation-based neuroimaging, and using deep learning to uncover nonlinear representations that link the electrophysiological and hemodynamic measurements.

Adaptive adjustment after conflict with group opinion: evidence from neural electrophysiology.

Individuals inherently seek social consensus when making decisions or judgments. Previous studies have consistently indicated that dissenting group opinions are perceived as social conflict that demands attitude adjustment. However, the neurocognitive processes of attitude adjustment are unclear. In this electrophysiological study, participants were recruited to perform a face attractiveness judgment task. After forming their own judgment of a face, participants were informed of a purported group judgment (either consistent or inconsistent with their judgment), and then, critically, the same face was presented again. The neural responses to the second presented faces were measured. The second presented faces evoked a larger late positive potential after conflict with group opinions than those that did not conflict, suggesting that more motivated attention was allocated to stimulus. Moreover, faces elicited greater midfrontal theta (4-7 Hz) power after conflict with group opinions than after consistency with group opinions, suggesting that cognitive control was initiated to support attitude adjustment. Furthermore, the mixed-effects model revealed that single-trial theta power predicted behavioral change in the Conflict condition, but not in the No-Conflict condition. These findings provide novel insights into the neurocognitive processes underlying attitude adjustment, which is crucial to behavioral change during conformity.

Decoding the task specificity of post-error adjustments: Features and determinants.

Errors typically trigger post-error adjustments aimed at improving subsequent reactions within a single task, but little work has focused on whether these adjustments are task-general or task-specific across different tasks. We collected behavioral and electrophysiological (EEG) data when participants performed a psychological refractory period paradigm. This paradigm required them to complete Task 1 and Task 2 separated by a variable stimulus onset asynchrony (SOA). Behaviorally, post-error slowing and post-error accuracy exhibited task-general features at short SOAs but some task-specific features at long SOAs. EEG results manifest that task-general adjustments had a short-lived effect, whereas task-specific adjustments were long-lasting. Moreover, error awareness specifically conduced to the improvement of subsequent sensory processing and behavior performance in Task 1 (the task where errors occurred). These findings demonstrate that post-error adjustments rely on both transient, task-general interference and longer-lasting, task-specific control mechanisms simultaneously, with error awareness playing a crucial role in determining these mechanisms. We further discuss the contribution of central resources to the task specificity of post-error adjustments.

Single-trial interindividual correlation shows semantic and visuospatial networks are fundamental for advanced mathematical learning.

Mathematical learning and ability are crucial for individual and national economic and technological development, but the neural mechanisms underlying advanced mathematical learning remain unclear. The current study used functional magnetic resonance imaging (fMRI) to investigate how brain networks were involved in advanced mathematical learning and transfer. We recorded fMRI data from 24 undergraduate students as they learned the advanced mathematical concept of a commutative mathematical group. After learning, participants were required to complete learning and transfer behavioural tests. Results of single-trial interindividual brain-behaviour correlation analysis found that brain activity in the semantic and visuospatial networks, and the functional connectivity within the semantic network during advanced mathematical learning were positively correlated with learning and transfer effects. Additionally, the functional connectivity between the semantic and visuospatial networks was negatively correlated with the learning and transfer effects. These findings suggest that advanced mathematical learning relies on both semantic and visuospatial networks.

Brain Evoked Response Qualification Using Multi-Set Consensus Clustering: Toward Single-Trial EEG Analysis.

In event-related potential (ERP) analysis, it is commonly assumed that individual trials from a subject share similar properties and originate from comparable neural sources, allowing reliable interpretation of group-averages. Nevertheless, traditional group-level ERP analysis methods, including cluster analysis, often overlook critical information about individual subjects' neural processes due to using fixed measurement intervals derived from averaging. We developed a multi-set consensus clustering pipeline to examine cognitive processes at the individual subject level. Initially, consensus clustering from diverse methods was applied to single-trial EEG epochs of individual subjects. Subsequently, a second level of consensus clustering was performed across the trials of each subject. A newly modified time window determination method was then employed to identify individual subjects' ERP(s) of interest. We validated our method with simulated data for ERP components N2 and P3, and real data from a visual oddball task to confirm the P3 component. Our findings revealed that estimated time windows for individual subjects provide precise ERP identification compared to fixed time windows across all subjects. Additionally, Monte Carlo simulations with synthetic single-trial data demonstrated stable scores for the N2 and P3 components, confirming the reliability of our method. The proposed method enhances the examination of brain-evoked responses at the individual subject level by considering single-trial EEG data, thereby extracting mutual information relevant to the neural process. This approach offers a significant improvement over conventional ERP analysis, which relies on the averaging mechanism and fixed measurement interval.

Behavior-related potentials from single-trial interindividual correlation between event related potentials and behavioral performance reveals right lateralized processing of numerosity.

Accumulated functional magnetic resonance imaging (fMRI) and electroencephalography evidence indicate that numerosity is first processed in the occipito-parietal cortex. fMRI evidence also indicates right-lateralized processing of numerosity, but there is no consistent evidence from event-related potential (ERP) studies. This study investigated the ERP of numerosity processing in the left, right, and bilateral visual fields. The single-trial ERP-behavioral correlation was applied to show how the ERP was associated with behavioral responses. The results showed a significant early behavioral-ERP correlation on the right N1 component when stimuli were presented in the left visual field rather than in the right visual field. The behavioral ERP correlation was termed BN1. There was bilateral BN1 based on the reaction time or error rate, but the right BN1 was larger than that the left BN1 when the stimulus was present in the bilateral visual field. Therefore, this study provided a new neural marker for individual differences in processing numerosity and suggested that processing numerosity was supported by the right occipito-parietal cortex.

Neural evidence for age-related deficits in the representation of state spaces.

Under high cognitive demands, older adults tend to resort to simpler, habitual, or model-free decision strategies. This age-related shift in decision behavior has been attributed to deficits in the representation of the cognitive maps, or state spaces, necessary for more complex model-based decision-making. Yet, the neural mechanisms behind this shift remain unclear. In this study, we used a modified 2-stage Markov task in combination with computational modeling and single-trial EEG analyses to establish neural markers of age-related changes in goal-directed decision-making under different demands on the representation of state spaces. Our results reveal that the shift to simpler decision strategies in older adults is due to (i) impairments in the representation of the transition structure of the task and (ii) a diminished signaling of the reward value associated with decision options. In line with the diminished state space hypothesis of human aging, our findings suggest that deficits in goal-directed, model-based behavior in older adults result from impairments in the representation of state spaces of cognitive tasks.

Noninvasive neuromagnetic single-trial analysis of human neocortical population spikes.

Neuronal spiking is commonly recorded by invasive sharp microelectrodes, whereas standard noninvasive macroapproaches (e.g., electroencephalography [EEG] and magnetoencephalography [MEG]) predominantly represent mass postsynaptic potentials. A notable exception are low-amplitude high-frequency (∼600 Hz) somatosensory EEG/MEG responses that can represent population spikes when averaged over hundreds of trials to raise the signal-to-noise ratio. Here, a recent leap in MEG technology-featuring a factor 10 reduction in white noise level compared with standard systems-is leveraged to establish an effective single-trial portrayal of evoked cortical population spike bursts in healthy human subjects. This time-resolved approach proved instrumental in revealing a significant trial-to-trial variability of burst amplitudes as well as time-correlated (∼10 s) fluctuations of burst response latencies. Thus, ultralow-noise MEG enables noninvasive single-trial analyses of human cortical population spikes concurrent with low-frequency mass postsynaptic activity and thereby could comprehensively characterize cortical processing, potentially also in diseases not amenable to invasive microelectrode recordings.

Single trial variability in neural activity during a working memory task reveals multiple distinct information processing sequences.

Successful encoding, maintenance, and retrieval of information stored in working memory requires persistent coordination of activity among multiple brain regions. It is generally assumed that the pattern of such coordinated activity remains consistent for a given task. Thus, to separate this task-relevant signal from noise, multiple trials of the same task are completed, and the neural response is averaged across trials to generate an event-related potential (ERP). However, from trial to trial, the neuronal activity recorded with electroencephalogram (EEG) is actually spatially and temporally diverse, conflicting with the assumption of a single pattern of activity for a given task. Here, we show that variability in neuronal activity among single time-locked trials arises from the presence of multiple forms of stimulus dependent synchronized activity (i.e., distinct ERPs). We develop a data-driven classification method based on community detection to identify three discrete spatio-temporal clusters, or subtypes, of trials with different patterns of activation that are further associated with differences in decision-making processes. These results demonstrate that differences in the patterns of neural activity during working memory tasks represent fluctuations in the engagement of distinct brain networks and cognitive processes, suggesting that the brain can choose from multiple mechanisms to perform a given task.

Inter-individual single-trial classification of MEG data using M-CCA.

Neuroscientific studies often involve some form of group analysis over multiple participants. This requires alignment of recordings across participants. A naive solution is to assume that participants' recordings can be aligned anatomically in sensor space. However, this assumption is likely violated due to anatomical and functional differences between individual brains. In magnetoencephalography (MEG) recordings the problem of inter-subject alignment is exacerbated by the susceptibility of MEG to individual cortical folding patterns as well as the inter-subject variability of sensor locations over the brain due to the use of a fixed helmet. Hence, an approach to combine MEG data over individual brains should relax the assumptions that a) brain anatomy and function are tightly linked and b) that the same sensors capture functionally comparable brain activation across individuals. Here we use multiset canonical correlation analysis (M-CCA) to find a common representation of MEG activations recorded from 15 participants performing a grasping task. The M-CCA algorithm was applied to transform the data of a set of multiple participants into a common space with maximum correlation between participants. Importantly, we derive a method to transform data from a new, previously unseen participant into this common representation. This makes it useful for applications that require transfer of models derived from a group of individuals to new individuals. We demonstrate the usefulness and superiority of the approach with respect to previously used approaches. Finally, we show that our approach requires only a small number of labeled data from the new participant. The proposed method demonstrates that functionally motivated common spaces have potential applications in reducing training time of online brain-computer interfaces, where models can be pre-trained on previous participants/sessions. Moreover, inter-subject alignment via M-CCA has the potential for combining data of different participants and could become helpful in future endeavors on large open datasets.

Single-trial ERP Quantification Using Neural Networks.

Traditional approaches to quantify components in event-related potentials (ERPs) are based on averaging EEG responses. However, this method ignores the trial-to-trial variability in the component's latency, resulting in a smeared version of the component and underestimates of its amplitude. Different techniques to quantify ERP components in single trials have therefore been described in literature. In this study, two approaches based on neural networks are proposed and their performance was compared with other techniques using simulated data and two experimental datasets. On the simulated dataset, the neural networks outperformed other techniques for most signal-to-noise ratios and resulted in better estimates of the topography and shape of the ERP component. In the first experimental dataset, the highest correlation values between the estimated latencies of the P300 component and the reaction times were obtained using the neural networks. In the second dataset, the single-trial latency estimation techniques showed an amplitude reduction of the N400 effect with age and ascertained this effect could not be attributed to differences in latency variability. These results illustrate the applicability and the added value of neural networks for the quantification of ERP components in individual trials. A limitation, however, is that simulated data is needed to train the neural networks, which can be difficult when the ERP components to be found are not known a priori. Nevertheless, the neural networks-based approaches offer more information on the variability of the timing of the component and result in better estimates of the shape and topography of ERP components.

Domain-Specific Processing Stage for Estimating Single-Trail Evoked Potential Improves CNN Performance in Detecting Error Potential.

We present a novel architecture designed to enhance the detection of Error Potential (ErrP) signals during ErrP stimulation tasks. In the context of predicting ErrP presence, conventional Convolutional Neural Networks (CNNs) typically accept a raw EEG signal as input, encompassing both the information associated with the evoked potential and the background activity, which can potentially diminish predictive accuracy. Our approach involves advanced Single-Trial (ST) ErrP enhancement techniques for processing raw EEG signals in the initial stage, followed by CNNs for discerning between ErrP and NonErrP segments in the second stage. We tested different combinations of methods and CNNs. As far as ST ErrP estimation is concerned, we examined various methods encompassing subspace regularization techniques, Continuous Wavelet Transform, and ARX models. For the classification stage, we evaluated the performance of EEGNet, CNN, and a Siamese Neural Network. A comparative analysis against the method of directly applying CNNs to raw EEG signals revealed the advantages of our architecture. Leveraging subspace regularization yielded the best improvement in classification metrics, at up to 14% in balanced accuracy and 13.4% in F1-score.

Quantifying rhythmicity in perceptual reports.

Several recent studies investigated the rhythmic nature of cognitive processes that lead to perception and behavioral report. These studies used different methods, and there has not yet been an agreement on a general standard. Here, we present a way to test and quantitatively compare these methods. We simulated behavioral data from a typical experiment and analyzed these data with several methods. We applied the main methods found in the literature, namely sine-wave fitting, the discrete Fourier transform (DFT) and the least square spectrum (LSS). DFT and LSS can be applied both on the average accuracy time course and on single trials. LSS is mathematically equivalent to DFT in the case of regular, but not irregular sampling - which is more common. LSS additionally offers the possibility to take into account a weighting factor which affects the strength of the rhythm, such as arousal. Statistical inferences were done either on the investigated sample (fixed-effects) or on the population (random-effects) of simulated participants. Multiple comparisons across frequencies were corrected using False Discovery Rate, Bonferroni, or the Max-Based approach. To perform a quantitative comparison, we calculated sensitivity, specificity and D-prime of the investigated analysis methods and statistical approaches. Within the investigated parameter range, single-trial methods had higher sensitivity and D-prime than the methods based on the average accuracy time course. This effect was further increased for a simulated rhythm of higher frequency. If an additional (observable) factor influenced detection performance, adding this factor as weight in the LSS further improved sensitivity and D-prime. For multiple comparison correction, the Max-Based approach provided the highest specificity and D-prime, closely followed by the Bonferroni approach. Given a fixed total amount of trials, the random-effects approach had higher D-prime when trials were distributed over a larger number of participants, even though this gave less trials per participant. Finally, we present the idea of using a dampened sinusoidal oscillator instead of a simple sinusoidal function, to further improve the fit to behavioral rhythmicity observed after a reset event.

Linking neurophysiological processes of action monitoring to post-response speed-accuracy adjustments in a neuro-cognitive diffusion model.

The cognitive system needs to continuously monitor actions and initiate adaptive measures aimed at increasing task performance and avoiding future errors. To investigate the link between the contributing cognitive processes, we introduce the neuro-cognitive diffusion model, a statistical approach that allows a combination of computational modelling of behavioural and electrophysiological data on a single-trial level. This unique combination of methods allowed us to demonstrate across three experimental datasets that early response monitoring (error negativity; Ne/c) was related to more response caution and increased attention on task-relevant features on the subsequent trial, thereby preventing future errors, whereas later response monitoring (error positivity, Pe/c) maintained the ability of responding fast under speed pressure. Our results suggest that Pe/c-related processes might keep Ne/c-related processes in check regarding their impact on post-response adaptation to reconcile the conflicting criteria of fast and accurate responding.

Spatiotemporally resolved multivariate pattern analysis for M/EEG.

An emerging goal in neuroscience is tracking what information is represented in brain activity over time as a participant completes some task. While electroencephalography (EEG) and magnetoencephalography (MEG) offer millisecond temporal resolution of how activity patterns emerge and evolve, standard decoding methods present significant barriers to interpretability as they obscure the underlying spatial and temporal activity patterns. We instead propose the use of a generative encoding model framework that simultaneously infers the multivariate spatial patterns of activity and the variable timing at which these patterns emerge on individual trials. An encoding model inversion maps from these parameters to the equivalent decoding model, allowing predictions to be made about unseen test data in the same way as in standard decoding methodology. These SpatioTemporally Resolved MVPA (STRM) models can be flexibly applied to a wide variety of experimental paradigms, including classification and regression tasks. We show that these models provide insightful maps of the activity driving predictive accuracy metrics; demonstrate behaviourally meaningful variation in the timing of pattern emergence on individual trials; and achieve predictive accuracies that are either equivalent or surpass those achieved by more widely used methods. This provides a new avenue for investigating the brain's representational dynamics and could ultimately support more flexible experimental designs in the future.

Single-trial averaging improves the physiological interpretation of contact heat evoked potentials.

Laser and contact heat evoked potentials (LEPs and CHEPs, respectively) provide an objective measure of pathways and processes involved in nociception. The majority of studies analyzing LEP or CHEP outcomes have done so based on conventional, across-trial averaging. With this approach, evoked potential components are potentially confounded by latency jitter and ignore relevant information contained within single trials. The current study addressed the advantage of analyzing nociceptive evoked potentials based on responses to noxious stimulations within each individual trial. Single-trial and conventional averaging were applied to data previously collected in 90 healthy subjects from 3 stimulation locations on the upper limb. The primary analysis focused on relationships between single and across-trial averaged CHEP outcomes (i.e., N2P2 amplitude and N2 and P2 latencies) and subject characteristics (i.e., age, sex, height, and rating of perceived intensity), which were examined by way of linear mixed model analysis. Single-trial averaging lead to larger N2P2 amplitudes and longer N2 and P2 latencies. Age and ratings of perceived intensity were the only subject level characteristics associated with CHEPs outcomes that significantly interacted with the method of analysis (conventional vs single-trial averaging). The strength of relationships for age and ratings of perceived intensity, measured by linear fit, were increased for single-trial compared to conventional across-trial averaged CHEP outcomes. By accounting for latency jitter, single-trial averaging improved the associations between CHEPs and physiological outcomes and should be incorporated as a standard analytical technique in future studies.

Stimulation artifact source separation (SASS) for assessing electric brain oscillations during transcranial alternating current stimulation (tACS).

Brain oscillations, e.g. measured by electro- or magnetoencephalography (EEG/MEG), are causally linked to brain functions that are fundamental for perception, cognition and learning. Recent advances in neurotechnology provide means to non-invasively target these oscillations using frequency-tuned amplitude-modulated transcranial alternating current stimulation (AM-tACS). However, online adaptation of stimulation parameters to ongoing brain oscillations remains an unsolved problem due to stimulation artifacts that impede such adaptation, particularly at the target frequency. Here, we introduce a real-time compatible artifact rejection algorithm (Stimulation Artifact Source Separation, SASS) that overcomes this limitation. SASS is a spatial filter (linear projection) removing EEG signal components that are maximally different in the presence versus absence of stimulation. This enables the reliable removal of stimulation-specific signal components, while leaving physiological signal components unaffected. For validation of SASS, we evoked brain activity with known phase and amplitude using 10 Hz visual flickers across 7 healthy human volunteers. 64-channel EEG was recorded during and in absence of 10 Hz AM-tACS targeting the visual cortex. Phase differences between AM-tACS and the visual stimuli were randomized, so that steady-state visually evoked potentials (SSVEPs) were phase-locked to the visual stimuli but not to the AM-tACS signal. For validation, distributions of single-trial amplitude and phase of EEG signals recorded during and in absence of AM-tACS were compared for each participant. When no artifact rejection method was applied, AM-tACS stimulation artifacts impeded assessment of single-trial SSVEP amplitude and phase. Using SASS, amplitude and phase of single trials recorded during and in absence of AM-tACS were comparable. These results indicate that SASS can be used to establish adaptive (closed-loop) AM-tACS, a potentially powerful tool to target various brain functions, and to investigate how AM-tACS interacts with electric brain oscillations.

State anxiety biases estimates of uncertainty and impairs reward learning in volatile environments.

Clinical and subclinical (trait) anxiety impairs decision making and interferes with learning. Less understood are the effects of temporary anxious states on learning and decision making in healthy populations, and whether these can serve as a model for clinical anxiety. Here we test whether anxious states in healthy individuals elicit a pattern of aberrant behavioural, neural, and physiological responses comparable with those found in anxiety disorders-particularly when processing uncertainty in unstable environments. In our study, both a state anxious and a control group learned probabilistic stimulus-outcome mappings in a volatile task environment while we recorded their electrophysiological (EEG) signals. By using a hierarchical Bayesian model of inference and learning, we assessed the effect of state anxiety on Bayesian belief updating with a focus on uncertainty estimates. State anxiety was associated with an underestimation of environmental uncertainty, and informational uncertainty about the reward tendency. Anxious individuals' beliefs about reward contingencies were more precise (had smaller uncertainty) and thus more resistant to updating, ultimately leading to impaired reward-based learning. State anxiety was also associated with greater uncertainty about volatility. We interpret this pattern as evidence that state anxious individuals are less tolerant to informational uncertainty about the contingencies governing their environment and more willing to be uncertain about the level of stability of the world itself. Further, we tracked the neural representation of belief update signals in the trial-by-trial EEG amplitudes. In control participants, lower-level precision-weighted prediction errors (pwPEs) about reward tendencies were represented in the ERP signals across central and parietal electrodes peaking at 496 ms, overlapping with the late P300 in classical ERP analysis. The state anxiety group did not exhibit a significant representation of low-level pwPEs, and there were no significant differences between the groups. Smaller variance in low-level pwPE about reward tendencies in state anxiety could partially account for the null results. Expanding previous computational work on trait anxiety, our findings establish that temporary anxious states in healthy individuals impair reward-based learning in volatile environments, primarily through changes in uncertainty estimates, which play a central role in current Bayesian accounts of perceptual inference and learning.

Single-trial regression of spatial exploration behavior indicates posterior EEG alpha modulation to reflect egocentric coding.

Learning to navigate uncharted terrain is a key cognitive ability that emerges as a deeply embodied process, with eye movements and locomotion proving most useful to sample the environment. We studied healthy human participants during active spatial learning of room-scale virtual reality (VR) mazes. In the invisible maze task, participants wearing a wireless electroencephalography (EEG) headset were free to explore their surroundings, only given the objective to build and foster a mental spatial representation of their environment. Spatial uncertainty was resolved by touching otherwise invisible walls that were briefly rendered visible inside VR, similar to finding your way in the dark. We showcase the capabilities of mobile brain/body imaging using VR, demonstrating several analysis approaches based on general linear models (GLMs) to reveal behavior-dependent brain dynamics. Confirming spatial learning via drawn sketch maps, we employed motion capture to image spatial exploration behavior describing a shift from initial exploration to subsequent exploitation of the mental representation. Using independent component analysis, the current work specifically targeted oscillations in response to wall touches reflecting isolated spatial learning events arising in deep posterior EEG sources located in the retrosplenial complex. Single-trial regression identified significant modulation of alpha oscillations by the immediate, egocentric, exploration behavior. When encountering novel walls, as well as with increasing walking distance between subsequent touches when encountering novel walls, alpha power decreased. We conclude that these oscillations play a prominent role during egocentric evidencing of allocentric spatial hypotheses.

Neural oscillatory responses to performance monitoring differ between high- and low-impulsive individuals, but are unaffected by TMS.

Higher impulsivity may arise from neurophysiological deficits of cognitive control in the prefrontal cortex. Cognitive control can be assessed by time-frequency decompositions of electrophysiological data. We aimed to clarify neuroelectric mechanisms of performance monitoring in connection with impulsiveness during a modified Eriksen flanker task in high- (n = 24) and low-impulsive subjects (n = 21) and whether these are modulated by double-blind, sham-controlled intermittent theta burst stimulation (iTBS). We found a larger error-specific peri-response beta power decrease over fronto-central sites in high-impulsive compared to low-impulsive participants, presumably indexing less effective motor execution processes. Lower parieto-occipital theta intertrial phase coherence (ITPC) preceding correct responses predicted higher reaction time (RT) and higher RT variability, potentially reflecting efficacy of cognitive control or general attention. Single-trial preresponse theta phase clustering was coupled to RT in correct trials (weighted ITPC), reflecting oscillatory dynamics that predict trial-specific behavior. iTBS did not modulate behavior or EEG time-frequency power. Performance monitoring was associated with time-frequency patterns reflecting cognitive control (parieto-occipital theta ITPC, theta weighted ITPC) as well as differential action planning/execution processes linked to trait impulsivity (frontal low beta power). Beyond that, results suggest no stimulation effect related to response-locked time-frequency dynamics with the current stimulation protocol. Neural oscillatory responses to performance monitoring differ between high- and low-impulsive individuals, but are unaffected by iTBS.