Brain decoding of the Human Connectome Project tasks in a dense individual fMRI dataset.

Brain decoding aims to infer cognitive states from patterns of brain activity. Substantial inter-individual variations in functional brain organization challenge accurate decoding performed at the group level. In this paper, we tested whether accurate brain decoding models can be trained entirely at the individual level. We trained several classifiers on a dense individual functional magnetic resonance imaging (fMRI) dataset for which six participants completed the entire Human Connectome Project (HCP) task battery >13 times over ten separate fMRI sessions. We evaluated nine decoding methods, from Support Vector Machines (SVM) and Multi-Layer Perceptron (MLP) to Graph Convolutional Neural Networks (GCN). All decoders were trained to classify single fMRI volumes into 21 experimental conditions simultaneously, using ∼7 h of fMRI data per participant. The best prediction accuracies were achieved with GCN and MLP models, whose performance (57-67 % accuracy) approached state-of-the-art accuracy (76 %) with models trained at the group level on >1 K hours of data from the original HCP sample. Our SVM model also performed very well (54-62 % accuracy). Feature importance maps derived from MLP -our best-performing model- revealed informative features in regions relevant to particular cognitive domains, notably in the motor cortex. We also observed that inter-subject classification achieved substantially lower accuracy than subject-specific models, indicating that our decoders learned individual-specific features. This work demonstrates that densely-sampled neuroimaging datasets can be used to train accurate brain decoding models at the individual level. We expect this work to become a useful benchmark for techniques that improve model generalization across multiple subjects and acquisition conditions.

Scientific progress: By-whom or for-whom?

When science makes cognitive progress, who or what is it that improves in the requisite way? According to a widespread and unchallenged assumption, it is the cognitive attitudes of scientists themselves, i.e. the agents by whom scientific progress is made, that improve during progressive episodes. This paper argues against this assumption and explores a different approach. Scientific progress should be defined in terms of potential improvements to the cognitive attitudes of those for whom progress is made, i.e. the receivers rather than the producers of scientific information. This includes not only scientists themselves, but also various other individuals who utilize scientific information in different ways for the benefit of society as a whole.

Functional annotation of human cognitive states using deep graph convolution.

A key goal in neuroscience is to understand brain mechanisms of cognitive functions. An emerging approach is "brain decoding", which consists of inferring a set of experimental conditions performed by a participant, using pattern classification of brain activity. Few works so far have attempted to train a brain decoding model that would generalize across many different cognitive tasks drawn from multiple cognitive domains. To tackle this problem, we proposed a multidomain brain decoder that automatically learns the spatiotemporal dynamics of brain response within a short time window using a deep learning approach. We evaluated the decoding model on a large population of 1200 participants, under 21 different experimental conditions spanning six different cognitive domains, acquired from the Human Connectome Project task-fMRI database. Using a 10s window of fMRI response, the 21 cognitive states were identified with a test accuracy of 90% (chance level 4.8%). Performance remained good when using a 6s window (82%). It was even feasible to decode cognitive states from a single fMRI volume (720ms), with the performance following the shape of the hemodynamic response. Moreover, a saliency map analysis demonstrated that the high decoding performance was driven by the response of biologically meaningful brain regions. Together, we provide an automated tool to annotate human brain activity with fine temporal resolution and fine cognitive granularity. Our model shows potential applications as a reference model for domain adaptation, possibly making contributions in a variety of domains, including neurological and psychiatric disorders.

A Systematic Review for Cognitive State-Based QoE/UX Evaluation.

Traditional evaluation of user experience is subjective by nature, for what is sought is to use data from physiological and behavioral sensors to interpret the relationship that the user's cognitive states have with the elements of a graphical interface and interaction mechanisms. This study presents the systematic review that was developed to determine the cognitive states that are being investigated in the context of Quality of Experience (QoE)/User Experience (UX) evaluation, as well as the signals and characteristics obtained, machine learning models used, evaluation architectures proposed, and the results achieved. Twenty-nine papers published in 2014-2019 were selected from eight online sources of information, of which 24% were related to the classification of cognitive states, 17% described evaluation architectures, and 41% presented correlations between different signals, cognitive states, and QoE/UX metrics, among others. The amount of identified studies was low in comparison with cognitive state research in other contexts, such as driving or other critical activities; however, this provides a starting point to analyze and interpret states such as mental workload, confusion, and mental stress from various human signals and propose more robust QoE/UX evaluation architectures.

Spatiotemporal analysis of event-related fMRI to reveal cognitive states.

Cognitive science has a rich history of developing theories of processing that characterize the mental steps involved in performance of many tasks. Recent work in neuroimaging and machine learning has greatly improved our ability to link cognitive processes with what is happening in the brain. This article analyzes a hidden semi-Markov model-multivoxel pattern-analysis (HSMM-MVPA) methodology that we have developed for inferring the sequence of brain states one traverses in the performance of a cognitive task. The method is applied to a functional magnetic resonance imaging (fMRI) experiment where task boundaries are known that should separate states. The method is able to accurately identify those boundaries. Then, applying the method to synthetic data, we explore more fully those factors that influence performance of the method: signal-to-noise ratio, numbers of states, state sojourn times, and numbers of underlying experimental conditions. The results indicate the types of experimental tasks where applications of the HSMM-MVPA method are likely to yield accurate and insightful results.

Sensor Networks for Aerospace Human-Machine Systems.

Intelligent automation and trusted autonomy are being introduced in aerospace cyber-physical systems to support diverse tasks including data processing, decision-making, information sharing and mission execution. Due to the increasing level of integration/collaboration between humans and automation in these tasks, the operational performance of closed-loop human-machine systems can be enhanced when the machine monitors the operator's cognitive states and adapts to them in order to maximise the effectiveness of the Human-Machine Interfaces and Interactions (HMI2). Technological developments have led to neurophysiological observations becoming a reliable methodology to evaluate the human operator's states using a variety of wearable and remote sensors. The adoption of sensor networks can be seen as an evolution of this approach, as there are notable advantages if these sensors collect and exchange data in real-time, while their operation is controlled remotely and synchronised. This paper discusses recent advances in sensor networks for aerospace cyber-physical systems, focusing on Cognitive HMI2 (CHMI2) implementations. The key neurophysiological measurements used in this context and their relationship with the operator's cognitive states are discussed. Suitable data analysis techniques based on machine learning and statistical inference are also presented, as these techniques allow processing both neurophysiological and operational data to obtain accurate cognitive state estimations. Lastly, to support the development of sensor networks for CHMI2 applications, the paper addresses the performance characterisation of various state-of-the-art sensors and the propagation of measurement uncertainties through a machine learning-based inference engine. Results show that a proper sensor selection and integration can support the implementation of effective human-machine systems for various challenging aerospace applications, including Air Traffic Management (ATM), commercial airliner Single-Pilot Operations (SIPO), one-to-many Unmanned Aircraft Systems (UAS), and space operations management.

Tracking ongoing cognition in individuals using brief, whole-brain functional connectivity patterns.

Functional connectivity (FC) patterns in functional MRI exhibit dynamic behavior on the scale of seconds, with rich spatiotemporal structure and limited sets of whole-brain, quasi-stable FC configurations (FC states) recurring across time and subjects. Based on previous evidence linking various aspects of cognition to group-level, minute-to-minute FC changes in localized connections, we hypothesized that whole-brain FC states may reflect the global, orchestrated dynamics of cognitive processing on the scale of seconds. To test this hypothesis, subjects were continuously scanned as they engaged in and transitioned between mental states dictated by tasks. FC states computed within windows as short as 22.5 s permitted robust tracking of cognition in single subjects with near perfect accuracy. Accuracy dropped markedly for subjects with the lowest task performance. Spatially restricting FC information decreased accuracy at short time scales, emphasizing the distributed nature of whole-brain FC dynamics, beyond univariate magnitude changes, as valuable markers of cognition.

A fuzzy integral method based on the ensemble of neural networks to analyze fMRI data for cognitive state classification across multiple subjects.

The huge number of voxels in fMRI over time poses a major challenge to for effective analysis. Fast, accurate, and reliable classifiers are required for estimating the decoding accuracy of brain activities. Although machine-learning classifiers seem promising, individual classifiers have their own limitations. To address this limitation, the present paper proposes a method based on the ensemble of neural networks to analyze fMRI data for cognitive state classification for application across multiple subjects. Similarly, the fuzzy integral (FI) approach has been employed as an efficient tool for combining different classifiers. The FI approach led to the development of a classifiers ensemble technique that performs better than any of the single classifier by reducing the misclassification, the bias, and the variance. The proposed method successfully classified the different cognitive states for multiple subjects with high accuracy of classification. Comparison of the performance improvement, while applying ensemble neural networks method, vs. that of the individual neural network strongly points toward the usefulness of the proposed method.

Are adolescents with anorexia nervosa better at reading minds?

INTRODUCTION: The present study aimed to investigate mindreading abilities in female adolescent patients with AN compared to healthy controls (HCs), analysing differences for emotional valence of facial stimuli. METHODS: The Eating Disorder Inventory) for evaluating psychological traits associated with eating disorders and the Children's version of the Reading the Mind in the Eyes Test for evaluating mindreading abilities were administered to 40 Italian female patients (mean age = 14.93; SD = 1.48) with restrictive diagnosis of anorexia nervosa (AN) and 40 healthy females (mean age = 14.88; SD = 0.56). RESULTS: No significant differences between the AN group and HCs for the Eyes Total score were found. Even when analysing emotional valence of the items, the two groups were equally successful in the facial recognition of positive, negative and neutral emotions. A significant difference was revealed for the percentage of correct responses of item 10 and item 15, where the AN group was less able to correctly identify the target descriptor (Not believing) over the foils than HCs. A significant difference was revealed in discriminating for affective emotions versus cognitive states; only for affective but not for cognitive states, patients with AN were found to perform better than controls on the mindreading task. CONCLUSIONS: Our study highlighted the importance of analysing and discriminating for different valences of facial stimuli when assessing mindreading abilities in adolescents with AN, so that more precise and specific treatment approaches could be developed for female adolescents with AN.

Decoding Subject-Driven Cognitive States from EEG Signals for Cognitive Brain-Computer Interface.

In this study, we investigated the feasibility of using electroencephalogram (EEG) signals to differentiate between four distinct subject-driven cognitive states: resting state, narrative memory, music, and subtraction tasks. EEG data were collected from seven healthy male participants while performing these cognitive tasks, and the raw EEG signals were transformed into time-frequency maps using continuous wavelet transform. Based on these time-frequency maps, we developed a convolutional neural network model (TF-CNN-CFA) with a channel and frequency attention mechanism to automatically distinguish between these cognitive states. The experimental results demonstrated that the model achieved an average classification accuracy of 76.14% in identifying these four cognitive states, significantly outperforming traditional EEG signal processing methods and other classical image classification algorithms. Furthermore, we investigated the impact of varying lengths of EEG signals on classification performance and found that TF-CNN-CFA demonstrates consistent performance across different window lengths, indicating its strong generalization capability. This study validates the ability of EEG to differentiate higher cognitive states, which could potentially offer a novel BCI paradigm.

Estimating Systemic Cognitive States from a Mixture of Physiological and Brain Signals.

As human-machine teams are being considered for a variety of mixed-initiative tasks, detecting and being responsive to human cognitive states, in particular systematic cognitive states, is among the most critical capabilities for artificial systems to ensure smooth interactions with humans and high overall team performance. Various human physiological parameters, such as heart rate, respiration rate, blood pressure, and skin conductance, as well as brain activity inferred from functional near-infrared spectroscopy or electroencephalogram, have been linked to different systemic cognitive states, such as workload, distraction, or mind-wandering among others. Whether these multimodal signals are indeed sufficient to isolate such cognitive states across individuals performing tasks or whether additional contextual information (e.g., about the task state or the task environment) is required for making appropriate inferences remains an important open problem. In this paper, we introduce an experimental and machine learning framework for investigating these questions and focus specifically on using physiological and neurophysiological measurements to learn classifiers associated with systemic cognitive states like cognitive load, distraction, sense of urgency, mind wandering, and interference. Specifically, we describe a multitasking interactive experimental setting used to obtain a comprehensive multimodal data set which provided the foundation for a first evaluation of various standard state-of-the-art machine learning techniques with respect to their effectiveness in inferring systemic cognitive states. While the classification success of these standard methods based on just the physiological and neurophysiological signals across subjects was modest, which is to be expected given the complexity of the classification problem and the possibility that higher accuracy rates might not in general be achievable, the results nevertheless can serve as a baseline for evaluating future efforts to improve classification, especially methods that take contextual aspects such as task and environmental states into account.

A new approach to econometric modeling in digitized consumer behavior.

Given how identifying motivational factors of online purchasing is critical to the success of online retailers, research on the antecedents of online customer experience (cognitive and affective experiential states) has attracted widespread attention. In this study, we conducted an extensive survey to identify major behavioral changes in the online buyer, and based on the age of the respondents we synthesized the findings into an econometric model to explain the impact of cultural, social, personal, and psychological traits on online purchasing. Our survey identified a myriad of motivational factors that influence online buyers' psychological perceptions and the impact of those factors has been reported. The proposed econometric model would help online retailers to better understand the motivational factors behind online customers' purchasing decisions. It also serves to inform the academic community of recent trends in this stream of research and shed light on future research.

Accessing the States of Enhanced Cognition in a Gaming Context: The Importance of Psychophysiological Arousal.

The goal of this study was to examine activities and experiences where enhanced cognitive states (ECSs), characterized by dramatic boosts in focused attention, could be elicited under specific gaming contexts. In Experiment 1, expert gamers were tested on the attentional blink task before and after playing games of different genres, varying on four game design dimensions (perspective, "adrenaline-rush," immersivity, and collaborative vs. individual context) and two cognitive dimensions (speed of processing and attentional focus). In Experiment 2, using ECG-HRV methodology, we examined the physiological markers of gaming dimensions found to be critical for accessing ECSs in Experiment 1. The findings suggest that ECSs are a universal phenomenon that demands focusing one's attention on a single task from the egocentric perspective, and ought to involve an adventurous "adrenaline-rush" type of activity. Furthermore, the results demonstrated that an underlying physiological mechanism of ECSs includes parasympathetic nervous system (PSNS) withdrawal-associated arousal. Specifically, the gaming dimensions leading to greater PSNS withdrawal-associated arousal resulted in greater improvements on the attentional blink task during ECSs. These findings suggest that individuals can transcend what was hitherto assumed to be a limitation of human cognition, granting new prospects for eliciting exceptional human performance.

The day-of-week (DOW) effect on liberalism-conservatism: Evidence from a large-scale online survey in China.

Introduction: Past research has shown that psychological states tend to fluctuate across the days of a week, which is referred to as the day-of-week (DOW) effect. This study investigated the DOW effect on liberalism-conservatism among Chinese people by testing two competing hypotheses. According to the cognitive states hypothesis, it was predicted that liberalism would be high on Mondays but gradually decrease to Fridays due to the depletion of cognitive resources over the weekdays. In contrast, the affective states hypothesis predicted the opposite, considering the more positive affect brought by the approaching weekends. Both hypotheses predicted the level of liberalism would peak at weekends. Methods: Data (n = 171,830) were collected via an online questionnaire, the Chinese Political Compass (CPC) survey, which includes 50 items to measure people' liberalism-conservatism in three domains (i.e., political, economic, and social). Results: The results showed the level of liberalism decreased gradually from Mondays until Wednesdays, rebounded from Wednesdays to Fridays, and peaked at weekends. Discussion: The V-shaped pattern suggested that the DOW fluctuation in liberalism-conservatism could derive from the synergy of both cognitive and affective processes, instead of either one alone. The findings have important implications for practice and policy-making, including the recent pilot scheme of 4-day workweek.

Impact of Cognitive Profile on Impulse Control Disorders Presence and Severity in Parkinson's Disease.

Background: Impulse control disorders (ICDs) and related behaviors are frequent in Parkinson's disease (PD). Mild cognitive impairment (PD-MCI) and dementia (PDD), both characterized by heterogeneous cognitive phenotypes, are also commonly reported in PD. However, the frequency and severity of ICD within PD cognitive states is unknown. Methods: Three hundred and twenty-six PD patients completed a comprehensive neuropsychological assessment and were classified as PD-MCI, PDD, or without cognitive alterations (PD-NC). The Minnesota impulsive disorders interview was used to ascertain the presence (ICD+) or absence (ICD-) of ICD. The Questionnaire for Impulsive-Compulsive Disorders in Parkinson's Disease-Rating Scale was used to assess ICD severity. A subsample of 286 patients evaluated with the same cognitive tasks was selected in order to investigate the characteristics of ICD in PD cognitive phenotypes. Results: ICDs were present in 55% of PD-NC, in 50% of PD-MCI, and in 42% of PDD patients. Frequencies of ICD+ with attentive (ICD+: 20% vs. ICD-: 4%; p = 0.031) and executive impairments (ICD+: 44% vs. ICD-: 30%; p = 0.027) were higher in the PD-MCI and PDD subgroups, respectively. As expected, no differences were observed in the PD-NC. PD-MCI with attentive impairments presented higher percentage of ICD+ with deficits in the Trail Making Test B-A but not in the Digit Span Sequencing task. In PDD, executive failures concerned Similarities task (ICD+: 67%; ICD-: 29%; p = 0.035), with no differences between ICD+ and ICD- in the Stroop task. Conclusions: Prevalence and severity of ICDs and related behaviors do not differ in PD with different cognitive states. However, ICD+ are more likely to show deficits, respectively in attentive and in executive domains, specifically in the Trail Making Test B-A task for the attention and working memory domain in PD-MCI and in the Similarities task for the executive function domain in PDD. Prospective studies should evaluate if these tests can be used as screening tool for ICDs in PD.

Operating in a Reverberating Regime Enables Rapid Tuning of Network States to Task Requirements.

Neural circuits are able to perform computations under very diverse conditions and requirements. The required computations impose clear constraints on their fine-tuning: a rapid and maximally informative response to stimuli in general requires decorrelated baseline neural activity. Such network dynamics is known as asynchronous-irregular. In contrast, spatio-temporal integration of information requires maintenance and transfer of stimulus information over extended time periods. This can be realized at criticality, a phase transition where correlations, sensitivity and integration time diverge. Being able to flexibly switch, or even combine the above properties in a task-dependent manner would present a clear functional advantage. We propose that cortex operates in a "reverberating regime" because it is particularly favorable for ready adaptation of computational properties to context and task. This reverberating regime enables cortical networks to interpolate between the asynchronous-irregular and the critical state by small changes in effective synaptic strength or excitation-inhibition ratio. These changes directly adapt computational properties, including sensitivity, amplification, integration time and correlation length within the local network. We review recent converging evidence that cortex in vivo operates in the reverberating regime, and that various cortical areas have adapted their integration times to processing requirements. In addition, we propose that neuromodulation enables a fine-tuning of the network, so that local circuits can either decorrelate or integrate, and quench or maintain their input depending on task. We argue that this task-dependent tuning, which we call "dynamic adaptive computation," presents a central organization principle of cortical networks and discuss first experimental evidence.