Cortical cognitive processing during reading captured using functional-near infrared spectroscopy.

Neuroimaging studies using functional magnetic resonance imaging (fMRI) have provided unparalleled insights into the fundamental neural mechanisms underlying human cognitive processing, such as high-level linguistic processes during reading. Here, we build upon this prior work to capture sentence reading comprehension outside the MRI scanner using functional near infra-red spectroscopy (fNIRS) in a large sample of participants (n = 82). We observed increased task-related hemodynamic responses in prefrontal and temporal cortical regions during sentence-level reading relative to the control condition (a list of non-words), replicating prior fMRI work on cortical recruitment associated with high-level linguistic processing during reading comprehension. These results lay the groundwork towards developing adaptive systems to support novice readers and language learners by targeting the underlying cognitive processes. This work also contributes to bridging the gap between laboratory findings and more real-world applications in the realm of cognitive neuroscience.

Friend or foe: classifying collaborative interactions using fNIRS.

To succeed, effective teams depend on both cooperative and competitive interactions between individual teammates. Depending on the context, cooperation and competition can amplify or neutralize a team's problem solving ability. Therefore, to assess successful collaborative problem solving, it is first crucial to distinguish competitive from cooperative interactions. We investigate the feasibility of using lightweight brain sensors to distinguish cooperative from competitive interactions in pairs of participants (N=84) playing a decision-making game involving uncertain outcomes. We measured brain activity using functional near-infrared spectroscopy (fNIRS) from social, motor, and executive areas during game play alone and in competition or cooperation with another participant. To distinguish competitive, cooperative, and alone conditions, we then trained support vector classifiers using combinations of features extracted from fNIRS data. We find that features from social areas of the brain outperform other features for discriminating competitive, cooperative, and alone conditions in cross-validation. Comparing the competitive and alone conditions, social features yield a 5% improvement over motor and executive features. Social features show promise as means of distinguishing competitive and cooperative environments in problem solving settings. Using fNIRS data provides a real-time measure of subjective experience in an ecologically valid environment. These results have the potential to inform intelligent team monitoring to provide better real-time feedback and improve team outcomes in naturalistic settings.

Interdisciplinary views of fNIRS: Current advancements, equity challenges, and an agenda for future needs of a diverse fNIRS research community.

Functional Near-Infrared Spectroscopy (fNIRS) is an innovative and promising neuroimaging modality for studying brain activity in real-world environments. While fNIRS has seen rapid advancements in hardware, software, and research applications since its emergence nearly 30 years ago, limitations still exist regarding all three areas, where existing practices contribute to greater bias within the neuroscience research community. We spotlight fNIRS through the lens of different end-application users, including the unique perspective of a fNIRS manufacturer, and report the challenges of using this technology across several research disciplines and populations. Through the review of different research domains where fNIRS is utilized, we identify and address the presence of bias, specifically due to the restraints of current fNIRS technology, limited diversity among sample populations, and the societal prejudice that infiltrates today's research. Finally, we provide resources for minimizing bias in neuroscience research and an application agenda for the future use of fNIRS that is equitable, diverse, and inclusive.

Taking a Deeper Look at the Brain: Predicting Visual Perceptual and Working Memory Load From High-Density fNIRS Data.

Predicting workload using physiological sensors has taken on a diffuse set of methods in recent years. However, the majority of these methods train models on small datasets, with small numbers of channel locations on the brain, limiting a model's ability to transfer across participants, tasks, or experimental sessions. In this paper, we introduce a new method of modeling a large, cross-participant and cross-session set of high density functional near infrared spectroscopy (fNIRS) data by using an approach grounded in cognitive load theory and employing a Bi-Directional Gated Recurrent Unit (BiGRU) incorporating attention mechanism and self-supervised label augmentation (SLA). We show that our proposed CNN-BiGRU-SLA model can learn and classify different levels of working memory load (WML) and visual processing load (VPL) across participants. Importantly, we leverage a multi-label classification scheme, where our models are trained to predict simultaneously occurring levels of WML and VPL. We evaluate our model using leave-one-participant-out (LOOCV) as well as 10-fold cross validation. Using LOOCV, for binary classification (off/on), we reached an F1-score of 0.9179 for WML and 0.8907 for VPL across 22 participants (each participant did 2 sessions). For multi-level (off, low, high) classification, we reached an F1-score of 0.7972 for WML and 0.7968 for VPL. Using 10-fold cross validation, for multi-level classification, we reached an F1-score of 0.7742 for WML and 0.7741 for VPL.

Using fNIRS to Identify Transparency- and Reliability-Sensitive Markers of Trust Across Multiple Timescales in Collaborative Human-Human-Agent Triads.

Intelligent agents are rapidly evolving from assistants into teammates as they perform increasingly complex tasks. Successful human-agent teams leverage the computational power and sensory capabilities of automated agents while keeping the human operator's expectation consistent with the agent's ability. This helps prevent over-reliance on and under-utilization of the agent to optimize its effectiveness. Research at the intersection of human-computer interaction, social psychology, and neuroergonomics has identified trust as a governing factor of human-agent interactions that can be modulated to maintain an appropriate expectation. To achieve this calibration, trust can be monitored continuously and unobtrusively using neurophysiological sensors. While prior studies have demonstrated the potential of functional near-infrared spectroscopy (fNIRS), a lightweight neuroimaging technology, in the prediction of social, cognitive, and affective states, few have successfully used it to measure complex social constructs like trust in artificial agents. Even fewer studies have examined the dynamics of hybrid teams of more than 1 human or 1 agent. We address this gap by developing a highly collaborative task that requires knowledge sharing within teams of 2 humans and 1 agent. Using brain data obtained with fNIRS sensors, we aim to identify brain regions sensitive to changes in agent behavior on a long- and short-term scale. We manipulated agent reliability and transparency while measuring trust, mental demand, team processes, and affect. Transparency and reliability levels are found to significantly affect trust in the agent, while transparency explanations do not impact mental demand. Reducing agent communication is shown to disrupt interpersonal trust and team cohesion, suggesting similar dynamics as human-human teams. Contrasts of General Linear Model analyses identify dorsal medial prefrontal cortex activation specific to assessing the agent's transparency explanations and characterize increases in mental demand as signaled by dorsal lateral prefrontal cortex and frontopolar activation. Short scale event-level data is analyzed to show that predicting whether an individual will trust the agent, with data from 15 s before their decision, is feasible with fNIRS data. Discussing our results, we identify targets and directions for future neuroergonomics research as a step toward building an intelligent trust-modulation system to optimize human-agent collaborations in real time.

The construct of state-level suspicion: a model and research agenda for automated and information technology (IT) contexts.

OBJECTIVE: The objective was to review and integrate available research about the construct of state-level suspicion as it appears in social science literatures and apply the resulting findings to information technology (IT) contexts. BACKGROUND: Although the human factors literature is replete with articles about trust (and distrust) in automation, there is little on the related, but distinct, construct of "suspicion" (in either automated or IT contexts). The construct of suspicion--its precise definition, theoretical correlates, and role in such applications--deserves further study. METHOD: Literatures that consider suspicion are reviewed and integrated. Literatures include communication, psychology, human factors, management, marketing, information technology, and brain/neurology. We first develop a generic model of state-level suspicion. Research propositions are then derived within IT contexts. RESULTS: Fundamental components of suspicion include (a) uncertainty, (b) increased cognitive processing (e.g., generation of alternative explanations for perceived discrepancies), and (c) perceptions of (mal)intent. State suspicion is defined as the simultaneous occurrence of these three components. Our analysis also suggests that trust inhibits suspicion, whereas distrust can be a catalyst of state-level suspicion. Based on a three-stage model of state-level suspicion, associated research propositions and questions are developed. These propositions and questions are intended to help guide future work on the measurement of suspicion (self-report and neurological), as well as the role of the construct of suspicion in models of decision making and detection of deception. CONCLUSION: The study of suspicion, including its correlates, antecedents, and consequences, is important. We hope that the social sciences will benefit from our integrated definition and model of state suspicion. The research propositions regarding suspicion in IT contexts should motivate substantial research in human factors and related fields.