NAPS Fusion: A framework to overcome experimental data limitations to predict human performance and cognitive task outcomes.

In the area of human performance and cognitive research, machine learning (ML) problems become increasingly complex due to limitations in the experimental design, resulting in the development of poor predictive models. More specifically, experimental study designs produce very few data instances, have large class imbalances and conflicting ground truth labels, and generate wide data sets due to the diverse amount of sensors. From an ML perspective these problems are further exacerbated in anomaly detection cases where class imbalances occur and there are almost always more features than samples. Typically, dimensionality reduction methods (e.g., PCA, autoencoders) are utilized to handle these issues from wide data sets. However, these dimensionality reduction methods do not always map to a lower dimensional space appropriately, and they capture noise or irrelevant information. In addition, when new sensor modalities are incorporated, the entire ML paradigm has to be remodeled because of new dependencies introduced by the new information. Remodeling these ML paradigms is time-consuming and costly due to lack of modularity in the paradigm design, which is not ideal. Furthermore, human performance research experiments, at times, creates ambiguous class labels because the ground truth data cannot be agreed upon by subject-matter experts annotations, making ML paradigm nearly impossible to model. This work pulls insights from Dempster-Shafer theory (DST), stacking of ML models, and bagging to address uncertainty and ignorance for multi-classification ML problems caused by ambiguous ground truth, low samples, subject-to-subject variability, class imbalances, and wide data sets. Based on these insights, we propose a probabilistic model fusion approach, Naive Adaptive Probabilistic Sensor (NAPS), which combines ML paradigms built around bagging algorithms to overcome these experimental data concerns while maintaining a modular design for future sensor (new feature integration) and conflicting ground truth data. We demonstrate significant overall performance improvements using NAPS (an accuracy of 95.29%) in detecting human task errors (a four class problem) caused by impaired cognitive states and a negligible drop in performance with the case of ambiguous ground truth labels (an accuracy of 93.93%), when compared to other methodologies (an accuracy of 64.91%). This work potentially sets the foundation for other human-centric modeling systems that rely on human state prediction modeling.

A Psychophysiological Model of Firearms Training in Police Officers: A Virtual Reality Experiment for Biocybernetic Adaptation.

Crucial elements for police firearms training include mastering very specific psychophysiological responses associated with controlled breathing while shooting. Under high-stress situations, the shooter is affected by responses of the sympathetic nervous system that can impact respiration. This research focuses on how frontal oscillatory brainwaves and cardiovascular responses of trained police officers (N = 10) are affected during a virtual reality (VR) firearms training routine. We present data from an experimental study wherein shooters were interacting in a VR-based training simulator designed to elicit psychophysiological changes under easy, moderate and frustrating difficulties. Outcome measures in this experiment include electroencephalographic and heart rate variability (HRV) parameters, as well as performance metrics from the VR simulator. Results revealed that specific frontal areas of the brain elicited different responses during resting states when compared with active shooting in the VR simulator. Moreover, sympathetic signatures were found in the HRV parameters (both time and frequency) reflecting similar differences. Based on the experimental findings, we propose a psychophysiological model to aid the design of a biocybernetic adaptation layer that creates real-time modulations in simulation difficulty based on targeted physiological responses.

Activation Complexity: A Cognitive Impairment Tool for Characterizing Neuro-isolation.

Electroencephalography (EEG) is a method for recording electrical activity, indicative of cortical brain activity from the scalp. EEG has been used to diagnose neurological diseases and to characterize impaired cognitive states. When the electrical activity of neurons are temporally synchronized, the likelihood to reach their threshold potential for the signal to propagate to the next neuron, increases. This phenomenon is typically analyzed as the spectral intensity increasing from the summation of these neurons firing. Non-linear analysis methods (e.g., entropy) have been explored to characterize neuronal firings, but only analyze temporal information and not the frequency spectrum. By examining temporal and spectral entropic relationships simultaneously, we can better characterize how neurons are isolated, (the signal's inability to propagate to adjacent neurons), an indicator of impairment. A novel time-frequency entropic analysis method, referred to as Activation Complexity (AC), was designed to quantify these dynamics from key EEG frequency bands. The data was collected during a cognitive impairment study at NASA Langley Research Center, involving hypoxia induction in 49 human test subjects. AC demonstrated significant changes in EEG firing patterns characterize within explanatory (p < 0.05) and predictive models (10% increase in accuracy). The proposed work sets the methodological foundation for quantifying neuronal isolation and introduces new potential technique to understand human cognitive impairment for a range of neurological diseases and insults.

Uncertainty in heart rate complexity metrics caused by R-peak perturbations.

Heart rate complexity (HRC) is a proven metric for gaining insight into human stress and physiological deterioration. To calculate HRC, the detection of the exact instance of when the heart beats, the R-peak, is necessary. Electrocardiogram (ECG) signals can often be corrupted by environmental noise (e.g., from electromagnetic interference, movement artifacts), which can potentially alter the HRC measurement, producing erroneous inputs which feed into decision support models. Current literature has only investigated how HRC is affected by noise when R-peak detection errors occur (false positives and false negatives). However, the numerical methods used to calculate HRC are also sensitive to the specific location of the fiducial point of the R-peak. This raises many questions regarding how this fiducial point is altered by noise, the resulting impact on the measured HRC, and how we can account for noisy HRC measures as inputs into our decision models. This work uses Monte Carlo simulations to systematically add white and pink noise at different permutations of signal-to-noise ratios (SNRs), time segments, sampling rates, and HRC measurements to characterize the influence of noise on the HRC measure by altering the fiducial point of the R-peak. Using the generated information from these simulations provides improved decision processes for system design which address key concerns such as permutation entropy being a more precise, reliable, less biased, and more sensitive measurement for HRC than sample and approximate entropy.

Redundancy analysis of autonomic and self-reported, responses to induced emotions.

The issue of concordance among the elements of emotional states has been prominent in the literature since Lang (1968) explored the topic in relation to therapy for anxiety. Since that time, a consensus has emerged that concordance among these components is relatively low. To address this issue, redundancy analysis, a technique for examining directional relationships between two sets of multivariate data, was applied to data from a previously published study (Stephens, Christie, & Friedman, 2010). Subjects in this study listened to emotion-inducing music and viewed affective films while a montage of autonomic variables, as well as self-reported affective responses, were recorded. Results indicated that approximately 27-28% of the variance in self-reported affect could be explained by autonomic variables, and vice-versa. When all of the constraints of this emotion research paradigm are considered, these levels of explained variance indicate substantial coherence between feelings and physiology during the emotion inductions. These results are considered vis-à-vis the low levels of coherence that have often been reported in the literature.

Autonomic specificity of basic emotions: evidence from pattern classification and cluster analysis.

Autonomic nervous system (ANS) specificity of emotion remains controversial in contemporary emotion research, and has received mixed support over decades of investigation. This study was designed to replicate and extend psychophysiological research, which has used multivariate pattern classification analysis (PCA) in support of ANS specificity. Forty-nine undergraduates (27 women) listened to emotion-inducing music and viewed affective films while a montage of ANS variables, including heart rate variability indices, peripheral vascular activity, systolic time intervals, and electrodermal activity, were recorded. Evidence for ANS discrimination of emotion was found via PCA with 44.6% of overall observations correctly classified into the predicted emotion conditions, using ANS variables (z=16.05, p<.001). Cluster analysis of these data indicated a lack of distinct clusters, which suggests that ANS responses to the stimuli were nomothetic and stimulus-specific rather than idiosyncratic and individual-specific. Collectively these results further confirm and extend support for the notion that basic emotions have distinct ANS signatures.