Enhancing performance with multisensory cues in a realistic target discrimination task.

Making decisions is an important aspect of people's lives. Decisions can be highly critical in nature, with mistakes possibly resulting in extremely adverse consequences. Yet, such decisions have often to be made within a very short period of time and with limited information. This can result in decreased accuracy and efficiency. In this paper, we explore the possibility of increasing speed and accuracy of users engaged in the discrimination of realistic targets presented for a very short time, in the presence of unimodal or bimodal cues. More specifically, we present results from an experiment where users were asked to discriminate between targets rapidly appearing in an indoor environment. Unimodal (auditory) or bimodal (audio-visual) cues could shortly precede the target stimulus, warning the users about its location. Our findings show that, when used to facilitate perceptual decision under time pressure, and in condition of limited information in real-world scenarios, spoken cues can be effective in boosting performance (accuracy, reaction times or both), and even more so when presented in bimodal form. However, we also found that cue timing plays a critical role and, if the cue-stimulus interval is too short, cues may offer no advantage. In a post-hoc analysis of our data, we also show that congruency between the response location and both the target location and the cues, can interfere with the speed and accuracy in the task. These effects should be taken in consideration, particularly when investigating performance in realistic tasks.

A meta-learning BCI for estimating decision confidence.

Objective.We investigated whether a recently introduced transfer-learning technique based on meta-learning could improve the performance of brain-computer interfaces (BCIs) for decision-confidence prediction with respect to more traditional machine learning methods.Approach.We adapted the meta-learning by biased regularisation algorithm to the problem of predicting decision confidence from electroencephalography (EEG) and electro-oculogram (EOG) data on a decision-by-decision basis in a difficult target discrimination task based on video feeds. The method exploits previous participants' data to produce a prediction algorithm that is then quickly tuned to new participants. We compared it with with the traditional single-subject training almost universally adopted in BCIs, a state-of-the-art transfer learning technique called domain adversarial neural networks, a transfer-learning adaptation of a zero-training method we used recently for a similar task, and with a simple baseline algorithm.Main results.The meta-learning approach was significantly better than other approaches in most conditions, and much better in situations where limited data from a new participant are available for training/tuning. Meta-learning by biased regularisation allowed our BCI to seamlessly integrate information from past participants with data from a specific user to produce high-performance predictors. Its robustness in the presence of small training sets is a real-plus in BCI applications, as new users need to train the BCI for a much shorter period.Significance.Due to the variability and noise of EEG/EOG data, BCIs need to be normally trained with data from a specific participant. This work shows that even better performance can be obtained using our version of meta-learning by biased regularisation.

Subject- and task-independent neural correlates and prediction of decision confidence in perceptual decision making.

Objective.In many real-world decision tasks, the information available to the decision maker is incomplete. To account for this uncertainty, we associate a degree of confidence to every decision, representing the likelihood of that decision being correct. In this study, we analyse electroencephalography (EEG) data from 68 participants undertaking eight different perceptual decision-making experiments. Our goals are to investigate (1) whether subject- and task-independent neural correlates of decision confidence exist, and (2) to what degree it is possible to build brain computer interfaces that can estimate confidence on a trial-by-trial basis. The experiments cover a wide range of perceptual tasks, which allowed to separate the task-related, decision-making features from the task-independent ones.Approach.Our systems train artificial neural networks to predict the confidence in each decision from EEG data and response times. We compare the decoding performance with three training approaches: (1) single subject, where both training and testing data were acquired from the same person; (2) multi-subject, where all the data pertained to the same task, but the training and testing data came from different users; and (3) multi-task, where the training and testing data came from different tasks and subjects. Finally, we validated our multi-task approach using data from two additional experiments, in which confidence was not reported.Main results.We found significant differences in the EEG data for different confidence levels in both stimulus-locked and response-locked epochs. All our approaches were able to predict the confidence between 15% and 35% better than the corresponding reference baselines.Significance.Our results suggest that confidence in perceptual decision making tasks could be reconstructed from neural signals even when using transfer learning approaches. These confidence estimates are based on the decision-making process rather than just the confidence-reporting process.

EEG Movement Artifact Suppression in Interactive Virtual Reality.

The integration of electroencephalogram (EEG) sensors into virtual reality (VR) headsets can provide the capability of tracking the user's cognitive state and eventually be used to increase the sense of immersion. Recent developments in wireless, room-scale VR tracking allow users to move freely in the physical and virtual spaces. Such motion can create significant movement artifacts in EEG sensors mounted to the VR headset. This study explores the removal of EEG movement artifacts caused by repetitive, stereotyped movements during an interactive VR task.

Estimating Cognitive Workload in an Interactive Virtual Reality Environment Using EEG.

With the recent surge of affordable, high-performance virtual reality (VR) headsets, there is unlimited potential for applications ranging from education, to training, to entertainment, to fitness and beyond. As these interfaces continue to evolve, passive user-state monitoring can play a key role in expanding the immersive VR experience, and tracking activity for user well-being. By recording physiological signals such as the electroencephalogram (EEG) during use of a VR device, the user's interactions in the virtual environment could be adapted in real-time based on the user's cognitive state. Current VR headsets provide a logical, convenient, and unobtrusive framework for mounting EEG sensors. The present study evaluates the feasibility of passively monitoring cognitive workload via EEG while performing a classical n-back task in an interactive VR environment. Data were collected from 15 participants and the spatio-spectral EEG features were analyzed with respect to task performance. The results indicate that scalp measurements of electrical activity can effectively discriminate three workload levels, even after suppression of a co-varying high-frequency activity.

Decoding Mental Workload in Virtual Environments: A fNIRS Study using an Immersive n-back Task.

Virtual Reality (VR) has emerged as a novel paradigm for immersive applications in training, entertainment, rehabilitation, and other domains. In this paper, we investigate the automatic classification of mental workload from brain activity measured through functional near-infrared spectroscopy (fNIRS) in VR. We present results from a study which implements the established n-back task in an immersive visual scene, including physical interaction. Our results show that user workload can be detected from fNIRS signals in immersive VR tasks both person-dependently and -adaptively.

Experimental verification of ion stopping power prediction from dual energy CT data in tissue surrogates.

We present an experimental verification of stopping-power-ratio (SPR) prediction from dual energy CT (DECT) with potential use for dose planning in proton and ion therapy. The approach is based on DECT images converted to electron density relative to water ϱe/ϱe, w and effective atomic number Zeff. To establish a parameterization of the I-value by Zeff, 71 tabulated tissue compositions were used. For the experimental assessment of the method we scanned 20 materials (tissue surrogates, polymers, aluminum, titanium) at 80/140Sn kVp and 100/140Sn kVp (Sn: additional tin filtration) and computed the ϱe/ϱe, w and Zeff with a purely image based algorithm. Thereby, we found that ϱe/ϱe, w (Zeff) could be determined with an accuracy of 0.4% (1.7%) for the tissue surrogates with known elemental compositions. SPRs were predicted from DECT images for all 20 materials using the presented approach and were compared to measured water-equivalent path lengths (closely related to SPR). For the tissue surrogates the presented DECT approach was found to predict the experimental values within 0.6%, for aluminum and titanium within an accuracy of 1.7% and 9.4% (from 16-bit reconstructed DECT images).

MRI-based treatment plan simulation and adaptation for ion radiotherapy using a classification-based approach.

BACKGROUND: In order to benefit from the highly conformal irradiation of tumors in ion radiotherapy, sophisticated treatment planning and simulation are required. The purpose of this study was to investigate the potential of MRI for ion radiotherapy treatment plan simulation and adaptation using a classification-based approach. METHODS: Firstly, a voxelwise tissue classification was applied to derive pseudo CT numbers from MR images using up to 8 contrasts. Appropriate MR sequences and parameters were evaluated in cross-validation studies of three phantoms. Secondly, ion radiotherapy treatment plans were optimized using both MRI-based pseudo CT and reference CT and recalculated on reference CT. Finally, a target shift was simulated and a treatment plan adapted to the shift was optimized on a pseudo CT and compared to reference CT optimizations without plan adaptation. RESULTS: The derivation of pseudo CT values led to mean absolute errors in the range of 81 - 95 HU. Most significant deviations appeared at borders between air and different tissue classes and originated from partial volume effects. Simulations of ion radiotherapy treatment plans using pseudo CT for optimization revealed only small underdosages in distal regions of a target volume with deviations of the mean dose of PTV between 1.4 - 3.1% compared to reference CT optimizations. A plan adapted to the target volume shift and optimized on the pseudo CT exhibited a comparable target dose coverage as a non-adapted plan optimized on a reference CT. CONCLUSIONS: We were able to show that a MRI-based derivation of pseudo CT values using a purely statistical classification approach is feasible although no physical relationship exists. Large errors appeared at compact bone classes and came from an imperfect distinction of bones and other tissue types in MRI. In simulations of treatment plans, it was demonstrated that these deviations are comparable to uncertainties of a target volume shift of 2 mm in two directions indicating that especially applications for adaptive ion radiotherapy are interesting.