Enhancing performance with multisensory cues in a realistic target discrimination task.

Making decisions is an important aspect of people's lives. Decisions can be highly critical in nature, with mistakes possibly resulting in extremely adverse consequences. Yet, such decisions have often to be made within a very short period of time and with limited information. This can result in decreased accuracy and efficiency. In this paper, we explore the possibility of increasing speed and accuracy of users engaged in the discrimination of realistic targets presented for a very short time, in the presence of unimodal or bimodal cues. More specifically, we present results from an experiment where users were asked to discriminate between targets rapidly appearing in an indoor environment. Unimodal (auditory) or bimodal (audio-visual) cues could shortly precede the target stimulus, warning the users about its location. Our findings show that, when used to facilitate perceptual decision under time pressure, and in condition of limited information in real-world scenarios, spoken cues can be effective in boosting performance (accuracy, reaction times or both), and even more so when presented in bimodal form. However, we also found that cue timing plays a critical role and, if the cue-stimulus interval is too short, cues may offer no advantage. In a post-hoc analysis of our data, we also show that congruency between the response location and both the target location and the cues, can interfere with the speed and accuracy in the task. These effects should be taken in consideration, particularly when investigating performance in realistic tasks.

A meta-learning BCI for estimating decision confidence.

Objective.We investigated whether a recently introduced transfer-learning technique based on meta-learning could improve the performance of brain-computer interfaces (BCIs) for decision-confidence prediction with respect to more traditional machine learning methods.Approach.We adapted the meta-learning by biased regularisation algorithm to the problem of predicting decision confidence from electroencephalography (EEG) and electro-oculogram (EOG) data on a decision-by-decision basis in a difficult target discrimination task based on video feeds. The method exploits previous participants' data to produce a prediction algorithm that is then quickly tuned to new participants. We compared it with with the traditional single-subject training almost universally adopted in BCIs, a state-of-the-art transfer learning technique called domain adversarial neural networks, a transfer-learning adaptation of a zero-training method we used recently for a similar task, and with a simple baseline algorithm.Main results.The meta-learning approach was significantly better than other approaches in most conditions, and much better in situations where limited data from a new participant are available for training/tuning. Meta-learning by biased regularisation allowed our BCI to seamlessly integrate information from past participants with data from a specific user to produce high-performance predictors. Its robustness in the presence of small training sets is a real-plus in BCI applications, as new users need to train the BCI for a much shorter period.Significance.Due to the variability and noise of EEG/EOG data, BCIs need to be normally trained with data from a specific participant. This work shows that even better performance can be obtained using our version of meta-learning by biased regularisation.

Anytime collaborative brain-computer interfaces for enhancing perceptual group decision-making.

In this paper we present, and test in two realistic environments, collaborative Brain-Computer Interfaces (cBCIs) that can significantly increase both the speed and the accuracy of perceptual group decision-making. The key distinguishing features of this work are: (1) our cBCIs combine behavioural, physiological and neural data in such a way as to be able to provide a group decision at any time after the quickest team member casts their vote, but the quality of a cBCI-assisted decision improves monotonically the longer the group decision can wait; (2) we apply our cBCIs to two realistic scenarios of military relevance (patrolling a dark corridor and manning an outpost at night where users need to identify any unidentified characters that appear) in which decisions are based on information conveyed through video feeds; and (3) our cBCIs exploit Event-Related Potentials (ERPs) elicited in brain activity by the appearance of potential threats but, uniquely, the appearance time is estimated automatically by the system (rather than being unrealistically provided to it). As a result of these elements, in the two test environments, groups assisted by our cBCIs make both more accurate and faster decisions than when individual decisions are integrated in more traditional manners.

Subject- and task-independent neural correlates and prediction of decision confidence in perceptual decision making.

Objective.In many real-world decision tasks, the information available to the decision maker is incomplete. To account for this uncertainty, we associate a degree of confidence to every decision, representing the likelihood of that decision being correct. In this study, we analyse electroencephalography (EEG) data from 68 participants undertaking eight different perceptual decision-making experiments. Our goals are to investigate (1) whether subject- and task-independent neural correlates of decision confidence exist, and (2) to what degree it is possible to build brain computer interfaces that can estimate confidence on a trial-by-trial basis. The experiments cover a wide range of perceptual tasks, which allowed to separate the task-related, decision-making features from the task-independent ones.Approach.Our systems train artificial neural networks to predict the confidence in each decision from EEG data and response times. We compare the decoding performance with three training approaches: (1) single subject, where both training and testing data were acquired from the same person; (2) multi-subject, where all the data pertained to the same task, but the training and testing data came from different users; and (3) multi-task, where the training and testing data came from different tasks and subjects. Finally, we validated our multi-task approach using data from two additional experiments, in which confidence was not reported.Main results.We found significant differences in the EEG data for different confidence levels in both stimulus-locked and response-locked epochs. All our approaches were able to predict the confidence between 15% and 35% better than the corresponding reference baselines.Significance.Our results suggest that confidence in perceptual decision making tasks could be reconstructed from neural signals even when using transfer learning approaches. These confidence estimates are based on the decision-making process rather than just the confidence-reporting process.

Neurotechnologies for Human Cognitive Augmentation: Current State of the Art and Future Prospects.

Recent advances in neuroscience have paved the way to innovative applications that cognitively augment and enhance humans in a variety of contexts. This paper aims at providing a snapshot of the current state of the art and a motivated forecast of the most likely developments in the next two decades. Firstly, we survey the main neuroscience technologies for both observing and influencing brain activity, which are necessary ingredients for human cognitive augmentation. We also compare and contrast such technologies, as their individual characteristics (e.g., spatio-temporal resolution, invasiveness, portability, energy requirements, and cost) influence their current and future role in human cognitive augmentation. Secondly, we chart the state of the art on neurotechnologies for human cognitive augmentation, keeping an eye both on the applications that already exist and those that are emerging or are likely to emerge in the next two decades. Particularly, we consider applications in the areas of communication, cognitive enhancement, memory, attention monitoring/enhancement, situation awareness and complex problem solving, and we look at what fraction of the population might benefit from such technologies and at the demands they impose in terms of user training. Thirdly, we briefly review the ethical issues associated with current neuroscience technologies. These are important because they may differentially influence both present and future research on (and adoption of) neurotechnologies for human cognitive augmentation: an inferior technology with no significant ethical issues may thrive while a superior technology causing widespread ethical concerns may end up being outlawed. Finally, based on the lessons learned in our analysis, using past trends and considering other related forecasts, we attempt to forecast the most likely future developments of neuroscience technology for human cognitive augmentation and provide informed recommendations for promising future research and exploitation avenues.

Brain⁻Computer Interfaces for Human Augmentation.

The field of brain⁻computer interfaces (BCIs) has grown rapidly in the last few decades, allowing the development of ever faster and more reliable assistive technologies for converting brain activity into control signals for external devices for people with severe disabilities [...].

Collaborative Brain-Computer Interfaces to Enhance Group Decisions in an Outpost Surveillance Task.

We present a two-layered collaborative Brain-Computer Interface (cBCI) to aid groups making decisions under time constraints in a realistic video surveillance setting - the very first cBCI application of this type. The cBCI first uses response times (RTs) to estimate the decision confidence the user would report after each decision. Such an estimate is then used with neural features extracted from EEG to refine the decision confidence so that it better correlates with the correctness of the decision. The refined confidence is then used to weigh individual responses and obtain group decisions. Results obtained with 10 participants indicate that cBCI-assisted groups are significantly more accurate than groups using standard majority or weighing decisions using reported confidence values. This two-layer architecture allows the cBCI to not only further enhance group performance but also speed up the decision process, as the cBCI does not have to wait for all users to report their confidence after each decision.

Towards augmented human memory: Retrieval-induced forgetting and retrieval practice in an interactive, end-of-day review.

The authors report 6 experiments that examined the contention that an end-of-day review could lead to augmentation in human memory. In Experiment 1, participants in the study phase were presented with a campus tour of different to-be-remembered objects in different university locations. Each to-be-remembered object was presented with an associated specific comment. Participants were then shown the location name and photographs of half of the objects from half of the locations, and they were asked to try to name the object and recall the associated comment specific to each item. Following a filled delay, participants were presented with the name of each campus location and were asked to free recall the to-be-remembered objects. Relative to the recall from the unpracticed location categories, participants recalled the names of significantly more objects that they practiced (retrieval practice) and significantly fewer unpracticed objects from the practiced locations (retrieval-induced forgetting, RIF). These findings were replicated in Experiment 2 using a campus scavenger hunt in which participants selected their own stimuli from experimenter's categories. Following an examination of factors that maximized the effects of RIF and retrieval practice in the laboratory (Experiment 3), the authors applied these findings to the campus scavenger hunt task to create different retrieval practice schedules to maximize and minimize recall of items based on experimenter-selected (Experiment 4) and participant-selected items using both category-cued free recall (Experiment 5) and item-specific cues (Experiment 6). Their findings support the claim that an interactive, end-of-day review could lead to augmentation in human memory. (PsycINFO Database Record

Serial position, output order, and list length effects for words presented on smartphones over very long intervals.

Three experiments examined whether or not benchmark findings observed in the immediate retrieval from episodic memory are similarly observed over much greater time-scales. Participants were presented with experimentally-controlled lists of words at the very slow rate of one word every hour using an iPhone recall application, RECAPP, which was also used to recall the words in either any order (free recall: Experiments 1 to 3) or the same order as presented (serial recall: Experiment 3). We found strong temporal contiguity effects, weak serial position effects with very limited recency, and clear list length effects in free recall; clear primacy effects and classic error gradients in serial recall; and recency effects in a final two-alternative forced choice recognition task (Experiments 2 and 3). Our findings extend the timescales over which temporal contiguity effects have been observed, but failed to find consistent evidence for strong long-term recency effects with experimenter-controlled stimuli.

Group Augmentation in Realistic Visual-Search Decisions via a Hybrid Brain-Computer Interface.

Groups have increased sensing and cognition capabilities that typically allow them to make better decisions. However, factors such as communication biases and time constraints can lead to less-than-optimal group decisions. In this study, we use a hybrid Brain-Computer Interface (hBCI) to improve the performance of groups undertaking a realistic visual-search task. Our hBCI extracts neural information from EEG signals and combines it with response times to build an estimate of the decision confidence. This is used to weigh individual responses, resulting in improved group decisions. We compare the performance of hBCI-assisted groups with the performance of non-BCI groups using standard majority voting, and non-BCI groups using weighted voting based on reported decision confidence. We also investigate the impact on group performance of a computer-mediated form of communication between members. Results across three experiments suggest that the hBCI provides significant advantages over non-BCI decision methods in all cases. We also found that our form of communication increases individual error rates by almost 50% compared to non-communicating observers, which also results in worse group performance. Communication also makes reported confidence uncorrelated with the decision correctness, thereby nullifying its value in weighing votes. In summary, best decisions are achieved by hBCI-assisted, non-communicating groups.

Enhancement of Group Perception via a Collaborative Brain-Computer Interface.

OBJECTIVE: We aimed at improving group performance in a challenging visual search task via a hybrid collaborative brain-computer interface (cBCI). METHODS: Ten participants individually undertook a visual search task where a display was presented for 250 ms, and they had to decide whether a target was present or not. Local temporal correlation common spatial pattern (LTCCSP) was used to extract neural features from response- and stimulus-locked EEG epochs. The resulting feature vectors were extended by including response times and features extracted from eye movements. A classifier was trained to estimate the confidence of each group member. cBCI-assisted group decisions were then obtained using a confidence-weighted majority vote. RESULTS: Participants were combined in groups of different sizes to assess the performance of the cBCI. Results show that LTCCSP neural features, response times, and eye movement features significantly improve the accuracy of the cBCI over what we achieved with previous systems. For most group sizes, our hybrid cBCI yields group decisions that are significantly better than majority-based group decisions. CONCLUSION: The visual task considered here was much harder than a task we used in previous research. However, thanks to a range of technological enhancements, our cBCI has delivered a significant improvement over group decisions made by a standard majority vote. SIGNIFICANCE: With previous cBCIs, groups may perform better than single non-BCI users. Here, cBCI-assisted groups are more accurate than identically sized non-BCI groups. This paves the way to a variety of real-world applications of cBCIs where reducing decision errors is vital.

Collaborative brain-computer interface for aiding decision-making.

We look at the possibility of integrating the percepts from multiple non-communicating observers as a means of achieving better joint perception and better group decisions. Our approach involves the combination of a brain-computer interface with human behavioural responses. To test ideas in controlled conditions, we asked observers to perform a simple matching task involving the rapid sequential presentation of pairs of visual patterns and the subsequent decision as whether the two patterns in a pair were the same or different. We recorded the response times of observers as well as a neural feature which predicts incorrect decisions and, thus, indirectly indicates the confidence of the decisions made by the observers. We then built a composite neuro-behavioural feature which optimally combines the two measures. For group decisions, we uses a majority rule and three rules which weigh the decisions of each observer based on response times and our neural and neuro-behavioural features. Results indicate that the integration of behavioural responses and neural features can significantly improve accuracy when compared with the majority rule. An analysis of event-related potentials indicates that substantial differences are present in the proximity of the response for correct and incorrect trials, further corroborating the idea of using hybrids of brain-computer interfaces and traditional strategies for improving decision making.

Novel protocols for P300-based brain-computer interfaces.

The oddball protocol is often used in brain-computer interfaces (BCIs) to induce P300 ERPs, although, recently, some issues have been shown to detrimentally effect its performance. In this paper, we study a new periodic protocol and explore whether it can compete with the standard oddball protocol within the context of a BCI mouse. We found that the new protocol consistently and significantly outperforms the standard oddball protocol in relation to information transfer rates (33 bits/min for the former and 22 bits/min for the latter, measured at 90% accuracy) as well as P300 amplitudes. Furthermore, we performed a comparison of two periodic protocols with two less conventional oddball-like protocols that reveals the importance of the interactions between task and sequence in determining the success of a protocol.

Exploring multiple protocols for a brain-computer interface mouse.

In recent years, various visual protocols have been explored for P300-based BCI. In stimulus-driven BCI paradigms such as P300 BCIs it is vital to optimise the stimulation protocol as much as possible in order to achieve the best performance. Due to the inherent variability between subjects and the complex nature of the brain it is unlikely that an optimal protocol will be identified through a single iteration of random exploration. That is why in this paper we explore 8 different visual protocol configurations based on recent literature, in the hope of identifying key features that can later be used to create further improved protocols. Results indicate that luminosity changes, the standard method of stimulation used in visual P300 BCI protocols, do provide the best performance of the variations presented here.

Eigenbrains: The free vibrational modes of the brain as a new representation for EEG.

We present a new transform for EEG signals whose basis functions are well suited to represent the large-scale dynamics associated with event related potentials. The method involves instantiating an approximate model of the electrical properties of the brain as a conductor medium and then studying the free vibrational modes of the model. These form a set of basis functions, which we call eigenbrains, that can be used to meaningfully re-represent the brain's electrical activity. Eigenbrains are compared to principal component analysis and independent component analysis to highlight differences and similarities.

Documenting, modelling and exploiting P300 amplitude changes due to variable target delays in Donchin's speller.

The P300 is an endogenous event-related potential (ERP) that is naturally elicited by rare and significant external stimuli. P300s are used increasingly frequently in brain-computer interfaces (BCIs) because the users of ERP-based BCIs need no special training. However, P300 waves are hard to detect and, therefore, multiple target stimulus presentations are needed before an interface can make a reliable decision. While significant improvements have been made in the detection of P300s, no particular attention has been paid to the variability in shape and timing of P300 waves in BCIs. In this paper we start filling this gap by documenting, modelling and exploiting a modulation in the amplitude of P300s related to the number of non-targets preceding a target in a Donchin speller. The basic idea in our approach is to use an appropriately weighted average of the responses produced by a classifier during multiple stimulus presentations, instead of the traditional plain average. This makes it possible to weigh more heavily events that are likely to be more informative, thereby increasing the accuracy of classification. The optimal weights are determined through a mathematical model that precisely estimates the accuracy of our speller as well as the expected performance improvement w.r.t. the traditional approach. Tests with two independent datasets show that our approach provides a marked statistically significant improvement in accuracy over the top-performing algorithm presented in the literature to date. The method and the theoretical models we propose are general and can easily be used in other P300-based BCIs with minimal changes.

Reaction-time binning: a simple method for increasing the resolving power of ERP averages.

Stimulus-locked, response-locked, and ERP-locked averaging are effective methods for reducing artifacts in ERP analysis. However, they suffer from a magnifying-glass effect: they increase the resolution of specific ERPs at the cost of blurring other ERPs. Here we propose an extremely simple technique-binning trials based on response times and then averaging-which can significantly alleviate the problems of other averaging methods. We have empirically evaluated the technique in an experiment where the task requires detecting a target in the presence of distractors. We have also studied the signal-to-noise ratio and the resolving power of averages with and without binning. Results indicate that the method produces clearer representations of ERPs than either stimulus-locked and response-locked averaging, revealing finer details of ERPs and helping in the evaluation of the amplitude and latency of ERP waves. The method is applicable to within-subject and between-subject averages.

Semantic activation and letter search: blocking or suppression?

Two experiments investigated associative priming with a letter-search prime task where either the prime and letter probe were presented simultaneously, or the letter probe appeared 200 ms (Experiment 1) or 300 ms (Experiment 2) after the prime. Weak associative priming was observed in both experiments, but unlike Stolz and Besner (1996) we found no evidence that priming was increased when the probe was delayed. However, strong associative priming was observed when a semantic decision had to be made on the prime (Experiment 3). Our results are consistent with an account where semantic activation of the prime occurs but its action on the target is suppressed by the prime task. The persistence of weak priming effects with the letter search task is explained in terms of the low-frequency items used.