Neural representations of contextual guidance in visual search of real-world scenes.

Exploiting scene context and object-object co-occurrence is critical in guiding eye movements and facilitating visual search, yet the mediating neural mechanisms are unknown. We used functional magnetic resonance imaging while observers searched for target objects in scenes and used multivariate pattern analyses (MVPA) to show that the lateral occipital complex (LOC) can predict the coarse spatial location of observers' expectations about the likely location of 213 different targets absent from the scenes. In addition, we found weaker but significant representations of context location in an area related to the orienting of attention (intraparietal sulcus, IPS) as well as a region related to scene processing (retrosplenial cortex, RSC). Importantly, the degree of agreement among 100 independent raters about the likely location to contain a target object in a scene correlated with LOC's ability to predict the contextual location while weaker but significant effects were found in IPS, RSC, the human motion area, and early visual areas (V1, V3v). When contextual information was made irrelevant to observers' behavioral task, the MVPA analysis of LOC and the other areas' activity ceased to predict the location of context. Thus, our findings suggest that the likely locations of targets in scenes are represented in various visual areas with LOC playing a key role in contextual guidance during visual search of objects in real scenes.

Feature-independent neural coding of target detection during search of natural scenes.

Visual search requires humans to detect a great variety of target objects in scenes cluttered by other objects or the natural environment. It is unknown whether there is a general purpose neural detection mechanism in the brain that codes the presence of a wide variety of categories of objects embedded in natural scenes. We provide evidence for a feature-independent coding mechanism for detecting behaviorally relevant targets in natural scenes in the dorsal frontoparietal network. Pattern classifiers using single-trial fMRI responses in the dorsal frontoparietal network reliably predicted the presence of 368 different target objects and also the observer's choices. Other vision-related areas such as the primary visual cortex, lateral occipital complex, the parahippocampal, and the fusiform gyri did not predict target presence, while high-level association areas related to general purpose decision making, including the dorsolateral prefrontal cortex and anterior cingulate, did. Activity in the intraparietal sulcus, a main area in the dorsal frontoparietal network, correlated with observers' decision confidence and with the task difficulty of individual images. These results cannot be explained by physical differences across images or eye movements. Thus, the dorsal frontoparietal network detects behaviorally relevant targets in natural scenes independent of their defining visual features and may be the human analog of the priority map in monkey lateral intraparietal cortex.

Reversing food preference through multisensory exposure.

Experiencing food craving is nearly ubiquitous and has several negative pathological impacts prompting an increase in recent craving-related research. Food cue-reactivity tasks are often used to study craving, but most paradigms ignore the individual food preferences of participants, which could confound the findings. We explored the neuropsychological correlates of food craving preference using psychophysical tasks on human participants considering their individual food preferences in a multisensory food exposure set-up. Participants were grouped into Liked Food Exposure (LFE), Disliked Food Exposure (DFE), and Neutral Control (NEC) based on their preference for sweet and savory items. Participants reported their momentary craving for the displayed food stimuli through the desire scale and bidding scale (willingness to pay) pre and post multisensory exposure. Participants were exposed to food items they either liked or disliked. Our results asserted the effect of the multisensory food exposure showing a statistically significant increase in food craving for DFE participants postexposure to disliked food items. Using computational models and statistical methods, we also show that the desire for food does not necessarily translate to a willingness to pay every time, and instantaneous subjective valuation of food craving is an important parameter for subsequent action. Our results further demonstrate the role of parietal N200 and centro-parietal P300 in reversing food preference and possibly point to the decrease of inhibitory control in up-regulating craving for disliked food.

Neural decoding of collective wisdom with multi-brain computing.

Group decisions and even aggregation of multiple opinions lead to greater decision accuracy, a phenomenon known as collective wisdom. Little is known about the neural basis of collective wisdom and whether its benefits arise in late decision stages or in early sensory coding. Here, we use electroencephalography and multi-brain computing with twenty humans making perceptual decisions to show that combining neural activity across brains increases decision accuracy paralleling the improvements shown by aggregating the observers' opinions. Although the largest gains result from an optimal linear combination of neural decision variables across brains, a simpler neural majority decision rule, ubiquitous in human behavior, results in substantial benefits. In contrast, an extreme neural response rule, akin to a group following the most extreme opinion, results in the least improvement with group size. Analyses controlling for number of electrodes and time-points while increasing number of brains demonstrate unique benefits arising from integrating neural activity across different brains. The benefits of multi-brain integration are present in neural activity as early as 200 ms after stimulus presentation in lateral occipital sites and no additional benefits arise in decision related neural activity. Sensory-related neural activity can predict collective choices reached by aggregating individual opinions, voting results, and decision confidence as accurately as neural activity related to decision components. Estimation of the potential for the collective to execute fast decisions by combining information across numerous brains, a strategy prevalent in many animals, shows large time-savings. Together, the findings suggest that for perceptual decisions the neural activity supporting collective wisdom and decisions arises in early sensory stages and that many properties of collective cognition are explainable by the neural coding of information across multiple brains. Finally, our methods highlight the potential of multi-brain computing as a technique to rapidly and in parallel gather increased information about the environment as well as to access collective perceptual/cognitive choices and mental states.

Wisdom of crowds benefits perceptual decision making across difficulty levels.

Decades of research on collective decision making has claimed that aggregated judgment of multiple individuals is more accurate than expert individual judgement. A longstanding problem in this regard has been to determine how decisions of individuals can be combined to form intelligent group decisions. Our study consisted of a random target detection task in natural scenes, where human subjects (18 subjects, 7 female) detected the presence or absence of a random target as indicated by the cue word displayed prior to stimulus display. Concurrently the neural activities (EEG signals) were recorded. A separate behavioural experiment was performed by different subjects (20 subjects, 11 female) on the same set of images to categorize the tasks according to their difficulty levels. We demonstrate that the weighted average of individual decision confidence/neural decision variables produces significantly better performance than the frequently used majority pooling algorithm. Further, the classification error rates from individual judgement were found to increase with increasing task difficulty. This error could be significantly reduced upon combining the individual decisions using group aggregation rules. Using statistical tests, we show that combining all available participants is unnecessary to achieve minimum classification error rate. We also try to explore if group aggregation benefits depend on the correlation between the individual judgements of the group and our results seem to suggest that reduced inter-subject correlation can improve collective decision making for a fixed difficulty level.

Predicting variations of perceptual performance across individuals from neural activity using pattern classifiers.

Within the past decade computational approaches adopted from the field of machine learning have provided neuroscientists with powerful new tools for analyzing neural data. For instance, previous studies have applied pattern classification algorithms to electroencephalography data to predict the category of presented visual stimuli, human observer decision choices and task difficulty. Here, we quantitatively compare the ability of pattern classifiers and three ERP metrics (peak amplitude, mean amplitude, and onset latency of the face-selective N170) to predict variations across individuals' behavioral performance in a difficult perceptual task identifying images of faces and cars embedded in noise. We investigate three different pattern classifiers (Classwise Principal Component Analysis, CPCA; Linear Discriminant Analysis, LDA; and Support Vector Machine, SVM), five training methods differing in the selection of training data sets and three analyses procedures for the ERP measures. We show that all three pattern classifier algorithms surpass traditional ERP measurements in their ability to predict individual differences in performance. Although the differences across pattern classifiers were not large, the CPCA method with training data sets restricted to EEG activity for trials in which observers expressed high confidence about their decisions performed the highest at predicting perceptual performance of observers. We also show that the neural activity predicting the performance across individuals was distributed through time starting at 120ms, and unlike the face-selective ERP response, sustained for more than 400ms after stimulus presentation, indicating that both early and late components contain information correlated with observers' behavioral performance. Together, our results further demonstrate the potential of pattern classifiers compared to more traditional ERP techniques as an analysis tool for modeling spatiotemporal dynamics of the human brain and relating neural activity to behavior.

Ideal observer analysis for task normalization of pattern classifier performance applied to EEG and fMRI data.

The application of multivariate techniques to neuroimaging and electrophysiological data has greatly enhanced the ability to detect where, when, and how functional neural information is processed during a variety of behavioral tasks. With the extension to single-trial analysis, neuroscientists are able to relate brain states to perceptual, cognitive, and motor processes. Using pattern classification methods, the neuroscientist can extract neural performance measures in a manner analogous to human behavioral performance, allowing for a consistent information content metric across measurement modalities. However, as with behavioral psychophysical performance, pattern classifier performances are a product of both the task-relevant information inherent in the brain and in the task/stimuli. Here, we argue for the use of an ideal observer framework with which the researcher can effectively normalize the observed neural performance given the task's inherent objective difficulty. We use data from a face versus car discrimination task and compare classifier performance applied to electroencephalography (EEG) and functional magnetic resonance imaging (fMRI) data with corresponding human behavior through the absolute and relative efficiency metrics. We show that confounding variables that can lead to erroneous interpretations of information content can be accounted for through comparisons to an ideal observer, allowing for more confident interpretation of the neural mechanisms involved in the task of interest. Finally, we discuss limitations of interpretation due to the transduction of indirect measures of neural activity, underlying assumptions in the optimality of the pattern classifiers, and dependence of efficiency results on signal contrast.

How Our Perception and Confidence Are Altered Using Decision Cues.

Understanding how individuals utilize social information while making perceptual decisions and how it affects their decision confidence is crucial in a society. To date, very little has been known about perceptual decision-making in humans and the associated neural mediators under social influence. The present study provides empirical evidence of how individuals are manipulated by others' decisions while performing a face/car identification task. Subjects were significantly influenced by what they perceived as the decisions of other subjects, while the cues, in reality, were manipulated independently from the stimulus. Subjects, in general, tend to increase their decision confidence when their individual decision and the cues coincide, while their confidence decreases when cues conflict with their individual judgments, often leading to reversal of decision. Using a novel statistical model, it was possible to rank subjects based on their propensity to be influenced by cues. This was subsequently corroborated by an analysis of their neural data. Neural time series analysis revealed no significant difference in decision-making using social cues in the early stages, unlike neural expectation studies with predictive cues. Multivariate pattern analysis of neural data alludes to a potential role of the frontal cortex in the later stages of visual processing, which appeared to code the effect of cues on perceptual decision-making. Specifically, the medial frontal cortex seems to play a role in facilitating perceptual decision preceded by conflicting cues.

A Minimalist Approach for Distinguishing Individual Lanthanide Ions Using Multivariate Pattern Analysis.

To discriminate among the 14 trivalent lanthanide ions, curcumin, a naturally occurring, nontoxic, off-the-shelf, commercially available compound containing a single fluorophore, was chosen as a probe in the water media at pH 6.8 and pH 8.2. By measuring the emission and absorption spectra of the probe, under the different pH conditions, and by performing linear discriminant analysis on the data, 14 Ln3+ ions were discriminated. Additionally, an easy tool for the nonspecialists was developed with easily available household substances, using a smartphone app, which added an extra advantage to this single probe. This probe possesses advantageous features in terms of low-cost and instant on-site detection of the lanthanide ions.

A computationally efficient order statistics based outlier detection technique for EEG signals.

Detecting artifacts in EEG data produced by muscle activity, eye blinks and electrical noise is a common and important problem in EEG applications. We present a novel outlier detection method based on order statistics. We propose a 2 step procedure comprising of detecting noisy EEG channels followed by detection of noisy epochs in the outlier channels. The performance of our method is tested systematically using simulated and real EEG data. Our technique produces significant improvement in detecting EEG artifacts over state-of-the-art outlier detection technique used in EEG applications. The proposed method can serve as a general outlier detection tool for different types of noisy signals.

Mental state estimation for brain--computer interfaces.

Mental state estimation is potentially useful for the development of asynchronous brain--computer interfaces. In this study, four mental states have been identified and decoded from the electrocorticograms (ECoGs) of six epileptic patients, engaged in a memory reach task. A novel signal analysis technique has been applied to high-dimensional, statistically sparse ECoGs recorded by a large number of electrodes. The strength of the proposed technique lies in its ability to jointly extract spatial and temporal patterns, responsible for encoding mental state differences. As such, the technique offers a systematic way of analyzing the spatiotemporal aspects of brain information processing and may be applicable to a wide range of spatiotemporal neurophysiological signals.

Using rapid visually evoked EEG activity for person identification.

We investigate the potential of using EEG recordings of observers performing a rapid visual categorization task for person identification. We examine a 0.5 s epoch of EEG data using machine learning techniques that, unlike most previous studies, analyze the data in a holistic manner and extracts discriminative spatio-temporal filters. The analysis of the filters suggest sparse feature representation spatially as well as temporally. The filters reveal that the neural activity that discriminates individuals is spatially localized to occipital electrodes located on the scalp above the visual cortex and temporally localized in the interval of 120-200 ms after presentation of the visual stimulus. The results demonstrate the feasibility of EEG-based person identification based on difficult perceptual tasks.

A classwise PCA-based recognition of neural data for brain-computer interfaces.

We present a simple, computationally efficient recognition algorithm that can systematically extract useful information from any large-dimensional neural datasets. The technique is based on classwise Principal Component Analysis, which employs the distribution characteristics of each class to discard non-informative subspace. We propose a two-step procedure, comprising of removal of sparse non-informative subspace of the large-dimensional data, followed by a linear combination of the data in the remaining subspace to extract meaningful features for efficient classification. Our method produces significant improvement over the standard discriminant analysis based methods. The classification results are given for iEEG and EEG signals recorded from the human brain.

Analysis of large-scale brain data for brain-computer interfaces.

We present a systematic technique for extraction of useful information from large-scale neural data in the context of brain-computer interfaces. The technique is based on a direct linear discriminant analysis, recently developed for face recognition problems. We show that this technique is capable of extracting useful information from brain data in a systematic fashion and can serve as a general analytical tool for other types of biomedical data, such as images and collections of images (movies). The performance of the method is tested on intracranial electroencephalographic data recorded from the human brain.