A fully spiking coupled model of a deep neural network and a recurrent attractor explains dynamics of decision making in an object recognition task.

Objective.Object recognition and making a choice regarding the recognized object is pivotal for most animals. This process in the brain contains information representation and decision making steps which both take different amount of times for different objects. While dynamics of object recognition and decision making are usually ignored in object recognition models, here we proposed a fully spiking hierarchical model, explaining the process of object recognition from information representation to making decision.Approach.Coupling a deep neural network and a recurrent attractor based decision making model beside using spike time dependent plasticity learning rules in several convolutional and pooling layers, we proposed a model which can resemble brain behaviors during an object recognition task. We also measured human choices and reaction times in a psychophysical object recognition task and used it as a reference to evaluate the model.Main results.The proposed model explains not only the probability of making a correct decision but also the time that it takes to make a decision. Importantly, neural firing rates in both feature representation and decision making levels mimic the observed patterns in animal studies (number of spikes (p-value < 10-173) and the time of the peak response (p-value < 10-31) are significantly modulated with the strength of the stimulus). Moreover, the speed-accuracy trade-off as a well-known characteristic of decision making process in the brain is also observed in the model (changing the decision bound significantly affect the reaction time (p-value < 10-59) and accuracy (p-value < 10-165)).Significance.We proposed a fully spiking deep neural network which can explain dynamics of making decision about an object in both neural and behavioral level. Results showed that there is a strong and significant correlation (r= 0.57) between the reaction time of the model and of human participants in the psychophysical object recognition task.

Interpersonal alignment of neural evidence accumulation to social exchange of confidence.

Private, subjective beliefs about uncertainty have been found to have idiosyncratic computational and neural substrates yet, humans share such beliefs seamlessly and cooperate successfully. Bringing together decision making under uncertainty and interpersonal alignment in communication, in a discovery plus pre-registered replication design, we examined the neuro-computational basis of the relationship between privately held and socially shared uncertainty. Examining confidence-speed-accuracy trade-off in uncertainty-ridden perceptual decisions under social vs isolated context, we found that shared (i.e. reported confidence) and subjective (inferred from pupillometry) uncertainty dynamically followed social information. An attractor neural network model incorporating social information as top-down additive input captured the observed behavior and demonstrated the emergence of social alignment in virtual dyadic simulations. Electroencephalography showed that social exchange of confidence modulated the neural signature of perceptual evidence accumulation in the central parietal cortex. Our findings offer a neural population model for interpersonal alignment of shared beliefs.

Functional sophistication in human escape.

Animals including humans must cope with immediate threat and make rapid decisions to survive. Without much leeway for cognitive or motor errors, this poses a formidable computational problem. Utilizing fully immersive virtual reality with 13 natural threats, we examined escape decisions in N = 59 humans. We show that escape goals are dynamically updated according to environmental changes. The decision whether and when to escape depends on time-to-impact, threat identity and predicted trajectory, and stable personal characteristics. Its implementation appears to integrate secondary goals such as behavioral affordances. Perturbance experiments show that the underlying decision algorithm exhibits planning properties and can integrate novel actions. In contrast, rapid information-seeking and foraging-suppression are only partly devaluation-sensitive. Instead of being instinctive or hardwired stimulus-response patterns, human escape decisions integrate multiple variables in a flexible computational architecture. Taken together, we provide steps toward a computational model of how the human brain rapidly solves survival challenges.

Resolving the neural mechanism of core object recognition in space and time: A computational approach.

The underlying mechanism of object recognition- a fundamental brain ability- has been investigated in various studies. However, balancing between the speed and accuracy of recognition is less explored. Most of the computational models of object recognition are not potentially able to explain the recognition time and, thus, only focus on the recognition accuracy because of two reasons: lack of a temporal representation mechanism for sensory processing and using non-biological classifiers for decision-making processing. Here, we proposed a hierarchical temporal model of object recognition using a spiking deep neural network coupled to a biologically plausible decision-making model for explaining both recognition time and accuracy. We showed that the response dynamics of the proposed model can resemble those of the brain. Firstly, in an object recognition task, the model can mimic human's and monkey's recognition time as well as accuracy. Secondly, the model can replicate different speed-accuracy trade-off regimes as observed in the literature. More importantly, we demonstrated that temporal representation of different abstraction levels (superordinate, midlevel, and subordinate) in the proposed model matched the brain representation dynamics observed in previous studies. We conclude that the accumulation of spikes, generated by a hierarchical feedforward spiking structure, to reach abound can well explain not even the dynamics of making a decision, but also the representations dynamics for different abstraction levels.

A temporal hierarchical feedforward model explains both the time and the accuracy of object recognition.

Brain can recognize different objects as ones it has previously experienced. The recognition accuracy and its processing time depend on different stimulus properties such as the viewing conditions, the noise levels, etc. Recognition accuracy can be explained well by different models. However, most models paid no attention to the processing time, and the ones which do, are not biologically plausible. By modifying a hierarchical spiking neural network (spiking HMAX), the input stimulus is represented temporally within the spike trains. Then, by coupling the modified spiking HMAX model, with an accumulation-to-bound decision-making model, the generated spikes are accumulated over time. The input category is determined as soon as the firing rates of accumulators reaches a threshold (decision bound). The proposed object recognition model accounts for both recognition time and accuracy. Results show that not only does the model follow human accuracy in a psychophysical task better than the well-known non-temporal models, but also it predicts human response time in each choice. Results provide enough evidence that the temporal representation of features is informative, since it can improve the accuracy of a biologically plausible decision maker over time. In addition, the decision bound is able to adjust the speed-accuracy trade-off in different object recognition tasks.

Inherent Importance of Early Visual Features in Attraction of Human Attention.

Local contrasts attract human attention to different areas of an image. Studies have shown that orientation, color, and intensity are some basic visual features which their contrasts attract our attention. Since these features are in different modalities, their contribution in the attraction of human attention is not easily comparable. In this study, we investigated the importance of these three features in the attraction of human attention in synthetic and natural images. Choosing 100% percent detectable contrast in each modality, we studied the competition between different features. Psychophysics results showed that, although single features can be detected easily in all trials, when features were presented simultaneously in a stimulus, orientation always attracts subject's attention. In addition, computational results showed that orientation feature map is more informative about the pattern of human saccades in natural images. Finally, using optimization algorithms we quantified the impact of each feature map in construction of the final saliency map.

Sequence-dependent sensitivity explains the accuracy of decisions when cues are separated with a gap.

Most decisions require information gathering from a stimulus presented with different gaps. However, the neural mechanism underlying this integration is ambiguous. Recently, it has been claimed that humans can optimally integrate the information of two discrete pulses independent of the temporal gap between them. Interestingly, subjects' performance on such a task, with two discrete pulses, is superior to what a perfect accumulator can predict. Although numerous neuronal and descriptive models have been proposed to explain the mechanism of perceptual decision-making, none can explain human behavior on this two-pulse task. In order to investigate the mechanism of decision-making on the noted tasks, a set of modified drift-diffusion models based on different hypotheses were used. Model comparisons clarified that, in a sequence of information arriving at different times, the accumulated information of earlier evidence affects the process of information accumulation of later evidence. It was shown that the rate of information extraction depends on whether the pulse is the first or the second one. Moreover, our findings suggest that a drift diffusion model with a dynamic drift rate can also explain the stronger effect of the second pulse on decisions as shown by Kiani et al. (Journal of Neuroscience, 33 (42), 16483-16489, 2013).

Residual Information of Previous Decision Affects Evidence Accumulation in Current Decision.

Bias in perceptual decisions can be generally defined as an effect which is controlled by factors other than the decision-relevant information (e.g., perceptual information in a perceptual task, when trials are independent). The literature on decision-making suggests two main hypotheses to account for this kind of bias: internal bias signals are derived from (a) the residual of motor signals generated to report a decision in the past, and (b) the residual of sensory information extracted from the stimulus in the past. Beside these hypotheses, this study suggests that making a decision in the past per se may bias the next decision. We demonstrate the validity of this assumption, first, by performing behavioral experiments based on the two-alternative forced-choice (TAFC) discrimination of motion direction paradigms and, then, we modified the pure drift-diffusion model (DDM) based on the accumulation-to-bound mechanism to account for the sequential effect. In both cases, the trace of the previous trial influences the current decision. Results indicate that the probability of being correct in the current decision increases if it is in line with the previously made decision even in the presence of feedback. Moreover, a modified model that keeps the previous decision information in the starting point of evidence accumulation provides a better fit to the behavioral data. Our findings suggest that the accumulated evidence in the decision-making process after crossing the bound in the previous decision can affect the parameters of information accumulation for the current decision in consecutive trials.

The impact of the lateral geniculate nucleus and corticogeniculate interactions on efficient coding and higher-order visual object processing.

Principles of efficient coding suggest that the peripheral units of any sensory processing system are designed for efficient coding. The function of the lateral geniculate nucleus (LGN) as an early stage in the visual system is not well understood. Some findings indicate that similar to the retina that decorrelates input signals spatially, the LGN tends to perform a temporal decorrelation. There is evidence suggesting that corticogeniculate connections may account for this decorrelation in the LGN. In this study, we propose a computational model based on biological evidence reported by Wang et al. (2006), who demonstrated that the influence pattern of V1 feedback is phase-reversed. The output of our model shows how corticogeniculate connections decorrelate LGN responses and make an efficient representation. We evaluated our model using criteria that have previously been tested on LGN neurons through cell recording experiments, including sparseness, entropy, power spectra, and information transfer. We also considered the role of the LGN in higher-order visual object processing, comparing the categorization performance of human subjects with a cortical object recognition model in the presence and absence of our LGN input-stage model. Our results show that the new model that considers the role of the LGN, more closely follows the categorization performance of human subjects.

Predicting the human reaction time based on natural image statistics in a rapid categorization task.

The human visual system is developed by viewing natural scenes. In controlled experiments, natural stimuli therefore provide a realistic framework with which to study the underlying information processing steps involved in human vision. Studying the properties of natural images and their effects on the visual processing can help us to understand underlying mechanisms of visual system. In this study, we used a rapid animal vs. non-animal categorization task to assess the relationship between the reaction times of human subjects and the statistical properties of images. We demonstrated that statistical measures, such as the beta and gamma parameters of a Weibull, fitted to the edge histogram of an image, and the image entropy, are effective predictors of subject reaction times. Using these three parameters, we proposed a computational model capable of predicting the reaction times of human subjects.