Chimpanzee uses manipulative gaze cues to conceal and reveal information to foraging competitor.

Tactical deception has been widely reported in primates on a functional basis, but details of behavioral mechanisms are usually unspecified. We tested a pair of chimpanzees (Pan troglodytes) in the informed forager paradigm, in which the subordinate saw the location of hidden food and the dominant did not. We employed cross-correlations to examine temporal contingencies between chimpanzees' behavior: specifically how the direction of the subordinate's gaze and movement functioned to manipulate the dominant's searching behavior through two tactics, withholding, and misleading information. In Experiment 1, not only did the informed subordinate tend to stop walking toward a single high value food, but she also refrained from gazing toward it, thus, withholding potentially revealing cues from her searching competitor. In a second experiment, in which a moderate value food was hidden in addition to the high value food, whenever the subordinate alternated her gaze between the dominant and the moderate value food, she often paused walking for 5 s; this frequently recruited the dominant to the inferior food, functioning as a "decoy." The subordinate flexibly concealed and revealed gaze toward a goal, which suggests that not only can chimpanzees use visual cues to make predictions about behavior, but also that chimpanzees may understand that other individuals can exploit their gaze direction. These results substantiate descriptive reports of how chimpanzees use gaze to manipulate others, and to our knowledge are the first quantitative data to identify behavioral mechanisms of tactical deception. RESEARCH HIGHLIGHTS: Cross correlations show a subordinate chimpanzee tactically deceived a dominant by not gazing toward a valuable food (withholding), and recruiting to a "decoy" food (misleading). Chimpanzees understand that others can exploit their gaze direction.

Using cross correlations to investigate how chimpanzees (Pan troglodytes) use conspecific gaze cues to extract and exploit information in a foraging competition.

In a dyadic informed forager task, chimpanzees (Pan troglodytes) are known to exploit the knowledge of informed subordinates; however, the behavioral mechanisms they employ are unknown. It is tempting to interpret outcome measures, such as which individual obtained the food, in a cognitively richer way than the outcomes may justify. We employed a different approach from prior research, asking how chimpanzees compete by maneuvering around each other, whether they use gaze cues to acquire information from others, and what information they use in moment-to-moment decision-making. We used cross correlations, which plot the correlation between two variables as a function of time, systematically to examine chimpanzee interactions in a series of dyadic informed forager contests. We used cross correlations as a "proof of concept" so as to determine whether the target actions were contingent on, or occurred in a time-locked pattern relative to, the referent actions. A subordinate individual was given privileged knowledge of food location. As expected, an ignorant dominant followed the informed subordinate's movement in the enclosure. The dominant also followed the subordinate's gaze direction: after she looked at the subordinate, she was more likely to gaze toward this same direction within one second. In contrast, the subordinate only occasionally followed the dominant's movement and gaze. The dominant also changed her own direction of movement to converge on the location to which the subordinate directed her gaze and movement. Cross correlation proves an effective technique for charting contingencies in social interactions, an important step in understanding the use of cognition in natural situations.

The effect of neural adaptation on population coding accuracy.

Most neurons in the primary visual cortex initially respond vigorously when a preferred stimulus is presented, but adapt as stimulation continues. The functional consequences of adaptation are unclear. Typically a reduction of firing rate would reduce single neuron accuracy as less spikes are available for decoding, but it has been suggested that on the population level, adaptation increases coding accuracy. This question requires careful analysis as adaptation not only changes the firing rates of neurons, but also the neural variability and correlations between neurons, which affect coding accuracy as well. We calculate the coding accuracy using a computational model that implements two forms of adaptation: spike frequency adaptation and synaptic adaptation in the form of short-term synaptic plasticity. We find that the net effect of adaptation is subtle and heterogeneous. Depending on adaptation mechanism and test stimulus, adaptation can either increase or decrease coding accuracy. We discuss the neurophysiological and psychophysical implications of the findings and relate it to published experimental data.

Implied motion activation in cortical area MT can be explained by visual low-level features.

To investigate form-related activity in motion-sensitive cortical areas, we recorded cell responses to animate implied motion in macaque middle temporal (MT) and medial superior temporal (MST) cortex and investigated these areas using fMRI in humans. In the single-cell studies, we compared responses with static images of human or monkey figures walking or running left or right with responses to the same human and monkey figures standing or sitting still. We also investigated whether the view of the animate figure (facing left or right) that elicited the highest response was correlated with the preferred direction for moving random dot patterns. First, figures were presented inside the cell's receptive field. Subsequently, figures were presented at the fovea while a dynamic noise pattern was presented at the cell's receptive field location. The results show that MT neurons did not discriminate between figures on the basis of the implied motion content. Instead, response preferences for implied motion correlated with preferences for low-level visual features such as orientation and size. No correlation was found between the preferred view of figures implying motion and the preferred direction for moving random dot patterns. Similar findings were obtained in a smaller population of MST cortical neurons. Testing human MT+ responses with fMRI further corroborated the notion that low-level stimulus features might explain implied motion activation in human MT+. Together, these results suggest that prior human imaging studies demonstrating animate implied motion processing in area MT+ can be best explained by sensitivity for low-level features rather than sensitivity for the motion implied by animate figures.

Visual stimulation decorrelates neuronal activity.

The accuracy of neuronal encoding depends on the response statistics of individual neurons and the correlation of the activity between different neurons. Here, the dynamics of the neuronal response statistics in the anterior superior temporal sulcus of the macaque monkey is described. A transient reduction in the normalized trial-by-trial variability and decorrelation of the responses with both the activity of other neurons and previous activity of the same neuron are found at response onset. The variability of neuronal activity and its correlation structure return to the levels observed in the resting state 50-100 ms after response onset, except for marked increases in the signal correlation between neurons. The transient changes in the response statistics are seen even if there is little or no stimulus-elicited activity, indicating the effect is due to network properties rather than to activity changes per se. Modeling also indicates that the observed variations in response variability and correlation structure of the neuronal activity over time cannot be attributed to changes in firing rate. However, a reset of the underlying spike-generating process, possibly due to the driving input changing from recurrent to feedforward inputs, captures most of the observed changes. The nonstationarity indicated by the changes in correlation structure around response onset increases coding efficiency: compared with the mutual information calculated without regard to the transitory changes, the decorrelation increases the information conveyed by the initial response of modeled neuronal pairs by ≤ 4% and suggests that an integration time of as little as 50 ms is sufficient to extract 95% the available information during the initial response period.

Adaptive integration in the visual cortex by depressing recurrent cortical circuits.

Neurons in the visual cortex receive a large amount of input from recurrent connections, yet the functional role of these connections remains unclear. Here we explore networks with strong recurrence in a computational model and show that short-term depression of the synapses in the recurrent loops implements an adaptive filter. This allows the visual system to respond reliably to deteriorated stimuli yet quickly to high-quality stimuli. For low-contrast stimuli, the model predicts long response latencies, whereas latencies are short for high-contrast stimuli. This is consistent with physiological data showing that in higher visual areas, latencies can increase more than 100 ms at low contrast compared to high contrast. Moreover, when presented with briefly flashed stimuli, the model predicts stereotypical responses that outlast the stimulus, again consistent with physiological findings. The adaptive properties of the model suggest that the abundant recurrent connections found in visual cortex serve to adapt the network's time constant in accordance with the stimulus and normalizes neuronal signals such that processing is as fast as possible while maintaining reliability.

The sensitivity of primate STS neurons to walking sequences and to the degree of articulation in static images.

We readily use the form of human figures to determine if they are moving. Human figures that have arms and legs outstretched (articulated) appear to be moving more than figures where the arms and legs are near the body (standing). We tested whether neurons in the macaque monkey superior temporal sulcus (STS), a region known to be involved in processing social stimuli, were sensitive to the degree of articulation of a static human figure. Additionally, we tested sensitivity to the same stimuli within forward and backward walking sequences. We found that 57% of cells that responded to the static image of a human figure was also sensitive to the degree of articulation of the figure. Some cells displayed selective responses for articulated postures, while others (in equal numbers) displayed selective responses for standing postures. Cells selective for static images of articulated figures were more likely to respond to movies of walking forwards than walking backwards. Cells selective for static images of standing figures were more likely to respond to movies of walking backwards than forwards. An association between form sensitivity and walking sensitivity could be consistent with an interpretation that cell responses to articulated figures act as an implied motion signal.

Emotional information processing in major depression remission and partial remission: faces come first.

Although there is considerable knowledge of the cognitive and perceptual deficits associated with the acute phases of major depressive disorder (MDD), the processes involved in remission and relapse are still being evaluated. In the present study emotional information processing in remission was investigated. A Stroop paradigm was used to compare responses from a group of remitted or partially remitted MDD patients with a matched control group. The stimuli consisted of lexical and visual facial stimuli, with one word (positive/negative) superimposed on a face (happy/sad), presented in the same trial, and being congruent or incongruent. The task was to identify the emotional content of either the face (ignoring the word), or vice versa. The results showed that both patients and controls had the same interference patterns when the target was defined by the word, and that when the target was defined by the facial expression, reaction times were faster for both groups. However, patients showed a reduced positive bias, possibly indicating dissociation between patients and control groups in terms of attention to complex emotional information. Future studies testing the sensitivity of the Emotional Stroop test in the investigation of attention to complex emotional information is needed. Clinical implications are discussed.

Contrast induced changes in response latency depend on stimulus specificity.

Neurones in visual cortex show increasing response latency with decreasing stimulus contrast. Neurophysiological recordings from neurones in inferior temporal cortex (IT) and the superior temporal sulcus (STS), show that the increment in response latency with decreasing stimulus contrast is considerably greater in higher visual areas than that seen in primary visual cortex. This suggests that the majority of the latency change is not retinal or V1 in origin, instead each cortical processing area adds latency at low contrast. I show that, as in earlier visual areas, response latency is more strongly dependent on stimulus contrast than stimulus identity. There is large variation in the extent to which response latency increases with decreasing stimulus contrast. I show that this between cell variability is, at least in part, related to the stimulus specificity of the neurones: the increase in response latency as stimulus contrast decreases is greater for neurones that respond to few stimuli compared to neurones that respond to many stimuli.

Modelling spike trains and extracting response latency with Bayesian binning.

The peristimulus time histogram (PSTH) and the spike density function (SDF) are commonly used in the analysis of neurophysiological data. The PSTH is usually obtained by binning spike trains, the SDF being a (Gaussian) kernel smoothed version of the PSTH. While selection of the bin width or kernel size is often relatively arbitrary there have been recent attempts to remedy this situation (Shimazaki and Shinomoto, 2007c,b,a). We further develop an exact Bayesian generative model approach to estimating PSTHs (Endres et al., 2008) and demonstrate its superiority to competing methods using data from early (LGN) and late (STSa) visual areas. We also highlight the advantages of our scheme's automatic complexity control and generation of error bars. Additionally, our approach allows extraction of excitatory and inhibitory response latency from spike trains in a principled way, both on repeated and single trial data. We show that the method can be applied to data with high background firing rates and inhibitory responses (LGN) as well as to data with low firing rate and excitatory responses (STSa). Furthermore, we demonstrate on simulated data that our latency extraction method works for a range of signal-to-noise ratios and background firing rates. While further studies are needed to examine the sensitivity of our method to, for example, gradual changes in firing rate and adaptation, the current results suggest that Bayesian binning is a powerful method for the estimation of firing rate and the extraction response latency from neuronal spike trains.

Visual adaptation to goal-directed hand actions.

Prolonged exposure to visual stimuli, or adaptation, often results in an adaptation "aftereffect" which can profoundly distort our perception of subsequent visual stimuli. This technique has been commonly used to investigate mechanisms underlying our perception of simple visual stimuli, and more recently, of static faces. We tested whether humans would adapt to movies of hands grasping and placing different weight objects. After adapting to hands grasping light or heavy objects, subsequently perceived objects appeared relatively heavier, or lighter, respectively. The aftereffects increased logarithmically with adaptation action repetition and decayed logarithmically with time. Adaptation aftereffects also indicated that perception of actions relies predominantly on view-dependent mechanisms. Adapting to one action significantly influenced the perception of the opposite action. These aftereffects can only be explained by adaptation of mechanisms that take into account the presence/absence of the object in the hand. We tested if evidence on action processing mechanisms obtained using visual adaptation techniques confirms underlying neural processing. We recorded monkey superior temporal sulcus (STS) single-cell responses to hand actions. Cells sensitive to grasping or placing typically responded well to the opposite action; cells also responded during different phases of the actions. Cell responses were sensitive to the view of the action and were dependent upon the presence of the object in the scene. We show here that action processing mechanisms established using visual adaptation parallel the neural mechanisms revealed during recording from monkey STS. Visual adaptation techniques can thus be usefully employed to investigate brain mechanisms underlying action perception.

Seeing the future: Natural image sequences produce "anticipatory" neuronal activity and bias perceptual report.

This paper relates human perception to the functioning of cells in the temporal cortex that are engaged in high-level pattern processing. We review historical developments concerning (a) the functional organization of cells processing faces and (b) the selectivity for faces in cell responses. We then focus on (c) the comparison of perception and cell responses to images of faces presented in sequences of unrelated images. Specifically the paper concerns the cell function and perception in circumstances where meaningful patterns occur momentarily in the context of a naturally or unnaturally changing visual environment. Experience of visual sequences allows anticipation, yet one sensory stimulus also "masks" perception and neural processing of subsequent stimuli. To understand this paradox we compared cell responses in monkey temporal cortex to body images presented individually, in pairs and in action sequences. Responses to one image suppressed responses to similar images for approximately 500 ms. This suppression led to responses peaking 100 ms earlier to image sequences than to isolated images (e.g., during head rotation, face-selective activity peaks before the face confronts the observer). Thus forward masking has unrecognized benefits for perception because it can transform neuronal activity to make it predictive during natural change.

Integrating neuronal coding into cognitive models: predicting reaction time distributions.

Neurophysiological studies have examined many aspects of neuronal activity in terms of neuronal codes and postulated roles for these codes in brain processing. There has been relatively little work, however, examining the relationship between different neuronal codes and the behavioural phenomena associated with cognitive processes. Here, predictions about reaction time distributions derived from an accumulator model incorporating known neurophysiological data in temporal lobe visual areas of the macaque are examined. Results from human experimental studies examining the effects of changing stimulus orientation, size and contrast are consistent with the model, including qualitatively different changes in reaction time distributions with different stimulus manipulations. The different changes in reaction time distributions depend on whether the image manipulation changes neuronal response latency or magnitude and can be related to parallel or serial cognitive processes respectively. The results indicate that neuronal coding can be productively incorporated into computational models to provide mechanistic accounts of behavioural results related to cognitive phenomena.

Integration of visual and auditory information by superior temporal sulcus neurons responsive to the sight of actions.

Processing of complex visual stimuli comprising facial movements, hand actions, and body movements is known to occur in the superior temporal sulcus (STS) of humans and nonhuman primates. The STS is also thought to play a role in the integration of multimodal sensory input. We investigated whether STS neurons coding the sight of actions also integrated the sound of those actions. For 23% of neurons responsive to the sight of an action, the sound of that action significantly modulated the visual response. The sound of the action increased or decreased the visually evoked response for an equal number of neurons. In the neurons whose visual response was increased by the addition of sound (but not those neurons whose responses were decreased), the audiovisual integration was dependent upon the sound of the action matching the sight of the action. These results suggest that neurons in the STS form multisensory representations of observed actions.