Gaze and Arrows: Does the Gaze-Following Patch in the Posterior Temporal Cortex Differentiate Social and Symbolic Spatial Cues?

The gaze-following patch (GFP) is located in the posterior temporal cortex and has been described as a cortical module dedicated to processing other people's gaze-direction in a domain-specific manner. Thus, it appears to be the neural correlate of Baron-Cohen's eye direction detector (EDD) which is one of the core modules in his mindreading system-a neurocognitive model for the theory of mind concept. Inspired by Jerry Fodor's ideas on the modularity of the mind, Baron-Cohen proposed that, among other things, the individual modules are domain specific. In the case of the EDD, this means that it exclusively processes eye-like stimuli to extract gaze-direction and that other stimuli, which may carry directional information as well, are processed elsewhere. If the GFP is indeed EDD's neural correlate, it must meet this expectation. To test this, we compared the GFP's BOLD activity during gaze-direction following with the activity during arrow-direction following in the present human fMRI study. Contrary to the expectation based on the assumption of domain specificity, we did not find a differentiation between gaze- and arrow-direction following. In fact, we were not able to reproduce the GFP as presented in the previous studies. A possible explanation is that in the present study-unlike the previous work-the gaze stimuli did not contain an obvious change of direction that represented a visual motion. Hence, the critical stimulus component responsible for the identification of the GFP in the previous experiments might have been visual motion.

A prosocial function of head-gaze aversion and head-cocking in common marmosets.

Gaze aversion is a behavior adopted by several mammalian and non-mammalian species in response to eye contact, and is usually interpreted as a reaction to a perceived threat. Unlike many other primate species, common marmosets (Callithrix jacchus) are thought to have a high tolerance for direct gaze, barely exhibiting gaze avoidance towards conspecifics and humans. Here we show that this does not hold for marmosets interacting with a familiar experimenter who suddenly establishes eye contact in a playful interaction (peekaboo). Video footage synchronously recorded from the perspective of the marmoset and the experimenter showed that the monkeys consistently alternated between eye contact and head-gaze aversion, and that these responses were often preceded by head-cocking. We hypothesize that this behavioral strategy helps marmosets to temporarily disengage from emotionally overwhelming social stimulation due to sight of another individual's face, in order to prepare for a new round of affiliative face-to-face interactions.

Cerebellar complex spikes multiplex complementary behavioral information.

Purkinje cell (PC) discharge, the only output of cerebellar cortex, involves 2 types of action potentials, high-frequency simple spikes (SSs) and low-frequency complex spikes (CSs). While there is consensus that SSs convey information needed to optimize movement kinematics, the function of CSs, determined by the PC's climbing fiber input, remains controversial. While initially thought to be specialized in reporting information on motor error for the subsequent amendment of behavior, CSs seem to contribute to other aspects of motor behavior as well. When faced with the bewildering diversity of findings and views unraveled by highly specific tasks, one may wonder if there is just one true function with all the other attributions wrong? Or is the diversity of findings a reflection of distinct pools of PCs, each processing specific streams of information conveyed by climbing fibers? With these questions in mind, we recorded CSs from the monkey oculomotor vermis deploying a repetitive saccade task that entailed sizable motor errors as well as small amplitude saccades, correcting them. We demonstrate that, in addition to carrying error-related information, CSs carry information on the metrics of both primary and small corrective saccades in a time-specific manner, with changes in CS firing probability coupled with changes in CS duration. Furthermore, we also found CS activity that seemed to predict the upcoming events. Hence PCs receive a multiplexed climbing fiber input that merges complementary streams of information on the behavior, separable by the recipient PC because they are staggered in time.

Shape-invariant encoding of dynamic primate facial expressions in human perception.

Dynamic facial expressions are crucial for communication in primates. Due to the difficulty to control shape and dynamics of facial expressions across species, it is unknown how species-specific facial expressions are perceptually encoded and interact with the representation of facial shape. While popular neural network models predict a joint encoding of facial shape and dynamics, the neuromuscular control of faces evolved more slowly than facial shape, suggesting a separate encoding. To investigate these alternative hypotheses, we developed photo-realistic human and monkey heads that were animated with motion capture data from monkeys and humans. Exact control of expression dynamics was accomplished by a Bayesian machine-learning technique. Consistent with our hypothesis, we found that human observers learned cross-species expressions very quickly, where face dynamics was represented largely independently of facial shape. This result supports the co-evolution of the visual processing and motor control of facial expressions, while it challenges appearance-based neural network theories of dynamic expression recognition.

A Naturalistic Dynamic Monkey Head Avatar Elicits Species-Typical Reactions and Overcomes the Uncanny Valley.

Research on social perception in monkeys may benefit from standardized, controllable, and ethologically valid renditions of conspecifics offered by monkey avatars. However, previous work has cautioned that monkeys, like humans, show an adverse reaction toward realistic synthetic stimuli, known as the "uncanny valley" effect. We developed an improved naturalistic rhesus monkey face avatar capable of producing facial expressions (fear grin, lip smack and threat), animated by motion capture data of real monkeys. For validation, we additionally created decreasingly naturalistic avatar variants. Eight rhesus macaques were tested on the various videos and avoided looking at less naturalistic avatar variants, but not at the most naturalistic or the most unnaturalistic avatar, indicating an uncanny valley effect for the less naturalistic avatar versions. The avoidance was deepened by motion and accompanied by physiological arousal. Only the most naturalistic avatar evoked facial expressions comparable to those toward the real monkey videos. Hence, our findings demonstrate that the uncanny valley reaction in monkeys can be overcome by a highly naturalistic avatar.

Reflexive gaze following in common marmoset monkeys.

The ability to extract the direction of the other's gaze allows us to shift our attention to an object of interest to the other and to establish joint attention. By mapping one's own intentions on the object of joint attention, humans develop a Theory of (the other's) Mind (TOM), a functional sequence possibly disrupted in autism. Gaze following of both humans and old world monkeys is orchestrated by very similar cortical architectures, strongly suggesting homology. Also new world monkeys, a primate suborder that split from the old world monkey line about 35 million years ago, have complex social structures and one member of this group, the common marmosets (Callithrix jacchus) are known to follow human head-gaze. However, the question is if they use gaze following to establish joint attention with conspecifics. Here we show that this is indeed the case. In a free choice task, head-restrained marmosets prefer objects gazed at by a conspecific and, moreover, they exhibit considerably shorter choice reaction times for the same objects. These findings support the assumption of an evolutionarily old domain specific faculty shared within the primate order and they underline the potential value of marmosets in studies of normal and disturbed joint attention.

Learning from the past: A reverberation of past errors in the cerebellar climbing fiber signal.

The cerebellum allows us to rapidly adjust motor behavior to the needs of the situation. It is commonly assumed that cerebellum-based motor learning is guided by the difference between the desired and the actual behavior, i.e., by error information. Not only immediate but also future behavior will benefit from an error because it induces lasting changes of parallel fiber synapses on Purkinje cells (PCs), whose output mediates the behavioral adjustments. Olivary climbing fibers, likewise connecting with PCs, are thought to transport information on instant errors needed for the synaptic modification yet not to contribute to error memory. Here, we report work on monkeys tested in a saccadic learning paradigm that challenges this concept. We demonstrate not only a clear complex spikes (CS) signature of the error at the time of its occurrence but also a reverberation of this signature much later, before a new manifestation of the behavior, suitable to improve it.

Short-term adaptation of saccades does not affect smooth pursuit eye movement initiation.

Scrutiny of the visual environment requires saccades that shift gaze to objects of interest. In case the object should be moving, smooth pursuit eye movements (SPEM) try to keep the image of the object within the confines of the fovea in order to ensure sufficient time for its analysis. Both saccades and SPEM can be adaptively changed by the experience of insufficiencies, compromising the precision of saccades or the minimization of object image slip in the case of SPEM. As both forms of adaptation rely on the cerebellar oculomotor vermis (OMV), most probably deploying a shared neuronal machinery, one might expect that the adaptation of one type of eye movement should affect the kinematics of the other. In order to test this expectation, we subjected two monkeys to a standard saccadic adaption paradigm with SPEM test trials at the end and, alternatively, the same two monkeys plus a third one to a random saccadic adaptation paradigm with interleaved trials of SPEM. In contrast to our expectation, we observed at best marginal transfer which, moreover, had little consistency across experiments and subjects. The lack of consistent transfer of saccadic adaptation decisively constrains models of the implementation of oculomotor learning in the OMV, suggesting an extensive separation of saccade- and SPEM-related synapses on P-cell dendritic trees.

Following Eye Gaze Activates a Patch in the Posterior Temporal Cortex That Is not Part of the Human "Face Patch" System.

Humans follow another person's eye gaze to objects of interest to the other, thereby establishing joint attention, a first step toward developing a theory of the other's mind. Previous functional MRI studies agree that a "gaze-following patch" (GFP) of cortex close to the posterior superior temporal sulcus (STS) is specifically implicated in eye gaze-following. The location of the GFP is in the vicinity of the posterior members of the core face-processing system that consists of distinct patches in ventral visual cortex, the STS, and frontal cortex, also involved in processing information on the eyes. To test whether the GFP might correspond to one of the posterior face patches, we compared the pattern of blood oxygenation level-dependent (BOLD) imaging contrasts reflecting the passive vision of static faces with the one evoked by shifts of attention guided by the eye gaze of others. The viewing of static faces revealed the face patch system. On the other hand, eye gaze-following activated a cortical patch (the GFP) with its activation maximum separated by more than 24 mm in the right and 19 mm in the left hemisphere from the nearest face patch, the STS face area (FA). This segregation supports a distinct function of the GFP, different from the elementary processing of facial information.

The same oculomotor vermal Purkinje cells encode the different kinematics of saccades and of smooth pursuit eye movements.

Saccades and smooth pursuit eye movements (SPEM) are two types of goal-directed eye movements whose kinematics differ profoundly, a fact that may have contributed to the notion that the underlying cerebellar substrates are separated. However, it is suggested that some Purkinje cells (PCs) in the oculomotor vermis (OMV) of monkey cerebellum may be involved in both saccades and SPEM, a puzzling finding in view of the different kinematic demands of the two types of eye movements. Such 'dual' OMV PCs might be oddities with little if any functional relevance. On the other hand, they might be representatives of a generic mechanism serving as common ground for saccades and SPEM. In our present study, we found that both saccade- and SPEM-related responses of individual PCs could be predicted well by linear combinations of eye acceleration, velocity and position. The relative weights of the contributions that these three kinematic parameters made depended on the type of eye movement. Whereas in the case of saccades eye position was the most important independent variable, it was velocity in the case of SPEM. This dissociation is in accordance with standard models of saccades and SPEM control which emphasize eye position and velocity respectively as the relevant controlled state variables.

Individual neurons in the caudal fastigial oculomotor region convey information on both macro- and microsaccades.

Recent studies have suggested that microsaccades, the small amplitude saccades made during fixation, are precisely controlled. Two lines of evidence suggest that the cerebellum plays a key role not only in improving the accuracy of macrosaccades but also of microsaccades. First, lesions of the fastigial oculomotor regions (FOR) cause horizontal dysmetria of both micro- and macrosaccades. Secondly, our previous work on Purkinje cell simple spikes in the oculomotor vermis (OV) has established qualitatively similar response preferences for these two groups of saccades. In this work, we investigated the control signals for micro- and macrosaccades in the FOR, the target of OV Purkinje cell axons. We found that the same FOR neurons discharged for micro- and macrosaccades. For both groups of saccades, FOR neurons exhibited very similar dependencies of their discharge strength on direction and amplitude and very similar burst onset time differences for ipsi- and contraversive saccades and, in both, response duration reflected saccade duration, at least at the population level. An intriguing characteristic of microsaccade-related responses is that immediate pre-saccadic firing rates decreased with distance to the target center, a pattern that strikingly parallels the eye position dependency of both microsaccade metrics and frequency, which may suggest a potential neural mechanism underlying the role of FOR in fixation. Irrespective of this specific consideration, our study supports the view that microsaccades and macrosaccades share the same cerebellar circuitry and, in general, further strengthens the notion of a microsaccade-macrosaccade continuum.

Monkeys head-gaze following is fast, precise and not fully suppressible.

Human eye-gaze is a powerful stimulus, drawing the observer's attention to places and objects of interest to someone else ('eye-gaze following'). The largely homogeneous eyes of monkeys, compromising the assessment of eye-gaze by conspecifics from larger distances, explain the absence of comparable eye-gaze following in these animals. Yet, monkeys are able to use peer head orientation to shift attention ('head-gaze following'). How similar are monkeys' head-gaze and human eye-gaze following? To address this question, we trained rhesus monkeys to make saccades to targets, either identified by the head-gaze of demonstrator monkeys or, alternatively, identified by learned associations between the demonstrators' facial identities and the targets (gaze versus identity following). In a variant of this task that occurred at random, the instruction to follow head-gaze or identity was replaced in the course of a trial by the new rule to detect a change of luminance of one of the saccade targets. Although this change-of-rule rendered the demonstrator portraits irrelevant, they nevertheless influenced performance, reflecting a precise redistribution of spatial attention. The specific features depended on whether the initial rule was head-gaze or identity following: head-gaze caused an insuppressible shift of attention to the target gazed at by the demonstrator, whereas identity matching prompted much later shifts of attention, however, only if the initial rule had been identity following. Furthermore, shifts of attention prompted by head-gaze were spatially precise. Automaticity and swiftness, spatial precision and limited executive control characterizing monkeys' head-gaze following are key features of human eye-gaze following. This similarity supports the notion that both may rely on the same conserved neural circuitry.

Assessing the precision of gaze following using a stereoscopic 3D virtual reality setting.

Despite the ecological importance of gaze following, little is known about the underlying neuronal processes, which allow us to extract gaze direction from the geometric features of the eye and head of a conspecific. In order to understand the neuronal mechanisms underlying this ability, a careful description of the capacity and the limitations of gaze following at the behavioral level is needed. Previous studies of gaze following, which relied on naturalistic settings have the disadvantage of allowing only very limited control of potentially relevant visual features guiding gaze following, such as the contrast of iris and sclera, the shape of the eyelids and--in the case of photographs--they lack depth. Hence, in order to get full control of potentially relevant features we decided to study gaze following of human observers guided by the gaze of a human avatar seen stereoscopically. To this end we established a stereoscopic 3D virtual reality setup, in which we tested human subjects' abilities to detect at which target a human avatar was looking at. Following the gaze of the avatar showed all the features of the gaze following of a natural person, namely a substantial degree of precision associated with a consistent pattern of systematic deviations from the target. Poor stereo vision affected performance surprisingly little (only in certain experimental conditions). Only gaze following guided by targets at larger downward eccentricities exhibited a differential effect of the presence or absence of accompanying movements of the avatar's eyelids and eyebrows.

Microsaccade control signals in the cerebellum.

Microsaccades, the small saccades made when we try to keep the eyes still, were once believed to be inconsequential for vision, but recent studies suggest that they can precisely relocate gaze to tiny visual targets. Because the cerebellum is necessary for motor precision, we investigated whether microsaccades may exploit this neural machinery in monkeys. Almost all vermal Purkinje cells, which provide the eye-related output of the cerebellar cortex, were found to increase or decrease their simple spike firing rate during microsaccades. At both the single-cell and population level, microsaccade-related activity was highly similar to macrosaccade-related activity and we observed a continuous representation of saccade amplitude that spanned both the macrosaccade and microsaccade domains. Our results suggest that the cerebellum's role in fine-tuning eye movements extends even to the oculomotor system's smallest saccades and add to a growing list of observations that call into question the classical categorical distinction between microsaccades and macrosaccades.

Disparate substrates for head gaze following and face perception in the monkey superior temporal sulcus.

Primates use gaze cues to follow peer gaze to an object of joint attention. Gaze following of monkeys is largely determined by head or face orientation. We used fMRI in rhesus monkeys to identify brain regions underlying head gaze following and to assess their relationship to the 'face patch' system, the latter being the likely source of information on face orientation. We trained monkeys to locate targets by either following head gaze or using a learned association of face identity with the same targets. Head gaze following activated a distinct region in the posterior STS, close to-albeit not overlapping with-the medial face patch delineated by passive viewing of faces. This 'gaze following patch' may be the substrate of the geometrical calculations needed to translate information on head orientation from the face patches into precise shifts of attention, taking the spatial relationship of the two interacting agents into account.

A vermal Purkinje cell simple spike population response encodes the changes in eye movement kinematics due to smooth pursuit adaptation.

Smooth pursuit adaptation (SPA) is an example of cerebellum-dependent motor learning that depends on the integrity of the oculomotor vermis (OMV). In an attempt to unveil the neuronal basis of the role of the OMV in SPA, we recorded Purkinje cell simple spikes (PC SS) of trained monkeys. Individual PC SS exhibited specific changes of their discharge patterns during the course of SPA. However, these individual changes did not provide a reliable explanation of the behavioral changes. On the other hand, the population response of PC SS perfectly reflected the changes resulting from adaptation. Population vector was calculated using all cells recorded independent of their location. A population code conveying the behavioral changes is in full accordance with the anatomical convergence of PC axons on target neurons in the cerebellar nuclei. Its computational advantage is the ease with which it can be adjusted to the needs of the behavior by changing the contribution of individual PC SS based on error feedback.

Encoding of smooth-pursuit eye movement initiation by a population of vermal Purkinje cells.

Lesion studies suggest that the oculomotor vermis (OMV) is critical for the initiation of smooth-pursuit eye movements (SPEMs); yet, its specific role has remained elusive. In this study, we tested the hypothesis that vermal Purkinje cells (PCs) may be needed to fine-tune the kinematic description of SPEM initiation. Recording from identified PCs from the monkey OMV, we observed that SPEM-related PCs were characterized by a formidable diversity of response profiles with typically only modest reflection of eye movement kinematics. In contrast, the PC population discharge could be perfectly predicted based on a linear combination of eye acceleration, velocity, and position. This finding is in full accord with a role of the OMV in shaping eye movement kinematics. It, moreover, supports the notion that this shaping action is based on a population code, whose anatomic basis is the convergence of PCs on target neurons in the cerebellar nuclei.

Pontine reference frames for the sensory guidance of movement.

The pontine nuclei (PN) are the major intermediary elements in the corticopontocerebellar pathway. Here we asked if the PN may help to adapt the spatial reference frames used by cerebrocortical neurons involved in the sensory guidance of movement to a format potentially more appropriate for the cerebellum. To this end, we studied movement-related neurons in the dorsal PN (DPN) of monkeys, most probably projecting to the cerebellum, executing fixed vector saccades or, alternatively, fixed vector hand reaches from different starting positions. The 83 task-related neurons considered fired movement-related bursts before saccades (saccade-related) or before hand movements (hand movement-related). About 40% of the SR neurons were "oculocentric," whereas the others were modulated by eye starting position. A third of the HMR neurons encoded hand reaches in hand-centered coordinates, whereas the remainder exhibited different types of dependencies on starting positions, reminiscent in general of cortical responses. All in all, pontine reference frames for the sensory guidance of movement seem to be very similar to those in cortex. Specifically, the frequency of orbital position gain fields of SR neurons is identical in the DPN and in one of their major cortical inputs, lateral intraparietal area (LIP).

Cortical processing of head- and eye-gaze cues guiding joint social attention.

Previous fMRI experiments showed an involvement of the STS in the processing of eye-gaze direction in joint attention. Since head-gaze direction can also be used for the assessment of another person's attentional focus, we compared the mechanisms underlying the processing of head- and eye-gaze direction using a combined psychophysical and fMRI approach. Subjects actively followed the head- or eye-gaze direction of a person in a photograph towards one of seven possible targets by moving their eyes. We showed that the right posterior superior temporal sulcus (STS) as well as the right fusiform gyrus (FSG) were involved in both processing of head- as well as eye-gaze direction. Another finding was a bilateral deactivation of a distinct area in the middle STS (mSTS) as well as the left anterior STS (aSTS), that was stronger when subjects followed eye-gaze direction than when they followed head-gaze direction. We assume that this deactivation is based on an active suppression of information arising from the distracting other directional cue, i.e. head-gaze direction in the eye-gaze direction task and eye-gaze direction in the head-gaze direction task. These results further support the hypothesis that the human equivalent of the gaze sensitive area in monkeys lies in more anterior parts of the STS than previously thought.