Emergence of brain-like mirror-symmetric viewpoint tuning in convolutional neural networks.

Primates can recognize objects despite 3D geometric variations such as in-depth rotations. The computational mechanisms that give rise to such invariances are yet to be fully understood. A curious case of partial invariance occurs in the macaque face-patch AL and in fully connected layers of deep convolutional networks in which neurons respond similarly to mirror-symmetric views (e.g. left and right profiles). Why does this tuning develop? Here, we propose a simple learning-driven explanation for mirror-symmetric viewpoint tuning. We show that mirror-symmetric viewpoint tuning for faces emerges in the fully connected layers of convolutional deep neural networks trained on object recognition tasks, even when the training dataset does not include faces. First, using 3D objects rendered from multiple views as test stimuli, we demonstrate that mirror-symmetric viewpoint tuning in convolutional neural network models is not unique to faces: it emerges for multiple object categories with bilateral symmetry. Second, we show why this invariance emerges in the models. Learning to discriminate among bilaterally symmetric object categories induces reflection-equivariant intermediate representations. AL-like mirror-symmetric tuning is achieved when such equivariant responses are spatially pooled by downstream units with sufficiently large receptive fields. These results explain how mirror-symmetric viewpoint tuning can emerge in neural networks, providing a theory of how they might emerge in the primate brain. Our theory predicts that mirror-symmetric viewpoint tuning can emerge as a consequence of exposure to bilaterally symmetric objects beyond the category of faces, and that it can generalize beyond previously experienced object categories.

Emergence of brain-like mirror-symmetric viewpoint tuning in convolutional neural networks.

Primates can recognize objects despite 3D geometric variations such as in-depth rotations. The computational mechanisms that give rise to such invariances are yet to be fully understood. A curious case of partial invariance occurs in the macaque face-patch AL and in fully connected layers of deep convolutional networks in which neurons respond similarly to mirror-symmetric views (e.g., left and right profiles). Why does this tuning develop? Here, we propose a simple learning-driven explanation for mirror-symmetric viewpoint tuning. We show that mirror-symmetric viewpoint tuning for faces emerges in the fully connected layers of convolutional deep neural networks trained on object recognition tasks, even when the training dataset does not include faces. First, using 3D objects rendered from multiple views as test stimuli, we demonstrate that mirror-symmetric viewpoint tuning in convolutional neural network models is not unique to faces: it emerges for multiple object categories with bilateral symmetry. Second, we show why this invariance emerges in the models. Learning to discriminate among bilaterally symmetric object categories induces reflection-equivariant intermediate representations. AL-like mirror-symmetric tuning is achieved when such equivariant responses are spatially pooled by downstream units with sufficiently large receptive fields. These results explain how mirror-symmetric viewpoint tuning can emerge in neural networks, providing a theory of how they might emerge in the primate brain. Our theory predicts that mirror-symmetric viewpoint tuning can emerge as a consequence of exposure to bilaterally symmetric objects beyond the category of faces, and that it can generalize beyond previously experienced object categories.

A causal relationship between face-patch activity and face-detection behavior.

The primate brain contains distinct areas densely populated by face-selective neurons. One of these, face-patch ML, contains neurons selective for contrast relationships between face parts. Such contrast-relationships can serve as powerful heuristics for face detection. However, it is unknown whether neurons with such selectivity actually support face-detection behavior. Here, we devised a naturalistic face-detection task and combined it with fMRI-guided pharmacological inactivation of ML to test whether ML is of critical importance for real-world face detection. We found that inactivation of ML impairs face detection. The effect was anatomically specific, as inactivation of areas outside ML did not affect face detection, and it was categorically specific, as inactivation of ML impaired face detection while sparing body and object detection. These results establish that ML function is crucial for detection of faces in natural scenes, performing a critical first step on which other face processing operations can build.

Face Patch Resting State Networks Link Face Processing to Social Cognition.

Faces transmit a wealth of social information. How this information is exchanged between face-processing centers and brain areas supporting social cognition remains largely unclear. Here we identify these routes using resting state functional magnetic resonance imaging in macaque monkeys. We find that face areas functionally connect to specific regions within frontal, temporal, and parietal cortices, as well as subcortical structures supporting emotive, mnemonic, and cognitive functions. This establishes the existence of an extended face-recognition system in the macaque. Furthermore, the face patch resting state networks and the default mode network in monkeys show a pattern of overlap akin to that between the social brain and the default mode network in humans: this overlap specifically includes the posterior superior temporal sulcus, medial parietal, and dorsomedial prefrontal cortex, areas supporting high-level social cognition in humans. Together, these results reveal the embedding of face areas into larger brain networks and suggest that the resting state networks of the face patch system offer a new, easily accessible venue into the functional organization of the social brain and into the evolution of possibly uniquely human social skills.

Dynamic representation of the temporal and sequential structure of rhythmic movements in the primate medial premotor cortex.

We determined the encoding properties of single cells and the decoding accuracy of cell populations in the medial premotor cortex (MPC) of Rhesus monkeys to represent in a time-varying fashion the duration and serial order of six intervals produced rhythmically during a synchronization-continuation tapping task. We found that MPC represented the temporal and sequential structure of rhythmic movements by activating small ensembles of neurons that encoded the duration or the serial order in rapid succession, so that the pattern of active neurons changed dramatically within each interval. Interestingly, the width of the encoding or decoding function for serial order increased as a function of duration. Finally, we found that the strength of correlation in spontaneous activity of the individual cells varied as a function of the timing of their recruitment. These results demonstrate the existence of dynamic representations in MPC for the duration and serial order of intervals produced rhythmically and suggest that this dynamic code depends on ensembles of interconnected neurons that provide a strong synaptic drive to the next ensemble in a consecutive chain of neural events.

The shift from low to high non-structural protein 1 expression in rotavirus-infected MA-104 cells.

A hallmark of group/species A rotavirus (RVA) replication in MA-104 cells is the logarithmic increase in viral mRNAs that occurs four-12 h post-infection. Viral protein synthesis typically lags closely behind mRNA synthesis but continues after mRNA levels plateau. However, RVA non-structural protein 1 (NSP1) is present at very low levels throughout viral replication despite showing robust protein synthesis. NSP1 has the contrasting properties of being susceptible to proteasomal degradation, but being stabilised against proteasomal degradation by viral proteins and/or viral mRNAs. We aimed to determine the kinetics of the accumulation and intracellular distribution of NSP1 in MA-104 cells infected with rhesus rotavirus (RRV). NSP1 preferentially localises to the perinuclear region of the cytoplasm of infected cells, forming abundant granules that are heterogeneous in size. Late in infection, large NSP1 granules predominate, coincident with a shift from low to high NSP1 expression levels. Our results indicate that rotavirus NSP1 is a late viral protein in MA-104 cells infected with RRV, presumably as a result of altered protein turnover.

The shift from low to high non-structural protein 1 expression in rotavirus-infected MA-104 cells

A hallmark of group/species A rotavirus (RVA) replication in MA-104 cells is the logarithmic increase in viral mRNAs that occurs four-12 h post-infection. Viral protein synthesis typically lags closely behind mRNA synthesis but continues after mRNA levels plateau. However, RVA non-structural protein 1 (NSP1) is present at very low levels throughout viral replication despite showing robust protein synthesis. NSP1 has the contrasting properties of being susceptible to proteasomal degradation, but being stabilised against proteasomal degradation by viral proteins and/or viral mRNAs. We aimed to determine the kinetics of the accumulation and intracellular distribution of NSP1 in MA-104 cells infected with rhesus rotavirus (RRV). NSP1 preferentially localises to the perinuclear region of the cytoplasm of infected cells, forming abundant granules that are heterogeneous in size. Late in infection, large NSP1 granules predominate, coincident with a shift from low to high NSP1 expression levels. Our results indicate that rotavirus NSP1 is a late viral protein in MA-104 cells infected with RRV, presumably as a result of altered protein turnover.

Interval tuning in the primate medial premotor cortex as a general timing mechanism.

The precise quantification of time during motor performance is critical for many complex behaviors, including musical execution, speech articulation, and sports; however, its neural mechanisms are primarily unknown. We found that neurons in the medial premotor cortex (MPC) of behaving monkeys are tuned to the duration of produced intervals during rhythmic tapping tasks. Interval-tuned neurons showed similar preferred intervals across tapping behaviors that varied in the number of produced intervals and the modality used to drive temporal processing. In addition, we found that the same population of neurons is able to multiplex the ordinal structure of a sequence of rhythmic movements and a wide range of durations in the range of hundreds of milliseconds. Our results also revealed a possible gain mechanism for encoding the total number of intervals in a sequence of temporalized movements, where interval-tuned cells show a multiplicative effect of their activity for longer sequences of intervals. These data suggest that MPC is part of a core timing network that uses interval tuning as a signal to represent temporal processing in a variety of behavioral contexts where time is explicitly quantified.

Measuring time with different neural chronometers during a synchronization-continuation task.

Temporal information processing is critical for many complex behaviors including speech and music cognition, yet its neural substrate remains elusive. We examined the neurophysiological properties of medial premotor cortex (MPC) of two Rhesus monkeys during the execution of a synchronization-continuation tapping task that includes the basic sensorimotor components of a variety of rhythmic behaviors. We show that time-keeping in the MPC is governed by separate cell populations. One group encoded the time remaining for an action, showing activity whose duration changed as a function of interval duration, reaching a peak at similar magnitudes and times with respect to the movement. The other cell group showed a response that increased in duration or magnitude as a function of the elapsed time from the last movement. Hence, the sensorimotor loops engaged during the task may depend on the cyclic interplay between different neuronal chronometers that quantify the time passed and the remaining time for an action.

Subsecond timing in primates: comparison of interval production between human subjects and rhesus monkeys.

This study describes the psychometric similarities and differences in motor timing performance between 20 human subjects and three rhesus monkeys during two timing production tasks. These tasks involved tapping on a push-button to produce the same set of intervals (range of 450 to 1,000 ms), but they differed in the number of intervals produced (single vs. multiple) and the modality of the stimuli (auditory vs. visual) used to define the time intervals. The data showed that for both primate species, variability increased as a function of the length of the produced target interval across tasks, a result in accordance with the scalar property. Interestingly, the temporal performance of rhesus monkeys was equivalent to that of human subjects during both the production of single intervals and the tapping synchronization to a metronome. Overall, however, human subjects were more accurate than monkeys and showed less timing variability. This was especially true during the self-pacing phase of the multiple interval production task, a behavior that may be related to complex temporal cognition, such as speech and music execution. In addition, the well-known human bias toward auditory as opposed to visual cues for the accurate execution of time intervals was not evident in rhesus monkeys. These findings validate the rhesus monkey as an appropriate model for the study of the neural basis of time production, but also suggest that the exquisite temporal abilities of humans, which peak in speech and music performance, are not all shared with macaques.

Behavioral and neurophysiological aspects of target interception.

This chapter focuses on the behavioral and neurophysiological aspects of manual interception. We review the most important elements of an interceptive action from the sensory and cognitive stage to the motor side of this behavior. We describe different spatial and temporal target parameters that can be used to control the interception movement, as well as the different strategies used by the subject to intercept a moving target. We review the neurophysiological properties of the parietofrontal system during target motion processing and during a particular experiment of target interception. Finally, we describe the neural responses associated with the temporal and spatial parameters of a moving target and the possible neurophysiological mechanisms used to integrate this information in order to trigger an interception movement.

The context of temporal processing is represented in the multidimensional relationships between timing tasks.

In the present study we determined the performance interrelations of ten different tasks that involved the processing of temporal intervals in the subsecond range, using multidimensional analyses. Twenty human subjects executed the following explicit timing tasks: interval categorization and discrimination (perceptual tasks), and single and multiple interval tapping (production tasks). In addition, the subjects performed a continuous circle-drawing task that has been considered an implicit timing paradigm, since time is an emergent property of the produced spatial trajectory. All tasks could be also classified as single or multiple interval paradigms. Auditory or visual markers were used to define the intervals. Performance variability, a measure that reflects the temporal and non-temporal processes for each task, was used to construct a dissimilarity matrix that quantifies the distances between pairs of tasks. Hierarchical clustering and multidimensional scaling were carried out on the dissimilarity matrix, and the results showed a prominent segregation of explicit and implicit timing tasks, and a clear grouping between single and multiple interval paradigms. In contrast, other variables such as the marker modality were not as crucial to explain the performance between tasks. Thus, using this methodology we revealed a probable functional arrangement of neural systems engaged during different timing behaviors.

Do we have a common mechanism for measuring time in the hundreds of millisecond range? Evidence from multiple-interval timing tasks.

In the present study we examined the performance variability of a group of 13 subjects in eight different tasks that involved the processing of temporal intervals in the subsecond range. These tasks differed in their sensorimotor processing (S; perception vs. production), the modality of the stimuli used to define the intervals (M; auditory vs. visual), and the number of intervals (N; one or four). Different analytical techniques were used to determine the existence of a central or distributed timing mechanism across tasks. The results showed a linear increase in performance variability as a function of the interval duration in all tasks. However, this compliance of the scalar property of interval timing was accompanied by a strong effect of S, N, and M and the interaction between these variables on the subjects' temporal accuracy. Thus the performance variability was larger not only in perceptual tasks than that in motor-timing tasks, but also using visual rather than auditory stimuli, and decreased as a function of the number of intervals. These results suggest the existence of a partially overlapping distributed mechanism underlying the ability to quantify time in different contexts.