Neuronal Mechanisms of Visual Categorization: An Abstract View on Decision Making.

Categorization is our ability to flexibly assign sensory stimuli into discrete, behaviorally relevant groupings. Categorical decisions can be used to study decision making more generally by dissociating category identity of stimuli from the actions subjects use to signal their decisions. Here we discuss the evidence for such abstract categorical encoding in the primate brain and consider the relationship with other perceptual decision paradigms. Recent work on visual categorization has examined neuronal activity across a hierarchically organized network of cortical areas in monkeys trained to group visual stimuli into arbitrary categories. This has revealed a transformation of visual-feature encoding in early visual cortical areas into more flexible categorical representations in downstream parietal and prefrontal areas. These neuronal category representations are encoded as abstract internal cognitive states because they are not rigidly linked with either specific sensory stimuli or the actions that the monkeys use to signal their categorical choices.

Tapered fibertrodes for optoelectrical neural interfacing in small brain volumes with reduced artefacts.

Deciphering the neural patterns underlying brain functions is essential to understanding how neurons are organized into networks. This deciphering has been greatly facilitated by optogenetics and its combination with optoelectronic devices to control neural activity with millisecond temporal resolution and cell type specificity. However, targeting small brain volumes causes photoelectric artefacts, in particular when light emission and recording sites are close to each other. We take advantage of the photonic properties of tapered fibres to develop integrated 'fibertrodes' able to optically activate small brain volumes with abated photoelectric noise. Electrodes are positioned very close to light emitting points by non-planar microfabrication, with angled light emission allowing the simultaneous optogenetic manipulation and electrical read-out of one to three neurons, with no photoelectric artefacts, in vivo. The unconventional implementation of two-photon polymerization on the curved taper edge enables the fabrication of recoding sites all around the implant, making fibertrodes a promising complement to planar microimplants.

Visuomotor adaptation changes stereoscopic depth perception and tactile discrimination.

Perceptual judgments of relative depth from binocular disparity are systematically distorted in humans, despite in principle having access to reliable 3D information. Interestingly, these distortions vanish at a natural grasping distance, as if perceived stereo depth is contingent on a specific reference distance for depth-disparity scaling that corresponds to the length of our arm. Here we show that the brain's representation of the arm indeed powerfully modulates depth perception, and that this internal calibration can be quickly updated. We used a classic visuomotor adaptation task in which subjects execute reaching movements with the visual feedback of their reaching finger displaced farther in depth, as if they had a longer arm. After adaptation, 3D perception changed dramatically, and became accurate at the "new" natural grasping distance, the updated disparity scaling reference distance. We further tested whether the rapid adaptive changes were restricted to the visual modality or were characteristic of sensory systems in general. Remarkably, we found an improvement in tactile discrimination consistent with a magnified internal image of the arm. This suggests that the brain integrates sensory signals with information about arm length, and quickly adapts to an artificially updated body structure. These adaptive processes are most likely a relic of the mechanisms needed to optimally correct for changes in size and shape of the body during ontogenesis.

Experience-dependent representation of visual categories in parietal cortex.

Categorization is a process by which the brain assigns meaning to sensory stimuli. Through experience, we learn to group stimuli into categories, such as 'chair', 'table' and 'vehicle', which are critical for rapidly and appropriately selecting behavioural responses. Although much is known about the neural representation of simple visual stimulus features (for example, orientation, direction and colour), relatively little is known about how the brain learns and encodes the meaning of stimuli. We trained monkeys to classify 360 degrees of visual motion directions into two discrete categories, and compared neuronal activity in the lateral intraparietal (LIP) and middle temporal (MT) areas, two interconnected brain regions known to be involved in visual motion processing. Here we show that neurons in LIP--an area known to be centrally involved in visuo-spatial attention, motor planning and decision-making-robustly reflect the category of motion direction as a result of learning. The activity of LIP neurons encoded directions of motion according to their category membership, and that encoding shifted after the monkeys were retrained to group the same stimuli into two new categories. In contrast, neurons in area MT were strongly direction selective but carried little, if any, explicit category information. This indicates that LIP might be an important nexus for the transformation of visual direction selectivity to more abstract representations that encode the behavioural relevance, or meaning, of stimuli.

Neurons in the orbitofrontal cortex encode economic value.

Economic choice is the behaviour observed when individuals select one among many available options. There is no intrinsically 'correct' answer: economic choice depends on subjective preferences. This behaviour is traditionally the object of economic analysis and is also of primary interest in psychology. However, the underlying mental processes and neuronal mechanisms are not well understood. Theories of human and animal choice have a cornerstone in the concept of 'value'. Consider, for example, a monkey offered one raisin versus one piece of apple: behavioural evidence suggests that the animal chooses by assigning values to the two options. But where and how values are represented in the brain is unclear. Here we show that, during economic choice, neurons in the orbitofrontal cortex (OFC) encode the value of offered and chosen goods. Notably, OFC neurons encode value independently of visuospatial factors and motor responses. If a monkey chooses between A and B, neurons in the OFC encode the value of the two goods independently of whether A is presented on the right and B on the left, or vice versa. This trait distinguishes the OFC from other brain areas in which value modulates activity related to sensory or motor processes. Our results have broad implications for possible psychological models, suggesting that economic choice is essentially choice between goods rather than choice between actions. In this framework, neurons in the OFC seem to be a good candidate network for value assignment underlying economic choice.

The representation of economic value in the orbitofrontal cortex is invariant for changes of menu.

Economic choice entails assigning values to the available options and is impaired by lesions to the orbitofrontal cortex (OFC). Recent results show that some neurons in the OFC encode the values that monkeys (Macaca mulatta) assign to different goods when they choose between them. A broad and fundamental question is how this neuronal representation of value depends on the behavioral context. Here we show that neuronal responses in the OFC are typically invariant for changes of menu. In other words, the activity of a neuron in response to one particular good usually does not depend on what other goods are available at the same time. Neurons in the OFC encode economic value, not relative preference. The fact that their responses are menu invariant suggests that transitivity, a fundamental trait of economic choice, may be rooted in the activity of individual neurons.

Dopamine mediates the bidirectional update of interval timing.

The role of dopamine (DA) as a reward prediction error (RPE) signal in reinforcement learning (RL) tasks has been well-established over the past decades. Recent work has shown that the RPE interpretation can also account for the effects of DA on interval timing by controlling the speed of subjective time. According to this theory, the timing of the dopamine signal relative to reward delivery dictates whether subjective time speeds up or slows down: Early DA signals speed up subjective time and late signals slow it down. To test this bidirectional prediction, we reanalyzed measurements of dopaminergic neurons in the substantia nigra pars compacta of mice performing a self-timed movement task. Using the slope of ramping dopamine activity as a readout of subjective time speed, we found that trial-by-trial changes in the slope could be predicted from the timing of dopamine activity on the previous trial. This result provides a key piece of evidence supporting a unified computational theory of RL and interval timing. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Temporal sequence of attentional modulation in the lateral intraparietal area and middle temporal area during rapid covert shifts of attention.

In the visual system, spatial attention enhances sensory responses to stimuli at attended locations relative to unattended locations. Which brain structures direct the locus of attention, and how is attentional modulation delivered to structures in the visual system? We trained monkeys on an attention-switch task designed to precisely measure the onset of attentional modulation during rapid shifts of spatial attention. Here we show that attentional modulation appears substantially earlier in the lateral intraparietal area (LIP) than in an anatomically connected lower visual area, the middle temporal area. This temporal sequence of attentional latencies demonstrates that endogenous changes of state can occur in higher visual areas before lower visual areas and satisfies a critical prediction of the hypothesis that LIP is a source of top-down attentional signals to early visual cortex.

Slowly evolving dopaminergic activity modulates the moment-to-moment probability of reward-related self-timed movements.

Clues from human movement disorders have long suggested that the neurotransmitter dopamine plays a role in motor control, but how the endogenous dopaminergic system influences movement is unknown. Here, we examined the relationship between dopaminergic signaling and the timing of reward-related movements in mice. Animals were trained to initiate licking after a self-timed interval following a start-timing cue; reward was delivered in response to movements initiated after a criterion time. The movement time was variable from trial-to-trial, as expected from previous studies. Surprisingly, dopaminergic signals ramped-up over seconds between the start-timing cue and the self-timed movement, with variable dynamics that predicted the movement/reward time on single trials. Steeply rising signals preceded early lick-initiation, whereas slowly rising signals preceded later initiation. Higher baseline signals also predicted earlier self-timed movements. Optogenetic activation of dopamine neurons during self-timing did not trigger immediate movements, but rather caused systematic early-shifting of movement initiation, whereas inhibition caused late-shifting, as if modulating the probability of movement. Consistent with this view, the dynamics of the endogenous dopaminergic signals quantitatively predicted the moment-by-moment probability of movement initiation on single trials. We propose that ramping dopaminergic signals, likely encoding dynamic reward expectation, can modulate the decision of when to move.

Distinct encoding of spatial and nonspatial visual information in parietal cortex.

It is well established that the primate parietal cortex plays an important role in visuospatial processing. Parietal cortex damage in both humans and monkeys can lead to behavioral deficits in spatial processing, and many parietal neurons, such as in the macaque lateral intraparietal area (LIP), are strongly influenced by visual-spatial factors. Several recent studies have shown that LIP neurons can also convey robust signals related to nonspatial factors, such as color, shape, and the behavioral context or rule that is relevant for solving the task at hand. But what is the relationship between the encoding of spatial factors and more abstract, nonspatial, influences in LIP? To examine this, we trained monkeys to group visual motion patterns into two arbitrary categories, and recorded the activity of LIP neurons while monkeys performed a categorization task in which stimuli were presented either inside each neuron's receptive field (RF) or at a location in the opposite visual field. While the activity of nearly all LIP neurons showed strong spatial dependence (i.e., greater responses when stimuli were presented within neurons' RFs), we also found that many LIP neurons also showed reliable encoding of the category membership of stimuli even when the stimuli were presented away from neurons' RFs. This suggests that both spatial and nonspatial information can be encoded by individual LIP neurons, and that parietal cortex may be a nexus for the integration of visuospatial signals and more abstract task-dependent information during complex visually based behaviors.

Neural activity in the middle temporal area and lateral intraparietal area during endogenously cued shifts of attention.

We measured the behavioral time course of endogenously cued attentional shifts while recording from neurons in the middle temporal area (MT) and lateral intraparietal area (LIP) of two macaque monkeys. The monkeys were required to detect a subtle speed change of one of two continuously moving stimuli. The likely location of the speed change was cued throughout each trial but could switch at an unpredictable time. Attention was evident as an improvement in detection ability and reaction time at the cued location, and the focus of attention shifted over a 400 ms period in response to a switch of the cued stimulus. Attention modulated the ongoing neural response in both MT and LIP, and the sign of this modulation also rapidly shifted after a cue switch. Our data provide a framework for understanding the link between the neural and behavioral effects of attention. The responses of single neurons to the test stimulus in MT and LIP were correlated with stimulus detection and reaction time and, at the population level, a spike-rate threshold model was able to account for the effect of attention on detection rate and reaction time. In this view, the time course of the attentional shift can be understood as an interaction between the emerging attentional modulation and the neural response to the test stimulus in LIP. We also present evidence that the threshold model is not wholly explained by sensory (feedforward) information but may also be influenced by cognitive (feedback) processes at the time of stimulus detection.

The effect of microsaccades on the correlation between neural activity and behavior in middle temporal, ventral intraparietal, and lateral intraparietal areas.

It is widely reported that the activity of single neurons in visual cortex is correlated with the perceptual decision of the subject. The strength of this correlation has implications for the neuronal populations generating the percepts. Here we asked whether microsaccades, which are small, involuntary eye movements, contribute to the correlation between neural activity and behavior. We analyzed data from three different visual detection experiments, with neural recordings from the middle temporal (MT), lateral intraparietal (LIP), and ventral intraparietal (VIP) areas. All three experiments used random dot motion stimuli, with the animals required to detect a transient or sustained change in the speed or strength of motion. We found that microsaccades suppressed neural activity and inhibited detection of the motion stimulus, contributing to the correlation between neural activity and detection behavior. Microsaccades accounted for as much as 19% of the correlation for area MT, 21% for area LIP, and 17% for VIP. While microsaccades only explain part of the correlation between neural activity and behavior, their effect has implications when considering the neuronal populations underlying perceptual decisions.

A cognitive signal for the proactive timing of action in macaque LIP.

Natural movements often occur without any immediate external event to cause them. In contrast to reactive movements, which are directly triggered by external cues, it is less clear how these proactive actions are initiated or when they will be made. We found that single neurons in the macaque's lateral intraparietal area (LIP) exhibit gradual firing rate elevations that reach a consistent value--which may correspond to a threshold--at the time of proactive, but not reactive, arm movements. This activity differs from sensory- and motor-related activity recorded in nearby cortical areas and could provide an internal trigger for action when abrupt external triggers in the visual input are unavailable.

Parietal activity and the perceived direction of ambiguous apparent motion.

We recorded from parietal neurons in monkeys (Macacca mulatta) trained to report the direction of an apparent motion stimulus consisting of regularly spaced columns of dots surrounded by an aperture. Displacing the dots by half their inter-column spacing produced vivid apparent motion that could be perceived in either the preferred or anti-preferred direction for each neuron. Many neurons in the lateral intraparietal area (LIP) responded more strongly on trials in which the animals reported perceiving the neurons' preferred direction, independent of the hand movement used to report their percept. This selectivity was less common in the medial superior temporal area (MST) and virtually absent in the middle temporal area (MT). Variations in activity of LIP and MST neurons just before motion onset were also predictive of the animals' subsequent perceived direction. These data suggest a hierarchy of representation in parietal cortex, whereby neuronal responses become more aligned with subjective perception in higher parietal areas.

Parietal area 5 and the initiation of self-timed movements versus simple reactions.

The timing of action has been studied extensively in reaction-time tasks in which an abrupt sensory stimulus triggers a movement. In these experiments, neurophysiologists have attempted to explain variability in movement time with variability in neuronal activity. However, in natural settings, movements are not usually triggered by abrupt sensory cues. What underlies the timing of action under such circumstances, when movements are uncoupled or only weakly coupled to abrupt events in the external world? We trained monkeys to perform the same arm movement either in direct reaction to a salient visual event, or as a self-timed action, less coupled to any obvious external trigger. Neurons in cortical area 5 exhibited phasic discharge modulations that were generally comparable for both modes of action, with some neurons increasing and others decreasing their firing rates with movement. For self-timed movements, however, there was an additional, slow ramp-up or ramp-down of activity in the few hundred milliseconds before the phasic discharge. These ramping modulations occurred well before any detectable changes in arm-muscle activity and their time course bore a striking resemblance to activity in the putamen preceding self-timed movements, observed previously. Together, the results suggest a possible mechanism for the internal timing of action within the motor system. In this model, reverberant activity in corticobasal-ganglia circuits reaches a threshold level resulting in much larger perimovement discharges within the same network, consequently driving the initiation of action.

Neural coding of behavioral relevance in parietal cortex.

Flexible control of behavior requires the selective processing of task-relevant sensory information and the appropriate linkage of sensory input to action. A great deal of evidence suggests a central role for the parietal cortex in these functions. Recent results from neurophysiological studies in non-human primates and neuroimaging experiments in humans illuminate the importance of parietal cortex for attention, and suggest how parietal neurons might allow the dynamic representation of behaviorally relevant information.

Relative timing: from behaviour to neurons.

Processing of temporal information is critical to behaviour. Here, we review the phenomenology and mechanism of relative timing, ordinal comparisons between the timing of occurrence of events. Relative timing can be an implicit component of particular brain computations or can be an explicit, conscious judgement. Psychophysical measurements of explicit relative timing have revealed clues about the interaction of sensory signals in the brain as well as in the influence of internal states, such as attention, on those interactions. Evidence from human neurophysiological and functional imaging studies, neuropsychological examination in brain-lesioned patients, and temporary disruptive interventions such as transcranial magnetic stimulation (TMS), point to a role of the parietal cortex in relative timing. Relative timing has traditionally been modelled as a 'race' between competing neural signals. We propose an updated race process based on the integration of sensory evidence towards a decision threshold rather than simple signal propagation. The model suggests a general approach for identifying brain regions involved in relative timing, based on looking for trial-by-trial correlations between neural activity and temporal order judgements (TOJs). Finally, we show how the paradigm can be used to reveal signals related to TOJs in parietal cortex of monkeys trained in a TOJ task.

Multipoint-emitting optical fibers for spatially addressable in vivo optogenetics.

Optical stimulation and silencing of neural activity is a powerful technique for elucidating the structure and function of neural circuitry. In most in vivo optogenetic experiments, light is delivered into the brain through a single optical fiber. However, this approach limits illumination to a fixed volume of the brain. Here a focused ion beam is used to pattern multiple light windows on a tapered optical fiber. We show that such fibers allow selective and dynamic illumination of different brain regions along the taper. Site selection is achieved by a simple coupling strategy at the fiber input, and the use of a single tapered waveguide minimizes the implant invasiveness. We demonstrate the effectiveness of this approach for multipoint optical stimulation in the mammalian brain in vivo by coupling the fiber to a microelectrode array and performing simultaneous extracellular recording and stimulation at multiple sites in the mouse striatum and cerebral cortex.

Biased associative representations in parietal cortex.

Neurons in cortical sensory areas respond selectively to sensory stimuli, and the preferred stimulus typically varies among neurons so as to continuously span the sensory space. However, some neurons reflect sensory features that are learned or task dependent. For example, neurons in the lateral intraparietal area (LIP) reflect learned associations between visual stimuli. One might expect that roughly even numbers of LIP neurons would prefer each set of associated stimuli. However, in two associative learning experiments and a perceptual decision experiment, we found striking asymmetries: nearly all neurons recorded from an animal had a similar order of preference among associated stimuli. Behavioral factors could not account for these neuronal biases. A recent computational study proposed that population-firing patterns in parietal cortex have one-dimensional dynamics on long timescales, a possible consequence of recurrent connections that could drive persistent activity. One-dimensional dynamics would predict the biases in selectivity that we observed.