Association Cortex Is Essential to Reverse Hemianopia by Multisensory Training.

Hemianopia induced by unilateral visual cortex lesions can be resolved by repeatedly exposing the blinded hemifield to auditory-visual stimuli. This rehabilitative "training" paradigm depends on mechanisms of multisensory plasticity that restore the lost visual responsiveness of multisensory neurons in the ipsilesional superior colliculus (SC) so that they can once again support vision in the blinded hemifield. These changes are thought to operate via the convergent visual and auditory signals relayed to the SC from association cortex (the anterior ectosylvian sulcus [AES], in cat). The present study tested this assumption by cryogenically deactivating ipsilesional AES in hemianopic, anesthetized cats during weekly multisensory training sessions. No signs of visual recovery were evident in this condition, even after providing animals with up to twice the number of training sessions required for effective rehabilitation. Subsequent training under the same conditions, but with AES active, reversed the hemianopia within the normal timeframe. These results indicate that the corticotectal circuit that is normally engaged in SC multisensory plasticity has to be operational for the brain to use visual-auditory experience to resolve hemianopia.

Multisensory Integration and the Society for Neuroscience: Then and Now.

The operation of our multiple and distinct sensory systems has long captured the interest of researchers from multiple disciplines. When the Society was founded 50 years ago to bring neuroscience research under a common banner, sensory research was largely divided along modality-specific lines. At the time, there were only a few physiological and anatomical observations of the multisensory interactions that powerfully influence our everyday perception. Since then, the neuroscientific study of multisensory integration has increased exponentially in both volume and diversity. From initial studies identifying the overlapping receptive fields of multisensory neurons, to subsequent studies of the spatial and temporal principles that govern the integration of multiple sensory cues, our understanding of this phenomenon at the single-neuron level has expanded to include a variety of dimensions. We now can appreciate how multisensory integration can alter patterns of neural activity in time, and even coordinate activity among populations of neurons across different brain areas. There is now a growing battery of sophisticated empirical and computational techniques that are being used to study this process in a number of models. These advancements have not only enhanced our understanding of this remarkable process in the normal adult brain, but also its underlying circuitry, requirements for development, susceptibility to malfunction, and how its principles may be used to mitigate malfunction.

Development of multisensory integration from the perspective of the individual neuron.

The ability to use cues from multiple senses in concert is a fundamental aspect of brain function. It maximizes the brain's use of the information available to it at any given moment and enhances the physiological salience of external events. Because each sense conveys a unique perspective of the external world, synthesizing information across senses affords computational benefits that cannot otherwise be achieved. Multisensory integration not only has substantial survival value but can also create unique experiences that emerge when signals from different sensory channels are bound together. However, neurons in a newborn's brain are not capable of multisensory integration, and studies in the midbrain have shown that the development of this process is not predetermined. Rather, its emergence and maturation critically depend on cross-modal experiences that alter the underlying neural circuit in such a way that optimizes multisensory integrative capabilities for the environment in which the animal will function.

Behavioral response inhibition and maturation of goal representation in prefrontal cortex after puberty.

Executive functions including behavioral response inhibition mature after puberty, in tandem with structural changes in the prefrontal cortex. Little is known about how activity of prefrontal neurons relates to this profound cognitive development. To examine this, we tracked neuronal responses of the prefrontal cortex in monkeys as they transitioned from puberty into adulthood and compared activity at different developmental stages. Performance of the antisaccade task greatly improved in this period. Among neural mechanisms that could facilitate it, reduction of stimulus-driven activity, increased saccadic activity, or enhanced representation of the opposing goal location, only the latter was evident in adulthood. Greatly accentuated in adults, this neural correlate of vector inversion may be a prerequisite to the formation of a motor plan to look away from the stimulus. Our results suggest that the prefrontal mechanisms that underlie mature performance on the antisaccade task are more strongly associated with forming an alternative plan of action than with suppressing the neural impact of the prepotent stimulus.

When the simplest voluntary decisions appear patently suboptimal.

Rahnev & Denison (R&D) catalog numerous experiments in which performance deviates, often in subtle ways, from the theoretical ideal. We discuss an extreme case, an elementary behavior (reactive saccades to single targets) for which a simple contextual manipulation results in responses that are dramatically different from those expected based on reward maximization - and yet are highly informative and amenable to mechanistic examination.

Age-dependent changes in prefrontal intrinsic connectivity.

The prefrontal cortex continues to mature after puberty and into early adulthood, mirroring the time course of maturation of cognitive abilities. However, the way in which prefrontal activity changes during peri- and postpubertal cortical maturation is largely unknown. To address this question, we evaluated the developmental stage of peripubertal rhesus monkeys with a series of morphometric, hormonal, and radiographic measures, and conducted behavioral and neurophysiological tests as the monkeys performed working memory tasks. We compared firing rate and the strength of intrinsic functional connectivity between neurons in peripubertal vs. adult monkeys. Notably, analyses of spike train cross-correlations demonstrated that the average magnitude of functional connections measured between neurons was lower overall in the prefrontal cortex of peripubertal monkeys compared with adults. The difference resulted because negative functional connections (indicative of inhibitory interactions) were stronger and more prevalent in peripubertal compared with adult monkeys, whereas the positive connections showed similar distributions in the two groups. Our results identify changes in the intrinsic connectivity of prefrontal neurons, particularly that mediated by inhibition, as a possible substrate for peri- and postpubertal advances in cognitive capacity.

Task dependence of decision- and choice-related activity in monkey oculomotor thalamus.

Oculomotor signals circulate within putative recurrent feedback loops that include the frontal eye field (FEF) and the oculomotor thalamus (OcTh). To examine how OcTh contributes to visuomotor control, and perceptually informed saccadic choices in particular, neural correlates of perceptual judgment and motor selection in OcTh were evaluated and compared with those previously reported for FEF in the same subjects. Monkeys performed three tasks: a choice task in which perceptual decisions are urgent, a choice task in which identical decisions are made without time pressure, and a single-target, delayed saccade task. The OcTh yielded far fewer task-responsive neurons than the FEF, but across responsive pools, similar neuron types were found, ranging from purely visual to purely saccade related. Across such types, the impact of the perceptual information relevant to saccadic choices was qualitatively the same in FEF and OcTh. However, distinct from that in FEF, activity in OcTh was strongly task dependent, typically being most vigorous in the urgent task, less so in the easier choice task, and least in the single-target task. This was true for responsive and nonresponsive cells alike. Neurons with exclusively motor-related activity showed strong task dependence, fired less, and differed most patently from their FEF counterparts, whereas those that combined visual and motor activity fired most similarly to their FEF counterparts. The results suggest that OcTh activity is more distantly related to saccade production per se, because its degree of commitment to a motor choice varies markedly as a function of ongoing cognitive or behavioral demands.

What does a neuron learn from multisensory experience?

The brain's ability to integrate information from different senses is acquired only after extensive sensory experience. However, whether early life experience instantiates a general integrative capacity in multisensory neurons or one limited to the particular cross-modal stimulus combinations to which one has been exposed is not known. By selectively restricting either visual-nonvisual or auditory-nonauditory experience during the first few months of life, the present study found that trisensory neurons in cat superior colliculus (as well as their bisensory counterparts) became adapted to the cross-modal stimulus combinations specific to each rearing environment. Thus, even at maturity, trisensory neurons did not integrate all cross-modal stimulus combinations to which they were capable of responding, but only those that had been linked via experience to constitute a coherent spatiotemporal event. This selective maturational process determines which environmental events will become the most effective targets for superior colliculus-mediated shifts of attention and orientation.

The countermanding task revisited: fast stimulus detection is a key determinant of psychophysical performance.

The countermanding task is a standard method for assessing cognitive/inhibitory control over action and for investigating its neural correlates. In it, the subject plans a movement and either executes it, if no further instruction is given, or attempts to prevent it, if a stop signal is shown. Through various experimental manipulations, many studies have sought to characterize the inhibitory mechanisms thought to be at work in the task, typically using an inferred, model-dependent metric called the stop-signal reaction time. This approach has consistently overlooked the impact of perceptual evaluation on performance. Through analytical work and computer simulations, here we show that psychophysical performance in the task can be easily understood as the result of an ongoing motor plan that is modified (decelerated) by the outcome of a rapid sensory detection process. Notably, no specific assumptions about hypothetical inhibitory mechanisms are needed. This modeling framework achieves four things: (1) it replicates and reconciles behavioral results in numerous variants of the countermanding task; (2) it provides a new, objective metric for characterizing task performance that is more effective than the stop-signal reaction time; (3) it shows that the time window over which detection of a high-visibility stimulus effectively occurs is extremely short (∼20 ms); and (4) it indicates that modulating neuronal latencies and the buildup rates of developing motor plans are two key neural mechanisms for controlling action. The results suggest that manipulations of the countermanding task often cause changes in perceptual detection processes, and not necessarily in inhibition.

Perceptual modulation of motor--but not visual--responses in the frontal eye field during an urgent-decision task.

Neuronal activity in the frontal eye field (FEF) ranges from purely motor (related to saccade production) to purely visual (related to stimulus presence). According to numerous studies, visual responses correlate strongly with early perceptual analysis of the visual scene, including the deployment of spatial attention, whereas motor responses do not. Thus, functionally, the consensus is that visually responsive FEF neurons select a target among visible objects, whereas motor-related neurons plan specific eye movements based on such earlier target selection. However, these conclusions are based on behavioral tasks that themselves promote a serial arrangement of perceptual analysis followed by motor planning. So, is the presumed functional hierarchy in FEF an intrinsic property of its circuitry or does it reflect just one possible mode of operation? We investigate this in monkeys performing a rapid-choice task in which, crucially, motor planning always starts ahead of task-critical perceptual analysis, and the two relevant spatial locations are equally informative and equally likely to be target or distracter. We find that the choice is instantiated in FEF as a competition between oculomotor plans, in agreement with model predictions. Notably, although perception strongly influences the motor neurons, it has little if any measurable impact on the visual cells; more generally, the more dominant the visual response, the weaker the perceptual modulation. The results indicate that, contrary to expectations, during rapid saccadic choices perceptual information may directly modulate ongoing saccadic plans, and this process is not contingent on prior selection of the saccadic goal by visually driven FEF responses.

Noise-rearing disrupts the maturation of multisensory integration.

It is commonly believed that the ability to integrate information from different senses develops according to associative learning principles as neurons acquire experience with co-active cross-modal inputs. However, previous studies have not distinguished between requirements for co-activation versus co-variation. To determine whether cross-modal co-activation is sufficient for this purpose in visual-auditory superior colliculus (SC) neurons, animals were reared in constant omnidirectional noise. By masking most spatiotemporally discrete auditory experiences, the noise created a sensory landscape that decoupled stimulus co-activation and co-variance. Although a near-normal complement of visual-auditory SC neurons developed, the vast majority could not engage in multisensory integration, revealing that visual-auditory co-activation was insufficient for this purpose. That experience with co-varying stimuli is required for multisensory maturation is consistent with the role of the SC in detecting and locating biologically significant events, but it also seems likely that this is a general requirement for multisensory maturation throughout the brain.

Conditional independence as a statistical assessment of evidence integration processes.

Intuitively, combining multiple sources of evidence should lead to more accurate decisions than considering single sources of evidence individually. In practice, however, the proper computation may be difficult, or may require additional data that are inaccessible. Here, based on the concept of conditional independence, we consider expressions that can serve either as recipes for integrating evidence based on limited data, or as statistical benchmarks for characterizing evidence integration processes. Consider three events, A, B, and C. We find that, if A and B are conditionally independent with respect to C, then the probability that C occurs given that both A and B are known, P(C|A, B), can be easily calculated without the need to measure the full three-way dependency between A, B, and C. This simplified approach can be used in two general ways: to generate predictions by combining multiple (conditionally independent) sources of evidence, or to test whether separate sources of evidence are functionally independent of each other. These applications are demonstrated with four computer-simulated examples, which include detecting a disease based on repeated diagnostic testing, inferring biological age based on multiple biomarkers of aging, discriminating two spatial locations based on multiple cue stimuli (multisensory integration), and examining how behavioral performance in a visual search task depends on selection histories. Besides providing a sound prescription for predicting outcomes, this methodology may be useful for analyzing experimental data of many types.

Coupling of saccade plans to endogenous attention during urgent choices.

The neural mechanisms that willfully direct attention to specific locations in space are closely related to those for generating targeting eye movements (saccades). However, the degree to which the voluntary deployment of attention to a location necessarily activates a corresponding saccade plan remains unclear. One problem is that attention and saccades are both automatically driven by salient sensory events; another is that the underlying processes unfold within tens of milliseconds only. Here, we use an urgent task design to resolve the evolution of a visuomotor choice on a moment-by-moment basis while independently controlling the endogenous (goal-driven) and exogenous (salience-driven) contributions to performance. Human participants saw a peripheral cue and, depending on its color, either looked at it (prosaccade) or looked at a diametrically opposite, uninformative non-cue (antisaccade). By varying the luminance of the stimuli, the exogenous contributions could be cleanly dissociated from the endogenous process guiding the choice over time. According to the measured timecourses, generating a correct antisaccade requires about 30 ms more processing time than generating a correct prosaccade based on the same perceptual signal. The results indicate that saccade plans elaborated during fixation are biased toward the location where attention is endogenously deployed, but the coupling is weak and can be willfully overridden very rapidly.

Location- and feature-based selection histories make independent, qualitatively distinct contributions to urgent visuomotor performance.

Attention mechanisms that guide visuomotor behaviors are classified into three broad types according to their reliance on stimulus salience, current goals, and selection histories (i.e., recent experience with events of many sorts). These forms of attentional control are clearly distinct and multifaceted, but what is largely unresolved is how they interact dynamically to determine impending visuomotor choices. To investigate this, we trained two macaque monkeys to perform an urgent version of an oddball search task in which a red target appears among three green distracters, or vice versa. By imposing urgency, performance can be tracked continuously as it transitions from uninformed guesses to informed choices, and this, in turn, permits assessment of attentional control as a function of time. We found that the probability of making a correct choice was strongly modulated by the histories of preceding target colors and target locations. Crucially, although both effects were gated by success (or reward), the two variables played dynamically distinct roles: whereas location history promoted an early motor bias, color history modulated the later perceptual evaluation. Furthermore, target color and location influenced performance independently of each other. The results show that, when combined, selection histories can give rise to enormous swings in visuomotor performance even in simple tasks with highly discriminable stimuli.

Parallel signatures of cognitive maturation in primate antisaccade performance and prefrontal activity.

The ability to suppress inappropriate actions and respond rapidly to appropriate ones matures late in life, after puberty. We investigated the development of this capability in monkeys trained to look away from a lone, bright stimulus (antisaccade task). We evaluated behavioral performance and recorded neural activity in the prefrontal cortex both before and after the transition from puberty to adulthood. Compared to when young, adult monkeys processed the stimulus more rapidly, resisted more effectively the involuntary urge to look at it, and adhered to the task rule more consistently. The spatially selective visuomotor neurons in the prefrontal cortex provided neural correlates of these behavioral changes indicative of a faster transition from stimulus-driven (exogenous) to goal-driven (endogenous) control within the time course of each trial. The results reveal parallel signatures of cognitive maturation in behavior and prefrontal activity that are consistent with improvements in attentional allocation after adolescence.

Brain structure and activity predicting cognitive maturation in adolescence.

Cognitive abilities of primates, including humans, continue to improve through adolescence 1,2. While a range of changes in brain structure and connectivity have been documented 3,4, how they affect neuronal activity that ultimately determines performance of cognitive functions remains unknown. Here, we conducted a multilevel longitudinal study of monkey adolescent neurocognitive development. The developmental trajectory of neural activity in the prefrontal cortex accounted remarkably well for working memory improvements. While complex aspects of activity changed progressively during adolescence, such as the rotation of stimulus representation in multidimensional neuronal space, which has been implicated in cognitive flexibility, even simpler attributes, such as the baseline firing rate in the period preceding a stimulus appearance had predictive power over behavior. Unexpectedly, decreases in brain volume and thickness, which are widely thought to underlie cognitive changes in humans 5 did not predict well the trajectory of neural activity or cognitive performance changes. Whole brain cortical volume in particular, exhibited an increase and reached a local maximum in late adolescence, at a time of rapid behavioral improvement. Maturation of long-distance white matter tracts linking the frontal lobe with areas of the association cortex and subcortical regions best predicted changes in neuronal activity and behavior. Our results provide evidence that optimization of neural activity depending on widely distributed circuitry effects cognitive development in adolescence.

Neural correlates of a decision variable before learning to perform a match/non-match task.

The lateral prefrontal cortex plays an important role in working memory and decision-making, although little is known about how neural correlates of these functions are shaped by learning. To understand the effect of learning on the neuronal representation of decision-making, we recorded single neurons from the lateral prefrontal cortex of monkeys before and after they were trained to judge whether two stimuli appeared at matching spatial locations. After training, and in agreement with previous studies, a population of neurons exhibited activity that was modulated depending on whether the second stimulus constituted a match or not, which had predictive ability for the monkey's choice. However, even before training, prefrontal neurons displayed modulation depending on the match or non-match status of a stimulus, with approximately equal percentages of neurons preferring a match or a non-match. The difference in firing rate and discriminability for match and non-match stimuli before training was of comparable magnitude as that after training. Changes observed after training involved an increase in the percentage of neurons exhibiting this effect, a greater proportion of neurons preferring non-match stimuli, and a greater percentage of neurons representing information about the first stimulus during the presentation of the second stimulus. Our results suggest that the neuronal activity representing some match/non-match judgments is present in the lateral prefrontal cortex even when subjects are not required to perform a comparison and before any training.

Incorporating cross-modal statistics in the development and maintenance of multisensory integration.

Development of multisensory integration capabilities in superior colliculus (SC) neurons was examined in cats whose visual-auditory experience was restricted to a circumscribed period during early life (postnatal day 30-8 months). Animals were periodically exposed to visual and auditory stimuli appearing either randomly in space and time, or always in spatiotemporal concordance. At all other times animals were maintained in darkness. Physiological testing was initiated at ∼2 years of age. Exposure to random visual and auditory stimuli proved insufficient to spur maturation of the ability to integrate cross-modal stimuli, but exposure to spatiotemporally concordant cross-modal stimuli was highly effective. The multisensory integration capabilities of neurons in the latter group resembled those of normal animals and were retained for >16 months in the absence of subsequent visual-auditory experience. Furthermore, the neurons were capable of integrating stimuli having physical properties differing significantly from those in the exposure set. These observations suggest that acquiring the rudiments of multisensory integration requires little more than exposure to consistent relationships between the modality-specific components of a cross-modal event, and that continued experience with such events is not necessary for their maintenance. Apparently, the statistics of cross-modal experience early in life define the spatial and temporal filters that determine whether the components of cross-modal stimuli are to be integrated or treated as independent events, a crucial developmental process that determines the spatial and temporal rules by which cross-modal stimuli are integrated to enhance both sensory salience and the likelihood of eliciting an SC-mediated motor response.

Working memory performance and neural activity in prefrontal cortex of peripubertal monkeys.

The dorsolateral prefrontal cortex matures late into adolescence or early adulthood. This pattern of maturation mirrors working memory abilities, which continue to improve into adulthood. However, the nature of the changes that prefrontal neuronal activity undergoes during this process is poorly understood. We investigated behavioral performance and neural activity in working memory tasks around the time of puberty, a developmental event associated with the release of sex hormones and significant neurological change. The developmental stages of male rhesus monkeys were evaluated with a series of morphometric, hormonal, and radiographic measures. Peripubertal monkeys were trained to perform an oculomotor delayed response task and a variation of this task involving a distractor stimulus. We found that the peripubertal monkeys tended to abort a relatively large fraction of trials, and these were associated with low levels of task-related neuronal activity. However, for completed trials, accuracy in the delayed saccade task was high and the appearance of a distractor stimulus did not impact performance significantly. In correct trials delay period activity was robust and was not eliminated by the presentation of a distracting stimulus, whereas in trials that resulted in errors the sustained cue-related activity was significantly weaker. Our results show that in peripubertal monkeys the prefrontal cortex is capable of generating robust persistent activity in the delay periods of working memory tasks, although in general it may be more prone to stochastic failure than in adults.

Multisensory integration: current issues from the perspective of the single neuron.

For thousands of years science philosophers have been impressed by how effectively the senses work together to enhance the salience of biologically meaningful events. However, they really had no idea how this was accomplished. Recent insights into the underlying physiological mechanisms reveal that, in at least one circuit, this ability depends on an intimate dialogue among neurons at multiple levels of the neuraxis; this dialogue cannot take place until long after birth and might require a specific kind of experience. Understanding the acquisition and usage of multisensory integration in the midbrain and cerebral cortex of mammals has been aided by a multiplicity of approaches. Here we examine some of the fundamental advances that have been made and some of the challenging questions that remain.