Sustained Activation of PV+ Interneurons in Core Auditory Cortex Enables Robust Divisive Gain Control for Complex and Naturalistic Stimuli.

Sensory cortices must flexibly adapt their operations to internal states and external requirements. Sustained modulation of activity levels in different inhibitory interneuron populations may provide network-level mechanisms for adjustment of sensory cortical processing on behaviorally relevant timescales. However, understanding of the computational roles of inhibitory interneuron modulation has mostly been restricted to effects at short timescales, through the use of phasic optogenetic activation and transient stimuli. Here, we investigated how modulation of inhibitory interneurons affects cortical computation on longer timescales, by using sustained, network-wide optogenetic activation of parvalbumin-positive interneurons (the largest class of cortical inhibitory interneurons) to study modulation of auditory cortical responses to prolonged and naturalistic as well as transient stimuli. We found highly conserved spectral and temporal tuning in auditory cortical neurons, despite a profound reduction in overall network activity. This reduction was predominantly divisive, and consistent across simple, complex, and naturalistic stimuli. A recurrent network model with power-law input-output functions replicated our results. We conclude that modulation of parvalbumin-positive interneurons on timescales typical of sustained neuromodulation may provide a means for robust divisive gain control conserving stimulus representations.

Cortical control of arm movements: a dynamical systems perspective.

Our ability to move is central to everyday life. Investigating the neural control of movement in general, and the cortical control of volitional arm movements in particular, has been a major research focus in recent decades. Studies have involved primarily either attempts to account for single-neuron responses in terms of tuning for movement parameters or attempts to decode movement parameters from populations of tuned neurons. Even though this focus on encoding and decoding has led to many seminal advances, it has not produced an agreed-upon conceptual framework. Interest in understanding the underlying neural dynamics has recently increased, leading to questions such as how does the current population response determine the future population response, and to what purpose? We review how a dynamical systems perspective may help us understand why neural activity evolves the way it does, how neural activity relates to movement parameters, and how a unified conceptual framework may result.

State-dependent population coding in primary auditory cortex.

Sensory function is mediated by interactions between external stimuli and intrinsic cortical dynamics that are evident in the modulation of evoked responses by cortical state. A number of recent studies across different modalities have demonstrated that the patterns of activity in neuronal populations can vary strongly between synchronized and desynchronized cortical states, i.e., in the presence or absence of intrinsically generated up and down states. Here we investigated the impact of cortical state on the population coding of tones and speech in the primary auditory cortex (A1) of gerbils, and found that responses were qualitatively different in synchronized and desynchronized cortical states. Activity in synchronized A1 was only weakly modulated by sensory input, and the spike patterns evoked by tones and speech were unreliable and constrained to a small range of patterns. In contrast, responses to tones and speech in desynchronized A1 were temporally precise and reliable across trials, and different speech tokens evoked diverse spike patterns with extremely weak noise correlations, allowing responses to be decoded with nearly perfect accuracy. Restricting the analysis of synchronized A1 to activity within up states yielded similar results, suggesting that up states are not equivalent to brief periods of desynchronization. These findings demonstrate that the representational capacity of A1 depends strongly on cortical state, and suggest that cortical state should be considered as an explicit variable in all studies of sensory processing.

The equivalence of information-theoretic and likelihood-based methods for neural dimensionality reduction.

Stimulus dimensionality-reduction methods in neuroscience seek to identify a low-dimensional space of stimulus features that affect a neuron's probability of spiking. One popular method, known as maximally informative dimensions (MID), uses an information-theoretic quantity known as "single-spike information" to identify this space. Here we examine MID from a model-based perspective. We show that MID is a maximum-likelihood estimator for the parameters of a linear-nonlinear-Poisson (LNP) model, and that the empirical single-spike information corresponds to the normalized log-likelihood under a Poisson model. This equivalence implies that MID does not necessarily find maximally informative stimulus dimensions when spiking is not well described as Poisson. We provide several examples to illustrate this shortcoming, and derive a lower bound on the information lost when spiking is Bernoulli in discrete time bins. To overcome this limitation, we introduce model-based dimensionality reduction methods for neurons with non-Poisson firing statistics, and show that they can be framed equivalently in likelihood-based or information-theoretic terms. Finally, we show how to overcome practical limitations on the number of stimulus dimensions that MID can estimate by constraining the form of the non-parametric nonlinearity in an LNP model. We illustrate these methods with simulations and data from primate visual cortex.

Five key factors determining pairwise correlations in visual cortex.

The responses of cortical neurons to repeated presentation of a stimulus are highly variable, yet correlated. These "noise correlations" reflect a low-dimensional structure of population dynamics. Here, we examine noise correlations in 22,705 pairs of neurons in primary visual cortex (V1) of anesthetized cats, during ongoing activity and in response to artificial and natural visual stimuli. We measured how noise correlations depend on 11 factors. Because these factors are themselves not independent, we distinguished their influences using a nonlinear additive model. The model revealed that five key factors play a predominant role in determining pairwise correlations. Two of these are distance in cortex and difference in sensory tuning: these are known to decrease correlation. A third factor is firing rate: confirming most earlier observations, it markedly increased pairwise correlations. A fourth factor is spike width: cells with a broad spike were more strongly correlated amongst each other. A fifth factor is spike isolation: neurons with worse isolation were more correlated, even if they were recorded on different electrodes. For pairs of neurons with poor isolation, this last factor was the main determinant of correlations. These results were generally independent of stimulus type and timescale of analysis, but there were exceptions. For instance, pairwise correlations depended on difference in orientation tuning more during responses to gratings than to natural stimuli. These results consolidate disjoint observations in a vast literature on pairwise correlations and point towards regularities of population coding in sensory cortex.

Outlier responses reflect sensitivity to statistical structure in the human brain.

We constantly look for patterns in the environment that allow us to learn its key regularities. These regularities are fundamental in enabling us to make predictions about what is likely to happen next. The physiological study of regularity extraction has focused primarily on repetitive sequence-based rules within the sensory environment, or on stimulus-outcome associations in the context of reward-based decision-making. Here we ask whether we implicitly encode non-sequential stochastic regularities, and detect violations therein. We addressed this question using a novel experimental design and both behavioural and magnetoencephalographic (MEG) metrics associated with responses to pure-tone sounds with frequencies sampled from a Gaussian distribution. We observed that sounds in the tail of the distribution evoked a larger response than those that fell at the centre. This response resembled the mismatch negativity (MMN) evoked by surprising or unlikely events in traditional oddball paradigms. Crucially, responses to physically identical outliers were greater when the distribution was narrower. These results show that humans implicitly keep track of the uncertainty induced by apparently random distributions of sensory events. Source reconstruction suggested that the statistical-context-sensitive responses arose in a temporo-parietal network, areas that have been associated with attention orientation to unexpected events. Our results demonstrate a very early neurophysiological marker of the brain's ability to implicitly encode complex statistical structure in the environment. We suggest that this sensitivity provides a computational basis for our ability to make perceptual inferences in noisy environments and to make decisions in an uncertain world.

Residual dynamics resolves recurrent contributions to neural computation.

Relating neural activity to behavior requires an understanding of how neural computations arise from the coordinated dynamics of distributed, recurrently connected neural populations. However, inferring the nature of recurrent dynamics from partial recordings of a neural circuit presents considerable challenges. Here we show that some of these challenges can be overcome by a fine-grained analysis of the dynamics of neural residuals-that is, trial-by-trial variability around the mean neural population trajectory for a given task condition. Residual dynamics in macaque prefrontal cortex (PFC) in a saccade-based perceptual decision-making task reveals recurrent dynamics that is time dependent, but consistently stable, and suggests that pronounced rotational structure in PFC trajectories during saccades is driven by inputs from upstream areas. The properties of residual dynamics restrict the possible contributions of PFC to decision-making and saccade generation and suggest a path toward fully characterizing distributed neural computations with large-scale neural recordings and targeted causal perturbations.

Learning shapes cortical dynamics to enhance integration of relevant sensory input.

Adaptive sensory behavior is thought to depend on processing in recurrent cortical circuits, but how dynamics in these circuits shapes the integration and transmission of sensory information is not well understood. Here, we study neural coding in recurrently connected networks of neurons driven by sensory input. We show analytically how information available in the network output varies with the alignment between feedforward input and the integrating modes of the circuit dynamics. In light of this theory, we analyzed neural population activity in the visual cortex of mice that learned to discriminate visual features. We found that over learning, slow patterns of network dynamics realigned to better integrate input relevant to the discrimination task. This realignment of network dynamics could be explained by changes in excitatory-inhibitory connectivity among neurons tuned to relevant features. These results suggest that learning tunes the temporal dynamics of cortical circuits to optimally integrate relevant sensory input.

Depth-dependent temporal response properties in core auditory cortex.

The computational role of cortical layers within auditory cortex has proven difficult to establish. One hypothesis is that interlaminar cortical processing might be dedicated to analyzing temporal properties of sounds; if so, then there should be systematic depth-dependent changes in cortical sensitivity to the temporal context in which a stimulus occurs. We recorded neural responses simultaneously across cortical depth in primary auditory cortex and anterior auditory field of CBA/Ca mice, and found systematic depth dependencies in responses to second-and-later noise bursts in slow (1-10 bursts/s) trains of noise bursts. At all depths, responses to noise bursts within a train usually decreased with increasing train rate; however, the rolloff with increasing train rate occurred at faster rates in more superficial layers. Moreover, in some recordings from mid-to-superficial layers, responses to noise bursts within a 3-4 bursts/s train were stronger than responses to noise bursts in slower trains. This non-monotonicity with train rate was especially pronounced in more superficial layers of the anterior auditory field, where responses to noise bursts within the context of a slow train were sometimes even stronger than responses to the noise burst at train onset. These findings may reflect depth dependence in suppression and recovery of cortical activity following a stimulus, which we suggest could arise from laminar differences in synaptic depression at feedforward and recurrent synapses.

Learning stable, regularised latent models of neural population dynamics.

Ongoing advances in experimental technique are making commonplace simultaneous recordings of the activity of tens to hundreds of cortical neurons at high temporal resolution. Latent population models, including Gaussian-process factor analysis and hidden linear dynamical system (LDS) models, have proven effective at capturing the statistical structure of such data sets. They can be estimated efficiently, yield useful visualisations of population activity, and are also integral building-blocks of decoding algorithms for brain-machine interfaces (BMI). One practical challenge, particularly to LDS models, is that when parameters are learned using realistic volumes of data the resulting models often fail to reflect the true temporal continuity of the dynamics; and indeed may describe a biologically-implausible unstable population dynamic that is, it may predict neural activity that grows without bound. We propose a method for learning LDS models based on expectation maximisation that constrains parameters to yield stable systems and at the same time promotes capture of temporal structure by appropriate regularisation. We show that when only little training data is available our method yields LDS parameter estimates which provide a substantially better statistical description of the data than alternatives, whilst guaranteeing stable dynamics. We demonstrate our methods using both synthetic data and extracellular multi-electrode recordings from motor cortex.

Deep learning, reinforcement learning, and world models.

Deep learning (DL) and reinforcement learning (RL) methods seem to be a part of indispensable factors to achieve human-level or super-human AI systems. On the other hand, both DL and RL have strong connections with our brain functions and with neuroscientific findings. In this review, we summarize talks and discussions in the "Deep Learning and Reinforcement Learning" session of the symposium, International Symposium on Artificial Intelligence and Brain Science. In this session, we discussed whether we can achieve comprehensive understanding of human intelligence based on the recent advances of deep learning and reinforcement learning algorithms. Speakers contributed to provide talks about their recent studies that can be key technologies to achieve human-level intelligence.

Learning and attention increase visual response selectivity through distinct mechanisms.

Selectivity of cortical neurons for sensory stimuli can increase across days as animals learn their behavioral relevance and across seconds when animals switch attention. While both phenomena occur in the same circuit, it is unknown whether they rely on similar mechanisms. We imaged primary visual cortex as mice learned a visual discrimination task and subsequently performed an attention switching task. Selectivity changes due to learning and attention were uncorrelated in individual neurons. Selectivity increases after learning mainly arose from selective suppression of responses to one of the stimuli but from selective enhancement and suppression during attention. Learning and attention differentially affected interactions between excitatory and PV, SOM, and VIP inhibitory cells. Circuit modeling revealed that cell class-specific top-down inputs best explained attentional modulation, while reorganization of local functional connectivity accounted for learning-related changes. Thus, distinct mechanisms underlie increased discriminability of relevant sensory stimuli across longer and shorter timescales.

Dynamics on the manifold: Identifying computational dynamical activity from neural population recordings.

The question of how the collective activity of neural populations gives rise to complex behaviour is fundamental to neuroscience. At the core of this question lie considerations about how neural circuits can perform computations that enable sensory perception, decision making, and motor control. It is thought that such computations are implemented through the dynamical evolution of distributed activity in recurrent circuits. Thus, identifying dynamical structure in neural population activity is a key challenge towards a better understanding of neural computation. At the same time, interpreting this structure in light of the computation of interest is essential for linking the time-varying activity patterns of the neural population to ongoing computational processes. Here, we review methods that aim to quantify structure in neural population recordings through a dynamical system defined in a low-dimensional latent variable space. We discuss advantages and limitations of different modelling approaches and address future challenges for the field.

Dendritic calcium signals in rhesus macaque motor cortex drive an optical brain-computer interface.

Calcium imaging is a powerful tool for recording from large populations of neurons in vivo. Imaging in rhesus macaque motor cortex can enable the discovery of fundamental principles of motor cortical function and can inform the design of next generation brain-computer interfaces (BCIs). Surface two-photon imaging, however, cannot presently access somatic calcium signals of neurons from all layers of macaque motor cortex due to photon scattering. Here, we demonstrate an implant and imaging system capable of chronic, motion-stabilized two-photon imaging of neuronal calcium signals from macaques engaged in a motor task. By imaging apical dendrites, we achieved optical access to large populations of deep and superficial cortical neurons across dorsal premotor (PMd) and gyral primary motor (M1) cortices. Dendritic signals from individual neurons displayed tuning for different directions of arm movement. Combining several technical advances, we developed an optical BCI (oBCI) driven by these dendritic signalswhich successfully decoded movement direction online. By fusing two-photon functional imaging with CLARITY volumetric imaging, we verified that many imaged dendrites which contributed to oBCI decoding originated from layer 5 output neurons, including a putative Betz cell. This approach establishes new opportunities for studying motor control and designing BCIs via two photon imaging.

Effects of category-specific costs on neural systems for perceptual decision-making.

Perceptual judgments are often biased by prospective losses, leading to changes in decision criteria. Little is known about how and where sensory evidence and cost information interact in the brain to influence perceptual categorization. Here we show that prospective losses systematically bias the perception of noisy face-house images. Asymmetries in category-specific cost were associated with enhanced blood-oxygen-level-dependent signal in a frontoparietal network. We observed selective activation of parahippocampal gyrus for changes in category-specific cost in keeping with the hypothesis that loss functions enact a particular task set that is communicated to visual regions. Across subjects, greater shifts in decision criteria were associated with greater activation of the anterior cingulate cortex (ACC). Our results support a hypothesis that costs bias an intermediate representation between perception and action, expressed via general effects on frontal cortex, and selective effects on extrastriate cortex. These findings indicate that asymmetric costs may affect a neural implementation of perceptual decision making in a similar manner to changes in category expectation, constituting a step toward accounting for how prospective losses are flexibly integrated with sensory evidence in the brain.

A structured model of video reproduces primary visual cortical organisation.

The visual system must learn to infer the presence of objects and features in the world from the images it encounters, and as such it must, either implicitly or explicitly, model the way these elements interact to create the image. Do the response properties of cells in the mammalian visual system reflect this constraint? To address this question, we constructed a probabilistic model in which the identity and attributes of simple visual elements were represented explicitly and learnt the parameters of this model from unparsed, natural video sequences. After learning, the behaviour and grouping of variables in the probabilistic model corresponded closely to functional and anatomical properties of simple and complex cells in the primary visual cortex (V1). In particular, feature identity variables were activated in a way that resembled the activity of complex cells, while feature attribute variables responded much like simple cells. Furthermore, the grouping of the attributes within the model closely parallelled the reported anatomical grouping of simple cells in cat V1. Thus, this generative model makes explicit an interpretation of complex and simple cells as elements in the segmentation of a visual scene into basic independent features, along with a parametrisation of their moment-by-moment appearances. We speculate that such a segmentation may form the initial stage of a hierarchical system that progressively separates the identity and appearance of more articulated visual elements, culminating in view-invariant object recognition.

Movement initiation and grasp representation in premotor and primary motor cortex mirror neurons.

Pyramidal tract neurons (PTNs) within macaque rostral ventral premotor cortex (F5) and (M1) provide direct input to spinal circuitry and are critical for skilled movement control. Contrary to initial hypotheses, they can also be active during action observation, in the absence of any movement. A population-level understanding of this phenomenon is currently lacking. We recorded from single neurons, including identified PTNs, in (M1) (n = 187), and F5 (n = 115) as two adult male macaques executed, observed, or withheld (NoGo) reach-to-grasp actions. F5 maintained a similar representation of grasping actions during both execution and observation. In contrast, although many individual M1 neurons were active during observation, M1 population activity was distinct from execution, and more closely aligned to NoGo activity, suggesting this activity contributes to withholding of self-movement. M1 and its outputs may dissociate initiation of movement from representation of grasp in order to flexibly guide behaviour.

The consequences of response nonlinearities for interpretation of spectrotemporal receptive fields.

Neurons in the central auditory system are often described by the spectrotemporal receptive field (STRF), conventionally defined as the best linear fit between the spectrogram of a sound and the spike rate it evokes. An STRF is often assumed to provide an estimate of the receptive field of a neuron, i.e., the spectral and temporal range of stimuli that affect the response. However, when the true stimulus-response function is nonlinear, the STRF will be stimulus dependent, and changes in the stimulus properties can alter estimates of the sign and spectrotemporal extent of receptive field components. We demonstrate analytically and in simulations that, even when uncorrelated stimuli are used, interactions between simple neuronal nonlinearities and higher-order structure in the stimulus can produce STRFs that show contributions from time-frequency combinations to which the neuron is actually insensitive. Only when spectrotemporally independent stimuli are used does the STRF reliably indicate features of the underlying receptive field, and even then it provides only a conservative estimate. One consequence of these observations, illustrated using natural stimuli, is that a stimulus-induced change in an STRF could arise from a consistent but nonlinear neuronal response to stimulus ensembles with differing higher-order dependencies. Thus, although the responses of higher auditory neurons may well involve adaptation to the statistics of different stimulus ensembles, stimulus dependence of STRFs alone, or indeed of any overly constrained stimulus-response mapping, cannot demonstrate the nature or magnitude of such effects.

Nonlinearities and contextual influences in auditory cortical responses modeled with multilinear spectrotemporal methods.

The relationship between a sound and its neural representation in the auditory cortex remains elusive. Simple measures such as the frequency response area or frequency tuning curve provide little insight into the function of the auditory cortex in complex sound environments. Spectrotemporal receptive field (STRF) models, despite their descriptive potential, perform poorly when used to predict auditory cortical responses, showing that nonlinear features of cortical response functions, which are not captured by STRFs, are functionally important. We introduce a new approach to the description of auditory cortical responses, using multilinear modeling methods. These descriptions simultaneously account for several nonlinearities in the stimulus-response functions of auditory cortical neurons, including adaptation, spectral interactions, and nonlinear sensitivity to sound level. The models reveal multiple inseparabilities in cortical processing of time lag, frequency, and sound level, and suggest functional mechanisms by which auditory cortical neurons are sensitive to stimulus context. By explicitly modeling these contextual influences, the models are able to predict auditory cortical responses more accurately than are STRF models. In addition, they can explain some forms of stimulus dependence in STRFs that were previously poorly understood.

Techniques for extracting single-trial activity patterns from large-scale neural recordings.

Large, chronically implanted arrays of microelectrodes are an increasingly common tool for recording from primate cortex and can provide extracellular recordings from many (order of 100) neurons. While the desire for cortically based motor prostheses has helped drive their development, such arrays also offer great potential to advance basic neuroscience research. Here we discuss the utility of array recording for the study of neural dynamics. Neural activity often has dynamics beyond that driven directly by the stimulus. While governed by those dynamics, neural responses may nevertheless unfold differently for nominally identical trials, rendering many traditional analysis methods ineffective. We review recent studies - some employing simultaneous recording, some not - indicating that such variability is indeed present both during movement generation and during the preceding premotor computations. In such cases, large-scale simultaneous recordings have the potential to provide an unprecedented view of neural dynamics at the level of single trials. However, this enterprise will depend not only on techniques for simultaneous recording but also on the use and further development of analysis techniques that can appropriately reduce the dimensionality of the data, and allow visualization of single-trial neural behavior.