Active intrinsic conductances in recurrent networks allow for long-lasting transients and sustained activity with realistic firing rates as well as robust plasticity.

Recurrent neural networks of spiking neurons can exhibit long lasting and even persistent activity. Such networks are often not robust and exhibit spike and firing rate statistics that are inconsistent with experimental observations. In order to overcome this problem most previous models had to assume that recurrent connections are dominated by slower NMDA type excitatory receptors. Usually, the single neurons within these networks are very simple leaky integrate and fire neurons or other low dimensional model neurons. However real neurons are much more complex, and exhibit a plethora of active conductances which are recruited both at the sub and supra threshold regimes. Here we show that by including a small number of additional active conductances we can produce recurrent networks that are both more robust and exhibit firing-rate statistics that are more consistent with experimental results. We show that this holds both for bi-stable recurrent networks, which are thought to underlie working memory and for slowly decaying networks which might underlie the estimation of interval timing. We also show that by including these conductances, such networks can be trained to using a simple learning rule to predict temporal intervals that are an order of magnitude larger than those that can be trained in networks of leaky integrate and fire neurons.

Persistent increased PKMζ in long-term and remote spatial memory.

PKMζ is an autonomously active PKC isoform that is thought to maintain both LTP and long-term memory. Whereas persistent increases in PKMζ protein sustain the kinase's action in LTP, the molecular mechanism for the persistent action of PKMζ during long-term memory has not been characterized. PKMζ inhibitors disrupt spatial memory when introduced into the dorsal hippocampus from 1day to 1month after training. Therefore, if the mechanisms of PKMζ's persistent action in LTP maintenance and long-term memory were similar, persistent increases in PKMζ would last for the duration of the memory, far longer than most other learning-induced gene products. Here we find that spatial conditioning by aversive active place avoidance or appetitive radial arm maze induces PKMζ increases in dorsal hippocampus that persist from 1day to 1month, coinciding with the strength and duration of memory retention. Suppressing the increase by intrahippocampal injections of PKMζ-antisense oligodeoxynucleotides prevents the formation of long-term memory. Thus, similar to LTP maintenance, the persistent increase in the amount of autonomously active PKMζ sustains the kinase's action during long-term and remote spatial memory maintenance.

On the origin of sensory errors: Contrast discrimination under temporal constraint.

Estimation of perceptual variables is imprecise and prone to errors. Although the properties of these perceptual errors are well characterized, the physiological basis for these errors is unknown. One previously proposed explanation for these errors is the trial-by-trial variability of the responses of sensory neurons that encode the percept. In order to test this hypothesis, we developed a mathematical formalism that allows us to find the statistical characteristics of the physiological system responsible for perceptual errors, as well as the time scale over which the visual information is integrated. Crucially, these characteristics can be estimated solely from a behavioral experiment performed here. We demonstrate that the physiological basis of perceptual error has a constant level of noise (i.e., independent of stimulus intensity and duration). By comparing these results to previous physiological measurements, we show that perceptual errors cannot be due to the variability during the encoding stage. We also find that the time window over which perceptual evidence is integrated lasts no more than ∼230 ms. Finally, we discuss sources of error that may be consistent with our behavioral measurements.

A Simple Network Architecture Accounts for Diverse Reward Time Responses in Primary Visual Cortex.

Many actions performed by animals and humans depend on an ability to learn, estimate, and produce temporal intervals of behavioral relevance. Exemplifying such learning of cued expectancies is the observation of reward-timing activity in the primary visual cortex (V1) of rodents, wherein neural responses to visual cues come to predict the time of future reward as behaviorally experienced in the past. These reward-timing responses exhibit significant heterogeneity in at least three qualitatively distinct classes: sustained increase or sustained decrease in firing rate until the time of expected reward, and a class of cells that reach a peak in firing at the expected delay. We elaborate upon our existing model by including inhibitory and excitatory units while imposing simple connectivity rules to demonstrate what role these inhibitory elements and the simple architectures play in sculpting the response dynamics of the network. We find that simply adding inhibition is not sufficient for obtaining the different distinct response classes, and that a broad distribution of inhibitory projections is necessary for obtaining peak-type responses. Furthermore, although changes in connection strength that modulate the effects of inhibition onto excitatory units have a strong impact on the firing rate profile of these peaked responses, the network exhibits robustness in its overall ability to predict the expected time of reward. Finally, we demonstrate how the magnitude of expected reward can be encoded at the expected delay in the network and how peaked responses express this reward expectancy. SIGNIFICANCE STATEMENT: Heterogeneity in single-neuron responses is a common feature of neuronal systems, although sometimes, in theoretical approaches, it is treated as a nuisance and seldom considered as conveying a different aspect of a signal. In this study, we focus on the heterogeneous responses in the primary visual cortex of rodents trained with a predictable delayed reward time. We describe under what conditions this heterogeneity can arise by self-organization, and what information it can convey. This study, while focusing on a specific system, provides insight onto how heterogeneity can arise in general while also shedding light onto mechanisms of reinforcement learning using realistic biological assumptions.

Atypical PKCs in memory maintenance: the roles of feedback and redundancy.

Memories that last a lifetime are thought to be stored, at least in part, as persistent enhancement of the strength of particular synapses. The synaptic mechanism of these persistent changes, late long-term potentiation (L-LTP), depends on the state and number of specific synaptic proteins. Synaptic proteins, however, have limited dwell times due to molecular turnover and diffusion, leading to a fundamental question: how can this transient molecular machinery store memories lasting a lifetime? Because the persistent changes in efficacy are synapse-specific, the underlying molecular mechanisms must to a degree reside locally in synapses. Extensive experimental evidence points to atypical protein kinase C (aPKC) isoforms as key components involved in memory maintenance. Furthermore, it is evident that establishing long-term memory requires new protein synthesis. However, a comprehensive model has not been developed describing how these components work to preserve synaptic efficacies over time. We propose a molecular model that can account for key empirical properties of L-LTP, including its protein synthesis dependence, dependence on aPKCs, and synapse-specificity. Simulations and empirical data suggest that either of the two aPKC subtypes in hippocampal neurons, PKMζ and PKCι/λ, can maintain L-LTP, making the system more robust. Given genetic compensation at the level of synthesis of these PKC subtypes as in knockout mice, this system is able to maintain L-LTP and memory when one of the pathways is eliminated.

Networks that learn the precise timing of event sequences.

Neuronal circuits can learn and replay firing patterns evoked by sequences of sensory stimuli. After training, a brief cue can trigger a spatiotemporal pattern of neural activity similar to that evoked by a learned stimulus sequence. Network models show that such sequence learning can occur through the shaping of feedforward excitatory connectivity via long term plasticity. Previous models describe how event order can be learned, but they typically do not explain how precise timing can be recalled. We propose a mechanism for learning both the order and precise timing of event sequences. In our recurrent network model, long term plasticity leads to the learning of the sequence, while short term facilitation enables temporally precise replay of events. Learned synaptic weights between populations determine the time necessary for one population to activate another. Long term plasticity adjusts these weights so that the trained event times are matched during playback. While we chose short term facilitation as a time-tracking process, we also demonstrate that other mechanisms, such as spike rate adaptation, can fulfill this role. We also analyze the impact of trial-to-trial variability, showing how observational errors as well as neuronal noise result in variability in learned event times. The dynamics of the playback process determines how stochasticity is inherited in learned sequence timings. Future experiments that characterize such variability can therefore shed light on the neural mechanisms of sequence learning.

A biophysically-based neuromorphic model of spike rate- and timing-dependent plasticity.

Current advances in neuromorphic engineering have made it possible to emulate complex neuronal ion channel and intracellular ionic dynamics in real time using highly compact and power-efficient complementary metal-oxide-semiconductor (CMOS) analog very-large-scale-integrated circuit technology. Recently, there has been growing interest in the neuromorphic emulation of the spike-timing-dependent plasticity (STDP) Hebbian learning rule by phenomenological modeling using CMOS, memristor or other analog devices. Here, we propose a CMOS circuit implementation of a biophysically grounded neuromorphic (iono-neuromorphic) model of synaptic plasticity that is capable of capturing both the spike rate-dependent plasticity (SRDP, of the Bienenstock-Cooper-Munro or BCM type) and STDP rules. The iono-neuromorphic model reproduces bidirectional synaptic changes with NMDA receptor-dependent and intracellular calcium-mediated long-term potentiation or long-term depression assuming retrograde endocannabinoid signaling as a second coincidence detector. Changes in excitatory or inhibitory synaptic weights are registered and stored in a nonvolatile and compact digital format analogous to the discrete insertion and removal of AMPA or GABA receptor channels. The versatile Hebbian synapse device is applicable to a variety of neuroprosthesis, brain-machine interface, neurorobotics, neuromimetic computation, machine learning, and neural-inspired adaptive control problems.

KIBRA anchoring the action of PKMζ maintains the persistence of memory.

How can short-lived molecules selectively maintain the potentiation of activated synapses to sustain long-term memory? Here, we find kidney and brain expressed adaptor protein (KIBRA), a postsynaptic scaffolding protein genetically linked to human memory performance, complexes with protein kinase Mzeta (PKMζ), anchoring the kinase's potentiating action to maintain late-phase long-term potentiation (late-LTP) at activated synapses. Two structurally distinct antagonists of KIBRA-PKMζ dimerization disrupt established late-LTP and long-term spatial memory, yet neither measurably affects basal synaptic transmission. Neither antagonist affects PKMζ-independent LTP or memory that are maintained by compensating PKCs in ζ-knockout mice; thus, both agents require PKMζ for their effect. KIBRA-PKMζ complexes maintain 1-month-old memory despite PKMζ turnover. Therefore, it is not PKMζ alone, nor KIBRA alone, but the continual interaction between the two that maintains late-LTP and long-term memory.

Scaling of perceptual errors can predict the shape of neural tuning curves.

Weber's law, first characterized in the 19th century, states that errors estimating the magnitude of perceptual stimuli scale linearly with stimulus intensity. This linear relationship is found in most sensory modalities, generalizes to temporal interval estimation, and even applies to some abstract variables. Despite its generality and long experimental history, the neural basis of Weber's law remains unknown. This work presents a simple theory explaining the conditions under which Weber's law can result from neural variability and predicts that the tuning curves of neural populations which adhere to Weber's law will have a log-power form with parameters that depend on spike-count statistics. The prevalence of Weber's law suggests that it might be optimal in some sense. We examine this possibility, using variational calculus, and show that Weber's law is optimal only when observed real-world variables exhibit power-law statistics with a specific exponent. Our theory explains how physiology gives rise to the behaviorally characterized Weber's law and may represent a general governing principle relating perception to neural activity.

Learning to Express Reward Prediction Error-like Dopaminergic Activity Requires Plastic Representations of Time.

The dominant theoretical framework to account for reinforcement learning in the brain is temporal difference (TD) reinforcement learning. The TD framework predicts that some neuronal elements should represent the reward prediction error (RPE), which means they signal the difference between the expected future rewards and the actual rewards. The prominence of the TD theory arises from the observation that firing properties of dopaminergic neurons in the ventral tegmental area appear similar to those of RPE model-neurons in TD learning. Previous implementations of TD learning assume a fixed temporal basis for each stimulus that might eventually predict a reward. Here we show that such a fixed temporal basis is implausible and that certain predictions of TD learning are inconsistent with experiments. We propose instead an alternative theoretical framework, coined FLEX (Flexibly Learned Errors in Expected Reward). In FLEX, feature specific representations of time are learned, allowing for neural representations of stimuli to adjust their timing and relation to rewards in an online manner. In FLEX dopamine acts as an instructive signal which helps build temporal models of the environment. FLEX is a general theoretical framework that has many possible biophysical implementations. In order to show that FLEX is a feasible approach, we present a specific biophysically plausible model which implements the principles of FLEX. We show that this implementation can account for various reinforcement learning paradigms, and that its results and predictions are consistent with a preponderance of both existing and reanalyzed experimental data.

On the precision of quasi steady state assumptions in stochastic dynamics.

Many biochemical networks have complex multidimensional dynamics and there is a long history of methods that have been used for dimensionality reduction for such reaction networks. Usually a deterministic mass action approach is used; however, in small volumes, there are significant fluctuations from the mean which the mass action approach cannot capture. In such cases stochastic simulation methods should be used. In this paper, we evaluate the applicability of one such dimensionality reduction method, the quasi-steady state approximation (QSSA) [L. Menten and M. Michaelis, "Die kinetik der invertinwirkung," Biochem. Z 49, 333369 (1913)] for dimensionality reduction in case of stochastic dynamics. First, the applicability of QSSA approach is evaluated for a canonical system of enzyme reactions. Application of QSSA to such a reaction system in a deterministic setting leads to Michaelis-Menten reduced kinetics which can be used to derive the equilibrium concentrations of the reaction species. In the case of stochastic simulations, however, the steady state is characterized by fluctuations around the mean equilibrium concentration. Our analysis shows that a QSSA based approach for dimensionality reduction captures well the mean of the distribution as obtained from a full dimensional simulation but fails to accurately capture the distribution around that mean. Moreover, the QSSA approximation is not unique. We have then extended the analysis to a simple bistable biochemical network model proposed to account for the stability of synaptic efficacies; the substrate of learning and memory [J. E. Lisman, "A mechanism of memory storage insensitive to molecular turnover: A bistable autophosphorylating kinase," Proc. Natl. Acad. Sci. U.S.A. 82, 3055-3057 (1985)]. Our analysis shows that a QSSA based dimensionality reduction method results in errors as big as two orders of magnitude in predicting the residence times in the two stable states.

Learning reward timing in cortex through reward dependent expression of synaptic plasticity.

The ability to represent time is an essential component of cognition but its neural basis is unknown. Although extensively studied both behaviorally and electrophysiologically, a general theoretical framework describing the elementary neural mechanisms used by the brain to learn temporal representations is lacking. It is commonly believed that the underlying cellular mechanisms reside in high order cortical regions but recent studies show sustained neural activity in primary sensory cortices that can represent the timing of expected reward. Here, we show that local cortical networks can learn temporal representations through a simple framework predicated on reward dependent expression of synaptic plasticity. We assert that temporal representations are stored in the lateral synaptic connections between neurons and demonstrate that reward-modulated plasticity is sufficient to learn these representations. We implement our model numerically to explain reward-time learning in the primary visual cortex (V1), demonstrate experimental support, and suggest additional experimentally verifiable predictions.

Conditions for Synaptic Specificity during the Maintenance Phase of Synaptic Plasticity.

Activity-dependent modifications of synaptic efficacies are a cellular substrate of learning and memory. Experimental evidence shows that these modifications are synapse specific and that the long-lasting effects are associated with the sustained increase in concentration of specific proteins like PKMζ However, such proteins are likely to diffuse away from their initial synaptic location and spread out to neighboring synapses, potentially compromising synapse specificity. In this article, we address the issue of synapse specificity during memory maintenance. Assuming that the long-term maintenance of synaptic plasticity is accomplished by a molecular switch, we carry out analytical calculations and perform simulations using the reaction-diffusion package in NEURON to determine the limits of synapse specificity during maintenance. Moreover, we explore the effects of the diffusion and degradation rates of proteins and of the geometrical characteristics of dendritic spines on synapse specificity. We conclude that the necessary conditions for synaptic specificity during maintenance require that molecular switches reside in dendritic spines. The requirement for synaptic specificity when the molecular switch resides in spines still imposes strong limits on the diffusion and turnover of rates of maintenance molecules, as well as on the morphologic properties of synaptic spines. These constraints are quite general and apply to most existing models suggested for maintenance. The parameter values can be experimentally evaluated, and if they do not fit the appropriate predicted range, the validity of this class of maintenance models would be challenged.

A single spiking neuron that can represent interval timing: analysis, plasticity and multi-stability.

The ability to represent interval timing is crucial for many common behaviors, such as knowing whether to stop when the light turns from green to yellow. Neural representations of interval timing have been reported in the rat primary visual cortex and we have previously presented a computational framework describing how they can be learned by a network of neurons. Recent experimental and theoretical results in entorhinal cortex have shown that single neurons can exhibit persistent activity, previously thought to be generated by a network of neurons. Motivated by these single neuron results, we propose a single spiking neuron model that can learn to compute and represent interval timing. We show that a simple model, reduced analytically to a single dynamical equation, captures the average behavior of the complete high dimensional spiking model very well. Variants of this model can be used to produce bi-stable or multi-stable persistent activity. We also propose a plasticity rule by which this model can learn to represent different intervals and different levels of persistent activity.

A network of spiking neurons that can represent interval timing: mean field analysis.

Despite the vital importance of our ability to accurately process and encode temporal information, the underlying neural mechanisms are largely unknown. We have previously described a theoretical framework that explains how temporal representations, similar to those reported in the visual cortex, can form in locally recurrent cortical networks as a function of reward modulated synaptic plasticity. This framework allows networks of both linear and spiking neurons to learn the temporal interval between a stimulus and paired reward signal presented during training. Here we use a mean field approach to analyze the dynamics of non-linear stochastic spiking neurons in a network trained to encode specific time intervals. This analysis explains how recurrent excitatory feedback allows a network structure to encode temporal representations.

Behavioral Time Scale Plasticity of Place Fields: Mathematical Analysis.

Traditional synaptic plasticity experiments and models depend on tight temporal correlations between pre- and postsynaptic activity. These tight temporal correlations, on the order of tens of milliseconds, are incompatible with significantly longer behavioral time scales, and as such might not be able to account for plasticity induced by behavior. Indeed, recent findings in hippocampus suggest that rapid, bidirectional synaptic plasticity which modifies place fields in CA1 operates at behavioral time scales. These experimental results suggest that presynaptic activity generates synaptic eligibility traces both for potentiation and depression, which last on the order of seconds. These traces can be converted to changes in synaptic efficacies by the activation of an instructive signal that depends on naturally occurring or experimentally induced plateau potentials. We have developed a simple mathematical model that is consistent with these observations. This model can be fully analyzed to find the fixed points of induced place fields and how these fixed points depend on system parameters such as the size and shape of presynaptic place fields, the animal's velocity during induction, and the parameters of the plasticity rule. We also make predictions about the convergence time to these fixed points, both for induced and pre-existing place fields.

Learning precise spatiotemporal sequences via biophysically realistic learning rules in a modular, spiking network.

Multiple brain regions are able to learn and express temporal sequences, and this functionality is an essential component of learning and memory. We propose a substrate for such representations via a network model that learns and recalls discrete sequences of variable order and duration. The model consists of a network of spiking neurons placed in a modular microcolumn based architecture. Learning is performed via a biophysically realistic learning rule that depends on synaptic 'eligibility traces'. Before training, the network contains no memory of any particular sequence. After training, presentation of only the first element in that sequence is sufficient for the network to recall an entire learned representation of the sequence. An extended version of the model also demonstrates the ability to successfully learn and recall non-Markovian sequences. This model provides a possible framework for biologically plausible sequence learning and memory, in agreement with recent experimental results.

Translational switch for long-term maintenance of synaptic plasticity.

Memory can last a lifetime, yet synaptic contacts that contribute to the storage of memory are composed of proteins that have much shorter lifetimes. A physiological model of memory formation, long-term potentiation (LTP), has a late protein-synthesis-dependent phase (L-LTP) that can last for many hours in slices or even for days in vivo. Could the activity-dependent synthesis of new proteins account for the persistence of L-LTP and memory? Here, we examine the proposal that a self-sustaining regulation of translation can form a bistable switch that can persistently regulate the on-site synthesis of plasticity-related proteins. We show that an alpha CaMKII-CPEB1 molecular pair can operate as a bistable switch. Our results imply that L-LTP should produce an increase in the total amount of alpha CaMKII at potentiated synapses. This study also proposes an explanation for why the application of protein synthesis and alphaCaMKII inhibitors at the induction and maintenance phases of L-LTP result in very different outcomes.

Network dynamics of Broca's area during word selection.

Current models of word-production in Broca's area (i.e. left ventro-lateral prefrontal cortex, VLPFC) posit that sequential and staggered semantic, lexical, phonological and articulatory processes precede articulation. Using millisecond-resolution intra-cranial recordings, we evaluated spatiotemporal dynamics and high frequency functional interconnectivity between left VLPFC regions during single-word production. Through the systematic variation of retrieval, selection, and phonological loads, we identified specific activation profiles and functional coupling patterns between these regions that fit within current psycholinguistic theories of word production. However, network interactions underpinning these processes activate in parallel (not sequentially), while the processes themselves are indexed by specific changes in network state. We found evidence that suggests that pars orbitalis is coupled with pars triangularis during lexical retrieval, while lexical selection is terminated via coupled activity with M1 at articulation onset. Taken together, this work reveals that speech production relies on very specific inter-regional couplings in rapid sequence in the language dominant hemisphere.