Information capacity and robustness of encoding in the medial prefrontal cortex are modulated by the bioavailability of serotonin and the time elapsed from the cue during a reward-driven task.

Serotonin (5-HT) is a key neuromodulator of medial prefrontal cortex (mPFC) functions. Pharmacological manipulation of systemic 5-HT bioavailability alters the electrical activity of mPFC neurons. However, 5-HT modulation at the population level is not well characterized. In the present study, we made single neuron extracellular recordings in the mPFC of rats performing an operant conditioning task, and analyzed the effect of systemic administration of fluoxetine (a selective serotonin reuptake inhibitor) on the information encoded in the firing activity of the neural population. Chronic (longer than 15 days), but not acute (less than 15 days), fluoxetine administration reduced the firing rate of mPFC neurons. Moreover, fluoxetine treatment enhanced pairwise entropy but diminished noise correlation and redundancy in the information encoded, thus showing how mPFC differentially encodes information as a function of 5-HT bioavailability. Information about the occurrence of the reward-predictive stimulus was maximized during reward consumption, around 3 to 4 s after the presentation of the cue, and it was higher under chronic fluoxetine treatment. However, the encoded information was less robust to noise corruption when compared to control conditions.

Probing the structure-function relationship with neural networks constructed by solving a system of linear equations.

Neural network models are an invaluable tool to understand brain function since they allow us to connect the cellular and circuit levels with behaviour. Neural networks usually comprise a huge number of parameters, which must be chosen carefully such that networks reproduce anatomical, behavioural, and neurophysiological data. These parameters are usually fitted with off-the-shelf optimization algorithms that iteratively change network parameters and simulate the network to evaluate its performance and improve fitting. Here we propose to invert the fitting process by proceeding from the network dynamics towards network parameters. Firing state transitions are chosen according to the transition graph associated with the solution of a task. Then, a system of linear equations is constructed from the network firing states and membrane potentials, in a way that guarantees the consistency of the system. This allows us to uncouple the dynamical features of the model, like its neurons firing rate and correlation, from the structural features, and the task-solving algorithm implemented by the network. We employed our method to probe the structure-function relationship in a sequence memory task. The networks obtained showed connectivity and firing statistics that recapitulated experimental observations. We argue that the proposed method is a complementary and needed alternative to the way neural networks are constructed to model brain function.

Serotonergic modulation of basolateral amygdala nucleus in the extinction of reward-driven learning: The role of 5-HT bioavailability and 5-HT1A receptor.

Serotonin (5-HT) neurotransmission has been associated with reward-related behaviour. Moreover, the serotonergic system modulates the basolateral amygdala (BLA), a structure involved in reward encoding, and reward prediction error. However, the role played by 5-HT on BLA during a reward-driven task has not been fully elucidated. In this paper, we investigated whether serotonergic modulation of the BLA is involved in reward-driven learning. To this end, we trained Long Evans rats in an operant conditioning task, and examined the effects of fluoxetine treatment (a selective serotonin reuptake inhibitor, 10 mg/kg) in combination with BLA lesions with NMDA (20 mg/mL) on extinction learning. We also investigated whether intra-BLA injection of the serotonergic 5-HT1A receptor agonist 8-OH DPAT, or antagonist WAY-100635, alters extinction performance. We found that fluoxetine treatment strongly accelerated extinction learning, while BLA lesions partially reverted this effect and slightly impaired consolidation of extinction. Stimulation and inhibition of 5-HT1A receptors in BLA induced opposite effects to those of fluoxetine, impairing or accelerating extinction performance, respectively. Our findings suggest that 5-HT modulates reward-driven learning, and 5-HT1A receptors located in the BLA are relevant for extinction.

A Computational Theory for the Emergence of Grammatical Categories in Cortical Dynamics.

A general agreement in psycholinguistics claims that syntax and meaning are unified precisely and very quickly during online sentence processing. Although several theories have advanced arguments regarding the neurocomputational bases of this phenomenon, we argue that these theories could potentially benefit by including neurophysiological data concerning cortical dynamics constraints in brain tissue. In addition, some theories promote the integration of complex optimization methods in neural tissue. In this paper we attempt to fill these gaps introducing a computational model inspired in the dynamics of cortical tissue. In our modeling approach, proximal afferent dendrites produce stochastic cellular activations, while distal dendritic branches-on the other hand-contribute independently to somatic depolarization by means of dendritic spikes, and finally, prediction failures produce massive firing events preventing formation of sparse distributed representations. The model presented in this paper combines semantic and coarse-grained syntactic constraints for each word in a sentence context until grammatically related word function discrimination emerges spontaneously by the sole correlation of lexical information from different sources without applying complex optimization methods. By means of support vector machine techniques, we show that the sparse activation features returned by our approach are well suited-bootstrapping from the features returned by Word Embedding mechanisms-to accomplish grammatical function classification of individual words in a sentence. In this way we develop a biologically guided computational explanation for linguistically relevant unification processes in cortex which connects psycholinguistics to neurobiological accounts of language. We also claim that the computational hypotheses established in this research could foster future work on biologically-inspired learning algorithms for natural language processing applications.

Phonetic acquisition in cortical dynamics, a computational approach.

Many computational theories have been developed to improve artificial phonetic classification performance from linguistic auditory streams. However, less attention has been given to psycholinguistic data and neurophysiological features recently found in cortical tissue. We focus on a context in which basic linguistic units-such as phonemes-are extracted and robustly classified by humans and other animals from complex acoustic streams in speech data. We are especially motivated by the fact that 8-month-old human infants can accomplish segmentation of words from fluent audio streams based exclusively on the statistical relationships between neighboring speech sounds without any kind of supervision. In this paper, we introduce a biologically inspired and fully unsupervised neurocomputational approach that incorporates key neurophysiological and anatomical cortical properties, including columnar organization, spontaneous micro-columnar formation, adaptation to contextual activations and Sparse Distributed Representations (SDRs) produced by means of partial N-Methyl-D-aspartic acid (NMDA) depolarization. Its feature abstraction capabilities show promising phonetic invariance and generalization attributes. Our model improves the performance of a Support Vector Machine (SVM) classifier for monosyllabic, disyllabic and trisyllabic word classification tasks in the presence of environmental disturbances such as white noise, reverberation, and pitch and voice variations. Furthermore, our approach emphasizes potential self-organizing cortical principles achieving improvement without any kind of optimization guidance which could minimize hypothetical loss functions by means of-for example-backpropagation. Thus, our computational model outperforms multiresolution spectro-temporal auditory feature representations using only the statistical sequential structure immerse in the phonotactic rules of the input stream.

High mutual cooperation rates in rats learning reciprocal altruism: The role of payoff matrix.

Cooperation is one of the most studied paradigms for the understanding of social interactions. Reciprocal altruism -a special type of cooperation that is taught by means of the iterated prisoner dilemma game (iPD)- has been shown to emerge in different species with different success rates. When playing iPD against a reciprocal opponent, the larger theoretical long-term reward is delivered when both players cooperate mutually. In this work, we trained rats in iPD against an opponent playing a Tit for Tat strategy, using a payoff matrix with positive and negative reinforcements, that is food and timeout respectively. We showed for the first time, that experimental rats were able to learn reciprocal altruism with a high average cooperation rate, where the most probable state was mutual cooperation (85%). Although when subjects defected, the most probable behavior was to go back to mutual cooperation. When we modified the matrix by increasing temptation rewards (T) or by increasing cooperation rewards (R), the cooperation rate decreased. In conclusion, we observe that an iPD matrix with large positive reward improves less cooperation than one with small rewards, shown that satisfying the relationship among iPD reinforcement was not enough to achieve high mutual cooperation behavior. Therefore, using positive and negative reinforcements and an appropriate contrast between rewards, rats have cognitive capacity to learn reciprocal altruism. This finding allows to infer that the learning of reciprocal altruism has early appeared in evolution.

Putative dopamine neurons in the ventral tegmental area enhance information coding in the prefrontal cortex.

It has been proposed that neuronal populations in the prefrontal cortex (PFC) robustly encode task-relevant information through an interplay with the ventral tegmental area (VTA). Yet, the precise computation underlying such functional interaction remains elusive. Here, we conducted simultaneous recordings of single-unit activity in PFC and VTA of rats performing a GO/NoGO task. We found that mutual information between stimuli and neural activity increases in the PFC as soon as stimuli are presented. Notably, it is the activity of putative dopamine neurons in the VTA that contributes critically to enhance information coding in the PFC. The higher the activity of these VTA neurons, the better the conditioned stimuli are encoded in the PFC.

A spherical treadmill system to train head-fixed adult rats.

BACKGROUND: While spherical treadmills are widely used in mouse models, there are only a few experimental setups suitable for adult rats, and none of them include head-fixation. NEW METHOD: We introduce a novel spherical treadmill apparatus for head-fixed rats that allows a wide repertory of natural responses. The rat is secured to a frame and placed on a freely rotating sphere. While being head-fixed, it can walk in any direction and perform different motor tasks. COMPARISON WITH EXISTING METHODS: Instead of being air-lifted, which is acceptable for light animals, the treadmill is sustained by three spherical bearings ensuring a smooth rotation in any direction. Movement detection is accomplished using a video camera that registers a dot pattern plotted on the sphere. RESULTS: Long Evans rats were trained to perform an auditory discrimination task in a Go/No-Go (walking/not-walking) paradigm. Animals were able to successfully discriminate between a 1 kHz and a 8 kHz auditory stimulus and execute the correct response, reaching the learning criterion (80% of correct responses) in approximately 20 training sessions. CONCLUSIONS: Our system broadens the possibilities of head-fixation experiments in adult rats making them compatible with spatial navigation on a spherical treadmill.

Efficient enhancement of information in the prefrontal cortex during the presence of reward predicting stimuli.

The prefrontal cortex (PFC) is a key brain structure for decision making, behavioural flexibility and working memory. Neurons in PFC encode relevant stimuli through changes in their firing rate, although the metabolic cost of spiking activity puts strong constrains to neural codes based on firing rate modulation. Thus, how PFC neural populations code relevant information in an efficient way is not clearly understood. To address this issue we made single unit recordings in the PFC of rats performing a GO/NOGO discrimination task and analysed how entropy between pairs of neurons changes during cue presentation. We found that entropy rises only during reward-predicting cues. Moreover, this change in entropy occurred along an increase in the efficiency of the whole process. We studied possible mechanisms behind the efficient gain in entropy by means of a two neuron leaky integrate-and-fire model, and found that a precise relationship between synaptic efficacy and firing rate is required to explain the experimentally observed results.

Exploring the limits of learning: Segregation of information integration and response selection is required for learning a serial reversal task.

Animals are proposed to learn the latent rules governing their environment in order to maximize their chances of survival. However, rules may change without notice, forcing animals to keep a memory of which one is currently at work. Rule switching can lead to situations in which the same stimulus/response pairing is positively and negatively rewarded in the long run, depending on variables that are not accessible to the animal. This fact raises questions on how neural systems are capable of reinforcement learning in environments where the reinforcement is inconsistent. Here we address this issue by asking about which aspects of connectivity, neural excitability and synaptic plasticity are key for a very general, stochastic spiking neural network model to solve a task in which rules change without being cued, taking the serial reversal task (SRT) as paradigm. Contrary to what could be expected, we found strong limitations for biologically plausible networks to solve the SRT. Especially, we proved that no network of neurons can learn a SRT if it is a single neural population that integrates stimuli information and at the same time is responsible of choosing the behavioural response. This limitation is independent of the number of neurons, neuronal dynamics or plasticity rules, and arises from the fact that plasticity is locally computed at each synapse, and that synaptic changes and neuronal activity are mutually dependent processes. We propose and characterize a spiking neural network model that solves the SRT, which relies on separating the functions of stimuli integration and response selection. The model suggests that experimental efforts to understand neural function should focus on the characterization of neural circuits according to their connectivity, neural dynamics, and the degree of modulation of synaptic plasticity with reward.

Dopamine bioavailability in the mPFC modulates operant learning performance in rats: an experimental study with a computational interpretation.

Dopamine encodes reward and its prediction in reinforcement learning. Catechol-O-methyltransferase (COMT) activity in the medial prefrontal cortex (mPFC) has been shown to influence cognitive abilities by modifying dopamine clearance. Nevertheless, it is unknown how COMT in the mPFC influences operant learning. Systemic entacapone (50mg/kg), as well as local entacapone (3 pg) and recombinant COMT (17 µg) in the mPFC were administered to male Long Evans rats prior to training in an operant conditioning task. We found that systemic and local administration of the COMT inhibitor entacapone significantly improves learning performance. Conversely, recombinant COMT administration totally impaired learning. These data have been interpreted through a computational model where the phasic firing of dopaminergic neurons was computed by means of a temporal difference algorithm and dopamine bioavailability in the mPFC was simulated with a gating window. The duration of this window was selected to simulate the effects of inhibited or enhanced COMT activity (by entacapone or recombinant COMT respectively). The model accounts for an improved performance reproducing the entacapone effects, and a detrimental impact on learning when the clearance is increased reproducing the recombinant COMT effects. The experimental and computational results show that learning performance can be deeply influenced by COMT manipulations in the mPFC.

A computational theory for the learning of equivalence relations.

Equivalence relations (ERs) are logical entities that emerge concurrently with the development of language capabilities. In this work we propose a computational model that learns to build ERs by learning simple conditional rules. The model includes visual areas, dopaminergic, and noradrenergic structures as well as prefrontal and motor areas, each of them modeled as a group of continuous valued units that simulate clusters of real neurons. In the model, lateral interaction between neurons of visual structures and top-down modulation of prefrontal/premotor structures over the activity of neurons in visual structures are necessary conditions for learning the paradigm. In terms of the number of neurons and their interaction, we show that a minimal structural complexity is required for learning ERs among conditioned stimuli. Paradoxically, the emergence of the ER drives a reduction in the number of neurons needed to maintain those previously specific stimulus-response learned rules, allowing an efficient use of neuronal resources.

Blood pressure long term regulation: a neural network model of the set point development.

BACKGROUND: The notion of the nucleus tractus solitarius (NTS) as a comparator evaluating the error signal between its rostral neural structures (RNS) and the cardiovascular receptor afferents into it has been recently presented. From this perspective, stress can cause hypertension via set point changes, so offering an answer to an old question. Even though the local blood flow to tissues is influenced by circulating vasoactive hormones and also by local factors, there is yet significant sympathetic control. It is well established that the state of maturation of sympathetic innervation of blood vessels at birth varies across animal species and it takes place mostly during the postnatal period. During ontogeny, chemoreceptors are functional; they discharge when the partial pressures of oxygen and carbon dioxide in the arterial blood are not normal. METHODS: The model is a simple biological plausible adaptative neural network to simulate the development of the sympathetic nervous control. It is hypothesized that during ontogeny, from the RNS afferents to the NTS, the optimal level of each sympathetic efferent discharge is learned through the chemoreceptors' feedback. Its mean discharge leads to normal oxygen and carbon dioxide levels in each tissue. Thus, the sympathetic efferent discharge sets at the optimal level if, despite maximal drift, the local blood flow is compensated for by autoregulation. Such optimal level produces minimum chemoreceptor output, which must be maintained by the nervous system. Since blood flow is controlled by arterial blood pressure, the long-term mean level is stabilized to regulate oxygen and carbon dioxide levels. After development, the cardiopulmonary reflexes play an important role in controlling efferent sympathetic nerve activity to the kidneys and modulating sodium and water excretion. RESULTS: Starting from fixed RNS afferents to the NTS and random synaptic weight values, the sympathetic efferents converged to the optimal values. When learning was completed, the output from the chemoreceptors became zero because the sympathetic efferents led to normal partial pressures of oxygen and carbon dioxide. CONCLUSIONS: We introduce here a simple simulating computational theory to study, from a neurophysiologic point of view, the sympathetic development of cardiovascular regulation due to feedback signals sent off by cardiovascular receptors. The model simulates, too, how the NTS, as emergent property, acts as a comparator and how its rostral afferents behave as set point.

A theory about a role of the hyper direct pathway in pattern expression by the basal ganglia.

The Basal Ganglia (BG) are a group of nuclei, in the brain of mammalians and other vertebrates, strongly connected with the cerebral cortex, thalamus and other brain areas. The BG are associated with several brain functions including learning and motor control. When there is cortical activation, there is a strong synchronization between BG and cortex, i.e. when a given task is being executed or in the case of Parkinson disease[1], [2]. If we consider the internal segment of the Globus Pallidus (GPi) there is synchronism between GPi-cortex at frequencies as low as 3Hz to as high as 85Hz [1], [3]. In the other hand, in a delta sleep or in an anesthetized case, a very low frequency correlation is observed (1-10 Hz), but no high frequency correlation between GPi-cortex [1], [2], [3]. It is unknown why this decorrelation happens. But It is agreement that when there is no pattern to select, like in delta sleep or with an anesthetized model, the BG network would maintain the GPi and cortex decorrelated at high frequencies. Many thalamus-BG and thalamus-BG-cortex loops are modulators of the BG activity. Particularly there exists an anatomic thalamus-BG loop, formed by GPi, intralaminar thalamic nuclei (IL) and Subthalamic Nucleus (STN) [4]. Using a computational model, based on an "Integrate and Fire" neural network, we analyzed the IL nucleus as a modulator of the so-called hyper direct pathway. Our results show that, in an anesthetic case, this thalamic path could be relevant to allow a high frequency decorrelated state between the GPi and cortex.

Neural set point for the control of arterial pressure: role of the nucleus tractus solitarius.

BACKGROUND: Physiological experiments have shown that the mean arterial blood pressure (MAP) can not be regulated after chemo and cardiopulmonary receptor denervation. Neuro-physiological information suggests that the nucleus tractus solitarius (NTS) is the only structure that receives information from its rostral neural nuclei and from the cardiovascular receptors and projects to nuclei that regulate the circulatory variables. METHODS: From a control theory perspective, to answer if the cardiovascular regulation has a set point, we should find out whether in the cardiovascular control there is something equivalent to a comparator evaluating the error signal (between the rostral projections to the NTS and the feedback inputs). The NTS would function as a comparator if: a) its lesion suppresses cardiovascular regulation; b) the negative feedback loop still responds normally to perturbations (such as mechanical or electrical) after cutting the rostral afferent fibers to the NTS; c) perturbation of rostral neural structures (RNS) to the NTS modifies the set point without changing the dynamics of the elicited response; and d) cardiovascular responses to perturbations on neural structures within the negative feedback loop compensate for much faster than perturbations on the NTS rostral structures. RESULTS: From the control theory framework, experimental evidence found currently in the literature plus experimental results from our group was put together showing that the above-mentioned conditions (to show that the NTS functions as a comparator) are satisfied. CONCLUSIONS: Physiological experiments suggest that long-term blood pressure is regulated by the nervous system. The NTS functions as a comparator (evaluating the error signal) between its RNS and the cardiovascular receptor afferents and projects to nuclei that regulate the circulatory variables. The mean arterial pressure (MAP) is regulated by the feedback of chemo and cardiopulmonary receptors and the baroreflex would stabilize the short term pressure value to the prevailing carotid MAP. The discharge rates of rostral neural projections to the NTS would function as the set point of the closed and open loops of cardiovascular control. No doubt, then, the RNS play a functional role not only under steady-state conditions, but also in different behaviors and pathologies.

Bang-bang control of feeding: role of hypothalamic and satiety signals.

Rats, people, and many other omnivores eat in meals rather than continuously. We show by experimental test that eating in meals is regulated by a simple bang-bang control system, an idea foreshadowed by Le Magnen and many others, shown by us to account for a wide range of behavioral data, but never explicitly tested or tied to neurophysiological facts. The hypothesis is simply that the tendency to eat rises with time at a rate determined by satiety signals. When these signals fall below a set point, eating begins, in on-off fashion. The delayed sequelae of eating increment the satiety signals, which eventually turn eating off. Thus, under free conditions, the organism eats in bouts separated by noneating activities. We report an experiment with rats to test novel predictions about meal patterns that are not explained by existing homeostatic approaches. Access to food was systematically but unpredictably interrupted just as the animal tried to start a new meal. A simple bang-bang model fits the resulting meal-pattern data well, and its elements can be identified with neurophysiological processes. Hypothalamic inputs can provide the set point for longer-term regulation carried out by a comparator in the hindbrain. Delayed gustatory and gastrointestinal aftereffects of eating act via the nucleus of the solitary tract and other hindbrain regions as neural feedback governing short-term regulation. In this way, the model forges real links between a functioning feedback mechanism, neuro-hormonal data, and both short-term (meals) and long-term (eating-rate regulation) behavioral data.