An open-source behavior controller for associative learning and memory (B-CALM).

Associative learning and memory, i.e., learning and remembering the associations between environmental stimuli, self-generated actions, and outcomes such as rewards or punishments, are critical for the well-being of animals. Hence, the neural mechanisms underlying these processes are extensively studied using behavioral tasks in laboratory animals. Traditionally, these tasks have been controlled using commercial hardware and software, which limits scalability and accessibility due to their cost. More recently, due to the revolution in microcontrollers or microcomputers, several general-purpose and open-source solutions have been advanced for controlling neuroscientific behavioral tasks. While these solutions have great strength due to their flexibility and general-purpose nature, for the same reasons, they suffer from some disadvantages including the need for considerable programming expertise, limited online visualization, or slower than optimal response latencies for any specific task. Here, to mitigate these concerns, we present an open-source behavior controller for associative learning and memory (B-CALM). B-CALM provides an integrated suite that can control a host of associative learning and memory behaviors. As proof of principle for its applicability, we show data from head-fixed mice learning Pavlovian conditioning, operant conditioning, discrimination learning, as well as a timing task and a choice task. These can be run directly from a user-friendly graphical user interface (GUI) written in MATLAB that controls many independently running Arduino Mega microcontrollers in parallel (one per behavior box). In sum, B-CALM will enable researchers to execute a wide variety of associative learning and memory tasks in a scalable, accurate, and user-friendly manner.

Few-shot learning: temporal scaling in behavioral and dopaminergic learning.

How do we learn associations in the world (e.g., between cues and rewards)? Cue-reward associative learning is controlled in the brain by mesolimbic dopamine1-4. It is widely believed that dopamine drives such learning by conveying a reward prediction error (RPE) in accordance with temporal difference reinforcement learning (TDRL) algorithms5. TDRL implementations are "trial-based": learning progresses sequentially across individual cue-outcome experiences. Accordingly, a foundational assumption-often considered a mere truism-is that the more cue-reward pairings one experiences, the more one learns this association. Here, we disprove this assumption, thereby falsifying a foundational principle of trial-based learning algorithms. Specifically, when a group of head-fixed mice received ten times fewer experiences over the same total time as another, a single experience produced as much learning as ten experiences in the other group. This quantitative scaling also holds for mesolimbic dopaminergic learning, with the increase in learning rate being so high that the group with fewer experiences exhibits dopaminergic learning in as few as four cue-reward experiences and behavioral learning in nine. An algorithm implementing reward-triggered retrospective learning explains these findings. The temporal scaling and few-shot learning observed here fundamentally changes our understanding of the neural algorithms of associative learning.

Mesolimbic dopamine release conveys causal associations.

Learning to predict rewards based on environmental cues is essential for survival. It is believed that animals learn to predict rewards by updating predictions whenever the outcome deviates from expectations, and that such reward prediction errors (RPEs) are signaled by the mesolimbic dopamine system-a key controller of learning. However, instead of learning prospective predictions from RPEs, animals can infer predictions by learning the retrospective cause of rewards. Hence, whether mesolimbic dopamine instead conveys a causal associative signal that sometimes resembles RPE remains unknown. We developed an algorithm for retrospective causal learning and found that mesolimbic dopamine release conveys causal associations but not RPE, thereby challenging the dominant theory of reward learning. Our results reshape the conceptual and biological framework for associative learning.

Serotonin receptors contribute to dopamine depression of lateral inhibition in the nucleus accumbens.

Dopamine modulation of nucleus accumbens (NAc) circuitry is central to theories of reward seeking and reinforcement learning. Despite decades of effort, the acute dopamine actions on the NAc microcircuitry remain puzzling. Here, we dissect out the direct actions of dopamine on lateral inhibition between medium spiny neurons (MSNs) in mouse brain slices and find that they are pathway specific. Dopamine potently depresses GABAergic transmission from presynaptic dopamine D2 receptor-expressing MSNs (D2-MSNs), whereas it potentiates transmission from presynaptic dopamine D1 receptor-expressing MSNs (D1-MSNs) onto other D1-MSNs. To our surprise, presynaptic D2 receptors mediate only half of the depression induced by endogenous and exogenous dopamine. Presynaptic serotonin 5-HT1B receptors are responsible for a significant component of dopamine-induced synaptic depression. This study clarifies the mechanistic understanding of dopamine actions in the NAc by showing pathway-specific modulation of lateral inhibition and involvement of D2 and 5-HT1B receptors in dopamine depression of D2-MSN synapses.

Striatal Local Circuitry: A New Framework for Lateral Inhibition.

This Perspective will examine the organization of intrastriatal circuitry, review recent findings in this area, and discuss how the pattern of connectivity between striatal neurons might give rise to the behaviorally observed synergism between the direct/indirect pathway neurons. The emphasis of this Perspective is on the underappreciated role of lateral inhibition between striatal projection cells in controlling neuronal firing and shaping the output of this circuit. We review some classic studies in combination with more recent anatomical and functional findings to lay out a framework for an updated model of the intrastriatal lateral inhibition, where we explore its contribution to the formation of functional units of processing and the integration and filtering of inputs to generate motor patterns and learned behaviors.

Cognitive and Behavioral Impairments Evoked by Low-Level Exposure to Tobacco Smoke Components: Comparison with Nicotine Alone.

Active maternal smoking has adverse effects on neurobehavioral development of the offspring, with nicotine (Nic) providing much of the underlying causative mechanism. To determine whether the lower exposures caused by second-hand smoke are deleterious, we administered tobacco smoke extract (TSE) to pregnant rats starting preconception and continued through the second postnatal week, corresponding to all 3 trimesters of fetal brain development. Dosing was adjusted to produce maternal plasma Nic concentrations encountered with second-hand smoke, an order of magnitude below those seen in active smokers. We then compared TSE effects to those of an equivalent dose of Nic alone, and to a 10-fold higher Nic dose. Gestational exposure to TSE and Nic significantly disrupted cognitive and behavioral function in behavioral tests given during adolescence and adulthood (postnatal weeks 4-40), producing hyperactivity, working memory deficits, and impairments in emotional processing, even at the low exposure levels corresponding to second-hand smoke. Although TSE effects were highly correlated with those of Nic, the effects of TSE were much larger than could be attributed to just the Nic in the mixture. Indeed, TSE effects more closely resembled those of the 10-fold higher Nic levels, but still exceeded their magnitude. In combination with our earlier findings, this study thus completes the chain of causation to prove that second-hand smoke exposure causes neurodevelopmental deficits, originating in disruption of neurodifferentiation, leading to miswiring of neuronal circuits, and as shown here, culminating in behavioral dysfunction. As low level exposure to Nic alone produced neurobehavioral teratology, 'harm reduction' Nic products do not abolish the potential for neurodevelopmental damage.

Role of nicotinic receptors in the lateral habenula in the attenuation of amphetamine-induced prepulse inhibition deficits of the acoustic startle response in rats.

RATIONALE: Prepulse inhibition (PPI) refers to the reduction of the startle response magnitude when a startling stimulus is closely preceded by a weak stimulus. PPI is commonly used to measure sensorimotor gating. In rats, the PPI reduction induced by the dopamine agonist apomorphine can be reversed by systemic administration of nicotine. A high concentration of nicotinic receptors is found in the lateral habenula (LHb), an epithalamic structure with efferent projections to brain regions involved in the modulation of PPI, which has been shown to regulate the activity of midbrain dopamine neurons. OBJECTIVES: The prospective role of nicotinic receptors in the LHb in the regulation of PPI was assessed in this study, using different pharmacological models of sensorimotor gating deficits. METHODS: Interactions between systemic amphetamine and haloperidol and intra-LHb infusions of mecamylamine (10 µg/side) or nicotine (30 µg/side) on PPI were analyzed in Experiments 1 and 2. Intra-LHb infusions of different nicotine doses (25, and 50 µg/side) and their interactions with systemic administration of amphetamine or dizocilpine on PPI were examined in Experiments 3 and 4. RESULTS: Infusions of nicotine into the LHb dose-dependently attenuated amphetamine-induced PPI deficits but had no effect on PPI disruptions caused by dizocilpine. Intra-LHb mecamylamine infusions did not affect PPI nor interact with dopaminergic manipulations. CONCLUSIONS: These results are congruent with previous reports of systemic nicotine effects on PPI, suggesting a role of the LHb in the attenuation of sensorimotor gating deficits caused by the hyperactivity of dopamine systems.

Effects of tobacco smoke constituents, anabasine and anatabine, on memory and attention in female rats.

Nicotine has been well characterized to improve memory and attention. Nicotine is the primary, but not only neuroactive compound in tobacco. Other tobacco constituents such as anabasine and anatabine also have agonist actions on nicotinic receptors. The current study investigated the effects of anabasine and anatabine on memory and attention. Adult female Sprague-Dawley rats were trained on a win-shift spatial working and reference memory task in the 16-arm radial maze or a visual signal detection operant task to test attention. Acute dose-effect functions of anabasine and anatabine over two orders of magnitude were evaluated for both tasks. In the radial-arm maze memory test, anabasine but not anatabine significantly reduced the memory impairment caused by the NMDA antagonist dizocilpine (MK-801). In the signal detection attentional task, anatabine but not anabasine significantly attenuated the attentional impairment caused by dizocilpine. These studies show that non-nicotine nicotinic agonists in tobacco, similar to nicotine, can significantly improve memory and attentional function. Both anabasine and anatabine produced cognitive improvement, but their effectiveness differed with regard to memory and attention. Follow-up studies with anabasine and anatabine are called for to determine their efficacy as therapeutics for memory and attentional dysfunction.

Decreasing nicotinic receptor activity and the spatial learning impairment caused by the NMDA glutamate antagonist dizocilpine in rats.

Nicotinic systems have been shown by a variety of studies to be involved in cognitive function. Nicotinic receptors have an inherent property to become desensitized after activation. The relative role of nicotinic receptor activation vs. net receptor inactivation by desensitization in the cognitive effects of nicotinic drugs remains to be fully understood. In these studies, we tested the effects of the α7 nicotinic receptor antagonist methyllycaconitine (MLA), the α4ß2 nicotinic receptor antagonist dihydro-ß-erythroidine (DHßE), the nonspecific nicotinic channel blocker mecamylamine and the α4ß2 nicotinic receptor desensitizing agent sazetidine-A on learning in a repeated acquisition test. Adult female Sprague-Dawley rats were trained on a repeated acquisition learning procedure in an 8-arm radial maze. MLA (1-4mg/kg), DHßE (1-4mg/kg), mecamylamine (0.125-0.5mg/kg) or sazetidine-A (1 and 3mg/kg) were administered in four different studies either alone or together with the NMDA glutamate antagonist dizocilpine (0.05 and 0.10mg/kg). MLA significantly counteracted the learning impairment caused by dizocilpine. The overall choice accuracy impairment caused by dizocilpine was significantly attenuated by co-administration of DHßE. Low doses of the non-specific nicotinic antagonist mecamylamine also reduced dizocilpine-induced repeated acquisition impairment. Sazetidine-A reversed the accuracy impairment caused by dizocilpine. These studies provide evidence that a net decrease in nicotinic receptor activity can improve learning by attenuating learning impairment induced by NMDA glutamate blockade. This adds to evidence in cognitive tests that nicotinic antagonists can improve cognitive function. Further research characterizing the efficacy and mechanisms underlying nicotinic antagonist and desensitization induced cognitive improvement is warranted.