An On-Demand Drug Delivery System for Control of Epileptiform Seizures.

Drug delivery systems have the potential to deliver high concentrations of drug to target areas on demand, while elsewhere and at other times encapsulating the drug, to limit unwanted actions. Here we show proof of concept in vivo and ex vivo tests of a novel drug delivery system based on hollow-gold nanoparticles tethered to liposomes (HGN-liposomes), which become transiently permeable when activated by optical or acoustic stimulation. We show that laser or ultrasound simulation of HGN-liposomes loaded with the GABAA receptor agonist, muscimol, triggers rapid and repeatable release in a sufficient concentration to inhibit neurons and suppress seizure activity. In particular, laser-stimulated release of muscimol from previously injected HGN-liposomes caused subsecond hyperpolarizations of the membrane potential of hippocampal pyramidal neurons, measured by whole cell intracellular recordings with patch electrodes. In hippocampal slices and hippocampal-entorhinal cortical wedges, seizure activity was immediately suppressed by muscimol release from HGN-liposomes triggered by laser or ultrasound pulses. After intravenous injection of HGN-liposomes in whole anesthetized rats, ultrasound stimulation applied to the brain through the dura attenuated the seizure activity induced by pentylenetetrazol. Ultrasound alone, or HGN-liposomes without ultrasound stimulation, had no effect. Intracerebrally-injected HGN-liposomes containing kainic acid retained their contents for at least one week, without damage to surrounding tissue. Thus, we demonstrate the feasibility of precise temporal control over exposure of neurons to the drug, potentially enabling therapeutic effects without continuous exposure. For future application, studies on the pharmacokinetics, pharmacodynamics, and toxicity of HGN-liposomes and their constituents, together with improved methods of targeting, are needed, to determine the utility and safety of the technology in humans.

Responses of putative medium spiny neurons and fast-spiking interneurons to reward-related sensory signals in Wistar and genetically hypertensive rats.

Medium spiny neurons (MSN) are the primary output neurons of the striatum. Their activity is modulated by exogenous afferents and local circuit inputs, including fast-spiking interneurons (FSI). Altered responses of MSN and FSI may account for altered reward-driven behaviour in hyperactive rat strains, such as the genetically hypertensive (GH) rat. To investigate whether striatal neuron responses differ between GH and Wistar rats, we recorded putative MSNs (pMSN) and FSI (pFSI) from freely moving GH and Wistar rats in a classically conditioned (Pavlovian) cue-reward association paradigm. Here, the same auditory cue signal predicted reward delivery in one block of trials, but was not followed by reward in another. The significance of the cue as a reward predictor was indicated during each block by an environmental context provided by the house light. The results showed that pMSN in GH rats, but not Wistar rats, were more sensitive to the auditory signal in the context indicating no-reward, than in the reward context. Such enhanced sensitivity to cues in a no-reward context may contribute to a specific deficit in instrumental behaviour seen in GH rats, which maintain higher levels of instrumental responding in a context that indicates responding will not be rewarded. In addition, pFSI also responded to auditory signals, but there was no significant effect of reward context. Surprisingly, given their known feed-forward role, pFSI responded at longer latency than pMSN, suggesting that relative timing of activity in the two populations may be task specific.

Altered Recruitment of Motor Cortex Neuronal Activity During the Grasping Phase of Skilled Reaching in a Chronic Rat Model of Unilateral Parkinsonism.

Parkinson's disease causes prominent difficulties in the generation and execution of voluntary limb movements, including regulation of distal muscles and coordination of proximal and distal movement components to achieve accurate grasping. Difficulties with manual dexterity have a major impact on activities of daily living. We used extracellular single neuron recordings to investigate the neural underpinnings of parkinsonian movement deficits in the motor cortex of chronic unilateral 6-hydroxydopamine lesion male rats performing a skilled reach-to-grasp task the. Both normal movements and parkinsonian deficits in this task have striking homology to human performance. In lesioned animals there were several differences in the activity of cortical neurons during reaches by the affected limb compared with control rats. These included an increase in proportions of neurons showing rate decreases, along with increased amplitude of their average rate-decrease response at specific times during the reach, suggesting a shift in the balance of net excitation and inhibition of cortical neurons; a significant increase in the duration of rate-increase responses, which could result from reduced coupling of cortical activity to specific movement components; and changes in the timing and incidence of neurons with pure rate-increase or biphasic responses, particularly at the end of reach when grasping would normally be occurring. The changes in cortical activity may account for the deficits that occur in skilled distal motor control following dopamine depletion, and highlight the need for treatment strategies targeted toward modulating cortical mechanisms for fine distal motor control in patients.SIGNIFICANCE STATEMENT We show for the first time in a chronic lesion rat model of Parkinson's disease movement deficits that there are specific changes in motor cortex neuron activity associated with the grasping phase of a skilled motor task. Such changes provide a possible mechanism underpinning the problems with manual dexterity seen in Parkinson's patients and highlight the need for treatment strategies targeted toward distal motor control.

Role of homeostatic feedback mechanisms in modulating methylphenidate actions on phasic dopamine signaling in the striatum of awake behaving rats.

Methylphenidate is an established treatment for attention-deficit hyperactivity disorder that also has abuse potential. Both properties may relate to blocking dopamine and norepinephrine reuptake. We measured the effects of methylphenidate on dopamine dynamics in freely moving rats. Methylphenidate alone had no effect on the amplitude of phasic responses to cues or reward. However, when administered with the D2 receptor antagonist raclopride, methylphenidate increased dopamine responses, while raclopride alone had no effect. Using brain slices of substantia nigra or striatum, we confirmed that methylphenidate effects on firing rate of nigral dopamine neurons and dopamine release from terminals are constrained by negative feedback. A computational model using physiologically relevant parameters revealed that actions of methylphenidate on norepinephrine and dopamine transporters, and the effects of changes in tonic dopamine levels on D2 receptors, are necessary and sufficient to account for the experimental findings. In addition, non-linear fitting of the model to the data from freely moving animals revealed that methylphenidate significantly slowed the initial cue response dynamics. These results show that homeostatic regulation of dopamine release in the face of changing tonic levels of extracellular dopamine should be taken into account to understand the therapeutic benefits and abuse potential of methylphenidate.

Dopamine release in mushroom bodies of the honey bee (Apis mellifera L.) in response to aversive stimulation.

In Drosophila melanogaster, aversive (electric shock) stimuli have been shown to activate subpopulations of dopaminergic neurons with terminals in the mushroom bodies (MBs) of the brain. While there is compelling evidence that dopamine (DA)-induced synaptic plasticity underpins the formation of aversive memories in insects, the mechanisms involved have yet to be fully resolved. Here we take advantage of the accessibility of MBs in the brain of the honey bee to examine, using fast scan cyclic voltammetry, the kinetics of DA release and reuptake in vivo in response to electric shock, and to investigate factors that modulate the release of this amine. DA increased transiently in the MBs in response to electric shock stimuli. The magnitude of release varied depending on stimulus duration and intensity, and a strong correlation was identified between DA release and the intensity of behavioural responses to shock. With repeated stimulation, peak DA levels increased. However, the amount of DA released on the first stimulation pulse typically exceeded that evoked by subsequent pulses. No signal was detected in response to odour alone. Interestingly, however, if odour presentation was paired with electric shock, DA release was enhanced. These results set the stage for analysing the mechanisms that modulate DA release in the MBs of the bee.

Complex Multiplexing of Reward-Cue- and Licking-Movement-Related Activity in Single Midline Thalamus Neurons.

Midline thalamus is implicated in linking visceral and exteroceptive sensory information with behavior. However, whether neuronal activity is modulated with temporal precision by cues and actions in real time is unknown. Using single-neuron recording and a Pavlovian visual-cue/liquid-reward association task in rats, we discovered phasic responses to sensory cues, appropriately timed to modify information processing in output targets, as well as tonic modulations within and between trials that were differentially reward modulated, which may have distinct arousal functions. Many of the cue-responsive neurons also responded to repetitive licks, consistent with sensorimotor integration. Further, some lick-related neurons were activated only by the first rewarded lick and only if that lick were also part of a conditioned response sequence initiated earlier, consistent with binding action decisions to their ensuing outcome. This rich repertoire of responses provides electrophysiological evidence for midline thalamus as a site of complex information integration for reward-mediated behavior. SIGNIFICANCE STATEMENT: Disparate brain circuits are involved in sensation, movement, and reward information. These must interact in order for the relationships between cues, actions, and outcomes to be learned. We found that responses of single neurons in midline thalamus to sensory cues are increased when associated with reward. This output may amplify similar signals generated in parallel by the dopamine system. In addition, some neurons coded a three-factor decision in which the neuron fired only if there was a movement, if it was the first one after the reward becoming available, and if it was part of a sequence triggered in response to a preceding cue. These data highlight midline thalamus as an important node integrating multiple types of information for linking sensation, actions, and rewards.

Oxytocin excites nucleus accumbens shell neurons in vivo.

Oxytocin modulates reward-related behaviors. The nucleus accumbens shell (NAcSh) is a major relay in the brain reward pathway and expresses oxytocin receptors, but the effects of oxytocin on the activity of NAcSh neurons in vivo are unknown. Hence, we used in vivo extracellular recording to show that intracerebroventricular (ICV) oxytocin administration (0.2µg) robustly increased medial NAcSh neuron mean firing rate; this increase was almost exclusively evident in slow-firing neurons and was not associated with any change in firing pattern. To determine whether oxytocin excitation of medial NAcSh neurons is modulated by drugs that impact the brain reward pathway, we next tested the effects of ICV oxytocin following repeated morphine treatment. In morphine-treated rats, ICV oxytocin did not affect the mean firing rate of medial NAcSh neurons. Taken together, these results show that oxytocin excites medial NAcSh neurons but does not do so after repeated morphine. This could be an important factor in oxytocin modulation of reward-related behaviors, such as drug addiction.

Deficit in late-stage contingent negative variation provides evidence for disrupted movement preparation in patients with conversion paresis.

Conversion paresis is the presence of unexplained weakness without detectable neuropathology that is not feigned. To examine the 'abnormal preparation' and 'disrupted execution' hypotheses proposed to explain the movement deficits in conversion paresis, electroencephalographic, electromyographic and kinematic measures were recorded during motor preparation and execution. Six patients with unilateral upper limb conversion weakness, 24 participants feigning weakness and 12 control participants performed a 2-choice precued reaction time task. Precues provided advance information about the responding hand or finger. Patients and feigners demonstrated similar diminished force, longer movement time and extended duration of muscle activity in their symptomatic limb. Patients showed significantly suppressed contingent negative variation (CNV) amplitudes, but only when the symptomatic limb was precued. Despite the similarity in performance measures, this CNV suppression was not seen in feigners. Diminished CNV for symptomatic hand precues may reflect engagement of an inhibitory mechanism suppressing cortical activity related to preparatory processes.

Oxytocin enhances the expression of morphine-induced conditioned place preference in rats.

Drug addiction is characterized by drug-seeking and drug-taking and has devastating consequences on addicts as well as on society. Environmental contexts previously associated with drug use can elicit continued drug use and facilitate relapse. Accumulating evidence suggests that the neuropeptide oxytocin might be a potential treatment for behavioral disorders, including drug addiction. Here, we investigated the effects of central oxytocin administration on the acquisition and expression of morphine-induced conditioned place preference (CPP), a model for measuring the rewarding effects of drugs of abuse, in male Wistar rats. Intracerebroventricular (ICV) administration of oxytocin (0.2µg) or the specific oxytocin receptor antagonist (OTA), desGly-NH2, d(CH2)5[Tyr(Me)(2), Thr(4)] OVT, (0.75µg), on the conditioning days did not affect the acquisition of morphine-induced CPP. By contrast, ICV oxytocin, but not OTA, administration immediately prior to the post-conditioning session enhanced the expression of morphine-induced CPP, possibly by activation of oxytocin receptors in the nucleus accumbens shell (NAcSh). The oxytocin enhancement of morphine-induced CPP was not associated with any changes in the locomotor activity of morphine-conditioned rats. Together, these data suggest that central administration of exogenous oxytocin enhances the expression of morphine-induced CPP, at least in part, via activation of oxytocin receptors within the NAcSh.

Patterned, but not tonic, optogenetic stimulation in motor thalamus improves reaching in acute drug-induced Parkinsonian rats.

High-frequency deep brain stimulation (DBS) in motor thalamus (Mthal) ameliorates tremor but not akinesia in Parkinson's disease. The aim of this study was to investigate whether there are effective methods of Mthal stimulation to treat akinesia. Glutamatergic Mthal neurons, transduced with channelrhodopsin-2 by injection of lentiviral vector (Lenti.CaMKII.hChR2(H134R).mCherry), were selectively stimulated with blue light (473 nm) via a chronically implanted fiber-optic probe. Rats performed a reach-to-grasp task in either acute drug-induced parkinsonian akinesia (0.03-0.07 mg/kg haloperidol, s.c.) or control (vehicle injection) conditions, and the number of reaches was recorded for 5 min before, during, and after stimulation. We compared the effect of DBS using complex physiological patterns previously recorded in the Mthal of a control rat during reaching or exploring behavior, with tonic DBS delivering the same number of stimuli per second (rate-control 6.2 or 1.8 Hz, respectively) and with stimulation patterns commonly used in other brain regions to treat neurological conditions (tonic 130 Hz, theta burst (TBS), and tonic 15 Hz rate-control for TBS). Control rats typically executed >150 reaches per 5 min, which was unaffected by any of the stimulation patterns. Acute parkinsonian rats executed <20 reaches, displaying marked akinesia, which was significantly improved by stimulating with the physiological reaching pattern or TBS (both p < 0.05), whereas the exploring and all tonic patterns failed to improve reaching. Data indicate that the Mthal may be an effective site to treat akinesia, but the pattern of stimulation is critical for improving reaching in parkinsonian rats.

Reduced reach-related modulation of motor thalamus neural activity in a rat model of Parkinson's disease.

Motor thalamus (Mthal) is a key node in the corticobasal ganglia (BG) loop that controls complex, cognitive aspects of movement. In Parkinson's disease (PD), profound alterations in neuronal activity occur in BG nuclei and cortex. Because Mthal is located between these two structures, altered Mthal activity has been assumed to underlie the pathogenesis of PD motor deficits. However, to date, inconsistent changes in neuronal firing rate and pattern have been reported in parkinsonian animals. Moreover, although a distinct firing pattern of Mthal neurons, called low-threshold calcium spike bursts (LTS bursts), is observed in reduced preparations, it remains unknown whether they occur or what their role might be in behaving animals. We recorded Mthal spiking activity in control and unilateral 6-hydroxydopamine lesioned rats performing a skilled forelimb-reaching task. We show for the first time that Mthal firing rate in control rats is modulated in a temporally precise pattern during reach-to-grasp movements, with a peak at the time of the reach-end and troughs just before and after it. We identified LTS-like events on the basis of LTS burst characteristics. These were rare, but also modulated, decreasing in incidence just after reach-end. The inhibitory modulations in firing rate and LTS-like events were abolished in parkinsonian rats. These data confirm that nigrostriatal dopamine depletion is accompanied by profound and specific deficits in movement-related Mthal activity. These changes would severely impair Mthal contributions to motor program development in motor cortex and are likely to be an important factor underlying the movement deficits of PD.

Motor thalamus integration of cortical, cerebellar and basal ganglia information: implications for normal and parkinsonian conditions.

Motor thalamus (Mthal) is implicated in the control of movement because it is strategically located between motor areas of the cerebral cortex and motor-related subcortical structures, such as the cerebellum and basal ganglia (BG). The role of BG and cerebellum in motor control has been extensively studied but how Mthal processes inputs from these two networks is unclear. Specifically, there is considerable debate about the role of BG inputs on Mthal activity. This review summarizes anatomical and physiological knowledge of the Mthal and its afferents and reviews current theories of Mthal function by discussing the impact of cortical, BG and cerebellar inputs on Mthal activity. One view is that Mthal activity in BG and cerebellar-receiving territories is primarily "driven" by glutamatergic inputs from the cortex or cerebellum, respectively, whereas BG inputs are modulatory and do not strongly determine Mthal activity. This theory is steeped in the assumption that the Mthal processes information in the same way as sensory thalamus, through interactions of modulatory inputs with a single driver input. Another view, from BG models, is that BG exert primary control on the BG-receiving Mthal so it effectively relays information from BG to cortex. We propose a new "super-integrator" theory where each Mthal territory processes multiple driver or driver-like inputs (cortex and BG, cortex and cerebellum), which are the result of considerable integrative processing. Thus, BG and cerebellar Mthal territories assimilate motivational and proprioceptive motor information previously integrated in cortico-BG and cortico-cerebellar networks, respectively, to develop sophisticated motor signals that are transmitted in parallel pathways to cortical areas for optimal generation of motor programmes. Finally, we briefly review the pathophysiological changes that occur in the BG in parkinsonism and generate testable hypotheses about how these may affect processing of inputs in the Mthal.

Distinct modulation of event-related potentials during motor preparation in patients with motor conversion disorder.

OBJECTIVE: Conversion paresis patients and healthy people feigning weakness both exhibit weak voluntary movement without detectable neuropathology. Uniquely, conversion patients lack a sense of conscious awareness of the origin of their impairment. We investigated whether conversion paresis patients show distinct electroencephalographic (EEG) markers associated with their unconscious movement deficits. METHODS: Six unilateral upper limb conversion paresis patients, 12 feigning participants asked to mimic weakness and 12 control participants performed a precued reaction time task, requiring movements of either hand, depending on precue information. Performance measures (force, reaction and movement time), and event-related EEG potentials (ERP) were compared, between groups and across hands or hemisphere, using linear mixed models. RESULTS: Feigners generated the same inter-hand difference in reaction and movement time as expressed by patients, even though no specific targets were set nor feedback given on these measures. We found novel ERP signatures specific to patients. When the symptomatic hand was precued, the P3 ERP component accompanying the precue was dramatically larger in patients than in feigning participants. Additionally, in patients the earlier N1 ERP component was diminished when the precue signalled either the symptomatic or asymptomatic hand. CONCLUSIONS: These results are consistent with previous suggestions that lack of awareness of the origin of their symptoms in conversion disorder patients may result from suppression of brain activity normally related to self-agency. In patients the diminished N1 to all precues is consistent with a generalised reduction in cognitive processing of movement-related precues. The P3 enhancement in patients is unlikely to simply reflect changes required for generation of impaired movements, because it was not seen in feigners showing the same behavioural deficits. Rather, this P3 enhancement in patients may represent a neural biomarker of unconscious processes, including additional emotional loading, related to active suppression of brain circuits involved in the attribution of self-agency.

Association with reward negatively modulates short latency phasic conditioned responses of dorsal raphe nucleus neurons in freely moving rats.

The dorsal raphe nucleus (DRN) is implicated in mood regulation, control of impulsive behavior, and in processing aversive and reward-related signals. DRN neurons show phasic responses to sensory stimuli, but whether association with reward modulates these responses is unknown. We recorded DRN neurons from rats in a contextual conditioned approach paradigm in which an auditory cue was either followed or not followed by reward, depending on a global context signal. Conditioned approach (licking) occurred after cues in the reward context, but was suppressed in the no-reward context. Many DRN neurons showed short-latency phasic activations in response to the cues. There was striking contextual bias, with more and stronger excitations in the no-reward context than in the reward context. Therefore, DRN activity scaled inversely with cue salience and with the probability of subsequent conditioned approach. Tonic changes were similarly discriminatory, with increases being dominant after cues in the no-reward context, when licking was suppressed, and tonic decreases in rate dominant after reward-predictive cues during expression of conditioned licking. Phasic and tonic DRN responses thus provide signals of consistent valence but over different timescales. The tonic changes in activity are consistent with previous data and hypotheses relating DRN activity to response suppression and impulse control. Phasic responses could contribute to this via online modulation of attention allocation through projections to sensory-processing regions.

Neural control of dopamine neurotransmission: implications for reinforcement learning.

In the past few decades there has been remarkable convergence of machine learning with neurobiological understanding of reinforcement learning mechanisms, exemplified by temporal difference (TD) learning models. The anatomy of the basal ganglia provides a number of potential substrates for instantiation of the TD mechanism. In contrast to the traditional concept of direct and indirect pathway outputs from the striatum, we emphasize that projection neurons of the striatum are branched and individual striatofugal neurons innervate both globus pallidus externa and globus pallidus interna/substantia nigra (GPi/SNr). This suggests that the GPi/SNr has the necessary inputs to operate as the source of a TD signal. We also discuss the mechanism for the timing processes necessary for learning in the TD framework. The TD framework has been particularly successful in analysing electrophysiogical recordings from dopamine (DA) neurons during learning, in terms of reward prediction error. However, present understanding of the neural control of DA release is limited, and hence the neural mechanisms involved are incompletely understood. Inhibition is very conspicuously present among the inputs to the DA neurons, with inhibitory synapses accounting for the majority of synapses on DA neurons. Furthermore, synchronous firing of the DA neuron population requires disinhibition and excitation to occur together in a coordinated manner. We conclude that the inhibitory circuits impinging directly or indirectly on the DA neurons play a central role in the control of DA neuron activity and further investigation of these circuits may provide important insight into the biological mechanisms of reinforcement learning.

Animal models to guide clinical drug development in ADHD: lost in translation?

We review strategies for developing animal models for examining and selecting compounds with potential therapeutic benefit in attention-deficit hyperactivity disorder (ADHD). ADHD is a behavioural disorder of unknown aetiology and pathophysiology. Current understanding suggests that genetic factors play an important role in the aetiology of ADHD. The involvement of dopaminergic and noradrenergic systems in the pathophysiology of ADHD is probable. We review the clinical features of ADHD including inattention, hyperactivity and impulsivity and how these are operationalized for laboratory study. Measures of temporal discounting (but not premature responding) appear to predict known drug effects well (treatment validity). Open-field measures of overactivity commonly used do not have treatment validity in human populations. A number of animal models have been proposed that simulate the symptoms of ADHD. The most commonly used are the spontaneously hypertensive rat (SHR) and the 6-hydroxydopamine-lesioned (6-OHDA) animals. To date, however, the SHR lacks treatment validity, and the effects of drugs on symptoms of impulsivity and inattention have not been studied extensively in 6-OHDA-lesioned animals. At the present stage of development, there are no in vivo models of proven effectiveness for examining and selecting compounds with potential therapeutic benefit in ADHD. However, temporal discounting is an emerging theme in theories of ADHD, and there is good evidence of increased value of delayed reward following treatment with stimulant drugs. Therefore, operant behaviour paradigms that measure the effects of drugs in situations of delayed reinforcement, whether in normal rats or selected models, show promise for the future.

Power fluctuations in beta and gamma frequencies in rat globus pallidus: association with specific phases of slow oscillations and differential modulation by dopamine D1 and D2 receptors.

Modulation of oscillatory activity through basal ganglia-cortical loops in specific frequency bands is thought to reflect specific functional states of neural networks. A specific negative correlation between beta and gamma sub-bands has been demonstrated in human basal ganglia and may be key for normal basal ganglia function. However, these studies were limited to Parkinson's disease patients. To confirm that this interaction is a feature of normal basal ganglia, we recorded local field potential (LFP) from electrodes in globus pallidus (GP) of intact rats. We found significant negative correlation between specific frequencies within gamma (≈ 60 Hz) and beta (≈ 14 Hz) bands. Furthermore, we show that fluctuations in power at these frequencies are differentially nested within slow (≈ 3 Hz) oscillations in the delta band, showing maximum power at distinct and different phases of delta. These results suggest a hierarchical organization of LFP frequencies in the rat GP, in which a low-frequency signal in the basal ganglia can predict the timing and interaction of power fluctuations across higher frequencies. Finally, we found that dopamine D(1) and D(2) receptor antagonists differentially affected power in gamma and beta bands and also had different effects on correlation between them and the nesting within delta, indicating an important role for endogenous dopamine acting on direct and indirect pathway neurons in the maintenance of the hierarchical organization of frequency bands. Disruption of this hierarchical organization and subsequent disordered beta-gamma balance in basal ganglia disorders such as Parkinson's disease may be important in the pathogenesis of their symptoms.

Sensitivity to delay of reinforcement in two animal models of attention deficit hyperactivity disorder (ADHD).

An altered response to reinforcement has been proposed as a mechanism underlying many of the symptoms of attention deficit hyperactivity disorder (ADHD). We measured sensitivity to delay of reinforcement in two animal models of ADHD, the spontaneously hypertensive rat (SHR) and a newly proposed model, the genetically hypertensive (GH) rat. A task previously used to measure effects of delay of reinforcement in children with ADHD was adapted for use in the present experiment. The SHR and GH rats were compared to their respective genetic control strains, Wistar-Kyoto (WKY), and Wistar (WI). The experimental task required pressing one of two available levers each trial. One lever delivered an immediate reinforcement, and the other lever a delayed reinforcement. Both the SHR and GH strains allocated significantly more responses to the immediately reinforced lever than their genetic control strains. Individual instances of reinforcement differentially affected response allocation in the GH but not the SHR. These findings support the use of the SHR and GH rat to model altered response to reinforcement, and demonstrate the additional value of the GH strain to model the effects of individual instances of reinforcement in children with ADHD.

Tripartite mechanism of extinction suggested by dopamine neuron activity and temporal difference model.

Extinction of behavior enables adaptation to a changing world and is crucial for recovery from disorders such as phobias and drug addiction. However, the brain mechanisms underlying behavioral extinction remain poorly understood. Midbrain dopamine (DA) neurons appear to play a central role in most acquisition processes of appetitive conditioning. Here, we show that the responses of putative DA neurons to conditioned reward predicting cues also dynamically encode two classical features of extinction: decrement in amplitude of previously learned excitatory responses and rebound of responding on subsequent retesting (spontaneous recovery). Crucially, this encoding involves development of inhibitory responses in the DA neurons, reflecting new, extinction-specific learning in the brain. We explored the implications of this finding by adding such inhibitory inputs to a standard temporal difference model of DA cell activity. We found that combining extinction-triggered plasticity of these inputs with a time-dependent spontaneous decay of weights, equivalent to a forgetting process as described in classical behavioral extinction literature, enabled the model to simulate several classical features of extinction. A key requirement to achieving spontaneous recovery was differential rates of spontaneous decay for weights representing original conditioning and for subsequent extinction learning. A testable prediction of the model is thus that differential decay properties exist within the wider circuits regulating DA cell activity. These findings are consistent with the hypothesis that extinction processes at both cellular and behavioral levels involve a dynamic interaction between new (inhibitory) learning, forgetting, and unlearning.

Striatal contributions to reward and decision making: making sense of regional variations in a reiterated processing matrix.

The striatum is the major input nucleus of the basal ganglia. It is thought to play a key role in learning on the basis of positive reinforcement and in action selection. One view of the striatum conceives it as comprising a reiterated matrix of processing units that perform common operations in different striatal regions, namely synaptic plasticity according to a three-factor rule, and lateral inhibition. These operations are required for reinforcement learning and selection of previously reinforced actions. Analysis of the behavioral effects of circumscribed lesions of the striatum, however, suggests regional specialization of learning and decision-making operations. We consider how a basic processing unit may be modified by regional variations in neurochemical parameters, for example, by the gradient in density of dopamine terminals from dorsal to ventral striatum. These variations suggest subtle differences between dorsolateral and ventromedial striatal regions in the temporal properties of dopamine signaling, which are superimposed on regional differences in connectivity. We propose that these variations make sense in relation to the temporal structure of activity in striatal inputs from different regions, and the requirements of different learning operations. Dorsolateral striatal (DLS) regions may be subject to brief, precisely timed pulses of dopamine, whereas ventromedial striatal regions integrate dopamine signals over a longer time course. These differences may be important for understanding regional variations in the contribution to reinforcement of habits, versus incentive processes that are sensitive to the value of expected rewards.