Learning in deep neural networks and brains with similarity-weighted interleaved learning.

Understanding how the brain learns throughout a lifetime remains a long-standing challenge. In artificial neural networks (ANNs), incorporating novel information too rapidly results in catastrophic interference, i.e., abrupt loss of previously acquired knowledge. Complementary Learning Systems Theory (CLST) suggests that new memories can be gradually integrated into the neocortex by interleaving new memories with existing knowledge. This approach, however, has been assumed to require interleaving all existing knowledge every time something new is learned, which is implausible because it is time-consuming and requires a large amount of data. We show that deep, nonlinear ANNs can learn new information by interleaving only a subset of old items that share substantial representational similarity with the new information. By using such similarity-weighted interleaved learning (SWIL), ANNs can learn new information rapidly with a similar accuracy level and minimal interference, while using a much smaller number of old items presented per epoch (fast and data-efficient). SWIL is shown to work with various standard classification datasets (Fashion-MNIST, CIFAR10, and CIFAR100), deep neural network architectures, and in sequential learning frameworks. We show that data efficiency and speedup in learning new items are increased roughly proportionally to the number of nonoverlapping classes stored in the network, which implies an enormous possible speedup in human brains, which encode a high number of separate categories. Finally, we propose a theoretical model of how SWIL might be implemented in the brain.

Early impairment of thalamocortical circuit activity and coherence in a mouse model of Huntington's disease.

Huntington's disease (HD) is a progressive, fatal neurodegenerative disorder characterized by motor, cognitive, and psychiatric disturbances. There is no known cure for HD, but its progressive nature allows for early therapeutic intervention. Currently, much of the research has focused on the striatum, however, there is evidence suggesting that disruption of thalamocortical circuits could underlie some of the early symptoms of HD. Loss of both cortical pyramidal neurons (CPNs) and thalamic neurons occurs in HD patients, and cognitive, somatosensory, and attention deficits precede motor abnormalities. However, the role of thalamocortical pathways in HD progression has been understudied. Here, we measured single unit activity and local field potentials (LFPs) from electrode arrays implanted in the thalamus and primary motor cortex of 4-5 month-old male and female Q175 mice. We assessed neuronal activity under baseline conditions as well as during presentation of rewards delivered via actuation of an audible solenoid valve. HD mice showed a significantly delayed licking response to the reward stimulus. At the same time, neuronal activation to the reward was delayed in thalamic neurons, CPNs and fast-spiking cortical interneurons (FSIs) of HD mice. In addition, thalamocortical coherence increased at lower frequencies in HD relative to wildtype mice. Together, these data provide evidence that impaired cortical and thalamic responses to reward stimuli, and impaired thalamocortical coherence, may play an important early role in motor, cognitive, and learning deficits in HD patients.

Selective Modulation of Orbitofrontal Network Activity during Negative Occasion Setting.

Discrete cues can gain powerful control over behavior to help an animal anticipate and cope with upcoming events. This is important in conditions where understanding the relationship between complex stimuli provides a means to resolving situational ambiguity. However, it is unclear how cortical circuits generate and maintain these signals that conditionally regulate behavior. To address this, we established a Pavlovian serial feature-negative conditioning paradigm, where male mice are trained on a trial in which a conditioned stimulus (CS) is presented alone and followed by reward, or a feature-negative trial in which the CS is preceded by a feature cue indicating there is no reward. Mice learn to respond with anticipatory licking to a solitary CS, but significantly suppress their responding to the same cue during feature-negative trials. We show that the feature cue forms a selective association with its paired CS, because the ability of the feature to transfer its suppressive properties to a separately rewarded cue is limited. Next, to examine the underlying neural dynamics, we conduct recordings in the orbitofrontal cortex (OFC). We find that the feature cue significantly and selectively inhibits CS-evoked activity. Finally, we find that the feature triggers a distinct OFC network state during the delay period between the feature and CS, establishing a potential link between the feature and future events. Together, our findings suggest that OFC dynamics are modulated by the feature cue and its associated conditioned stimulus in a manner consistent with an occasion setting model.SIGNIFICANCE STATEMENT The ability of patterned cues to form an inhibitory relationship with ambiguously rewarded outcomes has been appreciated since early studies on learning and memory. However, it was often assumed that these cues, despite their hierarchical nature, still made direct associative links with neural rewarding events. This model was significantly challenged, largely by the work of Holland and colleagues, who demonstrated that under certain conditions cues can inherit occasion setting properties whereby they modulate the ability of a paired cue to elicit its conditioned response. Here we provide some of the first evidence that the activity of a cortical circuit is selectively modulated by such cues, thereby providing insight into the mechanisms of higher order learning.

Differential Encoding of Time by Prefrontal and Striatal Network Dynamics.

Telling time is fundamental to many forms of learning and behavior, including the anticipation of rewarding events. Although the neural mechanisms underlying timing remain unknown, computational models have proposed that the brain represents time in the dynamics of neural networks. Consistent with this hypothesis, changing patterns of neural activity dynamically in a number of brain areas-including the striatum and cortex-has been shown to encode elapsed time. To date, however, no studies have explicitly quantified and contrasted how well different areas encode time by recording large numbers of units simultaneously from more than one area. Here, we performed large-scale extracellular recordings in the striatum and orbitofrontal cortex of mice that learned the temporal relationship between a stimulus and a reward and reported their response with anticipatory licking. We used a machine-learning algorithm to quantify how well populations of neurons encoded elapsed time from stimulus onset. Both the striatal and cortical networks encoded time, but the striatal network outperformed the orbitofrontal cortex, a finding replicated both in simultaneously and nonsimultaneously recorded corticostriatal datasets. The striatal network was also more reliable in predicting when the animals would lick up to ∼1 s before the actual lick occurred. Our results are consistent with the hypothesis that temporal information is encoded in a widely distributed manner throughout multiple brain areas, but that the striatum may have a privileged role in timing because it has a more accurate "clock" as it integrates information across multiple cortical areas. SIGNIFICANCE STATEMENT: The neural representation of time is thought to be distributed across multiple functionally specialized brain structures, including the striatum and cortex. However, until now, the neural code for time has not been compared quantitatively between these areas. Here, we performed large-scale recordings in the striatum and orbitofrontal cortex of mice trained on a stimulus-reward association task involving a delay period and used a machine-learning algorithm to quantify how well populations of simultaneously recorded neurons encoded elapsed time from stimulus onset. We found that, although both areas encoded time, the striatum consistently outperformed the orbitofrontal cortex. These results suggest that the striatum may refine the code for time by integrating information from multiple inputs.

Brain activity mapping at multiple scales with silicon microprobes containing 1,024 electrodes.

The coordinated activity of neural ensembles across multiple interconnected regions has been challenging to study in the mammalian brain with cellular resolution using conventional recording tools. For instance, neural systems regulating learned behaviors often encompass multiple distinct structures that span the brain. To address this challenge we developed a three-dimensional (3D) silicon microprobe capable of simultaneously measuring extracellular spike and local field potential activity from 1,024 electrodes. The microprobe geometry can be precisely configured during assembly to target virtually any combination of four spatially distinct neuroanatomical planes. Here we report on the operation of such a device built for high-throughput monitoring of neural signals in the orbitofrontal cortex and several nuclei in the basal ganglia. We perform analysis on systems-level dynamics and correlations during periods of conditioned behavioral responding and rest, demonstrating the technology's ability to reveal functional organization at multiple scales in parallel in the mouse brain.

Small G proteins exhibit pattern sensitivity in MAPK activation during the induction of memory and synaptic facilitation in Aplysia.

Memory formation is highly sensitive to specific patterns of training, but the cellular and molecular mechanisms underlying pattern sensitivity are not well understood. We explored this general question by using Aplysia californica as a model system. We examined the regulation of MAPK (ERK1/2) activation by small G proteins in the CNS by using different patterns of analog stimuli that mimic different patterns of behavioral training for memory induction. We first cloned and characterized the Aplysia homologs of the small G proteins, Ras and Rap1 (ApRas and ApRap, respectively). We next examined changes in ApRas and ApRap activity that accompany MAPK activation. Last, by delivering recombinant ApRas and ApRap into the CNS, we directly manipulated their activity and examined the resultant MAPK activation. We found that MAPK activation induced by analog training depends on the combined activity of ApRas and ApRap, rather than the individual activity of either one alone. Also, ApRas and ApRap have a complex role in MAPK activation: they can act as activators or inhibitors, depending on the specific pattern of the training. The pattern-sensitive regulation of MAPK by interactive ApRas and ApRap activity that we have identified could contribute to the molecular routing of different downstream effects of spatially localized MAPK required for the induction of specific pattern-sensitive forms of synaptic facilitation and memory.

Intermediate-term processes in memory formation.

Neuroscientists have invested considerable effort in attempting to elucidate the molecular mechanisms that mediate short-term and long-term forms of learning and memory. For instance, the discovery of long-term potentiation inspired a field that has produced hundreds of studies examining both early and late forms of long-term potentiation. And at the behavioral level, most neuroscientists investigate either short- or long-term forms of memory or some combination of the two. The general belief that plasticity was restricted to short- and long-term temporal domains lasted for many years because of the apparent continuity of memory and its molecular characterization from one domain to the other. In cellular studies of plasticity, the short-term stage typically lasts in the range of minutes, and requires modification of pre-existing proteins, whereas long-term changes, such as synaptic growth, last for hours to days and require transcription and translation. As both behavioral and cellular studies covered a wider range of temporal domains, from the initiation of brief memory to the expression of long-lasting memory, it was at least tacitly assumed that these studies also captured any intervening domains as well. However, between these two temporal extremes lies a unique form of intermediate-term synaptic plasticity and memory, which mechanistically is a blend of the early and late forms.

Parvalbumin Interneurons Modulate Striatal Output and Enhance Performance during Associative Learning.

The prevailing view is that striatal parvalbumin (PV)-positive interneurons primarily function to downregulate medium spiny projection neuron (MSN) activity via monosynaptic inhibitory signaling. Here, by combining in vivo neural recordings and optogenetics, we unexpectedly find that both suppressing and over-activating PV cells attenuates spontaneous MSN activity. To account for this, we find that, in addition to monosynaptic coupling, PV-MSN interactions are mediated by a competing disynaptic inhibitory circuit involving a variety of neuropeptide Y-expressing interneurons. Next we use optogenetic and chemogenetic approaches to show that dorsolateral striatal PV interneurons influence the initial expression of reward-conditioned responses but that their contribution to performance declines with experience. Consistent with this, we observe with large-scale recordings in behaving animals that the relative contribution of PV cells on MSN activity diminishes with training. Together, this work provides a possible mechanism by which PV interneurons modulate striatal output and selectively enhance performance early in learning.

CREB regulates memory allocation in the insular cortex.

The molecular and cellular mechanisms of memory storage have attracted a great deal of attention. By comparison, little is known about memory allocation, the process that determines which specific neurons in a neural network will store a given memory. Previous studies demonstrated that memory allocation is not random in the amygdala; these studies showed that amygdala neurons with higher levels of the cyclic-AMP-response-element-binding protein (CREB) are more likely to be recruited into encoding and storing fear memory. To determine whether specific mechanisms also regulate memory allocation in other brain regions and whether CREB also has a role in this process, we studied insular cortical memory representations for conditioned taste aversion (CTA). In this task, an animal learns to associate a taste (conditioned stimulus [CS]) with the experience of malaise (such as that induced by LiCl; unconditioned stimulus [US]). The insular cortex is required for CTA memory formation and retrieval. CTA learning activates a subpopulation of neurons in this structure, and the insular cortex and the basolateral amygdala (BLA) interact during CTA formation. Here, we used a combination of approaches, including viral vector transfections of insular cortex, arc fluorescence in situ hybridization (FISH), and designer receptors exclusively activated by designer drugs (DREADD) system, to show that CREB levels determine which insular cortical neurons go on to encode a given conditioned taste memory.

Temporal phases of activity-dependent plasticity and memory are mediated by compartmentalized routing of MAPK signaling in aplysia sensory neurons.

An activity-dependent form of intermediate memory (AD-ITM) for sensitization is induced in Aplysia by a single tail shock that gives rise to plastic changes (AD-ITF) in tail sensory neurons (SNs) via the interaction of action potential firing in the SN coupled with the release of serotonin in the CNS. Activity-dependent long-term facilitation (AD-LTF, lasting >24hr) requires protein synthesis dependent persistent mitogen-activated protein kinase (MAPK) activation and translocation to the SN nucleus. We now show that the induction of the earlier temporal phase (AD-ITM and AD-ITF), which is translation and transcription independent, requires the activation of a compartmentally distinct novel signaling cascade that links second messengers, MAPK and PKC into a unified pathway within tail SNs. Since both AD-ITM and AD-LTM require MAPK activity, these collective findings suggest that presynaptic SNs route the flow of molecular information to distinct subcellular compartments during the induction of activity-dependent long-lasting memories.