Learning in deep neural networks and brains with similarity-weighted interleaved learning.

Understanding how the brain learns throughout a lifetime remains a long-standing challenge. In artificial neural networks (ANNs), incorporating novel information too rapidly results in catastrophic interference, i.e., abrupt loss of previously acquired knowledge. Complementary Learning Systems Theory (CLST) suggests that new memories can be gradually integrated into the neocortex by interleaving new memories with existing knowledge. This approach, however, has been assumed to require interleaving all existing knowledge every time something new is learned, which is implausible because it is time-consuming and requires a large amount of data. We show that deep, nonlinear ANNs can learn new information by interleaving only a subset of old items that share substantial representational similarity with the new information. By using such similarity-weighted interleaved learning (SWIL), ANNs can learn new information rapidly with a similar accuracy level and minimal interference, while using a much smaller number of old items presented per epoch (fast and data-efficient). SWIL is shown to work with various standard classification datasets (Fashion-MNIST, CIFAR10, and CIFAR100), deep neural network architectures, and in sequential learning frameworks. We show that data efficiency and speedup in learning new items are increased roughly proportionally to the number of nonoverlapping classes stored in the network, which implies an enormous possible speedup in human brains, which encode a high number of separate categories. Finally, we propose a theoretical model of how SWIL might be implemented in the brain.

A shared neural ensemble links distinct contextual memories encoded close in time.

Recent studies suggest that a shared neural ensemble may link distinct memories encoded close in time. According to the memory allocation hypothesis, learning triggers a temporary increase in neuronal excitability that biases the representation of a subsequent memory to the neuronal ensemble encoding the first memory, such that recall of one memory increases the likelihood of recalling the other memory. Here we show in mice that the overlap between the hippocampal CA1 ensembles activated by two distinct contexts acquired within a day is higher than when they are separated by a week. Several findings indicate that this overlap of neuronal ensembles links two contextual memories. First, fear paired with one context is transferred to a neutral context when the two contexts are acquired within a day but not across a week. Second, the first memory strengthens the second memory within a day but not across a week. Older mice, known to have lower CA1 excitability, do not show the overlap between ensembles, the transfer of fear between contexts, or the strengthening of the second memory. Finally, in aged mice, increasing cellular excitability and activating a common ensemble of CA1 neurons during two distinct context exposures rescued the deficit in linking memories. Taken together, these findings demonstrate that contextual memories encoded close in time are linked by directing storage into overlapping ensembles. Alteration of these processes by ageing could affect the temporal structure of memories, thus impairing efficient recall of related information.

Early impairment of thalamocortical circuit activity and coherence in a mouse model of Huntington's disease.

Huntington's disease (HD) is a progressive, fatal neurodegenerative disorder characterized by motor, cognitive, and psychiatric disturbances. There is no known cure for HD, but its progressive nature allows for early therapeutic intervention. Currently, much of the research has focused on the striatum, however, there is evidence suggesting that disruption of thalamocortical circuits could underlie some of the early symptoms of HD. Loss of both cortical pyramidal neurons (CPNs) and thalamic neurons occurs in HD patients, and cognitive, somatosensory, and attention deficits precede motor abnormalities. However, the role of thalamocortical pathways in HD progression has been understudied. Here, we measured single unit activity and local field potentials (LFPs) from electrode arrays implanted in the thalamus and primary motor cortex of 4-5 month-old male and female Q175 mice. We assessed neuronal activity under baseline conditions as well as during presentation of rewards delivered via actuation of an audible solenoid valve. HD mice showed a significantly delayed licking response to the reward stimulus. At the same time, neuronal activation to the reward was delayed in thalamic neurons, CPNs and fast-spiking cortical interneurons (FSIs) of HD mice. In addition, thalamocortical coherence increased at lower frequencies in HD relative to wildtype mice. Together, these data provide evidence that impaired cortical and thalamic responses to reward stimuli, and impaired thalamocortical coherence, may play an important early role in motor, cognitive, and learning deficits in HD patients.

Synaptic tagging during memory allocation.

There is now compelling evidence that the allocation of memory to specific neurons (neuronal allocation) and synapses (synaptic allocation) in a neurocircuit is not random and that instead specific mechanisms, such as increases in neuronal excitability and synaptic tagging and capture, determine the exact sites where memories are stored. We propose an integrated view of these processes, such that neuronal allocation, synaptic tagging and capture, spine clustering and metaplasticity reflect related aspects of memory allocation mechanisms. Importantly, the properties of these mechanisms suggest a set of rules that profoundly affect how memories are stored and recalled.

Selective Modulation of Orbitofrontal Network Activity during Negative Occasion Setting.

Discrete cues can gain powerful control over behavior to help an animal anticipate and cope with upcoming events. This is important in conditions where understanding the relationship between complex stimuli provides a means to resolving situational ambiguity. However, it is unclear how cortical circuits generate and maintain these signals that conditionally regulate behavior. To address this, we established a Pavlovian serial feature-negative conditioning paradigm, where male mice are trained on a trial in which a conditioned stimulus (CS) is presented alone and followed by reward, or a feature-negative trial in which the CS is preceded by a feature cue indicating there is no reward. Mice learn to respond with anticipatory licking to a solitary CS, but significantly suppress their responding to the same cue during feature-negative trials. We show that the feature cue forms a selective association with its paired CS, because the ability of the feature to transfer its suppressive properties to a separately rewarded cue is limited. Next, to examine the underlying neural dynamics, we conduct recordings in the orbitofrontal cortex (OFC). We find that the feature cue significantly and selectively inhibits CS-evoked activity. Finally, we find that the feature triggers a distinct OFC network state during the delay period between the feature and CS, establishing a potential link between the feature and future events. Together, our findings suggest that OFC dynamics are modulated by the feature cue and its associated conditioned stimulus in a manner consistent with an occasion setting model.SIGNIFICANCE STATEMENT The ability of patterned cues to form an inhibitory relationship with ambiguously rewarded outcomes has been appreciated since early studies on learning and memory. However, it was often assumed that these cues, despite their hierarchical nature, still made direct associative links with neural rewarding events. This model was significantly challenged, largely by the work of Holland and colleagues, who demonstrated that under certain conditions cues can inherit occasion setting properties whereby they modulate the ability of a paired cue to elicit its conditioned response. Here we provide some of the first evidence that the activity of a cortical circuit is selectively modulated by such cues, thereby providing insight into the mechanisms of higher order learning.

Differential Encoding of Time by Prefrontal and Striatal Network Dynamics.

Telling time is fundamental to many forms of learning and behavior, including the anticipation of rewarding events. Although the neural mechanisms underlying timing remain unknown, computational models have proposed that the brain represents time in the dynamics of neural networks. Consistent with this hypothesis, changing patterns of neural activity dynamically in a number of brain areas-including the striatum and cortex-has been shown to encode elapsed time. To date, however, no studies have explicitly quantified and contrasted how well different areas encode time by recording large numbers of units simultaneously from more than one area. Here, we performed large-scale extracellular recordings in the striatum and orbitofrontal cortex of mice that learned the temporal relationship between a stimulus and a reward and reported their response with anticipatory licking. We used a machine-learning algorithm to quantify how well populations of neurons encoded elapsed time from stimulus onset. Both the striatal and cortical networks encoded time, but the striatal network outperformed the orbitofrontal cortex, a finding replicated both in simultaneously and nonsimultaneously recorded corticostriatal datasets. The striatal network was also more reliable in predicting when the animals would lick up to ∼1 s before the actual lick occurred. Our results are consistent with the hypothesis that temporal information is encoded in a widely distributed manner throughout multiple brain areas, but that the striatum may have a privileged role in timing because it has a more accurate "clock" as it integrates information across multiple cortical areas. SIGNIFICANCE STATEMENT: The neural representation of time is thought to be distributed across multiple functionally specialized brain structures, including the striatum and cortex. However, until now, the neural code for time has not been compared quantitatively between these areas. Here, we performed large-scale recordings in the striatum and orbitofrontal cortex of mice trained on a stimulus-reward association task involving a delay period and used a machine-learning algorithm to quantify how well populations of simultaneously recorded neurons encoded elapsed time from stimulus onset. We found that, although both areas encoded time, the striatum consistently outperformed the orbitofrontal cortex. These results suggest that the striatum may refine the code for time by integrating information from multiple inputs.

Transforming growth factor ß recruits persistent MAPK signaling to regulate long-term memory consolidation in Aplysia californica.

In this study, we explore the mechanistic relationship between growth factor signaling and kinase activity that supports the protein synthesis-dependent phase of long-term memory (LTM) consolidation for sensitization ofAplysia Specifically, we examine LTM for tail shock-induced sensitization of the tail-elicited siphon withdrawal (T-SW) reflex, a form of memory that requires both (i) extracellular signal-regulated kinase (ERK1/2; MAPK) activity within identified sensory neurons (SNs) that mediate the T-SW and (ii) the activation of transforming growth factor ß (TGFß) signaling. We now report that repeated tail shocks that induce intermediate-term (ITM) and LTM for sensitization, also induce a sustained post-training phase of MAPK activity in SNs (lasting at least 1 h). We identified two mechanistically distinct phases of post-training MAPK: (i) an immediate phase that does not require ongoing protein synthesis or TGFß signaling, and (ii) a sustained phase that requires both protein synthesis and extracellular TGFß signaling. We find that LTM consolidation requires sustained MAPK, and is disrupted by inhibitors of protein synthesis and TGFß signaling during the consolidation window. These results provide strong evidence that TGFß signaling sustains MAPK activity as an essential mechanistic step for LTM consolidation.

Brain activity mapping at multiple scales with silicon microprobes containing 1,024 electrodes.

The coordinated activity of neural ensembles across multiple interconnected regions has been challenging to study in the mammalian brain with cellular resolution using conventional recording tools. For instance, neural systems regulating learned behaviors often encompass multiple distinct structures that span the brain. To address this challenge we developed a three-dimensional (3D) silicon microprobe capable of simultaneously measuring extracellular spike and local field potential activity from 1,024 electrodes. The microprobe geometry can be precisely configured during assembly to target virtually any combination of four spatially distinct neuroanatomical planes. Here we report on the operation of such a device built for high-throughput monitoring of neural signals in the orbitofrontal cortex and several nuclei in the basal ganglia. We perform analysis on systems-level dynamics and correlations during periods of conditioned behavioral responding and rest, demonstrating the technology's ability to reveal functional organization at multiple scales in parallel in the mouse brain.

Small G proteins exhibit pattern sensitivity in MAPK activation during the induction of memory and synaptic facilitation in Aplysia.

Memory formation is highly sensitive to specific patterns of training, but the cellular and molecular mechanisms underlying pattern sensitivity are not well understood. We explored this general question by using Aplysia californica as a model system. We examined the regulation of MAPK (ERK1/2) activation by small G proteins in the CNS by using different patterns of analog stimuli that mimic different patterns of behavioral training for memory induction. We first cloned and characterized the Aplysia homologs of the small G proteins, Ras and Rap1 (ApRas and ApRap, respectively). We next examined changes in ApRas and ApRap activity that accompany MAPK activation. Last, by delivering recombinant ApRas and ApRap into the CNS, we directly manipulated their activity and examined the resultant MAPK activation. We found that MAPK activation induced by analog training depends on the combined activity of ApRas and ApRap, rather than the individual activity of either one alone. Also, ApRas and ApRap have a complex role in MAPK activation: they can act as activators or inhibitors, depending on the specific pattern of the training. The pattern-sensitive regulation of MAPK by interactive ApRas and ApRap activity that we have identified could contribute to the molecular routing of different downstream effects of spatially localized MAPK required for the induction of specific pattern-sensitive forms of synaptic facilitation and memory.

Intermediate-term processes in memory formation.

Neuroscientists have invested considerable effort in attempting to elucidate the molecular mechanisms that mediate short-term and long-term forms of learning and memory. For instance, the discovery of long-term potentiation inspired a field that has produced hundreds of studies examining both early and late forms of long-term potentiation. And at the behavioral level, most neuroscientists investigate either short- or long-term forms of memory or some combination of the two. The general belief that plasticity was restricted to short- and long-term temporal domains lasted for many years because of the apparent continuity of memory and its molecular characterization from one domain to the other. In cellular studies of plasticity, the short-term stage typically lasts in the range of minutes, and requires modification of pre-existing proteins, whereas long-term changes, such as synaptic growth, last for hours to days and require transcription and translation. As both behavioral and cellular studies covered a wider range of temporal domains, from the initiation of brief memory to the expression of long-lasting memory, it was at least tacitly assumed that these studies also captured any intervening domains as well. However, between these two temporal extremes lies a unique form of intermediate-term synaptic plasticity and memory, which mechanistically is a blend of the early and late forms.

Parvalbumin Interneurons Modulate Striatal Output and Enhance Performance during Associative Learning.

The prevailing view is that striatal parvalbumin (PV)-positive interneurons primarily function to downregulate medium spiny projection neuron (MSN) activity via monosynaptic inhibitory signaling. Here, by combining in vivo neural recordings and optogenetics, we unexpectedly find that both suppressing and over-activating PV cells attenuates spontaneous MSN activity. To account for this, we find that, in addition to monosynaptic coupling, PV-MSN interactions are mediated by a competing disynaptic inhibitory circuit involving a variety of neuropeptide Y-expressing interneurons. Next we use optogenetic and chemogenetic approaches to show that dorsolateral striatal PV interneurons influence the initial expression of reward-conditioned responses but that their contribution to performance declines with experience. Consistent with this, we observe with large-scale recordings in behaving animals that the relative contribution of PV cells on MSN activity diminishes with training. Together, this work provides a possible mechanism by which PV interneurons modulate striatal output and selectively enhance performance early in learning.

Retrieval and Reconsolidation Accounts of Fear Extinction.

Extinction is the primary mode for the treatment of anxiety disorders. However, extinction memories are prone to relapse. For example, fear is likely to return when a prolonged time period intervenes between extinction and a subsequent encounter with the fear-provoking stimulus (spontaneous recovery). Therefore there is considerable interest in the development of procedures that strengthen extinction and to prevent such recovery of fear. We contrasted two procedures in rats that have been reported to cause such deepened extinction. One where extinction begins before the initial consolidation of fear memory begins (immediate extinction) and another where extinction begins after a brief exposure to the consolidated fear stimulus. The latter is thought to open a period of memory vulnerability similar to that which occurs during initial consolidation (reconsolidation update). We also included a standard extinction treatment and a control procedure that reversed the brief exposure and extinction phases. Spontaneous recovery was only found with the standard extinction treatment. In a separate experiment we tested fear shortly after extinction (i.e., within 6 h). All extinction procedures, except reconsolidation update reduced fear at this short-term test. The findings suggest that strengthened extinction can result from alteration in both retrieval and consolidation processes.

Intermediate-term memory for site-specific sensitization in aplysia is maintained by persistent activation of protein kinase C.

Recent studies of long-term synaptic plasticity and long-term memory have demonstrated that the same functional endpoint, such as long-term potentiation, can be induced through distinct signaling pathways engaged by different patterns of stimulation. A critical question raised by these studies is whether different induction pathways either converge onto a common molecular mechanism or engage different molecular cascades for the maintenance of long-term plasticity. We directly examined this issue in the context of memory for sensitization in the marine mollusk Aplysia. In this system, training with a single tail shock normally induces short-term memory (<30 min) for sensitization of tail-elicited siphon withdrawal, whereas repeated spaced shocks induce both intermediate-term memory (ITM) (>90 min) and long-term memory (>24 hr). We now show that a single tail shock can also induce ITM that is expressed selectively at the trained site (site-specific ITM). Although phenotypically similar to the form of ITM induced by repeated trials, the mechanisms by which site-specific ITM is induced and maintained are distinct. Unlike repeated-trial ITM, site-specific ITM requires neither protein synthesis nor PKA activity for induction or maintenance. Rather, the induction of site-specific ITM requires calpain-dependent proteolysis of activated PKC, yielding a persistently active PKC catalytic fragment (PKM) that also serves to maintain the memory in the intermediateterm temporal domain. Thus, two unique forms of ITM that have different induction requirements also use distinct molecular mechanisms for their maintenance.

Differential role of mitogen-activated protein kinase in three distinct phases of memory for sensitization in Aplysia.

The mitogen-activated protein kinase (MAPK) pathway has been implicated recently in synaptic plasticity and memory. Here we used tail shock-induced sensitization of the tail-elicited siphon withdrawal reflex in Aplysia to examine the role of MAPK in three different phases of memory. We show that a specific pattern of serotonin (5-HT) application that produces intermediate-term and long-term synaptic facilitation (ITF and LTF, respectively) of the sensory-motor (SN-MN) synapses in Aplysia leads to sustained activation of extracellular signal-regulated kinase in the ventrocaudal cluster sensory neurons (SNs), which include the tail SNs. Furthermore, repeated tail shocks that induce intermediate-term and long-term memory (ITM and LTM, respectively) for sensitization also lead to sustained MAPK activation in the SNs. Given these results, we next examined the requirement of MAPK activity in (1) SN-MN synaptic facilitation and (2) memory for sensitization in Aplysia, by inhibiting MEK, the upstream kinase that phosphorylates and activates MAPK. In cellular experiments, we show that MAPK activity is required for ITF of tail SN-tail MN synapses, and, in parallel behavioral experiments, we show that ITM requires MAPK activity for its induction but not its expression. In contrast, short-term memory for sensitization does not require MAPK activity. Finally, 5-HT-induced LTF has been shown previously to require MAPK activity. Here we show that LTM for sensitization also requires MAPK activity. These results provide evidence that MAPK plays important roles specifically in long-lasting phases of synaptic plasticity and memory.

A hybrid fibronectin motif protein as an integrin targeting selective tumor vascular thrombogen.

Targeted thrombotic eradication of solid tumors is a novel therapeutic strategy. The feasibility, efficacy, selectivity, and safety are dependent on multiple variables of protein design, molecular assembly, vascular target, and exclusive restriction of function to the tumor vasculature. To advance this strategy, we describe a design of an integrin targeting selective tumor vascular thrombogen. We adopted the fibronectin structural motif of tandem repeating modules with four type III repeat modules of fibronectin followed by two structurally homologous modules of the extracellular domain of tissue factor. This hybrid protein of six tandem modules recognizes integrins and selectively docks and initiates the thrombogenic protease cascade locally on the target cell surfaces. The protein is inactive in blood but is functionally active once assembled on integrin-positive cells. When administered i.v. to tumor-bearing mice, it selectively induces extensive local microthrombosis of the tumor microvasculature. The principles are addressed from the perspective of protein structural design for a class of selective tumor vascular thrombogen proteins that, through interaction with tumor angiogenic endothelium, elicit thrombotic occlusion rather than apoptosis or arrest of angiogenesis. This response can produce local tumor infarction followed by intratumoral ischemia-reperfusion injury, inflammation, and a local host tumor eradicative response.

CREB regulates memory allocation in the insular cortex.

The molecular and cellular mechanisms of memory storage have attracted a great deal of attention. By comparison, little is known about memory allocation, the process that determines which specific neurons in a neural network will store a given memory. Previous studies demonstrated that memory allocation is not random in the amygdala; these studies showed that amygdala neurons with higher levels of the cyclic-AMP-response-element-binding protein (CREB) are more likely to be recruited into encoding and storing fear memory. To determine whether specific mechanisms also regulate memory allocation in other brain regions and whether CREB also has a role in this process, we studied insular cortical memory representations for conditioned taste aversion (CTA). In this task, an animal learns to associate a taste (conditioned stimulus [CS]) with the experience of malaise (such as that induced by LiCl; unconditioned stimulus [US]). The insular cortex is required for CTA memory formation and retrieval. CTA learning activates a subpopulation of neurons in this structure, and the insular cortex and the basolateral amygdala (BLA) interact during CTA formation. Here, we used a combination of approaches, including viral vector transfections of insular cortex, arc fluorescence in situ hybridization (FISH), and designer receptors exclusively activated by designer drugs (DREADD) system, to show that CREB levels determine which insular cortical neurons go on to encode a given conditioned taste memory.

Molecular and cellular approaches to memory allocation in neural circuits.

Although memory allocation is a subject of active research in computer science, little is known about how the brain allocates information within neural circuits. There is an extensive literature on how specific types of memory engage different parts of the brain, and how neurons in these regions process and store information. Until recently, however, the mechanisms that determine how specific cells and synapses within a neural circuit (and not their neighbors) are recruited during learning have received little attention. Recent findings suggest that memory allocation is not random, but rather specific mechanisms regulate where information is stored within a neural circuit. New methods that allow tagging, imaging, activation, and inactivation of neurons in behaving animals promise to revolutionize studies of brain circuits, including memory allocation. Results from these studies are likely to have a considerable impact on computer science, as well as on the understanding of memory and its disorders.

Temporal phases of activity-dependent plasticity and memory are mediated by compartmentalized routing of MAPK signaling in aplysia sensory neurons.

An activity-dependent form of intermediate memory (AD-ITM) for sensitization is induced in Aplysia by a single tail shock that gives rise to plastic changes (AD-ITF) in tail sensory neurons (SNs) via the interaction of action potential firing in the SN coupled with the release of serotonin in the CNS. Activity-dependent long-term facilitation (AD-LTF, lasting >24hr) requires protein synthesis dependent persistent mitogen-activated protein kinase (MAPK) activation and translocation to the SN nucleus. We now show that the induction of the earlier temporal phase (AD-ITM and AD-ITF), which is translation and transcription independent, requires the activation of a compartmentally distinct novel signaling cascade that links second messengers, MAPK and PKC into a unified pathway within tail SNs. Since both AD-ITM and AD-LTM require MAPK activity, these collective findings suggest that presynaptic SNs route the flow of molecular information to distinct subcellular compartments during the induction of activity-dependent long-lasting memories.

The role of PKA, CaMKII, and PKC in avoidance conditioning: permissive or instructive?

This article explores the causal and correlative relationships between kinases and learning and memory. Specifically, the contributions of three kinases-protein kinase A (PKA), calcium calmodulin-dependent kinase II (CaMKII), and protein kinase C (PKC)-are assessed during the consolidation phase of avoidance conditioning. The following sources of evidence are considered: inhibitor data, activity monitoring, and transgenic studies. An exhaustive effort is made to address several issues regarding the participation of these kinases in (a) posttraining timing and magnitude, (b) location across many brain regions, and (c) the use of multiple pharmacological agents and assays. In addition, this article attempts to integrate the behavioral data with the purported role of kinases in long-term potentiation (LTP).