Multi-layer network utilizing rewarded spike time dependent plasticity to learn a foraging task.

Neural networks with a single plastic layer employing reward modulated spike time dependent plasticity (STDP) are capable of learning simple foraging tasks. Here we demonstrate advanced pattern discrimination and continuous learning in a network of spiking neurons with multiple plastic layers. The network utilized both reward modulated and non-reward modulated STDP and implemented multiple mechanisms for homeostatic regulation of synaptic efficacy, including heterosynaptic plasticity, gain control, output balancing, activity normalization of rewarded STDP and hard limits on synaptic strength. We found that addition of a hidden layer of neurons employing non-rewarded STDP created neurons that responded to the specific combinations of inputs and thus performed basic classification of the input patterns. When combined with a following layer of neurons implementing rewarded STDP, the network was able to learn, despite the absence of labeled training data, discrimination between rewarding patterns and the patterns designated as punishing. Synaptic noise allowed for trial-and-error learning that helped to identify the goal-oriented strategies which were effective in task solving. The study predicts a critical set of properties of the spiking neuronal network with STDP that was sufficient to solve a complex foraging task involving pattern classification and decision making.

Interneuron-mediated inhibition synchronizes neuronal activity during slow oscillation.

The signature of slow-wave sleep in the electroencephalogram (EEG) is large-amplitude fluctuation of the field potential, which reflects synchronous alternation of activity and silence across cortical neurons. While initiation of the active cortical states during sleep slow oscillation has been intensively studied, the biological mechanisms which drive the network transition from an active state to silence remain poorly understood. In the current study, using a combination of in vivo electrophysiology and thalamocortical network simulation, we explored the impact of intrinsic and synaptic inhibition on state transition during sleep slow oscillation. We found that in normal physiological conditions, synaptic inhibition controls the duration and the synchrony of active state termination. The decline of interneuron-mediated inhibition led to asynchronous downward transition across the cortical network and broke the regular slow oscillation pattern. Furthermore, in both in vivo experiment and computational modelling, we revealed that when the level of synaptic inhibition was reduced significantly, it led to a recovery of synchronized oscillations in the form of seizure-like bursting activity. In this condition, the fast active state termination was mediated by intrinsic hyperpolarizing conductances. Our study highlights the significance of both intrinsic and synaptic inhibition in manipulating sleep slow rhythms.

A model of bi-directional interactions between complementary learning systems for memory consolidation of sequential experiences.

The standard theory of memory consolidation posits a dual-store memory system: a fast-learning fast-decaying hippocampus that transfers memories to slow-learning long-term cortical storage. Hippocampal lesions interrupt this transfer, so recent memories are more likely to be lost than more remote memories. Existing models of memory consolidation that simulate this temporally graded retrograde amnesia operate only on static patterns or unitary variables as memories and study only one-way interaction from the hippocampus to the cortex. However, the mechanisms underlying the consolidation of episodes, which are sequential in nature and comprise multiple events, are not well-understood. The representation of learning for sequential experiences in the cortical-hippocampal network as a self-consistent dynamical system is not sufficiently addressed in prior models. Further, there is evidence for a bi-directional interaction between the two memory systems during offline periods, whereby the reactivation of waking neural patterns originating in the cortex triggers time-compressed sequential replays in the hippocampus, which in turn drive the consolidation of the pertinent sequence in the cortex. We have developed a computational model of memory encoding, consolidation, and recall for storing temporal sequences that explores the dynamics of this bi-directional interaction and time-compressed replays in four simulation experiments, providing novel insights into whether hippocampal learning needs to be suppressed for stable memory consolidation and into how new and old memories compete for limited replay opportunities during offline periods. The salience of experienced events, based on factors such as recency and frequency of use, is shown to have considerable impact on memory consolidation because it biases the relative probability that a particular event will be cued in the cortex during offline periods. In the presence of hippocampal learning during sleep, our model predicts that the fast-forgetting hippocampus can continually refresh the memory traces of a given episodic sequence if there are no competing experiences to be replayed.

Transcranial Current Stimulation During Sleep Facilitates Insight into Temporal Rules, but does not Consolidate Memories of Individual Sequential Experiences.

Slow-wave sleep (SWS) is known to contribute to memory consolidation, likely through the reactivation of previously encoded waking experiences. Contemporary studies demonstrate that when auditory or olfactory stimulation is administered during memory encoding and then reapplied during SWS, memory consolidation can be enhanced, an effect that is believed to rely on targeted memory reactivation (TMR) induced by the sensory stimulation. Here, we show that transcranial current stimulations (tCS) during sleep can also be used to induce TMR, resulting in the facilitation of high-level cognitive processes. Participants were exposed to repeating sequences in a realistic 3D immersive environment while being stimulated with particular tCS patterns. A subset of these tCS patterns was then reapplied during sleep stages N2 and SWS coupled to slow oscillations in a closed-loop manner. We found that in contrast to our initial hypothesis, performance for the sequences corresponding to the reapplied tCS patterns was no better than for other sequences that received stimulations only during wake or not at all. In contrast, we found that the more stimulations participants received overnight, the more likely they were to detect temporal regularities governing the learned sequences the following morning, with tCS-induced beta power modulations during sleep mediating this effect.

One-Shot Tagging During Wake and Cueing During Sleep With Spatiotemporal Patterns of Transcranial Electrical Stimulation Can Boost Long-Term Metamemory of Individual Episodes in Humans.

Targeted memory reactivation (TMR) during slow-wave oscillations (SWOs) in sleep has been demonstrated with sensory cues to achieve about 5-12% improvement in post-nap memory performance on simple laboratory tasks. But prior work has not yet addressed the one-shot aspect of episodic memory acquisition, or dealt with the presence of interference from ambient environmental cues in real-world settings. Further, TMR with sensory cues may not be scalable to the multitude of experiences over one's lifetime. We designed a novel non-invasive non-sensory paradigm that tags one-shot experiences of minute-long naturalistic episodes in immersive virtual reality (VR) with unique spatiotemporal amplitude-modulated patterns (STAMPs) of transcranial electrical stimulation (tES). In particular, we demonstrated that these STAMPs can be re-applied as brief pulses during SWOs in sleep to achieve about 10-20% improvement in the metamemory of targeted episodes compared to the control episodes at 48 hours after initial viewing. We found that STAMPs can not only facilitate but also impair metamemory for the targeted episodes based on an interaction between pre-sleep metamemory and the number of STAMP applications during sleep. Overnight metamemory improvements were mediated by spectral power increases following the offset of STAMPs in the slow-spindle band (8-12 Hz) for left temporal areas in the scalp electroencephalography (EEG) during sleep. These results prescribe an optimal strategy to leverage STAMPs for boosting metamemory and suggest that real-world episodic memories can be modulated in a targeted manner even with coarser, non-invasive spatiotemporal stimulation.

Dose-Dependent Effects of Closed-Loop tACS Delivered During Slow-Wave Oscillations on Memory Consolidation.

Sleep is critically important to consolidate information learned throughout the day. Slow-wave sleep (SWS) serves to consolidate declarative memories, a process previously modulated with open-loop non-invasive electrical stimulation, though not always effectively. These failures to replicate could be explained by the fact that stimulation has only been performed in open-loop, as opposed to closed-loop where phase and frequency of the endogenous slow-wave oscillations (SWOs) are matched for optimal timing. The current study investigated the effects of closed-loop transcranial Alternating Current Stimulation (tACS) targeting SWOs during sleep on memory consolidation. 21 participants took part in a three-night, counterbalanced, randomized, single-blind, within-subjects study, investigating performance changes (correct rate and F1 score) on images in a target detection task over 24 h. During sleep, 1.5 mA closed-loop tACS was delivered in phase over electrodes at F3 and F4 and 180° out of phase over electrodes at bilateral mastoids at the frequency (range 0.5-1.2 Hz) and phase of ongoing SWOs for a duration of 5 cycles in each discrete event throughout the night. Data were analyzed in a repeated measures ANOVA framework, and results show that verum stimulation improved post-sleep performance specifically on generalized versions of images used in training at both morning and afternoon tests compared to sham, suggesting the facilitation of schematization of information, but not of rote, veridical recall. We also found a surprising inverted U-shaped dose effect of sleep tACS, which is interpreted in terms of tACS-induced faciliatory and subsequent refractory dynamics of SWO power in scalp EEG. This is the first study showing a selective modulation of long-term memory generalization using a novel closed-loop tACS approach, which holds great potential for both healthy and neuropsychiatric populations.

Network models of frequency modulated sweep detection.

Frequency modulated (FM) sweeps are common in species-specific vocalizations, including human speech. Auditory neurons selective for the direction and rate of frequency change in FM sweeps are present across species, but the synaptic mechanisms underlying such selectivity are only beginning to be understood. Even less is known about mechanisms of experience-dependent changes in FM sweep selectivity. We present three network models of synaptic mechanisms of FM sweep direction and rate selectivity that explains experimental data: (1) The 'facilitation' model contains frequency selective cells operating as coincidence detectors, summing up multiple excitatory inputs with different time delays. (2) The 'duration tuned' model depends on interactions between delayed excitation and early inhibition. The strength of delayed excitation determines the preferred duration. Inhibitory rebound can reinforce the delayed excitation. (3) The 'inhibitory sideband' model uses frequency selective inputs to a network of excitatory and inhibitory cells. The strength and asymmetry of these connections results in neurons responsive to sweeps in a single direction of sufficient sweep rate. Variations of these properties, can explain the diversity of rate-dependent direction selectivity seen across species. We show that the inhibitory sideband model can be trained using spike timing dependent plasticity (STDP) to develop direction selectivity from a non-selective network. These models provide a means to compare the proposed synaptic and spectrotemporal mechanisms of FM sweep processing and can be utilized to explore cellular mechanisms underlying experience- or training-dependent changes in spectrotemporal processing across animal models. Given the analogy between FM sweeps and visual motion, these models can serve a broader function in studying stimulus movement across sensory epithelia.

A spiking network model of decision making employing rewarded STDP.

Reward-modulated spike timing dependent plasticity (STDP) combines unsupervised STDP with a reinforcement signal that modulates synaptic changes. It was proposed as a learning rule capable of solving the distal reward problem in reinforcement learning. Nonetheless, performance and limitations of this learning mechanism have yet to be tested for its ability to solve biological problems. In our work, rewarded STDP was implemented to model foraging behavior in a simulated environment. Over the course of training the network of spiking neurons developed the capability of producing highly successful decision-making. The network performance remained stable even after significant perturbations of synaptic structure. Rewarded STDP alone was insufficient to learn effective decision making due to the difficulty maintaining homeostatic equilibrium of synaptic weights and the development of local performance maxima. Our study predicts that successful learning requires stabilizing mechanisms that allow neurons to balance their input and output synapses as well as synaptic noise.

Non-homogeneous extracellular resistivity affects the current-source density profiles of up-down state oscillations.

Rhythmic local field potential (LFP) oscillations observed during deep sleep are the result of synchronized electrical activities of large neuronal ensembles, which consist of alternating periods of activity and silence, termed 'up' and 'down' states, respectively. Current-source density (CSD) analysis indicates that the up states of these slow oscillations are associated with current sources in superficial cortical layers and sinks in deep layers, while the down states display the opposite pattern of source-sink distribution. We show here that a network model of up and down states displays this CSD profile only if a frequency-filtering extracellular medium is assumed. When frequency filtering was modelled as inhomogeneous conductivity, this simple model had considerably more power in slow frequencies, resulting in significant differences in LFP and CSD profiles compared with the constant-resistivity model. These results suggest that the frequency-filtering properties of extracellular media may have important consequences for the interpretation of the results of CSD analysis.