Uncertainty-based modulation for lifelong learning.

The creation of machine learning algorithms for intelligent agents capable of continuous, lifelong learning is a critical objective for algorithms being deployed on real-life systems in dynamic environments. Here we present an algorithm inspired by neuromodulatory mechanisms in the human brain that integrates and expands upon Stephen Grossberg's ground-breaking Adaptive Resonance Theory proposals. Specifically, it builds on the concept of uncertainty, and employs a series of "neuromodulatory" mechanisms to enable continuous learning, including self-supervised and one-shot learning. Algorithm components were evaluated in a series of benchmark experiments that demonstrate stable learning without catastrophic forgetting. We also demonstrate the critical role of developing these systems in a closed-loop manner where the environment and the agent's behaviors constrain and guide the learning process. To this end, we integrated the algorithm into an embodied simulated drone agent. The experiments show that the algorithm is capable of continuous learning of new tasks and under changed conditions with high classification accuracy (>94%) in a virtual environment, without catastrophic forgetting. The algorithm accepts high dimensional inputs from any state-of-the-art detection and feature extraction algorithms, making it a flexible addition to existing systems. We also describe future development efforts focused on imbuing the algorithm with mechanisms to seek out new knowledge as well as employ a broader range of neuromodulatory processes.

Short Duration Repetitive Transcranial Electrical Stimulation During Sleep Enhances Declarative Memory of Facts.

Transcranial electrical stimulation (tES) during sleep has been shown to successfully modulate memory consolidation. Here, we tested the effect of short duration repetitive tES (SDR-tES) during a daytime nap on the consolidation of declarative memory of facts in healthy individuals. We use a previously described approach to deliver the stimulation at regular intervals during non-rapid eye movement (NREM) sleep, specifically stage NREM2 and NREM3. Similar to previous studies using tES, we find enhanced memory performance compared to sham both after sleep and 48 h later. We also observed an increase in the proportion of time spent in NREM3 sleep and SDR-tES boosted the overall rate of slow oscillations (SOs) during NREM2/NREM3 sleep. Retrospective investigation of brain activity immediately preceding stimulation suggests that increases in the SO rate are more likely when stimulation is delivered during quiescent and asynchronous periods of activity in contrast to other closed-loop approaches which target phasic stimulation during ongoing SOs.

Closed-Loop Targeted Memory Reactivation during Sleep Improves Spatial Navigation.

Sounds associated with newly learned information that are replayed during non-rapid eye movement (NREM) sleep can improve recall in simple tasks. The mechanism for this improvement is presumed to be reactivation of the newly learned memory during sleep when consolidation takes place. We have developed an EEG-based closed-loop system to precisely deliver sensory stimulation at the time of down-state to up-state transitions during NREM sleep. Here, we demonstrate that applying this technology to participants performing a realistic navigation task in virtual reality results in a significant improvement in navigation efficiency after sleep that is accompanied by increases in the spectral power especially in the fast (12-15 Hz) sleep spindle band. Our results show promise for the application of sleep-based interventions to drive improvement in real-world tasks.

The impact of cerebellar transcranial direct current stimulation (tDCS) on learning fine-motor sequences.

The cerebellum has been shown to be important for skill learning, including the learning of motor sequences. We investigated whether cerebellar transcranial direct current stimulation (tDCS) would enhance learning of fine motor sequences. Because the ability to generalize or transfer to novel task variations or circumstances is a crucial goal of real world training, we also examined the effect of tDCS on performance of novel sequences after training. In Study 1, participants received either anodal, cathodal or sham stimulation while simultaneously practising three eight-element key press sequences in a non-repeating, interleaved order. Immediately after sequence practice with concurrent tDCS, a transfer session was given in which participants practised three interleaved novel sequences. No stimulation was given during transfer. An inhibitory effect of cathodal tDCS was found during practice, such that the rate of learning was slowed in comparison to the anodal and sham groups. In Study 2, participants received anodal or sham stimulation and a 24 h delay was added between the practice and transfer sessions to reduce mental fatigue. Although this consolidation period benefitted subsequent transfer for both tDCS groups, anodal tDCS enhanced transfer performance. Together, these studies demonstrate polarity-specific effects on fine motor sequence learning and generalization.This article is part of the themed issue 'New frontiers for statistical learning in the cognitive sciences'.

Cerebellar activation during motor sequence learning is associated with subsequent transfer to new sequences.

Effective learning results not only in improved performance on a practiced task, but also in the ability to transfer the acquired knowledge to novel, similar tasks. Using a modified serial reaction time (RT) task, the authors examined the ability to transfer to novel sequences after practicing sequences in a repetitive order versus a nonrepeating interleaved order. Interleaved practice resulted in better performance on new sequences than repetitive practice. In a second study, participants practiced interleaved sequences in a functional MRI (fMRI) scanner and received a transfer test of novel sequences. Transfer ability was positively correlated with cerebellar blood oxygen level dependent activity during practice, indicating that greater cerebellar engagement during training resulted in better subsequent transfer performance. Interleaved practice may thus result in a more generalized representation that is robust to interference, and the degree of activation in the cerebellum may be a reflection of the instantiation and engagement of internal models. (PsycINFO Database Record