Uncertainty-based modulation for lifelong learning.

The creation of machine learning algorithms for intelligent agents capable of continuous, lifelong learning is a critical objective for algorithms being deployed on real-life systems in dynamic environments. Here we present an algorithm inspired by neuromodulatory mechanisms in the human brain that integrates and expands upon Stephen Grossberg's ground-breaking Adaptive Resonance Theory proposals. Specifically, it builds on the concept of uncertainty, and employs a series of "neuromodulatory" mechanisms to enable continuous learning, including self-supervised and one-shot learning. Algorithm components were evaluated in a series of benchmark experiments that demonstrate stable learning without catastrophic forgetting. We also demonstrate the critical role of developing these systems in a closed-loop manner where the environment and the agent's behaviors constrain and guide the learning process. To this end, we integrated the algorithm into an embodied simulated drone agent. The experiments show that the algorithm is capable of continuous learning of new tasks and under changed conditions with high classification accuracy (>94%) in a virtual environment, without catastrophic forgetting. The algorithm accepts high dimensional inputs from any state-of-the-art detection and feature extraction algorithms, making it a flexible addition to existing systems. We also describe future development efforts focused on imbuing the algorithm with mechanisms to seek out new knowledge as well as employ a broader range of neuromodulatory processes.

Short Duration Repetitive Transcranial Electrical Stimulation During Sleep Enhances Declarative Memory of Facts.

Transcranial electrical stimulation (tES) during sleep has been shown to successfully modulate memory consolidation. Here, we tested the effect of short duration repetitive tES (SDR-tES) during a daytime nap on the consolidation of declarative memory of facts in healthy individuals. We use a previously described approach to deliver the stimulation at regular intervals during non-rapid eye movement (NREM) sleep, specifically stage NREM2 and NREM3. Similar to previous studies using tES, we find enhanced memory performance compared to sham both after sleep and 48 h later. We also observed an increase in the proportion of time spent in NREM3 sleep and SDR-tES boosted the overall rate of slow oscillations (SOs) during NREM2/NREM3 sleep. Retrospective investigation of brain activity immediately preceding stimulation suggests that increases in the SO rate are more likely when stimulation is delivered during quiescent and asynchronous periods of activity in contrast to other closed-loop approaches which target phasic stimulation during ongoing SOs.

Spatio-temporal structure of sleep slow oscillations on the electrode manifold and its relation to spindles.

Electrophysiological sleep rhythms have been shown to impact human waking cognition, but their spatio-temporal dynamics are not understood. We investigated how slow oscillations (SOs; 0.5-4 Hz) are organized during a night of polysomnographically-recorded sleep, focusing on the scalp electrode manifold. We detected troughs of SOs at all electrodes independently and analyzed the concurrent SO troughs found in every other electrode within ±400 ms. We used a k-clustering algorithm to categorize the spatial patterns of SO trough co-occurrence into three types (Global, Local or Frontal) depending on their footprint on the electrode manifold during the considered time window. When comparing the clusters across non-rapid eye movement (NREM) sleep stages, we found a relatively larger fraction of Local SOs in slow wave sleep (SWS) compared to stage 2, and larger fraction of Global SOs in stage 2 compared to SWS. The probability of SO detection in time between two electrodes showed that SO troughs of all types co-occurred at some nearby electrodes, but only Global troughs had traveling wave profiles, moving anteriorly to posteriorly. Global SOs also had larger amplitudes at frontal electrodes and stronger coupling with fast spindles (12.5-16 Hz). Indeed, SO-spindle complexes were more likely to be detected following a Global SO trough compared to SOs in other clusters. Also, the phase-amplitude modulation of SOs over spindles (modulation vector) was higher for Global SOs across the electrode manifold. Given the recent evidence of a link between thalamocortical coupling and cognition, our findings suggest stronger cognitive relevance of Global SOs as compared to other SO types in sleep memory processing. Clinical Trials: No clinical trial is related to this study.

Closed-Loop Targeted Memory Reactivation during Sleep Improves Spatial Navigation.

Sounds associated with newly learned information that are replayed during non-rapid eye movement (NREM) sleep can improve recall in simple tasks. The mechanism for this improvement is presumed to be reactivation of the newly learned memory during sleep when consolidation takes place. We have developed an EEG-based closed-loop system to precisely deliver sensory stimulation at the time of down-state to up-state transitions during NREM sleep. Here, we demonstrate that applying this technology to participants performing a realistic navigation task in virtual reality results in a significant improvement in navigation efficiency after sleep that is accompanied by increases in the spectral power especially in the fast (12-15 Hz) sleep spindle band. Our results show promise for the application of sleep-based interventions to drive improvement in real-world tasks.

Toward a brain-based componential semantic representation.

Componential theories of lexical semantics assume that concepts can be represented by sets of features or attributes that are in some sense primitive or basic components of meaning. The binary features used in classical category and prototype theories are problematic in that these features are themselves complex concepts, leaving open the question of what constitutes a primitive feature. The present availability of brain imaging tools has enhanced interest in how concepts are represented in brains, and accumulating evidence supports the claim that these representations are at least partly "embodied" in the perception, action, and other modal neural systems through which concepts are experienced. In this study we explore the possibility of devising a componential model of semantic representation based entirely on such functional divisions in the human brain. We propose a basic set of approximately 65 experiential attributes based on neurobiological considerations, comprising sensory, motor, spatial, temporal, affective, social, and cognitive experiences. We provide normative data on the salience of each attribute for a large set of English nouns, verbs, and adjectives, and show how these attribute vectors distinguish a priori conceptual categories and capture semantic similarity. Robust quantitative differences between concrete object categories were observed across a large number of attribute dimensions. A within- versus between-category similarity metric showed much greater separation between categories than representations derived from distributional (latent semantic) analysis of text. Cluster analyses were used to explore the similarity structure in the data independent of a priori labels, revealing several novel category distinctions. We discuss how such a representation might deal with various longstanding problems in semantic theory, such as feature selection and weighting, representation of abstract concepts, effects of context on semantic retrieval, and conceptual combination. In contrast to componential models based on verbal features, the proposed representation systematically relates semantic content to large-scale brain networks and biologically plausible accounts of concept acquisition.

Caffeine-induced synaptic potentiation in hippocampal CA2 neurons.

Caffeine enhances cognition, but even high non-physiological doses have modest effects on synapses. A(1) adenosine receptors (A(1)Rs) are antagonized by caffeine and are most highly enriched in hippocampal CA2, which has not been studied in this context. We found that physiological doses of caffeine in vivo or A(1)R antagonists in vitro induced robust, long-lasting potentiation of synaptic transmission in rat CA2 without affecting other regions of the hippocampus.

RGS14 is a natural suppressor of both synaptic plasticity in CA2 neurons and hippocampal-based learning and memory.

Learning and memory have been closely linked to strengthening of synaptic connections between neurons (i.e., synaptic plasticity) within the dentate gyrus (DG)-CA3-CA1 trisynaptic circuit of the hippocampus. Conspicuously absent from this circuit is area CA2, an intervening hippocampal region that is poorly understood. Schaffer collateral synapses on CA2 neurons are distinct from those on other hippocampal neurons in that they exhibit a perplexing lack of synaptic long-term potentiation (LTP). Here we demonstrate that the signaling protein RGS14 is highly enriched in CA2 pyramidal neurons and plays a role in suppression of both synaptic plasticity at these synapses and hippocampal-based learning and memory. RGS14 is a scaffolding protein that integrates G protein and H-Ras/ERK/MAP kinase signaling pathways, thereby making it well positioned to suppress plasticity in CA2 neurons. Supporting this idea, deletion of exons 2-7 of the RGS14 gene yields mice that lack RGS14 (RGS14-KO) and now express robust LTP at glutamatergic synapses in CA2 neurons with no impact on synaptic plasticity in CA1 neurons. Treatment of RGS14-deficient CA2 neurons with a specific MEK inhibitor blocked this LTP, suggesting a role for ERK/MAP kinase signaling pathways in this process. When tested behaviorally, RGS14-KO mice exhibited marked enhancement in spatial learning and in object recognition memory compared with their wild-type littermates, but showed no differences in their performance on tests of nonhippocampal-dependent behaviors. These results demonstrate that RGS14 is a key regulator of signaling pathways linking synaptic plasticity in CA2 pyramidal neurons to hippocampal-based learning and memory but distinct from the canonical DG-CA3-CA1 circuit.

Regional differences in hippocampal calcium handling provide a cellular mechanism for limiting plasticity.

Although much is known about the mechanisms underlying synaptic plasticity, the cellular mechanisms that negatively regulate plasticity in some brain regions are considerably less studied. One region where neurons do not reliably express long-term potentiation (LTP) is the CA2 subfield of the hippocampus. Given the connection between synaptic plasticity and increases in postsynaptic [Ca(2+)], and that CA2 neurons express a large number of calcium-regulating proteins, we tested the hypothesis that the relative lack of LTP in CA2 results from differences in the calcium dynamics of these neurons. By measuring calcium-dependent fluorescence transients in dendritic spines, we show that CA2 neurons have smaller action potential-evoked intracellular Ca(2+) transients because of a higher endogenous Ca(2+)-buffering capacity and significantly higher rates of Ca(2+) extrusion when compared with CA1 and CA3 neurons. Perfusion with higher external [Ca(2+)] during induction restores LTP to CA2 neurons, suggesting that they possess the cellular machinery required for plasticity, but that the restriction of postsynaptic [Ca(2+)] limits its expression. Camstatin, an analogue of the calcium-modulating protein Pep-19 strongly expressed in CA2 neurons, blocked LTP and increased Ca(2+) extrusion in CA1 neurons, suggesting a role for extrusion in the regulation of plasticity in CA2. In agreement with this idea, we found that intracellular introduction of a PMCA pump inhibitor (carboxyeosin) allows for the induction of LTP in CA2 neurons. Our results indicate that regulation of postsynaptic [Ca(2+)] through modulation of extrusion and/or buffering regulates expression of LTP in CA2 and potentially other brain regions.

Ipsilateral input modifies the primary somatosensory cortex response to contralateral skin flutter.

We recorded the optical intrinsic signal response of squirrel monkey primary somatosensory cortex (SI) to 25 Hz vibrotactile ("flutter") stimulation applied independently to the thenar eminence on each hand and also to bilateral (simultaneous) stimulation of both thenars. The following observations were obtained in every subject (n = 5). (1) Ipsilateral stimulation was accompanied by an increase in absorbance within the SI hand region substantially smaller than the absorbance increase evoked by contralateral stimulation. (2) The absorbance increase evoked by simultaneous bilateral stimulation was smaller (by approximately 30%) than that evoked by contralateral stimulation. (3) The spatiointensive pattern of the SI response to bilateral flutter was distinctly different than the pattern that accompanied contralateral flutter stimulation: with contralateral flutter, the center of the responding region of SI underwent a large increase in absorbance, whereas absorbance decreased in the surrounding region; in contrast, during bilateral flutter, absorbance decreased (relative to that evoked by contralateral flutter) in the central region of SI but increased in the surround. The results raise the possibility that somatosensory perceptual experiences specific to bimanual tactile object exploration derive, at least in part, from the unique spatiointensive activity pattern evoked in SI when the stimulus makes contact with both hands. It is suggested that modulatory influences evoked by ipsilateral thenar flutter stimulation reach SI via a two-stage pathway involving interhemispheric (callosal) connections between information processing levels higher than SI and subsequently via intrahemispheric (corticocortical) projections to the SI hand region.

Amplitude-dependency of response of SI cortex to flutter stimulation.

BACKGROUND: It is established that increasing the amplitude of a flutter stimulus increases its perceived intensity. Although many studies have examined this phenomenon with regard to the responding afferent population, the way in which the intensity of a stimulus is coded in primary somatosensory cortex (SI) remains unclear. RESULTS: Optical intrinsic signal (OIS) imaging was used to study the evoked responses in SI of anesthetized squirrel monkeys by 25 Hz sinusoidal vertical skin displacement stimulation. Stimuli were 10 sec duration with a 50 sec inter-stimulus interval. Stimulus amplitude ranged from 50 to 400 microns and different amplitudes were interleaved. Control levels of activity were measured in the absence of stimulation, and used to compare with activation levels evoked by the different stimulus amplitudes. Stimulation of a discrete skin site on the forelimb evoked a prominent increase in absorbance within the forelimb representational region in cytoarchitectonic areas 3b and 1 of the contralateral hemisphere. An increase in stimulus amplitude led to a proportional increase in the magnitude of the absorbance increase in this region of areas 3b and 1 while surrounding cortex underwent a decrease in absorbance. Correlation maps revealed that as stimulus amplitude is increased, the spatial extent of the activated region in SI remains relatively constant, and the activity within this region increases progressively. Additionally, as stimulus amplitude is increased to suprathreshold levels, activity in the surround of the activated SI territory decreases, suggesting an increase in inhibition of neuronal activity within these regions. CONCLUSION: Increasing the amplitude of a flutter stimulus leads to a proportional increase in absorbance within the forelimb representational region of SI. This most likely reflects an increase in the firing rate of neurons in this region of SI. The relatively constant spatial extent of this stimulus-evoked increase in absorbance suggests that an increase in the amplitude of a 25 Hz skin stimulus does not evoke a larger area of SI neuronal activation due to an amplitude-dependent lateral inhibitory effect that spatially funnels the responding SI neuronal population.

Response of SI cortex to ipsilateral, contralateral and bilateral flutter stimulation in the cat.

BACKGROUND: While SII cortex is considered to be the first cortical stage of the pathway that integrates tactile information arising from both sides of the body, SI cortex is generally not considered as a region in which neuronal response is modulated by simultaneous stimulation of bilateral (and mirror-image) skin sites. RESULTS: Optical intrinsic signal imaging was used to evaluate the response of SI and SII in the same hemisphere to 25 Hz sinusoidal vertical skin displacement stimulation ("skin flutter") applied contralaterally, ipsilaterally, and bilaterally (simultaneously) to the central pads of the forepaws. A localized increase in absorbance in both SI and SII occurred in response to both contralateral and bilateral flutter stimulation. Ipsilateral flutter stimulation evoked a localized increase in absorbance in SII, but little or no change in SI absorbance. In the forepaw representational region of SI, however, bilateral stimulation of the central pads evoked a response substantially smaller (approximately 30-35% smaller) than the response to flutter stimulation of the contralateral central pad. CONCLUSION: The finding that the response of SI cortex to bilateral central pad flutter stimulation is substantially smaller than the response evoked by a contralateral flutter stimulus, together with the recently published observation that a region located posteriorly in SII responds with a substantially larger response to a bilateral flutter stimulus than the response evoked from the contralateral central pad, lead us to propose that the SI activity evoked by contralateral skin stimulation is suppressed/inhibited (via corticocortical connections between SII and SI in the same hemisphere) by the activity a simultaneous ipsilateral skin stimulus evokes in posterior SII.

Response of SII cortex to ipsilateral, contralateral and bilateral flutter stimulation in the cat.

BACKGROUND: A distinctive property of SII is that it is the first cortical stage of the somatosensory projection pathway that integrates information arising from both sides of the body. However, there is very little known about how inputs across the body mid-line are processed within SII. RESULTS: Optical intrinsic signal imaging was used to evaluate the response of primary somatosensory cortex (SI and SII in the same hemisphere) to 25 Hz sinusoidal vertical skin displacement stimulation ("skin flutter") applied contralaterally, ipsilaterally, and bilaterally to the central pads of the forepaws. A localized increase in absorbance in both SI and SII was evoked by both contralateral and bilateral flutter stimulation. Ipsilateral flutter stimulation evoked a localized increase in absorbance in SII, but not in SI. The SII region that responded with an increase in absorbance to ipsilateral stimulation was posterior to the region in which absorbance increased maximally in response to stimulation of the contralateral central pad. Additionally, in the posterior SII region that responded maximally to ipsilateral stimulation of the central pad, bilateral central pad stimulation approximated a linear summation of the SII responses to independent stimulation of the contralateral and ipsilateral central pads. Conversely, in anterior SII (the region that responded maximally to contralateral stimulation), bilateral stimulation was consistently less than the response evoked from the contralateral central pad. CONCLUSIONS: The results indicate that two regions located at neighboring, but distinctly different A-P levels of the anterior ectosylvian gyrus process input from opposite sides of the body midline in very different ways. The results suggest that the SII cortex, in the cat, can be subdivided into at least two functionally distinct regions and that these functionally distinct regions demonstrate a laterality preference within SII.

Epicardial fiber organization in swine right ventricle and its impact on propagation.

Fiber organization is important for myocardial excitation and contraction. It can be a major factor in arrhythmogenesis and current distribution during defibrillation shocks. In this study, we report the discovery of a previously undetected thin epicardial layer in swine right ventricle (RV) with distinctly different fiber orientation, which significantly affects epicardial propagation. Experiments were conducted in isolated coronary-perfused right ventricular free wall preparations (n=8) stained with the voltage-sensitive dye di-4-ANEPPS. Optical signals were recorded from the epicardium with a CCD video camera at 800 fps. Preparations were sectioned parallel to the epicardial surface with a resolution of 50 mum or better. To link the histological data with the observed activation patterns, resulting fiber angles were introduced into a 3D computer model to simulate the electrical activation and voltage-dependent optical signals. In all preparations, we detected a thin epicardial layer with almost no depth-dependent fiber rotation. The thickness of this layer (z(0)) varied from 110 to 930 microm. At the boundary of this layer, we observed an abrupt change in fiber angle by 64+/-13 degrees followed by a gradual fiber rotation in the underlying layers. In preparations with z(0) <700 microm, optical mapping during epicardial stimulation revealed unusual diamond- and rectangular-shaped activation fronts with two axes of fast conduction. Computer simulations accurately predicted the features of the experimentally recorded activation fronts. The free wall of swine RV has a thin epicardial layer with distinctly different fiber orientation, which can significantly affect propagation and give rise to unusually shaped activation fronts. This is important for understanding electrical propagation in the heart, and further refines the existing knowledge of myocardial fiber architecture.