A domain-agnostic approach for characterization of lifelong learning systems.

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of (1) Continuous Learning, (2) Transfer and Adaptation, and (3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.

Uncertainty-based modulation for lifelong learning.

The creation of machine learning algorithms for intelligent agents capable of continuous, lifelong learning is a critical objective for algorithms being deployed on real-life systems in dynamic environments. Here we present an algorithm inspired by neuromodulatory mechanisms in the human brain that integrates and expands upon Stephen Grossberg's ground-breaking Adaptive Resonance Theory proposals. Specifically, it builds on the concept of uncertainty, and employs a series of "neuromodulatory" mechanisms to enable continuous learning, including self-supervised and one-shot learning. Algorithm components were evaluated in a series of benchmark experiments that demonstrate stable learning without catastrophic forgetting. We also demonstrate the critical role of developing these systems in a closed-loop manner where the environment and the agent's behaviors constrain and guide the learning process. To this end, we integrated the algorithm into an embodied simulated drone agent. The experiments show that the algorithm is capable of continuous learning of new tasks and under changed conditions with high classification accuracy (>94%) in a virtual environment, without catastrophic forgetting. The algorithm accepts high dimensional inputs from any state-of-the-art detection and feature extraction algorithms, making it a flexible addition to existing systems. We also describe future development efforts focused on imbuing the algorithm with mechanisms to seek out new knowledge as well as employ a broader range of neuromodulatory processes.

A newly identified nociresponsive region in the transitional zone (TZ) in rat sensorimotor cortex.

The primary somatosensory cortex (S1) comprises a number of functionally distinct regions, reflecting the diversity of somatosensory receptor submodalities innervating the body. In particular, two spatially and functionally distinct nociceptive regions have been described in primate S1 (Vierck et al., 2013; Whitsel et al., 2019). One region is located mostly in Brodmann cytoarchitectonic area 1, where a subset of neurons exhibit functional characteristics associated with myelinated Aδ nociceptors and perception of 1st/sharp, discriminative pain. The second region is located at the transition between S1 and primary motor cortex (M1) in area 3a, where neurons exhibit functional characteristics associated with unmyelinated C nociceptors and perception of 2nd/slow, burning pain. To test the hypothesis that in rats the transitional zone (TZ) - which is a dysgranular region at the transition between M1 and S1 - is the functional equivalent of the nociresponsive region of area 3a in primates, extracellular spike discharge activity was recorded from TZ neurons in rats under general isoflurane anesthesia. Thermonoxious stimuli were applied by lowering the contralateral forepaw or hindpaw into a 48-51 °C heated water bath for 5-10 s. Neurons in TZ were found to be minimally affected by non-noxious somatosensory stimuli, but highly responsive to thermonoxious skin stimuli in a slow temporal summation manner closely resembling that of nociresponsive neurons in primate area 3a. Selective inactivation of TZ by topical lidocaine application suppressed or delayed the nociceptive withdrawal reflex, suggesting that TZ exerts a tonic facilitatory influence over spinal cord neurons producing this reflex. In conclusion, TZ appears to be a rat homolog of the nociresponsive part of monkey area 3a. A possibility is considered that this region might be primarily engaged in autonomic aspects of nociception.