Sleep-like unsupervised replay reduces catastrophic forgetting in artificial neural networks.
Nat Commun
; 13(1): 7742, 2022 12 15.
Article
in En
| MEDLINE
| ID: mdl-36522325
ABSTRACT
Artificial neural networks are known to suffer from catastrophic forgetting when learning multiple tasks sequentially, they perform well on the most recent task at the expense of previously learned tasks. In the brain, sleep is known to play an important role in incremental learning by replaying recent and old conflicting memory traces. Here we tested the hypothesis that implementing a sleep-like phase in artificial neural networks can protect old memories during new training and alleviate catastrophic forgetting. Sleep was implemented as off-line training with local unsupervised Hebbian plasticity rules and noisy input. In an incremental learning framework, sleep was able to recover old tasks that were otherwise forgotten. Previously learned memories were replayed spontaneously during sleep, forming unique representations for each class of inputs. Representational sparseness and neuronal activity corresponding to the old tasks increased while new task related activity decreased. The study suggests that spontaneous replay simulating sleep-like dynamics can alleviate catastrophic forgetting in artificial neural networks.
Full text:
1
Collection:
01-internacional
Database:
MEDLINE
Main subject:
Neural Networks, Computer
/
Learning
Language:
En
Journal:
Nat Commun
Journal subject:
BIOLOGIA
/
CIENCIA
Year:
2022
Document type:
Article
Affiliation country: