Representations and generalization in artificial and brain neural networks.
Proc Natl Acad Sci U S A
; 121(27): e2311805121, 2024 Jul 02.
Article
em En
| MEDLINE
| ID: mdl-38913896
ABSTRACT
Humans and animals excel at generalizing from limited data, a capability yet to be fully replicated in artificial intelligence. This perspective investigates generalization in biological and artificial deep neural networks (DNNs), in both in-distribution and out-of-distribution contexts. We introduce two hypotheses First, the geometric properties of the neural manifolds associated with discrete cognitive entities, such as objects, words, and concepts, are powerful order parameters. They link the neural substrate to the generalization capabilities and provide a unified methodology bridging gaps between neuroscience, machine learning, and cognitive science. We overview recent progress in studying the geometry of neural manifolds, particularly in visual object recognition, and discuss theories connecting manifold dimension and radius to generalization capacity. Second, we suggest that the theory of learning in wide DNNs, especially in the thermodynamic limit, provides mechanistic insights into the learning processes generating desired neural representational geometries and generalization. This includes the role of weight norm regularization, network architecture, and hyper-parameters. We will explore recent advances in this theory and ongoing challenges. We also discuss the dynamics of learning and its relevance to the issue of representational drift in the brain.
Palavras-chave
Texto completo:
1
Coleções:
01-internacional
Base de dados:
MEDLINE
Assunto principal:
Encéfalo
/
Redes Neurais de Computação
Limite:
Animals
/
Humans
Idioma:
En
Revista:
Proc Natl Acad Sci U S A
Ano de publicação:
2024
Tipo de documento:
Article
País de publicação:
EEUU
/
ESTADOS UNIDOS
/
ESTADOS UNIDOS DA AMERICA
/
EUA
/
UNITED STATES
/
UNITED STATES OF AMERICA
/
US
/
USA