Symbiotic Graph Neural Networks for 3D Skeleton-Based Human Action Recognition and Motion Prediction.
IEEE Trans Pattern Anal Mach Intell
; 44(6): 3316-3333, 2022 06.
Article
en En
| MEDLINE
| ID: mdl-33481706
ABSTRACT
3D skeleton-based action recognition and motion prediction are two essential problems of human activity understanding. In many previous works 1) they studied two tasks separately, neglecting internal correlations; and 2) they did not capture sufficient relations inside the body. To address these issues, we propose a symbiotic model to handle two tasks jointly; and we propose two scales of graphs to explicitly capture relations among body-joints and body-parts. Together, we propose symbiotic graph neural networks, which contain a backbone, an action-recognition head, and a motion-prediction head. Two heads are trained jointly and enhance each other. For the backbone, we propose multi-branch multiscale graph convolution networks to extract spatial and temporal features. The multiscale graph convolution networks are based on joint-scale and part-scale graphs. The joint-scale graphs contain actional graphs, capturing action-based relations, and structural graphs, capturing physical constraints. The part-scale graphs integrate body-joints to form specific parts, representing high-level relations. Moreover, dual bone-based graphs and networks are proposed to learn complementary features. We conduct extensive experiments for skeleton-based action recognition and motion prediction with four datasets, NTU-RGB+D, Kinetics, Human3.6M, and CMU Mocap. Experiments show that our symbiotic graph neural networks achieve better performances on both tasks compared to the state-of-the-art methods.
Texto completo:
1
Colección:
01-internacional
Banco de datos:
MEDLINE
Asunto principal:
Algoritmos
/
Reconocimiento de Normas Patrones Automatizadas
Tipo de estudio:
Prognostic_studies
/
Risk_factors_studies
Límite:
Humans
Idioma:
En
Revista:
IEEE Trans Pattern Anal Mach Intell
Asunto de la revista:
INFORMATICA MEDICA
Año:
2022
Tipo del documento:
Article