Complementary multi-modality molecular self-supervised learning via non-overlapping masking for property prediction.
Brief Bioinform
; 25(4)2024 May 23.
Article
in En
| MEDLINE
| ID: mdl-38801702
ABSTRACT
Self-supervised learning plays an important role in molecular representation learning because labeled molecular data are usually limited in many tasks, such as chemical property prediction and virtual screening. However, most existing molecular pre-training methods focus on one modality of molecular data, and the complementary information of two important modalities, SMILES and graph, is not fully explored. In this study, we propose an effective multi-modality self-supervised learning framework for molecular SMILES and graph. Specifically, SMILES data and graph data are first tokenized so that they can be processed by a unified Transformer-based backbone network, which is trained by a masked reconstruction strategy. In addition, we introduce a specialized non-overlapping masking strategy to encourage fine-grained interaction between these two modalities. Experimental results show that our framework achieves state-of-the-art performance in a series of molecular property prediction tasks, and a detailed ablation study demonstrates efficacy of the multi-modality framework and the masking strategy.
Key words
Full text:
1
Collection:
01-internacional
Database:
MEDLINE
Main subject:
Supervised Machine Learning
Language:
En
Journal:
Brief Bioinform
Journal subject:
BIOLOGIA
/
INFORMATICA MEDICA
Year:
2024
Type:
Article
Affiliation country:
China