Self-supervised learning on millions of primary RNA sequences from 72 vertebrates improves sequence-based RNA splicing prediction.
Brief Bioinform
; 25(3)2024 Mar 27.
Article
de En
| MEDLINE
| ID: mdl-38605640
ABSTRACT
Language models pretrained by self-supervised learning (SSL) have been widely utilized to study protein sequences, while few models were developed for genomic sequences and were limited to single species. Due to the lack of genomes from different species, these models cannot effectively leverage evolutionary information. In this study, we have developed SpliceBERT, a language model pretrained on primary ribonucleic acids (RNA) sequences from 72 vertebrates by masked language modeling, and applied it to sequence-based modeling of RNA splicing. Pretraining SpliceBERT on diverse species enables effective identification of evolutionarily conserved elements. Meanwhile, the learned hidden states and attention weights can characterize the biological properties of splice sites. As a result, SpliceBERT was shown effective on several downstream tasks zero-shot prediction of variant effects on splicing, prediction of branchpoints in humans, and cross-species prediction of splice sites. Our study highlighted the importance of pretraining genomic language models on a diverse range of species and suggested that SSL is a promising approach to enhance our understanding of the regulatory logic underlying genomic sequences.
Mots clés
Texte intégral:
1
Collection:
01-internacional
Base de données:
MEDLINE
Sujet principal:
Vertébrés
/
Épissage des ARN
Limites:
Animals
/
Humans
Langue:
En
Journal:
Brief Bioinform
/
Brief. bioinform
/
Briefings in bioinformatics
Sujet du journal:
BIOLOGIA
/
INFORMATICA MEDICA
Année:
2024
Type de document:
Article
Pays d'affiliation:
Chine
Pays de publication:
Royaume-Uni