Proformer: a hybrid macaron transformer model predicts expression values from promoter sequences.

The breakthrough high-throughput measurement of the cis-regulatory activity of millions of randomly generated promoters provides an unprecedented opportunity to systematically decode the cis-regulatory logic that determines the expression values. We developed an end-to-end transformer encoder architecture named Proformer to predict the expression values from DNA sequences. Proformer used a Macaron-like Transformer encoder architecture, where two half-step feed forward (FFN) layers were placed at the beginning and the end of each encoder block, and a separable 1D convolution layer was inserted after the first FFN layer and in front of the multi-head attention layer. The sliding k-mers from one-hot encoded sequences were mapped onto a continuous embedding, combined with the learned positional embedding and strand embedding (forward strand vs. reverse complemented strand) as the sequence input. Moreover, Proformer introduced multiple expression heads with mask filling to prevent the transformer models from collapsing when training on relatively small amount of data. We empirically determined that this design had significantly better performance than the conventional design such as using the global pooling layer as the output layer for the regression task. These analyses support the notion that Proformer provides a novel method of learning and enhances our understanding of how cis-regulatory sequences determine the expression values.

IRX3/5 regulate mitotic chromatid segregation and limb bud shape.

Pattern formation is influenced by transcriptional regulation as well as by morphogenetic mechanisms that shape organ primordia, although factors that link these processes remain under-appreciated. Here we show that, apart from their established transcriptional roles in pattern formation, IRX3/5 help to shape the limb bud primordium by promoting the separation and intercalation of dividing mesodermal cells. Surprisingly, IRX3/5 are required for appropriate cell cycle progression and chromatid segregation during mitosis, possibly in a nontranscriptional manner. IRX3/5 associate with, promote the abundance of, and share overlapping functions with co-regulators of cell division such as the cohesin subunits SMC1, SMC3, NIPBL and CUX1. The findings imply that IRX3/5 coordinate early limb bud morphogenesis with skeletal pattern formation.

Single cell lineage reconstruction using distance-based algorithms and the R package, DCLEAR.

BACKGROUND: DCLEAR is an R package used for single cell lineage reconstruction. The advances of CRISPR-based gene editing technologies have enabled the prediction of cell lineage trees based on observed edited barcodes from each cell. However, the performance of existing reconstruction methods of cell lineage trees was not accessed until recently. In response to this problem, the Allen Institute hosted the Cell Lineage Reconstruction Dream Challenge in 2020 to crowdsource relevant knowledge from across the world. Our team won sub-challenges 2 and 3 in the challenge competition. RESULTS: The DCLEAR package contained the R codes, which was submitted in response to sub-challenges 2 and 3. Our method consists of two steps: (1) distance matrix estimation and (2) the tree reconstruction from the distance matrix. We proposed two novel methods for distance matrix estimation as outlined in the DCLEAR package. Using our method, we find that two of the more sophisticated distance methods display a substantially improved level of performance compared to the traditional Hamming distance method. DCLEAR is open source and freely available from R CRAN and from under the GNU General Public License, version 3. CONCLUSIONS: DCLEAR is a powerful resource for single cell lineage reconstruction.

Sall4 regulates neuromesodermal progenitors and their descendants during body elongation in mouse embryos.

Bi-potential neuromesodermal progenitors (NMPs) produce both neural and paraxial mesodermal progenitors in the trunk and tail during vertebrate body elongation. We show that Sall4, a pluripotency-related transcription factor gene, has multiple roles in regulating NMPs and their descendants in post-gastrulation mouse embryos. Sall4 deletion using TCre caused body/tail truncation, reminiscent of early depletion of NMPs, suggesting a role of Sall4 in NMP maintenance. This phenotype became significant at the time of the trunk-to-tail transition, suggesting that Sall4 maintenance of NMPs enables tail formation. Sall4 mutants exhibit expanded neural and reduced mesodermal tissues, indicating a role of Sall4 in NMP differentiation balance. Mechanistically, we show that Sall4 promotion of WNT/ß-catenin signaling contributes to NMP maintenance and differentiation balance. RNA-Seq and SALL4 ChIP-Seq analyses support the notion that Sall4 regulates both mesodermal and neural development. Furthermore, in the mesodermal compartment, genes regulating presomitic mesoderm differentiation are downregulated in Sall4 mutants. In the neural compartment, we show that differentiation of NMPs towards post-mitotic neuron is accelerated in Sall4 mutants. Our results collectively provide evidence supporting the role of Sall4 in regulating NMPs and their descendants.

ETV2 (Ets Variant Transcription Factor 2)-Rhoj Cascade Regulates Endothelial Progenitor Cell Migration During Embryogenesis.

OBJECTIVE: Endothelial progenitors migrate early during embryogenesis to form the primary vascular plexus. The regulatory mechanisms that govern their migration are not completely defined. Here, we describe a novel role for ETV2 (Ets variant transcription factor 2) in cell migration and provide evidence for an ETV2-Rhoj network as a mechanism responsible for this process. Approach and Results: Analysis of RNAseq datasets showed robust enrichment of migratory/motility pathways following overexpression of ETV2 during mesodermal differentiation. We then analyzed ETV2 chromatin immunoprecipitation-seq and assay for transposase accessible chromatin-seq datasets, which showed enrichment of chromatin immunoprecipitation-seq peaks with increased chromatin accessibility in migratory genes following overexpression of ETV2. Migratory assays showed that overexpression of ETV2 enhanced cell migration in mouse embryonic stem cells, embryoid bodies, and mouse embryonic fibroblasts. Knockout of Etv2 led to migratory defects of Etv2-EYFP+ angioblasts to their predefined regions of developing embryos relative to wild-type controls at embryonic day (E) 8.5, supporting its role during migration. Mechanistically, we showed that ETV2 binds the promoter region of Rhoj serving as an upstream regulator of cell migration. Single-cell RNAseq analysis of Etv2-EYFP+ sorted cells revealed coexpression of Etv2 and Rhoj in endothelial progenitors at E7.75 and E8.25. Overexpression of ETV2 led to a robust increase in Rhoj in both embryoid bodies and mouse embryonic fibroblasts, whereas, its expression was abolished in the Etv2 knockout embryoid bodies. Finally, shRNA-mediated knockdown of Rhoj resulted in migration defects, which were partially rescued by overexpression of ETV2. CONCLUSIONS: These results define an ETV2-Rhoj cascade, which is important for the regulation of endothelial progenitor cell migration.

ETV2-null porcine embryos survive to post-implantation following incomplete enucleation.

Blind enucleation is used in porcine somatic cell nuclear transfer (SCNT) to remove the metaphase II (MII) spindle from the oocyte. Deviation of the MII spindle location, however, leads to incomplete enucleation (IE). Here, we report that the rate of complete enucleation (CE) using the blind method was 80.2 ± 1.7%, although this significantly increased when the polar body-MII deviation was minimized (≦45°). While it is established that IE embryos will not survive to full term, the effect of IE on early stage development is unknown. We have previously demonstrated in mice and pigs that ETV2 deletion results in embryonic lethality due to the lack of hematoendothelial lineages. We observed that ETV2-null cloned embryos derived from blindly and incompletely enucleated oocytes had both WT and mutant sequences at E18 and, using FISH analysis, we observed triploidy. We also compared SCNT embryos generated from either CE or intentionally IE oocytes using the spindle viewer system. We observed a higher in vitro blastocyst rate in the IE versus the CE-SCNT embryos (31.9 ± 3.2% vs 21.0 ± 2.1%). Based on known processes in normal fertilization, we infer that the IE-SCNT embryos extruded the haploid second PB after fusion with donor fibroblasts and formed a near-triploid aneuploid nucleus in each blastomere. These studies demonstrate the peri-implantation survival of residual haploid nuclei following IE and emphasize the need for complete enucleation especially for the analysis of SCNT embryos in the peri-implantation stage and will, further, impact the field of reverse xenotransplantation.

PVT1 dependence in cancer with MYC copy-number increase.

'Gain' of supernumerary copies of the 8q24.21 chromosomal region has been shown to be common in many human cancers and is associated with poor prognosis. The well-characterized myelocytomatosis (MYC) oncogene resides in the 8q24.21 region and is consistently co-gained with an adjacent 'gene desert' of approximately 2 megabases that contains the long non-coding RNA gene PVT1, the CCDC26 gene candidate and the GSDMC gene. Whether low copy-number gain of one or more of these genes drives neoplasia is not known. Here we use chromosome engineering in mice to show that a single extra copy of either the Myc gene or the region encompassing Pvt1, Ccdc26 and Gsdmc fails to advance cancer measurably, whereas a single supernumerary segment encompassing all four genes successfully promotes cancer. Gain of PVT1 long non-coding RNA expression was required for high MYC protein levels in 8q24-amplified human cancer cells. PVT1 RNA and MYC protein expression correlated in primary human tumours, and copy number of PVT1 was co-increased in more than 98% of MYC-copy-increase cancers. Ablation of PVT1 from MYC-driven colon cancer line HCT116 diminished its tumorigenic potency. As MYC protein has been refractory to small-molecule inhibition, the dependence of high MYC protein levels on PVT1 long non-coding RNA provides a much needed therapeutic target.

DrImpute: imputing dropout events in single cell RNA sequencing data.

BACKGROUND: The single cell RNA sequencing (scRNA-seq) technique begin a new era by allowing the observation of gene expression at the single cell level. However, there is also a large amount of technical and biological noise. Because of the low number of RNA transcriptomes and the stochastic nature of the gene expression pattern, there is a high chance of missing nonzero entries as zero, which are called dropout events. RESULTS: We develop DrImpute to impute dropout events in scRNA-seq data. We show that DrImpute has significantly better performance on the separation of the dropout zeros from true zeros than existing imputation algorithms. We also demonstrate that DrImpute can significantly improve the performance of existing tools for clustering, visualization and lineage reconstruction of nine published scRNA-seq datasets. CONCLUSIONS: DrImpute can serve as a very useful addition to the currently existing statistical tools for single cell RNA-seq analysis. DrImpute is implemented in R and is available at https://github.com/gongx030/DrImpute .

Pathologic Stimulus Determines Lineage Commitment of Cardiac C-kit+ Cells.

BACKGROUND: Although cardiac c-kit+ cells are being tested in clinical trials, the circumstances that determine lineage differentiation of c-kit+ cells in vivo are unknown. Recent findings suggest that endogenous cardiac c-kit+ cells rarely contribute cardiomyocytes to the adult heart. We assessed whether various pathological stimuli differentially affect the eventual cell fates of c-kit+ cells. METHODS: We used single-cell sequencing and genetic lineage tracing of c-kit+ cells to determine whether various pathological stimuli would result in different fates of c-kit+ cells. RESULTS: Single-cell sequencing of cardiac CD45-c-kit+ cells showed innate heterogeneity, indicative of the existence of vascular and mesenchymal c-kit+ cells in normal hearts. Cardiac pressure overload resulted in a modest increase in c-kit-derived cardiomyocytes, with significant increases in the numbers of endothelial cells and fibroblasts. Doxorubicin-induced acute cardiotoxicity did not increase c-kit-derived endothelial cell fates but instead induced cardiomyocyte differentiation. Mechanistically, doxorubicin-induced DNA damage in c-kit+ cells resulted in expression of p53. Inhibition of p53 blocked cardiomyocyte differentiation in response to doxorubicin, whereas stabilization of p53 was sufficient to increase c-kit-derived cardiomyocyte differentiation. CONCLUSIONS: These results demonstrate that different pathological stimuli induce different cell fates of c-kit+ cells in vivo. Although the overall rate of cardiomyocyte formation from c-kit+ cells is still below clinically relevant levels, we show that p53 is central to the ability of c-kit+ cells to adopt cardiomyocyte fates, which could lead to the development of strategies to preferentially generate cardiomyocytes from c-kit+ cells.

Kelch Repeat and BTB Domain Containing Protein 5 (Kbtbd5) Regulates Skeletal Muscle Myogenesis through the E2F1-DP1 Complex.

We have previously isolated a muscle-specific Kelch gene, Kelch repeat and BTB domain containing protein 5 (Kbtbd5)/Kelch-like protein 40 (Klhl40). In this report, we identified DP1 as a direct interacting factor for Kbtbd5 using a yeast two-hybrid screen and in vitro binding assays. Our studies demonstrate that Kbtbd5 interacts and regulates the cytoplasmic localization of DP1. GST pulldown assays demonstrate that the dimerization domain of DP1 interacts with all three of the Kbtbd5 domains. We further show that Kbtbd5 promotes the ubiquitination and degradation of DP1, thereby inhibiting E2F1-DP1 activity. To investigate the in vivo function of Kbtbd5, we used gene disruption technology and engineered Kbtbd5 null mice. Targeted deletion of Kbtbd5 resulted in postnatal lethality. Histological studies reveal that the Kbtbd5 null mice have smaller muscle fibers, a disorganized sarcomeric structure, increased extracellular matrix, and decreased numbers of mitochondria compared with wild-type controls. RNA sequencing and quantitative PCR analyses demonstrate the up-regulation of E2F1 target apoptotic genes (Bnip3 and p53inp1) in Kbtbd5 null skeletal muscle. Consistent with these observations, the cellular apoptosis in Kbtbd5 null mice was increased. Breeding of Kbtbd5 null mouse into the E2F1 null background rescues the lethal phenotype of the Kbtbd5 null mice but not the growth defect. The expression of Bnip3 and p53inp1 in Kbtbd5 mutant skeletal muscle are also restored to control levels in the E2F1 null background. In summary, our studies demonstrate that Kbtbd5 regulates skeletal muscle myogenesis through the regulation of E2F1-DP1 activity.

Inferring dynamic gene regulatory networks in cardiac differentiation through the integration of multi-dimensional data.

BACKGROUND: Decoding the temporal control of gene expression patterns is key to the understanding of the complex mechanisms that govern developmental decisions during heart development. High-throughput methods have been employed to systematically study the dynamic and coordinated nature of cardiac differentiation at the global level with multiple dimensions. Therefore, there is a pressing need to develop a systems approach to integrate these data from individual studies and infer the dynamic regulatory networks in an unbiased fashion. RESULTS: We developed a two-step strategy to integrate data from (1) temporal RNA-seq, (2) temporal histone modification ChIP-seq, (3) transcription factor (TF) ChIP-seq and (4) gene perturbation experiments to reconstruct the dynamic network during heart development. First, we trained a logistic regression model to predict the probability (LR score) of any base being bound by 543 TFs with known positional weight matrices. Second, four dimensions of data were combined using a time-varying dynamic Bayesian network model to infer the dynamic networks at four developmental stages in the mouse [mouse embryonic stem cells (ESCs), mesoderm (MES), cardiac progenitors (CP) and cardiomyocytes (CM)]. Our method not only infers the time-varying networks between different stages of heart development, but it also identifies the TF binding sites associated with promoter or enhancers of downstream genes. The LR scores of experimentally verified ESCs and heart enhancers were significantly higher than random regions (p <10(-100)), suggesting that a high LR score is a reliable indicator for functional TF binding sites. Our network inference model identified a region with an elevated LR score approximately -9400 bp upstream of the transcriptional start site of Nkx2-5, which overlapped with a previously reported enhancer region (-9435 to -8922 bp). TFs such as Tead1, Gata4, Msx2, and Tgif1 were predicted to bind to this region and participate in the regulation of Nkx2-5 gene expression. Our model also predicted the key regulatory networks for the ESC-MES, MES-CP and CP-CM transitions. CONCLUSION: We report a novel method to systematically integrate multi-dimensional -omics data and reconstruct the gene regulatory networks. This method will allow one to rapidly determine the cis-modules that regulate key genes during cardiac differentiation.

Cooperative interaction of Etv2 and Gata2 regulates the development of endothelial and hematopoietic lineages.

Regulatory mechanisms that govern lineage specification of the mesodermal progenitors to become endothelial and hematopoietic cells remain an area of intense interest. Both Ets and Gata factors have been shown to have important roles in the transcriptional regulation in endothelial and hematopoietic cells. We previously reported Etv2 as an essential regulator of vasculogenesis and hematopoiesis. In the present study, we demonstrate that Gata2 is co-expressed and interacts with Etv2 in the endothelial and hematopoietic cells in the early stages of embryogenesis. Our studies reveal that Etv2 interacts with Gata2 in vitro and in vivo. The protein-protein interaction between Etv2 and Gata2 is mediated by the Ets and Gata domains. Using the embryoid body differentiation system, we demonstrate that co-expression of Gata2 augments the activity of Etv2 in promoting endothelial and hematopoietic lineage differentiation. We also identify Spi1 as a common downstream target gene of Etv2 and Gata2. We provide evidence that Etv2 and Gata2 bind to the Spi1 promoter in vitro and in vivo. In summary, we propose that Gata2 functions as a cofactor of Etv2 in the transcriptional regulation of mesodermal progenitors during embryogenesis.

Evaluation and optimization of sequence-based gene regulatory deep learning models.

Neural networks have emerged as immensely powerful tools in predicting functional genomic regions, notably evidenced by recent successes in deciphering gene regulatory logic. However, a systematic evaluation of how model architectures and training strategies impact genomics model performance is lacking. To address this gap, we held a DREAM Challenge where competitors trained models on a dataset of millions of random promoter DNA sequences and corresponding expression levels, experimentally determined in yeast, to best capture the relationship between regulatory DNA and gene expression. For a robust evaluation of the models, we designed a comprehensive suite of benchmarks encompassing various sequence types. While some benchmarks produced similar results across the top-performing models, others differed substantially. All top-performing models used neural networks, but diverged in architectures and novel training strategies, tailored to genomics sequence data. To dissect how architectural and training choices impact performance, we developed the Prix Fixe framework to divide any given model into logically equivalent building blocks. We tested all possible combinations for the top three models and observed performance improvements for each. The DREAM Challenge models not only achieved state-of-the-art results on our comprehensive yeast dataset but also consistently surpassed existing benchmarks on Drosophila and human genomic datasets. Overall, we demonstrate that high-quality gold-standard genomics datasets can drive significant progress in model development.

SeATAC: a tool for exploring the chromatin landscape and the role of pioneer factors.

Assay for Transposase-Accessible Chromatin with sequencing (ATAC-seq) reveals chromatin accessibility across the genome. Currently, no method specifically detects differential chromatin accessibility. Here, SeATAC uses a conditional variational autoencoder model to learn the latent representation of ATAC-seq V-plots and outperforms MACS2 and NucleoATAC on six separate tasks. Applying SeATAC to several pioneer factor-induced differentiation or reprogramming ATAC-seq datasets suggests that induction of these factors not only relaxes the closed chromatin but also decreases chromatin accessibility of 20% to 30% of their target sites. SeATAC is a novel tool to accurately reveal genomic regions with differential chromatin accessibility from ATAC-seq data.

MCHeart: Multi-Channel-Based Heart Signal Processing Scheme for Heart Noise Detection Using Deep Learning.

In this study, we constructed a model to predict abnormal cardiac sounds using a diverse set of auscultation data collected from various auscultation positions. Abnormal heart sounds were identified by extracting features such as peak intervals and noise characteristics during systole and diastole. Instead of using raw signal data, we transformed them into log-mel 2D spectrograms, which were employed as input variables for the CNN model. The advancement of our model involves integrating a deep learning architecture with feature extraction techniques based on existing knowledge of cardiac data. Specifically, we propose a multi-channel-based heart signal processing (MCHeart) scheme, which incorporates our proposed features into the deep learning model. Additionally, we introduce the ReLCNN model by applying residual blocks and MHA mechanisms to the LCNN architecture. By adding murmur features with a smoothing function and training the ReLCNN model, the weighted accuracy of the model increased from 79.6% to 83.6%, showing a performance improvement of approximately 4% point compared to the LCNN baseline model.

ETV2 and VEZF1 interaction and regulation of the hematoendothelial lineage during embryogenesis.

Ets variant 2 (Etv2), a member of the Ets factor family, has an essential role in the formation of endothelial and hematopoietic cell lineages during embryonic development. The functional role of ETS transcription factors is, in part, dependent on the interacting proteins. There are relatively few studies exploring the coordinated interplay between ETV2 and its interacting proteins that regulate mesodermal lineage determination. In order to identify novel ETV2 interacting partners, a yeast two-hybrid analysis was performed and the C2H2 zinc finger transcription factor VEZF1 (vascular endothelial zinc finger 1) was identified as a binding factor, which was specifically expressed within the endothelium during vascular development. To confirm this interaction, co-immunoprecipitation and GST pull down assays demonstrated the direct interaction between ETV2 and VEZF1. During embryoid body differentiation, Etv2 achieved its peak expression at day 3.0 followed by rapid downregulation, on the other hand Vezf1 expression increased through day 6 of EB differentiation. We have previously shown that ETV2 potently activated Flt1 gene transcription. Using a Flt1 promoter-luciferase reporter assay, we demonstrated that VEZF1 co-activated the Flt1 promoter. Electrophoretic mobility shift assay and Chromatin immunoprecipitation established VEZF1 binding to the Flt1 promoter. Vezf1 knockout embryonic stem cells had downregulation of hematoendothelial marker genes when undergoing embryoid body mediated mesodermal differentiation whereas overexpression of VEZF1 induced the expression of hematoendothelial genes during differentiation. These current studies provide insight into the co-regulation of the hemato-endothelial lineage development via a co-operative interaction between ETV2 and VEZF1.

FOXK1 regulates Wnt signalling to promote cardiogenesis.

AIMS: Congenital heart disease (CHD) is the most common genetic birth defect, which has considerable morbidity and mortality. We focused on deciphering key regulators that govern cardiac progenitors and cardiogenesis. FOXK1 is a forkhead/winged helix transcription factor known to regulate cell cycle kinetics and is restricted to mesodermal progenitors, somites, and heart. In the present study, we define an essential role for FOXK1 during cardiovascular development. METHODS AND RESULTS: We used the mouse embryoid body system to differentiate control and Foxk1 KO embryonic stem cells into mesodermal, cardiac progenitor cells and mature cardiac cells. Using flow cytometry, immunohistochemistry, cardiac beating, transcriptional and chromatin immunoprecipitation quantitative polymerase chain reaction assays, bulk RNA sequencing (RNAseq) and assay for transposase-accessible chromatin using sequencing (ATACseq) analyses, FOXK1 was observed to be an important regulator of cardiogenesis. Flow cytometry analyses revealed perturbed cardiogenesis in Foxk1 KO embryoid bodies (EBs). Bulk RNAseq analysis at two developmental stages showed a significant reduction of the cardiac molecular program in Foxk1 KO EBs compared to the control EBs. ATACseq analysis during EB differentiation demonstrated that the chromatin landscape nearby known important regulators of cardiogenesis was significantly relaxed in control EBs compared to Foxk1 KO EBs. Furthermore, we demonstrated that in the absence of FOXK1, cardiac differentiation was markedly impaired by assaying for cardiac Troponin T expression and cardiac contractility. We demonstrate that FOXK1 is an important regulator of cardiogenesis by repressing the Wnt/ß-catenin signalling pathway and thereby promoting differentiation. CONCLUSION: These results identify FOXK1 as an essential transcriptional and epigenetic regulator of cardiovascular development. Mechanistically, FOXK1 represses Wnt signalling to promote the development of cardiac progenitor cells.

ETV2 functions as a pioneer factor to regulate and reprogram the endothelial lineage.

The vasculature is an essential organ for the delivery of blood and oxygen to all tissues of the body and is thus relevant to the treatment of ischaemic diseases, injury-induced regeneration and solid tumour growth. Previously, we demonstrated that ETV2 is an essential transcription factor for the development of cardiac, endothelial and haematopoietic lineages. Here we report that ETV2 functions as a pioneer factor that relaxes closed chromatin and regulates endothelial development. By comparing engineered embryonic stem cell differentiation and reprogramming models with multi-omics techniques, we demonstrated that ETV2 was able to bind nucleosomal DNA and recruit BRG1. BRG1 recruitment remodelled chromatin around endothelial genes and helped to maintain an open configuration, resulting in increased H3K27ac deposition. Collectively, these results will serve as a platform for the development of therapeutic initiatives directed towards cardiovascular diseases and solid tumours.