Extensible benchmarking of methods that identify and quantify polyadenylation sites from RNA-seq data.

The tremendous rate with which data is generated and analysis methods emerge makes it increasingly difficult to keep track of their domain of applicability, assumptions, limitations, and consequently, of the efficacy and precision with which they solve specific tasks. Therefore, there is an increasing need for benchmarks, and for the provision of infrastructure for continuous method evaluation. APAeval is an international community effort, organized by the RNA Society in 2021, to benchmark tools for the identification and quantification of the usage of alternative polyadenylation (APA) sites from short-read, bulk RNA-sequencing (RNA-seq) data. Here, we reviewed 17 tools and benchmarked eight on their ability to perform APA identification and quantification, using a comprehensive set of RNA-seq experiments comprising real, synthetic, and matched 3'-end sequencing data. To support continuous benchmarking, we have incorporated the results into the OpenEBench online platform, which allows for continuous extension of the set of methods, metrics, and challenges. We envisage that our analyses will assist researchers in selecting the appropriate tools for their studies, while the containers and reproducible workflows could easily be deployed and extended to evaluate new methods or data sets.

High-throughput data and modeling reveal insights into the mechanisms of cooperative DNA-binding by transcription factor proteins.

Cooperative DNA-binding by transcription factor (TF) proteins is critical for eukaryotic gene regulation. In the human genome, many regulatory regions contain TF-binding sites in close proximity to each other, which can facilitate cooperative interactions. However, binding site proximity does not necessarily imply cooperative binding, as TFs can also bind independently to each of their neighboring target sites. Currently, the rules that drive cooperative TF binding are not well understood. In addition, it is oftentimes difficult to infer direct TF-TF cooperativity from existing DNA-binding data. Here, we show that in vitro binding assays using DNA libraries of a few thousand genomic sequences with putative cooperative TF-binding events can be used to develop accurate models of cooperativity and to gain insights into cooperative binding mechanisms. Using factors ETS1 and RUNX1 as our case study, we show that the distance and orientation between ETS1 sites are critical determinants of cooperative ETS1-ETS1 binding, while cooperative ETS1-RUNX1 interactions show more flexibility in distance and orientation and can be accurately predicted based on the affinity and sequence/shape features of the binding sites. The approach described here, combining custom experimental design with machine-learning modeling, can be easily applied to study the cooperative DNA-binding patterns of any TFs.

Extensible benchmarking of methods that identify and quantify polyadenylation sites from RNA-seq data.

The tremendous rate with which data is generated and analysis methods emerge makes it increasingly difficult to keep track of their domain of applicability, assumptions, and limitations and consequently, of the efficacy and precision with which they solve specific tasks. Therefore, there is an increasing need for benchmarks, and for the provision of infrastructure for continuous method evaluation. APAeval is an international community effort, organized by the RNA Society in 2021, to benchmark tools for the identification and quantification of the usage of alternative polyadenylation (APA) sites from short-read, bulk RNA-sequencing (RNA-seq) data. Here, we reviewed 17 tools and benchmarked eight on their ability to perform APA identification and quantification, using a comprehensive set of RNA-seq experiments comprising real, synthetic, and matched 3'-end sequencing data. To support continuous benchmarking, we have incorporated the results into the OpenEBench online platform, which allows for seamless extension of the set of methods, metrics, and challenges. We envisage that our analyses will assist researchers in selecting the appropriate tools for their studies. Furthermore, the containers and reproducible workflows generated in the course of this project can be seamlessly deployed and extended in the future to evaluate new methods or datasets.

GENESIS: Gene-Specific Machine Learning Models for Variants of Uncertain Significance Found in Catecholaminergic Polymorphic Ventricular Tachycardia and Long QT Syndrome-Associated Genes.

BACKGROUND: Cardiac channelopathies such as catecholaminergic polymorphic tachycardia and long QT syndrome predispose patients to fatal arrhythmias and sudden cardiac death. As genetic testing has become common in clinical practice, variants of uncertain significance (VUS) in genes associated with catecholaminergic polymorphic ventricular tachycardia and long QT syndrome are frequently found. The objective of this study was to predict pathogenicity of catecholaminergic polymorphic ventricular tachycardia-associated RYR2 VUS and long QT syndrome-associated VUS in KCNQ1, KCNH2, and SCN5A by developing gene-specific machine learning models and assessing them using cross-validation, cellular electrophysiological data, and clinical correlation. METHODS: The GENe-specific EnSemble grId Search framework was developed to identify high-performing machine learning models for RYR2, KCNQ1, KCNH2, and SCN5A using variant- and protein-specific inputs. Final models were applied to datasets of VUS identified from ClinVar and exome sequencing. Whole cell patch clamp and clinical correlation of selected VUS was performed. RESULTS: The GENe-specific EnSemble grId Search models outperformed alternative methods, with area under the receiver operating characteristics up to 0.87, average precisions up to 0.83, and calibration slopes as close to 1.0 (perfect) as 1.04. Blinded voltage-clamp analysis of HEK293T cells expressing 2 predicted pathogenic variants in KCNQ1 each revealed an ≈80% reduction of peak Kv7.1 current compared with WT. Normal Kv7.1 function was observed in KCNQ1-V241I HEK cells as predicted. Though predicted benign, loss of Kv7.1 function was observed for KCNQ1-V106D HEK cells. Clinical correlation of 9/10 variants supported model predictions. CONCLUSIONS: Gene-specific machine learning models may have a role in post-genetic testing diagnostic analyses by providing high performance prediction of variant pathogenicity.

Machine Learning Algorithms to Predict Mortality and Allocate Palliative Care for Older Patients With Hip Fracture.

OBJECTIVES: To evaluate a machine learning model designed to predict mortality for Medicare beneficiaries aged >65 years treated for hip fracture in Inpatient Rehabilitation Facilities (IRFs). DESIGN: Retrospective design/cohort analysis of Centers for Medicare & Medicaid Services Inpatient Rehabilitation Facility-Patient Assessment Instrument data. SETTING AND PARTICIPANTS: A total of 17,140 persons admitted to Medicare-certified IRFs in 2015 following hospitalization for hip fracture. MEASURES: Patient characteristics include sociodemographic (age, gender, race, and social support) and clinical factors (functional status at admission, chronic conditions) and IRF length of stay. Outcomes were 30-day and 1-year all-cause mortality. We trained and evaluated 2 classification models, logistic regression and a multilayer perceptron (MLP), to predict the probability of 30-day and 1-year mortality and evaluated the calibration, discrimination, and precision of the models. RESULTS: For 30-day mortality, MLP performed well [acc = 0.74, area under the receiver operating characteristic curve (AUROC) = 0.76, avg prec = 0.10, slope = 1.14] as did logistic regression (acc = 0.78, AUROC = 0.76, avg prec = 0.09, slope = 1.20). For 1-year mortality, the performances were similar for both MLP (acc = 0.68, AUROC = 0.75, avg prec = 0.32, slope = 0.96) and logistic regression (acc = 0.68, AUROC = 0.75, avg prec = 0.32, slope = 0.95). CONCLUSION AND IMPLICATIONS: A scoring system based on logistic regression may be more feasible to run in current electronic medical records. But MLP models may reduce cognitive burden and increase ability to calibrate to local data, yielding clinical specificity in mortality prediction so that palliative care resources may be allocated more effectively.