Combining Reinforcement Learning and Tensor Networks, with an Application to Dynamical Large Deviations.

We present a framework to integrate tensor network (TN) methods with reinforcement learning (RL) for solving dynamical optimization tasks. We consider the RL actor-critic method, a model-free approach for solving RL problems, and introduce TNs as the approximators for its policy and value functions. Our "actor-critic with tensor networks" (ACTeN) method is especially well suited to problems with large and factorizable state and action spaces. As an illustration of the applicability of ACTeN we solve the exponentially hard task of sampling rare trajectories in two paradigmatic stochastic models, the East model of glasses and the asymmetric simple exclusion process, the latter being particularly challenging to other methods due to the absence of detailed balance. With substantial potential for further integration with the vast array of existing RL methods, the approach introduced here is promising both for applications in physics and to multi-agent RL problems more generally.

Reinforcement learning of rare diffusive dynamics.

We present a method to probe rare molecular dynamics trajectories directly using reinforcement learning. We consider trajectories that are conditioned to transition between regions of configuration space in finite time, such as those relevant in the study of reactive events, and trajectories exhibiting rare fluctuations of time-integrated quantities in the long time limit, such as those relevant in the calculation of large deviation functions. In both cases, reinforcement learning techniques are used to optimize an added force that minimizes the Kullback-Leibler divergence between the conditioned trajectory ensemble and a driven one. Under the optimized added force, the system evolves the rare fluctuation as a typical one, affording a variational estimate of its likelihood in the original trajectory ensemble. Low variance gradients employing value functions are proposed to increase the convergence of the optimal force. The method we develop employing these gradients leads to efficient and accurate estimates of both the optimal force and the likelihood of the rare event for a variety of model systems.

Comparison of splice sites reveals that long noncoding RNAs are evolutionarily well conserved.

Large-scale RNA sequencing has revealed a large number of long mRNA-like transcripts (lncRNAs) that do not code for proteins. The evolutionary history of these lncRNAs has been notoriously hard to study systematically due to their low level of sequence conservation that precludes comprehensive homology-based surveys and makes them nearly impossible to align. An increasing number of special cases, however, has been shown to be at least as old as the vertebrate lineage. Here we use the conservation of splice sites to trace the evolution of lncRNAs. We show that >85% of the human GENCODE lncRNAs were already present at the divergence of placental mammals and many hundreds of these RNAs date back even further. Nevertheless, we observe a fast turnover of intron/exon structures. We conclude that lncRNA genes are evolutionary ancient components of vertebrate genomes that show an unexpected and unprecedented evolutionary plasticity. We offer a public web service (http://splicemap.bioinf.uni-leipzig.de) that allows to retrieve sets of orthologous splice sites and to produce overview maps of evolutionarily conserved splice sites for visualization and further analysis. An electronic supplement containing the ncRNA data sets used in this study is available at http://www.bioinf.uni-leipzig.de/publications/supplements/12-001.

Application of an NGS-based 28-gene panel in myeloproliferative neoplasms reveals distinct mutation patterns in essential thrombocythaemia, primary myelofibrosis and polycythaemia vera.

Molecular routine diagnostics for BCR-ABL1-negative myeloproliferative neoplasms (MPN) currently focusses on mutations in JAK2, CALR and MPL. In recent years, recurrent mutations in MPNs have been identified in several other genes. We here present the validation of a next generation sequencing (NGS)-based 28-gene panel and its use in MPN. We analysed the mutation status of 28 genes in 100 MPN patients [40 essential thrombocythaemia (ET), 30 primary myelofibrosis (PMF), 30 polycythaemia vera (PV)] and found two or more mutated genes in 53 patients. Moreover, significantly more mutated splicing genes (SF3B1, SRSF2 and U2AF1) were present in PMF (0·60 mutated genes/patient) compared to ET (0·15) while no mutations in splicing genes were found in PV. Additionally, chromatin modification genes (ASXL1 and EZH2) were frequently mutated in PMF patients (0·50) and, to a significantly lesser extent, in ET (0·13) and PV (0·07). Contrarily, DNA methylation genes (DNMT3A, IDH1, IDH2 and TET2) were mutated most often in PV (0·5) and less frequently in ET (0·23) and PMF (0·20), but without reaching statistical significance. Our results demonstrate the feasibility and utility of NGS-based panel diagnostics for MPN. With 53% of the patients bearing two or more mutated genes, their prognostic relevance needs further studies.

BlockClust: efficient clustering and classification of non-coding RNAs from short read RNA-seq profiles.

SUMMARY: Non-coding RNAs (ncRNAs) play a vital role in many cellular processes such as RNA splicing, translation, gene regulation. However the vast majority of ncRNAs still have no functional annotation. One prominent approach for putative function assignment is clustering of transcripts according to sequence and secondary structure. However sequence information is changed by post-transcriptional modifications, and secondary structure is only a proxy for the true 3D conformation of the RNA polymer. A different type of information that does not suffer from these issues and that can be used for the detection of RNA classes, is the pattern of processing and its traces in small RNA-seq reads data. Here we introduce BlockClust, an efficient approach to detect transcripts with similar processing patterns. We propose a novel way to encode expression profiles in compact discrete structures, which can then be processed using fast graph-kernel techniques. We perform both unsupervised clustering and develop family specific discriminative models; finally we show how the proposed approach is scalable, accurate and robust across different organisms, tissues and cell lines. AVAILABILITY: The whole BlockClust galaxy workflow including all tool dependencies is available at http://toolshed.g2.bx.psu.edu/view/rnateam/blockclust_workflow.

CRISPRmap: an automated classification of repeat conservation in prokaryotic adaptive immune systems.

Central to Clustered Regularly Interspaced Short Palindromic Repeat (CRISPR)-Cas systems are repeated RNA sequences that serve as Cas-protein-binding templates. Classification is based on the architectural composition of associated Cas proteins, considering repeat evolution is essential to complete the picture. We compiled the largest data set of CRISPRs to date, performed comprehensive, independent clustering analyses and identified a novel set of 40 conserved sequence families and 33 potential structure motifs for Cas-endoribonucleases with some distinct conservation patterns. Evolutionary relationships are presented as a hierarchical map of sequence and structure similarities for both a quick and detailed insight into the diversity of CRISPR-Cas systems. In a comparison with Cas-subtypes, I-C, I-E, I-F and type II were strongly coupled and the remaining type I and type III subtypes were loosely coupled to repeat and Cas1 evolution, respectively. Subtypes with a strong link to CRISPR evolution were almost exclusive to bacteria; nevertheless, we identified rare examples of potential horizontal transfer of I-C and I-E systems into archaeal organisms. Our easy-to-use web server provides an automated assignment of newly sequenced CRISPRs to our classification system and enables more informed choices on future hypotheses in CRISPR-Cas research: http://rna.informatik.uni-freiburg.de/CRISPRmap.

New State Records of Mosquitoes for Colorado.

The 1967 treatment of the Mosquitoes of Colorado by Harmston and Lawson and subsequent publications have recorded 46 culicid species from Colorado. As part of a study to create an updated synopsis of the mosquitoes of Colorado, adult trapping at numerous localities was conducted in Colorado during the summers of 2013 and 2014. This review also included an examination of mosquito specimens in various relevant museum collections. Aedes (Ochlerotatus) niphadopsis and Ae. (Och.) spencerii spencerii were collected during the 2013 and 2014 field seasons. Records for Ae. (Och.) canadensis canadensis, Ae. (Stegomyia) aegypti, and Uranotaenia (Pseudoficalbia) anhydor syntheta were obtained from examination of museum specimens. These species constitute new state records for Colorado, with 51 species now known from the state.

GraphClust: alignment-free structural clustering of local RNA secondary structures.

MOTIVATION: Clustering according to sequence-structure similarity has now become a generally accepted scheme for ncRNA annotation. Its application to complete genomic sequences as well as whole transcriptomes is therefore desirable but hindered by extremely high computational costs. RESULTS: We present a novel linear-time, alignment-free method for comparing and clustering RNAs according to sequence and structure. The approach scales to datasets of hundreds of thousands of sequences. The quality of the retrieved clusters has been benchmarked against known ncRNA datasets and is comparable to state-of-the-art sequence-structure methods although achieving speedups of several orders of magnitude. A selection of applications aiming at the detection of novel structural ncRNAs are presented. Exemplarily, we predicted local structural elements specific to lincRNAs likely functionally associating involved transcripts to vital processes of the human nervous system. In total, we predicted 349 local structural RNA elements. AVAILABILITY: The GraphClust pipeline is available on request.

Computational discovery of human coding and non-coding transcripts with conserved splice sites.

MOTIVATION: Long non-coding RNAs (lncRNAs) resemble protein-coding mRNAs but do not encode proteins. Most lncRNAs are under lower sequence constraints than protein-coding genes and lack conserved secondary structures, making it hard to predict them computationally. RESULTS: We introduce an approach to predict spliced lncRNAs in vertebrate genomes combining comparative genomics and machine learning. It is based on detecting signatures of characteristic splice site evolution in vertebrate whole genome alignments. First, we predict individual splice sites, then assemble compatible sites into exon candidates, and finally predict multi-exon transcripts. Using a novel method to evaluate typical splice site substitution patterns that explicitly takes the species phylogeny into account, we show that individual splice sites can be accurately predicted. Since our approach relies only on predicted splice sites, it can uncover both coding and non-coding exons. We show that our predicted exons and partial transcripts are mostly non-coding and lack conserved secondary structures. These exons are of particular interest, since existing computational approaches cannot detect them. Transcriptome sequencing data indicate tissue-specific expression patterns of predicted exons and there is evidence that increasing sequencing depth and breadth will validate additional predictions. We also found a significant enrichment of predicted exons that form multi-exon transcript parts, and we experimentally validate such a novel multi-exon gene. Overall, we obtain 336 novel multi-exon transcript predictions from human intergenic regions. Our results indicate the existence of novel human transcripts that are conserved in evolution and our approach contributes to the completion of the human transcript catalog. AVAILABILITY AND IMPLEMENTATION: Predicted human splice sites, exons and gene structures together with a Perl implementation of the tree-based log-odds scoring and a supplementary PDF file containing additional figures and tables are available at: http://www.bioinf.uni-leipzig.de/publications/supplements/10-010. The five experimentally confirmed partial transcript isoforms have been deposited in GenBank under accession numbers HM587422-HM587426.

Hierarchical classical metastability in an open quantum East model.

We study in detail an open quantum generalization of a classical kinetically constrained model-the East model-known to exhibit slow glassy dynamics stemming from a complex hierarchy of metastable states with distinct lifetimes. Using the recently introduced theory of classical metastability for open quantum systems, we show that the driven open quantum East model features a hierarchy of classical metastabilities at low temperature and weak driving field. We find that the effective long-time description of its dynamics not only is classical, but shares many properties with the classical East model, such as obeying an effective detailed balance condition and lacking static interactions between excitations, but with this occurring within a modified set of metastable phases which are coherent, and with an effective temperature that is dependent on the coherent drive.

Non-coding RNA annotation of the genome of Trichoplax adhaerens.

A detailed annotation of non-protein coding RNAs is typically missing in initial releases of newly sequenced genomes. Here we report on a comprehensive ncRNA annotation of the genome of Trichoplax adhaerens, the presumably most basal metazoan whose genome has been published to-date. Since blast identified only a small fraction of the best-conserved ncRNAs--in particular rRNAs, tRNAs and some snRNAs--we developed a semi-global dynamic programming tool, GotohScan, to increase the sensitivity of the homology search. It successfully identified the full complement of major and minor spliceosomal snRNAs, the genes for RNase P and MRP RNAs, the SRP RNA, as well as several small nucleolar RNAs. We did not find any microRNA candidates homologous to known eumetazoan sequences. Interestingly, most ncRNAs, including the pol-III transcripts, appear as single-copy genes or with very small copy numbers in the Trichoplax genome.

Entomological Investigation Following a Zika Outbreak in Brownsville, Texas.

In November and December 2016, an outbreak of locally transmitted Zika occurred in Brownsville, TX. The Texas Department of State Health Services requested for a Centers for Disease Control and Prevention (CDC) Epi Aid, and as part of that Epi Aid a team of CDC entomologists was deployed in January 2017. The mission was to improve mosquito-based arbovirus surveillance and evaluate the possibility of continuing local Zika virus (ZIKV) transmission in the city. The mosquito-based arbovirus surveillance program was expanded from 4 to 40 BG-Sentinel traps evenly distributed throughout the city. Over a 2-wk period, 15 mosquito species were detected; the most abundant species were Culex quinquefasciatus, Aedes aegypti, and Ae. albopictus, which accounted for 66.7%, 16.2%, and 5.7% of the total mosquito collection, respectively. The relative abundance of Ae. aegypti (1.0 mosquitoes/trap/day) and Ae. albopictus (0.4 mosquitoes/trap/day) was very low and unlikely to initiate and/or sustain ZIKV transmission. Zika virus was not detected in the mosquitoes collected, suggesting no or extremely low ZIKV transmission at that time.

Experimental Infection of Amblyomma americanum (Acari: Ixodidae) With Bourbon Virus (Orthomyxoviridae: Thogotovirus).

Following the recent discovery of Bourbon virus (BRBV) as a human pathogen, and the isolation of the virus from Amblyomma americanum (L.) collected near the location of a fatal human case, we undertook a series of experiments to assess the laboratory vector competence of this tick species for BRBV. Larval ticks were infected using an immersion technique, and transstadial transmission of virus to the nymphal and then to the adult stages was demonstrated. Transstadially infected nymphs transmitted virus to adult ticks at very high rates during cofeeding, indicating the presence of infectious virus in the saliva of engorging ticks. Vertical transmission by transstadially infected females to their progeny occurred, but at a low rate. Rabbits fed on by infected ticks of all active life stages developed high titers of antibody to the virus, demonstrating host exposure to BRBV antigens/live virus during tick blood feeding. These results demonstrate that A. americanum is a competent vector of BRBV and indicate that cofeeding could be critical for enzootic maintenance.

Homology-based annotation of non-coding RNAs in the genomes of Schistosoma mansoni and Schistosoma japonicum.

BACKGROUND: Schistosomes are trematode parasites of the phylum Platyhelminthes. They are considered the most important of the human helminth parasites in terms of morbidity and mortality. Draft genome sequences are now available for Schistosoma mansoni and Schistosoma japonicum. Non-coding RNA (ncRNA) plays a crucial role in gene expression regulation, cellular function and defense, homeostasis, and pathogenesis. The genome-wide annotation of ncRNAs is a non-trivial task unless well-annotated genomes of closely related species are already available. RESULTS: A homology search for structured ncRNA in the genome of S. mansoni resulted in 23 types of ncRNAs with conserved primary and secondary structure. Among these, we identified rRNA, snRNA, SL RNA, SRP, tRNAs and RNase P, and also possibly MRP and 7SK RNAs. In addition, we confirmed five miRNAs that have recently been reported in S. japonicum and found two additional homologs of known miRNAs. The tRNA complement of S. mansoni is comparable to that of the free-living planarian Schmidtea mediterranea, although for some amino acids differences of more than a factor of two are observed: Leu, Ser, and His are overrepresented, while Cys, Meth, and Ile are underrepresented in S. mansoni. On the other hand, the number of tRNAs in the genome of S. japonicum is reduced by more than a factor of four. Both schistosomes have a complete set of minor spliceosomal snRNAs. Several ncRNAs that are expected to exist in the S. mansoni genome were not found, among them the telomerase RNA, vault RNAs, and Y RNAs. CONCLUSION: The ncRNA sequences and structures presented here represent the most complete dataset of ncRNA from any lophotrochozoan reported so far. This data set provides an important reference for further analysis of the genomes of schistosomes and indeed eukaryotic genomes at large.

NcDNAlign: plausible multiple alignments of non-protein-coding genomic sequences.

Genome-wide multiple sequence alignments (MSAs) are a necessary prerequisite for an increasingly diverse collection of comparative genomic approaches. Here we present a versatile method that generates high-quality MSAs for non-protein-coding sequences. The NcDNAlign pipeline combines pairwise BLAST alignments to create initial MSAs, which are then locally improved and trimmed. The program is optimized for speed and hence is particulary well-suited to pilot studies. We demonstrate the practical use of NcDNAlign in three case studies: the search for ncRNAs in gammaproteobacteria and the analysis of conserved noncoding DNA in nematodes and teleost fish, in the latter case focusing on the fate of duplicated ultra-conserved regions. Compared to the currently widely used genome-wide alignment program TBA, our program results in a 20- to 30-fold reduction of CPU time necessary to generate gammaproteobacterial alignments. A showcase application of bacterial ncRNA prediction based on alignments of both algorithms results in similar sensitivity, false discovery rates, and up to 100 putatively novel ncRNA structures. Similar findings hold for our application of NcDNAlign to the identification of ultra-conserved regions in nematodes and teleosts. Both approaches yield conserved sequences of unknown function, result in novel evolutionary insights into conservation patterns among these genomes, and manifest the benefits of an efficient and reliable genome-wide alignment package. The software is available under the GNU Public License at http://www.bioinf.uni-leipzig.de/Software/NcDNAlign/.

Impact of Wearing and Washing/Drying of Permethrin-Treated Clothing on Their Contact Irritancy and Toxicity for Nymphal Ixodes scapularis (Acari: Ixodidae) Ticks.

Permethrin-treated clothing is available as consumer products to prevent bites by tick and insect pests. We used bioassays to examine the impact of wearing and washing/drying of permethrin-treated shirts, pants, and socks, and wearing of treated shoes, on their contact irritancy and toxicity for nymphal Ixodes scapularis Say (Acari: Ixodidae) ticks, the primary vectors in the eastern United States of the causative agents of Lyme disease, human anaplasmosis, and human babesiosis. Pristine permethrin-treated clothing displayed strong contact irritancy and toxicity toward I. scapularis nymphs, with 0-30% of ticks across clothing types and tick sources displaying normal movement 1 h after forced contact for 30-120 s with treated textile. Following 16 d of wear and 16 rounds of machine washing and drying, we recorded reduced concentrations (by 50-90%) of permethrin, compared with pristine treated clothing, from shirts, pants, and socks. This loss of permethrin was associated with reduced contact irritancy and toxicity for ticks after forced contact with worn and washed/dried treated clothing: 31-67% of ticks displayed normal movement 1 h after contact. Nevertheless, the worn and washed/dried treated clothing was still superior to nontreated textile, for which 90-100% of ticks displayed normal movement. Treated shoes, which were worn but not washed, remained as toxic to the ticks as pristine treated shoes. We caution that these laboratory bioassay results should not be interpreted as being directly indicative of the outcome of using washed/worn permethrin-treated clothing in daily life. Although wear and washing/drying did reduce the irritancy and toxicity of permethrin-treated clothing for I. scapularis nymphs more than we had expected, the remaining effect might still reduce the risk of tick bites in a real-life scenario.

Duplicated RNA genes in teleost fish genomes.

Teleost fishes share a duplication of their entire genomes. We report here on a computational survey of structured non-coding RNAs (ncRNAs) in teleost genomes, focusing on the fate of fish-specific duplicates. As in other metazoan groups, we find evidence of a large number (11,543) of structured RNAs, most of which (~86%) are clade-specific or evolve so fast that their tetrapod homologs cannot be detected. In surprising contrast to protein-coding genes, the fish-specific genome duplication did not lead to a large number of paralogous ncRNAs: only 188 candidates, mostly microRNAs, appear in a larger copy number in teleosts than in tetrapods, suggesting that large-scale gene duplications do not play a major role in the expansion of the vertebrate ncRNA inventory.

Spectral properties of simple classical and quantum reset processes.

We study the spectral properties of classical and quantum Markovian processes that are reset at random times to a specific configuration or state with a reset rate that is independent of the current state of the system. We demonstrate that this simple reset dynamics causes a uniform shift in the eigenvalues of the Markov generator, excluding the zero mode corresponding to the stationary state, which has the effect of accelerating or even inducing relaxation to a stationary state. Based on this result, we provide expressions for the stationary state and probability current of the reset process in terms of weighted sums over dynamical modes of the reset-free process. We also discuss the effect of resets on processes that display metastability. We illustrate our results with two classical stochastic processes, the totally asymmetric random walk and the one-dimensional Brownian motion, as well as two quantum models: a particle coherently hopping on a chain and the dissipative transverse field Ising model, known to exhibit metastability.