Exploring Trade-Offs in Spiking Neural Networks.

Spiking neural networks (SNNs) have emerged as a promising alternative to traditional deep neural networks for low-power computing. However, the effectiveness of SNNs is not solely determined by their performance but also by their energy consumption, prediction speed, and robustness to noise. The recent method Fast & Deep, along with others, achieves fast and energy-efficient computation by constraining neurons to fire at most once. Known as time-to-first-spike (TTFS), this constraint, however, restricts the capabilities of SNNs in many aspects. In this work, we explore the relationships of performance, energy consumption, speed, and stability when using this constraint. More precisely, we highlight the existence of trade-offs where performance and robustness are gained at the cost of sparsity and prediction latency. To improve these trade-offs, we propose a relaxed version of Fast & Deep that allows for multiple spikes per neuron. Our experiments show that relaxing the spike constraint provides higher performance while also benefiting from faster convergence, similar sparsity, comparable prediction latency, and better robustness to noise compared to TTFS SNNs. By highlighting the limitations of TTFS and demonstrating the advantages of unconstrained SNNs, we provide valuable insight for the development of effective learning strategies for neuromorphic computing.

Identification of organophosphorus simulants for the development of next-generation detection technologies.

Organophosphorus (OP) chemical warfare agents (CWAs) represent an ongoing threat but the understandable widespread prohibition of their use places limitations on the development of technologies to counter the effects of any OP CWA release. Herein, we describe new, accessible methods for the identification of appropriate molecular simulants to mimic the hydrogen bond accepting capacity of the P[double bond, length as m-dash]O moiety, common to every member of this class of CWAs. Using the predictive methodologies developed herein, we have identified OP CWA hydrogen bond acceptor simulants for soman and sarin. It is hoped that the effective use of these physical property specific simulants will aid future countermeasure developments.

Minimal Spiking Neuron for Solving Multilabel Classification Tasks.

The multispike tempotron (MST) is a powersul, single spiking neuron model that can solve complex supervised classification tasks. It is also internally complex, computationally expensive to evaluate, and unsuitable for neuromorphic hardware. Here we aim to understand whether it is possible to simplify the MST model while retaining its ability to learn and process information. To this end, we introduce a family of generalized neuron models (GNMs) that are a special case of the spike response model and much simpler and cheaper to simulate than the MST. We find that over a wide range of parameters, the GNM can learn at least as well as the MST does. We identify the temporal autocorrelation of the membrane potential as the most important ingredient of the GNM that enables it to classify multiple spatiotemporal patterns. We also interpret the GNM as a chemical system, thus conceptually bridging computation by neural networks with molecular information processing. We conclude the letter by proposing alternative training approaches for the GNM, including error trace learning and error backpropagation.

Translation elongation can control translation initiation on eukaryotic mRNAs.

Synonymous codons encode the same amino acid, but differ in other biophysical properties. The evolutionary selection of codons whose properties are optimal for a cell generates the phenomenon of codon bias. Although recent studies have shown strong effects of codon usage changes on protein expression levels and cellular physiology, no translational control mechanism is known that links codon usage to protein expression levels. Here, we demonstrate a novel translational control mechanism that responds to the speed of ribosome movement immediately after the start codon. High initiation rates are only possible if start codons are liberated sufficiently fast, thus accounting for the observation that fast codons are overrepresented in highly expressed proteins. In contrast, slow codons lead to slow liberation of the start codon by initiating ribosomes, thereby interfering with efficient translation initiation. Codon usage thus evolved as a means to optimise translation on individual mRNAs, as well as global optimisation of ribosome availability.

Performance limits and trade-offs in entropy-driven biochemical computers.

It is now widely accepted that biochemical reaction networks can perform computations. Examples are kinetic proof reading, gene regulation, or signalling networks. For many of these systems it was found that their computational performance is limited by a trade-off between the metabolic cost, the speed and the accuracy of the computation. In order to gain insight into the origins of these trade-offs, we consider entropy-driven computers as a model of biochemical computation. Using tools from stochastic thermodynamics, we show that entropy-driven computation is subject to a trade-off between accuracy and metabolic cost, but does not involve time-trade-offs. Time trade-offs appear when it is taken into account that the result of the computation needs to be measured in order to be known. We argue that this measurement process, although usually ignored, is a major contributor to the cost of biochemical computation.

Limited by sensing - A minimal stochastic model of the lag-phase during diauxic growth.

Many microbes when grown on a mixture of two carbon sources utilise first and exclusively the preferred sugar, before switching to the less preferred carbon source. This results in two distinct exponential growth phases, often interrupted by a lag-phase of reduced growth termed the lag-phase. While the lag-phase appears to be an evolved feature, it is not clear what drives its evolution, as it comes with a substantial up-front fitness penalty due to lost growth. In this article a minimal mathematical model based on a master-equation approach is proposed. This model can explain many empirically observed phenomena. It suggests that the lag-phase can be understood as a manifestation of the trade-off between switching speed and switching efficiency. Moreover, the model predicts heterogeneity of the population during the lag-phase. Finally, it is shown that the switch from one carbon source to another one is a sensing problem and the lag-phase is a manifestation of known fundamental limitations of biological sensors.

In silico evolution of diauxic growth.

BACKGROUND: The glucose effect is a well known phenomenon whereby cells, when presented with two different nutrients, show a diauxic growth pattern, i.e. an episode of exponential growth followed by a lag phase of reduced growth followed by a second phase of exponential growth. Diauxic growth is usually thought of as a an adaptation to maximise biomass production in an environment offering two or more carbon sources. While diauxic growth has been studied widely both experimentally and theoretically, the hypothesis that diauxic growth is a strategy to increase overall growth has remained an unconfirmed conjecture. METHODS: Here, we present a minimal mathematical model of a bacterial nutrient uptake system and metabolism. We subject this model to artificial evolution to test under which conditions diauxic growth evolves. RESULTS: As a result, we find that, indeed, sequential uptake of nutrients emerges if there is competition for nutrients and the metabolism/uptake system is capacity limited. DISCUSSION: However, we also find that diauxic growth is a secondary effect of this system and that the speed-up of nutrient uptake is a much larger effect. Notably, this speed-up of nutrient uptake coincides with an overall reduction of efficiency. CONCLUSIONS: Our two main conclusions are: (i) Cells competing for the same nutrients evolve rapid but inefficient growth dynamics. (ii) In the deterministic models we use here no substantial lag-phase evolves. This suggests that the lag-phase is a consequence of stochastic gene expression.

The architecture of eukaryotic translation.

Translation in baker's yeast involves the coordinated interaction of 200,000 ribosomes, 3,000,000 tRNAs and between 15,000 and 60,000 mRNAs. It is currently unknown whether this specific constellation of components has particular relevance for the requirements of the yeast proteome, or whether this is simply a frozen accident. Our study uses a computational simulation model of the genome-wide translational apparatus of yeast to explore quantitatively which combinations of mRNAs, ribosomes and tRNAs can produce viable proteomes. Surprisingly, we find that if we only consider total translational activity over time without regard to composition of the proteome, then there are many and widely differing combinations that can generate equivalent synthesis yields. In contrast, translational activity required for generating specific proteomes can only be achieved within a much more constrained parameter space. Furthermore, we find that strongly ribosome limited regimes are optimal for cells in that they are resource efficient and simplify the dynamics of the system.

Low-variance Forward Gradients using Direct Feedback Alignment and momentum.

Supervised learning in deep neural networks is commonly performed using error backpropagation. However, the sequential propagation of errors during the backward pass limits its scalability and applicability to low-powered neuromorphic hardware. Therefore, there is growing interest in finding local alternatives to backpropagation. Recently proposed methods based on forward-mode automatic differentiation suffer from high variance in large deep neural networks, which affects convergence. In this paper, we propose the Forward Direct Feedback Alignment algorithm that combines Activity-Perturbed Forward Gradients with Direct Feedback Alignment and momentum. We provide both theoretical proofs and empirical evidence that our proposed method achieves lower variance than forward gradient techniques. In this way, our approach enables faster convergence and better performance when compared to other local alternatives to backpropagation and opens a new perspective for the development of online learning algorithms compatible with neuromorphic systems.

A novel and versatile computational tool to model translation.

MOTIVATION: Much is now known about the mechanistic details of gene translation. There are also rapid advances in high-throughput technologies to determine quantitative aspects of the system. As a consequence-realistic and system-wide simulation models of translation are now feasible. Such models are also needed as devices to integrate a large volume of highly fragmented data known about translation. Software: In this application note, we present a novel, highly efficient software tool to model translation. The tool represents the main aspects of translation. Features include a representation of exhaustible tRNA pools, ribosome-ribosome interactions and differential initiation rates for different mRNA species. The tool is written in Java, and is hence portable and can be parameterized for any organism. AVAILABILITY: The model can be obtained from the authors or directly downloaded from the authors' home-page (http://goo.gl/JUWvI).

The role of tRNA and ribosome competition in coupling the expression of different mRNAs in Saccharomyces cerevisiae.

Protein synthesis translates information from messenger RNAs into functional proteomes. Because of the finite nature of the resources required by the translational machinery, both the overall protein synthesis activity of a cell and activity on individual mRNAs are controlled by the allocation of limiting resources. Upon introduction of heterologous sequences into an organism-for example for the purposes of bioprocessing or synthetic biology-limiting resources may also become overstretched, thus negatively affecting both endogenous and heterologous gene expression. In this study, we present a mean-field model of translation in Saccharomyces cerevisiae for the investigation of two particular translational resources, namely ribosomes and aminoacylated tRNAs. We firstly use comparisons of experiments with heterologous sequences and simulations of the same conditions to calibrate our model, and then analyse the behaviour of the translational system in yeast upon introduction of different types of heterologous sequences. Our main findings are that: competition for ribosomes, rather than tRNAs, limits global translation in this organism; that tRNA aminoacylation levels exert, at most, weak control over translational activity; and that decoding speeds and codon adaptation exert strong control over local (mRNA specific) translation rates.

Programming Molecular Systems To Emulate a Learning Spiking Neuron.

Hebbian theory seeks to explain how the neurons in the brain adapt to stimuli to enable learning. An interesting feature of Hebbian learning is that it is an unsupervised method and, as such, does not require feedback, making it suitable in contexts where systems have to learn autonomously. This paper explores how molecular systems can be designed to show such protointelligent behaviors and proposes the first chemical reaction network (CRN) that can exhibit autonomous Hebbian learning across arbitrarily many input channels. The system emulates a spiking neuron, and we demonstrate that it can learn statistical biases of incoming inputs. The basic CRN is a minimal, thermodynamically plausible set of microreversible chemical equations that can be analyzed with respect to their energy requirements. However, to explore how such chemical systems might be engineered de novo, we also propose an extended version based on enzyme-driven compartmentalized reactions. Finally, we show how a purely DNA system, built upon the paradigm of DNA strand displacement, can realize neuronal dynamics. Our analysis provides a compelling blueprint for exploring autonomous learning in biological settings, bringing us closer to realizing real synthetic biological intelligence.

Constraints on Hebbian and STDP learned weights of a spiking neuron.

We analyse mathematically the constraints on weights resulting from Hebbian and STDP learning rules applied to a spiking neuron with weight normalisation. In the case of pure Hebbian learning, we find that the normalised weights equal the promotion probabilities of weights up to correction terms that depend on the learning rate and are usually small. A similar relation can be derived for STDP algorithms, where the normalised weight values reflect a difference between the promotion and demotion probabilities of the weight. These relations are practically useful in that they allow checking for convergence of Hebbian and STDP algorithms. Another application is novelty detection. We demonstrate this using the MNIST dataset.

Modeling fimbriae mediated parasite-host interactions.

Type 1 fimbriae are a known virulence factor in a number of pathogenic enterobacteriaceae, including Salmonella, Shigella and E. coli. Yet, they are also expressed by some commensal strains, notably of E. coli. One hypothesis of the role of fimbriae in commensals is that they evoke a small but tolerable host immune response in order to have the host release sialic acid, which is a valuable nutrient. Genetic evidence suggests that sialic acid down-regulates fimbriation. This has been believed to enable the cells to reduce virulence when the host response is increasing, thus avoiding a full activation of host defenses. In this article we assess the plausibility of this hypothesis using mathematical models. Our models lead us to two main conclusions: A slight activation of host defenses is only possible with a carefully tuned set of parameters, whereas under a wide range of parameters and assumptions, the model predicts the host defenses to be activated to at least half their potential in response to fimbriation. Secondly, the fact that fimbriation is suppressed by sialic acid seems irrelevant for the global qualitative properties.

Hidden patterns of codon usage bias across kingdoms.

The genetic code is necessarily degenerate with 64 possible nucleotide triplets being translated into 20 amino acids. Eighteen out of the 20 amino acids are encoded by multiple synonymous codons. While synonymous codons are clearly equivalent in terms of the information they carry, it is now well established that they are used in a biased fashion. There is currently no consensus as to the origin of this bias. Drawing on ideas from stochastic thermodynamics we derive from first principles a mathematical model describing the statistics of codon usage bias. We show that the model accurately describes the distribution of codon usage bias of genomes in the fungal and bacterial kingdoms. Based on it, we derive a new computational measure of codon usage bias-the distance D capturing two aspects of codon usage bias: (i) differences in the genome-wide frequency of codons and (ii) apparent non-random distributions of codons across mRNAs. By means of large scale computational analysis of over 900 species across two kingdoms of life, we demonstrate that our measure provides novel biological insights. Specifically, we show that while codon usage bias is clearly based on heritable traits and closely related species show similar degrees of bias, there is considerable variation in the magnitude of D within taxonomic classes suggesting that the contribution of sequence-level selection to codon bias varies substantially within relatively confined taxonomic groups. Interestingly, commonly used model organisms are near the median for values of D for their taxonomic class, suggesting that they may not be good representative models for species with more extreme D, which comprise organisms of medical and agricultural interest. We also demonstrate that amino acid specific patterns of codon usage are themselves quite variable between branches of the tree of life, and that some of this variability correlates with organismal tRNA content.

Towards the Prediction of Antimicrobial Efficacy for Hydrogen Bonded, Self-Associating Amphiphiles.

Herein we report 50 structurally related supramolecular self-associating amphiphilic (SSA) salts and related compounds. These SSAs are shown to act as antimicrobial agents, active against model Gram-positive (methicillin-resistant Staphylococcus aureus) and/or Gram-negative (Escherichia coli) bacteria of clinical interest. Through a combination of solution-state, gas-phase, solid-state and in silico measurements, we determine 14 different physicochemical parameters for each of these 50 structurally related compounds. These parameter sets are then used to identify molecular structure-physicochemical property-antimicrobial activity relationships for our model Gram-negative and Gram-positive bacteria, while simultaneously providing insight towards the elucidation of SSA mode of antimicrobial action.

Models of transcription factor binding: sensitivity of activation functions to model assumptions.

We present three models of how transcription factors (TFs) bind to their specific binding sites on the DNA: a model based on statistical physics, a Markov-chain model and a computational simulation. Comparison of these models suggests that the effect of non-specific binding can be significant. We also investigate possible mechanisms for cooperativity. The simulation model suggests that direct interactions between TFs are unlikely to be the main source of cooperativity between specific binding sites, because such interactions tend to lead to the formation of clusters on the DNA with undesirable side-effects.

The evolution of group-level pathogenic traits.

A group-selection model for the evolutionary origin of phase-variation in E. coli is proposed. Populations of commensal strains of E. coli populating mammalian hosts modulate its immune defenses through population-level control of the expression of fimbriae. At any time only a proportion of the population expresses these cell-surface adhesins. Collectively they elicit a host-based nutrient release if the fimbriae expression is low. Too high levels of fimbriation would provoke an inflammatory response and thus intolerable conditions for the cells. The optimal level of fimbriation is a group property and its evolution is difficult to explain by naive individual selection scenarios. This article presents a computational model to simulate the evolution of fimbriae. The two main conclusions of this contribution are: (i) the evolution of this group property requires the population to be partitioned into weakly interacting sub-populations. (ii) Given certain scenarios evolution consistently under-performs, in the sense that it does not find the optimal level of fimbriation.

Thermodynamics of quasideterministic digital computers.

A central result of stochastic thermodynamics is that irreversible state transitions of Markovian systems entail a cost in terms of an infinite entropy production. A corollary of this is that strictly deterministic computation is not possible. Using a thermodynamically consistent model, we show that quasideterministic computation can be achieved at finite, and indeed modest cost with accuracies that are indistinguishable from deterministic behavior for all practical purposes. Concretely, we consider the entropy production of stochastic (Markovian) systems that behave like and and a not gates. Combinations of these gates can implement any logical function. We require that these gates return the correct result with a probability that is very close to 1, and additionally, that they do so within finite time. The central component of the model is a machine that can read and write binary tapes. We find that the error probability of the computation of these gates falls with the power of the system size, whereas the cost only increases linearly with the system size.

A thermodynamically consistent model of finite-state machines.

Finite-state machines (FSMs) are a theoretically and practically important model of computation. We propose a general, thermodynamically consistent model of FSMs and characterize the resource requirements of these machines. We model FSMs as time-inhomogeneous Markov chains. The computation is driven by instantaneous manipulations of the energy levels of the states. We calculate the entropy production of the machine, its error probability, and the time required to complete one update step. We find that a sequence of generalized bit-setting operations is sufficient to implement any FSM.