Robustness and complexity.

When a system robustly corrects component-level errors, the direct pressure on component performance declines. Components become less reliable, maintain more genetic variability, or drift neutrally, creating new forms of complexity. Examples include the hourglass pattern of biological development and the hourglass architecture for robustly complex systems in engineering.

Disease from opposing forces in regulatory control.

Danger requires a strong rapid response. Speedy triggers are prone to false signals. False alarms can be costly, requiring strong negative regulators to oppose the initial triggers. Strongly opposed forces can easily be perturbed, leading to imbalance and disease. For example, immunity and fear response balance strong rapid triggers against widespread slow negative regulators. Diseases of immunity and behavior arise from imbalance. A different opposition of forces occurs in mammalian growth, which balances strong paternally expressed accelerators against maternally expressed suppressors. Diseases of overgrowth or undergrowth arise from imbalance. Other examples of opposing forces and disease include control of dopamine expression and male versus female favored traits.

Precise Traits from Sloppy Components: Perception and the Origin of Phenotypic Response.

Organisms perceive their environment and respond. The origin of perception-response traits presents a puzzle. Perception provides no value without response. Response requires perception. Recent advances in machine learning may provide a solution. A randomly connected network creates a reservoir of perceptive information about the recent history of environmental states. In each time step, a relatively small number of inputs drives the dynamics of the relatively large network. Over time, the internal network states retain a memory of past inputs. To achieve a functional response to past states or to predict future states, a system must learn only how to match states of the reservoir to the target response. In the same way, a random biochemical or neural network of an organism can provide an initial perceptive basis. With a solution for one side of the two-step perception-response challenge, evolving an adaptive response may not be so difficult. Two broader themes emerge. First, organisms may often achieve precise traits from sloppy components. Second, evolutionary puzzles often follow the same outlines as the challenges of machine learning. In each case, the basic problem is how to learn, either by artificial computational methods or by natural selection.

Evolutionary explanations for heterogeneous behavior in clonal bacterial populations.

Cellular heterogeneity in clonal bacterial populations is widespread. Division of labor and bet hedging are common adaptive explanations for the function of such heterogeneity. We suggest group-level phenotypes via shareable molecules and variation in cellular vigor as two alternative evolutionary explanations for bacterial cellular heterogeneity.

Optimizing differential equations to fit data and predict outcomes.

Many scientific problems focus on observed patterns of change or on how to design a system to achieve particular dynamics. Those problems often require fitting differential equation models to target trajectories. Fitting such models can be difficult because each evaluation of the fit must calculate the distance between the model and target patterns at numerous points along a trajectory. The gradient of the fit with respect to the model parameters can be challenging to compute. Recent technical advances in automatic differentiation through numerical differential equation solvers potentially change the fitting process into a relatively easy problem, opening up new possibilities to study dynamics. However, application of the new tools to real data may fail to achieve a good fit. This article illustrates how to overcome a variety of common challenges, using the classic ecological data for oscillations in hare and lynx populations. Models include simple ordinary differential equations (ODEs) and neural ordinary differential equations (NODEs), which use artificial neural networks to estimate the derivatives of differential equation systems. Comparing the fits obtained with ODEs versus NODEs, representing small and large parameter spaces, and changing the number of variable dimensions provide insight into the geometry of the observed and model trajectories. To analyze the quality of the models for predicting future observations, a Bayesian-inspired preconditioned stochastic gradient Langevin dynamics (pSGLD) calculation of the posterior distribution of predicted model trajectories clarifies the tendency for various models to underfit or overfit the data. Coupling fitted differential equation systems with pSGLD sampling provides a powerful way to study the properties of optimization surfaces, raising an analogy with mutation-selection dynamics on fitness landscapes.

Robustness increases heritability: implications for familial disease.

Robustness protects organisms in two ways. Homeostatic buffering lowers the variation of traits caused by internal or external perturbations. Tolerance reduces the consequences of bad situations, such as extreme phenotypes or infections. This article shows that both types of robustness increase the heritability of protected traits. Additionally, robustness strongly increases the heritability of disease. The natural tendency for organisms to protect robustly against perturbations may partly explain the high heritability that occurs for some diseases.

Optimization of Transcription Factor Genetic Circuits.

Transcription factors (TFs) affect the production of mRNAs. In essence, the TFs form a large computational network that controls many aspects of cellular function. This article introduces a computational method to optimize TF networks. The method extends recent advances in artificial neural network optimization. In a simple example, computational optimization discovers a four-dimensional TF network that maintains a circadian rhythm over many days, successfully buffering strong stochastic perturbations in molecular dynamics and entraining to an external day-night signal that randomly turns on and off at intervals of several days. This work highlights the similar challenges in understanding how computational TF and neural networks gain information and improve performance.

The number of neutral mutants in an expanding Luria-Delbrück population is approximately Fréchet.

Background: A growing population of cells accumulates mutations. A single mutation early in the growth process carries forward to all descendant cells, causing the final population to have a lot of mutant cells. When the first mutation happens later in growth, the final population typically has fewer mutants. The number of mutant cells in the final population follows the Luria-Delbrück distribution. The mathematical form of the distribution is known only from its probability generating function. For larger populations of cells, one typically uses computer simulations to estimate the distribution. Methods: This article searches for a simple approximation of the Luria-Delbrück distribution, with an explicit mathematical form that can be used easily in calculations. Results: The Fréchet distribution provides a good approximation for the Luria-Delbrück distribution for neutral mutations, which do not cause a growth rate change relative to the original cells. Conclusions: The Fréchet distribution apparently provides a good match through its description of extreme value problems for multiplicative processes such as exponential growth.

The Fundamental Equations of Change in Statistical Ensembles and Biological Populations.

A recent article in Nature Physics unified key results from thermodynamics, statistics, and information theory. The unification arose from a general equation for the rate of change in the information content of a system. The general equation describes the change in the moments of an observable quantity over a probability distribution. One term in the equation describes the change in the probability distribution. The other term describes the change in the observable values for a given state. We show the equivalence of this general equation for moment dynamics with the widely known Price equation from evolutionary theory, named after George Price. We introduce the Price equation from its biological roots, review a mathematically abstract form of the equation, and discuss the potential for this equation to unify diverse mathematical theories from different disciplines. The new work in Nature Physics and many applications in biology show that this equation also provides the basis for deriving many novel theoretical results within each discipline.

Developmental Mutators and Early Onset Cancer.

A new hypothesis suggests that somatic genome remodeling during normal development can cause mutations that explain many early onset cancers in children and adults.

Simple unity among the fundamental equations of science.

The Price equation describes the change in populations. Change concerns some value, such as biological fitness, information or physical work. The Price equation reveals universal aspects for the nature of change, independently of the meaning ascribed to values. By understanding those universal aspects, we can see more clearly why fundamental mathematical results in different disciplines often share a common form. We can also interpret more clearly the meaning of key results within each discipline. For example, the mathematics of natural selection in biology has a form closely related to information theory and physical entropy. Does that mean that natural selection is about information or entropy? Or do natural selection, information and entropy arise as interpretations of a common underlying abstraction? The Price equation suggests the latter. The Price equation achieves its abstract generality by partitioning change into two terms. The first term naturally associates with the direct forces that cause change. The second term naturally associates with the changing frame of reference. In the Price equation's canonical form, total change remains zero because the conservation of total probability requires that all probabilities invariantly sum to one. Much of the shared common form for the mathematics of different disciplines may arise from that seemingly trivial invariance of total probability, which leads to the partitioning of total change into equal and opposite components of the direct forces and the changing frame of reference. This article is part of the theme issue 'Fifty years of the Price equation'.

Sexual antagonism leads to a mosaic of X-autosome conflict.

Males and females have different optimal values for some traits, such as body size. When the same genes control these traits in both sexes, selection pushes in opposite directions in males and females. Alleles at autosomal loci spend equal amounts of time in males and females, suggesting that the sexually antagonistic selective forces may approximately balance between the opposing optima. Frank and Crespi noted that alleles on the X chromosome spend twice as much time in diploid females as in haploid males. That distinction between the sexes may tend to favor X-linked genes that push more strongly toward the female optimum than the male optimum. The female bias of X-linked genes opposes the intermediate optimum of autosomal genes, potentially creating a difference between the direction of selection on traits favored by X chromosomes and autosomes. Patten has recently argued that explicit genetic assumptions about dominance and the relative magnitude of allelic effects may lead X-linked genes to favor the male rather than the female optimum, contradicting Frank and Crespi. This article combines the insights of those prior analyses into a new, more general theory. We find some parameter combinations for X-linked loci that favor a female bias and other parameter combinations that favor a male bias. We conclude that the X likely contains a mosaic pattern of loci that differ with autosomes over sexually antagonistic traits. The overall tendency for a female or male bias on the X depends on prior assumptions about the distribution of key parameters across X-linked loci. Those parameters include the dominance coefficient and the way in which ploidy influences the magnitude of allelic effects.

The common patterns of abundance: the log series and Zipf's law.

In a language corpus, the probability that a word occurs n times is often proportional to 1/ n2. Assigning rank, s, to words according to their abundance, log s vs log n typically has a slope of minus one. That simple Zipf's law pattern also arises in the population sizes of cities, the sizes of corporations, and other patterns of abundance. By contrast, for the abundances of different biological species, the probability of a population of size n is typically proportional to 1/ n, declining exponentially for larger n, the log series pattern. This article shows that the differing patterns of Zipf's law and the log series arise as the opposing endpoints of a more general theory. The general theory follows from the generic form of all probability patterns as a consequence of conserved average values and the associated invariances of scale. To understand the common patterns of abundance, the generic form of probability distributions plus the conserved average abundance is sufficient. The general theory includes cases that are between the Zipf and log series endpoints, providing a broad framework for analyzing widely observed abundance patterns.

How to Understand Behavioral Patterns in Big Data: The Case of Human Collective Memory.

Simple patterns often arise from complex systems. For example, human perception of similarity decays exponentially with perceptual distance. The ranking of word usage versus the frequency at which the words are used has a log-log slope of minus one. Recent advances in big data provide an opportunity to characterize the commonly observed patterns of behavior. Those observed regularities set the challenge of understanding the mechanistic processes that generate common behaviors. This article illustrates the problem with the recent big data analysis of collective memory. Collective memory follows a simple biexponential pattern of decay over time. An initial rapid decay is followed by a slower, longer lasting decay. Candia et al. successfully fit a two stage model of mechanistic process to that pattern. Although that fit is useful, this article emphasizes the need, in big data analyses, to consider a broad set of alternative causal explanations. In this case, the method of signal frequency analysis yields several simple alternative models that generate exactly the same observed pattern of collective memory decay. This article concludes that the full potential of big data analyses in the behavioral sciences will require better methods for developing alternative, empirically testable causal models.

Evolutionary design of regulatory control. II. Robust error-correcting feedback increases genetic and phenotypic variability.

As systems become more robust against perturbations, they can compensate for greater sloppiness in the performance of their components. That robust compensation reduces the force of natural selection on the system's components, leading to component decay. The paradoxical coupling of robustness and decay predicts that robust systems evolve cheaper, lower performing components, which accumulate greater mutational genetic variability and which have greater phenotypic stochasticity in trait expression. Previous work noted the paradox of robustness. However, no general theory for the evolutionary dynamics of system robustness and component decay has been developed. This article takes a first step by linking engineering control theory with the genetic theory of evolutionary dynamics. Control theory emphasizes error-correcting feedback as the single greatest principle in robust system design. Linking control theory to evolution leads to a theory for the evolutionary dynamics of error-correcting feedback, a unifying approach for the evolutionary analysis of robust systems. This article shows how increasingly robust systems accumulate more genetic variability and greater stochasticity of expression in their components. The theory predicts different levels of variability between different regulatory control architectures and different levels of variability between different components within a particular regulatory control system. The theory also shows that increasing robustness reduces the frequency of system failures associated with disease and, simultaneously, causes a strong increase in the heritability of disease. Thus, robust error correction in biological regulatory control may partly explain the puzzlingly high heritability of disease and, more generally, the surprisingly high heritability of fitness.

Invariance in ecological pattern.

Background: The abundance of different species in a community often follows the log series distribution. Other ecological patterns also have simple forms. Why does the complexity and variability of ecological systems reduce to such simplicity? Common answers include maximum entropy, neutrality, and convergent outcome from different underlying biological processes.  Methods: This article proposes a more general answer based on the concept of invariance, the property by which a pattern remains the same after transformation. Invariance has a long tradition in physics. For example, general relativity emphasizes the need for the equations describing the laws of physics to have the same form in all frames of reference.  Results: By bringing this unifying invariance approach into ecology, we show that the log series pattern dominates when the consequences of processes acting on abundance are invariant to the addition or multiplication of abundance by a constant. The lognormal pattern dominates when the processes acting on net species growth rate obey rotational invariance (symmetry) with respect to the summing up of the individual component processes. Conclusions: Recognizing how these invariances connect pattern to process leads to a synthesis of previous approaches. First, invariance provides a simpler and more fundamental maximum entropy derivation of the log series distribution. Second, invariance provides a simple derivation of the key result from neutral theory: the log series at the metacommunity scale and a clearer form of the skewed lognormal at the local community scale. The invariance expressions are easy to understand because they uniquely describe the basic underlying components that shape pattern.

Evolutionary design of regulatory control. I. A robust control theory analysis of tradeoffs.

The evolutionary design of regulatory control balances various tradeoffs in performance. Fast reaction to environmental change tends to favor plastic responsiveness at the expense of greater sensitivity to perturbations that degrade homeostatic control. Greater homeostatic stability against unpredictable disturbances tends to reduce performance in tracking environmental change. This article applies the classic principles of engineering control theory to the evolutionary design of regulatory systems. The engineering theory clarifies the conceptual aspects of evolutionary tradeoffs and provides analytic methods for developing specific predictions. On the conceptual side, this article clarifies the meanings of integral control, feedback, and design, concepts that have been discussed in a confusing way within the biological literature. On the analytic side, this article presents extensive methods and examples to study error-correcting feedback, which is perhaps the single greatest principle of design in both human-engineered and naturally designed systems. The broad framework and associated software code provide a comprehensive how-to guide for making models that focus on functional aspects of regulatory control and for making comparative predictions about regulatory design in response to various kinds of environmental challenge. The second article in this series analyzes how alternative regulatory designs influence the relative levels of genetic variability, stochasticity of trait expression, and heritability of disease.

Measurement invariance explains the universal law of generalization for psychological perception.

The universal law of generalization describes how animals discriminate between alternative sensory stimuli. On an appropriate perceptual scale, the probability that an organism perceives two stimuli as similar typically declines exponentially with the difference on the perceptual scale. Exceptions often follow a Gaussian probability pattern rather than an exponential pattern. Previous explanations have been based on underlying theoretical frameworks such as information theory, Kolmogorov complexity, or empirical multidimensional scaling. This article shows that the few inevitable invariances that must apply to any reasonable perceptual scale provide a sufficient explanation for the universal exponential law of generalization. In particular, reasonable measurement scales of perception must be invariant to shift by a constant value, which by itself leads to the exponential form. Similarly, reasonable measurement scales of perception must be invariant to multiplication, or stretch, by a constant value, which leads to the conservation of the slope of discrimination with perceptual difference. In some cases, an additional assumption about exchangeability or rotation of underlying perceptual dimensions leads to a Gaussian pattern of discrimination, which can be understood as a special case of the more general exponential form. The three measurement invariances of shift, stretch, and rotation provide a sufficient explanation for the universally observed patterns of perceptual generalization. All of the additional assumptions and language associated with information, complexity, and empirical scaling are superfluous with regard to the broad patterns of perception.

A biochemical logarithmic sensor with broad dynamic range.

Sensory perception often scales logarithmically with the input level. Similarly, the output response of biochemical systems sometimes scales logarithmically with the input signal that drives the system. How biochemical systems achieve logarithmic sensing remains an open puzzle. This article shows how a biochemical logarithmic sensor can be constructed from the most basic principles of chemical reactions. Assuming that reactions follow the classic Michaelis-Menton kinetics of mass action or the more generalized and commonly observed Hill equation response, the summed output of several simple reactions with different sensitivities to the input will often give an aggregate output response that logarithmically transforms the input. The logarithmic response is robust to stochastic fluctuations in parameter values. This model emphasizes the simplicity and robustness by which aggregate chemical circuits composed of sloppy components can achieve precise response characteristics. Both natural and synthetic designs gain from the power of this aggregate approach.