Precise Traits from Sloppy Components: Perception and the Origin of Phenotypic Response.

Organisms perceive their environment and respond. The origin of perception-response traits presents a puzzle. Perception provides no value without response. Response requires perception. Recent advances in machine learning may provide a solution. A randomly connected network creates a reservoir of perceptive information about the recent history of environmental states. In each time step, a relatively small number of inputs drives the dynamics of the relatively large network. Over time, the internal network states retain a memory of past inputs. To achieve a functional response to past states or to predict future states, a system must learn only how to match states of the reservoir to the target response. In the same way, a random biochemical or neural network of an organism can provide an initial perceptive basis. With a solution for one side of the two-step perception-response challenge, evolving an adaptive response may not be so difficult. Two broader themes emerge. First, organisms may often achieve precise traits from sloppy components. Second, evolutionary puzzles often follow the same outlines as the challenges of machine learning. In each case, the basic problem is how to learn, either by artificial computational methods or by natural selection.

Measurement invariance explains the universal law of generalization for psychological perception.

The universal law of generalization describes how animals discriminate between alternative sensory stimuli. On an appropriate perceptual scale, the probability that an organism perceives two stimuli as similar typically declines exponentially with the difference on the perceptual scale. Exceptions often follow a Gaussian probability pattern rather than an exponential pattern. Previous explanations have been based on underlying theoretical frameworks such as information theory, Kolmogorov complexity, or empirical multidimensional scaling. This article shows that the few inevitable invariances that must apply to any reasonable perceptual scale provide a sufficient explanation for the universal exponential law of generalization. In particular, reasonable measurement scales of perception must be invariant to shift by a constant value, which by itself leads to the exponential form. Similarly, reasonable measurement scales of perception must be invariant to multiplication, or stretch, by a constant value, which leads to the conservation of the slope of discrimination with perceptual difference. In some cases, an additional assumption about exchangeability or rotation of underlying perceptual dimensions leads to a Gaussian pattern of discrimination, which can be understood as a special case of the more general exponential form. The three measurement invariances of shift, stretch, and rotation provide a sufficient explanation for the universally observed patterns of perceptual generalization. All of the additional assumptions and language associated with information, complexity, and empirical scaling are superfluous with regard to the broad patterns of perception.

The Fundamental Equations of Change in Statistical Ensembles and Biological Populations.

A recent article in Nature Physics unified key results from thermodynamics, statistics, and information theory. The unification arose from a general equation for the rate of change in the information content of a system. The general equation describes the change in the moments of an observable quantity over a probability distribution. One term in the equation describes the change in the probability distribution. The other term describes the change in the observable values for a given state. We show the equivalence of this general equation for moment dynamics with the widely known Price equation from evolutionary theory, named after George Price. We introduce the Price equation from its biological roots, review a mathematically abstract form of the equation, and discuss the potential for this equation to unify diverse mathematical theories from different disciplines. The new work in Nature Physics and many applications in biology show that this equation also provides the basis for deriving many novel theoretical results within each discipline.

Evolutionary design of regulatory control. II. Robust error-correcting feedback increases genetic and phenotypic variability.

As systems become more robust against perturbations, they can compensate for greater sloppiness in the performance of their components. That robust compensation reduces the force of natural selection on the system's components, leading to component decay. The paradoxical coupling of robustness and decay predicts that robust systems evolve cheaper, lower performing components, which accumulate greater mutational genetic variability and which have greater phenotypic stochasticity in trait expression. Previous work noted the paradox of robustness. However, no general theory for the evolutionary dynamics of system robustness and component decay has been developed. This article takes a first step by linking engineering control theory with the genetic theory of evolutionary dynamics. Control theory emphasizes error-correcting feedback as the single greatest principle in robust system design. Linking control theory to evolution leads to a theory for the evolutionary dynamics of error-correcting feedback, a unifying approach for the evolutionary analysis of robust systems. This article shows how increasingly robust systems accumulate more genetic variability and greater stochasticity of expression in their components. The theory predicts different levels of variability between different regulatory control architectures and different levels of variability between different components within a particular regulatory control system. The theory also shows that increasing robustness reduces the frequency of system failures associated with disease and, simultaneously, causes a strong increase in the heritability of disease. Thus, robust error correction in biological regulatory control may partly explain the puzzlingly high heritability of disease and, more generally, the surprisingly high heritability of fitness.

Evolutionary design of regulatory control. I. A robust control theory analysis of tradeoffs.

The evolutionary design of regulatory control balances various tradeoffs in performance. Fast reaction to environmental change tends to favor plastic responsiveness at the expense of greater sensitivity to perturbations that degrade homeostatic control. Greater homeostatic stability against unpredictable disturbances tends to reduce performance in tracking environmental change. This article applies the classic principles of engineering control theory to the evolutionary design of regulatory systems. The engineering theory clarifies the conceptual aspects of evolutionary tradeoffs and provides analytic methods for developing specific predictions. On the conceptual side, this article clarifies the meanings of integral control, feedback, and design, concepts that have been discussed in a confusing way within the biological literature. On the analytic side, this article presents extensive methods and examples to study error-correcting feedback, which is perhaps the single greatest principle of design in both human-engineered and naturally designed systems. The broad framework and associated software code provide a comprehensive how-to guide for making models that focus on functional aspects of regulatory control and for making comparative predictions about regulatory design in response to various kinds of environmental challenge. The second article in this series analyzes how alternative regulatory designs influence the relative levels of genetic variability, stochasticity of trait expression, and heritability of disease.

The Price Equation Program: Simple Invariances Unify Population Dynamics, Thermodynamics, Probability, Information and Inference.

The fundamental equations of various disciplines often seem to share the same basic structure. Natural selection increases information in the same way that Bayesian updating increases information. Thermodynamics and the forms of common probability distributions express maximum increase in entropy, which appears mathematically as loss of information. Physical mechanics follows paths of change that maximize Fisher information. The information expressions typically have analogous interpretations as the Newtonian balance between force and acceleration, representing a partition between the direct causes of change and the opposing changes in the frame of reference. This web of vague analogies hints at a deeper common mathematical structure. I suggest that the Price equation expresses that underlying universal structure. The abstract Price equation describes dynamics as the change between two sets. One component of dynamics expresses the change in the frequency of things, holding constant the values associated with things. The other component of dynamics expresses the change in the values of things, holding constant the frequency of things. The separation of frequency from value generalizes Shannon's separation of the frequency of symbols from the meaning of symbols in information theory. The Price equation's generalized separation of frequency and value reveals a few simple invariances that define universal geometric aspects of change. For example, the conservation of total frequency, although a trivial invariance by itself, creates a powerful constraint on the geometry of change. That constraint plus a few others seem to explain the common structural forms of the equations in different disciplines. From that abstract perspective, interpretations such as selection, information, entropy, force, acceleration, and physical work arise from the same underlying geometry expressed by the Price equation.

Evolution of robustness and cellular stochasticity of gene expression.

Gene expression varies widely in cells with the same genotype and environment. Predicting the patterns of stochastic cellular fluctuations remains an unsolved challenge. I propose that the degree to which varying cellular components combine to determine robust phenotypes may predict the amount of variability. Microbes provide excellent experimental models to analyze the relations between robust phenotypes and stochastic variability.

Nonheritable cellular variability accelerates the evolutionary processes of cancer.

Recent cancer studies emphasize that genetic and heritable epigenetic changes drive the evolutionary rate of cancer progression and drug resistance. We discuss the ways in which nonheritable aspects of cellular variability may significantly increase evolutionary rate. Nonheritable variability arises by stochastic fluctuations in cells and by physiological responses of cells to the environment. New approaches to drug design may be required to control nonheritable variability and the evolution of resistance to chemotherapy.

Pathology from evolutionary conflict, with a theory of X chromosome versus autosome conflict over sexually antagonistic traits.

Evolutionary conflicts cause opponents to push increasingly hard and in opposite directions on the regulation of traits. One can see only the intermediate outcome from the balance of the exaggerated and opposed forces. Intermediate expression hides the underlying conflict, potentially misleading one to conclude that trait regulation is designed to achieve efficient and robust expression, rather than arising by the precarious resolution of conflict. Perturbation often reveals the underlying nature of evolutionary conflict. Upon mutation or knockout of one side in the conflict, the other previously hidden and exaggerated push on the trait may cause extreme, pathological expression. In this regard, pathology reveals hidden evolutionary design. We first review several evolutionary conflicts between males and females, including conflicts over mating, fertilization, and the growth rate of offspring. Perturbations of these conflicts lead to infertility, misregulated growth, cancer, behavioral abnormalities, and psychiatric diseases. We then turn to antagonism between the sexes over traits present in both males and females. For many traits, the different sexes favor different phenotypic values, and constraints prevent completely distinct expression in the sexes. In this case of sexual antagonism, we present a theory of conflict between X-linked genes and autosomal genes. We suggest that dysregulation of the exaggerated conflicting forces between the X chromosome and the autosomes may be associated with various pathologies caused by extreme expression along the male-female axis. Rapid evolution of conflicting X-linked and autosomal genes may cause divergence between populations and speciation.

The origin of novel traits in cancer.

The traditional view of cancer emphasizes a genes-first process. Novel cancer traits arise by genetic mutations that spread to drive phenotypic change. However, recent data support a phenotypes-first process in which nonheritable cellular variability creates novel traits that later become heritably stabilized by genetic and epigenetic changes. Single-cell measurements reinforce the idea that phenotypes lead genotypes, showing how cancer evolution follows normal developmental plasticity and creates novel traits by recombining parts of different cellular developmental programs. In parallel, studies in evolutionary biology also support a phenotypes-first process driven by developmental plasticity and developmental recombination. These advances in cancer research and evolutionary biology mutually reinforce a revolution in our understanding of how cells and organisms evolve novel traits in response to environmental challenges.

Evolution in health and medicine Sackler colloquium: Somatic evolutionary genomics: mutations during development cause highly variable genetic mosaicism with risk of cancer and neurodegeneration.

Somatic mutations must happen often during development because of the large number of cell divisions to expand from a single-cell zygote to a full organism. A mutation in development carries forward to all descendant cells, causing genetic mosaicism. Widespread genetic mosaicism may influence diseases that derive from a few genetically altered cells, such as cancer. I show how to predict the expected amount of mosaicism and the variation in mosaicism between individuals. I then calculate the predicted risk of cancer derived from developmental mutations. The calculations show that a significant fraction of cancer in later life likely arises from developmental mutations in early life. In addition, much of the variation in the risk of cancer between individuals may arise from variation in the degree of genetic mosaicism set in early life. I also suggest that certain types of neurodegeneration, such as amyotrophic lateral sclerosis (ALS), may derive from a small focus of genetically altered cells. If so, then the risk of ALS would be influenced by developmental mutations and the consequent variation in genetic mosaicism. New technologies promise the ability to measure genetic mosaicism by sampling a large number of cellular genomes within an individual. The sampling of many genomes within an individual will eventually allow one to reconstruct the cell lineage history of genetic change in a single body. Somatic evolutionary genomics will follow from this technology, providing new insight into the origin and progression of disease with increasing age.

Robustness increases heritability: implications for familial disease.

Robustness protects organisms in two ways. Homeostatic buffering lowers the variation of traits caused by internal or external perturbations. Tolerance reduces the consequences of bad situations, such as extreme phenotypes or infections. This article shows that both types of robustness increase the heritability of protected traits. Additionally, robustness strongly increases the heritability of disease. The natural tendency for organisms to protect robustly against perturbations may partly explain the high heritability that occurs for some diseases.

Optimizing differential equations to fit data and predict outcomes.

Many scientific problems focus on observed patterns of change or on how to design a system to achieve particular dynamics. Those problems often require fitting differential equation models to target trajectories. Fitting such models can be difficult because each evaluation of the fit must calculate the distance between the model and target patterns at numerous points along a trajectory. The gradient of the fit with respect to the model parameters can be challenging to compute. Recent technical advances in automatic differentiation through numerical differential equation solvers potentially change the fitting process into a relatively easy problem, opening up new possibilities to study dynamics. However, application of the new tools to real data may fail to achieve a good fit. This article illustrates how to overcome a variety of common challenges, using the classic ecological data for oscillations in hare and lynx populations. Models include simple ordinary differential equations (ODEs) and neural ordinary differential equations (NODEs), which use artificial neural networks to estimate the derivatives of differential equation systems. Comparing the fits obtained with ODEs versus NODEs, representing small and large parameter spaces, and changing the number of variable dimensions provide insight into the geometry of the observed and model trajectories. To analyze the quality of the models for predicting future observations, a Bayesian-inspired preconditioned stochastic gradient Langevin dynamics (pSGLD) calculation of the posterior distribution of predicted model trajectories clarifies the tendency for various models to underfit or overfit the data. Coupling fitted differential equation systems with pSGLD sampling provides a powerful way to study the properties of optimization surfaces, raising an analogy with mutation-selection dynamics on fitness landscapes.

Disease from opposing forces in regulatory control.

Danger requires a strong rapid response. Speedy triggers are prone to false signals. False alarms can be costly, requiring strong negative regulators to oppose the initial triggers. Strongly opposed forces can easily be perturbed, leading to imbalance and disease. For example, immunity and fear response balance strong rapid triggers against widespread slow negative regulators. Diseases of immunity and behavior arise from imbalance. A different opposition of forces occurs in mammalian growth, which balances strong paternally expressed accelerators against maternally expressed suppressors. Diseases of overgrowth or undergrowth arise from imbalance. Other examples of opposing forces and disease include control of dopamine expression and male versus female favored traits.

Robustness and complexity.

When a system robustly corrects component-level errors, the direct pressure on component performance declines. Components become less reliable, maintain more genetic variability, or drift neutrally, creating new forms of complexity. Examples include the hourglass pattern of biological development and the hourglass architecture for robustly complex systems in engineering.

Evolutionary explanations for heterogeneous behavior in clonal bacterial populations.

Cellular heterogeneity in clonal bacterial populations is widespread. Division of labor and bet hedging are common adaptive explanations for the function of such heterogeneity. We suggest group-level phenotypes via shareable molecules and variation in cellular vigor as two alternative evolutionary explanations for bacterial cellular heterogeneity.

Optimization of Transcription Factor Genetic Circuits.

Transcription factors (TFs) affect the production of mRNAs. In essence, the TFs form a large computational network that controls many aspects of cellular function. This article introduces a computational method to optimize TF networks. The method extends recent advances in artificial neural network optimization. In a simple example, computational optimization discovers a four-dimensional TF network that maintains a circadian rhythm over many days, successfully buffering strong stochastic perturbations in molecular dynamics and entraining to an external day-night signal that randomly turns on and off at intervals of several days. This work highlights the similar challenges in understanding how computational TF and neural networks gain information and improve performance.

The number of neutral mutants in an expanding Luria-Delbrück population is approximately Fréchet.

Background: A growing population of cells accumulates mutations. A single mutation early in the growth process carries forward to all descendant cells, causing the final population to have a lot of mutant cells. When the first mutation happens later in growth, the final population typically has fewer mutants. The number of mutant cells in the final population follows the Luria-Delbrück distribution. The mathematical form of the distribution is known only from its probability generating function. For larger populations of cells, one typically uses computer simulations to estimate the distribution. Methods: This article searches for a simple approximation of the Luria-Delbrück distribution, with an explicit mathematical form that can be used easily in calculations. Results: The Fréchet distribution provides a good approximation for the Luria-Delbrück distribution for neutral mutations, which do not cause a growth rate change relative to the original cells. Conclusions: The Fréchet distribution apparently provides a good match through its description of extreme value problems for multiplicative processes such as exponential growth.

A general model of the public goods dilemma.

An individually costly act that benefits all group members is a public good. Natural selection favours individual contribution to public good [corrected] only when some benefit to the individual offsets the cost of contribution. Problems of sex ratio, parasite virulence, microbial metabolism, punishment of noncooperators, and nearly all aspects of sociality have been analysed as public goods shaped by kin and group selection. Here, I develop two general aspects of the public goods problem that have received relatively little attention. First, variation in individual resources favours selfish individuals to vary their allocation to public goods. Those individuals better endowed contribute their excess resources to public benefit, whereas those individuals with fewer resources contribute less to the public good. Thus, purely selfish behaviour causes individuals to stratify into upper classes that contribute greatly to public benefit and social cohesion and to lower classes that contribute little to the public good. Second, if group success absolutely requires production of the public good, then the pressure favouring production is relatively high. By contrast, if group success depends weakly on the public good, then the pressure favouring production is relatively weak. Stated in this way, it is obvious that the role of baseline success is important. However, discussions of public goods problems sometimes fail to emphasize this point sufficiently. The models here suggest simple tests for the roles of resource variation and baseline success. Given the widespread importance of public goods, better models and tests would greatly deepen our understanding of many processes in biology and sociality.