Optimal Decoding of Cellular Identities in a Genetic Network.

In developing organisms, spatially prescribed cell identities are thought to be determined by the expression levels of multiple genes. Quantitative tests of this idea, however, require a theoretical framework capable of exposing the rules and precision of cell specification over developmental time. We use the gap gene network in the early fly embryo as an example to show how expression levels of the four gap genes can be jointly decoded into an optimal specification of position with 1% accuracy. The decoder correctly predicts, with no free parameters, the dynamics of pair-rule expression patterns at different developmental time points and in various mutant backgrounds. Precise cellular identities are thus available at the earliest stages of development, contrasting the prevailing view of positional information being slowly refined across successive layers of the patterning network. Our results suggest that developmental enhancers closely approximate a mathematically optimal decoding strategy.

On the dimensionality of behavior.

There is a growing effort in the "physics of behavior" that aims at complete quantitative characterization of animal movements under more complex, naturalistic conditions. One reaction to the resulting explosion of high-dimensional data is the search for low-dimensional structure. Here I try to define more clearly what we mean by the dimensionality of behavior, where observable behavior may consist of either continuous trajectories or sequences of discrete states. This discussion also serves to isolate situations in which the dimensionality of behavior is effectively infinite.

Long Timescales, Individual Differences, and Scale Invariance in Animal Behavior.

The explosion of data on animal behavior in more natural contexts highlights the fact that these behaviors exhibit correlations across many timescales. However, there are major challenges in analyzing these data: records of behavior in single animals have fewer independent samples than one might expect. In pooling data from multiple animals, individual differences can mimic long-ranged temporal correlations; conversely, long-ranged correlations can lead to an overestimate of individual differences. We suggest an analysis scheme that addresses these problems directly, apply this approach to data on the spontaneous behavior of walking flies, and find evidence for scale-invariant correlations over nearly three decades in time, from seconds to one hour. Three different measures of correlation are consistent with a single underlying scaling field of dimension Δ=0.180±0.005.

Trading bits in the readout from a genetic network.

In the regulation of gene expression, information of relevance to the organism is represented by the concentrations of transcription factor molecules. To extract this information the cell must effectively "measure" these concentrations, but there are physical limits to the precision of these measurements. We use the gap gene network in the early fly embryo as an example of the tradeoff between the precision of concentration measurements and the transmission of relevant information. For thresholded measurements we find that lower thresholds are more important, and fine tuning is not required for near-optimal information transmission. We then consider general sensors, constrained only by a limit on their information capacity, and find that thresholded sensors can approach true information theoretic optima. The information theoretic approach allows us to identify the optimal sensor for the entire gap gene network and to argue that the physical limitations of sensing necessitate the observed multiplicity of enhancer elements, with sensitivities to combinations rather than single transcription factors.

Decomposing the Local Arrow of Time in Interacting Systems.

We show that the evidence for a local arrow of time, which is equivalent to the entropy production in thermodynamic systems, can be decomposed. In a system with many degrees of freedom, there is a term that arises from the irreversible dynamics of the individual variables, and then a series of non-negative terms contributed by correlations among pairs, triplets, and higher-order combinations of variables. We illustrate this decomposition on simple models of noisy logical computations, and then apply it to the analysis of patterns of neural activity in the retina as it responds to complex dynamic visual scenes. We find that neural activity breaks detailed balance even when the visual inputs do not, and that this irreversibility arises primarily from interactions between pairs of neurons.

Optimal Local Estimates of Visual Motion in a Natural Environment.

Many organisms use visual signals to estimate motion, and these estimates typically are biased. Here, we ask whether these biases may reflect physical rather than biological limitations. Using a camera-gyroscope system, we sample the joint distribution of images and rotational motions in a natural environment, and from this distribution we construct the optimal estimator of velocity based on local image intensities. Over most of the natural dynamic range, this estimator exhibits the biases observed in neural and behavioral responses. Thus, imputed errors in sensory processing may represent an optimal response to the physical signals sampled from the environment.

Coarse Graining, Fixed Points, and Scaling in a Large Population of Neurons.

We develop a phenomenological coarse-graining procedure for activity in a large network of neurons, and apply this to recordings from a population of 1000+ cells in the hippocampus. Distributions of coarse-grained variables seem to approach a fixed non-Gaussian form, and we see evidence of scaling in both static and dynamic quantities. These results suggest that the collective behavior of the network is described by a nontrivial fixed point.

Predictability and hierarchy in Drosophila behavior.

Even the simplest of animals exhibit behavioral sequences with complex temporal dynamics. Prominent among the proposed organizing principles for these dynamics has been the idea of a hierarchy, wherein the movements an animal makes can be understood as a set of nested subclusters. Although this type of organization holds potential advantages in terms of motion control and neural circuitry, measurements demonstrating this for an animal's entire behavioral repertoire have been limited in scope and temporal complexity. Here, we use a recently developed unsupervised technique to discover and track the occurrence of all stereotyped behaviors performed by fruit flies moving in a shallow arena. Calculating the optimally predictive representation of the fly's future behaviors, we show that fly behavior exhibits multiple time scales and is organized into a hierarchical structure that is indicative of its underlying behavioral programs and its changing internal states.

Perspectives on theory at the interface of physics and biology.

Theoretical physics is the search for simple and universal mathematical descriptions of the natural world. In contrast, much of modern biology is an exploration of the complexity and diversity of life. For many, this contrast is prima facie evidence that theory, in the sense that physicists use the word, is impossible in a biological context. For others, this contrast serves to highlight a grand challenge. I am an optimist, and believe (along with many colleagues) that the time is ripe for the emergence of a more unified theoretical physics of biological systems, building on successes in thinking about particular phenomena. In this essay I try to explain the reasons for my optimism, through a combination of historical and modern examples.

Predictive information in a sensory population.

Guiding behavior requires the brain to make predictions about the future values of sensory inputs. Here, we show that efficient predictive computation starts at the earliest stages of the visual system. We compute how much information groups of retinal ganglion cells carry about the future state of their visual inputs and show that nearly every cell in the retina participates in a group of cells for which this predictive information is close to the physical limit set by the statistical structure of the inputs themselves. Groups of cells in the retina carry information about the future state of their own activity, and we show that this information can be compressed further and encoded by downstream predictor neurons that exhibit feature selectivity that would support predictive computations. Efficient representation of predictive information is a candidate principle that can be applied at each stage of neural computation.

Thermodynamics and signatures of criticality in a network of neurons.

The activity of a neural network is defined by patterns of spiking and silence from the individual neurons. Because spikes are (relatively) sparse, patterns of activity with increasing numbers of spikes are less probable, but, with more spikes, the number of possible patterns increases. This tradeoff between probability and numerosity is mathematically equivalent to the relationship between entropy and energy in statistical physics. We construct this relationship for populations of up to N = 160 neurons in a small patch of the vertebrate retina, using a combination of direct and model-based analyses of experiments on the response of this network to naturalistic movies. We see signs of a thermodynamic limit, where the entropy per neuron approaches a smooth function of the energy per neuron as N increases. The form of this function corresponds to the distribution of activity being poised near an unusual kind of critical point. We suggest further tests of criticality, and give a brief discussion of its functional significance.

Morphogenesis at criticality.

Spatial patterns in the early fruit fly embryo emerge from a network of interactions among transcription factors, the gap genes, driven by maternal inputs. Such networks can exhibit many qualitatively different behaviors, separated by critical surfaces. At criticality, we should observe strong correlations in the fluctuations of different genes around their mean expression levels, a slowing of the dynamics along some but not all directions in the space of possible expression levels, correlations of expression fluctuations over long distances in the embryo, and departures from a Gaussian distribution of these fluctuations. Analysis of recent experiments on the gap gene network shows that all these signatures are observed, and that the different signatures are related in ways predicted by theory. Although there might be other explanations for these individual phenomena, the confluence of evidence suggests that this genetic network is tuned to criticality.

Social interactions dominate speed control in poising natural flocks near criticality.

Flocks of birds exhibit a remarkable degree of coordination and collective response. It is not just that thousands of individuals fly, on average, in the same direction and at the same speed, but that even the fluctuations around the mean velocity are correlated over long distances. Quantitative measurements on flocks of starlings, in particular, show that these fluctuations are scale-free, with effective correlation lengths proportional to the linear size of the flock. Here we construct models for the joint distribution of velocities in the flock that reproduce the observed local correlations between individuals and their neighbors, as well as the variance of flight speeds across individuals, but otherwise have as little structure as possible. These minimally structured or maximum entropy models provide quantitative, parameter-free predictions for the spread of correlations throughout the flock, and these are in excellent agreement with the data. These models are mathematically equivalent to statistical physics models for ordering in magnets, and the correct prediction of scale-free correlations arises because the parameters--completely determined by the data--are in the critical regime. In biological terms, criticality allows the flock to achieve maximal correlation across long distances with limited speed fluctuations.

Complexity of generic biochemical circuits: topology versus strength of interactions.

The historical focus on network topology as a determinant of biological function is still largely maintained today, illustrated by the rise of structure-only approaches to network analysis. However, biochemical circuits and genetic regulatory networks are defined both by their topology and by a multitude of continuously adjustable parameters, such as the strength of interactions between nodes, also recognized as important. Here we present a class of simple perceptron-based Boolean models within which comparing the relative importance of topology versus interaction strengths becomes a quantitatively well-posed problem. We quantify the intuition that for generic networks, optimization of interaction strengths is a crucial ingredient of achieving high complexity, defined here as the number of fixed points the network can accommodate. We propose a new methodology for characterizing the relative role of parameter optimization for topologies of a given class.

Positional information, in bits.

Cells in a developing embryo have no direct way of "measuring" their physical position. Through a variety of processes, however, the expression levels of multiple genes come to be correlated with position, and these expression levels thus form a code for "positional information." We show how to measure this information, in bits, using the gap genes in the Drosophila embryo as an example. Individual genes carry nearly two bits of information, twice as much as would be expected if the expression patterns consisted only of on/off domains separated by sharp boundaries. Taken together, four gap genes carry enough information to define a cell's location with an error bar of ~1 along the anterior/posterior axis of the embryo. This precision is nearly enough for each cell to have a unique identity, which is the maximum information the system can use, and is nearly constant along the length of the embryo. We argue that this constancy is a signature of optimality in the transmission of information from primary morphogen inputs to the output of the gap gene network.

Searching for collective behavior in a large network of sensory neurons.

Maximum entropy models are the least structured probability distributions that exactly reproduce a chosen set of statistics measured in an interacting network. Here we use this principle to construct probabilistic models which describe the correlated spiking activity of populations of up to 120 neurons in the salamander retina as it responds to natural movies. Already in groups as small as 10 neurons, interactions between spikes can no longer be regarded as small perturbations in an otherwise independent system; for 40 or more neurons pairwise interactions need to be supplemented by a global interaction that controls the distribution of synchrony in the population. Here we show that such "K-pairwise" models--being systematic extensions of the previously used pairwise Ising models--provide an excellent account of the data. We explore the properties of the neural vocabulary by: 1) estimating its entropy, which constrains the population's capacity to represent visual information; 2) classifying activity patterns into a small set of metastable collective modes; 3) showing that the neural codeword ensembles are extremely inhomogenous; 4) demonstrating that the state of individual neurons is highly predictable from the rest of the population, allowing the capacity for error correction.

Statistical mechanics for natural flocks of birds.

Flocking is a typical example of emergent collective behavior, where interactions between individuals produce collective patterns on the large scale. Here we show how a quantitative microscopic theory for directional ordering in a flock can be derived directly from field data. We construct the minimally structured (maximum entropy) model consistent with experimental correlations in large flocks of starlings. The maximum entropy model shows that local, pairwise interactions between birds are sufficient to correctly predict the propagation of order throughout entire flocks of starlings, with no free parameters. We also find that the number of interacting neighbors is independent of flock density, confirming that interactions are ruled by topological rather than metric distance. Finally, by comparing flocks of different sizes, the model correctly accounts for the observed scale invariance of long-range correlations among the fluctuations in flight direction.

Inverse spin glass and related maximum entropy problems.

If we have a system of binary variables and we measure the pairwise correlations among these variables, then the least structured or maximum entropy model for their joint distribution is an Ising model with pairwise interactions among the spins. Here we consider inhomogeneous systems in which we constrain, for example, not the full matrix of correlations, but only the distribution from which these correlations are drawn. In this sense, what we have constructed is an inverse spin glass: rather than choosing coupling constants at random from a distribution and calculating correlations, we choose the correlations from a distribution and infer the coupling constants. We argue that such models generate a block structure in the space of couplings, which provides an explicit solution of the inverse problem. This allows us to generate a phase diagram in the space of (measurable) moments of the distribution of correlations. We expect that these ideas will be most useful in building models for systems that are nonequilibrium statistical mechanics problems, such as networks of real neurons.

Searching for simplicity in the analysis of neurons and behavior.

What fascinates us about animal behavior is its richness and complexity, but understanding behavior and its neural basis requires a simpler description. Traditionally, simplification has been imposed by training animals to engage in a limited set of behaviors, by hand scoring behaviors into discrete classes, or by limiting the sensory experience of the organism. An alternative is to ask whether we can search through the dynamics of natural behaviors to find explicit evidence that these behaviors are simpler than they might have been. We review two mathematical approaches to simplification, dimensionality reduction and the maximum entropy method, and we draw on examples from different levels of biological organization, from the crawling behavior of Caenorhabditis elegans to the control of smooth pursuit eye movements in primates, and from the coding of natural scenes by networks of neurons in the retina to the rules of English spelling. In each case, we argue that the explicit search for simplicity uncovers new and unexpected features of the biological system and that the evidence for simplification gives us a language with which to phrase new questions for the next generation of experiments. The fact that similar mathematical structures succeed in taming the complexity of very different biological systems hints that there is something more general to be discovered.

Emergence of long timescales and stereotyped behaviors in Caenorhabditis elegans.

Animal behaviors often are decomposable into discrete, stereotyped elements, well separated in time. In one model, such behaviors are triggered by specific commands; in the extreme case, the discreteness of behavior is traced to the discreteness of action potentials in the individual command neurons. Here, we use the crawling behavior of the nematode Caenorhabditis elegans to demonstrate the opposite view, in which discreteness, stereotypy, and long timescales emerge from the collective dynamics of the behavior itself. In previous work, we found that as C. elegans crawls, its body moves through a "shape space" in which four dimensions capture approximately 95% of the variance in body shape. Here we show that stochastic dynamics within this shape space predicts transitions between attractors corresponding to abrupt reversals in crawling direction. With no free parameters, our inferred stochastic dynamical system generates reversal timescales and stereotyped trajectories in close agreement with experimental observations. We use the stochastic dynamics to show that the noise amplitude decreases systematically with increasing time away from food, resulting in longer bouts of forward crawling and suggesting that worms can use noise to modify their locomotory behavior.