Long Timescales, Individual Differences, and Scale Invariance in Animal Behavior.

The explosion of data on animal behavior in more natural contexts highlights the fact that these behaviors exhibit correlations across many timescales. However, there are major challenges in analyzing these data: records of behavior in single animals have fewer independent samples than one might expect. In pooling data from multiple animals, individual differences can mimic long-ranged temporal correlations; conversely, long-ranged correlations can lead to an overestimate of individual differences. We suggest an analysis scheme that addresses these problems directly, apply this approach to data on the spontaneous behavior of walking flies, and find evidence for scale-invariant correlations over nearly three decades in time, from seconds to one hour. Three different measures of correlation are consistent with a single underlying scaling field of dimension Δ=0.180±0.005.

Exactly solvable statistical physics models for large neuronal populations.

Maximum entropy methods provide a principled path connecting measurements of neural activity directly to statistical physics models, and this approach has been successful for populations of N~100 neurons. As N increases in new experiments, we enter an undersampled regime where we have to choose which observables should be constrained in the maximum entropy construction. The best choice is the one that provides the greatest reduction in entropy, defining a "minimax entropy" principle. This principle becomes tractable if we restrict attention to correlations among pairs of neurons that link together into a tree; we can find the best tree efficiently, and the underlying statistical physics models are exactly solved. We use this approach to analyze experiments on N~1500 neurons in the mouse hippocampus, and show that the resulting model captures the distribution of synchronous activity in the network.

Long time scales, individual differences, and scale invariance in animal behavior.

The explosion of data on animal behavior in more natural contexts highlights the fact that these behaviors exhibit correlations across many time scales. But there are major challenges in analyzing these data: records of behavior in single animals have fewer independent samples than one might expect; in pooling data from multiple animals, individual differences can mimic long-ranged temporal correlations; conversely long-ranged correlations can lead to an over-estimate of individual differences. We suggest an analysis scheme that addresses these problems directly, apply this approach to data on the spontaneous behavior of walking flies, and find evidence for scale invariant correlations over nearly three decades in time, from seconds to one hour. Three different measures of correlation are consistent with a single underlying scaling field of dimension $\Delta = 0.180\pm 0.005$.

Scale invariance in early embryonic development.

The body plan of the fruit fly is determined by the expression of just a handful of genes. We show that the spatial patterns of expression for several of these genes scale precisely with the size of the embryo. Concretely, discrete positional markers such as the peaks in striped patterns have absolute positions along the anterior-posterior axis that are proportional to embryo length, with better than 1% accuracy. Further, the information (in bits) that graded patterns of expression provide about position can be decomposed into information about fractional or scaled position and information about absolute position or embryo length; all of the available information is about scaled position, again with ~1% accuracy. These observations suggest that the underlying genetic network exhibits scale invariance in a deeper mathematical sense. Taking this mathematical statement seriously requires that the network dynamics have a zero mode, which connects to many other observations on this system.

Emergence of local irreversibility in complex interacting systems.

Living systems are fundamentally irreversible, breaking detailed balance and establishing an arrow of time. But how does the evident arrow of time for a whole system arise from the interactions among its multiple elements? We show that the local evidence for the arrow of time, which is the entropy production for thermodynamic systems, can be decomposed. First, it can be split into two components: an independent term reflecting the dynamics of individual elements and an interaction term driven by the dependencies among elements. Adapting tools from nonequilibrium physics, we further decompose the interaction term into contributions from pairs of elements, triplets, and higher-order terms. We illustrate our methods on models of cellular sensing and logical computations, as well as on patterns of neural activity in the retina as it responds to visual inputs. We find that neural activity can define the arrow of time even when the visual inputs do not, and that the dominant contribution to this breaking of detailed balance comes from interactions among pairs of neurons.

Decomposing the Local Arrow of Time in Interacting Systems.

We show that the evidence for a local arrow of time, which is equivalent to the entropy production in thermodynamic systems, can be decomposed. In a system with many degrees of freedom, there is a term that arises from the irreversible dynamics of the individual variables, and then a series of non-negative terms contributed by correlations among pairs, triplets, and higher-order combinations of variables. We illustrate this decomposition on simple models of noisy logical computations, and then apply it to the analysis of patterns of neural activity in the retina as it responds to complex dynamic visual scenes. We find that neural activity breaks detailed balance even when the visual inputs do not, and that this irreversibility arises primarily from interactions between pairs of neurons.

On the dimensionality of behavior.

There is a growing effort in the "physics of behavior" that aims at complete quantitative characterization of animal movements under more complex, naturalistic conditions. One reaction to the resulting explosion of high-dimensional data is the search for low-dimensional structure. Here I try to define more clearly what we mean by the dimensionality of behavior, where observable behavior may consist of either continuous trajectories or sequences of discrete states. This discussion also serves to isolate situations in which the dimensionality of behavior is effectively infinite.

Trading bits in the readout from a genetic network.

In the regulation of gene expression, information of relevance to the organism is represented by the concentrations of transcription factor molecules. To extract this information the cell must effectively "measure" these concentrations, but there are physical limits to the precision of these measurements. We use the gap gene network in the early fly embryo as an example of the tradeoff between the precision of concentration measurements and the transmission of relevant information. For thresholded measurements we find that lower thresholds are more important, and fine tuning is not required for near-optimal information transmission. We then consider general sensors, constrained only by a limit on their information capacity, and find that thresholded sensors can approach true information theoretic optima. The information theoretic approach allows us to identify the optimal sensor for the entire gap gene network and to argue that the physical limitations of sensing necessitate the observed multiplicity of enhancer elements, with sensitivities to combinations rather than single transcription factors.

Optimal Local Estimates of Visual Motion in a Natural Environment.

Many organisms use visual signals to estimate motion, and these estimates typically are biased. Here, we ask whether these biases may reflect physical rather than biological limitations. Using a camera-gyroscope system, we sample the joint distribution of images and rotational motions in a natural environment, and from this distribution we construct the optimal estimator of velocity based on local image intensities. Over most of the natural dynamic range, this estimator exhibits the biases observed in neural and behavioral responses. Thus, imputed errors in sensory processing may represent an optimal response to the physical signals sampled from the environment.

Coarse Graining, Fixed Points, and Scaling in a Large Population of Neurons.

We develop a phenomenological coarse-graining procedure for activity in a large network of neurons, and apply this to recordings from a population of 1000+ cells in the hippocampus. Distributions of coarse-grained variables seem to approach a fixed non-Gaussian form, and we see evidence of scaling in both static and dynamic quantities. These results suggest that the collective behavior of the network is described by a nontrivial fixed point.

Searching for collective behavior in a small brain.

In large neuronal networks, it is believed that functions emerge through the collective behavior of many interconnected neurons. Recently, the development of experimental techniques that allow simultaneous recording of calcium concentration from a large fraction of all neurons in Caenorhabditis elegans-a nematode with 302 neurons-creates the opportunity to ask whether such emergence is universal, reaching down to even the smallest brains. Here, we measure the activity of 50+ neurons in C. elegans, and analyze the data by building the maximum entropy model that matches the mean activity and pairwise correlations among these neurons. To capture the graded nature of the cells' responses, we assign each cell multiple states. These models, which are equivalent to a family of Potts glasses, successfully predict higher statistical structure in the network. In addition, these models exhibit signatures of collective behavior: the state of single cells can be predicted from the state of the rest of the network; the network, despite being sparse in a way similar to the structural connectome, distributes its response globally when locally perturbed; the distribution over network states has multiple local maxima, as in models of memory; and the parameters that describe the real network are close to a critical surface in this family of models.

Optimal Decoding of Cellular Identities in a Genetic Network.

In developing organisms, spatially prescribed cell identities are thought to be determined by the expression levels of multiple genes. Quantitative tests of this idea, however, require a theoretical framework capable of exposing the rules and precision of cell specification over developmental time. We use the gap gene network in the early fly embryo as an example to show how expression levels of the four gap genes can be jointly decoded into an optimal specification of position with 1% accuracy. The decoder correctly predicts, with no free parameters, the dynamics of pair-rule expression patterns at different developmental time points and in various mutant backgrounds. Precise cellular identities are thus available at the earliest stages of development, contrasting the prevailing view of positional information being slowly refined across successive layers of the patterning network. Our results suggest that developmental enhancers closely approximate a mathematically optimal decoding strategy.

Perspectives on theory at the interface of physics and biology.

Theoretical physics is the search for simple and universal mathematical descriptions of the natural world. In contrast, much of modern biology is an exploration of the complexity and diversity of life. For many, this contrast is prima facie evidence that theory, in the sense that physicists use the word, is impossible in a biological context. For others, this contrast serves to highlight a grand challenge. I am an optimist, and believe (along with many colleagues) that the time is ripe for the emergence of a more unified theoretical physics of biological systems, building on successes in thinking about particular phenomena. In this essay I try to explain the reasons for my optimism, through a combination of historical and modern examples.

Collective Behavior of Place and Non-place Neurons in the Hippocampal Network.

Discussions of the hippocampus often focus on place cells, but many neurons are not place cells in any given environment. Here we describe the collective activity in such mixed populations, treating place and non-place cells on the same footing. We start with optical imaging experiments on CA1 in mice as they run along a virtual linear track and use maximum entropy methods to approximate the distribution of patterns of activity in the population, matching the correlations between pairs of cells but otherwise assuming as little structure as possible. We find that these simple models accurately predict the activity of each neuron from the state of all the other neurons in the network, regardless of how well that neuron codes for position. Our results suggest that understanding the neural activity may require not only knowledge of the external variables modulating it but also of the internal network state.

PCA meets RG.

A system with many degrees of freedom can be characterized by a covariance matrix; principal components analysis (PCA) focuses on the eigenvalues of this matrix, hoping to find a lower dimensional description. But when the spectrum is nearly continuous, any distinction between components that we keep and those that we ignore becomes arbitrary; it then is natural to ask what happens as we vary this arbitrary cutoff. We argue that this problem is analogous to the momentum shell renormalization group (RG). Following this analogy, we can define relevant and irrelevant operators, where the role of dimensionality is played by properties of the eigenvalue density. These results also suggest an approach to the analysis of real data. As an example, we study neural activity in the vertebrate retina as it responds to naturalistic movies, and find evidence of behavior controlled by a nontrivial fixed point. Applied to financial data, our analysis separates modes dominated by sampling noise from a smaller but still macroscopic number of modes described by a non-Gaussian distribution.

Complexity of generic biochemical circuits: topology versus strength of interactions.

The historical focus on network topology as a determinant of biological function is still largely maintained today, illustrated by the rise of structure-only approaches to network analysis. However, biochemical circuits and genetic regulatory networks are defined both by their topology and by a multitude of continuously adjustable parameters, such as the strength of interactions between nodes, also recognized as important. Here we present a class of simple perceptron-based Boolean models within which comparing the relative importance of topology versus interaction strengths becomes a quantitatively well-posed problem. We quantify the intuition that for generic networks, optimization of interaction strengths is a crucial ingredient of achieving high complexity, defined here as the number of fixed points the network can accommodate. We propose a new methodology for characterizing the relative role of parameter optimization for topologies of a given class.

Predictability and hierarchy in Drosophila behavior.

Even the simplest of animals exhibit behavioral sequences with complex temporal dynamics. Prominent among the proposed organizing principles for these dynamics has been the idea of a hierarchy, wherein the movements an animal makes can be understood as a set of nested subclusters. Although this type of organization holds potential advantages in terms of motion control and neural circuitry, measurements demonstrating this for an animal's entire behavioral repertoire have been limited in scope and temporal complexity. Here, we use a recently developed unsupervised technique to discover and track the occurrence of all stereotyped behaviors performed by fruit flies moving in a shallow arena. Calculating the optimally predictive representation of the fly's future behaviors, we show that fly behavior exhibits multiple time scales and is organized into a hierarchical structure that is indicative of its underlying behavioral programs and its changing internal states.

Entropic effects in a nonequilibrium system: Flocks of birds.

When European starlings come together to form a flock, the distribution of their individual velocities narrows around the mean velocity of the flock. We argue that, in a broad class of models for the joint distribution of positions and velocities, this narrowing generates an entropic effect that opposes the cohesion of the flock. The strength of this effect depends strongly on the nature of the interactions among birds: If birds are coupled to a fixed number of neighbors, the entropic forces are weak, while if they couple to all other birds within a fixed distance, the entropic effects are sufficient to tear a flock apart.

Extending the dynamic range of transcription factor action by translational regulation.

A crucial step in the regulation of gene expression is binding of transcription factor (TF) proteins to regulatory sites along the DNA. But transcription factors act at nanomolar concentrations, and noise due to random arrival of these molecules at their binding sites can severely limit the precision of regulation. Recent work on the optimization of information flow through regulatory networks indicates that the lower end of the dynamic range of concentrations is simply inaccessible, overwhelmed by the impact of this noise. Motivated by the behavior of homeodomain proteins, such as the maternal morphogen Bicoid in the fruit fly embryo, we suggest a scheme in which transcription factors also act as indirect translational regulators, binding to the mRNA of other regulatory proteins. Intuitively, each mRNA molecule acts as an independent sensor of the input concentration, and averaging over these multiple sensors reduces the noise. We analyze information flow through this scheme and identify conditions under which it outperforms direct transcriptional regulation. Our results suggest that the dual role of homeodomain proteins is not just a historical accident, but a solution to a crucial physics problem in the regulation of gene expression.

Thermodynamics and signatures of criticality in a network of neurons.

The activity of a neural network is defined by patterns of spiking and silence from the individual neurons. Because spikes are (relatively) sparse, patterns of activity with increasing numbers of spikes are less probable, but, with more spikes, the number of possible patterns increases. This tradeoff between probability and numerosity is mathematically equivalent to the relationship between entropy and energy in statistical physics. We construct this relationship for populations of up to N = 160 neurons in a small patch of the vertebrate retina, using a combination of direct and model-based analyses of experiments on the response of this network to naturalistic movies. We see signs of a thermodynamic limit, where the entropy per neuron approaches a smooth function of the energy per neuron as N increases. The form of this function corresponds to the distribution of activity being poised near an unusual kind of critical point. We suggest further tests of criticality, and give a brief discussion of its functional significance.