Maximum likelihood estimators for colony-forming units.

Measuring the abundance of microbes in a sample is a common procedure with a long history, but best practices are not well-conserved across microbiological fields. Serial dilution methods are commonly used to dilute bacterial cultures to produce countable numbers of colonies, and from these counts, to infer bacterial concentrations measured in colony-forming units (CFUs). The most common methods to generate data for CFU point estimates involve plating bacteria on (or in) a solid growth medium and counting their resulting colonies or counting the number of tubes at a given dilution that have growth. Traditionally, these types of data have been analyzed separately using different analytic methods. Here, we build a direct correspondence between these approaches, which allows one to extend the use of the most probable number method from the liquid tubes experiments, for which it was developed, to the growth plates by viewing colony-sized patches of a plate as equivalent to individual tubes. We also discuss how to combine measurements taken at different dilutions, and we review several ways of analyzing colony counts, including the Poisson and truncated Poisson methods. We test all point estimate methods computationally using simulated data. For all methods, we discuss their relevant error bounds, assumptions, strengths, and weaknesses. We provide an online calculator for these estimators.Estimation of the number of microbes in a sample is an important problem with a long history. Yet common practices, such as combining results from different measurements, remain sub-optimal. We provide a comparison of methods for estimating abundance of microbes and detail a mapping between different methods, which allows to extend their range of applicability. This mapping enables higher precision estimates of colony-forming units (CFUs) using the same data already collected for traditional CFU estimation methods. Furthermore, we provide recommendations for how to combine measurements of colony counts taken across dilutions, correcting several misconceptions in the literature.

A mathematical model for ketosis-prone diabetes suggests the existence of multiple pancreatic ß-cell inactivation mechanisms.

Ketosis-prone diabetes mellitus (KPD) is a subtype of type 2 diabetes, which presents much like type 1 diabetes, with dramatic hyperglycemia and ketoacidosis. Although KPD patients are initially insulin-dependent, after a few months of insulin treatment, ~ 70% undergo near-normoglycemia remission and can maintain blood glucose without insulin, as in early type 2 diabetes or prediabetes. Here, we propose that these phenomena can be explained by the existence of a fast, reversible glucotoxicity process, which may exist in all people but be more pronounced in those susceptible to KPD. We develop a simple mathematical model of the pathogenesis of KPD, which incorporates this assumption, and show that it reproduces the phenomenology of KPD, including variations in the ability for patients to achieve and sustain remission. These results suggest that a variation of our model may be able to quantitatively describe variations in the course of remission among individuals with KPD.

Data Efficiency, Dimensionality Reduction, and the Generalized Symmetric Information Bottleneck.

The symmetric information bottleneck (SIB), an extension of the more familiar information bottleneck, is a dimensionality-reduction technique that simultaneously compresses two random variables to preserve information between their compressed versions. We introduce the generalized symmetric information bottleneck (GSIB), which explores different functional forms of the cost of such simultaneous reduction. We then explore the data set size requirements of such simultaneous compression. We do this by deriving bounds and root-mean-squared estimates of statistical fluctuations of the involved loss functions. We show that in typical situations, the simultaneous GSIB compression requires qualitatively less data to achieve the same errors compared to compressing variables one at a time. We suggest that this is an example of a more general principle that simultaneous compression is more data efficient than independent compression of each of the input variables.

Bayesian estimation of the Kullback-Leibler divergence for categorical systems using mixtures of Dirichlet priors.

In many applications in biology, engineering, and economics, identifying similarities and differences between distributions of data from complex processes requires comparing finite categorical samples of discrete counts. Statistical divergences quantify the difference between two distributions. However, their estimation is very difficult and empirical methods often fail, especially when the samples are small. We develop a Bayesian estimator of the Kullback-Leibler divergence between two probability distributions that makes use of a mixture of Dirichlet priors on the distributions being compared. We study the properties of the estimator on two examples: probabilities drawn from Dirichlet distributions and random strings of letters drawn from Markov chains. We extend the approach to the squared Hellinger divergence. Both estimators outperform other estimation techniques, with better results for data with a large number of categories and for higher values of divergences.

Neural criticality from effective latent variables.

Observations of power laws in neural activity data have raised the intriguing notion that brains may operate in a critical state. One example of this critical state is 'avalanche criticality', which has been observed in various systems, including cultured neurons, zebrafish, rodent cortex, and human EEG. More recently, power laws were also observed in neural populations in the mouse under an activity coarse-graining procedure, and they were explained as a consequence of the neural activity being coupled to multiple latent dynamical variables. An intriguing possibility is that avalanche criticality emerges due to a similar mechanism. Here, we determine the conditions under which latent dynamical variables give rise to avalanche criticality. We find that populations coupled to multiple latent variables produce critical behavior across a broader parameter range than those coupled to a single, quasi-static latent variable, but in both cases, avalanche criticality is observed without fine-tuning of model parameters. We identify two regimes of avalanches, both critical but differing in the amount of information carried about the latent variable. Our results suggest that avalanche criticality arises in neural systems in which activity is effectively modeled as a population driven by a few dynamical variables and these variables can be inferred from the population activity.

Extrinsic vs Intrinsic Criticality in Systems with Many Components.

Biological systems with many components often exhibit seemingly critical behaviors, characterized by atypically large correlated fluctuations. Yet the underlying causes remain unclear. Here we define and examine two types of criticality. Intrinsic criticality arises from interactions within the system which are fine-tuned to a critical point. Extrinsic criticality, in contrast, emerges without fine tuning when observable degrees of freedom are coupled to unobserved fluctuating variables. We unify both types of criticality using the language of learning and information theory. We show that critical correlations, intrinsic or extrinsic, lead to diverging mutual information between two halves of the system, and are a feature of learning problems, in which the unobserved fluctuations are inferred from the observable degrees of freedom. We argue that extrinsic criticality is equivalent to standard inference, whereas intrinsic criticality describes fractional learning, in which the amount to be learned depends on the system size. We show further that both types of criticality are on the same continuum, connected by a smooth crossover. In addition, we investigate the observability of Zipf's law, a power-law rank-frequency distribution often used as an empirical signature of criticality. We find that Zipf's law is a robust feature of extrinsic criticality but can be nontrivial to observe for some intrinsically critical systems, including critical mean-field models We further demonstrate that models with global dynamics, such as oscillatory models, can produce observable Zipf's law without relying on either external fluctuations or fine tuning. Our findings suggest that while possible in theory, fine tuning is not the only, nor the most likely, explanation for the apparent ubiquity of criticality in biological systems with many components. Our work offers an alternative interpretation in which criticality, specifically extrinsic criticality, results from the adaptation of collective behavior to external stimuli.

Inferring local structure from pairwise correlations.

To construct models of large, multivariate complex systems, such as those in biology, one needs to constrain which variables are allowed to interact. This can be viewed as detecting "local" structures among the variables. In the context of a simple toy model of two-dimensional natural and synthetic images, we show that pairwise correlations between the variables-even when severely undersampled-provide enough information to recover local relations, including the dimensionality of the data, and to reconstruct arrangement of pixels in fully scrambled images. This proves to be successful even though higher order interaction structures are present in our data. We build intuition behind the success, which we hope might contribute to modeling complex, multivariate systems and to explaining the success of modern attention-based machine learning approaches.

Low-probability states, data statistics, and entropy estimation.

A fundamental problem in the analysis of complex systems is getting a reliable estimate of the entropy of their probability distributions over the state space. This is difficult because unsampled states can contribute substantially to the entropy, while they do not contribute to the maximum likelihood estimator of entropy, which replaces probabilities by the observed frequencies. Bayesian estimators overcome this obstacle by introducing a model of the low-probability tail of the probability distribution. Which statistical features of the observed data determine the model of the tail, and hence the output of such estimators, remains unclear. Here we show that well-known entropy estimators for probability distributions on discrete state spaces model the structure of the low-probability tail based largely on a few statistics of the data: the sample size, the maximum likelihood estimate, the number of coincidences among the samples, and the dispersion of the coincidences. We derive approximate analytical entropy estimators for undersampled distributions based on these statistics, and we use the results to propose an intuitive understanding of how the Bayesian entropy estimators work.

A dynamical model of C. elegans thermal preference reveals independent excitatory and inhibitory learning pathways.

Caenorhabditis elegans is capable of learning and remembering behaviorally relevant cues such as smells, tastes, and temperature. This is an example of associative learning, a process in which behavior is modified by making associations between various stimuli. Since the mathematical theory of conditioning does not account for some of its salient aspects, such as spontaneous recovery of extinguished associations, accurate modeling of behavior of real animals during conditioning has turned out difficult. Here, we do this in the context of the dynamics of the thermal preference of C. elegans. We quantify C. elegans thermotaxis in response to various conditioning temperatures, starvation durations, and genetic perturbations using a high-resolution microfluidic droplet assay. We model these data comprehensively, within a biologically interpretable, multi-modal framework. We find that the strength of the thermal preference is composed of two independent, genetically separable contributions and requires a model with at least four dynamical variables. One pathway positively associates the experienced temperature independently of food and the other negatively associates with the temperature when food is absent. The multidimensional structure of the association strength provides an explanation for the apparent classical temperature-food association of C. elegans thermal preference and a number of longstanding questions in animal learning, including spontaneous recovery, asymmetric response to appetitive vs. aversive cues, latent inhibition, and generalization among similar cues.

Neural criticality from effective latent variables.

Observations of power laws in neural activity data have raised the intriguing notion that brains may operate in a critical state. One example of this critical state is "avalanche criticality," which has been observed in various systems, including cultured neurons, zebrafish, rodent cortex, and human EEG. More recently, power laws were also observed in neural populations in the mouse under an activity coarse-graining procedure, and they were explained as a consequence of the neural activity being coupled to multiple latent dynamical variables. An intriguing possibility is that avalanche criticality emerges due to a similar mechanism. Here, we determine the conditions under which latent dynamical variables give rise to avalanche criticality. We find that populations coupled to multiple latent variables produce critical behavior across a broader parameter range than those coupled to a single, quasi-static latent variable, but in both cases, avalanche criticality is observed without fine-tuning of model parameters. We identify two regimes of avalanches, both critical but differing in the amount of information carried about the latent variable. Our results suggest that avalanche criticality arises in neural systems in which activity is effectively modeled as a population driven by a few dynamical variables and these variables can be inferred from the population activity.

Statistical properties of large data sets with linear latent features.

Analytical understanding of how low-dimensional latent features reveal themselves in large-dimensional data is still lacking. We study this by defining a probabilistic linear latent features model with additive noise and by analytically and numerically computing the statistical distributions of pairwise correlations and eigenvalues of the data correlation matrix. This allows us to resolve the latent feature structure across a wide range of data regimes set by the number of recorded variables, observations, latent features, and the signal-to-noise ratio. We find a characteristic imprint of latent features in the distribution of correlations and eigenvalues and provide an analytic estimate for the boundary between signal and noise, even in the absence of a spectral gap.

Unsupervised Bayesian Ising Approximation for decoding neural activity and other biological dictionaries.

The problem of deciphering how low-level patterns (action potentials in the brain, amino acids in a protein, etc.) drive high-level biological features (sensorimotor behavior, enzymatic function) represents the central challenge of quantitative biology. The lack of general methods for doing so from the size of datasets that can be collected experimentally severely limits our understanding of the biological world. For example, in neuroscience, some sensory and motor codes have been shown to consist of precisely timed multi-spike patterns. However, the combinatorial complexity of such pattern codes have precluded development of methods for their comprehensive analysis. Thus, just as it is hard to predict a protein's function based on its sequence, we still do not understand how to accurately predict an organism's behavior based on neural activity. Here, we introduce the unsupervised Bayesian Ising Approximation (uBIA) for solving this class of problems. We demonstrate its utility in an application to neural data, detecting precisely timed spike patterns that code for specific motor behaviors in a songbird vocal system. In data recorded during singing from neurons in a vocal control region, our method detects such codewords with an arbitrary number of spikes, does so from small data sets, and accounts for dependencies in occurrences of codewords. Detecting such comprehensive motor control dictionaries can improve our understanding of skilled motor control and the neural bases of sensorimotor learning in animals. To further illustrate the utility of uBIA, we used it to identify the distinct sets of activity patterns that encode vocal motor exploration versus typical song production. Crucially, our method can be used not only for analysis of neural systems, but also for understanding the structure of correlations in other biological and nonbiological datasets.

Constrained brain volume in an efficient coding model explains the fraction of excitatory and inhibitory neurons in sensory cortices.

The number of neurons in mammalian cortex varies by multiple orders of magnitude across different species. In contrast, the ratio of excitatory to inhibitory neurons (E:I ratio) varies in a much smaller range, from 3:1 to 9:1 and remains roughly constant for different sensory areas within a species. Despite this structure being important for understanding the function of neural circuits, the reason for this consistency is not yet understood. While recent models of vision based on the efficient coding hypothesis show that increasing the number of both excitatory and inhibitory cells improves stimulus representation, the two cannot increase simultaneously due to constraints on brain volume. In this work, we implement an efficient coding model of vision under a constraint on the volume (using number of neurons as a surrogate) while varying the E:I ratio. We show that the performance of the model is optimal at biologically observed E:I ratios under several metrics. We argue that this happens due to trade-offs between the computational accuracy and the representation capacity for natural stimuli. Further, we make experimentally testable predictions that 1) the optimal E:I ratio should be higher for species with a higher sparsity in the neural activity and 2) the character of inhibitory synaptic distributions and firing rates should change depending on E:I ratio. Our findings, which are supported by our new preliminary analyses of publicly available data, provide the first quantitative and testable hypothesis based on optimal coding models for the distribution of excitatory and inhibitory neural types in the mammalian sensory cortices.

Latent Dynamical Variables Produce Signatures of Spatiotemporal Criticality in Large Biological Systems.

Understanding the activity of large populations of neurons is difficult due to the combinatorial complexity of possible cell-cell interactions. To reduce the complexity, coarse graining had been previously applied to experimental neural recordings, which showed over two decades of apparent scaling in free energy, activity variance, eigenvalue spectra, and correlation time, hinting that the mouse hippocampus operates in a critical regime. We model such data by simulating conditionally independent binary neurons coupled to a small number of long-timescale stochastic fields and then replicating the coarse-graining procedure and analysis. This reproduces the experimentally observed scalings, suggesting that they do not require fine-tuning of internal parameters, but will arise in any system, biological or not, where activity variables are coupled to latent dynamic stimuli. Parameter sweeps for our model suggest that emergence of scaling requires most of the cells in a population to couple to the latent stimuli, predicting that even the celebrated place cells must also respond to nonplace stimuli.

Inferring phenomenological models of first passage processes.

Biochemical processes in cells are governed by complex networks of many chemical species interacting stochastically in diverse ways and on different time scales. Constructing microscopically accurate models of such networks is often infeasible. Instead, here we propose a systematic framework for building phenomenological models of such networks from experimental data, focusing on accurately approximating the time it takes to complete the process, the First Passage (FP) time. Our phenomenological models are mixtures of Gamma distributions, which have a natural biophysical interpretation. The complexity of the models is adapted automatically to account for the amount of available data and its temporal resolution. The framework can be used for predicting behavior of FP systems under varying external conditions. To demonstrate the utility of the approach, we build models for the distribution of inter-spike intervals of a morphologically complex neuron, a Purkinje cell, from experimental and simulated data. We demonstrate that the developed models can not only fit the data, but also make nontrivial predictions. We demonstrate that our coarse-grained models provide constraints on more mechanistically accurate models of the involved phenomena.

Precise spatial memory in local random networks.

Self-sustained, elevated neuronal activity persisting on timescales of 10 s or longer is thought to be vital for aspects of working memory, including brain representations of real space. Continuous-attractor neural networks, one of the most well-known modeling frameworks for persistent activity, have been able to model crucial aspects of such spatial memory. These models tend to require highly structured or regular synaptic architectures. In contrast, we study numerical simulations of a geometrically embedded model with a local, but otherwise random, connectivity profile; imposing a global regulation of our system's mean firing rate produces localized, finely spaced discrete attractors that effectively span a two-dimensional manifold. We demonstrate how the set of attracting states can reliably encode a representation of the spatial locations at which the system receives external input, thereby accomplishing spatial memory via attractor dynamics without synaptic fine-tuning or regular structure. We then measure the network's storage capacity numerically and find that the statistics of retrievable positions are also equivalent to a full tiling of the plane, something hitherto achievable only with (approximately) translationally invariant synapses, and which may be of interest in modeling such biological phenomena as visuospatial working memory in two dimensions.

Randomly connected networks generate emergent selectivity and predict decoding properties of large populations of neurons.

Modern recording methods enable sampling of thousands of neurons during the performance of behavioral tasks, raising the question of how recorded activity relates to theoretical models. In the context of decision making, functional connectivity between choice-selective cortical neurons was recently reported. The straightforward interpretation of these data suggests the existence of selective pools of inhibitory and excitatory neurons. Computationally investigating an alternative mechanism for these experimental observations, we find that a randomly connected network of excitatory and inhibitory neurons generates single-cell selectivity, patterns of pairwise correlations, and the same ability of excitatory and inhibitory populations to predict choice, as in experimental observations. Further, we predict that, for this task, there are no anatomically defined subpopulations of neurons representing choice, and that choice preference of a particular neuron changes with the details of the task. We suggest that distributed stimulus selectivity and functional organization in population codes could be emergent properties of randomly connected networks.

Universal Properties of Concentration Sensing in Large Ligand-Receptor Networks.

Cells estimate concentrations of chemical ligands in their environment using a limited set of receptors. Recent work has shown that the temporal sequence of binding and unbinding events on just a single receptor can be used to estimate the concentrations of multiple ligands. Here, for a network of many ligands and many receptors, we show that such temporal sequences can be used to estimate the concentration of a few times as many ligand species as there are receptors. Crucially, we show that the spectrum of the inverse covariance matrix of these estimates has several universal properties, which we trace to properties of Vandermonde matrices. We argue that this can be used by cells in realistic biochemical decoding networks.

Physical Limit to Concentration Sensing in a Changing Environment.

Cells adapt to changing environments by sensing ligand concentrations using specific receptors. The accuracy of sensing is ultimately limited by the finite number of ligand molecules bound by receptors. Previously derived physical limits to sensing accuracy largely have assumed that the concentration was constant and ignored its temporal fluctuations. We formulate the problem of concentration sensing in a strongly fluctuating environment as a nonlinear field-theoretic problem, for which we find an excellent approximate Gaussian solution. We derive a new physical bound on the relative error in concentration c which scales as δc/c∼(Dacτ)^{-1/4} with ligand diffusivity D, receptor cross section a, and characteristic fluctuation timescale τ, in stark contrast to the usual Berg and Purcell bound δc/c∼(DacT)^{-1/2} for a perfect receptor sensing concentration during time T. We show how the bound can be achieved by a biochemical network downstream of the receptor that adapts the kinetics of signaling as a function of the square root of the sensed concentration.

Estimation of mutual information for real-valued data with error bars and controlled bias.

Estimation of mutual information between (multidimensional) real-valued variables is used in analysis of complex systems, biological systems, and recently also quantum systems. This estimation is a hard problem, and universally good estimators provably do not exist. We focus on the estimator introduced by Kraskov et al. [Phys. Rev. E 69, 066138 (2004)PLEEE81539-375510.1103/PhysRevE.69.066138] based on the statistics of distances between neighboring data points, which empirically works for a wide class of underlying probability distributions. First, we illustrate pitfalls of naively applying bootstrapping to estimate the variance of the mutual information estimate. Then we improve this estimator by (1) expanding its range of applicability and by providing (2) a self-consistent way of verifying the absence of bias, (3) a method for estimation of its variance, and (4) guidelines for choosing the values of the free parameter of the estimator. We demonstrate the performance of our estimator on synthetic data sets, as well as on neurophysiological and systems biology data sets.