Explicit mutual information for simple networks and neurons with lognormal activities.

Networks with stochastic variables described by heavy-tailed lognormal distribution are ubiquitous in nature, and hence they deserve an exact information-theoretic characterization. We derive analytical formulas for mutual information between elements of different networks with correlated lognormally distributed activities. In a special case, we find an explicit expression for mutual information between neurons when neural activities and synaptic weights are lognormally distributed, as suggested by experimental data. Comparison of this expression with the case when these two variables have short tails reveals that mutual information with heavy tails for neurons and synapses is generally larger and can diverge for some finite variances in presynaptic firing rates and synaptic weights. This result suggests that evolution might prefer brains with heterogeneous dynamics to optimize information processing.

Cooperativity, Information Gain, and Energy Cost During Early LTP in Dendritic Spines.

We investigate a mutual relationship between information and energy during the early phase of LTP induction and maintenance in a large-scale system of mutually coupled dendritic spines, with discrete internal states and probabilistic dynamics, within the framework of nonequilibrium stochastic thermodynamics. In order to analyze this computationally intractable stochastic multidimensional system, we introduce a pair approximation, which allows us to reduce the spine dynamics into a lower-dimensional manageable system of closed equations. We found that the rates of information gain and energy attain their maximal values during an initial period of LTP (i.e., during stimulation), and after that, they recover to their baseline low values, as opposed to a memory trace that lasts much longer. This suggests that the learning phase is much more energy demanding than the memory phase. We show that positive correlations between neighboring spines increase both a duration of memory trace and energy cost during LTP, but the memory time per invested energy increases dramatically for very strong, positive synaptic cooperativity, suggesting a beneficial role of synaptic clustering on memory duration. In contrast, information gain after LTP is the largest for negative correlations, and energy efficiency of that information generally declines with increasing synaptic cooperativity. We also find that dendritic spines can use sparse representations for encoding long-term information, as both energetic and structural efficiencies of retained information and its lifetime exhibit maxima for low fractions of stimulated synapses during LTP. Moreover, we find that such efficiencies drop significantly with increasing the number of spines. In general, our stochastic thermodynamics approach provides a unifying framework for studying, from first principles, information encoding, and its energy cost during learning and memory in stochastic systems of interacting synapses.

Information encoded in volumes and areas of dendritic spines is nearly maximal across mammalian brains.

Many experiments suggest that long-term information associated with neuronal memory resides collectively in dendritic spines. However, spines can have a limited size due to metabolic and neuroanatomical constraints, which should effectively limit the amount of encoded information in excitatory synapses. This study investigates how much information can be stored in the population of sizes of dendritic spines, and whether it is optimal in any sense. It is shown here, using empirical data for several mammalian brains across different regions and physiological conditions, that dendritic spines nearly maximize entropy contained in their volumes and surface areas for a given mean size in cortical and hippocampal regions. Although both short- and heavy-tailed fitting distributions approach [Formula: see text] of maximal entropy in the majority of cases, the best maximization is obtained primarily for short-tailed gamma distribution. We find that most empirical ratios of standard deviation to mean for spine volumes and areas are in the range [Formula: see text], which is close to the theoretical optimal ratios coming from entropy maximization for gamma and lognormal distributions. On average, the highest entropy is contained in spine length ([Formula: see text] bits per spine), and the lowest in spine volume and area ([Formula: see text] bits), although the latter two are closer to optimality. In contrast, we find that entropy density (entropy per spine size) is always suboptimal. Our results suggest that spine sizes are almost as random as possible given the constraint on their size, and moreover the general principle of entropy maximization is applicable and potentially useful to information and memory storing in the population of cortical and hippocampal excitatory synapses, and to predicting their morphological properties.

Energetics of stochastic BCM type synaptic plasticity and storing of accurate information.

Excitatory synaptic signaling in cortical circuits is thought to be metabolically expensive. Two fundamental brain functions, learning and memory, are associated with long-term synaptic plasticity, but we know very little about energetics of these slow biophysical processes. This study investigates the energy requirement of information storing in plastic synapses for an extended version of BCM plasticity with a decay term, stochastic noise, and nonlinear dependence of neuron's firing rate on synaptic current (adaptation). It is shown that synaptic weights in this model exhibit bistability. In order to analyze the system analytically, it is reduced to a simple dynamic mean-field for a population averaged plastic synaptic current. Next, using the concepts of nonequilibrium thermodynamics, we derive the energy rate (entropy production rate) for plastic synapses and a corresponding Fisher information for coding presynaptic input. That energy, which is of chemical origin, is primarily used for battling fluctuations in the synaptic weights and presynaptic firing rates, and it increases steeply with synaptic weights, and more uniformly though nonlinearly with presynaptic firing. At the onset of synaptic bistability, Fisher information and memory lifetime both increase sharply, by a few orders of magnitude, but the plasticity energy rate changes only mildly. This implies that a huge gain in the precision of stored information does not have to cost large amounts of metabolic energy, which suggests that synaptic information is not directly limited by energy consumption. Interestingly, for very weak synaptic noise, such a limit on synaptic coding accuracy is imposed instead by a derivative of the plasticity energy rate with respect to the mean presynaptic firing, and this relationship has a general character that is independent of the plasticity type. An estimate for primate neocortex reveals that a relative metabolic cost of BCM type synaptic plasticity, as a fraction of neuronal cost related to fast synaptic transmission and spiking, can vary from negligible to substantial, depending on the synaptic noise level and presynaptic firing.

Metabolic constraints on synaptic learning and memory.

Dendritic spines, the carriers of long-term memory, occupy a small fraction of cortical space, and yet they are the major consumers of brain metabolic energy. What fraction of this energy goes for synaptic plasticity, correlated with learning and memory? It is estimated here based on neurophysiological and proteomic data for rat brain that, depending on the level of protein phosphorylation, the energy cost of synaptic plasticity constitutes a small fraction of the energy used for fast excitatory synaptic transmission, typically 4.0-11.2%. Next, this study analyzes a metabolic cost of new learning and its memory trace in relation to the cost of prior memories, using a class of cascade models of synaptic plasticity. It is argued that these models must contain bidirectional cyclic motifs, related to protein phosphorylation, to be compatible with basic thermodynamic principles. For most investigated parameters longer memories generally require proportionally more energy to store. The exceptions are the parameters controlling the speed of molecular transitions (e.g., ATP-driven phosphorylation rate), for which memory lifetime per invested energy can increase progressively for longer memories. Furthermore, in general, a memory trace decouples dynamically from a corresponding synaptic metabolic rate such that the energy expended on new learning and its memory trace constitutes in most cases only a small fraction of the baseline energy associated with prior memories. Taken together, these empirical and theoretical results suggest a metabolic efficiency of synaptically stored information.NEW & NOTEWORTHY Learning and memory involve a sequence of molecular events in dendritic spines called synaptic plasticity. These events are physical in nature and require energy, which has to be supplied by ATP molecules. However, our knowledge of the energetics of these processes is very poor. This study estimates the empirical energy cost of synaptic plasticity and considers theoretically a metabolic rate of learning and its memory trace in a class of cascade models of synaptic plasticity.

Optimal synaptic signaling connectome for locomotory behavior in Caenorhabditis elegans: Design minimizing energy cost.

The detailed knowledge of C. elegans connectome for 3 decades has not contributed dramatically to our understanding of worm's behavior. One of main reasons for this situation has been the lack of data on the type of synaptic signaling between particular neurons in the worm's connectome. The aim of this study was to determine synaptic polarities for each connection in a small pre-motor circuit controlling locomotion. Even in this compact network of just 7 neurons the space of all possible patterns of connection types (excitation vs. inhibition) is huge. To deal effectively with this combinatorial problem we devised a novel and relatively fast technique based on genetic algorithms and large-scale parallel computations, which we combined with detailed neurophysiological modeling of interneuron dynamics and compared the theory to the available behavioral data. As a result of these massive computations, we found that the optimal connectivity pattern that matches the best locomotory data is the one in which all interneuron connections are inhibitory, even those terminating on motor neurons. This finding is consistent with recent experimental data on cholinergic signaling in C. elegans, and it suggests that the system controlling locomotion is designed to save metabolic energy. Moreover, this result provides a solid basis for a more realistic modeling of neural control in these worms, and our novel powerful computational technique can in principle be applied (possibly with some modifications) to other small-scale functional circuits in C. elegans.

Cortical Composition Hierarchy Driven by Spine Proportion Economical Maximization or Wire Volume Minimization.

The structure and quantitative composition of the cerebral cortex are interrelated with its computational capacity. Empirical data analyzed here indicate a certain hierarchy in local cortical composition. Specifically, neural wire, i.e., axons and dendrites take each about 1/3 of cortical space, spines and glia/astrocytes occupy each about (1/3)(2), and capillaries around (1/3)(4). Moreover, data analysis across species reveals that these fractions are roughly brain size independent, which suggests that they could be in some sense optimal and thus important for brain function. Is there any principle that sets them in this invariant way? This study first builds a model of local circuit in which neural wire, spines, astrocytes, and capillaries are mutually coupled elements and are treated within a single mathematical framework. Next, various forms of wire minimization rule (wire length, surface area, volume, or conduction delays) are analyzed, of which, only minimization of wire volume provides realistic results that are very close to the empirical cortical fractions. As an alternative, a new principle called "spine economy maximization" is proposed and investigated, which is associated with maximization of spine proportion in the cortex per spine size that yields equally good but more robust results. Additionally, a combination of wire cost and spine economy notions is considered as a meta-principle, and it is found that this proposition gives only marginally better results than either pure wire volume minimization or pure spine economy maximization, but only if spine economy component dominates. However, such a combined meta-principle yields much better results than the constraints related solely to minimization of wire length, wire surface area, and conduction delays. Interestingly, the type of spine size distribution also plays a role, and better agreement with the data is achieved for distributions with long tails. In sum, these results suggest that for the efficiency of local circuits wire volume may be more primary variable than wire length or temporal delays, and moreover, the new spine economy principle may be important for brain evolutionary design in a broader context.

Effect of temperature and glia in brain size enlargement and origin of allometric body-brain size scaling in vertebrates.

BACKGROUND: Brain signaling requires energy. The cost of maintaining and supporting energetically demanding neurons is the key constraint on brain size. The dramatic increase in brain size among mammals and birds cannot be understood without solving this conundrum: larger brains, with more neurons, consume more energy. RESULTS: Here we examined the intrinsic relationships between metabolism, body-brain size ratios and neuronal densities of both endothermic and ectothermic animals. We formulated a general model to elucidate the key factors that correlate with brain enlargement, and the origin of allometric body-brain size scaling. This framework identified temperature as a critical factor in brain enlargement via temperature-regulated changes in metabolism. Our framework predicts that ectothermic animals living in tropical climates should have brain sizes that are several times larger than those of ectothermic animals living in cold climates. This prediction was confirmed by data from experiments in fish brains. Our framework also suggests that a rapid increase in the number of less energy-demanding glial cells may be another important factor contributing to the ten-fold increase in the brain sizes of endotherms compared with ectotherms. CONCLUSIONS: This study thus provides a quantitative theory that predicts the brain sizes of all the major types of animals and quantifies the contributions of temperature-dependent metabolism, body size and neuronal density.

Constancy and trade-offs in the neuroanatomical and metabolic design of the cerebral cortex.

Mammalian brains span about four orders of magnitude in cortical volume and have to operate in different environments that require diverse behavioral skills. Despite these geometric and behavioral diversities, the examination of cerebral cortex across species reveals that it contains a substantial number of conserved characteristics that are associated with neuroanatomy and metabolism, i.e., with neuronal connectivity and function. Some of these cortical constants or invariants have been known for a long time but not sufficiently appreciated, and others were only recently discovered. The focus of this review is to present the cortical invariants and discuss their role in the efficient information processing. Global conservation in neuroanatomy and metabolism, as well as their correlated regional and developmental variability suggest that these two parallel systems are mutually coupled. It is argued that energetic constraint on cortical organization can be strong if cerebral blood supplied is either below or above a certain level, and it is rather soft otherwise. Moreover, because maximization or minimization of parameters associated with cortical connectivity, function and cost often leads to conflicts in design, it is argued that the architecture of the cerebral cortex is a result of structural and functional compromises.

Synaptic polarity of the interneuron circuit controlling C. elegans locomotion.

Caenorhabditis elegans is the only animal for which a detailed neural connectivity diagram has been constructed. However, synaptic polarities in this diagram, and thus, circuit functions are largely unknown. Here, we deciphered the likely polarities of seven pre-motor neurons implicated in the control of worm's locomotion, using a combination of experimental and computational tools. We performed single and multiple laser ablations in the locomotor interneuron circuit and recorded times the worms spent in forward and backward locomotion. We constructed a theoretical model of the locomotor circuit and searched its all possible synaptic polarity combinations and sensory input patterns in order to find the best match to the timing data. The optimal solution is when either all or most of the interneurons are inhibitory and forward interneurons receive the strongest input, which suggests that inhibition governs the dynamics of the locomotor interneuron circuit. From the five pre-motor interneurons, only AVB and AVD are equally likely to be excitatory, i.e., they have probably similar number of inhibitory and excitatory connections to distant targets. The method used here has a general character and thus can be also applied to other neural systems consisting of small functional networks.

Approximate invariance of metabolic energy per synapse during development in mammalian brains.

During mammalian development the cerebral metabolic rate correlates qualitatively with synaptogenesis, and both often exhibit bimodal temporal profiles. Despite these non-monotonic dependencies, it is found based on empirical data for different mammals that regional metabolic rate per synapse is approximately conserved from birth to adulthood for a given species (with a slight deviation from this constancy for human visual and temporal cortices during adolescence). A typical synapse uses about (7±2)×10(3) glucose molecules per second in primate cerebral cortex, and about five times of that amount in cat and rat visual cortices. A theoretical model for brain metabolic expenditure is used to estimate synaptic signaling and neural spiking activity during development. It is found that synaptic efficacy is generally inversely correlated with average firing rate, and, additionally, synapses consume a bulk of metabolic energy, roughly 50-90% during most of the developmental process (except human temporal cortex < 50%). Overall, these results suggest a tight regulation of brain electrical and chemical activities during the formation and consolidation of neural connections. This presumably reflects strong energetic constraints on brain development.

Scaling of brain metabolism and blood flow in relation to capillary and neural scaling.

Brain is one of the most energy demanding organs in mammals, and its total metabolic rate scales with brain volume raised to a power of around 5/6. This value is significantly higher than the more common exponent 3/4 relating whole body resting metabolism with body mass and several other physiological variables in animals and plants. This article investigates the reasons for brain allometric distinction on a level of its microvessels. Based on collected empirical data it is found that regional cerebral blood flow CBF across gray matter scales with cortical volume V as CBF ~ V(-1/6), brain capillary diameter increases as V(1/12), and density of capillary length decreases as V(-1/6). It is predicted that velocity of capillary blood is almost invariant (~V(ε)), capillary transit time scales as V(1/6), capillary length increases as V(1/6+ε), and capillary number as V(2/3-ε), where ε is typically a small correction for medium and large brains, due to blood viscosity dependence on capillary radius. It is shown that the amount of capillary length and blood flow per cortical neuron are essentially conserved across mammals. These results indicate that geometry and dynamics of global neuro-vascular coupling have a proportionate character. Moreover, cerebral metabolic, hemodynamic, and microvascular variables scale with allometric exponents that are simple multiples of 1/6, rather than 1/4, which suggests that brain metabolism is more similar to the metabolism of aerobic than resting body. Relation of these findings to brain functional imaging studies involving the link between cerebral metabolism and blood flow is also discussed.

Thermodynamic constraints on neural dimensions, firing rates, brain temperature and size.

There have been suggestions that heat caused by cerebral metabolic activity may constrain mammalian brain evolution, architecture, and function. This article investigates physical limits on brain wiring and corresponding changes in brain temperature that are imposed by thermodynamics of heat balance determined mainly by Na(+)/K(+)-ATPase, cerebral blood flow, and heat conduction. It is found that even moderate firing rates cause significant intracellular Na(+) build-up, and the ATP consumption rate associated with pumping out these ions grows nonlinearly with frequency. Surprisingly, the power dissipated by the Na(+)/K(+) pump depends biphasically on frequency, which can lead to the biphasic dependence of brain temperature on frequency as well. Both the total power of sodium pumps and brain temperature diverge for very small fiber diameters, indicating that too thin fibers are not beneficial for thermal balance. For very small brains blood flow is not a sufficient cooling mechanism deep in the brain. The theoretical lower bound on fiber diameter above which brain temperature is in the operational regime is strongly frequency dependent but finite due to synaptic depression. For normal neurophysiological conditions this bound is at least an order of magnitude smaller than average values of empirical fiber diameters, suggesting that neuroanatomy of the mammalian brains operates in the thermodynamically safe regime. Analytical formulas presented can be used to estimate average firing rates in mammals, and relate their changes to changes in brain temperature, which can have important practical applications. In general, activity in larger brains is found to be slower than in smaller brains.

Systems level circuit model of C. elegans undulatory locomotion: mathematical modeling and molecular genetics.

To establish the relationship between locomotory behavior and dynamics of neural circuits in the nematode C. elegans we combined molecular and theoretical approaches. In particular, we quantitatively analyzed the motion of C. elegans with defective synaptic GABA and acetylcholine transmission, defective muscle calcium signaling, and defective muscles and cuticle structures, and compared the data with our systems level circuit model. The major experimental findings are: (1) anterior-to-posterior gradients of body bending flex for almost all strains both for forward and backward motion, and for neuronal mutants, also analogous weak gradients of undulatory frequency, (2) existence of some form of neuromuscular (stretch receptor) feedback, (3) invariance of neuromuscular wavelength, (4) biphasic dependence of frequency on synaptic signaling, and (5) decrease of frequency with increase of the muscle time constant. Based on (1) we hypothesize that the Central Pattern Generator (CPG) is located in the head both for forward and backward motion. Points (1) and (2) are the starting assumptions for our theoretical model, whose dynamical patterns are qualitatively insensitive to the details of the CPG design if stretch receptor feedback is sufficiently strong and slow. The model reveals that stretch receptor coupling in the body wall is critical for generation of the neuromuscular wave. Our model agrees with our behavioral data (3), (4), and (5), and with other pertinent published data, e.g., that frequency is an increasing function of muscle gap-junction coupling.

Global and regional brain metabolic scaling and its functional consequences.

BACKGROUND: Information processing in the brain requires large amounts of metabolic energy, the spatial distribution of which is highly heterogeneous, reflecting the complex activity patterns in the mammalian brain. RESULTS: In this study, it was found, based on empirical data, that despite this heterogeneity, the volume-specific cerebral glucose metabolic rate of many different brain structures scales with brain volume with almost the same exponent: around -0.15. The exception is white matter, the metabolism of which seems to scale with a standard specific exponent of -1/4. The scaling exponents for the total oxygen and glucose consumptions in the brain in relation to its volume are identical, at 0.86 +/- 0.03, which is significantly larger than the exponents 3/4 and 2/3 that have been suggested for whole body basal metabolism on body mass. CONCLUSION: These findings show explicitly that in mammals: (i) volume-specific scaling exponents of the cerebral energy expenditure in different brain parts are approximately constant (except brain stem structures), and (ii) the total cerebral metabolic exponent against brain volume is greater than the much-cited Kleiber's 3/4 exponent. The neurophysiological factors that might account for the regional uniformity of the exponents and for the excessive scaling of the total brain metabolism are discussed, along with the relationship between brain metabolic scaling and computation.

Conservation rules, their breakdown, and optimality in Caenorhabditis sinusoidal locomotion.

Undulatory locomotion is common to nematodes as well as to limbless vertebrates, but its control is not understood in spite of the identification of hundred of genes involved in Caenorhabditis elegans locomotion. To reveal the mechanisms of nematode undulatory locomotion, we quantitatively analysed the movement of C. elegans with genetic perturbations to neurons, muscles, and skeleton (cuticle). We also compared locomotion of different Caenorhabditis species. We constructed a theoretical model that combines mechanics and biophysics, and that is constrained by the observations of propulsion and muscular velocities, as well as wavelength and amplitude of undulations. We find that normalized wavelength is a conserved quantity among wild-type C. elegans individuals, across mutants, and across different species. The velocity of forward propulsion scales linearly with the velocity of the muscular wave and the corresponding slope is also a conserved quantity and almost optimal; the exceptions are in some mutants affecting cuticle structure. In theoretical terms, the optimality of the slope is equivalent to the exact balance between muscular and visco-elastic body reaction bending moments. We find that the amplitude and frequency of undulations are inversely correlated and provide a theoretical explanation for this fact. These experimental results are valid both for young adults and for all larval stages of wild-type C. elegans. In particular, during development, the amplitude scales linearly with the wavelength, consistent with our theory. We also investigated the influence of substrate firmness on motion parameters, and found that it does not affect the above invariants. In general, our biomechanical model can explain the observed robustness of the mechanisms controlling nematode undulatory locomotion.

Model of the early development of thalamo-cortical connections and area patterning via signaling molecules.

The mammalian cortex is divided into architectonic and functionally distinct areas. There is growing experimental evidence that their emergence and development is controlled by both epigenetic and genetic factors. The latter were recently implicated as dominating the early cortical area specification. In this paper, we present a theoretical model that explicitly considers the genetic factors and that is able to explain several sets of experiments on cortical area regulation involving transcription factors Emx2 and Pax6, and fibroblast growth factor FGF8. The model consists of the dynamics of thalamo-cortical connections modulated by signaling molecules that are regulated genetically, and by axonal competition for neocortical space. The model can make predictions and provides a basic mathematical framework for the early development of the thalamo-cortical connections and area patterning that can be further refined as more experimental facts become known.

How does connectivity between cortical areas depend on brain size? Implications for efficient computation.

A formula for an average connectivity between cortical areas in mammals is derived. Based on comparative neuroanatomical data, it is found, surprisingly, that this connectivity is either only weakly dependent or independent of brain size. It is discussed how this formula can be used to estimate the average length of axons in white matter. Other allometric relations, such as cortical patches and area sizes vs. brain size, are also provided. Finally, some functional implications, with an emphasis on efficient cortical computation, are discussed as well.

Synchrony arising from a balanced synaptic plasticity in a network of heterogeneous neural oscillators.

We investigate the dynamics of a recurrent network of coupled heterogeneous neural oscillators with experimentally observed spike-timing-dependent synaptic plasticity. We show both theoretically and by computer simulations that, in a regime of a balance between synaptic potentiation and depression, the network of such oscillators converges to a stable synchronous state. The stability of this state is fostered by flexible synaptic weights which adjust themselves based on the relative timing of firing of pre- and postsynaptic oscillators.