Structured States of Disordered Proteins from Genomic Sequences.

Protein flexibility ranges from simple hinge movements to functional disorder. Around half of all human proteins contain apparently disordered regions with little 3D or functional information, and many of these proteins are associated with disease. Building on the evolutionary couplings approach previously successful in predicting 3D states of ordered proteins and RNA, we developed a method to predict the potential for ordered states for all apparently disordered proteins with sufficiently rich evolutionary information. The approach is highly accurate (79%) for residue interactions as tested in more than 60 known disordered regions captured in a bound or specific condition. Assessing the potential for structure of more than 1,000 apparently disordered regions of human proteins reveals a continuum of structural order with at least 50% with clear propensity for three- or two-dimensional states. Co-evolutionary constraints reveal hitherto unseen structures of functional importance in apparently disordered proteins.

Maximum entropy determination of mammalian proteome dynamics.

Full understanding of proteostasis and energy utilization in cells will require knowledge of the fraction of cell proteins being degraded with different half-lives and their rates of synthesis. We therefore developed a method to determine such information that combines mathematical analysis of protein degradation kinetics obtained in pulse-chase experiments with Bayesian data fitting using the maximum entropy principle. This approach will enable rapid analyses of whole-cell protein dynamics in different cell types, physiological states, and neurodegenerative disease. Using it, we obtained surprising insights about protein stabilities in cultured cells normally and upon activation of proteolysis by mTOR inhibition and increasing cAMP or cGMP. It revealed that >90% of protein content in dividing mammalian cell lines is long-lived, with half-lives of 24 to 200 h, and therefore comprises much of the proteins in daughter cells. The well-studied short-lived proteins (half-lives < 10 h) together comprise <2% of cell protein mass, but surprisingly account for 10 to 20% of measurable newly synthesized protein mass. Evolution thus appears to have minimized intracellular proteolysis except to rapidly eliminate misfolded and regulatory proteins.

Physical models of bacterial chromosomes.

The interplay between bacterial chromosome organization and functions such as transcription and replication can be studied in increasing detail using novel experimental techniques. Interpreting the resulting quantitative data, however, can be theoretically challenging. In this minireview, we discuss how connecting experimental observations to biophysical theory and modeling can give rise to new insights on bacterial chromosome organization. We consider three flavors of models of increasing complexity: simple polymer models that explore how physical constraints, such as confinement or plectoneme branching, can affect bacterial chromosome organization; bottom-up mechanistic models that connect these constraints to their underlying causes, for instance, chromosome compaction to macromolecular crowding, or supercoiling to transcription; and finally, data-driven methods for inferring interpretable and quantitative models directly from complex experimental data. Using recent examples, we discuss how biophysical models can both deepen our understanding of how bacterial chromosomes are structured and give rise to novel predictions about bacterial chromosome organization.

Enhancing computational enzyme design by a maximum entropy strategy.

Although computational enzyme design is of great importance, the advances utilizing physics-based approaches have been slow, and further progress is urgently needed. One promising direction is using machine learning, but such strategies have not been established as effective tools for predicting the catalytic power of enzymes. Here, we show that the statistical energy inferred from homologous sequences with the maximum entropy (MaxEnt) principle significantly correlates with enzyme catalysis and stability at the active site region and the more distant region, respectively. This finding decodes enzyme architecture and offers a connection between enzyme evolution and the physical chemistry of enzyme catalysis, and it deepens our understanding of the stability-activity trade-off hypothesis for enzymes. Overall, the strong correlations found here provide a powerful way of guiding enzyme design.

Optimizing the human learnability of abstract network representations.

Precisely how humans process relational patterns of information in knowledge, language, music, and society is not well understood. Prior work in the field of statistical learning has demonstrated that humans process such information by building internal models of the underlying network structure. However, these mental maps are often inaccurate due to limitations in human information processing. The existence of such limitations raises clear questions: Given a target network that one wishes for a human to learn, what network should one present to the human? Should one simply present the target network as-is, or should one emphasize certain parts of the network to proactively mitigate expected errors in learning? To investigate these questions, we study the optimization of network learnability in a computational model of human learning. Evaluating an array of synthetic and real-world networks, we find that learnability is enhanced by reinforcing connections within modules or clusters. In contrast, when networks contain significant core-periphery structure, we find that learnability is best optimized by reinforcing peripheral edges between low-degree nodes. Overall, our findings suggest that the accuracy of human network learning can be systematically enhanced by targeted emphasis and de-emphasis of prescribed sectors of information.

Digital computing through randomness and order in neural networks.

We propose that coding and decoding in the brain are achieved through digital computation using three principles: relative ordinal coding of inputs, random connections between neurons, and belief voting. Due to randomization and despite the coarseness of the relative codes, we show that these principles are sufficient for coding and decoding sequences with error-free reconstruction. In particular, the number of neurons needed grows linearly with the size of the input repertoire growing exponentially. We illustrate our model by reconstructing sequences with repertoires on the order of a billion items. From this, we derive the Shannon equations for the capacity limit to learn and transfer information in the neural population, which is then generalized to any type of neural network. Following the maximum entropy principle of efficient coding, we show that random connections serve to decorrelate redundant information in incoming signals, creating more compact codes for neurons and therefore, conveying a larger amount of information. Henceforth, despite the unreliability of the relative codes, few neurons become necessary to discriminate the original signal without error. Finally, we discuss the significance of this digital computation model regarding neurobiological findings in the brain and more generally with artificial intelligence algorithms, with a view toward a neural information theory and the design of digital neural networks.

The Structure of Hippocampal CA1 Interactions Optimizes Spatial Coding across Experience.

Although much is known about how single neurons in the hippocampus represent an animal's position, how circuit interactions contribute to spatial coding is less well understood. Using a novel statistical estimator and theoretical modeling, both developed in the framework of maximum entropy models, we reveal highly structured CA1 cell-cell interactions in male rats during open field exploration. The statistics of these interactions depend on whether the animal is in a familiar or novel environment. In both conditions the circuit interactions optimize the encoding of spatial information, but for regimes that differ in the informativeness of their spatial inputs. This structure facilitates linear decodability, making the information easy to read out by downstream circuits. Overall, our findings suggest that the efficient coding hypothesis is not only applicable to individual neuron properties in the sensory periphery, but also to neural interactions in the central brain.SIGNIFICANCE STATEMENT Local circuit interactions play a key role in neural computation and are dynamically shaped by experience. However, measuring and assessing their effects during behavior remains a challenge. Here, we combine techniques from statistical physics and machine learning to develop new tools for determining the effects of local network interactions on neural population activity. This approach reveals highly structured local interactions between hippocampal neurons, which make the neural code more precise and easier to read out by downstream circuits, across different levels of experience. More generally, the novel combination of theory and data analysis in the framework of maximum entropy models enables traditional neural coding questions to be asked in naturalistic settings.

Reliability of energy landscape analysis of resting-state functional MRI data.

Energy landscape analysis is a data-driven method to analyse multidimensional time series, including functional magnetic resonance imaging (fMRI) data. It has been shown to be a useful characterization of fMRI data in health and disease. It fits an Ising model to the data and captures the dynamics of the data as movement of a noisy ball constrained on the energy landscape derived from the estimated Ising model. In the present study, we examine test-retest reliability of the energy landscape analysis. To this end, we construct a permutation test that assesses whether or not indices characterizing the energy landscape are more consistent across different sets of scanning sessions from the same participant (i.e. within-participant reliability) than across different sets of sessions from different participants (i.e. between-participant reliability). We show that the energy landscape analysis has significantly higher within-participant than between-participant test-retest reliability with respect to four commonly used indices. We also show that a variational Bayesian method, which enables us to estimate energy landscapes tailored to each participant, displays comparable test-retest reliability to that using the conventional likelihood maximization method. The proposed methodology paves the way to perform individual-level energy landscape analysis for given data sets with a statistically controlled reliability.

Valence and interactions in judicial voting.

The collective statistics of voting on judicial courts present hints about their inner workings. Many approaches for studying these statistics, however, assume that judges' decisions are conditionally independent: a judge reaches a decision based on the case at hand and his or her personal views. In reality, judges interact. We develop a minimal model that accounts for judge bias, depending on the context of the case, and peer interaction. We apply the model to voting data from the US Supreme Court. We find strong evidence that interaction is an important factor across natural courts from 1946 to 2021. We also find that, after accounting for interaction, the recovered biases differ from highly cited ideological scores. Our method exemplifies how physics and complexity-inspired modelling can drive the development of theoretical models and improved measures for political voting. This article is part of the theme issue 'A complexity science approach to law and governance'.

Update of the geographic range of Humboldt's squirrel monkeys (Saimiri cassiquiarensis cassiquiarensis Lesson, 1840) using a model-based approach.

Saimiri cassiquiarensis cassiquiarensis (Cebidae) is a primate subspecies with a wide distribution in the Amazonian region of Brazil, Colombia, and Venezuela. However, the boundaries of its geographic range remain poorly defined. This study presents new occurrence localities for this subspecies and updates its distribution using a compiled data set of 140 occurrence records based on literature, specimens vouchered in scientific collections, and new field data to produce model-based range maps. After cleaning our data set, we updated the subspecies' extent of occurrence, which was used in model calibration. We then modeled the subspecies' range using a maximum entropy algorithm (MaxEnt). The final model was adjusted using a fixed threshold, and we revised this polygon based on known geographic barriers and parapatric congeneric ranges. Our findings indicate that this subspecies is strongly associated with lowland areas, with consistently high daily temperatures. We propose modifications to all range boundaries and estimate that 3% of the area of occupancy (AOO, as defined by IUCN) has already been lost due to deforestation, resulting in a current range of 224,469 km2. We also found that 54% of their AOO is currently covered by protected areas (PAs). Based on these results, we consider that this subspecies is currently properly classified as Least Concern, because it occupies an extensive range, which is relatively well covered by PAs, and is currently experiencing low rates of deforestation.

Saimiri cassiquiarensis cassiquiarensis (Cebidae) é uma subespécie de primata com ampla distribuição na região amazônica do Brasil, Colômbia e Venezuela. No entanto, os limites de sua distribuição geográfica permanecem mal definidos. Este estudo apresenta novas localidades de ocorrência para essa subespécie e atualiza sua distribuição usando 140 registros de ocorrência compilados com base na literatura, espécimes depositados em coleções científicas e novos registros de campo para produzir mapas de distribuição baseados em modelos. Após a limpeza do nosso banco de dados, atualizamos a extensão de ocorrência da subespécie, que foi usada na calibração do modelo. Em seguida, modelamos a área de distribuição da subespécie usando um algoritmo de entropia máxima (MaxEnt). O modelo final foi ajustado usando um limiar fixo e revisamos esse polígono com base em barreiras geográficas conhecidas e na distribuição de congêneres parapátricas. Nosso modelo sugere que a espécie é fortemente associada a áreas planas, com temperaturas diárias consistentemente altas. Propomos modificações em todos os limites da área de distribuição e estimamos que 3% da área de ocupação (AOO, conforme definida pela IUCN) da subespécie já foi perdida devido ao desmatamento, resultando em uma área de distribuição atual de 224,469 km2. Também estimamos que 54% de sua AOO encontra­se atualmente coberta por áreas protegidas. Com base nesses resultados, consideramos que a subespécie está apropriadamente classificada como Pouco Preocupante, pois ocupa uma área extensa, que é relativamente bem coberta por áreas protegidas e atualmente apresenta baixas taxas de desmatamento.

Field-theoretic density estimation for biological sequence space with applications to 5' splice site diversity and aneuploidy in cancer.

Density estimation in sequence space is a fundamental problem in machine learning that is also of great importance in computational biology. Due to the discrete nature and large dimensionality of sequence space, how best to estimate such probability distributions from a sample of observed sequences remains unclear. One common strategy for addressing this problem is to estimate the probability distribution using maximum entropy (i.e., calculating point estimates for some set of correlations based on the observed sequences and predicting the probability distribution that is as uniform as possible while still matching these point estimates). Building on recent advances in Bayesian field-theoretic density estimation, we present a generalization of this maximum entropy approach that provides greater expressivity in regions of sequence space where data are plentiful while still maintaining a conservative maximum entropy character in regions of sequence space where data are sparse or absent. In particular, we define a family of priors for probability distributions over sequence space with a single hyperparameter that controls the expected magnitude of higher-order correlations. This family of priors then results in a corresponding one-dimensional family of maximum a posteriori estimates that interpolate smoothly between the maximum entropy estimate and the observed sample frequencies. To demonstrate the power of this method, we use it to explore the high-dimensional geometry of the distribution of 5' splice sites found in the human genome and to understand patterns of chromosomal abnormalities across human cancers.

Detecting Spatial Patterns of Peatland Greenhouse Gas Sinks and Sources with Geospatial Environmental and Remote Sensing Data.

Peatlands play a key role in the circulation of the main greenhouse gases (GHG) - methane (CH4), carbon dioxide (CO2), and nitrous oxide (N2O). Therefore, detecting the spatial pattern of GHG sinks and sources in peatlands is pivotal for guiding effective climate change mitigation in the land use sector. While geospatial environmental data, which provide detailed spatial information on ecosystems and land use, offer valuable insights into GHG sinks and sources, the potential of directly using remote sensing data from satellites remains largely unexplored. We predicted the spatial distribution of three major GHGs (CH4, CO2, and N2O) sinks and sources across Finland. Utilizing 143 field measurements, we compared the predictive capacity of three different data sets with MaxEnt machine-learning modeling: (1) geospatial environmental data including climate, topography and habitat variables, (2) remote sensing data (Sentinel-1 and Sentinel-2), and (3) a combination of both. The combined dataset yielded the highest accuracy with an average test area under the receiver operating characteristic curve (AUC) of 0.845 and AUC stability of 0.928. A slightly lower accuracy was achieved using only geospatial environmental data (test AUC 0.810, stability AUC 0.924). In contrast, using only remote sensing data resulted in reduced predictive accuracy (test AUC 0.763, stability AUC 0.927). Our results suggest that (1) reliable estimates of GHG sinks and sources cannot be produced with remote sensing data only and (2) integrating multiple data sources is recommended to achieve accurate and realistic predictions of GHG spatial patterns.

Why Does Cross-Sectional Analyst Coverage Incorporate Market-Wide Information?

This paper shows that the empirical distribution of cross-sectional analyst coverage in China's stock markets follows an exponential law in a given month from 2011 to 2020. The findings hold in both the emerging (Shanghai) and the developed market (Hong Kong). Moreover, the unique distribution parameter (i.e., mean) is directly related to the amount of market-wide information. Average analyst coverage exhibits a significant negative predictive power for stock-market uncertainty, highlighting the role of security analysts in diminishing the total uncertainty. The exponential law can be derived from the maximum entropy principle (MEP). When analysts, who are constrained by average ability in generating information (i.e., the first-order moment), strive to maximize the amount of market-wide information, this objective yields the exponential distribution. Contrary to the conventional wisdom that security analysts specialize in the generation of firm-specific information, empirical findings suggest that analysts primarily produce market-wide information for 25 countries. Nevertheless, it remains unclear why cross-sectional analyst coverage reflects market-wide information, this paper provides an entropy-based explanation.

Maximum Geometric Quantum Entropy.

Any given density matrix can be represented as an infinite number of ensembles of pure states. This leads to the natural question of how to uniquely select one out of the many, apparently equally-suitable, possibilities. Following Jaynes' information-theoretic perspective, this can be framed as an inference problem. We propose the Maximum Geometric Quantum Entropy Principle to exploit the notions of Quantum Information Dimension and Geometric Quantum Entropy. These allow us to quantify the entropy of fully arbitrary ensembles and select the one that maximizes it. After formulating the principle mathematically, we give the analytical solution to the maximization problem in a number of cases and discuss the physical mechanism behind the emergence of such maximum entropy ensembles.

Towards Multi-Objective Object Push-Grasp Policy Based on Maximum Entropy Deep Reinforcement Learning under Sparse Rewards.

In unstructured environments, robots need to deal with a wide variety of objects with diverse shapes, and often, the instances of these objects are unknown. Traditional methods rely on training with large-scale labeled data, but in environments with continuous and high-dimensional state spaces, the data become sparse, leading to weak generalization ability of the trained models when transferred to real-world applications. To address this challenge, we present an innovative maximum entropy Deep Q-Network (ME-DQN), which leverages an attention mechanism. The framework solves complex and sparse reward tasks through probabilistic reasoning while eliminating the trouble of adjusting hyper-parameters. This approach aims to merge the robust feature extraction capabilities of Fully Convolutional Networks (FCNs) with the efficient feature selection of the attention mechanism across diverse task scenarios. By integrating an advantage function with the reasoning and decision-making of deep reinforcement learning, ME-DQN propels the frontier of robotic grasping and expands the boundaries of intelligent perception and grasping decision-making in unstructured environments. Our simulations demonstrate a remarkable grasping success rate of 91.6%, while maintaining excellent generalization performance in the real world.

Small and Simple Systems That Favor the Arrow of Time.

The 2nd law of thermodynamics yields an irreversible increase in entropy until thermal equilibrium is achieved. This irreversible increase is often assumed to require large and complex systems to emerge from the reversible microscopic laws of physics. We test this assumption using simulations and theory of a 1D ring of N Ising spins coupled to an explicit heat bath of N Einstein oscillators. The simplicity of this system allows the exact entropy to be calculated for the spins and the heat bath for any N, with dynamics that is readily altered from reversible to irreversible. We find thermal-equilibrium behavior in the thermodynamic limit, and in systems as small as N=2, but both results require microscopic dynamics that is intrinsically irreversible.

Maximum Entropy Criterion for Moment Indeterminacy of Probability Densities.

We deal with absolutely continuous probability distributions with finite all-positive integer-order moments. It is well known that any such distribution is either uniquely determined by its moments (M-determinate), or it is non-unique (M-indeterminate). In this paper, we follow the maximum entropy approach and establish a new criterion for the M-indeterminacy of distributions on the positive half-line (Stieltjes case). Useful corollaries are derived for M-indeterminate distributions on the whole real line (Hamburger case). We show how the maximum entropy is related to the symmetry property and the M-indeterminacy.

Statistical models of complex brain networks: a maximum entropy approach.

The brain is a highly complex system. Most of such complexity stems from the intermingled connections between its parts, which give rise to rich dynamics and to the emergence of high-level cognitive functions. Disentangling the underlying network structure is crucial to understand the brain functioning under both healthy and pathological conditions. Yet, analyzing brain networks is challenging, in part because their structure represents only one possible realization of a generative stochastic process which is in general unknown. Having a formal way to cope with such intrinsic variability is therefore central for the characterization of brain network properties. Addressing this issue entails the development of appropriate tools mostly adapted from network science and statistics. Here, we focus on a particular class of maximum entropy models for networks, i.e. exponential random graph models, as a parsimonious approach to identify the local connection mechanisms behind observed global network structure. Efforts are reviewed on the quest for basic organizational properties of human brain networks, as well as on the identification of predictive biomarkers of neurological diseases such as stroke. We conclude with a discussion on how emerging results and tools from statistical graph modeling, associated with forthcoming improvements in experimental data acquisition, could lead to a finer probabilistic description of complex systems in network neuroscience.

Hierarchical Bayesian modeling of the relationship between task-related hemodynamic responses and cortical excitability.

Investigating the relationship between task-related hemodynamic responses and cortical excitability is challenging because it requires simultaneous measurement of hemodynamic responses while applying noninvasive brain stimulation. Moreover, cortical excitability and task-related hemodynamic responses are both associated with inter-/intra-subject variability. To reliably assess such a relationship, we applied hierarchical Bayesian modeling. This study involved 16 healthy subjects who underwent simultaneous Paired Associative Stimulation (PAS10, PAS25, Sham) while monitoring brain activity using functional Near-Infrared Spectroscopy (fNIRS), targeting the primary motor cortex (M1). Cortical excitability was measured by Motor Evoked Potentials (MEPs), and the motor task-related hemodynamic responses were measured using fNIRS 3D reconstructions. We constructed three models to investigate: (1) PAS effects on the M1 excitability, (2) PAS effects on fNIRS hemodynamic responses to a finger tapping task, and (3) the correlation between PAS effects on M1 excitability and PAS effects on task-related hemodynamic responses. Significant increase in cortical excitability was found following PAS25, whereas a small reduction of the cortical excitability was shown after PAS10 and a subtle increase occurred after sham. Both HbO and HbR absolute amplitudes increased after PAS25 and decreased after PAS10. The probability of the positive correlation between modulation of cortical excitability and hemodynamic activity was 0.77 for HbO and 0.79 for HbR. We demonstrated that PAS stimulation modulates task-related cortical hemodynamic responses in addition to M1 excitability. Moreover, the positive correlation between PAS modulations of excitability and hemodynamics brought insight into understanding the fundamental properties of cortical function and cortical excitability.

Correcting prevalence estimation for biased sampling with testing errors.

Sampling for prevalence estimation of infection is subject to bias by both oversampling of symptomatic individuals and error-prone tests. This results in naïve estimators of prevalence (ie, proportion of observed infected individuals in the sample) that can be very far from the true proportion of infected. In this work, we present a method of prevalence estimation that reduces both the effect of bias due to testing errors and oversampling of symptomatic individuals, eliminating it altogether in some scenarios. Moreover, this procedure considers stratified errors in which tests have different error rate profiles for symptomatic and asymptomatic individuals. This results in easily implementable algorithms, for which code is provided, that produce better prevalence estimates than other methods (in terms of reducing and/or removing bias), as demonstrated by formal results, simulations, and on COVID-19 data from the Israeli Ministry of Health.