Equivalence of information production and generalised entropies in complex processes.

Complex systems with strong correlations and fat-tailed distribution functions have been argued to be incompatible with the Boltzmann-Gibbs entropy framework and alternatives, so-called generalised entropies, were proposed and studied. Here we show, that this perceived incompatibility is actually a misconception. For a broad class of processes, Boltzmann entropy -the log multiplicity- remains the valid entropy concept. However, for non-i.i.d. processes, Boltzmann entropy is not of Shannon form, -k∑ipi log pi, but takes the shape of generalised entropies. We derive this result for all processes that can be asymptotically mapped to adjoint representations reversibly where processes are i.i.d. In these representations the information production is given by the Shannon entropy. Over the original sampling space this yields functionals identical to generalised entropies. The problem of constructing adequate context-sensitive entropy functionals therefore can be translated into the much simpler problem of finding adjoint representations. The method provides a comprehensive framework for a statistical physics of strongly correlated systems and complex processes.

The Typical Set and Entropy in Stochastic Systems with Arbitrary Phase Space Growth.

The existence of the typical set is key for data compression strategies and for the emergence of robust statistical observables in macroscopic physical systems. Standard approaches derive its existence from a restricted set of dynamical constraints. However, given its central role underlying the emergence of stable, almost deterministic statistical patterns, a question arises whether typical sets exist in much more general scenarios. We demonstrate here that the typical set can be defined and characterized from general forms of entropy for a much wider class of stochastic processes than was previously thought. This includes processes showing arbitrary path dependence, long range correlations or dynamic sampling spaces, suggesting that typicality is a generic property of stochastic processes, regardless of their complexity. We argue that the potential emergence of robust properties in complex stochastic systems provided by the existence of typical sets has special relevance to biological systems.

Homophily-Based Social Group Formation in a Spin Glass Self-Assembly Framework.

Homophily, the tendency of humans to attract each other when sharing similar features, traits, or opinions, has been identified as one of the main driving forces behind the formation of structured societies. Here we ask to what extent homophily can explain the formation of social groups, particularly their size distribution. We propose a spin-glass-inspired framework of self-assembly, where opinions are represented as multidimensional spins that dynamically self-assemble into groups; individuals within a group tend to share similar opinions (intragroup homophily), and opinions between individuals belonging to different groups tend to be different (intergroup heterophily). We compute the associated nontrivial phase diagram by solving a self-consistency equation for "magnetization" (combined average opinion). Below a critical temperature, there exist two stable phases: one ordered with nonzero magnetization and large clusters, the other disordered with zero magnetization and no clusters. The system exhibits a first-order transition to the disordered phase. We analytically derive the group-size distribution that successfully matches empirical group-size distributions from online communities.

Antagonistic Functions of Androgen Receptor and NF-κB in Prostate Cancer-Experimental and Computational Analyses.

Prostate cancer is very frequent and is, in many countries, the third-leading cause of cancer related death in men. While early diagnosis and treatment by surgical removal is often curative, metastasizing prostate cancer has a very bad prognosis. Based on the androgen-dependence of prostate epithelial cells, the standard treatment is blockade of the androgen receptor (AR). However, nearly all patients suffer from a tumor relapse as the metastasizing cells become AR-independent. In our study we show a counter-regulatory link between AR and NF-κB both in human cells and in mouse models of prostate cancer, implying that inhibition of AR signaling results in induction of NF-κB-dependent inflammatory pathways, which may even foster the survival of metastasizing cells. This could be shown by reporter gene assays, DNA-binding measurements, and immune-fluorescence microscopy, and furthermore by a whole set of computational methods using a variety of datasets. Interestingly, loss of PTEN, a frequent genetic alteration in prostate cancer, also causes an upregulation of NF-κB and inflammatory activity. Finally, we present a mathematical model of a dynamic network between AR, NF-κB/IκB, PI3K/PTEN, and the oncogene c-Myc, which indicates that AR blockade may upregulate c-Myc together with NF-κB, and that combined anti-AR/anti-NF-κB and anti-PI3K treatment might be beneficial.

Empirical social triad statistics can be explained with dyadic homophylic interactions.

The remarkable robustness of many social systems has been associated with a peculiar triangular structure in the underlying social networks. Triples of people that have three positive relations (e.g., friendship) between each other are strongly overrepresented. Triples with two negative relations (e.g., enmity) and one positive relation are also overrepresented, and triples with one or three negative relations are drastically suppressed. For almost a century, the mechanism behind these very specific ("balanced") triad statistics remained elusive. Here, we propose a simple realistic adaptive network model, where agents tend to minimize social tension that arises from dyadic interactions. Both opinions of agents and their signed links (positive or negative relations) are updated in the dynamics. The key aspect of the model resides in the fact that agents only need information about their local neighbors in the network and do not require (often unrealistic) higher-order network information for their relation and opinion updates. We demonstrate the quality of the model on detailed temporal relation data of a society of thousands of players of a massive multiplayer online game where we can observe triangle formation directly. It not only successfully predicts the distribution of triangle types but also explains empirical group size distributions, which are essential for social cohesion. We discuss the details of the phase diagrams behind the model and their parameter dependence, and we comment on to what extent the results might apply universally in societies.

Balance and fragmentation in societies with homophily and social balance.

Recent attempts to understand the origin of social fragmentation on the basis of spin models include terms accounting for two social phenomena: homophily-the tendency for people with similar opinions to establish positive relations-and social balance-the tendency for people to establish balanced triadic relations. Spins represent attribute vectors that encode G different opinions of individuals whose social interactions can be positive or negative. Here we present a co-evolutionary Hamiltonian model of societies where people minimise their individual social stresses. We show that societies always reach stationary, balanced, and fragmented states, if-in addition to homophily-individuals take into account a significant fraction, q, of their triadic relations. Above a critical value, [Formula: see text], balanced and fragmented states exist for any number of opinions.

Thermodynamics of structure-forming systems.

Structure-forming systems are ubiquitous in nature, ranging from atoms building molecules to self-assembly of colloidal amphibolic particles. The understanding of the underlying thermodynamics of such systems remains an important problem. Here, we derive the entropy for structure-forming systems that differs from Boltzmann-Gibbs entropy by a term that explicitly captures clustered states. For large systems and low concentrations the approach is equivalent to the grand-canonical ensemble; for small systems we find significant deviations. We derive the detailed fluctuation theorem and Crooks' work fluctuation theorem for structure-forming systems. The connection to the theory of particle self-assembly is discussed. We apply the results to several physical systems. We present the phase diagram for patchy particles described by the Kern-Frenkel potential. We show that the Curie-Weiss model with molecule structures exhibits a first-order phase transition.

The effect of social balance on social fragmentation.

With the availability of internet, social media, etc., the interconnectedness of people within most societies has increased tremendously over the past decades. Across the same timespan, an increasing level of fragmentation of society into small isolated groups has been observed. With a simple model of a society, in which the dynamics of individual opinion formation is integrated with social balance, we show that these two phenomena might be tightly related. We identify a critical level of interconnectedness, above which society fragments into sub-communities that are internally cohesive and hostile towards other groups. This critical communication density necessarily exists in the presence of social balance, and arises from the underlying mathematical structure of a phase transition known from the theory of disordered magnets called spin glasses. We discuss the consequences of this phase transition for social fragmentation in society.

Boosting test-efficiency by pooled testing for SARS-CoV-2-Formula for optimal pool size.

In the current COVID19 crisis many national healthcare systems are confronted with an acute shortage of tests for confirming SARS-CoV-2 infections. For low overall infection levels in the population the pooling of samples can drastically amplify the testing capacity. Here we present a formula to estimate the optimal group-size for pooling, the efficiency gain (tested persons per test), and the expected upper bound of missed infections in pooled testing, all as a function of the population-wide infection levels and the false negative/positive rates of the currently used PCR tests. Assuming an infection level of 0.1% and a false negative rate of 2%, the optimal pool-size is about 34, and an efficiency gain of about 15 tested persons per test is possible. For an infection level of 1% the optimal pool-size is 11, the efficiency gain is 5.1 tested persons per test. For an infection level of 10% the optimal pool-size reduces to about 4, the efficiency gain is about 1.7 tested persons per test. For infection levels of 30% and higher there is no more benefit from pooling. To see to what extent replicates of the pooled tests improve the estimate of the maximal number of missed infections, we present results for 1 to 5 replicates.

The role of grammar in transition-probabilities of subsequent words in English text.

Sentence formation is a highly structured, history-dependent, and sample-space reducing (SSR) process. While the first word in a sentence can be chosen from the entire vocabulary, typically, the freedom of choosing subsequent words gets more and more constrained by grammar and context, as the sentence progresses. This sample-space reducing property offers a natural explanation of Zipf's law in word frequencies, however, it fails to capture the structure of the word-to-word transition probability matrices of English text. Here we adopt the view that grammatical constraints (such as subject-predicate-object) locally re-order the word order in sentences that are sampled by the word generation process. We demonstrate that superimposing grammatical structure-as a local word re-ordering (permutation) process-on a sample-space reducing word generation process is sufficient to explain both, word frequencies and word-to-word transition probabilities. We compare the performance of the grammatically ordered SSR model in reproducing several test statistics of real texts with other text generation models, such as the Bernoulli model, the Simon model, and the random typewriting model.

A network-based explanation of why most COVID-19 infection curves are linear.

Many countries have passed their first COVID-19 epidemic peak. Traditional epidemiological models describe this as a result of nonpharmaceutical interventions pushing the growth rate below the recovery rate. In this phase of the pandemic many countries showed an almost linear growth of confirmed cases for extended time periods. This new containment regime is hard to explain by traditional models where either infection numbers grow explosively until herd immunity is reached or the epidemic is completely suppressed. Here we offer an explanation of this puzzling observation based on the structure of contact networks. We show that for any given transmission rate there exists a critical number of social contacts, [Formula: see text], below which linear growth and low infection prevalence must occur. Above [Formula: see text] traditional epidemiological dynamics take place, e.g., as in susceptible-infected-recovered (SIR) models. When calibrating our model to empirical estimates of the transmission rate and the number of days being contagious, we find [Formula: see text] Assuming realistic contact networks with a degree of about 5, and assuming that lockdown measures would reduce that to household size (about 2.5), we reproduce actual infection curves with remarkable precision, without fitting or fine-tuning of parameters. In particular, we compare the United States and Austria, as examples for one country that initially did not impose measures and one that responded with a severe lockdown early on. Our findings question the applicability of standard compartmental models to describe the COVID-19 containment phase. The probability to observe linear growth in these is practically zero.

Time-energy uncertainty principle for irreversible heat engines.

Even though irreversibility is one of the major hallmarks of any real-life process, an actual understanding of irreversible processes remains still mostly semi-empirical. In this paper, we formulate a thermodynamic uncertainty principle for irreversible heat engines operating with an ideal gas as a working medium. In particular, we show that the time needed to run through such an irreversible cycle multiplied by the irreversible work lost in the cycle is bounded from below by an irreducible and process-dependent constant that has the dimension of an action. The constant in question depends on a typical scale of the process and becomes comparable to Planck's constant at the length scale of the order Bohr radius, i.e. the scale that corresponds to the smallest distance on which the ideal gas paradigm realistically applies. This article is part of the theme issue 'Fundamental aspects of nonequilibrium thermodynamics'.

Information Geometric Duality of ϕ-Deformed Exponential Families.

In the world of generalized entropies-which, for example, play a role in physical systems with sub- and super-exponential phase space growth per degree of freedom-there are two ways for implementing constraints in the maximum entropy principle: linear and escort constraints. Both appear naturally in different contexts. Linear constraints appear, e.g., in physical systems, when additional information about the system is available through higher moments. Escort distributions appear naturally in the context of multifractals and information geometry. It was shown recently that there exists a fundamental duality that relates both approaches on the basis of the corresponding deformed logarithms (deformed-log duality). Here, we show that there exists another duality that arises in the context of information geometry, relating the Fisher information of ϕ -deformed exponential families that correspond to linear constraints (as studied by J.Naudts) to those that are based on escort constraints (as studied by S.-I. Amari). We explicitly demonstrate this information geometric duality for the case of ( c , d ) -entropy, which covers all situations that are compatible with the first three Shannon-Khinchin axioms and that include Shannon, Tsallis, Anteneodo-Plastino entropy, and many more as special cases. Finally, we discuss the relation between the deformed-log duality and the information geometric duality and mention that the escort distributions arising in these two dualities are generally different and only coincide for the case of the Tsallis deformation.

How driving rates determine the statistics of driven non-equilibrium systems with stationary distributions.

Sample space reducing (SSR) processes offer a simple analytical way to understand the origin and ubiquity of power-laws in many path-dependent complex systems. SRR processes show a wide range of applications that range from fragmentation processes, language formation to search and cascading processes. Here we argue that they also offer a natural framework to understand stationary distributions of generic driven non-equilibrium systems that are composed of a driving- and a relaxing process. We show that the statistics of driven non-equilibrium systems can be derived from the understanding of the nature of the underlying driving process. For constant driving rates exact power-laws emerge with exponents that are related to the driving rate. If driving rates become state-dependent, or if they vary across the life-span of the process, the functional form of the state-dependence determines the statistics. Constant driving rates lead to exact power-laws, a linear state-dependence function yields exponential or Gamma distributions, a quadratic function produces the normal distribution. Logarithmic and power-law state dependence leads to log-normal and stretched exponential distribution functions, respectively. Also Weibull, Gompertz and Tsallis-Pareto distributions arise naturally from simple state-dependent driving rates. We discuss a simple physical example of consecutive elastic collisions that exactly represents a SSR process.

Correction: Fitting power-laws in empirical data with estimators that work for all exponents.

[This corrects the article DOI: 10.1371/journal.pone.0170920.].

Maximum Configuration Principle for Driven Systems with Arbitrary Driving.

Depending on context, the term entropy is used for a thermodynamic quantity, a measure of available choice, a quantity to measure information, or, in the context of statistical inference, a maximum configuration predictor. For systems in equilibrium or processes without memory, the mathematical expression for these different concepts of entropy appears to be the so-called Boltzmann-Gibbs-Shannon entropy, H. For processes with memory, such as driven- or self- reinforcing-processes, this is no longer true: the different entropy concepts lead to distinct functionals that generally differ from H. Here we focus on the maximum configuration entropy (that predicts empirical distribution functions) in the context of driven dissipative systems. We develop the corresponding framework and derive the entropy functional that describes the distribution of observable states as a function of the details of the driving process. We do this for sample space reducing (SSR) processes, which provide an analytically tractable model for driven dissipative systems with controllable driving. The fact that a consistent framework for a maximum configuration entropy exists for arbitrarily driven non-equilibrium systems opens the possibility of deriving a full statistical theory of driven dissipative systems of this kind. This provides us with the technical means needed to derive a thermodynamic theory of driven processes based on a statistical theory. We discuss the Legendre structure for driven systems.

Sample space reducing cascading processes produce the full spectrum of scaling exponents.

Sample Space Reducing (SSR) processes are simple stochastic processes that offer a new route to understand scaling in path-dependent processes. Here we define a cascading process that generalises the recently defined SSR processes and is able to produce power laws with arbitrary exponents. We demonstrate analytically that the frequency distributions of states are power laws with exponents that coincide with the multiplication parameter of the cascading process. In addition, we show that imposing energy conservation in SSR cascades allows us to recover Fermi's classic result on the energy spectrum of cosmic rays, with the universal exponent -2, which is independent of the multiplication parameter of the cascade. Applications of the proposed process include fragmentation processes or directed cascading diffusion on networks, such as rumour or epidemic spreading.

Fitting power-laws in empirical data with estimators that work for all exponents.

Most standard methods based on maximum likelihood (ML) estimates of power-law exponents can only be reliably used to identify exponents smaller than minus one. The argument that power laws are otherwise not normalizable, depends on the underlying sample space the data is drawn from, and is true only for sample spaces that are unbounded from above. Power-laws obtained from bounded sample spaces (as is the case for practically all data related problems) are always free of such limitations and maximum likelihood estimates can be obtained for arbitrary powers without restrictions. Here we first derive the appropriate ML estimator for arbitrary exponents of power-law distributions on bounded discrete sample spaces. We then show that an almost identical estimator also works perfectly for continuous data. We implemented this ML estimator and discuss its performance with previous attempts. We present a general recipe of how to use these estimators and present the associated computer codes.

Three faces of entropy for complex systems: Information, thermodynamics, and the maximum entropy principle.

There are at least three distinct ways to conceptualize entropy: entropy as an extensive thermodynamic quantity of physical systems (Clausius, Boltzmann, Gibbs), entropy as a measure for information production of ergodic sources (Shannon), and entropy as a means for statistical inference on multinomial processes (Jaynes maximum entropy principle). Even though these notions represent fundamentally different concepts, the functional form of the entropy for thermodynamic systems in equilibrium, for ergodic sources in information theory, and for independent sampling processes in statistical systems, is degenerate, H(p)=-∑_{i}p_{i}logp_{i}. For many complex systems, which are typically history-dependent, nonergodic, and nonmultinomial, this is no longer the case. Here we show that for such processes, the three entropy concepts lead to different functional forms of entropy, which we will refer to as S_{EXT} for extensive entropy, S_{IT} for the source information rate in information theory, and S_{MEP} for the entropy functional that appears in the so-called maximum entropy principle, which characterizes the most likely observable distribution functions of a system. We explicitly compute these three entropy functionals for three concrete examples: for Pólya urn processes, which are simple self-reinforcing processes, for sample-space-reducing (SSR) processes, which are simple history dependent processes that are associated with power-law statistics, and finally for multinomial mixture processes.