Unraveling Hidden Major Factors by Breaking Heterogeneity into Homogeneous Parts within Many-System Problems.

For a large ensemble of complex systems, a Many-System Problem (MSP) studies how heterogeneity constrains and hides structural mechanisms, and how to uncover and reveal hidden major factors from homogeneous parts. All member systems in an MSP share common governing principles of dynamics, but differ in idiosyncratic characteristics. A typical dynamic is found underlying response features with respect to covariate features of quantitative or qualitative data types. Neither all-system-as-one-whole nor individual system-specific functional structures are assumed in such response-vs-covariate (Re-Co) dynamics. We developed a computational protocol for identifying various collections of major factors of various orders underlying Re-Co dynamics. We first demonstrate the immanent effects of heterogeneity among member systems, which constrain compositions of major factors and even hide essential ones. Secondly, we show that fuller collections of major factors are discovered by breaking heterogeneity into many homogeneous parts. This process further realizes Anderson's "More is Different" phenomenon. We employ the categorical nature of all features and develop a Categorical Exploratory Data Analysis (CEDA)-based major factor selection protocol. Information theoretical measurements-conditional mutual information and entropy-are heavily used in two selection criteria: C1-confirmable and C2-irreplaceable. All conditional entropies are evaluated through contingency tables with algorithmically computed reliability against the finite sample phenomenon. We study one artificially designed MSP and then two real collectives of Major League Baseball (MLB) pitching dynamics with 62 slider pitchers and 199 fastball pitchers, respectively. Finally, our MSP data analyzing techniques are applied to resolve a scientific issue related to the Rosenberg Self-Esteem Scale.

Learned Practical Guidelines for Evaluating Conditional Entropy and Mutual Information in Discovering Major Factors of Response-vs.-Covariate Dynamics.

We reformulate and reframe a series of increasingly complex parametric statistical topics into a framework of response-vs.-covariate (Re-Co) dynamics that is described without any explicit functional structures. Then we resolve these topics' data analysis tasks by discovering major factors underlying such Re-Co dynamics by only making use of data's categorical nature. The major factor selection protocol at the heart of Categorical Exploratory Data Analysis (CEDA) paradigm is illustrated and carried out by employing Shannon's conditional entropy (CE) and mutual information (I[Re;Co]) as the two key Information Theoretical measurements. Through the process of evaluating these two entropy-based measurements and resolving statistical tasks, we acquire several computational guidelines for carrying out the major factor selection protocol in a do-and-learn fashion. Specifically, practical guidelines are established for evaluating CE and I[Re;Co] in accordance with the criterion called [C1:confirmable]. Following the [C1:confirmable] criterion, we make no attempts on acquiring consistent estimations of these theoretical information measurements. All evaluations are carried out on a contingency table platform, upon which the practical guidelines also provide ways of lessening the effects of the curse of dimensionality. We explicitly carry out six examples of Re-Co dynamics, within each of which, several widely extended scenarios are also explored and discussed.

Categorical Nature of Major Factor Selection via Information Theoretic Measurements.

Without assuming any functional or distributional structure, we select collections of major factors embedded within response-versus-covariate (Re-Co) dynamics via selection criteria [C1: confirmable] and [C2: irrepaceable], which are based on information theoretic measurements. The two criteria are constructed based on the computing paradigm called Categorical Exploratory Data Analysis (CEDA) and linked to Wiener-Granger causality. All the information theoretical measurements, including conditional mutual information and entropy, are evaluated through the contingency table platform, which primarily rests on the categorical nature within all involved features of any data types: quantitative or qualitative. Our selection task identifies one chief collection, together with several secondary collections of major factors of various orders underlying the targeted Re-Co dynamics. Each selected collection is checked with algorithmically computed reliability against the finite sample phenomenon, and so is each member's major factor individually. The developments of our selection protocol are illustrated in detail through two experimental examples: a simple one and a complex one. We then apply this protocol on two data sets pertaining to two somewhat related but distinct pitching dynamics of two pitch types: slider and fastball. In particular, we refer to a specific Major League Baseball (MLB) pitcher and we consider data of multiple seasons.

Social power, conflict policing, and the role of subordination signals in rhesus macaque society.

OBJECTIVES: Policing is a conflict-limiting mechanism observed in many primate species. It is thought to require a skewed distribution of social power for some individuals to have sufficiently high social power to stop others' fights, yet social power has not been examined in most species with policing behavior. We examined networks of subordination signals as a source of social power that permits policing behavior in rhesus macaques. MATERIALS AND METHODS: For each of seven captive groups of rhesus macaques, we (a) examined the structure of subordination signal networks and used GLMs to examine the relationship between (b) pairwise dominance certainty and subordination network pathways and (c) policing frequency and social power (group-level convergence in subordination signaling pathways). RESULTS: Networks of subordination signals had perfect linear transitivity, and pairs connected by both direct and indirect pathways of signals had more certain dominance relationships than pairs with no such network connection. Social power calculated using both direct and indirect network pathways showed a heavy-tailed distribution and positively predicted conflict policing. CONCLUSIONS: Our results empirically substantiate that subordination signaling is associated with greater dominance relationship certainty and further show that pairs who signal rarely (or not at all) may use information from others' signaling interactions to infer or reaffirm the relative certainty of their own relationships. We argue that the network of formal dominance relationships is central to societal stability because it is important for relationship stability and also supports the additional stabilizing mechanism of policing.

Computing Systemic Risk Using Multiple Behavioral and Keystone Networks: The Emergence of a Crisis in Primate Societies and Banks.

What do the behavior of monkeys in captivity and the financial system have in common? The nodes in such social systems relate to each other through multiple and keystone networks, not just one network. Each network in the system has its own topology, and the interactions among the system's networks change over time. In such systems, the lead into a crisis appears to be characterized by a decoupling of the networks from the keystone network. This decoupling can also be seen in the crumbling of the keystone's power structure toward a more horizontal hierarchy. This paper develops nonparametric methods for describing the joint model of the latent architecture of interconnected networks in order to describe this process of decoupling, and hence provide an early warning system of an impending crisis.

Complexity of possibly gapped histogram and analysis of histogram.

We demonstrate that gaps and distributional patterns embedded within real-valued measurements are inseparable biological and mechanistic information contents of the system. Such patterns are discovered through data-driven possibly gapped histogram, which further leads to the geometry-based analysis of histogram (ANOHT). Constructing a possibly gapped histogram is a complex problem of statistical mechanics due to the ensemble of candidate histograms being captured by a two-layer Ising model. This construction is also a distinctive problem of Information Theory from the perspective of data compression via uniformity. By defining a Hamiltonian (or energy) as a sum of total coding lengths of boundaries and total decoding errors within bins, this issue of computing the minimum energy macroscopic states is surprisingly resolved by applying the hierarchical clustering algorithm. Thus, a possibly gapped histogram corresponds to a macro-state. And then the first phase of ANOHT is developed for simultaneous comparison of multiple treatments, while the second phase of ANOHT is developed based on classical empirical process theory for a tree-geometry that can check the authenticity of branches of the treatment tree. The well-known Iris data are used to illustrate our technical developments. Also, a large baseball pitching dataset and a heavily right-censored divorce data are analysed to showcase the existential gaps and utilities of ANOHT.

From patterned response dependency to structured covariate dependency: Entropy based categorical-pattern-matching.

Data generated from a system of interest typically consists of measurements on many covariate features and possibly multiple response features across all subjects in a designated ensemble. Such data is naturally represented by one response-matrix against one covariate-matrix. A matrix lattice is an advantageous platform for simultaneously accommodating heterogeneous data types: continuous, discrete and categorical, and exploring hidden dependency among/between features and subjects. After each feature being individually renormalized with respect to its own histogram, the categorical version of mutual conditional entropy is evaluated for all pairs of response and covariate features according to the combinatorial information theory. Then, by applying Data Could Geometry (DCG) algorithmic computations on such a mutual conditional entropy matrix, multiple synergistic feature-groups are partitioned. Distinct synergistic feature-groups embrace distinct structures of dependency. The explicit details of dependency among members of synergistic features are seen through mutliscale compositions of blocks computed by a computing paradigm called Data Mechanics. We then propose a categorical pattern matching approach to establish a directed associative linkage: from the patterned response dependency to serial structured covariate dependency. The graphic display of such a directed associative linkage is termed an information flow and the degrees of association are evaluated via tree-to-tree mutual conditional entropy. This new universal way of discovering system knowledge is illustrated through five data sets. In each case, the emergent visible heterogeneity is an organization of discovered knowledge.

Social network community structure and the contact-mediated sharing of commensal E. coli among captive rhesus macaques (Macaca mulatta).

In group-living animals, heterogeneity in individuals' social connections may mediate the sharing of microbial infectious agents. In this regard, the genetic relatedness of individuals' commensal gut bacterium Escherichia coli may be ideal to assess the potential for pathogen transmission through animal social networks. Here we use microbial phylogenetics and population genetics approaches, as well as host social network reconstruction, to assess evidence for the contact-mediated sharing of E. coli among three groups of captively housed rhesus macaques (Macaca mulatta), at multiple organizational scales. For each group, behavioral data on grooming, huddling, and aggressive interactions collected for a six-week period were used to reconstruct social network communities via the Data Cloud Geometry (DCG) clustering algorithm. Further, an E. coli isolate was biochemically confirmed and genotypically fingerprinted from fecal swabs collected from each macaque. Population genetics approaches revealed that Group Membership, in comparison to intrinsic attributes like age, sex, and/or matriline membership of individuals, accounted for the highest proportion of variance in E. coli genotypic similarity. Social network approaches revealed that such sharing was evident at the community-level rather than the dyadic level. Specifically, although we found no links between dyadic E. coli similarity and social contact frequencies, similarity was significantly greater among macaques within the same social network communities compared to those across different communities. Moreover, tests for one of our study-groups confirmed that E. coli isolated from macaque rectal swabs were more genotypically similar to each other than they were to isolates from environmentally deposited feces. In summary, our results suggest that among frequently interacting, spatially constrained macaques with complex social relationships, microbial sharing via fecal-oral, social contact-mediated routes may depend on both individuals' direct connections and on secondary network pathways that define community structure. They lend support to the hypothesis that social network communities may act as bottlenecks to contain the spread of infectious agents, thereby encouraging disease control strategies to focus on multiple organizational scales. Future directions includeincreasing microbial sampling effort per individual to better-detect dyadic transmission events, and assessments of the co-evolutionary links between sociality, infectious agent risk, and host immune function.

Detection of social group instability among captive rhesus macaques using joint network modeling.

Social stability in group-living animals is an emergent property which arises from the interaction amongst multiple behavioral networks. However, pinpointing when a social group is at risk of collapse is difficult. We used a joint network modeling approach to examine the interdependencies between two behavioral networks, aggression and status signaling, from four stable and three unstable groups of rhesus macaques in order to identify characteristic patterns of network interdependence in stable groups that are readily distinguishable from unstable groups. Our results showed that the most prominent source of aggression-status network interdependence in stable social groups came from more frequent dyads than expected with opposite direction status-aggression (i.e. A threatens B and B signals acceptance of subordinate status). In contrast, unstable groups showed a decrease in opposite direction aggression-status dyads (but remained higher than expected) as well as more frequent than expected dyads with bidirectional aggression. These results demonstrate that not only was the stable joint relationship between aggression and status networks readily distinguishable from unstable time points, social instability manifested in at least two different ways. In sum, our joint modeling approach may prove useful in quantifying and monitoring the complex social dynamics of any wild or captive social system, as all social systems are composed of multiple interconnected networks.

Unravelling the geometry of data matrices: effects of water stress regimes on winemaking.

A new method is proposed for unravelling the patterns between a set of experiments and the features that characterize those experiments. The aims are to extract these patterns in the form of a coupling between the rows and columns of the corresponding data matrix and to use this geometry as a support for model testing. These aims are reached through two key steps, namely application of an iterative geometric approach to couple the metric spaces associated with the rows and columns, and use of statistical physics to generate matrices that mimic the original data while maintaining their inherent structure, thereby providing the basis for hypothesis testing and statistical inference. The power of this new method is illustrated on the study of the impact of water stress conditions on the attributes of 'Cabernet Sauvignon' Grapes, Juice, Wine and Bottled Wine from two vintages. The first step, named data mechanics, de-convolutes the intrinsic effects of grape berries and wine attributes due to the experimental irrigation conditions from the extrinsic effects of the environment. The second step provides an analysis of the associations of some attributes of the bottled wine with characteristics of either the matured grape berries or the resulting juice, thereby identifying statistically significant associations between the juice pH, yeast assimilable nitrogen, and sugar content and the bottled wine alcohol level.

Data mechanics and coupling geometry on binary bipartite networks.

We quantify the notion of pattern and formalize the process of pattern discovery under the framework of binary bipartite networks. Patterns of particular focus are interrelated global interactions between clusters on its row and column axes. A binary bipartite network is built into a thermodynamic system embracing all up-and-down spin configurations defined by product-permutations on rows and columns. This system is equipped with its ferromagnetic energy ground state under Ising model potential. Such a ground state, also called a macrostate, is postulated to congregate all patterns of interest embedded within the network data in a multiscale fashion. A new computing paradigm for indirect searching for such a macrostate, called Data Mechanics, is devised by iteratively building a surrogate geometric system with a pair of nearly optimal marginal ultrametrics on row and column spaces. The coupling measure minimizing the Gromov-Wasserstein distance of these two marginal geometries is also seen to be in the vicinity of the macrostate. This resultant coupling geometry reveals multiscale block pattern information that characterizes multiple layers of interacting relationships between clusters on row and on column axes. It is the nonparametric information content of a binary bipartite network. This coupling geometry is then demonstrated to shed new light and bring resolution to interaction issues in community ecology and in gene-content-based phylogenetics. Its implied global inferences are expected to have high potential in many scientific areas.

Single stock dynamics on high-frequency data: from a compressed coding perspective.

High-frequency return, trading volume and transaction number are digitally coded via a nonparametric computing algorithm, called hierarchical factor segmentation (HFS), and then are coupled together to reveal a single stock dynamics without global state-space structural assumptions. The base-8 digital coding sequence, which is capable of revealing contrasting aggregation against sparsity of extreme events, is further compressed into a shortened sequence of state transitions. This compressed digital code sequence vividly demonstrates that the aggregation of large absolute returns is the primary driving force for stimulating both the aggregations of large trading volumes and transaction numbers. The state of system-wise synchrony is manifested with very frequent recurrence in the stock dynamics. And this data-driven dynamic mechanism is seen to correspondingly vary as the global market transiting in and out of contraction-expansion cycles. These results not only elaborate the stock dynamics of interest to a fuller extent, but also contradict some classical theories in finance. Overall this version of stock dynamics is potentially more coherent and realistic, especially when the current financial market is increasingly powered by high-frequency trading via computer algorithms, rather than by individual investors.

biDCG: a new method for discovering global features of DNA microarray data via an iterative re-clustering procedure.

Biclustering techniques have become very popular in cancer genetics studies, as they are tools that are expected to connect phenotypes to genotypes, i.e. to identify subgroups of cancer patients based on the fact that they share similar gene expression patterns as well as to identify subgroups of genes that are specific to these subtypes of cancer and therefore could serve as biomarkers. In this paper we propose a new approach for identifying such relationships or biclusters between patients and gene expression profiles. This method, named biDCG, rests on two key concepts. First, it uses a new clustering technique, DCG-tree [Fushing et al, PLos One, 8, e56259 (2013)] that generates ultrametric topological spaces that capture the geometries of both the patient data set and the gene data set. Second, it optimizes the definitions of bicluster membership through an iterative two-way reclustering procedure in which patients and genes are reclustered in turn, based respectively on subsets of genes and patients defined in the previous round. We have validated biDCG on simulated and real data. Based on the simulated data we have shown that biDCG compares favorably to other biclustering techniques applied to cancer genomics data. The results on the real data sets have shown that biDCG is able to retrieve relevant biological information.

Bootstrapping on undirected binary networks via statistical mechanics.

We propose a new method inspired from statistical mechanics for extracting geometric information from undirected binary networks and generating random networks that conform to this geometry. In this method an undirected binary network is perceived as a thermodynamic system with a collection of permuted adjacency matrices as its states. The task of extracting information from the network is then reformulated as a discrete combinatorial optimization problem of searching for its ground state. To solve this problem, we apply multiple ensembles of temperature regulated Markov chains to establish an ultrametric geometry on the network. This geometry is equipped with a tree hierarchy that captures the multiscale community structure of the network. We translate this geometry into a Parisi adjacency matrix, which has a relative low energy level and is in the vicinity of the ground state. The Parisi adjacency matrix is then further optimized by making block permutations subject to the ultrametric geometry. The optimal matrix corresponds to the macrostate of the original network. An ensemble of random networks is then generated such that each of these networks conforms to this macrostate; the corresponding algorithm also provides an estimate of the size of this ensemble. By repeating this procedure at different scales of the ultrametric geometry of the network, it is possible to compute its evolution entropy, i.e. to estimate the evolution of its complexity as we move from a coarse to a ne description of its geometric structure. We demonstrate the performance of this method on simulated as well as real data networks.

Lewis Carroll's Doublets net of English words: network heterogeneity in a complex system.

Lewis Carroll's English word game Doublets is represented as a system of networks with each node being an English word and each connectivity edge confirming that its two ending words are equal in letter length, but different by exactly one letter. We show that this system, which we call the Doublets net, constitutes a complex body of linguistic knowledge concerning English word structure that has computable multiscale features. Distributed morphological, phonological and orthographic constraints and the language's local redundancy are seen at the node level. Phonological communities are seen at the network level. And a balancing act between the language's global efficiency and redundancy is seen at the system level. We develop a new measure of intrinsic node-to-node distance and a computational algorithm, called community geometry, which reveal the implicit multiscale structure within binary networks. Because the Doublets net is a modular complex cognitive system, the community geometry and computable multi-scale structural information may provide a foundation for understanding computational learning in many systems whose network structure has yet to be fully analyzed.

Joint modeling of multiple social networks to elucidate primate social dynamics: I. maximum entropy principle and network-based interactions.

In a complex behavioral system, such as an animal society, the dynamics of the system as a whole represent the synergistic interaction among multiple aspects of the society. We constructed multiple single-behavior social networks for the purpose of approximating from multiple aspects a single complex behavioral system of interest: rhesus macaque society. Instead of analyzing these networks individually, we describe a new method for jointly analyzing them in order to gain comprehensive understanding about the system dynamics as a whole. This method of jointly modeling multiple networks becomes valuable analytical tool for studying the complex nature of the interaction among multiple aspects of any system. Here we develop a bottom-up, iterative modeling approach based upon the maximum entropy principle. This principle is applied to a multi-dimensional link-based distributional framework, which is derived by jointly transforming the multiple directed behavioral social network data, for extracting patterns of synergistic inter-behavioral relationships. Using a rhesus macaque group as a model system, we jointly modeled and analyzed four different social behavioral networks at two different time points (one stable and one unstable) from a rhesus macaque group housed at the California National Primate Research Center (CNPRC). We report and discuss the inter-behavioral dynamics uncovered by our joint modeling approach with respect to social stability.

Multi-scale clustering by building a robust and self correcting ultrametric topology on data points.

The advent of high-throughput technologies and the concurrent advances in information sciences have led to an explosion in size and complexity of the data sets collected in biological sciences. The biggest challenge today is to assimilate this wealth of information into a conceptual framework that will help us decipher biological functions. A large and complex collection of data, usually called a data cloud, naturally embeds multi-scale characteristics and features, generically termed geometry. Understanding this geometry is the foundation for extracting knowledge from data. We have developed a new methodology, called data cloud geometry-tree (DCG-tree), to resolve this challenge. This new procedure has two main features that are keys to its success. Firstly, it derives from the empirical similarity measurements a hierarchy of clustering configurations that captures the geometric structure of the data. This hierarchy is then transformed into an ultrametric space, which is then represented via an ultrametric tree or a Parisi matrix. Secondly, it has a built-in mechanism for self-correcting clustering membership across different tree levels. We have compared the trees generated with this new algorithm to equivalent trees derived with the standard Hierarchical Clustering method on simulated as well as real data clouds from fMRI brain connectivity studies, cancer genomics, giraffe social networks, and Lewis Carroll's Doublets network. In each of these cases, we have shown that the DCG trees are more robust and less sensitive to measurement errors, and that they provide a better quantification of the multi-scale geometric structures of the data. As such, DCG-tree is an effective tool for analyzing complex biological data sets.

Multiscale community geometry in a network and its application.

We introduce a between-ness-based distance metric to extract local and global information for each pair of nodes (or "vertices" used interchangeably) located in a binary network. Since this distance then superimposes a weighted graph upon such a binary network, a multiscale clustering mechanism, called data cloud geometry, is applicable to discover hierarchical communities within a binary network. This approach resolves many shortcomings of community finding approaches, which are primarily based on modularity optimization. Using several contrived and real binary networks, our community hierarchies compare favorably with results derived from a recently proposed approach based on time-scale differences of random walks and has already demonstrated significant improvements over module-based approaches, especially on the multiscale and the determination of the number of communities.

Extracting multiscale pattern information of fMRI based functional brain connectivity with application on classification of autism spectrum disorders.

We employed a multi-scale clustering methodology known as "data cloud geometry" to extract functional connectivity patterns derived from functional magnetic resonance imaging (fMRI) protocol. The method was applied to correlation matrices of 106 regions of interest (ROIs) in 29 individuals with autism spectrum disorders (ASD), and 29 individuals with typical development (TD) while they completed a cognitive control task. Connectivity clustering geometry was examined at both "fine" and "coarse" scales. At the coarse scale, the connectivity clustering geometry produced 10 valid clusters with a coherent relationship to neural anatomy. A supervised learning algorithm employed fine scale information about clustering motif configurations and prevalence, and coarse scale information about intra- and inter-regional connectivity; the algorithm correctly classified ASD and TD participants with sensitivity of 82.8% and specificity of 82.8%. Most of the predictive power of the logistic regression model resided at the level of the fine-scale clustering geometry, suggesting that cellular versus systems level disturbances are more prominent in individuals with ASD. This article provides validation for this multi-scale geometric approach to extracting brain functional connectivity pattern information and for its use in classification of ASD.

Computing a ranking network with confidence bounds from a graph-based Beta random field.

We address two largely overlooked, fundamental issues in computing a ranking hierarchy within a society: which information in the network is relevant, and what effect chance has on the hierarchy. To properly account for uncertainty from limited data, we construct a random field in a matrix form having entry-wise posterior Beta distributions based on a graph of pairwise conflict outcomes. To evaluate relevant network information using information transitivity, another random matrix of synthesized transitive dominance odds is computed collectively along observed dominance paths. These two matrices are coupled together to fuse both direct and indirect dominance information. An ensemble of realizations of this fused random matrix facilitates an ensemble of optimal ranking networks by means of simulated annealing. Conditional statistical inferences regarding network features are derived, manifesting the effect of uncertainty. Our computational approach is suitable for large graphs of pairwise conflict outcomes, and can accommodate tremendous data heterogeneity-a typical feature in such studies. We also demonstrate the infeasibility of the classical maximum-likelihood approach, and expose the mechanistic flaws that stem from completely ignoring relevant information residing in the graph. We analyse two real datasets of decisive conflict outcomes, the first involving college football teams, and the second involving an adult rhesus macaque society in captivity.