Assigning credit where it is due: an information content score to capture the clinical value of multiplexed assays of variant effect.

BACKGROUND: A variant can be pathogenic or benign with relation to a human disease. Current classification categories from benign to pathogenic reflect a probabilistic summary of the current understanding. A primary metric of clinical utility for multiplexed assays of variant effect (MAVE) is the number of variants that can be reclassified from uncertain significance (VUS). However, a gap in this measure of utility is that it underrepresents the information gained from MAVEs. The aim of this study was to develop an improved quantification metric for MAVE utility. We propose adopting an information content approach that includes data that does not reclassify variants will better reflect true information gain. We adopted an information content approach to evaluate the information gain, in bits, for MAVEs of BRCA1, PTEN, and TP53. Here, one bit represents the amount of information required to completely classify a single variant starting from no information. RESULTS: BRCA1 MAVEs produced a total of 831.2 bits of information, 6.58% of the total missense information in BRCA1 and a 22-fold increase over the information that only contributed to VUS reclassification. PTEN MAVEs produced 2059.6 bits of information which represents 32.8% of the total missense information in PTEN and an 85-fold increase over the information that contributed to VUS reclassification. TP53 MAVEs produced 277.8 bits of information which represents 6.22% of the total missense information in TP53 and a 3.5-fold increase over the information that contributed to VUS reclassification. CONCLUSIONS: An information content approach will more accurately portray information gained through MAVE mapping efforts than by counting the number of variants reclassified. This information content approach may also help define the impact of guideline changes that modify the information definitions used to classify groups of variants.

Research on Division of Labor Decision and System Stability of Swarm Robots Based on Mutual Information.

In rational decision-making processes, the information interaction among individual robots is a critical factor influencing system stability. We establish a game-theoretic model based on mutual information to address division of labor decision-making and stability issues arising from differential information interaction among swarm robots. Firstly, a mutual information model is employed to measure the information interaction among robots and analyze its influence on the behavior of individual robots. Secondly, employing the Cournot model and the Stackelberg model, we model the diverse decision-making behaviors of swarm robots influenced by discrepancies in mutual information. The intricate decision dynamics exhibited by the system under the disparity mutual information values during the game process, along with the stability of Nash equilibrium points, are analyzed. Finally, dynamic complexity simulations of the game models are simulated under the disparity mutual information values: (1) When ν1 of the game model varies within a certain range, the Nash equilibrium point loses stability and enters a chaotic state. (2) As I(X;Y) increases, the decision-making pattern of robots transitions gradually from the Cournot game to the Stackelberg game. Concurrently, the sensitivity of swarm robotics systems to changes in decision parameter decreases, reducing the likelihood of the system entering a chaotic state.

Inferring the roles of individuals in collective systems using information-theoretic measures of influence.

In collective systems, influence of individuals can permeate an entire group through indirect interactionscom-plicating any scheme to understand individual roles from observations. A typical approach to understand an individuals influence on another involves consideration of confounding factors, for example, by conditioning on other individuals outside of the pair. This becomes unfeasible in many cases as the number of individuals increases. In this article, we review some of the unforeseen problems that arise in understanding individual influence in a collective such as single cells, as well as some of the recent works which address these issues using tools from information theory.

An Attacker-Defender Game Model with Constrained Strategies.

Recently, research interest in the field of infrastructure attack and defense scenarios has increased. Numerous methods have been proposed for studying strategy interactions that combine complex network theory and game theory. However, the unavoidable effect of constrained strategies in complex situations has not been considered in previous studies. This study introduces a novel approach to analyzing these interactions by including the effects of constrained strategies, a factor often neglected in traditional analyses. First, we introduce the rule of constraints on strategies, which depends on the average distance between selected nodes. As the average distance increases, the probability of choosing the corresponding strategy decreases. Second, we establish an attacker-defender game model with constrained strategies based on the above rule and using information theory to evaluate the uncertainty of these strategies. Finally, we present a method for solving this problem and conduct experiments based on a target network. The results highlight the unique characteristics of the Nash equilibrium when setting constraints, as these constraints influence decision makers' Nash equilibria. When considering the constrained strategies, both the attacker and the defender tend to select strategies with lower average distances. The effect of the constraints on their strategies becomes less apparent as the number of attackable or defendable nodes increases. This research advances the field by introducing a novel framework for examining strategic interactions in infrastructure defense and attack scenarios. By incorporating strategy constraints, our work offers a new perspective on the critical area of infrastructure security.

Understanding Higher-Order Interactions in Information Space.

Methods used in topological data analysis naturally capture higher-order interactions in point cloud data embedded in a metric space. This methodology was recently extended to data living in an information space, by which we mean a space measured with an information theoretical distance. One such setting is a finite collection of discrete probability distributions embedded in the probability simplex measured with the relative entropy (Kullback-Leibler divergence). More generally, one can work with a Bregman divergence parameterized by a different notion of entropy. While theoretical algorithms exist for this setup, there is a paucity of implementations for exploring and comparing geometric-topological properties of various information spaces. The interest of this work is therefore twofold. First, we propose the first robust algorithms and software for geometric and topological data analysis in information space. Perhaps surprisingly, despite working with Bregman divergences, our design reuses robust libraries for the Euclidean case. Second, using the new software, we take the first steps towards understanding the geometric-topological structure of these spaces. In particular, we compare them with the more familiar spaces equipped with the Euclidean and Fisher metrics.

Contrast Information Dynamics: A Novel Information Measure for Cognitive Modelling.

We present contrast information, a novel application of some specific cases of relative entropy, designed to be useful for the cognitive modelling of the sequential perception of continuous signals. We explain the relevance of entropy in the cognitive modelling of sequential phenomena such as music and language. Then, as a first step to demonstrating the utility of constrast information for this purpose, we empirically show that its discrete case correlates well with existing successful cognitive models in the literature. We explain some interesting properties of constrast information. Finally, we propose future work toward a cognitive architecture that uses it.

Conscious Causality, Observer-Observed Simultaneity, and the Problem of Time for Integrated Information Theory.

Without proven causal power, consciousness cannot be integrated with physics except as an epiphenomenon, hence the term 'hard problem'. Integrated Information Theory (IIT) side-steps the issue by stating that subjective experience must be identical to informational physical structures whose cause-and-effect power is greater than the sum of their parts. But the focus on spatially oriented structures rather than events in time introduces a deep conceptual flaw throughout its entire structure, including the measure of integrated information, known as Φ (phi). However, the problem can be corrected by incorporating the temporal feature of consciousness responsible for the hard problem, which can ultimately resolve it, namely, that experiencer and experienced are not separated in time but exist simultaneously. Simultaneous causation is not possible in physics, hence the hard problem, and yet it can be proven deductively that consciousness does have causal power because of this phenomenological simultaneity. Experiencing presence makes some facts logically possible that would otherwise be illogical. Bypassing the hard problem has caused much of the criticism that IIT has attracted, but by returning to its roots in complexity theory, it can repurpose its model to measure causal connections that are temporally rather than spatially related.

Information theory reveals physiological manifestations of COVID-19 that correlate with symptom density of illness.

Algorithms for the detection of COVID-19 illness from wearable sensor devices tend to implicitly treat the disease as causing a stereotyped (and therefore recognizable) deviation from healthy physiology. In contrast, a substantial diversity of bodily responses to SARS-CoV-2 infection have been reported in the clinical milieu. This raises the question of how to characterize the diversity of illness manifestations, and whether such characterization could reveal meaningful relationships across different illness manifestations. Here, we present a framework motivated by information theory to generate quantified maps of illness presentation, which we term "manifestations," as resolved by continuous physiological data from a wearable device (Oura Ring). We test this framework on five physiological data streams (heart rate, heart rate variability, respiratory rate, metabolic activity, and sleep temperature) assessed at the time of reported illness onset in a previously reported COVID-19-positive cohort (N = 73). We find that the number of distinct manifestations are few in this cohort, compared to the space of all possible manifestations. In addition, manifestation frequency correlates with the rough number of symptoms reported by a given individual, over a several-day period prior to their imputed onset of illness. These findings suggest that information-theoretic approaches can be used to sort COVID-19 illness manifestations into types with real-world value. This proof of concept supports the use of information-theoretic approaches to map illness manifestations from continuous physiological data. Such approaches could likely inform algorithm design and real-time treatment decisions if developed on large, diverse samples.

Comparative analyses of the information content of letters, characters, and inter-word spaces across writing systems.

One difference among writing systems is how orthographic cues are used to demarcate words; although most alphabetic scripts use inter-word spaces, some Asian scripts do not explicitly mark word boundaries (e.g., Chinese). It is unclear whether these differences are arbitrary or whether they are designed to maximize reading efficiency. Here, we show that spaces inserted between words in non-demarcated scripts provide less information about word boundaries than spaces in demarcated scripts. Furthermore, despite the fact that less information is contained by inter-word spaces than characters/letters of the same size, the information content of inter-word spaces in demarcated scripts is closer to that of characters/letters compared to the information content of inter-word spaces that are inserted in non-demarcated scripts. These results suggest that the conventions used to demarcate word boundaries are sufficient to support efficient reading. Our findings provide new insights into the universals and variation across writing systems and shed light on the mental processes that support skilled reading.

Response inhibition in premotor cortex corresponds to a complex reshuffle of the mesoscopic information network.

Recent studies have explored functional and effective neural networks in animal models; however, the dynamics of information propagation among functional modules under cognitive control remain largely unknown. Here, we addressed the issue using transfer entropy and graph theory methods on mesoscopic neural activities recorded in the dorsal premotor cortex of rhesus monkeys. We focused our study on the decision time of a Stop-signal task, looking for patterns in the network configuration that could influence motor plan maturation when the Stop signal is provided. When comparing trials with successful inhibition to those with generated movement, the nodes of the network resulted organized into four clusters, hierarchically arranged, and distinctly involved in information transfer. Interestingly, the hierarchies and the strength of information transmission between clusters varied throughout the task, distinguishing between generated movements and canceled ones and corresponding to measurable levels of network complexity. Our results suggest a putative mechanism for motor inhibition in premotor cortex: a topological reshuffle of the information exchanged among ensembles of neurons.

In this study, we investigated the dynamics of information transfer among functionally identified neural modules during cognitive motor control. Our focus was on mesoscopic neural activities in the dorsal premotor cortex of rhesus monkeys engaged in a Stop-signal task. Leveraging multivariate transfer entropy and graph theory, we uncovered insights on how behavioral control shapes the topology of information transmission in a local brain network. Task phases modulated the strength and hierarchy of information exchange between modules, revealing the nuanced interplay between neural populations during generated and canceled movements. Notably, during successful inhibition, the network displayed a distinctive configuration, unveiling a novel mechanism for motor inhibition in the premotor cortex: a topological reshuffle of information among neuronal ensembles.

Word Forms Reflect Trade-Offs Between Speaker Effort and Robust Listener Recognition.

How do cognitive pressures shape the lexicons of natural languages? Here, we reframe George Kingsley Zipf's proposed "law of abbreviation" within a more general framework that relates it to cognitive pressures that affect speakers and listeners. In this new framework, speakers' drive to reduce effort (Zipf's proposal) is counteracted by the need for low-frequency words to have word forms that are sufficiently distinctive to allow for accurate recognition by listeners. To support this framework, we replicate and extend recent work using the prevalence of subword phonemic sequences (phonotactic probability) to measure speakers' production effort in place of Zipf's measure of length. Across languages and corpora, phonotactic probability is more strongly correlated with word frequency than word length. We also show this measure of ease of speech production (phonotactic probability) is strongly correlated with a measure of perceptual difficulty that indexes the degree of competition from alternative interpretations in word recognition. This is consistent with the claim that there must be trade-offs between these two factors, and is inconsistent with a recent proposal that phonotactic probability facilitates both perception and production. To our knowledge, this is the first work to offer an explanation why long, phonotactically improbable word forms remain in the lexicons of natural languages.

Confirmation Bias through Selective Use of Evidence in Human Cortex.

Decision-makers often process new evidence selectively, depending on their current beliefs about the world. We asked whether such confirmation biases result from biases in the encoding of sensory evidence in the brain, or alternatively in the utilization of encoded evidence for behavior. Human participants estimated the source of a sequence of visual-spatial evidence samples while we measured cortical population activity with magnetoencephalography (MEG). Halfway through the sequence, participants were prompted to judge the more likely source category. Their processing of subsequent evidence depended on its consistency with the previously chosen category, but the encoding of evidence in cortical activity did not. Instead, the encoded evidence in parietal and primary visual cortex contributed less to the estimation report when that evidence was inconsistent with the previous choice. We conclude that confirmation bias originates from the way in which decision-makers utilize information encoded in the brain. This provides room for deliberative control.

KLD: a program to elucidate the localization of the Fermi and Coulomb holes in molecular systems.

CONTEXT: The electron localization is a concept that allows scientists to better understand the physical and chemical properties of electronic systems. It is associated with the propensity of electron pairs with opposite spins to accumulate as well as with their response to external perturbations. This paper contains a detailed description of the design and implementation of the program KLD, which was primarily developed in our research group to elucidate electron localization in molecular systems by evaluating the information content of electron-pair density functions. KLD employs two information-based functions as a real space measure of the Fermi and Coulomb holes for same-spin electrons and shows a better resolution as compared to other methods (i.e., ELF). Information about the acceleration of the code is also included in the present work, being noticeable the reduction of wall-time calculation and the error calculation between versions. METHODS: KLD was designed to be easy to use, extend, and maintain; thus, many principles of modern software development, extensive testing, and package management were adopted. The latest version of the KLD program was created utilizing the Compute Unified Device Architecture (CUDA) version, which allows it to use the computational capacity of NVIDIA Graphics Processing Units (GPUs) for processing purposes. The electron-pair conditional density was calculated from the canonical molecular orbitals obtained at the HF/6-31G(2df,p) level, or alternatively the natural orbitals in the case of explicit correlated wavefunctions computed at the MP2/6-31G(2df,p)//HF/6-31G(2df,p) level.

Rényi entropy-complexity causality space: a novel neurocomputational tool for detecting scale-free features in EEG/iEEG data.

Scale-free brain activity, linked with learning, the integration of different time scales, and the formation of mental models, is correlated with a metastable cognitive basis. The spectral slope, a key aspect of scale-free dynamics, was proposed as a potential indicator to distinguish between different sleep stages. Studies suggest that brain networks maintain a consistent scale-free structure across wakefulness, anesthesia, and recovery. Although differences in anesthetic sensitivity between the sexes are recognized, these variations are not evident in clinical electroencephalographic recordings of the cortex. Recently, changes in the slope of the power law exponent of neural activity were found to correlate with changes in Rényi entropy, an extended concept of Shannon's information entropy. These findings establish quantifiers as a promising tool for the study of scale-free dynamics in the brain. Our study presents a novel visual representation called the Rényi entropy-complexity causality space, which encapsulates complexity, permutation entropy, and the Rényi parameter q. The main goal of this study is to define this space for classical dynamical systems within theoretical bounds. In addition, the study aims to investigate how well different time series mimicking scale-free activity can be discriminated. Finally, this tool is used to detect dynamic features in intracranial electroencephalography (iEEG) signals. To achieve these goals, the study implementse the Bandt and Pompe method for ordinal patterns. In this process, each signal is associated with a probability distribution, and the causal measures of Rényi entropy and complexity are computed based on the parameter q. This method is a valuable tool for analyzing simulated time series. It effectively distinguishes elements of correlated noise and provides a straightforward means of examining differences in behaviors, characteristics, and classifications. For the iEEG experimental data, the REM state showed a greater number of significant sex-based differences, while the supramarginal gyrus region showed the most variation across different modes and analyzes. Exploring scale-free brain activity with this framework could provide valuable insights into cognition and neurological disorders. The results may have implications for understanding differences in brain function between the sexes and their possible relevance to neurological disorders.

Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysis.

We study stochastic linear contextual bandits (CB) where the agent observes a noisy version of the true context through a noise channel with unknown channel parameters. Our objective is to design an action policy that can "approximate" that of a Bayesian oracle that has access to the reward model and the noise channel parameter. We introduce a modified Thompson sampling algorithm and analyze its Bayesian cumulative regret with respect to the oracle action policy via information-theoretic tools. For Gaussian bandits with Gaussian context noise, our information-theoretic analysis shows that under certain conditions on the prior variance, the Bayesian cumulative regret scales as O˜(mT), where m is the dimension of the feature vector and T is the time horizon. We also consider the problem setting where the agent observes the true context with some delay after receiving the reward, and show that delayed true contexts lead to lower regret. Finally, we empirically demonstrate the performance of the proposed algorithms against baselines.

Reliability of a probabilistic knowledge structure.

Indexes for estimating the overall reliability of a test in the framework of knowledge space theory (KST) are proposed and analyzed. First, the possibility of applying in KST the existing classical test theory (CTT) methods, based on the ratio between the true score variance and the total variance of the measure, has been explored. However, these methods are not suitable because in KST error and true score are not independent. Therefore, two new indexes based on the concepts of entropy and conditional entropy are developed. One index is used to estimate the reliability of the response pattern given the knowledge state, while the second one refers to the reliability of the estimated knowledge state of a person. Some theoretical considerations as well as simulations and an empirical example on real data are provided within a study of the behavior of these indexes under a certain number of different conditions.

On the Mathematical Relationship Between Contextual Probability and N400 Amplitude.

Accounts of human language comprehension propose different mathematical relationships between the contextual probability of a word and how difficult it is to process, including linear, logarithmic, and super-logarithmic ones. However, the empirical evidence favoring any of these over the others is mixed, appearing to vary depending on the index of processing difficulty used and the approach taken to calculate contextual probability. To help disentangle these results, we focus on the mathematical relationship between corpus-derived contextual probability and the N400, a neural index of processing difficulty. Specifically, we use 37 contemporary transformer language models to calculate the contextual probability of stimuli from 6 experimental studies of the N400, and test whether N400 amplitude is best predicted by a linear, logarithmic, super-logarithmic, or sub-logarithmic transformation of the probabilities calculated using these language models, as well as combinations of these transformed metrics. We replicate the finding that on some datasets, a combination of linearly and logarithmically-transformed probability can predict N400 amplitude better than either metric alone. In addition, we find that overall, the best single predictor of N400 amplitude is sub-logarithmically-transformed probability, which for almost all language models and datasets explains all the variance in N400 amplitude otherwise explained by the linear and logarithmic transformations. This is a novel finding that is not predicted by any current theoretical accounts, and thus one that we argue is likely to play an important role in increasing our understanding of how the statistical regularities of language impact language comprehension.