Geometric Structures Induced by Deformations of the Legendre Transform.

The recent link discovered between generalized Legendre transforms and non-dually flat statistical manifolds suggests a fundamental reason behind the ubiquity of Rényi's divergence and entropy in a wide range of physical phenomena. However, these early findings still provide little intuition on the nature of this relationship and its implications for physical systems. Here we shed new light on the Legendre transform by revealing the consequences of its deformation via symplectic geometry and complexification. These findings reveal a novel common framework that leads to a principled and unified understanding of physical systems that are not well-described by classic information-theoretic quantities.

Neighbourhood Sum Degree-Based Indices and Entropy Measures for Certain Family of Graphene Molecules.

A topological index (TI) is a real number that defines the relationship between a chemical structure and its properties and remains invariant under graph isomorphism. TIs defined for chemical structures are capable of predicting physical properties, chemical reactivity and biological activity. Several kinds of TIs have been defined and studied for different molecular structures. Graphene is the thinnest material known to man and is also extremely strong while being a good conductor of heat and electricity. With such unique features, graphene and its derivatives have found commercial uses and have also fascinated theoretical chemists. In this article, the neighbourhood sum degree-based M-polynomial and entropy measures have been computed for graphene, graphyne and graphdiyne structures. The proper analytical expressions for these indices are derived. The obtained results will enable theoretical chemists to study these exciting structures further from a structural perspective.

Effect of functional and effective brain connectivity in identifying vowels from articulation imagery procedures.

Articulation imagery, a form of mental imagery, refers to the activity of imagining or speaking to oneself mentally without an articulation movement. It is an effective domain of research in speech impaired neural disorders, as speech imagination has high similarity to real voice communication. This work employs electroencephalography (EEG) signals acquired from articulation and articulation imagery in identifying the vowel being imagined during different tasks. EEG signals from chosen electrodes are decomposed using the empirical mode decomposition (EMD) method into a series of intrinsic mode functions. Brain connectivity estimators and entropy measures have been computed to analyze the functional cooperation and causal dependence between different cortical regions as well as the regularity in the signals. Using machine learning techniques such as multiclass support vector machine (MSVM) and random forest (RF), the vowels have been classified. Three different training and testing protocols (Articulation-AR, Articulation imagery-AI and Articulation vs Articulation imagery-AR vs AI) were employed for identifying the vowel being imagined of articulating. An overall classification accuracy of 80% was obtained for articulation imagery protocol which was found to be higher than the other two protocols. Also, MSVM techniques outperformed the RF technique in terms of the classification accuracy. The effect of brain connectivity estimators and machine learning techniques seems to be reliable in identifying the vowel from the subjects' thought and thereby assisting the people with speech impairment.

Essential Conditions for the Full Synergy of Probability of Occurrence Distributions.

In this contribution, we specify the conditions for assuring the validity of the synergy of the distribution of probabilities of occurrence. We also study the subsequent restriction on the maximal extension of the strict concavity region on the parameter space of Sharma-Mittal entropy measures, which has been derived in a previous paper in this journal. The present paper is then a necessary complement to that publication. Some applications of the techniques introduced here are applied to protein domain families (Pfam databases, versions 27.0 and 35.0). The results will show evidence of their usefulness for testing the classification work performed with methods of alignment that are used by expert biologists.

Multi-Class Classification of Medical Data Based on Neural Network Pruning and Information-Entropy Measures.

Medical data includes clinical trials and clinical data such as patient-generated health data, laboratory results, medical imaging, and different signals coming from continuous health monitoring. Some commonly used data analysis techniques are text mining, big data analytics, and data mining. These techniques can be used for classification, clustering, and machine learning tasks. Machine learning could be described as an automatic learning process derived from concepts and knowledge without deliberate system coding. However, finding a suitable machine learning architecture for a specific task is still an open problem. In this work, we propose a machine learning model for the multi-class classification of medical data. This model is comprised of two components-a restricted Boltzmann machine and a classifier system. It uses a discriminant pruning method to select the most salient neurons in the hidden layer of the neural network, which implicitly leads to a selection of features for the input patterns that feed the classifier system. This study aims to investigate whether information-entropy measures may provide evidence for guiding discriminative pruning in a neural network for medical data processing, particularly cancer research, by using three cancer databases: Breast Cancer, Cervical Cancer, and Primary Tumour. Our proposal aimed to investigate the post-training neuronal pruning methodology using dissimilarity measures inspired by the information-entropy theory; the results obtained after pruning the neural network were favourable. Specifically, for the Breast Cancer dataset, the reported results indicate a 10.68% error rate, while our error rates range from 10% to 15%; for the Cervical Cancer dataset, the reported best error rate is 31%, while our proposal error rates are in the range of 4% to 6%; lastly, for the Primary Tumour dataset, the reported error rate is 20.35%, and our best error rate is 31%.

Tsallis Entropy for Loss Models and Survival Models Involving Truncated and Censored Random Variables.

The aim of this paper consists in developing an entropy-based approach to risk assessment for actuarial models involving truncated and censored random variables by using the Tsallis entropy measure. The effect of some partial insurance models, such as inflation, truncation and censoring from above and truncation and censoring from below upon the entropy of losses is investigated in this framework. Analytic expressions for the per-payment and per-loss entropies are obtained, and the relationship between these entropies are studied. The Tsallis entropy of losses of the right-truncated loss random variable corresponding to the per-loss risk model with a deductible d and a policy limit u is computed for the exponential, Weibull, χ2 or Gamma distribution. In this context, the properties of the resulting entropies, such as the residual loss entropy and the past loss entropy, are studied as a result of using a deductible and a policy limit, respectively. Relationships between these entropy measures are derived, and the combined effect of a deductible and a policy limit is also analyzed. By investigating residual and past entropies for survival models, the entropies of losses corresponding to the proportional hazard and proportional reversed hazard models are derived. The Tsallis entropy approach for actuarial models involving truncated and censored random variables is new and more realistic, since it allows a greater degree of flexibility and improves the modeling accuracy.

Affective Computing on Machine Learning-Based Emotion Recognition Using a Self-Made EEG Device.

In this research, we develop an affective computing method based on machine learning for emotion recognition using a wireless protocol and a wearable electroencephalography (EEG) custom-designed device. The system collects EEG signals using an eight-electrode placement on the scalp; two of these electrodes were placed in the frontal lobe, and the other six electrodes were placed in the temporal lobe. We performed experiments on eight subjects while they watched emotive videos. Six entropy measures were employed for extracting suitable features from the EEG signals. Next, we evaluated our proposed models using three popular classifiers: a support vector machine (SVM), multi-layer perceptron (MLP), and one-dimensional convolutional neural network (1D-CNN) for emotion classification; both subject-dependent and subject-independent strategies were used. Our experiment results showed that the highest average accuracies achieved in the subject-dependent and subject-independent cases were 85.81% and 78.52%, respectively; these accuracies were achieved using a combination of the sample entropy measure and 1D-CNN. Moreover, our study investigates the T8 position (above the right ear) in the temporal lobe as the most critical channel among the proposed measurement positions for emotion classification through electrode selection. Our results prove the feasibility and efficiency of our proposed EEG-based affective computing method for emotion recognition in real-world applications.

Alternative Entropy Measures and Generalized Khinchin-Shannon Inequalities.

The Khinchin-Shannon generalized inequalities for entropy measures in Information Theory, are a paradigm which can be used to test the Synergy of the distributions of probabilities of occurrence in physical systems. The rich algebraic structure associated with the introduction of escort probabilities seems to be essential for deriving these inequalities for the two-parameter Sharma-Mittal set of entropy measures. We also emphasize the derivation of these inequalities for the special cases of one-parameter Havrda-Charvat's, Rényi's and Landsberg-Vedral's entropy measures.

Global Sensitivity Analysis Based on Entropy: From Differential Entropy to Alternative Measures.

Differential entropy can be negative, while discrete entropy is always non-negative. This article shows that negative entropy is a significant flaw when entropy is used as a sensitivity measure in global sensitivity analysis. Global sensitivity analysis based on differential entropy cannot have negative entropy, just as Sobol sensitivity analysis does not have negative variance. Entropy is similar to variance but does not have the same properties. An alternative sensitivity measure based on the approximation of the differential entropy using dome-shaped functionals with non-negative values is proposed in the article. Case studies have shown that new sensitivity measures lead to a rational structure of sensitivity indices with a significantly lower proportion of higher-order sensitivity indices compared to other types of distributional sensitivity analysis. In terms of the concept of sensitivity analysis, a decrease in variance to zero means a transition from the differential to discrete entropy. The form of this transition is an open question, which can be studied using other scientific disciplines. The search for new functionals for distributional sensitivity analysis is not closed, and other suitable sensitivity measures may be found.

The Entropy Universe.

About 160 years ago, the concept of entropy was introduced in thermodynamics by Rudolf Clausius. Since then, it has been continually extended, interpreted, and applied by researchers in many scientific fields, such as general physics, information theory, chaos theory, data mining, and mathematical linguistics. This paper presents The Entropy Universe, which aims to review the many variants of entropies applied to time-series. The purpose is to answer research questions such as: How did each entropy emerge? What is the mathematical definition of each variant of entropy? How are entropies related to each other? What are the most applied scientific fields for each entropy? We describe in-depth the relationship between the most applied entropies in time-series for different scientific fields, establishing bases for researchers to properly choose the variant of entropy most suitable for their data. The number of citations over the past sixteen years of each paper proposing a new entropy was also accessed. The Shannon/differential, the Tsallis, the sample, the permutation, and the approximate entropies were the most cited ones. Based on the ten research areas with the most significant number of records obtained in the Web of Science and Scopus, the areas in which the entropies are more applied are computer science, physics, mathematics, and engineering. The universe of entropies is growing each day, either due to the introducing new variants either due to novel applications. Knowing each entropy's strengths and of limitations is essential to ensure the proper improvement of this research field.

New Pythagorean Entropy Measure with Application in Multi-Criteria Decision Analysis.

The purpose of this paper is to propose a new Pythagorean fuzzy entropy for Pythagorean fuzzy sets, which is a continuation of the Pythagorean fuzzy entropy of intuitionistic sets. The Pythagorean fuzzy set continues the intuitionistic fuzzy set with the additional advantage that it is well equipped to overcome its imperfections. Its entropy determines the quantity of information in the Pythagorean fuzzy set. Thus, the proposed entropy provides a new flexible tool that is particularly useful in complex multi-criteria problems where uncertain data and inaccurate information are considered. The performance of the introduced method is illustrated in a real-life case study, including a multi-criteria company selection problem. In this example, we provide a numerical illustration to distinguish the entropy measure proposed from some existing entropies used for Pythagorean fuzzy sets and intuitionistic fuzzy sets. Statistical illustrations show that the proposed entropy measures are reliable for demonstrating the degree of fuzziness of both Pythagorean fuzzy set (PFS) and intuitionistic fuzzy sets (IFS). In addition, a multi-criteria decision-making method complex proportional assessment (COPRAS) was also proposed with weights calculated based on the proposed new entropy measure. Finally, to validate the reliability of the results obtained using the proposed entropy, a comparative analysis was performed with a set of carefully selected reference methods containing other generally used entropy measurement methods. The illustrated numerical example proves that the calculation results of the proposed new method are similar to those of several other up-to-date methods.

A Bayesian Entropy Approach to Sectoral Systemic Risk Modeling.

We investigate the dynamics of systemic risk of European companies using an approach that merges paradigmatic risk measures such as Marginal Expected Shortfall, CoVaR, and Delta CoVaR, with a Bayesian entropy estimation method. Our purpose is to bring to light potential spillover effects of the entropy indicator for the systemic risk measures computed on the 24 sectors that compose the STOXX 600 index. Our results show that several sectors have a high proclivity for generating spillovers. In general, the largest influences are delivered by Capital Goods, Banks, Diversified Financials, Insurance, and Real Estate. We also bring detailed evidence on the sectors that are the most pregnable to spillovers and on those that represent the main contributors of spillovers.

Comparison and Research on Diversified Portfolios with Several Entropy Measures Based on Different Psychological States.

In previous studies, there were few portfolio models involving investors' psychological states, market ambiguity and entropy. Some entropy can make the model have the effect of diversifying investment, which is very important. This paper mainly studies four kinds of entropy. First, we obtained four definitions of entropy from the literature, and gave the function of fuzzy entropy in different psychological states through strict mathematical proof. Then, we construct a fuzzy portfolio entropy decision model based on the investor's psychological states, and compared it with the possibilistic mean-variance model. Then we presented a numerical example and compared the five different models established. By comparing the results, we find that: (a) The possibilistic mean-Shannon entropy model solves the problem of the possibility of excessive concentration in the possibilistic mean-variance model, but the dispersion is not enough. Conversely, the possibilistic mean-Yager entropy is over-emphasized due to the definition of its own function, such that it gave an investment pattern of equal weight distribution or approximate average distribution. (b) The results of possibilistic mean-proportional entropy can be said to be the middle status of the portfolios of possibilistic mean-Shannon entropy and possibilistic mean-Yager entropy. This portfolio not only achieves a certain rate of return, but also disperses the risk to some extent. (c) The lines of satisfaction for portfolios derived from different models are approximately U-shaped with the increase in return preference. (d) The possibilistic mean-Shannon entropy model tends to have the highest portfolio satisfaction with the same psychological state of the investor.

Quantification of Axial Abnormality Due to Cerebellar Ataxia with Inertial Measurements.

Cerebellar Ataxia (CA) leads to deficiencies in muscle movement and lack of coordination that is often manifested as gait and balance disabilities. Conventional CA clinical assessments are subjective, cumbersome and provide less insight into the functional capabilities of patients. This cross-sectional study investigates the use of wearable inertial sensors strategically positioned on the front-chest and upper-back locations during the Romberg and Trunk tests for objective assessment of human postural balance due to CA. The primary aim of this paper is to quantify the performance of postural stability of 34 patients diagnosed with CA and 22 healthy subjects as controls. Several forms of entropy descriptions were considered to uncover characteristics of movements intrinsic to CA. Indeed, correlation with clinical observation is vital in ascertaining the validity of the inertial measurements in addition to capturing unique features of movements not typically observed by the practicing clinician. Both of these aspects form an integral part of the underlying objective assessment scheme. Uncertainty in the velocity contained a significant level of information with respect to truncal instability and, based on an extensive clustering and discrimination analysis, fuzzy entropy was identified as an effective measure in characterising the underlying disability. Front-chest measurements demonstrated a strong correlation with clinical assessments while the upper-back measurements performed better in classifying the two cohorts, inferring that the standard clinical assessments are relatively influenced by the frontal observations. The Romberg test was confirmed to be an effective test of neurological diagnosis as well as a potential candidate for objective assessment resulting in a significant correlation with the clinical assessments. In contrast, the Trunk test is observed to be relatively less informative.

Fuzzy and Sample Entropies as Predictors of Patient Survival Using Short Ventricular Fibrillation Recordings during out of Hospital Cardiac Arrest.

Optimal defibrillation timing guided by ventricular fibrillation (VF) waveform analysis would contribute to improved survival of out-of-hospital cardiac arrest (OHCA) patients by minimizing myocardial damage caused by futile defibrillation shocks and minimizing interruptions to cardiopulmonary resuscitation. Recently, fuzzy entropy (FuzzyEn) tailored to jointly measure VF amplitude and regularity has been shown to be an efficient defibrillation success predictor. In this study, 734 shocks from 296 OHCA patients (50 survivors) were analyzed, and the embedding dimension (m) and matching tolerance (r) for FuzzyEn and sample entropy (SampEn) were adjusted to predict defibrillation success and patient survival. Entropies were significantly larger in successful shocks and in survivors, and when compared to the available methods, FuzzyEn presented the best prediction results, marginally outperforming SampEn. The sensitivity and specificity of FuzzyEn were 83.3% and 76.7% when predicting defibrillation success, and 83.7% and 73.5% for patient survival. Sensitivities and specificities were two points above those of the best available methods, and the prediction accuracy was kept even for VF intervals as short as 2s. These results suggest that FuzzyEn and SampEn may be promising tools for optimizing the defibrillation time and predicting patient survival in OHCA patients presenting VF.

Using complexity-entropy planes to detect Parkinson's disease from short segments of haemodynamic signals.

Objective. There is emerging evidence that analysing the entropy and complexity of biomedical signals can detect underlying changes in physiology which may be reflective of disease pathology. This approach can be used even when only short recordings of biomedical signals are available. This study aimed to determine whether entropy and complexity measures can detect differences between subjects with Parkinsons disease and healthy controls (HCs).Approach. A method based on a diagram of entropy versus complexity, named complexity-entropy plane, was used to re-analyse a dataset of cerebral haemodynamic signals from subjects with Parkinsons disease and HCs obtained under poikilocapnic conditions. A probability distribution for a set of ordinal patterns, designed to capture regularities in a time series, was computed from each signal under analysis. Four types of entropy and ten types of complexity measures were estimated from these distributions. Mean values of entropy and complexity were compared and their classification power was assessed by evaluating the best linear separator on the corresponding complexity-entropy planes.Main results. Few linear separators obtained significantly better classification, evaluated as the area under the receiver operating characteristic curve, than signal mean values. However, significant differences in both entropy and complexity were detected between the groups of participants.Significance. Measures of entropy and complexity were able to detect differences between healthy volunteers and subjects with Parkinson's disease, in poikilocapnic conditions, even though only short recordings were available for analysis. Further work is needed to refine this promising approach, and to help understand the findings in the context of specific pathophysiological changes.

Application of nonlinear methods to discriminate fractionated electrograms in paroxysmal versus persistent atrial fibrillation.

BACKGROUND AND OBJECTIVE: Complex fractionated atrial electrograms (CFAE) may contain information concerning the electrophysiological substrate of atrial fibrillation (AF); therefore they are of interest to guide catheter ablation treatment of AF. Electrogram signals are shaped by activation events, which are dynamical in nature. This makes it difficult to establish those signal properties that can provide insight into the ablation site location. Nonlinear measures may improve information. To test this hypothesis, we used nonlinear measures to analyze CFAE. METHODS: CFAE from several atrial sites, recorded for a duration of 16 s, were acquired from 10 patients with persistent and 9 patients with paroxysmal AF. These signals were appraised using non-overlapping windows of 1-, 2- and 4-s durations. The resulting data sets were analyzed with Recurrence Plots (RP) and Recurrence Quantification Analysis (RQA). The data was also quantified via entropy measures. RESULTS: RQA exhibited unique plots for persistent versus paroxysmal AF. Similar patterns were observed to be repeated throughout the RPs. Trends were consistent for signal segments of 1 and 2 s as well as 4 s in duration. This was suggestive that the underlying signal generation process is also repetitive, and that repetitiveness can be detected even in 1-s sequences. The results also showed that most entropy metrics exhibited higher measurement values (closer to equilibrium) for persistent AF data. It was also found that Determinism (DET), Trapping Time (TT), and Modified Multiscale Entropy (MMSE), extracted from signals that were acquired from locations at the posterior atrial free wall, are highly discriminative of persistent versus paroxysmal AF data. CONCLUSIONS: Short data sequences are sufficient to provide information to discern persistent versus paroxysmal AF data with a significant difference, and can be useful to detect repeating patterns of atrial activation.

A hybrid model for EEG-based gender recognition.

The gender recognition is an important research field to study evidence regarding some personal characteristics in the information and data society. However, some current traditional methods such as vision and sound have been exposed their own security weaknesses. Recently, biometric gender recognition based on Electroencephalography (EEG) signals has been widely used in information safety and medical fields. It is necessary to explore potential of using EEG to present a more robust and accurate result with larger training data based on sophisticated machine learning approaches. In this contribution, we present an automated gender recognition system by a hybrid model based on EEG data of resting state from twenty-eight subjects. These data are useful and handy to get insights into assessing the differences in personal gender. For achieving a good performance and a strong robustness, the system develops a hybrid model of combining random forest and logistic regression, and employs four common entropy measures to analyze the non-stationary EEG signals. Result also suggests that the recognition performance achieve an improved progress with an accuracy of 0.9982 and AUC of 0.9926 based on a nested tenfold cross-validation loop, implying that show a significant potential applicability of the proposed approach and is capable of recognizing personal gender.

Comparison of selected measures of gait stability derived from center of pressure displacement signal during single and dual-task treadmill walking.

Steady state gait dynamics has been examined using the measures of regularity, local dynamic stability, and variability. This study investigates the relationship between these measures under increasing cognitive loads. Participants walked on an instrumented treadmill at 1 m/s under walk only and two dual-task conditions. The secondary tasks were visuomotor cognitive games (VCG) of increasing difficulty level. The center of pressure displacement in the mediolateral direction (ML COP-D) and cognitive game performance were recorded for analysis. The following measures were calculated: (1) sample entropy (SampEn) and quantized dynamical entropy (QDE) of the ML COP-D, (2) short-term largest Lyapunov exponent (LLE) of the ML COP-D, and (3) variability of inter-stride spatio-temporal gait variables. Entropy and variability measures significantly increased from walk only to both dual-task conditions. Whereas, the short-term LLE increased only during the easy VCG task. No measure was sensitive to the difficulty level of the VCG tasks. The variability of heel strike positions in the mediolateral direction was positively correlated with SampEn and QDE. However, there were no significant correlations between the short-term LLE and either variability measures or entropy measures. These findings confirm that each of these measures is representative of a different aspect of human gait dynamics.

Preserved cardiac autonomic dynamics during sleep in subjects with spinal cord injuries.

BACKGROUND: Spinal cord injuries (SCI) are associated with altered cardiovascular autonomic control (CAC). Sleep is characterized by modifications of autonomic control across sleep stages; however, no data are available in SCI subjects on CAC during sleep. We aim to assess cardiac autonomic modulation during sleep in subjects with SCI. PATIENTS AND METHODS: 27 participants with a neurological and radiological diagnosis of cervical (Cerv, n = 12, ie, tetraplegic) and thoracic SCI (Thor, n = 15, ie, paraplegic) and healthy subjects (Controls) were enrolled. Overnight polysomnographic (PSG) recordings were obtained in all participants. Electrocardiography and respiration were extracted from PSG, divided into sleep stages [wakefulness (W), non-REM sleep (NREM) and REM] for assessment of CAC, using symbolic analysis (SA) and corrected conditional entropy (CCE). SA identified indices of sympathetic and parasympathetic modulation and CCE evaluated the degree of complexity of the heart period time series. RESULTS: SA revealed a reduction of sympathetic and predominant parasympathetic control during NREM compared to W and REM in SCI patients, independent of the level of the lesion, similar to the Controls. In all three groups, complexity of autonomic regulation was higher in NREM compared to W and REM. CONCLUSIONS: In subjects with SCI, cardiac autonomic control changed across sleep stages, with a reduction of sympathetic and an increase of parasympathetic modulation during NREM compared to W and REM, and a parallel increase of complexity during NREM, which was similar to the Controls. Cardiac autonomic dynamics during sleep are maintained in SCI, independent of the level of the lesion.