Reweighted Alternating Direction Method of Multipliers for DNN weight pruning.

As Deep Neural Networks (DNNs) continue to grow in complexity and size, leading to a substantial computational burden, weight pruning techniques have emerged as an effective solution. This paper presents a novel method for dynamic regularization-based pruning, which incorporates the Alternating Direction Method of Multipliers (ADMM). Unlike conventional methods that employ simple and abrupt threshold processing, the proposed method introduces a reweighting mechanism to assign importance to the weights in DNNs. Compared to other ADMM-based methods, the new method not only achieves higher accuracy but also saves considerable time thanks to the reduced number of necessary hyperparameters. The method is evaluated on multiple architectures, including LeNet-5, ResNet-32, ResNet-56, and ResNet-50, using the MNIST, CIFAR-10, and ImageNet datasets, respectively. Experimental results demonstrate its superior performance in terms of compression ratios and accuracy compared to state-of-the-art pruning methods. In particular, on the LeNet-5 model for the MNIST dataset, it achieves compression ratios of 355.9× with a slight improvement in accuracy; on the ResNet-50 model trained with the ImageNet dataset, it achieves compression ratios of 4.24× without sacrificing accuracy.

A Multitask Network Robustness Analysis System Based on the Graph Isomorphism Network.

Despite various measures across different engineering and social systems, network robustness remains crucial for resisting random faults and malicious attacks. In this study, robustness refers to the ability of a network to maintain its functionality after a part of the network has failed. Existing methods assess network robustness using attack simulations, spectral measures, or deep neural networks (DNNs), which return a single metric as a result. Evaluating network robustness is technically challenging, while evaluating a single metric is practically insufficient. This article proposes a multitask analysis system based on the graph isomorphism network (GIN) model, abbreviated as GIN-MAS. First, a destruction-based robustness metric is formulated using the destruction threshold of the examined network. A multitask learning approach is taken to learn the network robustness metrics, including connectivity robustness, controllability robustness, destruction threshold, and the maximum number of connected components. Then, a five-layer GIN is constructed for evaluating the aforementioned four robustness metrics simultaneously. Finally, extensive experimental studies reveal that 1) GIN-MAS outperforms nine other methods, including three state-of-the-art convolutional neural network (CNN)-based robustness evaluators, with lower prediction errors for both known and unknown datasets from various directed and undirected, synthetic, and real-world networks; 2) the multitask learning scheme is not only capable of handling multiple tasks simultaneously but more importantly it enables the parameter and knowledge sharing across tasks, thus preventing overfitting and enhancing the performances; and 3) GIN-MAS performs multitasks significantly faster than other single-task evaluators. The excellent performance of GIN-MAS suggests that more powerful DNNs have great potentials for analyzing more complicated and comprehensive robustness evaluation tasks.

A chaotic spiking backpropagation approach to brain learning.

A highlight of the chaotic spiking backpropagation (CSBP) method, which is a powerful tool for directly training spiking neural networks and helps to understand the learning mechanisms of human brain.

Delay-driven phase transitions in an epidemic model on time-varying networks.

A complex networked system typically has a time-varying nature in interactions among its components, which is intrinsically complicated and therefore technically challenging for analysis and control. This paper investigates an epidemic process on a time-varying network with a time delay. First, an averaging theorem is established to approximate the delayed time-varying system using autonomous differential equations for the analysis of system evolution. On this basis, the critical time delay is determined, across which the endemic equilibrium becomes unstable and a phase transition to oscillation in time via Hopf bifurcation will appear. Then, numerical examples are examined, including a periodically time-varying network, a blinking network, and a quasi-periodically time-varying network, which are simulated to verify the theoretical results. Further, it is demonstrated that the existence of time delay can extend the network frequency range to generate Turing patterns, showing a facilitating effect on phase transitions.

An "Observable" horseshoe map.

In this article, a family of diffeomorphisms with growing horseshoes contained in global attracting regions is presented, where the dimension of the unstable direction can be any fixed integer and a growing horseshoe means that the number of the folds of the horseshoe is increasing as a parameter is varied. Moreover, it is demonstrated that the horseshoe-like attractors are observable for certain parameters.

Integrated Multiomics Reveals Silencing of has_circ_0006646 Promotes TRIM21-Mediated NCL Ubiquitination to Inhibit Hepatocellular Carcinoma Metastasis.

Recent studies suggest that circular RNA (circRNA)-mediated post-translational modification of RNA-binding proteins (RBP) plays a pivotal role in metastasis of hepatocellular carcinoma (HCC). However, the specific mechanism and potential clinical therapeutic significance remain vague. This study attempts to profile the regulatory networks of circRNA and RBP using a multi-omics approach. Has_circ_0006646 (circ0006646) is an unreported circRNA in HCC and is associated with a poor prognosis. Silencing of circ0006646 significantly hinders metastasis in vivo. Mechanistically, circ0006646 prevents the interaction between nucleolin (NCL) and the E3 ligase tripartite motif-containing 21 to reduce the proteasome-mediated degradation of NCL via K48-linked polyubiquitylation. Furthermore, the change of NCL expression is proven to affect the phosphorylation levels of multiple proteins and inhibit p53 translation. Moreover, patient-derived tumor xenograft and lentivirus injection, which is conducted to simulate clinical treatment confirmed the potential therapeutic value. Overall, this study describes the integrated multi-omics landscape of circRNA-mediated NCL ubiquitination degradation in HCC metastasis and provides a novel therapeutic target.

Heterogeneously Networked Evolutionary Games With Intergroup Conflicts.

Network games primarily explore the intricacies of individual interactions and adaptive strategies within a network. Building upon this framework, the present study delves into the modeling, analysis, and control of heterogeneously networked evolutionary games with intergroup conflicts heterogeneously networked evolutionary games with intergroup conflict (HNEG-IC), where attacking players possess area-monitoring capabilities with limited attacking power. To begin with, a mathematical model is introduced to capture intragroup strategy dynamics and intergroup conflicts of HNEGs-IC via the algebraic state space representationalgebraic state space representation (ASSR). A necessary and sufficient condition for achieving global cooperation of HNEGs-IC is established. Then, a criterion for verifying the κ -cooperation below a certain mortality is presented. Considering the HNEGs-IC with strategy feedback control, it is proven that the feedback control, subject to global cooperation, is robust to conflicts when the intersection of the strategy threshold set and the reachable set of the preset initial strategy profiles is empty. Finally, for verification and demonstration, the obtained results are applied to a simplified virtual game model of the NATO and the Warsaw Pact.

Distributed Discrete-Time Convex Optimization With Closed Convex Set Constraints: Linearly Convergent Algorithm Design.

The convergence rate and applicability to directed graphs with interaction topologies are two important features for practical applications of distributed optimization algorithms. In this article, a new kind of fast distributed discrete-time algorithms is developed for solving convex optimization problems with closed convex set constraints over directed interaction networks. Under the gradient tracking framework, two distributed algorithms are, respectively, designed over balanced and unbalanced graphs, where momentum terms and two time-scales are involved. Furthermore, it is demonstrated that the designed distributed algorithms attain linear speedup convergence rates provided that the momentum coefficients and the step size are appropriately selected. Finally, numerical simulations verify the effectiveness and the global accelerated effect of the designed algorithms.

Cooperative Time-Varying Formation Fuzzy Tracking Control of Multiple Heterogeneous Uncertain Marine Surface Vehicles With Actuator Failures.

This article addresses the cooperative time-varying formation fuzzy tracking control problem for a cluster of heterogeneous multiple marine surface vehicles subject to unknown nonlinearity and actuator failures. The proposed cooperative control scheme consists of two parts: 1) a distributed time-varying formation observer and 2) a decentralized adaptive fuzzy tracking controller. The distributed observer is designed to obtain a predefined time-varying formation pattern under a directed communication topology. Subsequently, based on the states of the distributed observer, a decentralized fuzzy tracking control law is developed using fuzzy-logic systems and the adaptive approach. Lyapunov functions are constructed to guarantee that the controlled marine vehicles attain the desired time-varying formation with asymptotical stability of tracking errors. Finally, simulation results are presented to validate the efficacy of the proposed control methodology.

Li-Yorke chaos of linear differential equations in a finite-dimensional space with a weak topology.

Li-Yorke chaos of linear differential equations in a finite-dimensional space with a weak topology is introduced. Based on this topology on the Euclidean space, a flow generated from a linear differential equation is proved to be Li-Yorke chaotic under certain conditions, which is in sharp contract to the well-known fact that linear differential equations cannot be chaotic in a finite-dimensional space with a strong topology.

Role of the immune system in liver transplantation and its implications for therapeutic interventions.

Liver transplantation (LT) stands as the gold standard for treating end-stage liver disease and hepatocellular carcinoma, yet postoperative complications continue to impact survival rates. The liver's unique immune system, governed by a microenvironment of diverse immune cells, is disrupted during processes like ischemia-reperfusion injury posttransplantation, leading to immune imbalance, inflammation, and subsequent complications. In the posttransplantation period, immune cells within the liver collaboratively foster a tolerant environment, crucial for immune tolerance and liver regeneration. While clinical trials exploring cell therapy for LT complications exist, a comprehensive summary is lacking. This review provides an insight into the intricacies of the liver's immune microenvironment, with a specific focus on macrophages and T cells as primary immune players. Delving into the immunological dynamics at different stages of LT, we explore the disruptions after LT and subsequent immune responses. Focusing on immune cell targeting for treating liver transplant complications, we provide a comprehensive summary of ongoing clinical trials in this domain, especially cell therapies. Furthermore, we offer innovative treatment strategies that leverage the opportunities and prospects identified in the therapeutic landscape. This review seeks to advance our understanding of LT immunology and steer the development of precise therapies for postoperative complications.

MEGA: Machine Learning-Enhanced Graph Analytics for Infodemic Risk Management.

The COVID-19 pandemic brought not only global devastation but also an unprecedented infodemic of false or misleading information that spread rapidly through online social networks. Network analysis plays a crucial role in the science of fact-checking by modeling and learning the risk of infodemics through statistical processes and computation on mega-sized graphs. This article proposes MEGA, Machine Learning-Enhanced Graph Analytics, a framework that combines feature engineering and graph neural networks to enhance the efficiency of learning performance involving massive graphs. Infodemic risk analysis is a unique application of the MEGA framework, which involves detecting spambots by counting triangle motifs and identifying influential spreaders by computing the distance centrality. The MEGA framework is evaluated using the COVID-19 pandemic Twitter dataset, demonstrating superior computational efficiency and classification accuracy.

Resilience of hybrid herbivore-plant-pollinator networks.

The concept of network resilience has gained increasing attention in the last few decades owing to its great potential in strengthening and maintaining complex systems. From network-based approaches, researchers have explored resilience of real ecological systems comprising diverse types of interactions, such as mutualism, antagonist, and predation, or mixtures of them. In this paper, we propose a dimension-reduction method for analyzing the resilience of hybrid herbivore-plant-pollinator networks. We qualitatively evaluate the contribution of species toward maintaining resilience of networked systems, as well as the distinct roles played by different categories of species. Our findings demonstrate that the strong contributors to network resilience within each category are more vulnerable to extinction. Notably, among the three types of species in consideration, plants exhibit a higher likelihood of extinction, compared to pollinators and herbivores.

Computing Persistent Homology by Spanning Trees and Critical Simplices.

Topological data analysis can extract effective information from higher-dimensional data. Its mathematical basis is persistent homology. The persistent homology can calculate topological features at different spatiotemporal scales of the dataset, that is, establishing the integrated taxonomic relation among points, lines, and simplices. Here, the simplicial network composed of all-order simplices in a simplicial complex is essential. Because the sequence of nested simplicial subnetworks can be regarded as a discrete Morse function from the simplicial network to real values, a method based on the concept of critical simplices can be developed by searching all-order spanning trees. Employing this new method, not only the Morse function values with the theoretical minimum number of critical simplices can be obtained, but also the Betti numbers and composition of all-order cavities in the simplicial network can be calculated quickly. Finally, this method is used to analyze some examples and compared with other methods, showing its effectiveness and feasibility.

Nonlinear strict distance and similarity measures for intuitionistic fuzzy sets with applications to pattern classification and medical diagnosis.

In this paper, we propose a new type of nonlinear strict distance and similarity measures for intuitionistic fuzzy sets (IFSs). Our proposed methods not only have good properties, but also improve the drawbacks proposed by Mahanta and Panda (Int J Intell Syst 36(2):615-627, 2021) in which, for example, their distance value of [Formula: see text] is always equal to the maximum value 1 for any intuitionistic fuzzy number [Formula: see text]. To resolve these problems in Mahanta and Panda (Int J Intell Syst 36(2):615-627, 2021), we establish a nonlinear parametric distance measure for IFSs and prove that it satisfies the axiomatic definition of strict intuitionistic fuzzy distances and preserves all advantages of distance measures. In particular, our proposed distance measure can effectively distinguish different IFSs with high hesitancy. Meanwhile, we obtain that the dual similarity measure and the induced entropy of our proposed distance measure satisfy the axiomatic definitions of strict intuitionistic fuzzy similarity measure and intuitionistic fuzzy entropy. Finally, we apply our proposed distance and similarity measures to pattern classification, decision making on the choice of a proper antivirus face mask for COVID-19, and medical diagnosis problems, to illustrate the effectiveness of the new methods.

Ranking cliques in higher-order complex networks.

Traditional network analysis focuses on the representation of complex systems with only pairwise interactions between nodes. However, the higher-order structure, which is beyond pairwise interactions, has a great influence on both network dynamics and function. Ranking cliques could help understand more emergent dynamical phenomena in large-scale complex networks with higher-order structures, regarding important issues, such as behavioral synchronization, dynamical evolution, and epidemic spreading. In this paper, motivated by multi-node interactions in a topological simplex, several higher-order centralities are proposed, namely, higher-order cycle (HOC) ratio, higher-order degree, higher-order H-index, and higher-order PageRank (HOP), to quantify and rank the importance of cliques. Experiments on both synthetic and real-world networks support that, compared with other traditional network metrics, the proposed higher-order centralities effectively reduce the dimension of a large-scale network and are more accurate in finding a set of vital nodes. Moreover, since the critical cliques ranked by the HOP and the HOC are scattered over a complex network, the HOP and the HOC outperform other metrics in ranking cliques that are vital in maintaining the network connectivity, thereby facilitating network dynamical synchronization and virus spread control in applications.

RGP: Neural Network Pruning Through Regular Graph With Edges Swapping.

Deep learning technology has found a promising application in lightweight model design, for which pruning is an effective means of achieving a large reduction in both model parameters and float points operations (FLOPs). The existing neural network pruning methods mostly start from the consideration of the importance of model parameters and design parameter evaluation metrics to perform parameter pruning iteratively. These methods were not studied from the perspective of network model topology, so they might be effective but not efficient, and they require completely different pruning for different datasets. In this article, we study the graph structure of the neural network and propose a regular graph pruning (RGP) method to perform a one-shot neural network pruning. Specifically, we first generate a regular graph and set its node-degree values to meet the preset pruning ratio. Then, we reduce the average shortest path-length (ASPL) of the graph by swapping edges to obtain the optimal edge distribution. Finally, we map the obtained graph to a neural network structure to realize pruning. Our experiments demonstrate that the ASPL of the graph is negatively correlated with the classification accuracy of the neural network and that RGP has a strong precision retention capability with high parameter reduction (more than 90%) and FLOPs reduction (more than 90%) (the code for quick use and reproduction is available at https://github.com/Holidays1999/Neural-Network-Pruning-through-its-RegularGraph-Structure).

Network Robustness Prediction: Influence of Training Data Distributions.

Network robustness refers to the ability of a network to continue its functioning against malicious attacks, which is critical for various natural and industrial networks. Network robustness can be quantitatively measured by a sequence of values that record the remaining functionality after a sequential node-or edge-removal attacks. Robustness evaluations are traditionally determined by attack simulations, which are computationally very time-consuming and sometimes practically infeasible. The convolutional neural network (CNN)-based prediction provides a cost-efficient approach to fast evaluating the network robustness. In this article, the prediction performances of the learning feature representation-based CNN (LFR-CNN) and PATCHY-SAN methods are compared through extensively empirical experiments. Specifically, three distributions of network size in the training data are investigated, including the uniform, Gaussian, and extra distributions. The relationship between the CNN input size and the dimension of the evaluated network is studied. Extensive experimental results reveal that compared to the training data of uniform distribution, the Gaussian and extra distributions can significantly improve both the prediction performance and the generalizability, for both LFR-CNN and PATCHY-SAN, and for various functionality robustness. The extension ability of LFR-CNN is significantly better than PATCHY-SAN, verified by extensive comparisons on predicting the robustness of unseen networks. In general, LFR-CNN outperforms PATCHY-SAN, and thus LFR-CNN is recommended over PATCHY-SAN. However, since both LFR-CNN and PATCHY-SAN have advantages for different scenarios, the optimal settings of the input size of CNN are recommended under different configurations.

Modeling the spread dynamics of multiple-variant coronavirus disease under public health interventions: A general framework.

The COVID-19 pandemic was caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), which is a single-stranded positive-stranded RNA virus with a high multi-directional mutation rate. Many new variants even have an immune-evading property, which means that some individuals with antibodies against one variant can be reinfected by other variants. As a result, the realistic is still suffering from new waves of COVID-19 by its new variants. How to control the transmission or even eradicate the COVID-19 pandemic remains a critical issue for the whole world. This work presents an epidemiological framework for mimicking the multi-directional mutation process of SARS-CoV-2 and the epidemic spread of COVID-19 under realistic scenarios considering multiple variants. The proposed framework is used to evaluate single and combined public health interventions, which include non-pharmaceutical interventions, pharmaceutical interventions, and vaccine interventions under the existence of multi-directional mutations of SARS-CoV-2. The results suggest that several combined intervention strategies give optimal results and are feasible, requiring only moderate levels of individual interventions. Furthermore, the results indicate that even if the mutation rate of SARS-CoV-2 decreased 100 times, the pandemic would still not be eradicated without appropriate public health interventions.

From Chaos to Pseudorandomness: A Case Study on the 2-D Coupled Map Lattice.

Applying the chaos theory for secure digital communications is promising and it is well acknowledged that in such applications the underlying chaotic systems should be carefully chosen. However, the requirements imposed on the chaotic systems are usually heuristic, without theoretic guarantee for the resultant communication scheme. Among all the primitives for secure communications, it is well accepted that (pseudo) random numbers are most essential. Taking the well-studied 2-D coupled map lattice (2D CML) as an example, this article performs a theoretical study toward pseudorandom number generation with the 2D CML. In so doing, an analytical expression of the Lyapunov exponent (LE) spectrum of the 2D CML is first derived. Using the LEs, one can configure system parameters to ensure the 2D CML only exhibits complex dynamic behavior, and then collect pseudorandom numbers from the system orbits. Moreover, based on the observation that least significant bit distributes more evenly in the (pseudo) random distribution, an extraction algorithm E is developed with the property that when applied to the orbits of the 2D CML, it can squeeze uniform bits. In implementation, if fixed-point arithmetic is used in binary format with a precision of z bits after the radix point, E can ensure that the deviation of the squeezed bits is bounded by 2-z . Further simulation results demonstrate that the new method not only guides the 2D CML model to exhibit complex dynamic behavior but also generates uniformly distributed independent bits with good efficiency. In particular, the squeezed pseudorandom bits can pass both NIST 800-22 and TestU01 test suites in various settings. This study thereby provides a theoretical basis for effectively applying the 2D CML to secure communications.