Improved Dual Correlation Reduction Network With Affinity Recovery.

Deep graph clustering, which aims to reveal the underlying graph structure and divide the nodes into different clusters without human annotations, is a fundamental yet challenging task. However, we observe that the existing methods suffer from the representation collapse problem and tend to encode samples with different classes into the same latent embedding. Consequently, the discriminative capability of nodes is limited, resulting in suboptimal clustering performance. To address this problem, we propose a novel deep graph clustering algorithm termed improved dual correlation reduction network (IDCRN) through improving the discriminative capability of samples. Specifically, by approximating the cross-view feature correlation matrix to an identity matrix, we reduce the redundancy between different dimensions of features, thus improving the discriminative capability of the latent space explicitly. Meanwhile, the cross-view sample correlation matrix is forced to approximate the designed clustering-refined adjacency matrix to guide the learned latent representation to recover the affinity matrix even across views, thus enhancing the discriminative capability of features implicitly. Moreover, we avoid the collapsed representation caused by the oversmoothing issue in graph convolutional networks (GCNs) through an introduced propagation regularization term, enabling IDCRN to capture the long-range information with the shallow network structure. Extensive experimental results on six benchmarks have demonstrated the effectiveness and efficiency of IDCRN compared with the existing state-of-the-art deep graph clustering algorithms. The code of IDCRN is released at https://github.com/yueliu1999/IDCRN. Besides, we share a collection of deep graph clustering, including papers, codes, and datasets at https://github.com/yueliu1999/Awesome-Deep-Graph-Clustering.

A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal.

Knowledge graph reasoning (KGR), aiming to deduce new facts from existing facts based on mined logic rules underlying knowledge graphs (KGs), has become a fast-growing research direction. It has been proven to significantly benefit the usage of KGs in many AI applications, such as question answering, recommendation systems, and etc. According to the graph types, existing KGR models can be roughly divided into three categories, i.e., static models, temporal models, and multi-modal models. Early works in this domain mainly focus on static KGR, and recent works try to leverage the temporal and multi-modal information, which are more practical and closer to real-world. However, no survey papers and open-source repositories comprehensively summarize and discuss models in this important direction. To fill the gap, we conduct a first survey for knowledge graph reasoning tracing from static to temporal and then to multi-modal KGs. Concretely, the models are reviewed based on bi-level taxonomy, i.e., top-level (graph types) and base-level (techniques and scenarios). Besides, the performances, as well as datasets, are summarized and presented. Moreover, we point out the challenges and potential opportunities to enlighten the readers. The corresponding open-source repository is shared on GitHub https://github.com/LIANGKE23/Awesome-Knowledge-Graph-Reasoning.

Self-Supervised Temporal Graph Learning With Temporal and Structural Intensity Alignment.

Temporal graph learning aims to generate high-quality representations for graph-based tasks with dynamic information, which has recently garnered increasing attention. In contrast to static graphs, temporal graphs are typically organized as node interaction sequences over continuous time rather than an adjacency matrix. Most temporal graph learning methods model current interactions by incorporating historical neighborhood. However, such methods only consider first-order temporal information while disregarding crucial high-order structural information, resulting in suboptimal performance. To address this issue, we propose a self-supervised method called S2T for temporal graph learning, which extracts both temporal and structural information to learn more informative node representations. Notably, the initial node representations combine first-order temporal and high-order structural information differently to calculate two conditional intensities. An alignment loss is then introduced to optimize the node representations, narrowing the gap between the two intensities and making them more informative. Concretely, in addition to modeling temporal information using historical neighbor sequences, we further consider structural knowledge at both local and global levels. At the local level, we generate structural intensity by aggregating features from high-order neighbor sequences. At the global level, a global representation is generated based on all nodes to adjust the structural intensity according to the active statuses on different nodes. Extensive experiments demonstrate that the proposed model S2T achieves at most 10.13% performance improvement compared with the state-of-the-art competitors on several datasets.

Revisiting Initializing Then Refining: An Incomplete and Missing Graph Imputation Network.

With the development of various applications, such as recommendation systems and social network analysis, graph data have been ubiquitous in the real world. However, graphs usually suffer from being absent during data collection due to copyright restrictions or privacy-protecting policies. The graph absence could be roughly grouped into attribute-incomplete and attribute-missing cases. Specifically, attribute-incomplete indicates that a portion of the attribute vectors of all nodes are incomplete, while attribute-missing indicates that all attribute vectors of partial nodes are missing. Although various graph imputation methods have been proposed, none of them is custom-designed for a common situation where both types of graph absence exist simultaneously. To fill this gap, we develop a novel graph imputation network termed revisiting initializing then refining (RITR), where both attribute-incomplete and attribute-missing samples are completed under the guidance of a novel initializing-then-refining imputation criterion. Specifically, to complete attribute-incomplete samples, we first initialize the incomplete attributes using Gaussian noise before network learning, and then introduce a structure-attribute consistency constraint to refine incomplete values by approximating a structure-attribute correlation matrix to a high-order structure matrix. To complete attribute-missing samples, we first adopt structure embeddings of attribute-missing samples as the embedding initialization, and then refine these initial values by adaptively aggregating the reliable information of attribute-incomplete samples according to a dynamic affinity structure. To the best of our knowledge, this newly designed method is the first end-to-end unsupervised framework dedicated to handling hybrid-absent graphs. Extensive experiments on six datasets have verified that our methods consistently outperform the existing state-of-the-art competitors. Our source code is available at https://github.com/WxTu/RITR.

Hierarchically Contrastive Hard Sample Mining for Graph Self-Supervised Pretraining.

Contrastive learning has recently emerged as a powerful technique for graph self-supervised pretraining (GSP). By maximizing the mutual information (MI) between a positive sample pair, the network is forced to extract discriminative information from graphs to generate high-quality sample representations. However, we observe that, in the process of MI maximization (Infomax), the existing contrastive GSP algorithms suffer from at least one of the following problems: 1) treat all samples equally during optimization and 2) fall into a single contrasting pattern within the graph. Consequently, the vast number of well-categorized samples overwhelms the representation learning process, and limited information is accumulated, thus deteriorating the learning capability of the network. To solve these issues, in this article, by fusing the information from different views and conducting hard sample mining in a hierarchically contrastive manner, we propose a novel GSP algorithm called hierarchically contrastive hard sample mining (HCHSM). The hierarchical property of this algorithm is manifested in two aspects. First, according to the results of multilevel MI estimation in different views, the MI-based hard sample selection (MHSS) module keeps filtering the easy nodes and drives the network to focus more on hard nodes. Second, to collect more comprehensive information for hard sample learning, we introduce a hierarchically contrastive scheme to sequentially force the learned node representations to involve multilevel intrinsic graph features. In this way, as the contrastive granularity goes finer, the complementary information from different levels can be uniformly encoded to boost the discrimination of hard samples and enhance the quality of the learned graph embedding. Extensive experiments on seven benchmark datasets indicate that the HCHSM performs better than other competitors on node classification and node clustering tasks. The source code of HCHSM is available at https://github.com/WxTu/HCHSM.

scDFC: A deep fusion clustering method for single-cell RNA-seq data.

Clustering methods have been widely used in single-cell RNA-seq data for investigating tumor heterogeneity. Since traditional clustering methods fail to capture the high-dimension methods, deep clustering methods have drawn increasing attention these years due to their promising strengths on the task. However, existing methods consider either the attribute information of each cell or the structure information between different cells. In other words, they cannot sufficiently make use of all of this information simultaneously. To this end, we propose a novel single-cell deep fusion clustering model, which contains two modules, i.e. an attributed feature clustering module and a structure-attention feature clustering module. More concretely, two elegantly designed autoencoders are built to handle both features regardless of their data types. Experiments have demonstrated the validity of the proposed approach, showing that it is efficient to fuse attributes, structure, and attention information on single-cell RNA-seq data. This work will be further beneficial for investigating cell subpopulations and tumor microenvironment. The Python implementation of our work is now freely available at https://github.com/DayuHuu/scDFC.

Simple Contrastive Graph Clustering.

Contrastive learning has recently attracted plenty of attention in deep graph clustering due to its promising performance. However, complicated data augmentations and time-consuming graph convolutional operations undermine the efficiency of these methods. To solve this problem, we propose a simple contrastive graph clustering (SCGC) algorithm to improve the existing methods from the perspectives of network architecture, data augmentation, and objective function. As to the architecture, our network includes two main parts, that is, preprocessing and network backbone. A simple low-pass denoising operation conducts neighbor information aggregation as an independent preprocessing, and only two multilayer perceptrons (MLPs) are included as the backbone. For data augmentation, instead of introducing complex operations over graphs, we construct two augmented views of the same vertex by designing parameter unshared Siamese encoders and perturbing the node embeddings directly. Finally, as to the objective function, to further improve the clustering performance, a novel cross-view structural consistency objective function is designed to enhance the discriminative capability of the learned network. Extensive experimental results on seven benchmark datasets validate our proposed algorithm's effectiveness and superiority. Significantly, our algorithm outperforms the recent contrastive deep clustering competitors with at least seven times speedup on average. The code of SCGC is released at SCGC. Besides, we share a collection of deep graph clustering, including papers, codes, and datasets at ADGC.

Deep Fusion Clustering Network With Reliable Structure Preservation.

Deep clustering, which can elegantly exploit data representation to seek a partition of the samples, has attracted intensive attention. Recently, combining auto-encoder (AE) with graph neural networks (GNNs) has accomplished excellent performance by introducing structural information implied among data in clustering tasks. However, we observe that there are some limitations of most existing works: 1) in practical graph datasets, there exist some noisy or inaccurate connections among nodes, which would confuse network learning and cause biased representations, thus leading to unsatisfied clustering performance; 2) lacking dynamic information fusion module to carefully combine and refine the node attributes and the graph structural information to learn more consistent representations; and 3) failing to exploit the two separated views' information for generating a more robust target distribution. To solve these problems, we propose a novel method termed deep fusion clustering network with reliable structure preservation (DFCN-RSP). Specifically, the random walk mechanism is introduced to boost the reliability of the original graph structure by measuring localized structure similarities among nodes. It can simultaneously filter out noisy connections and supplement reliable connections in the original graph. Moreover, we provide a transformer-based graph auto-encoder (TGAE) that can use a self-attention mechanism with the localized structure similarity information to fine-tune the fused topology structure among nodes layer by layer. Furthermore, we provide a dynamic cross-modality fusion strategy to combine the representations learned from both TGAE and AE. Also, we design a triplet self-supervision strategy and a target distribution generation measure to explore the cross-modality information. The experimental results on five public benchmark datasets reflect that DFCN-RSP is more competitive than the state-of-the-art deep clustering algorithms. The corresponding code is available at https://github.com/gongleii/DFCN-RSP.

Context-Integrated and Feature-Refined Network for Lightweight Object Parsing.

Semantic segmentation for lightweight object parsing is a very challenging task, because both accuracy and efficiency (e.g., execution speed, memory footprint or computational complexity) should all be taken into account. However, most previous works pay too much attention to one-sided perspective, either accuracy or speed, and ignore others, which poses a great limitation to actual demands of intelligent devices. To tackle this dilemma, we propose a novel lightweight architecture named Context-Integrated and Feature-Refined Network (CIFReNet). The core components of CIFReNet are the Long-skip Refinement Module (LRM) and the Multi-scale Context Integration Module (MCIM). The LRM is designed to ease the propagation of spatial information between low-level and high-level stages. Furthermore, channel attention mechanism is introduced into the process of long-skip learning to boost the quality of low-level feature refinement. Meanwhile, the MCIM consists of three cascaded Dense Semantic Pyramid (DSP) blocks with image-level features, which is presented to encode multiple context information and enlarge the field of view. Specifically, the proposed DSP block exploits a dense feature sampling strategy to enhance the information representations without significantly increasing the computation cost. Comprehensive experiments are conducted on three benchmark datasets for object parsing including Cityscapes, CamVid, and Helen. As indicated, the proposed method reaches a better trade-off between accuracy and efficiency compared with the other state-of-the-art methods.

High-performance supercapacitors based on hierarchically porous carbons with a three-dimensional conductive network structure.

Clews of polymer nanobelts (CsPNBs) have the advantages of inexpensive raw materials, simple synthesis and large output. Novel clews of carbon nanobelts (CsCNBs) have been successfully prepared by carbonizing CsPNBs and by KOH activation subsequently. From the optimized process, CsCNBs*4, with a specific surface area of 2291 m2 g-1 and a pore volume of up to 1.29 cm3 g-1, has been obtained. Fundamentally, the CsCNBs possess a three-dimensional conductive network structure, a hierarchically porous framework, and excellent hydrophilicity, which enable fast ion diffusion through channels and a large enough ion adsorption/desorption surface to improve electrochemical performance of supercapacitors. The product exhibits a high specific capacitance of 327.5 F g-1 at a current density of 0.5 A g-1 in a three-electrode system. The results also reveal a high-rate capacitance (72.2% capacitance retention at 500 mV s-1) and stable cycling lifetime (95% of initial capacitance after 15 000 cycles). Moreover, CsCNBs*4 provides a high energy density of 29.8 W h kg-1 at a power density of 345.4 W kg-1 in 1 M tetraethylammonium tetrafluoroborate/acetonitrile (TEABF4/AN) electrolyte. These inspiring results imply that this carbon material with a three-dimensional conductive network structure possesses excellent potential for energy storage.

A Universal Strategy To Prepare Sulfur-Containing Polymer Composites with Desired Morphologies for Lithium-Sulfur Batteries.

Lithium-sulfur (Li-S) batteries are probably the most promising candidates for the next-generation batteries owing to their high energy density. However, Li-S batteries face severe technical problems where the dissolution of intermediate polysulfides is the biggest problem because it leads to the degradation of the cathode and the lithium anode, and finally the fast capacity decay. Compared with the composites of elemental sulfur and other matrices, sulfur-containing polymers (SCPs) have strong chemical bonds to sulfur and therefore show low dissolution of polysulfides. Unfortunately, most SCPs have very low electron conductivity and their morphologies can hardly be controlled, which undoubtedly depress the battery performances of SCPs. To overcome these two weaknesses of SCPs, a new strategy was developed for preparing SCP composites with enhanced conductivity and desired morphologies. With this strategy, macroporous SCP composites were successfully prepared from hierarchical porous carbon. The composites displayed discharge/charge capacities up to 1218/1139, 949/922, and 796/785 mA h g-1 at the current rates of 5, 10, and 15 C, respectively. Considering the universality of this strategy and the numerous morphologies of carbon materials, this strategy opens many opportunities for making carbon/SCP composites with novel morphologies.

Facile Synthesis of Nitrogen and Oxygen Co-Doped Clews of Carbon Nanobelts for Supercapacitors with Excellent Rate Performance.

Facile synthesis of carbon materials with high heteroatom content, large specific surface area (SSA) and hierarchical porous structure is critical for energy storage applications. In this study, nitrogen and oxygen co-doped clews of carbon nanobelts (NCNBs) with hierarchical porous structures are successfully prepared by a carbonization and subsequent activation by using ladder polymer of hydroquinone and formaldehyde (LPHF) as the precursor and ammonia as the activating agent. The hierarchical porous structures and ultra-high SSA (up to 2994 m² g−1) can effectively facilitate the exchange and transportation of electrons and ions. Moreover, suitable heteroatom content is believed to modify the wettability of the carbon material. The as-prepared activated NCNBs-60 (the NCNBs activated by ammonia at 950 °C for 60 min) possess a high capacitance of 282 F g−1 at the current density of 0.25 A g−1, NCNBs-45 (the NCNBs are activated by ammonia at 950 °C for 45 min) and show an excellent capacity retention of 50.2% when the current density increase from 0.25 to 150 A g−1. Moreover, the NCNBs-45 electrode exhibits superior electrochemical stability with 96.2% capacity retention after 10,000 cycles at 5.0 A g−1. The newly prepared NCNBs thus show great potential in the field of energy storage.

Facile Synthesis of Ultrahigh-Surface-Area Hollow Carbon Nanospheres and their Application in Lithium-Sulfur Batteries.

Hollow carbon nanospheres (HCNs) with specific surface areas up to 2949 m2 g-1 and pore volume up to 2.9 cm3 g-1 were successfully synthesized from polyaniline-co-polypyrrole hollow nanospheres by carbonization and CO2 activation. The cavity diameter and wall thickness of HCNs can be easily controlled by activation time. Owing to their large inner cavity and enclosed structure, HCNs are desirable carriers for encapsulating sulfur. To better understand the effects of pore characteristics and sulfur contents on the performances of lithium-sulfur batteries, three composites of HCNs and sulfur are prepared and studied in detail. The composites of HCNs with moderate specific surface areas and suitable sulfur content present a better performance. The first discharge capacity of this composite reaches 1401 mAh g-1 at 0.2 C. Even after 200 cycles, the discharge capacity remains at 626 mAh g-1 .