Towards generalizable Graph Contrastive Learning: An information theory perspective.

Graph Contrastive Learning (GCL) is increasingly employed in graph representation learning with the primary aim of learning node/graph representations from a predefined pretext task that can generalize to various downstream tasks. Meanwhile, the transition from a specific pretext task to diverse and unpredictable downstream tasks poses a significant challenge for GCL's generalization ability. Most existing GCL approaches maximize mutual information between two views derived from the original graph, either randomly or heuristically. However, the generalization ability of GCL and its theoretical principles are still less studied. In this paper, we introduce a novel metric GCL-GE, to quantify the generalization gap between predefined pretext and agnostic downstream tasks. Given the inherent intractability of GCL-GE, we leverage concepts from information theory to derive a mutual information upper bound that is independent of the downstream tasks, thus enabling the metric's optimization despite the variability in downstream tasks. Based on the theoretical insight, we propose InfoAdv, a GCL framework to directly enhance generalization by jointly optimizing GCL-GE and InfoMax. Extensive experiments validate the capability of InfoAdv to enhance performance across a wide variety of downstream tasks, demonstrating its effectiveness in improving the generalizability of GCL.

Location-aware convolutional neural networks for graph classification.

Graph patterns play a critical role in various graph classification tasks, e.g., chemical patterns often determine the properties of molecular graphs. Researchers devote themselves to adapting Convolutional Neural Networks (CNNs) to graph classification due to their powerful capability in pattern learning. The varying numbers of neighbor nodes and the lack of canonical order of nodes on graphs pose challenges in constructing receptive fields for CNNs. Existing methods generally follow a heuristic ranking-based framework, which constructs receptive fields by selecting a fixed number of nodes and dropping the others according to predetermined rules. However, such methods may lose important structure information through dropping nodes, and they also cannot learn task-oriented graph patterns. In this paper, we propose a Location learning-based Convolutional Neural Networks (LCNN) for graph classification. LCNN constructs receptive fields by learning the location of each node according to its embedding that contains structures and features information, then standard CNNs are applied to capture graph patterns. Such a location learning mechanism not only retains the information of all nodes, but also provides the ability for task-oriented pattern learning. Experimental results show the effectiveness of the proposed LCNN, and visualization results further illustrate the valid pattern learning ability of our method for graph classification.

CRFs based de-identification of medical records.

De-identification is a shared task of the 2014 i2b2/UTHealth challenge. The purpose of this task is to remove protected health information (PHI) from medical records. In this paper, we propose a novel de-identifier, WI-deId, based on conditional random fields (CRFs). A preprocessing module, which tokenizes the medical records using regular expressions and an off-the-shelf tokenizer, is introduced, and three groups of features are extracted to train the de-identifier model. The experiment shows that our system is effective in the de-identification of medical records, achieving a micro-F1 of 0.9232 at the i2b2 strict entity evaluation level.