Empowering Simple Graph Convolutional Networks.

Many neural networks for graphs are based on the graph convolution (GC) operator, proposed more than a decade ago. Since then, many alternative definitions have been proposed, which tend to add complexity (and nonlinearity) to the model. Recently, however, a simplified GC operator, dubbed simple graph convolution (SGC), which aims to remove nonlinearities was proposed. Motivated by the good results reached by this simpler model, in this article we propose, analyze, and compare simple graph convolution operators of increasing complexity that rely on linear transformations or controlled nonlinearities, and that can be implemented in single-layer graph convolutional networks (GCNs). Their computational expressiveness is characterized as well. We show that the predictive performance of the proposed GC operators is competitive with the ones of other widely adopted models on the considered node classification benchmark datasets.

Multiresolution Reservoir Graph Neural Network.

Graph neural networks are receiving increasing attention as state-of-the-art methods to process graph-structured data. However, similar to other neural networks, they tend to suffer from a high computational cost to perform training. Reservoir computing (RC) is an effective way to define neural networks that are very efficient to train, often obtaining comparable predictive performance with respect to the fully trained counterparts. Different proposals of reservoir graph neural networks have been proposed in the literature. However, their predictive performances are still slightly below the ones of fully trained graph neural networks on many benchmark datasets, arguably because of the oversmoothing problem that arises when iterating over the graph structure in the reservoir computation. In this work, we aim to reduce this gap defining a multiresolution reservoir graph neural network (MRGNN) inspired by graph spectral filtering. Instead of iterating on the nonlinearity in the reservoir and using a shallow readout function, we aim to generate an explicit k -hop unsupervised graph representation amenable for further, possibly nonlinear, processing. Experiments on several datasets from various application areas show that our approach is extremely fast and it achieves in most of the cases comparable or even higher results with respect to state-of-the-art approaches.

Suppression of a SARS-CoV-2 outbreak in the Italian municipality of Vo'.

On 21 February 2020, a resident of the municipality of Vo', a small town near Padua (Italy), died of pneumonia due to severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection1. This was the first coronavirus disease 19 (COVID-19)-related death detected in Italy since the detection of SARS-CoV-2 in the Chinese city of Wuhan, Hubei province2. In response, the regional authorities imposed the lockdown of the whole municipality for 14 days3. Here we collected information on the demography, clinical presentation, hospitalization, contact network and the presence of SARS-CoV-2 infection in nasopharyngeal swabs for 85.9% and 71.5% of the population of Vo' at two consecutive time points. From the first survey, which was conducted around the time the town lockdown started, we found a prevalence of infection of 2.6% (95% confidence interval (CI): 2.1-3.3%). From the second survey, which was conducted at the end of the lockdown, we found a prevalence of 1.2% (95% CI: 0.8-1.8%). Notably, 42.5% (95% CI: 31.5-54.6%) of the confirmed SARS-CoV-2 infections detected across the two surveys were asymptomatic (that is, did not have symptoms at the time of swab testing and did not develop symptoms afterwards). The mean serial interval was 7.2 days (95% CI: 5.9-9.6). We found no statistically significant difference in the viral load of symptomatic versus asymptomatic infections (P = 0.62 and 0.74 for E and RdRp genes, respectively, exact Wilcoxon-Mann-Whitney test). This study sheds light on the frequency of asymptomatic SARS-CoV-2 infection, their infectivity (as measured by the viral load) and provides insights into its transmission dynamics and the efficacy of the implemented control measures.

Heterogeneous networks integration for disease-gene prioritization with node kernels.

MOTIVATION: The identification of disease-gene associations is a task of fundamental importance in human health research. A typical approach consists in first encoding large gene/protein relational datasets as networks due to the natural and intuitive property of graphs for representing objects' relationships and then utilizing graph-based techniques to prioritize genes for successive low-throughput validation assays. Since different types of interactions between genes yield distinct gene networks, there is the need to integrate different heterogeneous sources to improve the reliability of prioritization systems. RESULTS: We propose an approach based on three phases: first, we merge all sources in a single network, then we partition the integrated network according to edge density introducing a notion of edge type to distinguish the parts and finally, we employ a novel node kernel suitable for graphs with typed edges. We show how the node kernel can generate a large number of discriminative features that can be efficiently processed by linear regularized machine learning classifiers. We report state-of-the-art results on 12 disease-gene associations and on a time-stamped benchmark containing 42 newly discovered associations. AVAILABILITY AND IMPLEMENTATION: Source code: https://github.com/dinhinfotech/DiGI.git. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Learning With Kernels: A Local Rademacher Complexity-Based Analysis With Application to Graph Kernels.

When dealing with kernel methods, one has to decide which kernel and which values for the hyperparameters to use. Resampling techniques can address this issue but these procedures are time-consuming. This problem is particularly challenging when dealing with structured data, in particular with graphs, since several kernels for graph data have been proposed in literature, but no clear relationship among them in terms of learning properties is defined. In these cases, exhaustive search seems to be the only reasonable approach. Recently, the global Rademacher complexity (RC) and local Rademacher complexity (LRC), two powerful measures of the complexity of a hypothesis space, have shown to be suited for studying kernels properties. In particular, the LRC is able to bound the generalization error of an hypothesis chosen in a space by disregarding those ones which will not be taken into account by any learning procedure because of their high error. In this paper, we show a new approach to efficiently bound the RC of the space induced by a kernel, since its exact computation is an NP-Hard problem. Then we show for the first time that RC can be used to estimate the accuracy and expressivity of different graph kernels under different parameter configurations. The authors' claims are supported by experimental results on several real-world graph data sets.

Generative Kernels for Tree-Structured Data.

This paper presents a family of methods for the design of adaptive kernels for tree-structured data that exploits the summarization properties of hidden states of hidden Markov models for trees. We introduce a compact and discriminative feature space based on the concept of hidden states multisets and we discuss different approaches to estimate such hidden state encoding. We show how it can be used to build an efficient and general tree kernel based on Jaccard similarity. Furthermore, we derive an unsupervised convolutional generative kernel using a topology induced on the Markov states by a tree topographic mapping. This paper provides an extensive empirical assessment on a variety of structured data learning tasks, comparing the predictive accuracy and computational efficiency of state-of-the-art generative, adaptive, and syntactical tree kernels. The results show that the proposed generative approach has a good tradeoff between computational complexity and predictive performance, in particular when considering the soft matching introduced by the topographic mapping.

Scuba: scalable kernel-based gene prioritization.

BACKGROUND: The uncovering of genes linked to human diseases is a pressing challenge in molecular biology and precision medicine. This task is often hindered by the large number of candidate genes and by the heterogeneity of the available information. Computational methods for the prioritization of candidate genes can help to cope with these problems. In particular, kernel-based methods are a powerful resource for the integration of heterogeneous biological knowledge, however, their practical implementation is often precluded by their limited scalability. RESULTS: We propose Scuba, a scalable kernel-based method for gene prioritization. It implements a novel multiple kernel learning approach, based on a semi-supervised perspective and on the optimization of the margin distribution. Scuba is optimized to cope with strongly unbalanced settings where known disease genes are few and large scale predictions are required. Importantly, it is able to efficiently deal both with a large amount of candidate genes and with an arbitrary number of data sources. As a direct consequence of scalability, Scuba integrates also a new efficient strategy to select optimal kernel parameters for each data source. We performed cross-validation experiments and simulated a realistic usage setting, showing that Scuba outperforms a wide range of state-of-the-art methods. CONCLUSIONS: Scuba achieves state-of-the-art performance and has enhanced scalability compared to existing kernel-based approaches for genomic data. This method can be useful to prioritize candidate genes, particularly when their number is large or when input data is highly heterogeneous. The code is freely available at https://github.com/gzampieri/Scuba .

Tree-Based Kernel for Graphs With Continuous Attributes.

The availability of graph data with node attributes that can be either discrete or real-valued is constantly increasing. While existing Kernel methods are effective techniques for dealing with graphs having discrete node labels, their adaptation to nondiscrete or continuous node attributes has been limited, mainly for computational issues. Recently, a few kernels especially tailored for this domain, and that trade predictive performance for computational efficiency, have been proposed. In this brief, we propose a graph kernel for complex and continuous nodes' attributes, whose features are tree structures extracted from specific graph visits. The kernel manages to keep the same complexity of the state-of-the-art kernels while implicitly using a larger feature space. We further present an approximated variant of the kernel, which reduces its complexity significantly. Experimental results obtained on six real-world data sets show that the kernel is the best performing one on most of them. Moreover, in most cases, the approximated version reaches comparable performances to the current state-of-the-art kernels in terms of classification accuracy while greatly shortening the running times.

Learning Orthographic Structure With Sequential Generative Neural Networks.

Learning the structure of event sequences is a ubiquitous problem in cognition and particularly in language. One possible solution is to learn a probabilistic generative model of sequences that allows making predictions about upcoming events. Though appealing from a neurobiological standpoint, this approach is typically not pursued in connectionist modeling. Here, we investigated a sequential version of the restricted Boltzmann machine (RBM), a stochastic recurrent neural network that extracts high-order structure from sensory data through unsupervised generative learning and can encode contextual information in the form of internal, distributed representations. We assessed whether this type of network can extract the orthographic structure of English monosyllables by learning a generative model of the letter sequences forming a word training corpus. We show that the network learned an accurate probabilistic model of English graphotactics, which can be used to make predictions about the letter following a given context as well as to autonomously generate high-quality pseudowords. The model was compared to an extended version of simple recurrent networks, augmented with a stochastic process that allows autonomous generation of sequences, and to non-connectionist probabilistic models (n-grams and hidden Markov models). We conclude that sequential RBMs and stochastic simple recurrent networks are promising candidates for modeling cognition in the temporal domain.

An efficient topological distance-based tree kernel.

Tree kernels proposed in the literature rarely use information about the relative location of the substructures within a tree. As this type of information is orthogonal to the one commonly exploited by tree kernels, the two can be combined to enhance state-of-the-art accuracy of tree kernels. In this brief, our attention is focused on subtree kernels. We describe an efficient algorithm for injecting positional information into a tree kernel and present ways to enlarge its feature space without affecting its worst case complexity. The experimental results on several benchmark datasets are presented showing that our method is able to reach state-of-the-art performances, obtaining in some cases better performance than computationally more demanding tree kernels.

Compositional generative mapping for tree-structured data--part I: bottom-up probabilistic modeling of trees.

We introduce a novel compositional (recursive) probabilistic model for trees that defines an approximated bottom-up generative process from the leaves to the root of a tree. The proposed model defines contextual state transitions from the joint configuration of the children to the parent nodes. We argue that the bottom-up context postulates different probabilistic assumptions with respect to a top-down approach, leading to different representational capabilities. We discuss classes of applications that are best suited to a bottom-up approach. In particular, the bottom-up context is shown to better correlate and model the co-occurrence of substructures among the child subtrees of internal nodes. A mixed memory approximation is introduced to factorize the joint children-to-parent state transition matrix as a mixture of pairwise transitions. The proposed approach is the first practical bottom-up generative model for tree-structured data that maintains the same computational class of its top-down counterpart. Comparative experimental analyses exploiting synthetic and real-world datasets show that the proposed model can deal with deep structures better than a top-down generative model. The model is also shown to better capture structural information from real-world data comprising trees with a large out-degree. The proposed bottom-up model can be used as a fundamental building block for the development of other new powerful models.

Learning nonsparse kernels by self-organizing maps for structured data.

The development of neural network (NN) models able to encode structured input, and the more recent definition of kernels for structures, makes it possible to directly apply machine learning approaches to generic structured data. However, the effectiveness of a kernel can depend on its sparsity with respect to a specific data set. In fact, the accuracy of a kernel method typically reduces as the kernel sparsity increases. The sparsity problem is particularly common in structured domains involving discrete variables which may take on many different values. In this paper, we explore this issue on two well-known kernels for trees, and propose to face it by recurring to self-organizing maps (SOMs) for structures. Specifically, we show that a suitable combination of the two approaches, obtained by defining a new class of kernels based on the activation map of a SOM for structures, can be effective in avoiding the sparsity problem and results in a system that can be significantly more accurate for categorization tasks on structured data. The effectiveness of the proposed approach is demonstrated experimentally on two relatively large corpora of XML formatted data and a data set of user sessions extracted from website logs.

Combining selectivity and affinity predictions using an integrated Support Vector Machine (SVM) approach: An alternative tool to discriminate between the human adenosine A(2A) and A(3) receptor pyrazolo-triazolo-pyrimidine antagonists binding sites.

G Protein-coupled receptors (GPCRs) selectivity is an important aspect of drug discovery process, and distinguishing between related receptor subtypes is often the key to therapeutic success. Nowadays, very few valuable computational tools are available for the prediction of receptor subtypes selectivity. In the present study, we present an alternative application of the Support Vector Machine (SVM) and Support Vector Regression (SVR) methodologies to simultaneously describe both A(2A)R versus A(3)R subtypes selectivity profile and the corresponding receptor binding affinities. We have implemented an integrated application of SVM-SVR approach, based on the use of our recently reported autocorrelated molecular descriptors encoding for the Molecular Electrostatic Potential (autoMEP), to simultaneously discriminate A(2A)R versus A(3)R antagonists and to predict their binding affinity to the corresponding receptor subtype of a large dataset of known pyrazolo-triazolo-pyrimidine analogs. To validate our approach, we have synthetized 51 new pyrazolo-triazolo-pyrimidine derivatives anticipating both A(2A)R/A(3)R subtypes selectivity and receptor binding affinity profiles.

An introduction to recursive neural networks and kernel methods for cheminformatics.

The aim of this paper is to introduce the reader to new developments in Neural Networks and Kernel Machines concerning the treatment of structured domains. Specifically, we discuss the research on these relatively new models to introduce a novel and more general approach to QSPR/QSAR analysis. The focus is on the computational side and not on the experimental one.

Predicting physical-chemical properties of compounds from molecular structures by recursive neural networks.

In this paper, we report on the potential of a recently developed neural network for structures applied to the prediction of physical chemical properties of compounds. The proposed recursive neural network (RecNN) model is able to directly take as input a structured representation of the molecule and to model a direct and adaptive relationship between the molecular structure and target property. Therefore, it combines in a learning system the flexibility and general advantages of a neural network model with the representational power of a structured domain. As a result, a completely new approach to quantitative structure-activity relationship/quantitative structure-property relationship (QSPR/QSAR) analysis is obtained. An original representation of the molecular structures has been developed accounting for both the occurrence of specific atoms/groups and the topological relationships among them. Gibbs free energy of solvation in water, Delta(solv)G degrees , has been chosen as a benchmark for the model. The different approaches proposed in the literature for the prediction of this property have been reconsidered from a general perspective. The advantages of RecNN as a suitable tool for the automatization of fundamental parts of the QSPR/QSAR analysis have been highlighted. The RecNN model has been applied to the analysis of the Delta(solv)G degrees in water of 138 monofunctional acyclic organic compounds and tested on an external data set of 33 compounds. As a result of the statistical analysis, we obtained, for the predictive accuracy estimated on the test set, correlation coefficient R = 0.9985, standard deviation S = 0.68 kJ mol(-1), and mean absolute error MAE = 0.46 kJ mol(-1). The inherent ability of RecNN to abstract chemical knowledge through the adaptive learning process has been investigated by principal components analysis of the internal representations computed by the network. It has been found that the model recognizes the chemical compounds on the basis of a nontrivial combination of their chemical structure and target property.

The loading problem for recursive neural networks.

The present work deals with one of the major and not yet completely understood topics of supervised connectionist models. Namely, it investigates the relationships between the difficulty of a given learning task and the chosen neural network architecture. These relationships have been investigated and nicely established for some interesting problems in the case of neural networks used for processing vectors and sequences, but only a few studies have dealt with loading problems involving graphical inputs. In this paper, we present sufficient conditions which guarantee the absence of local minima of the error function in the case of learning directed acyclic graphs with recursive neural networks. We introduce topological indices which can be directly calculated from the given training set and that allows us to design the neural architecture with local minima free error function. In particular, we conceive a reduction algorithm that involves both the information attached to the nodes and the topology, which enlarges significantly the class of the problems with unimodal error function previously proposed in the literature.

Contextual processing of structured data by recursive cascade correlation.

This paper propose a first approach to deal with contextual information in structured domains by recursive neural networks. The proposed model, i.e., contextual recursive cascade correlation (CRCC), a generalization of the recursive cascade correlation (RCC) model, is able to partially remove the causality assumption by exploiting contextual information stored in frozen units. We formally characterize the properties of CRCC showing that it is able to compute contextual transductions and also some causal supersource transductions that RCC cannot compute. Experimental results on controlled sequences and on a real-world task involving chemical structures confirm the computational limitations of RCC, while assessing the efficiency and efficacy of CRCC in dealing both with pure causal and contextual prediction tasks. Moreover, results obtained for the real-world task show the superiority of the proposed approach versus RCC when exploring a task for which it is not known whether the structural causality assumption holds.

Recursive self-organizing network models.

Self-organizing models constitute valuable tools for data visualization, clustering, and data mining. Here, we focus on extensions of basic vector-based models by recursive computation in such a way that sequential and tree-structured data can be processed directly. The aim of this article is to give a unified review of important models recently proposed in literature, to investigate fundamental mathematical properties of these models, and to compare the approaches by experiments. We first review several models proposed in literature from a unifying perspective, thereby making use of an underlying general framework which also includes supervised recurrent and recursive models as special cases. We shortly discuss how the models can be related to different neuron lattices. Then, we investigate theoretical properties of the models in detail: we explicitly formalize how structures are internally stored in different context models and which similarity measures are induced by the recursive mapping onto the structures. We assess the representational capabilities of the models, and we shortly discuss the issues of topology preservation and noise tolerance. The models are compared in an experiment with time series data. Finally, we add an experiment for one context model for tree-structured data to demonstrate the capability to process complex structures.