De novo design of protein interactions with learned surface fingerprints.

Physical interactions between proteins are essential for most biological processes governing life1. However, the molecular determinants of such interactions have been challenging to understand, even as genomic, proteomic and structural data increase. This knowledge gap has been a major obstacle for the comprehensive understanding of cellular protein-protein interaction networks and for the de novo design of protein binders that are crucial for synthetic biology and translational applications2-9. Here we use a geometric deep-learning framework operating on protein surfaces that generates fingerprints to describe geometric and chemical features that are critical to drive protein-protein interactions10. We hypothesized that these fingerprints capture the key aspects of molecular recognition that represent a new paradigm in the computational design of novel protein interactions. As a proof of principle, we computationally designed several de novo protein binders to engage four protein targets: SARS-CoV-2 spike, PD-1, PD-L1 and CTLA-4. Several designs were experimentally optimized, whereas others were generated purely in silico, reaching nanomolar affinity with structural and mutational characterization showing highly accurate predictions. Overall, our surface-centric approach captures the physical and chemical determinants of molecular recognition, enabling an approach for the de novo design of protein interactions and, more broadly, of artificial proteins with function.

Message-Passing Monte Carlo: Generating low-discrepancy point sets via graph neural networks.

Discrepancy is a well-known measure for the irregularity of the distribution of a point set. Point sets with small discrepancy are called low discrepancy and are known to efficiently fill the space in a uniform manner. Low-discrepancy points play a central role in many problems in science and engineering, including numerical integration, computer vision, machine perception, computer graphics, machine learning, and simulation. In this work, we present a machine learning approach to generate a new class of low-discrepancy point sets named Message-Passing Monte Carlo (MPMC) points. Motivated by the geometric nature of generating low-discrepancy point sets, we leverage tools from Geometric Deep Learning and base our model on graph neural networks. We further provide an extension of our framework to higher dimensions, which flexibly allows the generation of custom-made points that emphasize the uniformity in specific dimensions that are primarily important for the particular problem at hand. Finally, we demonstrate that our proposed model achieves state-of-the-art performance superior to previous methods by a significant margin. In fact, MPMC points are empirically shown to be either optimal or near-optimal with respect to the discrepancy for low dimension and small number of points, i.e., for which the optimal discrepancy can be determined.

Utilizing graph machine learning within drug discovery and development.

Graph machine learning (GML) is receiving growing interest within the pharmaceutical and biotechnology industries for its ability to model biomolecular structures, the functional relationships between them, and integrate multi-omic datasets - amongst other data types. Herein, we present a multidisciplinary academic-industrial review of the topic within the context of drug discovery and development. After introducing key terms and modelling approaches, we move chronologically through the drug development pipeline to identify and summarize work incorporating: target identification, design of small molecules and biologics, and drug repurposing. Whilst the field is still emerging, key milestones including repurposed drugs entering in vivo studies, suggest GML will become a modelling framework of choice within biomedical machine learning.

Social interpretation inflexibility moderates emotional reactions to social situations in children and adolescents.

Interpretation biases and inflexibility (i.e., difficulties revising interpretations) have been linked to increased internalizing symptoms. Although adolescence is a developmental period characterized by novel social situations and increased vulnerability to internalizing disorders, no studies have examined interpretation inflexibility in adolescents. Additionally, no studies (on adolescents or adults) have examined interpretation flexibility as a protective factor against adverse outcomes of interpersonal events. Using a novel task and a 28-day diary we examined relations among interpretation bias and inflexibility, internalizing symptoms, and negative interpersonal events in a sample of children and adolescents (N = 159, ages 9-18). At baseline, negative interpretation bias was positively correlated with social anxiety symptoms, and positive interpretation bias negatively correlated with social anxiety and depressive symptoms. Inflexible positive interpretations were correlated with higher social anxiety and depressive symptoms, while inflexible negative interpretations were correlated with higher social anxiety. Finally, interpretation inflexibility moderated daily associations between negative interpersonal events and depressive symptoms in daily life, such that higher inflexibility was associated with stronger associations between interpersonal events and subsequent depressive symptoms, potentially increasing depressive symptom instability. These results suggest that interpretation biases and inflexibility may act as both risk and protective factors for adolescent anxiety and depression.

Emotion Regulation in the Classroom: A Network Approach to Model Relations among Emotion Regulation Difficulties, Engagement to Learn, and Relationships with Peers and Teachers.

Emotion regulation is theorized to shape students' engagement in learning activities, but the specific pathways via which this occurs remain unclear. This study examined how emotion regulation mechanisms are related to behavioral and emotional engagement as well as relations with peers and teachers. The sample included 136 secondary school students (59,7% girls; Mage = 14.93, SDage = 1.02, range: 13-18 years). Psychometric network models revealed that difficulties in emotional awareness, emotional clarity, and access to emotion regulation strategies were differentially related to behavioral and emotional engagement, establishing an indirect link with teacher and/or peer relations. Nonacceptance of emotional responses, emotional awareness, and impulse control difficulties were uniquely related to teacher and/or peer relations, establishing an indirect link with student engagement. Causal discovery analysis suggested that student emotional engagement is an empirically-plausible direct cause of increased access to emotion regulation strategies. These findings uncover potential pathways through which emotion regulation hampers or facilitates learning at school, providing information useful for the design of school curricula and teacher training programs.

Predicting anticancer hyperfoods with graph convolutional networks.

BACKGROUND: Recent efforts in the field of nutritional science have allowed the discovery of disease-beating molecules within foods based on the commonality of bioactive food molecules to FDA-approved drugs. The pioneering work in this field used an unsupervised network propagation algorithm to learn the systemic-wide effect on the human interactome of 1962 FDA-approved drugs and a supervised algorithm to predict anticancer therapeutics using the learned representations. Then, a set of bioactive molecules within foods was fed into the model, which predicted molecules with cancer-beating potential.The employed methodology consisted of disjoint unsupervised feature generation and classification tasks, which can result in sub-optimal learned drug representations with respect to the classification task. Additionally, due to the disjoint nature of the tasks, the employed approach proved cumbersome to optimize, requiring testing of thousands of hyperparameter combinations and significant computational resources.To overcome the technical limitations highlighted above, we represent each drug as a graph (human interactome) with its targets as binary node features on the graph and formulate the problem as a graph classification task. To solve this task, inspired by the success of graph neural networks in graph classification problems, we use an end-to-end graph neural network model operating directly on the graphs, which learns drug representations to optimize model performance in the prediction of anticancer therapeutics. RESULTS: The proposed model outperforms the baseline approach in the anticancer therapeutic prediction task, achieving an F1 score of 67.99%±2.52% and an AUPR of 73.91%±3.49%. It is also shown that the model is able to capture knowledge of biological pathways to predict anticancer molecules based on the molecules' effects on cancer-related pathways. CONCLUSIONS: We introduce an end-to-end graph convolutional model to predict cancer-beating molecules within food. The introduced model outperforms the existing baseline approach, and shows interpretability, paving the way to the future of a personalized nutritional science approach allowing the development of nutrition strategies for cancer prevention and/or therapeutics.

Network machine learning maps phytochemically rich "Hyperfoods" to fight COVID-19.

In this paper, we introduce a network machine learning method to identify potential bioactive anti-COVID-19 molecules in foods based on their capacity to target the SARS-CoV-2-host gene-gene (protein-protein) interactome. Our analyses were performed using a supercomputing DreamLab App platform, harnessing the idle computational power of thousands of smartphones. Machine learning models were initially calibrated by demonstrating that the proposed method can predict anti-COVID-19 candidates among experimental and clinically approved drugs (5658 in total) targeting COVID-19 interactomics with the balanced classification accuracy of 80-85% in 5-fold cross-validated settings. This identified the most promising drug candidates that can be potentially "repurposed" against COVID-19 including common drugs used to combat cardiovascular and metabolic disorders, such as simvastatin, atorvastatin and metformin. A database of 7694 bioactive food-based molecules was run through the calibrated machine learning algorithm, which identified 52 biologically active molecules, from varied chemical classes, including flavonoids, terpenoids, coumarins and indoles predicted to target SARS-CoV-2-host interactome networks. This in turn was used to construct a "food map" with the theoretical anti-COVID-19 potential of each ingredient estimated based on the diversity and relative levels of candidate compounds with antiviral properties. We expect this in silico predicted food map to play an important role in future clinical studies of precision nutrition interventions against COVID-19 and other viral diseases.

Biased and inflexible interpretations of ambiguous social situations: Associations with eating disorder symptoms and socioemotional functioning.

BACKGROUND: Research indicates that difficulties across multiple socioemotional functioning domains (e.g., social emotion expression/regulation, response to social elicitors of emotion) and negatively biased interpretations of ambiguous social situations may affect eating disorder symptoms. The impact of inflexible interpretations of social situations on eating disorder symptoms is less clear. The present study therefore examined relations between inflexible and biased social interpretations, socioemotional functioning, and eating disorder symptoms. METHOD: A total of 310 participants from the general population, recruited from an online crowdsourcing platform, completed measures of socioemotional functioning (e.g., rejection sensitivity, negative social exchange), eating disorder symptoms, and positive and negative interpretation bias and inflexibility on a single measurement occasion. RESULTS: Socioemotional functioning impairments (Pillai's trace = 0.11, p < .001), but not negative (ß = .07, p = .162) or positive (ß = -.01, p = .804) interpretation bias or inflexible interpretations (ß = .04, p = .446), were associated with eating disorder symptoms in multiple regression models. In network analyses controlling statistically for multiple markers of socioemotional functioning, eating disorder symptoms were directly associated with negative (but not positive) interpretation bias. Inflexible interpretations were indirectly linked to symptoms via co-dampening of positive emotions. Exploratory causal discovery analyses suggested that several socioemotional functioning variables (social anxiety, depression, negative social exchange) may cause eating disorder symptoms. CONCLUSIONS: Consistent with cognitive-interpersonal models of disordered eating, our results suggest that less accurate (biased, inflexible) interpretations of social information contribute to patterns of cognition (anxious anticipation of rejection) and emotion regulation (down-regulation of positive social emotion) thought to encourage disordered eating. PUBLIC SIGNIFICANCE: This study suggests that less accurate interpretations of ambiguous social information encourage anxious anticipation of rejection and downregulation of positive social emotions, both of which are thought to promote eating disorder symptoms. Knowledge provided by this study about the likely relations between interpretive processes, social/emotional functioning, and eating disorder symptoms may help inform treatments for eating disorders, particularly those that attempt to modify patterns of interpretation.

Automated landmarking for palatal shape analysis using geometric deep learning.

OBJECTIVES: To develop and evaluate a geometric deep-learning network to automatically place seven palatal landmarks on digitized maxillary dental casts. SETTINGS AND SAMPLE POPULATION: The sample comprised individuals with permanent dentition of various ethnicities. The network was trained from manual landmark annotations on 732 dental casts and evaluated on 104 dental casts. MATERIALS AND METHODS: A geometric deep-learning network was developed to hierarchically learn features from point-clouds representing the 3D surface of each cast. These features predict the locations of seven palatal landmarks. RESULTS: Repeat-measurement reliability was <0.3 mm for all landmarks on all casts. Accuracy is promising. The proportion of test subjects with errors less than 2 mm was between 0.93 and 0.68, depending on the landmark. Unusually shaped and large palates generate the highest errors. There was no evidence for a difference in mean palatal shape estimated from manual compared to the automatic landmarking. The automatic landmarking reduces sample variation around the mean and reduces measurements of palatal size. CONCLUSIONS: The automatic landmarking method shows excellent repeatability and promising accuracy, which can streamline patient assessment and research studies. However, landmark indications should be subject to visual quality control.

Exploring palatal and dental shape variation with 3D shape analysis and geometric deep learning.

OBJECTIVES: Palatal shape contains a lot of information that is of clinical interest. Moreover, palatal shape analysis can be used to guide or evaluate orthodontic treatments. A statistical shape model (SSM) is a tool that, by means of dimensionality reduction, aims at compactly modeling the variance of complex shapes for efficient analysis. In this report, we evaluate several competing approaches to constructing SSMs for the human palate. SETTING AND SAMPLE POPULATION: This study used a sample comprising digitized 3D maxillary dental casts from 1,324 individuals. MATERIALS AND METHODS: Principal component analysis (PCA) and autoencoders (AE) are popular approaches to construct SSMs. PCA is a dimension reduction technique that provides a compact description of shapes by uncorrelated variables. AEs are situated in the field of deep learning and provide a non-linear framework for dimension reduction. This work introduces the singular autoencoder (SAE), a hybrid approach that combines the most important properties of PCA and AEs. We assess the performance of the SAE using standard evaluation tools for SSMs, including accuracy, generalization, and specificity. RESULTS: We found that the SAE obtains equivalent results to PCA and AEs for all evaluation metrics. SAE scores were found to be uncorrelated and provided an optimally compact representation of the shapes. CONCLUSION: We conclude that the SAE is a promising tool for 3D palatal shape analysis, which effectively combines the power of PCA with the flexibility of deep learning. This opens future AI driven applications of shape analysis in orthodontics and other related clinical disciplines.

Mistiming of thought and perception predicts delusionality.

The timing of thoughts and perceptions plays an essential role in belief formation. Just as people can experience in-the-moment perceptual illusions, however, they can also be deceived about how events unfold in time. Here, we consider how a particular type of temporal distortion, in which the apparent future influences "earlier" events in conscious awareness, might affect people's most fundamental beliefs about themselves and the world. Making use of a task that has been shown to elicit such reversals in the temporal experience of prediction and observation, we find that people who are more prone to think that they predicted an event that they actually already observed are also more likely to report holding delusion-like beliefs. Moreover, this relationship appears to be specific to how people experience prediction and is not explained by domain-general deficits in temporal discrimination. These findings may help uncover low-level perceptual mechanisms underlying delusional belief or schizotypy more broadly and may ultimately prove useful as a tool for identifying those at risk for psychotic illness.

Identifying psychological predictors of SARS-CoV-2 vaccination: A machine learning study.

BACKGROUND: Major barriers to addressing SARS-CoV-2 vaccine hesitancy include limited knowledge of what causes delay/refusal of SARS-CoV-2 vaccination and limited ability to predict who will remain unvaccinated over significant time periods despite vaccine availability. The present study begins to address these barriers by developing a machine learning model that prospectively predicts who will persist in not vaccinating against SARS-CoV-2. METHOD: Unvaccinated individuals (n = 325) who completed a baseline survey were followed over the six-month period when vaccines against SARS-CoV-2 were first widely available (April-October 2021). A random forest model was used to predict who would remain unvaccinated against SARS-CoV-2 from their baseline measures, including demographic information (e.g., age), medical history (e.g., of influenza vaccination), Health-Belief Model constructs (e.g., perceived vaccine dangerousness), conspiracist ideation, and task-based metrics of vulnerability to conspiracist ideation (e.g., tendency toward illusory pattern perception). RESULTS: The resulting model significantly predicted vaccination status (AUC-PR = 0.77, 95%CI [0.56 0.90]). At the optimal probability threshold determined by the Generalized Threshold Shifting Protocol, the model was moderately precise (0.83) when identifying individuals who remained unvaccinated (n = 80), and had a very low rate (0.04) of false-positives (incorrectly suggesting that individuals remained unvaccinated). Permutational importance tests suggested that baseline SARS-CoV-2 vaccine intentions conveyed the most information about future SARS-CoV-2 vaccination status. Conspiracist ideation was the second most informative predictor, suggesting that misinformation influences vaccination behavior. Other important predictors included perceived vaccine dangerousness, as expected under the Health Belief Model, and influenza vaccination history. CONCLUSIONS: The model we developed can accurately and prospectively identify individuals who remain unvaccinated against SARS-CoV-2. It could therefore facilitate empirically-informed allocation of interventions that encourage vaccine uptake. The predictive value of conspiracist ideation, perceived vaccine dangerousness, and vaccine intentions in our model is consistent with potential causal relations between these variables and SARS-CoV-2 vaccine uptake.

Combinatorial prediction of therapeutic perturbations using causally-inspired neural networks.

As an alternative to target-driven drug discovery, phenotype-driven approaches identify compounds that counteract the overall disease effects by analyzing phenotypic signatures. Our study introduces a novel approach to this field, aiming to expand the search space for new therapeutic agents. We introduce PDGrapher, a causally-inspired graph neural network (GNN) designed to predict combinatorial perturbagens - sets of therapeutic targets - capable of reversing disease effects. Unlike methods that learn responses to perturbations, PDGrapher solves the inverse problem, which is to infer the perturbagens necessary to achieve a specific response - i.e., directly predicting perturbagens by learning which perturbations elicit a desired response. By encoding gene regulatory networks or protein-protein interactions, PDGrapher can predict unseen chemical or genetic perturbagens, aiding in the discovery of novel drugs or therapeutic targets. Experiments across nine cell lines with chemical perturbations show that PDGrapher successfully predicted effective perturbagens in up to 13.33% additional test samples and ranked therapeutic targets up to 35% higher than the competing methods, and the method shows competitive performance across ten genetic perturbation datasets. A key innovation of PDGrapher is its direct prediction capability, which contrasts with the indirect, computationally intensive models traditionally used in phenotype-driven drug discovery that only predict changes in phenotypes due to perturbations. The direct approach enables PDGrapher to train up to 25 times faster than methods like scGEN and CellOT, representing a considerable leap in efficiency. Our results suggest that PDGrapher can advance phenotype-driven drug discovery, offering a fast and comprehensive approach to identifying therapeutically useful perturbations.

Graph Kernel Neural Networks.

The convolution operator at the core of many modern neural architectures can effectively be seen as performing a dot product between an input matrix and a filter. While this is readily applicable to data such as images, which can be represented as regular grids in the Euclidean space, extending the convolution operator to work on graphs proves more challenging, due to their irregular structure. In this article, we propose to use graph kernels, i.e., kernel functions that compute an inner product on graphs, to extend the standard convolution operator to the graph domain. This allows us to define an entirely structural model that does not require computing the embedding of the input graph. Our architecture allows to plug-in any type of graph kernels and has the added benefit of providing some interpretability in terms of the structural masks that are learned during the training process, similar to what happens for convolutional masks in traditional convolutional neural networks (CNNs). We perform an extensive ablation study to investigate the model hyperparameters' impact and show that our model achieves competitive performance on standard graph classification and regression datasets.

Trajectories and predictors of response to social cognition training in people with schizophrenia: A proof-of-concept machine learning study.

BACKGROUND: Social cognition training (SCT) can improve social cognition deficits in schizophrenia. However, little is known about patterns of response to SCT or individual characteristics that predict response. METHODS: 76 adults with schizophrenia randomized to receive 8-12 weeks of remotely-delivered SCT were included in this analysis. Social cognition was measured with a composite of six assessments. Latent class growth analyses identified trajectories of social cognitive response to SCT. Random forest and logistic regression models were trained to predict membership in the trajectory group that showed improvement from baseline measures including symptoms, functioning, motivation, and cognition. RESULTS: Five trajectory groups were identified: Group 1 (29 %) began with slightly above average social cognition, and this ability significantly improved with SCT. Group 2 (9 %) had baseline social cognition approximately one standard deviation above the sample mean and did not improve with training. Groups 3 (18 %) and 4 (36 %) began with average to slightly below-average social cognition and showed non-significant trends toward improvement. Group 5 (8 %) began with social cognition approximately one standard deviation below the sample mean, and experienced significant deterioration in social cognition. The random forest model had the best performance, predicting Group 1 membership with an area under the curve of 0.73 (SD 0.24; 95 % CI [0.51-0.87]). CONCLUSIONS: Findings suggest that there are distinct patterns of response to SCT in schizophrenia and that those with slightly above average social cognition at baseline may be most likely to experience gains. Results may inform future research seeking to individualize SCT treatment for schizophrenia.

Exploring "dark-matter" protein folds using deep learning.

De novo protein design explores uncharted sequence and structure space to generate novel proteins not sampled by evolution. A main challenge in de novo design involves crafting "designable" structural templates to guide the sequence searches toward adopting target structures. We present a convolutional variational autoencoder that learns patterns of protein structure, dubbed Genesis. We coupled Genesis with trRosetta to design sequences for a set of protein folds and found that Genesis is capable of reconstructing native-like distance and angle distributions for five native folds and three novel, the so-called "dark-matter" folds as a demonstration of generalizability. We used a high-throughput assay to characterize the stability of the designs through protease resistance, obtaining encouraging success rates for folded proteins. Genesis enables exploration of the protein fold space within minutes, unrestricted by protein topologies. Our approach addresses the backbone designability problem, showing that small neural networks can efficiently learn structural patterns in proteins. A record of this paper's transparent peer review process is included in the supplemental information.

Improving Graph Neural Network Expressivity via Subgraph Isomorphism Counting.

While Graph Neural Networks (GNNs) have achieved remarkable results in a variety of applications, recent studies exposed important shortcomings in their ability to capture the structure of the underlying graph. It has been shown that the expressive power of standard GNNs is bounded by the Weisfeiler-Leman (WL) graph isomorphism test, from which they inherit proven limitations such as the inability to detect and count graph substructures. On the other hand, there is significant empirical evidence, e.g. in network science and bioinformatics, that substructures are often intimately related to downstream tasks. To this end, we propose "Graph Substructure Networks" (GSN), a topologically-aware message passing scheme based on substructure encoding. We theoretically analyse the expressive power of our architecture, showing that it is strictly more expressive than the WL test, and provide sufficient conditions for universality. Importantly, we do not attempt to adhere to the WL hierarchy; this allows us to retain multiple attractive properties of standard GNNs such as locality and linear network complexity, while being able to disambiguate even hard instances of graph isomorphism. We perform an extensive experimental evaluation on graph classification and regression tasks and obtain state-of-the-art results in diverse real-world settings including molecular graphs and social networks.

Differentiable Graph Module (DGM) for Graph Convolutional Networks.

Graph deep learning has recently emerged as a powerful ML concept allowing to generalize successful deep neural architectures to non-euclidean structured data. Such methods have shown promising results on a broad spectrum of applications ranging from social science, biomedicine, and particle physics to computer vision, graphics, and chemistry. One of the limitations of the majority of current graph neural network architectures is that they are often restricted to the transductive setting and rely on the assumption that the underlying graph is known and fixed. Often, this assumption is not true since the graph may be noisy, or partially and even completely unknown. In such cases, it would be helpful to infer the graph directly from the data, especially in inductive settings where some nodes were not present in the graph at training time. Furthermore, learning a graph may become an end in itself, as the inferred structure may provide complementary insights next to the downstream task. In this paper, we introduce Differentiable Graph Module (DGM), a learnable function that predicts edge probabilities in the graph which are optimal for the downstream task. DGM can be combined with convolutional graph neural network layers and trained in an end-to-end fashion. We provide an extensive evaluation of applications from the domains of healthcare (disease prediction), brain imaging (age prediction), computer graphics (3D point cloud segmentation), and computer vision (zero-shot learning). We show that our model provides a significant improvement over baselines both in transductive and inductive settings and achieves state-of-the-art results.

A new age in protein design empowered by deep learning.

The rapid progress in the field of deep learning has had a significant impact on protein design. Deep learning methods have recently produced a breakthrough in protein structure prediction, leading to the availability of high-quality models for millions of proteins. Along with novel architectures for generative modeling and sequence analysis, they have revolutionized the protein design field in the past few years remarkably by improving the accuracy and ability to identify novel protein sequences and structures. Deep neural networks can now learn and extract the fundamental features of protein structures, predict how they interact with other biomolecules, and have the potential to create new effective drugs for treating disease. As their applicability in protein design is rapidly growing, we review the recent developments and technology in deep learning methods and provide examples of their performance to generate novel functional proteins.