Deep reinforcement learning in chemistry: A review.

Reinforcement learning (RL) has been applied to various domains in computational chemistry and has found wide-spread success. In this review, we first motivate the application of RL to chemistry and list some broad application domains, for example, molecule generation, geometry optimization, and retrosynthetic pathway search. We set up some of the formalism associated with reinforcement learning that should help the reader translate their chemistry problems into a form where RL can be used to solve them. We then discuss the solution formulations and algorithms proposed in recent literature for these problems, the advantages of one over the other, together with the necessary details of the RL algorithms they employ. This article should help the reader understand the state of RL applications in chemistry, learn about some relevant actively-researched open problems, gain insight into how RL can be used to approach them and hopefully inspire innovative RL applications in Chemistry.

Carbodicarbenes and Striking Redox Transitions of their Conjugate Acids: Influence of NHC versus CAAC as Donor Substituents.

Herein, a new type of carbodicarbene (CDC) comprising two different classes of carbenes is reported; NHC and CAAC as donor substituents and compare the molecular structure and coordination to Au(I)Cl to those of NHC-only and CAAC-only analogues. The conjugate acids of these three CDCs exhibit notable redox properties. Their reactions with [NO][SbF6 ] were investigated. The reduction of the conjugate acid of CAAC-only based CDC with KC8 results in the formation of hydrogen abstracted/eliminated products, which proceed through a neutral radical intermediate, detected by EPR spectroscopy. In contrast, the reduction of conjugate acids of NHC-only and NHC/CAAC based CDCs led to intermolecular reductive (reversible) carbon-carbon sigma bond formation. The resulting relatively elongated carbon-carbon sigma bonds were found to be readily oxidized. They were, thus, demonstrated to be potent reducing agents, underlining their potential utility as organic electron donors and n-dopants in organic semiconductor molecules.

Benchmark study on deep neural network potentials for small organic molecules.

There has been tremendous advancement in machine learning (ML) applications in computational chemistry, particularly in neural network potentials (NNP). NNPs can approximate potential energy surface (PES) as a high dimensional function by learning from existing reference data, thereby circumventing the need to solve the electronic Schrödinger equation explicitly. As a result, ML accelerates chemical space exploration and property prediction compared to quantum mechanical methods. Novel ML methods have the potential to provide efficient means for predicting the properties of molecules. However, this potential has been limited by the lack of standard comparative evaluations. In this work, we compare four selected models, that is, ANI, PhysNet, SchNet, and BAND-NN, developed to represent the PES of small organic molecules. We evaluate these models for their accuracy and transferability on two different test sets (i) Small organic molecules of up to eight-heavy atoms on which ANI and SchNet achieve root mean square error (RMSE) of 0.55 and 0.60 kcal/mol, respectively. (ii) On random selection of molecules from the GDB-11 database with 10-heavy atoms, ANI achieves RMSE of 1.17 kcal/mol and SchNet achieves RMSE of 1.89 kcal/mol. We examine their ability to produce smooth meaningful surface by performing PES scans for bond stretch, angle bend, and dihedral rotations on relatively large molecules to assess their possible application in molecular dynamics simulations. We also evaluate their performance for yielding minimum energy structures via geometry optimization using various minimization algorithms. All these models were also able to accurately differentiate different isomers of the same empirical formula C10H20 . ANI and PhysNet achieve an RMSE of 0.29 and 0.52 kcal/mol, respectively, on C10H20 isomers.

Tetra-Coordinated Boron-Functionalized Phenanthroimidazole-Based Zinc Salen as a Photocatalyst for the Cycloaddition of CO2 and Epoxides.

A unique B-N coordinated phenanthroimidazole-based zinc salen was synthesized. The zinc salen thus synthesized acts as a photocatalyst for the cycloaddition of carbon dioxide with terminal epoxides under ambient conditions. DFT study of the cycloaddition of carbon dioxide with terminal epoxide indicates the preference of the reaction pathway when photocatalyzed by zinc salen. We anticipate that this strategy will help to design new photocatalysts for CO2 fixation.

BiRDS - Binding Residue Detection from Protein Sequences Using Deep ResNets.

Protein-drug interactions play important roles in many biological processes and therapeutics. Predicting the binding sites of a protein helps to discover such interactions. New drugs can be designed to optimize these interactions, improving protein function. The tertiary structure of a protein decides the binding sites available to the drug molecule, but the determination of the 3D structure is slow and expensive. Conversely, the determination of the amino acid sequence is swift and economical. Although quick and accurate prediction of the binding site using just the sequence is challenging, the application of Deep Learning, which has been hugely successful in several biochemical tasks, makes it feasible. BiRDS is a Residual Neural Network that predicts the protein's most active binding site using sequence information. SC-PDB, an annotated database of druggable binding sites, is used for training the network. Multiple Sequence Alignments of the proteins in the database are generated using DeepMSA, and features such as Position-Specific Scoring Matrix, Secondary Structure, and Relative Solvent Accessibility are extracted. During training, a weighted binary cross-entropy loss function is used to counter the substantial imbalance in the two classes of binding and nonbinding residues. A novel test set SC6K is introduced to compare binding-site prediction methods. BiRDS achieves an AUROC score of 0.87, and the center of 25% of its predicted binding sites lie within 4 Å of the center of the actual binding site.

MolGPT: Molecular Generation Using a Transformer-Decoder Model.

Application of deep learning techniques for de novo generation of molecules, termed as inverse molecular design, has been gaining enormous traction in drug design. The representation of molecules in SMILES notation as a string of characters enables the usage of state of the art models in natural language processing, such as Transformers, for molecular design in general. Inspired by generative pre-training (GPT) models that have been shown to be successful in generating meaningful text, we train a transformer-decoder on the next token prediction task using masked self-attention for the generation of druglike molecules in this study. We show that our model, MolGPT, performs on par with other previously proposed modern machine learning frameworks for molecular generation in terms of generating valid, unique, and novel molecules. Furthermore, we demonstrate that the model can be trained conditionally to control multiple properties of the generated molecules. We also show that the model can be used to generate molecules with desired scaffolds as well as desired molecular properties by conditioning the generation on scaffold SMILES strings of desired scaffolds and property values. Using saliency maps, we highlight the interpretability of the generative process of the model.

DeepPocket: Ligand Binding Site Detection and Segmentation using 3D Convolutional Neural Networks.

A structure-based drug design pipeline involves the development of potential drug molecules or ligands that form stable complexes with a given receptor at its binding site. A prerequisite to this is finding druggable and functionally relevant binding sites on the 3D structure of the protein. Although several methods for detecting binding sites have been developed beforehand, a majority of them surprisingly fail in the identification and ranking of binding sites accurately. The rapid adoption and success of deep learning algorithms in various sections of structural biology beckons the usage of such algorithms for accurate binding site detection. As a combination of geometry based software and deep learning, we report a novel framework, DeepPocket that utilizes 3D convolutional neural networks for the rescoring of pockets identified by Fpocket and further segments these identified cavities on the protein surface. Apart from this, we also propose another data set SC6K containing protein structures submitted in the Protein Data Bank (PDB) from January 1st, 2018, until February 28th, 2020, for ligand binding site (LBS) detection. DeepPocket's results on various binding site data sets and SC6K highlight its better performance over current state-of-the-art methods and good generalization ability over novel structures.

Host metabolic reprogramming in response to SARS-CoV-2 infection: A systems biology approach.

Understanding the pathogenesis of SARS-CoV-2 is essential for developing effective treatment strategies. Viruses hijack the host metabolism to redirect the resources for their replication and survival. The influence of SARS-CoV-2 on host metabolism is yet to be fully understood. In this study, we analyzed the transcriptomic data obtained from different human respiratory cell lines and patient samples (nasopharyngeal swab, peripheral blood mononuclear cells, lung biopsy, bronchoalveolar lavage fluid) to understand metabolic alterations in response to SARS-CoV-2 infection. We explored the expression pattern of metabolic genes in the comprehensive genome-scale network model of human metabolism, Recon3D, to extract key metabolic genes, pathways, and reporter metabolites under each SARS-CoV-2-infected condition. A SARS-CoV-2 core metabolic interactome was constructed for network-based drug repurposing. Our analysis revealed the host-dependent dysregulation of glycolysis, mitochondrial metabolism, amino acid metabolism, nucleotide metabolism, glutathione metabolism, polyamine synthesis, and lipid metabolism. We observed different pro- and antiviral metabolic changes and generated hypotheses on how the host metabolism can be targeted for reducing viral titers and immunomodulation. These findings warrant further exploration with more samples and in vitro studies to test predictions.

Learning Atomic Interactions through Solvation Free Energy Prediction Using Graph Neural Networks.

Solvation free energy is a fundamental property that influences various chemical and biological processes, such as reaction rates, protein folding, drug binding, and bioavailability of drugs. In this work, we present a deep learning method based on graph networks to accurately predict solvation free energies of small organic molecules. The proposed model, comprising three phases, namely, message passing, interaction, and prediction, is able to predict solvation free energies in any generic organic solvent with a mean absolute error of 0.16 kcal/mol. In terms of accuracy, the current model outperforms all of the proposed machine learning-based models so far. The atomic interactions predicted in an unsupervised manner are able to explain the trends of free energies consistent with chemical wisdom. Further, the robustness of the machine learning-based model has been tested thoroughly, and its capability to interpret the predictions has been verified with several examples.

MoleGuLAR: Molecule Generation Using Reinforcement Learning with Alternating Rewards.

The design of new inhibitors for novel targets is a very important problem especially in the current scenario with the world being plagued by COVID-19. Conventional approaches such as high-throughput virtual screening require extensive combing through existing data sets in the hope of finding possible matches. In this study, we propose a computational strategy for de novo generation of molecules with high binding affinities to the specified target and other desirable properties for druglike molecules using reinforcement learning. A deep generative model built using a stack-augmented recurrent neural network initially trained to generate druglike molecules is optimized using reinforcement learning to start generating molecules with desirable properties like LogP, quantitative estimate of drug likeliness, topological polar surface area, and hydration free energy along with the binding affinity. For multiobjective optimization, we have devised a novel strategy in which the property being used to calculate the reward is changed periodically. In comparison to the conventional approach of taking a weighted sum of all rewards, this strategy shows an enhanced ability to generate a significantly higher number of molecules with desirable properties.

DART: deep learning enabled topological interaction model for energy prediction of metal clusters and its application in identifying unique low energy isomers.

Recently, machine learning (ML) has proven to yield fast and accurate predictions of chemical properties to accelerate the discovery of novel molecules and materials. The majority of the work is on organic molecules, and much more work needs to be done for inorganic molecules, especially clusters. In the present work, we introduce a simple topological atomic descriptor called TAD, which encodes chemical environment information of each atom in the cluster. TAD is a simple and interpretable descriptor where each value represents the atom count in three shells. We also introduce the DART deep learning enabled topological interaction model, which uses TAD as a feature vector to predict energies of metal clusters, in our case gallium clusters with sizes ranging from 31 to 70 atoms. The DART model is designed based on the principle that the energy is a function of atomic interactions and allows us to model these complex atomic interactions to predict the energy. We further introduce a new dataset called GNC_31-70, which comprises structures and DFT optimized energies of gallium clusters with sizes ranging from 31 to 70 atoms. We show how DART can be used to accelerate the process of identification of low energy structures without geometry optimization. Albeit using a topological descriptor, DART achieves a mean absolute error (MAE) of 3.59 kcal mol-1 (0.15 eV) on the test set. We also show that our model can distinguish core and surface atoms in the Ga-70 cluster, which the model has never encountered earlier. Finally, we demonstrate the transferability of the DART model by predicting energies for about 6k unseen configurations picked up from molecular dynamics (MD) data for three cluster sizes (46, 57, and 60) within seconds. The DART model was able to reduce the load on DFT optimizations while identifying unique low energy structures from MD data.

Multiscale Modeling of Wobble to Watson-Crick-Like Guanine-Uracil Tautomerization Pathways in RNA.

Energetically unfavorable Watson-Crick (WC)-like tautomeric forms of nucleobases are known to introduce spontaneous mutations, and contribute to replication, transcription, and translation errors. Recent NMR relaxation dispersion techniques were able to show that wobble (w) G•U mispair exists in equilibrium with the short-lived, low-population WC-like enolic tautomers. Presently, we have investigated the wG•U → WC-like enolic reaction pathway using various theoretical methods: quantum mechanics (QM), molecular dynamics (MD), and combined quantum mechanics/molecular mechanics (QM/MM). The previous studies on QM gas phase calculations were inconsistent with experimental data. We have also explored the environmental effects on the reaction energies by adding explicit water. While the QM-profile clearly becomes endoergic in the presence of water, the QM/MM-profile remains consistently endoergic in the presence and absence of water. Hence, by including microsolvation and QM/MM calculations, the experimental data can be explained. For the G•Uenol→ Genol•U pathway, the latter appears to be energetically more favorable throughout all computational models. This study can be considered as a benchmark of various computational models of wG•U to WC-like tautomerization pathways with and without the environmental effects, and may contribute on further studies of other mispairs as well.

Desolvation of Peptide Bond by O to S Substitution Impacts Protein Stability.

Amino acid side chains are key to fine-tuning the microenvironment polarity in proteins composed of polar amide bonds. Here, we report that substituting an oxygen atom of the backbone amide bond with sulfur atom desolvates the thioamide bond, thereby increasing its lipophilicity. The impact of such local desolvation by O to S substitution in proteins was tested by synthesizing thioamidated variants of Pin1 WW domain. We observe that a thioamide acts in synergy with nonpolar amino acid side chains to reduce the microenvironment polarity and increase protein stability by more than 14 °C. Through favorable van der Waals and hydrogen bonding interactions, this single atom substitution significantly stabilizes proteins without altering the amino acid sequence and structure of the native protein.

BAND NN: A Deep Learning Framework for Energy Prediction and Geometry Optimization of Organic Small Molecules.

Recent advances in artificial intelligence along with the development of large data sets of energies calculated using quantum mechanical (QM)/density functional theory (DFT) methods have enabled prediction of accurate molecular energies at reasonably low computational cost. However, machine learning models that have been reported so far require the atomic positions obtained from geometry optimizations using high-level QM/DFT methods as input in order to predict the energies and do not allow for geometry optimization. In this study, a transferable and molecule size-independent machine learning model bonds (B), angles (A), nonbonded (N) interactions, and dihedrals (D) neural network (BAND NN) based on a chemically intuitive representation inspired by molecular mechanics force fields is presented. The model predicts the atomization energies of equilibrium and nonequilibrium structures as sum of energy contributions from bonds (B), angles (A), nonbonds (N), and dihedrals (D) at remarkable accuracy. The robustness of the proposed model is further validated by calculations that span over the conformational, configurational, and reaction space. The transferability of this model on systems larger than the ones in the data set is demonstrated by performing calculations on selected large molecules. Importantly, employing the BAND NN model, it is possible to perform geometry optimizations starting from nonequilibrium structures along with predicting their energies. © 2019 Wiley Periodicals, Inc.

Deep learning enabled inorganic material generator.

Recent years have witnessed utilization of modern machine learning approaches for predicting the properties of materials using available datasets. However, to identify potential candidates for material discovery, one has to systematically scan through a large chemical space and subsequently calculate the properties of all such samples. On the other hand, generative methods are capable of efficiently sampling the chemical space and can generate molecules/materials with desired properties. In this study, we report a deep learning based inorganic material generator (DING) framework consisting of a generator module and a predictor module. The generator module is developed based on conditional variational autoencoders (CVAEs) and the predictor module consists of three deep neural networks trained for predicting the enthalpy of formation, volume per atom and energy per atom chosen to demonstrate the proposed method. The predictor and generator modules have been developed using a one-hot key representation of the material composition. A series of tests were done to examine the robustness of the predictor models, to demonstrate the continuity of the latent material space, and its ability to generate materials exhibiting target property values. The DING architecture proposed in this paper can be extended to other properties based on which the chemical space can be efficiently explored for interesting materials/molecules.

Transition between [R]- and [S]-stereoisomers without bond breaking.

The fifty-year old proposal of a nondissociative racemization reaction of a tetracoordinated tetrahedral center from one enantiomer to another via a planar transition state by Hoffmann and coworkers has been explored by many research groups over the past five decades. A number of stable molecules with planar tetracoordinated and higher-coordinated centers have been designed and experimentally realized; however, there has not been a single example of a molecular system that can possibly undergo such racemization. Here we show examples of molecular species that undergo inversion of stereochemistry around tetrahedral centers (Si, Al- and P+) either via a planar transition state or an intermediate state using quantum mechanical, ab initio quasi-classical dynamics calculations, and Born-Oppenheimer molecular dynamics (BOMD) simulations. This work is expected to provide potential leads for future studies on this fundamental phenomenon in chemistry.

Machine Learning for Accurate Force Calculations in Molecular Dynamics Simulations.

The computationally expensive nature of ab initio molecular dynamics simulations severely limits its ability to simulate large system sizes and long time scales, both of which are necessary to imitate experimental conditions. In this work, we explore an approach to make use of the data obtained using the quantum mechanical density functional theory (DFT) on small systems and use deep learning to subsequently simulate large systems by taking liquid argon as a test case. A suitable vector representation was chosen to represent the surrounding environment of each Ar atom, and a Δ-NetFF machine learning model, where the neural network was trained to predict the difference in resultant forces obtained by DFT and classical force fields, was introduced. Molecular dynamics simulations were then performed using forces from the neural network for various system sizes and time scales depending on the properties we calculated. A comparison of properties obtained from the classical force field and the neural network model was presented alongside available experimental data to validate the proposed method.

Gold-Palladium Nanocluster Catalysts for Homocoupling: Electronic Structure and Interface Dynamics.

The gold-palladium (Au-Pd) bimetallic nanocluster (NC) catalyst in colloidal phase performs the homocoupling reaction of various aryl chlorides (Ar-Cl) under ambient conditions. We have systematically investigated various aspects of the Au-Pd NC catalysts with respect to this homocoupling reaction by using density functional theory (DFT) calculations, genetic algorithm (GA) approaches, and molecular dynamics (MD) simulations. Our findings include the geometric and electronic structures of the Au-Pd NC, the reactive Pd sites on the NC surface, the electron-donating effects of surrounding polymer matrix, the reaction mechanism of homocoupling reaction and rate-determining step, the inverse halogen dependence of the reaction, and the solvation dynamics at interface region between NC and polymer matrix in aqueous solution.

Computational modeling of the catalytic mechanism of hydroxymethylbilane synthase.

Hydroxymethylbilane synthase (HMBS), the third enzyme in the heme biosynthesis pathway, catalyzes the formation of 1-hydroxymethylbilane (HMB) by a stepwise polymerization of four molecules of porphobilinogen (PBG) using the dipyrromethane (DPM) cofactor. The mechanism by which HMBS polymerizes four units of PBG has not been elucidated to date. In vitro and in silico studies on HMBS have suggested certain residues with catalytic importance, but their specific role in the catalysis is unclear. To understand the catalytic mechanism of HMBS, quantum mechanical (QM) calculations were performed on model systems obtained from the active site of the human HMBS enzyme. The addition of one molecule of PBG to the DPM cofactor is carried out in four steps: (1) protonation of the substrate, PBG; (2) deamination of PBG; (3) electrophilic addition of the deaminated substrate to the terminal pyrrole ring of the enzyme-bound DPM cofactor and (4) deprotonation of the carbon atom at the α-position of the second ring of DPM. Based on the energy profiles from the QM calculations on cluster models, R26 is proposed to be the best suitable proton donor to the PBG moiety, which aids in the deamination of the substrate. During the electrophilic addition step, the intermediate formed is stabilized by the carboxylate side chain of the D99 residue. In the final deprotonation step, an extra proton from the second ring of DPM is transferred to R26 via the carboxylate side chain of D99, thus completing one cycle of the catalytic mechanism. The residues in the cluster model seem to play an important role in obtaining accurate energy barriers. All the stationary points along the reaction pathway have been characterized using QM calculations. The rate limiting step for the complete mechanism is found to be the deamination of the PBG moiety. The results of this study provide a detailed understanding of the catalytic mechanism and would help design future studies aimed at modulating the activity of HMBS.

Role of Urea-Aromatic Stacking Interactions in Stabilizing the Aromatic Residues of the Protein in Urea-Induced Denatured State.

A delicate balance of different types of intramolecular interactions makes the folded states of proteins marginally more stable than the unfolded states. Experiments use thermal, chemical, or mechanical stress to perturb the folding equilibrium for examining protein stability and the protein folding process. Elucidation of the mechanism by which chemical denaturants unfold proteins is crucial; this study explores the nature of urea-aromatic interactions relevant in urea-assisted protein denaturation. Free energy profiles corresponding to the unfolding of Trp-cage miniprotein in the presence and absence of urea at three different temperatures demonstrate the distortion of the hydrophobic core to be a crucial step. Exposure of the Trp6 residue to the solvent is found to be favored in the presence of urea. Previous experiments showed that urea has a high affinity for aromatic groups of proteins. We show here that this is due to the remarkable ability of urea to form stacking and NH-π interactions with aromatic groups of proteins. Urea-nucleobase stacking interactions have been shown to be crucial in urea-assisted RNA unfolding. Examination of these interactions using microsecond-long unrestrained simulations shows that urea-aromatic stacking interactions are stabilizing and long lasting. Further MD simulations, thermodynamic integration, and quantum mechanical calculations on aromatic model systems reveal that such interactions are possible for all the aromatic amino acid side-chains. Finally, we validate the ubiquitous nature of urea-aromatic stacking interactions by analyzing experimental structures of urea transporters and proteins crystallized in the presence of urea or urea derivatives.