COATI: Multimodal Contrastive Pretraining for Representing and Traversing Chemical Space.

Creating a successful small molecule drug is a challenging multiparameter optimization problem in an effectively infinite space of possible molecules. Generative models have emerged as powerful tools for traversing data manifolds composed of images, sounds, and text and offer an opportunity to dramatically improve the drug discovery and design process. To create generative optimization methods that are more useful than brute-force molecular generation and filtering via virtual screening, we propose that four integrated features are necessary: large, quantitative data sets of molecular structure and activity, an invertible vector representation of realistic accessible molecules, smooth and differentiable regressors that quantify uncertainty, and algorithms to simultaneously optimize properties of interest. Over the course of 12 months, Terray Therapeutics has collected a data set of 2 billion quantitative binding measurements of small molecules to therapeutic targets, which directly motivates multiparameter generative optimization of molecules conditioned on these data. To this end, we present contrastive optimization for accelerated therapeutic inference (COATI), a pretrained, multimodal encoder-decoder model of druglike chemical space. COATI is constructed without any human biasing of features, using contrastive learning from text and 3D representations of molecules to allow for downstream use with structural models. We demonstrate that COATI possesses many of the desired properties of universal molecular embedding: fixed-dimension, invertibility, autoencoding, accurate regression, and low computation cost. Finally, we present a novel metadynamics algorithm for generative optimization using a small subset of our proprietary data collected for a model protein, carbonic anhydrase, designing molecules that satisfy the multiparameter optimization task of potency, solubility, and drug likeness. This work sets the stage for fully integrated generative molecular design and optimization for small molecules.

Compressing physics with an autoencoder: Creating an atomic species representation to improve machine learning models in the chemical sciences.

We define a vector quantity which corresponds to atomic species identity by compressing a set of physical properties with an autoencoder. This vector, referred to here as the elemental modes, provides many advantages in downstream machine learning tasks. Using the elemental modes directly as the feature vector, we trained a neural network to predict formation energies of elpasolites with improved accuracy over previous works on the same task. Combining the elemental modes with geometric features used in high-dimensional neural network potentials (HD-NNPs) solves many problems of scaling and efficiency in the development of such neural network potentials. Whereas similar models in the past have been limited to typically four atomic species (H, C, N, and O), our implementation does not scale in cost by adding more atomic species and allows us to train an HD-NNP model which treats molecules containing H, C, N, O, F, P, S, Cl, Se, Br, and I. Finally, we establish that our implementation allows us to define feature vectors for alchemical intermediate states in the HD-NNP model, which opens up new possibilities for performing alchemical free energy calculations on systems where bond breaking/forming is important.

On the border between localization and delocalization: tris(iminoxolene)titanium(iv).

The tris(aminophenol) ligand tris(4-methyl-2-(3',5'-di-tert-butyl-2'-hydroxyphenylamino)phenyl)amine, MeClampH6, reacts with Ti(OiPr)4 to give, after exposure to air, the dark purple, neutral, diamagnetic complex (MeClamp)Ti. The compound is six-coordinate, with an uncoordinated central nitrogen (Ti-N = 2.8274(12) Å), and contains titanium(iv) and a doubly oxidized ligand, formally a bis(iminosemiquinone)-mono(amidophenoxide). The compound is unsymmetrical in the solid state, though the three ligands are equivalent on the NMR timescale in solution. Ab initio calculations indicate that the ground state is a multiconfigurational singlet, with a low-lying multiconfigurational triplet state. Variable-temperature NMR measurements are consistent with a singlet-triplet gap of 1200 ± 70 cm-1, in good agreement with calculations. The distortion from threefold symmetry allows a low-lying, partially populated ligand-centered π nonbonding orbital to mix with largely occupied metal-ligand π bonding orbitals. The energetic accessibility of this distortion is inversely related to the strength of the metal-ligand π bonding interaction.

Metadynamics for training neural network model chemistries: A competitive assessment.

Neural network model chemistries (NNMCs) promise to facilitate the accurate exploration of chemical space and simulation of large reactive systems. One important path to improving these models is to add layers of physical detail, especially long-range forces. At short range, however, these models are data driven and data limited. Little is systematically known about how data should be sampled, and "test data" chosen randomly from some sampling techniques can provide poor information about generality. If the sampling method is narrow, "test error" can appear encouragingly tiny while the model fails catastrophically elsewhere. In this manuscript, we competitively evaluate two common sampling methods: molecular dynamics (MD), normal-mode sampling, and one uncommon alternative, Metadynamics (MetaMD), for preparing training geometries. We show that MD is an inefficient sampling method in the sense that additional samples do not improve generality. We also show that MetaMD is easily implemented in any NNMC software package with cost that scales linearly with the number of atoms in a sample molecule. MetaMD is a black-box way to ensure samples always reach out to new regions of chemical space, while remaining relevant to chemistry near kbT. It is a cheap tool to address the issue of generalization.

The TensorMol-0.1 model chemistry: a neural network augmented with long-range physics.

Traditional force fields cannot model chemical reactivity, and suffer from low generality without re-fitting. Neural network potentials promise to address these problems, offering energies and forces with near ab initio accuracy at low cost. However a data-driven approach is naturally inefficient for long-range interatomic forces that have simple physical formulas. In this manuscript we construct a hybrid model chemistry consisting of a nearsighted neural network potential with screened long-range electrostatic and van der Waals physics. This trained potential, simply dubbed "TensorMol-0.1", is offered in an open-source Python package capable of many of the simulation types commonly used to study chemistry: geometry optimizations, harmonic spectra, open or periodic molecular dynamics, Monte Carlo, and nudged elastic band calculations. We describe the robustness and speed of the package, demonstrating its millihartree accuracy and scalability to tens-of-thousands of atoms on ordinary laptops. We demonstrate the performance of the model by reproducing vibrational spectra, and simulating the molecular dynamics of a protein. Our comparisons with electronic structure theory and experimental data demonstrate that neural network molecular dynamics is poised to become an important tool for molecular simulation, lowering the resource barrier to simulating chemistry.

Origin of the Size-Dependent Stokes Shift in CsPbBr3 Perovskite Nanocrystals.

The origin of the size-dependent Stokes shift in CsPbBr3 nanocrystals (NCs) is explained for the first time. Stokes shifts range from 82 to 20 meV for NCs with effective edge lengths varying from ∼4 to 13 nm. We show that the Stokes shift is intrinsic to the NC electronic structure and does not arise from extrinsic effects such as residual ensemble size distributions, impurities, or solvent-related effects. The origin of the Stokes shift is elucidated via first-principles calculations. Corresponding theoretical modeling of the CsPbBr3 NC density of states and band structure reveals the existence of an intrinsic confined hole state 260 to 70 meV above the valence band edge state for NCs with edge lengths from ∼2 to 5 nm. A size-dependent Stokes shift is therefore predicted and is in quantitative agreement with the experimental data. Comparison between bulk and NC calculations shows that the confined hole state is exclusive to NCs. At a broader level, the distinction between absorbing and emitting states in CsPbBr3 is likely a general feature of other halide perovskite NCs and can be tuned via NC size to enhance applications involving these materials.

Accelerating Realtime TDDFT with Block-Orthogonalized Manby-Miller Embedding Theory.

Realtime time-dependent density-functional theory (RT-TDDFT) is one of the most practical techniques available to simulate electronic dynamics of molecules and materials. Promising applications of RT-TDDFT to study nonlinear spectra and energy transport demand simulations of large solvated systems over long time scales, which are computationally quite costly. In this paper, we apply an embedding technique developed for ground-state SCF methods by Manby and Miller to accelerate realtime TDDFT. We assess the accuracy and speed of these approximations by studying the absorption spectra of solvated and covalently split chromophores. Our embedding approach is also compared with less accurate, less costly QM/MM charge embeddings. We find that by mixing levels of detail the embedded mean-field theory scheme is a simple, accurate, and effective way to accelerate RT-TDDFT simulations.

Intrinsic Bond Energies from a Bonds-in-Molecules Neural Network.

Neural networks are being used to make new types of empirical chemical models as inexpensive as force fields, but with accuracy similar to the ab initio methods used to build them. In this work, we present a neural network that predicts the energies of molecules as a sum of intrinsic bond energies. The network learns the total energies of the popular GDB9 database to a competitive MAE of 0.94 kcal/mol on molecules outside of its training set, is naturally linearly scaling, and applicable to molecules consisting of thousands of bonds. More importantly, it gives chemical insight into the relative strengths of bonds as a function of their molecular environment, despite only being trained on total energy information. We show that the network makes predictions of relative bond strengths in good agreement with measured trends and human predictions. A Bonds-in-Molecules Neural Network (BIM-NN) learns heuristic relative bond strengths like expert synthetic chemists, and compares well with ab initio bond order measures such as NBO analysis.

Detection of electron tunneling across plasmonic nanoparticle-film junctions using nitrile vibrations.

The significant electric field enhancements that occur in plasmonic nanogap junctions are instrumental in boosting the performance of spectroscopy, optoelectronics and catalysis. Electron tunneling, associated with quantum effects in small junctions, is reported to limit the electric field enhancement. However, observing and quantitatively determining how tunneling alters the electric fields within small gaps is challenging due to the nanoscale dimensions and heterogeneity present experimentally. Here, we report the use of a nitrile probe placed in the nanoparticle-film gap junctions to demonstrate that the change in the nitrile stretching band associated with the vibrational Stark effect can be directly correlated with the local electric field environment modulated by gap size variations. The emergence of Stark shifts correlates with plasmon resonance shifts associated with electron tunneling across the gap junction. Time dependent changes in the nitrile band with extended illumination further support a build up of charge associated with optical rectification in the coupled plasmon system. Computational models agree with our experimental observations that the frequency shifts arise from a vibrational Stark effect. Large local electric fields associated with the smallest gap junctions give rise to significant Stark shifts. These results indicate that nitrile Stark probes can measure the local field strengths in plasmonic junctions and monitor the subtle changes in the local electric fields resulting from electron tunneling.

The many-body expansion combined with neural networks.

Fragmentation methods such as the many-body expansion (MBE) are a common strategy to model large systems by partitioning energies into a hierarchy of decreasingly significant contributions. The number of calculations required for chemical accuracy is still prohibitively expensive for the ab initio MBE to compete with force field approximations for applications beyond single-point energies. Alongside the MBE, empirical models of ab initio potential energy surfaces have improved, especially non-linear models based on neural networks (NNs) which can reproduce ab initio potential energy surfaces rapidly and accurately. Although they are fast, NNs suffer from their own curse of dimensionality; they must be trained on a representative sample of chemical space. In this paper we examine the synergy of the MBE and NN's and explore their complementarity. The MBE offers a systematic way to treat systems of arbitrary size while reducing the scaling problem of large systems. NN's reduce, by a factor in excess of 106, the computational overhead of the MBE and reproduce the accuracy of ab initio calculations without specialized force fields. We show that for a small molecule extended system like methanol, accuracy can be achieved with drastically different chemical embeddings. To assess this we test a new chemical embedding which can be inverted to predict molecules with desired properties. We also provide our open-source code for the neural network many-body expansion, Tensormol.

Cost-effective description of strong correlation: Efficient implementations of the perfect quadruples and perfect hextuples models.

Novel implementations based on dense tensor storage are presented for the singlet-reference perfect quadruples (PQ) [J. A. Parkhill et al., J. Chem. Phys. 130, 084101 (2009)] and perfect hextuples (PH) [J. A. Parkhill and M. Head-Gordon, J. Chem. Phys. 133, 024103 (2010)] models. The methods are obtained as block decompositions of conventional coupled-cluster theory that are exact for four electrons in four orbitals (PQ) and six electrons in six orbitals (PH), but that can also be applied to much larger systems. PQ and PH have storage requirements that scale as the square, and as the cube of the number of active electrons, respectively, and exhibit quartic scaling of the computational effort for large systems. Applications of the new implementations are presented for full-valence calculations on linear polyenes (CnHn+2), which highlight the excellent computational scaling of the present implementations that can routinely handle active spaces of hundreds of electrons. The accuracy of the models is studied in the π space of the polyenes, in hydrogen chains (H50), and in the π space of polyacene molecules. In all cases, the results compare favorably to density matrix renormalization group values. With the novel implementation of PQ, active spaces of 140 electrons in 140 orbitals can be solved in a matter of minutes on a single core workstation, and the relatively low polynomial scaling means that very large systems are also accessible using parallel computing.

Nonradiative Relaxation in Real-Time Electronic Dynamics OSCF2: Organolead Triiodide Perovskite.

We apply our recently developed nonequilibrium real-time time-dependent density functional theory (OSCF2) to investigate the transient spectrum and relaxation dynamics of the tetragonal structure of methylammonium lead triiodide perovskite (MAPbI3). We obtain an estimate of the interband relaxation kinetics and identify multiple ultrafast cooling channels for hot electrons and hot holes that largely corroborate the dual valence-dual conduction model. The computed relaxation rates and absorption spectra are in good agreement with the existing experimental data. We present the first ab initio simulations of the perovskite transient absorption (TA) spectrum, substantiating the assignment of induced bleaches and absorptions including a Pauli-bleach signal. This paper validates both OSCF2 as a good qualitative model of electronic dynamics, and the dominant interpretation of the TA spectrum of this material.

Black-Box, Real-Time Simulations of Transient Absorption Spectroscopy.

We introduce an atomistic, all-electron, black-box electronic structure code to simulate transient absorption (TA) spectra and apply it to simulate pyrazole and a GFP-chromophore derivative. The method is an application of OSCF2, our dissipative extension of time-dependent density functional theory. We compare our simulated spectra directly with recent ultrafast spectroscopic experiments. We identify features in the TA spectra to Pauli-blocking, which may be missed without a first-principles model. An important ingredient in this method is the stationary-TDDFT correction scheme recently put forward by Fischer, Govind, and Cramer that allows us to overcome a limitation of adiabatic TDDFT. We demonstrate that OSCF2 is able to reproduce the energies of bleaches and induced absorptions as well as the decay of the transient spectrum with only the molecular structure as input.

Kinetic Energy of Hydrocarbons as a Function of Electron Density and Convolutional Neural Networks.

We demonstrate a convolutional neural network trained to reproduce the Kohn-Sham kinetic energy of hydrocarbons from an input electron density. The output of the network is used as a nonlocal correction to conventional local and semilocal kinetic functionals. We show that this approximation qualitatively reproduces Kohn-Sham potential energy surfaces when used with conventional exchange correlation functionals. The density which minimizes the total energy given by the functional is examined in detail. We identify several avenues to improve on this exploratory work, by reducing numerical noise and changing the structure of our functional. Finally we examine the features in the density learned by the neural network to anticipate the prospects of generalizing these models.

Nonadiabatic Dynamics for Electrons at Second-Order: Real-Time TDDFT and OSCF2.

We develop a new model to simulate nonradiative relaxation and dephasing by combining real-time Hartree-Fock and density functional theory (DFT) with our recent open-systems theory of electronic dynamics. The approach has some key advantages: it has been systematically derived and properly relaxes noninteracting electrons to a Fermi-Dirac distribution. This paper combines the new dissipation theory with an atomistic, all-electron quantum chemistry code and an atom-centered model of the thermal environment. The environment is represented nonempirically and is dependent on molecular structure in a nonlocal way. A production quality, O(N(3)) closed-shell implementation of our theory applicable to realistic molecular systems is presented, including timing information. This scaling implies that the added cost of our nonadiabatic relaxation model, time-dependent open self-consistent field at second order (OSCF2), is computationally inexpensive, relative to adiabatic propagation of real-time time-dependent Hartree-Fock (TDHF) or time-dependent density functional theory (TDDFT). Details of the implementation and numerical algorithm, including factorization and efficiency, are discussed. We demonstrate that OSCF2 approaches the stationary self-consistent field (SCF) ground state when the gap is large relative to k(b)T. The code is used to calculate linear-response spectra including the effects of bath dynamics. Finally, we show how our theory of finite-temperature relaxation can be used to correct ground-state DFT calculations.

How electronic dynamics with Pauli exclusion produces Fermi-Dirac statistics.

It is important that any dynamics method approaches the correct population distribution at long times. In this paper, we derive a one-body reduced density matrix dynamics for electrons in energetic contact with a bath. We obtain a remarkable equation of motion which shows that in order to reach equilibrium properly, rates of electron transitions depend on the density matrix. Even though the bath drives the electrons towards a Boltzmann distribution, hole blocking factors in our equation of motion cause the electronic populations to relax to a Fermi-Dirac distribution. These factors are an old concept, but we show how they can be derived with a combination of time-dependent perturbation theory and the extended normal ordering of Mukherjee and Kutzelnigg for a general electronic state. The resulting non-equilibrium kinetic equations generalize the usual Redfield theory to many-electron systems, while ensuring that the orbital occupations remain between zero and one. In numerical applications of our equations, we show that relaxation rates of molecules are not constant because of the blocking effect. Other applications to model atomic chains are also presented which highlight the importance of treating both dephasing and relaxation. Finally, we show how the bath localizes the electron density matrix.

Feynman's clock, a new variational principle, and parallel-in-time quantum dynamics.

We introduce a discrete-time variational principle inspired by the quantum clock originally proposed by Feynman and use it to write down quantum evolution as a ground-state eigenvalue problem. The construction allows one to apply ground-state quantum many-body theory to quantum dynamics, extending the reach of many highly developed tools from this fertile research area. Moreover, this formalism naturally leads to an algorithm to parallelize quantum simulation over time. We draw an explicit connection between previously known time-dependent variational principles and the time-embedded variational principle presented. Sample calculations are presented, applying the idea to a hydrogen molecule and the spin degrees of freedom of a model inorganic compound, demonstrating the parallel speedup of our method as well as its flexibility in applying ground-state methodologies. Finally, we take advantage of the unique perspective of this variational principle to examine the error of basis approximations in quantum dynamics.

Force-field functor theory: classical force-fields which reproduce equilibrium quantum distributions.

Feynman and Hibbs were the first to variationally determine an effective potential whose associated classical canonical ensemble approximates the exact quantum partition function. We examine the existence of a map between the local potential and an effective classical potential which matches the exact quantum equilibrium density and partition function. The usefulness of such a mapping rests in its ability to readily improve Born-Oppenheimer potentials for use with classical sampling. We show that such a map is unique and must exist. To explore the feasibility of using this result to improve classical molecular mechanics, we numerically produce a map from a library of randomly generated one-dimensional potential/effective potential pairs then evaluate its performance on independent test problems. We also apply the map to simulate liquid para-hydrogen, finding that the resulting radial pair distribution functions agree well with path integral Monte Carlo simulations. The surprising accessibility and transferability of the technique suggest a quantitative route to adapting Born-Oppenheimer potentials, with a motivation similar in spirit to the powerful ideas and approximations of density functional theory.

A correlated-polaron electronic propagator: open electronic dynamics beyond the Born-Oppenheimer approximation.

In this work, we develop an approach to treat correlated many-electron dynamics, dressed by the presence of a finite-temperature harmonic bath. Our theory combines a small polaron transformation with the second-order time-convolutionless master equation and includes both electronic and system-bath correlations on equal footing. Our theory is based on the ab initio Hamiltonian, and is thus well-defined apart from any phenomenological choice of basis states or electronic system-bath coupling model. The equation-of-motion for the density matrix we derive includes non-markovian and non-perturbative bath effects and can be used to simulate environmentally broadened electronic spectra and dissipative dynamics, which are subjects of recent interest. The theory also goes beyond the adiabatic Born-Oppenheimer approximation, but with computational cost scaling such as the Born-Oppenheimer approach. Example propagations with a developmental code are performed, demonstrating the treatment of electron-correlation in absorption spectra, vibronic structure, and decay in an open system. An untransformed version of the theory is also presented to treat more general baths and larger systems.