COATI: Multimodal Contrastive Pretraining for Representing and Traversing Chemical Space.

Creating a successful small molecule drug is a challenging multiparameter optimization problem in an effectively infinite space of possible molecules. Generative models have emerged as powerful tools for traversing data manifolds composed of images, sounds, and text and offer an opportunity to dramatically improve the drug discovery and design process. To create generative optimization methods that are more useful than brute-force molecular generation and filtering via virtual screening, we propose that four integrated features are necessary: large, quantitative data sets of molecular structure and activity, an invertible vector representation of realistic accessible molecules, smooth and differentiable regressors that quantify uncertainty, and algorithms to simultaneously optimize properties of interest. Over the course of 12 months, Terray Therapeutics has collected a data set of 2 billion quantitative binding measurements of small molecules to therapeutic targets, which directly motivates multiparameter generative optimization of molecules conditioned on these data. To this end, we present contrastive optimization for accelerated therapeutic inference (COATI), a pretrained, multimodal encoder-decoder model of druglike chemical space. COATI is constructed without any human biasing of features, using contrastive learning from text and 3D representations of molecules to allow for downstream use with structural models. We demonstrate that COATI possesses many of the desired properties of universal molecular embedding: fixed-dimension, invertibility, autoencoding, accurate regression, and low computation cost. Finally, we present a novel metadynamics algorithm for generative optimization using a small subset of our proprietary data collected for a model protein, carbonic anhydrase, designing molecules that satisfy the multiparameter optimization task of potency, solubility, and drug likeness. This work sets the stage for fully integrated generative molecular design and optimization for small molecules.

Compressing physics with an autoencoder: Creating an atomic species representation to improve machine learning models in the chemical sciences.

We define a vector quantity which corresponds to atomic species identity by compressing a set of physical properties with an autoencoder. This vector, referred to here as the elemental modes, provides many advantages in downstream machine learning tasks. Using the elemental modes directly as the feature vector, we trained a neural network to predict formation energies of elpasolites with improved accuracy over previous works on the same task. Combining the elemental modes with geometric features used in high-dimensional neural network potentials (HD-NNPs) solves many problems of scaling and efficiency in the development of such neural network potentials. Whereas similar models in the past have been limited to typically four atomic species (H, C, N, and O), our implementation does not scale in cost by adding more atomic species and allows us to train an HD-NNP model which treats molecules containing H, C, N, O, F, P, S, Cl, Se, Br, and I. Finally, we establish that our implementation allows us to define feature vectors for alchemical intermediate states in the HD-NNP model, which opens up new possibilities for performing alchemical free energy calculations on systems where bond breaking/forming is important.

Metadynamics for training neural network model chemistries: A competitive assessment.

Neural network model chemistries (NNMCs) promise to facilitate the accurate exploration of chemical space and simulation of large reactive systems. One important path to improving these models is to add layers of physical detail, especially long-range forces. At short range, however, these models are data driven and data limited. Little is systematically known about how data should be sampled, and "test data" chosen randomly from some sampling techniques can provide poor information about generality. If the sampling method is narrow, "test error" can appear encouragingly tiny while the model fails catastrophically elsewhere. In this manuscript, we competitively evaluate two common sampling methods: molecular dynamics (MD), normal-mode sampling, and one uncommon alternative, Metadynamics (MetaMD), for preparing training geometries. We show that MD is an inefficient sampling method in the sense that additional samples do not improve generality. We also show that MetaMD is easily implemented in any NNMC software package with cost that scales linearly with the number of atoms in a sample molecule. MetaMD is a black-box way to ensure samples always reach out to new regions of chemical space, while remaining relevant to chemistry near kbT. It is a cheap tool to address the issue of generalization.

Origin of the Size-Dependent Stokes Shift in CsPbBr3 Perovskite Nanocrystals.

The origin of the size-dependent Stokes shift in CsPbBr3 nanocrystals (NCs) is explained for the first time. Stokes shifts range from 82 to 20 meV for NCs with effective edge lengths varying from ∼4 to 13 nm. We show that the Stokes shift is intrinsic to the NC electronic structure and does not arise from extrinsic effects such as residual ensemble size distributions, impurities, or solvent-related effects. The origin of the Stokes shift is elucidated via first-principles calculations. Corresponding theoretical modeling of the CsPbBr3 NC density of states and band structure reveals the existence of an intrinsic confined hole state 260 to 70 meV above the valence band edge state for NCs with edge lengths from ∼2 to 5 nm. A size-dependent Stokes shift is therefore predicted and is in quantitative agreement with the experimental data. Comparison between bulk and NC calculations shows that the confined hole state is exclusive to NCs. At a broader level, the distinction between absorbing and emitting states in CsPbBr3 is likely a general feature of other halide perovskite NCs and can be tuned via NC size to enhance applications involving these materials.

Detection of electron tunneling across plasmonic nanoparticle-film junctions using nitrile vibrations.

The significant electric field enhancements that occur in plasmonic nanogap junctions are instrumental in boosting the performance of spectroscopy, optoelectronics and catalysis. Electron tunneling, associated with quantum effects in small junctions, is reported to limit the electric field enhancement. However, observing and quantitatively determining how tunneling alters the electric fields within small gaps is challenging due to the nanoscale dimensions and heterogeneity present experimentally. Here, we report the use of a nitrile probe placed in the nanoparticle-film gap junctions to demonstrate that the change in the nitrile stretching band associated with the vibrational Stark effect can be directly correlated with the local electric field environment modulated by gap size variations. The emergence of Stark shifts correlates with plasmon resonance shifts associated with electron tunneling across the gap junction. Time dependent changes in the nitrile band with extended illumination further support a build up of charge associated with optical rectification in the coupled plasmon system. Computational models agree with our experimental observations that the frequency shifts arise from a vibrational Stark effect. Large local electric fields associated with the smallest gap junctions give rise to significant Stark shifts. These results indicate that nitrile Stark probes can measure the local field strengths in plasmonic junctions and monitor the subtle changes in the local electric fields resulting from electron tunneling.

The many-body expansion combined with neural networks.

Fragmentation methods such as the many-body expansion (MBE) are a common strategy to model large systems by partitioning energies into a hierarchy of decreasingly significant contributions. The number of calculations required for chemical accuracy is still prohibitively expensive for the ab initio MBE to compete with force field approximations for applications beyond single-point energies. Alongside the MBE, empirical models of ab initio potential energy surfaces have improved, especially non-linear models based on neural networks (NNs) which can reproduce ab initio potential energy surfaces rapidly and accurately. Although they are fast, NNs suffer from their own curse of dimensionality; they must be trained on a representative sample of chemical space. In this paper we examine the synergy of the MBE and NN's and explore their complementarity. The MBE offers a systematic way to treat systems of arbitrary size while reducing the scaling problem of large systems. NN's reduce, by a factor in excess of 106, the computational overhead of the MBE and reproduce the accuracy of ab initio calculations without specialized force fields. We show that for a small molecule extended system like methanol, accuracy can be achieved with drastically different chemical embeddings. To assess this we test a new chemical embedding which can be inverted to predict molecules with desired properties. We also provide our open-source code for the neural network many-body expansion, Tensormol.

Nonradiative Relaxation in Real-Time Electronic Dynamics OSCF2: Organolead Triiodide Perovskite.

We apply our recently developed nonequilibrium real-time time-dependent density functional theory (OSCF2) to investigate the transient spectrum and relaxation dynamics of the tetragonal structure of methylammonium lead triiodide perovskite (MAPbI3). We obtain an estimate of the interband relaxation kinetics and identify multiple ultrafast cooling channels for hot electrons and hot holes that largely corroborate the dual valence-dual conduction model. The computed relaxation rates and absorption spectra are in good agreement with the existing experimental data. We present the first ab initio simulations of the perovskite transient absorption (TA) spectrum, substantiating the assignment of induced bleaches and absorptions including a Pauli-bleach signal. This paper validates both OSCF2 as a good qualitative model of electronic dynamics, and the dominant interpretation of the TA spectrum of this material.

Cost-effective description of strong correlation: Efficient implementations of the perfect quadruples and perfect hextuples models.

Novel implementations based on dense tensor storage are presented for the singlet-reference perfect quadruples (PQ) [J. A. Parkhill et al., J. Chem. Phys. 130, 084101 (2009)] and perfect hextuples (PH) [J. A. Parkhill and M. Head-Gordon, J. Chem. Phys. 133, 024103 (2010)] models. The methods are obtained as block decompositions of conventional coupled-cluster theory that are exact for four electrons in four orbitals (PQ) and six electrons in six orbitals (PH), but that can also be applied to much larger systems. PQ and PH have storage requirements that scale as the square, and as the cube of the number of active electrons, respectively, and exhibit quartic scaling of the computational effort for large systems. Applications of the new implementations are presented for full-valence calculations on linear polyenes (CnHn+2), which highlight the excellent computational scaling of the present implementations that can routinely handle active spaces of hundreds of electrons. The accuracy of the models is studied in the π space of the polyenes, in hydrogen chains (H50), and in the π space of polyacene molecules. In all cases, the results compare favorably to density matrix renormalization group values. With the novel implementation of PQ, active spaces of 140 electrons in 140 orbitals can be solved in a matter of minutes on a single core workstation, and the relatively low polynomial scaling means that very large systems are also accessible using parallel computing.

Feynman's clock, a new variational principle, and parallel-in-time quantum dynamics.

We introduce a discrete-time variational principle inspired by the quantum clock originally proposed by Feynman and use it to write down quantum evolution as a ground-state eigenvalue problem. The construction allows one to apply ground-state quantum many-body theory to quantum dynamics, extending the reach of many highly developed tools from this fertile research area. Moreover, this formalism naturally leads to an algorithm to parallelize quantum simulation over time. We draw an explicit connection between previously known time-dependent variational principles and the time-embedded variational principle presented. Sample calculations are presented, applying the idea to a hydrogen molecule and the spin degrees of freedom of a model inorganic compound, demonstrating the parallel speedup of our method as well as its flexibility in applying ground-state methodologies. Finally, we take advantage of the unique perspective of this variational principle to examine the error of basis approximations in quantum dynamics.

How electronic dynamics with Pauli exclusion produces Fermi-Dirac statistics.

It is important that any dynamics method approaches the correct population distribution at long times. In this paper, we derive a one-body reduced density matrix dynamics for electrons in energetic contact with a bath. We obtain a remarkable equation of motion which shows that in order to reach equilibrium properly, rates of electron transitions depend on the density matrix. Even though the bath drives the electrons towards a Boltzmann distribution, hole blocking factors in our equation of motion cause the electronic populations to relax to a Fermi-Dirac distribution. These factors are an old concept, but we show how they can be derived with a combination of time-dependent perturbation theory and the extended normal ordering of Mukherjee and Kutzelnigg for a general electronic state. The resulting non-equilibrium kinetic equations generalize the usual Redfield theory to many-electron systems, while ensuring that the orbital occupations remain between zero and one. In numerical applications of our equations, we show that relaxation rates of molecules are not constant because of the blocking effect. Other applications to model atomic chains are also presented which highlight the importance of treating both dephasing and relaxation. Finally, we show how the bath localizes the electron density matrix.

Exciton coherence lifetimes from electronic structure.

We model the coherent energy transfer of an electronic excitation within covalently linked aromatic homodimers from first-principles. Our results shed light on whether commonly used models of the bath calculated via detailed electronic structure calculations can reproduce the key dynamics. For the systems we model, the time scales of coherent transport are experimentally known from time-dependent polarization anisotropy measurements, and so we can directly assess whether current techniques are predictive for modeling coherent transport. The coupling of the electronic degrees of freedom to the nuclear degrees of freedom is calculated from first-principles rather than assumed, and the fluorescence anisotropy decay is directly reproduced. Surprisingly, we find that although time-dependent density functional theory absolute energies are routinely in error by orders of magnitude more than the coupling energy between monomers, the coherent transport properties of these dimers can be semi-quantitatively reproduced from these calculations. Future directions which must be pursued to yield predictive and reliable models of coherent transport are suggested.

A correlated-polaron electronic propagator: open electronic dynamics beyond the Born-Oppenheimer approximation.

In this work, we develop an approach to treat correlated many-electron dynamics, dressed by the presence of a finite-temperature harmonic bath. Our theory combines a small polaron transformation with the second-order time-convolutionless master equation and includes both electronic and system-bath correlations on equal footing. Our theory is based on the ab initio Hamiltonian, and is thus well-defined apart from any phenomenological choice of basis states or electronic system-bath coupling model. The equation-of-motion for the density matrix we derive includes non-markovian and non-perturbative bath effects and can be used to simulate environmentally broadened electronic spectra and dissipative dynamics, which are subjects of recent interest. The theory also goes beyond the adiabatic Born-Oppenheimer approximation, but with computational cost scaling such as the Born-Oppenheimer approach. Example propagations with a developmental code are performed, demonstrating the treatment of electron-correlation in absorption spectra, vibronic structure, and decay in an open system. An untransformed version of the theory is also presented to treat more general baths and larger systems.

Benchmark results for empirical post-GGA functionals: difficult exchange problems and independent tests.

Many of the most promising new density functionals have improved the treatment of non-local exchange effects with the help of semi-empirical information and more sophisticated recipes for combining Hartree-Fock and local exchange approximations. In order to quantify recent advancements and identify directions for improvement, we have examined a broad spectrum of test problems. We evaluate the performance of several new hybrid density functionals (ωB97, ωB97X, ωB97X-D, LRC-ωPBEh, M06, M06-2X, and M06-HF) on a variety of chemical problems, some sensitive to the treatment of exact exchange (which we have hoped to systematically improve) and some which require a balanced treatment of correlation. Since all of the functionals under consideration are parameterized with ground-state thermochemical data, the benchmark aims to determine the applicability of the new density functionals to cases that have not been considered in the optimization of the semi-empirical parameters. The first class of benchmarks includes the excitation energies of 21 molecules (83 states) primarily from a recent benchmark conducted by Tozer and co-workers, with some additional references from data made available from the groups of Thiel and Truhlar. We briefly examine the conformational preferences of a small peptide and complete our study with two recently published sets of data that have shown large, systematic errors in simple alkane thermochemistry. While our results indicate that the more general hybrids currently under development perform well for problems outside of their parameterization and improve over the standard hybrid density functionals in an essentially systematic way, there is still a significant self-interaction error in the more difficult cases. Functionals based on a range-separation of exchange and functionals depending on the kinetic-energy density both perform comparably, and there is evidence for complementary strengths.

The formulation and performance of a perturbative correction to the perfect quadruples model.

A recently published alternative hierarchy of coupled-cluster approximations is reformulated as a perturbative correction. A single variant, a model for the total electronic energy based on the perfect quadruples model, is explored in detail. The computational scaling of the method developed is the same as canonical second order Mo̸ller-Plesset perturbation theory (fifth order in the number of molecular orbitals), but its accuracy competes with the high-accuracy, high-cost standard CCSD(T), even when the latter is allowed to break spin-symmetry. The variation presented can be implemented without explicit calculation and storage of the most expensive energy contributions, thereby improving the range of systems which can be treated. The performance and scaling of the method are demonstrated with calculations on the water, fluorine, and oxirane molecules, and compared to the parent model.

A tractable and accurate electronic structure method for static correlations: the perfect hextuples model.

We present the next stage in a hierarchy of local approximations to complete active space self-consistent field (CASSCF) model in an active space of one active orbital per active electron based on the valence orbital-optimized coupled-cluster (VOO-CC) formalism. Following the perfect pairing (PP) model, which is exact for a single electron pair and extensive, and the perfect quadruples (PQ) model, which is exact for two pairs, we introduce the perfect hextuples (PH) model, which is exact for three pairs. PH is an approximation to the VOO-CC method truncated at hextuples containing all correlations between three electron pairs. While VOO-CCDTQ56 requires computational effort scaling with the 14th power of molecular size, PH requires only sixth power effort. Our implementation also introduces some techniques which reduce the scaling to fifth order and has been applied to active spaces roughly twice the size of the CASSCF limit without any symmetry. Because PH explicitly correlates up to six electrons at a time, it can faithfully model the static correlations of molecules with up to triple bonds in a size-consistent fashion and for organic reactions usually reproduces CASSCF with chemical accuracy. The convergence of the PP, PQ, and PH hierarchy is demonstrated on a variety of examples including symmetry breaking in benzene, the Cope rearrangement, the Bergman reaction, and the dissociation of fluorine.

A truncation hierarchy of coupled cluster models of strongly correlated systems based on perfect-pairing references: the singles+doubles models.

Paired, active-space treatments of static correlation are augmented with additional amplitudes to produce a hierarchy of parsimonious and efficient cluster truncations that approximate the total energy. The number of parameters introduced in these models grow with system size in a tractable way: two powers larger than the static correlation model it is built upon: for instance cubic for the models built on perfect pairing, fourth order for a perfect quadruples (PQ) reference, and fifth order for the models built on perfect hextuples. These methods are called singles+doubles (SD) corrections to perfect pairing, PQ, perfect hextuples, and two variants are explored. An implementation of the SD methods is compared to benchmark results for F(2) and H(2)O dissociation problems, the H(4) and H(8) model systems, and the insertion of beryllium into hydrogen. In the cases examined even the quartic number of parameters associated with PQSD is able to provide results which meaningfully improve on coupled-cluster singles doubles (CCSD) (which also has quartic amplitudes) and compete with existing multi-reference alternatives.

The numerical condition of electron correlation theories when only active pairs of electrons are spin-unrestricted.

The use of spin-unrestriction with high-quality correlation theory, such as coupled-cluster (CC) methods, is a common practice necessary to obtain high-quality potential energy surfaces. While this typically is a useful approach, we find that in the unrestricted limit of ROHF fragments (the unrestricted in active pair orbitals) the CC equations are singular if only the strongly correlated electrons are considered. Unstable amplitudes which do not represent the physics of the problem are easily found and could be unwittingly accepted without inspection. We use stability analysis and the condition number of the CC doubles Jacobian matrix to examine the problem, and present results for several molecular systems with a variety of unrestricted cluster models. Finally a regularization of the CC equations is proposed, using a dynamic penalty function, which allows us to apply CC, and Lagrangian gradient formulas even in the singular limit.

The perfect quadruples model for electron correlation in a valence active space.

A local approximation to the Schrodinger equation in a valence active space is suggested based on coupled cluster (CC) theory. Working in a pairing active space with one virtual orbital per occupied orbital, this perfect quadruples (PQ) model is defined such that electrons are strongly correlated up to "four-at-a-time" in up to two different (occupied-virtual) electron pairs. This is a truncation of the CC theory with up to quadruple substitutions (CCSDTQ) in the active space, such that the retained amplitudes in PQ are proportional to the fourth root of the number of CCSDTQ amplitudes. Despite the apparently drastic nature of the PQ truncation, in the cases examined this model is a very accurate approximation to complete active space self-consistent field. Examples include deformations of square H(4), dissociation of two single bonds (water), a double bond (ethene), and a triple bond (nitrogen). The computational scaling of the model (fourth order with molecule size) is less than integral transformation, so relatively large systems can be addressed with improved accuracy relative to earlier methods such as perfect and imperfect pairing, which are truncations of CCSD in an active space.

On the border between localization and delocalization: tris(iminoxolene)titanium(iv).

The tris(aminophenol) ligand tris(4-methyl-2-(3',5'-di-tert-butyl-2'-hydroxyphenylamino)phenyl)amine, MeClampH6, reacts with Ti(OiPr)4 to give, after exposure to air, the dark purple, neutral, diamagnetic complex (MeClamp)Ti. The compound is six-coordinate, with an uncoordinated central nitrogen (Ti-N = 2.8274(12) Å), and contains titanium(iv) and a doubly oxidized ligand, formally a bis(iminosemiquinone)-mono(amidophenoxide). The compound is unsymmetrical in the solid state, though the three ligands are equivalent on the NMR timescale in solution. Ab initio calculations indicate that the ground state is a multiconfigurational singlet, with a low-lying multiconfigurational triplet state. Variable-temperature NMR measurements are consistent with a singlet-triplet gap of 1200 ± 70 cm-1, in good agreement with calculations. The distortion from threefold symmetry allows a low-lying, partially populated ligand-centered π nonbonding orbital to mix with largely occupied metal-ligand π bonding orbitals. The energetic accessibility of this distortion is inversely related to the strength of the metal-ligand π bonding interaction.

The TensorMol-0.1 model chemistry: a neural network augmented with long-range physics.

Traditional force fields cannot model chemical reactivity, and suffer from low generality without re-fitting. Neural network potentials promise to address these problems, offering energies and forces with near ab initio accuracy at low cost. However a data-driven approach is naturally inefficient for long-range interatomic forces that have simple physical formulas. In this manuscript we construct a hybrid model chemistry consisting of a nearsighted neural network potential with screened long-range electrostatic and van der Waals physics. This trained potential, simply dubbed "TensorMol-0.1", is offered in an open-source Python package capable of many of the simulation types commonly used to study chemistry: geometry optimizations, harmonic spectra, open or periodic molecular dynamics, Monte Carlo, and nudged elastic band calculations. We describe the robustness and speed of the package, demonstrating its millihartree accuracy and scalability to tens-of-thousands of atoms on ordinary laptops. We demonstrate the performance of the model by reproducing vibrational spectra, and simulating the molecular dynamics of a protein. Our comparisons with electronic structure theory and experimental data demonstrate that neural network molecular dynamics is poised to become an important tool for molecular simulation, lowering the resource barrier to simulating chemistry.