MELLODDY: Cross-pharma Federated Learning at Unprecedented Scale Unlocks Benefits in QSAR without Compromising Proprietary Information.

Federated multipartner machine learning has been touted as an appealing and efficient method to increase the effective training data volume and thereby the predictivity of models, particularly when the generation of training data is resource-intensive. In the landmark MELLODDY project, indeed, each of ten pharmaceutical companies realized aggregated improvements on its own classification or regression models through federated learning. To this end, they leveraged a novel implementation extending multitask learning across partners, on a platform audited for privacy and security. The experiments involved an unprecedented cross-pharma data set of 2.6+ billion confidential experimental activity data points, documenting 21+ million physical small molecules and 40+ thousand assays in on-target and secondary pharmacodynamics and pharmacokinetics. Appropriate complementary metrics were developed to evaluate the predictive performance in the federated setting. In addition to predictive performance increases in labeled space, the results point toward an extended applicability domain in federated learning. Increases in collective training data volume, including by means of auxiliary data resulting from single concentration high-throughput and imaging assays, continued to boost predictive performance, albeit with a saturating return. Markedly higher improvements were observed for the pharmacokinetics and safety panel assay-based task subsets.

Developing machine-learned potentials to simultaneously capture the dynamics of excess protons and hydroxide ions in classical and path integral simulations.

The transport of excess protons and hydroxide ions in water underlies numerous important chemical and biological processes. Accurately simulating the associated transport mechanisms ideally requires utilizing ab initio molecular dynamics simulations to model the bond breaking and formation involved in proton transfer and path-integral simulations to model the nuclear quantum effects relevant to light hydrogen atoms. These requirements result in a prohibitive computational cost, especially at the time and length scales needed to converge proton transport properties. Here, we present machine-learned potentials (MLPs) that can model both excess protons and hydroxide ions at the generalized gradient approximation and hybrid density functional theory levels of accuracy and use them to perform multiple nanoseconds of both classical and path-integral proton defect simulations at a fraction of the cost of the corresponding ab initio simulations. We show that the MLPs are able to reproduce ab initio trends and converge properties such as the diffusion coefficients of both excess protons and hydroxide ions. We use our multi-nanosecond simulations, which allow us to monitor large numbers of proton transfer events, to analyze the role of hypercoordination in the transport mechanism of the hydroxide ion and provide further evidence for the asymmetry in diffusion between excess protons and hydroxide ions.

Optically Induced Anisotropy in Time-Resolved Scattering: Imaging Molecular-Scale Structure and Dynamics in Disordered Media with Experiment and Theory.

Time-resolved scattering experiments enable imaging of materials at the molecular scale with femtosecond time resolution. However, in disordered media they provide access to just one radial dimension thus limiting the study of orientational structure and dynamics. Here we introduce a rigorous and practical theoretical framework for predicting and interpreting experiments combining optically induced anisotropy and time-resolved scattering. Using impulsive nuclear Raman and ultrafast x-ray scattering experiments of chloroform and simulations, we demonstrate that this framework can accurately predict and elucidate both the spatial and temporal features of these experiments.

Machine learning-accelerated quantum mechanics-based atomistic simulations for industrial applications.

Atomistic simulations have become an invaluable tool for industrial applications ranging from the optimization of protein-ligand interactions for drug discovery to the design of new materials for energy applications. Here we review recent advances in the use of machine learning (ML) methods for accelerated simulations based on a quantum mechanical (QM) description of the system. We show how recent progress in ML methods has dramatically extended the applicability range of conventional QM-based simulations, allowing to calculate industrially relevant properties with enhanced accuracy, at reduced computational cost, and for length and time scales that would have otherwise not been accessible. We illustrate the benefits of ML-accelerated atomistic simulations for industrial R&D processes by showcasing relevant applications from two very different areas, drug discovery (pharmaceuticals) and energy materials. Writing from the perspective of both a molecular and a materials modeling scientist, this review aims to provide a unified picture of the impact of ML-accelerated atomistic simulations on the pharmaceutical, chemical, and materials industries and gives an outlook on the exciting opportunities that could emerge in the future.

AENET-LAMMPS and AENET-TINKER: Interfaces for accurate and efficient molecular dynamics simulations with machine learning potentials.

Machine-learning potentials (MLPs) trained on data from quantum-mechanics based first-principles methods can approach the accuracy of the reference method at a fraction of the computational cost. To facilitate efficient MLP-based molecular dynamics and Monte Carlo simulations, an integration of the MLPs with sampling software is needed. Here, we develop two interfaces that link the atomic energy network (ænet) MLP package with the popular sampling packages TINKER and LAMMPS. The three packages, ænet, TINKER, and LAMMPS, are free and open-source software that enable, in combination, accurate simulations of large and complex systems with low computational cost that scales linearly with the number of atoms. Scaling tests show that the parallel efficiency of the ænet-TINKER interface is nearly optimal but is limited to shared-memory systems. The ænet-LAMMPS interface achieves excellent parallel efficiency on highly parallel distributed-memory systems and benefits from the highly optimized neighbor list implemented in LAMMPS. We demonstrate the utility of the two MLP interfaces for two relevant example applications: the investigation of diffusion phenomena in liquid water and the equilibration of nanostructured amorphous battery materials.

Don't Overweight Weights: Evaluation of Weighting Strategies for Multi-Task Bioactivity Classification Models.

Machine learning models predicting the bioactivity of chemical compounds belong nowadays to the standard tools of cheminformaticians and computational medicinal chemists. Multi-task and federated learning are promising machine learning approaches that allow privacy-preserving usage of large amounts of data from diverse sources, which is crucial for achieving good generalization and high-performance results. Using large, real world data sets from six pharmaceutical companies, here we investigate different strategies for averaging weighted task loss functions to train multi-task bioactivity classification models. The weighting strategies shall be suitable for federated learning and ensure that learning efforts are well distributed even if data are diverse. Comparing several approaches using weights that depend on the number of sub-tasks per assay, task size, and class balance, respectively, we find that a simple sub-task weighting approach leads to robust model performance for all investigated data sets and is especially suited for federated learning.

How van der Waals interactions determine the unique properties of water.

Whereas the interactions between water molecules are dominated by strongly directional hydrogen bonds (HBs), it was recently proposed that relatively weak, isotropic van der Waals (vdW) forces are essential for understanding the properties of liquid water and ice. This insight was derived from ab initio computer simulations, which provide an unbiased description of water at the atomic level and yield information on the underlying molecular forces. However, the high computational cost of such simulations prevents the systematic investigation of the influence of vdW forces on the thermodynamic anomalies of water. Here, we develop efficient ab initio-quality neural network potentials and use them to demonstrate that vdW interactions are crucial for the formation of water's density maximum and its negative volume of melting. Both phenomena can be explained by the flexibility of the HB network, which is the result of a delicate balance of weak vdW forces, causing, e.g., a pronounced expansion of the second solvation shell upon cooling that induces the density maximum.

Nanostructured Ir-supported on Ti4O7 as a cost-effective anode for proton exchange membrane (PEM) electrolyzers.

PEM water electrolysis has recently emerged as one of the most promising technologies for large H2 production from a temporal surplus of renewable electricity; yet it is expensive, partly due to the use of large amounts of Ir present in the anode. Here we report the development and characterization of a cost-effective catalyst, which consists of metallic Ir nanoparticles supported on commercial Ti4O7. The catalyst is synthesized by reducing IrCl3 with NaBH4 in a suspension containing Ti4O7, cetyltrimethylammonium bromide (CTAB) and anhydrous ethanol. No thermal treatment was applied afterwards in order to preserve the high conductivity of Ti4O7 and the metallic properties of Ir. Electron microscopy images show an uniform distribution of mostly single Ir particles covering the electro-ceramic support, although some agglomerates are still present. X-ray diffraction (XRD) analysis reveals a cubic face centered structure of Ir nanoparticles with a crystallite size of ca. 1.8 nm. According to X-ray photoelectron spectroscopy (XPS), the ratio of metallic Ir and Ir-oxide, identified as Ir(3+), is 3 : 1 after the removal of surface contamination. Other surface properties such as primary particle size distribution and surface potential were determined by atomic force microscopy (AFM). Cyclic and linear voltammetric measurements were conducted to study the electrochemical surface and kinetics of Ir-black and Ir/Ti4O7. The developed catalyst outperforms the commercial Ir-black in terms of mass activity for the oxygen evolution reaction (OER) in acid medium by a factor of four, measured at 0.25 V overpotential and room temperature. In general, the Ir/Ti4O7 catalyst exhibits improved kinetics and higher turnover frequency (TOF) compared to Ir-black. The developed Ir/Ti4O7 catalyst allows reducing the precious metal loading in the anode of a PEM electrolyzer by taking advantage of the use of an electro-ceramic support.

Representing the potential-energy surface of protonated water clusters by high-dimensional neural network potentials.

Investigating the properties of protons in water is essential for understanding many chemical processes in aqueous solution. While important insights can in principle be gained by accurate and well-established methods like ab initio molecular dynamics simulations, the computational costs of these techniques are often very high. This prevents studying large systems on long time scales, which is severely limiting the applicability of computer simulations to address a wide range of interesting phenomena. Developing more efficient potentials enabling the simulation of water including dissociation and recombination events with first-principles accuracy is a very challenging task. In particular protonated water clusters have become important model systems to assess the reliability of such potentials, as the presence of the excess proton induces substantial changes in the local hydrogen bond patterns and many energetically similar isomers exist, which are extremely difficult to describe. In recent years it has been demonstrated for a number of systems including neutral water clusters of varying size that neural networks (NNs) can be used to construct potentials with close to first-principles accuracy. Based on density-functional theory (DFT) calculations, here we present a reactive full-dimensional NN potential for protonated water clusters up to the octamer. A detailed investigation of this potential shows that the energetic, structural, and vibrational properties are in excellent agreement with DFT results making the NN approach a very promising candidate for developing a high-quality potential for water. This finding is further supported by first preliminary but very encouraging NN-based simulations of the bulk liquid.

A density-functional theory-based neural network potential for water clusters including van der Waals corrections.

The fundamental importance of water for many chemical processes has motivated the development of countless efficient but approximate water potentials for large-scale molecular dynamics simulations, from simple empirical force fields to very sophisticated flexible water models. Accurate and generally applicable water potentials should fulfill a number of requirements. They should have a quality close to quantum chemical methods, they should explicitly depend on all degrees of freedom including all relevant many-body interactions, and they should be able to describe molecular dissociation and recombination. In this work, we present a high-dimensional neural network (NN) potential for water clusters based on density-functional theory (DFT) calculations, which is constructed using clusters containing up to 10 monomers and is in principle able to meet all these requirements. We investigate the reliability of specific parametrizations employing two frequently used generalized gradient approximation (GGA) exchange-correlation functionals, PBE and RPBE, as reference methods. We find that the binding energy errors of the NN potentials with respect to DFT are significantly lower than the typical uncertainties of DFT calculations arising from the choice of the exchange-correlation functional. Further, we examine the role of van der Waals interactions, which are not properly described by GGA functionals. Specifically, we incorporate the D3 scheme suggested by Grimme (J. Chem. Phys. 2010, 132, 154104) in our potentials and demonstrate that it can be applied to GGA-based NN potentials in the same way as to DFT calculations without modification. Our results show that the description of small water clusters provided by the RPBE functional is significantly improved if van der Waals interactions are included, while in case of the PBE functional, which is well-known to yield stronger binding than RPBE, van der Waals corrections lead to overestimated binding energies.

Novel Pyrrolidinium-Functionalized Styrene-b-ethylene-b-butylene-b-styrene Copolymer Based Anion Exchange Membrane with Flexible Spacers for Water Electrolysis.

Anion exchange membranes (AEM) are core components for alkaline electrochemical energy technologies, such as water electrolysis and fuel cells. They are regarded as promising alternatives for proton exchange membranes (PEM) due to the possibility of using platinum group metal (PGM)-free electrocatalysts. However, their chemical stability and conductivity are still of great concern, which is appearing to be a major challenge for developing AEM-based energy systems. Herein, we highlight an AEM with styrene-b-ethylene-b-butylene-b-styrene copolymer (SEBS) as a backbone and pyrrolidinium or piperidinium functional groups tethered on flexible ethylene oxide spacer side-chains (SEBS-Py2O6). This membrane reached 27.8 mS cm-1 hydroxide ion conductivity at room temperature, which is higher compared to previously obtained piperidinium-functionalized SEBS reaching up to 10.09 mS cm-1. The SEBS-Py206 combined with PGM-free electrodes in an AWE water electrolysis (AEMWE) cell achieves 520 mA cm-2 at 2 V in 0.1 M KOH and 171 mA cm-2 in ultra-pure water (UPW). This high performance indicates that SEBS-Py2O6 membranes are suitable for application in water electrolysis.

Magnesium Anode Protection by an Organic Artificial Solid Electrolyte Interphase for Magnesium-Sulfur Batteries.

In the search for post-lithium battery systems, magnesium-sulfur batteries have attracted research attention in recent years due to their high potential energy density, raw material abundance, and low cost. Despite significant progress, the system still lacks cycling stability mainly associated with the ongoing parasitic reduction of sulfur at the anode surface, resulting in the loss of active materials and passivating surface layer formation on the anode. In addition to sulfur retention approaches on the cathode side, the protection of the reductive anode surface by an artificial solid electrolyte interphase (SEI) represents a promising approach, which contrarily does not impede the sulfur cathode kinetics. In this study, an organic coating approach based on ionomers and polymers is pursued to combine the desired properties of mechanical flexibility and high ionic conductivity while enabling a facile and energy-efficient preparation. Despite exhibiting higher polarization overpotentials in Mg-Mg cells, the charge overpotential in Mg-S cells was decreased by the coated anodes with the initial Coulombic efficiency being significantly increased. Consequently, the discharge capacity after 300 cycles applying an Aquivion/PVDF-coated Mg anode was twice that of a pristine Mg anode, indicating effective polysulfide repulsion from the Mg surface by the artificial SEI. This was backed by operando imaging during long-term OCV revealing a non-colored separator, i.e. mitigated self-discharge. While SEM, AFM, IR and XPS were applied to gain further insights into the surface morphology and composition, scalable coating techniques were investigated in addition to ensure practical relevance. Remarkably therein, the Mg anode preparation and all surface coatings were prepared under ambient conditions, which facilitates future electrode and cell assembly. Overall, this study highlights the important role of Mg anode coatings to improve the electrochemical performance of magnesium-sulfur batteries.

A neural network potential-energy surface for the water dimer based on environment-dependent atomic energies and charges.

Understanding the unique properties of water still represents a significant challenge for theory and experiment. Computer simulations by molecular dynamics require a reliable description of the atomic interactions, and in recent decades countless water potentials have been reported in the literature. Still, most of these potentials contain significant approximations, for instance a frozen internal structure of the individual water monomers. Artificial neural networks (NNs) offer a promising way for the construction of very accurate potential-energy surfaces taking all degrees of freedom explicitly into account. These potentials are based on electronic structure calculations for representative configurations, which are then interpolated to a continuous energy surface that can be evaluated many orders of magnitude faster. We present a full-dimensional NN potential for the water dimer as a first step towards the construction of a NN potential for liquid water. This many-body potential is based on environment-dependent atomic energy contributions, and long-range electrostatic interactions are incorporated employing environment-dependent atomic charges. We show that the potential and derived properties like vibrational frequencies are in excellent agreement with the underlying reference density-functional theory calculations.

Towards Replacing Titanium with Copper in the Bipolar Plates for Proton Exchange Membrane Water Electrolysis.

For proton exchange membrane water electrolysis (PEMWE) to become competitive, the cost of stack components, such as bipolar plates (BPP), needs to be reduced. This can be achieved by using coated low-cost materials, such as copper as alternative to titanium. Herein we report on highly corrosion-resistant copper BPP coated with niobium. All investigated samples showed excellent corrosion resistance properties, with corrosion currents lower than 0.1 µA cm-2 in a simulated PEM electrolyzer environment at two different pH values. The physico-chemical properties of the Nb coatings are thoroughly characterized by scanning electron microscopy (SEM), electrochemical impedance spectroscopy (EIS), X-ray photoelectron spectroscopy (XPS), and atomic force microscopy (AFM). A 30 µm thick Nb coating fully protects the Cu against corrosion due to the formation of a passive oxide layer on its surface, predominantly composed of Nb2O5. The thickness of the passive oxide layer determined by both EIS and XPS is in the range of 10 nm. The results reported here demonstrate the effectiveness of Nb for protecting Cu against corrosion, opening the possibility to use it for the manufacturing of BPP for PEMWE. The latter was confirmed by its successful implementation in a single cell PEMWE based on hydraulic compression technology.

Exploiting Machine Learning to Efficiently Predict Multidimensional Optical Spectra in Complex Environments.

The excited-state dynamics of chromophores in complex environments determine a range of vital biological and energy capture processes. Time-resolved, multidimensional optical spectroscopies provide a key tool to investigate these processes. Although theory has the potential to decode these spectra in terms of the electronic and atomistic dynamics, the need for large numbers of excited-state electronic structure calculations severely limits first-principles predictions of multidimensional optical spectra for chromophores in the condensed phase. Here, we leverage the locality of chromophore excitations to develop machine learning models to predict the excited-state energy gap of chromophores in complex environments for efficiently constructing linear and multidimensional optical spectra. By analyzing the performance of these models, which span a hierarchy of physical approximations, across a range of chromophore-environment interaction strengths, we provide strategies for the construction of machine learning models that greatly accelerate the calculation of multidimensional optical spectra from first principles.

Parallel Multistream Training of High-Dimensional Neural Network Potentials.

Over the past years high-dimensional neural network potentials (HDNNPs), fitted to accurately reproduce ab initio potential energy surfaces, have become a powerful tool in chemistry, physics and materials science. Here, we focus on the training of the neural networks that lies at the heart of the HDNNP method. We present an efficient approach for optimizing the weight parameters of the neural network via multistream Kalman filtering, using potential energies and forces as reference data. In this procedure, the choice of the free parameters of the Kalman filter can have a significant impact on the fit quality. Carrying out a large parameter study, we determine optimal settings and demonstrate how to optimize training results of HDNNPs. Moreover, we illustrate our HDNNP training approach by revisiting previously presented fits for water and developing a new potential for copper sulfide. This material, accessible in computer simulations so far only via first-principles methods, forms a particularly complex solid structure at low temperatures and undergoes a phase transition to a superionic state upon heating. Analyzing MD simulations carried out with the Cu2S HDNNP, we confirm that the underlying ab initio reference method indeed reproduces this behavior.

Hiding in the Crowd: Spectral Signatures of Overcoordinated Hydrogen-Bond Environments.

Molecules with an excess number of hydrogen-bonding partners play a crucial role in fundamental chemical processes, ranging from anomalous diffusion in supercooled water to transport of aqueous proton defects and ordering of water around hydrophobic solutes. Here we show that overcoordinated hydrogen-bond environments can be identified in both the ambient and supercooled regimes of liquid water by combining experimental Raman multivariate curve resolution measurements and machine learning accelerated quantum simulations. In particular, we find that OH groups appearing in spectral regions usually associated with non-hydrogen-bonded species actually correspond to hydrogen bonds formed in overcoordinated environments. We further show that only these species exhibit a turnover in population as a function of temperature, which is robust and persists under both constant pressure and density conditions. This work thus provides a new tool to identify, interpret, and elucidate the spectral signatures of crowded hydrogen-bond networks.

Density anomaly of water at negative pressures from first principles.

Using molecular dynamics simulations based on ab initio trained high-dimensional neural network potentials, we study the equation of state of liquid water at negative pressures. From density isobars computed for various pressures down to p = -230 MPa we determine the line of density maxima for two potentials based on the BLYP and the RPBE functionals, respectively. In both cases, dispersion corrections are included to account for non-local long-range correlations that give rise to van der Waals forces. We have followed the density maximum down to negative pressures close to the spinodal instability. For both functionals, the temperature of maximum density increases with decreasing pressure under moderate stretching, but changes slope at [Formula: see text] MPa and [Formula: see text] MPa for BLYP and RPBE, respectively. Our calculations confirm qualitatively the retracing shape of the line of density maxima found for empirical water models, indicating that the spinodal line maintains a positive slope even at strongly negative pressures.

The Interplay of Structure and Dynamics in the Raman Spectrum of Liquid Water over the Full Frequency and Temperature Range.

While many vibrational Raman spectroscopy studies of liquid water have investigated the temperature dependence of the high-frequency O-H stretching region, few have analyzed the changes in the Raman spectrum as a function of temperature over the entire spectral range. Here, we obtain the Raman spectra of water from its melting to boiling point, both experimentally and from simulations using an ab initio-trained machine learning potential. We use these to assign the Raman bands and show that the entire spectrum can be well described as a combination of two temperature-independent spectra. We then assess which spectral regions exhibit strong dependence on the local tetrahedral order in the liquid. Further, this work demonstrates that changes in this structural parameter can be used to elucidate the temperature dependence of the Raman spectrum of liquid water and provides a guide to the Raman features that signal water ordering in more complex aqueous systems.

Quantitative in Situ Analysis of Ionomer Structure in Fuel Cell Catalytic Layers.

A quantitative in situ investigation of the structure of the catalytic layer of polymer electrolyte membrane fuel cells using material-sensitive and conductive atomic force microscopy is reported. The distribution and size of the ionomer phase at the surface of the catalytic layer is retrieved from adhesion force mappings, measured at high humidity and up to 75 °C. The average ionomer layer thickness varies between 7 and 13 nm for three differently prepared samples, as concluded from the histograms. Evidence of a lamellar structure of the thinner ionomer layers is presented. A significant thinning of the ionomer layers after long-term fuel cell operation is observed.