Hybrid programming-model strategies for GPU offloading of electronic structure calculation kernels.

To address the challenge of performance portability and facilitate the implementation of electronic structure solvers, we developed the basic matrix library (BML) and Parallel, Rapid O(N), and Graph-based Recursive Electronic Structure Solver (PROGRESS) library. The BML implements linear algebra operations necessary for electronic structure kernels using a unified user interface for various matrix formats (dense and sparse) and architectures (CPUs and GPUs). Focusing on density functional theory and tight-binding models, PROGRESS implements several solvers for computing the single-particle density matrix and relies on BML. In this paper, we describe the general strategies used for these implementations on various computer architectures, using OpenMP target functionalities on GPUs, in conjunction with third-party libraries to handle performance critical numerical kernels. We demonstrate the portability of this approach and its performance in benchmark problems.

A fast, dense Chebyshev solver for electronic structure on GPUs.

Matrix diagonalization is almost always involved in computing the density matrix needed in quantum chemistry calculations. In the case of modest matrix sizes (≲4000), performance of traditional dense diagonalization algorithms on modern GPUs is underwhelming compared to the peak performance of these devices. This motivates the exploration of alternative algorithms better suited to these types of architectures. We newly derive, and present in detail, an existing Chebyshev expansion algorithm [Liang et al., J. Chem. Phys. 119, 4117-4125 (2003)] whose number of required matrix multiplications scales with the square root of the number of terms in the expansion. Focusing on dense matrices of modest size, our implementation on GPUs results in large speed ups when compared to diagonalization. Additionally, we improve upon this existing method by capitalizing on the inherent task parallelism and concurrency in the algorithm. This improvement is implemented on GPUs by using CUDA and HIP streams via the MAGMA library and leads to a significant speed up over the serial-only approach for smaller (≲1000) matrix sizes. Finally, we apply our technique to a model system with a high density of states around the Fermi level, which typically presents significant challenges.

A methodology to generate crystal-based molecular structures for atomistic simulations.

We propose a systematic method to construct crystal-based molecular structures often needed as input for computational chemistry studies. These structures include crystal 'slabs' with periodic boundary conditions (PBCs) and non-periodic solids such as Wulff structures. We also introduce a method to build crystal slabs with orthogonal PBC vectors. These methods are integrated into our code,Los Alamos Crystal Cut(LCC), which is open source and thus fully available to the community. Examples showing the use of these methods are given throughout the manuscript.

Quantum Perturbation Theory Using Tensor Cores and a Deep Neural Network.

Time-independent quantum response calculations are performed using Tensor cores. This is achieved by mapping density matrix perturbation theory onto the computational structure of a deep neural network. The main computational cost of each deep layer is dominated by tensor contractions, i.e., dense matrix-matrix multiplications, in mixed-precision arithmetics, which achieves close to peak performance. Quantum response calculations are demonstrated and analyzed using self-consistent charge density-functional tight-binding theory as well as coupled-perturbed Hartree-Fock theory. For linear response calculations, a novel parameter-free convergence criterion is presented that is well-suited for numerically noisy low-precision floating point operations and we demonstrate a peak performance of almost 200 Tflops using the Tensor cores of two Nvidia A100 GPUs.

Bioelectrosynthesis systems.

Bioelectrosynthesis (BES) systems exploit extracellular electron transport pathways to augment cellular metabolism. This strategy can be used to improve the economic viability of bio-based syntheses versus conventional methods, most notably petrochemical-based syntheses. It also has the potential to reduce the carbon footprint of biomanufacturing processes. Efficient channeling of cathode-derived electrons towards biosynthesis requires a better understanding of the biological mechanisms of electron transport as well as detailed evaluation of all aspects of process performance. More advanced solutions may deploy cell free systems that use ex situ generated reducing equivalents to improve economic performance.

Bringing discrete-time Langevin splitting methods into agreement with thermodynamics.

In light of the recently published complete set of statistically correct Grønbech-Jensen (GJ) methods for discrete-time thermodynamics, we revise a differential operator splitting method for the Langevin equation in order to comply with the basic GJ thermodynamic sampling features, namely, the Boltzmann distribution and Einstein diffusion, in linear systems. This revision, which is based on the introduction of time scaling along with flexibility of a discrete-time velocity attenuation parameter, provides a direct link between the ABO splitting formalism and the GJ methods. This link brings about the conclusion that any GJ method has at least weak second order accuracy in the applied time step. It further helps identify a novel half-step velocity, which simultaneously produces both correct kinetic statistics and correct transport measures for any of the statistically sound GJ methods. Explicit algorithmic expressions are given for the integration of the new half-step velocity into the GJ set of methods. Numerical simulations, including quantum-based molecular dynamics (QMD) using the QMD suite Los Alamos Transferable Tight-Binding for Energetics, highlight the discussed properties of the algorithms as well as exhibit the direct application of robust, time-step-independent stochastic integrators to QMD.

Quantum-Based Molecular Dynamics Simulations Using Tensor Cores.

Tensor cores, along with tensor processing units, represent a new form of hardware acceleration specifically designed for deep neural network calculations in artificial intelligence applications. Tensor cores provide extraordinary computational speed and energy efficiency but with the caveat that they were designed for tensor contractions (matrix-matrix multiplications) using only low-precision floating-point operations. Despite this perceived limitation, we demonstrate how tensor cores can be applied with high efficiency to the challenging and numerically sensitive problem of quantum-based Born-Oppenheimer molecular dynamics, which requires highly accurate electronic structure optimizations and conservative force evaluations. The interatomic forces are calculated on-the-fly from an electronic structure that is obtained from a generalized deep neural network, where the computational structure naturally takes advantage of the exceptional processing power of the tensor cores and allows for high performance in excess of 100 Tflops on a single Nvidia A100 GPU. Stable molecular dynamics trajectories are generated using the framework of extended Lagrangian Born-Oppenheimer molecular dynamics, which combines computational efficiency with long-term stability, even when using approximate charge relaxations and force evaluations that are limited in accuracy by the numerically noisy conditions caused by the low-precision tensor core floating-point operations. A canonical ensemble simulation scheme is also presented, where the additional numerical noise in the calculated forces is absorbed into a Langevin-like dynamics.

Mixed Precision Fermi-Operator Expansion on Tensor Cores from a Machine Learning Perspective.

We present a second-order recursive Fermi-operator expansion scheme using mixed precision floating point operations to perform electronic structure calculations using tensor core units. A performance of over 100 teraFLOPs is achieved for half-precision floating point operations on Nvidia's A100 tensor core units. The second-order recursive Fermi-operator scheme is formulated in terms of a generalized, differentiable deep neural network structure, which solves the quantum mechanical electronic structure problem. We demonstrate how this network can be accelerated by optimizing the weight and bias values to substantially reduce the number of layers required for convergence. We also show how this machine learning approach can be used to optimize the coefficients of the recursive Fermi-operator expansion to accurately represent the fractional occupation numbers of the electronic states at finite temperatures.

Modeling Drug Absorption from the Dermis after an Injection.

A dermal absorption model for small and macromolecules was previously proposed by Ibrahim et al. This model estimated absorption of therapeutics from the dermal tissue based on their molecular size and protein binding through blood and lymphatics. Blood absorption followed a two-pore theory and the lymphatic absorption was limited by the constant lymphatic flow rate. Current work builds on this steady-state concept by modeling the absorption from the dermis immediately after an injection is given (unsteady state). An injection in the dermis creates a localized pressure gradient which resolves itself over time. This phenomenon is captured in the model to estimate the impact of injection volume on the absorption rate constant. Blood absorption follows the two-pore theory but is time-dependent and the lymphatic absorption is determined based on valve opening and pressure driven convective flow, returning to steady-state as the molecule is absorbed. A direct comparison of the steady-state analysis, experimental data and the current model is made. The results indicate that accounting for the localized time-varying pressure can better predict the experimental absorption rate constants. This work significantly improves the existing understanding of macromolecule uptake from the interstitial fluid following intradermal injection.

The challenge of stochastic Størmer-Verlet thermostats generating correct statistics.

In light of the recently developed complete GJ set of single random variable stochastic, discrete-time Størmer-Verlet algorithms for statistically accurate simulations of Langevin equations [N. Grønbech-Jensen, Mol. Phys. 118, e1662506 (2020)], we investigate two outstanding questions: (1) Are there any algorithmic or statistical benefits from including multiple random variables per time step and (2) are there objective reasons for using one or more methods from the available set of statistically correct algorithms? To address the first question, we assume a general form for the discrete-time equations with two random variables and then follow the systematic, brute-force GJ methodology by enforcing correct thermodynamics in linear systems. It is concluded that correct configurational Boltzmann sampling of a particle in a harmonic potential implies correct configurational free-particle diffusion and that these requirements only can be accomplished if the two random variables per time step are identical. We consequently submit that the GJ set represents all possible stochastic Størmer-Verlet methods that can reproduce time step-independent statistics of linear systems. The second question is thus addressed within the GJ set. Based on numerical simulations of complex molecular systems, as well as on analytic considerations, we analyze apparent friction-induced differences in the stability of the methods. We attribute these differences to an inherent, friction-dependent discrete-time scaling, which depends on the specific method. We suggest that the method with the simplest interpretation of temporal scaling, the GJ-I/GJF-2GJ method, be preferred for statistical applications.

Endogenous Bioelectric Signaling Networks: Exploiting Voltage Gradients for Control of Growth and Form.

Living systems exhibit remarkable abilities to self-assemble, regenerate, and remodel complex shapes. How cellular networks construct and repair specific anatomical outcomes is an open question at the heart of the next-generation science of bioengineering. Developmental bioelectricity is an exciting emerging discipline that exploits endogenous bioelectric signaling among many cell types to regulate pattern formation. We provide a brief overview of this field, review recent data in which bioelectricity is used to control patterning in a range of model systems, and describe the molecular tools being used to probe the role of bioelectrics in the dynamic control of complex anatomy. We suggest that quantitative strategies recently developed to infer semantic content and information processing from ionic activity in the brain might provide important clues to cracking the bioelectric code. Gaining control of the mechanisms by which large-scale shape is regulated in vivo will drive transformative advances in bioengineering, regenerative medicine, and synthetic morphology, and could be used to therapeutically address birth defects, traumatic injury, and cancer.

Planarian regeneration in space: Persistent anatomical, behavioral, and bacteriological changes induced by space travel.

Regeneration is regulated not only by chemical signals but also by physical processes, such as bioelectric gradients. How these may change in the absence of the normal gravitational and geomagnetic fields is largely unknown. Planarian flatworms were moved to the International Space Station for 5 weeks, immediately after removing their heads and tails. A control group in spring water remained on Earth. No manipulation of the planaria occurred while they were in orbit, and space-exposed worms were returned to our laboratory for analysis. One animal out of 15 regenerated into a double-headed phenotype-normally an extremely rare event. Remarkably, amputating this double-headed worm again, in plain water, resulted again in the double-headed phenotype. Moreover, even when tested 20 months after return to Earth, the space-exposed worms displayed significant quantitative differences in behavior and microbiome composition. These observations may have implications for human and animal space travelers, but could also elucidate how microgravity and hypomagnetic environments could be used to trigger desired morphological, neurological, physiological, and bacteriomic changes for various regenerative and bioengineering applications.