Porting fragmentation methods to GPUs using an OpenMP API: Offloading the resolution-of-the-identity second-order Møller-Plesset perturbation method.

Using an OpenMP Application Programming Interface, the resolution-of-the-identity second-order Møller-Plesset perturbation (RI-MP2) method has been off-loaded onto graphical processing units (GPUs), both as a standalone method in the GAMESS electronic structure program and as an electron correlation energy component in the effective fragment molecular orbital (EFMO) framework. First, a new scheme has been proposed to maximize data digestion on GPUs that subsequently linearizes data transfer from central processing units (CPUs) to GPUs. Second, the GAMESS Fortran code has been interfaced with GPU numerical libraries (e.g., NVIDIA cuBLAS and cuSOLVER) for efficient matrix operations (e.g., matrix multiplication, matrix decomposition, and matrix inversion). The standalone GPU RI-MP2 code shows an increasing speedup of up to 7.5× using one NVIDIA V100 GPU with one IBM 42-core P9 CPU for calculations on fullerenes of increasing size from 40 to 260 carbon atoms using the 6-31G(d)/cc-pVDZ-RI basis sets. A single Summit node with six V100s can compute the RI-MP2 correlation energy of a cluster of 175 water molecules using the correlation consistent basis sets cc-pVDZ/cc-pVDZ-RI containing 4375 atomic orbitals and 14 700 auxiliary basis functions in ∼0.85 h. In the EFMO framework, the GPU RI-MP2 component shows near linear scaling for a large number of V100s when computing the energy of an 1800-atom mesoporous silica nanoparticle in a bath of 4000 water molecules. The parallel efficiencies of the GPU RI-MP2 component with 2304 and 4608 V100s are 98.0% and 96.1%, respectively.

Toward an extreme-scale electronic structure system.

Electronic structure calculations have the potential to predict key matter transformations for applications of strategic technological importance, from drug discovery to material science and catalysis. However, a predictive physicochemical characterization of these processes often requires accurate quantum chemical modeling of complex molecular systems with hundreds to thousands of atoms. Due to the computationally demanding nature of electronic structure calculations and the complexity of modern high-performance computing hardware, quantum chemistry software has historically failed to operate at such large molecular scales with accuracy and speed that are useful in practice. In this paper, novel algorithms and software are presented that enable extreme-scale quantum chemistry capabilities with particular emphasis on exascale calculations. This includes the development and application of the multi-Graphics Processing Unit (GPU) library LibCChem 2.0 as part of the General Atomic and Molecular Electronic Structure System package and of the standalone Extreme-scale Electronic Structure System (EXESS), designed from the ground up for scaling on thousands of GPUs to perform high-performance accurate quantum chemistry calculations at unprecedented speed and molecular scales. Among various results, we report that the EXESS implementation enables Hartree-Fock/cc-pVDZ plus RI-MP2/cc-pVDZ/cc-pVDZ-RIFIT calculations on an ionic liquid system with 623 016 electrons and 146 592 atoms in less than 45 min using 27 600 GPUs on the Summit supercomputer with a 94.6% parallel efficiency.

Many-Body Dispersion.

A broad range of approaches to many-body dispersion are discussed, including empirical approaches with multiple fitted parameters, augmented density functional-based approaches, symmetry adapted perturbation theory, and a supermolecule approach based on coupled cluster theory. Differing definitions of "body" are considered, specifically atom-based vs molecule-based approaches.

Coupling of two curved polyaromatic radical-anions: stabilization of dimers by counterions.

In this study, a comprehensive theoretical investigation of both kinetic and thermodynamic stabilities was performed for dimeric dianionic systems (C20H10)22- and (C28H14)22-, neutralized by two alkali metal cations. The influence of the counterions was of primary interest. The impact of the additional/spectator ligand(s) was elucidated by considering adducts with four molecules of diglyme or two molecules of 18-crown-6 ether. Importantly, both types of systems - in the form of contact-ion pair (CIP) and solvent-separated ion pair (SSIP) - were considered. The SSIP set was augmented by the adduct, in which the dimeric dianionic species were neutralized with purely organic cations N(CH3)4+ and P(CH3)4+. Detailed analysis of the bonding revealed that the presence of the counterions made these systems thermodynamically stable. This finding is in sharp contrast with results obtained for isolated (PAH)22- systems, which were previously found to be thermodynamically unstable, but kinetically persistent. The introduction of the alkali metal cations to the system significantly increases the ionic term (ΔEelstat), whereas the repulsive ΔEPauli one was found to be substantially reduced. Considering that the orbital component (ΔEorb) exhibited only a moderate decrease and the preparation energy (ΔEprep) showed no changes, the above-mentioned changes in ΔEelstat and ΔEPauli provided a clear explanation for the increase of the thermodynamic stability of the target species. Importantly, a clear correlation between the size of the alkali metal cation and stability of the target dimeric product was established. Thermodynamic stability of the system rises with a decrease in the size of M+ due to enlargement of the ΔEorb. Evaluated energy barriers (as spin-crossing points between singlet and triplet energy surfaces) were found to be equal to +15.85 kcal mol-1 and +18.5 kcal mol-1 for [(Cs+)2{(C20H10)22-}] and [(Cs+)2{(C28H14)22-}], respectively, which is substantially higher than those calculated for isolated (PAH)22- systems (+10.00 kcal mol-1 for (C20H10)22- and +12.35 kcal mol-1 for (C28H14)22-). Thus, this study identified the presence of counterions as the key factor, which have a dramatic influence on the thermodynamic and kinetic stabilities of the aimed dianionic dimeric systems, which are formed by two curved polyaromatic monoanion-radicals.

Heterotrimetallic Mixed-Valent Molecular Precursors Containing Periodic Table Neighbors: Assignment of Metal Positions and Oxidation States.

A known trinuclear structure was used to design the heterobimetallic mixed-valent, mixed-ligand molecule [CoII (hfac)3 -Na-CoIII (acac)3 ] (1). This was used as a template structure to develop heterotrimetallic molecules [CoII (hfac)3 -Na-FeIII (acac)3 ] (2) and [NiII (hfac)3 -Na-CoIII (acac)3 ] (3) via isovalent site-specific substitution at either of the cobalt positions. Diffraction methods, synchrotron resonant diffraction, and multiple-wavelength anomalous diffraction were applied beyond simple structural investigation to provide an unambiguous assignment of the positions and oxidation states for the periodic table neighbors in the heterometallic assemblies. Molecules of 2 and 3 are true heterotrimetallic rather than a statistical mixture of two heterobimetallic counterparts. Trinuclear platform 1 exhibits flexibility in accommodating a variety of di- and trivalent metals, which can be further utilized in the design of molecular precursors for the NaMM'O4 functional oxide materials.

Mono-reduced Corannulene: To Couple and Not to Couple in One Crystal.

One-electron reduction of corannulene, C20 H10 , with Li metal in diglyme resulted in crystallization of [{Li+ (diglyme)2 }4 (C20 H10 .- )2 (C20 H10 -C20 H10 )2- ] (1), as revealed by single-crystal X-ray diffraction. This hybrid product contains two corannulene monoanion-radicals along with a dianionic dimer, crystallized with four Li+ ions wrapped by diglyme molecules. The dimeric (C20 H10 -C20 H10 )2- anion provides the first crystallographically confirmed example of spontaneous radical dimerization for C20 H10 .- . The C-C bond length between the two C20 H10 .- bowls of 1.588(5) Šis consistent with the single σ-bond character of the linker. The trans-disposition of two bowls in the centrosymmetric (C20 H10 -C20 H10 )2- dimer is observed with the torsion angle around the central C-C bond of 180°. Comprehensive theoretical analysis of formation/decomposition processes of the dimeric dianion has been carried out in order to evaluate the nature of bonding and energetics of the C20 H10 .- coupling. It is found that such σ-bonded dimers are thermodynamically unstable due to large preparation energy and repulsive Pauli component of the bonding, but kinetically persistent due to a high energy barrier provided by the existing spin-crossing point.

Many-Body Dispersion in Molecular Clusters.

Many-body dispersion has gained considerable attention over the past decade, particularly for condensed phase systems. However, quantitatively accurate studies of many-body dispersion have only recently become feasible due to challenges in reliability and accuracy. Currently available methodologies for calculating many-body dispersion have been challenged, with recent evidence suggesting, for example, that dispersion-corrected density functional theory (DFT) schemes cannot consistently predict many-body dispersion accurately. This study evaluates many-body dispersion energies using a composite approach that employs singles and doubles coupled cluster theory with perturbative/noniterative triples, CCSD(T), combined with an extrapolation to the complete basis set (CBS) limit. The combined CCSD(T)/CBS approach is applied to Arn and (H2O)n, n = 3-10, clusters, and a new data set called S22(3), which includes trimers generated based on the S22 data set. In these systems, the many-body dispersion provides a very small contribution to the total interaction energy of all of the systems studied, generally 3% or less of the total interaction energy. Two-body dispersion is the dominant dispersion contribution and many-body dispersion contributes no more than 5.7% of the total dispersion energy, generally staying below 2%.

Site-Directed Dimerization of Bowl-Shaped Radical Anions to Form a σ-Bonded Dibenzocorannulene Dimer.

Designed site-directed dimerization of the monoanion radicals of a π-bowl in the solid state is reported. Dibenzo[a,g]corannulene (C28 H14 ) was selected based on the asymmetry of the charge/spin localization in the C28 H14.- anion. Controlled one-electron reduction of C28 H14 with Cs metal in diglyme resulted in crystallization of a new dimer, [{Cs+ (diglyme)}2 (C28 H14 -C28 H14 )2- ] (1), as revealed by single crystal X-ray diffraction study performed in a broad range of temperatures. The C-C bond length between two C28 H14.- bowls (1.560(8) Å) measured at -143 °C does not significantly change upon heating of the crystal to +67 °C. The single σ-bond character of the C-C linker is confirmed by calculations. The trans-disposition of two bowls in 1 is observed with the torsion angles around the central C-C bond of 172.3(5)° and 173.5(5)°. A systematic theoretical evaluation of dimerization pathways of C28 H14.- radicals confirmed that the trans-isomer found in 1 is energetically favored.

Porting Fragmentation Methods to Graphical Processing Units Using an OpenMP Application Programming Interface: Offloading the Fock Build for Low Angular Momentum Functions.

A framework to offload four-index two-electron repulsion integrals to graphical processing units (GPUs) using OpenMP is discussed. The method has been applied to the Fock build for low angular momentum s and p functions in both the restricted Hartree-Fock (RHF) and in the effective fragment molecular orbital (EFMO) framework. Benchmark calculations for the GPU code for the pure RHF method show an increasing speedup relative to the existing OpenMP CPU code in GAMESS from 1.04 to 52× for clusters of 70-569 water molecules. The parallel efficiency on 24 NVIDIA V100 GPU boards also increases when increasing the system size: from 75 to 94% for water clusters that contain 303-1120 molecules. In the EFMO framework, the GPU Fock build shows a high linear scalability up to 4608 V100s with a parallel efficiency of 96% for calculations on a solvated mesoporous silica nanoparticle system with ∼67,000 basis functions.

Enabling Fortran Standard Parallelism in GAMESS for Accelerated Quantum Chemistry Calculations.

The performance of Fortran 2008 DO CONCURRENT (DC) relative to OpenACC and OpenMP target offloading (OTO) with different compilers is studied for the GAMESS quantum chemistry application. Specifically, DC and OTO are used to offload the Fock build, which is a computational bottleneck in most quantum chemistry codes, to GPUs. The DC Fock build performance is studied on NVIDIA A100 and V100 accelerators and compared with the OTO versions compiled by the NVIDIA HPC, IBM XL, and Cray Fortran compilers. The results show that DC can speed up the Fock build by 3.0× compared with that of the OTO model. With similar offloading efforts, DC is a compelling programming model for offloading Fortran applications to GPUs.

The General Atomic and Molecular Electronic Structure System (GAMESS): Novel Methods on Novel Architectures.

The primary focus of GAMESS over the last 5 years has been the development of new high-performance codes that are able to take effective and efficient advantage of the most advanced computer architectures, both CPU and accelerators. These efforts include employing density fitting and fragmentation methods to reduce the high scaling of well-correlated (e.g., coupled-cluster) methods as well as developing novel codes that can take optimal advantage of graphical processing units and other modern accelerators. Because accurate wave functions can be very complex, an important new functionality in GAMESS is the quasi-atomic orbital analysis, an unbiased approach to the understanding of covalent bonds embedded in the wave function. Best practices for the maintenance and distribution of GAMESS are also discussed.

Palladium-Catalyzed Intermolecular Alkene Carboacylation via Ester C-O Bond Activation.

We report palladium-catalyzed intermolecular carboacylation of alkenes with ester electrophiles and tetraarylborate nucleophiles. Bicyclic alkenes react with a variety of pentafluorophenyl benzoate and alkanoate esters and sodium tetraarylborates to form ketone products in ≤99% yields. These reactions occur in the absence of a directing group and demonstrate esters are competent acyl electrophiles for intermolecular alkene carboacylation reactions.

Faster Self-Consistent Field (SCF) Calculations on GPU Clusters.

A novel implementation of the self-consistent field (SCF) procedure specifically designed for high-performance execution on multiple graphics processing units (GPUs) is presented. The algorithm offloads to GPUs the three major computational stages of the SCF, namely, the calculation of one-electron integrals, the calculation and digestion of electron repulsion integrals, and the diagonalization of the Fock matrix, including SCF acceleration via DIIS. Performance results for a variety of test molecules and basis sets show remarkable speedups with respect to the state-of-the-art parallel GAMESS CPU code and relative to other widely used GPU codes for both single and multi-GPU execution. The new code outperforms all existing multi-GPU implementations when using eight V100 GPUs, with speedups relative to Terachem ranging from 1.2× to 3.3× and speedups of up to 28× over QUICK on one GPU and 15× using eight GPUs. Strong scaling calculations show nearly ideal scalability up to 8 GPUs while retaining high parallel efficiency for up to 18 GPUs.

Three to tango requires a site-specific substitution: heterotrimetallic molecular precursors for high-voltage rechargeable batteries.

Design of heterotrimetallic molecules, especially those containing at least two different metals with close atomic numbers, radii, and the same coordination number/environment is a challenging task. This quest is greatly facilitated by having a heterobimetallic parent molecule that features multiple metal sites with only some of those displaying substitutional flexibility. Recently, a unique heterobimetallic complex LiMn2(thd)5 (thd = 2,2,6,6-tetramethyl-3,5-heptanedionate) has been introduced as a single-source precursor for the preparation of a popular spinel cathode material, LiMn2O4. Theoretical calculations convincingly predict that in the above trinuclear molecule only one of the Mn sites is sufficiently flexible to be substituted with another 3d transition metal. Following those predictions, two heterotrimetallic complexes, LiMn2-x Co x (thd)5 (x = 1 (1a) and 0.5 (1b)), that represent full and partial substitution, respectively, of Co for Mn in the parent molecule, have been synthesized. X-ray structural elucidation clearly showed that only one transition metal position in the trinuclear molecule contains Co, while the other site remains fully occupied by Mn. A number of techniques have been employed for deciphering the structure and composition of heterotrimetallic compounds. Synchrotron resonant diffraction experiments unambiguously assigned 3d transition metal positions as well as provided a precise "site-specific Mn/Co elemental analysis" in a single crystal, even in an extremely difficult case of severely disordered structure formed by the superposition of two enantiomers. DART mass spectrometry and magnetic measurements clearly confirmed the presence of heterotrimetallic species LiMnCo(thd)5 rather than a statistical mixture of two heterobimetallic LiMn2(thd)5 and LiCo2(thd)5 molecules. Heterometallic precursors 1a and 1b were found to exhibit a clean decomposition yielding phase-pure LiMnCoO4 and LiMn1.5Co0.5O4 spinels, respectively, at the relatively low temperature of 400 °C. The latter oxide represents an important "5V spinel" cathode material for the lithium ion batteries. Transmission electron microscopy confirmed a homogeneous distribution of transition metals in quaternary oxides obtained by pyrolysis of single-source precursors.

A three body problem: a genuine heterotrimetallic molecule vs. a mixture of two parent heterobimetallic molecules.

This work raises a fundamental question about the "real" structure of molecular compounds containing three different metals: whether they consist of genuine heterotrimetallic species or of a mixture of parent heterobimetallic species. Heterotrimetallic complex Li2CoNi(tbaoac)6 (1, tbaoac = tert-butyl acetoacetate) has been designed based on the model tetranuclear structure featuring two transition metal sites in order to be utilized as a molecular precursor for the low-temperature preparation of the LiCo0.5Ni0.5O2 battery cathode material. An investigation of the structure of 1 appeared to be very challenging, since the Co and Ni atoms have very similar atomic numbers, monoisotopic masses, and radii as well as the same oxidation state and coordination number/environment. Using a statistical analysis of heavily overlaid isotope distribution patterns of the [Li2MM'L5]+ (M/M' = Co2, Ni2, and CoNi) ions in DART mass spectra, it was concluded that the reaction product 1 contains both heterotrimetallic and bimetallic species. A structural analogue approach has been applied to obtain Li2MMg(tbaoac)6 (M = Co (2) and Ni (3)) complexes that contain lighter, diamagnetic magnesium in the place of one of the 3d transition metals. X-ray crystallography, mass spectrometry, and NMR spectroscopy unambiguously confirmed the presence of three types of molecules in the reaction mixture that reaches an equilibrium, Li2M2L6 + Li2Mg2L6 ↔ 2Li2MMgL6, upon prolonged reflux in solution. The equilibrium mixture was shown to have a nearly statistical distribution of the three molecules, and this is fully supported by the results of theoretical calculations revealing that the stabilization energies of heterotrimetallic assemblies fall exactly in between those for the parent heterobimetallic species. The LiCo0.5Ni0.5O2 quaternary oxide has been obtained in its phase-pure form by thermal decomposition of heterometallic precursor 1 at temperatures as low as 450 °C. Its chemical composition, structure, morphology, and transition metal distribution have been studied by X-ray and electron diffraction techniques and compositional energy-dispersive X-ray mapping with nanometer resolution. The work clearly illustrates the advantages of heterometallic single-source precursors over the corresponding multi-source precursors.