Atoms to Phenotypes: Molecular Design Principles of Cellular Energy Metabolism.

We report a 100-million atom-scale model of an entire cell organelle, a photosynthetic chromatophore vesicle from a purple bacterium, that reveals the cascade of energy conversion steps culminating in the generation of ATP from sunlight. Molecular dynamics simulations of this vesicle elucidate how the integral membrane complexes influence local curvature to tune photoexcitation of pigments. Brownian dynamics of small molecules within the chromatophore probe the mechanisms of directional charge transport under various pH and salinity conditions. Reproducing phenotypic properties from atomistic details, a kinetic model evinces that low-light adaptations of the bacterium emerge as a spontaneous outcome of optimizing the balance between the chromatophore's structural integrity and robust energy conversion. Parallels are drawn with the more universal mitochondrial bioenergetic machinery, from whence molecular-scale insights into the mechanism of cellular aging are inferred. Together, our integrative method and spectroscopic experiments pave the way to first-principles modeling of whole living cells.

VMD as a Platform for Interactive Small Molecule Preparation and Visualization in Quantum and Classical Simulations.

Modeling and simulation of small molecules such as drugs and biological cofactors have been both a major focus of computational chemistry for decades and a growing need among computational biophysicists who seek to investigate the interaction of different types of ligands with biomolecules. Of particular interest in this regard are quantum mechanical (QM) calculations that are used to more accurately describe such small molecules, which can be of heterogeneous structures and chemistry, either in purely QM calculations or in hybrid QM/molecular mechanics (MM) simulations. QM programs are also used to develop MM force field parameters for small molecules to be used along with established force fields for biomolecules in classical simulations. With this growing need in mind, here we report a set of software tools developed and closely integrated within the broadly used molecular visualization/analysis program, VMD, that allow the user to construct, modify, and parametrize small molecules and prepare them for QM, hybrid QM/MM, or classical simulations. The tools also provide interactive analysis and visualization capabilities in an easy-to-use and integrated environment. In this paper, we briefly report on these tools and their major features and capabilities, along with examples of how they can facilitate molecular research in computational biophysics that might be otherwise prohibitively complex.

ANARI: A 3-D Rendering API Standard.

ANARI is a new 3-D rendering API, an emerging Khronos standard that enables visualization applications to leverage the state-of-the-art rendering techniques across diverse hardware platforms and rendering engines. Visualization applications have historically embedded custom-written renderers to enable them to provide the necessary combination of features, performance, and visual fidelity required by their users. As computing power, rendering algorithms, dedicated rendering hardware acceleration operations, and associated low-level APIs have advanced, the effort and costs associated with maintaining renderers within visualization applications have risen dramatically. The rising cost and complexity associated with renderer development creates an undesirable barrier for visualization applications to be able to fully benefit from the latest rendering methods and hardware. ANARI directly addresses these challenges by providing a high-level, visualization-oriented API that abstracts low-level rendering algorithms and hardware acceleration details while providing easy and efficient access to diverse ANARI implementations, thereby enabling visualization applications to support the state-of-the-art rendering capabilities.

Intelligent resolution: Integrating Cryo-EM with AI-driven multi-resolution simulations to observe the severe acute respiratory syndrome coronavirus-2 replication-transcription machinery in action.

The severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2) replication transcription complex (RTC) is a multi-domain protein responsible for replicating and transcribing the viral mRNA inside a human cell. Attacking RTC function with pharmaceutical compounds is a pathway to treating COVID-19. Conventional tools, e.g., cryo-electron microscopy and all-atom molecular dynamics (AAMD), do not provide sufficiently high resolution or timescale to capture important dynamics of this molecular machine. Consequently, we develop an innovative workflow that bridges the gap between these resolutions, using mesoscale fluctuating finite element analysis (FFEA) continuum simulations and a hierarchy of AI-methods that continually learn and infer features for maintaining consistency between AAMD and FFEA simulations. We leverage a multi-site distributed workflow manager to orchestrate AI, FFEA, and AAMD jobs, providing optimal resource utilization across HPC centers. Our study provides unprecedented access to study the SARS-CoV-2 RTC machinery, while providing general capability for AI-enabled multi-resolution simulations at scale.

NAMD goes quantum: an integrative suite for hybrid simulations.

Hybrid methods that combine quantum mechanics (QM) and molecular mechanics (MM) can be applied to studies of reaction mechanisms in locations ranging from active sites of small enzymes to multiple sites in large bioenergetic complexes. By combining the widely used molecular dynamics and visualization programs NAMD and VMD with the quantum chemistry packages ORCA and MOPAC, we created an integrated, comprehensive, customizable, and easy-to-use suite (http://www.ks.uiuc.edu/Research/qmmm). Through the QwikMD interface, setup, execution, visualization, and analysis are streamlined for all levels of expertise.

AI-driven multiscale simulations illuminate mechanisms of SARS-CoV-2 spike dynamics.

We develop a generalizable AI-driven workflow that leverages heterogeneous HPC resources to explore the time-dependent dynamics of molecular systems. We use this workflow to investigate the mechanisms of infectivity of the SARS-CoV-2 spike protein, the main viral infection machinery. Our workflow enables more efficient investigation of spike dynamics in a variety of complex environments, including within a complete SARS-CoV-2 viral envelope simulation, which contains 305 million atoms and shows strong scaling on ORNL Summit using NAMD. We present several novel scientific discoveries, including the elucidation of the spike's full glycan shield, the role of spike glycans in modulating the infectivity of the virus, and the characterization of the flexible interactions between the spike and the human ACE2 receptor. We also demonstrate how AI can accelerate conformational sampling across different systems and pave the way for the future application of such methods to additional studies in SARS-CoV-2 and other molecular systems.

Scalable molecular dynamics on CPU and GPU architectures with NAMD.

NAMDis a molecular dynamics program designed for high-performance simulations of very large biological objects on CPU- and GPU-based architectures. NAMD offers scalable performance on petascale parallel supercomputers consisting of hundreds of thousands of cores, as well as on inexpensive commodity clusters commonly found in academic environments. It is written in C++ and leans on Charm++ parallel objects for optimal performance on low-latency architectures. NAMD is a versatile, multipurpose code that gathers state-of-the-art algorithms to carry out simulations in apt thermodynamic ensembles, using the widely popular CHARMM, AMBER, OPLS, and GROMOS biomolecular force fields. Here, we review the main features of NAMD that allow both equilibrium and enhanced-sampling molecular dynamics simulations with numerical efficiency. We describe the underlying concepts utilized by NAMD and their implementation, most notably for handling long-range electrostatics; controlling the temperature, pressure, and pH; applying external potentials on tailored grids; leveraging massively parallel resources in multiple-copy simulations; and hybrid quantum-mechanical/molecular-mechanical descriptions. We detail the variety of options offered by NAMD for enhanced-sampling simulations aimed at determining free-energy differences of either alchemical or geometrical transformations and outline their applicability to specific problems. Last, we discuss the roadmap for the development of NAMD and our current efforts toward achieving optimal performance on GPU-based architectures, for pushing back the limitations that have prevented biologically realistic billion-atom objects to be fruitfully simulated, and for making large-scale simulations less expensive and easier to set up, run, and analyze. NAMD is distributed free of charge with its source code at www.ks.uiuc.edu.

Scalable Analysis of Authentic Viral Envelopes on FRONTERA.

Enveloped viruses, such as SARS-CoV-2, infect cells via fusion of their envelope with the host membrane. By employing molecular simulations to characterize viral envelopes, researchers can gain insights into key determinants of infection. Here, the Frontera supercomputer is leveraged for large-scale modeling and analysis of authentic viral envelopes, whose lipid compositions are complex and realistic. Visual Molecular Dynamics (VMD) with support for MPI is employed, overcoming previous computational limitations and enabling investigation into virus biology at an unprecedented scale. The techniques applied here to an authentic HIV-1 envelope at two levels of spatial resolution (29 million particles and 280 million atoms) are broadly applicable to the study of other viruses. The authors are actively employing these techniques to develop and characterize an authentic SARS-CoV-2 envelope. A general framework for carrying out scalable analysis of simulation trajectories on Frontera is presented, expanding the utility of the machine in humanity's ongoing fight against infectious diseases.

Multiscale modeling and cinematic visualization of photosynthetic energy conversion processes from electronic to cell scales.

Conversion of sunlight into chemical energy, namely photosynthesis, is the primary energy source of life on Earth. A visualization depicting this process, based on multiscale computational models from electronic to cell scales, is presented in the form of an excerpt from the fulldome show Birth of Planet Earth. This accessible visual narrative shows a lay audience, including children, how the energy of sunlight is captured, converted, and stored through a chain of proteins to power living cells. The visualization is the result of a multi-year collaboration among biophysicists, visualization scientists, and artists, which, in turn, is based on a decade-long experimental-computational collaboration on structural and functional modeling that produced an atomic detail description of a bacterial bioenergetic organelle, the chromatophore. Software advancements necessitated by this project have led to significant performance and feature advances, including hardware-accelerated cinematic ray tracing and instanced visualizations for efficient cell-scale modeling. The energy conversion steps depicted feature an integration of function from electronic to cell levels, spanning nearly 12 orders of magnitude in time scales. This atomic detail description uniquely enables a modern retelling of one of humanity's earliest stories-the interplay between light and life.

High-Performance Analysis of Biomolecular Containers to Measure Small-Molecule Transport, Transbilayer Lipid Diffusion, and Protein Cavities.

Compartmentalization is a central theme in biology. Cells are composed of numerous membrane-enclosed structures, evolved to facilitate specific biochemical processes; viruses act as containers of genetic material, optimized to drive infection. Molecular dynamics simulations provide a mechanism to study biomolecular containers and the influence they exert on their environments; however, trajectory analysis software generally lacks knowledge of container interior versus exterior. Further, many relevant container analyses involve large-scale particle tracking endeavors, which may become computationally prohibitive with increasing system size. Here, a novel method based on 3-D ray casting is presented, which rapidly classifies the space surrounding biomolecular containers of arbitrary shape, enabling fast determination of the identities and counts of particles (e.g., solvent molecules) found inside and outside. The method is broadly applicable to the study of containers and enables high-performance characterization of properties such as solvent density, small-molecule transport, transbilayer lipid diffusion, and topology of protein cavities. The method is implemented in VMD, a widely used simulation analysis tool that supports personal computers, clouds, and parallel supercomputers, including ORNL's Summit and Titan and NCSA's Blue Waters, where the method can be employed to efficiently analyze trajectories encompassing millions of particles. The ability to rapidly characterize the spatial relationships of particles relative to a biomolecular container over many trajectory frames, irrespective of large particle counts, enables analysis of containers on a scale that was previously unfeasible, at a level of accuracy that was previously unattainable.

TopoGromacs: Automated Topology Conversion from CHARMM to GROMACS within VMD.

Molecular dynamics (MD) simulation engines use a variety of different approaches for modeling molecular systems with force fields that govern their dynamics and describe their topology. These different approaches introduce incompatibilities between engines, and previously published software bridges the gaps between many popular MD packages, such as between CHARMM and AMBER or GROMACS and LAMMPS. While there are many structure building tools available that generate topologies and structures in CHARMM format, only recently have mechanisms been developed to convert their results into GROMACS input. We present an approach to convert CHARMM-formatted topology and parameters into a format suitable for simulation with GROMACS by expanding the functionality of TopoTools, a plugin integrated within the widely used molecular visualization and analysis software VMD. The conversion process was diligently tested on a comprehensive set of biological molecules in vacuo. The resulting comparison between energy terms shows that the translation performed was lossless as the energies were unchanged for identical starting configurations. By applying the conversion process to conventional benchmark systems that mimic typical modestly sized MD systems, we explore the effect of the implementation choices made in CHARMM, NAMD, and GROMACS. The newly available automatic conversion capability breaks down barriers between simulation tools and user communities and allows users to easily compare simulation programs and leverage their unique features without the tedium of constructing a topology twice.

Atomic Detail Visualization of Photosynthetic Membranes with GPU-Accelerated Ray Tracing.

The cellular process responsible for providing energy for most life on Earth, namely photosynthetic light-harvesting, requires the cooperation of hundreds of proteins across an organelle, involving length and time scales spanning several orders of magnitude over quantum and classical regimes. Simulation and visualization of this fundamental energy conversion process pose many unique methodological and computational challenges. We present, in two accompanying movies, light-harvesting in the photosynthetic apparatus found in purple bacteria, the so-called chromatophore. The movies are the culmination of three decades of modeling efforts, featuring the collaboration of theoretical, experimental, and computational scientists. We describe the techniques that were used to build, simulate, analyze, and visualize the structures shown in the movies, and we highlight cases where scientific needs spurred the development of new parallel algorithms that efficiently harness GPU accelerators and petascale computers.

Assembly of Nsp1 nucleoporins provides insight into nuclear pore complex gating.

Nuclear pore complexes (NPCs) form gateways for material transfer across the nuclear envelope of eukaryotic cells. Disordered proteins, rich in phenylalanine-glycine repeat motifs (FG-nups), form the central transport channel. Understanding how nups are arranged in the interior of the NPC may explain how NPC functions as a selectivity filter for transport of large molecules and a sieve-like filter for diffusion of small molecules (<9 nm or 40 kDa). We employed molecular dynamics to model the structures formed by various assemblies of one kind of nup, namely the 609-aa-long FG domain of Nsp1 (Nsp1-FG). The simulations started from different initial conformations and geometrical arrangements of Nsp1-FGs. In all cases Nsp1-FGs collectively formed brush-like structures with bristles made of bundles of 2-27 nups, however, the bundles being cross-linked through single nups leaving one bundle and joining a nearby one. The degree of cross-linking varies with different initial nup conformations and arrangements. Structural analysis reveals that FG-repeats of the nups not only involve formation of bundle structures, but are abundantly present in cross-linking regions where the epitopes of FG-repeats are highly accessible. Large molecules that are assisted by transport factors (TFs) are selectively transported through NPC apparently by binding to FG-nups through populated FG-binding pockets on the TF surface. Therefore, our finding suggests that TFs bind concertedly to multiple FGs in cross-linking regions and break-up the bundles to create wide pores for themselves and their cargoes to pass. In addition, the cross-linking between Nsp1-FG bundles, arising from simulations, is found to set a molecular size limit of <9 nm (40 kDa) for passive diffusion of molecules. Our simulations suggest that the NPC central channel, near the periphery where tethering of nups is dominant, features brush-like moderately cross-linked bundles, but in the central region, where tethering loses its effect, features a sieve-like structure of bundles and frequent cross-links.

Runtime and Architecture Support for Efficient Data Exchange in Multi-Accelerator Applications.

Heterogeneous parallel computing applications often process large data sets that require multiple GPUs to jointly meet their needs for physical memory capacity and compute throughput. However, the lack of high-level abstractions in previous heterogeneous parallel programming models force programmers to resort to multiple code versions, complex data copy steps and synchronization schemes when exchanging data between multiple GPU devices, which results in high software development cost, poor maintainability, and even poor performance. This paper describes the HPE runtime system, and the associated architecture support, which enables a simple, efficient programming interface for exchanging data between multiple GPUs through either interconnects or cross-node network interfaces. The runtime and architecture support presented in this paper can also be used to support other types of accelerators. We show that the simplified programming interface reduces programming complexity. The research presented in this paper started in 2009. It has been implemented and tested extensively in several generations of HPE runtime systems as well as adopted into the NVIDIA GPU hardware and drivers for CUDA 4.0 and beyond since 2011. The availability of real hardware that support key HPE features gives rise to a rare opportunity for studying the effectiveness of the hardware support by running important benchmarks on real runtime and hardware. Experimental results show that in a exemplar heterogeneous system, peer DMA and double-buffering, pinned buffers, and software techniques can improve the inter-accelerator data communication bandwidth by 2×. They can also improve the execution speed by 1.6× for a 3D finite difference, 2.5× for 1D FFT, and 1.6× for merge sort, all measured on real hardware. The proposed architecture support enables the HPE runtime to transparently deploy these optimizations under simple portable user code, allowing system designers to freely employ devices of different capabilities. We further argue that simple interfaces such as HPE are needed for most applications to benefit from advanced hardware features in practice.

Simulation of reaction diffusion processes over biologically relevant size and time scales using multi-GPU workstations.

Simulation of in vivo cellular processes with the reaction-diffusion master equation (RDME) is a computationally expensive task. Our previous software enabled simulation of inhomogeneous biochemical systems for small bacteria over long time scales using the MPD-RDME method on a single GPU. Simulations of larger eukaryotic systems exceed the on-board memory capacity of individual GPUs, and long time simulations of modest-sized cells such as yeast are impractical on a single GPU. We present a new multi-GPU parallel implementation of the MPD-RDME method based on a spatial decomposition approach that supports dynamic load balancing for workstations containing GPUs of varying performance and memory capacity. We take advantage of high-performance features of CUDA for peer-to-peer GPU memory transfers and evaluate the performance of our algorithms on state-of-the-art GPU devices. We present parallel e ciency and performance results for simulations using multiple GPUs as system size, particle counts, and number of reactions grow. We also demonstrate multi-GPU performance in simulations of the Min protein system in E. coli. Moreover, our multi-GPU decomposition and load balancing approach can be generalized to other lattice-based problems.

Stable small quantum dots for synaptic receptor tracking on live neurons.

We developed a coating method to produce functionalized small quantum dots (sQDs), about 9 nm in diameter, that were stable for over a month. We made sQDs in four emission wavelengths, from 527 to 655 nm and with different functional groups. AMPA receptors on live neurons were labeled with sQDs and postsynaptic density proteins were visualized with super-resolution microscopy. Their diffusion behavior indicates that sQDs access the synaptic clefts significantly more often than commercial QDs.

Lattice Microbes: high-performance stochastic simulation method for the reaction-diffusion master equation.

Spatial stochastic simulation is a valuable technique for studying reactions in biological systems. With the availability of high-performance computing (HPC), the method is poised to allow integration of data from structural, single-molecule and biochemical studies into coherent computational models of cells. Here, we introduce the Lattice Microbes software package for simulating such cell models on HPC systems. The software performs either well-stirred or spatially resolved stochastic simulations with approximated cytoplasmic crowding in a fast and efficient manner. Our new algorithm efficiently samples the reaction-diffusion master equation using NVIDIA graphics processing units and is shown to be two orders of magnitude faster than exact sampling for large systems while maintaining an accuracy of !0.1%. Display of cell models and animation of reaction trajectories involving millions of molecules is facilitated using a plug-in to the popular VMD visualization platform. The Lattice Microbes software is open source and available for download at http://www.scs.illinois.edu/schulten/lm

Dynamics of chromosome organization in a minimal bacterial cell.

Computational models of cells cannot be considered complete unless they include the most fundamental process of life, the replication and inheritance of genetic material. By creating a computational framework to model systems of replicating bacterial chromosomes as polymers at 10 bp resolution with Brownian dynamics, we investigate changes in chromosome organization during replication and extend the applicability of an existing whole-cell model (WCM) for a genetically minimal bacterium, JCVI-syn3A, to the entire cell-cycle. To achieve cell-scale chromosome structures that are realistic, we model the chromosome as a self-avoiding homopolymer with bending and torsional stiffnesses that capture the essential mechanical properties of dsDNA in Syn3A. In addition, the conformations of the circular DNA must avoid overlapping with ribosomes identitied in cryo-electron tomograms. While Syn3A lacks the complex regulatory systems known to orchestrate chromosome segregation in other bacteria, its minimized genome retains essential loop-extruding structural maintenance of chromosomes (SMC) protein complexes (SMC-scpAB) and topoisomerases. Through implementing the effects of these proteins in our simulations of replicating chromosomes, we find that they alone are sufficient for simultaneous chromosome segregation across all generations within nested theta structures. This supports previous studies suggesting loop-extrusion serves as a near-universal mechanism for chromosome organization within bacterial and eukaryotic cells. Furthermore, we analyze ribosome diffusion under the influence of the chromosome and calculate in silico chromosome contact maps that capture inter-daughter interactions. Finally, we present a methodology to map the polymer model of the chromosome to a Martini coarse-grained representation to prepare molecular dynamics models of entire Syn3A cells, which serves as an ultimate means of validation for cell states predicted by the WCM.

Application Experiences on a GPU-Accelerated Arm-based HPC Testbed.

This paper assesses and reports the experience of ten teams working to port, validate, and benchmark several High Performance Computing applications on a novel GPU-accelerated Arm testbed system. The testbed consists of eight NVIDIA Arm HPC Developer Kit systems, each one equipped with a server-class Arm CPU from Ampere Computing and two data center GPUs from NVIDIA Corp. The systems are connected together using InfiniBand interconnect. The selected applications and mini-apps are written using several programming languages and use multiple accelerator-based programming models for GPUs such as CUDA, OpenACC, and OpenMP offloading. Working on application porting requires a robust and easy-to-access programming environment, including a variety of compilers and optimized scientific libraries. The goal of this work is to evaluate platform readiness and assess the effort required from developers to deploy well-established scientific workloads on current and future generation Arm-based GPU-accelerated HPC systems. The reported case studies demonstrate that the current level of maturity and diversity of software and tools is already adequate for large-scale production deployments.

py-MCMD: Python Software for Performing Hybrid Monte Carlo/Molecular Dynamics Simulations with GOMC and NAMD.

py-MCMD, an open-source Python software, provides a robust workflow layer that manages communication of relevant system information between the simulation engines NAMD and GOMC and generates coherent thermodynamic properties and trajectories for analysis. To validate the workflow and highlight its capabilities, hybrid Monte Carlo/molecular dynamics (MC/MD) simulations are performed for SPC/E water in the isobaric-isothermal (NPT) and grand canonical (GC) ensembles as well as with Gibbs ensemble Monte Carlo (GEMC). The hybrid MC/MD approach shows close agreement with reference MC simulations and has a computational efficiency that is 2 to 136 times greater than traditional Monte Carlo simulations. MC/MD simulations performed for water in a graphene slit pore illustrate significant gains in sampling efficiency when the coupled-decoupled configurational-bias MC (CD-CBMC) algorithm is used compared with simulations using a single unbiased random trial position. Simulations using CD-CBMC reach equilibrium with 25 times fewer cycles than simulations using a single unbiased random trial position, with a small increase in computational cost. In a more challenging application, hybrid grand canonical Monte Carlo/molecular dynamics (GCMC/MD) simulations are used to hydrate a buried binding pocket in bovine pancreatic trypsin inhibitor. Water occupancies produced by GCMC/MD simulations are in close agreement with crystallographically identified positions, and GCMC/MD simulations have a computational efficiency that is 5 times better than MD simulations. py-MCMD is available on GitHub at https://github.com/GOMC-WSU/py-MCMD.