CodonBert large language model for mRNA vaccines.

mRNA-based vaccines and therapeutics are gaining popularity and usage across a wide range of conditions. One of the critical issues when designing such mRNAs is sequence optimization. Even small proteins or peptides can be encoded by an enormously large number of mRNAs. The actual mRNA sequence can have a large impact on several properties including expression, stability, immunogenicity, and more. To enable the selection of an optimal sequence, we developed CodonBERT, a large language model (LLM) for mRNAs. Unlike prior models, CodonBERT uses codons as inputs which enables it to learn better representations. CodonBERT was trained using more than 10 million mRNA sequences from a diverse set of organisms. The resulting model captures important biological concepts. CodonBERT can also be extended to perform prediction tasks for various mRNA properties. CodonBERT outperforms previous mRNA prediction methods including on a new flu vaccine dataset.

Representations of lipid nanoparticles using large language models for transfection efficiency prediction.

MOTIVATION: Lipid nanoparticles (LNPs) are the most widely used vehicles for mRNA vaccine delivery. The structure of the lipids composing the LNPs can have a major impact on the effectiveness of the mRNA payload. Several properties should be optimized to improve delivery and expression including biodegradability, synthetic accessibility, and transfection efficiency. RESULTS: To optimize LNPs, we developed and tested models that enable the virtual screening of LNPs with high transfection efficiency. Our best method uses the lipid Simplified Molecular-Input Line-Entry System (SMILES) as inputs to a large language model. Large language model-generated embeddings are then used by a downstream gradient-boosting classifier. As we show, our method can more accurately predict lipid properties, which could lead to higher efficiency and reduced experimental time and costs. AVAILABILITY AND IMPLEMENTATION: Code and data links available at: https://github.com/Sanofi-Public/LipoBART.

Structure prediction of epitaxial inorganic interfaces by lattice and surface matching with Ogre.

We present a new version of the Ogre open source Python package with the capability to perform structure prediction of epitaxial inorganic interfaces by lattice and surface matching. In the lattice matching step, a scan over combinations of substrate and film Miller indices is performed to identify the domain-matched interfaces with the lowest mismatch. Subsequently, surface matching is conducted by Bayesian optimization to find the optimal interfacial distance and in-plane registry between the substrate and the film. For the objective function, a geometric score function is proposed based on the overlap and empty space between atomic spheres at the interface. The score function reproduces the results of density functional theory (DFT) at a fraction of the computational cost. The optimized interfaces are pre-ranked using a score function based on the similarity of the atomic environment at the interface to the bulk environment. Final ranking of the top candidate structures is performed with DFT. Ogre streamlines DFT calculations of interface energies and electronic properties by automating the construction of interface models. The application of Ogre is demonstrated for two interfaces of interest for quantum computing and spintronics, Al/InAs and Fe/InSb.

Ogre: A Python package for molecular crystal surface generation with applications to surface energy and crystal habit prediction.

We present Ogre, an open-source code for generating surface slab models from bulk molecular crystal structures. Ogre is written in Python and interfaces with the FHI-aims code to calculate surface energies at the level of density functional theory (DFT). The input of Ogre is the geometry of the bulk molecular crystal. The surface is cleaved from the bulk structure with the molecules on the surface kept intact. A slab model is constructed according to the user specifications for the number of molecular layers and the length of the vacuum region. Ogre automatically identifies all symmetrically unique surfaces for the user-specified Miller indices and detects all possible surface terminations. Ogre includes utilities to analyze the surface energy convergence and Wulff shape of the molecular crystal. We present the application of Ogre to three representative molecular crystals: the pharmaceutical aspirin, the organic semiconductor tetracene, and the energetic material HMX. The equilibrium crystal shapes predicted by Ogre are in agreement with experimentally grown crystals, demonstrating that DFT produces satisfactory predictions of the crystal habit for diverse classes of molecular crystals.

Best practices for first-principles simulations of epitaxial inorganic interfaces.

At an interface between two materials physical properties and functionalities may be achieved, which would not exist in either material alone. Epitaxial inorganic interfaces are at the heart of semiconductor, spintronic, and quantum devices. First principles simulations based on density functional theory (DFT) can help elucidate the electronic and magnetic properties of interfaces and relate them to the structure and composition at the atomistic scale. Furthermore, DFT simulations can predict the structure and properties of candidate interfaces and guide experimental efforts in promising directions. However, DFT simulations of interfaces can be technically elaborate and computationally expensive. To help researchers embarking on such simulations, this review covers best practices for first principles simulations of epitaxial inorganic interfaces, including DFT methods, interface model construction, interface structure prediction, and analysis and visualization tools.