Exploiting redundancy in large materials datasets for efficient machine learning with less data.

Extensive efforts to gather materials data have largely overlooked potential data redundancy. In this study, we present evidence of a significant degree of redundancy across multiple large datasets for various material properties, by revealing that up to 95% of data can be safely removed from machine learning training with little impact on in-distribution prediction performance. The redundant data is related to over-represented material types and does not mitigate the severe performance degradation on out-of-distribution samples. In addition, we show that uncertainty-based active learning algorithms can construct much smaller but equally informative datasets. We discuss the effectiveness of informative data in improving prediction performance and robustness and provide insights into efficient data acquisition and machine learning training. This work challenges the "bigger is better" mentality and calls for attention to the information richness of materials data rather than a narrow emphasis on data volume.

14 examples of how LLMs can transform materials science and chemistry: a reflection on a large language model hackathon.

Large-language models (LLMs) such as GPT-4 caught the interest of many scientists. Recent studies suggested that these models could be useful in chemistry and materials science. To explore these possibilities, we organized a hackathon. This article chronicles the projects built as part of this hackathon. Participants employed LLMs for various applications, including predicting properties of molecules and materials, designing novel interfaces for tools, extracting knowledge from unstructured data, and developing new educational applications. The diverse topics and the fact that working prototypes could be generated in less than two days highlight that LLMs will profoundly impact the future of our fields. The rich collection of ideas and projects also indicates that the applications of LLMs are not limited to materials science and chemistry but offer potential benefits to a wide range of scientific disciplines.

Inverse Design of Next-Generation Superconductors Using Data-Driven Deep Generative Models.

Finding new superconductors with a high critical temperature (Tc) has been a challenging task due to computational and experimental costs. We present a diffusion model inspired by the computer vision community to generate new superconductors with unique structures and chemical compositions. Specifically, we used a crystal diffusion variational autoencoder (CDVAE) along with atomistic line graph neural network (ALIGNN) pretrained models and the Joint Automated Repository for Various Integrated Simulations (JARVIS) superconducting database of density functional theory (DFT) calculations to generate new superconductors with a high success rate. We started with a DFT data set of ∼1000 superconducting materials to train the diffusion model. We used the model to generate 3000 new structures, which along with pretrained ALIGNN screening results in 61 candidates. For the top candidates, we performed DFT calculations for validation. Such approaches go beyond funnel-like materials screening approaches and allow for the inverse design of next-generation materials.

Fast and accurate prediction of material properties with three-body tight-binding model for the periodic table.

Parametrized tight-binding models fit to first-principles calculations can provide an efficient and accurate quantum mechanical method for predicting properties of molecules and solids. However, well-tested parameter sets are generally only available for a limited number of atom combinations, making routine use of this method difficult. Furthermore, many previous models consider only simple two-body interactions, which limits accuracy. To tackle these challenges, we develop a density functional theory database of nearly 1 000 000 materials, which we use to fit a universal set of tight-binding parameters for 65 elements and their binary combinations. We include both two-body and three-body effective interaction terms in our model, plus self-consistent charge transfer, enabling our model to work for metallic, covalent, and ionic bonds with the same parameter set. To ensure predictive power, we adopt a learning framework where we repeatedly test the model on new low-energy crystal structures and then add them to the fitting data set, iterating until predictions improve. We distribute the materials database and tools developed in this paper publicly.

AtomVision: A Machine Vision Library for Atomistic Images.

Computer vision techniques have immense potential for materials design applications. In this work, we introduce an integrated and general-purpose AtomVision library that can be used to generate and curate microscopy image (such as scanning tunneling microscopy and scanning transmission electron microscopy) data sets and apply a variety of machine learning techniques. To demonstrate the applicability of this library, we (1) establish an atomistic image data set of about 10 000 materials with large structural and chemical diversity, (2) develop and compare convolutional and atomistic line graph neural network models to classify the Bravais lattices, (3) demonstrate the application of fully convolutional neural networks using U-Net architecture to pixelwise classify atom versus background, (4) use a generative adversarial network for super resolution, (5) curate an image data set on the basis of natural language processing using an open-access arXiv data set, and (6) integrate the computational framework with experimental microscopy images for Rh, Fe3O4, and SnS systems. The AtomVision library is available at https://github.com/usnistgov/atomvision.

MPpredictor: An Artificial Intelligence-Driven Web Tool for Composition-Based Material Property Prediction.

The applications of artificial intelligence, machine learning, and deep learning techniques in the field of materials science are becoming increasingly common due to their promising abilities to extract and utilize data-driven information from available data and accelerate materials discovery and design for future applications. In an attempt to assist with this process, we deploy predictive models for multiple material properties, given the composition of the material. The deep learning models described here are built using a cross-property deep transfer learning technique, which leverages source models trained on large data sets to build target models on small data sets with different properties. We deploy these models in an online software tool that takes a number of material compositions as input, performs preprocessing to generate composition-based attributes for each material, and feeds them into the predictive models to obtain up to 41 different material property values. The material property predictor is available online at http://ai.eecs.northwestern.edu/MPpredictor.

Systematic DFT+U and Quantum Monte Carlo Benchmark of Magnetic Two-Dimensional (2D) CrX3 (X = I, Br, Cl, F).

The search for two-dimensional (2D) magnetic materials has attracted a great deal of attention because of the experimental synthesis of 2D CrI3, which has a measured Curie temperature of 45 K. Often times, these monolayers have a higher degree of electron correlation and require more sophisticated methods beyond density functional theory (DFT). Diffusion Monte Carlo (DMC) is a correlated electronic structure method that has been demonstrated to be successful for calculating the electronic and magnetic properties of a wide variety of 2D and bulk systems, since it has a weaker dependence on the Hubbard parameter (U) and density functional. In this study, we designed a workflow that combines DFT +U and DMC in order to treat 2D correlated magnetic systems. We chose monolayer CrX3 (X = I, Br, Cl, F), with a stronger focus on CrI3 and CrBr3, as a case study due to the fact that they have been experimentally realized and have a finite critical temperature. With this DFT+U and DMC workflow and the analytical method of Torelli and Olsen, we estimated a maximum value of 43.56 K for the Tc of CrI3 and 20.78 K for the Tc of CrBr3, in addition to analyzing the spin densities and magnetic properties with DMC and DFT+U. We expect that running this workflow for a well-known material class will aid in the future discovery and characterization of lesser known and more complex correlated 2D magnetic materials.

High-Throughput DFT-Based Discovery of Next Generation Two-Dimensional (2D) Superconductors.

High-throughput density functional theory (DFT) calculations allow for a systematic search for conventional superconductors. With the recent interest in two-dimensional (2D) superconductors, we used a high-throughput workflow to screen over 1000 2D materials in the JARVIS-DFT database and performed electron-phonon coupling calculations, using the McMillan-Allen-Dynes formula to calculate the superconducting transition temperature (Tc) for 165 of them. Of these 165 materials, we identify 34 dynamically stable structures with transition temperatures above 5 K, including materials such as W2N3, NbO2, ZrBrO, TiClO, NaSn2S4, Mg2B4C2, and the previously unreported Mg2B4N2 (Tc = 21.8 K). Finally, we performed experiments to determine the Tc of selected layered superconductors (2H-NbSe2, 2H-NbS2, ZrSiS, FeSe) and discuss the measured results within the context of our DFT results. We aim that the outcome of this workflow can guide future computational and experimental studies of new and emerging 2D superconductors by providing a roadmap of high-throughput DFT data.

Predicting Lattice Vibrational Frequencies Using Deep Graph Neural Networks.

Lattice vibrational frequencies are related to many important materials properties such as thermal and electrical conductivity as well as superconductivity. However, computational calculation of vibrational frequencies using density functional theory methods is computationally too demanding for large number of samples in materials screening. Here we propose a deep graph neural network based algorithm for predicting crystal vibrational frequencies from crystal structures. Our algorithm addresses the variable dimension of vibrational frequency spectrum using the zero padding scheme. Benchmark studies on two data sets with 15,000 mixed-structure and 35,552 rhombohedra samples show that the aggregated R 2 scores of the prediction reach 0.554 and 0.724. We also evaluate the structural transferability by predicting the vibration frequencies for 239 individual cubic target structures. The R 2 scores for more than 40% of the targets are greater than 0.8 and can reach as high as 0.98 for the model trained with mixed samples, while the average mean absolute error is 43.69 Thz showing low transferability across structure types. Our work demonstrates the capability of deep graph neural networks to learn to predict lattice vibration frequency when sufficient number of training samples are available.

Large scale dataset of real space electronic charge density of cubic inorganic materials from density functional theory (DFT) calculations.

Driven by the big data science, material informatics has attracted enormous research interests recently along with many recognized achievements. To acquire knowledge of materials by previous experience, both feature descriptors and databases are essential for training machine learning (ML) models with high accuracy. In this regard, the electronic charge density ρ(r), which in principle determines the properties of materials at their ground state, can be considered as one of the most appropriate descriptors. However, the systematic electronic charge density ρ(r) database of inorganic materials is still in its infancy due to the difficulties in collecting raw data in experiment and the expensive first-principles based computational cost in theory. Herein, a real space electronic charge density ρ(r) database of 17,418 cubic inorganic materials is constructed by performing high-throughput density functional theory calculations. The displayed ρ(r) patterns show good agreements with those reported in previous studies, which validates our computations. Further statistical analysis reveals that it possesses abundant and diverse data, which could accelerate ρ(r) related machine learning studies. Moreover, the electronic charge density database will also assists chemical bonding identifications and promotes new crystal discovery in experiments.

Uncertainty Prediction for Machine Learning Models of Material Properties.

Uncertainty quantification in artificial intelligence (AI)-based predictions of material properties is of immense importance for the success and reliability of AI applications in materials science. While confidence intervals are commonly reported for machine learning (ML) models, prediction intervals, i.e., the evaluation of the uncertainty on each prediction, are not as frequently available. In this work, we compare three different approaches to obtain such individual uncertainty, testing them on 12 ML-physical properties. Specifically, we investigated using the quantile loss function, machine learning the prediction intervals directly, and using Gaussian processes. We identify each approach's advantages and disadvantages and end up slightly favoring the modeling of the individual uncertainties directly, as it is the easiest to fit and, in most of the cases, minimizes over- and underestimation of the predicted errors. All data for training and testing were taken from the publicly available JARVIS-DFT database, and the codes developed for computing the prediction intervals are available through the JARVIS-tools package.

Cross-property deep transfer learning framework for enhanced predictive analytics on small materials data.

Artificial intelligence (AI) and machine learning (ML) have been increasingly used in materials science to build predictive models and accelerate discovery. For selected properties, availability of large databases has also facilitated application of deep learning (DL) and transfer learning (TL). However, unavailability of large datasets for a majority of properties prohibits widespread application of DL/TL. We present a cross-property deep-transfer-learning framework that leverages models trained on large datasets to build models on small datasets of different properties. We test the proposed framework on 39 computational and two experimental datasets and find that the TL models with only elemental fractions as input outperform ML/DL models trained from scratch even when they are allowed to use physical attributes as input, for 27/39 (≈ 69%) computational and both the experimental datasets. We believe that the proposed framework can be widely useful to tackle the small data challenge in applying AI/ML in materials science.

OPTIMADE, an API for exchanging materials data.

The Open Databases Integration for Materials Design (OPTIMADE) consortium has designed a universal application programming interface (API) to make materials databases accessible and interoperable. We outline the first stable release of the specification, v1.0, which is already supported by many leading databases and several software packages. We illustrate the advantages of the OPTIMADE API through worked examples on each of the public materials databases that support the full API specification.

Quantum computation for predicting electron and phonon properties of solids.

Quantum chemistry is one of the most promising near-term applications of quantum computers. Quantum algorithms such as variational quantum eigen solver (VQE) and variational quantum deflation (VQD) algorithms have been mainly applied for molecular systems and there is a need to implement such methods for periodic solids. Using Wannier tight-binding Hamiltonian (WTBH) approaches, we demonstrate the application of VQE and VQD to accurately predict both electronic and phonon bandstructure properties of several elemental as well as multi-component solid-state materials. We apply VQE-VQD calculations for 307 spin-orbit coupling based electronic WTBHs and 933 finite-difference based phonon WTBHs. Also, we discuss a workflow for using VQD with lattice Green's function that can be used for solving dynamical mean-field theory problems. The WTBH model solvers can be used for testing other quantum algorithms and models also.

Database of Wannier tight-binding Hamiltonians using high-throughput density functional theory.

Wannier tight-binding Hamiltonians (WTBH) provide a computationally efficient way to predict electronic properties of materials. In this work, we develop a computational workflow for high-throughput Wannierization of density functional theory (DFT) based electronic band structure calculations. We apply this workflow to 1771 materials (1406 3D and 365 2D), and we create a database with the resulting WTBHs. We evaluate the accuracy of the WTBHs by comparing the Wannier band structures to directly calculated spin-orbit coupling DFT band structures. Our testing includes k-points outside the grid used in the Wannierization, providing an out-of-sample test of accuracy. We illustrate the use of WTBHs with a few example applications. We also develop a web-app that can be used to predict electronic properties on-the-fly using WTBH from our database. The tools to generate the Hamiltonian and the database of the WTB parameters are made publicly available through the websites https://github.com/usnistgov/jarvis and https://jarvis.nist.gov/jarviswtb .

Computational scanning tunneling microscope image database.

We introduce the systematic database of scanning tunneling microscope (STM) images obtained using density functional theory (DFT) for two-dimensional (2D) materials, calculated using the Tersoff-Hamann method. It currently contains data for 716 exfoliable 2D materials. Examples of the five possible Bravais lattice types for 2D materials and their Fourier-transforms are discussed. All the computational STM images generated in this work are made available on the JARVIS-STM website ( https://jarvis.nist.gov/jarvisstm ). We find excellent qualitative agreement between the computational and experimental STM images for selected materials. As a first example application of this database, we train a convolution neural network model to identify the Bravais lattice from the STM images. We believe the model can aid high-throughput experimental data analysis. These computational STM images can directly aid the identification of phases, analyzing defects and lattice-distortions in experimental STM images, as well as be incorporated in the autonomous experiment workflows.

Neural signatures of syntactic variation in speech planning.

Planning to speak is a challenge for the brain, and the challenge varies between and within languages. Yet, little is known about how neural processes react to these variable challenges beyond the planning of individual words. Here, we examine how fundamental differences in syntax shape the time course of sentence planning. Most languages treat alike (i.e., align with each other) the 2 uses of a word like "gardener" in "the gardener crouched" and in "the gardener planted trees." A minority keeps these formally distinct by adding special marking in 1 case, and some languages display both aligned and nonaligned expressions. Exploiting such a contrast in Hindi, we used electroencephalography (EEG) and eye tracking to suggest that this difference is associated with distinct patterns of neural processing and gaze behavior during early planning stages, preceding phonological word form preparation. Planning sentences with aligned expressions induces larger synchronization in the theta frequency band, suggesting higher working memory engagement, and more visual attention to agents than planning nonaligned sentences, suggesting delayed commitment to the relational details of the event. Furthermore, plain, unmarked expressions are associated with larger desynchronization in the alpha band than expressions with special markers, suggesting more engagement in information processing to keep overlapping structures distinct during planning. Our findings contrast with the observation that the form of aligned expressions is simpler, and they suggest that the global preference for alignment is driven not by its neurophysiological effect on sentence planning but by other sources, possibly by aspects of production flexibility and fluency or by sentence comprehension. This challenges current theories on how production and comprehension may affect the evolution and distribution of syntactic variants in the world's languages.

Density functional theory-based electric field gradient database.

The deviation of the electron density around the nuclei from spherical symmetry determines the electric field gradient (EFG), which can be measured by various types of spectroscopy. Nuclear Quadrupole Resonance (NQR) is particularly sensitive to the EFG. The EFGs, and by implication NQR frequencies, vary dramatically across materials. Consequently, searching for NQR spectral lines in previously uninvestigated materials represents a major challenge. Calculated EFGs can significantly aid at the search's inception. To facilitate this task, we have applied high-throughput density functional theory calculations to predict EFGs for 15187 materials in the JARVIS-DFT database. This database, which will include EFG as a standard entry, is continuously increasing. Given the large scope of the database, it is impractical to verify each calculation. However, we assess accuracy by singling out cases for which reliable experimental information is readily available and compare them to the calculations. We further present a statistical analysis of the results. The database and tools associated with our work are made publicly available by JARVIS-DFT ( https://www.ctcms.nist.gov/~knc6/JVASP.html ) and NIST-JARVIS API ( http://jarvis.nist.gov/ ).

Author Correction: Enhancing materials property prediction by leveraging computational and experimental data using deep transfer learning.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.