Material Design of Porous Hydroxyapatite Ceramics via Inverse Analysis of an Estimation Model for Bone-Forming Ability Based on Machine Learning and Experimental Validation of Biological Hard Tissue Responses.

Hydroxyapatite and ß-tricalcium phosphate have been clinically applied as artificial bone materials due to their high biocompatibility. The development of artificial bones requires the verification of safety and efficacy through animal experiments; however, from the viewpoint of animal welfare, it is necessary to reduce the number of animal experiments. In this study, we utilized machine learning to construct a model that estimates the bone-forming ability of bioceramics from material fabrication conditions, material properties, and in vivo experimental conditions. We succeeded in constructing two models: 'Model 1', which predicts material properties from their fabrication conditions, and 'Model 2', which predicts the bone-formation rate from material properties and in vivo experimental conditions. The inclusion of full width at half maximum (FWHM) in the feature of Model 2 showed an improvement in accuracy. Furthermore, the results of the feature importance showed that the FWHMs were the most important. By an inverse analysis of the two models, we proposed candidates for material fabrication conditions to achieve target values of the bone-formation rate. Under the proposed conditions, the material properties of the fabricated material were consistent with the estimated material properties. Furthermore, a comparison between bone-formation rates after 12 weeks of implantation in the porcine tibia and the estimated bone-formation rate. This result showed that the actual bone-formation rates existed within the error range of the estimated bone-formation rates, indicating that machine learning consistently predicts the results of animal experiments using material fabrication conditions. We believe that these findings will lead to the establishment of alternative animal experiments to replace animal experiments in the development of artificial bones.

Molecular Design of Novel Herbicide and Insecticide Seed Compounds with Machine Learning.

Pesticides are widely used to improve crop productivity by eliminating weeds and pests. Conventional pesticide development involves synthesizing compounds, testing their activities, and studying their effects on the ecosystem. However, as pesticide discovery has an extremely low success rate, many compounds must be synthesized and tested. To overcome the high human, financial, and time costs of this process, machine learning is attracting increasing attention. In this study, we used machine learning for the molecular design of novel seed compounds for herbicides and insecticides. Classification models were constructed by using compounds that had been tested as herbicides and insecticides, and an inverse analysis of the constructed models was conducted. In the molecular design of herbicides, we proposed 186 new samples as herbicides using ensemble learning and a method for expressing explanatory variables that consider the relationships among eight weed species. For the molecular design of insecticides, we used undersampling and ensemble learning for the analysis of unbalanced data. Based on approximately 340,000 compounds, 12 potential insecticides were proposed, of which 2 exhibited actual activity when tested. These results demonstrate the potential of the developed machine-learning method for rapidly identifying novel herbicides and insecticides.

Evaluation and Optimization Methods for Applicability Domain Methods and Their Hyperparameters, Considering the Prediction Performance of Machine Learning Models.

In molecular, material, and process design and control, the applicability domain (AD) of a mathematical model y = f(x) between properties, activities, and features x is constructed. As there are multiple AD methods, each with its own set of hyperparameters, it is necessary to select an appropriate AD method and hyperparameters for each data set and mathematical model. However, there is no method for optimizing the AD model. This study proposes a method for evaluating and optimizing the AD model for each data set and a mathematical model. Using the predictions of double cross-validation with all samples, the relationship between coverage and root-mean-squared error (RMSE) was calculated for all combinations of AD methods and their hyperparameters, and the area under the coverage and RMSE curve (AUCR) was calculated. The AD model with the lowest AUCR value was selected as the optimal fit for the mathematical model. The proposed method was validated using eight data sets, including molecules, materials, and spectra, demonstrating that the proposed method could generate optimal AD models for all data sets. The Python code for the proposed method is available at https://github.com/hkaneko1985/dcekit.

Enhancing the Search Performance of Bayesian Optimization by Creating Different Descriptor Datasets Using Density Functional Theory.

Descriptors calculated from molecular structure information can be used as explanatory variables in Bayesian optimization (BO). Even though structural and descriptor information can be obtained from various databases for general compounds, information on highly confidential compounds such as pharmaceutical intermediates and active pharmaceutical ingredients cannot be retrieved from these databases. In particular, determining the stable structure and electronic state of a compound via quantum chemical calculations from descriptor information requires considerable computational time. Although descriptor information can be obtained using density functional theory (DFT), which has a relatively light computational load, only conventional combinations of basis sets and functionals can be selected before experiments instead of the best ones. Few studies have discussed these effects on the search performance of BO, and good search performance is highly dependent on the application. Therefore, we developed a method to improve the search performance of BO by using descriptors computed from several combinations of basis sets and functionals. The dataset obtained from averaging multiple descriptor sets exhibited better BO search performance than that of a single descriptor dataset. In addition, the more descriptor sets used for averaging, the better the search performance. This method has a relatively small computational load and can be easily used by those who are unfamiliar with quantum chemical calculations.

Catalyst Design and Feature Engineering to Improve Selectivity and Reactivity in Two Simultaneous Cross-Coupling Reactions.

Highly active catalysts are required in numerous industrial fields; therefore, to minimize costs and development time, catalyst design using machine learning has attracted significant attention. This study focused on a reaction system where two types of cross-coupling reactions, namely, Buchwald-Hartwig type cross-coupling (BHCC) and Suzuki-Miyaura type cross-coupling (SMCC) reactions, occur simultaneously. Constructing a machine-learning model that considers all experimental conditions is essential to accurately predict the product yield for both the BHCC and the SMCC reactions. The objective of this study was to establish explanatory variables x that considered all experimental conditions within the reaction system involving simultaneous cross-couplings and to design catalysts that achieve the target yield and the development of novel reactions. To accomplish this, Bayesian optimization was combined with established variables x to design new catalysts and enhance reaction selectivity. Moreover, the catalyst design in this study successfully pioneered new reactions involving Cu, Rh, and Pt catalysts in a reaction system that did not previously react with transition metals other than Ni or Pd.

Robust Design of a Dimethyl Ether Production Process Using Process Simulation and Robust Bayesian Optimization.

As greenhouse gases such as CO2 continue to promote global warming, the reduction of CO2 emissions is attracting increasing attention. In this study, we design a process for producing dimethyl ether (DME), which is a promising means of using CO2 as a resource. Design variables such as temperature and pressure need to be optimized to reduce CO2 emissions while maintaining high product purity and DME production. Conventional process designs determine these design variables from the chemical background and through trial-and-error simulations, which are very time-consuming. The proposed method optimizes the design variables efficiently by repeating the process simulations and selecting promising candidates for the design variables using machine learning. For an adaptive design of experiments, Bayesian optimization is used to achieve the objectives of the DME process while efficiently optimizing the design variables. In addition, we also optimize the design variables considering variations in the temperature and pressure data, meaning robust Bayesian optimization. The proposed method successfully identifies design variables that satisfy all experimental targets in an average of 54 simulations while achieving 100% of the targets with product purity 0.95-1.00, amount of DME in the product 350-845 kmol/h, and CO2 emissions 0-835 kmol/h, confirming the effectiveness of the proposed robust Bayesian optimization method.

Predictive Modeling of HMG-CoA Reductase Inhibitory Activity and Design of New HMG-CoA Reductase Inhibitors.

As blood cholesterol increases, it accumulates in the intima of blood vessels, elevating the risk of atherosclerosis and coronary artery disease. Drugs that inhibit enzymes essential for cholesterol synthesis are effective in improving blood cholesterol levels. Statins are used to treat hypercholesterolemia as they inhibit 3-hydroxyl-3-methylglutaryl coenzyme A (HMG-CoA) reductase (HMGR), the rate-limiting enzyme in cholesterol synthesis. Statins are known to exert their effects by translocating to the liver, where they are taken up by the organic anion transporting polypeptide 1B1 (OATP1B1). Therefore, we hypothesized that a compound with high HMGR inhibitory activity and high affinity for OATP1B1 would be an excellent new therapeutic agent for hypercholesterolemia with increased liver selectivity and fewer side effects. In this study, we developed two models for predicting HMGR inhibitory activity and OATP1B1 affinity to propose the chemical structure of a new therapeutic agent for hypercholesterolemia with both high inhibitory activity and high liver selectivity. HMGR inhibitory activity and OATP1B1 affinity prediction models were constructed with high prediction accuracy for the test data: r2 = 0.772 and 0.768, respectively. New chemical structures were then input into these models to search for candidate compounds. We found compounds with higher HMGR inhibitory activity and OATP1B1 affinity than rosuvastatin, the most recently developed statin drug, and compounds that did not have a common structure of statins with high HMGR inhibitory activity.

Interpretation of Machine Learning Models for Data Sets with Many Features Using Feature Importance.

Feature importance (FI) is used to interpret the machine learning model y = f(x) constructed between the explanatory variables or features, x, and the objective variables, y. For a large number of features, interpreting the model in the order of increasing FI is inefficient when there are similarly important features. Therefore, in this study, a method is developed to interpret models by considering the similarities between the features in addition to the FI. The cross-validated permutation feature importance (CVPFI), which can be calculated using any machine learning method and can handle multicollinearity problems, is used as the FI, while the absolute correlation and maximal information coefficients are used as metrics of feature similarity. Machine learning models could be effectively interpreted by considering the features from the Pareto fronts, where CVPFI is large and the feature similarity is small. Analyses of actual molecular and material data sets confirm that the proposed method enables the accurate interpretation of machine learning models.

Nanoscale chemical patterning of graphite at different length scales.

Chemical patterning surfaces is relevant in several different domains of science and technology with exciting possibilities in electronics, catalysis, sensing, and photonics. Here, we present a novel strategy for chemical patterning of graphite using a combination of covalent and non-covalent approaches. Building on our previous work, where self-assembled monolayers of linear alkanes were used as sacrificial masks for directing the covalent anchoring of aryl groups to the graphite surface in sub-10 nm arrays, we present a modified design of a template alkane with alkoxy terminal groups which allowed better pattern transfer fidelity in comparison to simple linear alkanes. We also explored the use of chronoamperometry (CA) instead of previously used cyclic voltammetry (CV) for the functionalization process, which enabled patterning of the graphite surface at two-different length scales: few hundred nanometer circular patterns interspersed with sub-10 nm linear arrays. The covalent chemical patterning process has been studied in detail using CV and CA measurements whereas the patterned substrates have been thoroughly characterized using Raman spectroscopy, scanning tunnelling microscopy (STM) and atomic force microscopy (AFM). Based on the comparison between the pattern transfer fidelity of previously studied alkanes and newly synthesized alkoxy alkane, we discuss plausible molecular mechanism of pattern transfer.

Molecular Descriptors, Structure Generation, and Inverse QSAR/QSPR Based on SELFIES.

For inverse QSAR/QSPR in conventional molecular design, several chemical structures must be generated and their molecular descriptors must be calculated. However, there is no one-to-one correspondence between the generated chemical structures and molecular descriptors. In this paper, molecular descriptors, structure generation, and inverse QSAR/QSPR based on self-referencing embedded strings (SELFIES), a 100% robust molecular string representation, are proposed. A one-hot vector is converted from SELFIES to SELFIES descriptors x, and an inverse analysis of the QSAR/QSPR model y = f(x) with the objective variable y and molecular descriptor x is conducted. Thus, x values that achieve a target y value are obtained. Based on these values, SELFIES strings or molecules are generated, meaning that inverse QSAR/QSPR is performed successfully. The SELFIES descriptors and SELFIES-based structure generation are verified using datasets of actual compounds. The successful construction of SELFIES-descriptor-based QSAR/QSPR models with predictive abilities comparable to those of models based on other fingerprints is confirmed. A large number of molecules with one-to-one relationships with the values of the SELFIES descriptors are generated. Furthermore, as a case study of inverse QSAR/QSPR, molecules with target y values are generated successfully. The Python code for the proposed method is available at https://github.com/hkaneko1985/dcekit.

Spatially Controlled Aryl Radical Grafting of Graphite Surfaces Guided by Self-Assembled Molecular Networks of Linear Alkane Derivatives: The Importance of Conformational Dynamics.

The covalent functionalization of carbon surfaces with nanometer-scale precision is of interest because of its potential in a range of applications. We herein report the controlled grafting of graphite surfaces using electrochemically generated aryl radicals templated by self-assembled molecular networks (SAMNs) of bisalkylurea derivatives. A bisalkylurea derivative having two butoxy units acts as a template for the covalent functionalization of aryl groups in between self-assembled rows of this molecule. In contrast, grafting occurs without a spatial order when an SAMN of bis(tetradecyl)urea was used as a template. This indicates that a degree of dynamics at the alkyl termini is required to favor controlled covalent attachment, a situation that is suppressed by strong intrarow intermolecular interactions resulting from the hydrogen bonding of the urea groups, but favored by terminal short alkoxy groups. The present information is useful for understanding the mechanism of the template-guided aryl radical grafting and the molecular design of new generations of template molecules.

Selecting optimum miRNA panel for miRNA signature-based companion diagnostic model to predict the response of R-CHOP treatment in diffuse large B-cell lymphoma.

Diffuse large B-cell lymphoma (DLBCL) is the most common type of malignant lymphoma. Although the first-line treatment, R-CHOP treatment, shows efficacy in approximately 80% of patients with DLBCL, some patients have refractory disease or relapse after the initial response to therapy, resulting in a significantly poorer prognosis. In this study, we developed a microRNA (miRNA) signature-based companion diagnostic model to predict the response of patients with DLBCL to R-CHOP treatment by integrating two clinical study datasets. To select the optimum miRNA combination as a panel, we examined three feature selection methods (p-value-based ranking, stepwise method, and Boruta), together with 11 types of classifiers systematically. Boruta selection enabled a higher area under the curve (AUC) with a lower number of miRNAs compared with other feature selection methods, leading to an AUC of 0.751 via the random forest classifier using 36 miRNAs. Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway enrichment analysis suggested that Boruta avoided multiple selection of miRNAs with similar functions, thereby preventing the decrease in diagnostic ability via collinearity. The AUC value first increased with an increasing number of miRNAs and then became almost constant at approximately 30 miRNAs, suggesting the existence of the optimum number of miRNAs as a panel for future clinical translation of multiple miRNA-based diagnostics.

De Novo Direct Inverse QSPR/QSAR: Chemical Variational Autoencoder and Gaussian Mixture Regression Models.

Herein, we propose a de novo direct inverse quantitative structure-property relationship/quantitative structure-activity relationship (QSPR/QSAR) analysis method, based on the chemical variational autoencoder (VAE) and Gaussian mixture regression (GMR) models, to generate molecules with the desired target variables of interest for properties and activities (y). A data set of molecules was analyzed, and an encoder was used to transform the simplified molecular input line entry system (SMILES) strings to latent variables (x), while a decoder was used to transform x to SMILES strings. A chemical VAE model was used for analysis and a GMR model (between x and y) was constructed for direct inverse analysis. The target y values were input into the GMR model to directly predict the x values. Following this, the predicted x values were input into the decoder associated with the chemical VAE model and the SMILES string representations (or chemical structures of molecules) were obtained as the output, indicating that the proposed method could be used to selectively obtain the molecules that were characterized by the target y values. We confirmed that the proposed method can be used to generate molecules within the target y ranges even when the conventional chemical VAE model failed to generate the target molecules.

Initial Sample Selection in Bayesian Optimization for Combinatorial Optimization of Chemical Compounds.

An efficient search for optimal solutions in Bayesian optimization (BO) entails providing appropriate initial samples when building a Gaussian process regression model. For general experimental designs without compounds or molecular descriptors in explanatory variable x, selecting initial samples with a larger D-optimality allows little correlation between x in the selected samples, which leads to effective regression model building. However, in the case of experimental designs with compounds, a high correlation always exists between molecular descriptors calculated from chemical structures, and compounds with similar structures form clusters in the chemical space. Therefore, selecting the initial samples uniformly from each cluster is desirable for obtaining initial samples with maximum information on experimental conditions. As D-optimality does not work well with highly correlated molecular descriptors and does not consider information on clusters in sample selection, we propose an initial sample selection method based on clustering and apply it to the optimization of coupling reaction conditions with BO. We confirm that the proposed method reaches the optimal solution with up to 5% fewer experiments than random sampling or sampling based on D-optimality. This study makes a contribution to the initial sample selection method for BO, and we are convinced that the proposed method improves the search performance of BO in various fields of science and technology if initial samples can be determined using cluster information appropriately formed by utilizing domain knowledge.

Defect rate prediction and failure-cause diagnosis in a mass-production process for precision electric components.

Many defects occur during the mass production of precision electrical components. To control and manage them, process variables (PVs), such as the temperature, pressure, flow rate, and liquid level, are measured and time-series data analyzed. However, identification of point of defects is difficult as any operation can cause defects and multiple equipment units are used in parallel for some operations. This study considers the combination of unfavourable conditions between operations to predict the defect rate (DR) of products. A dataset measured in an actual mass-production process for precision electrical components is analysed to predict the DR of the products. Data analysis is performed on a dataset generated from an actual mass-production process for precision electrical components, and machine learning models. are constructed using ensemble learning methods, such as random forests, the gradient boosting decision tree, XGBoost, and LightGBM. Conventional univariate analyses only show a maximum correlation coefficient of 0.17 with a DR and process variables (PVs). In this study, we improved the correlation coefficient to 0.73 using a multivariate analysis, including the data of PVs that are not considered important in the process, and appropriately transformed PVs based on the domain knowledge of the process. Furthermore, PVs that were closely related to the DR could be diagnosed based on the feature importance of the constructed machine-learning models. This study confirms the importance of using domain knowledge to improve the prediction ability of machine learning models and the interpretation of constructed models.

Integration of Materials and Process Informatics: Metal Oxide and Process Design for CO2 Reduction.

In materials informatics, a mathematical model constructed between the synthesis conditions of materials and their properties and activities is used to design synthesis conditions in which the properties and activities have the desired values. In process informatics, a mathematical model constructed between the process conditions for devices and industrial plants and product quality and cost is used to design process conditions that can produce the desired products. In this study, we propose a method to simultaneously design the synthesis conditions of materials and the process conditions of products by integrating materials and process informatics in the reverse water-gas shift chemical looping (RWGS-CL) reaction, which produces CO from CO2 using metal oxides via the RWGS-CL process. Four methods: Gaussian process regression-Bayesian optimization (GPR-BO), Gaussian mixture regression-Bayesian optimization (GMR-BO), GMR-BO-multiple, and GPR-GMR-BO were investigated for the optimization. All four proposed methods outperformed the results of a random search. GPR-BO achieved the highest performance and proposed 27 promising candidates for the synthesis conditions and metal oxides. The selected metals did not include Cu and Ga, which tended to have high predicted CO2 and H2 conversion rates, but Fe and La, which had slightly lower predicted CO2 and H2 conversion rates. These results indicate that a combination of metal oxides with lower predicted CO2 and H2 conversion rates and optimized process conditions was important for the optimization of both materials and processes, which was achieved by integrating materials and process informatics via the proposed method. Thus, we confirmed that it is possible to simultaneously optimize the combination of metals, composition ratios, synthesis conditions of the material or the metal oxide, and the process conditions using experimental datasets, process simulations, and machine learning, such as GPR, GMR, BO, and multiobjective optimization with a genetic algorithm.

Design of Molecules with Low Hole and Electron Reorganization Energy Using DFT Calculations and Bayesian Optimization.

Materials exhibiting higher mobility than conventional organic semiconducting materials, such as fullerenes and fused thiophenes, are in high demand for applications in printed electronics. To discover new molecules that might show improved charge mobility, the adaptive design of experiments (DoE) to design molecules with low reorganization energy was performed by combining density functional theory (DFT) methods and machine learning techniques. DFT-calculated values of 165 molecules were used as an initial training dataset for a Gaussian process regression (GPR) model, and five rounds of molecular designs applying the GPR model and validation via DFT calculations were executed. As a result, new molecules whose reorganization energy is smaller than the lowest value in the initial training dataset were successfully discovered.

Symmetry and spacing controls in periodic covalent functionalization of graphite surfaces templated by self-assembled molecular networks.

We herein present the periodic covalent functionalization of graphite surfaces, creating a range of patterns of different symmetries and pitches at the nanoscale. Self-assembled molecular networks (SAMNs) of rhombic-shaped bis(dehydrobenzo[12]annulene) (bisDBA) derivatives having alkyl chain substituents of different lengths were used as templates for covalent grafting of electrochemically generated aryl radicals. Scanning tunneling microscopy (STM) observations at the 1,2,4-trichlorobenzene/graphite interface revealed that these molecules form a variety of networks that contain pores of different shapes and sizes. The covalently functionalized surfaces show hexagonal, oblique, and quasi-rectangular periodicities. This is attributed to the favorable aryl radical addition at the pore(s). We also confirmed the successful transmission of chirality information from the SAMNs to the alignment of the grafted aryls. In one case, the addition of a guest molecule was used to switch the SAMN symmetry and periodicity, leading to a change in the functionalized surface periodicity from oblique to hexagonal in the presence of the guest molecule. This contribution highlights the potential of SAMNs as templates for the controlled formation of nanopatterned carbon materials.

Design and Analysis of Metal Oxides for CO2 Reduction Using Machine Learning, Transfer Learning, and Bayesian Optimization.

We aim to achieve resource recycling by capturing and using CO2 generated in a chemical production and disposal process. We focused on CO2 conversion to CO by the reverse water gas shift-chemical looping (RWGS-CL) reaction. This reaction proceeds in two steps (H2 + MO x ⇆ H2O + MO x-1; CO2 + MO x-1 ⇆ CO + MO x ) via a metal oxide that acts as an oxygen carrier. High CO2 conversion can be achieved owing to a low H2O concentration in the second step, which causes an unwanted back reaction (H2 + CO2 ⇆ CO + H2O). However, the RWGS-CL process is difficult to control because of repeated thermochemical redox cycling, and the CO2 and H2 conversion extents vary depending on the metal oxide composition and experimental conditions. In this study, we developed metal oxides and simultaneously optimized experimental conditions to satisfy target CO2 and H2 conversion extents by using machine learning and Bayesian optimization. We used transfer learning to improve the prediction accuracy of the mathematical models by incorporating a data set and knowledge of oxygen vacancy formation energy. Furthermore, we analyzed the RWGS-CL reaction based on the prediction accuracy of each variable and the feature importance of the random forest regression model.

Genetic Algorithm-Based Partial Least-Squares with Only the First Component for Model Interpretation.

In the fields of molecular design, material design, process design, and process control, it is important not only to construct models with high predictive ability between explanatory variables X and objective variables y but also to interpret the constructed models to clarify phenomena and elucidate mechanisms in the fields. However, even in linear models, it is dangerous to use regression coefficients as contributions of X to y due to multicollinearity among X. Thus, the focus of this study is the model of partial least-squares with only the first component (PLSFC). It is possible to use regression coefficients as contributions of X to y for the PLSFC model. In addition, selecting the combination of X that can construct a predictive PLSFC model using a genetic algorithm (GA) is proposed, which is called GA-based PLSFC (GA-PLSFC). The constructed model would have both high predictive ability and high interpretability with regression coefficients that can be defined as contributions of X to y. The effectiveness of the proposed PLSFC and GA-PLSFC is verified using numerically simulated data sets and real material data sets. The proposed method was found to be capable of constructing predictive models with high interpretability. The Python codes for GA-PLSFC are available at https://github.com/hkaneko1985/dcekit.