Framework for Testing Robustness of Machine Learning-Based Classifiers.

There has been a rapid increase in the number of artificial intelligence (AI)/machine learning (ML)-based biomarker diagnostic classifiers in recent years. However, relatively little work has focused on assessing the robustness of these biomarkers, i.e., investigating the uncertainty of the AI/ML models that these biomarkers are based upon. This paper addresses this issue by proposing a framework to evaluate the already-developed classifiers with regard to their robustness by focusing on the variability of the classifiers' performance and changes in the classifiers' parameter values using factor analysis and Monte Carlo simulations. Specifically, this work evaluates (1) the importance of a classifier's input features and (2) the variability of a classifier's output and model parameter values in response to data perturbations. Additionally, it was found that one can estimate a priori how much replacement noise a classifier can tolerate while still meeting accuracy goals. To illustrate the evaluation framework, six different AI/ML-based biomarkers are developed using commonly used techniques (linear discriminant analysis, support vector machines, random forest, partial-least squares discriminant analysis, logistic regression, and multilayer perceptron) for a metabolomics dataset involving 24 measured metabolites taken from 159 study participants. The framework was able to correctly predict which of the classifiers should be less robust than others without recomputing the classifiers itself, and this prediction was then validated in a detailed analysis.

Altered metabolism of mothers of young children with Autism Spectrum Disorder: a case control study.

BACKGROUND: Previous research studies have demonstrated abnormalities in the metabolism of mothers of young children with autism. METHODS: Metabolic analysis was performed on blood samples from 30 mothers of young children with Autism Spectrum Disorder (ASD-M) and from 29 mothers of young typically-developing children (TD-M). Targeted metabolic analysis focusing on the folate one-carbon metabolism (FOCM) and the transsulfuration pathway (TS) as well as broad metabolic analysis were performed. Statistical analysis of the data involved both univariate and multivariate statistical methods. RESULTS: Univariate analysis revealed significant differences in 5 metabolites from the folate one-carbon metabolism and the transsulfuration pathway and differences in an additional 48 metabolites identified by broad metabolic analysis, including lower levels of many carnitine-conjugated molecules. Multivariate analysis with leave-one-out cross-validation allowed classification of samples as belonging to one of the two groups of mothers with 93% sensitivity and 97% specificity with five metabolites. Furthermore, each of these five metabolites correlated with 8-15 other metabolites indicating that there are five clusters of correlated metabolites. In fact, all but 5 of the 50 metabolites with the highest area under the receiver operating characteristic curve were associated with the five identified groups. Many of the abnormalities appear linked to low levels of folate, vitamin B12, and carnitine-conjugated molecules. CONCLUSIONS: Mothers of children with ASD have many significantly different metabolite levels compared to mothers of typically developing children at 2-5 years after birth.