CAFA-evaluator: a Python tool for benchmarking ontological classification methods.

We present CAFA-evaluator, a powerful Python program designed to evaluate the performance of prediction methods on targets with hierarchical concept dependencies. It generalizes multi-label evaluation to modern ontologies where the prediction targets are drawn from a directed acyclic graph and achieves high efficiency by leveraging matrix computation and topological sorting. The program requirements include a small number of standard Python libraries, making CAFA-evaluator easy to maintain. The code replicates the Critical Assessment of protein Function Annotation (CAFA) benchmarking, which evaluates predictions of the consistent subgraphs in Gene Ontology. Owing to its reliability and accuracy, the organizers have selected CAFA-evaluator as the official CAFA evaluation software. Availability and implementation: https://pypi.org/project/cafaeval.

Learning from the unknown: exploring the range of bacterial functionality.

Determining the repertoire of a microbe's molecular functions is a central question in microbial biology. Modern techniques achieve this goal by comparing microbial genetic material against reference databases of functionally annotated genes/proteins or known taxonomic markers such as 16S rRNA. Here, we describe a novel approach to exploring bacterial functional repertoires without reference databases. Our Fusion scheme establishes functional relationships between bacteria and assigns organisms to Fusion-taxa that differ from otherwise defined taxonomic clades. Three key findings of our work stand out. First, bacterial functional comparisons outperform marker genes in assigning taxonomic clades. Fusion profiles are also better for this task than other functional annotation schemes. Second, Fusion-taxa are robust to addition of novel organisms and are, arguably, able to capture the environment-driven bacterial diversity. Finally, our alignment-free nucleic acid-based Siamese Neural Network model, created using Fusion functions, enables finding shared functionality of very distant, possibly structurally different, microbial homologs. Our work can thus help annotate functional repertoires of bacterial organisms and further guide our understanding of microbial communities.

An Approach to Identifying and Quantifying Bias in Biomedical Data.

Data biases are a known impediment to the development of trustworthy machine learning models and their application to many biomedical problems. When biased data is suspected, the assumption that the labeled data is representative of the population must be relaxed and methods that exploit a typically representative unlabeled data must be developed. To mitigate the adverse effects of unrepresentative data, we consider a binary semi-supervised setting and focus on identifying whether the labeled data is biased and to what extent. We assume that the class-conditional distributions were generated by a family of component distributions represented at different proportions in labeled and unlabeled data. We also assume that the training data can be transformed to and subsequently modeled by a nested mixture of multivariate Gaussian distributions. We then develop a multi-sample expectation-maximization algorithm that learns all individual and shared parameters of the model from the combined data. Using these parameters, we develop a statistical test for the presence of the general form of bias in labeled data and estimate the level of this bias by computing the distance between corresponding class-conditional distributions in labeled and unlabeled data. We first study the new methods on synthetic data to understand their behavior and then apply them to real-world biomedical data to provide evidence that the bias estimation procedure is both possible and effective.

Incorporation of CT-based measurements of trunk anatomy into subject-specific musculoskeletal models of the spine influences vertebral loading predictions.

We created subject-specific musculoskeletal models of the thoracolumbar spine by incorporating spine curvature and muscle morphology measurements from computed tomography (CT) scans to determine the degree to which vertebral compressive and shear loading estimates are sensitive to variations in trunk anatomy. We measured spine curvature and trunk muscle morphology using spine CT scans of 125 men, and then created four different thoracolumbar spine models for each person: (i) height and weight adjusted (Ht/Wt models); (ii) height, weight, and spine curvature adjusted (+C models); (iii) height, weight, and muscle morphology adjusted (+M models); and (iv) height, weight, spine curvature, and muscle morphology adjusted (+CM models). We determined vertebral compressive and shear loading at three regions of the spine (T8, T12, and L3) for four different activities. Vertebral compressive loads predicted by the subject-specific CT-based musculoskeletal models were between 54% lower to 45% higher from those estimated using musculoskeletal models adjusted only for subject height and weight. The impact of subject-specific information on vertebral loading estimates varied with the activity and spinal region. Vertebral loading estimates were more sensitive to incorporation of subject-specific spinal curvature than subject-specific muscle morphology. Our results indicate that individual variations in spine curvature and trunk muscle morphology can have a major impact on estimated vertebral compressive and shear loads, and thus should be accounted for when estimating subject-specific vertebral loading. © 2017 Orthopaedic Research Society. Published by Wiley Periodicals, Inc. J Orthop Res 35:2164-2173, 2017.