Vascular Patterning as Integrative Readout of Complex Molecular and Physiological Signaling by VESsel GENeration Analysis.

The molecular signaling cascades that regulate angiogenesis and microvascular remodeling are fundamental to normal development, healthy physiology, and pathologies such as inflammation and cancer. Yet quantifying such complex, fractally branching vascular patterns remains difficult. We review application of NASA's globally available, freely downloadable VESsel GENeration (VESGEN) Analysis software to numerous examples of 2D vascular trees, networks, and tree-network composites. Upon input of a binary vascular image, automated output includes informative vascular maps and quantification of parameters such as tortuosity, fractal dimension, vessel diameter, area, length, number, and branch point. Previous research has demonstrated that cytokines and therapeutics such as vascular endothelial growth factor, basic fibroblast growth factor (fibroblast growth factor-2), transforming growth factor-beta-1, and steroid triamcinolone acetonide specify unique "fingerprint" or "biomarker" vascular patterns that integrate dominant signaling with physiological response. In vivo experimental examples described here include vascular response to keratinocyte growth factor, a novel vessel tortuosity factor; angiogenic inhibition in humanized tumor xenografts by the anti-angiogenesis drug leronlimab; intestinal vascular inflammation with probiotic protection by Saccharomyces boulardii, and a workflow programming of vascular architecture for 3D bioprinting of regenerative tissues from 2D images. Microvascular remodeling in the human retina is described for astronaut risks in microgravity, vessel tortuosity in diabetic retinopathy, and venous occlusive disease.

Learning classification models with soft-label information.

OBJECTIVE: Learning of classification models in medicine often relies on data labeled by a human expert. Since labeling of clinical data may be time-consuming, finding ways of alleviating the labeling costs is critical for our ability to automatically learn such models. In this paper we propose a new machine learning approach that is able to learn improved binary classification models more efficiently by refining the binary class information in the training phase with soft labels that reflect how strongly the human expert feels about the original class labels. MATERIALS AND METHODS: Two types of methods that can learn improved binary classification models from soft labels are proposed. The first relies on probabilistic/numeric labels, the other on ordinal categorical labels. We study and demonstrate the benefits of these methods for learning an alerting model for heparin induced thrombocytopenia. The experiments are conducted on the data of 377 patient instances labeled by three different human experts. The methods are compared using the area under the receiver operating characteristic curve (AUC) score. RESULTS: Our AUC results show that the new approach is capable of learning classification models more efficiently compared to traditional learning methods. The improvement in AUC is most remarkable when the number of examples we learn from is small. CONCLUSIONS: A new classification learning framework that lets us learn from auxiliary soft-label information provided by a human expert is a promising new direction for learning classification models from expert labels, reducing the time and cost needed to label data.

Learning classification models from multiple experts.

Building classification models from clinical data using machine learning methods often relies on labeling of patient examples by human experts. Standard machine learning framework assumes the labels are assigned by a homogeneous process. However, in reality the labels may come from multiple experts and it may be difficult to obtain a set of class labels everybody agrees on; it is not uncommon that different experts have different subjective opinions on how a specific patient example should be classified. In this work we propose and study a new multi-expert learning framework that assumes the class labels are provided by multiple experts and that these experts may differ in their class label assessments. The framework explicitly models different sources of disagreements and lets us naturally combine labels from different human experts to obtain: (1) a consensus classification model representing the model the group of experts converge to, as well as, and (2) individual expert models. We test the proposed framework by building a model for the problem of detection of the Heparin Induced Thrombocytopenia (HIT) where examples are labeled by three experts. We show that our framework is superior to multiple baselines (including standard machine learning framework in which expert differences are ignored) and that our framework leads to both improved consensus and individual expert models.

A Temporal Pattern Mining Approach for Classifying Electronic Health Record Data.

We study the problem of learning classification models from complex multivariate temporal data encountered in electronic health record systems. The challenge is to define a good set of features that are able to represent well the temporal aspect of the data. Our method relies on temporal abstractions and temporal pattern mining to extract the classification features. Temporal pattern mining usually returns a large number of temporal patterns, most of which may be irrelevant to the classification task. To address this problem, we present the Minimal Predictive Temporal Patterns framework to generate a small set of predictive and non-spurious patterns. We apply our approach to the real-world clinical task of predicting patients who are at risk of developing heparin induced thrombocytopenia. The results demonstrate the benefit of our approach in efficiently learning accurate classifiers, which is a key step for developing intelligent clinical monitoring systems.

Learning medical diagnosis models from multiple experts.

Building classification models from clinical data often requires labeling examples by human experts. However, it is difficult to obtain a perfect set of labels everyone agrees on because medical data are typically very complicated and it is quite common that different experts have different opinions on the same patient data. A solution that has been recently explored by the research community is learning from multiple experts/annotators. The objective of learning from multiple experts is to model different characteristics of the human experts and combine them to obtain a consensus model. In this work, we study and develop a new probabilistic approach for learning classification models from labels provided by multiple experts. Our method explicitly models and incorporates three characteristics of annotators into the learning process: their specific prediction model, consistency and bias. We show that in addition to building a superior classification model, our method also helps to model behavior of annotators. We applied the proposed method to learn different characteristics of Physicians labeling clinical records for Heparin Induced Thrombocytopenia (HIT) and combine them in order to obtain a final classifier.

Sampling Strategies to Evaluate the Performance of Unknown Predictors.

The focus of this paper is on how to select a small sample of examples for labeling that can help us to evaluate many different classification models unknown at the time of sampling. We are particularly interested in studying the sampling strategies for problems in which the prevalence of the two classes is highly biased toward one of the classes. The evaluation measures of interest we want to estimate as accurately as possible are those obtained from the contingency table. We provide a careful theoretical analysis on sensitivity, specificity, and precision and show how sampling strategies should be adapted to the rate of skewness in data in order to effectively compute the three aforementioned evaluation measures.

Factorized Diffusion Map Approximation.

Diffusion maps are among the most powerful Machine Learning tools to analyze and work with complex high-dimensional datasets. Unfortunately, the estimation of these maps from a finite sample is known to suffer from the curse of dimensionality. Motivated by other machine learning models for which the existence of structure in the underlying distribution of data can reduce the complexity of estimation, we study and show how the factorization of the underlying distribution into independent subspaces can help us to estimate diffusion maps more accurately. Building upon this result, we propose and develop an algorithm that can automatically factorize a high dimensional data space in order to minimize the error of estimation of its diffusion map, even in the case when the underlying distribution is not decomposable. Experiments on both the synthetic and real-world datasets demonstrate improved estimation performance of our method over the standard diffusion-map framework.

Sample-efficient learning with auxiliary class-label information.

Building classification models from clinical data collected for past patients often requires additional example labeling and annotation by a human expert. Since example labeling may require to review a complete electronic health record the process can be very time consuming and costly. To make the process more cost-efficient, the number of examples an expert needs to label should be reduced. We develop and test a new approach for the classification learning in which, in addition to class labels provided by an expert, the learner is provided with auxiliary information that reflects how strong the expert feels about the class label. We show that this information can be extremely useful for practical classification tasks based on human assessment and can lead to improved learning with a smaller number of examples. We develop a new classification approach based on the support vector machines and the learning to rank methodologies capable of utilizing the auxiliary information during the model learning process. We demonstrate the benefit of the approach on the problem of learning an alert model for Heparin Induced Thrombocytopenia (HIT) by showing an improved classification performance of the models that are trained on a smaller number of labeled examples.

Conditional Anomaly Detection with Soft Harmonic Functions.

In this paper, we consider the problem of conditional anomaly detection that aims to identify data instances with an unusual response or a class label. We develop a new non-parametric approach for conditional anomaly detection based on the soft harmonic solution, with which we estimate the confidence of the label to detect anomalous mislabeling. We further regularize the solution to avoid the detection of isolated examples and examples on the boundary of the distribution support. We demonstrate the efficacy of the proposed method on several synthetic and UCI ML datasets in detecting unusual labels when compared to several baseline approaches. We also evaluate the performance of our method on a real-world electronic health record dataset where we seek to identify unusual patient-management decisions.

Learning classification with auxiliary probabilistic information.

Finding ways of incorporating auxiliary information or auxiliary data into the learning process has been the topic of active data mining and machine learning research in recent years. In this work we study and develop a new framework for classification learning problem in which, in addition to class labels, the learner is provided with an auxiliary (probabilistic) information that reflects how strong the expert feels about the class label. This approach can be extremely useful for many practical classification tasks that rely on subjective label assessment and where the cost of acquiring additional auxiliary information is negligible when compared to the cost of the example analysis and labelling. We develop classification algorithms capable of using the auxiliary information to make the learning process more efficient in terms of the sample complexity. We demonstrate the benefit of the approach on a number of synthetic and real world data sets by comparing it to the learning with class labels only.

A Pattern Mining Approach for Classifying Multivariate Temporal Data.

We study the problem of learning classification models from complex multivariate temporal data encountered in electronic health record systems. The challenge is to define a good set of features that are able to represent well the temporal aspect of the data. Our method relies on temporal abstractions and temporal pattern mining to extract the classification features. Temporal pattern mining usually returns a large number of temporal patterns, most of which may be irrelevant to the classification task. To address this problem, we present the minimal predictive temporal patterns framework to generate a small set of predictive and non-spurious patterns. We apply our approach to the real-world clinical task of predicting patients who are at risk of developing heparin induced thrombocytopenia. The results demonstrate the benefit of our approach in learning accurate classifiers, which is a key step for developing intelligent clinical monitoring systems.