Linear optimal transport subspaces for point set classification.

Learning from point sets is an essential component in many computer vision and machine learning applications. Native, unordered, and permutation invariant set structure space is challenging to model, particularly for point set classification under spatial deformations. Here we propose a framework for classifying point sets experiencing certain types of spatial deformations, with a particular emphasis on datasets featuring affine deformations. Our approach employs the Linear Optimal Transport (LOT) transform to obtain a linear embedding of set-structured data. Utilizing the mathematical properties of the LOT transform, we demonstrate its capacity to accommodate variations in point sets by constructing a convex data space, effectively simplifying point set classification problems. Our method, which employs a nearest-subspace algorithm in the LOT space, demonstrates label efficiency, non-iterative behavior, and requires no hyper-parameter tuning. It achieves competitive accuracies compared to state-of-the-art methods across various point set classification tasks. Furthermore, our approach exhibits robustness in out-of-distribution scenarios where training and test distributions vary in terms of deformation magnitudes.

End-to-End Signal Classification in Signed Cumulative Distribution Transform Space.

This paper presents a new end-to-end signal classification method using the signed cumulative distribution transform (SCDT). We adopt a transport generative model to define the classification problem. We then make use of mathematical properties of the SCDT to render the problem easier in transform domain, and solve for the class of an unknown sample using a nearest local subspace (NLS) search algorithm in SCDT domain. Experiments show that the proposed method provides high accuracy classification results while being computationally cheap, data efficient, and robust to out-of-distribution samples with respect to the existing end-to-end classification methods. The implementation of the proposed method in Python language is integrated as a part of the software package PyTransKit.

Data-driven Identification of Parametric Governing Equations of Dynamical Systems Using the Signed Cumulative Distribution Transform.

This paper presents a novel data-driven approach to identify partial differential equation (PDE) parameters of a dynamical system. Specifically, we adopt a mathematical "transport" model for the solution of the dynamical system at specific spatial locations that allows us to accurately estimate the model parameters, including those associated with structural damage. This is accomplished by means of a newly-developed mathematical transform, the signed cumulative distribution transform (SCDT), which is shown to convert the general nonlinear parameter estimation problem into a simple linear regression. This approach has the additional practical advantage of requiring no a priori knowledge of the source of the excitation (or, alternatively, the initial conditions). By using training data, we devise a coarse regression procedure to recover different PDE parameters from the PDE solution measured at a single location. Numerical experiments show that the proposed regression procedure is capable of detecting and estimating PDE parameters with superior accuracy compared to a number of recently developed machine learning methods. Furthermore, a damage identification experiment conducted on a publicly available dataset provides strong evidence of the proposed method's effectiveness in structural health monitoring (SHM) applications. The Python implementation of the proposed system identification technique is integrated as a part of the software package PyTransKit [1].

Approach to Quantify Eye Movements to Augment Stroke Diagnosis With a Non-Calibrated Eye-Tracker.

Automated eye-tracking technology could enhance diagnosis for many neurological diseases, including stroke. Current literature focuses on gaze estimation through a form of calibration. However, patients with neuro-ocular abnormalities may have difficulty completing a calibration procedure due to inattention or other neurological deficits. OBJECTIVE: We investigated 1) the need for calibration to measure eye movement symmetry in healthy controls and 2) the potential of eye movement symmetry to distinguish between healthy controls and patients. METHODS: We analyzed fixations, smooth pursuits, saccades, and conjugacy measured by a Spearman correlation coefficient and utilized a linear mixed-effects model to estimate the effect of calibration. RESULTS: Healthy participants (n = 18) did not differ in correlations between calibrated and non-calibrated conditions for all tests. The calibration condition did not improve the linear mixed effects model (log-likelihood ratio test p = 0.426) in predicting correlation coefficients. Interestingly, the patient group (n = 17) differed in correlations for the DOT (0.844 [95% CI 0.602, 0.920] vs. 0.98 [95% CI 0.976, 0.985]), H (0.903 [95% CI 0.746, 0.958] vs. 0.979 [95% CI 0.971, 0.986]), and OKN (0.898 [95% CI 0.785, 0.958] vs. 0.993 [95% CI 0.987, 0.996]) tests compared to healthy controls along the x-axis. These differences were not observed along the y-axis. SIGNIFICANCE: This study suggests that automated eye tracking can be deployed without calibration to measure eye movement symmetry. It may be a good discriminator between normal and abnormal eye movement symmetry. Validation of these findings in larger populations is required.

Predicting Malignancy of Breast Imaging Findings Using Quantitative Analysis of Contrast-Enhanced Mammography (CEM).

We sought to develop new quantitative approaches to characterize the spatial distribution of mammographic density and contrast enhancement of suspicious contrast-enhanced mammography (CEM) findings to improve malignant vs. benign classifications of breast lesions. We retrospectively analyzed all breast lesions that underwent CEM imaging and tissue sampling at our institution from 2014-2020 in this IRB-approved study. A penalized linear discriminant analysis was used to classify lesions based on the averaged histograms of radial distributions of mammographic density and contrast enhancement. T-tests were used to compare the classification accuracies of density, contrast, and concatenated density and contrast histograms. Logistic regression and AUC-ROC analyses were used to assess if adding demographic and clinical data improved the model accuracy. A total of 159 suspicious findings were evaluated. Density histograms were more accurate in classifying lesions as malignant or benign than a random classifier (62.37% vs. 48%; p < 0.001), but the concatenated density and contrast histograms demonstrated a higher accuracy (71.25%; p < 0.001) than the density histograms alone. Including the demographic and clinical data in our models led to a higher AUC-ROC than concatenated density and contrast images (0.81 vs. 0.70; p < 0.001). In the classification of invasive vs. non-invasive malignancy, the concatenated density and contrast histograms demonstrated no significant improvement in accuracy over the density histograms alone (77.63% vs. 78.59%; p = 0.504). Our findings suggest that quantitative differences in the radial distribution of mammographic density could be used to discriminate malignant from benign breast findings; however, classification accuracy was significantly improved with the addition of contrast-enhanced imaging data from CEM. Adding patient demographic and clinical information further improved the classification accuracy.

Transport-based morphometry of nuclear structures of digital pathology images in cancers.

Alterations in nuclear morphology are useful adjuncts and even diagnostic tools used by pathologists in the diagnosis and grading of many tumors, particularly malignant tumors. Large datasets such as TCGA and the Human Protein Atlas, in combination with emerging machine learning and statistical modeling methods, such as feature extraction and deep learning techniques, can be used to extract meaningful knowledge from images of nuclei, particularly from cancerous tumors. Here we describe a new technique based on the mathematics of optimal transport for modeling the information content related to nuclear chromatin structure directly from imaging data. In contrast to other techniques, our method represents the entire information content of each nucleus relative to a template nucleus using a transport-based morphometry (TBM) framework. We demonstrate the model is robust to different staining patterns and imaging protocols, and can be used to discover meaningful and interpretable information within and across datasets and cancer types. In particular, we demonstrate morphological differences capable of distinguishing nuclear features along the spectrum from benign to malignant categories of tumors across different cancer tissue types, including tumors derived from liver parenchyma, thyroid gland, lung mesothelium, and skin epithelium. We believe these proof of concept calculations demonstrate that the TBM framework can provide the quantitative measurements necessary for performing meaningful comparisons across a wide range of datasets and cancer types that can potentially enable numerous cancer studies, technologies, and clinical applications and help elevate the role of nuclear morphometry into a more quantitative science. The source codes implementing our method is available at https://github.com/rohdelab/nuclear_morphometry.

Real-time intelligent classification of COVID-19 and thrombosis via massive image-based analysis of platelet aggregates.

Microvascular thrombosis is a typical symptom of COVID-19 and shows similarities to thrombosis. Using a microfluidic imaging flow cytometer, we measured the blood of 181 COVID-19 samples and 101 non-COVID-19 thrombosis samples, resulting in a total of 6.3 million bright-field images. We trained a convolutional neural network to distinguish single platelets, platelet aggregates, and white blood cells and performed classical image analysis for each subpopulation individually. Based on derived single-cell features for each population, we trained machine learning models for classification between COVID-19 and non-COVID-19 thrombosis, resulting in a patient testing accuracy of 75%. This result indicates that platelet formation differs between COVID-19 and non-COVID-19 thrombosis. All analysis steps were optimized for efficiency and implemented in an easy-to-use plugin for the image viewer napari, allowing the entire analysis to be performed within seconds on mid-range computers, which could be used for real-time diagnosis.

Invariance encoding in sliced-Wasserstein space for image classification with limited training data.

Deep convolutional neural networks (CNNs) are broadly considered to be state-of-the-art generic end-to-end image classification systems. However, they are known to underperform when training data are limited and thus require data augmentation strategies that render the method computationally expensive and not always effective. Rather than using a data augmentation strategy to encode invariances as typically done in machine learning, here we propose to mathematically augment a nearest subspace classification model in sliced-Wasserstein space by exploiting certain mathematical properties of the Radon Cumulative Distribution Transform (R-CDT), a recently introduced image transform. We demonstrate that for a particular type of learning problem, our mathematical solution has advantages over data augmentation with deep CNNs in terms of classification accuracy and computational complexity, and is particularly effective under a limited training data setting. The method is simple, effective, computationally efficient, non-iterative, and requires no parameters to be tuned. Python code implementing our method is available at https://github.com/rohdelab/mathematical augmentation. Our method is integrated as a part of the software package PyTransKit, which is available at https://github.com/rohdelab/PyTransKit.

Long-term effects of Pfizer-BioNTech COVID-19 vaccinations on platelets.

There is a global concern about the safety of COVID-19 vaccines associated with platelet function. However, their long-term effects on overall platelet activity remain poorly understood. Here we address this problem by image-based single-cell profiling and temporal monitoring of circulating platelet aggregates in the blood of healthy human subjects, before and after they received multiple Pfizer-BioNTech (BNT162b2) vaccine doses over a time span of nearly 1 year. Results show no significant or persisting platelet aggregation trends following the vaccine doses, indicating that any effects of vaccinations on platelet turnover, platelet activation, platelet aggregation, and platelet-leukocyte interaction was insignificant.

Human vs. Machine Learning Based Detection of Facial Weakness Using Video Analysis.

Background: Current EMS stroke screening tools facilitate early detection and triage, but the tools' accuracy and reliability are limited and highly variable. An automated stroke screening tool could improve stroke outcomes by facilitating more accurate prehospital diagnosis and delivery. We hypothesize that a machine learning algorithm using video analysis can detect common signs of stroke. As a proof-of-concept study, we trained a computer algorithm to detect presence and laterality of facial weakness in publically available videos with comparable accuracy, sensitivity, and specificity to paramedics. Methods and Results: We curated videos of people with unilateral facial weakness (n = 93) and with a normal smile (n = 96) from publicly available web-based sources. Three board certified vascular neurologists categorized the videos according to the presence or absence of weakness and laterality. Three paramedics independently analyzed each video with a mean accuracy, sensitivity and specificity of 92.6% [95% CI 90.1-94.7%], 87.8% [95% CI 83.9-91.7%] and 99.3% [95% CI 98.2-100%]. Using a 5-fold cross validation scheme, we trained a computer vision algorithm to analyze the same videos producing an accuracy, sensitivity and specificity of 88.9% [95% CI 83.5-93%], 90.3% [95% CI 82.4-95.5%] and 87.5 [95% CI 79.2-93.4%]. Conclusions: These preliminary results suggest that a machine learning algorithm using computer vision analysis can detect unilateral facial weakness in pre-recorded videos with an accuracy and sensitivity comparable to trained paramedics. Further research is warranted to pursue the concept of augmented facial weakness detection and external validation of this algorithm in independent data sets and prospective patient encounters.

Investigating impact of cardiorespiratory fitness in reducing brain tissue loss caused by ageing.

Mitigating the loss of brain tissue due to age is a major problem for an ageing population. Improving cardiorespiratory fitness has been suggested as a possible strategy, but the influenceon brain morphology has not been fully characterized. To investigate the dependent shifts in brain tissue distribution as a function of cardiorespiratory fitness, we used a 3D transport-based morphometry approach. In this study of 172 inactive older adults aged 58-81 (66.5 ± 5.7) years, cardiorespiratory fitness was determined by V O 2 peak (ml/kg/min) during graded exercise and brain morphology was assessed through structural magnetic resonance imaging. After correcting for covariates including age (in the fitness model), gender and level of education, we compared dependent tissue shifts with age to those due to V O 2   peak . We found a significant association between cardiorespiratory fitness and brain tissue distribution (white matter, r = 0.30, P = 0.003; grey matter, r = 0.40, P < 0.001) facilitated by direct visualization of the brain tissue shifts due to cardiorespiratory fitness through inverse transformation-a key capability of 3D transport-based morphometry. A strong statistical correlation was found between brain tissue changes related to ageing and those associated with lower cardiorespiratory fitness (white matter, r = 0.62, P < 0.001; grey matter, r = 0.74, P < 0.001). In both cases, frontotemporal regions shifted the most while basal ganglia shifted the least. Our results highlight the importance of cardiorespiratory fitness in maintaining brain health later in life. Furthermore, this work demonstrates 3D transport-based morphometry as a novel neuroinformatic technology that may aid assessment of therapeutic approaches for brain ageing and neurodegenerative diseases.

Massive image-based single-cell profiling reveals high levels of circulating platelet aggregates in patients with COVID-19.

A characteristic clinical feature of COVID-19 is the frequent incidence of microvascular thrombosis. In fact, COVID-19 autopsy reports have shown widespread thrombotic microangiopathy characterized by extensive diffuse microthrombi within peripheral capillaries and arterioles in lungs, hearts, and other organs, resulting in multiorgan failure. However, the underlying process of COVID-19-associated microvascular thrombosis remains elusive due to the lack of tools to statistically examine platelet aggregation (i.e., the initiation of microthrombus formation) in detail. Here we report the landscape of circulating platelet aggregates in COVID-19 obtained by massive single-cell image-based profiling and temporal monitoring of the blood of COVID-19 patients (n = 110). Surprisingly, our analysis of the big image data shows the anomalous presence of excessive platelet aggregates in nearly 90% of all COVID-19 patients. Furthermore, results indicate strong links between the concentration of platelet aggregates and the severity, mortality, respiratory condition, and vascular endothelial dysfunction level of COVID-19 patients.

Transport-based pattern recognition versus deep neural networks in underwater OAM communications.

Comparisons between machine learning and optimal transport-based approaches in classifying images are made in underwater orbital angular momentum (OAM) communications. A model is derived that justifies optimal transport for use in attenuated water environments. OAM pattern demultiplexing is performed using optimal transport and deep neural networks and compared to each other. Additionally, some of the complications introduced by signal attenuation are highlighted. The Radon cumulative distribution transform (R-CDT) is applied to OAM patterns to transform them to a linear subspace. The original OAM images and the R-CDT transformed patterns are used in several classification algorithms, and results are compared. The selected classification algorithms are the nearest subspace algorithm, a shallow convolutional neural network (CNN), and a deep neural network. It is shown that the R-CDT transformed images are more accurate than the original OAM images in pattern classification. Also, the nearest subspace algorithm performs better than the selected CNNs in OAM pattern classification in underwater environments.

Video-Based Facial Weakness Analysis.

OBJECTIVE: Facial weakness is a common sign of neurological diseases such as Bell's palsy and stroke. However, recognizing facial weakness still remains as a challenge, because it requires experience and neurological training. METHODS: We propose a framework for facial weakness detection, which models the temporal dynamics of both shape and appearance-based features of each target frame through a bi-directional long short-term memory network (Bi-LSTM). The system is evaluated on a "in-the-wild"video dataset that is verified by three board-certified neurologists. In addition, three emergency medical services (EMS) personnel and three upper level residents rated the dataset. We compare the evaluation of the proposed algorithm with other comparison methods as well as the human raters. RESULTS: Experimental evaluation demonstrates that: (1) the proposed algorithm achieves the accuracy, sensitivity, and specificity of 94.3%, 91.4%, and 95.7%, which outperforms other comparison methods and achieves the equal performance to paramedics; (2) the framework can provide visualizable and interpretable results that increases model transparency and interpretability; (3) a prototype is implemented as a proof-of-concept showcase to show the feasibility of an inexpensive solution for facial weakness detection. CONCLUSION: The experiment results suggest that the proposed framework can identify facial weakness effectively. SIGNIFICANCE: We provide a proof-of-concept study, showing that such technology could be used by non-neurologists to more readily identify facial weakness in the field, leading to increasing coverage and earlier treatment.

PARTITIONING SIGNAL CLASSES USING TRANSPORT TRANSFORMS FOR DATA ANALYSIS AND MACHINE LEARNING.

A relatively new set of transport-based transforms (CDT, R-CDT, LOT) have shown their strength and great potential in various image and data processing tasks such as parametric signal estimation, classification, cancer detection among many others. It is hence worthwhile to elucidate some of the mathematical properties that explain the successes of these transforms when they are used as tools in data analysis, signal processing or data classification. In particular, we give conditions under which classes of signals that are created by algebraic generative models are transformed into convex sets by the transport transforms. Such convexification of the classes simplify the classification and other data analysis and processing problems when viewed in the transform domain. More specifically, we study the extent and limitation of the convexification ability of these transforms under an algebraic generative modeling framework. We hope that this paper will serve as an introduction to these transforms and will encourage mathematicians and other researchers to further explore the theoretical underpinnings and algorithmic tools that will help understand the successes of these transforms and lay the groundwork for further successful applications.

Radon Cumulative Distribution Transform Subspace Modeling for Image Classification.

We present a new supervised image classification method applicable to a broad class of image deformation models. The method makes use of the previously described Radon Cumulative Distribution Transform (R-CDT) for image data, whose mathematical properties are exploited to express the image data in a form that is more suitable for machine learning. While certain operations such as translation, scaling, and higher-order transformations are challenging to model in native image space, we show the R-CDT can capture some of these variations and thus render the associated image classification problems easier to solve. The method - utilizing a nearest-subspace algorithm in the R-CDT space - is simple to implement, non-iterative, has no hyper-parameters to tune, is computationally efficient, label efficient, and provides competitive accuracies to state-of-the-art neural networks for many types of classification problems. In addition to the test accuracy performances, we show improvements (with respect to neural network-based methods) in terms of computational efficiency (it can be implemented without the use of GPUs), number of training samples needed for training, as well as out-of-distribution generalization. The Python code for reproducing our results is available at [1].

Enabling early detection of osteoarthritis from presymptomatic cartilage texture maps via transport-based learning.

Many diseases have no visual cues in the early stages, eluding image-based detection. Today, osteoarthritis (OA) is detected after bone damage has occurred, at an irreversible stage of the disease. Currently no reliable method exists for OA detection at a reversible stage. We present an approach that enables sensitive OA detection in presymptomatic individuals. Our approach combines optimal mass transport theory with statistical pattern recognition. Eighty-six healthy individuals were selected from the Osteoarthritis Initiative, with no symptoms or visual signs of disease on imaging. On 3-y follow-up, a subset of these individuals had progressed to symptomatic OA. We trained a classifier to differentiate progressors and nonprogressors on baseline cartilage texture maps, which achieved a robust test accuracy of 78% in detecting future symptomatic OA progression 3 y prior to symptoms. This work demonstrates that OA detection may be possible at a potentially reversible stage. A key contribution of our work is direct visualization of the cartilage phenotype defining predictive ability as our technique is generative. We observe early biochemical patterns of fissuring in cartilage that define future onset of OA. In the future, coupling presymptomatic OA detection with emergent clinical therapies could modify the outcome of a disease that costs the United States healthcare system $16.5 billion annually. Furthermore, our technique is broadly applicable to earlier image-based detection of many diseases currently diagnosed at advanced stages today.

Parametric Signal Estimation Using the Cumulative Distribution Transform.

We present a new method for estimating signal model parameters using the Cumulative Distribution Transform (CDT). Our approach minimizes the Wasserstein distance between measured and model signals. We derive some useful properties of the CDT and show that the resulting estimation problem, while nonlinear in the original signal domain, becomes a linear least squares problem in the transform domain. Furthermore, we discuss the properties of the estimator in the presence of noise and present a novel approach for mitigating the impact of the noise on the estimates. The proposed estimation approach is evaluated by applying it to a source localization problem and comparing its performance against traditional approaches.