Scalable Optimal Transport Methods in Machine Learning: A Contemporary Survey.

Optimal Transport (OT) is a mathematical framework that first emerged in the eighteenth century and has led to a plethora of methods for answering many theoretical and applied questions. The last decade has been a witness to the remarkable contributions of this classical optimization problem to machine learning. This paper is about where and how optimal transport is used in machine learning with a focus on the question of scalable optimal transport. We provide a comprehensive survey of optimal transport while ensuring an accessible presentation as permitted by the nature of the topic and the context. First, we explain the optimal transport background and introduce different flavors (i.e. mathematical formulations), properties, and notable applications. We then address the fundamental question of how to scale optimal transport to cope with the current demands of big and high dimensional data. We conduct a systematic analysis of the methods used in the literature for scaling OT and present the findings in a unified taxonomy. We conclude with presenting some open challenges and discussing potential future research directions. A live repository of related OT research papers is maintained in https://github.com/abdelwahed/OT_for_big_data.git.

HairNet2: deep learning to quantify cotton leaf hairiness, a complex genetic and environmental trait.

BACKGROUND: Cotton accounts for 80% of the global natural fibre production. Its leaf hairiness affects insect resistance, fibre yield, and economic value. However, this phenotype is still qualitatively assessed by visually attributing a Genotype Hairiness Score (GHS) to a leaf/plant, or by using the HairNet deep-learning model which also outputs a GHS. Here, we introduce HairNet2, a quantitative deep-learning model which detects leaf hairs (trichomes) from images and outputs a segmentation mask and a Leaf Trichome Score (LTS). RESULTS: Trichomes of 1250 images were annotated (AnnCoT) and a combination of six Feature Extractor modules and five Segmentation modules were tested alongside a range of loss functions and data augmentation techniques. HairNet2 was further validated on the dataset used to build HairNet (CotLeaf-1), a similar dataset collected in two subsequent seasons (CotLeaf-2), and a dataset collected on two genetically diverse populations (CotLeaf-X). The main findings of this study are that (1) leaf number, environment and image position did not significantly affect results, (2) although GHS and LTS mostly correlated for individual GHS classes, results at the genotype level revealed a strong LTS heterogeneity within a given GHS class, (3) LTS correlated strongly with expert scoring of individual images. CONCLUSIONS: HairNet2 is the first quantitative and scalable deep-learning model able to measure leaf hairiness. Results obtained with HairNet2 concur with the qualitative values used by breeders at both extremes of the scale (GHS 1-2, and 5-5+), but interestingly suggest a reordering of genotypes with intermediate values (GHS 3-4+). Finely ranking mild phenotypes is a difficult task for humans. In addition to providing assistance with this task, HairNet2 opens the door to selecting plants with specific leaf hairiness characteristics which may be associated with other beneficial traits to deliver better varieties.

Deep learning approaches for seizure video analysis: A review.

Seizure events can manifest as transient disruptions in the control of movements which may be organized in distinct behavioral sequences, accompanied or not by other observable features such as altered facial expressions. The analysis of these clinical signs, referred to as semiology, is subject to observer variations when specialists evaluate video-recorded events in the clinical setting. To enhance the accuracy and consistency of evaluations, computer-aided video analysis of seizures has emerged as a natural avenue. In the field of medical applications, deep learning and computer vision approaches have driven substantial advancements. Historically, these approaches have been used for disease detection, classification, and prediction using diagnostic data; however, there has been limited exploration of their application in evaluating video-based motion detection in the clinical epileptology setting. While vision-based technologies do not aim to replace clinical expertise, they can significantly contribute to medical decision-making and patient care by providing quantitative evidence and decision support. Behavior monitoring tools offer several advantages such as providing objective information, detecting challenging-to-observe events, reducing documentation efforts, and extending assessment capabilities to areas with limited expertise. The main applications of these could be (1) improved seizure detection methods; (2) refined semiology analysis for predicting seizure type and cerebral localization. In this paper, we detail the foundation technologies used in vision-based systems in the analysis of seizure videos, highlighting their success in semiology detection and analysis, focusing on work published in the last 7 years. We systematically present these methods and indicate how the adoption of deep learning for the analysis of video recordings of seizures could be approached. Additionally, we illustrate how existing technologies can be interconnected through an integrated system for video-based semiology analysis. Each module can be customized and improved by adapting more accurate and robust deep learning approaches as these evolve. Finally, we discuss challenges and research directions for future studies.

Curved Geometric Networks for Visual Anomaly Recognition.

Learning a latent embedding to understand the underlying nature of data distribution is often formulated in Euclidean spaces with zero curvature. However, the success of the geometry constraints, posed in the embedding space, indicates that curved spaces might encode more structural information, leading to better discriminative power and hence richer representations. In this work, we investigate the benefits of the curved space for analyzing anomalous, open-set, or out-of-distribution (OOD) objects in data. This is achieved by considering embeddings via three geometry constraints, namely, spherical geometry (with positive curvature), hyperbolic geometry (with negative curvature), or mixed geometry (with both positive and negative curvatures). Three geometric constraints can be chosen interchangeably in a unified design, given the task at hand. Tailored for the embeddings in the curved space, we also formulate functions to compute the anomaly score. Two types of geometric modules (i.e., geometric-in-one (GiO) and geometric-in-two (GiT) models) are proposed to plug in the original Euclidean classifier, and anomaly scores are computed from the curved embeddings. We evaluate the resulting designs under a diverse set of visual recognition scenarios, including image detection (multiclass OOD detection and one-class anomaly detection) and segmentation (multiclass anomaly segmentation and one-class anomaly segmentation). The empirical results show the effectiveness of our proposal through consistent improvement over various scenarios. The code is made available at https://github.com/JHome1/GiO-GiT.

Monitoring of Pigmented Skin Lesions Using 3D Whole Body Imaging.

BACKGROUND AND OBJECTIVES: Advanced artificial intelligence and machine learning have great potential to redefine how skin lesions are detected, mapped, tracked and documented. Here, we propose a 3D whole-body imaging system known as 3DSkin-mapper to enable automated detection, evaluation and mapping of skin lesions. METHODS: A modular camera rig arranged in a cylindrical configuration was designed to automatically capture images of the entire skin surface of a subject synchronously from multiple angles. Based on the images, we developed algorithms for 3D model reconstruction, data processing and skin lesion detection and tracking based on deep convolutional neural networks. We also introduced a customised, user-friendly, and adaptable interface that enables individuals to interactively visualise, manipulate, and annotate the images. The interface includes built-in features such as mapping 2D skin lesions onto the corresponding 3D model. RESULTS: The proposed system is developed for skin lesion screening, the focus of this paper is to introduce the system instead of clinical study. Using synthetic and real images we demonstrate the effectiveness of the proposed system by providing multiple views of a target skin lesion, enabling further 3D geometry analysis and longitudinal tracking. Skin lesions are identified as outliers which deserve more attention from a skin cancer physician. Our detector leverages expert annotated labels to learn representations of skin lesions, while capturing the effects of anatomical variability. It takes only a few seconds to capture the entire skin surface, and about half an hour to process and analyse the images. CONCLUSIONS: Our experiments show that the proposed system allows fast and easy whole body 3D imaging. It can be used by dermatological clinics to conduct skin screening, detect and track skin lesions over time, identify suspicious lesions, and document pigmented lesions. The system can potentially save clinicians time and effort significantly. The 3D imaging and analysis has the potential to change the paradigm of whole body photography with many applications in skin diseases, including inflammatory and pigmentary disorders. With reduced time requirements for recording and documenting high-quality skin information, doctors could spend more time providing better-quality treatment based on more detailed and accurate information.

HairNet: a deep learning model to score leaf hairiness, a key phenotype for cotton fibre yield, value and insect resistance.

BACKGROUND: Leaf hairiness (pubescence) is an important plant phenotype which regulates leaf transpiration, affects sunlight penetration, and provides increased resistance or susceptibility against certain insects. Cotton accounts for 80% of global natural fibre production, and in this crop leaf hairiness also affects fibre yield and value. Currently, this key phenotype is measured visually which is slow, laborious and operator-biased. Here, we propose a simple, high-throughput and low-cost imaging method combined with a deep-learning model, HairNet, to classify leaf images with great accuracy. RESULTS: A dataset of [Formula: see text] 13,600 leaf images from 27 genotypes of Cotton was generated. Images were collected from leaves at two different positions in the canopy (leaf 3 & leaf 4), from genotypes grown in two consecutive years and in two growth environments (glasshouse & field). This dataset was used to build a 4-part deep learning model called HairNet. On the whole dataset, HairNet achieved accuracies of 89% per image and 95% per leaf. The impact of leaf selection, year and environment on HairNet accuracy was then investigated using subsets of the whole dataset. It was found that as long as examples of the year and environment tested were present in the training population, HairNet achieved very high accuracy per image (86-96%) and per leaf (90-99%). Leaf selection had no effect on HairNet accuracy, making it a robust model. CONCLUSIONS: HairNet classifies images of cotton leaves according to their hairiness with very high accuracy. The simple imaging methodology presented in this study and the high accuracy on a single image per leaf achieved by HairNet demonstrates that it is implementable at scale. We propose that HairNet replaces the current visual scoring of this trait. The HairNet code and dataset can be used as a baseline to measure this trait in other species or to score other microscopic but important phenotypes.

Attention in Attention Networks for Person Retrieval.

This paper generalizes the Attention in Attention (AiA) mechanism, in P. Fang et al., 2019 by employing explicit mapping in reproducing kernel Hilbert spaces to generate attention values of the input feature map. The AiA mechanism models the capacity of building inter-dependencies among the local and global features by the interaction of inner and outer attention modules. Besides a vanilla AiA module, termed linear attention with AiA, two non-linear counterparts, namely, second-order polynomial attention and Gaussian attention, are also proposed to utilize the non-linear properties of the input features explicitly, via the second-order polynomial kernel and Gaussian kernel approximation. The deep convolutional neural network, equipped with the proposed AiA blocks, is referred to as Attention in Attention Network (AiA-Net). The AiA-Net learns to extract a discriminative pedestrian representation, which combines complementary person appearance and corresponding part features. Extensive ablation studies verify the effectiveness of the AiA mechanism and the use of non-linear features hidden in the feature map for attention design. Furthermore, our approach outperforms current state-of-the-art by a considerable margin across a number of benchmarks. In addition, state-of-the-art performance is also achieved in the video person retrieval task with the assistance of the proposed AiA blocks.

A survey on graph-based deep learning for computational histopathology.

With the remarkable success of representation learning for prediction problems, we have witnessed a rapid expansion of the use of machine learning and deep learning for the analysis of digital pathology and biopsy image patches. However, learning over patch-wise features using convolutional neural networks limits the ability of the model to capture global contextual information and comprehensively model tissue composition. The phenotypical and topological distribution of constituent histological entities play a critical role in tissue diagnosis. As such, graph data representations and deep learning have attracted significant attention for encoding tissue representations, and capturing intra- and inter- entity level interactions. In this review, we provide a conceptual grounding for graph analytics in digital pathology, including entity-graph construction and graph architectures, and present their current success for tumor localization and classification, tumor invasion and staging, image retrieval, and survival prediction. We provide an overview of these methods in a systematic manner organized by the graph representation of the input image, scale, and organ on which they operate. We also outline the limitations of existing techniques, and suggest potential future research directions in this domain.

Video-Based Inpatient Fall Risk Assessment: A Case Study.

Inpatient falls are a serious safety issue in hospitals and healthcare facilities. Recent advances in video analytics for patient monitoring provide a non-intrusive avenue to reduce this risk through continuous activity monitoring. However, in- bed fall risk assessment systems have received less attention in the literature. The majority of prior studies have focused on fall event detection, and do not consider the circumstances that may indicate an imminent inpatient fall. Here, we propose a video-based system that can monitor the risk of a patient falling, and alert staff of unsafe behaviour to help prevent falls before they occur. We propose an approach that leverages recent advances in human localisation and skeleton pose estimation to extract spatial features from video frames recorded in a simulated environment. We demonstrate that body positions can be effectively recognised and provide useful evidence for fall risk assessment. This work highlights the benefits of video-based models for analysing behaviours of interest, and demonstrates how such a system could enable sufficient lead time for healthcare professionals to respond and address patient needs, which is necessary for the development of fall intervention programs.

Attention-Based Real Image Restoration.

Deep convolutional neural networks perform better on images containing spatially invariant degradations, also known as synthetic degradations; however, their performance is limited on real-degraded photographs and requires multiple-stage network modeling. To advance the practicability of restoration algorithms, this article proposes a novel single-stage blind real image restoration network (R²Net) by employing a modular architecture. We use a residual on the residual structure to ease low-frequency information flow and apply feature attention to exploit the channel dependencies. Furthermore, the evaluation in terms of quantitative metrics and visual quality for four restoration tasks, i.e., denoising, super-resolution, raindrop removal, and JPEG compression on 11 real degraded datasets against more than 30 state-of-the-art algorithms, demonstrates the superiority of our R²Net. We also present the comparison on three synthetically generated degraded datasets for denoising to showcase our method's capability on synthetics denoising. The codes, trained models, and results are available on https://github.com/saeed-anwar/R2Net.

Towards Interpretable Attention Networks for Cervical Cancer Analysis.

Recent advances in deep learning have enabled the development of automated frameworks for analysing medical images and signals, including analysis of cervical cancer. Many previous works focus on the analysis of isolated cervical cells, or do not offer explainable methods to explore and understand how the proposed models reach their classification decisions on multi-cell images which contain multiple cells. Here, we evaluate various state-of-the-art deep learning models and attention-based frameworks to classify multiple cervical cells. Our aim is to provide interpretable deep learning models by comparing their explainability through the gradients visualization. We demonstrate the importance of using images that contain multiple cells over using isolated single-cell images. We show the effectiveness of the residual channel attention model for extracting important features from a group of cells, and demonstrate this model's efficiency for multiple cervical cells classification. This work highlights the benefits of attention networks to exploit relations and distributions within multi-cell images for cervical cancer analysis. Such an approach can assist clinicians in understanding a model's prediction by providing interpretable results.

Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future.

With the advances of data-driven machine learning research, a wide variety of prediction problems have been tackled. It has become critical to explore how machine learning and specifically deep learning methods can be exploited to analyse healthcare data. A major limitation of existing methods has been the focus on grid-like data; however, the structure of physiological recordings are often irregular and unordered, which makes it difficult to conceptualise them as a matrix. As such, graph neural networks have attracted significant attention by exploiting implicit information that resides in a biological system, with interacting nodes connected by edges whose weights can be determined by either temporal associations or anatomical junctions. In this survey, we thoroughly review the different types of graph architectures and their applications in healthcare. We provide an overview of these methods in a systematic manner, organized by their domain of application including functional connectivity, anatomical structure, and electrical-based analysis. We also outline the limitations of existing techniques and discuss potential directions for future research.

Attention Networks for Multi-Task Signal Analysis.

Recent advances in deep learning have enabled the development of automated frameworks for analysing medical images and signals. For analysis of physiological recordings, models based on temporal convolutional networks and recurrent neural networks have demonstrated encouraging results and an ability to capture complex patterns and dependencies in the data. However, representations that capture the entirety of the raw signal are suboptimal as not all portions of the signal are equally important. As such, attention mechanisms are proposed to divert focus to regions of interest, reducing computational cost and enhancing accuracy. Here, we evaluate attention-based frameworks for the classification of physiological signals in different clinical domains. We evaluated our methodology on three classification scenarios: neurogenerative disorders, neurological status and seizure type. We demonstrate that attention networks can outperform traditional deep learning models for sequence modelling by identifying the most relevant attributes of an input signal for decision making. This work highlights the benefits of attention-based models for analysing raw data in the field of biomedical research.

Neural Memory Networks for Seizure Type Classification.

Classification of seizure type is a key step in the clinical process for evaluating an individual who presents with seizures. It determines the course of clinical diagnosis and treatment, and its impact stretches beyond the clinical domain to epilepsy research and the development of novel therapies. Automated identification of seizure type may facilitate understanding of the disease, and seizure detection and prediction have been the focus of recent research that has sought to exploit the benefits of machine learning and deep learning architectures. Nevertheless, there is not yet a definitive solution for automating the classification of seizure type, a task that must currently be performed by an expert epileptologist. Inspired by recent advances in neural memory networks (NMNs), we introduce a novel approach for the classification of seizure type using electrophysiological data. We first explore the performance of traditional deep learning techniques which use convolutional and recurrent neural networks, and enhance these architectures by using external memory modules with trainable neural plasticity. We show that our model achieves a state-of-the-art weighted F1 score of 0.945 for seizure type classification on the TUH EEG Seizure Corpus with the IBM TUSZ preprocessed data. This work highlights the potential of neural memory networks to support the field of epilepsy research, along with biomedical research and signal analysis more broadly.

Globally-Optimal Inlier Set Maximisation for Camera Pose and Correspondence Estimation.

Estimating the 6-DoF pose of a camera from a single image relative to a 3D point-set is an important task for many computer vision applications. Perspective-n-point solvers are routinely used for camera pose estimation, but are contingent on the provision of good quality 2D-3D correspondences. However, finding cross-modality correspondences between 2D image points and a 3D point-set is non-trivial, particularly when only geometric information is known. Existing approaches to the simultaneous pose and correspondence problem use local optimisation, and are therefore unlikely to find the optimal solution without a good pose initialisation, or introduce restrictive assumptions. Since a large proportion of outliers and many local optima are common for this problem, we instead propose a robust and globally-optimal inlier set maximisation approach that jointly estimates the optimal camera pose and correspondences. Our approach employs branch-and-bound to search the 6D space of camera poses, guaranteeing global optimality without requiring a pose prior. The geometry of SE(3) is used to find novel upper and lower bounds on the number of inliers and local optimisation is integrated to accelerate convergence. The algorithm outperforms existing approaches on challenging synthetic and real datasets, reliably finding the global optimum, with a GPU implementation greatly reducing runtime.

Incorporating Network Built-in Priors in Weakly-Supervised Semantic Segmentation.

Pixel-level annotations are expensive and time consuming to obtain. Hence, weak supervision using only image tags could have a significant impact in semantic segmentation. Recently, CNN-based methods have proposed to fine-tune pre-trained networks using image tags. Without additional information, this leads to poor localization accuracy. This problem, however, was alleviated by making use of objectness priors to generate foreground/background masks. Unfortunately these priors either require pixel-level annotations/bounding boxes, or still yield inaccurate object boundaries. Here, we propose a novel method to extract accurate masks from networks pre-trained for the task of object recognition, thus forgoing external objectness modules. We first show how foreground/background masks can be obtained from the activations of higher-level convolutional layers of a network. We then show how to obtain multi-class masks by the fusion of foreground/background ones with information extracted from a weakly-supervised localization network. Our experiments evidence that exploiting these masks in conjunction with a weakly-supervised training loss yields state-of-the-art tag-based weakly-supervised semantic segmentation results.

Effect of fluoridated milk on enamel demineralization adjacent to fixed orthodontic appliances.

OBJECTIVE: To investigate the effect of daily intake of fluoridated milk on enamel demineralization adjacent to fixed orthodontic brackets assessed with quantitative light-induced fluorescence (QLF). MATERIALS AND METHODS: Sixty-four healthy adolescents (13-18 years) undergoing orthodontic treatment with fixed appliances were enrolled and randomly allocated to a randomized controlled trial with two parallel groups. The intervention group was instructed to drink one glass of milk (≈ 200 ml) supplemented with fluoride (5 ppm) once daily and the subjects of the control group to drink the same amount of milk without fluoride. The intervention period was 12 weeks and the end-point was mineral gain or loss in enamel, assessed by QLF on two selected sites from each individual. The attrition rate was 12.5% and 112 sites were included in the final evaluation. RESULTS: There was no statistically significant difference between the groups concerning fluorescence (ΔF) values and lesion area (A mm(2)) at baseline. After 12 weeks, a significant decrease (p < 0.05) in ΔF was registered in the fluoridated milk group and a significant increase in the non-fluoride control group (p < 0.05). The mean reduction in the test group was somewhat lower (14%) than the increase in the control group (18%), but individual variations were evident. Only minor alterations of lesion area were recorded over the 12-week period and no statistically significant differences compared with baseline were found in any of the groups. CONCLUSION: Daily intake of fluoridated milk may aid remineralization of white spot lesions adjacent to fixed orthodontic appliances.

The role of fluoride in the preventive management of dentin hypersensitivity and root caries.

OBJECTIVE: The objectives were to bring light on fluoride to control dentin hypersensitivity (DHS) and prevent root caries. MATERIALS AND METHODS: Search strategy included papers mainly published in PubMed, Medline from October 2000 to October 2011. RESULTS: Fluoride toothpaste shows a fair effect on sensitive teeth when combined with dentin fluid-obstructing agents such as different metal ions, potassium, and oxalates. Fluoride in solution, gel, and varnish give an instant and long-term relief of dentin and bleaching hypersensitivity. Combined with laser technology, a limited additional positive effect is achieved. Prevention of root caries is favored by toothpaste with 5,000 ppm F and by fluoride rinsing with 0.025-0.1 % F solutions, as the application of fluoride gel or fluoride varnish three to four times a year. Fluoride measures with tablets, chewing gum, toothpick, and flossing may be questioned because of unfavorable cost effectiveness ratio. CONCLUSION: Most fluoride preparations in combination with dentin fluid obstruction agents are beneficial to reduce DHS. Prevention of root caries is favorable with higher fluoride concentrations in, e.g., toothpaste. CLINICAL RELEVANCE: Fluoride is an effective agent to control DHS and to prevent root caries particularly when used in higher concentrations.

Reversal of primary root caries lesions after daily intake of milk supplemented with fluoride and probiotic lactobacilli in older adults.

OBJECTIVE: To evaluate the effect of milk supplemented with fluoride and/or probiotic bacteria on primary root caries lesions (PRCL) in older adults. MATERIALS AND METHODS: After informed consent, 160 healthy subjects, 58-84 years of age, with at least two PRCL were recruited and randomly assigned to one of four parallel study groups drinking 200 ml milk once daily for 15 months. Group A consumed standard milk (placebo); Group B ingested milk supplemented with 5 ppm F and probiotic bacteria (Lactobacillus rhamnosus LB21, 10(7) CFU/mL); Group C drank milk with only probiotic bacteria and group D milk contained only fluoride. Primary endpoints were Root Caries Index (RCI) and electric resistance measurements (ECM) carried out by one blinded single examiner. Secondary endpoints were mutans streptococci and lactobacilli counts in saliva and plaque estimated with chair-side tests. Data were compared within and between groups with non-parametric tests. RESULTS: The drop out rate was 38%. At baseline there were no statistical differences between the groups. Significantly higher numbers of RCI reversals were found in groups B, C and D compared with group A (p < 0.05). The mean ECM values increased significantly (p < 0.05) in all groups except for the placebo group A, indicating that remineralization occurred. The effect was most beneficial in the two groups that contained fluoride. No significant alterations were displayed regarding the microbial counts. No severe adverse effects were reported during intervention. CONCLUSION: Daily intake of milk supplemented with fluoride and/or probiotic bacteria may reverse soft and leathery PRCL in older adults.

Effect of topical applications of a novel chlorhexidine-thymol varnish formula on mutans streptococci and caries development in occlusal fissures of permanent molars.

OBJECTIVE: To evaluate the effect of a new formula of a chlorhexidine-thymol varnish on mutans streptococci (MS) colonization and fissure caries development. METHODS: The study group consisted of 58 healthy adolescents (12-17 years old) undergoing orthodontic treatment with fixed appliances. A double-blind split-mouth design was applied, and 116 pairs of molar teeth were randomly assigned to topical varnish applications with either the new Cervitec Plus or its predecessor Cervitec. Both varnishes contained 1% CHX and 1% thymol (CHX/T) as active ingredients, but differed with respect to adhesive properties. The varnishes were applied in the fissures at baseline, and then every sixth week throughout the 48-week study period. Endpoints were MS colonization (CRT test) and occlusal laser fluorescence (LF, DIAGNOdent) recordings carried out regularly during follow-up. RESULTS: A significant reduction in the levels of MS in the fissures after the initial treatment was displayed with both varnishes (p < 0.05), and the levels remained consistently suppressed throughout the follow-up period. A non-significant but clear tendency was noted in favor of the new formula after six and 12 weeks, with fewer teeth harboring high counts (> or = 10(5) CFU) of MS (6w:12% vs. 24%, 12w:16% vs. 26%). No new lesions were registered in any subject, and the mean LF recordings did not change significantly over time in the groups. CONCLUSION: There were no statistically significant differences between the CHX/T varnishes in terms of bacterial growth and caries prevention. For MS suppression, a tendency towards an initially superior effect was displayed with the new formula.