Phenotyping grapevine resistance to downy mildew: deep learning as a promising tool to assess sporulation and necrosis.

BACKGROUND: Downy mildew is a plant disease that affects all cultivated European grapevine varieties. The disease is caused by the oomycete Plasmopara viticola. The current strategy to control this threat relies on repeated applications of fungicides. The most eco-friendly and sustainable alternative solution would be to use bred-resistant varieties. During breeding programs, some wild Vitis species have been used as resistance sources to introduce resistance loci in Vitis vinifera varieties. To ensure the durability of resistance, resistant varieties are built on combinations of these loci, some of which are unfortunately already overcome by virulent pathogen strains. The development of a high-throughput machine learning phenotyping method is now essential for identifying new resistance loci. RESULTS: Images of grapevine leaf discs infected with P. viticola were annotated with OIV 452-1 values, a standard scale, traditionally used by experts to assess resistance visually. This descriptor takes two variables into account the complete phenotype of the symptom: sporulation and necrosis. This annotated dataset was used to train neural networks. Various encoders were used to incorporate prior knowledge of the scale's ordinality. The best results were obtained with the Swin transformer encoder which achieved an accuracy of 81.7%. Finally, from a biological point of view, the model described the studied trait and identified differences between genotypes in agreement with human observers, with an accuracy of 97% but at a high-throughput 650% faster than that of humans. CONCLUSION: This work provides a fast, full pipeline for image processing, including machine learning, to describe the symptoms of grapevine leaf discs infected with P. viticola using the OIV 452-1, a two-symptom standard scale that considers sporulation and necrosis. If symptoms are frequently assessed by visual observation, which is time-consuming, low-throughput, tedious, and expert dependent, the method developed sweeps away all these constraints. This method could be extended to other pathosystems studied on leaf discs where disease symptoms are scored with ordinal scales.

Toward robust and high-throughput detection of seed defects in X-ray images via deep learning.

BACKGROUND: The detection of internal defects in seeds via non-destructive imaging techniques is a topic of high interest to optimize the quality of seed lots. In this context, X-ray imaging is especially suited. Recent studies have shown the feasibility of defect detection via deep learning models in 3D tomography images. We demonstrate the possibility of performing such deep learning-based analysis on 2D X-ray radiography for a faster yet robust method via the X-Robustifier pipeline proposed in this article. RESULTS: 2D X-ray images of both defective and defect-free seeds were acquired. A deep learning model based on state-of-the-art object detection neural networks is proposed. Specific data augmentation techniques are introduced to compensate for the low ratio of defects and increase the robustness to variation of the physical parameters of the X-ray imaging systems. The seed defects were accurately detected (F1-score >90%), surpassing human performance in computation time and error rates. The robustness of these models against the principal distortions commonly found in actual agro-industrial conditions is demonstrated, in particular, the robustness to physical noise, dimensionality reduction and the presence of seed coating. CONCLUSION: This work provides a full pipeline to automatically detect common defects in seeds via 2D X-ray imaging. The method is illustrated on sugar beet and faba bean and could be efficiently extended to other species via the proposed generic X-ray data processing approach (X-Robustifier). Beyond a simple proof of feasibility, this constitutes important results toward the effective use in the routine of deep learning-based automatic detection of seed defects.

Superpixels meet essential spectra for fast Raman hyperspectral microimaging.

In the context of spectral unmixing, essential information corresponds to the most linearly dissimilar rows and/or columns of a two-way data matrix which are indispensable to reproduce the full data matrix in a convex linear way. Essential information has recently been shown accessible on-the-fly via a decomposition of the measured spectra in the Fourier domain and has opened new perspectives for fast Raman hyperspectral microimaging. In addition, when some spatial prior is available about the sample, such as the existence of homogeneous objects in the image, further acceleration for the data acquisition procedure can be achieved by using superpixels. The expected gain in acquisition time is shown to be around three order of magnitude on simulated and real data with very limited distortions of the estimated spectrum of each object composing the images.

Assessment of Essential Information in the Fourier Domain to Accelerate Raman Hyperspectral Microimaging.

In the context of multivariate curve resolution (MCR) and spectral unmixing, essential information (EI) corresponds to the most linearly dissimilar rows and/or columns of a two-way data matrix. In recent works, the assessment of EI has been revealed to be a very useful practical tool to select the most relevant spectral information before MCR analysis, key features being speed and compression ability. However, the canonical approach relies on the principal component analysis to evaluate the convex hull that encapsulates the data structure in the normalized score space. This implies that the evaluation of the essentiality of each spectrum can only be achieved after all the spectra have been acquired by the instrument. This paper proposes a new approach to extract EI in the Fourier domain (EIFD). Spectral information is transformed into Fourier coefficients, and EI is assessed from a convex hull analysis of the data point cloud in the 2D phasor plots of a few selected harmonics. Because the coordinate system of a phasor plot does not depend on the data themselves, the evaluation of the essentiality of the information carried by each spectrum can be achieved individually and independently from the others. As a result, time-consuming operations like Raman spectral imaging can be significantly accelerated exploiting a chemometric-driven (i.e., based on the EI content of a spectral pixel) procedure for data acquisition and targeted sampling. The usefulness of EIFD is shown by analyzing Raman hyperspectral microimaging data, demonstrating a potential 50-fold acceleration of Raman acquisition.

Attention-Based Fusion of Ultrashort Voice Utterances and Depth Videos for Multimodal Person Identification.

Multimodal deep learning, in the context of biometrics, encounters significant challenges due to the dependence on long speech utterances and RGB images, which are often impractical in certain situations. This paper presents a novel solution addressing these issues by leveraging ultrashort voice utterances and depth videos of the lip for person identification. The proposed method utilizes an amalgamation of residual neural networks to encode depth videos and a Time Delay Neural Network architecture to encode voice signals. In an effort to fuse information from these different modalities, we integrate self-attention and engineer a noise-resistant model that effectively manages diverse types of noise. Through rigorous testing on a benchmark dataset, our approach exhibits superior performance over existing methods, resulting in an average improvement of 10%. This method is notably efficient for scenarios where extended utterances and RGB images are unfeasible or unattainable. Furthermore, its potential extends to various multimodal applications beyond just person identification.

Quantitative Analysis of PcG-Associated Condensates by Stochastic Optical Reconstruction Microscopy (STORM).

The polycomb group (PcG) proteins play a central role in the maintenance of a repressive state of gene expression. Recent findings demonstrate that PcG components are organized into nuclear condensates, contributing to the reshaping of chromatin architecture in physiological and pathological conditions, thus affecting the nuclear mechanics. In this context, direct stochastic optical reconstruction microscopy (dSTORM) provides an effective tool to achieve a detailed characterization of PcG condensates by visualizing them at a nanometric level. Furthermore, by analyzing dSTORM datasets with cluster analysis algorithms, quantitative information can be yielded regarding protein numbers, grouping, and spatial organization. Here, we describe how to set up a dSTORM experiment and perform the data analysis to study PcG complexes' components in adhesion cells quantitatively.

GenoDrawing: An Autoencoder Framework for Image Prediction from SNP Markers.

Advancements in genome sequencing have facilitated whole-genome characterization of numerous plant species, providing an abundance of genotypic data for genomic analysis. Genomic selection and neural networks (NNs), particularly deep learning, have been developed to predict complex traits from dense genotypic data. Autoencoders, an NN model to extract features from images in an unsupervised manner, has proven to be useful for plant phenotyping. This study introduces an autoencoder framework, GenoDrawing, for predicting and retrieving apple images from a low-depth single-nucleotide polymorphism (SNP) array, potentially useful in predicting traits that are difficult to define. GenoDrawing demonstrates proficiency in its task using a small dataset of shape-related SNPs. Results indicate that the use of SNPs associated with visual traits has substantial impact on the generated images, consistent with biological interpretation. While using substantial SNPs is crucial, incorporating additional, unrelated SNPs results in performance degradation for simple NN architectures that cannot easily identify the most important inputs. The proposed GenoDrawing method is a practical framework for exploring genomic prediction in fruit tree phenotyping, particularly beneficial for small to medium breeding companies to predict economically substantial heritable traits. Although GenoDrawing has limitations, it sets the groundwork for future research in image prediction from genomic markers. Future studies should focus on using stronger models for image reproduction, SNP information extraction, and dataset balance in terms of phenotypes for more precise outcomes.

A new in vitro monitoring system reveals a specific influence of Arabidopsis nitrogen nutrition on its susceptibility to Alternaria brassicicola at the seedling stage.

BACKGROUND: Seedling growth is an early phase of plant development highly susceptible to environmental factors such as soil nitrogen (N) availability or presence of seed-borne pathogens. Whereas N plays a central role in plant-pathogen interactions, its role has never been studied during this early phase for the interaction between Arabidopsis thaliana and Alternaria brassicicola, a seed-transmitted necrotrophic fungus. The aim of the present work was to develop an in vitro monitoring system allowing to study the impact of the fungus on A. thaliana seedling growth, while modulating N nutrition. RESULTS: The developed system consists of square plates placed vertically and filled with nutrient agar medium allowing modulation of N conditions. Seeds are inoculated after sowing by depositing a droplet of conidial suspension. A specific semi-automated image analysis pipeline based on the Ilastik software was developed to quantify the impact of the fungus on seedling aerial development, calculating an index accounting for every aspect of fungal impact, namely seedling death, necrosis and developmental delay. The system also permits to monitor root elongation. The interest of the system was then confirmed by characterising how N media composition [0.1 and 5 mM of nitrate (NO3-), 5 mM of ammonium (NH4+)] affects the impact of the fungus on three A. thaliana ecotypes. Seedling development was strongly and negatively affected by the fungus. However, seedlings grown with 5 mM NO3- were less susceptible than those grown with NH4+ or 0.1 mM NO3-, which differed from what was observed with adult plants (rosette stage). CONCLUSIONS: The developed monitoring system allows accurate determination of seedling growth characteristics (both on aerial and root parts) and symptoms. Altogether, this system could be used to study the impact of plant nutrition on susceptibility of various genotypes to fungi at the seedling stage.

Machine learning-based detection of label-free cancer stem-like cell fate.

The detection of cancer stem-like cells (CSCs) is mainly based on molecular markers or functional tests giving a posteriori results. Therefore label-free and real-time detection of single CSCs remains a difficult challenge. The recent development of microfluidics has made it possible to perform high-throughput single cell imaging under controlled conditions and geometries. Such a throughput requires adapted image analysis pipelines while providing the necessary amount of data for the development of machine-learning algorithms. In this paper, we provide a data-driven study to assess the complexity of brightfield time-lapses to monitor the fate of isolated cancer stem-like cells in non-adherent conditions. We combined for the first time individual cell fate and cell state temporality analysis in a unique algorithm. We show that with our experimental system and on two different primary cell lines our optimized deep learning based algorithm outperforms classical computer vision and shallow learning-based algorithms in terms of accuracy while being faster than cutting-edge convolutional neural network (CNNs). With this study, we show that tailoring our deep learning-based algorithm to the image analysis problem yields better results than pre-trained models. As a result, such a rapid and accurate CNN is compatible with the rise of high-throughput data generation and opens the door to on-the-fly CSC fate analysis.

Plant disease symptom segmentation in chlorophyll fluorescence imaging with a synthetic dataset.

Despite the wide use of computer vision methods in plant health monitoring, little attention is paid to segmenting the diseased leaf area at its early stages. It can be explained by the lack of datasets of plant images with annotated disease lesions. We propose a novel methodology to generate fluorescent images of diseased plants with an automated lesion annotation. We demonstrate that a U-Net model aiming to segment disease lesions on fluorescent images of plant leaves can be efficiently trained purely by a synthetically generated dataset. The trained model showed 0.793% recall and 0.723% average precision against an empirical fluorescent test dataset. Creating and using such synthetic data can be a powerful technique to facilitate the application of deep learning methods in precision crop protection. Moreover, our method of generating synthetic fluorescent images is a way to improve the generalization ability of deep learning models.

CNN-LSTM Based Multimodal MRI and Clinical Data Fusion for Predicting Functional Outcome in Stroke Patients.

Clinical outcome prediction plays an important role in stroke patient management. From a machine learning point-of-view, one of the main challenges is dealing with heterogeneous data at patient admission, i.e. the image data which are multidimensional and the clinical data which are scalars. In this paper, a multimodal convolutional neural network - long short-term memory (CNN-LSTM) based ensemble model is proposed. For each MR image module, a dedicated network provides preliminary prediction of the clinical outcome using the modified Rankin scale (mRS). The final mRS score is obtained by merging the preliminary probabilities of each module dedicated to a specific type of MR image weighted by the clinical metadata, here age or the National Institutes of Health Stroke Scale (NIHSS). The experimental results demonstrate that the proposed model surpasses the baselines and offers an original way to automatically encode the spatio-temporal context of MR images in a deep learning architecture. The highest AUC (0.77) was achieved for the proposed model with NIHSS. Clinical Relevance- - We present the first deep learning approach predicting the clinical outcome of stroke patients treated by mechanical thrombectomy which integrates imaging data at the voxel level with key clinical metadata. Combining clinical and imaging data to evaluate the potential benefit from therapy closely mirrors the clinical decision process. Our promising results suggest our predictive model could assist in acute stroke management.

On the robustness of machine learning algorithms toward microfluidic distortions for cell classification via on-chip fluorescence microscopy.

Single-cell imaging and sorting are critical technologies in biology and clinical applications. The power of these technologies is increased when combined with microfluidics, fluorescence markers, and machine learning. However, this quest faces several challenges. One of these is the effect of the sample flow velocity on the classification performances. Indeed, cell flow speed affects the quality of image acquisition by increasing motion blur and decreasing the number of acquired frames per sample. We investigate how these visual distortions impact the final classification task in a real-world use-case of cancer cell screening, using a microfluidic platform in combination with light sheet fluorescence microscopy. We demonstrate, by analyzing both simulated and experimental data, that it is possible to achieve high flow speed and high accuracy in single-cell classification. We prove that it is possible to overcome the 3D slice variability of the acquired 3D volumes, by relying on their 2D sum z-projection transformation, to reach an efficient real time classification with an accuracy of 99.4% using a convolutional neural network with transfer learning from simulated data. Beyond this specific use-case, we provide a web platform to generate a synthetic dataset and to investigate the effect of flow speed on cell classification for any biological samples and a large variety of fluorescence microscopes (https://www.creatis.insa-lyon.fr/site7/en/MicroVIP).

A Generic Formula and Some Special Cases for the Kullback-Leibler Divergence between Central Multivariate Cauchy Distributions.

This paper introduces a closed-form expression for the Kullback-Leibler divergence (KLD) between two central multivariate Cauchy distributions (MCDs) which have been recently used in different signal and image processing applications where non-Gaussian models are needed. In this overview, the MCDs are surveyed and some new results and properties are derived and discussed for the KLD. In addition, the KLD for MCDs is showed to be written as a function of Lauricella D-hypergeometric series FD(p). Finally, a comparison is made between the Monte Carlo sampling method to approximate the KLD and the numerical value of the closed-form expression of the latter. The approximation of the KLD by Monte Carlo sampling method are shown to converge to its theoretical value when the number of samples goes to the infinity.

Segmentation of structural parts of rosebush plants with 3D point-based deep learning methods.

BACKGROUND: Segmentation of structural parts of 3D models of plants is an important step for plant phenotyping, especially for monitoring architectural and morphological traits. Current state-of-the art approaches rely on hand-crafted 3D local features for modeling geometric variations in plant structures. While recent advancements in deep learning on point clouds have the potential of extracting relevant local and global characteristics, the scarcity of labeled 3D plant data impedes the exploration of this potential. RESULTS: We adapted six recent point-based deep learning architectures (PointNet, PointNet++, DGCNN, PointCNN, ShellNet, RIConv) for segmentation of structural parts of rosebush models. We generated 3D synthetic rosebush models to provide adequate amount of labeled data for modification and pre-training of these architectures. To evaluate their performance on real rosebush plants, we used the ROSE-X data set of fully annotated point cloud models. We provided experiments with and without the incorporation of synthetic data to demonstrate the potential of point-based deep learning techniques even with limited labeled data of real plants. CONCLUSION: The experimental results show that PointNet++ produces the highest segmentation accuracy among the six point-based deep learning methods. The advantage of PointNet++ is that it provides a flexibility in the scales of the hierarchical organization of the point cloud data. Pre-training with synthetic 3D models boosted the performance of all architectures, except for PointNet.

Recent advances in E-monitoring of plant diseases.

Infectious plant diseases are caused by pathogenic microorganisms, such as fungi, oomycetes, bacteria, viruses, phytoplasma, and nematodes. Plant diseases have a significant effect on the plant quality and yield and they can destroy the entire plant if they are not controlled in time. To minimize disease-related losses, it is essential to identify and control pathogens in the early stages. Plant disease control is thus a fundamental challenge both for global food security and sustainable agriculture. Conventional methods for plant diseases control have given place to electronic control (E-monitoring) due to their lack of portability, being time consuming, need for a specialized user, etc. E-monitoring using electronic nose (e-nose), biosensors, wearable sensors, and 'electronic eyes' has attracted increasing attention in recent years. Detection, identification, and quantification of pathogens based on electronic sensors (E-sensors) are both convenient and practical and may be used in combination with conventional methods. This paper discusses recent advances made in E-sensors as component parts in combination with wearable sensors, in electronic sensing systems to control and detect viruses, bacteria, pathogens and fungi. In addition, future challenges using sensors to manage plant diseases are investigated.

Magnetic resonance elastography with guided pressure waves.

Magnetic resonance elastography aims to non-invasively and remotely characterize the mechanical properties of living tissues. To quantitatively and regionally map the shear viscoelastic moduli in vivo, the technique must achieve proper mechanical excitation throughout the targeted tissues. Although it is straightforward, ante manibus, in close organs such as the liver or the breast, which practitioners clinically palpate already, it is somewhat fortunately highly challenging to trick the natural protective barriers of remote organs such as the brain. So far, mechanical waves have been induced in the latter by shaking the surrounding cranial bones. Here, the skull was circumvented by guiding pressure waves inside the subject's buccal cavity so mechanical waves could propagate from within through the brainstem up to the brain. Repeatable, reproducible and robust displacement fields were recorded in phantoms and in vivo by magnetic resonance elastography with guided pressure waves such that quantitative mechanical outcomes were extracted in the human brain.

Enhancing the Tracking of Seedling Growth Using RGB-Depth Fusion and Deep Learning.

The use of high-throughput phenotyping with imaging and machine learning to monitor seedling growth is a tough yet intriguing subject in plant research. This has been recently addressed with low-cost RGB imaging sensors and deep learning during day time. RGB-Depth imaging devices are also accessible at low-cost and this opens opportunities to extend the monitoring of seedling during days and nights. In this article, we investigate the added value to fuse RGB imaging with depth imaging for this task of seedling growth stage monitoring. We propose a deep learning architecture along with RGB-Depth fusion to categorize the three first stages of seedling growth. Results show an average performance improvement of 5% correct recognition rate by comparison with the sole use of RGB images during the day. The best performances are obtained with the early fusion of RGB and Depth. Also, Depth is shown to enable the detection of growth stage in the absence of the light.

Impact of the reperfusion status for predicting the final stroke infarct using deep learning.

BACKGROUND: Predictive maps of the final infarct may help therapeutic decisions in acute ischemic stroke patients. Our objectives were to assess whether integrating the reperfusion status into deep learning models would improve their performance, and to compare them to current clinical prediction methods. METHODS: We trained and tested convolutional neural networks (CNNs) to predict the final infarct in acute ischemic stroke patients treated by thrombectomy in our center. When training the CNNs, non-reperfused patients from a non-thrombectomized cohort were added to the training set to increase the size of this group. Baseline diffusion and perfusion-weighted magnetic resonance imaging (MRI) were used as inputs, and the lesion segmented on day-6 MRI served as the ground truth for the final infarct. The cohort was dichotomized into two subsets, reperfused and non-reperfused patients, from which reperfusion status specific CNNs were developed and compared to one another, and to the clinically-used perfusion-diffusion mismatch model. Evaluation metrics included the Dice similarity coefficient (DSC), precision, recall, volumetric similarity, Hausdorff distance and area-under-the-curve (AUC). RESULTS: We analyzed 109 patients, including 35 without reperfusion. The highest DSC were achieved in both reperfused and non-reperfused patients (DSC = 0.44 ± 0.25 and 0.47 ± 0.17, respectively) when using the corresponding reperfusion status-specific CNN. CNN-based models achieved higher DSC and AUC values compared to those of perfusion-diffusion mismatch models (reperfused patients: AUC = 0.87 ± 0.13 vs 0.79 ± 0.17, P < 0.001; non-reperfused patients: AUC = 0.81 ± 0.13 vs 0.73 ± 0.14, P < 0.01, in CNN vs perfusion-diffusion mismatch models, respectively). CONCLUSION: The performance of deep learning models improved when the reperfusion status was incorporated in their training. CNN-based models outperformed the clinically-used perfusion-diffusion mismatch model. Comparing the predicted infarct in case of successful vs failed reperfusion may help in estimating the treatment effect and guiding therapeutic decisions in selected patients.

On the value of CTIS imagery for neural-network-based classification: a simulation perspective.

The computed tomography imaging spectrometer (CTIS) is a snapshot hyperspectral imaging system. Its output is a 2D image of multiplexed spatiospectral projections of the hyperspectral cube of the scene. Traditionally, the 3D cube is reconstructed from this image before further analysis. In this paper, we show that it is possible to learn information directly from the CTIS raw output, by training a neural network to perform binary classification on such images. The use case we study is an agricultural one, as snapshot imagery is used substantially in this field: the detection of apple scab lesions on leaves. To train the network appropriately and to study several degrees of scab infection, we simulated CTIS images of scabbed leaves. This was made possible with a novel CTIS simulator, where special care was taken to preserve realistic pixel intensities compared to true images. To the best of our knowledge, this is the first application of compressed learning on a simulated CTIS system.

Deep learning-based detection of seedling development.

BACKGROUND: Monitoring the timing of seedling emergence and early development via high-throughput phenotyping with computer vision is a challenging topic of high interest in plant science. While most studies focus on the measurements of leaf area index or detection of specific events such as emergence, little attention has been put on the identification of kinetics of events of early seedling development on a seed to seed basis. RESULT: Imaging systems screened the whole seedling growth process from the top view. Precise annotation of emergence out of the soil, cotyledon opening, and appearance of first leaf was conducted. This annotated data set served to train deep neural networks. Various strategies to incorporate in neural networks, the prior knowledge of the order of the developmental stages were investigated. Best results were obtained with a deep neural network followed with a long short term memory cell, which achieves more than 90% accuracy of correct detection. CONCLUSION: This work provides a full pipeline of image processing and machine learning to classify three stages of plant growth plus soil on the different accessions of two species of red clover and alfalfa but which could easily be extended to other crops and other stages of development.