Attention-Based Fusion of Ultrashort Voice Utterances and Depth Videos for Multimodal Person Identification.

Multimodal deep learning, in the context of biometrics, encounters significant challenges due to the dependence on long speech utterances and RGB images, which are often impractical in certain situations. This paper presents a novel solution addressing these issues by leveraging ultrashort voice utterances and depth videos of the lip for person identification. The proposed method utilizes an amalgamation of residual neural networks to encode depth videos and a Time Delay Neural Network architecture to encode voice signals. In an effort to fuse information from these different modalities, we integrate self-attention and engineer a noise-resistant model that effectively manages diverse types of noise. Through rigorous testing on a benchmark dataset, our approach exhibits superior performance over existing methods, resulting in an average improvement of 10%. This method is notably efficient for scenarios where extended utterances and RGB images are unfeasible or unattainable. Furthermore, its potential extends to various multimodal applications beyond just person identification.

Enhancing the Tracking of Seedling Growth Using RGB-Depth Fusion and Deep Learning.

The use of high-throughput phenotyping with imaging and machine learning to monitor seedling growth is a tough yet intriguing subject in plant research. This has been recently addressed with low-cost RGB imaging sensors and deep learning during day time. RGB-Depth imaging devices are also accessible at low-cost and this opens opportunities to extend the monitoring of seedling during days and nights. In this article, we investigate the added value to fuse RGB imaging with depth imaging for this task of seedling growth stage monitoring. We propose a deep learning architecture along with RGB-Depth fusion to categorize the three first stages of seedling growth. Results show an average performance improvement of 5% correct recognition rate by comparison with the sole use of RGB images during the day. The best performances are obtained with the early fusion of RGB and Depth. Also, Depth is shown to enable the detection of growth stage in the absence of the light.

Toward Joint Acquisition-Annotation of Images with Egocentric Devices for a Lower-Cost Machine Learning Application to Apple Detection.

Since most computer vision approaches are now driven by machine learning, the current bottleneck is the annotation of images. This time-consuming task is usually performed manually after the acquisition of images. In this article, we assess the value of various egocentric vision approaches in regard to performing joint acquisition and automatic image annotation rather than the conventional two-step process of acquisition followed by manual annotation. This approach is illustrated with apple detection in challenging field conditions. We demonstrate the possibility of high performance in automatic apple segmentation (Dice 0.85), apple counting (88 percent of probability of good detection, and 0.09 true-negative rate), and apple localization (a shift error of fewer than 3 pixels) with eye-tracking systems. This is obtained by simply applying the areas of interest captured by the egocentric devices to standard, non-supervised image segmentation. We especially stress the importance in terms of time of using such eye-tracking devices on head-mounted systems to jointly perform image acquisition and automatic annotation. A gain of time of over 10-fold by comparison with classical image acquisition followed by manual image annotation is demonstrated.

Dielectric elastomer stack actuator-based autofocus fluid lens.

Extremely small cameras and many cell phones simply do not have enough room to allow users to move a rigid lens the distance required for a varying range of focal lengths. An adaptive liquid lens, however, enables small cameras to focus without needing extra room. An autofocus liquid lens provides several advantages over a traditional lens in terms of efficiency, cost, compactness, and flexibility. But one of the main challenges in these lenses is a high driving voltage requirement of around at least 1.8 kV. In this paper, we propose a new design of a liquid lens based on a dielectric elastomer stack actuator (DESA), which significantly overcomes the aforementioned existing problem. The lens consists of a frame (a thin DESA membrane with a hole in the middle), silicon oil, and water. A voltage range is applied on the membrane in order to change the hole dimension. Due to change of hole dimension, a change in meniscus occurs that changes the focal length of the lens. In this research work, various experimental results are achieved by configuring two DESA with different active areas. Depending on the active area of the membrane, the length of the laser beam on the plane varies from 6 to 35 mm, and the driving voltage is in the range of 50-750 V.

Toward robust and high-throughput detection of seed defects in X-ray images via deep learning.

BACKGROUND: The detection of internal defects in seeds via non-destructive imaging techniques is a topic of high interest to optimize the quality of seed lots. In this context, X-ray imaging is especially suited. Recent studies have shown the feasibility of defect detection via deep learning models in 3D tomography images. We demonstrate the possibility of performing such deep learning-based analysis on 2D X-ray radiography for a faster yet robust method via the X-Robustifier pipeline proposed in this article. RESULTS: 2D X-ray images of both defective and defect-free seeds were acquired. A deep learning model based on state-of-the-art object detection neural networks is proposed. Specific data augmentation techniques are introduced to compensate for the low ratio of defects and increase the robustness to variation of the physical parameters of the X-ray imaging systems. The seed defects were accurately detected (F1-score >90%), surpassing human performance in computation time and error rates. The robustness of these models against the principal distortions commonly found in actual agro-industrial conditions is demonstrated, in particular, the robustness to physical noise, dimensionality reduction and the presence of seed coating. CONCLUSION: This work provides a full pipeline to automatically detect common defects in seeds via 2D X-ray imaging. The method is illustrated on sugar beet and faba bean and could be efficiently extended to other species via the proposed generic X-ray data processing approach (X-Robustifier). Beyond a simple proof of feasibility, this constitutes important results toward the effective use in the routine of deep learning-based automatic detection of seed defects.

ROSE-X: an annotated data set for evaluation of 3D plant organ segmentation methods.

BACKGROUND: The production and availability of annotated data sets are indispensable for training and evaluation of automatic phenotyping methods. The need for complete 3D models of real plants with organ-level labeling is even more pronounced due to the advances in 3D vision-based phenotyping techniques and the difficulty of full annotation of the intricate 3D plant structure. RESULTS: We introduce the ROSE-X data set of 11 annotated 3D models of real rosebush plants acquired through X-ray tomography and presented both in volumetric form and as point clouds. The annotation is performed manually to provide ground truth data in the form of organ labels for the voxels corresponding to the plant shoot. This data set is constructed to serve both as training data for supervised learning methods performing organ-level segmentation and as a benchmark to evaluate their performance. The rosebush models in the data set are of high quality and complex architecture with organs frequently touching each other posing a challenge for the current plant organ segmentation methods. We report leaf/stem segmentation results obtained using four baseline methods. The best performance is achieved by the volumetric approach where local features are trained with a random forest classifier, giving Intersection of Union (IoU) values of 97.93% and 86.23% for leaf and stem classes, respectively. CONCLUSION: We provided an annotated 3D data set of 11 rosebush plants for training and evaluation of organ segmentation methods. We also reported leaf/stem segmentation results of baseline methods, which are open to improvement. The data set, together with the baseline results, has the potential of becoming a significant resource for future studies on automatic plant phenotyping.

Deep learning-based detection of seedling development.

BACKGROUND: Monitoring the timing of seedling emergence and early development via high-throughput phenotyping with computer vision is a challenging topic of high interest in plant science. While most studies focus on the measurements of leaf area index or detection of specific events such as emergence, little attention has been put on the identification of kinetics of events of early seedling development on a seed to seed basis. RESULT: Imaging systems screened the whole seedling growth process from the top view. Precise annotation of emergence out of the soil, cotyledon opening, and appearance of first leaf was conducted. This annotated data set served to train deep neural networks. Various strategies to incorporate in neural networks, the prior knowledge of the order of the developmental stages were investigated. Best results were obtained with a deep neural network followed with a long short term memory cell, which achieves more than 90% accuracy of correct detection. CONCLUSION: This work provides a full pipeline of image processing and machine learning to classify three stages of plant growth plus soil on the different accessions of two species of red clover and alfalfa but which could easily be extended to other crops and other stages of development.

Simulated perfusion MRI data to boost training of convolutional neural networks for lesion fate prediction in acute stroke.

The problem of final tissue outcome prediction of acute ischemic stroke is assessed from physically realistic simulated perfusion magnetic resonance images. Different types of simulations with a focus on the arterial input function are discussed. These simulated perfusion magnetic resonance images are fed to convolutional neural network to predict real patients. Performances close to the state-of-the-art performances are obtained with a patient specific approach. This approach consists in training a model only from simulated images tuned to the arterial input function of a tested real patient. This demonstrates the added value of physically realistic simulated images to predict the final infarct from perfusion.

Machine Learning-Based Classification of the Health State of Mice Colon in Cancer Study from Confocal Laser Endomicroscopy.

In this article, we address the problem of the classification of the health state of the colon's wall of mice, possibly injured by cancer with machine learning approaches. This problem is essential for translational research on cancer and is a priori challenging since the amount of data is usually limited in all preclinical studies for practical and ethical reasons. Three states considered including cancer, health, and inflammatory on tissues. Fully automated machine learning-based methods are proposed, including deep learning, transfer learning, and shallow learning with SVM. These methods addressed different training strategies corresponding to clinical questions such as the automatic clinical state prediction on unseen data using a pre-trained model, or in an alternative setting, real-time estimation of the clinical state of individual tissue samples during the examination. Experimental results show the best performance of 99.93% correct recognition rate obtained for the second strategy as well as the performance of 98.49% which were achieved for the more difficult first case.

Local spatio-temporal encoding of raw perfusion MRI for the prediction of final lesion in stroke.

We address the medical image analysis issue of predicting the final lesion in stroke from early perfusion magnetic resonance imaging. The classical processing approach for the dynamical perfusion images consists in a temporal deconvolution to improve the temporal signals associated with each voxel before performing prediction. We demonstrate here the value of exploiting directly the raw perfusion data by encoding the local environment of each voxel as a spatio-temporal texture, with an observation scale larger than the voxel. As a first illustration for this approach, the textures are characterized with local binary patterns and the classification is performed using a standard support vector machine (SVM). This simple machine learning classification scheme demonstrates results with 95% accuracy on average while working only raw perfusion data. We discuss the influence of the observation scale and evaluate the interest of using post-processed perfusion data with this approach.