Two-stream Graph Convolutional Networks with Task-specific Loss for Dual-task Gait Analysis.

Dual-task gait systems can be utilized to assess elderly patients for cognitive decline. Although numerous research studies have been conducted to estimate cognitive scores, this field still faces two significant challenges. Firstly, it is crucial to fully utilize dual-task cost representations for diagnosis. Secondly, the design of optimal strategies for effectively extracting dual-task cost representations remains a challenge. To address these issues, in this paper, we propose a deep learning-based framework that implements a spatio-temporal graph convolutional neural network (ST-GCN) with single-task and dual-task pathways for cognitive impairment detection in gait. We also introduce a novel loss, termed task-specific loss, to ensure that single-task and dual-task representations are distinguishable from each other. Furthermore, dual-task cost representations are calculated as the difference between dual-task and single-task representations, which are resilient to individual differences and contribute to the robustness of the framework. These representations provide a comprehensive view of single-task and dual-task gait information to generate task predictions. The proposed framework outperforms existing approaches with a sensitivity of 0.969 and a specificity of 0.940 for cognitive impairment detection.

Discrete Search Photometric Stereo for Fast and Accurate Shape Estimation.

We consider the problem of estimating surface normals of a scene with spatially varying, general bidirectional reflectance distribution functions (BRDFs) observed by a static camera under varying distant illuminations. Unlike previous approaches that rely on continuous optimization of surface normals, we cast the problem as a discrete search problem over a set of finely discretized surface normals. In this setting, we show that the expensive processes can be precomputed in a scene-independent manner, resulting in accelerated inference. We discuss two variants of our discrete search photometric stereo (DSPS), one working with continuous linear combinations of BRDF bases and the other working with discrete BRDFs sampled from a BRDF space. Experiments show that DSPS has comparable accuracy to state-of-the-art exemplar-based photometric stereo methods while achieving 10-100x acceleration.

3D modeling and reconstruction of plants and trees: A cross-cutting review across computer graphics, vision, and plant phenotyping.

This paper reviews the past and current trends of three-dimensional (3D) modeling and reconstruction of plants and trees. These topics have been studied in multiple research fields, including computer vision, graphics, plant phenotyping, and forestry. This paper, therefore, provides a cross-cutting review. Representations of plant shape and structure are first summarized, where every method for plant modeling and reconstruction is based on a shape/structure representation. The methods were then categorized into 1) creating non-existent plants (modeling) and 2) creating models from real-world plants (reconstruction). This paper also discusses the limitations of current methods and possible future directions.

Exploring the mechanical and morphological rationality of tree branch structure based on 3D point cloud analysis and the finite element method.

Trees are thought to have acquired a mechanically optimized shape through evolution, but a scientific methodology to investigate the mechanical rationality of tree morphology remains to be established. The aim of this study was to develop a new method for 3D reconstruction of actual tree shape and to establish a theoretical formulation for elucidating the structure and function of tree branches. We obtained 3D point cloud data of tree shape of Japanese zelkova (Zelkova serrata) and Japanese larch (Larix kaempferi) using the NavVis Lidar scanner, then applied a cylinder structure extraction from point cloud data with error estimation. We then formulated the mechanical stress of branches under gravity using the elastic theory, and performed finite element method simulations to evaluate the mechanical characteristics. Subsequently, we constructed a mechanics-based theoretical formulation of branch development that ensures constant bending stress produces various branching patterns depending on growth properties. The derived theory recapitulates the trade-off among branch growth anisotropy, stress-gravity length, and branch shape, which may open the quantitative way to evaluate mechanical and morphological rationality of tree branches.

Early Detection of Low Cognitive Scores from Dual-task Performance Data Using a Spatio-temporal Graph Convolutional Neural Network.

Detecting low cognitive scores at an early stage is important for delaying the progress of dementia. Investigations of early-stage detection have employed automatic assessment using dual-task (i.e., performing two different tasks simultaneously). However, current approaches to dual-task-based detection are based on either simple features or limited motion information, which degrades the detection accuracy. To address this problem, we proposed a framework that uses graph convolutional networks to extract spatio-temporal features from dual-task performance data. Moreover, to make the proposed method robust against data imbalance, we devised a loss function that directly optimizes the summation of the sensitivity and specificity of the detection of low cognitive scores (i.e., score≤ 23 or score≤ 27). Our evaluation is based on 171 subjects from 6 different senior citizens' facilities. Our experimental results demonstrated that the proposed algorithm considerably outperforms the previous standard with respect to both the sensitivity and specificity of the detection of low cognitive scores.

Training instance segmentation neural network with synthetic datasets for crop seed phenotyping.

In order to train the neural network for plant phenotyping, a sufficient amount of training data must be prepared, which requires time-consuming manual data annotation process that often becomes the limiting step. Here, we show that an instance segmentation neural network aimed to phenotype the barley seed morphology of various cultivars, can be sufficiently trained purely by a synthetically generated dataset. Our attempt is based on the concept of domain randomization, where a large amount of image is generated by randomly orienting the seed object to a virtual canvas. The trained model showed 96% recall and 95% average Precision against the real-world test dataset. We show that our approach is effective also for various crops including rice, lettuce, oat, and wheat. Constructing and utilizing such synthetic data can be a powerful method to alleviate human labor costs for deploying deep learning-based analysis in the agricultural domain.

Statistical Analysis of Dual-task Gait Characteristics for Cognitive Score Estimation.

Traditional approaches for the screening of cognitive function are often based on paper tests, such as Mini-Mental State Examination (MMSE), that evaluate the degree of cognitive impairment and provide a score of patient's mental ability. Procedures for conducting paper tests require time investment involving a questioner and not suitable to be carried out frequently. Previous studies showed that dementia impaired patients are not capable of multi-tasking efficiently. Based on this observation an automated system utilizing Kinect device for collecting primarily patient's gait data who carry out locomotion and calculus tasks individually (i.e., single-tasks) and then simultaneously (i.e., dual-task) was introduced. We installed this system in three elderly facilities and collected 10,833 behavior data from 90 subjects. We conducted analyses of the acquired information extracting 12 features of single- and dual-task performance developed a method for automatic dementia score estimation to investigate determined which characteristics are the most important. In result, a machine learning algorithm using single and dual-task performance classified subjects with an MMSE score of 23 or lower with a recall 0.753 and a specificity 0.799. We found the gait characteristics were important features in the score estimation, and referring to both single and dual-task features was effective.

How Convolutional Neural Networks Diagnose Plant Disease.

Deep learning with convolutional neural networks (CNNs) has achieved great success in the classification of various plant diseases. However, a limited number of studies have elucidated the process of inference, leaving it as an untouchable black box. Revealing the CNN to extract the learned feature as an interpretable form not only ensures its reliability but also enables the validation of the model authenticity and the training dataset by human intervention. In this study, a variety of neuron-wise and layer-wise visualization methods were applied using a CNN, trained with a publicly available plant disease image dataset. We showed that neural networks can capture the colors and textures of lesions specific to respective diseases upon diagnosis, which resembles human decision-making. While several visualization methods were used as they are, others had to be optimized to target a specific layer that fully captures the features to generate consequential outputs. Moreover, by interpreting the generated attention maps, we identified several layers that were not contributing to inference and removed such layers inside the network, decreasing the number of parameters by 75% without affecting the classification accuracy. The results provide an impetus for the CNN black box users in the field of plant science to better understand the diagnosis process and lead to further efficient use of deep learning for plant disease diagnosis.