Less Is More: Adaptive Trainable Gradient Dropout for Deep Neural Networks.

The undeniable computational power of artificial neural networks has granted the scientific community the ability to exploit the available data in ways previously inconceivable. However, deep neural networks require an overwhelming quantity of data in order to interpret the underlying connections between them, and therefore, be able to complete the specific task that they have been assigned to. Feeding a deep neural network with vast amounts of data usually ensures efficiency, but may, however, harm the network's ability to generalize. To tackle this, numerous regularization techniques have been proposed, with dropout being one of the most dominant. This paper proposes a selective gradient dropout method, which, instead of relying on dropping random weights, learns to freeze the training process of specific connections, thereby increasing the overall network's sparsity in an adaptive manner, by driving it to utilize more salient weights. The experimental results show that the produced sparse network outperforms the baseline on numerous image classification datasets, and additionally, the yielded results occurred after significantly less training epochs.

Transformer-Based Fire Detection in Videos.

Fire detection in videos forms a valuable feature in surveillance systems, as its utilization can prevent hazardous situations. The combination of an accurate and fast model is necessary for the effective confrontation of this significant task. In this work, a transformer-based network for the detection of fire in videos is proposed. It is an encoder-decoder architecture that consumes the current frame that is under examination, in order to compute attention scores. These scores denote which parts of the input frame are more relevant for the expected fire detection output. The model is capable of recognizing fire in video frames and specifying its exact location in the image plane in real-time, as can be seen in the experimental results, in the form of segmentation mask. The proposed methodology has been trained and evaluated for two computer vision tasks, the full-frame classification task (fire/no fire in frames) and the fire localization task. In comparison with the state-of-the-art models, the proposed method achieves outstanding results in both tasks, with 97% accuracy, 20.4 fps processing time, 0.02 false positive rate for fire localization, and 97% for f-score and recall metrics in the full-frame classification task.

DeepSurf: a surface-based deep learning approach for the prediction of ligand binding sites on proteins.

MOTIVATION: The knowledge of potentially druggable binding sites on proteins is an important preliminary step toward the discovery of novel drugs. The computational prediction of such areas can be boosted by following the recent major advances in the deep learning field and by exploiting the increasing availability of proper data. RESULTS: In this article, a novel computational method for the prediction of potential binding sites is proposed, called DeepSurf. DeepSurf combines a surface-based representation, where a number of 3D voxelized grids are placed on the protein's surface, with state-of-the-art deep learning architectures. After being trained on the large database of scPDB, DeepSurf demonstrates superior results on three diverse testing datasets, by surpassing all its main deep learning-based competitors, while attaining competitive performance to a set of traditional non-data-driven approaches. AVAILABILITY AND IMPLEMENTATION: The source code of the method along with trained models are freely available at https://github.com/stemylonas/DeepSurf.git. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Continuous Sign Language Recognition through a Context-Aware Generative Adversarial Network.

Continuous sign language recognition is a weakly supervised task dealing with the identification of continuous sign gestures from video sequences, without any prior knowledge about the temporal boundaries between consecutive signs. Most of the existing methods focus mainly on the extraction of spatio-temporal visual features without exploiting text or contextual information to further improve the recognition accuracy. Moreover, the ability of deep generative models to effectively model data distribution has not been investigated yet in the field of sign language recognition. To this end, a novel approach for context-aware continuous sign language recognition using a generative adversarial network architecture, named as Sign Language Recognition Generative Adversarial Network (SLRGAN), is introduced. The proposed network architecture consists of a generator that recognizes sign language glosses by extracting spatial and temporal features from video sequences, as well as a discriminator that evaluates the quality of the generator's predictions by modeling text information at the sentence and gloss levels. The paper also investigates the importance of contextual information on sign language conversations for both Deaf-to-Deaf and Deaf-to-hearing communication. Contextual information, in the form of hidden states extracted from the previous sentence, is fed into the bidirectional long short-term memory module of the generator to improve the recognition accuracy of the network. At the final stage, sign language translation is performed by a transformer network, which converts sign language glosses to natural language text. Our proposed method achieved word error rates of 23.4%, 2.1% and 2.26% on the RWTH-Phoenix-Weather-2014 and the Chinese Sign Language (CSL) and Greek Sign Language (GSL) Signer Independent (SI) datasets, respectively.

Artificial Intelligence Technologies for Sign Language.

AI technologies can play an important role in breaking down the communication barriers of deaf or hearing-impaired people with other communities, contributing significantly to their social inclusion. Recent advances in both sensing technologies and AI algorithms have paved the way for the development of various applications aiming at fulfilling the needs of deaf and hearing-impaired communities. To this end, this survey aims to provide a comprehensive review of state-of-the-art methods in sign language capturing, recognition, translation and representation, pinpointing their advantages and limitations. In addition, the survey presents a number of applications, while it discusses the main challenges in the field of sign language technologies. Future research direction are also proposed in order to assist prospective researchers towards further advancing the field.

Implicit and Explicit Regularization for Optical Flow Estimation.

In this paper, two novel and practical regularizing methods are proposed to improve existing neural network architectures for monocular optical flow estimation. The proposed methods aim to alleviate deficiencies of current methods, such as flow leakage across objects and motion consistency within rigid objects, by exploiting contextual information. More specifically, the first regularization method utilizes semantic information during the training process to explicitly regularize the produced optical flow field. The novelty of this method lies in the use of semantic segmentation masks to teach the network to implicitly identify the semantic edges of an object and better reason on the local motion flow. A novel loss function is introduced that takes into account the objects' boundaries as derived from the semantic segmentation mask to selectively penalize motion inconsistency within an object. The method is architecture agnostic and can be integrated into any neural network without modifying or adding complexity at inference. The second regularization method adds spatial awareness to the input data of the network in order to improve training stability and efficiency. The coordinates of each pixel are used as an additional feature, breaking the invariance properties of the neural network architecture. The additional features are shown to implicitly regularize the optical flow estimation enforcing a consistent flow, while improving both the performance and the convergence time. Finally, the combination of both regularization methods further improves the performance of existing cutting edge architectures in a complementary way, both quantitatively and qualitatively, on popular flow estimation benchmark datasets.

DeepMoCap: Deep Optical Motion Capture Using Multiple Depth Sensors and Retro-Reflectors.

In this paper, a marker-based, single-person optical motion capture method (DeepMoCap) is proposed using multiple spatio-temporally aligned infrared-depth sensors and retro-reflective straps and patches (reflectors). DeepMoCap explores motion capture by automatically localizing and labeling reflectors on depth images and, subsequently, on 3D space. Introducing a non-parametric representation to encode the temporal correlation among pairs of colorized depthmaps and 3D optical flow frames, a multi-stage Fully Convolutional Network (FCN) architecture is proposed to jointly learn reflector locations and their temporal dependency among sequential frames. The extracted reflector 2D locations are spatially mapped in 3D space, resulting in robust 3D optical data extraction. The subject's motion is efficiently captured by applying a template-based fitting technique on the extracted optical data. Two datasets have been created and made publicly available for evaluation purposes; one comprising multi-view depth and 3D optical flow annotated images (DMC2.5D), and a second, consisting of spatio-temporally aligned multi-view depth images along with skeleton, inertial and ground truth MoCap data (DMC3D). The FCN model outperforms its competitors on the DMC2.5D dataset using 2D Percentage of Correct Keypoints (PCK) metric, while the motion capture outcome is evaluated against RGB-D and inertial data fusion approaches on DMC3D, outperforming the next best method by 4 . 5 % in total 3D PCK accuracy.

Deep Learning on Multi Sensor Data for Counter UAV Applications-A Systematic Review.

Usage of Unmanned Aerial Vehicles (UAVs) is growing rapidly in a wide range of consumer applications, as they prove to be both autonomous and flexible in a variety of environments and tasks. However, this versatility and ease of use also brings a rapid evolution of threats by malicious actors that can use UAVs for criminal activities, converting them to passive or active threats. The need to protect critical infrastructures and important events from such threats has brought advances in counter UAV (c-UAV) applications. Nowadays, c-UAV applications offer systems that comprise a multi-sensory arsenal often including electro-optical, thermal, acoustic, radar and radio frequency sensors, whose information can be fused to increase the confidence of threat's identification. Nevertheless, real-time surveillance is a cumbersome process, but it is absolutely essential to detect promptly the occurrence of adverse events or conditions. To that end, many challenging tasks arise such as object detection, classification, multi-object tracking and multi-sensor information fusion. In recent years, researchers have utilized deep learning based methodologies to tackle these tasks for generic objects and made noteworthy progress, yet applying deep learning for UAV detection and classification is considered a novel concept. Therefore, the need to present a complete overview of deep learning technologies applied to c-UAV related tasks on multi-sensor data has emerged. The aim of this paper is to describe deep learning advances on c-UAV related tasks when applied to data originating from many different sensors as well as multi-sensor information fusion. This survey may help in making recommendations and improvements of c-UAV applications for the future.

AI nutrition recommendation using a deep generative model and ChatGPT.

In recent years, major advances in artificial intelligence (AI) have led to the development of powerful AI systems for use in the field of nutrition in order to enhance personalized dietary recommendations and improve overall health and well-being. However, the lack of guidelines from nutritional experts has raised questions on the accuracy and trustworthiness of the nutritional advice provided by such AI systems. This paper aims to address this issue by introducing a novel AI-based nutrition recommendation method that leverages the speed and explainability of a deep generative network and the use of novel sophisticated loss functions to align the network with established nutritional guidelines. The use of a variational autoencoder to robustly model the anthropometric measurements and medical condition of users in a descriptive latent space, as well as the use of an optimizer to adjust meal quantities based on users' energy requirements enable the proposed method to generate highly accurate, nutritious and personalized weekly meal plans. Coupled with the ability of ChatGPT to provide an unparalleled pool of meals from various cuisines, the proposed method can achieve increased meal variety, accuracy and generalization capabilities. Extensive experiments on 3000 virtual user profiles and 84000 daily meal plans, as well as 1000 real profiles and 7000 daily meal plans, demonstrate the exceptional accuracy of the proposed diet recommendation method in generating weekly meal plans that are appropriate for the users in terms of energy intake and nutritional requirements, as well as the easiness with which it can be integrated into future diet recommendation systems.

Can ChatGPT provide appropriate meal plans for NCD patients?

OBJECTIVES: Dietary habits significantly affect health conditions and are closely related to the onset and progression of non-communicable diseases (NCDs). Consequently, a well-balanced diet plays an important role in lessening the effects of various disorders, including NCDs. Several artificial intelligence recommendation systems have been developed to propose healthy and nutritious diets. Most of these systems use expert knowledge and guidelines to provide tailored diets and encourage healthier eating habits. However, new advances in large language models such as ChatGPT, with their ability to produce human-like responses, have led individuals to search for advice in several tasks, including diet recommendations. This study aimed to determine the ability of ChatGPT models to generate appropriate personalized meal plans for patients with obesity, cardiovascular diseases, and type 2 diabetes. METHODS: Using a state-of-the-art knowledge-based recommendation system as a reference, we assessed the meal plans generated by two large language models in terms of energy intake, nutrient accuracy, and meal variability. RESULTS: Experimental results with different user profiles revealed the potential of ChatGPT models to provide personalized nutritional advice. CONCLUSION: Additional supervision and guidance by nutrition experts or knowledge-based systems are required to ensure meal appropriateness for users with NCDs.

PROTEIN AI Advisor: A Knowledge-Based Recommendation Framework Using Expert-Validated Meals for Healthy Diets.

AI-based software applications for personalized nutrition have recently gained increasing attention to help users follow a healthy lifestyle. In this paper, we present a knowledge-based recommendation framework that exploits an explicit dataset of expert-validated meals to offer highly accurate diet plans spanning across ten user groups of both healthy subjects and participants with health conditions. The proposed advisor is built on a novel architecture that includes (a) a qualitative layer for verifying ingredient appropriateness, and (b) a quantitative layer for synthesizing meal plans. The first layer is implemented as an expert system for fuzzy inference relying on an ontology of rules acquired by experts in Nutrition, while the second layer as an optimization method for generating daily meal plans based on target nutrient values and ranges. The system's effectiveness is evaluated through extensive experiments for establishing meal and meal plan appropriateness, meal variety, as well as system capacity for recommending meal plans. Evaluations involved synthetic data, including the generation of 3000 virtual user profiles and their weekly meal plans. Results reveal a high precision and recall for recommending appropriate ingredients in most user categories, while the meal plan generator achieved a total recommendation accuracy of 92% for all nutrient recommendations.

Users' Perspective on the AI-Based Smartphone PROTEIN App for Personalized Nutrition and Healthy Living: A Modified Technology Acceptance Model (mTAM) Approach.

The ubiquitous nature of smartphone ownership, its broad application and usage, along with its interactive delivery of timely feedback are appealing for health-related behavior change interventions via mobile apps. However, users' perspectives about such apps are vital in better bridging the gap between their design intention and effective practical usage. In this vein, a modified technology acceptance model (mTAM) is proposed here, to explain the relationship between users' perspectives when using an AI-based smartphone app for personalized nutrition and healthy living, namely, PROTEIN, and the mTAM constructs toward behavior change in their nutrition and physical activity habits. In particular, online survey data from 85 users of the PROTEIN app within a period of 2 months were subjected to confirmatory factor analysis (CFA) and regression analysis (RA) to reveal the relationship of the mTAM constructs, i.e., perceived usefulness (PU), perceived ease of use (PEoU), perceived novelty (PN), perceived personalization (PP), usage attitude (UA), and usage intention (UI) with the users' behavior change (BC), as expressed via the acceptance/rejection of six related hypotheses (H1-H6), respectively. The resulted CFA-related parameters, i.e., factor loading (FL) with the related p-value, average variance extracted (AVE), and composite reliability (CR), along with the RA results, have shown that all hypotheses H1-H6 can be accepted (p < 0.001). In particular, it was found that, in all cases, FL > 0.5, CR > 0.7, AVE > 0.5, indicating that the items/constructs within the mTAM framework have good convergent validity. Moreover, the adjusted coefficient of determination (R 2) was found within the range of 0.224-0.732, justifying the positive effect of PU, PEoU, PN, and PP on the UA, that in turn positively affects the UI, leading to the BC. Additionally, using a hierarchical RA, a significant change in the prediction of BC from UA when the UI is used as a mediating variable was identified. The explored mTAM framework provides the means for explaining the role of each construct in the functionality of the PROTEIN app as a supportive tool for the users to improve their healthy living by adopting behavior change in their dietary and physical activity habits. The findings herein offer insights and references for formulating new strategies and policies to improve the collaboration among app designers, developers, behavior scientists, nutritionists, physical activity/exercise physiology experts, and marketing experts for app design/development toward behavior change.

Recurrent neural network pruning using dynamical systems and iterative fine-tuning.

Network pruning techniques are widely employed to reduce the memory requirements and increase the inference speed of neural networks. This work proposes a novel RNN pruning method that considers the RNN weight matrices as collections of time-evolving signals. Such signals that represent weight vectors can be modelled using Linear Dynamical Systems (LDSs). In this way, weight vectors with similar temporal dynamics can be pruned as they have limited effect on the performance of the model. Additionally, during the fine-tuning of the pruned model, a novel discrimination-aware variation of the L2 regularization is introduced to penalize network weights (i.e., reduce the magnitude), whose impact on the output of an RNN network is minimal. Finally, an iterative fine-tuning approach is proposed that employs a bigger model to guide an increasingly smaller pruned one, as a steep decrease of the network parameters can irreversibly harm the performance of the pruned model. Extensive experimentation with different network architectures demonstrates the potential of the proposed method to create pruned models with significantly improved perplexity by at least 0.62% on the PTB dataset and improved F1-score by 1.39% on the SQuAD dataset, contrary to other state-of-the-art approaches that slightly improve or even deteriorate models' performance.

Volume-of-Interest Aware Deep Neural Networks for Rapid Chest CT-Based COVID-19 Patient Risk Assessment.

Since December 2019, the world has been devastated by the Coronavirus Disease 2019 (COVID-19) pandemic. Emergency Departments have been experiencing situations of urgency where clinical experts, without long experience and mature means in the fight against COVID-19, have to rapidly decide the most proper patient treatment. In this context, we introduce an artificially intelligent tool for effective and efficient Computed Tomography (CT)-based risk assessment to improve treatment and patient care. In this paper, we introduce a data-driven approach built on top of volume-of-interest aware deep neural networks for automatic COVID-19 patient risk assessment (discharged, hospitalized, intensive care unit) based on lung infection quantization through segmentation and, subsequently, CT classification. We tackle the high and varying dimensionality of the CT input by detecting and analyzing only a sub-volume of the CT, the Volume-of-Interest (VoI). Differently from recent strategies that consider infected CT slices without requiring any spatial coherency between them, or use the whole lung volume by applying abrupt and lossy volume down-sampling, we assess only the "most infected volume" composed of slices at its original spatial resolution. To achieve the above, we create, present and publish a new labeled and annotated CT dataset with 626 CT samples from COVID-19 patients. The comparison against such strategies proves the effectiveness of our VoI-based approach. We achieve remarkable performance on patient risk assessment evaluated on balanced data by reaching 88.88%, 89.77%, 94.73% and 88.88% accuracy, sensitivity, specificity and F1-score, respectively.

Validation of a Deep Learning System for the Full Automation of Bite and Meal Duration Analysis of Experimental Meal Videos.

Eating behavior can have an important effect on, and be correlated with, obesity and eating disorders. Eating behavior is usually estimated through self-reporting measures, despite their limitations in reliability, based on ease of collection and analysis. A better and widely used alternative is the objective analysis of eating during meals based on human annotations of in-meal behavioral events (e.g., bites). However, this methodology is time-consuming and often affected by human error, limiting its scalability and cost-effectiveness for large-scale research. To remedy the latter, a novel "Rapid Automatic Bite Detection" (RABiD) algorithm that extracts and processes skeletal features from videos was trained in a video meal dataset (59 individuals; 85 meals; three different foods) to automatically measure meal duration and bites. In these settings, RABiD achieved near perfect agreement between algorithmic and human annotations (Cohen's kappa κ = 0.894; F1-score: 0.948). Moreover, RABiD was used to analyze an independent eating behavior experiment (18 female participants; 45 meals; three different foods) and results showed excellent correlation between algorithmic and human annotations. The analyses revealed that, despite the changes in food (hash vs. meatballs), the total meal duration remained the same, while the number of bites were significantly reduced. Finally, a descriptive meal-progress analysis revealed that different types of food affect bite frequency, although overall bite patterns remain similar (the outcomes were the same for RABiD and manual). Subjects took bites more frequently at the beginning and the end of meals but were slower in-between. On a methodological level, RABiD offers a valid, fully automatic alternative to human meal-video annotations for the experimental analysis of human eating behavior, at a fraction of the cost and the required time, without any loss of information and data fidelity.

Fast and Precise Hippocampus Segmentation Through Deep Convolutional Neural Network Ensembles and Transfer Learning.

Automatic segmentation of the hippocampus from 3D magnetic resonance imaging mostly relied on multi-atlas registration methods. In this work, we exploit recent advances in deep learning to design and implement a fully automatic segmentation method, offering both superior accuracy and fast result. The proposed method is based on deep Convolutional Neural Networks (CNNs) and incorporates distinct segmentation and error correction steps. Segmentation masks are produced by an ensemble of three independent models, operating with orthogonal slices of the input volume, while erroneous labels are subsequently corrected by a combination of Replace and Refine networks. We explore different training approaches and demonstrate how, in CNN-based segmentation, multiple datasets can be effectively combined through transfer learning techniques, allowing for improved segmentation quality. The proposed method was evaluated using two different public datasets and compared favorably to existing methodologies. In the EADC-ADNI HarP dataset, the correspondence between the method's output and the available ground truth manual tracings yielded a mean Dice value of 0.9015, while the required segmentation time for an entire MRI volume was 14.8 seconds. In the MICCAI dataset, the mean Dice value increased to 0.8835 through transfer learning from the larger EADC-ADNI HarP dataset.

Computerized decision support for beneficial home-based exercise rehabilitation in patients with cardiovascular disease.

BACKGROUND: Exercise-based rehabilitation plays a key role in improving the health and quality of life of patients with Cardiovascular Disease (CVD). Home-based computer-assisted rehabilitation programs have the potential to facilitate and support physical activity interventions and improve health outcomes. OBJECTIVES: We present the development and evaluation of a computerized Decision Support System (DSS) for unsupervised exercise rehabilitation at home, aiming to show the feasibility and potential of such systems toward maximizing the benefits of rehabilitation programs. METHODS: The development of the DSS was based on rules encapsulating the logic according to which an exercise program can be executed beneficially according to international guidelines and expert knowledge. The DSS considered data from a prescribed exercise program, heart rate from a wristband device, and motion accuracy from a depth camera, and subsequently generated personalized, performance-driven adaptations to the exercise program. Communication interfaces in the form of RESTful web service operations were developed enabling interoperation with other computer systems. RESULTS: The DSS was deployed in a computer-assisted platform for exercise-based cardiac rehabilitation at home, and it was evaluated in simulation and real-world studies with CVD patients. The simulation study based on data provided from 10 CVD patients performing 45 exercise sessions in total, showed that patients can be trained within or above their beneficial HR zones for 67.1 ±â€¯22.1% of the exercise duration in the main phase, when they are guided with the DSS. The real-world study with 3 CVD patients performing 43 exercise sessions through the computer-assisted platform, showed that patients can be trained within or above their beneficial heart rate zones for 87.9 ±â€¯8.0% of the exercise duration in the main phase, with DSS guidance. CONCLUSIONS: Computerized decision support systems can guide patients to the beneficial execution of their exercise-based rehabilitation program, and they are feasible.

Three-dimensional shape-structure comparison method for protein classification.

In this paper, a 3D shape-based approach is presented for the efficient search, retrieval, and classification of protein molecules. The method relies primarily on the geometric 3D structure of the proteins, which is produced from the corresponding PDB files and secondarily on their primary and secondary structure. After proper positioning of the 3D structures, in terms of translation and scaling, the Spherical Trace Transform is applied to them so as to produce geometry-based descriptor vectors, which are completely rotation invariant and perfectly describe their 3D shape. Additionally, characteristic attributes of the primary and secondary structure of the protein molecules are extracted, forming attribute-based descriptor vectors. The descriptor vectors are weighted and an integrated descriptor vector is produced. Three classification methods are tested. A part of the FSSP/DALI database, which provides a structural classification of the proteins, is used as the ground truth in order to evaluate the classification accuracy of the proposed method. The experimental results show that the proposed method achieves more than 99 percent classification accuracy while remaining much simpler and faster than the DALI method.