A Review of Image-Based Food Recognition and Volume Estimation Artificial Intelligence Systems.

The daily healthy diet and balanced intake of essential nutrients play an important role in modern lifestyle. The estimation of a meal's nutrient content is an integral component of significant diseases, such as diabetes, obesity and cardiovascular disease. Lately, there has been an increasing interest towards the development and utilization of smartphone applications with the aim of promoting healthy behaviours. The semi - automatic or automatic, precise and in real-time estimation of the nutrients of daily consumed meals is approached in relevant literature as a computer vision problem using food images which are taken via a user's smartphone. Herein, we present the state-of-the-art on automatic food recognition and food volume estimation methods starting from their basis, i.e., the food image databases. First, by methodically organizing the extracted information from the reviewed studies, this review study enables the comprehensive fair assessment of the methods and techniques applied for segmenting food images, classifying their food content and computing the food volume, associating their results with the characteristics of the used datasets. Second, by unbiasedly reporting the strengths and limitations of these methods and proposing pragmatic solutions to the latter, this review can inspire future directions in the field of dietary assessment systems.

Weight Estimation of Mediterranean Food Images using Random Forest Regression Algorithm.

The daily nutrition management is one of the most important issues that concern individuals in the modern lifestyle. Over the years, the development of dietary assessment systems and applications based on food images has assisted experts to manage people's nutritional facts and eating habits. In these systems, the food volume estimation is the most important task for calculating food quantity and nutritional information. In this study, we present a novel methodology for food weight estimation based on a food image, using the Random Forest regression algorithm. The weight estimation model was trained on a unique dataset of 5,177 annotated Mediterranean food images, consisting of 50 different foods with a reference card placed next to the plate. Then, we created a data frame of 6,425 records from the annotated food images with features such as: food area, reference object area, food id, food category and food weight. Finally, using the Random Forest regression algorithm and applying nested cross validation and hyperparameters tuning, we trained the weight estimation model. The proposed model achieves 22.6 grams average difference between predicted and real weight values for each food item record in the data frame and 15.1% mean absolute percentage error for each food item, opening new perspectives in food image-based volume and nutrition estimation models and systems.Clinical Relevance- The proposed methodology is suitable for healthcare systems and applications that monitor an individual's malnutrition, offering the ability to estimate the energy and nutrients consumed using an image of the meal.

A novel approach to estimate the weight of food items based on features extracted from an image using boosting algorithms.

Managing daily nutrition is a prominent concern among individuals in contemporary society. The advancement of dietary assessment systems and applications utilizing images has facilitated the effective management of individuals' nutritional information and dietary habits over time. The determination of food weight or volume is a vital part in these systems for assessing food quantities and nutritional information. This study presents a novel methodology for evaluating the weight of food by utilizing extracted features from images and training them through advanced boosting regression algorithms. Α unique dataset of 23,052 annotated food images of Mediterranean cuisine, including 226 different dishes with a reference object placed next to the dish, was used to train the proposed pipeline. Then, using extracted features from the annotated images, such as food area, reference object area, food id, food category, and food weight, we built a dataframe with 24,996 records. Finally, we trained the weight estimation model by applying cross validation, hyperparameter tuning, and boosting regression algorithms such as XGBoost, CatBoost, and LightGBM. Between the predicted and actual weight values for each food in the proposed dataset, the proposed model achieves a mean weight absolute error 3.93 g, a mean absolute percentage error 3.73% and a root mean square error 6.05 g for the 226 food items of the Mediterranean Greek Food database (MedGRFood), setting new perspectives in food image-based weight and nutrition estimate models and systems.

An Automated Image-Based Dietary Assessment System for Mediterranean Foods.

Goal: The modern way of living has significantly influenced the daily diet. The ever-increasing number of people with obesity, diabetes and cardiovascular diseases stresses the need to find tools that could help in the daily intake of the necessary nutrients. Methods: In this paper, we present an automated image-based dietary assessment system of Mediterranean food, based on: 1) an image dataset of Mediterranean foods, 2) on a pre-trained Convolutional Neural Network (CNN) for food image classification, and 3) on stereo vision techniques for the volume and nutrition estimation of the food. We use a pre-trained CNN in the Food-101 dataset to train a deep learning classification model employing our dataset Mediterranean Greek Food (MedGRFood). Based on the EfficientNet family of CNNs, we use the EfficientNetB2 both for the pre-trained model and its weights evaluation, as well as for classifying food images in the MedGRFood dataset. Next, we estimate the volume of the food, through 3D food reconstruction of two images taken by a smartphone camera. The proposed volume estimation subsystem uses stereo vision techniques and algorithms, and needs the input of two food images to reconstruct the point cloud of the food and to compute its quantity. Results: The classification accuracy where true class matches with the most probable class predicted by the model (Top-1 accuracy) is 83.8%, while the accuracy where true class matches with any one of the 5 most probable classes predicted by the model (Top-5 accuracy) is 97.6%, for the food classification subsystem. The food volume estimation subsystem achieves an overall mean absolute percentage error 10.5% for 148 different food dishes. Conclusions: The proposed automated image-based dietary assessment system provides the capability of continuous recording of health data in real time.

Short Term Glucose Prediction in Patients with Type 1 Diabetes Mellitus.

Glucose prediction is used in diabetes self-management as it allows to take suitable actions for proper glycemic regulation of the patient. The aim of this work is the short-term personalized glucose prediction in patients with Type 1 diabetes mellitus (T1DM). In this scope, we compared two different models, an autoregressive moving average (ARMA) model and a long short-term memory (LSTM) model for different prediction horizons. The comparison of two models was performed using the evaluation metrics of root mean square error (RMSE) and mean absolute error (MAE). The models were trained and tested in 29 real patients. The results shown that the LSTM model had better performance than ARMA with RMSE 3.13, 6.41 and 8.81 mg/dL and MAE 1.98, 5.06 and 6.47 mg/dL for 5-, 15- and 30-minutes prediction horizon.

GlucoseML Mobile Application for Automated Dietary Assessment of Mediterranean Food.

Over the years and with the help of technology, the daily care of type 1 diabetes has been improved significantly. The increased adoption of continuous glucose monitoring, the continuous subcutaneous insulin injection and the accurate behavioral monitoring mHealth solutions have contributed to this phenomenon. In this study we present a mobile application for automated dietary assessment of Mediterranean food images as part of the GlucoseML system. Based on short-term predictive analysis of the glucose trajectory, GlucoseML is a type-1 diabetes self-management system. A computer vision approach is used as main part of the GlucoseML dietary assessment system calculating food carbohydrates, fats and proteins, relying on: (i) a deep learning subsystem for food image classification, and (ii) a 3D food image reconstruction subsystem for the volume estimation of food. The deep learning subsystem achieves 82.4% and 97.5% top-1 and top-5 accuracy, respectively, for food image classification while the subsystem for volume estimation of food achieves a mean absolute percentage error 10.7% for the four main categories of MedGRFood dataset.

Mediterranean Food Image Recognition Using Deep Convolutional Networks.

We present a new dataset of food images that can be used to evaluate food recognition systems and dietary assessment systems. The Mediterranean Greek food -MedGRFood dataset consists of food images from the Mediterranean cuisine, and mainly from the Greek cuisine. The dataset contains 42,880 food images belonging to 132 food classes which have been collected from the web. Based on the EfficientNet family of convolutional neural networks, specifically the EfficientNetB2, we propose a new deep learning schema that achieves 83.4% top-1 accuracy and 97.8% top-5 accuracy in the MedGRFood dataset for food recognition. This schema includes the use of the fine tuning, transfer learning and data augmentation technique.

Site specific prediction of PCI stenting based on imaging and biomechanics data using gradient boosting tree ensembles.

Cardiovascular diseases are nowadays considered as the main cause of morbidity and mortality worldwide. Coronary Artery Disease (CAD), the most typical form of cardiovascular disease is diagnosed by a variety of imaging modalities, both invasive and non-invasive, which involve either risk implications or high cost. Therefore, several attempts have been undertaken to early diagnose and predict either the high CAD risk patients or the cardiovascular events, implementing machine learning techniques. The purpose of this study is to present a classification scheme for the prediction of Percutaneous Coronary Intervention (PCI) stenting placement, using image-based data. The proposed classification model is a gradient boosting classifier, incorporated into a class imbalance handling technique, the Easy ensemble scheme and aims to classify coronary segments into high CAD risk and low CAD risk, based on their PCI placement. Through this study, we investigate the importance of image based features, concluding that the combination of the coronary degree of stenosis and the fractional flow reserve achieves accuracy 78%.

A Dynamic Bayesian Network Approach to Behavioral Modelling of Elderly People during a Home-based Augmented Reality Balance Physiotherapy Programme.

In this study, we propose a dynamic Bayesian network (DBN)-based approach to behavioral modelling of community dwelling older adults at risk for falls during the daily sessions of a hologram-enabled vestibular rehabilitation therapy programme. The component of human behavior being modelled is the level of frustration experienced by the user at each exercise, as it is assessed by the NASA Task Load Index. Herein, we present the topology of the DBN and test its inference performance on real-patient data.Clinical Relevance- Precise behavioral modelling will provide an indicator for tailoring the rehabilitation programme to each individual's personal psychological needs.

Predicting Lymphoma Development by Exploiting Genetic Variants and Clinical Findings in a Machine Learning-Based Methodology With Ensemble Classifiers in a Cohort of Sjögren's Syndrome Patients.

Lymphoma development constitutes one of the most serious clinico-pathological manifestations of patients with Sjögren's Syndrome (SS). Over the last decades the risk for lymphomagenesis in SS patients has been studied aiming to identify novel biomarkers and risk factors predicting lymphoma development in this patient population. Objective: The current study aims to explore whether genetic susceptibility profiles of SS patients along with known clinical, serological and histological risk factors enhance the accuracy of predicting lymphoma development in this patient population. Methods: The potential predicting role of both genetic variants, clinical and laboratory risk factors were investigated through a Machine Learning-based (ML) framework which encapsulates ensemble classifiers. Results: Ensemble methods empower the classification accuracy with approaches which are sensitive to minor perturbations in the training phase. The evaluation of the proposed methodology based on a 10-fold stratified cross validation procedure yielded considerable results in terms of balanced accuracy (GB: 0.7780 ± 0.1514, RF Gini: 0.7626 ± 0.1787, RF Entropy: 0.7590 ± 0.1837). Conclusions: The initial clinical, serological, histological and genetic findings at an early diagnosis have been exploited in an attempt to establish predictive tools in clinical practice and further enhance our understanding towards lymphoma development in SS.

Short-term prediction of glucose in type 1 diabetes using kernel adaptive filters.

This study aims at presenting a nonlinear, recursive, multivariate prediction model of the subcutaneous glucose concentration in type 1 diabetes. Nonlinear regression is performed in a reproducing kernel Hilbert space, by either the fixed budget quantized kernel least mean square (QKLMS-FB) or the approximate linear dependency kernel recursive least-squares (KRLS-ALD) algorithm, such that a sparse model structure is accomplished. A multivariate feature set (i.e., subcutaneous glucose, food carbohydrates, insulin regime and physical activity) is used and its influence on short-term glucose prediction is investigated. The method is evaluated using data from 15 patients with type 1 diabetes in free-living conditions. In the case when all the input variables are considered: (i) the average root mean squared error (RMSE) of QKLMS-FB increases from 13.1 mg dL-1 (mean absolute percentage error (MAPE) 6.6%) for a 15-min prediction horizon (PH) to 37.7 mg dL-1 (MAPE 20.8%) for a 60-min PH and (ii) the RMSE of KRLS-ALD, being predictably lower, increases from 10.5 mg dL-1 (MAPE 5.2%) for a 15-min PH to 31.8 mg dL-1 (MAPE 18.0%) for a 60-min PH. Multivariate data improve systematically both the regularity and the time lag of the predictions, reducing the errors in critical glucose value regions for a PH ≥ 30 min. Graphical abstract ᅟ.

Cohort Harmonization and Integrative Analysis From a Biomedical Engineering Perspective.

In this review, the critical parts and milestones for data harmonization, from the biomedical engineering perspective, are outlined. The need for data sharing between heterogeneous sources paves the way for cohort harmonization; thus, fostering data integration and interdisciplinary research. Unmet needs in chronic diseases, as well as in other diseases, can be addressed based on the integration of patient health records and the sharing of information of the clinical picture and outcome. The stratification of patients, the determination of various clinical and outcome features, and the identification of novel biomarkers for the different phenotypes of the disease characterize the impact of cohort harmonization in patient-centered clinical research and in precision medicine. Subsequently, the establishment of matching techniques and ontologies for the creation of data schemas are also presented. The exploitation of web technologies and data-collection tools supports the opportunities to achieve new levels of integration and interoperability. Ethical and legal issues that arise when sharing and harmonizing individual-level data are discussed in order to evaluate the harmonization potential. Use cases that shape and test the harmonization approach are explicitly analyzed along with their significant results on their research objectives. Finally, future trends and directions are discussed and critically reviewed toward a roadmap in cohort harmonization for clinical medicine.

A Machine Learning Approach for the Prediction of the Progression of Cardiovascular Disease based on Clinical and Non-Invasive Imaging Data.

Nowadays, cardiovascular diseases are very common and are considered as the main cause of morbidity and mortality worldwide. Coronary Artery Disease (CAD), the most typical cardiovascular disease is diagnosed by a variety of medical imaging modalities, which involve costs and complications. Therefore, several attempts have been undertaken to early diagnose and predict CAD status and progression through machine learning approaches. The purpose of this study is to present a machine learning technique for the prediction of CAD, using image-based data and clinical data. We investigate the effect of vascular anatomical features of the three coronary arteries on the graduation of CAD. A classification model is built to predict the future status of CAD, including cases of "no CAD" patients, "non-obstructive CAD" patients and "obstructive CAD" patients. The best accuracy was achieved by the implementation of a tree-based classifier, J48 classifier, after a ranking feature selection methodology. The majority of the selected features are the vessel geometry derived features, among the traditional risk factors. The combination of geometrical risk factors with the conventional ones constitutes a novel scheme for the CAD prediction.

Kernel-based adaptive learning improves accuracy of glucose predictive modelling in type 1 diabetes: A proof-of-concept study.

This study aims at demonstrating the need for nonlinear recursive models to the identification and prediction of the dynamic glucose system in type 1 diabetes. Nonlinear regression is performed in a reproducing kernel Hilbert space, by the Approximate Linear Dependency Kernel Recursive Least Squares (KRLS-ALD) algorithm, such that a sparse model structure is accomplished. The method is evaluated on seven people with type 1 diabetes in free-living conditions, where a change in glycaemic dynamics is forced by increasing the level of physical activity in the middle of the observational period. The univariate input allows for short-term (≤30 min) predictions with KRLS-ALD reaching an average root mean square error of 15.22±5.95 mgdL-1 and an average time lag of 17.14±2.67 min for an horizon of 30 min. Its performance is considerably better than that of time-invariant (regularized) linear regression models.

Non-linear dynamic modeling of glucose in type 1 diabetes with kernel adaptive filters.

We propose a non-linear recursive solution to the problem of short-term prediction of glucose in type 1 diabetes. The Fixed Budget Quantized Kernel Least Mean Square (QKLMS-FB) algorithm is employed to construct a univariate model of subcutaneous glucose concentration, which: (i) handles nonlinearities by transforming the input space into a high-dimensional Reproducing Kernel Hilbert Space and, (ii) finds a sparse solution by retaining a representative subset of the training input vectors. The dataset comes from the continuous multi-day recordings of 15 type 1 patients in free-living conditions. QKLMS-FB produces an average root mean squared error of 18.66±3.19 mg/dl for a prediction horizon of 30 min with 82.04% of hypoglycemic readings and 93.30% of hyperglycemic ones being classified as clinically accurate or with benign errors. The effect of the prediction horizon is more evident in the hypoglycemic range.

Evaluation of short-term predictors of glucose concentration in type 1 diabetes combining feature ranking with regression models.

Glucose concentration in type 1 diabetes is a function of biological and environmental factors which present high inter-patient variability. The objective of this study is to evaluate a number of features, which are extracted from medical and lifestyle self-monitoring data, with respect to their ability to predict the short-term subcutaneous (s.c.) glucose concentration of an individual. Random forests (RF) and RReliefF algorithms are first employed to rank the candidate feature set. Then, a forward selection procedure follows to build a glucose predictive model, where features are sequentially added to it in decreasing order of importance. Predictions are performed using support vector regression or Gaussian processes. The proposed method is validated on a dataset of 15 type diabetics in real-life conditions. The s.c. glucose profile along with time of the day and plasma insulin concentration are systematically highly ranked, while the effect of food intake and physical activity varies considerably among patients. Moreover, the average prediction error converges in less than d/2 iterations (d is the number of features). Our results suggest that RF and RReliefF can find the most informative features and can be successfully used to customize the input of glucose models.

Online prediction of glucose concentration in type 1 diabetes using extreme learning machines.

We propose an online machine-learning solution to the problem of nonlinear glucose time series prediction in type 1 diabetes. Recently, extreme learning machine (ELM) has been proposed for training single hidden layer feed-forward neural networks. The high accuracy and fast learning speed of ELM drive us to investigate its applicability to the glucose prediction problem. Given that diabetes self-monitoring data are received sequentially, we focus on online sequential ELM (OS-ELM) and online sequential ELM kernels (KOS-ELM). A multivariate feature set is utilized concerning subcutaneous glucose, insulin therapy, carbohydrates intake and physical activity. The dataset comes from the continuous multi-day recordings of 15 type 1 patients in free-living conditions. Assuming stationarity and evaluating the performance of the proposed method by 10-fold cross- validation, KOS-ELM were found to perform better than OS-ELM in terms of prediction error, temporal gain and regularity of predictions for a 30-min prediction horizon.

A glucose model based on support vector regression for the prediction of hypoglycemic events under free-living conditions.

BACKGROUND: The prevention of hypoglycemic events is of paramount importance in the daily management of insulin-treated diabetes. The use of short-term prediction algorithms of the subcutaneous (s.c.) glucose concentration may contribute significantly toward this direction. The literature suggests that, although the recent glucose profile is a prominent predictor of hypoglycemia, the overall patient's context greatly impacts its accurate estimation. The objective of this study is to evaluate the performance of a support vector for regression (SVR) s.c. glucose method on hypoglycemia prediction. MATERIALS AND METHODS: We extend our SVR model to predict separately the nocturnal events during sleep and the non-nocturnal (i.e., diurnal) ones over 30-min and 60-min horizons using information on recent glucose profile, meals, insulin intake, and physical activities for a hypoglycemic threshold of 70 mg/dL. We also introduce herein additional variables accounting for recurrent nocturnal hypoglycemia due to antecedent hypoglycemia, exercise, and sleep. SVR predictions are compared with those from two other machine learning techniques. RESULTS: The method is assessed on a dataset of 15 patients with type 1 diabetes under free-living conditions. Nocturnal hypoglycemic events are predicted with 94% sensitivity for both horizons and with time lags of 5.43 min and 4.57 min, respectively. As concerns the diurnal events, when physical activities are not considered, the sensitivity is 92% and 96% for a 30-min and 60-min horizon, respectively, with both time lags being less than 5 min. However, when such information is introduced, the diurnal sensitivity decreases by 8% and 3%, respectively. Both nocturnal and diurnal predictions show a high (>90%) precision. CONCLUSIONS: Results suggest that hypoglycemia prediction using SVR can be accurate and performs better in most diurnal and nocturnal cases compared with other techniques. It is advised that the problem of hypoglycemia prediction should be handled differently for nocturnal and diurnal periods as regards input variables and interpretation of results.

A predictive model of subcutaneous glucose concentration in type 1 diabetes based on Random Forests.

In this study, an individualized predictive model of the subcutaneous glucose concentration in type 1 diabetes is presented, which relies on the Random Forests regression technique. A multivariate dataset is utilized concerning the s.c. glucose profile, the plasma insulin concentration, the intestinal absorption of meal-derived glucose and the daily energy expenditure. In an attempt to capture daily rhythms in glucose metabolism, we also introduce a time feature in the predictive analysis. The dataset comes from the continuous multi-day recordings of 27 type 1 patients in free-living conditions. Evaluating the performance of the proposed method by 10-fold cross validation, an average RMSE of 6.60, 8.15, 9.25 and 10.83 mg/dl for 15, 30, 60 and 120 min prediction horizons, respectively, was attained.

Predictive modeling of glucose metabolism using free-living data of type 1 diabetic patients.

The aim of this study is to model the blood glucose metabolism of type 1 diabetic patients using free-living data. The proposed method considers the effect of diet, medication and exercise on blood glucose levels. Compartmental models are used to quantify the absorption of subcutaneously administered insulin, the absorption of glucose from the gut following a meal, as well as the effects of exercise on plasma glucose and insulin dynamics. Compartmental analysis is combined with a glucose predictive model which employs Support Vector machines for Regression to estimate the subcutaneous glucose concentrations. The model is trained and tested on real data recorded from two type 1 diabetic patients. The results obtained demonstrate the ability of the model to predict glucose response with a sufficient accuracy.