Can visceral adipose tissue and skeletal muscle predict recurrence of newly diagnosed Crohn's disease in different treatments.

BACKGROUND AND AIMS: It is crucial to manage the recurrence of Crohn's disease (CD). This study is aimed to explore whether visceral adipose tissue (VAT) and skeletal muscle (SM) are associated with the recurrence of CD upon different treatments. METHODS: All patients with a definite diagnosis of CD were retrospectively divided into three groups according to distinct treatment regimens: 5-amino salicylic acid group (Group A), steroids + azathioprine (Group B) and biologics (Group C). The pretreatment computerized tomography (CT) images and clinical data were collected. The VAT area, mesenteric fat index (MFI), the ratio of VAT area to fat mass (VAT area/FM) were assessed. The primary end point was the recurrence of CD within 1 year of follow-up. RESULTS: A total of 171 CD patients were enrolled, including 57 (33.33%) patients in Group A, 70 (40.94%) patients in Group B and 44 (25.73%) patients in Group C. Patients with 1-year recurrence had higher MFI (P = 0.011) and VAT area/FM (P = 0.000). ROC curve demonstrated that patients with the ratio of VAT area/FM and MFI higher than 0.578 and 1.394 tended to have recurrence with the AUC of 0.707 and 0.709. Similar results could be observed in Group A & B but not in Group C. CONCLUSIONS: High VAT area/FM and MFI are related to recurrence within 1 year for newly diagnosed CD patients treated by 5-amino salicylic or azathioprine + steroids rather than biologics. We could not observe any radiological data associated with the recurrence of CD patients under biological treatment.

Necroptosis in inflammatory bowel disease: A potential effective target. / ç‚Žç—‡æ€§è‚ ç— ä¸­çš„åæ­»æ€§å‡‹äº¡ï¼šä¸€ä¸ªæ½œåœ¨çš„æ²»ç–—é¶ç‚¹ï¼ˆè‹±æ–‡ï¼‰.

The morbidity of inflammatory bowel diseases (IBD) is rising rapidly but no curative therapies to prevent its recurrence. Cell death is crucial to maintaining homeostasis. Necroptosis is a newly identified programmed cell death and its roles played in IBD need to be explored. Necroptosis is mediated by receptor interacting protein kinase 1 (RIPK1), RIPK3, and mixed lineage kinase domain-like protein (MLKL), which resulted in cell swelling, plasma membrane rupture, intracellular content leaking, and eventually cell death as well as the promotion of inflammation. Studies have found that inhibiting necroptosis alleviated IBD in animal models and IBD patients with an increased level of necroptosis in inflammatory tissues, indicating that necroptosis is related to the pathogenesis of IBD. However, due to the complexity in regulation of necroptosis and the involvement of multiple functions of relevant signaling molecules, the specific mechanism remains elusive. Necroptosis may play a vital regulatory role in the pathogenesis of IBD, which provides a new idea and method for further exploring the therapeutic target of IBD.

Development and validation of machine learning models to predict the need for haemostatic therapy in acute upper gastrointestinal bleeding.

Background: Acute upper gastrointestinal bleeding (AUGIB) is a major cause of morbidity and mortality. This presentation however is not universally high risk as only 20-30% of bleeds require urgent haemostatic therapy. Nevertheless, the current standard of care is for all patients admitted to an inpatient bed to undergo endoscopy within 24 h for risk stratification which is invasive, costly and difficult to achieve in routine clinical practice. Objectives: To develop novel non-endoscopic machine learning models for AUGIB to predict the need for haemostatic therapy by endoscopic, radiological or surgical intervention. Design: A retrospective cohort study. Method: We analysed data from patients admitted with AUGIB to hospitals from 2015 to 2020 (n = 970). Machine learning models were internally validated to predict the need for haemostatic therapy. The performance of the models was compared to the Glasgow-Blatchford score (GBS) using the area under receiver operating characteristic (AUROC) curves. Results: The random forest classifier [AUROC 0.84 (0.80-0.87)] had the best performance and was superior to the GBS [AUROC 0.75 (0.72-0.78), p < 0.001] in predicting the need for haemostatic therapy in patients with AUGIB. A GBS cut-off of ⩾12 was associated with an accuracy, sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) of 0.74, 0.49, 0.81, 0.41 and 0.85, respectively. The Random Forrest model performed better with an accuracy, sensitivity, specificity, PPV and NPV of 0.82, 0.54, 0.90, 0.60 and 0.88, respectively. Conclusion: We developed and validated a machine learning algorithm with high accuracy and specificity in predicting the need for haemostatic therapy in AUGIB. This could be used to risk stratify high-risk patients to urgent endoscopy.

CauDR: A causality-inspired domain generalization framework for fundus-based diabetic retinopathy grading.

Diabetic retinopathy (DR) is the most common diabetic complication, which usually leads to retinal damage, vision loss, and even blindness. A computer-aided DR grading system has a significant impact on helping ophthalmologists with rapid screening and diagnosis. Recent advances in fundus photography have precipitated the development of novel retinal imaging cameras and their subsequent implementation in clinical practice. However, most deep learning-based algorithms for DR grading demonstrate limited generalization across domains. This inferior performance stems from variance in imaging protocols and devices inducing domain shifts. We posit that declining model performance between domains arises from learning spurious correlations in the data. Incorporating do-operations from causality analysis into model architectures may mitigate this issue and improve generalizability. Specifically, a novel universal structural causal model (SCM) was proposed to analyze spurious correlations in fundus imaging. Building on this, a causality-inspired diabetic retinopathy grading framework named CauDR was developed to eliminate spurious correlations and achieve more generalizable DR diagnostics. Furthermore, existing datasets were reorganized into 4DR benchmark for DG scenario. Results demonstrate the effectiveness and the state-of-the-art (SOTA) performance of CauDR. Diabetic retinopathy (DR) is the most common diabetic complication, which usually leads to retinal damage, vision loss, and even blindness. A computer-aided DR grading system has a significant impact on helping ophthalmologists with rapid screening and diagnosis. Recent advances in fundus photography have precipitated the development of novel retinal imaging cameras and their subsequent implementation in clinical practice. However, most deep learning-based algorithms for DR grading demonstrate limited generalization across domains. This inferior performance stems from variance in imaging protocols and devices inducing domain shifts. We posit that declining model performance between domains arises from learning spurious correlations in the data. Incorporating do-operations from causality analysis into model architectures may mitigate this issue and improve generalizability. Specifically, a novel universal structural causal model (SCM) was proposed to analyze spurious correlations in fundus imaging. Building on this, a causality-inspired diabetic retinopathy grading framework named CauDR was developed to eliminate spurious correlations and achieve more generalizable DR diagnostics. Furthermore, existing datasets were reorganized into 4DR benchmark for DG scenario. Results demonstrate the effectiveness and the state-of-the-art (SOTA) performance of CauDR.

Clinical value of endoscopic ultrasound sound speed in differential diagnosis of pancreatic solid lesion and prognosis of pancreatic cancer.

BACKGROUND: Differential diagnosis of pancreatic solid lesion (PSL) and prognosis of pancreatic cancer (PC) is a clinical challenge. We aimed to explore the differential diagnostic value of sound speed (SS) obtained from endoscopic ultrasonography (EUS) in PSL and the prognostic value of SS in PC. METHODS: Patients with PSL in The Third Xiangya Hospital of Central South University from March 2019 to October 2019 were prospectively enrolled, who obtained SS from PSL. Patients were divided into the PC group and the pancreatic benign lesion (PBL) group. SS1 is the SS of lesions and SS2 is the SS of normal tissues adjacent to lesions. Ratio1 is equal to SS1 divided by SS2 of PSL (ratio1 = SS1/SS2). RESULTS: Eighty patients were enrolled (24 PBL patients, 56 PC patients). SS1 and ratio1 in PC group were higher compared with PBL group (SS1:1568.00 vs. 1550.00, Z = -2.066, p = 0.039; ratio1: 1.0110 vs. 1.0051, Z = -3.391, p = 0.001). The SS1 in PC (Z = -6.503, p < 0.001) was higher compared to SS2. In the nonsurgical group of PC, low ratio1 predicted high overall survival (OS) (7.000 months vs. 4.000 months; p = 0.039). In the surgical group of PC, low SS1 was associated with low median OS (4.000 months vs. 12.000 months; p = 0.033). CONCLUSIONS: SS plays a vital role in distinguishing between PBL and PC. Higher SS1 and ratio1 obtained by EUS are more related to PC than PBL. In PC patients, high SS1 may predict pancreatic lesions. In the nonsurgical group of PC, low ratio1 may predict high OS. However, in the surgical group of PC, low SS1 may predict low OS.

Machine Learning and Optical-Coherence-Tomography-Derived Radiomics Analysis to Predict the Postoperative Anatomical Outcome of Full-Thickness Macular Hole.

Full-thickness macular hole (FTMH) leads to central vision loss. It is essential to identify patients with FTMH at high risk of postoperative failure accurately to achieve anatomical closure. This study aimed to construct a predictive model for the anatomical outcome of FTMH after surgery. A retrospective study was performed, analyzing 200 eyes from 197 patients diagnosed with FTMH. Radiomics features were extracted from optical coherence tomography (OCT) images. Logistic regression, support vector machine (SVM), and backpropagation neural network (BPNN) classifiers were trained and evaluated. Decision curve analysis and survival analysis were performed to assess the clinical implications. Sensitivity, specificity, F1 score, and area under the receiver operating characteristic curve (AUC) were calculated to assess the model effectiveness. In the training set, the AUC values were 0.998, 0.988, and 0.995, respectively. In the test set, the AUC values were 0.941, 0.943, and 0.968, respectively. The OCT-omics scores were significantly higher in the "Open" group than in the "Closed" group and were positively correlated with the minimum diameter (MIN) and base diameter (BASE) of FTMH. Therefore, in this study, we developed a model with robust discriminative ability to predict the postoperative anatomical outcome of FTMH.

Localization and recognition of human action in 3D using transformers.

Understanding a person's behavior from their 3D motion sequence is a fundamental problem in computer vision with many applications. An important component of this problem is 3D action localization, which involves recognizing what actions a person is performing, and when the actions occur in the sequence. To promote the progress of the 3D action localization community, we introduce a new, challenging, and more complex benchmark dataset, BABEL-TAL (BT), for 3D action localization. Important baselines and evaluating metrics, as well as human evaluations, are carefully established on this benchmark. We also propose a strong baseline model, i.e., Localizing Actions with Transformers (LocATe), that jointly localizes and recognizes actions in a 3D sequence. The proposed LocATe shows superior performance on BABEL-TAL as well as on the large-scale PKU-MMD dataset, achieving state-of-the-art performance by using only 10% of the labeled training data. Our research could advance the development of more accurate and efficient systems for human behavior analysis, with potential applications in areas such as human-computer interaction and healthcare.

Dietary Assessment with Multimodal ChatGPT: A Systematic Analysis.

Conventional approaches to dietary assessment are primarily grounded in self-reporting methods or structured interviews conducted under the supervision of dietitians. These methods, however, are often subjective, potentially inaccurate, and time-intensive. Although artificial intelligence (AI)-based solutions have been devised to automate the dietary assessment process, prior AI methodologies tackle dietary assessment in a fragmented landscape (e.g., merely recognizing food types or estimating portion size), and encounter challenges in their ability to generalize across a diverse range of food categories, dietary behaviors, and cultural contexts. Recently, the emergence of multimodal foundation models, such as GPT-4V, has exhibited transformative potential across a wide range of tasks (e.g., scene understanding and image captioning) in various research domains. These models have demonstrated remarkable generalist intelligence and accuracy, owing to their large-scale pre-training on broad datasets and substantially scaled model size. In this study, we explore the application of GPT-4V powering multimodal ChatGPT for dietary assessment, along with prompt engineering and passive monitoring techniques. We evaluated the proposed pipeline using a self-collected, semi free-living dietary intake dataset comprising 16 real-life eating episodes, captured through wearable cameras. Our findings reveal that GPT-4V excels in food detection under challenging conditions without any fine-tuning or adaptation using food-specific datasets. By guiding the model with specific language prompts (e.g., African cuisine), it shifts from recognizing common staples like rice and bread to accurately identifying regional dishes like banku and ugali. Another GPT-4V's standout feature is its contextual awareness. GPT-4V can leverage surrounding objects as scale references to deduce the portion sizes of food items, further facilitating the process of dietary assessment.

Egocentric Image Captioning for Privacy-Preserved Passive Dietary Intake Monitoring.

Camera-based passive dietary intake monitoring is able to continuously capture the eating episodes of a subject, recording rich visual information, such as the type and volume of food being consumed, as well as the eating behaviors of the subject. However, there currently is no method that is able to incorporate these visual clues and provide a comprehensive context of dietary intake from passive recording (e.g., is the subject sharing food with others, what food the subject is eating, and how much food is left in the bowl). On the other hand, privacy is a major concern while egocentric wearable cameras are used for capturing. In this article, we propose a privacy-preserved secure solution (i.e., egocentric image captioning) for dietary assessment with passive monitoring, which unifies food recognition, volume estimation, and scene understanding. By converting images into rich text descriptions, nutritionists can assess individual dietary intake based on the captions instead of the original images, reducing the risk of privacy leakage from images. To this end, an egocentric dietary image captioning dataset has been built, which consists of in-the-wild images captured by head-worn and chest-worn cameras in field studies in Ghana. A novel transformer-based architecture is designed to caption egocentric dietary images. Comprehensive experiments have been conducted to evaluate the effectiveness and to justify the design of the proposed architecture for egocentric dietary image captioning. To the best of our knowledge, this is the first work that applies image captioning for dietary intake assessment in real-life settings.

Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation.

Medical image analysis plays an important role in clinical diagnosis. In this paper, we examine the recent Segment Anything Model (SAM) on medical images, and report both quantitative and qualitative zero-shot segmentation results on nine medical image segmentation benchmarks, covering various imaging modalities, such as optical coherence tomography (OCT), magnetic resonance imaging (MRI), and computed tomography (CT), as well as different applications including dermatology, ophthalmology, and radiology. Those benchmarks are representative and commonly used in model development. Our experimental results indicate that while SAM presents remarkable segmentation performance on images from the general domain, its zero-shot segmentation ability remains restricted for out-of-distribution images, e.g., medical images. In addition, SAM exhibits inconsistent zero-shot segmentation performance across different unseen medical domains. For certain structured targets, e.g., blood vessels, the zero-shot segmentation of SAM completely failed. In contrast, a simple fine-tuning of it with a small amount of data could lead to remarkable improvement of the segmentation quality, showing the great potential and feasibility of using fine-tuned SAM to achieve accurate medical image segmentation for a precision diagnostics. Our study indicates the versatility of generalist vision foundation models on medical imaging, and their great potential to achieve desired performance through fine-turning and eventually address the challenges associated with accessing large and diverse medical datasets in support of clinical diagnostics.

Large AI Models in Health Informatics: Applications, Challenges, and the Future.

Large AI models, or foundation models, are models recently emerging with massive scales both parameter-wise and data-wise, the magnitudes of which can reach beyond billions. Once pretrained, large AI models demonstrate impressive performance in various downstream tasks. A prime example is ChatGPT, whose capability has compelled people's imagination about the far-reaching influence that large AI models can have and their potential to transform different domains of our lives. In health informatics, the advent of large AI models has brought new paradigms for the design of methodologies. The scale of multi-modal data in the biomedical and health domain has been ever-expanding especially since the community embraced the era of deep learning, which provides the ground to develop, validate, and advance large AI models for breakthroughs in health-related areas. This article presents a comprehensive review of large AI models, from background to their applications. We identify seven key sectors in which large AI models are applicable and might have substantial influence, including: 1) bioinformatics; 2) medical diagnosis; 3) medical imaging; 4) medical informatics; 5) medical education; 6) public health; and 7) medical robotics. We examine their challenges, followed by a critical discussion about potential future directions and pitfalls of large AI models in transforming the field of health informatics.

Evaluation of Acceptability, Functionality, and Validity of a Passive Image-Based Dietary Intake Assessment Method in Adults and Children of Ghanaian and Kenyan Origin Living in London, UK.

BACKGROUND: Accurate estimation of dietary intake is challenging. However, whilst some progress has been made in high-income countries, low- and middle-income countries (LMICs) remain behind, contributing to critical nutritional data gaps. This study aimed to validate an objective, passive image-based dietary intake assessment method against weighed food records in London, UK, for onward deployment to LMICs. METHODS: Wearable camera devices were used to capture food intake on eating occasions in 18 adults and 17 children of Ghanaian and Kenyan origin living in London. Participants were provided pre-weighed meals of Ghanaian and Kenyan cuisine and camera devices to automatically capture images of the eating occasions. Food images were assessed for portion size, energy, nutrient intake, and the relative validity of the method compared to the weighed food records. RESULTS: The Pearson and Intraclass correlation coefficients of estimates of intakes of food, energy, and 19 nutrients ranged from 0.60 to 0.95 and 0.67 to 0.90, respectively. Bland-Altman analysis showed good agreement between the image-based method and the weighed food record. Under-estimation of dietary intake by the image-based method ranged from 4 to 23%. CONCLUSIONS: Passive food image capture and analysis provides an objective assessment of dietary intake comparable to weighed food records.

Antibiotics-induced depression in mice via the microbiota-gut-brain axis.

BACKGROUND: Intestinal dysbacteriosis is associated with depression. This study aimed to establish an antibiotics-induced depression mouse model and explore the mechanism of antibiotic-induced depression. METHODS: C57BL/6 J mice were treated with antibiotics to prepare the antibiotic-induced depression mouse model. Behavioral tests and depression-related bio-markers were examined. To understand the abundance of different bacteria in intestinal flora and screen out the predominant bacterial species, metagenomic analysis of feces was carried out. Finally, we detected the expression of NF-κB-p65 and p-NF-κB-p65 in PFC and the hippocampus using Western blot. RESULTS: Mixtures A and B caused depression-like behavior in mice. Norepinephrine, 5-hydroxytryptamine, and brain-derived neurotrophic factor in hippocampus and PFC of antibiotic-induced depression mice significantly decreased. The serum adrenocorticotropic hormone and corticosterone concentrations increased. The abundance values of Bacteroides thetaiotaomicron, Klebsiella oxytoca, and Klebsiella aerogenes in antibiotic-induced depression mice significantly increased, and the characteristic KO genes and metabolic pathways in antibiotic-induced depression mice were significantly different with in CUMS depression mice (the positive control) and normal mice. The relative levels of p-NF-κB-p65 in antibiotics-induced depression mice were significantly higher than in normal mice. LIMITATIONS: How dysbacteriosis induces inflammation in the central nervous system is unclear. CONCLUSIONS: Specific antibiotic mixture can cause depression-like behavior and changes of depression-related bio-markers in mice. The antibiotic-induced depression mice display changes in the species and metabolism of intestinal bacterial flora. The activation of NF-κB inflammatory signaling pathway in the central nervous system may act as one of the mechanisms in the development of antibiotic-induced depression.

Endoscopists' Acceptance on the Implementation of Artificial Intelligence in Gastrointestinal Endoscopy: Development and Case Analysis of a Scale.

Background: The purpose of this paper is to develop and validate a standardized endoscopist acceptance scale for the implementation of artificial intelligence (AI) in gastrointestinal endoscopy. Methods: After investigating endoscopists who have previously used AI and consulting with AI experts, we developed a provisional scale to measure the acceptance of AI as used in gastrointestinal endoscopy that was then distributed to a sample of endoscopists who have used AI. After analyzing the feedback data collected on the provisional scale, we developed a new formal scale with four factors. Cronbach's alpha, confirmatory factor analysis (CFA), content validity, and related validity were conducted to test the reliability and validity of the formal scale. We also constructed a receiver operating characteristic (ROC) curve in order to determine the scale's ability to distinguish higher acceptance and satisfaction. Results: A total of 210 valid formal scale data points were collected. The overall Cronbach's alpha was 0.904. All the factor loadings were >0.50, of which the highest factor loading was 0.86 and the lowest was 0.54 (AVE = 0.580, CR = 0.953). The correlation coefficient between the total score of the scale and the satisfaction score was 0.876, and the area under the ROC curve was 0.949 ± 0.031. Endoscopists with a score higher than 50 tend to be accepting and satisfied with AI. Conclusion: This study yielded a viable questionnaire to measure the acceptance among endoscopists of the implementation of AI in gastroenterology.

Identification of lncRNA expression profiles and analysis of ceRNA in the hippocampus of perinatal glyphosate-exposed mice.

OBJECTIVE: In order to understand the role of long noncoding RNAs (lncRNAs) played in the mechanisms of glyphosate neurotoxicity in neuronal development. METHODS: Perinatal glyphosate exposure (PGE) mouse model was constructed, and a lncRNA microarray was used to study the lncRNA expression changes in the hippocampus tissue of perinatal glyphosate exposure mice. Then we used GO (Gene Ontology) and KEGG (Kyoto Encyclopedia of Genes and Genomes) databases to analyze the function of the differentially expressed mRNAs and lncRNAs. RESULTS: LncRNA microarray analysis revealed that 1759 lncRNAs and 759 mRNAs were differentially expressed in the perinatal glyphosate exposure (PGE) mice group (G group) compared with the normal control mice group (C group). The functions of the DEmRNAs are involved in the cellular response to hormone stimulus. The ceRNA analysis showed that some interaction networks existed, including (ENSMUST00000137546, ENSMUST00000160950)/(miR-34a-3p, miR-130a-3p)/(Il12b, Irf1). Further analysis of the target mRNAs of miRNAs indicated that the possible functions involved the neuroactive ligand-receptor interaction and calcium signaling pathway, which are involved in perinatal glyphosate exposure-induced neurotoxicity. CONCLUSION: The aberrant expression of lncRNAs is related to the perinatal glyphosate-exposed neurotoxicity. These lncRNAs affect the target gene expression level, might by regulating neuroactive ligand-receptor interactions. The (ENSMUST00000137546, ENSMUST00000160950)/ (miRNA-34a-5p, miR-130a-3p) / mRNAs (e.g., Il12b, Irf1) interaction network may functions in perinatal glyphosate exposure-induced neurotoxicity.

Comparison of Efficacy of Anti-interleukin-17 in the Treatment of Psoriasis Between Caucasians and Asians: A Systematic Review and Meta-Analysis.

BACKGROUND: Interleukin-17 (IL-17) monoclonal antibody drugs have been increasingly significant in the treatment of psoriasis, but it is not clear whether the efficacy is equivalent across ethnicities. OBJECTIVE: To explore the differences of short-term efficacy of IL-17 inhibitors between Caucasians and Asians. METHODS: The pooled log risk ratio (logRR) between the groups was estimated. The meta-regression analysis on the logRR was performed, with the proportion of Caucasian patients as the covariate. The subgroup analysis was performed by specific IL-17 inhibitors. RESULTS: Of the 1,569 potentially relevant studies, sixteen randomized controlled trials (RCTs) were included. For the Psoriasis Area and Severity Index 75 (PASI 75) response at week 12, the pooled logRR of the Asian group and the Caucasian group was 2.81 (95% CI: 2.27-3.35, p < 0.001) and 2.93 (95% CI: 2.71-3.16, p < 0.001), respectively, indicating no significant difference of efficacy between Asians and Caucasians. The meta-regression analysis did not show an association of the proportion of Caucasians with the effect size (ß = 0.3203, p = 0.334). In the subgroup analysis, the comparison results of secukinumab were consistent with the main analysis. LIMITATIONS: Only the short-term efficacy was explored. The data from Asian countries were limited. CONCLUSIONS: The short-term efficacy of IL-17 inhibitors in the treatment of psoriasis has no significant difference between Caucasians and Asians. SYSTEMATIC REVIEW REGISTRATION: PROSPERO, identifier CRD42020201994, https://www.crd.york.ac.uk/prospero/.

Counting Bites and Recognizing Consumed Food from Videos for Passive Dietary Monitoring.

Assessing dietary intake in epidemiological studies are predominantly based on self-reports, which are subjective, inefficient, and also prone to error. Technological approaches are therefore emerging to provide objective dietary assessments. Using only egocentric dietary intake videos, this work aims to provide accurate estimation on individual dietary intake through recognizing consumed food items and counting the number of bites taken. This is different from previous studies that rely on inertial sensing to count bites, and also previous studies that only recognize visible food items but not consumed ones. As a subject may not consume all food items visible in a meal, recognizing those consumed food items is more valuable. A new dataset that has 1,022 dietary intake video clips was constructed to validate our concept of bite counting and consumed food item recognition from egocentric videos. 12 subjects participated and 52 meals were captured. A total of 66 unique food items, including food ingredients and drinks, were labelled in the dataset along with a total of 2,039 labelled bites. Deep neural networks were used to perform bite counting and food item recognition in an end-to-end manner. Experiments have shown that counting bites directly from video clips can reach 74.15% top-1 accuracy (classifying between 0-4 bites in 20-second clips), and a MSE value of 0.312 (when using regression). Our experiments on video-based food recognition also show that recognizing consumed food items is indeed harder than recognizing visible ones, with a drop of 25% in F1 score.

Image-Based Food Classification and Volume Estimation for Dietary Assessment: A Review.

A daily dietary assessment method named 24-hour dietary recall has commonly been used in nutritional epidemiology studies to capture detailed information of the food eaten by the participants to help understand their dietary behaviour. However, in this self-reporting technique, the food types and the portion size reported highly depends on users' subjective judgement which may lead to a biased and inaccurate dietary analysis result. As a result, a variety of visual-based dietary assessment approaches have been proposed recently. While these methods show promises in tackling issues in nutritional epidemiology studies, several challenges and forthcoming opportunities, as detailed in this study, still exist. This study provides an overview of computing algorithms, mathematical models and methodologies used in the field of image-based dietary assessment. It also provides a comprehensive comparison of the state of the art approaches in food recognition and volume/weight estimation in terms of their processing speed, model accuracy, efficiency and constraints. It will be followed by a discussion on deep learning method and its efficacy in dietary assessment. After a comprehensive exploration, we found that integrated dietary assessment systems combining with different approaches could be the potential solution to tackling the challenges in accurate dietary intake assessment.

Development and Validation of an Objective, Passive Dietary Assessment Method for Estimating Food and Nutrient Intake in Households in Low- and Middle-Income Countries: A Study Protocol.

Malnutrition is a major concern in low- and middle-income countries (LMIC), but the full extent of nutritional deficiencies remains unknown largely due to lack of accurate assessment methods. This study seeks to develop and validate an objective, passive method of estimating food and nutrient intake in households in Ghana and Uganda. Household members (including under-5s and adolescents) are assigned a wearable camera device to capture images of their food intake during waking hours. Using custom software, images captured are then used to estimate an individual's food and nutrient (i.e., protein, fat, carbohydrate, energy, and micronutrients) intake. Passive food image capture and assessment provides an objective measure of food and nutrient intake in real time, minimizing some of the limitations associated with self-reported dietary intake methods. Its use in LMIC could potentially increase the understanding of a population's nutritional status, and the contribution of household food intake to the malnutrition burden. This project is registered at clinicaltrials.gov (NCT03723460).