AI-based digital image dietary assessment methods compared to humans and ground truth: a systematic review.

OBJECTIVE: Human error estimating food intake is a major source of bias in nutrition research. Artificial intelligence (AI) methods may reduce bias, but the overall accuracy of AI estimates is unknown. This study was a systematic review of peer-reviewed journal articles comparing fully automated AI-based (e.g. deep learning) methods of dietary assessment from digital images to human assessors and ground truth (e.g. doubly labelled water). MATERIALS AND METHODS: Literature was searched through May 2023 in four electronic databases plus reference mining. Eligible articles reported AI estimated volume, energy, or nutrients. Independent investigators screened articles and extracted data. Potential sources of bias were documented in absence of an applicable risk of bias assessment tool. RESULTS: Database and hand searches identified 14,059 unique publications; fifty-two papers (studies) published from 2010 to 2023 were retained. For food detection and classification, 79% of papers used a convolutional neural network. Common ground truth sources were calculation using nutrient tables (51%) and weighed food (27%). Included papers varied widely in food image databases and results reported, so meta-analytic synthesis could not be conducted. Relative errors were extracted or calculated from 69% of papers. Average overall relative errors (AI vs. ground truth) ranged from 0.10% to 38.3% for calories and 0.09% to 33% for volume, suggesting similar performance. Ranges of relative error were lower when images had single/simple foods. CONCLUSIONS: Relative errors for volume and calorie estimations suggest that AI methods align with - and have the potential to exceed - accuracy of human estimations. However, variability in food image databases and results reported prevented meta-analytic synthesis. The field can advance by testing AI architectures on a limited number of large-scale food image and nutrition databases that the field determines to be adequate for training and testing and by reporting accuracy of at least absolute and relative error for volume or calorie estimations.

These results suggest that AI methods are in line with ­ and have the potential to exceed ­ accuracy of human estimations of nutrient content based on digital food images.Variability in food image databases used and results reported prevented meta-analytic synthesis.The field can advance by testing AI architectures on a limited number of large-scale food image and nutrition databases that the field determines to be accurate and by reporting accuracy of at least absolute and relative error for volume or calorie estimations.Overall, the tools currently available need more development before deployment as stand-alone dietary assessment methods in nutrition research or clinical practice.

An Individualized Machine Learning Approach for Human Body Weight Estimation Using Smart Shoe Insoles.

Rapid significant weight fluctuations can indicate severe health conditions such as edema due to congestive heart failure or severe dehydration that could require prompt intervention. Daily body weighing does not accurately represent the patient's body weight fluctuations occurring within a day. The patient's lack of compliance with tracking their weight measurements is also a predominant issue. Using shoe insole sensors embedded into footwear could achieve accurate real-time monitoring systems for estimating continuous body weight changes. Here, the machine learning models' predictive capabilities for continuous real-time weight estimation using the insole data are presented. The lack of availability of public datasets to feed these models is also addressed by introducing two novel datasets. The proposed framework is designed to adapt to the patient, considering several unique factors such as shoe type, posture, foot shape, and gait pattern. The proposed framework estimates the mean absolute percentage error of 0.61% and 0.74% and the MAE of 1.009 lbs. and 1.154 lbs. for the less controlled and more controlled experimental settings, respectively. This will help researchers utilize machine learning techniques for more accurate real-time continuous weight estimation using sensor data and enable more reliable aging-in-place monitoring and telehealth.

QCNN-H: Single-Image Dehazing Using Quaternion Neural Networks.

Single-image haze removal is challenging due to its ill-posed nature. The breadth of real-world scenarios makes it difficult to find an optimal dehazing approach that works well for various applications. This article addresses this challenge by utilizing a novel robust quaternion neural network architecture for single-image dehazing applications. The architecture's performance to dehaze images and its impact on real applications, such as object detection, is presented. The proposed single-image dehazing network is based on an encoder-decoder architecture capable of taking advantage of quaternion image representation without interrupting the quaternion dataflow end-to-end. We achieve this by introducing a novel quaternion pixel-wise loss function and quaternion instance normalization layer. The performance of the proposed QCNN-H quaternion framework is evaluated on two synthetic datasets, two real-world datasets, and one real-world task-oriented benchmark. Extensive experiments confirm that the QCNN-H outperforms state-of-the-art haze removal procedures in visual quality and quantitative metrics. Furthermore, the evaluation shows increased accuracy and recall of state-of-the-art object detection in hazy scenes using the presented QCNN-H method. This is the first time the quaternion convolutional network has been applied to the haze removal task.

Deep Perceptual Image Enhancement Network for Exposure Restoration.

Image restoration techniques process degraded images to highlight obscure details or enhance the scene with good contrast and vivid color for the best possible visibility. Poor illumination condition causes issues, such as high-level noise, unlikely color or texture distortions, nonuniform exposure, halo artifacts, and lack of sharpness in the images. This article presents a novel end-to-end trainable deep convolutional neural network called the deep perceptual image enhancement network (DPIENet) to address these challenges. The novel contributions of the proposed work are: 1) a framework to synthesize multiple exposures from a single image and utilizing the exposure variation to restore the image and 2) a loss function based on the approximation of the logarithmic response of the human eye. Extensive computer simulations on the benchmark MIT-Adobe FiveK and user studies performed using Google high dynamic range, DIV2K, and low light image datasets show that DPIENet has clear advantages over state-of-the-art techniques. It has the potential to be useful for many everyday applications such as modernizing traditional camera technologies that currently capture images/videos with under/overexposed regions due to their sensors limitations, to be used in consumer photography to help the users capture appealing images, or for a variety of intelligent systems, including automated driving and video surveillance applications.

Tufts Dental Database: A Multimodal Panoramic X-Ray Dataset for Benchmarking Diagnostic Systems.

The application of Artificial Intelligence in dental healthcare has a very promising role due to the abundance of imagery and non-imagery-based clinical data. Expert analysis of dental radiographs can provide crucial information for clinical diagnosis and treatment. In recent years, Convolutional Neural Networks have achieved the highest accuracy in various benchmarks, including analyzing dental X-ray images to improve clinical care quality. The Tufts Dental Database, a new X-ray panoramic radiography image dataset, has been presented in this paper. This dataset consists of 1000 panoramic dental radiography images with expert labeling of abnormalities and teeth. The classification of radiography images was performed based on five different levels: anatomical location, peripheral characteristics, radiodensity, effects on the surrounding structure, and the abnormality category. This first-of-its-kind multimodal dataset also includes the radiologist's expertise captured in the form of eye-tracking and think-aloud protocol. The contributions of this work are 1) publicly available dataset that can help researchers to incorporate human expertise into AI and achieve more robust and accurate abnormality detection; 2) a benchmark performance analysis for various state-of-the-art systems for dental radiograph image enhancement and image segmentation using deep learning; 3) an in-depth review of various panoramic dental image datasets, along with segmentation and detection systems. The release of this dataset aims to propel the development of AI-powered automated abnormality detection and classification in dental panoramic radiographs, enhance tooth segmentation algorithms, and the ability to distill the radiologist's expertise into AI.

Healthy Aging-Nutrition Matters: Start Early and Screen Often.

The amount of time spent in poor health at the end of life is increasing. This narrative review summarizes consistent evidence indicating that healthy dietary patterns and maintenance of a healthy weight in the years leading to old age are associated with broad prevention of all the archetypal diseases and impairments associated with aging including: noncommunicable diseases, sarcopenia, cognitive decline and dementia, osteoporosis, age-related macular degeneration, diabetic retinopathy, hearing loss, obstructive sleep apnea, urinary incontinence, and constipation. In addition, randomized clinical trials show that disease-specific nutrition interventions can attenuate progression-and in some cases effectively treat-many established aging-associated conditions. However, middle-aged and older adults are vulnerable to unhealthy dietary patterns, and typically consume diets with inadequate servings of healthy food groups and essential nutrients, along with an abundance of energy-dense but nutrient-weak foods that contribute to obesity. However, based on menu examples, diets that are nutrient-dense, plant-based, and with a moderately low glycemic load are better equipped to meet the nutritional needs of many older adults than current recommendations in US Dietary Guidelines. These summary findings indicate that healthy nutrition is more important for healthy aging than generally recognized. Improved public health messaging about nutrition and aging, combined with routine screening and medical referrals for age-related conditions that can be treated with a nutrition prescription, should form core components of a national nutrition roadmap to reduce the epidemic of unhealthy aging.

Automated Detection of COVID-19 Cases on Radiographs using Shape-Dependent Fibonacci-p Patterns.

The coronavirus (COVID-19) pandemic has been adversely affecting people's health globally. To diminish the effect of this widespread pandemic, it is essential to detect COVID-19 cases as quickly as possible. Chest radiographs are less expensive and are a widely available imaging modality for detecting chest pathology compared with CT images. They play a vital role in early prediction and developing treatment plans for suspected or confirmed COVID-19 chest infection patients. In this paper, a novel shape-dependent Fibonacci-p patterns-based feature descriptor using a machine learning approach is proposed. Computer simulations show that the presented system (1) increases the effectiveness of differentiating COVID-19, viral pneumonia, and normal conditions, (2) is effective on small datasets, and (3) has faster inference time compared to deep learning methods with comparable performance. Computer simulations are performed on two publicly available datasets; (a) the Kaggle dataset, and (b) the COVIDGR dataset. To assess the performance of the presented system, various evaluation parameters, such as accuracy, recall, specificity, precision, and f1-score are used. Nearly 100% differentiation between normal and COVID-19 radiographs is observed for the three-class classification scheme using the lung area-specific Kaggle radiographs. While Recall of 72.65 ± 6.83 and specificity of 77.72 ± 8.06 is observed for the COVIDGR dataset.

A Comprehensive Database for Benchmarking Imaging Systems.

Cross-modality face recognition is an emerging topic due to the wide-spread usage of different sensors in day-to-day life applications. The development of face recognition systems relies greatly on existing databases for evaluation and obtaining training examples for data-hungry machine learning algorithms. However, currently, there is no publicly available face database that includes more than two modalities for the same subject. In this work, we introduce the Tufts Face Database that includes images acquired in various modalities: photograph images, thermal images, near infrared images, a recorded video, a computerized facial sketch, and 3D images of each volunteer's face. An Institutional Research Board protocol was obtained and images were collected from students, staff, faculty, and their family members at Tufts University. The database includes over 10,000 images from 113 individuals from more than 15 different countries, various gender identities, ages, and ethnic backgrounds. The contributions of this work are: 1) Detailed description of the content and acquisition procedure for images in the Tufts Face Database; 2) The Tufts Face Database is publicly available to researchers worldwide, which will allow assessment and creation of more robust, consistent, and adaptable recognition algorithms; 3) A comprehensive, up-to-date review on face recognition systems and face datasets.

Initial Evaluation of Computer-Assisted Radiologic Assessment for Renal Mass Edge Detection as an Indication of Tumor Roughness to Predict Renal Cancer Subtypes.

OBJECTIVE: To develop software to assess the potential aggressiveness of an incidentally detected renal mass using images. METHODS: Thirty randomly selected patients who underwent nephrectomy for renal cell carcinoma (RCC) had their images independently reviewed by engineers. Tumor "Roughness" was based on image algorithm of tumor topographic features visualized on computed tomography (CT) scans. Univariant and multivariant statistical analyses are utilized for analysis. RESULTS: We investigated 30 subjects that underwent partial or radical nephrectomy. After excluding poor image-rendered images, 27 patients remained (benign cyst = 1, oncocytoma = 2, clear cell RCC = 15, papillary RCC = 7, and chromophobe RCC = 2). The mean roughness score for each mass is 1.18, 1.16, 1.27, 1.52, and 1.56 units, respectively (p < 0.004). Renal masses were correlated with tumor roughness (Pearson's, p=0.02). However, tumor size itself was larger in benign tumors (p=0.1). Linear regression analysis noted that the roughness score is the most influential on the model with all other demographics being equal including tumor size (p=0.003). CONCLUSION: Using basic CT imaging software, tumor topography ("roughness") can be quantified and correlated with histologies such as RCC subtype and could lead to determining aggressiveness of small renal masses.

Choosing the optimal spatial domain measure of enhancement for mammogram images.

Medical imaging systems often require image enhancement, such as improving the image contrast, to provide medical professionals with the best visual image quality. This helps in anomaly detection and diagnosis. Most enhancement algorithms are iterative processes that require many parameters be selected. Poor or nonoptimal parameter selection can have a negative effect on the enhancement process. In this paper, a quantitative metric for measuring the image quality is used to select the optimal operating parameters for the enhancement algorithms. A variety of measures evaluating the quality of an image enhancement will be presented along with each measure's basis for analysis, namely, on image content and image attributes. We also provide guidelines for systematically choosing the proper measure of image quality for medical images.

Nonreference medical image edge map measure.

Edge detection is a key step in medical image processing. It is widely used to extract features, perform segmentation, and further assist in diagnosis. A poor quality edge map can result in false alarms and misses in cancer detection algorithms. Therefore, it is necessary to have a reliable edge measure to assist in selecting the optimal edge map. Existing reference based edge measures require a ground truth edge map to evaluate the similarity between the generated edge map and the ground truth. However, the ground truth images are not available for medical images. Therefore, a nonreference edge measure is ideal for medical image processing applications. In this paper, a nonreference reconstruction based edge map evaluation (NREM) is proposed. The theoretical basis is that a good edge map keeps the structure and details of the original image thus would yield a good reconstructed image. The NREM is based on comparing the similarity between the reconstructed image with the original image using this concept. The edge measure is used for selecting the optimal edge detection algorithm and optimal parameters for the algorithm. Experimental results show that the quantitative evaluations given by the edge measure have good correlations with human visual analysis.

Non-linear direct multi-scale image enhancement based on the luminance and contrast masking characteristics of the human visual system.

Image enhancement is a crucial pre-processing step for various image processing applications and vision systems. Many enhancement algorithms have been proposed based on different sets of criteria. However, a direct multi-scale image enhancement algorithm capable of independently and/or simultaneously providing adequate contrast enhancement, tonal rendition, dynamic range compression, and accurate edge preservation in a controlled manner has yet to be produced. In this paper, a multi-scale image enhancement algorithm based on a new parametric contrast measure is presented. The parametric contrast measure incorporates not only the luminance masking characteristic, but also the contrast masking characteristic of the human visual system. The formulation of the contrast measure can be adapted for any multi-resolution decomposition scheme in order to yield new human visual system-inspired multi-scale transforms. In this article, it is exemplified using the Laplacian pyramid, discrete wavelet transform, stationary wavelet transform, and dual-tree complex wavelet transform. Consequently, the proposed enhancement procedure is developed. The advantages of the proposed method include: 1) the integration of both the luminance and contrast masking phenomena; 2) the extension of non-linear mapping schemes to human visual system inspired multi-scale contrast coefficients; 3) the extension of human visual system-based image enhancement approaches to the stationary and dual-tree complex wavelet transforms, and a direct means of; 4) adjusting overall brightness; and 5) achieving dynamic range compression for image enhancement within a direct multi-scale enhancement framework. Experimental results demonstrate the ability of the proposed algorithm to achieve simultaneous local and global enhancements.

Nonlinear unsharp masking for mammogram enhancement.

This paper introduces a new unsharp masking (UM) scheme, called nonlinear UM (NLUM), for mammogram enhancement. The NLUM offers users the flexibility 1) to embed different types of filters into the nonlinear filtering operator; 2) to choose different linear or nonlinear operations for the fusion processes that combines the enhanced filtered portion of the mammogram with the original mammogram; and 3) to allow the NLUM parameter selection to be performed manually or by using a quantitative enhancement measure to obtain the optimal enhancement parameters. We also introduce a new enhancement measure approach, called the second-derivative-like measure of enhancement, which is shown to have better performance than other measures in evaluating the visual quality of image enhancement. The comparison and evaluation of enhancement performance demonstrate that the NLUM can improve the disease diagnosis by enhancing the fine details in mammograms with no a priori knowledge of the image contents. The human-visual-system-based image decomposition is used for analysis and visualization of mammogram enhancement.

Parameterized logarithmic framework for image enhancement.

Image processing technologies such as image enhancement generally utilize linear arithmetic operations to manipulate images. Recently, Jourlin and Pinoli successfully used the logarithmic image processing (LIP) model for several applications of image processing such as image enhancement and segmentation. In this paper, we introduce a parameterized LIP (PLIP) model that spans both the linear arithmetic and LIP operations and all scenarios in between within a single unified model. We also introduce both frequency- and spatial-domain PLIP-based image enhancement methods, including the PLIP Lee's algorithm, PLIP bihistogram equalization, and the PLIP alpha rooting. Computer simulations and comparisons demonstrate that the new PLIP model allows the user to obtain improved enhancement performance by changing only the PLIP parameters, to yield better image fusion results by utilizing the PLIP addition or image multiplication, to represent a larger span of cases than the LIP and linear arithmetic cases by changing parameters, and to utilize and illustrate the logarithmic exponential operation for image fusion and enhancement.

Boolean derivatives with application to edge detection for imaging systems.

This paper introduces a new concept of Boolean derivatives as a fusion of partial derivatives of Boolean functions (PDBFs). Three efficient algorithms for the calculation of PDBFs are presented. It is shown that Boolean function derivatives are useful for the application of identifying the location of edge pixels in binary images. The same concept is extended to the development of a new edge detection algorithm for grayscale images, which yields competitive results, compared with those of traditional methods. Furthermore, a new measure is introduced to automatically determine the parameter values used in the thresholding portion of the binarization procedure. Through computer simulations, demonstrations of Boolean derivatives and the effectiveness of the presented edge detection algorithm, compared with traditional edge detection algorithms, are shown using several synthetic and natural test images. In order to make quantitative comparisons, two quantitative measures are used: one based on the recovery of the original image from the output edge map and the Pratt's figure of merit.

Mammogram enhancement using alpha weighted quadratic filter.

Mammograms are widely used to detect breast cancer in women. The quality of the image may suffer from poor resolution or low contrast due to the limitations of the X-ray hardware systems. Image enhancement is a powerful tool to improve the visual quality of mammograms. This paper introduces a new powerful nonlinear filter called the alpha weighted quadratic filter for mammogram enhancement. The user has the flexibility to design the filter by selecting all of the parameters manually or using an existing quantitative measure to select the optimal enhancement parameters. Computer simulations show that excellent enhancement results can be obtained with no apriori knowledge of the mammogram contents. The filter can also be used for automatic segmentation.

A lossless encryption method for medical images using edge maps.

Image encryption is an effective approach for providing security and privacy protection for medical images. This paper introduces a new lossless approach, called EdgeCrypt, to encrypt medical images using the information contained within an edge map. The algorithm can fully protect the selected objects/regions within medical images or the entire medical images. It can also encrypt other types of images such as grayscale images or color images. The algorithm can be used for privacy protection in the real-time medical applications such as wireless medical networking and mobile medical services.

Human visual system-based image enhancement and logarithmic contrast measure.

Varying scene illumination poses many challenging problems for machine vision systems. One such issue is developing global enhancement methods that work effectively across the varying illumination. In this paper, we introduce two novel image enhancement algorithms: edge-preserving contrast enhancement, which is able to better preserve edge details while enhancing contrast in images with varying illumination, and a novel multihistogram equalization method which utilizes the human visual system (HVS) to segment the image, allowing a fast and efficient correction of nonuniform illumination. We then extend this HVS-based multihistogram equalization approach to create a general enhancement method that can utilize any combination of enhancement algorithms for an improved performance. Additionally, we propose new quantitative measures of image enhancement, called the logarithmic Michelson contrast measure (AME) and the logarithmic AME by entropy. Many image enhancement methods require selection of operating parameters, which are typically chosen using subjective methods, but these new measures allow for automated selection. We present experimental results for these methods and make a comparison against other leading algorithms.

Transform coefficient histogram-based image enhancement algorithms using contrast entropy.

Many applications of histograms for the purposes of image processing are well known. However, applying this process to the transform domain by way of a transform coefficient histogram has not yet been fully explored. This paper proposes three methods of image enhancement: a) logarithmic transform histogram matching, b) logarithmic transform histogram shifting, and c) logarithmic transform histogram shaping using Gaussian distributions. They are based on the properties of the logarithmic transform domain histogram and histogram equalization. The presented algorithms use the fact that the relationship between stimulus and perception is logarithmic and afford a marriage between enhancement qualities and computational efficiency. A human visual system-based quantitative measurement of image contrast improvement is also defined. This helps choose the best parameters and transform for each enhancement. A number of experimental results are presented to illustrate the performance of the proposed algorithms.