Editorial for the Special Issue "Sensing-Based Biomedical Communication and Intelligent Identification for Healthcare".

The integration of sensor technology in healthcare has become crucial for disease diagnosis and treatment [...].

Feasibility of the automatic ingestion monitor (AIM-2) for infant feeding assessment: a pilot study among breast-feeding mothers from Ghana.

OBJECTIVE: Passive, wearable sensors can be used to obtain objective information in infant feeding, but their use has not been tested. Our objective was to compare assessment of infant feeding (frequency, duration and cues) by self-report and that of the Automatic Ingestion Monitor-2 (AIM-2). DESIGN: A cross-sectional pilot study was conducted in Ghana. Mothers wore the AIM-2 on eyeglasses for 1 d during waking hours to assess infant feeding using images automatically captured by the device every 15 s. Feasibility was assessed using compliance with wearing the device. Infant feeding practices collected by the AIM-2 images were annotated by a trained evaluator and compared with maternal self-report via interviewer-administered questionnaire. SETTING: Rural and urban communities in Ghana. PARTICIPANTS: Participants were thirty eight (eighteen rural and twenty urban) breast-feeding mothers of infants (child age ≤7 months). RESULTS: Twenty-five mothers reported exclusive breast-feeding, which was common among those < 30 years of age (n 15, 60 %) and those residing in urban communities (n 14, 70 %). Compliance with wearing the AIM-2 was high (83 % of wake-time), suggesting low user burden. Maternal report differed from the AIM-2 data, such that mothers reported higher mean breast-feeding frequency (eleven v. eight times, P = 0·041) and duration (18·5 v. 10 min, P = 0·007) during waking hours. CONCLUSION: The AIM-2 was a feasible tool for the assessment of infant feeding among mothers in Ghana as a passive, objective method and identified overestimation of self-reported breast-feeding frequency and duration. Future studies using the AIM-2 are warranted to determine validity on a larger scale.

Improved Wearable Devices for Dietary Assessment Using a New Camera System.

An unhealthy diet is strongly linked to obesity and numerous chronic diseases. Currently, over two-thirds of American adults are overweight or obese. Although dietary assessment helps people improve nutrition and lifestyle, traditional methods for dietary assessment depend on self-report, which is inaccurate and often biased. In recent years, as electronics, information, and artificial intelligence (AI) technologies advanced rapidly, image-based objective dietary assessment using wearable electronic devices has become a powerful approach. However, research in this field has been focused on the developments of advanced algorithms to process image data. Few reports exist on the study of device hardware for the particular purpose of dietary assessment. In this work, we demonstrate that, with the current hardware design, there is a considerable risk of missing important dietary data owing to the common use of rectangular image screen and fixed camera orientation. We then present two designs of a new camera system to reduce data loss by generating circular images using rectangular image sensor chips. We also present a mechanical design that allows the camera orientation to be adjusted, adapting to differences among device wearers, such as gender, body height, and so on. Finally, we discuss the pros and cons of rectangular versus circular images with respect to information preservation and data processing using AI algorithms.

A Novel Approach to Dining Bowl Reconstruction for Image-Based Food Volume Estimation.

Knowing the amounts of energy and nutrients in an individual's diet is important for maintaining health and preventing chronic diseases. As electronic and AI technologies advance rapidly, dietary assessment can now be performed using food images obtained from a smartphone or a wearable device. One of the challenges in this approach is to computationally measure the volume of food in a bowl from an image. This problem has not been studied systematically despite the bowl being the most utilized food container in many parts of the world, especially in Asia and Africa. In this paper, we present a new method to measure the size and shape of a bowl by adhering a paper ruler centrally across the bottom and sides of the bowl and then taking an image. When observed from the image, the distortions in the width of the paper ruler and the spacings between ruler markers completely encode the size and shape of the bowl. A computational algorithm is developed to reconstruct the three-dimensional bowl interior using the observed distortions. Our experiments using nine bowls, colored liquids, and amorphous foods demonstrate high accuracy of our method for food volume estimation involving round bowls as containers. A total of 228 images of amorphous foods were also used in a comparative experiment between our algorithm and an independent human estimator. The results showed that our algorithm overperformed the human estimator who utilized different types of reference information and two estimation methods, including direct volume estimation and indirect estimation through the fullness of the bowl.

An automatic electronic instrument for accurate measurements of food volume and density.

OBJECTIVE: Accurate measurements of food volume and density are often required as 'gold standards' for calibration of image-based dietary assessment and food database development. Currently, there is no specialised laboratory instrument for these measurements. We present the design of a new volume of density (VD) meter to bridge this technological gap. DESIGN: Our design consists of a turntable, a load sensor, a set of cameras and lights installed on an arc-shaped stationary support, and a microcomputer. It acquires an array of food images, reconstructs a 3D volumetric model, weighs the food and calculates both food volume and density, all in an automatic process controlled by the microcomputer. To adapt to the complex shapes of foods, a new food surface model, derived from the electric field of charged particles, is developed for 3D point cloud reconstruction of either convex or concave food surfaces. RESULTS: We conducted two experiments to evaluate the VD meter. The first experiment utilised computer-synthesised 3D objects with prescribed convex and concave surfaces of known volumes to investigate different food surface types. The second experiment was based on actual foods with different shapes, colours and textures. Our results indicated that, for synthesised objects, the measurement error of the electric field-based method was <1 %, significantly lower compared with traditional methods. For real-world foods, the measurement error depended on the types of food volumes (detailed discussion included). The largest error was approximately 5 %. CONCLUSION: The VD meter provides a new electronic instrument to support advanced research in nutrition science.

Image-based food portion size estimation using a smartphone without a fiducial marker.

OBJECTIVE: Current approaches to food volume estimation require the person to carry a fiducial marker (e.g. a checkerboard card), to be placed next to the food before taking a picture. This procedure is inconvenient and post-processing of the food picture is time-consuming and sometimes inaccurate. These problems keep people from using the smartphone for self-administered dietary assessment. The current bioengineering study presents a novel smartphone-based imaging approach to table-side estimation of food volume which overcomes current limitations. DESIGN: We present a new method for food volume estimation without a fiducial marker. Our mathematical model indicates that, using a special picture-taking strategy, the smartphone-based imaging system can be calibrated adequately if the physical length of the smartphone and the output of the motion sensor within the device are known. We also present and test a new virtual reality method for food volume estimation using the International Food Unit™ and a training process for error control. RESULTS: Our pilot study, with sixty-nine participants and fifteen foods, indicates that the fiducial-marker-free approach is valid and that the training improves estimation accuracy significantly (P0·05). CONCLUSIONS: Elimination of a fiducial marker and application of virtual reality, the International Food Unit™ and an automated training allowed quick food volume estimation and control of the estimation error. The estimated volume could be used to search a nutrient database and determine energy and nutrients in the diet.

Reliability and validity of food portion size estimation from images using manual flexible digital virtual meshes.

OBJECTIVE: The eButton takes frontal images at 4s intervals throughout the day. A three-dimensional manually administered wire mesh procedure has been developed to quantify portion sizes from the two-dimensional images. The present paper reports a test of the inter-rater reliability and validity of use of the wire mesh procedure. DESIGN: Seventeen foods of diverse shapes and sizes served on plates, bowls and cups were selected to rigorously test the portion assessment procedure. A dietitian not involved in inter-rater reliability assessment used standard cups to independently measure the quantities of foods to generate the 'true' value for a total of seventy-five 'served' and seventy-five smaller 'left' images with diverse portion sizes. SETTING: The images appeared on the computer to which the digital wire meshes were applied. SUBJECTS: Two dietitians and three engineers independently estimated portion size of the larger ('served') and smaller ('left') images for the same foods. RESULTS: The engineers had higher reliability and validity than the dietitians. The dietitians had lower reliabilities and validities for the smaller more irregular images, but the engineers did not, suggesting training could overcome this limitation. The lower reliabilities and validities for foods served in bowls, compared with plates, suggest difficulties with the curved nature of the bowls. CONCLUSIONS: The wire mesh procedure is an important step forward in quantifying portion size, which has been subject to substantial self-report error. Improved training procedures are needed to overcome the identified problems.

Automatic food detection in egocentric images using artificial intelligence technology.

OBJECTIVE: To develop an artificial intelligence (AI)-based algorithm which can automatically detect food items from images acquired by an egocentric wearable camera for dietary assessment. DESIGN: To study human diet and lifestyle, large sets of egocentric images were acquired using a wearable device, called eButton, from free-living individuals. Three thousand nine hundred images containing real-world activities, which formed eButton data set 1, were manually selected from thirty subjects. eButton data set 2 contained 29 515 images acquired from a research participant in a week-long unrestricted recording. They included both food- and non-food-related real-life activities, such as dining at both home and restaurants, cooking, shopping, gardening, housekeeping chores, taking classes, gym exercise, etc. All images in these data sets were classified as food/non-food images based on their tags generated by a convolutional neural network. RESULTS: A cross data-set test was conducted on eButton data set 1. The overall accuracy of food detection was 91·5 and 86·4 %, respectively, when one-half of data set 1 was used for training and the other half for testing. For eButton data set 2, 74·0 % sensitivity and 87·0 % specificity were obtained if both 'food' and 'drink' were considered as food images. Alternatively, if only 'food' items were considered, the sensitivity and specificity reached 85·0 and 85·8 %, respectively. CONCLUSIONS: The AI technology can automatically detect foods from low-quality, wearable camera-acquired real-world egocentric images with reasonable accuracy, reducing both the burden of data processing and privacy concerns.

A Hierarchical Deep Fusion Framework for Egocentric Activity Recognition Using a Wearable Hybrid Sensor System.

Recently, egocentric activity recognition has attracted considerable attention in the pattern recognition and artificial intelligence communities because of its wide applicability in medical care, smart homes, and security monitoring. In this study, we developed and implemented a deep-learning-based hierarchical fusion framework for the recognition of egocentric activities of daily living (ADLs) in a wearable hybrid sensor system comprising motion sensors and cameras. Long short-term memory (LSTM) and a convolutional neural network are used to perform egocentric ADL recognition based on motion sensor data and photo streaming in different layers, respectively. The motion sensor data are used solely for activity classification according to motion state, while the photo stream is used for further specific activity recognition in the motion state groups. Thus, both motion sensor data and photo stream work in their most suitable classification mode to significantly reduce the negative influence of sensor differences on the fusion results. Experimental results show that the proposed method not only is more accurate than the existing direct fusion method (by up to 6%) but also avoids the time-consuming computation of optical flow in the existing method, which makes the proposed algorithm less complex and more suitable for practical application.

Utility of eButton images for identifying food preparation behaviors and meal-related tasks in adolescents.

BACKGROUND: Food preparation skills may encourage healthy eating. Traditional assessment of child food preparation employs self- or parent proxy-reporting methods, which are prone to error. The eButton is a wearable all-day camera that has promise as an objective, passive method for measuring child food preparation practices. PURPOSE: This paper explores the feasibility of the eButton to reliably capture home food preparation behaviors and practices in a sample of pre- and early adolescents (ages 9 to 13). METHODS: This is a secondary analysis of two eButton pilot projects evaluating the dietary intake of pre- and early adolescents in or around Houston, Texas. Food preparation behaviors were coded into seven major categories including: browsing, altering food/adding seasoning, food media, meal related tasks, prep work, cooking and observing. Inter-coder reliability was measured using Cohen's kappa and percent agreement. RESULTS: Analysis was completed on data for 31 participants. The most common activity was browsing in the pantry or fridge. Few participants demonstrated any food preparation work beyond unwrapping of food packages and combining two or more ingredients; actual cutting or measuring of foods were rare. CONCLUSIONS: Although previous research suggests children who "help" prepare meals may obtain some dietary benefit, accurate assessment tools of food preparation behavior are lacking. The eButton offers a feasible approach to food preparation behavior measurement among pre- and early adolescents. Follow up research exploring the validity of this method in a larger sample, and comparisons between cooking behavior and dietary intake are needed.

Deep Learning for Classification of Normal Swallows in Adults.

Cervical auscultation is a method for assessing swallowing performance. However, its ability to serve as a classification tool for a practical clinical assessment method is not fully understood. In this study, we utilized neural network classification methods in the form of Deep Belief networks in order to classify swallows. We specifically utilized swallows that did not result in clinically significant aspiration and classified them on whether they originated from healthy subjects or unhealthy patients. Dual-axis swallowing vibrations from 1946 discrete swallows were recorded from 55 healthy and 53 unhealthy subjects. The Fourier transforms of both signals were used as inputs to the networks of various sizes. We found that single and multi-layer Deep Belief networks perform nearly identically when analyzing only a single vibration signal. However, multi-layered Deep Belief networks demonstrated approximately a 5% to 10% greater accuracy and sensitivity when both signals were analyzed concurrently, indicating that higher-order relationships between these vibrations are important for classification and assessment.

Position paper on the need for portion-size education and a standardised unit of measurement.

Large portion sizes contribute to weight gain in western societies. Portion-size interventions, aids and education can be effective in helping prevent weight gain, but consumers are unsure what appropriate portions are and express confusion about existing guidelines. A lack of clarity about suggested serving size recommendations is a major barrier to food portion-size control. Therefore, standardised measurement units and unambiguous terminologies are required. This position paper summarises the evidence regarding the impact and importance of portion-size education and estimation, and outlines strategies for improving consumer understanding and application of this through the development of an international food measurement system and a range of appropriate portion control tools. In this position paper, the authors call for the standardisation of food volume measurement terminologies, units, implementation recommendations, as well as consumer education. The target audience for this paper includes nutrition and behavioural researchers, policy makers, and stakeholders who potentially influence and implement changes in national food measurement systems, which in turn impact on consumer choice.

Simultaneous Wireless Power Transfer and Data Communication Using Synchronous Pulse-Controlled Load Modulation.

Wireless Power Transfer (WPT) and wireless data communication are both important problems of research with various applications, especially in medicine. However, these two problems are usually studied separately. In this work, we present a joint study of both problems. Most medical electronic devices, such as smart implants, must have both a power supply to allow continuous operation and a communication link to pass information. Traditionally, separate wireless channels for power transfer and communication are utilized, which complicate the system structure, increase power consumption and make device miniaturization difficult. A more effective approach is to use a single wireless link with both functions of delivering power and passing information. We present a design of such a wireless link in which power and data travel in opposite directions. In order to aggressively miniaturize the implant and reduce power consumption, we eliminate the traditional multi-bit Analog-to-Digital Converter (ADC), digital memory and data transmission circuits all together. Instead, we use a pulse stream, which is obtained from the original biological signal, by a sigma-delta converter and an edge detector, to alter the load properties of the WPT channel. The resulting WPT signal is synchronized with the load changes therefore requiring no memory elements to record inter-pulse intervals. We take advantage of the high sensitivity of the resonant WPT to the load change, and the system dynamic response is used to transfer each pulse. The transient time of the WPT system is analyzed using the coupling mode theory (CMT). Our experimental results show that the memoryless approach works well for both power delivery and data transmission, providing a new wireless platform for the design of future miniaturized medical implants.

ARCH: Adaptive recurrent-convolutional hybrid networks for long-term action recognition.

Recognition of human actions from digital video is a challenging task due to complex interfering factors in uncontrolled realistic environments. In this paper, we propose a learning framework using static, dynamic and sequential mixed features to solve three fundamental problems: spatial domain variation, temporal domain polytrope, and intra- and inter-class diversities. Utilizing a cognitive-based data reduction method and a hybrid "network upon networks" architecture, we extract human action representations which are robust against spatial and temporal interferences and adaptive to variations in both action speed and duration. We evaluated our method on the UCF101 and other three challenging datasets. Our results demonstrated a superior performance of the proposed algorithm in human action recognition.

Cross-trees, Edge and Superpixel Priors-based Cost aggregation for Stereo matching.

In this paper, we propose a novel cross-trees structure to perform the nonlocal cost aggregation strategy, and the cross-trees structure consists of a horizontal-tree and a vertical-tree. Compared to other spanning trees, the significant superiorities of the cross-trees are that the trees' constructions are efficient and the trees are exactly unique since the constructions are independent on any local or global property of the image itself. Additionally, two different priors: edge prior and superpixel prior, are proposed to tackle the false cost aggregations which cross the depth boundaries. Hence, our method contains two different algorithms in terms of cross-trees+prior. By traversing the two crossed trees successively, a fast non-local cost aggregation algorithm is performed twice to compute the aggregated cost volume. Performance evaluation on the 27 Middlebury data sets shows that both our algorithms outperform the other two tree-based non-local methods, namely minimum spanning tree (MST) and segment-tree (ST).

Multiview stereo and silhouette fusion via minimizing generalized reprojection error.

Accurate reconstruction of 3D geometrical shape from a set of calibrated 2D multiview images is an active yet challenging task in computer vision. The existing multiview stereo methods usually perform poorly in recovering deeply concave and thinly protruding structures, and suffer from several common problems like slow convergence, sensitivity to initial conditions, and high memory requirements. To address these issues, we propose a two-phase optimization method for generalized reprojection error minimization (TwGREM), where a generalized framework of reprojection error is proposed to integrate stereo and silhouette cues into a unified energy function. For the minimization of the function, we first introduce a convex relaxation on 3D volumetric grids which can be efficiently solved using variable splitting and Chambolle projection. Then, the resulting surface is parameterized as a triangle mesh and refined using surface evolution to obtain a high-quality 3D reconstruction. Our comparative experiments with several state-of-the-art methods show that the performance of TwGREM based 3D reconstruction is among the highest with respect to accuracy and efficiency, especially for data with smooth texture and sparsely sampled viewpoints.

An adaptive Hidden Markov model for activity recognition based on a wearable multi-sensor device.

Human activity recognition is important in the study of personal health, wellness and lifestyle. In order to acquire human activity information from the personal space, many wearable multi-sensor devices have been developed. In this paper, a novel technique for automatic activity recognition based on multi-sensor data is presented. In order to utilize these data efficiently and overcome the big data problem, an offline adaptive-Hidden Markov Model (HMM) is proposed. A sensor selection scheme is implemented based on an improved Viterbi algorithm. A new method is proposed that incorporates personal experience into the HMM model as a priori information. Experiments are conducted using a personal wearable computer eButton consisting of multiple sensors. Our comparative study with the standard HMM and other alternative methods in processing the eButton data have shown that our method is more robust and efficient, providing a useful tool to evaluate human activity and lifestyle.

Fast single image haze removal via local atmospheric light veil estimation.

In this study, a novel single-image based dehazing framework is proposed to remove haze artifacts from images through local atmospheric light estimation. We use a novel strategy based on a physical model where the extreme intensity of each RGB pixel is used to define an initial atmospheric veil (local atmospheric light veil). Across bilateral filter is applied to each veil to achieve both local smoothness and edge preservation. A transmission map and a reflection component of each RGB channel are constructed from the physical atmospheric scattering model. The proposed approach avoids adverse effects caused by the error in estimating the global atmospheric light. Experimental results on outdoor hazy images demonstrate that the proposed method produces image output with satisfactory visual quality and color fidelity. Our comparative study demonstrates a higher performance of our method over several state-of-the-art methods.

Accuracy of food portion size estimation from digital pictures acquired by a chest-worn camera.

OBJECTIVE: Accurate estimation of food portion size is of paramount importance in dietary studies. We have developed a small, chest-worn electronic device called eButton which automatically takes pictures of consumed foods for objective dietary assessment. From the acquired pictures, the food portion size can be calculated semi-automatically with the help of computer software. The aim of the present study is to evaluate the accuracy of the calculated food portion size (volumes) from eButton pictures. DESIGN: Participants wore an eButton during their lunch. The volume of food in each eButton picture was calculated using software. For comparison, three raters estimated the food volume by viewing the same picture. The actual volume was determined by physical measurement using seed displacement. SETTING: Dining room and offices in a research laboratory. SUBJECTS: Seven lab member volunteers. RESULTS: Images of 100 food samples (fifty Western and fifty Asian foods) were collected and each food volume was estimated from these images using software. The mean relative error between the estimated volume and the actual volume over all the samples was -2·8 % (95 % CI -6·8 %, 1·2 %) with sd of 20·4 %. For eighty-five samples, the food volumes determined by computer differed by no more than 30 % from the results of actual physical measurements. When the volume estimates by the computer and raters were compared, the computer estimates showed much less bias and variability. CONCLUSIONS: From the same eButton pictures, the computer-based method provides more objective and accurate estimates of food volume than the visual estimation method.

Obstacle classification and 3D measurement in unstructured environments based on ToF cameras.

Inspired by the human 3D visual perception system, we present an obstacle detection and classification method based on the use of Time-of-Flight (ToF) cameras for robotic navigation in unstructured environments. The ToF camera provides 3D sensing by capturing an image along with per-pixel 3D space information. Based on this valuable feature and human knowledge of navigation, the proposed method first removes irrelevant regions which do not affect robot's movement from the scene. In the second step, regions of interest are detected and clustered as possible obstacles using both 3D information and intensity image obtained by the ToF camera. Consequently, a multiple relevance vector machine (RVM) classifier is designed to classify obstacles into four possible classes based on the terrain traversability and geometrical features of the obstacles. Finally, experimental results in various unstructured environments are presented to verify the robustness and performance of the proposed approach. We have found that, compared with the existing obstacle recognition methods, the new approach is more accurate and efficient.