Surveying Nutrient Assessment with Photographs of Meals (SNAPMe): A Benchmark Dataset of Food Photos for Dietary Assessment.

Photo-based dietary assessment is becoming more feasible as artificial intelligence methods improve. However, advancement of these methods for dietary assessment in research settings has been hindered by the lack of an appropriate dataset against which to benchmark algorithm performance. We conducted the Surveying Nutrient Assessment with Photographs of Meals (SNAPMe) study (ClinicalTrials ID: NCT05008653) to pair meal photographs with traditional food records. Participants were recruited nationally, and 110 enrollment meetings were completed via web-based video conferencing. Participants uploaded and annotated their meal photos using a mobile phone app called Bitesnap and completed food records using the Automated Self-Administered 24-h Dietary Assessment Tool (ASA24®) version 2020. Participants included photos before and after eating non-packaged and multi-serving packaged meals, as well as photos of the front and ingredient labels for single-serving packaged foods. The SNAPMe Database (DB) contains 3311 unique food photos linked with 275 ASA24 food records from 95 participants who photographed all foods consumed and recorded food records in parallel for up to 3 study days each. The use of the SNAPMe DB to evaluate ingredient prediction demonstrated that the publicly available algorithms FB Inverse Cooking and Im2Recipe performed poorly, especially for single-ingredient foods and beverages. Correlations between nutrient estimates common to the Bitesnap and ASA24 dietary assessment tools indicated a range in predictive capacity across nutrients (cholesterol, adjusted R2 = 0.85, p < 0.0001; food folate, adjusted R2 = 0.21, p < 0.05). SNAPMe DB is a publicly available benchmark for photo-based dietary assessment in nutrition research. Its demonstrated utility suggested areas of needed improvement, especially the prediction of single-ingredient foods and beverages.

Canonical correlation analysis of brain prefrontal activity measured by functional near infra-red spectroscopy (fNIRS) during a moral judgment task.

Individuals differ in the extent to which they make decisions in different moral dilemmas. In this study, we investigated the relationship between functional brain activities during moral decision making and psychopathic personality traits in a healthy population. We measured the hemodynamic activities of the brain by functional near-infrared spectroscopy (fNIRS). FNIRS is an evolving non-invasive neuroimaging modality which is relatively inexpensive, patient friendly and robust to subject movement. Psychopathic traits were evaluated through a self-report questionnaire called the Psychopathic Personality Inventory Revised (PPI-R). We recorded functional brain activities of 30 healthy subjects while they performed a moral judgment (MJ) task. Regularized canonical correlation analysis (R-CCA) was applied to find the relationships between activation in different regions of prefrontal cortex (PFC) and the core psychopathic traits. Our results showed a significant canonical correlation between PFC activation and PPI-R content scale (PPI-R-CS). Specifically, coldheartedness and carefree non-planfulness were the only PPI-R-CS factors that were highly correlated with PFC activation during personal (emotionally salient) MJ, while Machiavellian egocentricity, rebellious nonconformity, coldheartedness, and carefree non-planfulness were the core traits that exhibited the same dynamics as PFC activation during impersonal (more logical) MJ. Furthermore, ventromedial prefrontal cortex (vmPFC) and left lateral PFC were the most positively correlated regions with PPI-R-CS traits during personal MJ, and the right vmPFC and right lateral PFC in impersonal MJ.

Cross-Modal Scene Networks.

People can recognize scenes across many different modalities beyond natural images. In this paper, we investigate how to learn cross-modal scene representations that transfer across modalities. To study this problem, we introduce a new cross-modal scene dataset. While convolutional neural networks can categorize scenes well, they also learn an intermediate representation not aligned across modalities, which is undesirable for cross-modal transfer applications. We present methods to regularize cross-modal convolutional neural networks so that they have a shared representation that is agnostic of the modality. Our experiments suggest that our scene representation can help transfer representations across modalities for retrieval. Moreover, our visualizations suggest that units emerge in the shared representation that tend to activate on consistent concepts independently of the modality.

Opti-acoustic stereo imaging: on system calibration and 3-D target reconstruction.

Utilization of an acoustic camera for range measurements is a key advantage for 3-D shape recovery of underwater targets by opti-acoustic stereo imaging, where the associated epipolar geometry of optical and acoustic image correspondences can be described in terms of conic sections. In this paper, we propose methods for system calibration and 3-D scene reconstruction by maximum likelihood estimation from noisy image measurements. The recursive 3-D reconstruction method utilized as initial condition a closed-form solution that integrates the advantages of two other closed-form solutions, referred to as the range and azimuth solutions. Synthetic data tests are given to provide insight into the merits of the new target imaging and 3-D reconstruction paradigm, while experiments with real data confirm the findings based on computer simulations, and demonstrate the merits of this novel 3-D reconstruction paradigm.