MoodCapture: Depression Detection Using In-the-Wild Smartphone Images.

MoodCapture presents a novel approach that assesses depression based on images automatically captured from the front-facing camera of smartphones as people go about their daily lives. We collect over 125,000 photos in the wild from N=177 participants diagnosed with major depressive disorder for 90 days. Images are captured naturalistically while participants respond to the PHQ-8 depression survey question: "I have felt down, depressed, or hopeless". Our analysis explores important image attributes, such as angle, dominant colors, location, objects, and lighting. We show that a random forest trained with face landmarks can classify samples as depressed or non-depressed and predict raw PHQ-8 scores effectively. Our post-hoc analysis provides several insights through an ablation study, feature importance analysis, and bias assessment. Importantly, we evaluate user concerns about using MoodCapture to detect depression based on sharing photos, providing critical insights into privacy concerns that inform the future design of in-the-wild image-based mental health assessment tools.

Predicting Brain Functional Connectivity Using Mobile Sensing.

Brain circuit functioning and connectivity between specific regions allow us to learn, remember, recognize and think as humans. In this paper, we ask the question if mobile sensing from phones can predict brain functional connectivity. We study the brain resting-state functional connectivity (RSFC) between the ventromedial prefrontal cortex (vmPFC) and the amygdala, which has been shown by neuroscientists to be associated with mental illness such as anxiety and depression. We discuss initial results and insights from the NeuroSence study, an exploratory study of 105 first year college students using neuroimaging and mobile sensing across one semester. We observe correlations between several behavioral features from students' mobile phones and connectivity between vmPFC and amygdala, including conversation duration (r=0.365, p<0.001), sleep onset time (r=0.299, p<0.001) and the number of phone unlocks (r=0.253, p=0.029). We use a support vector classifier and 10-fold cross validation and show that we can classify whether students have higher (i.e., stronger) or lower (i.e., weaker) vmPFC-amygdala RSFC purely based on mobile sensing data with an F1 score of 0.793. To the best of our knowledge, this is the first paper to report that resting-state brain functional connectivity can be predicted using passive sensing data from mobile phones.