Diversity, equity, and inclusivity in observational ambulatory assessment: Recommendations from two decades of Electronically Activated Recorder (EAR) research.

Ambient audio sampling methods such as the Electronically Activated Recorder (EAR) have become increasingly prominent in clinical and social sciences research. These methods record snippets of naturalistically assessed audio from participants' daily lives, enabling novel observational research about the daily social interactions, identities, environments, behaviors, and speech of populations of interest. In practice, these scientific opportunities are equaled by methodological challenges: researchers' own cultural backgrounds and identities can easily and unknowingly permeate the collection, coding, analysis, and interpretation of social data from daily life. Ambient audio sampling poses unique and significant challenges to cultural humility, diversity, equity, and inclusivity (DEI) in scientific research that require systematized attention. Motivated by this observation, an international consortium of 21 researchers who have used ambient audio sampling methodologies created a workgroup with the aim of improving upon existing published guidelines. We pooled formally and informally documented challenges pertaining to DEI in ambient audio sampling from our collective experience on 40+ studies (most of which used the EAR app) in clinical and healthy populations ranging from children to older adults. This article presents our resultant recommendations and argues for the incorporation of community-engaged research methods in observational ambulatory assessment designs looking forward. We provide concrete recommendations across each stage typical of an ambient audio sampling study (recruiting and enrolling participants, developing coding systems, training coders, handling multi-linguistic participants, data analysis and interpretation, and dissemination of results) as well as guiding questions that can be used to adapt these recommendations to project-specific constraints and needs.

Automatic Behavior Assessment from Uncontrolled Everyday Audio Recordings by Deep Learning.

The manual categorization of behavior from sensory observation data to facilitate further analyses is a very expensive process. To overcome the inherent subjectivity of this process, typically, multiple domain experts are involved, resulting in increased efforts for the labeling. In this work, we investigate whether social behavior and environments can automatically be coded based on uncontrolled everyday audio recordings by applying deep learning. Recordings of daily living were obtained from healthy young and older adults at randomly selected times during the day by using a wearable device, resulting in a dataset of uncontrolled everyday audio recordings. For classification, a transfer learning approach based on a publicly available pretrained neural network and subsequent fine-tuning was implemented. The results suggest that certain aspects of social behavior and environments can be automatically classified. The ambient noise of uncontrolled audio recordings, however, poses a hard challenge for automatic behavior assessment, in particular, when coupled with data sparsity.

With a little help from familiar interlocutors: real-world language use in young and older adults.

OBJECTIVES: Functional psychologists are concerned with the performance of cognitive activities in the real world in relation to cognitive changes in older age. Conversational contexts may mitigate the influence of cognitive aging on the cognitive activity of language production. This study examined effects of familiarity with interlocutors, as a context, on language production in the real world. METHOD: We collected speech samples using iPhones, where an audio recording app (i.e. Electronically Activated Recorder [EAR]) was installed. Over 31,300 brief audio files (30-second long) were randomly collected across four days from 61 young and 48 healthy older adults in Switzerland. We transcribed the audio files that included participants' speech and manually coded for familiar interlocutors (i.e. significant other, friends, family members) and strangers. We computed scores of vocabulary richness and grammatical complexity from the transcripts using computational linguistics techniques. RESULTS: Bayesian multilevel analyses showed that participants used richer vocabulary and more complex grammar when talking with familiar interlocutors than with strangers. Young adults used more diverse vocabulary than older adults and the age effects remained stable across contexts. Furthermore, older adults produced equally complex grammar as young adults did with the significant other, but simpler grammar than young adults with friends and family members. CONCLUSION: Familiarity with interlocutors is a promising contextual factor for research on aging and language complexity in the real world. Results were discussed in the context of cognitive aging.

Social Reminiscence in Older Adults' Everyday Conversations: Automated Detection Using Natural Language Processing and Machine Learning.

BACKGROUND: Reminiscence is the act of thinking or talking about personal experiences that occurred in the past. It is a central task of old age that is essential for healthy aging, and it serves multiple functions, such as decision-making and introspection, transmitting life lessons, and bonding with others. The study of social reminiscence behavior in everyday life can be used to generate data and detect reminiscence from general conversations. OBJECTIVE: The aims of this original paper are to (1) preprocess coded transcripts of conversations in German of older adults with natural language processing (NLP), and (2) implement and evaluate learning strategies using different NLP features and machine learning algorithms to detect reminiscence in a corpus of transcripts. METHODS: The methods in this study comprise (1) collecting and coding of transcripts of older adults' conversations in German, (2) preprocessing transcripts to generate NLP features (bag-of-words models, part-of-speech tags, pretrained German word embeddings), and (3) training machine learning models to detect reminiscence using random forests, support vector machines, and adaptive and extreme gradient boosting algorithms. The data set comprises 2214 transcripts, including 109 transcripts with reminiscence. Due to class imbalance in the data, we introduced three learning strategies: (1) class-weighted learning, (2) a meta-classifier consisting of a voting ensemble, and (3) data augmentation with the Synthetic Minority Oversampling Technique (SMOTE) algorithm. For each learning strategy, we performed cross-validation on a random sample of the training data set of transcripts. We computed the area under the curve (AUC), the average precision (AP), precision, recall, as well as F1 score and specificity measures on the test data, for all combinations of NLP features, algorithms, and learning strategies. RESULTS: Class-weighted support vector machines on bag-of-words features outperformed all other classifiers (AUC=0.91, AP=0.56, precision=0.5, recall=0.45, F1=0.48, specificity=0.98), followed by support vector machines on SMOTE-augmented data and word embeddings features (AUC=0.89, AP=0.54, precision=0.35, recall=0.59, F1=0.44, specificity=0.94). For the meta-classifier strategy, adaptive and extreme gradient boosting algorithms trained on word embeddings and bag-of-words outperformed all other classifiers and NLP features; however, the performance of the meta-classifier learning strategy was lower compared to other strategies, with highly imbalanced precision-recall trade-offs. CONCLUSIONS: This study provides evidence of the applicability of NLP and machine learning pipelines for the automated detection of reminiscence in older adults' everyday conversations in German. The methods and findings of this study could be relevant for designing unobtrusive computer systems for the real-time detection of social reminiscence in the everyday life of older adults and classifying their functions. With further improvements, these systems could be deployed in health interventions aimed at improving older adults' well-being by promoting self-reflection and suggesting coping strategies to be used in the case of dysfunctional reminiscence cases, which can undermine physical and mental health.

The mood-enhancement function of autobiographical memories: Comparisons with other functions in terms of emotional valence.

In two studies, we examined the emotional valence of memories used for mood-enhancement in relation to memories serving self, social and directive functions. Our sample included a total of 263 participants aged between 45 and 82 years. In Study 1, participants recalled memories in response to 51 cue words. In Study 2, participants recalled 32 memories that served the four functions (eight memories per function). We used multilevel modeling to take into consideration the hierarchical nature of our datasets (memories nested within individuals). Study 1 showed that emotional valence was positively associated with mood-enhancement and social functions, whereas negatively related to self and directive functions. This relation was strongest for the mood-enhancement function. In Study 2, mood-enhancing memories were rated as more positive than self, social and directive memories. We discussed results in terms of the tripartite model of memory functions and proposed that mood-enhancement should represent a distinct function.

Consistency of adults' earliest memories across two years.

The consistency of earliest memories in content, dating, and memory qualities was investigated. A total of 84 (27 males; Mage = 24.93, SD = 1.36) adults reported earliest memories, estimated ages, and rated their recollections on memory qualities with a two-year time lag. At Time 2, their original reports at Time 1 were presented and they were asked to report whether the earliest memories they recalled at Time 2 were the same. Fifty-six per cent of the participants reported the same earliest memories and those remembering the same events had earlier memories than those remembering different ones. Although no significant differences were observed in estimated ages on the basis of mean ages, a predating bias of later memories and a tendency to postdate earlier memories were observed on the basis of a 48-month cut-off point. Thus, how the data is analysed is critical in detecting dating biases or errors affecting conclusions and interpretations about the dating consistency of earliest memories. Finally, memory qualities of earliest memories displayed a high level of consistency with a two-year time lag regardless of remembering the same versus different event.

Identifying distinct sets of predictors of specific functions of autobiographical memory.

Grounded in the ecological approach, research has charted several adaptive functions of autobiographical remembering. Each represents a rather different psychosocial domain (i.e., self, social, directive). The goal of this research was to determine the contributions of each of a set of variables, controlling for all others, in predicting use of autobiographical memory to serve each specific function. In two studies, participants (N = 100; N = 195) rated frequency of functional use of specific event memories and completed a brief battery of memory-related measures. Most Study One results were replicated in Study Two. Self-relevance of memories was related to their functional use regardless of domain. Each function was also, however, predicted by a unique set of variables consistent with its use in a given psychosocial domain. Findings emphasise how a combination of factors come into play to allow humans to use autobiographical memory to serve various different functions in navigating daily life.

The psychological distance of memories: Examining causal relations with mood and self-esteem in young, middle-aged and older adults.

Three studies examined the self-enhancement function of autobiographical memory (measured with subjective temporal distance of memories). Participants recalled a memory of an attained and a failed goal and rated the subjective distance between each memory and the present. Study 1 showed that young adults with higher self-esteem felt closer to memories of attained goals and farther from failure memories than those with lower self-esteem. In Study 2, young, middle-aged and older adults with higher self-esteem felt closer to success memories, whereas self-esteem was unrelated to the temporal distance of failure memories. In both studies, feeling closer to success memories (and far from failure) led to enhanced mood. In Study 3, state self-esteem was experimentally manipulated. The manipulation had no effect on young and older adults, but middle-aged adults whose self-esteem was decreased, felt closer to success memories than failure memories. Results are discussed in relation to the temporal self-appraisal theory.

Michael Jackson, Bin Laden and I: functions of positive and negative, public and private flashbulb memories.

This study examined the perceived psychosocial functions of flashbulb memories: It compared positive and negative public flashbulb memories (positive: Bin Laden's death, negative: Michael Jackson's death) with private ones (positive: pregnancy, negative: death of a loved one). A sample of n = 389 young and n = 176 middle-aged adults answered canonical category questions used to identify flashbulb memories and rated the personal significance, the psychological temporal distance, and the functions of each memory (i.e., self-continuity, social-boding, directive functions). Hierarchical regressions showed that, in general, private memories were rated more functional than public memories. Positive and negative private memories were comparable in self-continuity and directionality, but the positive private memory more strongly served social functions. In line with the positivity bias in autobiographical memory, positive flashbulb memories felt psychologically closer than negative ones. Finally, middle-aged adults rated their memories as less functional regarding self-continuity and social-bonding than young adults. Results are discussed regarding the tripartite model of autobiographical memory functions.

Autobiographical memory functions in young Japanese men and women.

The present study examined whether the three major functions of autobiographical memory observed in Western societies (i.e., directing-behaviour, social-bonding and self-continuity) also exist in an East Asian society. Two self-report measures were used to assess the autobiographical memory functions of Japanese men and women. Japanese young adults (N = 451, ages 17-28 years) first completed the original Thinking About Life Experiences (TALE) Questionnaire. They subsequently received three TALE items that represented memory functions and attempted to recall a specific instance of memory recall for each item. Confirmatory factor analyses on the TALE showed that the three functions were replicated in the current sample. However, Japanese participants reported lower levels of all three functions than American participants in a previous study. We also explored whether there was an effect of gender in this Japanese sample. Women reported higher levels of the self-continuity and social-bonding functions than men. Finally, participants recalled more specific instances of memory recall for the TALE items that had received higher ratings on the TALE, suggesting that the findings on the first measure were supported by the second measure. Results are discussed in relation to the functional approach to autobiographical memory in a cross-cultural context.

Understanding reminiscence and its negative functions in the everyday conversations of young adults: A machine learning approach.

Reminiscence is the act of recalling or telling others about relevant personal past experiences. It is an important activity for all individuals, young and old alike. In fact, reminiscence can serve different functions that can support or be detrimental to one's well-being. Although previous studies have extensively investigated older adults' recalling of autobiographical memories, the evidence for young adults remains scarce. Therefore, in this work, we analyze young adults' production of reminiscence and their functions with a naturalistic observation method. Furthermore, we demonstrate that natural language processing and machine learning can automatically detect reminiscence and its negative functions in young adults' everyday conversations. We interpret machine learning model results using Shapley explanations. Our results indicate that young adults reminisce in everyday life mostly to connect with others through conversation, to compensate for a lack of stimulation or to recall difficult past experiences. Moreover, our models improve existing benchmarks from the literature on the automated detection of older adults' reminiscence in everyday life. Finally, our results may support the development of digital health intervention programs that detect reminiscence and its functions in young adults to support their well-being.

Predicting Working Memory in Healthy Older Adults Using Real-Life Language and Social Context Information: A Machine Learning Approach.

BACKGROUND: Language use and social interactions have demonstrated a close relationship with cognitive measures. It is important to improve the understanding of language use and behavioral indicators from social context to study the early prediction of cognitive decline among healthy populations of older adults. OBJECTIVE: This study aimed at predicting an important cognitive ability, working memory, of 98 healthy older adults participating in a 4-day-long naturalistic observation study. We used linguistic measures, part-of-speech (POS) tags, and social context information extracted from 7450 real-life audio recordings of their everyday conversations. METHODS: The methods in this study comprise (1) the generation of linguistic measures, representing idea density, vocabulary richness, and grammatical complexity, as well as POS tags with natural language processing (NLP) from the transcripts of real-life conversations and (2) the training of machine learning models to predict working memory using linguistic measures, POS tags, and social context information. We measured working memory using (1) the Keep Track test, (2) the Consonant Updating test, and (3) a composite score based on the Keep Track and Consonant Updating tests. We trained machine learning models using random forest, extreme gradient boosting, and light gradient boosting machine algorithms, implementing repeated cross-validation with different numbers of folds and repeats and recursive feature elimination to avoid overfitting. RESULTS: For all three prediction routines, models comprising linguistic measures, POS tags, and social context information improved the baseline performance on the validation folds. The best model for the Keep Track prediction routine comprised linguistic measures, POS tags, and social context variables. The best models for prediction of the Consonant Updating score and the composite working memory score comprised POS tags only. CONCLUSIONS: The results suggest that machine learning and NLP may support the prediction of working memory using, in particular, linguistic measures and social context information extracted from the everyday conversations of healthy older adults. Our findings may support the design of an early warning system to be used in longitudinal studies that collects cognitive ability scores and records real-life conversations unobtrusively. This system may support the timely detection of early cognitive decline. In particular, the use of a privacy-sensitive passive monitoring technology would allow for the design of a program of interventions to enable strategies and treatments to decrease or avoid early cognitive decline.

The relation of the conceptual self to recent and distant autobiographical memories.

Based on the self-memory system model (SMS; Conway, Singer, & Tagini, 2004) of autobiographical memory, this study uses a large sample of young and middle-aged adults to investigate the relation between individuals' current self-characteristics and the content of both their earliest childhood memory and a recent memory. In the first session, participants' current self-characteristics were assessed. In the second session, individuals provided a written narrative of their earliest childhood memory and a more recent memory (within-participants design) and rated the self themes present in each memory. In keeping with the SMS model, findings show that current self-characteristics were reflected in individuals' memories. As predicted, however, recent memories were more frequently linked to current self-characteristics than were earliest memories. All six current self-characteristics predicted the inclusion of these themes in recent memories, but only four self-characteristics were associated with memory themes in earliest memories. The relation between current self-characteristics and memory themes did not differ across young and middle-aged adults, suggesting developmental stability in these relations. Findings provide general support for the SMS model but also suggest possibilities for its extension and refinement.

Linear and Nonlinear Age Trajectories of Language Use: A Laboratory Observation Study of Couples' Conflict Conversations.

OBJECTIVES: This study investigated linear and nonlinear age effects on language use with speech samples that were representative of naturally occurring conversations. METHOD: Using a corpus-based approach, we examined couples' conflict conversations in the laboratory. The conversations, from a total of 364 community-dwelling German-speaking heterosexual couples (aged 19-82), were videotaped and transcribed. We examined usage of lower-frequency words, grammatical complexity, and utterance of filled pauses (e.g., äh ["um"]). RESULTS: Multilevel models showed that age effects on the usage of lower-frequency words were nonsignificant. Grammatical complexity increased until middle age (i.e., 54) and then declined. The utterance of filled pauses increased until old age (i.e., 70) and then decreased. DISCUSSION: Results are discussed in relation to cognitive aging research.

Examining the life story account of the reminiscence bump: why we remember more from young adulthood.

The reminiscence bump is a robust finding demonstrated mostly with the cue-word method in Western cultures. The first aim of the study was to replicate the reminiscence bump using a life history timeline method and to extend reminiscence bump research to a Turkish sample. The second aim was to empirically examine the recently proposed life story account (Gluck & Bluck, 2007) for the reminiscence bump. The sample consisted of 40 women and 32 men aged 52 to 66 years. Participants' lives were divided into 5-year intervals and they verbally reported as many memories as possible in a standard timeframe from each interval (in random order) and provided ratings of several memory characteristics. As expected, the lifespan distribution of the resulting 6373 memories demonstrated a reminiscence bump. In support of the life story account, bump memories were found to be more novel, more important for identity development, more distinct, and more likely to involve developmental transitions than memories from other age periods. Findings are discussed in terms of the life story account, which synthesises lifespan developmental theory and life story theory.

Reminiscence in Everyday Conversations: A Naturalistic Observation Study of Older Adults.

OBJECTIVES: We examined older adults' social reminiscence behavior in everyday life, and the relation between reminiscence functions and well-being. METHOD: The sample included 2,164 sound snippets that included speech from 45 healthy older adults. We examined reminiscence in daily conversations using the Electronically Activated Recorder. Across four days, we collected a random sample of about 280 sound files (30 seconds long) per participant. Participants' utterances were coded for whether they included reminiscence, for their functions and conversation partners. Participants completed mood and life satisfaction measures. RESULTS: Participants reminisced in 5% of their utterances (range: 0%-29%). They reminisced in 40% of cases with friends, 32.8% with their partner and 8% with their children/relatives. Three reminiscence functions were observed: identity, teaching/informing, and conversation. Participants' reminiscence served the identity function while they were reminiscing with their partner and children. Participants reminisced to teach/inform while reminiscing with their children and strangers. Reminiscing for conversation occurred mainly with partner and friends. We found positive relations between life satisfaction and identity, teach/inform, and conversation functions. Mood had a negative relation with identity and teach/inform functions. DISCUSSION: This is the first study to take a naturalistic observation approach to reminiscence and to build on self-report data.

Real-Life Language Use Across Different Interlocutors: A Naturalistic Observation Study of Adults Varying in Age.

Amid the growing interest in studying language use in real life, this study, for the first time, examined age effects on real-life language use, as well as within-person variations across different interlocutors. We examined speech samples collected via the Electronically Activated Recorder (i.e., portable audio recorder that periodically records ambient sounds) for a larger project. This existing dataset included more than 18,000 sound snippets (50-s long) from 53 American couples (breast cancer patients and their spouses; aged 24 to 94 years) in their natural environments. Sound snippets that included participant speech were coded for different interlocutors and given scores on three linguistic measures that are associated with age-related cognitive changes: usage of unique words, usage of uncommon words, and grammatical complexity. Multilevel models showed that there were no age effects on the three linguistic measures when interlocutors were not taken into account. We found that interlocutors influenced usage of unique words and grammatical complexity. More specifically, compared to talking with their spouse, participants used fewer unique words with children and friends; and used simpler grammatical structures with children, strangers, and in multiparty conversations. Next, we found that interlocutors influenced the associations between age and language use. More specifically, young adults used more unique words and more uncommon words with children than older adults. They used more uncommon words with friends and uttered more complex grammatical structures with strangers than older adults. Our results offer preliminary evidence for a new perspective to understand real-life language use: focusing not only on individual characteristics (i.e., age), but also context (i.e., interlocutors). This perspective should be useful to researchers who are interested in collecting "big data" and understanding cognitive activities in real life.

Conversational Time Travel: Evidence of a Retrospective Bias in Real Life Conversations.

We examined mental time travel reflected onto individuals' utterances in real-life conversations using a naturalistic observation method: Electronically Activated Recorder (EAR, a portable audio recorder that periodically and unobtrusively records snippets of ambient sounds and speech). We introduced the term conversational time travel and examined, for the first time, how much individuals talked about their personal past versus personal future in real life. Study 1 included 9,010 sound files collected from 51 American adults who carried the EAR over 1 weekend and were recorded every 9 min for 50 s. Study 2 included 23,103 sound files from 33 young and 48 healthy older adults from Switzerland who carried the EAR for 4 days (2 weekdays and 1 weekend, counterbalanced). 30-s recordings occurred randomly throughout the day. We developed a new coding scheme for conversational time travel: We listened to all sound files and coded each file for whether the participant was talking or not. Those sound files that included participant speech were also coded in terms of their temporal focus (e.g., past, future, present, time-independent) and autobiographical nature (i.e., about the self, about others). We, first, validated our coding scheme using the text analysis tool, Linguistic Inquiry and Word Count. Next, we compared the percentages of past- and future-oriented utterances about the self (to tap onto conversational time travel). Results were consistent across all samples and showed that participants talked about their personal past two to three times as much as their personal future (i.e., retrospective bias). This is in contrast to research showing a prospective bias in thinking behavior, based on self-report and experience-sampling methods. Findings are discussed in relation to the social functions of recalling the personal past (e.g., sharing memories to bond with others, to update each other, to teach, to give advice) and to the directive functions of future-oriented thought (e.g., planning, decision making, goal setting that are more likely to happen privately in the mind). In sum, the retrospective bias in conversational time travel seems to be a functional and universal phenomenon across persons and across real-life situations.