AI-readiness for Biomedical Data: Bridge2AI Recommendations.

Biomedical research and clinical practice are in the midst of a transition toward significantly increased use of artificial intelligence (AI) and machine learning (ML) methods. These advances promise to enable qualitatively deeper insight into complex challenges formerly beyond the reach of analytic methods and human intuition while placing increased demands on ethical and explainable artificial intelligence (XAI), given the opaque nature of many deep learning methods. The U.S. National Institutes of Health (NIH) has initiated a significant research and development program, Bridge2AI, aimed at producing new "flagship" datasets designed to support AI/ML analysis of complex biomedical challenges, elucidate best practices, develop tools and standards in AI/ML data science, and disseminate these datasets, tools, and methods broadly to the biomedical community. An essential set of concepts to be developed and disseminated in this program along with the data and tools produced are criteria for AI-readiness of data, including critical considerations for XAI and ethical, legal, and social implications (ELSI) of AI technologies. NIH Bridge to Artificial Intelligence (Bridge2AI) Standards Working Group members prepared this article to present methods for assessing the AI-readiness of biomedical data and the data standards perspectives and criteria we have developed throughout this program. While the field is rapidly evolving, these criteria are foundational for scientific rigor and the ethical design and application of biomedical AI methods.

Considerations for the Design of Informed Consent in Digital Health Research: Participant Perspectives.

The research team, prospective participants, and written materials all influence the success of the informed consent process. As digital health research becomes more prevalent, new challenges for successful informed consent are introduced. This exploratory research utilized a human centered design process in which 19 people were enrolled to participate in one of four online focus-groups. Participants discussed their experiences with informed consent, preferences for receiving study information and ideas about alternative consent approaches. Data were analyzed using qualitative methods. Six major themes and sixteen sub-themes were identified that included study information that prospective participants would like to receive, preferences for accessing information and a desire to connect with research team members. Specific to digital health, participants expressed a need to understand how the technologies worked and how the volume of granular personal information would be collected, stored, and shared.

Remote data collection of infant activity and sleep patterns via wearable sensors in the HEALthy Brain and Child Development Study (HBCD).

The HEALthy Brain and Child Development (HBCD) Study, a multi-site prospective longitudinal cohort study, will examine human brain, cognitive, behavioral, social, and emotional development beginning prenatally and planned through early childhood. Wearable and remote sensing technologies have advanced data collection outside of laboratory settings to enable exploring, in more detail, the associations of early experiences with brain development and social and health outcomes. In the HBCD Study, the Novel Technology/Wearable Sensors Working Group (WG-NTW) identified two primary data types to be collected: infant activity (by measuring leg movements) and sleep (by measuring heart rate and leg movements). These wearable technologies allow for remote collection in the natural environment. This paper illustrates the collection of such data via wearable technologies and describes the decision-making framework, which led to the currently deployed study design, data collection protocol, and derivatives, which will be made publicly available. Moreover, considerations regarding actual and potential challenges to adoption and use, data management, privacy, and participant burden were examined. Lastly, the present limitations in the field of wearable sensor data collection and analysis will be discussed in terms of extant validation studies, the difficulties in comparing performance across different devices, and the impact of evolving hardware/software/firmware.

Achieving Inclusive Healthcare through Integrating Education and Research with AI and Personalized Curricula.

Precision medicine promises significant health benefits but faces challenges such as the need for complex data management and analytics, interdisciplinary collaboration, and education of researchers, healthcare professionals, and participants. Addressing these needs requires the integration of computational experts, engineers, designers, and healthcare professionals to develop user-friendly systems and shared terminologies. The widespread adoption of large language models (LLMs) like GPT-4 and Claude 3 highlights the importance of making complex data accessible to non-specialists. The Stanford Data Ocean (SDO) strives to mitigate these challenges through a scalable, cloud-based platform that supports data management for various data types, advanced research, and personalized learning in precision medicine. SDO provides AI tutors and AI-powered data visualization tools to enhance educational and research outcomes and make data analysis accessible for users from diverse educational backgrounds. By extending engagement and cutting-edge research capabilities globally, SDO particularly benefits economically disadvantaged and historically marginalized communities, fostering interdisciplinary biomedical research and bridging the gap between education and practical application in the biomedical field.

The Digital Therapeutics Real-World Evidence Framework: An Approach for Guiding Evidence-Based Digital Therapeutics Design, Development, Testing, and Monitoring.

Digital therapeutics (DTx) are a promising way to provide safe, effective, accessible, sustainable, scalable, and equitable approaches to advance individual and population health. However, developing and deploying DTx is inherently complex in that DTx includes multiple interacting components, such as tools to support activities like medication adherence, health behavior goal-setting or self-monitoring, and algorithms that adapt the provision of these according to individual needs that may change over time. While myriad frameworks exist for different phases of DTx development, no single framework exists to guide evidence production for DTx across its full life cycle, from initial DTx development to long-term use. To fill this gap, we propose the DTx real-world evidence (RWE) framework as a pragmatic, iterative, milestone-driven approach for developing DTx. The DTx RWE framework is derived from the 4-phase development model used for behavioral interventions, but it includes key adaptations that are specific to the unique characteristics of DTx. To ensure the highest level of fidelity to the needs of users, the framework also incorporates real-world data (RWD) across the entire life cycle of DTx development and use. The DTx RWE framework is intended for any group interested in developing and deploying DTx in real-world contexts, including those in industry, health care, public health, and academia. Moreover, entities that fund research that supports the development of DTx and agencies that regulate DTx might find the DTx RWE framework useful as they endeavor to improve how DTxcan advance individual and population health.

Evaluating Users' Experiences of a Child Multimodal Wearable Device: Mixed Methods Approach.

BACKGROUND: Wearable devices permit the continuous, unobtrusive collection of data from children in their natural environments and can transform our understanding of child development. Although the use of wearable devices has begun to emerge in research involving children, few studies have considered families' experiences and perspectives of participating in research of this kind. OBJECTIVE: Through a mixed methods approach, we assessed parents' and children's experiences of using a new wearable device in the home environment. The wearable device was designed specifically for use with infants and young children, and it integrates audio, electrocardiogram, and motion sensors. METHODS: In study 1, semistructured phone interviews were conducted with 42 parents of children aged 1 month to 9.5 years who completed 2 day-long recordings using the device, which the children wore on a specially designed shirt. In study 2, a total of 110 parents of children aged 2 months to 5.5 years responded to a questionnaire assessing their experience of completing 3 day-long device recordings in the home. Guided by the Digital Health Checklist, we assessed parental responses from both studies in relation to the following three key domains: (1) access and usability, (2) privacy, and (3) risks and benefits. RESULTS: In study 1, most parents viewed the device as easy to use and safe and remote visits as convenient. Parents' views on privacy related to the audio recordings were more varied. The use of machine learning algorithms (vs human annotators) in the analysis of the audio data, the ability to stop recordings at any time, and the view that the recordings reflected ordinary family life were some reasons cited by parents who expressed minimal, if any, privacy concerns. Varied risks and benefits were also reported, including perceived child comfort or discomfort, the need to adjust routines to accommodate the study, the understanding gained from the study procedures, and the parent's and child's enjoyment of study participation. In study 2, parents' ratings on 5 close-ended items yielded a similar pattern of findings. Compared with a "neutral" rating, parents were significantly more likely to agree that (1) device instructions were helpful and clear (t109=-45.98; P<.001), (2) they felt comfortable putting the device on their child (t109=-22.22; P<.001), and (3) they felt their child was safe while wearing the device (t109=-34.48; P<.001). They were also less likely to worry about the audio recordings gathered by the device (t108=6.14; P<.001), whereas parents' rating of the burden of the study procedures did not differ significantly from a "neutral" rating (t109=-0.16; P=.87). CONCLUSIONS: On the basis of parents' feedback, several concrete changes can be implemented to improve this new wearable platform and, ultimately, parents' and children's experiences of using child wearable devices in the home setting.

Ethical, legal, and social implications of digital health: A needs assessment from the Society of Behavioral Medicine to inform capacity building for behavioral scientists.

The ethical, legal, and social implications (ELSIs) of digital health are important when researchers and practitioners are using technology to collect, process, or store personal health data. Evidence underscores a strong need for digital health ELSI training, yet little is known about the specific ELSI topic areas that researchers and practitioners would most benefit from learning. To identify ELSI educational needs, a needs assessment survey was administered to the members of the Society of Behavioral Medicine (SBM). We sought to identify areas of ELSI proficiency and training need, and also evaluate interest and expertise in ELSI topics by career level and prior ELSI training history. The 14-item survey distributed to SBM members utilized the Digital Health Checklist tool (see recode.health/tools) and included items drawn from the four-domain framework: data management, access and usability, privacy and risk to benefit assessment. Respondents (N = 66) were majority faculty (74.2%) from psychology or public health. Only 39.4% reported receiving "formal" ELSI training. ELSI topics of greatest interest included practices that supported participant engagement, and dissemination and implementation of digital tools beyond the research setting. Respondents were least experienced in managing "bystander" data, having discussions about ELSIs, and reviewing terms of service agreements and privacy policies with participants and patients. There is opportunity for formalized ELSI training across career levels. Findings serve as an evidence base for continuous and ongoing evaluation of ELSI training needs to support scientists in conducting ethical and impactful digital health research.

New technologies are increasingly used in research and practice, which introduce new ethical, legal, and social implications (ELSIs). While there are scholars who study ELSIs in research, it is important that behavioral scientists have ELSI training in order to identify and mitigate possible harms and maximize benefits among their patients/participants, particularly when using technologies that collect personal health information. ELSI training opportunities are limited and, because ELSI is a broad complicated field, we know very little about the specific topics that researchers/practitioners would benefit from learning. To understand ELSI training needs specific to the field of digital health, we asked the members of the Society of Behavioral Medicine, a multidisciplinary nonprofit organization, to tell us about which ELSI areas they are most interested in. We found that 39.4% of members received formal ELSI training. Members were most interested in using technology to help patients/participants stay engaged in their treatments, and developing technologies that can be used outside of research (in the "real world"). Members were least experienced in reviewing terms of service/privacy policies and handling information collected from non-patient/participants (people in the backgrounds of voice recordings/videos). Training interests differed by career level (faculty vs. students), and so future ELSI trainings could be more beneficial if they were mindful of prior experiences.

Digital exposure notification tools: A global landscape analysis.

BACKGROUND: As the COVID-19 pandemic continues, digital exposure notification systems are increasingly used to support traditional contact tracing and other preventive strategies. Likewise, a plethora of COVID-19 mobile applications have emerged. Objective: To characterize the global landscape of pandemic related mobile applications, including digital exposure notification and contact tracing tools. DATA SOURCES AND METHODS: The following queries were entered into the Google search engine: "(*country name* COVID app) OR (COVID app *country name*) OR (COVID app *country name*+) OR (*country name*+ COVID app)". The App Store, Google Play, and official government websites were then accessed to collect descriptive data for each application. Descriptive data were qualified and quantified using standard methods. COVID-19 Exposure Notification Systems (ENS) and non-Exposure Notification products were categorized and summarized to provide a global landscape review. RESULTS: Our search resulted in a global count of 224 COVID-19 mobile applications, in 127 countries. Of these 224 apps, 128 supported exposure notification, with 75 employing the Google Apple Exposure Notification (GAEN) application programming interface (API). Of the 75 apps using the GAEN API, 15 apps were developed using Exposure Notification Express, a GAEN turnkey solution. COVID-19 applications that did not include exposure notifications (n = 96) focused on COVID-19 Self-Assessment (35·4%), COVID-19 Statistics and Information (32·3%), and COVID-19 Health Advice (29·2%). CONCLUSIONS: The digital response to COVID-19 generated diverse and novel solutions to support non-pharmacologic public health interventions. More research is needed to evaluate the extent to which these services and strategies were useful in reducing viral transmission.

Exposure notification system activity as a leading indicator for SARS-COV-2 caseload forecasting.

PURPOSE: Digital methods to augment traditional contact tracing approaches were developed and deployed globally during the COVID-19 pandemic. These "Exposure Notification (EN)" systems present new opportunities to support public health interventions. To date, there have been attempts to model the impact of such systems, yet no reports have explored the value of real-time system data for predictive epidemiological modeling. METHODS: We investigated the potential to short-term forecast COVID-19 caseloads using data from California's implementation of the Google Apple Exposure Notification (GAEN) platform, branded as CA Notify. CA Notify is a digital public health intervention leveraging resident's smartphones for anonymous EN. We extended a published statistical model that uses prior case counts to investigate the possibility of predicting short-term future case counts and then added EN activity to test for improved forecast performance. Additional predictive value was assessed by comparing the pandemic forecasting models with and without EN activity to the actual reported caseloads from 1-7 days in the future. RESULTS: Observation of time series presents noticeable evidence for temporal association of system activity and caseloads. Incorporating earlier ENs in our model improved prediction of the caseload counts. Using Bayesian inference, we found nonzero influence of EN terms with probability one. Furthermore, we found a reduction in both the mean absolute percentage error and the mean squared prediction error, the latter of at least 5% and up to 32% when using ENs over the model without. CONCLUSIONS: This preliminary investigation suggests smartphone based ENs can significantly improve the accuracy of short-term forecasting. These predictive models can be readily deployed as local early warning systems to triage resources and interventions.

"Click and mortar" opportunities for digitization and consumerism in trials.

BACKGROUND: Digitization (using novel digital tools and strategies) and consumerism (taking a consumer-oriented approach) are increasingly commonplace in clinical trials, but the implications of these changes are not well described. METHODS: We assembled a group of trial experts from academia, industry, non-profit, and government to discuss implications of this changing trial landscape and provide guidance. RESULTS: Digitization and consumerism can increase the volume and diversity of trial participants and expedite recruitment. However, downstream bottlenecks, challenges with retention, and serious issues with equity, ethics, and security can result. A "click and mortar" approach, combining approaches from novel and traditional trials with the thoughtful use of technology, may optimally balance opportunities and challenges facing many trials. CONCLUSION: We offer expert guidance and three "click and mortar" approaches to digital, consumer-oriented trials. More guidance and research are needed to navigate the associated opportunities and challenges.

Evaluating Mobile Apps Targeting Older Adults: Descriptive Study.

BACKGROUND: Smartphone use has increased dramatically and, in parallel, a market for mobile apps, including health apps, has emerged. The business model of targeted mobile app advertisements allows for the collection of personal and potentially sensitive information, often without user knowledge. Older adults comprise a rapidly growing demographic that is potentially vulnerable to exploitation by those accessing data collected via these apps. OBJECTIVE: This research examined apps that claimed to be useful to older adults with a goal of (1) classifying the functionality of each app, (2) identifying whether a privacy policy existed and was accessible, and (3) evaluating evidence that could support claims of value to older adults. METHODS: An environmental scan was conducted using the Google search engine and typing "apps for older adults." The first 25 sites that this search returned comprised the primary data for this study. Data were organized by descriptive features of purpose (eg, health, finance, and utility), the existence of an electronically accessible privacy policy, price, and evidence supporting each recommended mobile app. RESULTS: A total of 133 mobile apps were identified and promoted as being the best "apps for older adults." Of these 133 mobile apps, 83% (n=110) included a privacy policy. Fewer apps designated in the "medical" category included a privacy policy than those classified otherwise. CONCLUSIONS: The results suggest that most mobile apps targeting older adults include a privacy policy. Research is needed to determine whether these privacy policies are readable, succinct, and incorporate accessible data use and sharing practices to mitigate potential risks, particularly when collecting potentially sensitive health information.

The Independent Walking for Brain Health Intervention for Older Adults: Protocol for a Pilot Randomized Controlled Trial.

BACKGROUND: Extensive research suggests that physical activity (PA) is important for brain and cognitive health and may help to delay or prevent Alzheimer's disease and related dementias. Most PA interventions designed to improve brain health in older adults have been conducted in laboratory, gym, or group settings that require extensive resources and travel to the study site or group sessions. Research is needed to develop novel interventions that leverage mobile health (mHealth) technologies to help older adults increase their engagement in PA in free-living environments, reducing participant burden and increasing generalizability of research findings. Moreover, promoting engagement in moderate-to-vigorous PA (MVPA) may be most beneficial to brain health; thus, using mHealth to help older adults increase time spent in MVPA in free-living environments may help to offset the burden of Alzheimer's disease and related dementias and improve quality of life in older age. OBJECTIVE: We developed a novel PA intervention that leverages mHealth to help older adults achieve more minutes of MVPA independently. This pilot study was a 12-week randomized controlled trial to investigate the feasibility of providing just-in-time (JIT) feedback about PA intensity during free-living exercise sessions to help older adults meet current PA recommendations (150 minutes per week of MVPA). METHODS: Participants were eligible if they were cognitively healthy English speakers aged between 65 and 80 years without major cardiovascular, neurologic, or mental health conditions; could ambulate independently; and undergo magnetic resonance imaging. Enrollment occurred from October 2017 to March 2020. Participants randomized to the PA condition received an individualized exercise prescription and an mHealth device that provided heart rate-based JIT feedback on PA intensity, allowing them to adjust their behavior in real time to maintain MVPA during exercise sessions. Participants assigned to the healthy aging education condition received a reading prescription consisting of healthy aging topics and completed weekly quizzes based on the materials. RESULTS: In total, 44 participants were randomized to the intervention. A follow-up manuscript will describe the results of the intervention as well as discuss screening, recruitment, adverse events, and participants' opinions regarding their participation in the intervention. CONCLUSIONS: The long-term goal of this intervention is to better understand how MVPA affects brain and cognitive health in the real world and extend laboratory findings to everyday life. This pilot randomized controlled trial was conducted to determine the feasibility of using JIT heart rate zone feedback to help older adults independently increase time spent in MVPA while collecting data on the plausible mechanisms of change (frontal and medial temporal cerebral blood flow and cardiorespiratory fitness) that may affect cognition (memory and executive function) to help refine a planned stage 2 behavioral trial. TRIAL REGISTRATION: ClinicalTrials.gov NCT03058146; https://clinicaltrials.gov/ct2/show/NCT03058146. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/42980.

Defining Key Performance Indicators for the California COVID-19 Exposure Notification System (CA Notify).

OBJECTIVES: Toward common methods for system monitoring and evaluation, we proposed a key performance indicator framework and discussed lessons learned while implementing a statewide exposure notification (EN) system in California during the COVID-19 epidemic. MATERIALS AND METHODS: California deployed the Google Apple Exposure Notification framework, branded CA Notify, on December 10, 2020, to supplement traditional COVID-19 contact tracing programs. For system evaluation, we defined 6 key performance indicators: adoption, retention, sharing of unique codes, identification of potential contacts, behavior change, and impact. We aggregated and analyzed data from December 10, 2020, to July 1, 2021, in compliance with the CA Notify privacy policy. RESULTS: We estimated CA Notify adoption at nearly 11 million smartphone activations during the study period. Among 1 654 201 CA Notify users who received a positive test result for SARS-CoV-2, 446 634 (27%) shared their unique code, leading to ENs for other CA Notify users who were in close proximity to the SARS-CoV-2-positive individual. We identified at least 122 970 CA Notify users as contacts through this process. Contact identification occurred a median of 4 days after symptom onset or specimen collection date of the user who received a positive test result for SARS-CoV-2. PRACTICE IMPLICATIONS: Smartphone-based EN systems are promising new tools to supplement traditional contact tracing and public health interventions, particularly when efficient scaling is not feasible for other approaches. Methods to collect and interpret appropriate measures of system performance must be refined while maintaining trust and privacy.

Perceptions of Pharmacy Graduate Students Toward Research Ethics Education: A Cross-Sectional Study from a Developing Country.

Despite the potential value of graduate-level research ethics training, most Middle East countries, including Jordan, do not routinely offer formal research ethics training. In students enrolled in Jordanian master's level graduate program in pharmacy, the current study assessed: 1- differences in pre- and post-enrollment exposure to research ethics core themes, 2- whether this exposure was through a formal course or in an informal setting, and 3- student attitudes towards research ethics education and the need for integrating a dedicated research ethics course into pharmacy graduate programs. A 12-item on-line survey was developed by the authors and disseminated to a convenience sample of current and former master-level pharmacy students in Jordan. A total of 61 eligible respondents completed the survey. A minority of respondents (38%) acknowledged receiving research ethics training prior to enrollment into a postgraduate pharmacy program with nearly half (16%) describing this training as informal. In comparison, a larger percentage of the total respondents (56%) had received research ethics training during their postgraduate program enrollment, with nearly half of those (25%) indicating that this training was informal. A majority of respondents reported a strong need for integrating a formal research ethics course into postgraduate pharmacy curriculum (90%) to support their research training and thesis writing (89%). Overall, the study revealed a notable lack of research ethics education for graduate-level pharmacy students in Jordan.

Optimizing Ethics Engagement in Research: Learning from the Ethical Complexities of Studying Opioid Use in Pregnancy.

Research on opioid use in pregnancy is critically important to understand how the opioid epidemic has affected a generation of children, but also raises significant ethical and legal challenges. Embedded ethicists can help to fill the gaps in ethics oversight for such research, but further guidance is needed to help strike the balance between integration and independence.

Social Determinants of Health Data Availability for Patients with Eye Conditions.

Purpose: To quantify and characterize social determinants of health (SDoH) data coverage using single-center electronic health records (EHRs) and the National Institutes of Health All of Us research program. Design: Retrospective cohort study from June 2014 through June 2021. Participants: Adults 18 years of age or older with a diagnosis of diabetic retinopathy, glaucoma, cataracts, or age-related macular degeneration. Methods: For All of Us, research participants completed online survey forms as part of a nationwide prospective cohort study. In local EHRs, patients were selected based on diagnosis codes. Main Outcome Measures: Social determinants of health data coverage, characterized by the proportion of each disease cohort with available data regarding demographics and socioeconomic factors. Results: In All of Us, we identified 23 806 unique adult patients, of whom 2246 had a diagnosis of diabetic retinopathy, 13 448 had a diagnosis of glaucoma, 6634 had a diagnosis of cataracts, and 1478 had a diagnosis of age-related macular degeneration. Survey completion rates were high (99.5%-100%) across all cohorts for demographic information, overall health, income, education, and lifestyle. However, health care access (12.7%-29.4%), housing (0.7%-1.1%), social isolation (0.2%-0.3%), and food security (0-0.1%) showed significantly lower response rates. In local EHRs, we identified 80 548 adult patients, of whom 6616 had a diagnosis of diabetic retinopathy, 26 793 had a diagnosis of glaucoma, 40 427 had a diagnosis of cataracts, and 6712 had a diagnosis of age-related macular degeneration. High data coverage was found across all cohorts for variables related to tobacco use (82.84%-89.07%), alcohol use (77.45%-83.66%), and intravenous drug use (84.76%-93.14%). However, low data coverage (< 50% completion) was found for all other variables, including education, finances, social isolation, stress, physical activity, food insecurity, and transportation. We used chi-square testing to assess whether the data coverage varied across different disease cohorts and found that all fields varied significantly (P < 0.001). Conclusions: The limited and highly variable data coverage in both local EHRs and All of Us highlights the need for researchers and providers to develop SDoH data collection strategies and to assemble complete datasets.

Automated Analysis of Drawing Process to Estimate Global Cognition in Older Adults: Preliminary International Validation on the US and Japan Data Sets.

BACKGROUND: With the aging of populations worldwide, early detection of cognitive impairments has become a research and clinical priority, particularly to enable preventive intervention for dementia. Automated analysis of the drawing process has been studied as a promising means for lightweight, self-administered cognitive assessment. However, this approach has not been sufficiently tested for its applicability across populations. OBJECTIVE: The aim of this study was to evaluate the applicability of automated analysis of the drawing process for estimating global cognition in community-dwelling older adults across populations in different nations. METHODS: We collected drawing data with a digital tablet, along with Montreal Cognitive Assessment (MoCA) scores for assessment of global cognition, from 92 community-dwelling older adults in the United States and Japan. We automatically extracted 6 drawing features that characterize the drawing process in terms of the drawing speed, pauses between drawings, pen pressure, and pen inclinations. We then investigated the association between the drawing features and MoCA scores through correlation and machine learning-based regression analyses. RESULTS: We found that, with low MoCA scores, there tended to be higher variability in the drawing speed, a higher pause:drawing duration ratio, and lower variability in the pen's horizontal inclination in both the US and Japan data sets. A machine learning model that used drawing features to estimate MoCA scores demonstrated its capability to generalize from the US dataset to the Japan dataset (R2=0.35; permutation test, P<.001). CONCLUSIONS: This study presents initial empirical evidence of the capability of automated analysis of the drawing process as an estimator of global cognition that is applicable across populations. Our results suggest that such automated analysis may enable the development of a practical tool for international use in self-administered, automated cognitive assessment.

Are Mental Health Apps Adequately Equipped to Handle Users in Crisis?

Background: Mental health (MH) apps are growing in popularity. While MH apps may be helpful, less is known about how crises such as suicidal ideation are addressed in apps. Aims: We examined the proportion of MH apps that contained language mentioning suicide or suicidal ideation and how apps communicated these policies and directed users to MH resources through app content, terms of services, and privacy policies. Method: We chose apps using an Internet search of "top mental health apps," similar to how a user might find an app, and extracted information about how crisis language was presented in these apps. Results: We found that crisis language was inconsistent among apps. Overall, 35% of apps provided crisis-specific resources in their app interface and 10.5% contained crisis language in terms of service or privacy policies. Limitations: This study employed a nonsystematic approach to sampling apps, and therefore the findings may not broadly represent apps for MH. Conclusion: To address the inconsistency of crisis resources, crisis language should be included as part of app evaluation frameworks, and internationally accessible, vetted resources should be provided to app users.

Do Words Matter? Detecting Social Isolation and Loneliness in Older Adults Using Natural Language Processing.

Introduction: Social isolation and loneliness (SI/L) are growing problems with serious health implications for older adults, especially in light of the COVID-19 pandemic. We examined transcripts from semi-structured interviews with 97 older adults (mean age 83 years) to identify linguistic features of SI/L. Methods: Natural Language Processing (NLP) methods were used to identify relevant interview segments (responses to specific questions), extract the type and number of social contacts and linguistic features such as sentiment, parts-of-speech, and syntactic complexity. We examined: (1) associations of NLP-derived assessments of social relationships and linguistic features with validated self-report assessments of social support and loneliness; and (2) important linguistic features for detecting individuals with higher level of SI/L by using machine learning (ML) models. Results: NLP-derived assessments of social relationships were associated with self-reported assessments of social support and loneliness, though these associations were stronger in women than in men. Usage of first-person plural pronouns was negatively associated with loneliness in women and positively associated with emotional support in men. ML analysis using leave-one-out methodology showed good performance (F1 = 0.73, AUC = 0.75, specificity = 0.76, and sensitivity = 0.69) of the binary classification models in detecting individuals with higher level of SI/L. Comparable performance were also observed when classifying social and emotional support measures. Using ML models, we identified several linguistic features (including use of first-person plural pronouns, sentiment, sentence complexity, and sentence similarity) that most strongly predicted scores on scales for loneliness and social support. Discussion: Linguistic data can provide unique insights into SI/L among older adults beyond scale-based assessments, though there are consistent gender differences. Future research studies that incorporate diverse linguistic features as well as other behavioral data-streams may be better able to capture the complexity of social functioning in older adults and identification of target subpopulations for future interventions. Given the novelty, use of NLP should include prospective consideration of bias, fairness, accountability, and related ethical and social implications.