Mobile toolbox (MTB) remote measures of executive function and processing speed: development and validation.

OBJECTIVE: The ability to remotely monitor cognitive skills is increasing with the ubiquity of smartphones. The Mobile Toolbox (MTB) is a new measurement system that includes measures assessing Executive Functioning (EF) and Processing Speed (PS): Arrow Matching, Shape-Color Sorting, and Number-Symbol Match. The purpose of this study was to assess their psychometric properties. METHOD: MTB measures were developed for smartphone administration based on constructs measured in the NIH Toolbox® (NIHTB). Psychometric properties of the resulting measures were evaluated in three studies with participants ages 18 to 90. In Study 1 (N = 92), participants completed MTB measures in the lab and were administered both equivalent NIH TB measures and other external measures of similar cognitive constructs. In Study 2 (N = 1,021), participants completed the equivalent NIHTB measures in the lab and then took the MTB measures on their own, remotely. In Study 3 (N = 168), participants completed MTB measures twice remotely, two weeks apart. RESULTS: All three measures exhibited very high internal consistency and strong test-retest reliability, as well as moderately high correlations with comparable NIHTB tests and moderate correlations with external measures of similar constructs. Phone operating system (iOS vs. Android) had a significant impact on performance for Arrow Matching and Shape-Color Sorting, but no impact on either validity or reliability. CONCLUSIONS: Results support the reliability and convergent validity of MTB EF and PS measures for use across the adult lifespan in remote, self-administered designs.

Intensive longitudinal assessment following index trauma to predict development of PTSD using machine learning.

There are significant challenges to identifying which individuals require intervention following exposure to trauma, and a need for strategies to identify and provide individuals at risk for developing PTSD with timely interventions. The present study seeks to identify a minimal set of trauma-related symptoms, assessed during the weeks following traumatic exposure, that can accurately predict PTSD. Participants were 2185 adults (Mean age=36.4 years; 64% women; 50% Black) presenting for emergency care following traumatic exposure. Participants received a 'flash survey' with 6-8 varying symptoms (from a pool of 26 trauma symptoms) several times per week for eight weeks following the trauma exposure (each symptom assessed ∼6 times). Features (mean, sd, last, worst, peak-end scores) from the repeatedly assessed symptoms were included as candidate variables in a CART machine learning analysis to develop a pragmatic predictive algorithm. PTSD (PCL-5 ≥38) was present for 669 (31%) participants at the 8-week follow-up. A classification tree with three splits, based on mean scores of nervousness, rehashing, and fatigue, predicted PTSD with an Area Under the Curve of 0.836. Findings suggest feasibility for a 3-item assessment protocol, delivered once per week, following traumatic exposure to assess and potentially facilitate follow-up care for those at risk.

The Use of Mobile Assessments for Monitoring Mental Health in Youth: Umbrella Review.

BACKGROUND: Improving mental health in youth is a major concern. Future approaches to monitor and intervene in youth mental health problems should rely on mobile tools that allow for the daily monitoring of mental health both actively (eg, using ecological momentary assessments [EMAs]) and passively (eg, digital phenotyping) by capturing individuals' data. OBJECTIVE: This umbrella review aims to (1) report the main characteristics of existing reviews on mental health and young people, including mobile approaches to mental health; (2) describe EMAs and trace data and the mental health conditions investigated; (3) report the main results; and (4) outline promises, limitations, and directions for future research. METHODS: A systematic literature search was carried out in 9 scientific databases (Communication & Mass Media Complete, Psychology and Behavioral Sciences Collection, PsycINFO, CINAHL, ERIC, MEDLINE, the ProQuest Sociology Database, Web of Science, and PubMed) on January 30, 2022, coupled with a hand search and updated in July 2022. We included (systematic) reviews of EMAs and trace data in the context of mental health, with a specific focus on young populations, including children, adolescents, and young adults. The quality of the included reviews was evaluated using the AMSTAR (Assessment of Multiple Systematic Reviews) checklist. RESULTS: After the screening process, 30 reviews (published between 2016 and 2022) were included in this umbrella review, of which 21 (70%) were systematic reviews and 9 (30%) were narrative reviews. The included systematic reviews focused on symptoms of depression (5/21, 24%); bipolar disorders, schizophrenia, or psychosis (6/21, 29%); general ill-being (5/21, 24%); cognitive abilities (2/21, 9.5%); well-being (1/21, 5%); personality (1/21, 5%); and suicidal thoughts (1/21, 5%). Of the 21 systematic reviews, 15 (71%) summarized studies that used mobile apps for tracing, 2 (10%) summarized studies that used them for intervention, and 4 (19%) summarized studies that used them for both intervention and tracing. Mobile tools used in the systematic reviews were smartphones only (8/21, 38%), smartphones and wearable devices (6/21, 29%), and smartphones with other tools (7/21, 33%). In total, 29% (6/21) of the systematic reviews focused on EMAs, including ecological momentary interventions; 33% (7/21) focused on trace data; and 38% (8/21) focused on both. Narrative reviews mainly focused on the discussion of issues related to digital phenotyping, existing theoretical frameworks used, new opportunities, and practical examples. CONCLUSIONS: EMAs and trace data in the context of mental health assessments and interventions are promising tools. Opportunities (eg, using mobile approaches in low- and middle-income countries, integration of multimodal data, and improving self-efficacy and self-awareness on mental health) and limitations (eg, absence of theoretical frameworks, difficulty in assessing the reliability and effectiveness of such approaches, and need to appropriately assess the quality of the studies) were further discussed. TRIAL REGISTRATION: PROSPERO CRD42022347717; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=347717.

Rapid, reliable mobile assessment of affect-related motor processing.

Mobile technologies can be used for behavioral assessments to associate changes in behavior with environmental context and its influence on mental health and disease. Research on real-time motor control with a joystick, analyzed using a computational proportion-derivative (PD) modeling approach, has shown that model parameters can be estimated with high reliability and are related both to self-reported fear and to brain structures important for affective regulation, such as the anterior cingulate cortex. Here we introduce a mobile version of this paradigm, the rapid assessment of motor processing (RAMP) paradigm, and show that it provides robust, reliable, and accessible behavioral measurements relevant to mental health. A smartphone version of a previous joystick sensorimotor task was developed in which participants control a virtual car to a stop sign and stop. A sample of 89 adults performed the task, with 66 completing a second retest session. A PD modeling approach was applied to compute Kp (drive) and Kd (damping) parameters. Both Kp and Kd exhibited high test-retest reliabilities (ICC .81 and .78, respectively). Replicating a previous finding from a different sample with the joystick version of the task, both Kp and Kd were negatively associated with self-reported fear. The RAMP paradigm, a mobile sensorimotor assessment, can be used to assess drive and damping during motor control, which is robustly associated with subjective affect. This paradigm could be useful for examining dynamic contextual modulation of affect-related processing, which could improve assessment of the effects of interventions for psychiatric disorders in a real-world context.

Development of a Mobile Assessment Tool for Understanding Social Comparison Processes Among Individuals With Schizophrenia: Two-Phase Survey Study.

BACKGROUND: Digital tools may help to address social deficits in schizophrenia, particularly those that engage social comparison processes (ie, evaluating oneself relative to others). Yet, little is known about social comparison processes in schizophrenia or how best to capture between- versus within-person variability, which is critical to engaging comparisons in digital interventions. OBJECTIVE: The goals of this pilot study were to (1) better understand affective responses to social comparisons among individuals with schizophrenia, relative to healthy controls, using a validated global self-report measure; and (2) test a new brief, mobile assessment of affective responses to social comparison among individuals with schizophrenia, relative to the full measure. This study was conducted in 2 phases. METHODS: We first compared self-reported affective responses to social comparisons between individuals with schizophrenia (n=39) and healthy controls (n=38) using a traditional self-report measure, at 2 time points. We examined the temporal stability in responses and differences between groups. We then evaluated the performance of brief, mobile assessment of comparison responses among individuals with schizophrenia, completed over 12 weeks (n=31). RESULTS: Individuals with schizophrenia showed greater variability in affective responses to social comparison than controls on traditional measures and completed an average of 7.46 mobile assessments over 12 weeks. Mobile assessments captured within-person variability in affective responses in the natural environment (intraclass correlation coefficients of 0.40-0.60). Average scores for mobile assessments were positively correlated with responses to traditional measures. CONCLUSIONS: Affective responses to social comparison vary both between and within individuals with schizophrenia and capturing this variability via smartphone surveys shows some evidence of feasibility. As affective variability is a potential indicator of poor outcomes among individuals with mental health conditions, in the future, a brief, mobile assessment of affective responses to social comparisons may be useful for screening among individuals with schizophrenia. Further research on this process is needed to identify when specific comparison messaging may be most effective in digital interventions and could suggest new therapeutic targets for illnesses such as schizophrenia.

Reliability and Minimal Detectable Change for a Smartphone-Based Motor-Cognitive Assessment: Implications for Concussion Management.

Our purpose was to investigate the reliability and minimal detectable change characteristics of a smartphone-based assessment of single- and dual-task gait and cognitive performance. Uninjured adolescent athletes (n = 17; mean age = 16.6, SD = 1.3 y; 47% female) completed assessments initially and again 4 weeks later. The authors collected data via an automated smartphone-based application while participants completed a series of tasks under (1) single-task cognitive, (2) single-task gait, and (3) dual-task cognitive-gait conditions. The cognitive task was a series of continuous auditory Stroop cues. Average gait speed was consistent between testing sessions in single-task (0.98, SD = 0.21 vs 0.96, SD = 0.19 m/s; P = .60; r = .89) and dual-task (0.92, SD = 0.22 vs 0.89, SD = 0.22 m/s; P = .37; r = .88) conditions. Response accuracy was moderately consistent between assessments in single-task standing (82.3% accurate, SD = 17.9% vs 84.6% accurate, SD = 20.1%; P = .64; r = .52) and dual-task gait (89.4% accurate, SD = 15.9% vs 85.8% accurate, SD = 20.2%; P = .23; r = .81) conditions. Our results indicate automated motor-cognitive dual-task outcomes obtained within a smartphone-based assessment are consistent across a 1-month period. Further research is required to understand how this assessment performs in the setting of sport-related concussion. Given the relative reliability of values obtained, a smartphone-based evaluation may be considered for use to evaluate changes across time among adolescents, postconcussion.

Effectiveness and Utility of Mobile Device Assessment of Subjective Craving during Residential Opioid Dependence Treatment.

Background: Craving is a dynamic state that is both theoretically and empirically linked to relapse in addiction. Static measures cannot adequately capture the dynamic nature of craving, and research has shown that these measures are limited in their capacity to link craving to treatment outcomes. Methods: The current study reports on assessments of craving collected 4x-day across 12 days from 73 patients in residential treatment for opioid dependence. Analyses investigated whether the within-person assessments yielded expected across- and within-day variability, whether levels of craving changed across and within days, and, finally, whether individual differences in craving variability predicted post-residential treatment relapse. Results: Preliminary analyses found acceptable levels of data entry compliance and reliability. Consistent with expectations, craving varied both between (46%) and within persons, with most within-person variance (over 40%) existing within days. Other patterns that emerged indicated that, on average, craving declined across the 12-days of assessment, and was generally strongest at mid-day. Analyses also found that patients' person-level craving variability predicted post-treatment relapse, above and beyond their mean levels of craving. Conclusion: Analyses support the reliability, sensitivity, and potential utility of the 4x-day, 12-day assessment protocol for measuring craving during residential treatment.

Artificial Intelligence-Empowered Mobilization of Assessments in COVID-19-like Pandemics: A Case Study for Early Flattening of the Curve.

The global outbreak of the Coronavirus Disease 2019 (COVID-19) pandemic has uncovered the fragility of healthcare and public health preparedness and planning against epidemics/pandemics. In addition to the medical practice for treatment and immunization, it is vital to have a thorough understanding of community spread phenomena as related research reports 17.9-30.8% confirmed cases to remain asymptomatic. Therefore, an effective assessment strategy is vital to maximize tested population in a short amount of time. This article proposes an Artificial Intelligence (AI)-driven mobilization strategy for mobile assessment agents for epidemics/pandemics. To this end, a self-organizing feature map (SOFM) is trained by using data acquired from past mobile crowdsensing (MCS) campaigns to model mobility patterns of individuals in multiple districts of a city so to maximize the assessed population with minimum agents in the shortest possible time. Through simulation results for a real street map on a mobile crowdsensing simulator and considering the worst case analysis, it is shown that on the 15th day following the first confirmed case in the city under the risk of community spread, AI-enabled mobilization of assessment centers can reduce the unassessed population size down to one fourth of the unassessed population under the case when assessment agents are randomly deployed over the entire city.

Feasibility and acceptability of using an IVRS to assess decision making about sun protection.

OBJECTIVE: We developed an interactive voice response system (IVRS), an automated telephone survey technology, to assess real-time decision making about sun protection. We examined the feasibility and acceptability of IVRS in this electronic health (eHealth) context. METHODS: Melanoma patients who underwent surgery referred their first-degree relatives (FDRs) for participation. Eligible FDRs were contacted twice daily (12:30 pm; 5:00 pm) over 14 consecutive days via IVRS to complete a survey about their sun protection behaviors and decisions about those behaviors. RESULTS: Of the 81 eligible FDRs, 69 (85%) consented to the study, and 53 (77%) completed the study. We assessed adherence with the IVRS via the number and pattern of missing survey items across all answered IVRS calls. About 80% of scheduled IVRS calls were answered (1316/1652). Most surveys (93%) of the IVRS-answered calls were completed. To examine acceptability, we analyzed the program satisfaction survey data collected at the end of the study. Most participants viewed the IVRS to be highly acceptable and easy to use. CONCLUSIONS: These findings illustrate that use of real-time IVRS data collection regarding sun protection decision making is feasible and acceptable to higher-risk research participants and could thus be used with time and location-sensitive eHealth support to enhance sun protection decision making.

What Do People With Schizophrenia Do All Day? Ecological Momentary Assessment of Real-World Functioning in Schizophrenia.

Schizophrenia is a major cause of disability worldwide. As new treatments for functioning are tested, the need grows to demonstrate real-world functioning gains. Ecological momentary assessment (EMA) may provide a more ecologically valid measure of functioning. In this study, smartphone-based EMA was used to signal participants with schizophrenia (N = 100) and controls (N = 71) 7 times a day for 7 days to respond to brief questionnaires about social interactions and functioning behaviors. Excellent adherence was found, with both groups completing an average of 85% of surveys and only 3% of participants with schizophrenia excluded for poor adherence. Four-week test-retest reliability was high (r = .83 for total productive behaviors). Relative to controls, participants with schizophrenia reported significantly less total productive activity (d = 1.2), fewer social interactions (d = 0.3), more nonproductive behaviors (d = 1.0; watching TV, resting), and more time at home (d = 0.8). Within the schizophrenia group, participants living independently showed better functioning on EMA relative to participants in supported housing (d = 0.8) and participants engaged in vocational activities showed better functioning than individuals not engaged in vocational activities (d = 0.55). Modest correlations were found between EMA and an in-lab self-report measure of functioning activities performed in the community, but not between EMA and measures of functional capacity or potential. This study demonstrated the feasibility, sensitivity reliability, and validity of EMA methods to assess functioning in schizophrenia. EMA provides a much-needed measure of what individuals with schizophrenia are actually doing in real-world contexts. These results also suggest that there may be important disjunctions between indices of abilities and actual real-world functioning.

If at first you don't succeed adoption of iPad marking for high-stakes assessments.

Large-scale interview and simulation-based assessments such as objective structured clinical examinations (OSCEs) and multiple mini interviews (MMIs) are logistically complex to administer, generate large volumes of assessment data, and are strong candidates for the adoption of computer-based marking systems. Adoption of new technologies can be challenging, and technical failures, which are relatively commonplace, can delay and/or create resistance to ongoing implementation.This paper reports on the adoption process of an electronic marking system for OSCEs and MMIs following an unsuccessful initial trial. It describes how, after the initial setback, a staged implementation, progressing from small to larger-scale assessments, single to multiple assessment types, and lower to higher stakes assessments, was used to successfully adopt and embed iPad-based marking within our medical school.Critical factors in the success of this approach included thorough appraisal and selection of technologies, rigorous assurance of system reliability and security, constant review and refinement, and careful attention to implementation and end-user training. Engagement of stakeholders is also crucial, especially in the case of previous failures or setbacks. The early identification and recruitment of staff to provide specific expertise and support for adoption of an innovation helps to facilitate this process with four key roles proposed; those of innovation advocate, champion, expert and sponsor.

Mobile technology for mental health assessment.

Assessment and outcome monitoring are critical for the effective detection and treatment of mental illness. Traditional methods of capturing social, functional, and behavioral data are limited to the information that patients report back to their health care provider at selected points in time. As a result, these data are not accurate accounts of day-to-day functioning, as they are often influenced by biases in self-report. Mobile technology (mobile applications on smartphones, activity bracelets) has the potential to overcome such problems with traditional assessment and provide information about patient symptoms, behavior, and functioning in real time. Although the use of sensors and apps are widespread, several questions remain in the field regarding the reliability of off-the-shelf apps and sensors, use of these tools by consumers, and provider use of these data in clinical decision-making.

La evaluación y la supervisión de los resultados son esenciales para la detección efectiva y el tratamiento de la enfermedad mental. Los métodos tradicionales de captación de datos sociales, funcionales y conductuales están limitados a la información que los pacientes reportan retrospectivamente a sus proveedores de atención de salud en momentos seleccionados en el tiempo. Como resultado, esta información no constituye datos precisos del funcionamiento día a día, ya que a menudo están influenciados por sesgos en el auto-informe. La tecnología móvil (como aplicaciones móviles en teléfonos inteligentes, braceletes de actividad) tiene el potencial de superar los problemas de la evaluación tradicional y aporta información acerca de los síntomas, conductas y funcionamiento de los pacientes en tiempo real. Aunque el empleo de estos sensores y aplicaciones está muy extendido, aun persisten algunas preguntas en este campo en cuanto a la fiabilidad de las aplicaciones y sensores comerciales, el uso de estas herramientas por los consumidores y el empleo por los proveedores de esta información en la toma de decisiones clínicas.

L'évaluation et la surveillance de l'évolution sont essentielles pour un diagnostic et un traitement efficaces de la maladie mentale. Les méthodes traditionnellement utilisées pour recueillir les données comportementales, fonctionnelles et sociales sont limitées aux informations rapportées ponctuellement par les patients à leur médecin traitant. Ces données ne reflètent donc pas exactement le fonctionnement quotidien car elles sont souvent biaisées par l'auto-évaluation. La technologie mobile (applications pour smartphones, bracelets de contrôle d'activité) est capable de surmonter ces difficultés de l'évaluation traditionnelle et de fournir en temps réel des informations sur les symptômes des patients, leur comportement et leur fonctionnement. L'utilisation de capteurs et d'applications est répandue mais plusieurs questions restent en suspens en ce qui concerne la fiabilité des applications et capteurs standard, leur utilisation par les consommateurs et l'usage de ces données dans la prise de décision médicale.

Development and Verification of a Mobile Shelter Assessment System "Rapid Assessment System of Evacuation Center Condition Featuring Gonryo and Miyagi (RASECC-GM)" for Major Disasters.

UNLABELLED: Introduction There were 5,385 deceased and 710 missing in the Ishinomaki medical zone following the Great East Japan Earthquake that occurred in Japan on March 11, 2011. The Ishinomaki Zone Joint Relief Team (IZJRT) was formed to unify the relief teams of all organizations joining in support of the Ishinomaki area. The IZJRT expanded relief activity as they continued to manually collect and analyze assessments of essential information for maintaining health in all 328 shelters using a paper-type survey. However, the IZJRT spent an enormous amount of time and effort entering and analyzing these data because the work was vastly complex. Therefore, an assessment system must be developed that can tabulate shelter assessment data correctly and efficiently. The objective of this report was to describe the development and verification of a system to rapidly assess evacuation centers in preparation for the next major disaster. Report Based on experiences with the complex work during the disaster, software called the "Rapid Assessment System of Evacuation Center Condition featuring Gonryo and Miyagi" (RASECC-GM) was developed to enter, tabulate, and manage the shelter assessment data. Further, a verification test was conducted during a large-scale Self-Defense Force (SDF) training exercise to confirm its feasibility, usability, and accuracy. The RASECC-GM comprises three screens: (1) the "Data Entry screen," allowing for quick entry on tablet devices of 19 assessment items, including shelter administrator, living and sanitary conditions, and a tally of the injured and sick; (2) the "Relief Team/Shelter Management screen," for registering information on relief teams and shelters; and (3) the "Data Tabulation screen," which allows tabulation of the data entered for each shelter, as well as viewing and sorting from a disaster headquarters' computer. During the verification test, data of mock shelters entered online were tabulated quickly and accurately on a mock disaster headquarters' computer. Likewise, data entered offline also were tabulated quickly on the mock disaster headquarters' computer when the tablet device was moved into an online environment. CONCLUSIONS: The RASECC-GM, a system for rapidly assessing the condition of evacuation centers, was developed. Tests verify that users of the system would be able to easily, quickly, and accurately assess vast quantities of data from multiple shelters in a major disaster and immediately manage the inputted data at the disaster headquarters. Ishii T , Nakayama M , Abe M , Takayama S , Kamei T , Abe Y , Yamadera J , Amito K , Morino K . Development and verification of a mobile shelter assessment system "Rapid Assessment System of Evacuation Center Condition featuring Gonryo and Miyagi (RASECC-GM)" for major disasters. Prehosp Disaster Med. 2016;31(5):539-546.

Ecological momentary assessment versus standard assessment instruments for measuring mindfulness, depressed mood, and anxiety among older adults.

As mobile data capture tools for patient-reported outcomes proliferate in clinical research, a key dimension of measure performance is sensitivity to change. This study compared performance of patient-reported measures of mindfulness, depression, and anxiety symptoms using traditional paper-and-pencil forms versus real-time, ambulatory measurement of symptoms via ecological momentary assessment (EMA). Sixty-seven emotionally distressed older adults completed paper-and-pencil measures of mindfulness, depression, and anxiety along with two weeks of identical items reported during ambulatory monitoring via EMA before and after participation in a randomized trial of Mindfulness-Based Stress Reduction (MBSR) or a health education intervention. We calculated effect sizes for these measures across both measurement approaches and estimated the Number-Needed-to-Treat (NNT) in both measurement conditions. Study outcomes greatly differed depending on which measurement method was used. When EMA was used to measure clinical symptoms, older adults who participated in the MBSR intervention had significantly higher mindfulness and significantly lower depression and anxiety than participants in the health education intervention at post-treatment. However, these significant changes in symptoms were not found when outcomes were measured with paper-and-pencil measures. The NNT for mindfulness and depression measures administered through EMA were approximately 25-50% lower than NNTs derived from paper-and-pencil administration. Sensitivity to change in anxiety was similar across administration modes. In conclusion, EMA measures of depression and mindfulness substantially outperformed paper-and-pencil measures with the same items. The additional resources associated with EMA in clinical trials would seem to be offset by its greater sensitivity to detect change in key outcome variables.