Development and validation of an episodic memory measure in the Mobile Toolbox (MTB): Arranging Pictures.

INTRODUCTION: Arranging Pictures is a new episodic memory test based on the NIH Toolbox (NIHTB) Picture Sequence Memory measure and optimized for self-administration on a personal smartphone within the Mobile Toolbox (MTB). We describe evidence from three distinct validation studies. METHOD: In Study 1, 92 participants self-administered Arranging Pictures on study-provided smartphones in the lab and were administered external measures of similar and dissimilar constructs by trained examiners to assess validity under controlled circumstances. In Study 2, 1,021 participants completed the external measures in the lab and self-administered Arranging Pictures remotely on their personal smartphones to assess validity in real-world contexts. In Study 3, 141 participants self-administered Arranging Pictures remotely twice with a two-week delay on personal iOS smartphones to assess test-retest reliability and practice effects. RESULTS: Internal consistency was good across samples (ρxx = .80 to .85, p < .001). Test-retest reliability was marginal (ICC = .49, p < .001) and there were significant practice effects after a two-week delay (ΔM = 3.21 (95% CI [2.56, 3.88]). As expected, correlations with convergent measures were significant and moderate to large in magnitude (ρ = .44 to .76, p < .001), while correlations with discriminant measures were small (ρ = .23 to .27, p < .05) or nonsignificant. Scores demonstrated significant negative correlations with age (ρ = -.32 to -.21, p < .001). Mean performance was slightly higher in the iOS compared to the Android group (MiOS = 18.80, NiOS = 635; MAndroid = 17.11, NAndroid = 386; t(757.73) = 4.17, p < .001), but device type did not significantly influence the psychometric properties of the measure. Indicators of potential cheating were mixed; average scores were significantly higher in the remote samples (F(2, 850) = 11.415, p < .001), but there were not significantly more perfect scores. CONCLUSION: The MTB Arranging Pictures measure demonstrated evidence of reliability and validity when self-administered on personal device. Future research should examine the potential for cheating in remote settings and the properties of the measure in clinical samples.

Development and Validation of a Vocabulary Measure in the Mobile Toolbox.

OBJECTIVE: We describe the development of a new computer adaptive vocabulary test, Mobile Toolbox (MTB) Word Meaning, and validity evidence from 3 studies. METHOD: Word Meaning was designed to be a multiple-choice synonym test optimized for self-administration on a personal smartphone. The items were first calibrated online in a sample of 7,525 participants to create the computer-adaptive test algorithm for the Word Meaning measure within the MTB app. In Study 1, 92 participants self-administered Word Meaning on study-provided smartphones in the lab and were administered external measures by trained examiners. In Study 2, 1,021 participants completed the external measures in the lab and Word Meaning was self-administered remotely on their personal smartphones. In Study 3, 141 participants self-administered Word Meaning remotely twice with a 2-week delay on personal iPhones. RESULTS: The final bank included 1363 items. Internal consistency was adequate to good across samples (ρxx = 0.78 to 0.81, p < .001). Test-retest reliability was good (ICC = 0.65, p < .001), and the mean theta score was not significantly different upon the second administration. Correlations were moderate to large with measures of similar constructs (ρ = 0.67-0.75, p < .001) and non-significant with measures of dissimilar constructs. Scores demonstrated small to moderate correlations with age (ρ = 0.35 to 0.45, p < .001) and education (ρ = 0.26, p < .001). CONCLUSION: The MTB Word Meaning measure demonstrated evidence of reliability and validity in three samples. Further validation studies in clinical samples are necessary.

Predictive Utility of Irritability "In Context": Proof-of-Principle for an Early Childhood Mental Health Risk Calculator.

OBJECTIVE: We provide proof-of-principle for a mental health risk calculator advancing clinical utility of the irritability construct for identification of young children at high risk for common, early onsetting syndromes. METHOD: Data were harmonized from two longitudinal early childhood subsamples (total N = 403; 50.1% Male; 66.7% Nonwhite; Mage = 4.3 years). The independent subsamples were clinically enriched via disruptive behavior and violence (Subsample 1) and depression (Subsample 2). In longitudinal models, epidemiologic risk prediction methods for risk calculators were applied to test the utility of the transdiagnostic indicator, early childhood irritability, in the context of other developmental and social-ecological indicators to predict risk of internalizing/externalizing disorders at preadolescence (Mage = 9.9 years). Predictors were retained when they improved model discrimination (area under the receiver operating characteristic curve [AUC] and integrated discrimination index [IDI]) beyond the base demographic model. RESULTS: Compared to the base model, the addition of early childhood irritability and adverse childhood experiences significantly improved the AUC (0.765) and IDI slope (0.192). Overall, 23% of preschoolers went on to develop a preadolescent internalizing/externalizing disorder. For preschoolers with both elevated irritability and adverse childhood experiences, the likelihood of an internalizing/externalizing disorder was 39-66%. CONCLUSIONS: Predictive analytic tools enable personalized prediction of psychopathological risk for irritable young children, holding transformative potential for clinical translation.

Calibrating temper loss severity in the transition to toddlerhood: Implications for developmental science.

The integration of neurodevelopmental perspectives into clinical science has identified irritability as an early dimensional marker of lifespan mental health risk. Elucidating the developmental patterning of irritable behavior is key to differentiating normative variation from risk markers. Accounting for dysregulation and contextual features of irritability is useful for differentiation at preschool age, laying the groundwork for even earlier characterization. We provide initial evidence for the validity of the Multidimensional Assessment Profile of Disruptive Behavior Temper Loss Scale, Infant-Toddler version in two independent samples of 12-18-month-olds from the US. We calibrated the measure using item response theory in a large representative sample, then validated within an independent sample. We characterized the developmental patterning of irritable behaviors and their dimensional spectrum, and demonstrated test-retest reliability, and convergent validity. The MAP-DB-IT is a standardized, dimensional survey assessing irritability that serves as a tool for characterizing the developmental expression of early mental health risk.