Computerized assessment of executive functioning: Validation of the CNS Vital Signs executive functioning scores in a sample of community-dwelling older adults.

Objective: Computerized assessment of cognitive functioning has gained significant popularity over recent years, yet options for clinical assessment of executive functioning (EF) are lacking. One computerized testing platform, CNS Vital Signs (CNS-VS), offers tests designed to measure EF but requires further validation. The goal of the present study was to validate CNS-VS executive scores against standard clinical measures of EF. We also sought to determine whether a modified CNS-VS composite score that included variables purported to measure inhibition, switching, and working memory would outperform the currently available CNS-VS Executive Function Index. Method: A sample of 73 cognitively healthy older adults completed four tests from the Delis-Kaplan Executive Function System, the Digit Span subtest from the Wechsler Adult Intelligence Scale-fourth edition, and three CNS-VS tasks purported to measure inhibition, switching, and working memory. Results: Performances on the CNS-VS tests were predicted by performances on standard paper-and-pencil measures. Although the currently available CNS-VS Executive Function Index predicted unique variance in a well-validated paper-and-pencil EF composite score, our Modified CNS-VS EF composite accounted for unique variance above and beyond the original CNS-VS Executive Function Index, while the reverse was not true. Conclusions: The present results support the construct validity of CNS-VS EF tests but also suggest that modifications to their current composite scores would improve the prediction of EF performance.

Six elements test vs D-KEFS: what does "Ecological Validity" tell us?

OBJECTIVE: Extensive research shows that tests of executive functioning (EF) predict instrumental activities of daily living (IADLs) but are nevertheless often criticized for having poor ecological validity. The Modified Six Elements Test (MSET) is a pencil-and-paper test that was developed to mimic the demands of daily life, with the assumption that this would result in a more ecologically valid test. Although the MSET has been extensively validated in its ability to capture cognitive deficits in various populations, support for its ability to predict functioning in daily life is mixed. This study aimed to examine the MSET's ability to predict IADLs assessed via three different modalities relative to traditional EF measures. METHOD: Participants (93 adults aged 60 - 85) completed the MSET, traditional measures of EF (Delis-Kaplan Executive Function System; D-KEFS), and self-reported and performance-based IADLs in the lab. Participants then completed three weeks of IADL tasks at home, using the Daily Assessment of Independent Living and Executive Skills (DAILIES) protocol. RESULTS: The MSET predicted only IADLs completed at home, while the D-KEFS predicted IADLs across all three modalities. Further, the D-KEFS predicted home-based IADLs beyond the MSET when pitted against each other, whereas the MSET did not contribute beyond the D-KEFS. CONCLUSIONS: Traditional EF tests (D-KEFS) appear to be superior to the MSET in predicting IADLs in community-dwelling older adults. The present results argue against replacing traditional measures with the MSET when addressing functional independence of generally high-functioning and cognitive healthy older adult patients.

Pitfalls in research on ecological validity of novel executive function tests: A systematic review and a call to action.

The term "ecological validity" (EV) has traditionally referred to test scores' ability to predict real-world functioning. However, a test's similarity to real-world tasks is sometimes mistaken for evidence of its ability to predict daily life, sometimes bypassing rigorous validation research. The goal of this systematic review was to examine the type and quality of evidence used to support claims of EV of novel face-valid tests of executive functions (EF). MEDLINE and PsychINFO databases were searched using the following terms: ecologic* AND neuropsychol* AND (executive function* OR executive dysfunction OR executive abilit*). Thirty-two articles that explicitly stated that their results demonstrated EV of a novel face-valid test of EF were identified. Results showed that only 60% of studies based their claims about EV on test scores' ability to predict functional outcomes, with the remaining 40% relying on other evidence (e.g., correlations with other measures, participant feedback, group differences). Among the studies that did base their conclusions on test scores' ability to predict outcomes (n = 19), an overwhelming majority relied on behavioral rating scales, utilized small sample sizes and participant-to-variable ratios, and failed to control for covariates and multiple comparisons. Poor scientific rigor was particularly pronounced in studies of "naturalistic" tests. The present systematic review reveals significant conceptual, methodological, and statistical flaws among an overwhelming majority of studies that claim to have found support for the EV of a novel face-valid test of EF. We call upon authors, reviewers, and editors to safeguard the scientific rigor of research in this area. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Conceptualization of the term "ecological validity" in neuropsychological research on executive function assessment: a systematic review and call to action.

OBJECTIVE: "Ecological validity" (EV) is classically defined as test's ability to predict real-world functioning, either alone or together with test's similarity to real-world tasks. In neuropsychological literature on assessment of executive functions (EF), EV is conceptualized inconsistently, leading to misconceptions about the utility of tests. The goal of this systematic review was to examine how EV is conceptualized in studies of EF tests described as ecologically valid. METHOD: MEDLINE and PsychINFO Databases were searched. PRISMA guidelines were observed. After applying inclusion and exclusion criteria, this search yielded 90 articles. Deductive content analysis was employed to determine how the term EV was used. RESULTS: About 1/3 of the studies conceptualized EV as the test's ability to predict functional outcomes, 1/3 as both the ability to predict functional outcome and similarity to real-world tasks, and 1/3 were either unclear about the meaning of the term or relied on notions unrelated to classical definitions (e.g., similarity to real-world tasks alone, association with other tests, or the ability to discriminate between populations). CONCLUSIONS: Conceptualizations of the term EV in literature on EF assessment vary grossly, subsuming the notions of criterion, construct, and face validity, as well as sensitivity/specificity. Such inconsistency makes it difficult to interpret clinical utility of tests that are described as ecologically valid. We call on the field to require that, at minimum, the term EV be clearly defined in all publications, or replaced with more concrete terminology (e.g., criterion validity).

Test of Practical Judgment (TOP-J): Construct, Criterion, and Incremental Validity in a Community Sample of Older Adults.

OBJECTIVE: The Test of Practical Judgment (TOP-J) is a stand-alone judgment measure that is considered to tap into aspects of executive functioning (EF) and inform clinical predictions of daily functioning in older adults. Past validation research is variable and has some limitations. The present study sought to examine the reliability and construct, criterion, and incremental validities of scores on TOP-J 9-item version (TOP-J/9). METHOD: Participants were 95 community-dwelling older adults aged 60 to 85. Participants completed TOP-J/9, measures of EF and global cognition, and three different modalities of instrumental activities of daily living (IADLs) (self-report, performance-based tasks conducted in the laboratory, and performance-based tasks completed at home over 3 weeks). RESULTS: TOP-J/9 scores showed adequate internal consistency (α = 0.73) after correcting for the low number of items. TOP-J/9 was correlated with global cognition and EF, although EF did not survive correction for lower-order processes. Finally, although TOP-J/9 scores were associated with home-based IADL tasks (but not with self-report and laboratory-based IADLs), providing some evidence of criterion validity, they did not incrementally contribute to home-based IADL performance beyond other cognitive measures. However, when two items pertaining to social/ethical judgment were removed, this modified version of TOP-J did relate to EF beyond lower-order processes and contributed uniquely to prediction of home-based IADLs beyond other measures. CONCLUSION: Results suggest that TOP-J/9 taps into global cognitive status (but not necessarily EF) and predicts "real-world" functioning (but not above and beyond other cognitive measures). TOP-J psychometrics may be improved by removing two social/ethical items.

It's complicated: Executive functioning moderates impacts of daily busyness on everyday functioning in community-dwelling older adults.

OBJECTIVE: Research shows that cognitively healthy older adults with mild executive function (EF) weaknesses are vulnerable to the negative impacts of life complexity (or daily busyness) when performing instrumental activities of daily living (IADLs). However, past research assessed life complexity only at one timepoint, not capturing daily fluctuations. Importantly, fluctuations in busyness can themselves have deleterious impacts on functioning. This study extended past research by examining whether (1) variability in daily busyness would be more detrimental than level of busyness to performance of IADLs, and (2) EF assessed at home would moderate deleterious impact of busyness on IADLs. METHOD: Fifty-two community-dwelling older adults aged 60 to 95 completed daily IADL tasks and daily measures of EF and busyness via ecological momentary assessment, independently at home for 18 days. RESULTS: (1) In a subset of participants with mild EF weaknesses, high variability in busyness across days was associated with fewer tasks completed correctly; and (2) across all participants (regardless of EF), high levels of daily busyness were associated with fewer tasks completed on time. CONCLUSIONS: Findings indicate that high variability in daily busyness, potentially reflecting a lack of daily routine, was associated with IADL errors among cognitively healthy older adults with mild EF weaknesses. Additionally, consistently high levels of busyness were associated with failures to complete tasks, or failures to complete them on time, regardless of EF. These results further support the Contextually Valid Executive Assessment (ConVExA) model, which posits that EF and contextual factors interact to predict functional outcomes.

Was the COVID-19 Pandemic Associated with Gender Disparities in Authorship of Manuscripts Submitted to Clinical Neuropsychology Journals?

OBJECTIVE: The COVID-19 pandemic exacerbated gender disparities in some academic disciplines. This study examined the association of the pandemic with gender authorship disparities in clinical neuropsychology (CN) journals. METHOD: Author bylines of 1,018 initial manuscript submissions to four major CN journals from March 15 through September 15 of both 2019 and 2020 were coded for binary gender. Additionally, authorship of 40 articles published on pandemic-related topics (COVID-19, teleneuropsychology) across nine CN journals were coded for binary gender. RESULTS: Initial submissions to these four CN journals increased during the pandemic (+27.2%), with comparable increases in total number of authors coded as either women (+23.0%) or men (+25.4%). Neither the average percentage of women on manuscript bylines nor the proportion of women who were lead and/or corresponding authors differed significantly across time. Moreover, the representation of women as authors of pandemic-related articles did not differ from expected frequencies in the field. CONCLUSIONS: Findings suggest that representation of women as authors of peer-reviewed manuscript submissions to some CN journals did not change during the initial months of the COVID-19 pandemic. Future studies might examine how risk and protective factors may have influenced individual differences in scientific productivity during the pandemic.

Intra-individual variability identifies individuals vulnerable to contextually induced executive lapses.

Contextual stressors, such as engagement in burdensome emotion regulation known as expressive suppression (ES), can result in transient but clinically meaningful decrement in performance on measures of executive functioning (EF). The goal of the present investigation was to examine whether intra-individual variability (IIV-I), which has been identified as an indicator of cognitive weakness, could serve as a marker of vulnerability to EF decrements due to both naturally-occurring and experimentally-manipulated ES.In Study 1, 180 cognitively healthy older adults completed the Push-Turn-Taptap (PTT) task to assess IIV-I, four Delis-Kaplan Executive Function System (D-KEFS) subtests to assess EF, and the Burden of State Emotion Regulation Questionnaire (B-SERQ) to assess naturally-occurring ES. In Study 2, a subset (n = 81) of participants underwent experimental manipulation to induce ES, followed by second administration of the D-KEFS to examine ES-induced decrements in EF.In Study 1, hierarchical linear regression yielded a significant interaction between ES and IIV-I as predictors of EF performance, demonstrating that high ES was associated with low EF only among individuals with high IIV-I. In Study 2, repeated measures ANOVA demonstrated an interaction between time (pre- vs. post- manipulation), group (ES vs. control), and IIV-I (high vs. low), such that only individuals who exhibited high IIV-I were negatively impacted by the ES manipulation.IIV-I moderates the association between ES and EF, such that only individuals with high IIV-I exhibit vulnerability to the impact of ES. Thus, IIV-I may act as a marker of vulnerability to temporary EF depletion.

Ecological validity reconsidered: the Night Out Task versus the D-KEFS.

INTRODUCTION: Although executive functioning (EF) correlates with execution of instrumental activities of daily living (IADLs), tests of EF have been criticized for having poor ecological validity. Attempts have been made to develop new tests that approximate naturalistic daily tasks. However, the incremental utility of such tests has not been convincingly demonstrated. The Night Out Task (NOT) is a novel measure designed to increase ecological validity. This study examined whether the NOT correlates with traditional lab- and home-based measures of EF and IADLs, and whether it outperforms traditional measures of EF in predicting IADLs. METHOD: Participants (50 adults aged 60 to 95) completed (1) the Delis Kaplan Executive Function System (D-KEFS) and IADLs in the laboratory, and (2) ecological momentary assessment of EF and daily IADL tasks at home across three weeks (using the Daily Assessment of Independent Living and Executive Skills protocol; DAILIES). RESULTS: The NOT correlated with a lab-based measure of EF beyond covariates, and lab-based IADLs beyond covariates and beyond the D-KEFS. However, it was unrelated to at-home variables beyond covariates. In contrast, the D-KEFS was a significant predictor of at-home IADLs, and this association was mediated by at-home EF performance. CONCLUSION: This study provides a preliminary validation of the NOT as a correlate of office-based performances in a primarily college educated white sample. Despite its high face validity, the NOT does not appear to sufficiently tap EF processes needed for home-based IADLs as measured by the DAILIES, although small sample size limits the interpretability of this negative finding.

Reappraising cognitive reappraisal: The taxing impact of emotion regulation on executive functioning in older adults.

OBJECTIVE: Cognitive reappraisal (CR) and expressive suppression (ES) are two common emotion regulation strategies that share similar cognitive and neural underpinnings. Prior research has consistently shown that recent engagement in ES (both self-reported and experimentally manipulated) is associated with subsequent temporary decrements in executive functioning (EF). Thus far, only one study has examined the association between CR and EF, with null results. However, that study was limited by examining only zero-order correlations and by assessing only the speed, not accuracy, of EF performance. The present study examined multivariate relationships among recent CR, recent ES, and EF (both speed and accuracy), as well as the potential impacts of more chronic engagements in, and trait-level preferences between, the two emotion regulation strategies. METHOD: Participants were 201 community-dwelling older adults aged 60 to 93 who had participated in three separate studies examining the relationship between self-reported emotion regulation and EF. RESULTS: Recent CR was associated with EF performance accuracy above and beyond chronic CR. Both recent CR and ES contributed to EF performance accuracy uniquely beyond each other and beyond chronic and preferred emotion regulation. CONCLUSIONS: Both recent ES and CR appear to have a deleterious impact on EF performance accuracy, potentially due to utilization of similar resources; both should be accounted for when assessing emotion regulation and its impacts on EF.

Saving cognitive outcome data in Alzheimer's disease clinical trials during the COVID-19 pandemic: Commentary on the virtual administration of the ADAS-Cog.

Elderly participants in Alzheimer's disease (AD) clinical trials are at high risk of morbidity and mortality with interpersonal exposure to COVID-19, a situation that is likely to continue for the foreseeable future. Yet, in-person neuropsychological assessments remain the mainstay primary outcomes for clinical trials seeking prevention and cure for AD. The Alzheimer's Disease Assessment Scale-Cognitive (ADAS-Cog) is among the most commonly used cognitive assessment in AD clinical trials, and though currently lacking specific guidelines for virtual administrations, it can be used remotely with appropriate modifications and considerations. Here we propose a novel method of virtual administration of the ADAS-Cog, which considers workarounds for technological and human limitations imposed when the participant is no longer sitting across from the test administrator.

A pilot protocol to assess the feasibility of a virtual multiple crossover, randomized controlled trial design using methylphenidate in mild cognitive impairment.

BACKGROUND: The conventional clinical trial design in Alzheimer's disease (AD) and AD-related disorders (ADRDs) is the parallel-group randomized controlled trial. However, in heterogeneous disorders like AD/ADRDs, this design requires large sample sizes to detect meaningful effects in an "average" patient. They are very costly and, despite many attempts, have not yielded new treatments for many years. An alternative, the multi-crossover, randomized control trial (MCRCT) is a design in which each patient serves as their own control across successive, randomized blocks of active treatment and placebo. This design overcomes many limitations of parallel-group trials, yielding an unbiased assessment of treatment effect at the individual level ("N-of-1") regardless of unique patient characteristics. The goal of the present study is to pilot a MCRCT of a potential symptomatic treatment, methylphenidate, for mild-stage AD/ADRDs, testing feasibility and compliance of participants in this design and efficacy of the drug using both standard and novel outcome measures suited for this design. METHODS: Ten participants with mild cognitive impairment or mild-stage dementia due to AD/ADRDs will undergo a 4-week lead-in period followed by three, month-long treatment blocks (2 weeks of treatment with methylphenidate, 2 weeks placebo in random order). This trial will be conducted entirely virtually with an optional in-person screening visit. The primary outcome of interest is feasibility as measured by compliance and retention, with secondary and exploratory outcomes including cognition as measured by neuropsychological assessment at the end of each treatment period and daily brain games played throughout the study, actigraphy, and neuropsychiatric and functional assessments. DISCUSSION: This pilot study will gauge the feasibility of conducting a virtual MCRCT for symptomatic treatment in early AD/ADRD. It will also compare home-based daily brain games with standard neuropsychological measures within a clinical trial for AD/ADRD. Particular attention will be paid to compliance, tolerability of drug and participation, learning effects, trends and stability of daily measures across blocks, medication carryover effects, and correlations between standard and brief daily assessments. These data will provide guidance for more efficient trial design and the use of potentially more robust, ecological outcome measures in AD/ADRD research. TRIAL REGISTRATION: ClinicalTrials.gov, NCT03811847 . Registered on 21 January 2019.