RETRACTED - Intra-individual Cognitive Variability: An Examination of ANAM4 TBI-MIL Simple Reaction Time Data from Service Members with and without Mild Traumatic Brain Injury.

Objectives: The Automated Neuropsychological Assessment Metrics 4 TBI-MIL (ANAM4) is a computerized cognitive test often used in post-concussion assessments with U.S. service members (SMs). Existing evidence, however, remains mixed regarding ANAM4's ability to identify cognitive issues following mild traumatic brain injury (mTBI). Studies typically examine ANAM4 by comparing mean scores to baseline or normative scores. A more fine-grained approach involves examining inconsistency within an individual's performance. Methods: Data from a sample of 231 were healthy control SMs and 100 SMs within 7 days of mTBI who took the ANAM4 were included in analyses. We examine each individual's performance on a simple reaction time (SRT) subtest that is administered at the beginning (SRT1) and end (SRT2) of the ANAM4 battery, and calculate the standard deviation of difference scores by trial across administrations. Results: Multivariate analysis of variance and univariate analyses revealed group differences across all comparisons (p<.001) with pairwise comparisons revealing higher intra-individual variability and slower raw reaction time for the mTBI group compared with controls. Effect sizes were small though exceeded the recommended minimum practical effect size (ES>0.41). Conclusions: While inconsistencies in performance are often viewed as noise or test error, the results suggest intra-individual cognitive variability may be more sensitive than central tendency measures (i.e., comparison of means) in detecting changes in cognitive function in mTBI. Additionally, the findings highlight the utility of ANAM4's repeating a subtest at two points in a battery to explore within-subject differences in performance. (JINS, 2017, 23, 1-6).

Feasibility of Mild Traumatic Brain Injury Assessment Based on Cardiovascular Response to Postural Change.

OBJECTIVE: To determine the feasibility of short-term cardiovascular responses to postural change as a screening tool for mild traumatic brain injury (mTBI), using heart rate metrics that can be measured with a wearable electrocardiogram sensor. SETTING: Military TBI clinic. DESIGN: Data collected from active-duty service members who had sustained a medically diagnosed mTBI within the prior 72 hours and from age- and sex-matched controls. Cardiac data collected while participants performed a sequence of postural changes. MAIN MEASURES: Model classification compared with clinical mTBI diagnosis. RESULTS: Cardiac biomarkers of mTBI were identified and logistic regression classifiers for mTBI were developed from different subsets of biomarkers. The best model achieved 90% sensitivity and 69% specificity using data from 2 different postural changes. CONCLUSION: Noninvasive measurement of cardiovascular response to postural change is a promising approach for field-deployable post-mTBI screening.

Intraindividual Cognitive Variability: An Examination of ANAM4 TBI-MIL Simple Reaction Time Data from Service Members with and without Mild Traumatic Brain Injury.

OBJECTIVES: The Automated Neuropsychological Assessment Metrics 4 TBI-MIL (ANAM4) is a computerized cognitive test often used in post-concussion assessments with U.S. service members (SMs). However, existing evidence remains mixed regarding ANAM4's ability to identify cognitive issues following mild traumatic brain injury (mTBI). Studies typically examine ANAM4 using standardized scores and/ or comparisons to a baseline. A more fine-grained approach involves examining inconsistency within an individual's performance (i.e., intraindividual variability). METHODS: Data from 237 healthy control SMs and 105 SMs within seven days of mTBI who took the ANAM4 were included in analyses. Using each individual's raw scores on a simple reaction time (RT) subtest (SRT1) that is repeated at the end of the battery (SRT2), we calculated mean raw RT and the intraindividual standard deviation (ISD) of trial-by-trial RT. Analyses investigated differences between groups in mean RT, RT variability (i.e., ISD), and change in ISD from SRT1 and SRT2. RESULTS: Using regression residuals to control for demographic variables, analysis of variance, and pairwise comparisons revealed the control group had faster mean RT and smaller ISD compared to the mTBI group. Furthermore, the mTBI group had a significant increase in ISD from SRT1 to SRT2, with effect sizes exceeding the minimum practical effect for comparisons of ISD in SRT2 and change in ISD from SRT1 to SRT2. CONCLUSIONS: While inconsistencies in performance are often viewed as test error, the results suggest intraindividual cognitive variability may be more sensitive than traditional metrics in detecting changes in cognitive function after mTBI. Additionally, the findings highlight the utility of the ANAM4's repeating a RT subtest at two points in the same session for exploring within-subject differences in performance variability. (JINS, 2018, 24, 156-162).

An Initial Psychometric Analysis of the Brain Gauge Tactile-Based Test Battery and Its Potential for Clinical Use Assessing Patients With Acute Mild Traumatic Brain Injury.

OBJECTIVE: A new brief computerized test battery that uses tactile stimulation, Brain Gauge (BG), has been proposed as a cognitive assessment aid and its developers have reported an almost perfect ability to distinguish acute mild traumatic brain injury (mTBI) patients from healthy controls. This investigation attempted to replicate those results and serve as an initial psychometrically and clinically focused analysis of BG. METHODS: BG scores from 73 military service members (SM) assessed within 7 days after having a clinically diagnosed mTBI were compared to 100 healthy SMs. Mean scores were compared, score distributions were examined, and univariate and multivariate base rate analyses of low scores were performed. RESULTS: SMs with mTBI had statistically significantly worse performance on both BG Reaction Time (RT) tests and the Sequential Amplitude Discrimination test as reflected by higher mean RT and RT variability and higher minimum detectable amplitude difference. SMs with mTBI also had a significantly lower whole-battery composite (i.e., Cortical Metric Symptom Score). Larger proportions of SMs with mTBI had lower overall performance than controls. However, at most only 26.9% of those with mTBI performed at potentially clinically meaningful cutoffs that were defined as various numbers of low scores that were prevalent in no more than 10% of the control group, which is equivalent to specificity ≥90% and sensitivity ≤26.9% for mTBI. CONCLUSION: Our analysis did not replicate the high level of classification accuracy reported by BG's developers. Pending further psychometric development, BG may have limited clinical utility for assessing mTBI patients.

An Investigation of the Accuracy of Reaction Time Measurements on ANAM4 TBI-MIL Across Three Computer Platforms.

STUDY DESIGN: Using two observational methods and a within-subjects, counterbalanced design, this study aimed to determine if a computer's hardware and software settings significantly affected reaction time (RT) on the Automated Neuropsychological Assessment Metrics (Version 4) Traumatic Brain Injury Military (ANAM4 TBI-MIL). METHODS: Three computer platforms were investigated: Platform 1-older computers recommended for ANAM4 TBI-MIL administration, Platform 2-newer computers with settings downgraded to run like the older computers, and Platform 3-newer computers with default settings. Two observational methods were used to compare measured RT to observed RT on all three platforms: 1, a high-speed video analysis to compare the timing of stimulus onset and response to the measured RT and 2, comparing a preset RT delivered by a robotic key actuator activated by optic detector to the measured RT. Additionally, healthy active duty service members (n = 169) were administered a brief version of the ANAM4 TBI-MIL battery on each of the three platforms. RESULTS: RT differences were observed with both the high-speed video and robotic arm analyses across all three computer platforms, with the smallest discrepancies between observed and measured RT on Platform 1, followed by Platform 2, then Platform 3. When simple reaction time (SRT) raw and standardized scores obtained from the participants were compared across platforms, statistically significant and clinically meaningful differences were seen, especially between Platforms 1 and 3. CONCLUSIONS: A computer's configurations have a meaningful impact on ANAM SRT scores. The difference in an individual's performance across platforms could be misinterpreted as clinically meaningful change.

Using Rates of Low Scores to Assess Agreement between Brief Computerized Neuropsychological Assessment Batteries: A Clinically-based Approach for Psychometric Comparisons.

OBJECTIVE: To assess agreement between four brief computerized neurocognitive assessment tools (CNTs), ANAM, CogState, CNS Vital Signs, and ImPACT, by comparing rates of low scores. METHODS: Four hundred and six US Army service members (SMs) with and without acute mild traumatic brain injury completed two randomly assigned CNTs with order of administration also randomly assigned. We performed a base rate analysis for each CNT to determine the proportions of SMs in the control and mTBI groups who had various numbers of scores that were 1.0+, 1.5+, and 2.0+ standard deviations below the normative mean. We used these results to identify a hierarchy of low score levels ranging from poorest to least poor performance. We then compared the agreement between every low score level from each CNT pair administered to the SMs. RESULTS: More SMs in the mTBI group had low scores on all CNTs than SMs in the control group. As performance worsened, the association with mTBI became stronger for all CNTs. Most if not all SMs who performed at the worst level on any given CNT also had low scores on the other CNTs they completed but not necessarily at an equally low level. CONCLUSION: These results suggest that all of the CNTs we examined are broadly similar but still retain some psychometric differences that need to be better understood. Furthermore, the base rates of low scores we present could themselves be useful to clinicians and researchers as a guide for interpreting results from the CNTs.

A Comparison of Four Computerized Neurocognitive Assessment Tools to a Traditional Neuropsychological Test Battery in Service Members with and without Mild Traumatic Brain Injury.

OBJECTIVE: Computerized neurocognitive assessment tools (NCATS) are often used as a screening tool to identify cognitive deficits after mild traumatic brain injury (mTBI). However, differing methodology across studies renders it difficult to identify a consensus regarding the validity of NCATs. Thus, studies where multiple NCATs are administered in the same sample using the same methodology are warranted. METHOD: We investigated the validity of four NCATs: the ANAM4, CNS-VS, CogState, and ImPACT. Two NCATs were randomly assigned and a battery of traditional neuropsychological (NP) tests administered to healthy control active duty service members (n = 272) and to service members within 7 days of an mTBI (n = 231). Analyses included correlations between NCAT and the NP test scores to investigate convergent and discriminant validity, and regression analyses to identify the unique variance in NCAT and NP scores attributed to group status. Effect sizes (Cohen's f2) were calculated to guide interpretation of data. RESULTS: Only 37 (0.6%) of the 5,655 correlations calculated between NCATs and NP tests are large (i.e. r ≥ 0.50). The majority of correlations are small (i.e. 0.30 > r ≥ 0.10), with no clear patterns suggestive of convergent or discriminant validity between the NCATs and NP tests. Though there are statistically significant group differences across most NCAT and NP test scores, the unique variance accounted for by group status is minimal (i.e. semipartial R2 ≤ 0.033, 0.024, 0.062, and 0.011 for ANAM4, CNS-VS, CogState, and ImPACT, respectively), with effect sizes indicating small to no meaningful effect. CONCLUSION: Though the results are not overly promising for the validity of the four NCATs we investigated, traditional methods of investigating psychometric properties may not be appropriate for computerized tests. We offer several conceptual and methodological considerations for future studies regarding the validity of NCATs.

A review of the validity of computerized neurocognitive assessment tools in mild traumatic brain injury assessment.

Computerized neurocognitive assessment tools (NCATs) offer potential advantages over traditional neuropsychological tests in postconcussion assessments. However, their psychometric properties and clinical utility are still questionable. The body of research regarding the validity and clinical utility of NCATs suggests some support for aspects of validity (e.g., convergent validity) and some ability to distinguish between concussed individuals and controls, though there are still questions regarding the validity of these tests and their clinical utility, especially outside of the acute injury timeframe. In this paper, we provide a comprehensive summary of the existing validity literature for four commonly used and studied NCATs (automated neuropsychological assessment metrics, CNS vital signs, cogstate and immediate post-concussion and cognitive testing) and lay the groundwork for future investigations.

The impact of administration order in studies of computerized neurocognitive assessment tools (NCATs).

Computerized neurocognitive assessment tools (NCATs) have become a common way to assess postconcussion symptoms. As there is increasing research directly comparing multiple NCATs to each other, it is important to consider the impact that order of test administration may have on the integrity of the results. This study investigates the impact of administration order in a study of four different NCATs; Automated Neuropsychological Assessment Metrics (ANAM4), CNS Vital Signs (CNS-VS), CogState, and Immediate Post-Concussion Assessment and Cognitive Test (ImPACT). A total of 272 healthy active duty Service Members were enrolled into this study. All participants were randomly assigned to take two of the four NCATs with order of administration counterbalanced. Analyses attempted to investigate the effect of administration order alone (e.g., Time 1 versus Time 2), the effect of administration order combined with the impact of the specific NCAT received at Time 1, and only the impact of the Time 1 NCAT on Time 2 score variability. Specifically, independent samples t tests were used to compare Time 1 and Time 2 scores within each NCAT. Additional t tests compared Time 1 to Time 2 scores with Time 2 scores grouped by the NCAT received at Time 1. One-way analysis of variance (ANOVA) was used to compare only an NCAT's Time 2 scores grouped by the NCAT received at Time 1. Cohen's d effect sizes were calculated for all comparisons. The results from this study revealed statistically significant order effects for CogState and CNS-VS, though with effect sizes generally indicating minimum practical value, and marginal or absent order effects for ANAM4 and ImPACT with no clinically meaningful implications. Despite finding minimal order effects, clinicians should be mindful of the impact of administering multiple NCATs in a single session. Future studies should continue to be designed to minimize the potential effect of test administration order.

[Formula: see text]Interpreting change on the neurobehavioral symptom inventory and the PTSD checklist in military personnel.

OBJECTIVE: The purpose of this study was to examine the prevalence and stability of symptom reporting in a healthy military sample and to develop reliable change indices for two commonly used self-report measures in the military health care system. PARTICIPANTS AND METHOD: Participants were 215 U.S. active duty service members recruited from Fort Bragg, NC as normal controls as part of a larger study. Participants completed the Neurobehavioral Symptom Inventory (NSI) and Posttraumatic Checklist (PCL) twice, separated by approximately 30 days. RESULTS: Depending on the endorsement level used (i.e. ratings of 'mild' or greater vs. ratings of 'moderate' or greater), approximately 2-15% of this sample met DSM-IV symptom criteria for Postconcussional Disorder across time points, while 1-6% met DSM-IV symptom criteria for Posttraumatic Stress Disorder. Effect sizes for change from Time 1 to Time 2 on individual symptoms were small (Cohen's d = .01 to .13). The test-retest reliability for the NSI total score was r = .78 and the PCL score was r = .70. An eight-point change in symptom reporting represented reliable change on the NSI total score, with a seven-point change needed on the PCL. CONCLUSIONS: Postconcussion-like symptoms are not unique to mild TBI and are commonly reported in a healthy soldier sample. It is important for clinicians to use normative data when evaluating a service member or veteran and when evaluating the likelihood that a change in symptom reporting is reliable and clinically meaningful.

Test-retest reliability of four computerized neurocognitive assessment tools in an active duty military population.

Computerized neurocognitive assessment tools (NCATs) are increasingly used for baseline and post-concussion assessments. To date, NCATs have not demonstrated strong test-retest reliabilities. Most studies have used non-military populations and different methodologies, complicating the determination of the utility of NCATs in military populations. The test-retest reliability of four NCATs (Automated Neuropsychological Assessment Metrics 4 [ANAM4], CNS-Vital Signs, CogState, and Immediate Post-Concussion Assessment and Cognitive Test [ImPACT]) was investigated in a healthy active duty military sample. Four hundred and nineteen Service Members were randomly assigned to take one NCAT and 215 returned after approximately 30 days for retest. Participants deemed to have inadequate effort during one or both testing sessions, according to the NCATs scoring algorithms, were removed from analyses. Each NCAT had at least one reliability score (intraclass correlation) in the "adequate" range (.70-.79), only ImPACT had one score considered "high" (.80-.89), and no scores met "very high" criteria (.90-.99). However, overall test-retest reliabilities in four NCATs in a military sample are consistent with reliabilities reported in the literature and are lower than desired for clinical decision-making.