Performance Validity Testing in Justice-Involved Adults with Fetal Alcohol Spectrum Disorder.

OBJECTIVES: A number of commonly used performance validity tests (PVTs) may be prone to high failure rates when used for individuals with severe neurocognitive deficits. This study investigated the validity of 10 PVT scores in justice-involved adults with fetal alcohol spectrum disorder (FASD), a neurodevelopmental disability stemming from prenatal alcohol exposure and linked with severe neurocognitive deficits. METHOD: The sample comprised 80 justice-involved adults (ages 19-40) including 25 with confirmed or possible FASD and 55 where FASD was ruled out. Ten PVT scores were calculated, derived from Word Memory Test, Genuine Memory Impairment Profile, Advanced Clinical Solutions (Word Choice), the Wechsler Adult Intelligence Scale - Fourth Edition (Reliable Digit Span and age-corrected scaled scores (ACSS) from Digit Span, Coding, Symbol Search, Coding - Symbol Search, Vocabulary - Digit Span), and the Wechsler Memory Scale - Fourth Edition (Logical Memory II Recognition). RESULTS: Participants with diagnosed/possible FASD were more likely to fail any single PVT, and failed a greater number of PVTs overall, compared to those without FASD. They were also more likely to fail based on Word Memory Test, Digit Span ACSS, Coding ACSS, Symbol Search ACSS, and Logical Memory II Recognition, compared to controls (35-76%). Across both groups, substantially more participants with IQ <70 failed two or more PVTs (90%), compared to those with an IQ ≥70 (44%). CONCLUSIONS: Results highlight the need for additional research examining the use of PVTs in justice-involved populations with FASD.

Differentiating poor validity from probable impairment on the medical symptom validity test: a cross-validation study.

AIMS: In neuropsychological evaluations, it is often difficult to ascertain whether poor performance on measures of validity is due to poor effort or malingering, or whether there is genuine cognitive impairment. Dunham and Denney created an algorithm to assess this question using the Medical Symptom Validity Test (MSVT). We assessed the ability of their algorithm to detect poor validity versus probable impairment, and concordance of failure on the MSVT with other freestanding tests of performance validity. METHODS: Two previously published datasets (n = 153 and n = 641, respectively) from outpatient neuropsychological evaluations were used to test Dunham and Denney's algorithm, and to assess concordance of failure rates with the Test of Memory Malingering and the forced choice measure of the California Verbal Learning Test, two commonly used performance validity tests. RESULTS: In both datasets, none of the four cutoff scores for failure on the MSVT (70%, 75%, 80%, or 85%) identified a poor validity group with proportionally aligned failure rates on other freestanding measures of performance validity. Additionally, the protocols with probable impairment did not differ from those with poor validity on cognitive measures. CONCLUSIONS: Despite what appeared to be a promising approach to evaluating failure on the easy MSVT subtests when clinical data are unavailable (as recommended in the advanced interpretation program, or advanced interpretation [AI], of the MSVT), the current findings indicate the AI remains the gold standard for doing so. Future research should build on this effort to address shortcomings in measures of effort in neuropsychological evaluations.

Limited usefulness of the Rey Fifteen-Item Test in detection of invalid performance in veterans suspected of mild traumatic brain injury.

OBJECTIVE: This study explored using the FIT as a measure of performance validity among veterans undergoing neuropsychological evaluation for mild traumatic brain injury (mTBI). BACKGROUND: The Rey Fifteen-Item Memory Test (FIT) is a performance validity measure criticized for poor sensitivity. METHODS: Two hundred and fifty-seven veterans completed the FIT and Wechsler Adult Intelligence Scale, Fourth Edition, Digit Span (DS); 109 of whom completed the Test of Memory Malingering (TOMM). FIT cut-offs of <9, <8 and stricter cut-offs were examined using DS and/or TOMM as criterion performance validity measures. RESULTS: Only four participants scored below the standard cut score of 9 on the FIT. Among the 13 veterans failing both criterion tests, only two scored below 9 on the FIT. Regardless of which FIT cut-off was used, the FIT had poor diagnostic accuracy. CONCLUSION: Despite its popularity, the FIT is not supported as an appropriate measure of performance validity in veterans undergoing evaluation for possible mTBI. Therefore, inferences regarding neuropsychological data reliability with adequate statistical certainty require use of other measures of performance validity with greater sensitivity.

Effort, symptom validity testing, performance validity testing and traumatic brain injury.

BACKGROUND: To understand the neurocognitive effects of brain injury, valid neuropsychological test findings are paramount. REVIEW: This review examines the research on what has been referred to a symptom validity testing (SVT). Above a designated cut-score signifies a 'passing' SVT performance which is likely the best indicator of valid neuropsychological test findings. Likewise, substantially below cut-point performance that nears chance or is at chance signifies invalid test performance. Significantly below chance is the sine qua non neuropsychological indicator for malingering. However, the interpretative problems with SVT performance below the cut-point yet far above chance are substantial, as pointed out in this review. This intermediate, border-zone performance on SVT measures is where substantial interpretative challenges exist. Case studies are used to highlight the many areas where additional research is needed. Historical perspectives are reviewed along with the neurobiology of effort. Reasons why performance validity testing (PVT) may be better than the SVT term are reviewed. CONCLUSIONS: Advances in neuroimaging techniques may be key in better understanding the meaning of border zone SVT failure. The review demonstrates the problems with rigidity in interpretation with established cut-scores. A better understanding of how certain types of neurological, neuropsychiatric and/or even test conditions may affect SVT performance is needed.

Ability of the Wisconsin Card-Sorting Test-64 as an embedded measure to identify noncredible neurocognitive performance in mild traumatic brain injury litigants.

OBJECTIVE: To investigate the ability of selective measures on the Wisconsin Card Sorting Test-64 (WCST-64) to predict noncredible neurocognitive dysfunction in a large sample of mild traumatic brain injury (mTBI) litigants. METHOD: Participants included 114 adults who underwent a comprehensive neuropsychological examination. Criterion groups were formed based upon their performance on stand-alone measures of cognitive performance validity (PVT). RESULTS: Participants failing PVTs performed worse across all WCST-64 dependent variables of interest compared to participants who passed PVTs. Receiver operating curve analysis revealed that only categories completed was a significant predictors of PVT status. Multivariate logistic regression did not add to classification accuracy. CONCLUSION: Consideration of noncredible executive functioning may be warranted in mild traumatic brain injury (mTBI) litigants who complete ≤ 1 category on the WCST-64.

Predictive value of performance validity testing and symptom validity testing in psychoeducational assessment.

Using archival data from 2463 psychoeducational assessments of postsecondary students we investigated whether failure on either symptom or performance validity tests (SVTs or PVTs) was associated with score differences on various cognitive, achievement, or executive functioning performance measures or on symptom report measures related to mental health or attention complaints. In total, 14.6% of students failed one or more PVT, 33.6% failed one or more SVT, and 41.6% failed at least one validity test. Individuals who failed SVTs tended to have the highest levels of self-reported symptoms relative to other groups but did not score worse on performance-based psychological tests. Those who failed PVTs scored worse on performance-based tests relative to other groups. Failure on at least one PVT and one SVT resulted in both performance and self-reported symptoms suggestive of greater impairment compared with those who passed all validity measures. Findings also highlight the need for domain-specific SVTs; failing ADHD SVTs was associated only with extreme reports of ADHD and executive functioning symptoms while failing mental health SVTs related only to extreme reports of mental health complaints. Results support using at least one PVT and one SVT in psychoeducational assessments to aid in diagnostic certainty, given the frequency of non-credible presentation in this population of postsecondary students.

Detection of noncredible cognitive performance with Wechsler Memory Scale-IV measures in mild traumatic brain injury litigants.

OBJECTIVE: To investigate the operating characteristics of selective measures on the Wechsler Memory Scale-IV (WMS-IV) to predict noncredible neurocognitive dysfunction in a sample of mild traumatic brain injury (mTBI) litigants. METHOD: Participants included 110 adults who underwent a comprehensive neuropsychological examination. Criterion groups were formed based upon their performance on stand-alone measures of cognitive performance validity testing (PVT). RESULTS: Participants failing two stand-alone PVTs exhibited significantly lower scores across all WMS-IV dependent variables of interest compared to participants who passed both PVTs. Participants who failed one PVT were excluded. Bivariate logistic regression revealed that all six dependent variables were significant predictors of PVT status. The best prediction model consisted of three WMS-IV variables including Logical Memory Delayed Recall (LM2), Logical Memory Recognition (LMR), and Visual Reproduction Recognition (VRR). This model demonstrated an accuracy of 90.2%, 0.89 sensitivity, 0.92 specificity, and a Receiver Operating Curve (ROC) of 0.957. CONCLUSION: The current empirically-derived cutscores and logit equation for the WMS-IV may be an additional consideration in analyzing database validity and noncredible performance in mTBI personal injury litigants ages 18-69.

Case study of invalid to valid shift in cognitive performance following successful treatment of psychogenic nonepileptic seizure events.

Patients with psychogenic nonepileptic seizure (PNES) who fail performance validity testing (PVT) may appear to produce non-valid cognitive profiles. Consequently, they may not get referred to treatment and events persist, with worsening disability and high resource utilization. As a result, we report pre- and post-treatment neuropsychological evaluation findings in a 59-year-old woman with a confirmed diagnosis of PNES established using video-EEG monitoring. At pre-treatment baseline neuropsychological evaluation, PNES events occurred weekly to daily. Performance was impaired across PVTs and across multiple cognitive domains. After behavioral intervention specific to PNES, these events substantially reduced in frequency to rare stress-induced flares. Post-treatment neuropsychological evaluation revealed marked improvement of most cognitive and behavioral scores from baseline, and valid PVT scores. We review predisposing, precipitating, and perpetuating factors for PNES and cognitive impairment in this case and discuss the patient's outcome from treatment. Effectively managing PNES events and dissociative tendencies while reducing unnecessary pharmacological interventions appears to have allowed this patient to function closer to her optimal state. This case illustrates the complexity of Functional Neurologic Disorder (FND) clinical presentation and challenges the assumption that suboptimal neuropsychological performance predicts poor treatment engagement and outcome. We showcase the reversibility of PNES and cognitive manifestations of FND using targeted psychotherapeutic interventions, which resulted in reduced disability and associated healthcare costs, as well as re-engagement in life.

Response time measures on the Word Memory Test do not add incremental validity to accuracy scores in predicting noncredible neurocognitive dysfunction in mild traumatic brain injury litigants.

The objective of the current study was to investigate whether response time measures on the Word Memory Test (WMT) increase predictive validity on determining noncredible neurocognitive dysfunction in a large sample of mild traumatic brain injury (MTBI) litigants. Participants included 203 adults who underwent a comprehensive neuropsychological examination. Criterion groups were formed based upon their performance on stand-alone measures of cognitive performance validity (PVT). Participants failing PVTs exhibited significantly slower response times and lower accuracy on the WMT compared to participants who passed PVTs. Response time measures did not add significant incremental validity beyond that afforded by WMT accuracy measures alone. The best predictor of PVT status was the WMT Consistency Score (CNS) which was associated with an extremely large effect size (d = 16.44), followed by Immediate Recognition (IR: d = 10.68) and Delayed Recognition (DR: d = 10.10).

An enhanced delayed recognition measure for the Logical Memory subtest of the Wechsler Memory Scale - IV.

We provide a supplemental measure based on the Logical Memory (LM) subtest of the Wechsler Memory Scale - IV (Wechsler, 2008) to assist in distinguishing deficient memory storage from compromised retrieval operations. A 20-item five-option multiple choice delayed recognition test for the LM stories is described, followed by descriptive data based on a normative sample of 273 neuropsychologically normal outpatient referrals to a neuropsychology clinic. The analysis indicated that about 43% to 48% of the neuropsychology referrals exhibited retrieval difficulties and were able to store more information in long-term memory than they were able to retrieve on the delayed free recall trial. The 20-item measure, freely available to qualified clinicians, provides useful information regarding a person's ability to store and access newly acquired information.

The minimal effect of depression on cognitive functioning when accounting for TOMM performance in a sample of U.S. veterans.

While many studies have demonstrated a relationship between depression and cognitive deficits, most have neglected to include measurements of performance validity. This study examined the relationship between depression and cognition after accounting for noncredible performance. Participants were veterans referred for outpatient clinical evaluation. The first set of regression analyses (N = 187) included age, sex, and education in Model 1, Beck Depression Inventory-2 (BDI-2) added in Model 2, and pass/failure of Test of Memory Malingering (TOMM) added in Model 3 as predictors of 12 neuropsychological test indices. The second set of analyses (N = 559) mirrored the first but with Major Depressive Disorder (MDD) diagnosis in Models 2 and 3. In the first analyses, after including TOMM in the model, only the relationship between BDI-2 and verbal fluency remained significant, but this did not survive a Bonferroni correction. In the second analyses, after including TOMM and Bonferroni correction, MDD diagnosis was a significant predictor only for CVLT-II Short Delay Free Recall. Therefore, the relationship between depression and cognition may not be driven by frank cognitive impairment, but rather by psychological mechanisms, which has implications for addressing depressed individuals' concerns about their cognitive functioning and suggest the value of providing psychoeducation and reassurance.

Cross-validation of three Advanced Clinical Solutions performance validity tests: Examining combinations of measures to maximize classification of invalid performance.

Use of multiple performance validity tests (PVTs) may best identify invalid performance, though few studies have examined the utility and accuracy of combining PVTs. This study examined the following PVTs in the Advanced Clinical Solutions (ACS) package to determine their utility alone and in concert: Word Choice Test (WCT), Reliable Digit Span (RDS), and Logical Memory Recognition (LMR). Ninety-three veterans participated in clinical neuropsychological evaluations to determine presence of cognitive impairment; 25% of the performances were deemed invalid via criterion PVTs. Classification accuracy of the ACS measures was assessed via receiver operating characteristic curves, while logistic regressions determined utility of combining these PVTs. The WCT demonstrated superior classification accuracy compared to the two embedded measures of the ACS, even in veterans with cognitive impairment. The two embedded measures (even when used in concert) exhibited inadequate classification accuracy. A combined model with all three ACS PVTs similarly demonstrated little benefit of the embedded indicators over the WCT alone. Results suggest the ACS WCT has utility for detecting invalid performance in a clinical sample with likely cognitive impairment, though the embedded ACS measures (RDS and LMR) may have limited incremental utility, particularly in individuals with cognitive impairment.

Rates of Performance Validity Test Failure for ADHD Evaluations in a Military Population: An Exploratory Study.

Objective: Investigate rates of Performance Validity Test (PVT) failures in a group of active duty (AD) military participants referred for ADHD evaluations and other characteristics of those referred for evaluation. Method: AD service members referred for ADHD evaluations were given PVTs as part of assessment. PVT pass/fail groups were compared on age, estimated premorbid intelligence, history of ADHD, other mental health problems, and functional impairment. Results: PVT failure rate was consistent with other studies. Fail group was younger and lower estimated premorbid intelligence than those that passed. Groups based on other variables did not significantly differ. Failure group also performed worse on a measure of attention. Many individuals were referred with no childhood history of ADHD or objective evidence of impaired function. Conclusion: Rates of PVT failures were similar to those found in civilian university and veteran military settings. Screening candidates for specific Diagnostic and Statistical Manual of Mental Disorders (5th ed.) criteria for ADHD was only minimally followed.

A pilot study on the adequacy of the TOMM in detecting invalid performance in patients with substance use disorders.

Objective: The validity of neuropsychological test performance has scarcely been studied in patients with substance use disorders (SUD), despite the possibility that some SUD individuals may distort their performance for compensation seeking (e.g., work leave, permanent or total disability, or the designation of services such as sheltered housing). Thus, the aim of the present study is to determine: (i) the possible utility of the Test of Memory Malingering (TOMM) in detecting invalid test results among SUD patients, and (ii) the percentage of individuals that underperform the TOMM among SUD patients seeking compensation.Method: Seventy-seven patients (59 men and 18 women) with SUD in outpatient treatment with an age range between 20 and 59 years were divided into two groups: SUD patients without compensation seeking (n = 41), and SUD patients with compensation seeking (n = 36). Participants performed a neuropsychological assessment with measures of processing speed, working memory, inhibition, verbal memory, cognitive flexibility, and decision-making, that also included the TOMM.Results: Our results demonstrate that there were no significant differences on TOMM performance between the two groups. Only one participant underperformed the TOMM (1.3% of the sample). Results showed cognitive impairment in the sample, but no differences between the groups on the different neuropsychological measures.Conclusions: This constitutes the first study to explore the capacity of a forced-choice test to detect invalid test results in an SUD population. There was a high performance on the TOMM among the SUD participants of our study despite their cognitive impairment, indicating adequate effort in their neuropsychological test performance. Further, these findings suggest that the probability of showing invalid neuropsychological performance among SUD compensation seeking patients is much lower than what has been found in other populations, such as in mild brain injury patients.

Rey 15 item test plus recognition trial and TOMM in a community pediatric sample.

In pediatric evaluations, performance validity test (PVT) selection is often constrained by reading level, developmental appropriateness of stimuli, and administration time. The Rey 15 Item Test (FIT) addresses these constraints, and ranks among the most frequently used PVTs. Unfortunately, research indicates poor sensitivity of the FIT recall trial. Boone et al. developed a FIT recognition trial and demonstrated in an adult sample that its use increased sensitivity while maintaining high specificity. These results are promising, but, to the authors' knowledge, have only been replicated once in a pediatric sample. The present study examined the FIT plus recognition trial in a sample of 72 young athletes ages 8-16 years. All data for the present study were collected during baseline cognitive evaluations. The Test of Memory Malingering (TOMM) was used as the comparison criterion. Receiver operating characteristic curve analyses showed the addition of the recognition trial did not substantially improve sensitivity of the FIT. There was a surprising lack of concordance between TOMM and FIT scores, and, whereas the FIT correlated with multiple cognitive measures, the TOMM did not correlate with any other measures. Results suggest the FIT is not appropriate for pediatric clinical care, even with the additional recognition trial.

Classification accuracy of the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Effort Index (EI) and Effort Scale (ES) relative to the Test Of Memory Malingering (TOMM) in a mixed clinical sample.

Performance Validity Testing has become an integral part of neuropsychological assessment, and a variety of embedded screening measures for performance validity have been proposed. Several performance validity indexes have been developed for the Repeatable Battery for the Assessment of Neuropsychological Status. Although the RBANS Effort Index (EI) and Effort Scale (ES) have garnered some empirical support, other research has raised questions regarding their accuracy in a number of assessment contexts. We evaluated the classification accuracy of the EI and ES against a standalone performance validity measure (Test of Memory Malingering; TOMM) in a mixed clinical patient sample. Our results showed limited utility for the ES in a mixed sample and modest classification accuracy for the EI, raising concerns about the appropriate scope of use for these scales in general clinical practice, which suggested that an alternate EI cutoff score of >0 may be most appropriate.

Comparing two models of performance validity assessment in patients with Parkinson's disease who are candidates for deep brain stimulation surgery.

Utility of standalone and embedded performance validity tests (PVTs) as well as the decision-making algorithms used to reach clinical conclusions about credible and noncredible performance can be population specific. To better understand PVT utility in Parkinson's disease candidates for deep brain stimulation (DBS) we present on two aims: 1) establishing the frequency data of below-criterion responding for the Medical Symptom Validity Test and three embedded PVTs in a sample of 47 patients with Parkinson's disease, and 2) comparing the efficacy of two models for clinical-decision making regarding noncredible performance. Consistent with expectations from previous studies and desired specificity values, our retrospective analysis indicated that in this sample of presumably well-motived patients, the rate of below-criterion responding was less than 10% for all PVTs administered. Regarding our model comparison, we compared a typical PVT battery that required administration of a standalone measure in all cases against a recently proposed low risk algorithm that attempts to lower testing burden by relying more heavily on embedded PVTs with administration of a standalone measure only in the event of below-criterion performance on an embedded indicator. Results suggest that for patients with Parkinson's disease judged to be at limited risk for noncredible performance, a low risk PVT model may prove both more efficient and less prone to error than a more typical model. Implications for clinical decision-making are discussed, as are limitations of the study and its generalizability.

Feigning memory impairment in a forced-choice task: Evidence from event-related potentials.

Strategies of malingering detection have brought about a wealth of neuropsychological studies in the last decades. However, the investigation of physiological measures to reliably differentiate between authentic and manipulated symptom presentations is still in its infancy. The present study examined event-related potentials (ERP) to identify feigned memory impairment. We tested instructed malingerers (n = 25) and control participants (n = 22) with a recognition task similar to the Test of Memory Malingering. No differences between groups were found for P1 (70-110 ms) but for N1 (120-170 ms) and P300 components, with lower amplitudes for instructed malingerers. Behavioral data showed a typical pattern of unrealistically high errors in a forced-choice recognition task and less overall recalled stimuli in instructed malingerers. We also found study-phase repetition and old/new effects in the P300, but no interactions with groups (control vs. malingering). Post-hoc analyses revealed that the P300 effect is greater when participants reported an attention-based faking strategy, as opposed to response-based malingerers and controls. The employment of physiological measures can yield additional information on the validity of test data without the need to perform additional tests.

Test of variables of attention performance among ADHD children with credible vs. non-credible PVT performance.

The assessment of effort is a crucial step in the evaluation of children and adolescents who present with symptoms of an Attention-Deficit/Hyperactivity Disorder (ADHD). Studies with adults have found that a large percentage of individuals claiming to have ADHD fail performance validity measures. In children, failure on PVT measures is associated with lower scores on a wide array of neuropsychological measures. The current study examined the performance of 50 children diagnosed with ADHD on the basis of whether they passed (N = 25) versus failed (N = 25) a standalone PVT, on the Test of Variables of Attention (TOVA), the Wisconsin Card Sorting Test - 64 (WCST) and the Tower of London: Drexel (TOL). Subjects who failed one or more PVTs scored significantly below those who passed, on the Omission scores of the TOVA and on several dimensions of the WCST. No significant differences were found on the TOL scores. Specifically, subjects who failed PVTs scored more than two standard deviations below the mean on the first half TOVA Omission errors score, whereas those who passed PVTs scored within the Average range. It is proposed that first half Omission scores on the TOVA may represent an embedded measure of effort.

Performance validity testing in children and adolescents: A descriptive study comparing direct and embedded measures.

Effort testing is a standard element in adult neuropsychological assessment. Research examining performance validity tests (PVTs) has focused on adults. The purpose of this descriptive study was to examine direct and embedded PVTs in children and adolescents. The Test of Memory Malingering (TOMM), a stand-alone PVT, was compared to two embedded measures of effort: California Verbal Learning Test-Children's Version/Second Edition (CVLT-C/II) Recognition Discriminability and Reliable Digit Span (RDS). The sample consisted of 119 children and adolescents referred for outpatient assessment. Cut-off scores used for PVTs were based on previous studies. Results revealed 3/119 failures on the TOMM, 14/119 failures on the RDS, and 34/119 failures on the CVLT-C/II. There was a significant difference between failures on the TOMM and the RDS, as well as between failures on the TOMM and the CVLT-C/II and the RDS and the CVLT-C/II. The results of this study demonstrate that PVTs commonly used with adults may require modifications, including adjusted cut-off scores, to be appropriate with children and adolescents. Results of this study also suggest that relying on more than one measure likely provides the most utility.