Cross-validation of RAVLT performance validity indicators and the RAVLT/RO discriminant function in a large known groups sample.

OBJECTIVE: To cross-validate RAVLT performance validity cut-offs and the RAVLT/RO discriminant function in a large neuropsychological sample. METHOD: RAVLT scores and the RAVLT/RO discriminant function were compared in credible (n = 100) and noncredible (n = 353) neuropsychology referrals. RESULTS: Noncredible patients scored lower than credible patients on RAVLT scores and the RAVLT/RO discriminant function. With cut-offs set to ≥90% specificity, highest sensitivities were observed for the discriminant function (cut-off ≤.064; 55.8%), recognition total (cut-off ≤9; 53.1%), the recognition combination score (≤10; 47.7%), and total learning across trials (cut-off ≤31; 45.3%). Individuals with histories of learning difficulties were over-represented in the 10% of credible patients exceeding cut-offs. When these individuals were removed, cut-offs could be tightened while still maintaining at least 90% specificity, and thereby increasing sensitivity (e.g., recognition total cut-off ≤10, 65% sensitivity; RAVLT/RO discriminant function cut-off ≤.176, 58% sensitivity). When three of the most sensitive, non-overlapping scores were considered in combination, 17% of credible patients failed ≥1 of the three cut-offs, while 3% failed two, and only 1% failed all three. In contrast, in the noncredible sample, more than two-thirds failed one or more of the three cut-offs, nearly half failed ≥2, and nearly a quarter failed all three. CONCLUSIONS: RAVLT PVT cut-offs and the RAVLT/RO discriminant function achieve approximately 50% sensitivity, and approach 65% sensitivity when cut-offs specific to samples without histories of learning problems are employed, confirming that RAVLT cut-offs and the RAVLT/RO discriminant function continue to be valuable techniques in the identification of performance invalidity.

American Academy of Clinical Neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering.

Objective: Citation and download data pertaining to the 2009 AACN consensus statement on validity assessment indicated that the topic maintained high interest in subsequent years, during which key terminology evolved and relevant empirical research proliferated. With a general goal of providing current guidance to the clinical neuropsychology community regarding this important topic, the specific update goals were to: identify current key definitions of terms relevant to validity assessment; learn what experts believe should be reaffirmed from the original consensus paper, as well as new consensus points; and incorporate the latest recommendations regarding the use of validity testing, as well as current application of the term 'malingering.' Methods: In the spring of 2019, four of the original 2009 work group chairs and additional experts for each work group were impaneled. A total of 20 individuals shared ideas and writing drafts until reaching consensus on January 21, 2021. Results: Consensus was reached regarding affirmation of prior salient points that continue to garner clinical and scientific support, as well as creation of new points. The resulting consensus statement addresses definitions and differential diagnosis, performance and symptom validity assessment, and research design and statistical issues. Conclusions/Importance: In order to provide bases for diagnoses and interpretations, the current consensus is that all clinical and forensic evaluations must proactively address the degree to which results of neuropsychological and psychological testing are valid. There is a strong and continually-growing evidence-based literature on which practitioners can confidently base their judgments regarding the selection and interpretation of validity measures.

Malingering Detection of Cognitive Impairment With the b Test Is Boosted Using Machine Learning.

Objective: Here we report an investigation on the accuracy of the b Test, a measure to identify malingering of cognitive symptoms, in detecting malingerers of mild cognitive impairment. Method: Three groups of participants, patients with Mild Neurocognitive Disorder (n = 21), healthy elders (controls, n = 21), and healthy elders instructed to simulate mild cognitive disorder (malingerers, n = 21) were administered two background neuropsychological tests (MMSE, FAB) as well as the b Test. Results: Malingerers performed significantly worse on all error scores as compared to patients and controls, and performed poorly than controls, but comparably to patients, on the time score. Patients performed significantly worse than controls on all scores, but both groups showed the same pattern of more omission than commission errors. By contrast, malingerers exhibited the opposite pattern with more commission errors than omission errors. Machine learning models achieve an overall accuracy higher than 90% in distinguishing patients from malingerers on the basis of b Test results alone. Conclusions: Our findings suggest that b Test error scores accurately distinguish patients with Mild Neurocognitive Disorder from malingerers and may complement other validated procedures such as the Medical Symptom Validity Test.

Cross-validation of the Dot Counting Test in a large sample of credible and non-credible patients referred for neuropsychological testing.

OBJECTIVE: To cross-validate the Dot Counting Test in a large neuropsychological sample. METHOD: Dot Counting Test scores were compared in credible (n = 142) and non-credible (n = 335) neuropsychology referrals. RESULTS: Non-credible patients scored significantly higher than credible patients on all Dot Counting Test scores. While the original E-score cut-off of ≥17 achieved excellent specificity (96.5%), it was associated with mediocre sensitivity (52.8%). However, the cut-off could be substantially lowered to ≥13.80, while still maintaining adequate specificity (≥90%), and raising sensitivity to 70.0%. Examination of non-credible subgroups revealed that Dot Counting Test sensitivity in feigned mild traumatic brain injury (mTBI) was 55.8%, whereas sensitivity was 90.6% in patients with non-credible cognitive dysfunction in the context of claimed psychosis, and 81.0% in patients with non-credible cognitive performance in depression or severe TBI. Thus, the Dot Counting Test may have a particular role in detection of non-credible cognitive symptoms in claimed psychiatric disorders. Alternative to use of the E-score, failure on ≥1 cut-offs applied to individual Dot Counting Test scores (≥6.0″ for mean grouped dot counting time, ≥10.0″ for mean ungrouped dot counting time, and ≥4 errors), occurred in 11.3% of the credible sample, while nearly two-thirds (63.6%) of the non-credible sample failed one of more of these cut-offs. CONCLUSIONS: An E-score cut-off of 13.80, or failure on ≥1 individual score cut-offs, resulted in few false positive identifications in credible patients, and achieved high sensitivity (64.0-70.0%), and therefore appear appropriate for use in identifying neurocognitive performance invalidity.

Examination of the Modified Somatic Perception Questionnaire (MSPQ) in a large sample of credible and noncredible patients referred for neuropsychological testing.

OBJECTIVE: The current study evaluated MSPQ sensitivity to noncredible PVT performance in the context of external incentive, and examined MSPQ false positive rates in noncompensation-seeking neuropsychology patients; and investigated effects of ethnicity/culture, gender, and somatoform diagnosis on MSPQ scores, and relationships with PVT and MMPI-2-RF data. METHOD: MSPQ scores were compared in credible (n = 110) and noncredible (n = 153) neuropsychology referrals. RESULTS: Noncredible patients scored higher than credible patients. When the credible group was divided into those with somatoform orientation (n = 39) versus those without (n = 71), the credible nonsomatoform group scored lower than the other two groups, who did not differ from each other. MSPQ elevations were found in ethnic minorities, and in individuals who learned English as a second language or concurrently with another language. MSPQ elevations were also associated with chronic systemic diseases, neurologic illness, and substance abuse. Women scored higher than men, but men and women were equally represented among those patients scoring beyond cut-offs. MSPQ scores were minimally related to PVT data but were more strongly correlated with MMPI-2-RF scales, particularly over-report validity scales, RC1, and Somatic/Cognitive scales, with more widespread relationships observed in noncredible patients. CONCLUSIONS: A cut-off of 18 resulted in few false positives in credible nonsomatoform patients, and appears appropriate for identifying physical symptom over-report (due to malingering or somatoform orientations), with associated sensitivity of 29%. However, clinicians are cautioned regarding using the MSPQ in patients with systemic, neurologic, and substance abuse conditions, and in ethnic minorities and non-monolingual English-speakers.

The MMPI-2/MMPI-2-RF Symptom Validity Scale (FBS/FBS-r) is not a measure of 'litigation response syndrome': commentary on Nichols and Gass (2015).

OBJECTIVES: To address (1) Whether there is empirical evidence for the contention of Nichols and Gass that the MMPI-2/MMPI-2-RF FBS/FBS-r Symptom Validity Scale is a measure of Litigation Response Syndrome (LRS), representing a credible set of responses and reactions of claimants to the experience of being in litigation, rather than a measure of non-credible symptom report, as the scale is typically used; and (2) to address their stated concerns about the validity of FBS/FBS-r meta-analytic results, and the risk of false positive elevations in persons with bona-fide medical conditions. METHOD: Review of published literature on the FBS/FBS-r, focusing in particular on associations between scores on this symptom validity test and scores on performance validity tests (PVTs), and FBS/FBS-r score elevations in patients with genuine neurologic, psychiatric and medical problems. RESULTS: (1) several investigations show significant associations between FBS/FBS-r scores and PVTs measuring non-credible performance; (2) litigants who pass PVTs do not produce significant elevations on FBS/FBS-r; (3) non-litigating medical patients (bariatric surgery candidates, persons with sleep disorders, and patients with severe traumatic brain injury) who have multiple physical, emotional and cognitive symptoms do not produce significant elevations on FBS/FBS-r. Two meta-analytic studies show large effect sizes for FBS/FBS-r of similar magnitude. CONCLUSIONS: FBS/FBS-r measures non-credible symptom report rather than legitimate experience of litigation stress. Importantly, the absence of significant FBS/FBS-r elevations in litigants who pass PVTs demonstrating credible performance, directly contradicts the contention of Nichols and Gass that the scale measures LRS. These data, meta-analytic publications, and recent test use surveys support the admissibility of FBS/FBS-r under both Daubert and the older Frye criteria.

The validity of the MMPI-2/MMPI-2-RF Symptom Validity Scale (FBS/FBS-r) is established: reply to Nichols (2017).

We reply to Nichols' (2017) critique of our commentary on the MMPI-2/MMPI-2-RF Symptom Validity Scale (FBS/FBS-r) as a measure of symptom exaggeration versus a measure of litigation response syndrome (LRS). Nichols claims that we misrepresented the thrust of the original paper he co-authored with Gass; namely, that they did not represent that the FBS/FBS-r were measures of LRS but rather, intended to convey that the FBS/RBS-r were indeterminate as to whether the scales measured LRS or measured symptom exaggeration. Our original commentary offered statistical support from published literature that (1) FBS/FBS-r were associated with performance validity test (PVT) failure, establishing the scales as measures of symptom exaggeration, and (2) persons in litigation who passed PVTs did not produce clinically significant elevations on the scales, contradicting that FBS/FBS-r were measures of LRS. In the present commentary, we draw a distinction between the psychometric data we present supporting the validity of FBS/FBS-r, and the conceptual, non-statistical arguments presented by Nichols, who does not refute our original empirically based conclusions.

Specificity data for the b Test, Dot Counting Test, Rey-15 Item Plus Recognition, and Rey Word Recognition Test in monolingual Spanish-speakers.

The current study provides specificity data on a large sample (n = 115) of young to middle-aged, male, monolingual Spanish speakers of lower educational level and low acculturation to mainstream US culture for four neurocognitive performance validity tests (PVTs): the Dot Counting, the b Test, Rey Word Recognition, and Rey 15-Item Plus Recognition. Individuals with 0 to 6 years of education performed more poorly than did participants with 7 to 10 years of education on several Rey 15-Item scores (combination equation, recall intrusion errors, and recognition false positives), Rey Word Recognition total correct, and E-score and omission errors on the b Test, but no effect of educational level was observed for Dot Counting Test scores. Cutoff scores are provided that maintain approximately 90% specificity for the education subgroups separately. Some of these cutoffs match, or are even more stringent than, those recommended for use in US test takers who are primarily Caucasian, are tested in English, and have a higher educational level (i.e., Rey Word Recognition correct false-positive errors; Rey 15-Item recall intrusions and recognition false-positive errors; b Test total time; and Dot Counting E-score and grouped dot counting time). Thus, performance on these PVT variables in particular appears relatively robust to cultural/language/educational factors.

Relational integration and executive function in Alzheimer's disease.

Executive functions depend on the ability to represent relations between objects and events, and the prefrontal cortex provides the neural substrate for this capacity. Patients with probable Alzheimer's disease (AD) and control participants were administered measures of working memory and reasoning that varied systematically in their relational complexity. AD patients showed impairment on reasoning measures that required the online integration of relations but performed as well as control participants on nonrelational items and items requiring the processing of only single relations. When AD patients were divided into subgroups based on their performance on relational reasoning measures, the subgroup that showed significant impairment on relational integration measures exhibited a neuropsychological profile consistent with prefrontal cortical dysfunction.

Apparent effect of type of compensation seeking (disability versus litigation) on performance validity test scores may be due to other factors.

Neuropsychologists use performance validity tests (PVTs; Larrabee, 2012 ) to ensure that results of testing are reflective of the test taker's true neurocognitive ability, and their use is recommended in all compensation-seeking settings. However, whether the type of compensation context (e.g., personal injury litigation versus disability seeking) impacts the nature and extent of neurocognitive symptom feigning has not been adequately investigated. PVT performance was compared in an archival data set of noncredible individuals in either a personal injury litigation (n = 163) or a disability-seeking context (n = 201). Individuals were deemed noncredible based on meeting Slick, Sherman, and Iverson's ( 1999 ) criteria including failure on at least two PVTs and a lack of congruency between their low cognitive scores and normal function in activities of daily living (ADLs). In general, disability seekers tended to perform in a less sophisticated manner than did litigants (i.e., they failed more indicators and did so more extensively). Upon further investigation, these differences were in part accounted for by type of diagnoses feigned; those seeking compensation for mental health diagnoses were more likely to feign or exaggerate a wide variety of cognitive deficits, whereas those with claimed medical diagnoses (i.e., traumatic brain injury) were more targeted in their attempts to feign and/or exaggerate neurocognitive compromise.

Cross validation of the Lu and colleagues (2003) Rey-Osterrieth Complex Figure Test effort equation in a large known-group sample.

A Rey-Osterrieth Complex Figure Test (ROCFT) equation incorporating copy and recognition was found to be useful in detecting negative response bias in neuropsychological assessments (ROCFT Effort Equation; Lu, P. H., Boone, K. B., Cozolino, L., & Mitchell, C. (2003). Effectiveness of the Rey-Osterrieth Complex Figure Test and the Meyers and Meyers recognition trial in the detection of suspect effort. Clinical Neuropsychologist, 17, 426-440). In the current cross validation of this validity, the credible patient group (n = 146; 124 with equation data) outperformed the noncredible group (n = 157; 115 with equation data) on copy, 3-min recall, total recognition correct and the Effort Equation, but the latter was most effective in classifying subjects. A cut-off of ≤50 maintained specificity of 90% and achieved sensitivity of 80%. Results of the current cross validation provide corroboration that the ROCFT Effort Equation is an effective measure of neurocognitive response bias.

Non-credible language deficits following mild traumatic brain injury.

Identification of non-credible memory and other cognitive symptoms has received widespread attention within the past two decades. However, minimal information is available regarding patterns of non-credible language symptoms. We present the case of a 36-year-old female civil litigant who displayed delayed onset, severe, relatively focal speech and language symptoms, including difficulties with articulation, dysfluent speech, expressive language impairments with minor receptive difficulties, and lack of prosody, subsequent to a minor head trauma. On neuropsychological evaluation 3 years post injury, the patient presented with the same speech/language characteristics, but additionally exhibited a vague "foreign accent." Cognitive scores generally were normal with the exception of poor performance on many language tasks and processing/motor speed. The patient showed passing performance on most measures of response bias, but she failed effort indicators requiring rapid letter discrimination (b Test), rapid verbal repetition (timed forward digit span), and sensory function (finger agnosia errors) while passing effort indicators falling within the domains of memory, math/number skills, visuo-constructional ability, and attention. Thus, the type of failed effort indicators predicted the categories of standard cognitive tests on which she underperformed. Personality testing revealed patterns generally consistent with hysterical personality orientation. Given her long-standing history of multiple unexplained medical symptoms, the patient was diagnosed with a somatization disorder, as well as a current conversion disorder involving language symptoms. However, given her compensation-seeking status, the possibility of additional conscious feigning of symptoms could not be ruled out. The need for validation of additional measures to detect feigned language impairment is recommended.

Sensitivity and specificity of a digit symbol recognition trial in the identification of response bias.

Recently published practice standards recommend that multiple effort indicators be interspersed throughout neuropsychological evaluations to assess for response bias, which is most efficiently accomplished through use of effort indicators from standard cognitive tests already included in test batteries. The present study examined the utility of a timed recognition trial added to standard administration of the WAIS-III Digit Symbol subtest in a large sample of "real world" noncredible patients (n=82) as compared with credible neuropsychology clinic patients (n=89). Scores from the recognition trial were more sensitive in identifying poor effort than were standard Digit Symbol scores, and use of an equation incorporating Digit Symbol Age-Corrected Scaled Scores plus accuracy and time scores from the recognition trial was associated with nearly 80% sensitivity at 88.7% specificity. Thus, inclusion of a brief recognition trial to Digit Symbol administration has the potential to provide accurate assessment of response bias.

The Warrington Recognition Memory Test for words as a measure of response bias: total score and response time cutoffs developed on "real world" credible and noncredible subjects.

Several studies have examined the usefulness of the Warrington Recognition Memory Test-Words as a measure to detect suspect effort, although samples have generally been small and/or comprised of simulators rather than "real world" credible and noncredible patients. The current study examined the Warrington Recognition Memory Test-Words total score and response time of "real world" noncredible patients (as determined by motive to feign, failure on > or =2 independent measures of response bias, low cognitive scores inconsistent with normal ADLs; n = 190) versus credible patients (as determined by no motive to feign, failure of < or =1 measure of response bias; n = 124) derived from an archival database of individuals from the Harbor-UCLA Medical Center, Department of Psychiatry, Outpatient Neuropsychology Service, and the private practice of the second author. Noncredible patients obtained significantly lower total scores and longer times to complete the task. A total correct cutoff of < or =42 was found to have excellent specificity (91.9%) and sensitivity (88.9%), whereas a time cutoff of > or =207'' was associated with 65.5% sensitivity at 90.7% specificity, and when the time cut-score was used in combination with the total score cutoff, an additional 5% of the noncredible participants were captured, raising overall sensitivity to 93.7% (at 87.1% specificity). Thus, the Warrington Recognition Memory Test-Words, although not originally created for the purposes of measuring suspect effort, appears to be an excellent measure for detecting response bias on neuropsychological testing.

Examination of various WMS-III logical memory scores in the assessment of response bias.

The assessment of response validity during neuropsychological evaluation is an integral part of the testing process. Research has increasingly focused on the use of "embedded" effort measures (derived from standard neuropsychological tasks) because they do not require additional administration time and are less likely to be identified as effort indicators by test takers because of their primary focus as measures of cognitive function. The current study examined the clinical utility of various WMS-III Logical Memory scores in detecting response bias, as well as the Rarely Missed Index, an embedded effort indicator derived from the WMS-III Logical Memory Delayed Recognition subtest. The Rarely Missed Index cut-off only identified 24.1% of 63 non-credible participants (at >/=90% specificity in 125 credible patients), and cut-offs for other Logical Memory variables were in fact found to be more sensitive to non-credible performance. A new indicator, consisting of the weighted combination of the two most sensitive Logical Memory subtest scores (Logical Memory II raw score and Logical Memory Delayed Recognition raw score), was associated with 53% to 60% sensitivity, and thus may be an effective adjunct when utilized in conjunction with other validated effort indicators and collateral information in identifying non-credible performance.

Interpreting the meaning of multiple symptom validity test failure.

While it is recommended that judgments regarding the credibility of test performance be based on the results of more than one effort indicator, and recent efforts have been made to improve interpretation of multiple effort test failure, the field currently lacks adequate guidelines for using multiple measures of effort in concert with one another. A total of 103 patients were referred for outpatient neuropsychological evaluation, which included multiple measures of negative response bias embedded in standard test batteries. Using any pairwise failure combination to predict diagnostic classification was superior (sensitivity = 83.8%, specificity = 93.9%, overall hit rate = 90.3%) to using any one test by itself and to using any three-test failure combination. Further, the results were comparable to the results of logistical regression analyses using the embedded indicators as continuous predictors. Given its parsimony and clinical utility, the pairwise failure model is therefore a recommended criterion for identifying non-credible performance; however, there are of course other important contextual factors and influences to consider, which are also discussed.

Dementia and effort test performance.

Research on the performance of patients with dementia on tests of effort is particularly limited. We examined archival data from 214 non-litigating patients with dementia on 18 effort indices derived from 12 tests (WAIS-III/WAIS-R Digit Span and Vocabulary, Dot Counting Test, Warrington Recognition Memory Test-Words, WMS-III Logical Memory, Rey Word Recognition Memory Test, Finger Tapping, b-Test, Rey 15-Item, Test of Memory Malingering, Rey Auditory Verbal Learning Test, and Rey Complex Figure Test). Results indicated that recommended cut-offs for Digit Span indicators (Vocabulary Minus Digit Span and four-digit forward span time score) provided > or =90% specificity across participants, while the majority of other effort tests displayed specificities in the 30-70% range. Analyses of test specificity as a function of Mini Mental Status Examination (MMSE) score and specific dementia diagnosis are provided, as well as adjustments to cut-offs to maintain specificity where feasible.

The relationship of IQ to effort test performance.

The relationship between IQ and nine effort indicators was examined in a sample of 189 neuropsychology clinic outpatients who were not in litigation or attempting to obtain disability. Participants with the lowest IQ (50-59) failed approximately 60% of the effort tests, while patients with an IQ of 60 to 69 failed 44% of effort indicators, and individuals with borderline IQ (70 to 79) exhibited a 17% failure rate. All patients with IQ < 70 failed at least one effort test. Cutoffs for the Warrington Recognition Memory Test (Words) and Finger Tapping maintained the highest specificities in low IQ samples.

Examination of the impact of ethnicity on the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Fake Bad Scale.

The Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Fake Bad Scale (FBS; Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) has been shown to be sensitive to somatic over-endorsement. However, the impact of ethnicity has not been examined on the FBS, which is of concern given some studies that show increased rates of somatic endorsement in particular ethnic groups. We evaluated the FBS performance of 190 Caucasian American, Hispanic, and African American outpatients who were obtained from two different clinical settings, excluding those who were applying for disability or in litigation. We failed to find significant ethnic differences in mean FBS performance or in cut-off specificity rates. We did find evidence of a gender effect, supporting continued use of gender-specific FBS cutoffs.