Cross-validation of RAVLT performance validity indicators and the RAVLT/RO discriminant function in a large known groups sample.

OBJECTIVE: To cross-validate RAVLT performance validity cut-offs and the RAVLT/RO discriminant function in a large neuropsychological sample. METHOD: RAVLT scores and the RAVLT/RO discriminant function were compared in credible (n = 100) and noncredible (n = 353) neuropsychology referrals. RESULTS: Noncredible patients scored lower than credible patients on RAVLT scores and the RAVLT/RO discriminant function. With cut-offs set to ≥90% specificity, highest sensitivities were observed for the discriminant function (cut-off ≤.064; 55.8%), recognition total (cut-off ≤9; 53.1%), the recognition combination score (≤10; 47.7%), and total learning across trials (cut-off ≤31; 45.3%). Individuals with histories of learning difficulties were over-represented in the 10% of credible patients exceeding cut-offs. When these individuals were removed, cut-offs could be tightened while still maintaining at least 90% specificity, and thereby increasing sensitivity (e.g., recognition total cut-off ≤10, 65% sensitivity; RAVLT/RO discriminant function cut-off ≤.176, 58% sensitivity). When three of the most sensitive, non-overlapping scores were considered in combination, 17% of credible patients failed ≥1 of the three cut-offs, while 3% failed two, and only 1% failed all three. In contrast, in the noncredible sample, more than two-thirds failed one or more of the three cut-offs, nearly half failed ≥2, and nearly a quarter failed all three. CONCLUSIONS: RAVLT PVT cut-offs and the RAVLT/RO discriminant function achieve approximately 50% sensitivity, and approach 65% sensitivity when cut-offs specific to samples without histories of learning problems are employed, confirming that RAVLT cut-offs and the RAVLT/RO discriminant function continue to be valuable techniques in the identification of performance invalidity.

Wait, There's a Baby in this Bath Water! Update on Quantitative and Qualitative Cut-Offs for Rey 15-Item Recall and Recognition.

OBJECTIVE: Evaluate the effectiveness of Rey 15-item plus recognition data in a large neuropsychological sample. METHOD: Rey 15-item plus recognition scores were compared in credible (n = 138) and noncredible (n = 353) neuropsychology referrals. RESULTS: Noncredible patients scored significantly worse than credible patients on all Rey 15-item plus recognition scores. When cut-offs were selected to maintain at least 89.9% specificity, cut-offs could be made more stringent, with the highest sensitivity found for recognition correct (cut-off ≤11; 62.6% sensitivity) and the combination score (recall + recognition - false positives; cut-off ≤22; 60.6% sensitivity), followed by recall correct (cut-off ≤11; 49.3% sensitivity), and recognition false positive errors (≥3; 17.9% sensitivity). A cut-off of ≥4 applied to a summed qualitative error score for the recall trial resulted in 19.4% sensitivity. Approximately 10% of credible subjects failed either recall correct or recognition correct, whereas two-thirds of noncredible patients (67.7%) showed this pattern. Thirteen percent of credible patients failed either recall correct, recognition correct, or the recall qualitative error score, whereas nearly 70% of noncredible patients failed at least one of the three. Some individual qualitative recognition errors had low false positive rates (<2%) indicating that their presence was virtually pathognomonic for noncredible performance. Older age (>50) and IQ < 80 were associated with increased false positive rates in credible patients. CONCLUSIONS: Data on a larger sample than that available in the 2002 validation study show that Rey 15-item plus recognition cut-offs can be made more stringent, and thereby detect up to 70% of noncredible test takers, but the test should be used cautiously in older individuals and in individuals with lowered IQ.

Cross-validation of the Dot Counting Test in a large sample of credible and non-credible patients referred for neuropsychological testing.

OBJECTIVE: To cross-validate the Dot Counting Test in a large neuropsychological sample. METHOD: Dot Counting Test scores were compared in credible (n = 142) and non-credible (n = 335) neuropsychology referrals. RESULTS: Non-credible patients scored significantly higher than credible patients on all Dot Counting Test scores. While the original E-score cut-off of ≥17 achieved excellent specificity (96.5%), it was associated with mediocre sensitivity (52.8%). However, the cut-off could be substantially lowered to ≥13.80, while still maintaining adequate specificity (≥90%), and raising sensitivity to 70.0%. Examination of non-credible subgroups revealed that Dot Counting Test sensitivity in feigned mild traumatic brain injury (mTBI) was 55.8%, whereas sensitivity was 90.6% in patients with non-credible cognitive dysfunction in the context of claimed psychosis, and 81.0% in patients with non-credible cognitive performance in depression or severe TBI. Thus, the Dot Counting Test may have a particular role in detection of non-credible cognitive symptoms in claimed psychiatric disorders. Alternative to use of the E-score, failure on ≥1 cut-offs applied to individual Dot Counting Test scores (≥6.0″ for mean grouped dot counting time, ≥10.0″ for mean ungrouped dot counting time, and ≥4 errors), occurred in 11.3% of the credible sample, while nearly two-thirds (63.6%) of the non-credible sample failed one of more of these cut-offs. CONCLUSIONS: An E-score cut-off of 13.80, or failure on ≥1 individual score cut-offs, resulted in few false positive identifications in credible patients, and achieved high sensitivity (64.0-70.0%), and therefore appear appropriate for use in identifying neurocognitive performance invalidity.

Examination of the Modified Somatic Perception Questionnaire (MSPQ) in a large sample of credible and noncredible patients referred for neuropsychological testing.

OBJECTIVE: The current study evaluated MSPQ sensitivity to noncredible PVT performance in the context of external incentive, and examined MSPQ false positive rates in noncompensation-seeking neuropsychology patients; and investigated effects of ethnicity/culture, gender, and somatoform diagnosis on MSPQ scores, and relationships with PVT and MMPI-2-RF data. METHOD: MSPQ scores were compared in credible (n = 110) and noncredible (n = 153) neuropsychology referrals. RESULTS: Noncredible patients scored higher than credible patients. When the credible group was divided into those with somatoform orientation (n = 39) versus those without (n = 71), the credible nonsomatoform group scored lower than the other two groups, who did not differ from each other. MSPQ elevations were found in ethnic minorities, and in individuals who learned English as a second language or concurrently with another language. MSPQ elevations were also associated with chronic systemic diseases, neurologic illness, and substance abuse. Women scored higher than men, but men and women were equally represented among those patients scoring beyond cut-offs. MSPQ scores were minimally related to PVT data but were more strongly correlated with MMPI-2-RF scales, particularly over-report validity scales, RC1, and Somatic/Cognitive scales, with more widespread relationships observed in noncredible patients. CONCLUSIONS: A cut-off of 18 resulted in few false positives in credible nonsomatoform patients, and appears appropriate for identifying physical symptom over-report (due to malingering or somatoform orientations), with associated sensitivity of 29%. However, clinicians are cautioned regarding using the MSPQ in patients with systemic, neurologic, and substance abuse conditions, and in ethnic minorities and non-monolingual English-speakers.

Effectiveness of the Comalli Stroop Test as a measure of negative response bias.

Practice guidelines recommend the use of multiple performance validity tests (PVTs) to detect noncredible performance during neuropsychological evaluations, and PVTs embedded in standard cognitive tests achieve this goal most efficiently. The present study examined the utility of the Comalli version of the Stroop Test as a measure of response bias in a large sample of "real world" noncredible patients (n = 129) as compared with credible neuropsychology clinic patients (n=233). The credible group performed significantly better than the noncredible group on all trials, but particularly on word-reading (Stroop A) and color-naming (Stroop B); cut-scores for Stroop A and Stroop B trials were associated with moderate sensitivity (49-53%) as compared to the low sensitivity found for the color interference trial (29%). Some types of diagnoses (including learning disability, severe traumatic brain injury, psychosis, and depression), very advanced age (⩾80), and lowered IQ were associated with increased rates of false positive identifications, suggesting the need for some adjustments to cut-offs in these subgroups. Despite some previous reports of an inverted Stroop effect (i.e., color-naming worse than color interference) in noncredible subjects, individual Stroop word reading and color naming trials were much more effective in identifying response bias.

Cross validation of the Lu and colleagues (2003) Rey-Osterrieth Complex Figure Test effort equation in a large known-group sample.

A Rey-Osterrieth Complex Figure Test (ROCFT) equation incorporating copy and recognition was found to be useful in detecting negative response bias in neuropsychological assessments (ROCFT Effort Equation; Lu, P. H., Boone, K. B., Cozolino, L., & Mitchell, C. (2003). Effectiveness of the Rey-Osterrieth Complex Figure Test and the Meyers and Meyers recognition trial in the detection of suspect effort. Clinical Neuropsychologist, 17, 426-440). In the current cross validation of this validity, the credible patient group (n = 146; 124 with equation data) outperformed the noncredible group (n = 157; 115 with equation data) on copy, 3-min recall, total recognition correct and the Effort Equation, but the latter was most effective in classifying subjects. A cut-off of ≤50 maintained specificity of 90% and achieved sensitivity of 80%. Results of the current cross validation provide corroboration that the ROCFT Effort Equation is an effective measure of neurocognitive response bias.

The Warrington Recognition Memory Test for words as a measure of response bias: total score and response time cutoffs developed on "real world" credible and noncredible subjects.

Several studies have examined the usefulness of the Warrington Recognition Memory Test-Words as a measure to detect suspect effort, although samples have generally been small and/or comprised of simulators rather than "real world" credible and noncredible patients. The current study examined the Warrington Recognition Memory Test-Words total score and response time of "real world" noncredible patients (as determined by motive to feign, failure on > or =2 independent measures of response bias, low cognitive scores inconsistent with normal ADLs; n = 190) versus credible patients (as determined by no motive to feign, failure of < or =1 measure of response bias; n = 124) derived from an archival database of individuals from the Harbor-UCLA Medical Center, Department of Psychiatry, Outpatient Neuropsychology Service, and the private practice of the second author. Noncredible patients obtained significantly lower total scores and longer times to complete the task. A total correct cutoff of < or =42 was found to have excellent specificity (91.9%) and sensitivity (88.9%), whereas a time cutoff of > or =207'' was associated with 65.5% sensitivity at 90.7% specificity, and when the time cut-score was used in combination with the total score cutoff, an additional 5% of the noncredible participants were captured, raising overall sensitivity to 93.7% (at 87.1% specificity). Thus, the Warrington Recognition Memory Test-Words, although not originally created for the purposes of measuring suspect effort, appears to be an excellent measure for detecting response bias on neuropsychological testing.

Examination of various WMS-III logical memory scores in the assessment of response bias.

The assessment of response validity during neuropsychological evaluation is an integral part of the testing process. Research has increasingly focused on the use of "embedded" effort measures (derived from standard neuropsychological tasks) because they do not require additional administration time and are less likely to be identified as effort indicators by test takers because of their primary focus as measures of cognitive function. The current study examined the clinical utility of various WMS-III Logical Memory scores in detecting response bias, as well as the Rarely Missed Index, an embedded effort indicator derived from the WMS-III Logical Memory Delayed Recognition subtest. The Rarely Missed Index cut-off only identified 24.1% of 63 non-credible participants (at >/=90% specificity in 125 credible patients), and cut-offs for other Logical Memory variables were in fact found to be more sensitive to non-credible performance. A new indicator, consisting of the weighted combination of the two most sensitive Logical Memory subtest scores (Logical Memory II raw score and Logical Memory Delayed Recognition raw score), was associated with 53% to 60% sensitivity, and thus may be an effective adjunct when utilized in conjunction with other validated effort indicators and collateral information in identifying non-credible performance.

Examination of the impact of ethnicity on the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Fake Bad Scale.

The Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Fake Bad Scale (FBS; Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) has been shown to be sensitive to somatic over-endorsement. However, the impact of ethnicity has not been examined on the FBS, which is of concern given some studies that show increased rates of somatic endorsement in particular ethnic groups. We evaluated the FBS performance of 190 Caucasian American, Hispanic, and African American outpatients who were obtained from two different clinical settings, excluding those who were applying for disability or in litigation. We failed to find significant ethnic differences in mean FBS performance or in cut-off specificity rates. We did find evidence of a gender effect, supporting continued use of gender-specific FBS cutoffs.