Attorney demands for protected psychological test information: Is access necessary for cross examination or does it lead to misinformation? An interorganizational* position paper.

Objective: Some attorneys claim that to adequately cross examine neuropsychological experts, they require direct access to protected test information, rather than having test data analyzed by retained neuropsychological experts. The objective of this paper is to critically examine whether direct access to protected test materials by attorneys is indeed necessary, appropriate, and useful to the trier-of-fact. Method: Examples are provided of the types of nonscientific misinformation that occur when attorneys, who lack adequate training in testing, attempt to independently interpret neurocognitive/psychological test data. Results: Release of protected test information to attorneys introduces inaccurate information to the trier of fact, and jeopardizes future use of tests because non-psychologists are not ethically bound to protect test content. Conclusion: The public policy underlying the right of attorneys to seek possibly relevant documents should not outweigh the damage to tests and resultant misinformation that arise when protected test information is released directly to attorneys. The solution recommended by neuropsychological/psychological organizations and test publishers is to have protected psychological test information exchanged directly and only between clinical psychologist/neuropsychologist experts.

Are there differences in performance validity test scores between African American and White American neuropsychology clinic patients?

OBJECTIVE: The purpose of the present study was to compare performance on a wide range of PVTs in a neuropsychology clinic sample of African Americans and White Americans to determine if there are differences in mean scores or cut-off failure rates between the two groups, and to identify factors that may account for false positive PVT results in African American patients. METHOD: African American and White American non-compensation-seeking neuropsychology clinic patients were compared on a wide range of standalone and embedded PVTs: Dot Counting Test, b Test, Warrington Recognition Memory Test, Rey 15-item plus recognition, Rey Word Recognition Test, Digit Span (ACSS, RDS, 3-digit time, 4-digit time), WAIS-III Picture Completion (Most discrepant index), WAIS-III Digit Symbol/Coding (recognition equation), Rey Auditory Verbal Learning Test, Rey Complex figure, WMS-III Logical Memory, Comalli Stroop Test, Trails A, and Wisconsin Card Sorting Test. RESULTS: When groups were equated for age and education, African Americans obtained mean performances significantly worse than White Americans on only four of 25 PVT scores across the 14 different measures (Stroop Word Reading and Color Naming, Trails A, Digit Span 3-digit time); however, FSIQ was also significantly higher in White American patients. When subjects with borderline IQ (FSIQ = 70 to 79) were excluded (resulting in 74 White Americans and 25 African Americans), groups no longer differed in IQ and only continued to differ on a single PVT cutoff (Trails A). Further, specificity rates in African Americans were comparable to those of White Americans with the exception of the b Test, the Dot Counting Test, and Stroop B. CONCLUSIONS: PVT performance generally does not differ as a function of Black versus White race once the impact of intellectual level is controlled, and most PVT cutoffs appear appropriate for use in African Americans of low average IQ or higher.

Cross-validation of RAVLT performance validity indicators and the RAVLT/RO discriminant function in a large known groups sample.

OBJECTIVE: To cross-validate RAVLT performance validity cut-offs and the RAVLT/RO discriminant function in a large neuropsychological sample. METHOD: RAVLT scores and the RAVLT/RO discriminant function were compared in credible (n = 100) and noncredible (n = 353) neuropsychology referrals. RESULTS: Noncredible patients scored lower than credible patients on RAVLT scores and the RAVLT/RO discriminant function. With cut-offs set to ≥90% specificity, highest sensitivities were observed for the discriminant function (cut-off ≤.064; 55.8%), recognition total (cut-off ≤9; 53.1%), the recognition combination score (≤10; 47.7%), and total learning across trials (cut-off ≤31; 45.3%). Individuals with histories of learning difficulties were over-represented in the 10% of credible patients exceeding cut-offs. When these individuals were removed, cut-offs could be tightened while still maintaining at least 90% specificity, and thereby increasing sensitivity (e.g., recognition total cut-off ≤10, 65% sensitivity; RAVLT/RO discriminant function cut-off ≤.176, 58% sensitivity). When three of the most sensitive, non-overlapping scores were considered in combination, 17% of credible patients failed ≥1 of the three cut-offs, while 3% failed two, and only 1% failed all three. In contrast, in the noncredible sample, more than two-thirds failed one or more of the three cut-offs, nearly half failed ≥2, and nearly a quarter failed all three. CONCLUSIONS: RAVLT PVT cut-offs and the RAVLT/RO discriminant function achieve approximately 50% sensitivity, and approach 65% sensitivity when cut-offs specific to samples without histories of learning problems are employed, confirming that RAVLT cut-offs and the RAVLT/RO discriminant function continue to be valuable techniques in the identification of performance invalidity.

Wait, There's a Baby in this Bath Water! Update on Quantitative and Qualitative Cut-Offs for Rey 15-Item Recall and Recognition.

OBJECTIVE: Evaluate the effectiveness of Rey 15-item plus recognition data in a large neuropsychological sample. METHOD: Rey 15-item plus recognition scores were compared in credible (n = 138) and noncredible (n = 353) neuropsychology referrals. RESULTS: Noncredible patients scored significantly worse than credible patients on all Rey 15-item plus recognition scores. When cut-offs were selected to maintain at least 89.9% specificity, cut-offs could be made more stringent, with the highest sensitivity found for recognition correct (cut-off ≤11; 62.6% sensitivity) and the combination score (recall + recognition - false positives; cut-off ≤22; 60.6% sensitivity), followed by recall correct (cut-off ≤11; 49.3% sensitivity), and recognition false positive errors (≥3; 17.9% sensitivity). A cut-off of ≥4 applied to a summed qualitative error score for the recall trial resulted in 19.4% sensitivity. Approximately 10% of credible subjects failed either recall correct or recognition correct, whereas two-thirds of noncredible patients (67.7%) showed this pattern. Thirteen percent of credible patients failed either recall correct, recognition correct, or the recall qualitative error score, whereas nearly 70% of noncredible patients failed at least one of the three. Some individual qualitative recognition errors had low false positive rates (<2%) indicating that their presence was virtually pathognomonic for noncredible performance. Older age (>50) and IQ < 80 were associated with increased false positive rates in credible patients. CONCLUSIONS: Data on a larger sample than that available in the 2002 validation study show that Rey 15-item plus recognition cut-offs can be made more stringent, and thereby detect up to 70% of noncredible test takers, but the test should be used cautiously in older individuals and in individuals with lowered IQ.

Cross-validation of the Dot Counting Test in a large sample of credible and non-credible patients referred for neuropsychological testing.

OBJECTIVE: To cross-validate the Dot Counting Test in a large neuropsychological sample. METHOD: Dot Counting Test scores were compared in credible (n = 142) and non-credible (n = 335) neuropsychology referrals. RESULTS: Non-credible patients scored significantly higher than credible patients on all Dot Counting Test scores. While the original E-score cut-off of ≥17 achieved excellent specificity (96.5%), it was associated with mediocre sensitivity (52.8%). However, the cut-off could be substantially lowered to ≥13.80, while still maintaining adequate specificity (≥90%), and raising sensitivity to 70.0%. Examination of non-credible subgroups revealed that Dot Counting Test sensitivity in feigned mild traumatic brain injury (mTBI) was 55.8%, whereas sensitivity was 90.6% in patients with non-credible cognitive dysfunction in the context of claimed psychosis, and 81.0% in patients with non-credible cognitive performance in depression or severe TBI. Thus, the Dot Counting Test may have a particular role in detection of non-credible cognitive symptoms in claimed psychiatric disorders. Alternative to use of the E-score, failure on ≥1 cut-offs applied to individual Dot Counting Test scores (≥6.0″ for mean grouped dot counting time, ≥10.0″ for mean ungrouped dot counting time, and ≥4 errors), occurred in 11.3% of the credible sample, while nearly two-thirds (63.6%) of the non-credible sample failed one of more of these cut-offs. CONCLUSIONS: An E-score cut-off of 13.80, or failure on ≥1 individual score cut-offs, resulted in few false positive identifications in credible patients, and achieved high sensitivity (64.0-70.0%), and therefore appear appropriate for use in identifying neurocognitive performance invalidity.

Examination of the Modified Somatic Perception Questionnaire (MSPQ) in a large sample of credible and noncredible patients referred for neuropsychological testing.

OBJECTIVE: The current study evaluated MSPQ sensitivity to noncredible PVT performance in the context of external incentive, and examined MSPQ false positive rates in noncompensation-seeking neuropsychology patients; and investigated effects of ethnicity/culture, gender, and somatoform diagnosis on MSPQ scores, and relationships with PVT and MMPI-2-RF data. METHOD: MSPQ scores were compared in credible (n = 110) and noncredible (n = 153) neuropsychology referrals. RESULTS: Noncredible patients scored higher than credible patients. When the credible group was divided into those with somatoform orientation (n = 39) versus those without (n = 71), the credible nonsomatoform group scored lower than the other two groups, who did not differ from each other. MSPQ elevations were found in ethnic minorities, and in individuals who learned English as a second language or concurrently with another language. MSPQ elevations were also associated with chronic systemic diseases, neurologic illness, and substance abuse. Women scored higher than men, but men and women were equally represented among those patients scoring beyond cut-offs. MSPQ scores were minimally related to PVT data but were more strongly correlated with MMPI-2-RF scales, particularly over-report validity scales, RC1, and Somatic/Cognitive scales, with more widespread relationships observed in noncredible patients. CONCLUSIONS: A cut-off of 18 resulted in few false positives in credible nonsomatoform patients, and appears appropriate for identifying physical symptom over-report (due to malingering or somatoform orientations), with associated sensitivity of 29%. However, clinicians are cautioned regarding using the MSPQ in patients with systemic, neurologic, and substance abuse conditions, and in ethnic minorities and non-monolingual English-speakers.

Apparent effect of type of compensation seeking (disability versus litigation) on performance validity test scores may be due to other factors.

Neuropsychologists use performance validity tests (PVTs; Larrabee, 2012 ) to ensure that results of testing are reflective of the test taker's true neurocognitive ability, and their use is recommended in all compensation-seeking settings. However, whether the type of compensation context (e.g., personal injury litigation versus disability seeking) impacts the nature and extent of neurocognitive symptom feigning has not been adequately investigated. PVT performance was compared in an archival data set of noncredible individuals in either a personal injury litigation (n = 163) or a disability-seeking context (n = 201). Individuals were deemed noncredible based on meeting Slick, Sherman, and Iverson's ( 1999 ) criteria including failure on at least two PVTs and a lack of congruency between their low cognitive scores and normal function in activities of daily living (ADLs). In general, disability seekers tended to perform in a less sophisticated manner than did litigants (i.e., they failed more indicators and did so more extensively). Upon further investigation, these differences were in part accounted for by type of diagnoses feigned; those seeking compensation for mental health diagnoses were more likely to feign or exaggerate a wide variety of cognitive deficits, whereas those with claimed medical diagnoses (i.e., traumatic brain injury) were more targeted in their attempts to feign and/or exaggerate neurocognitive compromise.

Comparison of credible patients of very low intelligence and non-credible patients on neurocognitive performance validity indicators.

The purpose of this archival study was to identify performance validity tests (PVTs) and standard IQ and neurocognitive test scores, which singly or in combination, differentiate credible patients of low IQ (FSIQ ≤ 75; n = 55) from non-credible patients. We compared the credible participants against a sample of 74 non-credible patients who appeared to have been attempting to feign low intelligence specifically (FSIQ ≤ 75), as well as a larger non-credible sample (n = 383) unselected for IQ. The entire non-credible group scored significantly higher than the credible participants on measures of verbal crystallized intelligence/semantic memory and manipulation of overlearned information, while the credible group performed significantly better on many processing speed and memory tests. Additionally, credible women showed faster finger-tapping speeds than non-credible women. The credible group also scored significantly higher than the non-credible subgroup with low IQ scores on measures of attention, visual perceptual/spatial tasks, processing speed, verbal learning/list learning, and visual memory, and credible women continued to outperform non-credible women on finger tapping. When cut-offs were selected to maintain approximately 90% specificity in the credible group, sensitivity rates were highest for verbal and visual memory measures (i.e., TOMM trials 1 and 2; Warrington Words correct and time; Rey Word Recognition Test total; RAVLT Effort Equation, Trial 5, total across learning trials, short delay, recognition, and RAVLT/RO discriminant function; and Digit Symbol recognition), followed by select attentional PVT scores (i.e., b Test omissions and time to recite four digits forward). When failure rates were tabulated across seven most sensitive scores, a cut-off of ≥ 2 failures was associated with 85.4% specificity and 85.7% sensitivity, while a cut-off of ≥ 3 failures resulted in 95.1% specificity and 66.0% sensitivity. Results are discussed in light of extant literature and directions for future research.

Effectiveness of the Comalli Stroop Test as a measure of negative response bias.

Practice guidelines recommend the use of multiple performance validity tests (PVTs) to detect noncredible performance during neuropsychological evaluations, and PVTs embedded in standard cognitive tests achieve this goal most efficiently. The present study examined the utility of the Comalli version of the Stroop Test as a measure of response bias in a large sample of "real world" noncredible patients (n = 129) as compared with credible neuropsychology clinic patients (n=233). The credible group performed significantly better than the noncredible group on all trials, but particularly on word-reading (Stroop A) and color-naming (Stroop B); cut-scores for Stroop A and Stroop B trials were associated with moderate sensitivity (49-53%) as compared to the low sensitivity found for the color interference trial (29%). Some types of diagnoses (including learning disability, severe traumatic brain injury, psychosis, and depression), very advanced age (⩾80), and lowered IQ were associated with increased rates of false positive identifications, suggesting the need for some adjustments to cut-offs in these subgroups. Despite some previous reports of an inverted Stroop effect (i.e., color-naming worse than color interference) in noncredible subjects, individual Stroop word reading and color naming trials were much more effective in identifying response bias.

Cross validation of the Lu and colleagues (2003) Rey-Osterrieth Complex Figure Test effort equation in a large known-group sample.

A Rey-Osterrieth Complex Figure Test (ROCFT) equation incorporating copy and recognition was found to be useful in detecting negative response bias in neuropsychological assessments (ROCFT Effort Equation; Lu, P. H., Boone, K. B., Cozolino, L., & Mitchell, C. (2003). Effectiveness of the Rey-Osterrieth Complex Figure Test and the Meyers and Meyers recognition trial in the detection of suspect effort. Clinical Neuropsychologist, 17, 426-440). In the current cross validation of this validity, the credible patient group (n = 146; 124 with equation data) outperformed the noncredible group (n = 157; 115 with equation data) on copy, 3-min recall, total recognition correct and the Effort Equation, but the latter was most effective in classifying subjects. A cut-off of ≤50 maintained specificity of 90% and achieved sensitivity of 80%. Results of the current cross validation provide corroboration that the ROCFT Effort Equation is an effective measure of neurocognitive response bias.

Cross validation of the b Test in a large known groups sample.

The b Test (Boone, Lu, & Herzberg, 2002a) is a measure of cognitive performance validity originally validated on 91 non-credible participants and 7 credible clinical comparison groups (total n = 161). The purpose of the current study was to provide cross-validation data for the b Test on a known groups sample of non-credible participants (n = 212) and credible heterogeneous neuropsychological clinic patients (n = 103). The new data showed that while the original E-score cut-off of ≥ 155 achieved excellent specificity (99%), it was associated with relatively poor sensitivity (41%). However, the cut-off could be substantially lowered to ≥82, while still maintaining adequate specificity (≥90%) and raising sensitivity to 68%. Examination of non-credible subgroups revealed that b Test sensitivity in feigned mild traumatic brain injury (mTBI) was 58%, whereas in non-credible patients claiming depression and psychosis, cut-off sensitivity was 76% and 67%, respectively. These data suggest that the b Test may have a particular role in detection of non-credible cognitive symptoms associated with feigned psychiatric symptoms, and that fabricated deficits in processing speed and vigilance/visual scanning, detected by the b Test, are more prominent in feigned psychiatric presentations than in feigned mTBI. Further, b Test failures in patients with somatoform disorders were common, indicating that the b Test may have a specific use in detection of non-consciously created cognitive dysfunction associated with somatoform conditions.

Re-examination of the Rey Word Recognition Test.

The Rey Word Recognition Test, a brief and simple to administer free-standing neurocognitive performance validity test, was examined in a large known-groups sample (122 credible patients and 134 non-credible patients). Total correctly recognized was the most sensitive score, identifying 54% of non-credible participants using a cut-off of ≤6, while maintaining specificity of approximately 90%. However, specifically rates were somewhat lower in credible individuals with <12 years of education or borderline intelligence, or who were bilingual (spoke English as a second language, or learned English concurrently with another language), indicating that cut-offs may require minor adjustment in these groups. Sensitivity rates were much higher in non-credible female versus male mild traumatic brain injury patients (mTBI; 68% versus 48% for total correct), suggesting that the Rey Word Recognition Test is particularly effective in identifying performance invalidity in female mTBI compensation seekers.

Use of the WAIS-III picture completion subtest as an embedded measure of response bias.

In the present study a large sample of credible patients (n = 172) scored significantly higher than a large sample of noncredible participants (n = 195) on several WAIS-III Picture Completion variables: Age Adjusted Scaled Score, raw score, a "Rarely Missed" index (the nine items least often missed by credible participants), a "Rarely Correct" index (nine items correct <26% of the time in noncredible participants and with at least a 25 percentage-point lower endorsement rate as compared to credible participants), and a "Most Discrepant" index (the six items that were the most discrepant in correct endorsement between groups-at least a 40 percentage point difference). Comparison of the various scores showed that the "Most Discrepant" index outperformed all the others in identifying response bias (nearly 65% sensitivity at 92.8% specificity as compared to at most 59% sensitivity for the other scores). While no differences in Picture Completion scores were observed between less-educated (<12 years) and better-educated (≥12 years) credible participants, noncredible participants with <12 years of education scored significantly poorer than noncredible participants with 12 or more years of education. On the "Most Discrepant" index, 76.7% of less-educated noncredible participants were detected as compared to 58.3% of better-educated noncredible participants. Results of the current study suggest that the Picture Completion subtest of the WAIS-III is an effective measure of response bias, and that it may have a unique role in identifying suboptimal effort in less-educated test takers.

Sensitivity and specificity of a digit symbol recognition trial in the identification of response bias.

Recently published practice standards recommend that multiple effort indicators be interspersed throughout neuropsychological evaluations to assess for response bias, which is most efficiently accomplished through use of effort indicators from standard cognitive tests already included in test batteries. The present study examined the utility of a timed recognition trial added to standard administration of the WAIS-III Digit Symbol subtest in a large sample of "real world" noncredible patients (n=82) as compared with credible neuropsychology clinic patients (n=89). Scores from the recognition trial were more sensitive in identifying poor effort than were standard Digit Symbol scores, and use of an equation incorporating Digit Symbol Age-Corrected Scaled Scores plus accuracy and time scores from the recognition trial was associated with nearly 80% sensitivity at 88.7% specificity. Thus, inclusion of a brief recognition trial to Digit Symbol administration has the potential to provide accurate assessment of response bias.

The Warrington Recognition Memory Test for words as a measure of response bias: total score and response time cutoffs developed on "real world" credible and noncredible subjects.

Several studies have examined the usefulness of the Warrington Recognition Memory Test-Words as a measure to detect suspect effort, although samples have generally been small and/or comprised of simulators rather than "real world" credible and noncredible patients. The current study examined the Warrington Recognition Memory Test-Words total score and response time of "real world" noncredible patients (as determined by motive to feign, failure on > or =2 independent measures of response bias, low cognitive scores inconsistent with normal ADLs; n = 190) versus credible patients (as determined by no motive to feign, failure of < or =1 measure of response bias; n = 124) derived from an archival database of individuals from the Harbor-UCLA Medical Center, Department of Psychiatry, Outpatient Neuropsychology Service, and the private practice of the second author. Noncredible patients obtained significantly lower total scores and longer times to complete the task. A total correct cutoff of < or =42 was found to have excellent specificity (91.9%) and sensitivity (88.9%), whereas a time cutoff of > or =207'' was associated with 65.5% sensitivity at 90.7% specificity, and when the time cut-score was used in combination with the total score cutoff, an additional 5% of the noncredible participants were captured, raising overall sensitivity to 93.7% (at 87.1% specificity). Thus, the Warrington Recognition Memory Test-Words, although not originally created for the purposes of measuring suspect effort, appears to be an excellent measure for detecting response bias on neuropsychological testing.

Examination of various WMS-III logical memory scores in the assessment of response bias.

The assessment of response validity during neuropsychological evaluation is an integral part of the testing process. Research has increasingly focused on the use of "embedded" effort measures (derived from standard neuropsychological tasks) because they do not require additional administration time and are less likely to be identified as effort indicators by test takers because of their primary focus as measures of cognitive function. The current study examined the clinical utility of various WMS-III Logical Memory scores in detecting response bias, as well as the Rarely Missed Index, an embedded effort indicator derived from the WMS-III Logical Memory Delayed Recognition subtest. The Rarely Missed Index cut-off only identified 24.1% of 63 non-credible participants (at >/=90% specificity in 125 credible patients), and cut-offs for other Logical Memory variables were in fact found to be more sensitive to non-credible performance. A new indicator, consisting of the weighted combination of the two most sensitive Logical Memory subtest scores (Logical Memory II raw score and Logical Memory Delayed Recognition raw score), was associated with 53% to 60% sensitivity, and thus may be an effective adjunct when utilized in conjunction with other validated effort indicators and collateral information in identifying non-credible performance.

The role of cognitive impairment and caregiver support in diabetes management of older outpatients.

OBJECTIVES: To examine the role of cognitive impairment and caregiver support in diabetes care adherence and glycemic control. METHODS: Fifty-one veteran male outpatients (27 with caregivers) aged 60 years and older with type 2 diabetes were evaluated for cognitive impairment with the Cognitive Abilities Screening Instrument. Patients or caregivers completed diabetes self-care and depression scales. Medical morbidity information and HbA1c plasma levels at baseline and 1 year later were obtained from electronic medical records. RESULTS: Greater cognitive impairment (F = 5.1, p < .05), and presence of a caregiver (F = 5.3, p < .05), were independently associated with worse diabetes care adherence (adjusting for age, education, medical comorbidity, and depression). In addition, Mean HbA1c levels were worse in the cognitively impaired group with caregivers relative to the three other groups (F = 4.10, p < .05, eta2 = .09). One year later, mean HbA1c levels rose from 7.7 to 8.2% in the cognitively impaired group with caregivers. CONCLUSION: Cognitive impairment is associated with worse diabetes care management. Surprisingly, the presence of a caregiver is not protective. Further research is necessary to examine the healthcare needs of cognitively impaired, diabetic patients and their caregivers.

Interpreting the meaning of multiple symptom validity test failure.

While it is recommended that judgments regarding the credibility of test performance be based on the results of more than one effort indicator, and recent efforts have been made to improve interpretation of multiple effort test failure, the field currently lacks adequate guidelines for using multiple measures of effort in concert with one another. A total of 103 patients were referred for outpatient neuropsychological evaluation, which included multiple measures of negative response bias embedded in standard test batteries. Using any pairwise failure combination to predict diagnostic classification was superior (sensitivity = 83.8%, specificity = 93.9%, overall hit rate = 90.3%) to using any one test by itself and to using any three-test failure combination. Further, the results were comparable to the results of logistical regression analyses using the embedded indicators as continuous predictors. Given its parsimony and clinical utility, the pairwise failure model is therefore a recommended criterion for identifying non-credible performance; however, there are of course other important contextual factors and influences to consider, which are also discussed.