Performance Validity Test Failure in the Clinical Population: A Systematic Review and Meta-Analysis of Prevalence Rates.

Performance validity tests (PVTs) are used to measure the validity of the obtained neuropsychological test data. However, when an individual fails a PVT, the likelihood that failure truly reflects invalid performance (i.e., the positive predictive value) depends on the base rate in the context in which the assessment takes place. Therefore, accurate base rate information is needed to guide interpretation of PVT performance. This systematic review and meta-analysis examined the base rate of PVT failure in the clinical population (PROSPERO number: CRD42020164128). PubMed/MEDLINE, Web of Science, and PsychINFO were searched to identify articles published up to November 5, 2021. Main eligibility criteria were a clinical evaluation context and utilization of stand-alone and well-validated PVTs. Of the 457 articles scrutinized for eligibility, 47 were selected for systematic review and meta-analyses. Pooled base rate of PVT failure for all included studies was 16%, 95% CI [14, 19]. High heterogeneity existed among these studies (Cochran's Q = 697.97, p < .001; I2 = 91%; τ2 = 0.08). Subgroup analysis indicated that pooled PVT failure rates varied across clinical context, presence of external incentives, clinical diagnosis, and utilized PVT. Our findings can be used for calculating clinically applied statistics (i.e., positive and negative predictive values, and likelihood ratios) to increase the diagnostic accuracy of performance validity determination in clinical evaluation. Future research is necessary with more detailed recruitment procedures and sample descriptions to further improve the accuracy of the base rate of PVT failure in clinical practice.

Performance Validity and Outcome of Cognitive Behavior Therapy in Patients with Chronic Fatigue Syndrome.

OBJECTIVE: There is limited research examining the impact of the validity of cognitive test performance on treatment outcome. All known studies to date have operationalized performance validity dichotomously, leading to the loss of predictive information. Using the range of scores on a performance validity test (PVT), we hypothesized that lower performance at baseline was related to a worse treatment outcome following cognitive behavioral therapy (CBT) in patients with Chronic Fatigue Syndrome (CFS) and to lower adherence to treatment. METHOD: Archival data of 1081 outpatients treated with CBT for CFS were used in this study. At baseline, all patients were assessed with a PVT, the Amsterdam Short-Term Memory test (ASTM). Questionnaires assessing fatigue, physical disabilities, psychological distress, and level of functional impairment were administered before and after CBT. RESULTS: Our main hypothesis was not confirmed: the total ASTM score was not significantly associated with outcomes at follow-up. However, patients with a missing follow-up assessment had a lower ASTM performance at baseline, reported higher levels of physical limitations, and completed fewer therapy sessions. CONCLUSIONS: CFS patients who scored low on the ASTM during baseline assessment are more likely to complete fewer therapy sessions and not to complete follow-up assessment, indicative of limited adherence to treatment. However, if these patients were retained in the intervention, their response to CBT for CFS was comparable with subjects who score high on the ASTM. This finding calls for more research to better understand the impact of performance validity on engagement with treatment and outcomes.

Providing a brief corrective statement does not improve test performance in patients invalidating testing: A multisite, single-blind randomized controlled trial.

Objective: Performance below the actual abilities of the examinee can be measured using performance validity tests (PVTs). PVT failure negatively impacts the quality of the neuropsychological assessment. In our study, we addressed this issue by providing a brief corrective statement regarding invalidity to improve test-taking behavior. Methods: This study is a multisite single-blind randomized controlled trial in a consecutive sample of clinically referred adult patients (N = 196) in a general hospital setting. Patients who failed a PVT (n = 71) were randomly allocated to a corrective statement approach (CS; n = 39), in which a brief verbal corrective statement was given by the technician, or received no corrective statement upon indications of invalid performance (NO-CS; n = 32). Both groups (CS and NO-CS) were provided with the same subsequently repeated and newly administered tests. Results: There were no group (CS vs. NO-CS) differences on both the repeated and single-administered PVTs and standard cognitive tests. Furthermore, invalid performing participants benefited significantly less from the repeated test administration compared to the valid performing group. Conclusions: Our study found that a brief corrective within-session statement, to address PVT failure and improve test-taking behavior, did not improve consequent test performance. These results suggest limited value of a brief verbal corrective statement to influence performance below best of capabilities. It highlights the need for more research to identify more effective approaches that can enhance patients test-taking behavior. Ultimately, such efforts are critical in ensuring accurate diagnosis and effective treatment recommendations for patients.

Feedback on underperformance in patients with Chronic Fatigue Syndrome: The impact on subsequent neuropsychological test performance.

Performance Validity Tests (PVTs) are used to measure the credibility of neuropsychological test results. Until now, however, a minimal amount is known about the effects of feedback upon noncredible results (i.e., underperformance) on subsequent neuropsychological test performance. The purpose of this study was to investigate the effects of feedback on underperformance in Chronic Fatigue Syndrome (CFS) patients. A subset of these patients received feedback on Amsterdam Short-Term Memory (ASTM) failure (i.e., feedback [FB] group). After matching, the final sample consisted of two comparable groups (i.e., FB and No FB; both n = 33). At baseline and follow-up assessment, the patients completed the ASTM and two measurements of information processing speed (Complex Reaction Time [CRT] and Symbol Digit Test [SDT]). Results indicated that the patients in the FB group improved significantly on the CRT, compared to the No FB group. Although not significant, a comparable trend-like effect was observed for the SDT. Independent of the feedback intervention there was a substantial improvement on ASTM performance at re-administration. A limited feedback intervention upon underperformance in CFS patients may result in improvement on information processing speed performance. This implies that such an intervention might be clinically relevant, since it maximizes the potential of examining the patients' actual level of cognitive abilities.

A case of misdiagnosis of mild cognitive impairment: The utility of symptom validity testing in an outpatient memory clinic.

Noncredible symptom reports hinder the diagnostic process. This fact is especially the case for medical conditions that rely on subjective report of symptoms instead of objective measures. Mild cognitive impairment (MCI) primarily relies on subjective report, which makes it potentially susceptible to erroneous diagnosis. In this case report, we describe a 59-year-old female patient diagnosed with MCI 10 years previously. The patient was referred to the neurology department for reexamination by her general practitioner because of cognitive complaints and persistent fatigue. This case study used information from the medical file, a new magnetic resonance imaging brain scan, and neuropsychological assessment. Current neuropsychological assessment, including symptom validity tests, clearly indicated noncredible test performance, thereby invalidating the obtained neuropsychological test data. We conclude that a blind spot for noncredible symptom reports existed in the previous diagnostic assessments. This case highlights the usefulness of formal symptom validity testing in the diagnostic assessment of MCI.