ABSTRACT
OBJECTIVE: The evaluation of self-reported symptoms is a standard component of concussion assessment and management. Clinicians typically evaluate a total symptom severity score rather than scores corresponding to specific symptom domains (i.e., cognitive, sleep-arousal, physical, and affective symptoms). This study examined (i) whether elevations in specific symptom domains would be missed when interpreting only the total symptom severity score and (ii) if a single symptom domain elevation was more common than having elevated symptoms across multiple domains. METHOD: Adolescent student-athletes (N = 1,008) with concussion history (i.e., ≥6 months since last concussion) completed the Post-Concussion Symptom Scale (PCSS). The PCSS total score and cognitive, sleep-arousal, physical, and affective domain scores were calculated. To determine if symptoms were elevated, scores were compared to normative data matched on gender and pre-existing conditions, with scores considered elevated if they were ≥84th percentile. The frequency of total and domain score elevations were calculated and stratified by gender and number of prior concussions (i.e., 1 or ≥2 prior concussions). RESULTS: Overall, 26% of student-athletes had an elevated symptom domain score without being elevated on the total score. The most common symptom presentation was to have a single elevated symptom domain (21%), followed by two (11%), three (8%), or four elevated domains (6%). CONCLUSIONS: Interpreting PCSS symptom domains may be beneficial in detecting student-athletes with elevated symptoms following a remote concussion. Roughly a quarter of student-athletes have domain-specific symptom elevations that would be missed by interpreting the total score alone.
ABSTRACT
This study was designed to expand on a recent meta-analysis that identified ≤42 as the optimal cutoff on the Word Choice Test (WCT). We examined the base rate of failure and the classification accuracy of various WCT cutoffs in four independent clinical samples (N = 252) against various psychometrically defined criterion groups. WCT ≤ 47 achieved acceptable combinations of specificity (.86-.89) at .49 to .54 sensitivity. Lowering the cutoff to ≤45 improved specificity (.91-.98) at a reasonable cost to sensitivity (.39-.50). Making the cutoff even more conservative (≤42) disproportionately sacrificed sensitivity (.30-.38) for specificity (.98-1.00), while still classifying 26.7% of patients with genuine and severe deficits as non-credible. Critical item (.23-.45 sensitivity at .89-1.00 specificity) and time-to-completion cutoffs (.48-.71 sensitivity at .87-.96 specificity) were effective alternative/complementary detection methods. Although WCT ≤ 45 produced the best overall classification accuracy, scores in the 43 to 47 range provide comparable objective psychometric evidence of non-credible responding. Results question the need for designating a single cutoff as "optimal," given the heterogeneity of signal detection environments in which individual assessors operate. As meta-analyses often fail to replicate, ongoing research is needed on the classification accuracy of various WCT cutoffs.
Subject(s)
Neuropsychological Tests , Humans , Sensitivity and Specificity , Psychometrics , Reproducibility of ResultsABSTRACT
BACKGROUND: The Post-Concussion Symptom Scale (PCSS) is a self-report questionnaire measuring symptoms that commonly occur after a concussion; however, these symptoms are nonspecific and can be related to co-occurring orthopaedic injuries (eg, cervical strain) or patient characteristics and preexisting conditions, even in the absence of a recent injury. As such, clinicians may have difficulty determining whether symptom elevations are attributable to a recent concussion as opposed to a confounding injury or a preexisting condition, which may be especially difficult when preinjury baseline symptom data are unavailable. PURPOSE: This study aimed to further validate the 4-factor model of the PCSS (ie, cognitive, sleep-arousal, physical, and affective symptoms) with adolescent student-athletes and provide normative reference data for each factor and the total score, stratified by gender and preexisting health conditions. STUDY DESIGN: Cross-sectional study; Level of evidence, 3. METHODS: Participants were 9358 adolescent student-athletes who completed the PCSS during a preseason baseline evaluation (mean age, 14.9 years; SD, 1.3 years [range, 13-18 years]; 49.3% boys). The 4-factor model of the PCSS was tested for the full sample and separately for boys and girls using confirmatory factor analysis. Symptom severity percentiles were created for the PCSS total score and each factor, stratified by gender and preexisting conditions (ie, attention-deficit/hyperactivity disorder, mental health history, headache/migraine history, learning disability/dyslexia, academic problems, and concussion history). RESULTS: The 4-factor model of the PCSS replicated in the full sample (comparative fit index [CFI] = 0.959) and in both gender groups (boys: CFI = 0.961; girls: CFI = 0.960). The total PCSS score at the 84th percentile varied by preexisting conditions as follows: healthy participants = 8, attention-deficit/hyperactivity disorder = 18, mental health history = 26, headache/migraine history = 18, learning disability = 19, and academic problems = 17. On all PCSS subscales, participants with a mental health history had the highest scores, and high scores were associated with having >1 preexisting condition. Girls had higher scores than boys for each stratification. CONCLUSION: The 4-factor model of the PCSS replicates for adolescent student-athletes. Gender, number of preexisting conditions, and mental health history are important factors to account for when interpreting PCSS symptom severity. The normative data provided herein could assist clinicians in determining whether an adolescent student-athlete is presenting with persistent postconcussion symptoms or a typical symptom experience based on their gender and personal health history.
Subject(s)
Athletic Injuries , Brain Concussion , Learning Disabilities , Migraine Disorders , Post-Concussion Syndrome , Male , Female , Humans , Adolescent , Post-Concussion Syndrome/diagnosis , Preexisting Condition Coverage , Athletic Injuries/diagnosis , Athletic Injuries/complications , Cross-Sectional Studies , Neuropsychological Tests , Brain Concussion/diagnosis , Brain Concussion/complications , Athletes , Learning Disabilities/complications , Students , Headache/complicationsABSTRACT
ObjectiveThe study was designed to expand on the results of previous investigations on the D-KEFS Stroop as a performance validity test (PVT), which produced diverging conclusions. Method The classification accuracy of previously proposed validity cutoffs on the D-KEFS Stroop was computed against four different criterion PVTs in two independent samples: patients with uncomplicated mild TBI (n = 68) and disability benefit applicants (n = 49). Results Age-corrected scaled scores (ACSSs) ≤6 on individual subtests often fell short of specificity standards. Making the cutoffs more conservative improved specificity, but at a significant cost to sensitivity. In contrast, multivariate models (≥3 failures at ACSS ≤6 or ≥2 failures at ACSS ≤5 on the four subtests) produced good combinations of sensitivity (.39-.79) and specificity (.85-1.00), correctly classifying 74.6-90.6% of the sample. A novel validity scale, the D-KEFS Stroop Index correctly classified between 78.7% and 93.3% of the sample. Conclusions A multivariate approach to performance validity assessment provides a methodological safeguard against sample- and instrument-specific fluctuations in classification accuracy, strikes a reasonable balance between sensitivity and specificity, and mitigates the invalid before impaired paradox.
Subject(s)
Patients , Humans , Neuropsychological Tests , Psychometrics , Reproducibility of Results , Sensitivity and SpecificityABSTRACT
Base rates of failure (BRFail) on performance validity tests (PVTs) were examined in university students with limited English proficiency (LEP). BRFail was calculated for several free-standing and embedded PVTs. All free-standing PVTs and certain embedded indicators were robust to LEP. However, LEP was associated with unacceptably high BRFail (20-50%) on several embedded PVTs with high levels of verbal mediation (even multivariate models of PVT could not contain BRFail). In conclusion, failing free-standing/dedicated PVTs cannot be attributed to LEP. However, the elevated BRFail on several embedded PVTs in university students suggest an unacceptably high overall risk of false positives associated with LEP.
Subject(s)
Limited English Proficiency , Humans , Neuropsychological Tests , Cross-Cultural Comparison , Reproducibility of ResultsABSTRACT
This study was designed to examine alternative validity cutoffs on the Boston Naming Test (BNT).Archival data were collected from 206 adults assessed in a medicolegal setting following a motor vehicle collision. Classification accuracy was evaluated against three criterion PVTs.The first cutoff to achieve minimum specificity (.87-.88) was T ≤ 35, at .33-.45 sensitivity. T ≤ 33 improved specificity (.92-.93) at .24-.34 sensitivity. BNT validity cutoffs correctly classified 67-85% of the sample. Failing the BNT was unrelated to self-reported emotional distress. Although constrained by its low sensitivity, the BNT remains a useful embedded PVT.
Subject(s)
Emotions , Adult , Humans , Language Tests , Neuropsychological Tests , Reproducibility of Results , Self ReportABSTRACT
OBJECTIVE: This project was designed to cross-validate existing performance validity cutoffs embedded within measures of verbal fluency (FAS and animals) and develop new ones for the Emotion Word Fluency Test (EWFT), a novel measure of category fluency. METHOD: The classification accuracy of the verbal fluency tests was examined in two samples (70 cognitively healthy university students and 52 clinical patients) against psychometrically defined criterion measures. RESULTS: A demographically adjusted T-score of ≤31 on the FAS was specific (.88-.97) to noncredible responding in both samples. Animals T ≤ 29 achieved high specificity (.90-.93) among students at .27-.38 sensitivity. A more conservative cutoff (T ≤ 27) was needed in the patient sample for a similar combination of sensitivity (.24-.45) and specificity (.87-.93). An EWFT raw score ≤5 was highly specific (.94-.97) but insensitive (.10-.18) to invalid performance. Failing multiple cutoffs improved specificity (.90-1.00) at variable sensitivity (.19-.45). CONCLUSIONS: Results help resolve the inconsistency in previous reports, and confirm the overall utility of existing verbal fluency tests as embedded validity indicators. Multivariate models of performance validity assessment are superior to single indicators. The clinical utility and limitations of the EWFT as a novel measure are discussed.
Subject(s)
Emotions , Humans , Neuropsychological Tests , Reproducibility of ResultsABSTRACT
OBJECTIVE: This study was designed to evaluate the classification accuracy of the recently introduced forced-choice recognition trial to the Hopkins Verbal Learning Test - Revised (FCRHVLT-R) as a performance validity test (PVT) in a clinical sample. Time-to-completion (T2C) for FCRHVLT-R was also examined. METHOD: Forty-three students were assigned to either the control or the experimental malingering (expMAL) condition. Archival data were collected from 52 adults clinically referred for neuropsychological assessment. Invalid performance was defined using expMAL status, two free-standing PVTs and two validity composites. RESULTS: Among students, FCRHVLT-R ≤11 or T2C ≥45 seconds was specific (0.86-0.93) to invalid performance. Among patients, an FCRHVLT-R ≤11 was specific (0.94-1.00), but relatively insensitive (0.38-0.60) to non-credible responding0. T2C ≥35 s produced notably higher sensitivity (0.71-0.89), but variable specificity (0.83-0.96). The T2C achieved superior overall correct classification (81-86%) compared to the accuracy score (68-77%). The FCRHVLT-R provided incremental utility in performance validity assessment compared to previously introduced validity cutoffs on Recognition Discrimination. CONCLUSIONS: Combined with T2C, the FCRHVLT-R has the potential to function as a quick, inexpensive and effective embedded PVT. The time-cutoff effectively attenuated the low ceiling of the accuracy scores, increasing sensitivity by 19%. Replication in larger and more geographically and demographically diverse samples is needed before the FCRHVLT-R can be endorsed for routine clinical application.
Subject(s)
Malingering , Recognition, Psychology , Adult , Humans , Malingering/diagnosis , Neuropsychological Tests , Psychometrics , Reproducibility of Results , Verbal LearningABSTRACT
This study was designed to cross-validate the V-5, a quick psychiatric screener, across administration formats and levels of examinee acculturation. The V-5 was administered twice (once at the beginning and once at the end of the testing session) to three samples (N = 277) with varying levels of symptom severity and English language proficiency, varying type of administration, alongside traditional self-reported symptom inventories as criterion measures. The highest rest-retest reliability was observed on the Depression (.84) and Pain scales (.85). The V-5 was sensitive to the variability in symptom severity. Classification accuracy was driven by the base rate of the target construct, and was invariant across administration format (in-person or online) or level of English proficiency. The V-5 demonstrated promise as a cross-culturally robust screening instrument that is sensitive to change over time, lends itself to online administration, and is suitable for examinees with limited English proficiency.
Subject(s)
Mental Disorders , Humans , Language , Pain Measurement , Psychometrics , Reproducibility of Results , Surveys and QuestionnairesABSTRACT
OBJECTIVE: This study was designed to replicate previous research on embedded validity indicators (EVIs) in the Hopkins Verbal Learning Test - Revised (HVLT-R) and introduce a new forced choice recognition trial (FCR). METHOD: Undergraduate research volunteers were randomly assigned (MAge = 21.7; MEducation = 14.5 years, 85% female) to either the control or experimental malingering condition, and were administered a brief battery of neuropsychological tests. RESULTS: Recognition memory based EVIs (both existing and newly introduced) effectively discriminated credible and non-credible response sets. An FCR ≤11 produced .59 sensitivity and perfect specificity to invalid responding. A Recognition Discrimination (RD) score ≤8 also produced a good combination of sensitivity (.35) and specificity (.96). The FCR trial made unique contributions to performance validity assessment above and beyond previously published EVIs. CONCLUSIONS: RD achieved ≥.90 specificity at higher cutoffs than previously reported. The newly introduced FCR trial has the potential to enhance the existing arsenal of EVIs within the HVLT-R. However, it must demonstrate its ability to differentiate genuine impairment from non-credible responding before it can be recommended for clinical use.
Subject(s)
Malingering , Recognition, Psychology , Adult , Female , Humans , Male , Malingering/diagnosis , Neuropsychological Tests , Students , Verbal Learning , Young AdultABSTRACT
OBJECTIVE: This study was designed to evaluate the classification accuracy of a multivariate model of performance validity assessment using embedded validity indicators (EVIs) within the Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV). METHOD: Archival data were collected from 100 adults with traumatic brain injury (TBI) consecutively referred for neuropsychological assessment in a clinical setting. The classification accuracy of previously published individual EVIs nested within the WAIS-IV and a composite measure based on six independent EVIs were evaluated against psychometrically defined non-credible performance. RESULTS: Univariate validity cutoffs based on age-corrected scaled scores on Coding, Symbol Search, Digit Span, Letter-Number-Sequencing, Vocabulary minus Digit Span, and Coding minus Symbol Search were strong predictors of psychometrically defined non-credible responding. Failing ≥3 of these six EVIs at the liberal cutoff improved specificity (.91-.95) over univariate cutoffs (.78-.93). Conversely, failing ≥2 EVIs at the more conservative cutoff increased and stabilized sensitivity (.43-.67) compared to univariate cutoffs (.11-.63) while maintaining consistently high specificity (.93-.95). CONCLUSIONS: In addition to being a widely used test of cognitive functioning, the WAIS-IV can also function as a measure of performance validity. Consistent with previous research, combining information from multiple EVIs enhanced the classification accuracy of individual cutoffs and provided more stable parameter estimates. If the current findings are replicated in larger, diagnostically and demographically heterogeneous samples, the WAIS-IV has the potential to become a powerful multivariate model of performance validity assessment. BRIEF SUMMARY: Using a combination of multiple performance validity indicators embedded within the subtests of theWechsler Adult Intelligence Scale, the credibility of the response set can be establishedwith a high level of confidence. Multivariatemodels improve classification accuracy over individual tests. Relying on existing test data is a cost-effective approach to performance validity assessment.
Subject(s)
Brain Injuries, Traumatic/diagnosis , Cognitive Dysfunction/diagnosis , Neuropsychological Tests/standards , Psychometrics/standards , Wechsler Scales/standards , Adult , Brain Injuries, Traumatic/complications , Cognitive Dysfunction/etiology , Female , Humans , Male , Middle Aged , Reproducibility of ResultsABSTRACT
OBJECTIVES: Obstructive sleep apnea (OSA) is associated with cognitive impairment but the relationships between specific biomarkers and neurocognitive domains remain unclear. The present study examined the influence of common health comorbidities on these relationships. Adults with suspected OSA (N=60; 53% male; M age=52 years; SD=14) underwent neuropsychological evaluation before baseline polysomnography (PSG). Apneic syndrome severity, hypoxic strain, and sleep architecture disturbance were assessed through PSG. METHODS: Depression (Center for Epidemiological Studies Depression Scale, CESD), pain, and medical comorbidity (Charlson Comorbidity Index) were measured via questionnaires. Processing speed, attention, vigilance, memory, executive functioning, and motor dexterity were evaluated with cognitive testing. A winnowing approach identified 9 potential moderation models comprised of a correlated PSG variable, comorbid health factor, and cognitive performance. RESULTS: Regression analyses identified one significant moderation model: average blood oxygen saturation (AVO2) and depression predicting recall memory, accounting for 31% of the performance variance, p<.001. Depression was a significant predictor of recall memory, p<.001, but AVO2 was not a significant predictor. The interaction between depression and AVO2 was significant, accounting for an additional 10% of the variance, p<.001. The relationship between low AVO2 and low recall memory performance emerged when depression severity ratings approached a previously established clinical cutoff score (CESD=16). CONCLUSIONS: This study examined sleep biomarkers with specific neurocognitive functions among individuals with suspected OSA. Findings revealed that depression burden uniquely influence this pathophysiological relationship, which may aid clinical management. (JINS, 2018, 28, 864-875).
Subject(s)
Cognition , Sleep Apnea, Obstructive/complications , Sleep Apnea, Obstructive/psychology , Sleep , Adult , Aged , Arousal , Attention , Biomarkers , Cognitive Dysfunction , Comorbidity , Depression/complications , Depression/psychology , Executive Function , Female , Humans , Male , Mental Recall , Middle Aged , Neuropsychological Tests , Oxygen/blood , Polysomnography , Psychomotor Performance , Reaction Time , Young AdultABSTRACT
Importance: Estimated base rates of invalid performance on baseline testing (base rates of failure) for the management of sport-related concussion range from 6.1% to 40.0%, depending on the validity indicator used. The instability of this key measure represents a challenge in the clinical interpretation of test results that could undermine the utility of baseline testing. Objectives: To determine the prevalence of invalid performance on baseline testing and to assess whether the prevalence varies as a function of age and validity indicator. Design, Setting, and Participants: This retrospective, cross-sectional study included data collected between January 1, 2012, and December 31, 2016, from a clinical referral center in the Midwestern United States. Participants included 7897 consecutively tested, equivalently proportioned male and female athletes aged 10 to 21 years, who completed baseline neurocognitive testing for the purpose of concussion management. Interventions: Baseline assessment was conducted with the Immediate Postconcussion Assessment and Cognitive Testing (ImPACT), a computerized neurocognitive test designed for assessment of concussion. Main Outcomes and Measures: Base rates of failure on published ImPACT validity indicators were compared within and across age groups. Hypotheses were developed after data collection but prior to analyses. Results: Of the 7897 study participants, 4086 (51.7%) were male, mean (SD) age was 14.71 (1.78) years, 7820 (99.0%) were primarily English speaking, and the mean (SD) educational level was 8.79 (1.68) years. The base rate of failure ranged from 6.4% to 47.6% across individual indicators. Most of the sample (55.7%) failed at least 1 of 4 validity indicators. The base rate of failure varied considerably across age groups (117 of 140 [83.6%] for those aged 10 years to 14 of 48 [29.2%] for those aged 21 years), representing a risk ratio of 2.86 (95% CI, 2.60-3.16; P < .001). Conclusions and Relevance: The results for base rate of failure were surprisingly high overall and varied widely depending on the specific validity indicator and the age of the examinee. The strong age association, with 3 of 4 participants aged 10 to 12 years failing validity indicators, suggests that the clinical interpretation and utility of baseline testing in this age group is questionable. These findings underscore the need for close scrutiny of performance validity indicators on baseline testing across age groups.
Subject(s)
Athletic Injuries/diagnosis , Athletic Injuries/psychology , Brain Concussion/diagnosis , Brain Concussion/psychology , Mental Status and Dementia Tests/standards , Adolescent , Athletic Injuries/epidemiology , Brain Concussion/epidemiology , Child , Cross-Sectional Studies , Female , Humans , Male , Neuropsychological Tests/standards , Prevalence , Reproducibility of Results , Retrospective Studies , Sports/psychology , Young AdultABSTRACT
OBJECTIVE: The Forced Choice Recognition (FCR) trial of the California Verbal Learning Test-Second Edition (CVLT-II) was designed to serve as a performance validity test (PVT). The present study was designed to compare the classification accuracy of a more liberal alternative (≤15) to the de facto FCR cutoff (≤14). METHOD: The classification accuracy of the two cutoffs was computed in reference to psychometrically defined invalid performance, across various criterion measures, in a sample of 104 adults with TBI clinically referred for neuropsychological assessment. RESULTS: The FCR was highly predictive (AUC: .71-.83) of Pass/Fail status on reference PVTs, but unrelated to performance on measures known to be sensitive to TBI. On average, FCR ≤15 correctly identified an additional 6% of invalid response sets compared to FCR ≤14, while maintaining .92 specificity. Patients who failed the FCR reported higher levels of emotional distress. CONCLUSIONS: Results suggest that even a single error on the FCR is a reliable indicator of invalid responding. Further research is needed to investigate the clinical significance of the relationship between failing the FCR and level of self-reported psychiatric symptoms.
Subject(s)
Brain Injuries, Traumatic/psychology , Choice Behavior/physiology , Neuropsychological Tests , Recognition, Psychology/physiology , Adult , Female , Humans , Male , Middle Aged , Psychometrics , Sensitivity and Specificity , Young AdultABSTRACT
This study was designed to examine the clinical utility of critical items within the Recognition Memory Test (RMT) and the Word Choice Test (WCT). Archival data were collected from a mixed clinical sample of 202 patients clinically referred for neuropsychological testing (54.5% male; mean age = 45.3 years; mean level of education = 13.9 years). The credibility of a given response set was psychometrically defined using three separate composite measures, each of which was based on multiple independent performance validity indicators. Critical items improved the classification accuracy of both tests. They increased sensitivity by correctly identifying an additional 2-17% of the invalid response sets that passed the traditional cutoffs based on total score. They also increased specificity by providing additional evidence of noncredible performance in response sets that failed the total score cutoff. The combination of failing the traditional cutoff, but passing critical items was associated with increased risk of misclassifying the response set as invalid. Critical item analysis enhances the diagnostic power of both the RMT and WCT. Given that critical items require no additional test material or administration time, but help reduce both false positive and false negative errors, they represent a versatile, valuable, and time- and cost-effective supplement to performance validity assessment.
Subject(s)
Choice Behavior/physiology , Memory Disorders/diagnosis , Memory Disorders/physiopathology , Neuropsychological Tests , Recognition, Psychology/physiology , Adult , Aged , Analysis of Variance , Female , Humans , Male , Malingering , Middle Aged , Psychometrics , Reproducibility of Results , Sensitivity and Specificity , Signal Detection, PsychologicalABSTRACT
INTRODUCTION: The Recognition Memory Test (RMT) and Word Choice Test (WCT) are structurally similar, but psychometrically different. Previous research demonstrated that adding a time-to-completion cutoff improved the classification accuracy of the RMT. However, the contribution of WCT time-cutoffs to improve the detection of invalid responding has not been investigated. The present study was designed to evaluate the classification accuracy of time-to-completion on the WCT compared to the accuracy score and the RMT. METHOD: Both tests were administered to 202 adults (Mage = 45.3 years, SD = 16.8; 54.5% female) clinically referred for neuropsychological assessment in counterbalanced order as part of a larger battery of cognitive tests. RESULTS: Participants obtained lower and more variable scores on the RMT (M = 44.1, SD = 7.6) than on the WCT (M = 46.9, SD = 5.7). Similarly, they took longer to complete the recognition trial on the RMT (M = 157.2 s,SD = 71.8) than the WCT (M = 137.2 s, SD = 75.7). The optimal cutoff on the RMT (≤43) produced .60 sensitivity at .87 specificity. The optimal cutoff on the WCT (≤47) produced .57 sensitivity at .87 specificity. Time-cutoffs produced comparable classification accuracies for both RMT (≥192 s; .48 sensitivity at .88 specificity) and WCT (≥171 s; .49 sensitivity at .91 specificity). They also identified an additional 6-10% of the invalid profiles missed by accuracy score cutoffs, while maintaining good specificity (.93-.95). Functional equivalence was reached at accuracy scores ≤43 (RMT) and ≤47 (WCT) or time-to-completion ≥192 s (RMT) and ≥171 s (WCT). CONCLUSIONS: Time-to-completion cutoffs are valuable additions to both tests. They can function as independent validity indicators or enhance the sensitivity of accuracy scores without requiring additional measures or extending standard administration time.
Subject(s)
Choice Behavior/physiology , Neuropsychological Tests , Recognition, Psychology/physiology , Adult , Female , Humans , Male , Middle Aged , Psychometrics , Sensitivity and Specificity , Time FactorsABSTRACT
Research suggests that select processing speed measures can also serve as embedded validity indicators (EVIs). The present study examined the diagnostic utility of Wechsler Adult Intelligence Scale-Fourth Edition (WAIS-IV) subtests as EVIs in a mixed clinical sample of 205 patients medically referred for neuropsychological assessment (53.3% female, mean age = 45.1). Classification accuracy was calculated against 3 composite measures of performance validity as criterion variables. A PSI ≤79 produced a good combination of sensitivity (.23-.56) and specificity (.92-.98). A Coding scaled score ≤5 resulted in good specificity (.94-1.00), but low and variable sensitivity (.04-.28). A Symbol Search scaled score ≤6 achieved a good balance between sensitivity (.38-.64) and specificity (.88-.93). A Coding-Symbol Search scaled score difference ≥5 produced adequate specificity (.89-.91) but consistently low sensitivity (.08-.12). A 2-tailed cutoff on the Coding/Symbol Search raw score ratio (≤1.41 or ≥3.57) produced acceptable specificity (.87-.93), but low sensitivity (.15-.24). Failing ≥2 of these EVIs produced variable specificity (.81-.93) and sensitivity (.31-.59). Failing ≥3 of these EVIs stabilized specificity (.89-.94) at a small cost to sensitivity (.23-.53). Results suggest that processing speed based EVIs have the potential to provide a cost-effective and expedient method for evaluating the validity of cognitive data. Given their generally low and variable sensitivity, however, they should not be used in isolation to determine the credibility of a given response set. They also produced unacceptably high rates of false positive errors in patients with moderate-to-severe head injury. Combining evidence from multiple EVIs has the potential to improve overall classification accuracy. (PsycINFO Database Record
Subject(s)
Brain Injuries, Traumatic/psychology , Cognition , Epilepsy/psychology , Mental Disorders/psychology , Stroke/psychology , Wechsler Scales , Adult , Female , Humans , Intelligence , Male , Middle Aged , Neuropsychological Tests , Reproducibility of ResultsABSTRACT
Despite growing affective-memory research, only 2 potential clinical measures have been published, each with limitations. We describe the development and piloting of an integrated memory measure for neutral and affectively valenced words, the Cognitive-Affective Verbal Learning Test (C-AVLT). The C-AVLT and mood self-report measures were administered to 124 healthy university students in Study 1, with readministration to 40 students after 1 week. In Study 2, the C-AVLT and other neuropsychological measures of memory and emotion were administered to 61 patients referred for polysomnogram evaluation of obstructive sleep apnea (OSA). Study 1 supported the C-AVLT's internal and test-retest reliabilities, as well as concurrent validity, that is, the affective-bias scores but not performance scores correlated with self-reported mood. In Study 2, convergent, criterion (specifically cross-sectional concurrent validity), and incremental validity were supported with regard to both performance and affective-bias scores within the OSA sample. We demonstrated the C-AVLT is a reliable and clinically useful measure of both memory and affective-processing bias in 2 samples. Future clinical and research recommendations for the C-AVLT are discussed, including broadening normative data and criterion validity data in psychiatric and neurological samples. (PsycINFO Database Record
Subject(s)
Affect , Cognition , Emotions , Sleep Apnea, Obstructive/psychology , Students/psychology , Verbal Learning , Adolescent , Adult , Aged , Cross-Sectional Studies , Female , Humans , Male , Middle Aged , Neuropsychological Tests , Reproducibility of Results , Young AdultABSTRACT
OBJECTIVE: This study compared failure rates on performance validity tests (PVTs) across liberal and conservative cutoffs in a sample of undergraduate students participating in academic research. METHOD: Participants (n = 120) were administered four free-standing PVTs (Test of Memory Malingering, Word Memory Test, Rey 15-Item Test, Hiscock Forced-Choice Procedure) and three embedded PVTs (Digit Span, letter and category fluency). Participants also reported their perceived level of effort during testing. RESULTS: At liberal cutoffs, 36.7% of the sample failed ≥1 PVTs, 6.7% failed ≥2, and .8% failed 3. At conservative cutoffs, 18.3% of the sample failed ≥1 PVTs, 2.5% failed ≥2, and .8% failed 3. Participants were 3 to 5 times more likely to fail embedded (15.8-30.8%) compared to free-standing PVTs (3.3-10.0%). There was no significant difference in failure rates between native and non-native English speaking participants at either liberal or conservative cutoffs. Additionally, there was no relation between self-reported effort and PVT failure rates. CONCLUSIONS: Although PVT failure rates varied as a function of PVTs and cutoffs, between a third and a fifth of the sample failed ≥1 PVTs, consistent with high initial estimates of invalid performance in this population. Embedded PVTs had notably higher failure rates than free-standing PVTs. Assuming optimal effort in research using students as participants without a formal assessment of performance validity introduces a potentially significant confound in the study design.
Subject(s)
Motivation , Neuropsychological Tests , Students/psychology , Adult , Female , Healthy Volunteers , Humans , Male , Malingering/diagnosis , Memory , Perception , Reproducibility of Results , Research Design , Self Report , Young AdultABSTRACT
INTRODUCTION: The clinical assessment of affective functioning has been slow to incorporate findings from affective neuroscience. Of particular interest in the current study is the assessment of affective word production. METHOD: In a series of four studies, we examined test-retest and interrater reliability for the Emotion Word Fluency Test (EWFT), basic construct validity with existing verbal fluency measures, physiological responses across verbal fluency tasks, and a novel scoring method to examine qualitative aspects of participant response sets. RESULTS: Results demonstrated interrater and test-retest reliability values that were comparable to those of other commonly used verbal fluency tests. Construct validity was demonstrated by relations between the EWFT and other verbal fluency tests as well as through physiological evidence that performance on the EWFT is related to greater sympathetic activity than traditional verbal fluency tasks. Lastly, some of the novel scoring metrics related to two self-report measures of emotional functioning. CONCLUSIONS: Taken together, our findings provide initial support for the use of the EWFT as a measure of emotion word generation ability in young adults. This measure may prove to be useful in the assessment of affective language production in patient populations.