IAT faking indices revisited: Aspects of replicability and differential validity.

Research demonstrates that IATs are fakeable. Several indices [either slowing down or speeding up, and increasing errors or reducing errors in congruent and incongruent blocks; Combined Task Slowing (CTS); Ratio 150-10000] have been developed to detect faking. Findings on these are inconclusive, but previous studies have used small samples, suggesting they were statistically underpowered. Further, the stability of the results, the unique predictivity of the indices, the advantage of combining indices, and the dependency on how faking success is computed have yet to be examined. Therefore, we reanalyzed a large data set (N = 750) of fakers and non-fakers who completed an extraversion IAT. Results showed that faking strategies depend on the direction of faking. It was possible to detect faking of low scores due to slowing down on the congruent block, and somewhat less with CTS-both strategies led to faking success. In contrast, the strategy of increasing errors on the congruent block was observed but was not successful in altering the IAT effect in the desired direction. Fakers of high scores could be detected due to slowing down on the incongruent block, increasing errors on the incongruent block, and with CTS-all three strategies led to faking success. The results proved stable in subsamples and generally across different computations of faking success. Using regression analyses and machine learning, increasing errors had the strongest impact on the classification. Apparently, fakers use various goal-dependent strategies and not all are successful. To detect faking, we recommend combining indices depending on the context (and examining convergence).

Challenging response latencies in faking detection: The case of few items and no warnings.

AbstractFaking detection is an ongoing challenge in psychological assessment. A notable approach for detecting fakers involves the inspection of response latencies and is based on the congruence model of faking. According to this model, respondents who fake good will provide favorable responses (i.e., congruent answers) faster than they provide unfavorable (i.e., incongruent) responses. Although the model has been validated in various experimental faking studies, to date, research supporting the congruence model has focused on scales with large numbers of items. Furthermore, in this previous research, fakers have usually been warned that faking could be detected. In view of the trend to use increasingly shorter scales in assessment, it becomes important to investigate whether the congruence model also applies to self-report measures with small numbers of items. In addition, it is unclear whether warning participants about faking detection is necessary for a successful application of the congruence model. To address these issues, we reanalyzed data sets of two studies that investigated faking good and faking bad on extraversion (n = 255) and need for cognition (n = 146) scales. Reanalyses demonstrated that having only a few items per scale and not warning participants represent a challenge for the congruence model. The congruence model of faking was only partly confirmed under such conditions. Although faking good on extraversion was associated with the expected longer latencies for incongruent answers, all other conditions remained nonsignificant. Thus, properties of the measurement and properties of the procedure affect the successful application of the congruence model.

Lying on the Dissection Table: Anatomizing Faked Responses.

Research has shown that even experts cannot detect faking above chance, but recent studies have suggested that machine learning may help in this endeavor. However, faking differs between faking conditions, previous efforts have not taken these differences into account, and faking indices have yet to be integrated into such approaches. We reanalyzed seven data sets (N = 1,039) with various faking conditions (high and low scores, different constructs, naïve and informed faking, faking with and without practice, different measures [self-reports vs. implicit association tests; IATs]). We investigated the extent to which and how machine learning classifiers could detect faking under these conditions and compared different input data (response patterns, scores, faking indices) and different classifiers (logistic regression, random forest, XGBoost). We also explored the features that classifiers used for detection. Our results show that machine learning has the potential to detect faking, but detection success varies between conditions from chance levels to 100%. There were differences in detection (e.g., detecting low-score faking was better than detecting high-score faking). For self-reports, response patterns and scores were comparable with regard to faking detection, whereas for IATs, faking indices and response patterns were superior to scores. Logistic regression and random forest worked about equally well and outperformed XGBoost. In most cases, classifiers used more than one feature (faking occurred over different pathways), and the features varied in their relevance. Our research supports the assumption of different faking processes and explains why detecting faking is a complex endeavor.

Trying to separate the wheat from the chaff: Construct- and faking-related variance on the Implicit Association Test (IAT).

Recent research has indicated that diffusion model analyses allow the user to decompose the traditional IAT effect (D measure) into three newly developed IAT effects: IATv, which has already been shown to be significantly related to the construct-related variance of the IAT effect, and IATa and IATt0, both of which have been assumed to provide an indication of faking. But research on the impacts of faking on IATv, IATa, and IATt0 is still warranted. By reanalyzing a data set containing both faked and unfaked IAT effects, we investigated whether diffusion model analyses could be used to separate construct-related variance from faking-related variance on the IAT. Our results revealed that this separation is not yet possible. As had already been shown for the traditional IAT effect, IATv was affected by faking. Interestingly, it was affected by faking only under more difficult faking conditions (i.e., when participants were asked to fake without being given recommended strategies for how to do so, and when they were requested to fake high scores). By contrast, IATa was affected by faking only in the comparably easy faking condition (i.e., when participants had been informed about possible faking strategies and were asked to fake low scores). IATt0 was not affected by faking at all. Our results show that although diffusion model analyses cannot yet provide a clear separation between construct- and faking-related variance, they allow us to peer into the black box of the faking process itself, and thus provide a useful tool for analyzing and interpreting IAT scores.

Can People With Higher Versus Lower Scores on Impression Management or Self-Monitoring Be Identified Through Different Traces Under Faking?

According to faking models, personality variables and faking are related. Most prominently, people's tendency to try to make an appropriate impression (impression management; IM) and their tendency to adjust the impression they make (self-monitoring; SM) have been suggested to be associated with faking. Nevertheless, empirical findings connecting these personality variables to faking have been contradictory, partly because different studies have given individuals different tests to fake and different faking directions (to fake low vs. high scores). Importantly, whereas past research has focused on faking by examining test scores, recent advances have suggested that the faking process could be better understood by analyzing individuals' responses at the item level (response pattern). Using machine learning (elastic net and random forest regression), we reanalyzed a data set (N = 260) to investigate whether individuals' faked response patterns on extraversion (features; i.e., input variables) could reveal their IM and SM scores. We found that individuals had similar response patterns when they faked, irrespective of their IM scores (excluding the faking of high scores when random forest regression was used). Elastic net and random forest regression converged in revealing that individuals higher on SM differed from individuals lower on SM in how they faked. Thus, response patterns were able to reveal individuals' SM, but not IM. Feature importance analyses showed that whereas some items were faked differently by individuals with higher versus lower SM scores, others were faked similarly. Our results imply that analyses of response patterns offer valuable new insights into the faking process.

A Diffusion Model Approach for Understanding the Impact of 17 Interventions on the Race Implicit Association Test.

Performance on implicit measures reflects construct-specific and nonconstruct-specific processes. This creates an interpretive issue for understanding interventions to change implicit measures: Change in performance could reflect changes in the constructs of interest or changes in other mental processes. We reanalyzed data from six studies (N = 23,342) to examine the process-level effects of 17 interventions and one sham intervention to change race implicit association test (IAT) performance. Diffusion models decompose overall IAT performance (D-scores) into construct-specific (ease of decision-making) and nonconstruct-specific processes (speed-accuracy trade-offs, non-decision-related processes like motor execution). Interventions that effectively reduced D-scores changed ease of decision-making on compatible and incompatible trials. They also eliminated differences in speed-accuracy trade-offs between compatible and incompatible trials. Non-decision-related processes were affected by two interventions only. There was little evidence that interventions had any long-term effects. These findings highlight the value of diffusion modeling for understanding the mechanisms by which interventions affect implicit measure performance.

Exaggeration is harder than understatement, but practice makes perfect!

Previous research on the fakeability of the Implicit Association Test (IAT) yielded inconsistent results. The present study simultaneously analyses several relevant factors: faking direction, type of instructions, and practice. Furthermore, it takes baseline individual differences into account. After a baseline assessment in a self-esteem IAT without faking instructions (t0), participants in the faking conditions then (t1) faked high or low scores without being provided with recommended strategies on how to do so (i.e., individual strategies). At t2 and t3, they were asked to fake the IAT after having received information on recommended faking strategies. At t4, faking direction was reversed. Without the recommended strategies, faking high scores was not possible, but faking low scores was. With the recommended strategies, participants needed additional practice to fake high scores. When faking directions were reversed, participants were successful without additional practice, suggesting a transfer in faking skills. In most of the faking attempts, faking success was moderated by individual differences in baseline implicit self-esteem. This suggests that the complex interplay of factors influencing faking success should be taken into account when considering the issue of fakeability of the IAT.