A Causal View on Bias in Missing Data Imputation: The Impact of Evil Auxiliary Variables on Norming of Test Scores.

Among the most important merits of modern missing data techniques such as multiple imputation (MI) and full-information maximum likelihood estimation is the possibility to include additional information about the missingness process via auxiliary variables. During the past decade, the choice of auxiliary variables has been investigated under a variety of different conditions and more recent research points to the potentially biasing effect of certain auxiliary variables, particularly colliders (Thoemmes & Rose, 2014). In this article, we further extend biasing mechanisms of certain auxiliary variables considered in previous research and thereby focus on their effects on individual diagnosis based on norming, in which the whole distribution of a variable is of interest rather than average coefficients (e.g., means). For this, we first provide the theoretical underpinnings of the mechanisms under study and then provide two focused simulations that (i) directly expand on the collider scenario in Thoemmes and Rose (2014, appendix A) by considering outcomes that are relevant to norming and (ii) extend the scenarios under consideration by instrumental variable mechanisms. We illustrate the bias mechanisms for two different norming approaches and exemplify the procedures by means of an empirical example. We end by discussing limitations and implications of our research.

The Predictive Validity of Item Effect Variables in the Satisfaction With Life Scale for Psychological and Physical Health.

Although the Satisfaction with Life Scale strives to capture a single dimension, describing respondents' satisfaction with life as a whole, individual items might also capture unique aspects of life satisfaction leading to some form of multidimensionality. Such systematic item-specific variance can be viewed as a content-laden secondary trait. Information on the nomological net and predictive validity can be useful to aid the interpretation of these item-specific effects. Therefore, the present study on N = 2,543 Dutch respondents adopts revised latent state-trait theory to disentangle common construct variance, random measurement error, and person-specific item effects in the Satisfaction with Life Scale across three measurement occasions. The reported analyses not only demonstrate how to examine item-specific multidimensionality in longitudinal data but also emphasize how different identification constraints for the latent variable lead to different interpretations. Moreover, the predictive validity of item effect variables for the prediction of psychological and physical health is examined. A cross-validation with the same sample at a later measurement period and robustness checks with incomplete data, support our findings on the substantive value of a multidimensional specification of the Satisfaction with Life Scale for substantive analyses. Finally, the contributions of person-specific item effects for psychological assessments are discussed.

Studying item-effect variables and their correlation patterns with multi-construct multi-state models.

Method effects on the item level can be modeled as latent difference variables in longitudinal data. These item-effect variables represent interindividual differences associated with responses to a specific item when assessing a common construct with multi-item scales. In latent variable analyses, their inclusion substantially improves model fits in comparison to classical unidimensional measurement models. More importantly, covariations between different item-effect variables and with other constructs can provide valuable insights, for example, into the structure of the studied instrument or the response process. Therefore, we introduce a multi-construct multi-state model with item-effect variables for systematic investigations of these correlation patterns within and between constructs. The implementation of this model is demonstrated using a sample of N = 2,529 Dutch respondents that provided measures of life satisfaction and positive affect at five measurement occasions. Our results confirm non-negligible item effects in two ostensibly unidimensional scales, indicating the importance of modeling interindividual differences on the item level. The correlation pattern between constructs indicated rather specific effects for individual items and no common causes, but the correlations within a construct align with the item content and support a substantive meaning. These analyses exemplify how multi-construct multi-state models allow the systematic examination of item effects to improve substantive and psychometric research.

Causal effect analysis in nonrandomized data with latent variables and categorical indicators: The implementation and benefits of EffectLiteR.

Instead of using manifest proxies for a latent outcome or latent covariates in a causal effect analysis, the R package EffectLiteR facilitates a direct integration of latent variables based on structural equation models (SEM). The corresponding framework considers latent interactions and provides various effect estimates for evaluating the differential effectiveness of treatments. In addition, a user-friendly graphical interface customizes the implementation of the complex models. We aim to enable applications of EffectLiteR in more contexts, and therefore generalize the framework for incorporating latent variables measured with categorical indicators. This refers, for instance, to achievement tests in educational large-scale assessments (LSAs), which are typically constructed in the tradition of item response theory (IRT). We review different modeling strategies for incorporating latent variables from IRT models in an effect analysis (i.e., individual score estimates, plausible values, SEM for categorical indicators). The strategies differ in the handling of measurement error and, thus, have different implications for the accuracy and efficiency of causal effect estimates. We describe our extensions of EffectLiteR based on SEM for categorical indicators and illustrate the model specification step-by-step. In addition, we present a hands-on example, where we apply EffectLiteR in LSA data. The practical benefit of using latent variables in comparison to proficiency scores is of special interest in the application and discussion. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Compensation and Amplification of Attenuation Bias in Causal Effect Estimates.

Covariate-adjusted treatment effects are commonly estimated in non-randomized studies. It has been shown that measurement error in covariates can bias treatment effect estimates when not appropriately accounted for. So far, these delineations primarily assumed a true data generating model that included just one single covariate. It is, however, more plausible that the true model consists of more than one covariate. We evaluate when a further covariate may reduce bias due to measurement error in another covariate and in which cases it is not recommended to include a further covariate. We analytically derive the amount of bias related to the fallible covariate's reliability and systematically disentangle bias compensation and amplification due to an additional covariate. With a fallible covariate, it is not always beneficial to include an additional covariate for adjustment, as the additional covariate can extensively increase the bias. The mechanisms for an increased bias due to an additional covariate can be complex, even in a simple setting of just two covariates. A high reliability of the fallible covariate or a high correlation between the covariates cannot in general prevent from substantial bias. We show distorting effects of a fallible covariate in an empirical example and discuss adjustment for latent covariates as a possible solution.

When does measurement error in covariates impact causal effect estimates? Analytic derivations of different scenarios and an empirical illustration.

The average causal treatment effect (ATE) can be estimated from observational data based on covariate adjustment. Even if all confounding covariates are observed, they might not necessarily be reliably measured and may fail to obtain an unbiased ATE estimate. Instead of fallible covariates, the respective latent covariates can be used for covariate adjustment. But is it always necessary to use latent covariates? How well do analysis of covariance (ANCOVA) or propensity score (PS) methods estimate the ATE when latent covariates are used? We first analytically delineate the conditions under which latent instead of fallible covariates are necessary to obtain the ATE. Then we empirically examine the difference between ATE estimates when adjusting for fallible or latent covariates in an applied example. We discuss the issue of fallible covariates within a stochastic theory of causal effects and analyse data of a within-study comparison with recently developed ANCOVA and PS procedures that allow for latent covariates. We show that fallible covariates do not necessarily bias ATE estimates, but point out different scenarios in which adjusting for latent covariates is required. In our empirical application, we demonstrate how latent covariates can be incorporated for ATE estimation in ANCOVA and in PS analysis.

Evaluative conditioning increases with temporal contiguity. The influence of stimulus order and stimulus interval on evaluative conditioning.

Evaluative conditioning (EC) is a change in valence that is due to pairing a conditioned stimulus (CS) with another, typically valent, unconditioned stimulus (US). This paper investigates how basic presentation parameters moderate EC effects. In two studies we tested the effectiveness of different temporal relations of the CS and the US, that is, the order in which the stimuli were presented and the temporal distance between them. Both studies showed that the size of EC effects was independent of the presentation order of CS and US within a stimulus pair. Contrary to classical conditioning effects, EC effects are thus not most pronounced after CS-first presentations. Furthermore, as shown in Experiment 2, EC effects increased in magnitude as the temporal interval between CS and US presentations decreased. Experiment 1 showed largest EC effects in the condition with simultaneous presentations - which can be seen as the condition with the temporally closest presentation. In this Experiment stimuli were presented in two different modalities, which might have facilitated simultaneous processing. In Experiment 2, in which all stimuli were presented visually, this advantage of simultaneous presentation was not found. We discuss practical and theoretical implications of our findings.