Evaluation of frequentist test statistics using constrained statistical inference in the context of the generalized linear model.

When faced with a binary or count outcome, informative hypotheses can be tested in the generalized linear model using the distance statistic as well as modified versions of the Wald, the Score and the likelihood-ratio test (LRT). In contrast to classical null hypothesis testing, informative hypotheses allow to directly examine the direction or the order of the regression coefficients. Since knowledge about the practical performance of informative test statistics is missing in the theoretically oriented literature, we aim at closing this gap using simulation studies in the context of logistic and Poisson regression. We examine the effect of the number of constraints as well as the sample size on type I error rates when the hypothesis of interest can be expressed as a linear function of the regression parameters. The LRT shows the best performance in general, followed by the Score test. Furthermore, both the sample size and especially the number of constraints impact the type I error rates considerably more in logistic compared to Poisson regression. We provide an empirical data example together with R code that can be easily adapted by applied researchers. Moreover, we discuss informative hypothesis testing about effects of interest, which are a non-linear function of the regression parameters. We demonstrate this by means of a second empirical data example.

A Small Sample Correction for Factor Score Regression.

Factor score regression (FSR) is widely used as a convenient alternative to traditional structural equation modeling (SEM) for assessing structural relations between latent variables. But when latent variables are simply replaced by factor scores, biases in the structural parameter estimates often have to be corrected, due to the measurement error in the factor scores. The method of Croon (MOC) is a well-known bias correction technique. However, its standard implementation can render poor quality estimates in small samples (e.g. less than 100). This article aims to develop a small sample correction (SSC) that integrates two different modifications to the standard MOC. We conducted a simulation study to compare the empirical performance of (a) standard SEM, (b) the standard MOC, (c) naive FSR, and (d) the MOC with the proposed SSC. In addition, we assessed the robustness of the performance of the SSC in various models with a different number of predictors and indicators. The results showed that the MOC with the proposed SSC yielded smaller mean squared errors than SEM and the standard MOC in small samples and performed similarly to naive FSR. However, naive FSR yielded more biased estimates than the proposed MOC with SSC, by failing to account for measurement error in the factor scores.

Multilevel SEM with random slopes in discrete data using the pairwise maximum likelihood.

Pairwise maximum likelihood (PML) estimation is a promising method for multilevel models with discrete responses. Multilevel models take into account that units within a cluster tend to be more alike than units from different clusters. The pairwise likelihood is then obtained as the product of bivariate likelihoods for all within-cluster pairs of units and items. In this study, we investigate the PML estimation method with computationally intensive multilevel random intercept and random slope structural equation models (SEM) in discrete data. In pursuing this, we first reconsidered the general 'wide format' (WF) approach for SEM models and then extend the WF approach with random slopes. In a small simulation study we the determine accuracy and efficiency of the PML estimation method by varying the sample size (250, 500, 1000, 2000), response scales (two-point, four-point), and data-generating model (mediation model with three random slopes, factor model with one and two random slopes). Overall, results show that the PML estimation method is capable of estimating computationally intensive random intercept and random slopes multilevel models in the SEM framework with discrete data and many (six or more) latent variables with satisfactory accuracy and efficiency. However, the condition with 250 clusters combined with a two-point response scale shows more bias.

A Note on the Connection Between Trek Rules and Separable Nonlinear Least Squares in Linear Structural Equation Models.

We show that separable nonlinear least squares (SNLLS) estimation is applicable to all linear structural equation models (SEMs) that can be specified in RAM notation. SNLLS is an estimation technique that has successfully been applied to a wide range of models, for example neural networks and dynamic systems, often leading to improvements in convergence and computation time. It is applicable to models of a special form, where a subset of parameters enters the objective linearly. Recently, Kreiberg et al. (Struct Equ Model Multidiscip J 28(5):725-739, 2021. https://doi.org/10.1080/10705511.2020.1835484 ) have shown that this is also the case for factor analysis models. We generalize this result to all linear SEMs. To that end, we show that undirected effects (variances and covariances) and mean parameters enter the objective linearly, and therefore, in the least squares estimation of structural equation models, only the directed effects have to be obtained iteratively. For model classes without unknown directed effects, SNLLS can be used to analytically compute least squares estimates. To provide deeper insight into the nature of this result, we employ trek rules that link graphical representations of structural equation models to their covariance parametrization. We further give an efficient expression for the gradient, which is crucial to make a fast implementation possible. Results from our simulation indicate that SNLLS leads to improved convergence rates and a reduced number of iterations.

A structural after measurement approach to structural equation modeling.

In structural equation modeling (SEM), the measurement and structural parts of the model are usually estimated simultaneously. In this article, we revisit the long-standing idea that we should first estimate the measurement part, and then estimate the structural part. We call this the "structural-after-measurement" (SAM) approach to SEM. We describe a formal framework for the SAM approach under settings where the latent variables and their indicators are continuous. We review earlier SAM methods and establish how they are specific instances of the SAM framework. Decoupled estimation for the measurement and structural parts using SAM possesses three key advantages over simultaneous estimation in standard SEM. First, estimates are more robust against local model misspecifications. Second, estimation routines are less vulnerable to convergence issues in small samples. Third, estimates exhibit smaller finite sample biases under correctly specified models. We propose two variants of the SAM approach. "Local" SAM expresses the mean vector and variance-covariance matrix of the latent variables as a function of the observed summary statistics and the parameters of the measurement model. "Global" SAM holds the parameters of the measurement part fixed while estimating the parameters of the structural part. Our framework includes two-step corrected standard errors, and permits computing both local and global fit measures. Nonetheless, the SAM approach is an estimation strategy, and should not be regarded as a model-building tool. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Overview and evaluation of various frequentist test statistics using constrained statistical inference in the context of linear regression.

Within the framework of constrained statistical inference, we can test informative hypotheses, in which, for example, regression coefficients are constrained to have a certain direction or be in a specific order. A large amount of frequentist informative test statistics exist that each come with different versions, strengths and weaknesses. This paper gives an overview about these statistics, including the Wald, the LRT, the Score, the F ¯ - and the D-statistic. Simulation studies are presented that clarify their performance in terms of type I and type II error rates under different conditions. Based on the results, it is recommended to use the Wald and F ¯ -test rather than the LRT and Score test as the former need less computing time. Furthermore, it is favorable to use the degrees of freedom corrected rather than the naive mean squared error when calculating the test statistics as well as using the F ¯ - rather than the χ ¯ 2 -distribution when calculating the p-values.

Exploratory Graph Analysis for Factor Retention: Simulation Results for Continuous and Binary Data.

Exploratory graph analysis (EGA) is a commonly applied technique intended to help social scientists discover latent variables. Yet, the results can be influenced by the methodological decisions the researcher makes along the way. In this article, we focus on the choice regarding the number of factors to retain: We compare the performance of the recently developed EGA with various traditional factor retention criteria. We use both continuous and binary data, as evidence regarding the accuracy of such criteria in the latter case is scarce. Simulation results, based on scenarios resulting from varying sample size, communalities from major factors, interfactor correlations, skewness, and correlation measure, show that EGA outperforms the traditional factor retention criteria considered in most cases in terms of bias and accuracy. In addition, we show that factor retention decisions for binary data are preferably made using Pearson, instead of tetrachoric, correlations, which is contradictory to popular belief.

Computation and application of generalized linear mixed model derivatives using lme4.

Maximum likelihood estimation of generalized linear mixed models (GLMMs) is difficult due to marginalization of the random effects. Derivative computations of a fitted GLMM's likelihood are also difficult, especially because the derivatives are not by-products of popular estimation algorithms. In this paper, we first describe theoretical results related to GLMM derivatives along with a quadrature method to efficiently compute the derivatives, focusing on fitted lme4 models with a single clustering variable. We describe how psychometric results related to item response models are helpful for obtaining the derivatives, as well as for verifying the derivatives' accuracies. We then provide a tutorial on the many possible uses of these derivatives, including robust standard errors, score tests of fixed effect parameters, and likelihood ratio tests of non-nested models. The derivative computation methods and applications described in the paper are all available in easily obtained R packages.

The Conceptual, Cunning, and Conclusive Experiment in Psychology.

The ideal experiment in physics must be conceptual, cunning, and conclusive. Adoption of these same standards in psychology has led to experiments that are uninformative and frivolous. We explain why we believe that psychology is fundamentally incompatible with hypothesis-driven theoretical science and conclude that this erodes the logic behind recent proposals to improve psychological research, such as stricter statistical standards, preregistration, and replication. The failure of psychology is not that it is somehow insufficiently scientific but rather that it makes inordinate use of methods that are a mismatch for the aspirations of researchers in the field, at the expense of valuable empirical research.

Development of the Reading Comprehension Strategies Questionnaire (RCSQ) for late elementary school students.

Late elementary education constitutes a critical period in the development of reading comprehension strategies, a key competence in today's society. However, to date, appropriate measurements to map late elementary students' reading strategies are lacking. In this respect, the present article first describes the development and validation of the 26-item reading comprehension strategies questionnaire (RCSQ). To this aim, exploratory (sample 1: n = 1585 students) and confirmatory (sample 2: n = 1585 students) factor analyses were conducted. These analyses resulted in the RCSQ, consisting of five subscales: (1) overt cognitive reading strategies, (2) covert cognitive reading strategies, (3) monitoring, and (4) evaluating. For non-native and bilingual students, a fifth subscale 'using home language in view of comprehending texts' emerged. Second, multilevel analyses were performed to explore individual differences in late elementary students' reading comprehension strategy use. Implications for practice and future research are discussed.

A Wilcoxon-Mann-Whitney Test for Latent Variables.

We propose an extension of the Wilcoxon-Mann-Whitney test to compare two groups when the outcome variable is latent. We empirically demonstrate that the test can have superior power properties relative to tests based on Structural Equation Modeling for a variety of settings. In addition, several other advantages of the Wilcoxon-Mann-Whitney test are retained such as robustness to outliers and good small sample performance. We demonstrate the proposed methodology on a case study.

Estimating Structural Equation Models Using James-Stein Type Shrinkage Estimators.

We propose a two-step procedure to estimate structural equation models (SEMs). In a first step, the latent variable is replaced by its conditional expectation given the observed data. This conditional expectation is estimated using a James-Stein type shrinkage estimator. The second step consists of regressing the dependent variables on this shrinkage estimator. In addition to linear SEMs, we also derive shrinkage estimators to estimate polynomials. We empirically demonstrate the feasibility of the proposed method via simulation and contrast the proposed estimator with ML and MIIV estimators under a limited number of simulation scenarios. We illustrate the method on a case study.

Multilevel Factor Score Regression.

Multilevel SEM is an increasingly popular technique to analyze data that are both hierarchical and contain latent variables. The parameters are usually jointly estimated using a maximum likelihood estimator (MLE). This has the disadvantage that a large sample size is needed and misspecifications in one part of the model may influence the whole model. We propose an alternative stepwise estimation method, which is an extension of the Croon method for factor score regression. In this article, we extend this method to the multilevel setting. A simulation study was used to compare this new estimation method to the standard MLE. The Croon method outperformed MLE with regard to convergence rate, bias, MSE, and coverage, in particular when models contained a structural misspecification. In conclusion, the Croon method seems to be a promising alternative to MLE.

New Developments in Factor Score Regression: Fit Indices and a Model Comparison Test.

Factor score regression (FSR) is a popular alternative for structural equation modeling. Naively applying FSR induces bias for the estimators of the regression coefficients. Croon proposed a method to correct for this bias. Next to estimating effects without bias, interest often lies in inference of regression coefficients or in the fit of the model. In this article, we propose fit indices for FSR that can be used to inspect the model fit. We also introduce a model comparison test based on one of these newly proposed fit indices that can be used for inference of the estimators on the regression coefficients. In a simulation study we compare FSR with Croon's corrections and structural equation modeling in terms of bias of the regression coefficients, Type I error rate and power.

Computing Bayes factors from data with missing values.

The Bayes factor is increasingly used for the evaluation of hypotheses. These may be traditional hypotheses specified using equality constraints among the parameters of the statistical model of interest or informative hypotheses specified using equality and inequality constraints. Thus far, no attention has been given to the computation of Bayes factors from data with missing values. A key property of such a Bayes factor should be that it is only based on the information in the observed values. This article will show that such a Bayes factor can be obtained using multiple imputations of the missing values. After introduction of the general framework elaborations for Bayes factors based on default or subjective prior distributions and Bayes factors based on priors specified using training data will be given. It will be illustrated that the approach proposed can be applied using R packages for multiple imputation in combination with the Bayes factor packages Bain and BayesFactor. It will furthermore be illustrated that Bayes factors computed using a single imputation of the data are very inaccurate approximations of the correct Bayes factor. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The Effect of Faking on the Correlation Between Two Ordinal Variables: Some Population and Monte Carlo Results.

Correlational measures are probably the most spread statistical tools in psychological research. They are used by researchers to investigate, for example, relations between self-report measures usually collected using paper-pencil or online questionnaires. Like many other statistical analysis, also correlational measures can be seriously affected by specific sources of bias which constitute serious threats to the final observed results. In this contribution, we will focus on the impact of the fake data threat on the interpretation of statistical results for two well-know correlational measures (the Pearson product-moment correlation and the Spearman rank-order correlation). By using the Sample Generation by Replacement (SGR) approach, we analyze uncertainty in inferences based on possible fake data and evaluate the implications of fake data for correlational results. A population-level analysis and a Monte Carlo simulation are performed to study different modulations of faking on bivariate discrete variables with finite supports and varying sample sizes. We show that by using our paradigm it is always possible, under specific faking conditions, to increase (resp. decrease) the original correlation between two discrete variables in a predictable and systematic manner.

The effect of the "Follow in my Green Food Steps" programme on cooking behaviours for improved iron intake: a quasi-experimental randomized community study.

BACKGROUND: Nutritional iron deficiency is one of the leading factors for disease, disability and death. A quasi-experimental randomized community study in South-West Nigeria explored whether a branded behaviour change programme increased the use of green leafy vegetables (greens) and iron-fortified bouillon cubes in stews for improved iron intake. METHODS: A coinflip assigned the intervention to Ile-Ife (Intervention town). Osogbo (Control town) received no information. At baseline 602 mother-daughter pairs (daughters aged 12-18) were enrolled (Intervention: 300; Control: 302). A Food Frequency Questionnaire assessed the addition of cubes and greens to stews, the primary outcome. Secondary outcomes were the addition of cubes and greens to soups and changes in behavioural determinants measured using the Theory of Planned Behaviour. Structural Equation Modelling (SEM) evaluated the impact of the intervention on behavioural determinants and behaviour. RESULTS: The data of 527 pairs was used (Intervention: 240; Control: 287). The increase in greens added to stews was larger in the Intervention town compared to the Control town (MIntervention = 0.3 [SE = 0.03]; MControl = 0.0 [SE = 0.04], p < 0.001, r = 0.36). Change in iron-fortified cubes added to stews did not differ between towns (p = 0.07). The increase in cubes added to soups was larger in the Intervention town compared to the Control Town (MIntervention = 0.9 [SE = 0.2] vs MControl = 0.4 [SE = 0.1], p < .0001, r = 0.20). Unexpectedly, change in greens added to soups was larger in the Control town compared to the Intervention town (MIntervention = - 0.1 [SE = 0.1]; MControl = 0.5 [SE = 0.1], p = 0.003, r = 0.15). The intervention positively influenced awareness of anaemia and the determinants of behaviour in the Intervention town, with hardly any change in the Control town. Baseline SEMs could not be established, so no mediation analyses were done. Post-intervention SEMs highlighted the role of habit in cooking stews. CONCLUSIONS: The behaviour change programme increased the amount of green leafy vegetables added to stews and iron-fortified cubes added to soups. Future research should assess the long-term impact and the efficacy of the programme as it is scaled up and rolled out.

Local fit evaluation of structural equation models using graphical criteria.

Evaluation of model fit is critically important for every structural equation model (SEM), and sophisticated methods have been developed for this task. Among them are the χ² goodness-of-fit test, decomposition of the χ², derived measures like the popular root mean square error of approximation (RMSEA) or comparative fit index (CFI), or inspection of residuals or modification indices. Many of these methods provide a global approach to model fit evaluation: A single index is computed that quantifies the fit of the entire SEM to the data. In contrast, graphical criteria like d-separation or trek-separation allow derivation of implications that can be used for local fit evaluation, an approach that is hardly ever applied. We provide an overview of local fit evaluation from the viewpoint of SEM practitioners. In the presence of model misfit, local fit evaluation can potentially help in pinpointing where the problem with the model lies. For models that do fit the data, local tests can identify the parts of the model that are corroborated by the data. Local tests can also be conducted before a model is fitted at all, and they can be used even for models that are globally underidentified. We discuss appropriate statistical local tests, and provide applied examples. We also present novel software in R that automates this type of local fit evaluation. (PsycINFO Database Record

The EffectLiteR Approach for Analyzing Average and Conditional Effects.

We present a framework for estimating average and conditional effects of a discrete treatment variable on a continuous outcome variable, conditioning on categorical and continuous covariates. Using the new approach, termed the EffectLiteR approach, researchers can consider conditional treatment effects given values of all covariates in the analysis and various aggregates of these conditional treatment effects such as average effects, effects on the treated, or aggregated conditional effects given values of a subset of covariates. Building on structural equation modeling, key advantages of the new approach are (1) It allows for latent covariates and outcome variables; (2) it permits (higher order) interactions between the treatment variable and categorical and (latent) continuous covariates; and (3) covariates can be treated as stochastic or fixed. The approach is illustrated by an example, and open source software EffectLiteR is provided, which makes a detailed analysis of effects conveniently accessible for applied researchers.