Percentage of variance accounted for in structural equation models: The rediscovery of the goodness of fit index.

This article delves into the often-overlooked metric of percentage of variance accounted for in structural equation models (SEM). The goodness of fit index (GFI) provides the percentage of variance of the sum of squared covariances explained by the model. Despite being introduced over four decades ago, the GFI has been overshadowed in favor of fit indices that prioritize distinctions between close and nonclose fitting models. Similar to R² in regression, the GFI should not be used to this aim but rather to quantify the model's utility. The central aim of this study is to reintroduce the GFI, introducing a novel approach to computing the GFI using mean and mean-and-variance corrected test statistics, specifically designed for nonnormal data. We use an extensive simulation study to evaluate the precision of inferences on the GFI, including point estimates and confidence intervals. The findings demonstrate that the GFI can be very accurately estimated, even with nonnormal data, and that confidence intervals exhibit reasonable accuracy across diverse conditions, including large models and nonnormal data scenarios. The article provides methods and code for estimating the GFI in any SEM, urging researchers to reconsider the reporting of the percentage of variance accounted for as an essential tool for model assessment and selection. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A Bayesian Random Weights Linear Logistic Test Model for Within-Test Practice Effects.

The present paper introduces a random weights linear logistic test model for the measurement of individual differences in operation-specific practice effects within a single administration of a test. The proposed model is an extension of the linear logistic test model of learning developed by Spada (1977) in which the practice effects are considered random effects varying across examinees. A Bayesian framework was used for model estimation and evaluation. A simulation study was conducted to examine the behavior of the model in combination with the Bayesian procedures. The results demonstrated the good performance of the estimation and evaluation methods. Additionally, an empirical study was conducted to illustrate the applicability of the model to real data. The model was applied to a sample of responses from a logical ability test providing evidence of individual differences in operation-specific practice effects.

A Bayesian General Model to Account for Individual Differences in Operation-Specific Learning Within a Test.

The present paper introduces a general multidimensional model to measure individual differences in learning within a single administration of a test. Learning is assumed to result from practicing the operations involved in solving the items. The model accounts for the possibility that the ability to learn may manifest differently for correct and incorrect responses, which allows for distinguishing different types of learning effects in the data. Model estimation and evaluation is based on a Bayesian framework. A simulation study is presented that examines the performance of the estimation and evaluation methods. The results show accuracy in parameter recovery as well as good performance in model evaluation and selection. An empirical study illustrates the applicability of the model to data from a logical ability test.

Overfactoring in rating scale data: A comparison between factor analysis and item response theory.

Educational and psychological measurement is typically based on dichotomous variables or rating scales comprising a few ordered categories. When the mean of the observed responses approaches the upper or the lower bound of the scale, the distribution of the data becomes skewed and, if a categorical factor model holds in the population, the Pearson correlation between variables is attenuated. The consequence of this correlation attenuation is that the traditional linear factor model renders an excessive number of factors. This article presents the results of a simulation study investigating the problem of overfactoring and some solutions. We compare five widely known approaches: (1) The maximum-likelihood factor analysis (FA) model for normal data, (2) the categorical factor analysis (FAC) model based on polychoric correlations and maximum likelihood (ML) estimation, (3) the FAC model estimated using a weighted least squares algorithm, (4) the mean corrected chi-square statistic by Satorra-Bentler to handle the lack of normality, and (5) the Samejima's graded response model (GRM) from item response theory (IRT). Likelihood-ratio chi-square, parallel analysis (PA), and categorical parallel analysis (CPA) are used as goodness-of-fit criteria to estimate the number of factors in the simulation study. Our results indicate that the maximum-likelihood estimation led to overfactoring in the presence of skewed variables both for the linear and categorical factor model. The Satorra-Bentler and GRM constitute the most reliable alternatives to estimate the number of factors.

What are the consequences of ignoring cross-loadings in bifactor models? A simulation study assessing parameter recovery and sensitivity of goodness-of-fit indices.

Bifactor latent models have gained popularity and are widely used to model construct multidimensionality. When adopting a confirmatory approach, a common practice is to assume that all cross-loadings take zero values. This article presents the results of a simulation study exploring the impact of ignoring non-zero cross-loadings on the performance of confirmatory bifactor analysis. The present work contributes to previous research by including study conditions that had not been examined before. For instance, a wider range of values of the factor loadings both for the group factors and the cross-loadings is considered. Parameter recovery is analyzed, but the focus of the study is on assessing the sensitivity of goodness-of-fit indices to detect the model misspecification that involves ignoring non-zero cross-loadings. Several commonly used SEM fit indices are examined: both biased estimators of the fit index (CFI, GFI, and SRMR) and unbiased estimators (RMSEA and SRMR). Results indicated that parameter recovery worsens when ignoring moderate and large cross-loading values and using small sample sizes, and that commonly used SEM fit indices are not useful to detect such model misspecifications. We recommend the use of the unbiased SRMR index with a cutoff value adjusted by the communality level (R 2), as it is the only fit index sensitive to the model misspecification due to ignoring non-zero cross-loadings in the bifactor model. The results of the present study provide insights into modeling cross-loadings in confirmatory bifactor models but also practical recommendations to researchers.

Bayesian Estimation and Testing of a Beta Factor Model for Bounded Continuous Variables.

The particularities of bounded data are often overlooked. This type of data is likely to display a pattern of skewness because of the existence of an upper and lower limit that cannot be exceeded. In the context of factor analysis, when variables are skewed in opposite directions, using normal-theory factor analysis might lead to over-factoring. We propose a Bayesian beta factor model to analyze doubly bounded data. A simulation study was conducted to evaluate the performance of the normal and beta factor models in the presence of skewed variables. Two Bayesian approaches to model evaluation methods are considered, posterior predictive checking and three information criterion measures (DIC, WAIC, and LOO). The number of estimated factors based on the Bayesian methods is compared for the normal and beta factor models. An application of the model using real data is also presented. We found that the beta factor model constitutes a suitable alternative to analyze data with a pattern of mixed skewness. Posterior predictive checking appears to be a viable option to select the optimal number of factors in Bayesian factor analysis.

Nominal Factor Analysis of Situational Judgment Tests: Evaluation of Latent Dimensionality and Factorial Invariance.

Situational judgment tests have gained popularity in educational and psychological measurement and are widely used in personnel assessment. A situational judgment item presents a hypothetical scenario and a list of actions, and the individuals are asked to select their most likely action for that scenario. Because actions have no explicit order, the item generates nominal responses consisting of the actions selected by the individuals. This article shows how to factor-analyze the nominal responses originated from such a test, including the estimation of the number of latent factors and a factor invariance analysis in a multiple group design. The method consists of applying the MNCM, a multidimensional extension of the nominal categories model by Bock. The article includes the results of two studies: (1) a simulation study about Type-I error rate, statistical power, and recovery of the parameters in a multigroup factorial invariance design and (2) a real data example using responses to a situational judgment test measuring gender stereotypes to illustrate the approach. Results suggest the use of the Akaike information criterion, Bayesian information criterion, and corrected Bayesian information criterion indices to guide the selection of the number of factors with nominal responses. All the analyses are conducted using the computer program Mplus. The code is included as Supplemental Material (available online) for the readers so that they can adapt it to their own purposes.

A Bayesian Generalized Explanatory Item Response Model to Account for Learning During the Test.

The present paper introduces a new explanatory item response model to account for the learning that takes place during a psychometric test due to the repeated use of the operations involved in the items. The proposed model is an extension of the operation-specific learning model (Fischer and Formann in Appl Psychol Meas 6:397-416, 1982; Scheiblechner in Z für Exp Angew Psychol 19:476-506, 1972; Spada in Spada and Kempf (eds.) Structural models of thinking and learning, Huber, Bern, Germany, pp 227-262, 1977). The paper discusses special cases of the model, which, together with the general formulation, differ in the type of response in which the model states that learning occurs: (1) correct and incorrect responses equally (non-contingent learning); (2) correct responses only (contingent learning); and (3) correct and incorrect responses to a different extent (differential contingent learning). A Bayesian framework is adopted for model estimation and evaluation. A simulation study is conducted to examine the performance of the estimation and evaluation methods in recovering the true parameters and selecting the true model. Finally, an empirical study is presented to illustrate the applicability of the model to detect learning effects using real data.

Validation of a Questionnaire for Personality Profiling using Cognitive Diagnostic Modeling.

This article describes the development of the ENCUIST (Extroversion, Neuroticism, Callous-Unemotional, Instability, Short-Test) questionnaire, which has been created to provide a personality profiling method based on a cognitive diagnostic modeling framework. The ENCUIST measures the attributes of extroversion, neuroticism, callous unemotionality and overt expressions of anger that are relevant in a forensic context. The scores provided by the ENCUIST are binary classifications of the individuals (high/low) in these attributes. The ENCUIST was developed using a sample of 516 subjects to study its validation through psychometric procedures, including factor analysis, cognitive diagnostic modeling and structural equation modeling. The results supported a four-factor structure. Linear regressions were used to evaluate the predictive validity of the scores provided by ENCUIST with respect to two external criteria that are relevant in the forensic context, namely behavioral activation and behavioral inhibition. The results showed that the extroversion dimension is positively related to behavioral activation, although the effect size is modest and the proportion of explained variance is only 11%. Moreover, the dimensions of neuroticism and anger expression are positively related to behavioral inhibition, with 7% of the variance explained. Together, these results suggest that cognitive diagnostic models are useful tools for the elaboration of personality profiles based on classifying subjects along binary attributes.

Bayesian Dimensionality Assessment for the Multidimensional Nominal Response Model.

This article introduces Bayesian estimation and evaluation procedures for the multidimensional nominal response model. The utility of this model is to perform a nominal factor analysis of items that consist of a finite number of unordered response categories. The key aspect of the model, in comparison with traditional factorial model, is that there is a slope for each response category on the latent dimensions, instead of having slopes associated to the items. The extended parameterization of the multidimensional nominal response model requires large samples for estimation. When sample size is of a moderate or small size, some of these parameters may be weakly empirically identifiable and the estimation algorithm may run into difficulties. We propose a Bayesian MCMC inferential algorithm to estimate the parameters and the number of dimensions underlying the multidimensional nominal response model. Two Bayesian approaches to model evaluation were compared: discrepancy statistics (DIC, WAICC, and LOO) that provide an indication of the relative merit of different models, and the standardized generalized discrepancy measure that requires resampling data and is computationally more involved. A simulation study was conducted to compare these two approaches, and the results show that the standardized generalized discrepancy measure can be used to reliably estimate the dimensionality of the model whereas the discrepancy statistics are questionable. The paper also includes an example with real data in the context of learning styles, in which the model is used to conduct an exploratory factor analysis of nominal data.

A multistage adaptive test of fluid intelligence.

BACKGROUND: Multistage adaptive testing has recently emerged as an alternative to the computerized adaptive test. The current study details a new multistage test to assess fluid intelligence. METHOD: An item pool of progressive matrices with constructed response format was developed, and divided into six subtests. The subtests were applied to a sample of 724 college students and their psychometric properties were studied (i.e., reliability, dimensionality and validity evidence). The item pool was calibrated under the graded response model, and two multistage structures were developed, based on the automatic test assembly principles. Finally, the test information provided by each structure was compared in order to select the most appropriate one. RESULTS: The item pool showed adequate psychometric properties. From the two compared multistage structures, the simplest structure (i.e., routing test and two modules in the next stages) were more informative across the latent trait continuum and were therefore kept. DISCUSSION: Taken together, the results of the two studies support the application of the FIMT (Fluid Intelligence Multistage Test), a multistage test to assess fluid intelligence accurately and innovatively.

Logistic response models with item interactions.

Items that are clustered according to shared content may violate the principle of conditional independence commonly used in item response theory. This paper investigates the capabilities of a logistic item response model in relation to locally dependent item responses. The model includes main effect and interaction parameters that are computed as linear functions of the latent trait. The paper explains the interpretation of the parameters, the maximum likelihood estimation algorithm, the information matrix and some results concerning parameter identifiability. The problem of over-fitting the data is addressed in a simulation study, and two real data examples are described to illustrate the approach, one from the context of a sample survey and the other from ability testing using testlets.

Are we more consistent when talking about ourselves than when behaving? Consistency differences through a questionnaire and an objective task.

The present paper aimed to examine questionnaire response patterns and objective task-based test behavioral patterns in order to analyze the differences people show in consistency. It is hypothesized that people tend to be more consistent when talking about themselves (when describing themselves through verbal statements) that when they solve a task (when behaving). Consistency is computed using the pi* statistic (Hernandez, Rubio, Revuelta, & Santacreu, 2006). According to this procedure, consistency is defined as the value and the dimensionality of the latent trait of an individual (theta) remaining invariant through out the test of. Participants who are consistent must show a constant theta and follow a given response pattern during the entire course of the test. A sample of 3,972 participants was used. Results reveal that 68% of participants showed a consistent response pattern when completing the questionnaire. When tackling the task-based test, the percentage was 66%. 45% of individuals showed a consistent pattern in both tests. Implications for personality and individual differences assessment are discussed.

Factorial invariance in a repeated measures design: an application to the study of person-organization fit.

An important methodological concern of any research based on a person-environment (P-E) fit approach is the operationalization of the fit, which imposes some measurement requirements that are rarely empirically tested with statistical methods. Among them, the assessment of the P and E components along commensurate dimensions is possibly the most cited one. This paper proposes to test the equivalence across the P and E measures by analyzing the measurement invariance of a multi-group confirmatory factor analysis model. From a methodological point of view, the distinct aspect of this approach within the context of P-E fit research is that measurement invariance is assessed in a repeated measures design. An example illustrating the procedure in a person-organization (P-O) fit dataset is provided. Measurement invariance was tested at five different hierarchical levels: (1) configural, (2) first-order factor loadings, (3) second-order factor loadings, (4) residual variances of observed variables, and (5) disturbances of first-order factors. The results supported the measurement invariance across the P and O measures at the third level. The implications of these findings for P-E fit studies are discussed.

Estimating the pi* goodness of fit index for finite mixtures of item response models.

Testing the fit of finite mixture models is a difficult task, since asymptotic results on the distribution of likelihood ratio statistics do not hold; for this reason, alternative statistics are needed. This paper applies the pi* goodness of fit statistic to finite mixture item response models. The pi* statistic assumes that the population is composed of two subpopulations - those that follow a parametric model and a residual group outside the model; pi* is defined as the proportion of population in the residual group. The population was divided into two or more groups, or classes. Several groups followed an item response model and there was also a residual group. The paper presents maximum likelihood algorithms for estimating item parameters, the probabilities of the groups and pi*. The paper also includes a simulation study on goodness of recovery for the two- and three-parameter logistic models and an example with real data from a multiple choice test.

Extending the CLAST sequential rule to one-way ANOVA under group sampling.

Several studies have demonstrated that the fixed-sample stopping rule (FSR), in which the sample size is determined in advance, is less practical and efficient than are sequential-stopping rules. The composite limited adaptive sequential test (CLAST) is one such sequential-stopping rule. Previous research has shown that CLAST is more efficient in terms of sample size and power than are the FSR and other sequential rules and that it reflects more realistically the practice of experimental psychology researchers. The CLAST rule has been applied only to the t test of mean differences with two matched samples and to the chi-square independence test for twofold contingency tables. The present work extends previous research on the efficiency of CLAST to multiple group statistical tests. Simulation studies were conducted to test the efficiency of the CLAST rule for the one-way ANOVA for fixed effects models. The ANOVA general test and two linear contrasts of multiple comparisons among treatment means are considered. The article also introduces four rules for allocating N observations to J groups under the general null hypothesis and three allocation rules for the linear contrasts. Results show that the CLAST rule is generally more efficient than the FSR in terms of sample size and power for one-way ANOVA tests. However, the allocation rules vary in their optimality and have a differential impact on sample size and power. Thus, selecting an allocation rule depends on the cost of sampling and the intended precision.

[Application of the pi* goodness-of-fit index to latent structure models]. / Aplicación del estadístico* de bondad de ajuste a modelos de estructura latente.

Testing model fit for latent structure models (latent trait models and latent class models) is difficult because of the lack of goodness-of-fit statistics with a known distribution. This paper describes the application of the pi* goodness-of-fit statistic to latent structure models. The statistic pi* is based on the concept of latent classes and has a natural interpretation when applied to these models. This statistic assumes that the population is made up of several classes that follow a parametric model, and a residual class outside the model. The value of pi* is the population proportion in the residual class. This paper describes the estimation algorithms of pi* for latent trait and latent class models and an empirical example with a scale of study habits. There are two latent classes in these data: bad and regular students, which are related to the student s responsibility.

Optimization of sample size in controlled experiments: the CLAST rule.

Sequential rules are explored in the context of null hypothesis significance testing. Several studies have demonstrated that the fixed-sample stopping rule, in which the sample size used by researchers is determined in advance, is less practical and less efficient than sequential stopping rules. It is proposed that a sequential stopping rule called CLAST (composite limited adaptive sequential test) is a superior variant of COAST (composite open adaptive sequential test), a sequential rule proposed by Frick (1998). Simulation studies are conducted to test the efficiency of the proposed rule in terms of sample size and power. Two statistical tests are used: the one-tailed t test of mean differences with two matched samples, and the chi-square independence test for twofold contingency tables. The results show that the CLAST rule is more efficient than the COAST rule and reflects more realistically the practice of experimental psychology researchers.

Lack of echocardiographic and Doppler abnormalities in psoriatic arthritis patients without clinically evident cardiovascular disease or classic atherosclerosis risk factors.

OBJECTIVE: To assess the prevalence of echocardiographic and Doppler abnormalities in psoriatic arthritis (PsA) patients without clinically evident cardiovascular manifestations or classic atherosclerosis risk factors. METHODS: Fifty PsA patients were recruited from Hospital Xeral-Calde, Lugo, Spain. Patients seen during the period of recruitment that had classic cardiovascular risk factors or had suffered cardiovascular or cerebrovascular events were excluded. Fifty healthy matched controls were also studied. Echocardiographic and Doppler studies were performed in all cases and controls. RESULTS: In PsA patients the frequency of aortic and tricuspid (10%) and mitral regurgitation (16%) was not different from that seen in matched controls (10, 4, and 12%). Also, the pulmonary artery systolic pressure was normal in the group of PsA patients (23.4+/-3.9 mm Hg). The prevalence of diastolic dysfunction, in all cases due to impaired relaxation, was similar in PsA patients (28%) and controls (24%) (P=0.65). In addition, no significant echocardiographic and Doppler differences were observed when PsA patients with polyarticular pattern were compared with the remaining PsA patients. CONCLUSIONS: The present study shows that actively treated PsA patients without cardiovascular risk factors or clinically evident cardiovascular disease do not exhibit silent subclinical echocardiographic abnormalities.

Increased prevalence of severe subclinical atherosclerotic findings in long-term treated rheumatoid arthritis patients without clinically evident atherosclerotic disease.

We conducted the current study to search for subclinical atherosclerosis in patients with rheumatoid arthritis (RA) without clinically evident atherosclerosis or its complications who had been treated for a long duration, and to assess whether demographic or clinical factors affect the development of atherosclerotic disease. Forty-seven white patients fulfilling the 1987 American College of Rheumatology classification criteria for RA were recruited from Hospital Xeral-Calde, Lugo, Spain. Patients were required to have been treated for at least 5 years, including current treatment with 1 or more disease-modifying antirheumatic drugs. Patients with diabetes mellitus, renal insufficiency, hypertension, cardiovascular or cerebrovascular disease, and smokers were excluded. Forty-seven matched controls were also studied. Carotid intima-media wall thickness (IMT) and carotid plaques were measured in the right common carotid artery. The study was performed using high-resolution B-mode ultrasound. Patients had greater carotid IMT (0.779 +/- 0.164 mm) than did controls (0.699 +/- 0.129 mm); (p = 0.010). Sixteen (34%) patients showed carotid plaques compared with only 7 (15%) controls (p = 0.031). There was a positive correlation between the age at the time of study and the carotid IMT. Patients with carotid plaques had significantly greater carotid IMT (0.859 +/- 0.116 mm) than those without plaques (0.739 +/- 0.171 mm) (p = 0.014). Also, RA patients with carotid plaques had a significantly longer disease duration (mean, 21.0 yr) and more extraarticular manifestations (63%) than those without plaques (mean, 12.7 yr and 26%, respectively). Age at the time of the study and disease duration were the best predictive factors for the development of severe morphologic expression of atherosclerotic disease. The present study confirms an increased frequency of severe subclinical atherosclerotic findings in long-term actively treated RA patients from northwest Spain.