Investigating the performance of level-specific fit indices in multilevel confirmatory factor analysis with dichotomous indicators: A Monte Carlo study.

We conducted a Monte Carlo study to examine the performance of level-specific χ2 test statistics and fit regarding their capacity to determine model fit at specific levels in multilevel confirmatory factor analysis with dichotomous indicators. Five design factors-numbers of groups (NG), group size (GS), intra-class correlation (ICC), thresholds of dichotomous indicators (THR), and factor loadings (FL)-were considered in this study. According to our simulation results, we recommend that practitioners should be aware that the performance of between-level-specific (b-l-s) χ2 and fit indices was mainly influenced by ICC and FL, followed by NG. At the same time, THR could slightly weigh in the performance of b-l-s fit indices in some conditions. Both b-l-s χ2 and fit indices were more promising indicators to correctly indicate model fit when ICC or FL increased. A small to medium NG (50-100) might be sufficient for b-l-s χ2 and fit indices only if both ICC and factor loadings were high, while in remaining conditions, an NG of 200 was needed. Moreover, practitioners could use within-level-specific (w-l-s) χ2 and fit indices (except for RMSEAW) along with traditional cut-off values to evaluate within-level models comprising dichotomous indicators. W-l-s χ2 and fit indices were more promising to determine model fit when FL increased. THR had a slight impact and could weigh in the performance of [Formula: see text], RMSEAW, CFIW, and TLIW. Unfortunately, RMSEAW was heavily affected by FL and THR and could determine model fit only when FL was high and THR was symmetric.

Evaluating fit indices in a multilevel latent growth curve model: A Monte Carlo study.

The multilevel latent growth curve model (MLGCM), which is subsumed by the multilevel structural equation modeling framework, has been advocated as a means of investigating individual and cluster trajectories. Still, how to evaluate the goodness of fit of MLGCMs has not been well addressed. The purpose of this study was to conduct a systematic Monte Carlo simulation to carefully investigate the effectiveness of (a) level-specific fit indices and (b) target-specific fit indices in an MLGCM, in terms of their independence from the sample size's influence and their sensitivity to misspecification in the MLGCM that occurs in either the between-covariance, between-mean, or within-covariance structure. The design factors included the number of clusters, the cluster size, and the model specification. We used Mplus 7.4 to generate simulated replications and estimate each of the models. We appropriately controlled the severity of misspecification when we generated the simulated replications. The simulation results suggested that applying RMSEAT_S_COV, TLIT _ S _ COV, and SRMRB maximizes the capacity to detect misspecifications in the between-covariance structure. Moreover, the use of RMSEAPS _ B, CFIPS _ B, and TLIPS _ B is recommended for evaluating the fit of the between-mean structure. Finally, we found that evaluation of the within-covariance structure turned out to be unexpectedly challenging, because none of the within-level-specific fit indices (RMSEAPS _ W, CFIPS _ W, TLIPS _ W, and SRMRW) had a practically significant sensitivity.

Analyzing individual growth with clustered longitudinal data: A comparison between model-based and design-based multilevel approaches.

To prevent biased estimates of intraindividual growth and interindividual variability when working with clustered longitudinal data (e.g., repeated measures nested within students; students nested within schools), individual dependency should be considered. A Monte Carlo study was conducted to examine to what extent two model-based approaches (multilevel latent growth curve model - MLGCM, and maximum model - MM) and one design-based approach (design-based latent growth curve model - D-LGCM) could produce unbiased and efficient parameter estimates of intraindividual growth and interindividual variability given clustered longitudinal data. The solutions of a single-level latent growth curve model (SLGCM) were also provided to demonstrate the consequences of ignoring individual dependency. Design factors considered in the present simulation study were as follows: number of clusters (NC = 10, 30, 50, 100, 150, 200, and 500) and cluster size (CS = 5, 10, and 20). According to our results, when intraindividual growth is of interest, researchers are free to implement MLGCM, MM, or D-LGCM. With regard to interindividual variability, MLGCM and MM were capable of producing accurate parameter estimates and SEs. However, when D-LGCM and SLGCM were applied, parameter estimates of interindividual variability were not comprised exclusively of the variability in individual (e.g., students) growth but instead were the combined variability of individual and cluster (e.g., school) growth, which cannot be interpreted. The take-home message is that D-LGCM does not qualify as an alternative approach to analyzing clustered longitudinal data if interindividual variability is of interest.

Detecting Misspecified Multilevel Structural Equation Models with Common Fit Indices: A Monte Carlo Study.

This study investigated the sensitivity of common fit indices (i.e., RMSEA, CFI, TLI, SRMR-W, and SRMR-B) for detecting misspecified multilevel SEMs. The design factors for the Monte Carlo study were numbers of groups in between-group models (100, 150, and 300), group size (10, 20, 30, and 60), intra-class correlation (low, medium, and high), and the types of model misspecification (Simple and Complex). The simulation results showed that CFI, TLI, and RMSEA could only identify the misspecification in the within-group model. Additionally, CFI, TLI, and RMSEA were more sensitive to misspecification in pattern coefficients while SRMR-W was more sensitive to misspecification in factor covariance. Moreover, TLI outperformed both CFI and RMSEA in terms of the hit rates of detecting the within-group misspecification in factor covariance. On the other hand, SRMR-B was the only fit index sensitive to misspecification in the between-group model and more sensitive to misspecification in factor covariance than misspecification in pattern coefficients. Finally, we found that the influence of ICC on the performance of targeted fit indices was trivial.

A systematic review of and reflection on the applications of factor mixture modeling.

Factor mixture modeling (FMM) incorporates both continuous latent variables and categorical latent variables in a single analytic model clustering items and observations simultaneously. After two decades since the introduction of FMM to psychological and behavioral science research, it is an opportune time to review FMM applications to understand how these applications are utilized in real-world research. We conducted a systematic review of 76 FMM applications. We developed a comprehensive coding scheme based on the current methodological literature of FMM and evaluated common usages and practices of FMM. Based on the review, we identify challenges and issues that applied researchers encounter in the practice of FMM and provide practical suggestions to promote well-informed decision making. Lastly, we discuss future methodological directions and suggest how FMM can be expanded beyond its typical use in applied studies. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The Optimal Starting Model to Search for the Accurate Growth Trajectory in Latent Growth Models.

This simulation study aims to propose an optimal starting model to search for the accurate growth trajectory in Latent Growth Models (LGM). We examine the performance of four different starting models in terms of the complexity of the mean and within-subject variance-covariance (V-CV) structures when there are time-invariant covariates embedded in the population models. Results showed that the model search starting with the fully saturated model (i.e., the most complex mean and within-subject V-CV model) recovers best for the true growth trajectory in simulations. Specifically, the fully saturated starting model with using ΔBIC and ΔAIC performed best (over 95%) and recommended for researchers. An illustration of the proposed method is given using the empirical secondary dataset. Implications of the findings and limitations are discussed.

The Impact of Intraclass Correlation on the Effectiveness of Level-Specific Fit Indices in Multilevel Structural Equation Modeling: A Monte Carlo Study.

Several researchers have recommended that level-specific fit indices should be applied to detect the lack of model fit at any level in multilevel structural equation models. Although we concur with their view, we note that these studies did not sufficiently consider the impact of intraclass correlation (ICC) on the performance of level-specific fit indices. Our study proposed to fill this gap in the methodological literature. A Monte Carlo study was conducted to investigate the performance of (a) level-specific fit indices derived by a partially saturated model method (e.g., [Formula: see text] and [Formula: see text]) and (b) [Formula: see text] and [Formula: see text] in terms of their performance in multilevel structural equation models across varying ICCs. The design factors included intraclass correlation (ICC: ICC1 = 0.091 to ICC6 = 0.500), numbers of groups in between-level models (NG: 50, 100, 200, and 1,000), group size (GS: 30, 50, and 100), and type of misspecification (no misspecification, between-level misspecification, and within-level misspecification). Our simulation findings raise a concern regarding the performance of between-level-specific partial saturated fit indices in low ICC conditions: the performances of both [Formula: see text] and [Formula: see text] were more influenced by ICC compared with [Formula: see text] and SRMRB . However, when traditional cutoff values (RMSEA≤ 0.06; CFI, TLI≥ 0.95; SRMR≤ 0.08) were applied, [Formula: see text] and [Formula: see text] were still able to detect misspecified between-level models even when ICC was as low as 0.091 (ICC1). On the other hand, both [Formula: see text] and [Formula: see text] were not recommended under low ICC conditions.

Evidence of convergent and discriminant validity of child, teacher, and peer reports of teacher-student support.

This study investigated the construct validity of measures of teacher-student support in a sample of 709 ethnically diverse 2nd- and 3rd-grade academically at-risk students. Confirmatory factor analysis investigated the convergent and discriminant validities of teacher, child, and peer reports of teacher-student support and child conduct problems. Results supported the convergent and discriminant validity of scores on the measures. Peer reports accounted for the largest proportion of trait variance and nonsignificant method variance. Child reports accounted for the smallest proportion of trait variance and the largest method variance. A model with 2 latent factors provided a better fit to the data than a model with 1 factor, providing further evidence of the discriminant validity of measures of teacher-student support. Implications for research, policy, and practice are discussed.