Dynamic structural equation models with binary and ordinal outcomes in Mplus.

Intensive longitudinal designs are increasingly popular, as are dynamic structural equation models (DSEM) to accommodate unique features of these designs. Many helpful resources on DSEM exist, though they focus on continuous outcomes while categorical outcomes are omitted, briefly mentioned, or considered as a straightforward extension. This viewpoint regarding categorical outcomes is not unwarranted for technical audiences, but there are non-trivial nuances in model building and interpretation with categorical outcomes that are not necessarily straightforward for empirical researchers. Furthermore, categorical outcomes are common given that binary behavioral indicators or Likert responses are frequently solicited as low-burden variables to discourage participant non-response. This tutorial paper is therefore dedicated to providing an accessible treatment of DSEM in Mplus exclusively for categorical outcomes. We cover the general probit model whereby the raw categorical responses are assumed to come from an underlying normal process. We cover probit DSEM and expound why existing treatments have considered categorical outcomes as a straightforward extension of the continuous case. Data from a motivating ecological momentary assessment study with a binary outcome are used to demonstrate an unconditional model, a model with disaggregated covariates, and a model for data with a time trend. We provide annotated Mplus code for these models and discuss interpretation of the results. We then discuss model specification and interpretation in the case of an ordinal outcome and provide an example to highlight differences between ordinal and binary outcomes. We conclude with a discussion of caveats and extensions.

Association of BMI trajectories with cardiometabolic risk among low-income Mexican American children.

BACKGROUND: The aim of this study was to identify distinct trajectories of BMI growth from 2 to 7.5 years and examine their associations with markers of cardiometabolic risk at age 7.5 years among a sample of low-income Mexican American children. METHODS: This longitudinal cohort study recruited 322 mother-child dyads to participate prenatally and at child age 2, 3, 4.5, 6, and 7.5 years. Child height/weight, waist circumference, and blood pressure were assessed at each time point. Blood was collected from child at 7.5 years. RESULTS: Covarying for birthweight, three BMI trajectories were identified: Low-Stable BMI (73% of the sample), High-Stable BMI (5.6% of the sample), and Increasing BMI over time (21.4% of the sample). The High-Stable and Increasing BMI classes had higher waist circumference and systolic blood pressure and lower HDL-c than the Low-Stable BMI class (ps < 0.05). Among children with BMIs below the 85th percentile, 16% had three or more cardiometabolic risk indicators. CONCLUSIONS: BMI classes were consistent with existing literature. For youth, standard medical practice is to examine cardiometabolic risk indicators when BMI is high; however, this practice would miss 16% of youth in our sample who exhibit cardiometabolic risk but do not screen in based on BMI. IMPACT: Research indicates Mexican American youth are at risk for cardiometabolic dysregulation relative to other ethnic groups, yet there is a paucity of longitudinal research. An Increasing BMI and a High-Stable BMI class were associated with larger waist circumference, higher systolic blood pressure, and lower HDL cholesterol than the Low-Stable BMI class. BMI trajectories in childhood predict cardiometabolic risk indicators. As the sole screener for deciding when to test cardiometabolic indicators, BMI alone will miss some children exhibiting cardiometabolic dysregulation.

Facilitating Growth Mixture Model Convergence in Preventive Interventions.

Growth mixture models (GMMs) are applied to intervention studies with repeated measures to explore heterogeneity in the intervention effect. However, traditional GMMs are known to be difficult to estimate, especially at sample sizes common in single-center interventions. Common strategies to coerce GMMs to converge involve post hoc adjustments to the model, particularly constraining covariance parameters to equality across classes. Methodological studies have shown that although convergence is improved with post hoc adjustments, they embed additional tenuous assumptions into the model that can adversely impact key aspects of the model such as number of classes extracted and the estimated growth trajectories in each class. To facilitate convergence without post hoc adjustments, this paper reviews the recent literature on covariance pattern mixture models, which approach GMMs from a marginal modeling tradition rather than the random effect modeling tradition used by traditional GMMs. We discuss how the marginal modeling tradition can avoid complexities in estimation encountered by GMMs that feature random effects, and we use data from a lifestyle intervention for increasing insulin sensitivity (a risk factor for type 2 diabetes) among 90 Latino adolescents with obesity to demonstrate our point. Specifically, GMMs featuring random effects-even with post hoc adjustments-fail to converge due to estimation errors, whereas covariance pattern mixture models following the marginal model tradition encounter no issues with estimation while maintaining the ability to answer all the research questions.

Generalizability of Dynamic Fit Index, Equivalence Testing, and Hu & Bentler Cutoffs for Evaluating Fit in Factor Analysis.

Factor analysis is often used to model scales created to measure latent constructs, and internal structure validity evidence is commonly assessed with indices like RMSEA, and CFI. These indices are essentially effect size measures and definitive benchmarks regarding which values connote reasonable fit have been elusive. Simulations from the 1990s suggesting possible benchmark values are among the most highly cited methodological papers across any discipline. However, simulations have suggested that fixed benchmarks do not generalize well - fit indices are systematically impacted by characteristics like the number of items and the magnitude of the loadings, so fixed benchmarks can confound misfit with model characteristics. Alternative frameworks for creating customized, model-specific benchmarks have recently been proposed to circumvent these issues but they have not been systematically evaluated. Motivated by two empirical applications where different methods yield inconsistent conclusions, two simulation studies are performed to assess the ability of three different approaches to correctly classify models that are correct or misspecified across different conditions. Results show that dynamic fit indices and equivalence testing both improved upon the traditional Hu & Bentler benchmarks and dynamic fit indices appeared to be least confounded with model characteristics in the conditions studied.

dynamic : An R Package for Deriving Dynamic Fit Index Cutoffs for Factor Analysis.

To evaluate the fit of a confirmatory factor analysis model, researchers often rely on fit indices such as SRMR, RMSEA, and CFI. These indices are frequently compared to benchmark values of .08, .06, and .96, respectively, established by Hu and Bentler (Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1-55). However, these indices are affected by model characteristics and their sensitivity to misfit can change across models. Decisions about model fit can therefore be improved by tailoring cutoffs to each model. The methodological literature has proposed methods for deriving customized cutoffs, although it can require knowledge of linear algebra and Monte Carlo simulation. Given that many empirical researchers do not have training in these technical areas, empirical studies largely continue to rely on fixed benchmarks even though they are known to generalize poorly and can be poor arbiters of fit. To address this, this paper introduces the R package, dynamic, to make computation of dynamic fit index cutoffs (which are tailored to the user's model) more accessible to empirical researchers. dynamic heavily automatizes this process and only requires a lavaan object to automatically conduct several custom Monte Carlo simulations and output fit index cutoffs designed to be sensitive to misfit with the user's model characteristics.

Psychometric properties of sum scores and factor scores differ even when their correlation is 0.98: A response to Widaman and Revelle.

Commentary in Widaman and Revelle (2022) argued that sum scoring is justified as long as unidimensionality holds because sum score reliability is defined. My response begins with a review of the literature supporting the perspective we adopted in the original article. I then conduct simulation studies to assess the psychometric properties of sum scores created using Widaman and Revelle's justification relative to scores created by the weighted factor score approach in the original article. In my simulations, I generate data where sum and factor scores are correlated at 0.96 or 0.98 because high factor-sum score correlations are often used to support the contention that sum and factor scores have interchangeable psychometric properties. I explore (a) correlations between estimated scores and true scores, (b) classification accuracy of sum and factor scores, and (c) reliability of sum and factor scores. Results show that factor scores have (a) higher correlations with true scores (Δ = 0.02-0.04), (b) higher sensitivity (Δ = 4-8 percentage points), and (c) higher reliability (Δ = 0.04-0.07). Factor score performance metrics also have less sampling variability in most conditions. Psychometric properties of sum scores-even when highly correlated with factor scores-remain less desirable than those of factor scores. Additional considerations like models with multiple factors and measurement invariance are also discussed. Essentially, even if accepting Widaman and Revelle's justification for sum scoring, it is uncertain whether researchers generally would want to sum score after fitting a factor analysis unless sum and factor scores correlate at (and not merely close to) 1.00.

Dynamic fit index cutoffs for one-factor models.

Assessing whether a multiple-item scale can be represented with a one-factor model is a frequent interest in behavioral research. Often, this is done in a factor analysis framework with approximate fit indices like RMSEA, CFI, or SRMR. These fit indices are continuous measures, so values indicating acceptable fit are up to interpretation. Cutoffs suggested by Hu and Bentler (1999) are a common guideline used in empirical research. However, these cutoffs were derived with intent to detect omitted cross-loadings or omitted factor covariances in multifactor models. These types of misspecifications cannot exist in one-factor models, so the appropriateness of using these guidelines in one-factor models is uncertain. This paper uses a simulation study to address whether traditional fit index cutoffs are sensitive to the types of misspecifications common in one-factor models. The results showed that traditional cutoffs have very poor sensitivity to misspecification in one-factor models and that the traditional cutoffs generalize poorly to one-factor contexts. As an alternative, we investigate the accuracy and stability of the recently introduced dynamic fit cutoff approach for creating fit index cutoffs for one-factor models. Simulation results indicated excellent performance of dynamic fit index cutoffs to classify correct or misspecified one-factor models and that dynamic fit index cutoffs are a promising approach for more accurate assessment of model fit in one-factor contexts.

Reducing Incidence of Nonpositive Definite Covariance Matrices in Mixed Effect Models.

Deciding which random effects to retain is a central decision in mixed effect models. Recent recommendations advise a maximal structure whereby all theoretically relevant random effects are retained. Nonetheless, including many random effects often leads to nonpositive definiteness. A typical remedy is to simplify the random effect structure by removing random effects or associated covariances. However, this practice is known to bias estimates of remaining covariance parameters and compromise fixed effect inferences. Cholesky decompositions frequently are suggested as an alternative and are automatically implemented in some software. Instead of Cholesky decompositions, we describe factor analytic structures as an approach to avoid nonpositive definiteness. This approach is occasionally employed in biosciences like plant breeding, but, ironically, has not been established in behavioral sciences despite the close historical connection with factor analysis in these fields. We discuss how a factor analytic structure facilitates estimation and conduct simulations to compare convergence and performance to simplifying the random effects structure or Cholesky decomposition approaches. Results show a lower rate of nonpositive definiteness with the factor analytic structure than Cholesky decomposition and suggest that factor analytic covariance structure may be useful to combating nonpositive definiteness, especially in models with many random effects.

Second-by-second infant and mother emotion regulation and coregulation processes.

Context-appropriate infant physiological functioning may support emotion regulation and mother-infant emotion coregulation. Among a sample of 210 low-income Mexican-origin mothers and their 24-week-old infants, dynamic structural equation modeling (DSEM) was used to examine whether within-infant vagal functioning accounted for between-dyad differences in within-dyad second-by-second emotion regulation and coregulation during free play. Vagal functioning was captured by within-infant mean and variability (standard deviation) of respiratory sinus arrhythmia (RSA) during free play. Infant emotion regulation was quantified as emotional equilibria (within-person mean), volatility (within-person deviation from equilibrium), carryover (how quickly equilibrium is restored following a disturbance), and feedback loops (the extent to which prior affect dampens or amplifies subsequent affect) in positive and negative affect during free play; coregulation was quantified as the influence of one partner's affect on the other's subsequent affect. Among infants with lower RSA variability, positive affect fluctuated around a higher equilibrium, and negative affect fluctuated around a lower equilibrium; these infants exhibited feedback loops where their positive affect dampened their subsequent negative affect. As expected, infants with higher mean RSA exhibited more volatility in positive affect, feedback loops between their positive and negative affect, and stronger mother-driven emotion coregulation. The results highlight differences in simultaneously occurring biological and emotion regulation.

Response heterogeneity to lifestyle intervention among Latino adolescents.

OBJECTIVE: To characterize the heterogeneity in response to lifestyle intervention among Latino adolescents with obesity. METHODS: We conducted secondary data analysis of 90 Latino adolescents (age 15.4 ± 0.9 y, female 56.7%) with obesity (BMI% 98.1 ± 1.5%) that were enrolled in a 3 month lifestyle intervention and were followed for a year. Covariance pattern mixture models identified response phenotypes defined by changes in insulin sensitivity as measured using a 2 hour oral glucose tolerance test. Baseline characteristics were compared across response phenotypes using one-way ANOVA and chi-square test. RESULTS: Three distinct response phenotypes (PH1, PH2, PH3) were identified. PH1 exhibited the most robust response defined by the greatest increase in insulin sensitivity over time (ß ± SE, linear 0.52 ± 0.17, P < .001; quadratic -0.03 ± 0.01, P = .001). PH2 showed non-significant changes, while PH3 demonstrated modest short-term increases in insulin sensitivity which were not sustained over time (linear 0.08 ± 0.03, P = .002; quadratic -0.01 ± 0.002, P = .003). At baseline, PH3 (1.1 ± 0.4) was the most insulin resistant phenotype and exhibited the highest BMI% (98.5 ± 1.1%), 2 hours glucose concentrations (144.0 ± 27.5 mg/dL), and lowest beta-cell function as estimated by the oral disposition index (4.5 ± 2.8). CONCLUSION: Response to lifestyle intervention varies among Latino youth with obesity and suggests that precision approaches are warranted to meet the prevention needs of high risk youth.

Social network isolation mediates associations between risky symptoms and substance use in the high school transition.

The current study examined whether social status and social integration, two related but distinct indicators of an adolescent's standing within a peer network, mediate the association between risky symptoms (depressive symptoms and deviant behavior) and substance use across adolescence. The sample of 6,776 adolescents participated in up to seven waves of data collection spanning 6th to 12th grades. Scores indexing social status and integration were derived from a social network analysis of six schools and subsequent psychometric modeling. Results of latent growth models showed that social integration and status mediated the relation between risky symptoms and substance use and that risky symptoms mediated the relation between social standing and substance use during the high school transition. Before this transition, pathways involving deviant behavior led to high social integration and status and in turn to substance use. After this transition, both deviant behavior and depressive symptoms led to low social integration and status and in turn greater substance use. These findings suggest that the high school transition is a risky time for substance use related to the interplay of increases in depressive symptoms and deviant behavior on the one hand and decreases in social status and integration on the other.

Estimating New Quantities from Longitudinal Test Scores to Improve Forecasts of Future Performance.

Psychometric models for longitudinal test scores typically estimate quantities associated with single-administration tests, like ability at each time-point. However, models for longitudinal tests have not considered opportunities to estimate new quantities that are unavailable from single-administration tests. Specifically, we discuss dynamic measurement models - which combine aspects of longitudinal IRT, nonlinear growth models, and dynamic assessment - to directly estimate capacity, defined as the expected future score once the construct has fully developed. After discussing the history and connecting these areas into a single framework, we apply the model to verbal test scores from the Intergenerational Studies, which follow 494 people from 3 to 72 years old. The goal is to predict adult verbal scores (Age ≥ 34) from adolescent scores (Age ≤ 20). We held-out the adult data for prediction and compared predictions from traditional longitudinal IRT ability scores and proposed dynamic measurement capacity scores from models fit to the adolescent data. Results showed that the R2 from capacity scores were 2.5 times larger than the R2 from longitudinal IRT ability scores (43% vs. 16%), providing some evidence that exploring new quantities available from longitudinal testing could be worthwhile when an interest in testing is forecasting future performance.

Covariance pattern mixture models: Eliminating random effects to improve convergence and performance.

Growth mixture models (GMMs) are prevalent for modeling unknown population heterogeneity via distinct latent classes. However, GMMs are riddled with convergence issues, often requiring researchers to atheoretically alter the model with cross-class constraints simply to obtain convergence. We discuss how within-class random effects in GMMs exacerbate convergence issues, even though these random effects rarely help answer typical research questions. That is, latent classes provide a discretization of continuous random effects, so including additional random effects within latent classes can unnecessarily complicate the model. These random effects are commonly included in order to properly specify the marginal covariance; however, random effects are inefficient for patterning a covariance matrix, resulting in estimation issues. Such a goal can be achieved more simply through covariance pattern models, which we extend to the mixture model context in this article (covariance pattern mixture models, or CPMMs). We provide evidence from theory, simulation, and an empirical example showing that employing CPMMs (even if they are misspecified) instead of GMMs can circumvent the computational difficulties that can plague GMMs, without sacrificing the ability to answer the types of questions commonly asked in empirical studies. Our results show the advantages of CPMMs with respect to improved class enumeration and less biased class-specific growth trajectories, in addition to their vastly improved convergence rates. The results also show that constraining the covariance parameters across classes in order to bypass convergence issues with GMMs leads to poor results. An extensive software appendix is included to assist researchers in running CPMMs in Mplus.

Thinking twice about sum scores.

A common way to form scores from multiple-item scales is to sum responses of all items. Though sum scoring is often contrasted with factor analysis as a competing method, we review how factor analysis and sum scoring both fall under the larger umbrella of latent variable models, with sum scoring being a constrained version of a factor analysis. Despite similarities, reporting of psychometric properties for sum scored or factor analyzed scales are quite different. Further, if researchers use factor analysis to validate a scale but subsequently sum score the scale, this employs a model that differs from validation model. By framing sum scoring within a latent variable framework, our goal is to raise awareness that (a) sum scoring requires rather strict constraints, (b) imposing these constraints requires the same type of justification as any other latent variable model, and (c) sum scoring corresponds to a statistical model and is not a model-free arithmetic calculation. We discuss how unjustified sum scoring can have adverse effects on validity, reliability, and qualitative classification from sum score cut-offs. We also discuss considerations for how to use scale scores in subsequent analyses and how these choices can alter conclusions. The general goal is to encourage researchers to more critically evaluate how they obtain, justify, and use multiple-item scale scores.

Poisson Multilevel Models with Small Samples.

Recent methodological studies have investigated the properties of multilevel models with small samples. Previous work has primarily focused on continuous outcomes and little attention has been paid to count outcomes. The estimation of count outcome models can be difficult because the likelihood has no closed-form solution, meaning that approximation methods are required. Although adaptive Gaussian quadrature (AGQ) is generally seen as the gold standard, its comparative performance has been investigated with larger samples. AGQ approximates the full likelihood, a function that is known to produce biased estimates with small samples with continuous outcomes. Conversely, penalized quasi-likelihood (PQL) is considered to be a less desirable approximation; however, it can approximate the restricted likelihood function, a function that is known to perform well with smaller samples with continuous outcomes. The goal of this paper is to compare the small sample bias of full likelihood methods to the linearization bias of PQL with restricted likelihood. Simulation results indicate that the linearization bias of PQL is preferable to the finite sample bias of AGQ with smaller samples.

Effect Partitioning in Cross-Sectionally Clustered Data Without Multilevel Models.

Effect partitioning is almost exclusively performed with multilevel models (MLMs) - so much so that some have considered the two to be synonymous. MLMs are able to provide estimates with desirable statistical properties when data come from a hierarchical structure; but the random effects included in MLMs are not always integral to the analysis. As a result, other methods with relaxed assumptions are viable options in many cases. Through empirical examples and simulations, we show how generalized estimating equations (GEEs) can be used to effectively partition effects without random effects. We show that more onerous steps of MLMs such as determining the number of random effects and the structure for their covariance can be bypassed with GEEs while still obtaining identical or near-identical results. Additionally, violations of distributional assumptions adversely affect estimates with MLMs but have no effect on GEEs because no such assumptions are made. This makes GEEs a flexible alternative to MLMs with minimal assumptions that may warrant consideration. Limitations of GEEs for partitioning effects are also discussed.

The Thorny Relation Between Measurement Quality and Fit Index Cutoffs in Latent Variable Models.

Latent variable modeling is a popular and flexible statistical framework. Concomitant with fitting latent variable models is assessment of how well the theoretical model fits the observed data. Although firm cutoffs for these fit indexes are often cited, recent statistical proofs and simulations have shown that these fit indexes are highly susceptible to measurement quality. For instance, a root mean square error of approximation (RMSEA) value of 0.06 (conventionally thought to indicate good fit) can actually indicate poor fit with poor measurement quality (e.g., standardized factors loadings of around 0.40). Conversely, an RMSEA value of 0.20 (conventionally thought to indicate very poor fit) can indicate acceptable fit with very high measurement quality (standardized factor loadings around 0.90). Despite the wide-ranging effect on applications of latent variable models, the high level of technical detail involved with this phenomenon has curtailed the exposure of these important findings to empirical researchers who are employing these methods. This article briefly reviews these methodological studies in minimal technical detail and provides a demonstration to easily quantify the large influence measurement quality has on fit index values and how greatly the cutoffs would change if they were derived under an alternative level of measurement quality. Recommendations for best practice are also discussed.

Differentiating between mixed-effects and latent-curve approaches to growth modeling.

In psychology, mixed-effects models and latent-curve models are both widely used to explore growth over time. Despite this widespread popularity, some confusion remains regarding the overlap of these different approaches. Recent articles have shown that the two modeling frameworks are mathematically equivalent in many cases, which is often interpreted to mean that one's choice of modeling framework is merely a matter of personal preference. However, some important differences in estimation and specification can lead to the models producing very different results when implemented in software. Thus, mathematical equivalence does not necessarily equate to practical equivalence in all cases. In this article, we discuss these two common approaches to growth modeling and highlight contexts in which the choice of the modeling framework (and, consequently, the software) can directly impact the model estimates, or in which certain analyses can be facilitated in one framework over the other. We show that, unless the data are pristine, with a large sample size, linear or polynomial growth, and no missing data, and unless the participants have the same number of measurements collected at the same set of time points, one framework is often more advantageous to adopt. We provide several empirical examples to illustrate these situations, as well as ample software code so that researchers can make informed decisions regarding which framework will be the most beneficial and most straightforward for their research interests.

Small Sample Methods for Multilevel Modeling: A Colloquial Elucidation of REML and the Kenward-Roger Correction.

Studies on small sample properties of multilevel models have become increasingly prominent in the methodological literature in response to the frequency with which small sample data appear in empirical studies. Simulation results generally recommend that empirical researchers employ restricted maximum likelihood estimation (REML) with a Kenward-Roger correction with small samples in frequentist contexts to minimize small sample bias in estimation and to prevent inflation of Type-I error rates. However, simulation studies focus on recommendations for best practice, and there is little to no explanation of why traditional maximum likelihood (ML) breaks down with smaller samples, what differentiates REML from ML, or how the Kenward-Roger correction remedies lingering small sample issues. Due to the complexity of these methods, most extant descriptions are highly mathematical and are intended to prove that the methods improve small sample performance as intended. Thus, empirical researchers have documentation that these methods are advantageous but still lack resources to help understand what the methods actually do and why they are needed. This tutorial explains why ML falters with small samples, how REML circumvents some issues, and how Kenward-Roger works. We do so without equations or derivations to support more widespread understanding and use of these valuable methods.

Nonlinear Growth Models as Measurement Models: A Second-Order Growth Curve Model for Measuring Potential.

Recent methodological work has highlighted the promise of nonlinear growth models for addressing substantive questions in the behavioral sciences. In this article, we outline a second-order nonlinear growth model in order to measure a critical notion in development and education: potential. Here, potential is conceptualized as having three components-ability, capacity, and availability-where ability is the amount of skill a student is estimated to have at a given timepoint, capacity is the maximum amount of ability a student is predicted to be able to develop asymptotically, and availability is the difference between capacity and ability at any particular timepoint. We argue that single timepoint measures are typically insufficient for discerning information about potential, and we therefore describe a general framework that incorporates a growth model into the measurement model to capture these three components. Then, we provide an illustrative example using the public-use Early Childhood Longitudinal Study-Kindergarten data set using a Michaelis-Menten growth function (reparameterized from its common application in biochemistry) to demonstrate our proposed model as applied to measuring potential within an educational context. The advantage of this approach compared to currently utilized methods is discussed as are future directions and limitations.