Evaluating SEM Model Fit with Small Degrees of Freedom.

Research has revealed that the performance of root mean square error of approximation (RMSEA) in assessing structural equation models with small degrees of freedom (df) is suboptimal, often resulting in the rejection of correctly specified or closely fitted models. This study investigates the performance of standardized root mean square residual (SRMR) and comparative fit index (CFI) in small df models with various levels of factor loadings, sample sizes, and model misspecifications. We find that, in comparison with RMSEA, population SRMR and CFI are less susceptible to the effects of df. In small df models, the sample SRMR and CFI could provide more useful information to differentiate models with various levels of misfit. The confidence intervals and p-values of a close fit were generally accurate for all three fit indices. We recommend researchers use caution when interpreting RMSEA for models with small df and to rely more on SRMR and CFI.

The impact of summer vacation on children's obesogenic behaviors and body mass index: a natural experiment.

BACKGROUND: Children's BMI gain accelerates during summer. The Structured Days Hypothesis posits that the lack of the school day during summer vacation negatively impacts children's obesogenic behaviors (i.e., physical activity, screen time, diet, sleep). This natural experiment examined the impact of summer vacation on children's obesogenic behaviors and body mass index (BMI). METHODS: Elementary-aged children (n = 285, 5-12 years, 48.7% male, 57.4% African American) attending a year-round (n = 97) and two match-paired traditional schools (n = 188) in the United States participated in this study. Rather than taking a long break from school during the summer like traditional schools, year-round schools take shorter and more frequent breaks from school. This difference in school calendars allowed for obesogenic behaviors to be collected during three conditions: Condition 1) all children attend school, Condition 2) year-round children attend school while traditional children were on summer vacation, and Condition 3) summer vacation for all children. Changes in BMI z-score were collected for the corresponding school years and summers. Multi-level mixed effects regressions estimated obesogenic behaviors and monthly zBMI changes. It was hypothesized that children would experience unhealthy changes in obesogenic behaviors when entering summer vacation because the absence of the school day (i.e., Condition 1 vs. 2 for traditional school children and 2 vs. 3 for year-round school children). RESULTS: From Condition 1 to 2 traditional school children experienced greater unhealthy changes in daily minutes sedentary (∆ = 24.2, 95CI = 10.2, 38.2), screen time minutes (∆ = 33.7, 95CI = 17.2, 50.3), sleep midpoint time (∆ = 73:43, 95CI = 65:33, 81:53), and sleep efficiency percentage (-∆ = 0.7, 95CI = -1.1, - 0.3) when compared to year-round school children. Alternatively, from Condition 2 to 3 year-round school children experienced greater unhealthy changes in daily minutes sedentary (∆ = 54.5, 95CI = 38.0, 70.9), light physical activity minutes (∆ = - 42.2, 95CI = -56.2, - 28.3) MVPA minutes (∆ = - 11.4, 95CI = -3.7, - 19.1), screen time minutes (∆ = 46.5, 95CI = 30.0, 63.0), and sleep midpoint time (∆ = 95:54, 95CI = 85:26, 106:22) when compared to traditional school children. Monthly zBMI gain accelerated during summer for traditional (∆ = 0.033 95CI = 0.019, 0.047) but not year-round school children (∆ = 0.004, 95CI = -0.014, 0.023). CONCLUSIONS: This study suggests that the lack of the school day during summer vacation negatively impacts sedentary behaviors, sleep timing, and screen time. Changes in sedentary behaviors, screen time, and sleep midpoint may contribute to accelerated summer BMI gain. Providing structured programming during summer vacation may positively impact these behaviors, and in turn, mitigate accelerated summer BMI gain. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT03397940 . Registered January 12th 2018.

Evaluating Factorial Invariance: An Interval Estimation Approach Using Bayesian Structural Equation Modeling.

In this study, we introduce an interval estimation approach based on Bayesian structural equation modeling to evaluate factorial invariance. For each tested parameter, the size of noninvariance with an uncertainty interval (i.e. highest density interval [HDI]) is assessed via Bayesian parameter estimation. By comparing the most credible values (i.e. 95% HDI) with a region of practical equivalence (ROPE), the Bayesian approach allows researchers to (1) support the null hypothesis of practical invariance, and (2) examine the practical importance of the noninvariant parameter. Compared to the traditional likelihood ratio test, simulation results suggested that the proposed Bayesian approach could offer additional insight into evaluating factorial invariance, thus, leading to more informative conclusions. We provide an empirical example to demonstrate the procedures necessary to implement the proposed method in applied research. The importance of and influences on the choice of an appropriate ROPE are discussed.

The Meaning of Aggression Varies Across Culture: Testing the Measurement Invariance of the Refined Aggression Questionnaire in Samples From Spain, the United States, and Hong Kong.

Cultural differences in aggression are still poorly understood. The purpose of this article is to assess whether a tool for measuring aggression has the same meaning across cultures. Analyzing samples from Spain (n = 262), the United States (n = 344), and Hong Kong (n = 645), we used confirmatory factor analysis to investigate measurement invariance of the refined version of the Aggression Questionnaire (Bryant & Smith, 2001 ). The measurement of aggression was more equivalent between the Chinese and Spanish versions than between these two and the U.S. version. Aggression does not show invariance at the cultural level. Cultural variables such as affective autonomy or individualism could influence the meaning of aggression. Aggressive behavior models can be improved by incorporating cultural variables.

The Relationship Between the Standardized Root Mean Square Residual and Model Misspecification in Factor Analysis Models.

We argue that the definition of close fitting models should embody the notion of substantially ignorable misspecifications (SIM). A SIM model is a misspecified model that might be selected, based on parsimony, over the true model should knowledge of the true model be available. Because in applications the true model (i.e., the data generating mechanism) is unknown, we investigate the relationship between the population standardized root mean square residual (SRMR) values and various model misspecifications in factor analysis models to better understand the magnitudes of the SRMR. Summary effect sizes of misfit such as the SRMR are necessarily insensitive to some non-ignorable localized misspecifications (i.e., the presence of a few large residual correlations in large models). Localized misspecifications may be identified by examining the largest standardized residual covariance. Based on the findings, our population reference values for close fit are based on a two-index strategy: (1) largest absolute value of standardized residual covariance ≤0.10, and (2) SRMR ≤0.05× R¯2 the average R2 of the manifest variables; for acceptable fit our values are 0.15 and 0.10× R¯2 , respectively.

Improving Psychological Measurement: Does It Make a Difference? A Comment on Nesselroade and Molenaar (2016).

Nesselroade and Molenaar advocate the use of an idiographic filter approach. This is a fixed-effects approach, which may limit the number of individuals that can be simultaneously modeled, and it is not clear how to model the presence of subpopulations. Most important, Nesselroade and Molenaar's proposal appears to be best suited for modeling long time series on a few variables for a few individuals. Long time series are not common in psychological applications. Can it be applied to the usual longitudinal data we face? These are characterized by short time series (four to five points in time), hundreds of individuals, and dozens of variables. If so, what do we gain? Applied settings most often involve between-individual decisions. I conjecture that their approach will not outperform common, simpler, methods. However, when intraindividual decisions are involved, their approach may have an edge.

There's More Than Catastrophizing in Chronic Pain: Low Frustration Tolerance and Self-Downing Also Predict Mental Health in Chronic Pain Patients.

Among the potential range of irrational beliefs that could be used as predictors of physical and mental health, catastrophizing is the process that has received most attention in chronic pain research. Other irrational processes such as demandingness, low frustration tolerance, and self-downing have rarely been studied. The goal of this study was to explore whether this wider range of beliefs is associated with health in chronic pain patients beyond catastrophizing. A total of 492 chronic pain patients completed a measure of irrational beliefs, a measure of physical and mental health, and a numerical rating scale designed to assess pain intensity and interference. Irrational processes were more strongly associated with mental than with physical health. Low frustration tolerance and self-downing were found to be significantly related to mental health even after controlling for the effect of catastrophizing. Processes other than catastrophizing appear to have potentially important relationships with the mental health of people with chronic pain. These results may offer new intervention targets for practitioners.

Assessing Approximate Fit in Categorical Data Analysis.

A family of Root Mean Square Error of Approximation (RMSEA) statistics is proposed for assessing the goodness of approximation in discrete multivariate analysis with applications to item response theory (IRT) models. The family includes RMSEAs to assess the approximation up to any level of association of the discrete variables. Two members of this family are RMSEA2, which uses up to bivariate moments, and the full information RMSEAn. The RMSEA2 is estimated using the M2 statistic of Maydeu-Olivares and Joe (2005, 2006), whereas for maximum likelihood estimation, RMSEAn is estimated using Pearson's X(2) statistic. Using IRT models, we provide cutoff criteria of adequate, good, and excellent fit using the RMSEA2. When the data are ordinal, we find a strong linear relationship between the RMSEA2 and the Standardized Root Mean Squared Residual goodness-of-fit index. We are unable to offer cutoff criteria for the RMSEAn as its population values decrease as the number of variables and categories increase.

Identifying the Source of Misfit in Item Response Theory Models.

When an item response theory model fails to fit adequately, the items for which the model provides a good fit and those for which it does not must be determined. To this end, we compare the performance of several fit statistics for item pairs with known asymptotic distributions under maximum likelihood estimation of the item parameters: (a) a mean and variance adjustment to bivariate Pearson's X(2), (b) a bivariate subtable analog to Reiser's (1996) overall goodness-of-fit test, (c) a z statistic for the bivariate residual cross product, and (d) Maydeu-Olivares and Joe's (2006) M2 statistic applied to bivariate subtables. The unadjusted Pearson's X(2) with heuristically determined degrees of freedom is also included in the comparison. For binary and ordinal data, our simulation results suggest that the z statistic has the best Type I error and power behavior among all the statistics under investigation when the observed information matrix is used in its computation. However, if one has to use the cross-product information, the mean and variance adjusted X(2) is recommended. We illustrate the use of pairwise fit statistics in 2 real-data examples and discuss possible extensions of the current research in various directions.

Environmental risk and protective factors of adolescents' and youths' mental health: differences between parents' appraisal and self-reports.

PURPOSE: We investigated the effect of parents' mental health, life events, and home life (among other factors) on adolescents'/youths' mental health, whether such an effect varies when several variables are assessed jointly, and also whether the informant source of the mental health problem modifies the estimations. METHODS: We studied a representative sample of 454 Spanish adolescents/youths studied longitudinally (2 assessments, 3 years apart). We considered factors associated with adolescents'/youths' mental health (conduct, emotional, and hyperactivity scores [SDQ]): risk factors (parents' mental health and life events) and mediators (social and financial support). Structural equation modeling was applied. We constructed two models: (a) with parents' SDQ responses and (b) with self-reported SDQ responses (in a subsample of N = 260). RESULTS: Model fit was adequate for parents' appraisal. Parents' mental health (p < 0.05) and undesirable life events (p < 0.05) were the most important risk factors. The same model showed poorer fit when self-reported measures were used. Home life exerted a stronger protective effect on adolescents'/youths' mental health when reported by adolescents/youths. The negative effect of parents' mental health was significantly protected by home life in emotional [-0.14 (0.07)] and hyperactivity scores [-0.2 (0.08)]. CONCLUSIONS: Even in the presence of other factors, parents' mental health has an important effect on adolescents'/youths' mental health. Good levels of home life are protective, especially when adolescents'/youths' mental health is self-reported.

Fitting a Thurstonian IRT model to forced-choice data using Mplus.

To counter response distortions associated with the use of rating scales (a.k.a. Likert scales), items can be presented in a comparative fashion, so that respondents are asked to rank the items within blocks (forced-choice format). However, classical scoring procedures for these forced-choice designs lead to ipsative data, which presents psychometric challenges that are well described in the literature. Recently, Brown and Maydeu-Olivares (Educational and Psychological Measurement 71: 460-502, 2011a) introduced a model based on Thurstone's law of comparative judgment, which overcomes the problems of ipsative data. Here, we provide a step-by-step tutorial for coding forced-choice responses, specifying a Thurstonian item response theory model that is appropriate for the design used, assessing the model's fit, and scoring individuals on psychological attributes. Estimation and scoring is performed using Mplus, and a very straightforward Excel macro is provided that writes full Mplus input files for any forced-choice design. Armed with these tools, using a forced-choice design is now as easy as using ratings.

Assessing the Accuracy of Errors of Measurement. Implications for Assessing Reliable Change in Clinical settings.

Item response theory (IRT) models are non-linear latent variable models for discrete measures, whereas factor analysis (FA) is a latent variable model for continuous measures. In FA, the standard error (SE) of individuals' scores is common for all individuals. In IRT, the SE depends on the individual's score, and the SE function is to be provided. The empirical standard deviation of the scores across discrete ranges should also be computed to inform the extent to which IRT SEs overestimate or underestimate the variability of the scores. Within the target range of scores the test was designed to measure, one should expect IRT SEs to be smaller and more precise than FA SEs, and therefore preferable to assess clinical change. Outside the target range, IRT SEs may be too large and more imprecise than FA SEs, and FA more precise to assess change. As a result, whether FA or IRT characterize reliable change more accurately in a sample will depend on the proportion of individuals within or outside the IRT target score range. An application is provided to illustrate these concepts.

Using the Standardized Root Mean Squared Residual (SRMR) to Assess Exact Fit in Structural Equation Models.

We examine the accuracy of p values obtained using the asymptotic mean and variance (MV) correction to the distribution of the sample standardized root mean squared residual (SRMR) proposed by Maydeu-Olivares to assess the exact fit of SEM models. In a simulation study, we found that under normality, the MV-corrected SRMR statistic provides reasonably accurate Type I errors even in small samples and for large models, clearly outperforming the current standard, that is, the likelihood ratio (LR) test. When data shows excess kurtosis, MV-corrected SRMR p values are only accurate in small models (p = 10), or in medium-sized models (p = 30) if no skewness is present and sample sizes are at least 500. Overall, when data are not normal, the MV-corrected LR test seems to outperform the MV-corrected SRMR. We elaborate on these findings by showing that the asymptotic approximation to the mean of the SRMR sampling distribution is quite accurate, while the asymptotic approximation to the standard deviation is not.

Dynamics of sleep, sedentary behavior, and moderate-to-vigorous physical activity on school versus nonschool days.

STUDY OBJECTIVES: Studies examining time-use activity behaviors (sleep, sedentary behavior, and physical activity) on school days compared with nonschool days have examined these behaviors independently, ignoring their interrelated nature, limiting our ability to optimize the health benefits of these behaviors. This study examines the associations of school-day (vs. nonschool day) with time-use activity behaviors. METHODS: Time series data (6,642 days) from Fitbits (Charge-2) were collected (n = 196, 53% female, 5-10 years). We used a variable-centered dynamic structural equation modeling approach to estimate day-to-day associations of time-use activity behaviors on school days for each child. We then used person-centered cluster analyses to group individuals based on these estimates. RESULTS: Within-participant analysis showed that on school days (vs. nonschool days), children (1) slept less (ß = -0.17, 95% CI = -0.21, -0.13), (2) were less sedentary (ß = -0.05, 95% CI = -0.09, -0.02), and (3) had comparable moderate-to-vigorous physical activity (MVPA; ß = -0.05, 95% CI = -0.11, 0.00). Between-participant analysis showed that, on school days, children with higher sleep carryover experienced greater decreases in sleep (ß = 0.44, 95% CI = 0.08, 0.71), children with higher body mass index z-score decreased sedentary behavior more (ß = -0.41, 95% CI = -0.64, -0.13), and children with lower MVPA increased MVPA more (ß = -0.41, 95% CI -0.64, -0.13). Cluster analysis demonstrated four distinct patterns of connections between time-use activity behaviors and school (High Activity, Sleep Resilient, High Sedentary, and Dysregulated Sleep). CONCLUSIONS: Using a combination of person-centered and more traditional variable-centered approaches, we identified patterns of interrelated behaviors that differed on school, and nonschool days. Findings can inform targeted intervention strategies tailored to children's specific behavior patterns.

Impact of a year-round school calendar on children's BMI and fitness: Final outcomes from a natural experiment.

BACKGROUND: Structure may mitigate children's accelerated summer BMI gain and cardiorespiratory-fitness (CRF) loss. OBJECTIVES: Examine BMI and CRF change during school and summer for year-round and traditional calendar school children. METHODS: Three schools (N = 2279, 1 year-round) participated in this natural experiment. Children's BMI z-score (zBMI) and CRF (PACER laps) were measured from 2017 to 2019 each May/August. Mixed effects regression estimated monthly zBMI and CRF change during school/summer. Secondary analyses examined differences by weight status and race. Spline regression models estimated zBMI and CRF growth from kindergarten-sixth grade. RESULTS: Compared to traditional school, children attending a year-round school gained more zBMI (difference = 0.015; 95CI = 0.002, 0.028) during school, and less zBMI (difference = -0.029; 95CI = -0.041, -0.018), and more CRF (difference = 0.834; 95CI = 0.575, 1.093) monthly during summer. Differences by weight status and race were observed during summer and school. Growth models demonstrated that the magnitude of overall zBMI and CRF change from kindergarten-sixth grade was similar for year-round or traditional school children. CONCLUSIONS: Contrary to traditional school children zBMI increased during the traditional 9-month school calendar and zBMI decreased during the traditional summer vacation for year-round school children. Structured summer programming may mitigate accelerated summer BMI gain and CRF loss especially for overweight or obese, and/or Black children.

Hypothesis testing for coefficient alpha: an SEM approach.

We show how to test hypotheses for coefficient alpha in three different situations: (1) hypothesis tests of whether coefficient alpha equals a prespecified value, (2) hypothesis tests involving two statistically independent sample alphas as may arise when testing the equality of coefficient alpha across groups, and (3) hypothesis tests involving two statistically dependent sample alphas as may arise when testing the equality of alpha across time or when testing the equality of alpha for two test scores within the same sample. We illustrate how these hypotheses may be tested in a structural equation-modeling framework under the assumption of normally distributed responses and also under asymptotically distribution free assumptions. The formulas for the hypothesis tests and computer code are given for four different applied examples. Supplemental materials for this article may be downloaded from http://brm.psychonomic-journals.org/content/supplemental.

The Effect of Estimation Methods on SEM Fit Indices.

We examined the effect of estimation methods, maximum likelihood (ML), unweighted least squares (ULS), and diagonally weighted least squares (DWLS), on three population SEM (structural equation modeling) fit indices: the root mean square error of approximation (RMSEA), the comparative fit index (CFI), and the standardized root mean square residual (SRMR). We considered different types and levels of misspecification in factor analysis models: misspecified dimensionality, omitting cross-loadings, and ignoring residual correlations. Estimation methods had substantial impacts on the RMSEA and CFI so that different cutoff values need to be employed for different estimators. In contrast, SRMR is robust to the method used to estimate the model parameters. The same criterion can be applied at the population level when using the SRMR to evaluate model fit, regardless of the choice of estimation method.

Estimating causal effects in linear regression models with observational data: The instrumental variables regression model.

Instrumental variable methods are an underutilized tool to enhance causal inference in psychology. By way of incorporating predictors of the predictors (called "instruments" in the econometrics literature) into the model, instrumental variable regression (IVR) is able to draw causal inferences of a predictor on an outcome. We show that by regressing the outcome y on the predictors x and the predictors on the instruments, and modeling correlated disturbance terms between the predictor and outcome, causal inferences can be drawn on y on x if the IVR model cannot be rejected in a structural equation framework. We provide a tutorial on how to apply this model using ML estimation as implemented in structural equation modeling (SEM) software. We additionally provide code to identify instruments given a theoretical model, to select the best subset of instruments when more than necessary are available, and we guide researchers on how to apply this model using SEM. Finally, we demonstrate how the IVR model can be estimated using a number of estimators developed in econometrics (e.g., 2-stage least squares regression) and point out that the latter is simply a multistage SEM estimator of the IVR model. (PsycINFO Database Record (c) 2020 APA, all rights reserved).