Evaluating SEM Model Fit with Small Degrees of Freedom.

Research has revealed that the performance of root mean square error of approximation (RMSEA) in assessing structural equation models with small degrees of freedom (df) is suboptimal, often resulting in the rejection of correctly specified or closely fitted models. This study investigates the performance of standardized root mean square residual (SRMR) and comparative fit index (CFI) in small df models with various levels of factor loadings, sample sizes, and model misspecifications. We find that, in comparison with RMSEA, population SRMR and CFI are less susceptible to the effects of df. In small df models, the sample SRMR and CFI could provide more useful information to differentiate models with various levels of misfit. The confidence intervals and p-values of a close fit were generally accurate for all three fit indices. We recommend researchers use caution when interpreting RMSEA for models with small df and to rely more on SRMR and CFI.

Assessing the Accuracy of Errors of Measurement. Implications for Assessing Reliable Change in Clinical settings.

Item response theory (IRT) models are non-linear latent variable models for discrete measures, whereas factor analysis (FA) is a latent variable model for continuous measures. In FA, the standard error (SE) of individuals' scores is common for all individuals. In IRT, the SE depends on the individual's score, and the SE function is to be provided. The empirical standard deviation of the scores across discrete ranges should also be computed to inform the extent to which IRT SEs overestimate or underestimate the variability of the scores. Within the target range of scores the test was designed to measure, one should expect IRT SEs to be smaller and more precise than FA SEs, and therefore preferable to assess clinical change. Outside the target range, IRT SEs may be too large and more imprecise than FA SEs, and FA more precise to assess change. As a result, whether FA or IRT characterize reliable change more accurately in a sample will depend on the proportion of individuals within or outside the IRT target score range. An application is provided to illustrate these concepts.

Impact of a year-round school calendar on children's BMI and fitness: Final outcomes from a natural experiment.

BACKGROUND: Structure may mitigate children's accelerated summer BMI gain and cardiorespiratory-fitness (CRF) loss. OBJECTIVES: Examine BMI and CRF change during school and summer for year-round and traditional calendar school children. METHODS: Three schools (N = 2279, 1 year-round) participated in this natural experiment. Children's BMI z-score (zBMI) and CRF (PACER laps) were measured from 2017 to 2019 each May/August. Mixed effects regression estimated monthly zBMI and CRF change during school/summer. Secondary analyses examined differences by weight status and race. Spline regression models estimated zBMI and CRF growth from kindergarten-sixth grade. RESULTS: Compared to traditional school, children attending a year-round school gained more zBMI (difference = 0.015; 95CI = 0.002, 0.028) during school, and less zBMI (difference = -0.029; 95CI = -0.041, -0.018), and more CRF (difference = 0.834; 95CI = 0.575, 1.093) monthly during summer. Differences by weight status and race were observed during summer and school. Growth models demonstrated that the magnitude of overall zBMI and CRF change from kindergarten-sixth grade was similar for year-round or traditional school children. CONCLUSIONS: Contrary to traditional school children zBMI increased during the traditional 9-month school calendar and zBMI decreased during the traditional summer vacation for year-round school children. Structured summer programming may mitigate accelerated summer BMI gain and CRF loss especially for overweight or obese, and/or Black children.

Using the Standardized Root Mean Squared Residual (SRMR) to Assess Exact Fit in Structural Equation Models.

We examine the accuracy of p values obtained using the asymptotic mean and variance (MV) correction to the distribution of the sample standardized root mean squared residual (SRMR) proposed by Maydeu-Olivares to assess the exact fit of SEM models. In a simulation study, we found that under normality, the MV-corrected SRMR statistic provides reasonably accurate Type I errors even in small samples and for large models, clearly outperforming the current standard, that is, the likelihood ratio (LR) test. When data shows excess kurtosis, MV-corrected SRMR p values are only accurate in small models (p = 10), or in medium-sized models (p = 30) if no skewness is present and sample sizes are at least 500. Overall, when data are not normal, the MV-corrected LR test seems to outperform the MV-corrected SRMR. We elaborate on these findings by showing that the asymptotic approximation to the mean of the SRMR sampling distribution is quite accurate, while the asymptotic approximation to the standard deviation is not.

Dynamics of sleep, sedentary behavior, and moderate-to-vigorous physical activity on school versus nonschool days.

STUDY OBJECTIVES: Studies examining time-use activity behaviors (sleep, sedentary behavior, and physical activity) on school days compared with nonschool days have examined these behaviors independently, ignoring their interrelated nature, limiting our ability to optimize the health benefits of these behaviors. This study examines the associations of school-day (vs. nonschool day) with time-use activity behaviors. METHODS: Time series data (6,642 days) from Fitbits (Charge-2) were collected (n = 196, 53% female, 5-10 years). We used a variable-centered dynamic structural equation modeling approach to estimate day-to-day associations of time-use activity behaviors on school days for each child. We then used person-centered cluster analyses to group individuals based on these estimates. RESULTS: Within-participant analysis showed that on school days (vs. nonschool days), children (1) slept less (ß = -0.17, 95% CI = -0.21, -0.13), (2) were less sedentary (ß = -0.05, 95% CI = -0.09, -0.02), and (3) had comparable moderate-to-vigorous physical activity (MVPA; ß = -0.05, 95% CI = -0.11, 0.00). Between-participant analysis showed that, on school days, children with higher sleep carryover experienced greater decreases in sleep (ß = 0.44, 95% CI = 0.08, 0.71), children with higher body mass index z-score decreased sedentary behavior more (ß = -0.41, 95% CI = -0.64, -0.13), and children with lower MVPA increased MVPA more (ß = -0.41, 95% CI -0.64, -0.13). Cluster analysis demonstrated four distinct patterns of connections between time-use activity behaviors and school (High Activity, Sleep Resilient, High Sedentary, and Dysregulated Sleep). CONCLUSIONS: Using a combination of person-centered and more traditional variable-centered approaches, we identified patterns of interrelated behaviors that differed on school, and nonschool days. Findings can inform targeted intervention strategies tailored to children's specific behavior patterns.

The impact of summer vacation on children's obesogenic behaviors and body mass index: a natural experiment.

BACKGROUND: Children's BMI gain accelerates during summer. The Structured Days Hypothesis posits that the lack of the school day during summer vacation negatively impacts children's obesogenic behaviors (i.e., physical activity, screen time, diet, sleep). This natural experiment examined the impact of summer vacation on children's obesogenic behaviors and body mass index (BMI). METHODS: Elementary-aged children (n = 285, 5-12 years, 48.7% male, 57.4% African American) attending a year-round (n = 97) and two match-paired traditional schools (n = 188) in the United States participated in this study. Rather than taking a long break from school during the summer like traditional schools, year-round schools take shorter and more frequent breaks from school. This difference in school calendars allowed for obesogenic behaviors to be collected during three conditions: Condition 1) all children attend school, Condition 2) year-round children attend school while traditional children were on summer vacation, and Condition 3) summer vacation for all children. Changes in BMI z-score were collected for the corresponding school years and summers. Multi-level mixed effects regressions estimated obesogenic behaviors and monthly zBMI changes. It was hypothesized that children would experience unhealthy changes in obesogenic behaviors when entering summer vacation because the absence of the school day (i.e., Condition 1 vs. 2 for traditional school children and 2 vs. 3 for year-round school children). RESULTS: From Condition 1 to 2 traditional school children experienced greater unhealthy changes in daily minutes sedentary (∆ = 24.2, 95CI = 10.2, 38.2), screen time minutes (∆ = 33.7, 95CI = 17.2, 50.3), sleep midpoint time (∆ = 73:43, 95CI = 65:33, 81:53), and sleep efficiency percentage (-∆ = 0.7, 95CI = -1.1, - 0.3) when compared to year-round school children. Alternatively, from Condition 2 to 3 year-round school children experienced greater unhealthy changes in daily minutes sedentary (∆ = 54.5, 95CI = 38.0, 70.9), light physical activity minutes (∆ = - 42.2, 95CI = -56.2, - 28.3) MVPA minutes (∆ = - 11.4, 95CI = -3.7, - 19.1), screen time minutes (∆ = 46.5, 95CI = 30.0, 63.0), and sleep midpoint time (∆ = 95:54, 95CI = 85:26, 106:22) when compared to traditional school children. Monthly zBMI gain accelerated during summer for traditional (∆ = 0.033 95CI = 0.019, 0.047) but not year-round school children (∆ = 0.004, 95CI = -0.014, 0.023). CONCLUSIONS: This study suggests that the lack of the school day during summer vacation negatively impacts sedentary behaviors, sleep timing, and screen time. Changes in sedentary behaviors, screen time, and sleep midpoint may contribute to accelerated summer BMI gain. Providing structured programming during summer vacation may positively impact these behaviors, and in turn, mitigate accelerated summer BMI gain. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT03397940 . Registered January 12th 2018.

The Effect of Estimation Methods on SEM Fit Indices.

We examined the effect of estimation methods, maximum likelihood (ML), unweighted least squares (ULS), and diagonally weighted least squares (DWLS), on three population SEM (structural equation modeling) fit indices: the root mean square error of approximation (RMSEA), the comparative fit index (CFI), and the standardized root mean square residual (SRMR). We considered different types and levels of misspecification in factor analysis models: misspecified dimensionality, omitting cross-loadings, and ignoring residual correlations. Estimation methods had substantial impacts on the RMSEA and CFI so that different cutoff values need to be employed for different estimators. In contrast, SRMR is robust to the method used to estimate the model parameters. The same criterion can be applied at the population level when using the SRMR to evaluate model fit, regardless of the choice of estimation method.

The potential of a year-round school calendar for maintaining children's weight status and fitness: Preliminary outcomes from a natural experiment.

Purpose: To evaluate the potential of a year-round school calendar (180-day school year distributed across 12 months) as an intervention compared to a traditional school calendar (180-day school year distributed across 9 months) for mitigating children's weight gain and fitness loss via a natural experiment. Methods: Height, weight, and cardiorespiratory fitness (CRF) (i.e., Fitnessgram Progressive Aerobic Cardiovascular Endurance Run) were measured in children (5-12 years old) in 3 schools (2 traditional, 1 year-round, n = 990 students, age = 8.6 ± 2.4 years, 53.1% male, 68.9% African American) from 1 school district. Structure (represented by the presence of a school day) was the independent variable. Changes in body mass index (BMI), age- and sex-specific BMI z-scores (zBMI), BMI percentile, percent of overweight or obese children, and CRF (Progressive Aerobic Cardiovascular Endurance Run laps completed) were assessed for summer 2017 (May-August 2017), school year 2017/2018 (August 2017-May 2018), and summer 2018 (May-August 2018). Primary analyses examined the overall change in weight and CRF from summer 2017 until summer 2018 via multilevel mixed effects regression, with group (traditional vs. year-round calendar), time, and a group-by-time interaction as the independent variables. Secondary regression analyses estimated differences in change within and between groups during each time period, separately. Results: Year-round students gained less BMI (difference in ∆ = -0.44, 95% confidence interval (CI): -0.67 to -0.03) and less CRF (difference in ∆ = -1.92, 95%CI: -3.56 to -0.28) than students attending a traditional school overall. Compared with traditional students, during both summers, year-round students gained less BMI (summer 2017 difference in ∆ = -0.15, 95%CI: -0.21 to -0.08; summer 2018 difference in ∆ = -0.16, 95%CI: -0.24 to -0.07) and zBMI (summer 2017 difference in ∆ = -0.032, 95%CI: -0.050 to -0.010; summer 2018 difference in ∆ = -0.033, 95%CI: -0.056 to -0.009), and increased CRF (summer 2017 difference in ∆ = 0.40, 95%CI: 0.02-0.85; summer 2018 difference in ∆ = 0.23, 95%CI: -0.25 to 0.74). However, the opposite was observed for the school year, with traditional students gaining less BMI and zBMI and increasing CRF compared with year-round students (difference in BMI ∆ = 0.05, 95%CI: 0.03-0.07; difference in zBMI ∆ = 0.012, 95%CI: 0.005-0.019; difference in Progressive Aerobic Cardiovascular Endurance Run laps ∆ = -0.43, 95%CI: -0.58 to -0.28). Conclusion: The year-round school calendar had a small beneficial impact on children's weight status but not CRF. It is unclear if this benefit to children's weight would be maintained because gains made in the summer were largely erased during the school year. Trajectories of weight and CRF gain/loss were consistent with the structured days hypothesis.

Fitting Ordinal Factor Analysis Models With Missing Data: A Comparison Between Pairwise Deletion and Multiple Imputation.

This study compares two missing data procedures in the context of ordinal factor analysis models: pairwise deletion (PD; the default setting in Mplus) and multiple imputation (MI). We examine which procedure demonstrates parameter estimates and model fit indices closer to those of complete data. The performance of PD and MI are compared under a wide range of conditions, including number of response categories, sample size, percent of missingness, and degree of model misfit. Results indicate that both PD and MI yield parameter estimates similar to those from analysis of complete data under conditions where the data are missing completely at random (MCAR). When the data are missing at random (MAR), PD parameter estimates are shown to be severely biased across parameter combinations in the study. When the percentage of missingness is less than 50%, MI yields parameter estimates that are similar to results from complete data. However, the fit indices (i.e., χ2, RMSEA, and WRMR) yield estimates that suggested a worse fit than results observed in complete data. We recommend that applied researchers use MI when fitting ordinal factor models with missing data. We further recommend interpreting model fit based on the TLI and CFI incremental fit indices.

Estimating causal effects in linear regression models with observational data: The instrumental variables regression model.

Instrumental variable methods are an underutilized tool to enhance causal inference in psychology. By way of incorporating predictors of the predictors (called "instruments" in the econometrics literature) into the model, instrumental variable regression (IVR) is able to draw causal inferences of a predictor on an outcome. We show that by regressing the outcome y on the predictors x and the predictors on the instruments, and modeling correlated disturbance terms between the predictor and outcome, causal inferences can be drawn on y on x if the IVR model cannot be rejected in a structural equation framework. We provide a tutorial on how to apply this model using ML estimation as implemented in structural equation modeling (SEM) software. We additionally provide code to identify instruments given a theoretical model, to select the best subset of instruments when more than necessary are available, and we guide researchers on how to apply this model using SEM. Finally, we demonstrate how the IVR model can be estimated using a number of estimators developed in econometrics (e.g., 2-stage least squares regression) and point out that the latter is simply a multistage SEM estimator of the IVR model. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Restricted Recalibration of Item Response Theory Models.

In item response theory (IRT), it is often necessary to perform restricted recalibration (RR) of the model: A set of (focal) parameters is estimated holding a set of (nuisance) parameters fixed. Typical applications of RR include expanding an existing item bank, linking multiple test forms, and associating constructs measured by separately calibrated tests. In the current work, we provide full statistical theory for RR of IRT models under the framework of pseudo-maximum likelihood estimation. We describe the standard error calculation for the focal parameters, the assessment of overall goodness-of-fit (GOF) of the model, and the identification of misfitting items. We report a simulation study to evaluate the performance of these methods in the scenario of adding a new item to an existing test. Parameter recovery for the focal parameters as well as Type I error and power of the proposed tests are examined. An empirical example is also included, in which we validate the pediatric fatigue short-form scale in the Patient-Reported Outcome Measurement Information System (PROMIS), compute global and local GOF statistics, and update parameters for the misfitting items.

The Meaning of Aggression Varies Across Culture: Testing the Measurement Invariance of the Refined Aggression Questionnaire in Samples From Spain, the United States, and Hong Kong.

Cultural differences in aggression are still poorly understood. The purpose of this article is to assess whether a tool for measuring aggression has the same meaning across cultures. Analyzing samples from Spain (n = 262), the United States (n = 344), and Hong Kong (n = 645), we used confirmatory factor analysis to investigate measurement invariance of the refined version of the Aggression Questionnaire (Bryant & Smith, 2001 ). The measurement of aggression was more equivalent between the Chinese and Spanish versions than between these two and the U.S. version. Aggression does not show invariance at the cultural level. Cultural variables such as affective autonomy or individualism could influence the meaning of aggression. Aggressive behavior models can be improved by incorporating cultural variables.

Understanding the Model Size Effect on SEM Fit Indices.

This study investigated the effect the number of observed variables (p) has on three structural equation modeling indices: the comparative fit index (CFI), the Tucker-Lewis index (TLI), and the root mean square error of approximation (RMSEA). The behaviors of the population fit indices and their sample estimates were compared under various conditions created by manipulating the number of observed variables, the types of model misspecification, the sample size, and the magnitude of factor loadings. The results showed that the effect of p on the population CFI and TLI depended on the type of specification error, whereas a higher p was associated with lower values of the population RMSEA regardless of the type of model misspecification. In finite samples, all three fit indices tended to yield estimates that suggested a worse fit than their population counterparts, which was more pronounced with a smaller sample size, higher p, and lower factor loading.

Evaluating Factorial Invariance: An Interval Estimation Approach Using Bayesian Structural Equation Modeling.

In this study, we introduce an interval estimation approach based on Bayesian structural equation modeling to evaluate factorial invariance. For each tested parameter, the size of noninvariance with an uncertainty interval (i.e. highest density interval [HDI]) is assessed via Bayesian parameter estimation. By comparing the most credible values (i.e. 95% HDI) with a region of practical equivalence (ROPE), the Bayesian approach allows researchers to (1) support the null hypothesis of practical invariance, and (2) examine the practical importance of the noninvariant parameter. Compared to the traditional likelihood ratio test, simulation results suggested that the proposed Bayesian approach could offer additional insight into evaluating factorial invariance, thus, leading to more informative conclusions. We provide an empirical example to demonstrate the procedures necessary to implement the proposed method in applied research. The importance of and influences on the choice of an appropriate ROPE are discussed.

Changes in children's sleep and physical activity during a 1-week versus a 3-week break from school: a natural experiment.

Study Objectives: To examine changes in elementary aged children's sleep and physical activity during a 1-week and a 3-week school break. Methods: Sleep and physical activity of elementary children (n = 154, age = 5-9 years, 44.8% female, 65.5% African American) were collected over 7 weeks that included a 1-week break in two schools and a 3-week break in a single school. Mixed regression models estimated sleep and physical activity changes within and between groups (i.e. 1-week vs. 3-weeks) during school and school break weeks. Results: Compared to school weeks, bed times shifted 72.7 (95% CI = 57.5, 87.9) and 75.4 (95% CI = 58.1, 92.7) minutes later on weekdays during the 1-week and 3-week break, respectively. Wake times shifted 111.6 (95% CI = 94.3, 128.9) and 99.8 (95% CI = 80.5, 119.1) minutes later on weekdays during 1-week and 3-week breaks. On weekdays during the 3-week break, children engaged in 33.1 (95% CI = 14.1, 52.2) more sedentary minutes and -12.2 (-20.2, -4.2) fewer moderate-to-vigorous physical activity minutes/day. No statistically significant changes in children's sedentary, light, or moderate-to-vigorous physical activity (MVPA) minutes were observed on weekdays during the 1-week break. Between-group differences in the change in time sedentary (32.1-95% CI = 5.8, 58.4), and moderate-to-vigorous (-13.0-95% CI = -23.9, -2.0) physical activity were observed. Conclusions: Children's sleep shifted later on both 1-week and 3-week breaks. Children's activity changed minimally on weekdays during a 1-week school break and more during a 3-week school break. Displaced sleep and reductions in activity are intervention targets for mitigating unhealthy weight gain during extended breaks from school.

The Relationship Between the Standardized Root Mean Square Residual and Model Misspecification in Factor Analysis Models.

We argue that the definition of close fitting models should embody the notion of substantially ignorable misspecifications (SIM). A SIM model is a misspecified model that might be selected, based on parsimony, over the true model should knowledge of the true model be available. Because in applications the true model (i.e., the data generating mechanism) is unknown, we investigate the relationship between the population standardized root mean square residual (SRMR) values and various model misspecifications in factor analysis models to better understand the magnitudes of the SRMR. Summary effect sizes of misfit such as the SRMR are necessarily insensitive to some non-ignorable localized misspecifications (i.e., the presence of a few large residual correlations in large models). Localized misspecifications may be identified by examining the largest standardized residual covariance. Based on the findings, our population reference values for close fit are based on a two-index strategy: (1) largest absolute value of standardized residual covariance ≤0.10, and (2) SRMR ≤0.05× R¯2 the average R2 of the manifest variables; for acceptable fit our values are 0.15 and 0.10× R¯2 , respectively.

Assessing the Size of Model Misfit in Structural Equation Models.

When a statistically significant mean difference is found, the magnitude of the difference is judged qualitatively using an effect size such as Cohen's d. In contrast, in a structural equation model (SEM), the result of the statistical test of model fit is often disregarded if significant, and inferences are drawn using "close" models retained based on point estimates of sample statistics (goodness-of-fit indices). However, when a SEM cannot be retained using a test of exact fit, all substantive inferences drawn from it are suspect. It is therefore important to determine the size of the model misfit. Standardized residual covariances and residual correlations provide standardized effect sizes of the misfit of SEM models. They can be summarized using the Standardized Root Mean Squared Residual (SRMSR) and the Correlation Root Mean Squared Residual (CRMSR) which can be used as overall effect sizes of the misfit. Statistical theory is provided that allows the construction of confidence intervals and tests of close fit based on the SRMSR and CRMSR. It is hoped that the use of standardized effect sizes of misfit will help reconcile current practices in SEM and elsewhere in statistics.

Improving Psychological Measurement: Does It Make a Difference? A Comment on Nesselroade and Molenaar (2016).

Nesselroade and Molenaar advocate the use of an idiographic filter approach. This is a fixed-effects approach, which may limit the number of individuals that can be simultaneously modeled, and it is not clear how to model the presence of subpopulations. Most important, Nesselroade and Molenaar's proposal appears to be best suited for modeling long time series on a few variables for a few individuals. Long time series are not common in psychological applications. Can it be applied to the usual longitudinal data we face? These are characterized by short time series (four to five points in time), hundreds of individuals, and dozens of variables. If so, what do we gain? Applied settings most often involve between-individual decisions. I conjecture that their approach will not outperform common, simpler, methods. However, when intraindividual decisions are involved, their approach may have an edge.