Evaluating Covariate Effects on ESM Measurement Model Changes with Latent Markov Factor Analysis: A Three-Step Approach.

Invariance of the measurement model (MM) between subjects and within subjects over time is a prerequisite for drawing valid inferences when studying dynamics of psychological factors in intensive longitudinal data. To conveniently evaluate this invariance, latent Markov factor analysis (LMFA) was proposed. LMFA combines a latent Markov model with mixture factor analysis: The Markov model captures changes in MMs over time by clustering subjects' observations into a few states and state-specific factor analyses reveal what the MMs look like. However, to estimate the model, Vogelsmeier, Vermunt, van Roekel, and De Roover (2019) introduced a one-step (full information maximum likelihood; FIML) approach that is counterintuitive for applied researchers and entails cumbersome model selection procedures in the presence of many covariates. In this paper, we simplify the complex LMFA estimation and facilitate the exploration of covariate effects on state memberships by splitting the estimation in three intuitive steps: (1) obtain states with mixture factor analysis while treating repeated measures as independent, (2) assign observations to the states, and (3) use these states in a discrete- or continuous-time latent Markov model taking into account classification errors. A real data example demonstrates the empirical value.

Simultaneous clustering and variable selection: A novel algorithm and model selection procedure.

The growing availability of high-dimensional data sets offers behavioral scientists an unprecedented opportunity to integrate the information hidden in the novel types of data (e.g., genetic data, social media data, and GPS tracks, etc.,) and thereby obtain a more detailed and comprehensive view towards their research questions. In the context of clustering, analyzing the large volume of variables could potentially result in an accurate estimation or a novel discovery of underlying subgroups. However, a unique challenge is that the high-dimensional data sets likely involve a significant amount of irrelevant variables. These irrelevant variables do not contribute to the separation of clusters and they may mask cluster partitions. The current paper addresses this challenge by introducing a new clustering algorithm, called Cardinality K-means or CKM, and by proposing a novel model selection strategy. CKM is able to perform simultaneous clustering and variable selection with high stability. In two simulation studies and an empirical demonstration with genetic data, CKM consistently outperformed competing methods in terms of recovering cluster partitions and identifying signaling variables. Meanwhile, our novel model selection strategy determines the number of clusters based on a subset of variables that are most likely to be signaling variables. Through a simulation study, this strategy was found to result in a more accurate estimation of the number of clusters compared to the conventional strategy that utilizes the full set of variables. Our proposed CKM algorithm, together with the novel model selection strategy, has been implemented in a freely accessible R package.

How to explore within-person and between-person measurement model differences in intensive longitudinal data with the R package lmfa.

Intensive longitudinal data (ILD) have become popular for studying within-person dynamics in psychological constructs (or between-person differences therein). Before investigating the dynamics, it is crucial to examine whether the measurement model (MM) is the same across subjects and time and, thus, whether the measured constructs have the same meaning. If the MM differs (e.g., because of changes in item interpretation or response styles), observations cannot be validly compared. Exploring differences in the MM for ILD can be done with latent Markov factor analysis (LMFA), which classifies observations based on the underlying MM (for many subjects and time points simultaneously) and thus shows which observations are comparable. However, the complexity of the method or the fact that no open-source software for LMFA existed until now may have hindered researchers from applying the method in practice. In this article, we provide a step-by-step tutorial for the new user-friendly software package lmfa, which allows researchers to easily perform the analysis LMFA in the freely available software R to investigate MM differences in their own ILD.

Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives.

This review summarizes the current state of the art of statistical and (survey) methodological research on measurement (non)invariance, which is considered a core challenge for the comparative social sciences. After outlining the historical roots, conceptual details, and standard procedures for measurement invariance testing, the paper focuses in particular on the statistical developments that have been achieved in the last 10 years. These include Bayesian approximate measurement invariance, the alignment method, measurement invariance testing within the multilevel modeling framework, mixture multigroup factor analysis, the measurement invariance explorer, and the response shift-true change decomposition approach. Furthermore, the contribution of survey methodological research to the construction of invariant measurement instruments is explicitly addressed and highlighted, including the issues of design decisions, pretesting, scale adoption, and translation. The paper ends with an outlook on future research perspectives.

Scale length does matter: Recommendations for measurement invariance testing with categorical factor analysis and item response theory approaches.

In social sciences, the study of group differences concerning latent constructs is ubiquitous. These constructs are generally measured by means of scales composed of ordinal items. In order to compare these constructs across groups, one crucial requirement is that they are measured equivalently or, in technical jargon, that measurement invariance (MI) holds across the groups. This study compared the performance of scale- and item-level approaches based on multiple group categorical confirmatory factor analysis (MG-CCFA) and multiple group item response theory (MG-IRT) in testing MI with ordinal data. In general, the results of the simulation studies showed that MG-CCFA-based approaches outperformed MG-IRT-based approaches when testing MI at the scale level, whereas, at the item level, the best performing approach depends on the tested parameter (i.e., loadings or thresholds). That is, when testing loadings equivalence, the likelihood ratio test provided the best trade-off between true-positive rate and false-positive rate, whereas, when testing thresholds equivalence, the χ2 test outperformed the other testing strategies. In addition, the performance of MG-CCFA's fit measures, such as RMSEA and CFI, seemed to depend largely on the length of the scale, especially when MI was tested at the item level. General caution is recommended when using these measures, especially when MI is tested for each item individually.

Detecting which variables alter component interpretation across multiple groups: A resampling-based method.

In psychology, many studies measure the same variables in different groups. In the case of a large number of variables when a strong a priori idea about the underlying latent construct is lacking, researchers often start by reducing the variables to a few principal components in an exploratory way. Herewith, one often wants to evaluate whether the components represent the same construct in the different groups. To this end, it makes sense to remove outlying variables that have significantly different loadings on the extracted components across the groups, hampering equivalent interpretations of the components. Moreover, identifying such outlying variables is important when testing theories about which variables behave similarly or differently across groups. In this article, we first scrutinize the lower bound congruence method (LBCM; De Roover, Timmerman, & Ceulemans in Behavior Research Methods, 49, 216-229, 2017), which was recently proposed for solving the outlying-variable detection problem. LBCM investigates how Tucker's congruence between the loadings of the obtained cluster-loading matrices improves when specific variables are discarded. We show that LBCM has the tendency to output outlying variables that either are false positives or concern very small, and thus practically insignificant, loading differences. To address this issue, we present a new heuristic: the lower and resampled upper bound congruence method (LRUBCM). This method uses a resampling technique to obtain a sampling distribution for the congruence coefficient, under the hypothesis that no outlying variable is present. In a simulation study, we show that LRUBCM outperforms LBCM. Finally, we illustrate the use of the method by means of empirical data.

How to detect which variables are causing differences in component structure among different groups.

When comparing the component structures of a multitude of variables across different groups, the conclusion often is that the component structures are very similar in general and differ in a few variables only. Detecting such "outlying variables" is substantively interesting. Conversely, it can help to determine what is common across the groups. This article proposes and evaluates two formal detection heuristics to determine which variables are outlying, in a systematic and objective way. The heuristics are based on clusterwise simultaneous component analysis, which was recently presented as a useful tool for capturing the similarities and differences in component structures across groups. The heuristics are evaluated in a simulation study and illustrated using cross-cultural data on values.

Subspace K-means clustering.

To achieve an insightful clustering of multivariate data, we propose subspace K-means. Its central idea is to model the centroids and cluster residuals in reduced spaces, which allows for dealing with a wide range of cluster types and yields rich interpretations of the clusters. We review the existing related clustering methods, including deterministic, stochastic, and unsupervised learning approaches. To evaluate subspace K-means, we performed a comparative simulation study, in which we manipulated the overlap of subspaces, the between-cluster variance, and the error variance. The study shows that the subspace K-means algorithm is sensitive to local minima but that the problem can be reasonably dealt with by using partitions of various cluster procedures as a starting point for the algorithm. Subspace K-means performs very well in recovering the true clustering across all conditions considered and appears to be superior to its competitor methods: K-means, reduced K-means, factorial K-means, mixtures of factor analyzers (MFA), and MCLUST. The best competitor method, MFA, showed a performance similar to that of subspace K-means in easy conditions but deteriorated in more difficult ones. Using data from a study on parental behavior, we show that subspace K-means analysis provides a rich insight into the cluster characteristics, in terms of both the relative positions of the clusters (via the centroids) and the shape of the clusters (via the within-cluster residuals).

Awareness Is Bliss: How Acquiescence Affects Exploratory Factor Analysis.

Assessing the measurement model (MM) of self-report scales is crucial to obtain valid measurements of individuals' latent psychological constructs. This entails evaluating the number of measured constructs and determining which construct is measured by which item. Exploratory factor analysis (EFA) is the most-used method to evaluate these psychometric properties, where the number of measured constructs (i.e., factors) is assessed, and, afterward, rotational freedom is resolved to interpret these factors. This study assessed the effects of an acquiescence response style (ARS) on EFA for unidimensional and multidimensional (un)balanced scales. Specifically, we evaluated (a) whether ARS is captured as an additional factor, (b) the effect of different rotation approaches on the content and ARS factors recovery, and (c) the effect of extracting the additional ARS factor on the recovery of factor loadings. ARS was often captured as an additional factor in balanced scales when it was strong. For these scales, ignoring extracting this additional ARS factor, or rotating to simple structure when extracting it, harmed the recovery of the original MM by introducing bias in loadings and cross-loadings. These issues were avoided by using informed rotation approaches (i.e., target rotation), where (part of) the rotation target is specified according to a priori expectations on the MM. Not extracting the additional ARS factor did not affect the loading recovery in unbalanced scales. Researchers should consider the potential presence of ARS when assessing the psychometric properties of balanced scales and use informed rotation approaches when suspecting that an additional factor is an ARS factor.

The dire disregard of measurement invariance testing in psychological science.

Self-report scales are widely used in psychology to compare means in latent constructs across groups, experimental conditions, or time points. However, for these comparisons to be meaningful and unbiased, the scales must demonstrate measurement invariance (MI) across compared time points or (experimental) groups. MI testing determines whether the latent constructs are measured equivalently across groups or time, which is essential for meaningful comparisons. We conducted a systematic review of 426 psychology articles with openly available data, to (a) examine common practices in conducting and reporting of MI testing, (b) assess whether we could reproduce the reported MI results, and (c) conduct MI tests for the comparisons that enabled sufficiently powerful MI testing. We identified 96 articles that contained a total of 929 comparisons. Results showed that only 4% of the 929 comparisons underwent MI testing, and the tests were generally poorly reported. None of the reported MI tests were reproducible, and only 26% of the 174 newly performed MI tests reached sufficient (scalar) invariance, with MI failing completely in 58% of tests. Exploratory analyses suggested that in nearly half of the comparisons where configural invariance was rejected, the number of factors differed between groups. These results indicate that MI tests are rarely conducted and poorly reported in psychological studies. We observed frequent violations of MI, suggesting that reported differences between (experimental) groups may not be solely attributed to group differences in the latent constructs. We offer recommendations aimed at improving reporting and computational reproducibility practices in psychology. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

On the relationship between valence and arousal in samples across the globe.

Affect is involved in many psychological phenomena, but a descriptive structure, long sought, has been elusive. Valence and arousal are fundamental, and a key question-the focus of the present study-is the relationship between them. Valence is sometimes thought to be independent of arousal, but, in some studies (representing too few societies in the world) arousal was found to vary with valence. One common finding is that arousal is lowest at neutral valence and increases with both positive and negative valence: a symmetric V-shaped relationship. In the study reported here of self-reported affect during a remembered moment (N = 8,590), we tested the valence-arousal relationship in 33 societies with 25 different languages. The two most common hypotheses in the literature-independence and a symmetric V-shaped relationship-were not supported. With data of all samples pooled, arousal increased with positive but not negative valence. Valence accounted for between 5% (Finland) and 43% (China Beijing) of the variance in arousal. Although there is evidence for a structural relationship between the two, there is also a large amount of variability in this relation. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

How to perform multiblock component analysis in practice.

To explore structural differences and similarities in multivariate multiblock data (e.g., a number of variables have been measured for different groups of subjects, where the data for each group constitute a different data block), researchers have a variety of multiblock component analysis and factor analysis strategies at their disposal. In this article, we focus on three types of multiblock component methods--namely, principal component analysis on each data block separately, simultaneous component analysis, and the recently proposed clusterwise simultaneous component analysis, which is a generic and flexible approach that has no counterpart in the factor analysis tradition. We describe the steps to take when applying those methods in practice. Whereas plenty of software is available for fitting factor analysis solutions, up to now no easy-to-use software has existed for fitting these multiblock component analysis methods. Therefore, this article presents the MultiBlock Component Analysis program, which also includes procedures for missing data imputation and model selection.

Mixture multigroup factor analysis for unraveling factor loading noninvariance across many groups.

Psychological research often builds on between-group comparisons of (measurements of) latent variables; for instance, to evaluate cross-cultural differences in neuroticism or mindfulness. A critical assumption in such comparative research is that the same latent variable(s) are measured in exactly the same way across all groups (i.e., measurement invariance). Otherwise, one would be comparing apples and oranges. Nowadays, measurement invariance is often tested across a large number of groups by means of multigroup factor analysis. When the assumption is untenable, one may compare group-specific measurement models to pinpoint sources of noninvariance, but the number of pairwise comparisons exponentially increases with the number of groups. This makes it hard to unravel invariances from noninvariances and for which groups they apply, and it elevates the chances of falsely detecting noninvariance. An intuitive solution is clustering the groups into a few clusters based on the measurement model parameters. Therefore, we present mixture multigroup factor analysis (MMG-FA) which clusters the groups according to a specific level of measurement invariance. Specifically, in this article, clusters of groups with metric invariance (i.e., equal factor loadings) are obtained by making the loadings cluster-specific, whereas other parameters (i.e., intercepts, factor (co)variances, residual variances) are still allowed to differ between groups within a cluster. MMG-FA was found to perform well in an extensive simulation study, but a larger sample size within groups is required for recovering more subtle loading differences. Its empirical value is illustrated for data on the social value of emotions and data on emotional acculturation. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Latent Markov Latent Trait Analysis for Exploring Measurement Model Changes in Intensive Longitudinal Data.

Drawing inferences about dynamics of psychological constructs from intensive longitudinal data requires the measurement model (MM)-indicating how items relate to constructs-to be invariant across subjects and time-points. When assessing subjects in their daily life, however, there may be multiple MMs, for instance, because subjects differ in their item interpretation or because the response style of (some) subjects changes over time. The recently proposed "latent Markov factor analysis" (LMFA) evaluates (violations of) measurement invariance by classifying observations into latent "states" according to the MM underlying these observations such that MMs differ between states but are invariant within one state. However, LMFA is limited to normally distributed continuous data and estimates may be inaccurate when applying the method to ordinal data (e.g., from Likert items) with skewed responses or few response categories. To enable researchers and health professionals with ordinal data to evaluate measurement invariance, we present "latent Markov latent trait analysis" (LMLTA), which builds upon LMFA but treats responses as ordinal. Our application shows differences in MMs of adolescents' affective well-being in different social contexts, highlighting the importance of studying measurement invariance for drawing accurate inferences for psychological science and practice and for further understanding dynamics of psychological constructs.

Family demographic profiles and their relationship with the quality of executive functioning subcomponents in kindergarten.

Executive functioning (EF), needed for goal-oriented behaviour, thoughts, and emotions, is important for various life domains. This study examined the relationship between family demographics and EF subcomponents. A kindergarten sample was tested on subcomponents of working memory, inhibition, and cognitive flexibility. Parents provided information on demographic variables. For 78 children both EF and demographic data were available. First, demographic profiles were identified within the sample. Two profiles were found: A low-risk profile of mainly two-biological-parent, high-income families with a highly educated mother who did not smoke during pregnancy and a high-risk profile of low-income families with a young, low-educated mother who more often smoked during pregnancy. Second, children with different demographic profiles were compared on EF subcomponents. Results indicate differential relations between family demographics and EF subcomponents: Whereas for most EF subcomponents no association with family demographics was found, high-risk children performed better on response shifting and tended to perform worse on verbal memory than low-risk children. Parenting stress decreased performance only for high-risk children. Although this study found limited impact of family demographics for EF, further longitudinal research can provide nuanced insights about which factors influence specific EF subcomponents during which developmental periods and guide targeted prevention of EF difficulties.

Older adults' affective experiences across 100 days are less variable and less complex than younger adults'.

Older adults are often described as being more emotionally competent than younger adults, and higher levels of affect complexity are seen as an indicator of this competence. We argue, however, that once age differences in affect variability are taken into account, older adults' everyday affective experiences will be characterized by lower affect complexity when compared with younger adults'. In addition, reduced affect complexity seems more likely from a theoretical point of view. We tested this hypothesis with a study in which younger and older adults reported their momentary affect on 100 days. Affect complexity was examined using clusterwise simultaneous component analysis based on covariance matrices to take into account differences in affect variability. We found that in the majority of older adults (55%), structures of affect were comparatively simpler than those of younger adults because they were reduced to a positive affect component. Most remaining older adults (35%) were characterized by differentiated rather than undifferentiated affective responding, as were a considerable number of younger adults (43%). When affect variability was made comparable across age groups, affect complexity also became comparable across age groups. It is interesting that individuals with the least complex structures had the highest levels of well-being. We conclude that affective experiences are not only less variable in the majority of older adults, but also less complex. Implications for understanding emotions across the life span are discussed.

Is valuing positive emotion associated with life satisfaction?

The experience of positive emotion is closely linked to subjective well-being. For this reason, campaigns aimed at promoting the value of positive emotion have become widespread. What is rarely considered are the cultural implications of this focus on happiness. Promoting positive emotions as important for "the good life" not only has implications for how individuals value these emotional states, but for how they believe others around them value these emotions also. Drawing on data from over 9,000 college students across 47 countries we examined whether individuals' life satisfaction is associated with living in contexts in which positive emotions are socially valued. The findings show that people report more life satisfaction in countries where positive emotions are highly valued and this is linked to an increased frequency of positive emotional experiences in these contexts. They also reveal, however, that increased life satisfaction in countries that place a premium on positive emotion is less evident for people who tend to experience less valued emotional states: people who experience many negative emotions, do not flourish to the same extent in these contexts. The findings demonstrate how the cultural value placed on certain emotion states may shape the relationship between emotional experiences and subjective well-being.

Switching principal component analysis for modeling means and covariance changes over time.

Many psychological theories predict that cognitions, affect, action tendencies, and other variables change across time in mean level as well as in covariance structure. Often such changes are rather abrupt, because they are caused by sudden events. To capture such changes, one may repeatedly measure the variables under study for a single individual and examine whether the resulting multivariate time series contains a number of phases with different means and covariance structures. The latter task is challenging, however. First, in many cases, it is unknown how many phases there are and when new phases start. Second, often a rather large number of variables is involved, complicating the interpretation of the covariance pattern within each phase. To take up this challenge, we present switching principal component analysis (PCA). Switching PCA detects phases of consecutive observations or time points (in single subject data) with similar means and/or covariation structures, and performs a PCA per phase to yield insight into its covariance structure. An algorithm for fitting switching PCA solutions as well as a model selection procedure are presented and evaluated in a simulation study. Finally, we analyze empirical data on cardiorespiratory recordings.

What's hampering measurement invariance: detecting non-invariant items using clusterwise simultaneous component analysis.

The issue of measurement invariance is ubiquitous in the behavioral sciences nowadays as more and more studies yield multivariate multigroup data. When measurement invariance cannot be established across groups, this is often due to different loadings on only a few items. Within the multigroup CFA framework, methods have been proposed to trace such non-invariant items, but these methods have some disadvantages in that they require researchers to run a multitude of analyses and in that they imply assumptions that are often questionable. In this paper, we propose an alternative strategy which builds on clusterwise simultaneous component analysis (SCA). Clusterwise SCA, being an exploratory technique, assigns the groups under study to a few clusters based on differences and similarities in the component structure of the items, and thus based on the covariance matrices. Non-invariant items can then be traced by comparing the cluster-specific component loadings via congruence coefficients, which is far more parsimonious than comparing the component structure of all separate groups. In this paper we present a heuristic for this procedure. Afterwards, one can return to the multigroup CFA framework and check whether removing the non-invariant items or removing some of the equality restrictions for these items, yields satisfactory invariance test results. An empirical application concerning cross-cultural emotion data is used to demonstrate that this novel approach is useful and can co-exist with the traditional CFA approaches.

A clusterwise simultaneous component method for capturing within-cluster differences in component variances and correlations.

This paper presents a clusterwise simultaneous component analysis for tracing structural differences and similarities between data of different groups of subjects. This model partitions the groups into a number of clusters according to the covariance structure of the data of each group and performs a simultaneous component analysis with invariant pattern restrictions (SCA-P) for each cluster. These restrictions imply that the model allows for between-group differences in the variances and the correlations of the cluster-specific components. As such, clusterwise SCA-P is more flexible than the earlier proposed clusterwise SCA-ECP model, which imposed equal average cross-products constraints on the component scores of the groups that belong to the same cluster. Using clusterwise SCA-P, a finer-grained, yet parsimonious picture of the group differences and similarities can be obtained. An algorithm for fitting clusterwise SCA-P solutions is presented and its performance is evaluated by means of a simulation study. The value of the model for empirical research is illustrated with data from psychiatric diagnosis research.