Accounting for careless and insufficient effort responding in large-scale survey data-development, evaluation, and application of a screen-time-based weighting procedure.

Careless and insufficient effort responding (C/IER) poses a major threat to the quality of large-scale survey data. Traditional indicator-based procedures for its detection are limited in that they are only sensitive to specific types of C/IER behavior, such as straight lining or rapid responding, rely on arbitrary threshold settings, and do not allow taking the uncertainty of C/IER classification into account. Overcoming these limitations, we develop a two-step screen-time-based weighting procedure for computer-administered surveys. The procedure allows considering the uncertainty in C/IER identification, is agnostic towards the specific types of C/IE response patterns, and can feasibly be integrated with common analysis workflows for large-scale survey data. In Step 1, we draw on mixture modeling to identify subcomponents of log screen time distributions presumably stemming from C/IER. In Step 2, the analysis model of choice is applied to item response data, with respondents' posterior class probabilities being employed to downweigh response patterns according to their probability of stemming from C/IER. We illustrate the approach on a sample of more than 400,000 respondents being administered 48 scales of the PISA 2018 background questionnaire. We gather supporting validity evidence by investigating relationships between C/IER proportions and screen characteristics that entail higher cognitive burden, such as screen position and text length, relating identified C/IER proportions to other indicators of C/IER as well as by investigating rank-order consistency in C/IER behavior across screens. Finally, in a re-analysis of the PISA 2018 background questionnaire data, we investigate the impact of the C/IER adjustments on country-level comparisons.

A Bayesian Approach to Estimating Reciprocal Effects with the Bivariate STARTS Model.

The bivariate Stable Trait, AutoRegressive Trait, and State (STARTS) model provides a general approach for estimating reciprocal effects between constructs over time. However, previous research has shown that this model is difficult to estimate using the maximum likelihood (ML) method (e.g., nonconvergence). In this article, we introduce a Bayesian approach for estimating the bivariate STARTS model and implement it in the software Stan. We discuss issues of model parameterization and show how appropriate prior distributions for model parameters can be selected. Specifically, we propose the four-parameter beta distribution as a flexible prior distribution for the autoregressive and cross-lagged effects. Using a simulation study, we show that the proposed Bayesian approach provides more accurate estimates than ML estimation in challenging data constellations. An example is presented to illustrate how the Bayesian approach can be used to stabilize the parameter estimates of the bivariate STARTS model.

A machine learning-based procedure for leveraging clickstream data to investigate early predictability of failure on interactive tasks.

Early detection of risk of failure on interactive tasks comes with great potential for better understanding how examinees differ in their initial behavior as well as for adaptively tailoring interactive tasks to examinees' competence levels. Drawing on procedures originating in shopper intent prediction on e-commerce platforms, we introduce and showcase a machine learning-based procedure that leverages early-window clickstream data for systematically investigating early predictability of behavioral outcomes on interactive tasks. We derive features related to the occurrence, frequency, sequentiality, and timing of performed actions from early-window clickstreams and use extreme gradient boosting for classification. Multiple measures are suggested to evaluate the quality and utility of early predictions. The procedure is outlined by investigating early predictability of failure on two PIAAC 2012 Problem Solving in Technology Rich Environments (PSTRE) tasks. We investigated early windows of varying size in terms of time and in terms of actions. We achieved good prediction performance at stages where examinees had, on average, at least two thirds of their solution process ahead of them, and the vast majority of examinees who failed could potentially be detected to be at risk before completing the task. In-depth analyses revealed different features to be indicative of success and failure at different stages of the solution process, thereby highlighting the potential of the applied procedure for gaining a finer-grained understanding of the trajectories of behavioral patterns on interactive tasks.

Causal Inference with Multilevel Data: A Comparison of Different Propensity Score Weighting Approaches.

Propensity score methods are a widely recommended approach to adjust for confounding and to recover treatment effects with non-experimental, single-level data. This article reviews propensity score weighting estimators for multilevel data in which individuals (level 1) are nested in clusters (level 2) and nonrandomly assigned to either a treatment or control condition at level 1. We address the choice of a weighting strategy (inverse probability weights, trimming, overlap weights, calibration weights) and discuss key issues related to the specification of the propensity score model (fixed-effects model, multilevel random-effects model) in the context of multilevel data. In three simulation studies, we show that estimates based on calibration weights, which prioritize balancing the sample distribution of level-1 and (unmeasured) level-2 covariates, should be preferred under many scenarios (i.e., treatment effect heterogeneity, presence of strong level-2 confounding) and can accommodate covariate-by-cluster interactions. However, when level-1 covariate effects vary strongly across clusters (i.e., under random slopes), and this variation is present in both the treatment and outcome data-generating mechanisms, large cluster sizes are needed to obtain accurate estimates of the treatment effect. We also discuss the implementation of survey weights and present a real-data example that illustrates the different methods.

Multiple imputation of missing data in multilevel models with the R package mdmb: a flexible sequential modeling approach.

Multilevel models often include nonlinear effects, such as random slopes or interaction effects. The estimation of these models can be difficult when the underlying variables contain missing data. Although several methods for handling missing data such as multiple imputation (MI) can be used with multilevel data, conventional methods for multilevel MI often do not properly take the nonlinear associations between the variables into account. In the present paper, we propose a sequential modeling approach based on Bayesian estimation techniques that can be used to handle missing data in a variety of multilevel models that involve nonlinear effects. The main idea of this approach is to decompose the joint distribution of the data into several parts that correspond to the outcome and explanatory variables in the intended analysis, thus generating imputations in a manner that is compatible with the substantive analysis model. In three simulation studies, we evaluate the sequential modeling approach and compare it with conventional as well as other substantive-model-compatible approaches to multilevel MI. We implemented the sequential modeling approach in the R package mdmb and provide a worked example to illustrate its application.

Analysis of Interactions and Nonlinear Effects with Missing Data: A Factored Regression Modeling Approach Using Maximum Likelihood Estimation.

When estimating multiple regression models with incomplete predictor variables, it is necessary to specify a joint distribution for the predictor variables. A convenient assumption is that this distribution is a multivariate normal distribution, which is also the default in many statistical software packages. This distribution will in general be misspecified if predictors with missing data have nonlinear effects (e.g., x2) or are included in interaction terms (e.g., x·z). In the present article, we introduce a factored regression modeling approach for estimating regression models with missing data that is based on maximum likelihood estimation. In this approach, the model likelihood is factorized into a part that is due to the model of interest and a part that is due to the model for the incomplete predictors. In three simulation studies, we showed that the factored regression modeling approach produced valid estimates of interaction and nonlinear effects in regression models with missing values on categorical or continuous predictor variables under a broad range of conditions. We developed the R package mdmb, which facilitates a user-friendly application of the factored regression modeling approach, and present a real-data example that illustrates the flexibility of the software.

School or Work? The Choice May Change Your Personality.

According to the social-investment principle, entering new environments is associated with new social roles that influence people's behaviors. In this study, we examined whether young adults' personality development is differentially related to their choice of either an academic or a vocational pathway (i.e., entering an academic-track school or beginning vocational training). The personality constructs of interest were Big Five personality traits and vocational-interest orientations. We used a longitudinal study design and propensity-score matching to create comparable groups before they entered one of the pathways and then tested the differences between these groups 6 years later. We expected the vocational pathway to reinforce more mature behavior and curtail investigative interest. Results indicated that choosing the vocational compared with the academic pathway was associated with higher conscientiousness and less interest in investigative, social, and enterprising activities.

Similarity of vocational interest profiles within families: A person-centered approach for examining associations between circumplex profiles.

OBJECTIVE: Our study addressed three questions concerning the similarity of vocational interests within families: (a) How similar are vocational interests of mothers and fathers? (b) How similar are vocational interests of parents and their children? (c) Is the inference about parent-child profile similarity affected by mother-father profile similarity? METHOD: Data from N=1,624 tenth graders and their parents were used to analyze interest profile similarity by means of a pseudo-coupling approach. Similarity was assessed on the level of observed profiles and model-based circumplex profiles. RESULTS: Interest profiles of mother-father and parent-child dyads were more similar to each other than those of corresponding arbitrarily paired dyads. However, when the similarity between the parents' interest profiles was accounted for, only same-sex parent-child dyads were more similar to each other than would be expected by chance. All findings were mirrored on the level of observed profiles and model-based circumplex profiles. CONCLUSIONS: In sum, our findings support the validity of the circumplex model of vocational interests and emphasize the benefits of explicitly considering its implications when analyzing profile similarity. Moreover, we were able to show that the statistical evaluation of profile similarities must account for normative profile components.

Integrating covariates into circumplex structures: an extension procedure for Browne's circular stochastic process model.

Circumplex structures are elements of various psychological domains. Most work focuses on assessing the circular ordering of circumplex indicators and their relationships with covariates. In this article, an extension procedure for Browne's circumplex model is presented. Our approach models the relationships among circumplex indicators and the relationships of covariates with a latent circumplex simultaneously without affecting the circumplex indicators' positions on the circumplex. The approach builds upon Browne's Fourier series parameterization of a correlation function, which is used to model the latent circumplex correlation structure. It extends the shape of the correlation function to the profile of each covariate's correlations with the circumplex. The model is specified in the framework of structural equation modeling, thereby making it possible to test various hypotheses. Procedures are presented for deriving interval estimates for the parameters that relate the covariates to the circumplex. The model is compared to other approaches for assessing the relationships of a circumplex with covariates. The results of the exemplary applications and a simulation study were in favor of the suggested model. The approach is furthermore illustrated with a real-data example, focusing on the relationships between the interpersonal circumplex and the rivalry and admiration aspects of narcissism.

Improved HPLC-method for estimation and correction of amino acid losses during hydrolysis of unknown samples.

Amino acid analysis, commonly done by acid hydrolysis of proteins and HPLC analysis, faces one major problem: incomplete hydrolysis of stable amino acids and degradation of unstable amino acids are causing amino acid losses. As a result, amino acid recovery of unknown samples cannot be estimated. Some methods have been reported for correction of these factors in the past. This paper shows an improved and integrated method to overcome this problem by using stillage as an exemplary unknown sample material. Amino acid recovery from an unknown sample can be estimated by standard addition of a known protein. If the sample does not cause matrix effects during amino acid hydrolysis, recoveries of the standard protein are transferable to the sample. If the sample does cause matrix effects correction of amino acid losses can instead be done by determination of hydrolysis kinetics. Therefore, first order kinetics were used for amino acids that undergo degradation during hydrolysis. For all stable amino acids higher order kinetics were used, a novel approach to determine hydrolysis kinetics. The presented method can be a helpful tool for scientists who want to optimize amino acid analysis of a particular biomass substrate.

Self-esteem development in the school context: The roles of intrapersonal and interpersonal social predictors.

OBJECTIVE: When considering that social inclusion is a basic human need, it makes sense that self-esteem is fueled by social feedback and the sense of being liked by others. This is particularly true with respect to early adolescence, when peers become increasingly important. In the current article, we tested which components of social inclusion are particularly beneficial for the development of self-esteem by differentiating between intrapersonal components (i.e., self-perceptions of social inclusion) and interpersonal components (i.e., perceiver and target effects of liking). METHOD: Using longitudinal data from 2,281 fifth graders and 1,766 eighth graders (TRAIN; Jonkmann et al., 2013), we tested mean-level self-esteem development and the role of intrapersonal components in this development. Using classroom round-robin data on liking from subsamples of 846 (689) fifth-(eighth-)grade students nested in 46 (39) classes, we tested effects of interpersonal relationship components on self-esteem development in the classroom context. RESULTS: The three major findings demonstrated, first, no consistent trends in mean levels of self-esteem in early to middle adolescence; second, constant positive effects of intrapersonal components between students and within students across time; and third, no stable effects of interpersonal components. CONCLUSIONS: The discussion highlights the role of intrapersonal components and the methodological challenges of our study.

Integrating Covariates into Social Relations Models: A Plausible Values Approach for Handling Measurement Error in Perceiver and Target Effects.

The Social Relations Model (SRM) is a conceptual and analytical approach to examining dyadic behaviors and interpersonal perceptions within groups. In an SRM, the perceiver effect describes a person's tendency to perceive other group members in a certain way, whereas the target effect measures the tendency to be perceived by others in certain ways. In SRM research, it is often of interest to relate these individual SRM effects to covariates. However, the estimated individual SRM effects might not provide a very reliable measure of the true, unobserved SRM effects, resulting in distorted estimates of associations with other variables. This article introduces a plausible values approach that allows users to correct for measurement error when assessing the association of individual SRM effects with other individual difference variables. In the plausible values approach, the latent, true individual SRM effects are treated as missing values and are imputed from an imputation model by applying Bayesian estimation techniques. In a simulation study, the statistical properties of the plausible values approach are compared with two approaches that have been used in previous research. A data example from educational psychology is presented to illustrate how the plausible values approach can be implemented with the software WinBUGS.

Whose "Storm and Stress" Is It? Parent and Child Reports of Personality Development in the Transition to Early Adolescence.

The present study investigated Big Five personality trait development in the transition to early adolescence (from the fifth to eighth grade). Personality traits were assessed in 2,761 (47% female) students over a 3-year period of time. Youths' self-reports and parent ratings were used to test for cross-informant agreement. Acquiescent responding and measurement invariance were established with latent variable modeling. Growth curve models revealed three main findings: (a) Normative mean-level changes occurred for youths' self-report data and parent ratings with modest effects in both cases. (b) Agreeableness and Openness decreased for self-reports and parent ratings, whereas data source differences were found for Conscientiousness (decreased for self-reports and remained stable for parent ratings), Extraversion (increased for self-reports and decreased for parent ratings), and Neuroticism (remained stable for self-reports and decreased for parent ratings). (c) Girls showed a more mature personality overall (self-reports and parent ratings revealed higher levels of Agreeableness, Conscientiousness, and Openness) and became more extraverted in the middle of adolescence (self-reports). Personality changes modestly during early adolescence whereby change does not occur in the direction of maturation, and substantial differences exist between parent ratings and self-reports.

Self-esteem Is Mostly Stable Across Young Adulthood: Evidence from Latent STARTS Models.

How stable is self-esteem? This long-standing debate has led to different conclusions across different areas of psychology. Longitudinal data and up-to-date statistical models have recently indicated that self-esteem has stable and autoregressive trait-like components and state-like components. We applied latent STARTS models with the goal of replicating previous findings in a longitudinal sample of young adults (N = 4,532; Mage = 19.60, SD = 0.85; 55% female). In addition, we applied multigroup models to extend previous findings on different patterns of stability for men versus women and for people with high versus low levels of depressive symptoms. We found evidence for the general pattern of a major proportion of stable and autoregressive trait variance and a smaller yet substantial amount of state variance in self-esteem across 10 years. Furthermore, multigroup models suggested substantial differences in the variance components: Females showed more state variability than males. Individuals with higher levels of depressive symptoms showed more state and less autoregressive trait variance in self-esteem. Results are discussed with respect to the ongoing trait-state debate and possible implications of the group differences that we found in the stability of self-esteem.

The Kind of Student You Were in Elementary School Predicts Mortality.

We examined the association of self-reported and teacher-rated student characteristics assessed at the end of primary school with all-cause mortality assessed through age 52. Data stem from a representative sample of students from Luxembourg assessed in 1968 (N = 2,543; M = 11.9 years, SD = 0.6; 49.9% female; N = 166 participants died). Results from logistic regression analyses showed that the self-reported responsible student scale (OR = .81; CI = [.70; .95]) and the teacher rating of studiousness (OR = .80; CI = [.67; .96]) were predictive for all-cause mortality even after controlling for IQ, parental SES, and sex. These findings indicate that both observer-rated and self-reported student behaviors are important life-course predictors for mortality and are perhaps more important than childhood IQ.

Majoring in Selection, and Minoring in Socialization: The Role of the College Experience in Goal Change Post-High School.

Though it is frequently assumed that the college experience can influence our life goals, this claim has been relatively understudied. The current study examined the role of goals in college major selection, as well as whether major selection influences later goal change. In addition, we examined whether a person's perceptions of his or her peers' goals influence goal setting. Using a sample of German students (Mage = 19 years; n = 3,023 at Wave 1), we assessed life goal levels and changes from high school into college across three assessment occasions. Participants reported their current aspirations, along with the perceived goals of their peers during the college assessments. Using latent growth curve models, findings suggest that life goals upon entering college significantly predict the majors students select. However, this major selection had limited influence on later changes in life goals. Stronger effects were found with respect to perceptions of peers' goals, with students tending to change their goals to better align with their peers. The current study provides evidence that life goals are relatively stable and yet can change during the emerging adult years, in ways that demonstrate the potential influence of the college experience.

Exploring Factor Model Parameters across Continuous Variables with Local Structural Equation Models.

Using an empirical data set, we investigated variation in factor model parameters across a continuous moderator variable and demonstrated three modeling approaches: multiple-group mean and covariance structure (MGMCS) analyses, local structural equation modeling (LSEM), and moderated factor analysis (MFA). We focused on how to study variation in factor model parameters as a function of continuous variables such as age, socioeconomic status, ability levels, acculturation, and so forth. Specifically, we formalized the LSEM approach in detail as compared with previous work and investigated its statistical properties with an analytical derivation and a simulation study. We also provide code for the easy implementation of LSEM. The illustration of methods was based on cross-sectional cognitive ability data from individuals ranging in age from 4 to 23 years. Variations in factor loadings across age were examined with regard to the age differentiation hypothesis. LSEM and MFA converged with respect to the conclusions. When there was a broad age range within groups and varying relations between the indicator variables and the common factor across age, MGMCS produced distorted parameter estimates. We discuss the pros of LSEM compared with MFA and recommend using the two tools as complementary approaches for investigating moderation in factor model parameters.

Multiple imputation of missing covariate values in multilevel models with random slopes: a cautionary note.

Multiple imputation (MI) has become one of the main procedures used to treat missing data, but the guidelines from the methodological literature are not easily transferred to multilevel research. For models including random slopes, proper MI can be difficult, especially when the covariate values are partially missing. In the present article, we discuss applications of MI in multilevel random-coefficient models, theoretical challenges posed by slope variation, and the current limitations of standard MI software. Our findings from three simulation studies suggest that (a) MI is able to recover most parameters, but is currently not well suited to capture slope variation entirely when covariate values are missing; (b) MI offers reasonable estimates for most parameters, even in smaller samples or when its assumptions are not met; and

A Bayesian Approach to More Stable Estimates of Group-Level Effects in Contextual Studies.

Multilevel analyses are often used to estimate the effects of group-level constructs. However, when using aggregated individual data (e.g., student ratings) to assess a group-level construct (e.g., classroom climate), the observed group mean might not provide a reliable measure of the unobserved latent group mean. In the present article, we propose a Bayesian approach that can be used to estimate a multilevel latent covariate model, which corrects for the unreliable assessment of the latent group mean when estimating the group-level effect. A simulation study was conducted to evaluate the choice of different priors for the group-level variance of the predictor variable and to compare the Bayesian approach with the maximum likelihood approach implemented in the software Mplus. Results showed that, under problematic conditions (i.e., small number of groups, predictor variable with a small ICC), the Bayesian approach produced more accurate estimates of the group-level effect than the maximum likelihood approach did.

Agency-communion and interest in prosocial behavior: social motives for assimilation and contrast explain sociocultural inconsistencies.

Identifying the "prosocial personality" is a classic project in personality psychology. However, personality traits have been elusive predictors of prosocial behavior, with personality-prosociality relations varying widely across sociocultural contexts. We propose the social motives perspective to account for such sociocultural inconsistencies. According to this perspective, a focal quality of agency (e.g., competence, independence, openness) is the motive to swim against the social tide-agentic social contrast. Conversely, a focal quality of communion (e.g., warmth, interdependence, agreeableness) is the motive to swim with the social tide-communal social assimilation. We report two cross-sectional studies. Study 1 (N = 131,562) defined social context at the country level (11 European countries), whereas Study 2 (N = 56,395) defined it at the country level (11 European countries) and the city level (296 cities within these countries). Communion predicted interest in prosocial behavior comparatively strongly in sociocultural contexts where such interest was common and comparatively weakly where such interest was uncommon. Agency predicted interest in prosocial behavior comparatively strongly in sociocultural contexts where such interest was uncommon and comparatively weakly where such interest was common. The results supported the social motives perspective. Also, the findings help to reestablish the importance of personality for understanding prosociality.