Detecting Preknowledge Cheating via Innovative Measures: A Mixture Hierarchical Model for Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts.

Preknowledge cheating jeopardizes the validity of inferences based on test results. Many methods have been developed to detect preknowledge cheating by jointly analyzing item responses and response times. Gaze fixations, an essential eye-tracker measure, can be utilized to help detect aberrant testing behavior with improved accuracy beyond using product and process data types in isolation. As such, this study proposes a mixture hierarchical model that integrates item responses, response times, and visual fixation counts collected from an eye-tracker (a) to detect aberrant test takers who have different levels of preknowledge and (b) to account for nuances in behavioral patterns between normally-behaved and aberrant examinees. A Bayesian approach to estimating model parameters is carried out via an MCMC algorithm. Finally, the proposed model is applied to experimental data to illustrate how the model can be used to identify test takers having preknowledge on the test items.

Using Simulated Annealing to Investigate Sensitivity of SEM to External Model Misspecification.

Sensitivity analyses encompass a broad set of post-analytic techniques that are characterized as measuring the potential impact of any factor that has an effect on some output variables of a model. This research focuses on the utility of the simulated annealing algorithm to automatically identify path configurations and parameter values of omitted confounders in structural equation modeling (SEM). An empirical example based on a past published study is used to illustrate how strongly related an omitted variable must be to model variables for the conclusions of an analysis to change. The algorithm is outlined in detail and the results stemming from the sensitivity analysis are discussed.

Nonconvergence, covariance constraints, and class enumeration in growth mixture models.

Growth mixture models (GMMs) are a popular method to identify latent classes of growth trajectories. One shortcoming of GMMs is nonconvergence, which often leads researchers to apply covariance equality constraints to simplify estimation, though this may be a dubious assumption. Alternative model specifications have been proposed to reduce nonconvergence without imposing covariance equality constraints. These methods perform well when the correct number of classes is known, but research has not yet examined their use when the number of classes is unknown. Given the importance of selecting the number of classes, more information about class enumeration performance is crucial to assess the potential utility of these methods. We conducted an extensive simulation to explore class enumeration and classification accuracy of model specifications that are more robust to nonconvergence. Results show that the typical approach of applying covariance equality constraints performs quite poorly. Instead, we recommended covariance pattern GMMs because they (a) had the highest convergence rates, (b) were most likely to identify the correct number of classes, and (c) had the highest classification accuracy in many conditions, even with modest sample sizes. An analysis of empirical posttraumatic stress disorder (PTSD) data is provided to show that the typical four-class solution found in many empirical PTSD studies may be an artifact of the covariance equality constraint method that has permeated this literature. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Bridging Models of Biometric and Psychometric Assessment: A Three-Way Joint Modeling Approach of Item Responses, Response Times, and Gaze Fixation Counts.

Recently, joint models of item response data and response times have been proposed to better assess and understand test takers' learning processes. This article demonstrates how biometric information such as gaze fixation counts obtained from an eye-tracking machine can be integrated into the measurement model. The proposed joint modeling framework accommodates the relations among a test taker's latent ability, working speed and test engagement level via a person-side variance-covariance structure, while simultaneously permitting the modeling of item difficulty, time-intensity, and the engagement intensity through an item-side variance-covariance structure. A Bayesian estimation scheme is used to fit the proposed model to data. Posterior predictive model checking based on three discrepancy measures corresponding to various model components are introduced to assess model-data fit. Findings from a Monte Carlo simulation and results from analyzing experimental data demonstrate the utility of the model.

A Comparison of Label Switching Algorithms in the Context of Growth Mixture Models.

Simulation studies involving mixture models inevitably aggregate parameter estimates and other output across numerous replications. A primary issue that arises in these methodological investigations is label switching. The current study compares several label switching corrections that are commonly used when dealing with mixture models. A growth mixture model is used in this simulation study, and the design crosses three manipulated variables-number of latent classes, latent class probabilities, and class separation, yielding a total of 18 conditions. Within each of these conditions, the accuracy of a priori identifiability constraints, a priori training of the algorithm, and four post hoc algorithms developed by Tueller et al.; Cho; Stephens; and Rodriguez and Walker are tested to determine their classification accuracy. Findings reveal that, of all a priori methods, training of the algorithm leads to the most accurate classification under all conditions. In a case where an a priori algorithm is not selected, Rodriguez and Walker's algorithm is an excellent choice if interested specifically in aggregating class output without consideration as to whether the classes are accurately ordered. Using any of the post hoc algorithms tested yields improvement over baseline accuracy and is most effective under two-class models when class separation is high. This study found that if the class constraint algorithm was used a priori, it should be combined with a post hoc algorithm for accurate classification.

Assessing Preknowledge Cheating via Innovative Measures: A Multiple-Group Analysis of Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts.

Many approaches have been proposed to jointly analyze item responses and response times to understand behavioral differences between normally and aberrantly behaved test-takers. Biometric information, such as data from eye trackers, can be used to better identify these deviant testing behaviors in addition to more conventional data types. Given this context, this study demonstrates the application of a new method for multiple-group analysis that concurrently models item responses, response times, and visual fixation counts collected from an eye-tracker. It is hypothesized that differences in behavioral patterns between normally behaved test-takers and those who have different levels of preknowledge about the test items will manifest in latent characteristics of the different data types. A Bayesian estimation scheme is used to fit the proposed model to experimental data and the results are discussed.

African American English and Early Literacy: A Comparison of Approaches to Quantifying Nonmainstream Dialect Use.

Purpose Many studies have found a correlation between overall usage rates of nonmainstream forms and reading scores, but less is known about which dialect differences are most predictive. Here, we consider different methods of characterizing African American English use from existing assessments and examine which methods best predict literacy achievement. Method Kindergarten and first-grade students who speak African American English received two assessments of dialect use and two assessments of decoding at the beginning and end of the school year. Item-level analyses of the dialect-use assessments were used to compute measures of dialect usage: (a) an overall feature rate measure based on the Diagnostic Evaluation of Language Variation-Screening Test, (b) a subscore analysis of the Diagnostic Evaluation of Language Variation-Screening Test based on items that pattern together, (c) an alternative assessment where children repeat and translate sentences, and (d) "repertoire" measures based on a categorical distinction of whether a child used a particular feature of mainstream American English. Results Models using feature rate measures provided better data-model fit than those with repertoire measures, and baseline performance on a sentence repetition task was a positive predictor of reading score at the end of the school year. For phonological subscores, change from the beginning to end of the school year predicted reading at the end of the school year, whereas baseline scores were most predictive for grammatical subscores. Conclusions The addition of a sentence imitation task is useful for understanding a child's dialect and anticipating potential areas for support in early literacy. We observed some support for the idea that morphological dialect differences (i.e., irregular verb morphology) have a particularly close tie to later literacy, but future work will be necessary to confirm this finding. Supplemental Material https://doi.org/10.23641/asha.13425968.

Improving convergence in growth mixture models without covariance structure constraints.

Growth mixture models are a popular method to uncover heterogeneity in growth trajectories. Harnessing the power of growth mixture models in applications is difficult given the prevalence of nonconvergence when fitting growth mixture models to empirical data. Growth mixture models are rooted in the random effect tradition, and nonconvergence often leads researchers to modify their intended model with constraints in the random effect covariance structure to facilitate estimation. While practical, doing so has been shown to adversely affect parameter estimates, class assignment, and class enumeration. Instead, we advocate specifying the models with a marginal approach to prevent the widespread practice of sacrificing class-specific covariance structures to appease nonconvergence. A simulation is provided to show the importance of modeling class-specific covariance structures and builds off existing literature showing that applying constraints to the covariance leads to poor performance. These results suggest that retaining class-specific covariance structures should be a top priority and that marginal models like covariance pattern growth mixture models that model the covariance structure without random effects are well-suited for such a purpose, particularly with modest sample sizes and attrition commonly found in applications. An application to PTSD data with such characteristics is provided to demonstrate (a) convergence difficulties with random effect models, (b) how covariance structure constraints improve convergence but to the detriment of performance, and (c) how covariance pattern growth mixture models may provide a path forward that improves convergence without forfeiting class-specific covariance structures.

Piecewise latent growth models: beyond modeling linear-linear processes.

Piecewise latent growth models (LGMs) for linear-linear processes have been well-documented and studied in recent years. However, in the latent growth modeling literature, advancements to other functional forms as well as to multiple changepoints or knots have been nearly non-existent. This manuscript deals with three extensions. The first is to a piecewise latent growth model incorporating higher-order polynomials. The second is to extend the basic framework to three phases. The last extension is to inherently nonlinear functions. In these extensions, the changepoint(s) is a parameter to be estimated and may be fixed or allowed to vary across subjects as an application warrants. The approaches are developed and two illustrative empirical examples from psychology are used to highlight the methodological nuances. Annotated statistical software is provided to make these elaborations accessible to practitioners and methodologists.

Heterogeneity in the dynamic arousal and modulation of fear in young foster children.

Guided by emotional security theory, we explored how child and context-related factors were associated with heterogeneity in young foster children's organized patterns of fear response to distress. Results from group-based trajectory modeling used to analyze observational data from a fear-eliciting task showed that children from our sample (mean age = 62 months, SD = 9) were classified into 3 specific fear regulation patterns differentiated by the emotional response parameters of onset intensity, peak intensity, and rise time. A descriptive examination of child's emotion knowledge, aggressive behaviors, and attention problems, as well as length of time in current foster home, placement transitions, and caregiver responsiveness and modeling showed class-specific differences in means. Moreover, the likelihood of class membership was significantly predicted by children's emotion knowledge, aggressive behaviors, and foster mothers' responsiveness and modeling of appropriate boundaries. Results show promising support for the implementation of individualized, child-directed interventions targeting specific patterns of response parameters of emotion regulation for young foster children. Further, parenting intervention services need to promote the emotion socialization skills of foster parents that are tailored toward each specific trajectory pattern of emotion arousal and modulation.

Two-part mixed-effects location scale models.

Longitudinal time use data afford the opportunity to study within- and between-individual differences, but can present challenges in data analysis. Often the response set includes a large number of zeros representing those who did not engage in the target behavior. Coupled with this is a continuous measure of time use for those who did engage. The latter is strictly positive and skewed to the right if relatively few individuals engage in the behavior to a greater extent. Data analysis is further complicated for repeated measures, because within-individual responses are typically correlated, and some respondents may have missing data. This combination of zeros and positive responses is characteristic of a type of semicontinuous data in which the response is equal to a discrete value and is otherwise continuous. Two-part models have been successfully applied to cross-sectional time use data when the research goals distinguish between a respondent's likelihood to engage in a behavior and the time spent conditional on any time being spent, as these models allow different covariates to relate to each distinct aspect of a behavior. Two-part mixed-effects models extend two-part models for analysis of longitudinal semicontinuous data to simultaneously address longitudinal decisions to engage in a behavior and time spent conditional on any time spent. Heterogeneity between and within individuals can be studied in unique ways. This paper presents applications of these models to daily diary data to study individual differences in time spent relaxing or engaged in leisure activities for an adult sample.

Joint Modeling of Compensatory Multidimensional Item Responses and Response Times.

Computer-based testing (CBT) is becoming increasingly popular in assessing test-takers' latent abilities and making inferences regarding their cognitive processes. In addition to collecting item responses, an important benefit of using CBT is that response times (RTs) can also be recorded and used in subsequent analyses. To better understand the structural relations between multidimensional cognitive attributes and the working speed of test-takers, this research proposes a joint-modeling approach that integrates compensatory multidimensional latent traits and response speediness using item responses and RTs. The joint model is cast as a multilevel model in which the structural relation between working speed and accuracy are connected through their variance-covariance structures. The feasibility of this modeling approach is investigated via a Monte Carlo simulation study using a Bayesian estimation scheme. The results indicate that integrating RTs increased model parameter recovery and precision. In addition, Program of International Student Assessment (PISA) 2015 mathematics standard unit items are analyzed to further evaluate the feasibility of the approach to recover model parameters.

Latent Growth Models with Floors, Ceilings, and Random Knots.

In longitudinal/developmental studies, individual growth trajectories are sometimes bounded by a floor at the beginning of the observation period and/or a ceiling toward the end of the observation period (or vice versa), resulting in inherently nonlinear growth patterns. If the trajectories between the floor and ceiling are approximately linear, such longitudinal growth patterns can be described with a linear piecewise (spline) model in which segments join at knots. In these scenarios, it may be of specific interest for researchers to examine the timing when transition occurs, and in some occasions also to examine the levels of the floors and/or ceilings if they are not known and fixed. In the current study, we propose a reparameterized piecewise latent growth curve model so that a direct estimation of the random knots (and, if needed, a direct estimation of random floors and ceilings) is possible. We derive the model reparameterization using a 4-step structured latent curve modeling approach. We provide two illustrative examples to demonstrate how the proposed reparameterized models can be fitted to longitudinal growth data using the popular SEM software Mplus and we supply the full coding for applied researchers' reference.

Exploring Symptom Clusters in People With Heart Failure.

Patients with heart failure (HF) experience multiple symptoms or symptom clusters. The purposes of this study were to (a) determine if distinct latent classes of HF symptoms could be identified, and (b) explore whether sociodemographic and clinical characteristics influenced symptom cluster membership. A total of 4,011 HF patients recruited from outpatient setting completed the Minnesota Living With Heart Failure Questionnaire (MLHFQ), including five physical symptoms (edema, shortness of breath, fatigue-increased need to rest, fatigue-low energy, and sleep difficulties) and three psychological symptoms (worrying, feeling depressed, and cognitive problems). Four distinct classes using latent class profile analysis were identified: low distress (Class 1), physical distress (Class 2), psychological distress (Class 3), and high distress (Class 4). Significant differences among the four latent classes were found for age, education level, and comorbidities. Symptom clusters are useful for recognition of HF symptoms, allowing for the development of strategies that target symptom groups.

Negative Binomial Models for Visual Fixation Counts on Test Items.

With the development of technology-enhanced learning platforms, eye-tracking biometric indicators can be recorded simultaneously with students item responses. In the current study, visual fixation, an essential eye-tracking indicator, is modeled to reflect the degree of test engagement when a test taker solves a set of test questions. Three negative binomial regression models are proposed for modeling visual fixation counts of test takers solving a set of items. These models follow a similar structure to the lognormal response time model and the two-parameter logistic item response model. The proposed modeling structures include individualized latent person parameters reflecting the level of engagement of each test taker and two item parameters indicating the visual attention intensity and discriminating power of each test item. A Markov chain Monte Carlo estimation method is implemented for parameter estimation. Real data are fitted to the three proposed models, and the results are discussed.

The effects of getting a new teacher on the consistency of personality.

OBJECTIVE: In the present research, we examined the effect of getting a new teacher on consistency in students' personality measures, including trait and social cognitive constructs. METHOD: To test the effect of this kind of situational transition, we analyzed two large longitudinal samples (N = 5,628; N = 2,458) with quasi-experimental study designs. We used two consistency measures (i.e., rank-order clations and changes in variance over time) to compare students who got a new teacher with students who kept the same teacher. RESULTS: Multiple-group latent variable analyses showed no differences in the rank-order correlations for the math-related social cognitive constructs of interest, effort, self-concept, self-regulation, anxiety, and the Big Five personality traits. Significantly lower rank-order correlations were found for some of the German- and English-related social cognitive constructs (i.e., effort measures) for the group of students who got a new teacher. Regarding the changes in variance (over time), we found no systematic differences between groups in both studies. CONCLUSIONS: We found partial support for the idea that social cognitive variables are more susceptible to environmental changes (i.e., getting a new teacher) than the Big Five personality traits are.

A brief parent-focused intervention to improve preschoolers' conversational skills and school readiness.

Preschool children's use of decontextualized language, or talk about abstract topics beyond the here-and-now, is predictive of their kindergarten readiness and is associated with the frequency of parents' own use of decontextualized language. Does a brief, parent-focused intervention conveying the importance of decontextualized language cause parents to increase their use of these conversations, and as a result, their children's? We examined this question by randomly assigning 36 parents of 4-year-old children into a decontextualized language training condition or a no-treatment control condition and used mixed effects modeling to measure change (from baseline) in parent and child decontextualized language at 4 subsequent home mealtimes during the following month (N = 174 interactions including the baseline). Compared with the control condition, training condition dyads significantly increased their decontextualized talk and maintained these gains for the month following implementation. Furthermore, trained dyads generalized the program content to increase their use of similarly decontextualized, yet untrained conversations. These results demonstrate that an abstract feature of the input is malleable, and introduces a simple, scalable, and replicable approach to increase a feature of child language known to be foundational for children's school readiness. (PsycINFO Database Record

Using phantom variables in structural equation modeling to assess model sensitivity to external misspecification.

External misspecification, the omission of key variables from a structural model, can fundamentally alter the inferences one makes without such variables present. This article presents 2 strategies for dealing with omitted variables, the first a fixed parameter approach incorporating the omitted variable into the model as a phantom variable where all associated parameter values are fixed, and the other a random parameter approach specifying prior distributions for all of the phantom variable's associated parameter values under a Bayesian framework. The logic and implementation of these methods are discussed and demonstrated on an applied example from the educational psychology literature. The argument is made that such external misspecification sensitivity analyses should become a routine part of measured and latent variable modeling where the inclusion of all salient variables might be in question. (PsycINFO Database Record

The Effect of Extreme Response and Non-extreme Response Styles on Testing Measurement Invariance.

Extreme and non-extreme response styles (RSs) are prevalent in survey research using Likert-type scales. Their effects on measurement invariance (MI) in the context of confirmatory factor analysis are systematically investigated here via a Monte Carlo simulation study. Using the parameter estimates obtained from analyzing a 2007 Trends in International Mathematics and Science Study data set, a population model was constructed. Original and contaminated data with one of two RSs were generated and analyzed via multi-group confirmatory factor analysis with different constraints of MI. The results indicated that the detrimental effects of response style on MI have been underestimated. More specifically, these two RSs had a substantially negative impact on both model fit and parameter recovery, suggesting that the lack of MI between groups may have been caused by the RSs, not the measured factors of focal interest. Practical implications are provided to help practitioners to detect RSs and determine whether RSs are a serious threat to MI.