Modification Indices for Diagnostic Classification Models.

Diagnostic classification models (DCMs) are psychometric models for evaluating a student's mastery of the essential skills in a content domain based upon their responses to a set of test items. Currently, diagnostic model and/or Q-matrix misspecification is a known problem with limited avenues for remediation. To address this problem, this paper defines a one-sided score statistic that is a computationally efficient method for detecting under-specification at the item level of both the Q-matrix and the model parameters of the particular DCM chosen in an analysis. This method is analogous to the modification indices widely used in structural equation modeling. The results of a simulation study show the Type I error rate of modification indices for DCMs are acceptably close to the nominal significance level when the appropriate mixture χ2 reference distribution is used. The simulation results indicate that modification indices are very powerful in the detection of an under-specified Q-matrix and have ample power to detect the omission of model parameters in large samples or when the items are highly discriminating. An application of modification indices for DCMs to an analysis of response data from a large-scale administration of a diagnostic test demonstrates how they can be useful in diagnostic model refinement.

Using Diagnostic Classification Models to Validate Attribute Hierarchies and Evaluate Model Fit in Bayesian Networks.

We investigate the relationship between Bayesian inference networks (BayesNets) and diagnostic classification models (DCMs). Specifically, we demonstrate and empirically examine the equivalency of parameterizations between BayesNets and DCMs. Then, we propose a model-comparison framework for testing the model fit of BayesNets, in which we show how BayesNets are nested within the saturated DCM structural models. Additionally, we show when attributes feature a linear hierarchy, the Hierarchical DCM is nested within both BayesNets and saturated DCMs. The usefulness of proposed framework and model-fit testing strategy was supported by the results of analyzing both simulated and empirical data.

The Power to Detect and Predict Individual Differences in Intra-Individual Variability Using the Mixed-Effects Location-Scale Model.

Our goal is to provide empirical scientists with practical tools and advice with which to test hypotheses related to individual differences in intra-individual variability using the mixed-effects location-scale model. To that end, we evaluate Type I error rates and power to detect and predict individual differences in intra-individual variability using this model and provide empirically-based guidelines for building scale models that include random and/or systematically-varying fixed effects. We also provide two power simulation programs that allow researchers to conduct a priori empirical power analyses. Our results aligned with statistical power theory, in that, greater power was observed for designs with more individuals, more repeated occasions, greater proportions of variance available to be explained, and larger effect sizes. In addition, our results indicated that Type I error rates were acceptable in situations when individual differences in intra-individual variability were not initially detectable as well as when the scale-model individual-level predictor explained all initially detectable individual differences in intra-individual variability. We conclude our paper by providing study design and model building advice for those interested in using the mixed-effects location-scale model in practice.

Direct Estimation of Diagnostic Classification Model Attribute Mastery Profiles via a Collapsed Gibbs Sampling Algorithm.

This paper proposes a novel collapsed Gibbs sampling algorithm that marginalizes model parameters and directly samples latent attribute mastery patterns in diagnostic classification models. This estimation method makes it possible to avoid boundary problems in the estimation of model item parameters by eliminating the need to estimate such parameters. A simulation study showed the collapsed Gibbs sampling algorithm can accurately recover the true attribute mastery status in various conditions. A second simulation showed the collapsed Gibbs sampling algorithm was computationally more efficient than another MCMC sampling algorithm, implemented by JAGS. In an analysis of real data, the collapsed Gibbs sampling algorithm indicated good classification agreement with results from a previous study.

Modeling Psychometric Relational Data in Social Networks: Latent Interdependence Models.

In the current paper, we propose a latent interdependence approach to modeling psychometric data in social networks. The idea of latent interdependence is adopted from social relations models (SRMs), which formulate a mutual-rating process by both dyad members' characteristics. Under the framework of the latent interdependence approach, we introduce two psychometric models: The first model includes the main effects of both rating-sender and rating-receiver, and the second model includes a latent distance effect to assess the influence from the dissimilarity between the latent characteristics of both sides. The latent distance effect is quantified by the Euclidean distance between both sides' trait scores. Both models use Bayesian estimation via Markov chain Monte Carlo. How accurately model parameters were estimated was evaluated in a simulation study. Parameter recovery results showed that all parameters were accurately recovered under most of the conditions investigated. As expected, the accuracy of model estimation was significantly improved as network size grew. Also, through analyzing empirical data, we showed how to use the estimates of model parameters to predict the latent weight of connections among group members and rebuild either a univariate or multivariate network at a latent trait level. Finally, we discuss issues regarding model comparison and offer suggestions for future studies.

Bridging gaps by including culture: Development and empirical test of the culturally informed theory for disordered eating among Black women.

BACKGROUND: Leading eating disorder (ED) theories were informed primarily by samples of White females. Therefore, ED theories lack consideration of sociocultural factors that may impact ED symptom development among Black women. The current study proposed the first culturally informed theory for disordered eating among Black women, positing that ethnic discrimination, strong black woman (SBW) ideology (cultural and societal expectations of strength), and culturally informed appearance satisfaction may significantly impact stress. Stress may be associated with coping-motivated eating behaviors, which may lead to maladaptive weight control behaviors. METHODS: Black women (N = 208) completed surveys assessing socio-cultural factors, stress, commensal and binge eating, and maladaptive weight control behaviors. Path analysis was used to test the proposed theory. RESULTS: The final model had a good fit for the data. Findings overall supported the hypothesized model. Specifically, higher ethnic discrimination (ß = 0.044, p = .003), greater endorsement of SBW ideology (ß = 0.074, p =< .001), and lower culturally informed appearance satisfaction (ß = -0.032, p = .025) were associated with greater stress. Stress was positively associated with binge eating (ß = 0.457, p = .046), and binge eating was significantly associated with excessive exercise (=0.152, p = .008) and purging (ß = 0.273, p = <.001). In contrast, commensal eating was not associated with stress or weight control behaviors (p values = .697 to .749). CONCLUSIONS: The current study found that stress, as influenced by sociocultural factors, may play a role in binge eating, and subsequently, weight control behaviors among Black women. This theory is a starting point for future research on the specialized conceptualization of eating and maladaptive weight control behaviors among Black women.

Longitudinal measurement invariance of the Working Alliance Inventory - Short form across coaching sessions.

BACKGROUND: Throughout the psychotherapeutic and coaching literature, the client-therapist or coach-coachee working alliance has been highlighted as key force driving positive outcome. The Working Alliance Inventory Short form (WAI-S) for coaching charts the quality of working alliance throughout coaching sessions and is broadly applied in coaching research. Due to a shortfall in research on psychometric properties of the WAI-S, the purpose of this study was to examine (a) if the theorized three-factor structure of the 12-item WAI-S forms a solid representation of the dimensions of working alliance in coaching, and (b) longitudinal measurement invariance (LMI) of the WAI-S. METHOD: Data were collected in a two-wave study design comprising a main study sample of N = 690 Dutch coachees that completed the questionnaire at the first measurement, of which N = 490 also completed the questionnaire at the second measurement. Post hoc sensitivity analysis was performed based on the original sample, lacking additional information on covariates, and included both completers and dropouts, comprising N = 1986 respondents at T1, and N = 1020 respondents at T2. RESULTS: Confirmatory factor analyses evidenced best fit of the three-factor model in comparison to one-, and two-factor models at both time points. Despite the fact that multigroup confirmatory factor analysis detected non-invariant intercepts, our findings overall supported measurement invariance across coaching sessions. CONCLUSIONS: As decisions in both clinical and scientific practices generally rely on outcome assessment of interpersonal change in scores on the same measure over time, we believe our findings to be of contributing value to the consolidation of interpretation and accuracy of scorings on the WAI-S in coaching.

Examining the psychometric properties of the Integrative Hope Scale's English translation in a mixed-diagnostic community health sample.

OBJECTIVE: Our objective is to provide preliminary evidence of the English translation of the Integrative Hope Scale (IHS). Hope is a critical concept for recovery. Synthesizing from other hope models, Schrank and colleagues developed the IHS. Although translated into five languages, no known studies assess the IHS's English translation within a clinical sample. Additionally, no known studies investigate the IHS's relationships with mental health measures in a mixed-diagnostic clinical sample. METHOD: To address these gaps in the literature, we used confirmatory factor analyses, alpha, and omega reliability coefficients, and correlational analyses to assess the IHS within a suburban, mixed-diagnostic intensive outpatient community mental health sample (n = 125) in the midwestern United States. RESULTS: While poorest fit was found within the one-factor model, the four-factor oblique, higher-order, and bifactor models showed improved fit. Reliability for the total score was good, with subscales ranging from acceptable to good. Significant relationships were found for the IHS in expected directions with measures of hope and depression at a large effect size and anxiety and stress at a moderate effect size. CONCLUSIONS AND IMPLICATIONS FOR PRACTICE: This study provides preliminary evidence that the IHS may have the potential to serve as a central measure of hope. Given hope's role within recovery and given its relationships with mental health measures shown in this mixed-diagnostic clinical sample, the IHS should continue to be investigated by researchers, clinicians, and clients, especially in recovery-focused programs. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Corrigendum: Integrating a Statistical Topic Model and a Diagnostic Classification Model for Analyzing Items in a Mixed Format Assessment.

[This corrects the article DOI: 10.3389/fpsyg.2020.579199.].

Integrating a Statistical Topic Model and a Diagnostic Classification Model for Analyzing Items in a Mixed Format Assessment.

Selected response items and constructed response (CR) items are often found in the same test. Conventional psychometric models for these two types of items typically focus on using the scores for correctness of the responses. Recent research suggests, however, that more information may be available from the CR items than just scores for correctness. In this study, we describe an approach in which a statistical topic model along with a diagnostic classification model (DCM) was applied to a mixed item format formative test of English and Language Arts. The DCM was used to estimate students' mastery status of reading skills. These mastery statuses were then included in a topic model as covariates to predict students' use of each of the latent topics in their written answers to a CR item. This approach enabled investigation of the effects of mastery status of reading skills on writing patterns. Results indicated that one of the skills, Integration of Knowledge and Ideas, helped detect and explain students' writing patterns with respect to students' use of individual topics.

Psychometric Study of the Social Responsiveness Scale in Phelan-McDermid Syndrome.

The Social Responsiveness Scale-2 (SRS-2) is a quantitative measure used to characterize symptoms of autism spectrum disorder (ASD). However, research suggests that SRS-2 scores are significantly influenced by language ability and intellectual disability (ID). Efforts to refine the SRS-2 by Sturm, Kuhfeld, Kasari, and Mccracken [Journal of Child Psychology and Psychiatry, 58(9), 1053-1061] yielded a shortened form, yet its psychometric properties in populations with severe ID remain unknown. This study aims to examine the psychometric properties of the SRS-2 in Phelan-McDermid syndrome (PMS), a genetic condition associated with ASD and ID, thereby guiding score interpretation in this population and future development of targeted scales. Analyses, including Item Response Theory (IRT), were conducted on a sample of individuals with PMS (n = 91) recruited at six sites nationally. Psychometric properties evaluated include measures of reliability (internal consistency, test-retest reliability) and validity (structural, construct, content). While both SRS-2 forms are reliable, the shortened SRS-2 shows superior validity to the full SRS-2 for measuring ASD symptoms in PMS. On IRT analysis, the shortened SRS-2 shows excellent discrimination and precisely evaluates respondents across a wide range of ASD symptomatology but interpretation is limited by uncertain content validity and small sample size. The shortened SRS-2 shows some promise for use in PMS, but future refinements and additions are needed to develop items that are tailored to identify ASD in children with severe ID and specifically PMS. LAY SUMMARY: This study determined that a shortened form of the Social Responsiveness Scale, Second Edition (SRS-2) shows both promise and limitations for the characterization of autism symptomatology in individuals with Phelan-McDermid syndrome (PMS), a population characterized by intellectual disability (ID). Caution should be used when interpreting SRS-2 scores in individuals with ID and future research should modify existing items and develop new items to improve the SRS-2's ability to accurately characterize autism symptomatology in PMS. Autism Res 2020, 13: 1383-1396. © 2020 International Society for Autism Research, Wiley Periodicals, Inc.

Gibbs Samplers for Logistic Item Response Models via the Pólya-Gamma Distribution: A Computationally Efficient Data-Augmentation Strategy.

Fully Bayesian estimation of item response theory models with logistic link functions suffers from low computational efficiency due to posterior density functions that do not have known forms. To improve algorithmic computational efficiency, this paper proposes a Bayesian estimation method by adopting a new data-augmentation strategy in uni- and multidimensional IRT models. The strategy is based on the Pólya-Gamma family of distributions which provides a closed-form posterior distribution for logistic-based models. In this paper, an overview of Pólya-Gamma distributions is described within a logistic regression framework. In addition, we provide details about deriving conditional distributions of IRT, incorporating Pólya-Gamma distributions into the conditional distributions for Bayesian samplers' construction, and random drawing from the samplers such that a faster convergence can be achieved. Simulation studies and applications to real datasets were conducted to demonstrate the efficiency and utility of the proposed method.

Fibromyalgia: the role of sleep in affect and in negative event reactivity and recovery.

OBJECTIVE: Fibromyalgia (FM) syndrome is a chronic pain condition characterized by diffuse muscle pain, increased negative mood, and sleep disturbance. Until recently, sleep disturbance in persons with FM has been modeled as the result of the disease process or its associated pain. The current study examined sleep disturbance (i.e., sleep duration and sleep quality) as a predictor of daily affect, stress reactivity, and stress recovery. DESIGN AND MEASURES: A hybrid of daily diary and ecological momentary assessment methodology was used to evaluate the psychosocial functioning of 89 women with FM. Participants recorded numeric ratings of pain, fatigue, and positive and negative affect 3 times throughout the day for 30 consecutive days. At the end of each day, participants completed daily diary records of positive and negative life events. In addition, participants reported on their sleep duration and sleep quality each morning. RESULTS: After accounting for the effects of positive events, negative events, and pain on daily affect scores, it was found that sleep duration and quality were prospectively related to affect and fatigue. Furthermore, the effects of inadequate sleep on negative affect were cumulative. In addition, an inadequate amount of sleep prevented affective recovery from days with a high number of negative events. CONCLUSIONS: These results lend support to the hypothesis that sleep is a component of allostatic load and has an upstream role in daily functioning.

Measurement of psychological disorders using cognitive diagnosis models.

Cognitive diagnosis models are constrained (multiple classification) latent class models that characterize the relationship of questionnaire responses to a set of dichotomous latent variables. Having emanated from educational measurement, several aspects of such models seem well suited to use in psychological assessment and diagnosis. This article presents the development of a new cognitive diagnosis model for use in psychological assessment--the DINO (deterministic input; noisy "or" gate) model--which, as an illustrative example, is applied to evaluate and diagnose pathological gamblers. As part of this example, a demonstration of the estimates obtained by cognitive diagnosis models is provided. Such estimates include the probability an individual meets each of a set of dichotomous Diagnostic and Statistical Manual of Mental Disorders (text revision [DSM-IV-TR]; American Psychiatric Association, 2000) criteria, resulting in an estimate of the probability an individual meets the DSM-IV-TR definition for being a pathological gambler. Furthermore, a demonstration of how the hypothesized underlying factors contributing to pathological gambling can be measured with the DINO model is presented, through use of a covariance structure model for the tetrachoric correlation matrix of the dichotomous latent variables representing DSM-IV-TR criteria.

Intra-Individual Variability of Physical Activity in Older Adults With and Without Mild Alzheimer's Disease.

Physical activity shows promise for protection against cognitive decline in older adults with and without Alzheimer's disease (AD). To better understand barriers to adoption of physical activity in this population, a clear understanding of daily and weekly activity patterns is needed. Most accelerometry studies report average physical activity over an entire wear period without considering the potential importance of the variability of physical activity. This study evaluated individual differences in the amount and intra-individual variability of physical activity and determined whether these differences could be predicted by AD status, day of wear, age, gender, education, and cardiorespiratory capacity. Physical activity was measured via accelerometry (Actigraph GT3X+) over one week in 86 older adults with and without AD (n = 33 and n = 53, respectively). Mixed-effects location-scale models were estimated to evaluate and predict individual differences in the amount and intra-individual variability of physical activity. Results indicated that compared to controls, participants with AD averaged 21% less activity, but averaged non-significantly greater intra-individual variability. Women and men averaged similar amounts of physical activity, but women were significantly less variable. The amount of physical activity differed significantly across days of wear. Increased cardiorespiratory capacity was associated with greater average amounts of physical activity. Investigation of individual differences in the amount and intra-individual variability of physical activity provided insight into differences by AD status, days of monitor wear, gender, and cardiovascular capacity. All individuals regardless of AD status were equally consistent in their physical activity, which may have been due to a highly sedentary sample and/or the early disease stage of those participants with AD. These results highlight the value of considering individual differences in both the amount and intra-individual variability of physical activity.

A Latent Transition Analysis Model for Assessing Change in Cognitive Skills.

Latent transition analysis (LTA) was initially developed to provide a means of measuring change in dynamic latent variables. In this article, we illustrate the use of a cognitive diagnostic model, the DINA model, as the measurement model in a LTA, thereby demonstrating a means of analyzing change in cognitive skills over time. An example is presented of an instructional treatment on a sample of seventh-grade students in several classrooms in a Midwestern school district. In the example, it is demonstrated how hypotheses could be framed and then tested regarding the form of the change in different groups within the population. Both manifest and latent groups also are defined and used to test additional hypotheses about change specific to particular subpopulations. Results suggest that the use of a DINA measurement model expands the utility of LTA to practical problems in educational measurement research.

Comparing the Two- and Three-Parameter Logistic Models via Likelihood Ratio Tests: A Commonly Misunderstood Problem.

Selection of an appropriate item response model is critical in the measurement of latent examinee ability. The one-, two-, and three-parameter logistic (1PL, 2PL, and 3PL) models are nested, and as such can be compared using likelihood ratio (LR) tests. The null hypothesis in the LR test for selection among the 2PL and 3PL models sets the guessing parameters to their lower bound of 0. This violates one of the assumptions of the LR test and renders the usual χ2 reference distribution inappropriate for the comparison. A review of the current literature revealed that this problem is not well understood in the educational measurement field. Ignoring this issue can lead to selection of an overly simplified model, with implications for the ability estimates. In this article, the use of the LR test for item response model selection is investigated, with the goal of providing practitioners with an appropriate method of selecting the most parsimonious model. The results of simulation studies indicate the nature of the problem, with inaccurate Type I error rates for cases where the inappropriate null distribution was used. An analysis of data from a statewide mathematics test showed differences pertinent to subsequent analyses.