Good Fit Is Weak Evidence of Replication: Increasing Rigor Through Prior Predictive Similarity Checking.

Replication provides a confrontation of psychological theory, not only in experimental research, but also in model-based research. Goodness of fit (GOF) of the original model to the replication data is routinely provided as meaningful evidence of replication. We demonstrate, however, that GOF obscures important differences between the original and replication studies. As an alternative, we present Bayesian prior predictive similarity checking: a tool for rigorously evaluating the degree to which the data patterns and parameter estimates of a model replication study resemble those of the original study. We apply this method to original and replication data from the National Comorbidity Survey. Both data sets yielded excellent GOF, but the similarity checks often failed to support close or approximate empirical replication, especially when examining covariance patterns and indicator thresholds. We conclude with recommendations for applied research, including registered reports of model-based research, and provide extensive annotated R code to facilitate future applications of prior predictive similarity checking.

Improving the Walktrap Algorithm Using K-Means Clustering.

The walktrap algorithm is one of the most popular community-detection methods in psychological research. Several simulation studies have shown that it is often effective at determining the correct number of communities and assigning items to their proper community. Nevertheless, it is important to recognize that the walktrap algorithm relies on hierarchical clustering because it was originally developed for networks much larger than those encountered in psychological research. In this paper, we present and demonstrate a computational alternative to the hierarchical algorithm that is conceptually easier to understand. More importantly, we show that better solutions to the sum-of-squares optimization problem that is heuristically tackled by hierarchical clustering in the walktrap algorithm can often be obtained using exact or approximate methods for K-means clustering. Three simulation studies and analyses of empirical networks were completed to assess the impact of better sum-of-squares solutions.

The Brief Emotion Dysregulation Scale: Development, Preliminary Validation, and Recommendations for Use.

Emotion dysregulation is a multi-faceted, transdiagnostic construct, and its assessment is crucial for characterizing its role in the development, maintenance, and treatment of psychiatric problems. We developed the Brief Emotion Dysregulation Scale (BEDS) to capture four components of emotion dysregulation: sensitivity, lability, reactivity, and consequences. We examined factor structure and construct validity in four independent samples of college students (N = 1,485). We elected to treat consequences as a separate index of problems associated with emotion dysregulation. Exploratory and confirmatory factor analyses did not support the reactivity subscale and instead supported a well-fitting two-factor solution for sensitivity and lability. Multi-group analyses demonstrated strong factorial invariance by gender. The resulting 12-item BEDS includes sensitivity and lability subscales and a separate consequences scale to indicate associated problems. Convergent correlations suggested good construct validity. This provides preliminary support for the BEDS as a brief transdiagnostic screening tool for emotion dysregulation and associated consequences.

Examining the validity of the addictions neuroclinical assessment domains in a crowdsourced sample of adults with current alcohol use.

Several dimensional frameworks for characterizing heterogeneity in alcohol use disorder (AUD) have been proposed, including the Addictions Neuroclinical Assessment (ANA). The ANA is a framework for assessing individual variability within AUD across three domains corresponding to the proposed stages of the addiction cycle: reward (binge-intoxication stage), negative emotionality (withdrawal-negative affect stage), and cognitive control (preoccupation-anticipation stage). Recent work has evaluated the ANA's three-factor structure and construct validity, primarily in treatment-seekers with AUD. We extended this research by examining the factor structure, bias across alcohol use severity, longitudinal invariance, and concurrent and predictive validity of a novel assessment of the ANA domains in adults with past 12-month regular (10 + alcohol units/week) alcohol use. Participants recruited from Prolific (N = 732), a crowdsourced data collection platform, completed various self-report measures. A test-retest subsample (n = 234) completed these measures 30 days later. Split-half exploratory factor analysis and confirmatory factor analysis supported the three-factor structure of the ANA. The overall factor structure was invariant across 30 days. Concurrently and prospectively, ANA domains demonstrated convergent validity concerning theoretically aligned alcohol-related, psychological, and personality measures. However, there was evidence of poor discriminant validity, and several cognitive control and reward items demonstrated bias across alcohol use severity. Future research is needed to improve the measurement of ANA domains using multimodal indicators, examine longitudinal changes in domains and their relationship with alcohol use severity, characterize phenotypic subgroups based on relative levels of domains, and compare the utility of the ANA with other proposed frameworks for measuring AUD heterogeneity. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Methods for quantifying the heterogeneity of psychopathology.

OBJECTIVES: Specifiers for a major depressive disorder (MDE) are supposed to reduce diagnostic heterogeneity. However, recent literature challenges the idea that the atypical and melancholic specifiers identify more homogenous or coherent subgroups. We introduce the usage of distance metrics to characterize symptom heterogeneity. We attempt to replicate prior findings and explore whether symptom heterogeneity is reduced using specifier subgroups. METHODS: We used data derived from the National Epidemiological Survey on Alcohol and Related Conditions (NESARC Wave I; N = 5,749) and the Sequenced Treatment Alternatives to Relieve Depression study (STAR*D; N = 2,498). We computed Hamming and Manhattan distances from study participants' unique symptom profiles. Distances were standardized from 0-1 and compared by their within- and between-group similarities to their non-specifier counterparts for the melancholic and atypical specifiers. RESULTS: There was no evidence of statistically significant differences in heterogeneity for specifier (i.e., melancholic or atypical) vs. non-specifier designations (i.e., non-melancholic vs. non-atypical). CONCLUSION: Replicating prior work, melancholic and atypical depression specifiers appear to have limited utility in reducing heterogeneity. The current study does not support the claim that specifiers create more coherent subgroups as operationalized by similarity in the number of symptoms and their severity. Distance metrics are useful for quantifying symptom heterogeneity.

Alcohol use disorder conceptualizations and diagnoses reflect their sociopolitical context.

The present paper highlights how alcohol use disorder (AUD) conceptualizations and resulting diagnostic criteria have evolved over time in correspondence with interconnected sociopolitical influences in the United States. We highlight four illustrative examples of how DSM-defined alcoholism, abuse/dependence, and AUD have been influenced by sociopolitical factors. In doing so, we emphasize the importance of recognizing and understanding such sociopolitical factors in the application of AUD diagnoses. Last, we offer a roadmap to direct the process of future efforts toward the improved diagnosis of AUD, with an emphasis on pursuing falsifiability, acknowledging researchers' assumptions about human behavior, and collaborating across subfields. Such efforts that center the numerous mechanisms and functions of behavior, rather than signs or symptoms, have the potential to minimize sociopolitical influences in the development of diagnostic criteria and maximize the treatment utility of diagnoses.

Psychopathology as dynamic markers of alcohol initiation across development: A three-year longitudinal examination.

Sipping, an early form of alcohol initiation, is associated with aspects of psychopathology and personality that reflect long-term risk for harmful alcohol use. In the Adolescent Brain and Cognitive Development cohort (N = 11,872), sipping by age 9-10 was concurrently associated with impulsivity, other aspects of externalizing, and prodromal schizophrenia symptoms. Still, these associations were cross-sectional in nature, leaving open the possibility that these features of psychopathology and personality might not reflect long-term risk for alcohol consumption and related harm across development. Here, we attempted to replicate baseline concurrent associations across three waves of data to extend concurrent associations to prospective ones. Most cross-sectional associations replicated across waves, such that impulsivity, other aspects of externalizing, reward sensitivity (e.g., surgency, sensation seeking), and prodromal schizophrenia symptoms were associated with increased odds of having sipped alcohol by the age of 12. Nevertheless, not all concurrent associations replicated prospectively; impulsigenic features did not reflect long-term risk for sipping. Thus, some psychopathology features appeared to reflect stable risk factors, whereas others appeared to reflect state-dependent risk factors. All told, sipping might not reflect long-term risk for harmful alcohol use, and the nature of sipping may change across development.

Factor analysis in personality disorders research: Modern issues and illustrations of practical recommendations.

The development of factor analysis is uniquely situated within psychology, and the development of many psychological theories and measures are likewise tethered to the common use of factor analysis. In this article, we review modern methodological controversies and developments of factor analytic techniques through concrete demonstrations that span the exploratory-confirmatory continuum. Also, we provide recommendations for working through common challenges in personality disorders research. To help researchers conduct riskier tests of their theory-implied models, we review what factor analysis is and is not, as well as some dos and don'ts for engaging in the process of model evaluation and selection. Throughout, we also emphasize the need for closer alignment between factor models and our theories, as well as clearer statements about which criteria would support or refute the theories being tested. Consideration of these themes appears promising in terms of advances in theory, research, and treatment surrounding the nature and impact of personality disorders. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Alcohol use disorder criteria exhibit different comorbidity patterns.

BACKGROUND AND AIMS: Alcohol use disorder is comorbid with numerous other forms of psychopathology, including externalizing disorders (e.g. conduct disorder) and, to a lesser extent, internalizing conditions (e.g. depression, anxiety). Much of the time, overlap among alcohol use disorder and other conditions is explored at the disorder level, assuming that criteria are co-equal indicators of other psychopathology, even though alcohol use disorder criteria span numerous varied domains. Emerging evidence suggests that there are symptom clusters within the construct of alcohol use disorder that relate differentially with important external criteria, including psychopathology and allied personality traits (e.g. impulsivity, novelty-seeking). The present study mapped individual alcohol use disorder criteria onto internalizing and externalizing dimensions. DESIGN AND PARTICIPANTS: We used multivariate and factor analytical modeling and data from two large nationally representative samples of past year drinkers (n = 25 604; 19 454). SETTING: United States. MEASUREMENTS: Psychopathology was assessed using the Alcohol Use Disorder and Associated Disabilities Interview Schedule, yielding alcohol use disorder criteria, internalizing diagnoses (i.e. major depressive disorder, dysthymia, social anxiety disorder, generalized anxiety disorder, specific phobia, agoraphobia and panic disorder) and externalizing diagnoses and symptoms (i.e. antisocial personality disorder, conduct disorder and three impulsivity items drawn from borderline personality disorder criteria). Alcohol consumption was assessed in terms of past-year drinking frequency, usual amount of alcohol consumed on drinking days, binge drinking frequency, intoxication frequency, and maximum number of drinks in a 24-hour period. FINDINGS: Four different patterns emerged. First, several alcohol use disorder criteria were relatively weakly associated with externalizing and internalizing. Secondly, withdrawal was associated with internalizing, but this association was not specific to distress. Thirdly, there was a general lack of specificity between alcohol use disorder criteria and narrower forms of internalizing, despite what might be predicted by modern models of addiction. Fourthly, recurrent use in hazardous situations reflected higher degrees of externalizing and lower internalizing liability. CONCLUSIONS: Different symptom combinations appear to yield differential expressions of alcohol use disorder that are disorder-specific, or reflect broader tendencies toward externalizing, internalizing or both.

The Basic Psychological Need Satisfaction and Frustration Scales probably do not validly measure need frustration.

In basic psychological needs theory (BPNT), the separable constructs of need satisfaction and need frustration are theorized as pivotally related to psychopathology and broader aspects of well-being. The Basic Psychological Need Satisfaction and Frustration Scales (BPNSFS; Chen et al., 2015) have rapidly emerged as the dominant self-report measure in the BPNT domain, with translated versions available in a wide range of languages and a plethora of versions adapted for specific populations and life contexts. Through (a) an extended conceptual discussion of the BPNSFS and (b) a collection of complementary data analyses in eight samples, we demonstrate that the BPNSFS probably does not validly measure need frustration. Most importantly, we conclude that the ostensible distinction between need frustration and need satisfaction in the BPNSFS is likely primarily a method artifact caused by different item keying directions, given the way the measure currently assesses the intended constructs. If so, then the use of the BPNSFS may be generating misleading conclusions, obstructing sound investigation of current BPNT. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

On maximization of the modularity index in network psychometrics.

The modularity index (Q) is an important criterion for many community detection heuristics used in network psychometrics and its subareas (e.g., exploratory graph analysis). Some heuristics seek to directly maximize Q, whereas others, such as the walktrap algorithm, only use the modularity index post hoc to determine the number of communities. Researchers in network psychometrics have typically not employed methods that are guaranteed to find a partition that maximizes Q, perhaps because of the complexity of the underlying mathematical programming problem. In this paper, for networks of the size commonly encountered in network psychometrics, we explore the utility of finding the partition that maximizes Q via formulation and solution of a clique partitioning problem (CPP). A key benefit of the CPP is that the number of communities is naturally determined by its solution and, therefore, need not be prespecified in advance. The results of two simulation studies comparing maximization of Q to two other methods that seek to maximize modularity (fast greedy and Louvain), as well as one popular method that does not (walktrap algorithm), provide interesting insights as to the relative performances of the methods with respect to identification of the correct number of communities and the recovery of underlying community structure.

A comparison of logistic regression methods for Ising model estimation.

The Ising model has received significant attention in network psychometrics during the past decade. A popular estimation procedure is IsingFit, which uses nodewise l1-regularized logistic regression along with the extended Bayesian information criterion to establish the edge weights for the network. In this paper, we report the results of a simulation study comparing IsingFit to two alternative approaches: (1) a nonregularized nodewise stepwise logistic regression method, and (2) a recently proposed global l1-regularized logistic regression method that estimates all edge weights in a single stage, thus circumventing the need for nodewise estimation. MATLAB scripts for the methods are provided as supplemental material. The global l1-regularized logistic regression method generally provided greater accuracy and sensitivity than IsingFit, at the expense of lower specificity and much greater computation time. The stepwise approach showed considerable promise. Relative to the l1-regularized approaches, the stepwise method provided better average specificity for all experimental conditions, as well as comparable accuracy and sensitivity at the largest sample size.

Focusing Narrowly on Model Fit in Factor Analysis Can Mask Construct Heterogeneity and Model Misspecification: Applied Demonstrations across Sample and Assessment Types.

This study builds upon research indicating that focusing narrowly on model fit when evaluating factor analytic models can lead to problematic inferences regarding the nature of item sets, as well as how models should be applied to inform measure development and validation. To advance research in this area, we present concrete examples relevant to researchers in clinical, personality, and related subfields highlighting two specific scenarios when an overreliance on model fit may be problematic. Specifically, we present data analytic examples showing that focusing narrowly on model fit may lead to (a) incorrect conclusions that heterogeneous item sets reflect narrower homogeneous constructs and (b) the retention of potentially problematic items when developing assessment measures. We use both interview data from adult outpatients (N = 2,149) and self-report data from adults recruited online (N = 547) to demonstrate the importance of these issues across sample types and assessment methods. Following demonstrations with these data, we make recommendations focusing on how other model characteristics (e.g., factor loading patterns; carefully considering the content and nature of factor indicators) should be considered in addition to information provided by model fit indices when evaluating factor analytic models.

Psychopathy facets, perceived power, and forms of aggression.

Despite research indicating that exerting dominance and control is characteristic of psychopathy, no research has examined the role that feelings of and desire for power plays in psychopathy-related aggression. Borrowing from various literatures and novel conceptualizations, we investigated the contributions of feeling powerful and/or desiring power and distinct psychopathy facets in explaining aggression manifested in different forms (i.e., physical, verbal, indirect) across 4 samples. Results from regression analyses within each sample and a meta-analysis across the samples indicated that the impulsive facet of psychopathy was generally related to multiple forms of aggression, and the unique variance in the affective facet was primarily related to physical aggression across samples. In contrast, the unique variance of the interpersonal facet showed a primary relationship with indirect aggression (e.g., relational, passive). Desiring power made unique contributions in relation to multiple forms of aggression, whereas feeling powerful was generally unrelated and/or negatively related to aggression. In sum, the unique variance in the psychopathy facets showed fairly specialized relationships with forms of aggression, and desire for power may be an independent explanatory construct for multiple forms of aggression proneness. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

New approaches to deep phenotyping in addictions.

OBJECTIVE: The causes of substance use disorders (SUDs) are largely unknown and the effectiveness of their treatments is limited. One crucial impediment to research and treatment progress surrounds how SUDs are classified and diagnosed. Given the substantial heterogeneity among individuals diagnosed with a given SUD (e.g., alcohol use disorder [AUD]), identifying novel research and treatment targets and developing new study designs is daunting. METHOD: In this article, we review and integrate two recently developed frameworks, the National Institute on Drug Abuse's Phenotyping Assessment Battery (NIDA PhAB) and the Hierarchical Taxonomy of Psychopathology (HiTOP), that hope to accelerate progress in understanding the causes and consequences of psychopathology by means of deep phenotyping, or finer-grained analysis of phenotypes. RESULTS AND CONCLUSIONS: NIDA PhAB focuses on addiction-related processes across multiple units of analysis, whereas HiTOP focuses on clinical phenotypes and covers a broader range of psychopathology. We highlight that NIDA PhAB and HiTOP together provide deep and broad characterizations of people diagnosed with SUDs and complement each other in their efforts to address widely known limitations of traditional classification systems and their diagnostic categories. Next, we show how NIDA PhAB and HiTOP can be integrated to facilitate optimal rich phenotyping of addiction-related phenomena. Finally, we argue that such deep phenotyping promises to advance our understanding of the neurobiology of SUD and addiction, which will guide the development of personalized medicine and interventions. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A broad internalizing dimension accounts for the genetic associations between personality and individual internalizing disorders.

Much research has demonstrated that psychopathology can be described in terms of broad dimensions, representing liability for multiple psychiatric disorders. Broad spectra of psychopathology (e.g., internalizing and externalizing) are increasingly used as targets for research investigating the development, etiology, and course of psychopathology because they account for patterns of relatedness among disorders that were once presumed distinct. Thus, these spectra represent alluring targets due to their comprehensive and parsimonious nature. Nevertheless, little research has established the role of individual disorders over and above broad dimensions in the study of psychopathology. In the current study, we investigate whether there are unique etiological associations between individual internalizing disorders and personality traits after accounting for their etiological associations with a broad internalizing dimension. We used a community sample of twins (Npairs = 448) ages 4 to 19 to examine the etiological associations between internalizing psychopathology and Big Five personality dimensions. In terms of genetic covariation, a broad internalizing dimension was positively associated with neuroticism and negatively associated with extraversion, agreeableness, and conscientiousness. Moreover, internalizing accounted for most of the genetic variance shared between individual internalizing disorders and personality traits. Nevertheless, there were unique genetic associations between the following pairs of personality traits and disorders: neuroticism and social anxiety, extraversion and social anxiety, agreeableness and depression, and conscientiousness and compulsions. There was little evidence of environmental influences shared between internalizing and personality. In sum, a broad internalizing dimension adequately accounted for almost all of the etiologic covariation between internalizing disorders and personality, with several interesting exceptions. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

How Robust Is the p Factor? Using Multitrait-Multimethod Modeling to Inform the Meaning of General Factors of Youth Psychopathology.

We used multitrait-multimethod (MTMM) modeling to examine general factors of psychopathology in three samples of youth (Ns = 2119, 303, 592) for whom three informants reported on the youth's psychopathology (e.g., child, parent, teacher). Empirical support for the p-factor diminished in multi-informant models compared with mono-informant models: the correlation between externalizing and internalizing factors decreased and the general factor in bifactor models essentially reflected externalizing. Widely used MTMM-informed approaches for modeling multi-informant data cannot distinguish between competing interpretations of the patterns of effects we observed, including that the p-factor reflects, in part, evaluative consistency bias or that psychopathology manifests differently across contexts (e.g., home vs. school). Ultimately, support for the p-factor may be stronger in mono-informant designs, although it is does not entirely vanish in multi-informant models. Instead, the general factor of psychopathology in any given mono-informant model likely reflects a complex mix of variances, some substantive and some methodological.

Conceptual, methodological, and measurement factors that disqualify use of measurement invariance techniques to detect informant discrepancies in youth mental health assessments.

On page 1 of his classic text, Millsap (2011) states, "Measurement invariance is built on the notion that a measuring device should function the same way across varied conditions, so long as those varied conditions are irrelevant [emphasis added] to the attribute being measured." By construction, measurement invariance techniques require not only detecting varied conditions but also ruling out that these conditions inform our understanding of measured domains (i.e., conditions that do not contain domain-relevant information). In fact, measurement invariance techniques possess great utility when theory and research inform their application to specific, varied conditions (e.g., cultural, ethnic, or racial background of test respondents) that, if not detected, introduce measurement biases, and, thus, depress measurement validity (e.g., academic achievement and intelligence). Yet, we see emerging bodies of work where scholars have "put the cart before the horse" when it comes to measurement invariance, and they apply these techniques to varied conditions that, in fact, may reflect domain-relevant information. These bodies of work highlight a larger problem in measurement that likely cuts across many areas of scholarship. In one such area, youth mental health, researchers commonly encounter a set of conditions that nullify the use of measurement invariance, namely discrepancies between survey reports completed by multiple informants, such as parents, teachers, and youth themselves (i.e., informant discrepancies). In this paper, we provide an overview of conceptual, methodological, and measurement factors that should prevent researchers from applying measurement invariance techniques to detect informant discrepancies. Along the way, we cite evidence from the last 15 years indicating that informant discrepancies reflect domain-relevant information. We also apply this evidence to recent uses of measurement invariance techniques in youth mental health. Based on prior evidence, we highlight the implications of applying these techniques to multi-informant data, when the informant discrepancies observed within these data might reflect domain-relevant information. We close by calling for a moratorium on applying measurement invariance techniques to detect informant discrepancies in youth mental health assessments. In doing so, we describe how the state of the science would need to fundamentally "flip" to justify applying these techniques to detect informant discrepancies in this area of work.

Model fit is a fallible indicator of model quality in quantitative psychopathology research: A reply to Bader and Moshagen.

As evidenced by our exchange with Bader and Moshagen (2022), the degree to which model fit indices can and should be used for the purpose of model selection remains a contentious topic. Here, we make three core points. First, we discuss the common misconception about fit statistics' abilities to identify the "best model," arguing that mechanical application of model fit indices contributes to faulty inferences in the field of quantitative psychopathology. We illustrate the consequences of this practice through examples in the literature. Second, we highlight the parsimony-adjacent concept of fitting propensity, which is not accounted for by commonly used fit statistics. Finally, we present specific strategies to overcome interpretative bias and increase generalizability of study results and stress the importance of carefully balancing substantive and statistical criteria in model selection scenarios. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A modified approach to fitting relative importance networks.

Most researchers have estimated the edge weights for relative importance networks using a well-established measure of general dominance for multiple regression. This approach has several desirable properties including edge weights that represent R² contributions, in-degree centralities that correspond to R² for each item when using other items as predictors, and strong replicability. We endorse the continued use of relative importance networks and believe they have a valuable role in network psychometrics. However, to improve their utility, we introduce a modified approach that uses best-subsets regression as a preceding step to select an appropriate subset of predictors for each item. The benefits of this modification include: (a) computation time savings that can enable larger relative importance networks to be estimated, (b) a principled approach to edge selection that can significantly improve specificity, (c) the provision of a signed network if desired, (d) the potential use of the best-subsets regression approach for estimating Gaussian graphical models, and (e) possible generalization to best-subsets logistic regression for Ising models. We describe, evaluate, and demonstrate the proposed approach and discuss its strengths and limitations. (PsycInfo Database Record (c) 2022 APA, all rights reserved).