Detecting non-content-based response styles in survey data: An application of mixture factor analysis.

It is common for some participants in self-report surveys to be careless, inattentive, or lacking in effort. Data quality can be severely compromised by responses that are not based on item content (non-content-based [nCB] responses), leading to strong biases in the results of data analysis and misinterpretation of individual scores. In this study, we propose a specification of factor mixture analysis (FMA) to detect nCB responses. We investigated the usefulness and effectiveness of the FMA model in detecting nCB responses using both simulated data (Study 1) and real data (Study 2). In the first study, FMA showed reasonably robust sensitivity (.60 to .86) and excellent specificity (.96 to .99) on mixed-worded scales, suggesting that FMA had superior properties as a screening tool under different sample conditions. However, FMA performance was poor on scales composed of only positive items because of the difficulty in distinguishing acquiescent patterns from valid responses representing high levels of the trait. In Study 2 (real data), FMA detected a minority of cases (6.5%) with highly anomalous response patterns. Removing these cases resulted in a large increase in the fit of the unidimensional model and a substantial reduction in spurious multidimensionality.

Psychometric properties of the Alcohol Use Disorders Identification Test (AUDIT) across cross-cultural subgroups, genders, and sexual orientations: Findings from the International Sex Survey (ISS).

INTRODUCTION: Despite being a widely used screening questionnaire, there is no consensus on the most appropriate measurement model for the Alcohol Use Disorders Identification Test (AUDIT). Furthermore, there have been limited studies on its measurement invariance across cross-cultural subgroups, genders, and sexual orientations. AIMS: The present study aimed to examine the fit of different measurement models for the AUDIT and its measurement invariance across a wide range of subgroups by country, language, gender, and sexual orientation. METHODS: Responses concerning past-year alcohol use from the participants of the cross-sectional International Sex Survey were considered (N = 62,943; Mage: 32.73; SD = 12.59). Confirmatory factor analysis, as well as measurement invariance tests were performed for 21 countries, 14 languages, three genders, and four sexual-orientation subgroups that met the minimum sample size requirement for inclusion in these analyses. RESULTS: A two-factor model with factors describing 'alcohol use' (items 1-3) and 'alcohol problems' (items 4-10) showed the best model fit across countries, languages, genders, and sexual orientations. For the former two, scalar and latent mean levels of invariance were reached considering different criteria. For gender and sexual orientation, a latent mean level of invariance was reached. CONCLUSIONS: In line with the two-factor model, the calculation of separate alcohol-use and alcohol-problem scores is recommended when using the AUDIT. The high levels of measurement invariance achieved for the AUDIT support its use in cross-cultural research, capable also of meaningful comparisons among genders and sexual orientations.

Wording Effects in Assessment: Missing the Trees for the Forest.

This article examines wording effects when positive and negative worded items are included in psychological assessment. Wordings effects have been analyzed in the literature using statistical approaches based on population homogeneity assumptions (i.e. CFA, SEM), commonly adopting the bifactor model to separate trait variance and wording effects. This article presents an alternative approach by explicitly modeling population heterogeneity through a latent profile model, based on the idea that a subset of individuals exhibits wording effects. This kind of mixture model allows simultaneously to classify respondents, substantively characterize the differences in their response profiles, and report respondents' results in a comparable manner. Using the Rosenberg's self-esteem scale data from the LISS Panel (N = 6,762) in three studies, we identify a subgroup of participants who respond differentially according to item-wording and examine the impact of its responses in the estimation of the RSES measurement model, in terms of global and individual fit, under one-factor and bifactor models.The results of these analyses support the interpretation of wording effects in terms of a theoretically-proposed differential pattern of response to positively and negatively worded items, introducing a valuable tool for examining the artifactual or substantive interpretations of such wording effects.

An in-depth psychometric analysis of the Connor-Davidson Resilience Scale: calibration with Rasch-Andrich model.

BACKGROUND: The Connor-Davidson Resilience Scale (CD-RISC) is inarguably one of the best-known instruments in the field of resilience assessment. However, the criteria for the psychometric quality of the instrument were based only on classical test theory. METHOD: The aim of this paper has focused on the calibration of the CD-RISC with a nonclinical sample of 444 adults using the Rasch-Andrich Rating Scale Model, in order to clarify its structure and analyze its psychometric properties at the level of item. RESULTS: Two items showed misfit to the model and were eliminated. The remaining 22 items form basically a unidimensional scale. The CD-RISC has good psychometric properties. The fit of both the items and the persons to the Rasch model was good, and the response categories were functioning properly. Two of the items showed differential item functioning. CONCLUSIONS: The CD-RISC has an obvious ceiling effect, which suggests to include more difficult items in future versions of the scale.

Psychometric evidence of the Acceptance and Action Questionnaire-II (AAQ-II): an item response theory analysis in university students from Chile.

BACKGROUND: Experiential avoidance (EA) is a psychological mechanism associated with several mental health disorders and is regarded as a relevant target by third-generation cognitive behavioral therapies. It has been mainly assessed through self-report questionnaires, and the AAQ-II is the most used tool. Its psychometric evidence has been mostly tested through the classical test theory (CTT) and very scarcely assessed through Item Response Theory (IRT). METHODS: We used the Graded Response Model to examine its psychometric properties in Spanish-speaking university students (n = 1503; women = 995 (66.2%), mean age = 19.29, SD = 2.45). We tested whether the empirical data fit the model's predictions and estimated the dispersion of persons and items along the experiential avoidance continuum. Moreover, we examined category probability curves to identify the response probability of each answer. Likewise, an item-person map was made where the measurement of persons and items, both on the same scale and along the experiential avoidance continuum, could be observed jointly. Finally, we tested the gender invariance of the scale. RESULTS: We found that the values of the individuals and the items were in the established range to be considered an adequate measure of EA. Additionally, we observed high discrimination indices for all items. The current version with seven answer options could not be optimal and should be tested in future studies. Finally, we found evidence of differential functioning by gender in one of the seven items of the instrument. CONCLUSIONS: Our results indicate that the AAQ-II is a suitable tool for measuring EA and accurately classifying and differentiating EA levels in university students.

Why Do People Watch Pornography? Cross-Cultural Validation of the Pornography Use Motivations Scale (PUMS) and Its Short Form (PUMS-8).

Motivations for pornography use may vary across gender identities, sexual orientations, and geographical regions, warranting examination to promote individual and public health. The aims of this study were to validate the Pornography Use Motivations Scale (PUMS) in a diverse, multicultural sample, and develop a short form (PUMS-8) that can assess a wide range of pornography use motivations. Using data from 42 countries (N = 75,117; Mage = 32.07; SDage = 12.37), enabled us to thoroughly evaluate the dimensionality, validity, and reliability of the Pornography Use Motivations Scale (PUMS), leading to the development of the more concise PUMS-8 short scale. Additionally, language-, nationality-, gender-, and sexual-orientation-based measurement invariance tests were conducted to test the comparability across groups. Both the PUMS and the PUMS-8 assess eight pornography use motivations, and both demonstrated excellent psychometric properties. Sexual Pleasure emerged as the most frequent motivation for pornography use across countries, genders, and sexual orientations, while differences were observed concerning other motivations (e.g. self-exploration was more prevalent among gender-diverse individuals than men or women). The motivational background of pornography use showed high similarity in the examined countries. Both the PUMS and the PUMS-8 are reliable and valid measurement tools to assess different types of motivations for pornography use across countries, genders, and sexual orientations. Both scales are recommended for use in research and clinical settings.

Evaluating the factor structure and measurement invariance of the 20-item short version of the UPPS-P Impulsive Behavior Scale across multiple countries, languages, and gender identities.

The UPPS-P Impulsive Behavior Model and the various psychometric instruments developed and validated based on this model are well established in clinical and research settings. However, evidence regarding the psychometric validity, reliability, and equivalence across multiple countries of residence, languages, or gender identities, including gender-diverse individuals, is lacking to date. Using data from the International Sex Survey (N = 82,243), confirmatory factor analyses and measurement invariance analyses were performed on the preestablished five-factor structure of the 20-item short version of the UPPS-P Impulsive Behavior Scale to examine whether (a) psychometric validity and reliability and (b) psychometric equivalence hold across 34 country-of-residence-related, 22 language-related, and three gender-identity-related groups. The results of the present study extend the latter psychometric instrument's well-established relevance to 26 countries, 13 languages, and three gender identities. Most notably, psychometric validity and reliability were evidenced across nine novel translations included in the present study (i.e., Croatian, English, German, Hebrew, Korean, Macedonian, Polish, Portuguese-Portugal, and Spanish-Latin American) and psychometric equivalence was evidenced across all three gender identities included in the present study (i.e., women, men, and gender-diverse individuals).

Cross-cultural validation and measurement invariance of anxiety and depression symptoms: A study of the Brief Symptom Inventory (BSI) in 42 countries.

BACKGROUND: Depression and anxiety are among the most prevalent mental health issues experienced worldwide. However, whereas cross-cultural studies utilize psychometrically valid and reliable scales, fewer can meaningfully compare these conditions across different groups. To address this gap, the current study aimed to psychometrically assess the Brief Symptomatology Index (BSI) in 42 countries. METHODS: Using data from the International Sex Survey (N = 82,243; Mage = 32.39; SDage = 12.52; women: n = 46,874; 57 %), we examined the reliability of depression and anxiety symptom scores of the BSI-18, as well as evaluated evidence of construct, invariance, and criterion-related validity in predicting clinically relevant variables across countries, languages, genders, and sexual orientations. RESULTS: Results corroborated an invariant, two-factor structure across all groups tested, exhibiting excellent reliability estimates for both subscales. The 'caseness' criterion effectively discriminated among those at low and high risk of depression and anxiety, yielding differential effects on the clinical criteria examined. LIMITATIONS: The predictive validation was not made against a clinical diagnosis, and the full BSI-18 scale was not examined (excluding the somatization sub-dimension), limiting the validation scope of the BSI-18. Finally, the study was conducted online, mainly by advertisements through social media, ultimately skewing our sample towards women, younger, and highly educated populations. CONCLUSIONS: The results support that the BSI-12 is a valid and reliable assessment tool for assessing depression and anxiety symptoms across countries, languages, genders, and sexual orientations. Further, its caseness criterion can discriminate well between participants at high and low risk of depression and anxiety.

Cross-Cultural Adult ADHD Assessment in 42 Countries Using the Adult ADHD Self-Report Scale Screener.

OBJECTIVE: We analyzed adult ADHD symptoms in a cross-cultural context, including investigating the occurrence and potential correlates of adult ADHD and psychometric examination of the Adult ADHD Self-Report Scale (ASRS) Screener. METHOD: Our analysis is based on a large-scale research project involving 42 countries (International Sex Survey, N=72,627, 57% women, Mage=32.84; SDage=12.57). RESULTS: The ASRS Screener demonstrated good reliability and validity, along with partial invariance across different languages, countries, and genders. The occurrence of being at risk for adult ADHD was relatively high (21.4% for women, 18.1% for men). The highest scores were obtained in the US, Canada, and other English-speaking Western countries, with significantly lower scores among East Asian and non-English-speaking European countries. Moreover, ADHD symptom severity and occurrence were especially high among gender-diverse individuals. Significant associations between adult ADHD symptoms and age, mental and sexual health, and socioeconomic status were observed. CONCLUSIONS: Present results show significant cross-cultural variability in adult ADHD occurrence as well as highlight important factors related to adult ADHD. Moreover, the importance of further research on adult ADHD in previously understudied populations (non-Western countries) and minority groups (gender-diverse individuals) is stressed. Lastly, the present analysis is consistent with previous evidence showing low specificity of adult ADHD screening instruments and contributes to the current discussion on accurate adult ADHD screening and diagnosis.

The short version of the Sexual Distress Scale (SDS-3): Measurement invariance across countries, gender identities, and sexual orientations.

Background: The three-item Sexual Distress Scale (SDS-3) has been frequently used to assess distress related to sexuality in public health surveys and research on sexual wellbeing. However, its psychometric properties and measurement invariance across cultural, gender and sexual subgroups have not yet been examined. This multinational study aimed to validate the SDS-3 and test its psychometric properties, including measurement invariance across language, country, gender identity, and sexual orientation groups. Methods: We used global survey data from 82,243 individuals (Mean age=32.39 years; 40.3 % men, 57.0 % women, 2.8 % non-binary, and 0.6 % other genders) participating in the International Sexual Survey (ISS; https://internationalsexsurvey.org/) across 42 countries and 26 languages. Participants completed the SDS-3, as well as questions regarding sociodemographic characteristics, including gender identity and sexual orientation. Results: Confirmatory factor analysis (CFA) supported a unidimensional factor structure for the SDS-3, and multi-group CFA (MGCFA) suggested that this factor structure was invariant across countries, languages, gender identities, and sexual orientations. Cronbach's α for the unidimensional score was 0.83 (range between 0.76 and 0.89), and McDonald's ω was 0.84 (range between 0.76 and 0.90). Participants who did not experience sexual problems had significantly lower SDS-3 total scores (M = 2.99; SD=2.54) compared to those who reported sexual problems (M = 5.60; SD=3.00), with a large effect size (Cohen's d = 1.01 [95 % CI=-1.03, -0.98]; p < 0.001). Conclusion: The SDS-3 has a unidimensional factor structure and appears to be valid and reliable for measuring sexual distress among individuals from different countries, gender identities, and sexual orientations.

The ephemeral nature of wording effects.

This article explores the analysis and interpretation of wording effects associated with using direct and reverse items in psychological assessment. Previous research using bifactor models has suggested a substantive nature of this effect. The present study uses mixture modeling to systematically examine an alternative hypothesis and surpass recognized limitations in the bifactor modeling approach. In preliminary supplemental Studies S1 and S2, we examined the presence of participants who exhibited wording effects and evaluated their impact on the dimensionality of Rosenberg's Self-Esteem and the Revised Life Orientation Test, confirming the ubiquity of wording effects in scales containing direct and reverse items. Then, after analyzing the data for both scales (n = 5,953), we found that, despite a significant association between wording factors (Study 1), a low proportion of participants simultaneously exhibited asymmetric responses in both scales (Study 2). Similarly, despite finding both longitudinal invariance and temporal stability of this effect in three waves (n = 3,712, Study 3), a small proportion of participants was identified with asymmetric responses over time (Study 4), reflected in lower transition parameters compared to the other patterns of profiles examined. In both cases, we illustrate how bifactor models capitalize on the responses of individuals who do not even exhibit wording effects, yielding spurious correlations suggesting a substantive nature of the wording effect. These findings support the notion of an ephemeral nature underlying wording effects. The discussion focuses on alternative hypotheses to understand these findings and emphasizes the utility of including reverse items in psychological assessment. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The eleven-item Alcohol, Smoking and Substance Involvement Screening Test (ASSIST-11): Cross-cultural psychometric evaluation across 42 countries.

The Alcohol, Smoking and Substance Involvement Screening Test (ASSIST) is an instrument to screen substance-use-related health risks. However, little is known whether the ASSIST could be further shortened while remaining psychometrically sound across different countries, languages, gender identities, and sexual-orientation-based groups. The study aimed to validate a shortened 11-item ASSIST (ASSIST-11). Using the International Sex Survey data, 82,243 participants (M age = 32.39 years) across 42 countries and 26 languages completed questions from the ASSIST-11 regarding gender identity, sexual orientation, and other information. Confirmatory factor analysis (CFA) and multigroup CFA (MGCFA) evaluated the ASSIST-11's structure and tested measurement invariance across groups. Cronbach's α and McDonald's ω were used to examine the internal consistency. Cohen's d and independent t-tests were used to examine known-group validity. The ASSIST-11 was unidimensional across countries, languages, age groups, gender identities (i.e., men, women, and gender-diverse individuals), and sexual orientations (i.e., heterosexual and sexual minority individuals). Cronbach's α was 0.63 and McDonald's ω was 0.68 for the ASSIST-11. Known-group validity was supported by Cohen's d (range between 0.23 and 0.40) with significant differences (p-values<0.001). The ASSIST-11 is a modified instrument with a unidimensional factor structure across different languages, age groups, countries, gender identities, and sexual orientations. The low internal consistency of the ASSIST-11 might be acceptable as it assesses a broad concept (i.e., use of several different substances). Healthcare providers and researchers may use the ASSIST-11 to quickly assess substance-use information from general populations and evaluate the need to follow up with more detailed questions about substance use.

Habituation of the eyeblink response in humans with stimuli presented in a sequence of incremental intensity.

In an experiment we examined whether the repeated presentation of tones of gradually increasing intensities produces greater decrement in the eyeblink reflex response in humans than the repetition of tones of constant intensities. Two groups of participants matched for their initial level of response were exposed to 110 tones of 100-ms duration. For the participants in the incremental group, the tones increased from 60- to 90- dB in 3-dB steps, whereas participants in the constant group received the tones at a fixed 90-dB intensity. The results indicated that the level of response in the last block of 10 trials, in which both groups received 90-dB tones, was significantly lower in the incremental group than in the constant group. These findings support the data presented by Davis and Wagner (7) with the acoustic response in rats, but differ from several reports with autonomic responses in humans, where the advantage of the incremental condition has not been observed unambiguously. The discussion analyzes theoretical approaches to this phenomenon and the possible involvement of separate neural circuits.

Can the stimulus processing assumptions of the sometimes-opponent-process (SOP) model explain instances of contextual learning?

One of the most persisting assertions in Allan Wagner's view of conditioning is that the environment or context in which significant events occur can develop an association with these events, more or less in the same way as conditioned and unconditioned stimuli become associated with each other. He was drawn to this idea by evidence of contextual fear conditioning, contingency effects, some instances of context-specificity of long-term habituation, and latent inhibition. From a theoretical point of view, however, homologizing contexts to conditioned stimuli is not as simple as it seems, especially when quantitative theories are involved, as is the case of Wagner's work. It might be, for instance, that contexts cannot be represented merely as long-duration conditioned stimuli, in which case, no net contextual learning can occur due to the context being less correlated with reinforcement than with nonreinforcement. In this article, we use Wagner's sometimes-opponent-process model of conditioning to comment on the quantitative nature of this challenge. Also, based on an idea sketched by Mazur and Wagner, we describe a set of quantitative strategies that might be usefully considered to solve this dilemma within the general framework of Wagner's theory. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

A valid and reliable measure of nothing: disentangling the "Gavagai effect" in survey data.

BACKGROUND: In three recent studies, Maul demonstrated that sets of nonsense items can acquire excellent psychometric properties. Our aim was to find out why responses to nonsense items acquire a well-defined structure and high internal consistency. METHOD: We designed two studies. In the first study, 610 participants responded to eight items where the central term (intelligence) was replaced by the term "gavagai". In the second study, 548 participants responded to seven items whose content was totally invented. We asked the participants if they gave any meaning to "gavagai", and conducted analyses aimed at uncovering the most suitable structure for modeling responses to meaningless items. RESULTS: In the first study, 81.3% of the sample gave "gavagai" meaning, while 18.7% showed they had given it no interpretation. The factorial structures of the two groups were very different from each other. In the second study, the factorial model fitted almost perfectly. However, further analysis revealed that the structure of the data was not continuous but categorical with three unordered classes very similar to midpoint, disacquiescent, and random response styles. DISCUSSION: Apparently good psychometric properties on meaningless scales may be due to (a) respondents actually giving an interpretation to the item and responding according to that interpretation, or (b) a false positive because the statistical fit of the factorial model is not sensitive to cases where the actual structure of the data does not come from a common factor. In conclusion, the problem is not in factor analysis, but in the ability of the researcher to elaborate substantive hypotheses about the structure of the data, to employ analytical procedures congruent with those hypotheses, and to understand that a good fit in factor analysis does not have a univocal interpretation and is not sufficient evidence of either validity nor good psychometric properties.

The development and present status of the SOP model of associative learning.

The Sometimes Opponent Processes (SOP) model in its original form was especially calculated to address how expected unconditioned stimulus (US) and conditioned stimulus (CS) are rendered less effective than their novel counterparts in Pavlovian conditioning. Its several elaborations embracing the essential notion have extended the scope of the model to integrate a much greater number of phenomena of Pavlovian conditioning. Here, we trace the development of the model and add further thoughts about its extension and refinement.

A Quantitative Account of the Behavioral Characteristics of Habituation: The Sometimes Opponent Processes Model of Stimulus Processing.

Habituation is defined as a decline in responding to a repeated stimulus. After more than 80 years of research, there is an enduring consensus among researchers on the existence of 9-10 behavioral regularities or parameters of habituation. There is no similar agreement, however, on the best approach to explain these facts. In this paper, we demonstrate that the Sometimes Opponent Processes (SOP) model of stimulus processing accurately describes all of these regularities. This model was proposed by Allan Wagner as a quantitative elaboration of priming theory, which states that the processing of a stimulus, and therefore its capacity to provoke its response, depends inversely on the degree to which the stimulus is pre-represented in short-term memory. Using computer simulations, we show that all the facts involving within-session effects or short-term habituation might be the result of priming from recent presentations of the stimulus (self-generated priming). The characteristics involving between-sessions effects or long-term habituation would result from the retrieval of the representation of the stimulus from memory by the associated context (associatively generated priming).

Bifactor Models of Attention-Deficit/Hyperactivity Disorder (ADHD): An Evaluation of Three Necessary but Underused Psychometric Indexes.

BACKGROUND: In the past decade, the bifactor model of attention-deficit/hyperactivity disorder (ADHD) has been extensively researched. This model consists of an ADHD general dimension and two specific factors: inattention and hyperactivity/impulsivity. All studies conclude that the bifactor is superior to the traditional two-correlated factors model, according to the fit obtained by factor analysis. However, the proper interpretation of a bifactor not only depends on the fit but also on the quality of the measurement model. OBJECTIVE: To evaluate the model-based reliability, distribution of common variance and construct replicability of general and specific ADHD factors. METHOD: We estimated expected common variance, omega hierarchical/subscale and H-index from standardized factor loadings of 31 ADHD bifactor models previously published. RESULTS AND CONCLUSION: The ADHD general factor explained most of the common variance. Given the low reliable variance ratios, the specific factors were difficult to interpret. However, in clinical samples, inattention acquired sufficient specificity and stability for interpretation beyond the general factor. Implications for research and clinical practice are discussed.

A theoretical analysis of transfer of occasion setting: SOP with replaced elements.

The available data on occasion setting led Susan Brandon and Allan Wagner (Brandon and Wagner, 1998; Wagner and Brandon, 2001) to formulate what has come to be known as a replaced-elements conception (REM) of context-dependent cues within the SOP model (Wagner, 1981). In the present paper, we review the development of the theory, and show how, with a few congenial assumptions about shared cues, it can address some of the major regularities concerning when the transfer of occasion setting does or does not occur. Among the particular examples are the relatively unique transfers that have been reported to occur between separate serial discriminations and between targets that have been trained with the same versus different reinforcers.

General and specific attention-deficit/hyperactivity disorder factors of children 4 to 6 years of age: An exploratory structural equation modeling approach to assessing symptom multidimensionality.

We tested first-order factor and bifactor models of attention-deficit/hyperactivity disorder (ADHD) using confirmatory factor analysis (CFA) and exploratory structural equation modeling (ESEM) to adequately summarize the Diagnostic and Statistical Manual of Mental Disorders, 4th Edition, (DSM-IV-TR) symptoms observed in a Spanish sample of preschoolers and kindergarteners. Six ESEM and CFA models were estimated based on teacher evaluations of the behavior of 638 children 4 to 6 years of age. An ESEM bifactor model with a central dimension plus 3 specific factors (inattention, hyperactivity, and impulsivity) showed the best fit and interpretability. Strict invariance between the sexes was observed. The bifactor model provided a solution to previously encountered inconsistencies in the factorial models of ADHD in young children. However, the low reliability of the specific factors casts doubt on the utility of the subscales for ADHD measurement. More research is necessary to clarify the nature of G and S factors of ADHD.