How well can we measure chronic pain impact in existing longitudinal cohort studies? Lessons learned.

Multiple large longitudinal cohorts provide opportunities to address questions about predictors of pain and pain trajectories, even when not anticipated in design of the historical databases. This focus article uses two empirical examples to illustrate the processes of assessing the measurement properties of data from large cohort studies to answer questions about pain. In both examples, data were screened to select candidate variables that captured the impact of chronic pain on self-care activities, productivity and social activities. We describe a series of steps to select candidate items and evaluate their psychometric characteristics in relation to the measurement of pain impact proposed. In UK Biobank, a general lack of internal consistency of variables selected prevented the identification of a satisfactory measurement model, with lessons for the measurement of chronic pain impact. In the English Longitudinal Study of Ageing, a measurement model for chronic pain impact was identified, albeit limited to capturing the impact of pain on self-care and productivity but lacking coverage related to social participation. In conjunction with its supplementary material, this focus article aims to encourage exploration of these valuable prospectively collected data; to support researchers to make explicit the relationships between items in the databases and constructs of interest in pain research; and to use empirical methods to estimate the possible biases in these variables. PERSPECTIVE: This focus article outlines a theory-driven approach for fitting new measurement models to data from large cohort studies, and evaluating their psychometric properties. This aims to help researchers develop an empirical understanding of the gains and limitations connected with the process of re-purposing the data stored in these datasets.

Using the multidimensional nominal response model to model faking in questionnaire data: The importance of item desirability characteristics.

Faking in self-report personality questionnaires describes a deliberate response distortion aimed at presenting oneself in an overly favorable manner. Unless the influence of faking on item responses is taken into account, faking can harm multiple psychometric properties of a test. In the present article, we account for faking using an extension of the multidimensional nominal response model (MNRM), which is an item response theory (IRT) model that offers a flexible framework for modeling different kinds of response biases. Particularly, we investigated under which circumstances the MNRM can adequately adjust substantive trait scores and latent correlations for the influence of faking and examined the role of variation in the way item content is related to social desirability (i.e., item desirability characteristics) in facilitating the modeling of faking and counteracting its detrimental effects. Using a simulation, we found that the inclusion of a faking dimension in the model can overall improve the recovery of substantive trait person parameters and latent correlations between substantive traits, especially when the impact of faking in the data is high. Item desirability characteristics moderated the effect of modeling faking and were themselves associated with different levels of parameter recovery. In an empirical demonstration with N = 1070 test-takers, we also showed that the faking modeling approach in combination with different item desirability characteristics can prove successful in empirical questionnaire data. We end the article with a discussion of implications for psychological assessment.

Investigating Heterogeneity in Response Strategies: A Mixture Multidimensional IRTree Approach.

To improve the validity of self-report measures, researchers should control for response style (RS) effects, which can be achieved with IRTree models. A traditional IRTree model considers a response as a combination of distinct decision-making processes, where the substantive trait affects the decision on response direction, while decisions about choosing the middle category or extreme categories are largely determined by midpoint RS (MRS) and extreme RS (ERS). One limitation of traditional IRTree models is the assumption that all respondents utilize the same set of RS in their response strategies, whereas it can be assumed that the nature and the strength of RS effects can differ between individuals. To address this limitation, we propose a mixture multidimensional IRTree (MM-IRTree) model that detects heterogeneity in response strategies. The MM-IRTree model comprises four latent classes of respondents, each associated with a different set of RS traits in addition to the substantive trait. More specifically, the class-specific response strategies involve (1) only ERS in the "ERS only" class, (2) only MRS in the "MRS only" class, (3) both ERS and MRS in the "2RS" class, and (4) neither ERS nor MRS in the "0RS" class. In a simulation study, we showed that the MM-IRTree model performed well in recovering model parameters and class memberships, whereas the traditional IRTree approach showed poor performance if the population includes a mixture of response strategies. In an application to empirical data, the MM-IRTree model revealed distinct classes with noticeable class sizes, suggesting that respondents indeed utilize different response strategies.

A Review of Some of the History of Factorial Invariance and Differential Item Functioning.

The concept of factorial invariance has evolved since it originated in the 1930s as a criterion for the usefulness of the multiple factor model; it has become a form of analysis supporting the validity of inferences about group differences on underlying latent variables. The analysis of differential item functioning (DIF) arose in the literature of item response theory (IRT), where its original purpose was the detection and removal of test items that are differentially difficult for, or biased against, one subpopulation or another. The two traditions merge at the level of the underlying latent variable model, but their separate origins and different purposes have led them to differ in details of terminology and procedure. This review traces some aspects of the histories of the two traditions, ultimately drawing some conclusions about how analysts may draw on elements of both, and how the nature of the research question determines the procedures used. Whether statistical tests are grouped by parameter (as in studies of factorial invariance) or across parameters by variable (as in DIF analysis) depends on the context and is independent of the model, as are subtle aspects of the order of the tests. In any case in which DIF or partial invariance is a possibility, the invariant parameters, or anchor items in DIF analysis, are best selected in an interplay between the statistics and judgment about what is being measured.

Item response theory for before-after designs in interprofessional education research.

Although Item Response Theory (IRT) has been recommended for helping advance interprofessional education (IPE) research, its use remains limited. This may be partly explained by potential misconceptions regarding IRT`s "limitation" to cross-sectional data. The aim of this study is to demonstrate how Item Response Theory (IRT) can be applied effectively in before-and-after designs in IPE research. Specifically, a two-week before-after design with survey methodology using the Extended Professional Identity Scale (EPIS), an interprofessional identity measure, was conducted among n = 146 mixed health-science students. Results indicated that EPIS increased significantly before-after intervention by .74 standardised mean differences, t146 = 7.73, p < .05. The before-after IRT model also gave a test-retest reliability estimate of .60 which was considered acceptable. Comparison of the IRT model with a conventional paired-t-test indicated similar effect size estimates of Cohen's d = .56 and .54, respectively. We demonstrate IRT`s flexibility to before-after studies in IPE. Application of this model can yield accurate changes in target IPE constructs, and it is advantageous to classical test theory vis-à-vis baseline differences.

Cross-cultural adaptation of the awareness and beliefs about cancer measure for Hispanics/Latinos living in the United States.

Introduction: The purpose of this study is to culturally adapt the Awareness and Beliefs about Cancer (ABC) measure for use in the Hispanic/Latino population living in the United States (US). Methods: In accordance with Patient Reported Outcomes (PRO) Consortium guidelines for cross-cultural adaptation of measures for content and linguistic validity, we conducted: two forward-translations, reconciliation, two back-translations, revision and harmonization, six cognitive interviews, revision, external expert review, and finalization of the version. We used a mixed methods approach, conducting cognitive interviews with Hispanic/Latino community members while also convening an expert panel of six clinicians, health professionals, and community representatives and including the in the entire process. After cross-culturally adapting the ABC measure, we assessed the psychometric properties of the instrument using item response theory analysis. Item parameters, discrimination and category thresholds, and standard errors were calculated. For each of the adapted subdomains, we used item information curves to report the graphical profile of item effectiveness. Results: Twenty-two Hispanic/Latino community members were enrolled in cognitive interviews, and Hispanics/Latinos fluent in Spanish completed the measure to assess its psychometric properties. Cognitive interviews revealed opportunities to improve items. Key changes from the original measure include the inclusion of gender inclusive language and an inquiry into e-cigarette use on items related to smoking habits. Psychometric property analyses revealed that the anticipated delay in seeking medical help, general cancer beliefs, and cancer screening beliefs and behaviors subdomains had some slope parameters that were < 1; this implies that those items were not able to adequately discriminate the latent trait and had poor performance. Discussion: The adapted ABC measure for US Hispanics/Latinos meets content and linguistic validity standards, with construct validity confirmed for cancer symptom recognition and barriers to symptomatic presentation subdomains, but revisions are necessary for others, highlighting the need for ongoing refinement to ensure the cultural appropriateness of instruments.

Psychometric analysis and the implications for the use of the scoliosis research society questionnaire (SRS-22r English) for individuals with adolescent idiopathic scoliosis.

Background: Despite widespread usage of the SRS-22r questionnaire (Scoliosis Research Society Questionnaire-22r), the English version has only sparingly been subjected to analysis using modern psychometric techniques for patients with adolescent idiopathic scoliosis (AIS). The study purpose was to improve interpretation and clinical utility of the SRS-22r for adolescents with AIS by generating additional robust evidence, using modern statistical techniques. Questions about (1) Structure and (2) Item and Scale Functioning are addressed and interpreted for clinicians and researchers. Methods: This retrospective case review analyzed SRS-22r data collected from 1823 patients (mean age 14.9±2.2years) with a primary diagnosis of AIS who clinically completed an SRS-22r questionnaire.Individual SRS-22r questions and domain scores were retrieved through data queries. Patient information collected through chart review included diagnosis, age at assessment, sex, race and radiographic parameters. From 6044 SRS-22r assessments, 1 assessment per patient was randomly selected. Exploratory structural equation modeling (ESEM) and item response theory (IRT) techniques were used for data modeling, item calibration, and reliability assessment. Results: ESEM demonstrated acceptable fit to the data: χ2 (130)=343.73, p<.001; RMSEA=0.035; CFI=0.98; TLI=0.96; SRMR=0.02. Several items failed to adequately load onto their assigned factor. Item fit was adequate for all items except SRSq10 (Self-Image), SRSq16 (Mental Health), and SRSq20 (Mental Health). IRT models found item discriminations are within normal levels for items in psychological measures, except items SRSq1 (pain), SRSq2 (pain), and SRSq16 (mental health). Estimated reliability of the Function domain (ρ=0.69) was low, however, Pain, Self-Image and Mental Health domains exhibited high (ρ>0.80) reliability. Conclusions: Modern psychometric assessment of the SRS-22r, in adolescent patients with AIS, are presented and interpreted to assist clinicians and researchers in understanding its strengths and limitations. Overall, the SRS-22r demonstrated good psychometric properties in all domains except function. Cautious interpretation of the total score is suggested, as it does not reflect a single HRQoL construct.

Measurement Invariance of the Perceived Invalidation of Emotion Scale: An Examination Across Race/Ethnicity, Gender, Sexual Orientation, and Intersection of Identities.

The Perceived Invalidation of Emotions Scale (PIES), developed to measure emotional invalidation, could aid research efforts on various internalizing disorders and minority mental health. A prerequisite for its use includes its psychometric evaluation in diverse samples; thus, the current study aimed to evaluate the psychometric properties of the PIES in a combined sample of minoritized adults (N = 876). Results supported a unidimensional structure of the PIES that was invariant across the two minoritized samples, race/ethnicity, gender, sexual orientation, and intersections of race/ethnicity and sexual orientation. A reduced 7- and 4-item PIES with improved unidimensionality and consequentially more interpretable total scores were generated using item response theory analyses. Significant correlations observed between theoretically relevant constructs of adverse mental health outcomes and the PIES above and beyond identity-based discrimination supported the construct validity of the PIES. Implications include the disproportionate amount of emotional invalidation experienced by individuals with minoritized sexual orientation, which may reflect the recent increases in discrimination faced by these individuals.

Psychometric properties of computerized adaptive testing for chronic obstructive pulmonary disease patient-reported outcome measurement.

BACKGROUND: Computerized adaptive testing (CAT) is an effective way to reduce time, repetitious redundancy, and respond burden, and has been used to measure outcomes in many diseases. This study aimed to develop and validate a comprehensive disease-specific CAT for chronic obstructive pulmonary disease (COPD) patient-reported outcome measurement. METHODS: The discrimination and difficulty of the items from the modified patient-reported outcome scale for COPD (mCOPD-PRO) were analyzed using item response theory. Then the initial item, item selection method, ability estimation method, and stopping criteria were further set based on Concerto platform to form the CAT. Finally, the reliability and validity were validated. RESULTS: The item discrimination ranged from 1.05 to 2.71, and the item difficulty ranged from - 3.08 to 3.65. The measurement reliability of the CAT ranged from 0.910 to 0.922 using random method, while that ranged from 0.910 to 0.924 using maximum Fisher information (MFI) method. The content validity was good. The correlation coefficient between theta of the CAT and COPD assessment test and modified Medical Research Council dyspnea scale scores using random method was 0.628 and 0.540 (P < 0.001; P < 0.001) respectively, while that using MFI method was 0.347 and 0.328 (P = 0.007; P = 0.010) respectively. About 11 items (reducing by 59.3%) on average were tested using random method, while about seven items (reducing by 74.1%) on average using MFI method. The correlation coefficient between theta of the CAT and mCOPD-PRO total scores using random method was 0.919 (P < 0.001), while that using MFI method was 0.760 (P < 0.001). CONCLUSIONS: The comprehensive disease-specific CAT for COPD patient-reported outcome measurement is well developed with good psychometric properties, which can provide an efficient, accurate, and user-friendly measurement for patient-reported outcome of COPD.

Item response theory validation of the Oxford Knee Score and Activity and Participation Questionnaire: a step towards a common metric.

OBJECTIVE: The Oxford Knee Score (OKS) and OKS Activity and Participation Questionnaire (OKS-APQ) are patient reported outcome measures used to assess people undergoing knee replacement surgery. They have not explicitly been tested for unidimensionality (whether they measure one underlying trait such as 'knee health'). This study applied item response theory (IRT) to improve the validity of the instruments to optimise for ongoing use. STUDY DESIGN AND SETTING: Participants undergoing primary total knee replacement (TKR) provided pre-operative and post-operative responses for OKS and OKS-APQ. Confirmatory factor analyses (CFA) were performed on the OKS and OKS-APQ separately, and then on both when pooled into one. An IRT model was fitted to the data. RESULTS: 2972 individual response patterns were analysed. CFA demonstrated that when combining OKS and OKS-APQ as one instrument, they measure one latent health trait. A user-friendly, free-to-use, web app has been developed to allow clinicians to upload raw data and instantly receive IRT scores. CONCLUSIONS: The OKS and OKS-APQ can be combined to use effectively as a single instrument (producing a single score). For the separate OKS and OKS-APQ the original items and response options can continue to be posed to patients, and this study has confirmed the suitability of IRT-weighted scoring. Applying IRT to existing responses, converts traditional sum scores into continuous measurements with greater granularity including individual measurement error.

Pre-natal and early life lead exposure and childhood inhibitory control: an item response theory approach to improve measurement precision of inhibitory control.

BACKGROUND: Neurodevelopmental performance tasks are often separately analyzed, even when they tap into a similar construct. This may yield mixed findings for associations of an exposure-neurobehavioral outcome. We develop an item response theory (IRT) approach to integrate multiple task variables together to improve measurement precision of the underlying construct. We apply this approach to create an integrative measure of childhood inhibitory control, and study impacts of pre/post-natal lead exposure. METHODS: Using data from a prospective cohort based in Mexico (N = 533), we created an inhibitory control scale that integrates accuracy and reaction time information from four inhibitory control tasks (Go/NoGo Letter, Go/NoGo Neutral, Go/NoGo Happy, Delis-Kaplan Executive Function System (D-KEFS) Color-Word Interference Test, Condition 3). Using a generalized partial credit item response theory model, we estimated an inhibitory control index for each participant. We then assessed adjusted associations between umbilical cord blood and 4-year lead and childhood inhibitory control. We developed a resampling approach to incorporate error estimates from the inhibitory control variable to confirm the consistency of the lead-inhibitory control associations. We modeled time-varying associations of lead with each inhibitory control measure separately. RESULTS: Participants had a median age of 9 years; 51.4% were males. Umbilical cord blood [-0.06 (95% CI: -0.11, -0.01)] and 4-year lead [-0.07 (95% CI: -0.12, -0.02)] were associated with inhibitory control index at 8-10 years. A resampling approach confirmed that 4-year lead was consistently associated with childhood inhibitory control index. Umbilical cord blood and 4-year lead were each associated with 3 out of 8 measures in separate models. CONCLUSION: This is the first application of IRT in environmental epidemiology to create a latent variable for inhibitory control that integrates accuracy and reaction time information from multiple, related tasks. This framework can be applied to other correlated neurobehavioral assessments or other phenotype data.

A longitudinal examination of the psychometric properties of the English perceived stress scale- four (PSS-4) in mental health counsellors using item response theory.

We examined the longitudinal psychometric properties of the Perceived Stress Scale - 4 items version (PSS-4) using item response theory with a sample of 361 mental health counsellors. Participants completed the PSS-4 at three timepoints at six-month intervals in a one-year period. There were 290 participants who (80.3%) identified as female, 51 (14.1%) identified as male, eight (2.2%) identified as gender variant/non-conforming, seven (1.9%) wrote in their own gender identity (e.g., genderqueer, gender expansive), three (0.8%) identified as Transgender male, and two (0.6%) did not respond to the item. The racial and ethnic backgrounds were as follows: White (87.3%), Multiracial (5.5%), Latino or Hispanic or Spanish (2.8%), Black or African American (1.4%), Asian (0.8%), Middle Eastern (0.8%), and five did not respond to the item (1.4%). We found unidimensionality evidence of the PSS-4 across all three timepoints and response categories were monotonically ordered. We also found that across all timepoints, the average person location was lower than the average item location, suggesting that the PSS-4 may not be well-targeted for this sample of mental health counsellors. We observed no significant interactions between timepoints, hours worked per week, and length of employment. Implications of the findings, including a discussion of the utility of the PSS-4 as a global measure of stress and with mental health counsellors.

Non-Gaussian Liability Distribution for Depression in the General Population.

Unlike depression sum scores, the underlying risk for depression is typically assumed to be normally distributed across the general population. To assess the true empirical shape of depression risk, we created a continuous-valued estimate of the latent depression density, using the Davidian-Curve Item Response Theory (DC-IRT) and the National Health and Nutrition Examination Survey (NHANES) cohorts from 2005 to 2018 (n = 36,244 on the Nine-item Patient Health Questionnaire; PHQ-9). We conducted simulations to investigate the performance of DC-IRT for large samples and realistic items. The method can recover complex latent-risk distributions even when they are not evident from sum scores. However, estimation accuracy for different sample sizes depends on the method of model selection. In addition to full-data analysis, random samples of a few thousand observations were drawn for analysis. The latent shape of depression was left-skewed and bimodal in both investigations, indicating that the latent-normality assumption does not hold for depression.

Chinese version of the Death Anxiety Beliefs and Behaviours Scale: Psychometric properties based on CTT and IRT.

OBJECTIVES: The Death Anxiety Beliefs and Behaviour Scale (DABBS) is a unique tool designed to assess the detrimental beliefs and avoidant behaviors linked to death anxiety. This study aimed to adapt the DABBS into Chinese and verify its psychometric characteristics within a community-dwelling older adult population. METHODS: This study used a cross-sectional design and translated the DABBS into Chinese. The psychometric properties of 437 community-dwelling older persons were assessed using the Classical Test Theory (CTT) and Item Response Theory (IRT). RESULTS: The DABBS consisted of affect, beliefs, and behaviours, with 18 entries in 3 dimensions. The I-CVI of the DABBS ranged from 0.857 to 1.000, and the S-CVI was 0.968; Cronbach's alpha of 0.905. Rasch analysis results showed that the 3 dimensions of the scale possessed good unidimensionality, and the entries were well-fitted to the dimensions in which they were located; each entry Infit MNSQ and Outfit MNSQ were in the range of 0.50 to 1.50; the analysis of the functional differences of items in different characteristic subgroups (gender) showed that the absolute value of DIF Contrast was <0.50. The results of the Wright map showed that the ability of the participants was normally distributed, and the difficulty of the scale's entries was adapted to the average ability level of older adults. CONCLUSIONS: The present data indicate that the revised DABBS is a valid and reliable tool for assessing affect, beliefs, and behaviors associated with death anxiety in community-dwelling older individuals.

An Item Response Theory Analysis of the Psychosis Screening Questionnaire in Ethiopia.

AIM: Few psychosis screening instruments have been tested for use in Africa, yet appropriate tools can increase the detection of self-reported psychotic symptoms, improve the detection of psychosis and impact its prognosis. METHOD: The construct validity and factor structure of Psychosis Screening Questionnaire (PSQ) were tested with confirmatory factor analysis (CFA) and item response theory (IRT) in a sample of 1928 Ethiopian adults without any history of psychosis. We tested a unidimensional model with and without an item on mania. For IRT, unidimensional latent structure one-parameter logistic (1PL) and two-parameter (2PL) logistic models were tested and compared for relative fit using a likelihood-ratio test. RESULT AND DISCUSSION: The prevalence of lifetime positive screens was 2.8% in an Ethiopian sample of adults from a general medical setting. A unidimensional model demonstrated good fit for the PSQ, (CFI = 0.993, TLI = 0.986 and RMSEA = 0.025). For IRT, a 2PL model was the best fitting one. IRT tests of item difficulty and discrimination parameters showed that paranoia had the highest discrimination α = 4.59 $$ \left(\alpha =4.59\right) $$ and lowest difficulty ( ß = 2.53 $$ \beta =2.53 $$ ), likely to be endorsed at low levels of psychotic features. Thought insertion had the highest item difficulty ( ß = 2.98 $$ \beta =2.98 $$ ). Overall, the measure captures the psychosis construct at higher levels of the latent trait and may be suited for detecting moderate to severe levels of psychosis. CONCLUSION: The PSQ is found to have good construct validity in screening for psychosis among Ethiopian adults. Future studies may focus on the diagnostic validity of the PSQ comparing it with a structured clinical interview.

Multilevel Semiparametric Latent Variable Modeling in R with "galamm".

We present the R package galamm, whose goal is to provide common ground between structural equation modeling and mixed effect models. It supports estimation of models with an arbitrary number of crossed or nested random effects, smoothing splines, mixed response types, factor structures, heteroscedastic residuals, and data missing at random. Implementation using sparse matrix methods and automatic differentiation ensures computational efficiency. We here briefly present the implemented methodology, give an overview of the package and an example demonstrating its use.

Establishing and evaluating the gradient of item naming difficulty in post-stroke aphasia and semantic dementia.

Anomia is a common consequence following brain damage and a central symptom in semantic dementia (SD) and post-stroke aphasia (PSA), for instance. Picture naming tests are often used in clinical assessments and experience suggests that items vary systematically in their difficulty. Despite clinical intuitions and theoretical accounts, however, the existence and determinants of such a naming difficulty gradient remain to be empirically established and evaluated. Seizing the unique opportunity of two large-scale datasets of semantic dementia and post-stroke aphasia patients assessed with the same picture naming test, we applied an Item Response Theory (IRT) approach and we (a) established that an item naming difficulty gradient exists, which (b) partly differs between patient groups, and is (c) related in part to a limited number of psycholinguistic properties - frequency and familiarity for SD, frequency and word length for PSA. Our findings offer exciting future avenues for new, adaptive, time-efficient, and patient-tailored approaches to naming assessment and therapy.

A Note on Standard Errors for Multidimensional Two-Parameter Logistic Models Using Gaussian Variational Estimation.

Accurate item parameters and standard errors (SEs) are crucial for many multidimensional item response theory (MIRT) applications. A recent study proposed the Gaussian Variational Expectation Maximization (GVEM) algorithm to improve computational efficiency and estimation accuracy (Cho et al., 2021). However, the SE estimation procedure has yet to be fully addressed. To tackle this issue, the present study proposed an updated supplemented expectation maximization (USEM) method and a bootstrap method for SE estimation. These two methods were compared in terms of SE recovery accuracy. The simulation results demonstrated that the GVEM algorithm with bootstrap and item priors (GVEM-BSP) outperformed the other methods, exhibiting less bias and relative bias for SE estimates under most conditions. Although the GVEM with USEM (GVEM-USEM) was the most computationally efficient method, it yielded an upward bias for SE estimates.

Item Response Modeling of Clinical Instruments With Filter Questions: Disentangling Symptom Presence and Severity.

Clinical instruments that use a filter/follow-up response format often produce data with excess zeros, especially when administered to nonclinical samples. When the unidimensional graded response model (GRM) is then fit to these data, parameter estimates and scale scores tend to suggest that the instrument measures individual differences only among individuals with severe levels of the psychopathology. In such scenarios, alternative item response models that explicitly account for excess zeros may be more appropriate. The multivariate hurdle graded response model (MH-GRM), which has been previously proposed for handling zero-inflated questionnaire data, includes two latent variables: susceptibility, which underlies responses to the filter question, and severity, which underlies responses to the follow-up question. Using both simulated and empirical data, the current research shows that compared to unidimensional GRMs, the MH-GRM is better able to capture individual differences across a wider range of psychopathology, and that when unidimensional GRMs are fit to data from questionnaires that include filter questions, individual differences at the lower end of the severity continuum largely go unmeasured. Practical implications are discussed.

Examining measures of schizotypy for gender and racial bias using item response theory and differential item functioning.

Schizotypy involves schizophrenia-like traits and symptoms, with the Wisconsin Schizotypy Scales (WSS) being frequently used in previous research. There is some evidence that schizophrenia-spectrum symptom levels, including when using the WSS, might vary both by gender and by race and ethnicity. However, previous research has rarely examined to what extent the WSS show gender and racial bias. Further, this has not been previously examined for the Cognitive Slippage Scale, a measure of disorganized schizotypy. In this study, we examined biases for a subset of items from the WSS in a large sample of undergraduate students (n = 21,829). Using item response theory to test for levels of differential item functioning (DIF), we found some evidence of problematic DIF for all scales, including for negative, positive, and disorganized schizotypy scales. There was evidence of problematic DIF especially by gender and for Black and Multiracial participants. Overall, the current results suggest that gender and/or racial bias on these scales should be an important consideration in using these scales and our results could have implications for assessment of schizophrenia-spectrum symptoms.