Re-assessing the assessment of fears of positive and negative evaluation: Scale development and psychometric evaluation of the Bivalent Fear of Evaluation Scale (BFOES).

The bivalent fear of evaluation (BFOE) model of social anxiety divides fear of evaluation into two distinct valences: fear of positive evaluation (FPE) and fear of negative evaluation (FNE). However, there is evidence that the two most widely utilized and psychometrically supported measures of FNE and FPE contain items which are ambiguous with regard to valence of evaluative fear. To formally address this, the BFOE Scale (BFOES) was developed, by merging items from measures of FNE and FPE into a single scale with an integrated response format. The present studies examined the psychometric profile of the BFOES across a large pooled archival dataset (N = 2216), which included approximately 10 % (n = 224) patients with social anxiety disorder (SAD). The factorial validity, internal consistency, and construct validity of the BFOES were examined. Additionally, item response theory analyses were employed for the purpose of merging items from self-report scales which utilized different Likert-type response formats. Results from both studies provided support for the psychometric profile of the BFOES. The implications of the BFOES for the assessment of social anxiety, and theoretical models of fear of evaluation and SAD, are discussed.

Evaluating Longitudinal Anchoring Methods for Rasch Models.

Because modern, simultaneously estimated longitudinal Rasch models are unable to handle many timepoints, new methods of producing person and item estimates and evaluating test function are necessary. Longitudinal anchoring, in which a common scale of item parameters is used to estimate trait levels over multiple occasions, is a potential solution. With proper anchoring procedures, person and item estimates can be obtained without limiting the number of timepoints that can be analyzed. A simulation study examining the performance of six longitudinal anchoring methods (Floated, Racked, Time One, Mean, Random, and Stacked) was conducted. The Mean and the Stacked anchoring methods best recovered the population change over time, person and item estimates, and model fit. The Racked method could not produce reliable change estimates and should be avoided. Longitudinal anchoring is an easily implemented solution when analyzing large longitudinal datasets and shows promise as a low-computation method of producing latent trait estimates.

Creation of a Novel Digital Rectal Examination Evaluation Instrument to Teach and Assess Prostate Examination Proficiency.

OBJECTIVE: To create a validated tool to measure digital rectal examination proficiency and aid with teaching of the examination. DESIGN: The Digital Rectal Examination Clinical Tool was created using a modified Delphi method with 5 urologists and 5 radiation oncologists. The instrument was then validated in a population of preclinical medical students examining male urological teaching associates, and clinical trainees (third- and fourth-year medical students and urology resident physicians) examining prospectively enrolled subjects. Trainees completed paired examinations with an attending urologist, and responses were scored with reference to the attending responses. SETTING: The instrument was validated at the University of Virginia in the urology clinic, endoscopic operating room, and main operating room settings. PARTICIPANTS: We tested the instrument on consenting subjects consisting of male urologic teaching associates (n = 12), clinic patients (n = 4), and operating room patients (n = 64). The participants were undergraduate (n = 302) and graduate (n = 9) medical trainees. RESULTS: In preclerkship trainees, improved scores in subjects without abnormal compared to those with abnormal findings demonstrated validity. In clinical trainees, scores on the Digital Rectal Examination Clinical Tool increased by 2% for each additional year of training, demonstrating construct validity. CONCLUSIONS: We used an expert panel to create a novel instrument for measuring digital rectal examination proficiency and validated it with preclinical and clinical trainee cohorts at our institution.

The e-MSWS-12: improving the multiple sclerosis walking scale using item response theory.

BACKGROUND: The Multiple Sclerosis Walking Scale (MSWS-12) is the predominant patient-reported measure of multiple sclerosis (MS) -elated walking ability, yet it had not been analyzed using item response theory (IRT), the emerging standard for patient-reported outcome (PRO) validation. This study aims to reduce MSWS-12 measurement error and facilitate computerized adaptive testing by creating an IRT model of the MSWS-12 and distributing it online. METHODS: MSWS-12 responses from 284 subjects with MS were collected by mail and used to fit and compare several IRT models. Following model selection and assessment, subpopulations based on age and sex were tested for differential item functioning (DIF). RESULTS: Model comparison favored a one-dimensional graded response model (GRM). This model met fit criteria and explained 87 % of response variance. The performance of each MSWS-12 item was characterized using category response curves (CRCs) and item information. IRT-based MSWS-12 scores correlated with traditional MSWS-12 scores (r = 0.99) and timed 25-foot walk (T25FW) speed (r =  -0.70). Item 2 showed DIF based on age (χ 2 = 19.02, df = 5, p < 0.01), and Item 11 showed DIF based on sex (χ 2 = 13.76, df = 5, p = 0.02). CONCLUSIONS: MSWS-12 measurement error depends on walking ability, but could be lowered by improving or replacing items with low information or DIF. The e-MSWS-12 includes IRT-based scoring, error checking, and an estimated T25FW derived from MSWS-12 responses. It is available at https://ms-irt.shinyapps.io/e-MSWS-12 .

What You Don't Know Can Hurt You: Missing Data and Partial Credit Model Estimates.

The authors investigated the effect of missing completely at random (MCAR) item responses on partial credit model (PCM) parameter estimates in a longitudinal study of Positive Affect. Participants were 307 adults from the older cohort of the Notre Dame Study of Health and Well-Being (Bergeman and Deboeck, 2014) who completed questionnaires including Positive Affect items for 56 days. Additional missing responses were introduced to the data, randomly replacing 20%, 50%, and 70% of the responses on each item and each day with missing values, in addition to the existing missing data. Results indicated that item locations and person trait level measures diverged from the original estimates as the level of degradation from induced missing data increased. In addition, standard errors of these estimates increased with the level of degradation. Thus, MCAR data does damage the quality and precision of PCM estimates.

Assessing psychopathy among justice involved adolescents with the PCL:YV: an item response theory examination across gender.

This study used an item response theory (IRT) model and a large adolescent sample of justice involved youth (N = 1,007, 38% female) to examine the item functioning of the Psychopathy Checklist-Youth Version (PCL: YV). Items that were most discriminating (or most sensitive to changes) of the latent trait (thought to be psychopathy) among adolescents included "glibness/superficial charm," "lack of remorse," and "need for stimulation," whereas items that were least discriminating included "pathological lying," "failure to accept responsibility," and "lacks goals." The items "impulsivity" and "irresponsibility" were the most likely to be rated high among adolescents, whereas "parasitic lifestyle," and "glibness/superficial charm" were the most likely to be rated low. Evidence of differential item functioning (DIF) on 4 of the 13 items was found between boys and girls. "Failure to accept responsibility" and "impulsivity" were endorsed more frequently to describe adolescent girls than boys at similar levels of the latent trait, and vice versa for "grandiose sense of self-worth" and "lacks goals." The DIF findings suggest that 4 PCL: YV items function differently between boys and girls.

A Rasch Analysis of the KeyMath-3 Diagnostic Assessment.

Effectively assessing children's academic development can help school professionals make placement decisions and prepare appropriate instructional supports. The KeyMath-3 Diagnostic Assessment (Connolly, 2008) is a widely used assessment of children's mathematical abilities; however, despite much use, the measurement properties of the KeyMath-3 DA have not been examined, aside from the development and standardization phases. The current study conducted a Rasch analysis of the Basic Concepts content area of the KeyMath-3 DA in a diverse sample of 308 young children to assess the quality of the assessment. Rasch analytic procedures examined unidimensionality, item and person fit statistics, reliability, and item hierarchy. Misfitting items were further examined, and response patterns were modified. In general, results show that the Basic Concepts subscale is a good measure of the underlying construct of young children's understanding of the basic concepts in mathematics. Implications are discussed.

Using the Partial Credit Model to Evaluate the Student Engagement in Mathematics Scale.

The Student Engagement in Mathematics Scale (SEMS) is a self-report measure that was created to assess three dimensions of student engagement (social, emotional, and cognitive) in mathematics based on a single day of class. In the current study, the SEMS was administered to a sample of 360 fifth graders from a large Mid-Atlantic district. The Rasch partial credit model (PCM) was used to analyze the psychometric properties of each sub-dimension of the SEMS. Misfitting items were removed from the final analysis. In general, items represented a range of engagement levels. Results show that the SEMS is an effective measure for researchers and practitioners to assess upper elementary school students' perception of their engagement in math. The paper concludes with several recommendations for researchers considering using the SEMS.

Measuring positive and negative affect in older adults over 56 days: comparing trait level scoring methods using the partial credit model.

Positive (PA) and negative affect (NA) are important constructs in health and well-being research. Good longitudinal measurement is crucial to conducting meaningful research on relationships between affect, health, and well-being across the lifespan. One common affect measure, the PANAS, has been evaluated thoroughly with factor analysis, but not with Racsh-based latent trait models (RLTMs) such as the Partial Credit Model (PCM), and not longitudinally. Current longitudinal RLTMs can computationally handle few occasions of data. The present study compares four methods of anchoring PCMs across 56 occasions to longitudinally evaluate the psychometric properties of the PANAS plus additional items. Anchoring item parameters on mean parameter values across occasions produced more desirable results than using no anchor, using first occasion parameters as anchors, or allowing anchor values to vary across occasions. Results indicated problems with NA items, including poor category utilization, gaps in the item distribution, and a lack of easy-to-endorse items. PA items had much more desirable psychometric qualities.

Psychometric properties of the hypoglycemia fear survey-ii for adults with type 1 diabetes.

OBJECTIVE: To perform the first comprehensive psychometric evaluation of the Hypoglycemia Fear Survey-II (HFS-II), a measure of the behavioral and affective dimensions of fear of hypoglycemia, using modern test-theory methods, including item-response theory (IRT). RESEARCH DESIGN AND METHODS: Surveys completed in four previous studies by 777 adults with type 1 diabetes were aggregated for analysis, with 289 subjects completing both subscales of the HFS-II and 488 subjects completing only the Worry subscale. The aggregated sample (53.3% female, 44.4% using insulin pumps) had a mean age of 41.9 years, diabetes duration of 23.8 years, HbA(1c) value of 7.7%, and 1.4 severe hypoglycemic episodes in the past year. Data analysis included exploratory factor analysis using polychoric correlations and IRT. Factors were analyzed for fit, trait-level locations, point-measure correlations, and separation values. RESULTS: Internal and test-retest reliability was good, as well as convergent validity, as demonstrated by significant correlations with other measures of psychological distress. Scores were significantly higher in subjects who had experienced severe hypoglycemia in the past year. Factor analyses validated the two subscales of the HFS-II. Item analyses showed that 12 of 15 items on the Behavior subscale, and all of the items on the Worry subscale had good-fit statistics. CONCLUSIONS: The HFS-II is a reliable and valid measure of the fear of hypoglycemia in adults with type 1 diabetes, and factor analyses and IRT support the two separate subscales of the survey.

The Rainbow Families Scale (RFS): a measure of experiences among individuals with lesbian and gay parents.

According to two decades of research, parental sexual orientation does not affect overall child development. Researchers have not found significant differences between offspring of heterosexual parents and those of lesbian and gay parents in terms of their cognitive, psychological, or emotional adjustment. Still, there are gaps in the literature regarding social experiences specific to offspring of lesbian and gay parents. This study's objective was to construct a measure of those experiences. The Rainbow Families Scale (RFS) was created on the basis of focus group discussions (N = 9 participants), and then piloted (N = 24) and retested with a new sample (N = 91) to examine its psychometric properties. Exploratory factor analyses uncovered secondary dimensions and Rasch analytic procedures examined item fit, reliability, and category usage. Misfitting items were eliminated where necessary, yielding a psychometrically sound measurement tool to aid in the study of individuals with lesbian and gay parents.

Using LinLog and FACETS to model item components in the LLTM.

The current study investigates the performance of two Rasch measurement programs and their parameter estimations on the linear logistic test model (LLTM; Fischer, 1973). These two programs, LinLog (Whitely & Nieh, 1981) and FACETS (Linacre, 2002), are used to investigate within-item complexity factors in a spatial memory measure tool. LinLog uses conditional maximum likelihood to estimate person and item parameters and is an LLTM specific program. FACETS is usually reserved for the many-facet Rasch model (MFRM; Linacre, 1989), however in the case of specifically designed within-item solution processes, a multifaceted approach makes good sense. It is possible to consider each dimension within the item as a separate facet, just as if there were multiple raters for each item. Simulations of 500 and 1000 persons expand the original data set (114 persons) to better examine each estimation technique. LinLog and FACETS analyses show strikingly similar results in both the simulation and original data conditions, indicating that the FACETS program produces accurate LLTM parameter estimates.

Rasch analysis examining processing mechanisms of the object location memory test revised.

This study's objective was the construction and examination of the Object Location Memory Revised (OLM-R), an instrument designed to measure spatial memory. The OLM-R measured a participant's (N=111) ability to inspect a spatial array and recall image identities and positions after a distractor task. Rasch methodology and regression analyses were employed to explore the influence a priori design factors have on performance and item difficulty. Rasch analyses revealed that, while the OLM-R has a misfitting item, overall the instrument shows good measurement properties. Specific analyses examining complexity factors indicated that Object Manipulation (e.g., moving, replacing, or unchanging) and the Number of Items in an array were leading influences of OLM-R performance.

Ethnic identification biases responses to the Padua Inventory for obsessive-compulsive disorder.

The authors report differential item functioning (DIF) between Black and White participants completing the 60-item Padua Inventory (PI) for obsessive-compulsive disorder (OCD). The authors use an Internet-generated sample that included 105 Blacks, 67 Hispanics, 582 Whites, and 136 additional participants reporting an OCD diagnosis. Factor analysis replicated prior work indicating the PI consists of four factors: contamination fears, checking behaviors, impaired control over thoughts, and fear of losing control over impulses. On the contamination subscale, nonclinical Black and Hispanic mean scores were as high as the OCD group. Comparing Blacks to Whites, the authors applied an item response theory, DIF-graded response model to each factor and found significant DIF on eight items, with biased items in each factor. Results suggest that extraneous factors contribute to racial differences on scores. Cultural practices and fear of being negatively stereotyped may contribute to item bias.