A New Approach to Desirable Responding: Multidimensional Item Response Model of Overclaiming Data.

A variety of approaches have been presented for assessing desirable responding in self-report measures. Among them, the overclaiming technique asks respondents to rate their familiarity with a large set of real and nonexistent items (foils). The application of signal detection formulas to the endorsement rates of real items and foils yields indices of (a) knowledge accuracy and (b) knowledge bias. This overclaiming technique reflects both cognitive ability and personality. Here, we develop an alternative measurement model based on multidimensional item response theory (MIRT). We report three studies demonstrating this new model's capacity to analyze overclaiming data. First, a simulation study illustrates that MIRT and signal detection theory yield comparable indices of accuracy and bias-although MIRT provides important additional information. Two empirical examples-one based on mathematical terms and one based on Chinese idioms-are then elaborated. Together, they demonstrate the utility of this new approach for group comparisons and item selection. The implications of this research are illustrated and discussed.

Error patterns of facial emotion recognition in patients with schizophrenia.

Error patterns of facial emotion recognition (FER) indicate how individuals misinterpret others' facial expressions, which helps clinicians to manage related deficits. However, previous investigations are limited and may have been biased due to methodological issues (e.g., no consideration of response bias). This study aimed to propose a detectability index (d') for adjusting response bias and examine the error patterns of FER in patients with schizophrenia. Responses to 168 photos showing seven basic emotions, obtained from 351 patients with schizophrenia and 101 healthy adults, were extracted from a previous study. The differences in the d's between the two groups (Δd') were calculated to examine the error patterns of FER among the seven emotions. The findings were generally overlapped with those identified by the traditional confusion matrix. Four error patterns were found. First, the patients were insensitive to some negative emotions (i.e., sadness [Δd' = 0.83] and fear [Δd' = 0.72]). Second, they misrecognized happy faces as showing negative emotions (e.g., disgust [Δd' = 0.43] and sadness [Δd' = 0.37]). Third, they misinterpreted surprised faces as all the other emotions (Δd' = 0.41-0.87), except neutral. Fourth, they confused some negative emotions (e.g., misrecognizing fear as anger [Δd' = 0.87]). Our findings suggest that patients with schizophrenia show four error patterns of FER compared to healthy adults. Accordingly, interventions could be selected to improve their sensitivity to faces with negative emotions, differentiation of faces among positive and negative emotions, understanding of surprised faces, and discrimination of faces with negative emotions.

Improving test efficiency for a grid multidimensional computerized classification test by the application of a conditional latent-trait distribution to a sequential probability ratio test.

The measurement efficiency of a multidimensional computerized adaptive testing (MCAT) can be improved by taking the correlations between the dimensions into account during the item selection and latent-trait estimation procedures (Segall, 1996; Wang & Chen, 2004). Although a multidimensional computerized classification test (MCCT), which was based on a multidimensional itemresponse model, was previously found more efficient than its unidimensional counterpart, the difference was negligible (Seitz & Frey, 2013); the researchers had adopted a sequential probability ratio test (SPRT) as the termination criterion in this MCCT study. To make a classification decision on each dimension, which is called a grid classification (Wang et al., 2019), only items that loaded on that dimension were used to calculate the likelihood ratio, which squandered the available information of the correlations between the dimensions. The current study utilizes such useful information to improve the measurement efficiency of the MCCT by applying a conditional distribution of the latent-trait estimates and then including all the administered items to calculate the likelihood ratio in the SPRT. The performance of this newly proposed method was evaluated through a series of simulation studies. The results showed that the proposed method can sizably improve the measurement efficiency of an MCCT by saving 1% to 32% of the test length in comparison with the SPRT when the two test dimensions are at least moderately correlated. The findings and further applications of this study are discussed.

A psychometric analysis of a short form of the Chinese version of the ISPCAN child abuse screening tools - Children's home version (SC-ICAST-CH) using multidimensional item response theory.

BACKGROUND: The development of measures of child maltreatment for the Chinese population were limited until the Chinese version of the ISPCAN Child Abuse Screening Tools - Children's Home version (ICAST-CH-C) was proposed. Although the ICAST-CH-C was found to be effective in assessing the scope and prevalence of child maltreatment, it has several potential drawbacks. The time that is required to complete the ICAST-CH-C scale is longer than usual for a 36-item scale, because many of its items have one or more follow-up questions. Moreover, each item requires child victims to recall unpleasant experiences. Both phenomena can cause increases in invalid responses and in turn damage the data quality. OBJECTIVE: The goal of this study was to propose a short form of the ICAST-CH-C (called the SC-ICAST-CH) to reduce the test length and response time in order to improve the measurement quality. PARTICIPANTS AND SETTING: A dataset from a national survey of 5236 adolescents in Taiwan was used. METHODS: A multidimensional version of the rating scale model (MRSM) was fitted to the data. The model parameters were estimated with the ConQuest software. RESULTS: The results indicated the reliability of the SC-ICAST-CH was fairly good, with only 61 % of the original test length. Disordered thresholds were found in all five subscales; the underlying reasons for this phenomenon need further investigation. Specific cultural differences related to item retention/removal decisions were also discussed. CONCLUSION: The efficient, shorter SC-ICAST-CH was shown to be a valid and reliable instrument for assessing the prevalence of child maltreatment.

An Exploratory Strategy to Identify and Define Sources of Differential Item Functioning.

The sources of differential item functioning (DIF) items are usually identified through a qualitative content review by a panel of experts. However, the differential functioning for some DIF items might have been caused by reasons outside of the experts' experiences, leading to the sources for these DIF items possibly being misidentified. Quantitative methods can help to provide useful information, such as the DIF status and the number of sources of the DIF, which in turn help the item review and revision process to be more efficient and precise. However, the current quantitative methods assume all possible sources should be known in advance and collected to accompany the item response data, which is not always the case in reality. To this end, an exploratory strategy, combined with the MIMIC (multiple-indicator multiple-cause) method, that can be used to identify and name new sources of DIF is proposed in this study. The performance of this strategy was investigated through simulation. The results showed that when a set of DIF-free items can be correctly identified to define the main dimension, the proposed exploratory MIMIC method can accurately recover a number of possible sources of DIF and the items that belong to each. A real data analysis was also implemented to demonstrate how this strategy can be used in reality. The results and findings of this study are further discussed.

Examining unidimensionality and improving reliability for the eight subscales of the SF-36 in opioid-dependent patients using Rasch analysis.

PURPOSE: The Medical Outcome Study Short Form 36 (SF-36) is one of the most commonly used questionnaires for monitoring the Health-Related Quality of Life (HRQOL) of opioid-dependent patients. However, the unidimensionality and reliability of the SF-36 have not been verified in opioid-dependent patients. The aim of this study was to examine the unidimensionality and to improve the test reliability of the SF-36 for use in opioid-dependent patients. METHODS: A total of 583 opioid-dependent patients were recruited in the study. Unidimensionality was examined by conducting unidimensional Rasch analysis. Item fit statistics and principle component analysis were used to check the item-model fit in each of the eight subscales of the SF-36. Reliability was evaluated by applying both unidimensional and multidimensional Rasch analyses. RESULTS: After three misfitting items were excluded, the remaining items of each subscale in the SF-36 represented a single construct. The test reliabilities (0.80-0.87) yielded by the multidimensional approach were much higher than those (0.68-0.82) produced by the unidimensional approach. CONCLUSION: The remaining 32 items of the SF-36 are appropriate for evaluating the HRQOL in opioid-dependent patients in terms of unidimensionality. Additionally, the test scores produced by the multidimensional approach were more accurate than those obtained by the unidimensional approach.

A prospective study of the responsiveness of the original and the short form Berg Balance Scale in people with stroke.

OBJECTIVE: The study aim was to examine the responsiveness of the Berg Balance Scale (BBS) and that of its short form (SFBBS) at both the individual person level and the group level. DESIGN: A repeated-measurements design. SETTING: Hospital and home setting. PARTICIPANTS: Patients with stroke. RESULTS: Totals of 226, 202, and 168 patients with stroke were assessed with the BBS at 14, 30, and 90 days after stroke, respectively. The SFBBS data were extracted from the patients' responses on the BBS. At the group level, the BBS and the SFBBS had sufficient and similar responsiveness. For the Rasch scores, the effect sizes of the three change scores for the BBS and the SFBBS, respectively, had similar ranges between 0.38 and 0.88 and between 0.39 and 0.85, respectively. The standardized response means of the three change scores for the BBS and the SFBBS ranged from 0.74 to 1.33 and from 0.72 to 1.13, respectively. At the individual person level, the BBS detected significant balance improvement in about twice as many patients as the SFBBS detected. CONCLUSION: The responsiveness of the BBS at the individual person level was better than that of the SFBBS in patients with stroke. The BBS is recommended as an outcome measure to better detect changes in individual patients.

Is the long form of the Fugl-Meyer motor scale more responsive than the short form in patients with stroke?

OBJECTIVE: To compare the responsiveness of the Rasch-calibrated 37-item Fugl-Meyer motor Scale with that of the 12-item Fugl-Meyer motor scale at both an individual and a group level. DESIGN: Repeated-measurements design. SETTING: Medical center. PARTICIPANTS: Patients (N=301) 14 days after stroke. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: 50-item Fugl-Meyer motor scale, 37-item Fugl-Meyer motor scale, and 12-item Fugl-Meyer motor scale. RESULTS: The patients were assessed with the original 50-item Fugl-Meyer motor scale 4 times, at 14, 30, 90, and 180 days after stroke onset. The patients' responses were used for estimating the Rasch scores of the 37-item Fugl-Meyer motor scale and 12-item Fugl-Meyer motor scale. The effect size, standardized response mean, and paired t test were used to compare the group-based responsiveness of the 3 forms (50-item Fugl-Meyer motor scale, 37-item Fugl-Meyer motor scale, 12-item Fugl-Meyer motor scale). Individual-level responsiveness was compared based on the significance of change between the 37-item Fugl-Meyer motor scale and 12-item Fugl-Meyer motor scale. Because up to 13 items of the 50-item Fugl-Meyer motor scale did not meet the Rasch model's assumptions, the significance of change of the 50-item Fugl-Meyer motor scale was not calculated. At the group level, the FM-37 and FM-12 Fugl-Meyer motor scale had sufficient and similar responsiveness. At the individual level, the FM-37 Fugl-Meyer motor scale detected more patients with significant improvement than the FM-12 Fugl-Meyer motor scale. The SC values and category distribution of the FM-37 Fugl-Meyer motor scale were significantly better than those of the FM-12 Fugl-Meyer motor scale (P<.001). CONCLUSIONS: Although the group-level responsiveness of the 12-item Fugl-Meyer motor scale was sufficient and very similar to that of the 37-item Fugl-Meyer motor scale, the 37-item Fugl-Meyer motor scale had better individual-level responsiveness. The 37-item Fugl-Meyer motor scale is suggested as an outcome measure for both clinicians and researchers.

Validating and improving the reliability of the EORTC qlq-c30 using a multidimensional Rasch model.

OBJECTIVE: The reliability and validity of the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire (EORTC QLQ-C30) has not been examined while taking into account the correlation between subscales. The reliability of the EORTC QLQ-C30 subscales is modest, thus limiting their utility in both clinical and research settings. The purpose of this study was to validate the factor structure of multiple-item subscales of the EORTC QLQ-C30 and to improve their reliability by means of an item response analysis by using the multidimensional partial credit model. METHODS: A total of 2295 patients with complete data were used for the analysis. One- and nine-dimensional partial credit models were used to fit the data to validate the construct validity of the multiple-item subscales of the QLQ-C30. RESULTS: The model comparison showed that the nine-dimensional factor structure of multiple-item subscales was satisfactory. The multidimensional partial credit model fit data of the multiple-item subscales of the QLQ-C30 reasonably well. The estimated test reliabilities of each domain obtained from the multidimensional approach were higher than those obtained from the unidimensional approach. CONCLUSIONS: The constructs represented by the multiple-item subscales of the QLQ-C30 were validated. The improved reliability of the multiple-item subscales of the QLQ-C30 under the multidimensional approach can facilitate their applications in clinical and research settings.

Development of a computerized adaptive testing system of the Fugl-Meyer motor scale in stroke patients.

OBJECTIVE: To develop a computerized adaptive testing system of the Fugl-Meyer motor scale (CAT-FM) to efficiently and reliably assess motor function in patients with stroke. DESIGN: First, a simulation study was used to develop and examine the psychometric properties of the CAT-FM. Second, a field study was employed to determine the administration efficiency of the CAT-FM. SETTING: One medical center and 1 teaching hospital. PARTICIPANTS: Patients' responses (n=301) were used for the simulation study; 49 patients participated in the field study. INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: The 2 CAT-FM item banks (upper extremity and lower extremity) include 37 items from the original Fugl-Meyer scale. The reliability, validity, and responsiveness of the CAT-FM were determined by the simulation study. RESULTS: Two stopping rules (reliability ≥0.9 or an increase of reliability <.01 after testing an item) were used. The simulation study showed that the CAT-FM had high reliability (≥.93 for upper-extremity and lower-extremity subscales) and concurrent validity (Pearson r≥.91 for the upper-extremity and lower-extremity subscales and motor scale). The responsiveness was moderate (standardized response mean for the upper extremity=.67, lower extremity=.79, and motor=.77) for the 226 patients who completed both assessments at 14 and 90 days after stroke. The field study found that, on average, the time needed to administer the CAT-FM was 242 seconds with 4.7 items. CONCLUSIONS: The CAT-FM is an efficient, reliable, valid, and responsive clinical tool for assessing motor function in patients with stroke.

Development and initial validation of the Preschooler Gross Motor Quality Scale.

Motor skills have great impact on children in adapting to an environment and developing interpersonal interaction, cognition, and social behavior. Understanding what children can do and how they perform it is essential. Most motor tests seldom contain quality evaluation in the items or criteria. The purpose of this study was to develop and construct validity of the Preschooler Gross Motor Quality Scale (PGMQ) that included 17 items in three subscales to evaluate the quality of fundamental motor skills for the preschooler. The study designed PGMQ items and subscales according to well-described procedures, and recruited 174 children, aged 3-6 (89 boys and 85 girls) from 17 kindergartens. Two independent evaluators who were unaware of each other's results evaluated all the children separately in their kindergartens using PGMQ standardized setting and procedures and Gross Motor Scales of Peabody Developmental Motor Scales II (PDMS-II). Results of this study found significant differences between different ages and genders (Wilks'Λ=0.221, p<.001 and Wilks'Λ=0.690, p<.001 respectively). Total scores and raw scores in three subscales of PGMQ increased as age increased. Confirmatory factor analysis revealed three factors, including locomotion, manipulative ability and balance to explain 51.06% of the variance. The correlations between PGMQ and PDMS-II subscales were moderate to high (r=0.544-0.868, p<.01). The results showed satisfactory validity of PGMQ. PGMQ is useful to evaluate the development and quality of fundamental motor skills for the preschoolers.