Brief Report: Does the Number of Response Options Matter for the BFI-2? Conceptual Replication and Extension.

We evaluated how the number of response options affects the psychometric properties of the Big Five Inventory-2 (BFI-2). Using two large samples collected from a market research company (Ns = 893 and 1,213), we tested how different response options of the BFI-2 influenced scale score distributions, internal consistency estimates, convergent validity correlations, and criterion validity correlations. Results suggest that score distributions were impacted by the number of response options such that ceiling and floor effects were more common when using two or three response options than when using more options. Estimates of Cronbach's alpha were generally lower with fewer scale points as compared with more scale points, but these effects disappeared when ordinal alpha was used. There were no systematic effects of response options on convergent validity and criterion validity correlations. Given these results, there seems to be few psychometric reasons for deciding whether to administer personality items with five, six, or seven scale points.

How Does the Number of Response Options Impact the Psychometric Properties of the Rosenberg Self-Esteem Scale?

The Rosenberg Self-Esteem Scale is the most frequently used measure of self-esteem in the social sciences. These items are often administered with a different number of response options, but it is unclear how the number of response options impacts the psychometric properties of this measure. Across three experiments (Ns = 739, 2,358, and 1,461), we evaluated how different response options of the Rosenberg influenced (a) coefficient alpha estimates, (b) distributions of scores, and (c) associations with criterion-related variables. Observed coefficient alpha estimates were lowest for a 2-point format compared with response formats with more options. However, supplemental analyses using ordinal alpha pointed to similar estimates across conditions. Using four or more response options better approximated a normal distribution for observed summary scores. We found no consistent evidence that criterion-related correlations increased with more response options. Collectively, these results suggest that the Rosenberg should be administered with at least four response options and we favor a 5-point Likert-type response format.

Measurement Equality of Frequency and Severity Item Response Options on Depression and Generalized Anxiety Scales.

Rating scales are commonly used in psychological research and practice. It is important to consider how different item response options used on rating scales, such as those based on frequency or severity, may affect how they measure psychological constructs. In this study, participants were randomly assigned to conditions where they completed two depression and two generalized anxiety rating scales that used either frequency or severity item response options. Descriptive statistics and reliability estimates across depression and generalized anxiety scales using different item response options were similar. Measurement invariance tests indicated that depression and generalized anxiety scales with different item response options could be considered practically invariant. Finally, depression and generalized anxiety scales with different item response options had similar correlations with measures of impairment and other psychological constructs. There were essentially no differences in psychometric and measurement properties of these depression and generalized anxiety scales when using different item response options, suggesting this may not substantially affect the measurement of these constructs.

Development and equivalence of new faces for inclusion in the Childhood Asthma Control Test (C-ACT) response scale.

BACKGROUND: Accurate symptom monitoring is vital when managing pediatric asthma, providing an opportunity to improve control and relieve associated burden. The CHILDHOOD ASTHMA CONTROL TEST (C-ACT) has been validated for asthma control assessment in children; however, there are concerns that response option images used in the C-ACT are not culturally universal and could be misinterpreted. This cross-sectional, qualitative study developed and evaluated alternative response option images using interviews with children with asthma aged 4-11 years (and their parents/caregivers) in the United States, Spain, Poland, and Argentina. Interviews were conducted in two stages (with expert input) to evaluate the appropriateness, understanding and qualitative equivalence of the alternative images (both on paper and electronically). This included comparing the new images with the original C-ACT response scale, to provide context for equivalence results. RESULTS: Alternative response option images included scale A (simple faces), scale B (circles of decreasing size), and scale C (squares of decreasing quantity). In Stage 1, most children logically ranked images using scales A, B and C (66.7%, 79.0% and 70.6%, respectively). However, some children ranked the images in scales B (26.7%) and C (58.3%) in reverse order. Slightly more children could interpret the images within the context of their asthma in scale B (68.4%) than A (55.6%) and C (47.5%). Based on Stage 1 results, experts recommended scales A (with slight modifications) and B be investigated further. In Stage 2, similar proportions of children logically ranked the images used in modified scales A (69.7%) and B (75.7%). However, a majority of children ranked the images in scale B in the reverse order (60.0%). Slightly more children were able to interpret the images in the context of their asthma using scale B (57.6%) than modified scale A (48.5%). Children and parents/caregivers preferred modified scale A over scale B (78.8% and 90.9%, respectively). Compared with the original C-ACT, most children selected the same response option on items using both scales, supporting equivalency. Following review of Stage 2 results, all five experts agreed modified scale A was the optimal response scale. CONCLUSIONS: This study developed alternative response option images for use in the C-ACT and provides qualitative evidence of the equivalency of these response options to the originals.

Accurate monitoring of the symptoms associated with pediatric asthma is important when managing the condition. The CHILDHOOD ASTHMA CONTROL TEST (C-ACT) is a questionnaire widely used to measure asthma severity in young children (aged 4­11 years). Each question answered by the child in the C-ACT has four possible answer choices. To help children answer, each choice is presented alongside an image of a male child's face ranging from sad to happy. However, there are concerns that the images used are not culturally universal and could be misinterpreted­due to difficulties translating to electronic formats and a lack of differentiation between the images used. Through interviewing children with asthma, we aimed to address these concerns by developing and testing new images. Alternative image options developed included simpler faces, circles of decreasing size and squares of decreasing quantity. Children aged 4­11 years old were interviewed to test whether they understood the response scale using the new images and if they answered in the same way as with the original images. Interviews were conducted in two stages, with expert guidance at key stages. Results showed that children can interpret and understand the newly developed images and that they answer the questions the same as they would using the original images. These new images have the advantages of being culturally neutral and easier to implement on an electronic device.

Patient perspectives on recall period and response options in patient-reported outcome measures for chronic rhinosinusitis symptomatology: A pilot study.

OBJECTIVE: Patient-reported outcome measures (PROMs) for assessment of chronic rhinosinusitis (CRS) employ a variety of recall periods and response scales for reporting CRS symptom burden. CRS patient perspective is unknown with respect to recall periods and response scales in PROMs. DESIGN: Cross-sectional study. SETTING: Tertiary rhinology clinic. PARTICIPANTS: Sixty three adults with CRS. MAIN OUTCOME MEASURES: Participants were asked to choose which CRS symptom recall period-1 day, 2 weeks, 1 month or greater than 1 month-was most reflective of their current disease state and best to base treatment recommendations (including surgery) upon. Participants were also asked to report which of six response scales (one visual analogue scale [VAS] and five Likert scales ranging from four to eight items) were easiest to use and understand, and most preferred. RESULTS: A majority of participants felt the current state of their CRS symptoms was best reflected by a recall period of 2 weeks to 1 month; however, patients preferred that recommendations about treatments, including endoscopic sinus surgery, be determined by symptoms experienced over at least a one-month period. Participants generally found the VAS and seven-item Likert scale to be the easiest to use and understand, and their most preferred scales. No patient characteristics associated with preferences for recall periods or response scales. CONCLUSION: Future PROMs for CRS symptoms should consider assessment of symptoms over a one-month time frame and use either a VAS or seven-item Likert response scale to optimally balance reflection of current disease state, need for intervention and patient preference.

The role of response domain and scale label in the quantitative interpretation of patient-reported outcome measure response options.

PURPOSE: Uncertainties exist in how respondents interpret response options in patient-reported outcome measures (PROMs), particularly across different domains and for different scale labels. The current study assessed how respondents quantitatively interpret common response options. METHODS: Members of the general public were recruited to this study via an online panel, stratified by age, gender, and having English as a first language. Participants completed background questions and were randomised to answer questions on one of three domains (i.e. loneliness (negatively phrased), happiness or activities (positively phrased)). Participants were asked to provide quantitative interpretations of response options (e.g. how many times per week is equal to "often") and to order several common response options (e.g. occasionally, sometimes) on a 0-100 slider scale. Chi-squared tests and regression analyses were used to assess whether response options were interpreted consistently across domains and respondent characteristics. RESULTS: Data from 1377 participants were analysed. There was general consistency in quantifying the number of times over the last 7 days to which each response option referred. Response options were consistently assigned a lower value in the loneliness than happiness and activities domains. Individual differences, such as age and English as a second language, explained some significant variation in responses, but less than domain. CONCLUSION: Members of the public quantify common response options in a similar way, but their quantification is not equivalent across domains or every type of respondent. Recommendations for the use of certain scale labels over others in PROM development are provided.

Psychometric Properties of the Diener Satisfaction With Life Scale With Five Response Options Applied to the Colombian Population.

Introduction: The Satisfaction with Life Scale (SWLS), developed by Diener, Emmons, Larsen, and Griffin in 1985, comprises five items with seven response options in terms of agreement-disagreement. Recently, there has been a suggestion to reduce the response options of the SWLS to optimize its applicability in different cultural contexts. Objective: The study aims to assess the psychometric properties of the SWLS with five response options in the Colombian population. Specifically, we studied the dimensionality, invariance by gender and age (among a group of adolescents and emerging adults under 25 years and a group of adults of intermediate age and established adulthood under 59 years), convergent validity (with optimism), and divergent (with pessimism) and concurrent validity with other measures of well-being (flourishing, positive, and negative affects). Methodology: This project was a cross-sectional study using a non-probabilistic sample of the general population. Participants were included if they identified themselves as Colombian and were at least 18 years of age. The final sample comprised 1,255 participants. The average age was 25.62 years (SD = 8.60) ranging from 18 to 67 years of age, and 35.8% of the participants were men. In addition to SWLS, we used the Flourishing Scale (FS), Life Orientation Test-Revised (LOT-R), and Scale of Positive and Negative Experience (SPANE). Results: Cronbach's alpha coefficient (0.842), composite reliability (0.851), and average variance extracted (0.537) showed very good values. CFA was conducted to test the one-dimensional structure of FS, showing excellent goodness of fit [ χ(5)2 = 15.774, p < 0.001, CFI = 0.992, TLI = 0.985, RMSEA = 0.042, 90% RMSEA CI (0.020, 0.066), and SRMR = 0.016]. The correlations calculated among life satisfaction (SWLS) with flourishing (FS), positive and negative affects (SPANE), optimism, and pessimism (LOT-R) were statistically significant and as expected. Configural, metric, and scalar invariance across gender and age were confirmed. Percentiles were provided for the total score and for age. Conclusions: The SWLS with five response options has adequate psychometric properties in the Colombian population, and the use of this version (with 5 response options) is recommended due to its greater applicability.

Measuring User Experience With 3, 5, 7, or 11 Points : Does It Matter?

OBJECTIVE: To assess versions of the shorter form variant of Usability Metric for User Experience (UMUX-LITE) questionnaire differing in the number of response options for the items (3, 5, 7, and 11). BACKGROUND: The UMUX-LITE is an efficient (two-item) standardized questionnaire that measures perceived usability. A growing body of evidence shows it closely corresponds to one of the most widely used standardized usability questionnaires, the System Usability Scale (SUS), with regard to both correlation and magnitude of concurrently collected means. Although the "standard" version of the UMUX-LITE uses items with seven response options, there is some variance in practice. METHOD: Members of a corporate user experience panel (n = 242) completed surveys rating a recent Web site experience with the SUS and UMUX-LITE, also providing ratings of overall experience and likelihood-to-recommend. RESULTS: Scale reliabilities were acceptable (coefficient α >.70) with the exception of UMUX-LITE with three response options. All UMUX-LITE correlations with SUS, overall experience, and likelihood-to-recommend were highly significant. For likelihood-to-recommend, there was a significant difference in the magnitude of correlations, with 11 response options higher than three. Although some statistically significant differences were observed in correspondence between SUS and UMUX-LITE scores, these did not seem to translate to practically significant differences. CONCLUSION: The number of UMUX-LITE response options does not matter much, especially in practice. Because the version with three response options showed some weakness with regard to reliability and correlation with likelihood-to-recommend, practitioners should avoid it. APPLICATION: Unless there is a strong reason to do otherwise, use the "standard" version with seven response options.

Does the single-item self-rated health measure the same thing across different wordings? Construct validity study.

PURPOSE: The self-rated health (SRH) item is frequently used in health surveys but variations of its form (wording, response options) may hinder comparisons between versions over time or across surveys. The objectives were to determine (a) whether three SRH forms are equivalent, (b) the form with the best construct validity and (c) the best coding scheme to maximize equivalence across forms. METHODS: We used data from 58,023 respondents of the Swiss Health Survey. Three SRH forms were used. Response options varied across forms and we explored four coding schemes (two considering SRH as continuous, two as dichotomous). Construct validity of the SRH was assessed using 34 health predictors to estimate the explained variance. RESULTS: Distributions of response options were similar across SRH forms, except for the "good" and "very good" options ("good" in form 1: 58.6%, form 2: 65.0% and form 3: 44.1%). Explained variances differed across SRH forms, with form 3 providing the best overall explained variance, regardless of coding schemes. The linear coding scheme maximised the equivalence across SRH forms. CONCLUSION: The three SRH forms were not equivalent in terms of construct validity. Studies examining the evolution of SRH over time with surveys using different forms should use the linear coding scheme to maximise equivalence between SRH forms.

Do the unlabeled response categories of the Minnesota Living with Heart Failure Questionnaire satisfy the monotonicity assumption of simple-summated scoring?

PURPOSE: Half of the 21-item Minnesota Living with Heart Failure Questionnaire (MLHFQ) response categories are labeled (0 = No, 1 = Very little, 5 = Very much) and half are not (2, 3, and 4). We hypothesized that the unlabeled response options would not be more likely to be chosen at some place along the scale continuum than other response options and, therefore, not satisfy the monotonicity assumption of simple-summated scoring. METHODS: We performed exploratory and confirmatory factor analyses of the MLHFQ items in a sample of 1437 adults in the Better Effectiveness After Transition-Heart Failure study. We evaluated the unlabeled response options using item characteristic curves from item response theory-graded response models for MLHFQ physical and emotional health scales. Then, we examined the impact of collapsing response options on correlations of scale scores with other variables. RESULTS: The sample was 46% female; 71% aged 65 or older; 11% Hispanic, 22% Black, 54% White, and 12% other. The unlabeled response options were rarely chosen. The standard approach to scoring and scores obtained by collapsing adjacent response categories yielded similar associations with other variables, indicating that the existing response options are problematic. CONCLUSIONS: The unlabeled MLHFQ response options do not meet the assumptions of simple-summated scoring. Further assessment of the performance of the unlabeled response options and evaluation of alternative scoring approaches is recommended. Adding labels for response options in future administrations of the MLHFQ should be considered.

Response scale selection in adult pain measures: results from a literature review.

BACKGROUND: The purpose of this literature review was to examine the existing patient-reported outcome measurement literature to understand the empirical evidence supporting response scale selection in pain measurement for the adult population. METHODS: The search strategy involved a comprehensive, structured, literature review with multiple search objectives and search terms. RESULTS: The searched yielded 6918 abstracts which were reviewed against study criteria for eligibility across the adult pain objective. The review included 42 review articles, consensus guidelines, expert opinion pieces, and primary research articles providing insights into optimal response scale selection for pain assessment in the adult population. Based on the extensive and varied literature on pain assessments, the adult pain studies typically use simple response scales with single-item measures of pain-a numeric rating scale, visual analog scale, or verbal rating scale. Across 42 review articles, consensus guidelines, expert opinion pieces, and primary research articles, the NRS response scale was most often recommended in these guidance documents. When reviewing the empirical basis for these recommendations, we found that the NRS had slightly superior measurement properties (e.g., reliability, validity, responsiveness) across a wide variety of contexts of use as compared to other response scales. CONCLUSIONS: Both empirical studies and review articles provide evidence that the 11-point NRS is likely the optimal response scale to evaluate pain among adult patients without cognitive impairment.