The Effects of Questionnaire Length on the Relative Impact of Response Styles in Ambulatory Assessment.

Ambulatory assessment (AA) is becoming an increasingly popular research method in the fields of psychology and life science. Nevertheless, knowledge about the effects that design choices, such as questionnaire length (i.e., number of items per questionnaire), have on AA data quality is still surprisingly restricted. Additionally, response styles (RS), which threaten data quality, have hardly been analyzed in the context of AA. The aim of the current research was to experimentally manipulate questionnaire length and investigate the association between questionnaire length and RS in an AA study. We expected that the group with the longer (82-item) questionnaire would show greater reliance on RS relative to the substantive traits than the group with the shorter (33-item) questionnaire. Students (n = 284) received questionnaires three times a day for 14 days. We used a multigroup two-dimensional item response tree model in a multilevel structural equation modeling framework to estimate midpoint and extreme RS in our AA study. We found that the long questionnaire group showed a greater reliance on RS relative to trait-based processes than the short questionnaire group. Although further validation of our findings is necessary, we hope that researchers consider our findings when planning an AA study in the future.

Predicting response time on self-report rating scale assessments of noncognitive constructs.

Methodological studies of response time on noncognitive assessments have separately demonstrated the relevance of content trait level and response styles as predictive factors. In this paper we examine the simultaneous relevance of both factors as well as the potential for omitted predictor bias when ignoring either factor. Using response time data from several different noncognitive assessments, we demonstrate how a multilevel regression model that attends simultaneously to content and response style factors leads to consistent findings that support the simultaneous relevance of both factors. The average effects of response style consistently emerge as stronger, although also show greater respondent-level variability, possibly due to the multiple different underlying causes of response style behavior. Some implications for the use of response times in noncognitive measurement are considered.

A Stan tutorial on Bayesian IRTree models: Conventional models and explanatory extension.

IRTree models have been receiving increasing attention. However, to date, there are limited sources that provide a systematic introduction to Bayesian modeling techniques using modern probabilistic programming frameworks for the implementation of IRTree models. To facilitate the research and application of IRTree models, this paper introduces how to perform two families of Bayesian IRTree models (i.e., response tree models and latent tree models) in Stan and how to extend them in an explanatory way. Some suggestions on executing Stan codes and checking convergence are also provided. An empirical study based on the Oxford Achieving Resilience during COVID-19 data was conducted as an example to further illustrate how to apply Bayesian IRTree models to address research questions. Finally, strengths and future directions are discussed.

Co-occurring dominance and ideal point processes: A general IRTree framework for multidimensional item responding.

Responding to rating scale items is a multidimensional process, since not only the substantive trait being measured but also additional personal characteristics can affect the respondents' category choices. A flexible model class for analyzing such multidimensional responses are IRTree models, in which rating responses are decomposed into a sequence of sub-decisions. Different response processes can be involved in item responding both sequentially across those sub-decisions and as co-occurring processes within sub-decisions. In the previous literature, modeling co-occurring processes has been exclusively limited to dominance models, where higher trait levels are associated with higher expected scores. However, some response processes may rather follow an ideal point rationale, where the expected score depends on the proximity of a person's trait level to the item's location. Therefore, we propose a new multidimensional IRT model of co-occurring dominance and ideal point processes (DI-MIRT model) as a flexible framework for parameterizing IRTree sub-decisions with multiple dominance processes, multiple ideal point processes, and combinations of both. The DI-MIRT parameterization opens up new application areas for the IRTree model class and allows the specification of a wide range of theoretical assumptions regarding the cognitive processing of item responding. A simulation study shows that IRTree models with DI-MIRT parameterization provide excellent parameter recovery and accurately reflect co-occurring dominance and ideal point processes. In addition, a clear advantage over traditional IRTree models with purely sequential processes is demonstrated. Two application examples from the field of response style analysis highlight the benefits of the general IRTree framework under real-world conditions.

Psychometric benefits of self-chosen rating scales over given rating scales.

Rating scales are susceptible to response styles that undermine the scale quality. Optimizing a rating scale can tailor it to individuals' cognitive abilities, thereby preventing the occurrence of response styles related to a suboptimal response format. However, the discrimination ability of individuals in a sample may vary, suggesting that different rating scales may be appropriate for different individuals. This study aims to examine (1) whether response styles can be avoided when individuals are allowed to choose a rating scale and (2) whether the psychometric properties of self-chosen rating scales improve compared to given rating scales. To address these objectives, data from the flourishing scale were used as an illustrative example. MTurk workers from Amazon's Mechanical Turk platform (N = 7042) completed an eight-item flourishing scale twice: (1) using a randomly assigned four-, six-, or 11-point rating scale, and (2) using a self-chosen rating scale. Applying the restrictive mixed generalized partial credit model (rmGPCM) allowed examination of category use across the conditions. Correlations with external variables were calculated to assess the effects of the rating scales on criterion validity. The results revealed consistent use of self-chosen rating scales, with approximately equal proportions of the three response styles. Ordinary response behavior was observed in 55-58% of individuals, which was an increase of 12-15% compared to assigned rating scales. The self-chosen rating scales also exhibited superior psychometric properties. The implications of these findings are discussed.

Moving beyond Likert and Traditional Forced-Choice Scales: A Comprehensive Investigation of the Graded Forced-Choice Format.

The graded forced-choice (FC) format has recently emerged as an alternative that may preserve the advantages and overcome the issues of the dichotomous FC measures. The current study presented the first large-scale evaluation of the performance of three types of FC measures (FC2, FC4 and FC5 with 2, 4 and 5 response options, respectively) and compared their performance to their Likert (LK) counterparts (LK2, LK4, and LK5) on (1) psychometric properties, (2) respondent reactions, and (3) susceptibility to response styles. Results showed that, compared to LK measures with the same number of response options, the three FC scales provided better support for the hypothesized factor structure, were perceived as more faking-resistant and cognitive demanding, and were less susceptible to response styles. FC4/5 and LK4/5 demonstrated similarly good reliability, while LK2 provided more reliable scores than FC2. When compared across the three FC measures, FC4 and FC5 displayed comparable psychometric performance and respondent reactions. FC4 exhibited a moderate presence of extreme response style, while FC5 had a weak presence of both extreme and middle response styles. Based on these findings, the study recommends the use of graded FC over dichotomous FC and LK, particularly FC5 when extreme response style is a concern.

Pathways to Depressive Symptoms in a Digital Environment: The Role of Trait Affectivity and Mediation of Media Response Styles in Adolescence.

Trait negative affectivity and trait extraversion/positive affectivity are predictive of both responses to affect and depressive symptoms in adolescence. Furthermore, differences in the use of responses to affect are associated with different levels of depressive symptoms. Despite the central role of media content in adolescents' daily lives, responses to affect have not yet been extended to affect in this media content. It is thus unclear whether trait affectivity is predictive of responses to affective media content, and whether such media response styles are predictive of depressive symptoms. A 3-wave longitudinal panel study with 3-month intervals among 318 adolescents (Mage = 16.5 years, SDage = 1.11, 72.5% boys) investigated the mediating role of media response styles in associations between trait affectivity and depressive symptoms. Trait negative affectivity predicted media rumination and media dampening, yet only media rumination predicted greater levels of depressive symptoms over time. Trait positive affectivity was associated with concurrent media distraction and media-enhancing. The media response styles did not mediate the associations between trait affectivity and depressive symptoms over time. These findings suggest that individuals higher in trait negative affectivity tend to engage in maladaptive emotion regulation strategies during sad media content consumption, whereas adolescents higher in trait positive affectivity turn to more adaptive strategies during sad or happy media content consumption. Yet, media response styles are not robustly associated with depressive symptoms over time.

A Comparison of Limited-Information Test Statistics for a Response Style MIRT Model.

An increased use of models for measuring response styles is apparent in recent years with the multidimensional nominal response model (MNRM) as one prominent example. Inclusion of latent constructs representing extreme (ERS) or midpoint response style (MRS) often improves model fit according to information criteria. However, a test of absolute model fit is often not reported even though it could comprise an important piece of validity evidence. Limited information test statistics are candidates for this task, including the full (M2), ordinal (M2*), and mixed (C2) statistics, which differ in whether additional collapsing of univariate or bivariate contingency tables is conducted. Such collapsing makes sense when item categories are ordinal, which may not hold under the MNRM. More generally, limited information test statistics have gone unevaluated under nominal data and non-ordinal latent trait models. We present a simulation study evaluating the performance of M2, M2*, and C2 with the MNRM. Manipulated conditions included sample size, presence and type of response style, and strength of item slopes on substantive and style dimensions. We found that M2 sometimes had inflated Type I error rates, M2* always had little power, and C2 lacked power under some conditions. M2 and C2 may provide complementary and valuable information regarding model fit.

Do response styles affect the predictive validity of self-rated health among Chinese older men and women?

PURPOSE: This study intended to examine whether correcting for response styles of the widely used self-rated health (SRH) question would affect its predictive validity in estimating respondents' mortality risks. METHODS: We used data on a sample of 3319 men and 3250 women aged 60 and above from a nationally representative survey in China. Response styles were estimated with both ratings on two health anchoring vignettes and self-rated items in domains unrelated to health. Gender-stratified logistic regressions were used to estimate the effects of SRH on 4-year mortality risks. We compared the results before and after adjusting for response styles. RESULTS: In the unconditional model, the effects of the five-point SRH on 4-year mortality risks were significant in both older women and men, and stronger among Chinese older men based on discrete change in probabilities of mortality at means. Gender difference was observed in response styles. Men assigned lower ratings to the vignettes than women, reflecting their higher standards in defining good health. Women presented higher proportions of extreme responses and lower proportions of midpoint responses. Correcting for response styles had no effect on the predictive validity of SRH for Chinese older men, but had some positive effects for Chinese older women when no other covariates were included. CONCLUSION: Adjusting for response styles contributed to a small improvement in predicting mortality risks only among Chinese older women under restricted conditions, but not men.

The redemption and contamination research form: exploring relations with narrative identity, personality traits, response styles, and life satisfaction.

Narrative identity is typically assessed by collecting participants' autobiographical scenes and then coding these stories for themes including redemption (negative beginning, positive ending) and contamination (positive beginning, negative ending). Complimenting this approach, we introduce a self-report measure capturing the degree to which individuals explicitly view their lives and social worlds in redemptive and contaminated ways - the Redemption and Contamination Research Form (RCRF). In Studies 1 and 2, participants completed the RCRF and a measure of life satisfaction. In Study 2, participants also provided three autobiographical scenes, later coded for redemption and contamination. Across studies, our novel self-rated redemptive mindset variable corresponded positively with life satisfaction and, in Study 2, the redemption present in scenes. Relations remained significant after considering several covariates (e.g., traits, response styles). These results, which illustrate the utility of self-rated redemptive mindsets, carry implications for the multi-method assessment of constructs indigenous to narrative identity.

A New Model for Acquiescence at the Interface of Psychometrics and Cognitive Psychology.

When measuring psychological traits, one has to consider that respondents often show content-unrelated response behavior in answering questionnaires. To disentangle the target trait and two such response styles, extreme responding and midpoint responding, Böckenholt ( 2012a ) developed an item response model based on a latent processing tree structure. We propose a theoretically motivated extension of this model to also measure acquiescence, the tendency to agree with both regular and reversed items. Substantively, our approach builds on multinomial processing tree (MPT) models that are used in cognitive psychology to disentangle qualitatively distinct processes. Accordingly, the new model for response styles assumes a mixture distribution of affirmative responses, which are either determined by the underlying target trait or by acquiescence. In order to estimate the model parameters, we rely on Bayesian hierarchical estimation of MPT models. In simulations, we show that the model provides unbiased estimates of response styles and the target trait, and we compare the new model and Böckenholt's model in a recovery study. An empirical example from personality psychology is used for illustrative purposes.

Responses to positive affect, life satisfaction and self-esteem: A cross-lagged panel analysis during middle adolescence.

During middle adolescence, elevated stress and a greater presence of psychological disorders have been documented. The research has paid little attention to the regulation of positive affective states. Fredrickson's broaden-and-build theory suggests that cultivating positive emotions helps to build resources that boost well-being. The current research aimed to examine the longitudinal associations between responses to positive affect (emotion-focused positive rumination, self-focused positive rumination, and dampening) and psychological adjustment (self-esteem and life satisfaction) during middle adolescence. A longitudinal study with two waves separated by one year was conducted, assessing 977 adolescents (M = 13.81, SD = 0.79; 51.5% boys) with self-report measures. A cross-lagged panel analysis was performed by including within the same model the relationships between all of the variables in the two assessment points. The results indicated cross-lagged positive relationships of self-focused positive rumination with both self-esteem and life satisfaction, while dampening showed a negative cross-lagged relationship with self-esteem. Moreover, higher self-esteem predicted more emotion-focused positive rumination, and more dampening predicted lower life satisfaction. Thus, the use of adaptive responses to positive affect and a better psychological adjustment were found to be prospectively interrelated at the one-year follow-up during middle adolescence. The discussion argues for the need to implement programmes to promote more adaptive responses to positive affect to enhance psychological adjustment in the adolescent transition to adulthood.

General mixture item response models with different item response structures: Exposition with an application to Likert scales.

This article proposes a general mixture item response theory (IRT) framework that allows for classes of persons to differ with respect to the type of processes underlying the item responses. Through the use of mixture models, nonnested IRT models with different structures can be estimated for different classes, and class membership can be estimated for each person in the sample. If researchers are able to provide competing measurement models, this mixture IRT framework may help them deal with some violations of measurement invariance. To illustrate this approach, we consider a two-class mixture model, where a person's responses to Likert-scale items containing a neutral middle category are either modeled using a generalized partial credit model, or through an IRTree model. In the first model, the middle category ("neither agree nor disagree") is taken to be qualitatively similar to the other categories, and is taken to provide information about the person's endorsement. In the second model, the middle category is taken to be qualitatively different and to reflect a nonresponse choice, which is modeled using an additional latent variable that captures a person's willingness to respond. The mixture model is studied using simulation studies and is applied to an empirical example.

Psychometric properties of the Children's Response Styles Questionnaire in a Hong Kong Chinese community sample.

BACKGROUND: The Children's Response Styles Questionnaire (CRSQ) is a widely-adopted inventory that assesses response styles in youths. It is useful in examining how coping styles (particularly rumination) may relate to depressive vulnerability in youths. Despite its utility, little is known about its applicability in non-Western cultures and CRSQ has not been evaluated using current psychometric methods including item response theory (IRT). The present study assessed the properties using IRT methods in a Chinese youth sample. METHODS: Students in Grades 4-6 were recruited from seven public primary schools in Hong Kong, and a total of 581 children (280 boys and 301 girls) between 8 and 14 years of age participated in the study. A Chinese version of CRSQ was administered to them in groups at school after receiving written parental consent as well as students' assent. RESULTS: Confirmatory factor analysis revealed a two-factor structure that was comparable to that identified in Western samples, namely, the rumination and distraction/problem-solving subscales. IRT analysis suggested that items varied in levels of item discrimination and item severity, and in precision/usefulness for assessing the underlying latent trait levels. Test information analysis indicated that rumination subscale was more useful than the distraction and problem-solving subscale in assessing the latent trait over a broader range of levels. For gender-based Differential Item Functioning (DIF) analysis, item 1 "When I am sad, I think about how alone I feel" was found to exhibit higher discriminating power for girls than boys. CONCLUSIONS: The study presents the first attempt to examine CRSQ item properties using IRT analysis and supports its validity beyond the Western cultures. The factor structure of CRSQ was found to be comparable to the West in our Chinese sample. Differential Item Functioning (DIF) evaluation suggested all but one item in the rumination subscale of the CRSQ apply equally well to both boys and girls.

A two-year longitudinal study of gender differences in responses to positive affect and depressive symptoms during middle adolescence.

This study aimed to analyze the prospective associations during adolescence between depressive symptoms and response styles to positive affect and to examine gender differences. A longitudinal study was conducted with three waves separated by 1 year each to assess a non-clinical sample of 622 Spanish adolescents who were 13 and 14 years old (50.2% boys, 49.8% girls). The participants completed self-report measures of depressive symptoms and responses to positive affect (emotion-focused positive rumination, self-focused positive rumination and dampening of positive emotion). The results showed that the increase in depressive symptoms was associated with an increase in dampening and decreases in emotion-focused and self-focused positive rumination. Furthermore, girls presented more depressive symptoms, as well as higher dampening and lower self-focused positive rumination, than boys. The conclusions highlight the need to consider responses to positive affect in explaining gender differences in depressive symptoms during mid-adolescence, as well as in designing prevention programs.

Response styles in factual items: Personal, contextual and cultural correlates.

This study investigated response styles in factual items and explored their associations with personal, contextual and cultural factors. Responses on various factual questions, cognitive tests and interviewers' observational data from a total of 152,514 respondents in 22 countries in the Programme for the International Assessment of Adult Competencies (PIAAC) were analysed. Indexes of extreme, midpoint and acquiescent response styles were extracted from Likert-scale and dichotomous responses of factual items. A general response style (GRS) with a positive loading of extreme response style and negative loadings of midpoint and acquiescent response styles was confirmed. This factor showed a similar cross-cultural patterning as another general factor from attitudinal and self-evaluative items of Likert scales in a previous study, which indicated the pervasiveness of response styles irrespective of types of survey items. In a multilevel analysis, the individual-level GRS was found to be negatively related to being male, educational level and literacy competency, and positively related to 3rd-person presence and background noise, and at country level negatively associated with socioeconomic development. Cross-level interactions were also found. Implications on the pervasiveness and nature of response styles are discussed.

Linking extreme response style to response processes: A cross-cultural mixed methods approach.

The aim of this study is to contribute to a better understanding of extreme response style in cross-cultural research by integrating quantitative and qualitative evidence in a mixed methods design. In the quantitative phase, indexes of extreme response style, derived from quality of life measures from different international studies, were compared between Spain and the Netherlands. Results indicated that extreme responding was more common among Spanish than among Dutch in endorsement of items, but that the opposite was found for frequency scales including never as a response anchor. In the qualitative phase, cognitive interviews were conducted with 25 participants in each country. The integration of quantitative results and qualitative findings suggests that country differences in extreme response style may stem from various sources, including the more independent evaluation of each item by Dutch, the stronger connotations of never for Spanish and stronger emotions triggered by specific topics such as work satisfaction that was more strongly associated with insecurity for Spanish. It is concluded that the integration of quantitative and qualitative evidence can help to understand cross-cultural similarities and differences in extreme response style.

Cultural differences in survey responding: Issues and insights in the study of response biases.

This paper introduces the special section "Cultural differences in questionnaire responding" and discusses central topics in the research on response biases in cross-cultural survey research. Based on current conceptions of acquiescent, extreme, and socially desirable responding, the author considers current data on the correlated nature of response biases and the conditions under which different response styles they emerge. Based on evidence relating different response styles to the cultural dimension of individualism-collectivism, the paper explores how research presented as part of this special section might help resolves some tensions in this literature. The paper concludes by arguing that response styles should not be treated merely as measurement error, but as cultural behaviors in themselves.

Thinking about rumination: the scholarly contributions and intellectual legacy of Susan Nolen-Hoeksema.

Our article reviews and celebrates Susan Nolen-Hoeksema's remarkable contributions to psychological and clinical science, focusing on her vast body of theoretical and empirical work and her influence on colleagues and students. Susan spent her career trying to understand how and why a style of regulating emotions called rumination increases vulnerability to depression and exacerbates and perpetuates negative moods. More broadly, we describe research by Susan and her colleagues on the predictors of depression in childhood and adolescence; gender differences in depression and rumination in adolescence and adulthood; roots, correlates, and adverse consequences of ruminative response styles; and rumination as a transdiagnostic risk factor for not only depression but also a host of psychological disorders, including anxiety, substance abuse, and eating disorders. Susan's intellectual legacy is evident in her impressive publication and citation record, the clinical applications of her work, and the flourishing careers of the students she mentored.

A psychometric evaluation of the Swedish version of the Responses to Positive Affect questionnaire.

BACKGROUND: Previous research mainly focused on responses to negative affect in relation to depression, and less on responses to positive affect. Cognitive responses to positive affect are interesting in the context of emotion regulation and emotion disorders: positive rumination is associated to hypomania risk and bipolar disorder. There is to date no questionnaire in Swedish that captures the phenomena of cognitive response styles. AIMS: The aim of this study was to investigate the replicability of the Responses to Positive Affect questionnaire (RPA) in a newly translated Swedish version and to test its psychometric properties. METHODS: Swedish undergraduates (n = 111) completed a set of self-report questionnaires in a fixed order. RESULTS: The hypothesized three-factor model was largely replicated in the subscales Self-focused positive rumination, Emotion-focused positive rumination and Dampening. The two positive rumination subscales were strongly associated with each other and current positive affect. The subscales showed acceptable convergent and incremental validity with concurrent measures of depression, hypomania, anxiety, repetitive negative thinking, and positive and negative affect. The model explained 25% of the variance in hypomania, but fell short in the explanation of depression. CONCLUSIONS: The Swedish version of the RPA shows satisfactory reliability and initial findings from a student sample indicate that it is a valid measure comparable with the original RPA questionnaire. RESULTS give emphasis to the importance of further exploration of cognitive response styles in relation to psychopathology.