Estimating anchor-based minimal important change using longitudinal confirmatory factor analysis.

PURPOSE: The minimal important change (MIC) is defined as the smallest within-individual change in a patient-reported outcome measure (PROM) that patients on average perceive as important. We describe a method to estimate this value based on longitudinal confirmatory factor analysis (LCFA). The method is evaluated and compared with a recently published method based on longitudinal item response theory (LIRT) in simulated and real data. We also examined the effect of sample size on bias and precision of the estimate. METHODS: We simulated 108 samples with various characteristics in which the true MIC was simulated as the mean of individual MICs, and estimated MICs based on LCFA and LIRT. Additionally, both MICs were estimated in existing PROMIS Pain Behavior data from 909 patients. In another set of 3888 simulated samples with sample sizes of 125, 250, 500, and 1000, we estimated LCFA-based MICs. RESULTS: The MIC was equally well recovered with the LCFA-method as using the LIRT-method, but the LCFA analyses were more than 50 times faster. In the Pain Behavior data (with higher scores indicating more pain behavior), an LCFA-based MIC for improvement was estimated to be 2.85 points (on a simple sum scale ranging 14-42), whereas the LIRT-based MIC was estimated to be 2.60. The sample size simulations showed that smaller sample sizes decreased the precision of the LCFA-based MIC and increased the risk of model non-convergence. CONCLUSION: The MIC can accurately be estimated using LCFA, but sample sizes need to be preferably greater than 125.

Effect of present state bias on minimal important change estimates: a simulation study.

PURPOSE: The minimal important change (MIC) in a patient-reported outcome measure is often estimated using patient-reported transition ratings as anchor. However, transition ratings are often more heavily weighted by the follow-up state than by the baseline state, a phenomenon known as "present state bias" (PSB). It is unknown if and how PSB affects the estimation of MICs using various methods. METHODS: We simulated 3240 samples in which the true MIC was simulated as the mean of individual MICs, and PSB was created by basing transition ratings on a "weighted change", differentially weighting baseline and follow-up states. In each sample we estimated MICs based on the following methods: mean change (MC), receiver operating characteristic (ROC) analysis, predictive modeling (PM), adjusted predictive modeling (APM), longitudinal item response theory (LIRT), and longitudinal confirmatory factor analysis (LCFA). The latter two MICs were estimated with and without constraints on the transition item slope parameters (LIRT) or factor loadings (LCFA). RESULTS: PSB did not affect MIC estimates based on MC, ROC, and PM but these methods were biased by other factors. PSB caused imprecision in the MIC estimates based on APM, LIRT and LCFA with constraints, if the degree of PSB was substantial. However, the unconstrained LIRT- and LCFA-based MICs recovered the true MIC without bias and with high precision, independent of the degree of PSB. CONCLUSION: We recommend the unconstrained LIRT- and LCFA-based MIC methods to estimate anchor-based MICs, irrespective of the degree of PSB. The APM-method is a feasible alternative if PSB is limited.

Effect of Context Specificity on Response to the Shortened WOMAC Function Scale in Patients Undergoing Total Knee Arthroplasty.

OBJECTIVE: To determine, in patients undergoing total knee arthroplasty (TKA), whether increasing context specificity of selected items of the shortened version of the Western Ontario and McMaster Universities Osteoarthritis Index function (WOMAC-F) scale (ShortMAC-F) (1) enhanced the convergent validity of the ShortMAC-F with performance-based mobility measures (ii) affected mean scale score, structural validity, reliability, and interpretability. DESIGN: Secondary analysis of randomized clinical trial data. SETTING: A tertiary teaching hospital. PARTICIPANTS: Patients undergoing TKA (N=114). INTERVENTIONS: Not applicable. MAIN OUTCOME MEASURES: The ShortMAC-F was modified by specifying the "ascending stairs" and "rising from sitting" items to enquire about difficulty in performing the tasks without reliance on compensatory strategies, whereas the modified "level walking" item enquired about difficulty in walking 400 m. Before and 12 weeks after TKA, patients completed the WOMAC-F questionnaire, modified ShortMAC-F questionnaire, knee pain scale questionnaire, sit-to-stand test, fast gait speed test, and stair climb test. Interpretability was evaluated by calculating anchor-based substantial clinical benefit estimates. RESULTS: The modified ShortMAC-F correlated significantly more strongly than ShortMAC-F or WOMAC-F with pooled performance measures (differences in correlation values, 0.12-0.14). Increasing item context specificity of the ShortMAC-F did not influence its psychometric properties of unidimensionality (comparative fit and Tucker-Lewis indices, >0.95; root mean square error of approximation, 0.05-0.08), reliability (Cronbach's α, 0.75-0.83), correlation with pain intensity (correlation values, 0.48-0.52), and substantial clinical benefit estimates (16 percentage points); however, it resulted in lower mean score (4.5-4.8 points lower). CONCLUSIONS: The modified ShortMAC-F showed sufficient measurement properties for clinical application, and it seemed more adept than WOMAC-F at correlating with performance-based measures in TKA.

Establishing thresholds for meaningful within-individual change using longitudinal item response theory.

PURPOSE: Thresholds for meaningful within-individual change (MWIC) are useful for interpreting patient-reported outcome measures (PROM). Transition ratings (TR) have been recommended as anchors to establish MWIC. Traditional statistical methods for analyzing MWIC such as mean change analysis, receiver operating characteristic (ROC) analysis, and predictive modeling ignore problems of floor/ceiling effects and measurement error in the PROM scores and the TR item. We present a novel approach to MWIC estimation for multi-item scales using longitudinal item response theory (LIRT). METHODS: A Graded Response LIRT model for baseline and follow-up PROM data was expanded to include a TR item measuring latent change. The LIRT threshold parameter for the TR established the MWIC threshold on the latent metric, from which the observed PROM score MWIC threshold was estimated. We compared the LIRT approach and traditional methods using an example data set with baseline and three follow-up assessments differing by magnitude of score improvement, variance of score improvement, and baseline-follow-up score correlation. RESULTS: The LIRT model provided good fit to the data. LIRT estimates of observed PROM MWIC varied between 3 and 4 points score improvement. In contrast, results from traditional methods varied from 2 to 10 points-strongly associated with proportion of self-rated improvement. Best agreement between methods was seen when approximately 50% rated their health as improved. CONCLUSION: Results from traditional analyses of anchor-based MWIC are impacted by study conditions. LIRT constitutes a promising and more robust analytic approach to identifying thresholds for MWIC.

Estimating meaningful thresholds for multi-item questionnaires using item response theory.

PURPOSE: Meaningful thresholds are needed to interpret patient-reported outcome measure (PROM) results. This paper introduces a new method, based on item response theory (IRT), to estimate such thresholds. The performance of the method is examined in simulated datasets and two real datasets, and compared with other methods. METHODS: The IRT method involves fitting an IRT model to the PROM items and an anchor item indicating the criterion state of interest. The difficulty parameter of the anchor item represents the meaningful threshold on the latent trait. The latent threshold is then linked to the corresponding expected PROM score. We simulated 4500 item response datasets to a 10-item PROM, and an anchor item. The datasets varied with respect to the mean and standard deviation of the latent trait, and the reliability of the anchor item. The real datasets consisted of a depression scale with a clinical depression diagnosis as anchor variable and a pain scale with a patient acceptable symptom state (PASS) question as anchor variable. RESULTS: The new IRT method recovered the true thresholds accurately across the simulated datasets. The other methods, except one, produced biased threshold estimates if the state prevalence was smaller or greater than 0.5. The adjusted predictive modeling method matched the new IRT method (also in the real datasets) but showed some residual bias if the prevalence was smaller than 0.3 or greater than 0.7. CONCLUSIONS: The new IRT method perfectly recovers meaningful (interpretational) thresholds for multi-item questionnaires, provided that the data satisfy the assumptions for IRT analysis.

Meaningful thresholds for patient-reported outcomes following interventions for anterior cruciate ligament tear or traumatic meniscus injury: a systematic review for the OPTIKNEE consensus.

OBJECTIVE: We synthesised and assessed credibility (ie, trustworthiness) of thresholds that define meaningful scores for patient-reported outcome measures (PROMs) following interventions for anterior cruciate ligament (ACL) tear or traumatic meniscus injury. DESIGN: Systematic review, narrative synthesis. DATA SOURCES: We searched five databases, handsearched references of included studies and tracked citations. ELIGIBILITY: Included studies investigated: individuals with ACL tear or meniscus injury; mean age <35 years; and PROM thresholds calculated using any method to define a minimal important change (MIC) or a meaningful post-treatment score (Patient Acceptable Symptom State (PASS) or Treatment Failure). RESULTS: We included 18 studies (15 ACL, 3 meniscus). Three different methods were used to calculate anchor-based MICs across 9 PROMs, PASS thresholds across 4 PROMs and treatment failure for 1 PROM. Credibility was rated 'high' for only one study-an MIC of 18 for the Knee injury and Osteoarthritis Outcome Score Quality-of-life (KOOS-QOL) subscale (using the MID Credibility Assessment Tool). Where multiple thresholds were calculated among 'low' credibility thresholds in ACL studies, MICs converged to within a 10-point range for KOOS-Symptoms (-1.2 to 5.4) and function in daily living (activities of daily living, ADL 0.5-8.1) subscales, and the International Knee Documentation Committee Subjective Knee Form (7.1-16.2). Other PROM thresholds differed up to 30 points. PASS thresholds converged to within a 10-point range in KOOS-ADL for ACL tears (92.3-100), and KOOS-Symptoms (73-78) and KOOS-QOL (53-57) in meniscus injuries. CONCLUSION: Meaningful PROM thresholds were highly susceptible to study heterogeneity. While PROM thresholds can aid interpretability in research and clinical practice, they should be cautiously interpreted.

Barriers and Facilitators for Return to Work from the Perspective of Workers with Common Mental Disorders with Short, Medium and Long-Term Sickness Absence: A Longitudinal Qualitative Study.

Purpose Although common mental disorders (CMDs) highly impact individuals and society, a knowledge gap exists on how sickness absence can be prevented in workers with CMDs. This study explores: (1) workers' perceived causes of sickness absence; (2) perceived return to work (RTW) barriers and facilitators; and (3) differences between workers with short, medium and long-term sickness absence. Methods A longitudinal qualitative study was conducted involving 34 workers with CMDs. Semi-structured interviews were held at two time-points during their RTW process. The 68 interviews were audio-taped, transcribed and thematically analyzed to explore workers' perspective on sickness absence causes, RTW barriers and facilitators, and compare data across the three sub-groups of workers. Results Workers reported various causes for their absence, including: (1) high work pressure; (2) poor work relationships; (3) unhelpful thoughts and feelings, e.g. lacking self-insight; and (4) ineffective coping behaviors. According to workers, RTW was facilitated by work adjustments, fulfilling relationships with supervisors, and adequate occupational health guidance. Workers with short-term leave more often reported favorable work conditions, and proactive coping behavior. In contrast, the long-term group reported reactive coping behavior and dissatisfaction with their work. Conclusion Supporting workers with CMDs in gaining self-awareness and regaining control, discussing the value of their work, and creating work conditions that enable workers to do valuable work, seem central for successful RTW and might prevent sickness absence. Supervisors play a key role in enabling workers to do valuable work and further research should focus on how supervisors can be supported in this task.

Interpretation threshold values for the Oxford Knee Score in patients undergoing unicompartmental knee arthroplasty.

BACKGROUND AND PURPOSE: Developing meaningful thresholds for the Oxford Knee Score (OKS) advances its clinical use. We determined the minimal important change (MIC), patient acceptable symptom state (PASS), and treatment failure (TF) values as meaningful thresholds for the OKS at 3-, 12-, and 24-month follow-up in patients undergoing unicompartmental knee arthroplasty (UKA). PATIENTS AND METHODS: This is a cohort study with data from patients undergoing UKA collected at a hospital in Denmark between February 2016 and September 2021. The OKS was completed preoperatively and at 3, 12, and 24 months postoperatively. Interpretation threshold values were calculated with the anchor-based adjusted predictive modeling method. Non-parametric bootstrapping was used to derive 95% confidence intervals (CI). RESULTS: Complete 3-, 12-, and 24-month postoperative data was obtained for 331 of 423 (78%), 340 of 479 (71%), and 235 of 338 (70%) patients, median age of 68-69 years (58-59% females). Adjusted OKS MIC values were 4.7 (CI 3.3-6.0), 7.1 (CI 5.2-8.6), and 5.4 (CI 3.4- 7.3), adjusted OKS PASS values were 28.9 (CI 27.6-30.3), 32.7 (CI 31.5-33.9), and 31.3 (CI 29.1-33.3), and adjusted OKS TF values were 24.4 (CI 20.7-27.4), 29.3 (CI 27.3-31.1), and 28.5 (CI 26.0-30.5) at 3, 12, and 24 months postoperatively, respectively. All values statistically significantly increased from 3 to 12 months but not from 12 to 24 months. INTERPRETATION: The UKA-specific measurement properties and clinical thresholds for the OKS can improve the interpretation of UKA outcome and assist quality assessment in institutional and national registries.

Assessing baseline dependency of anchor-based minimal important change (MIC): don't stratify on the baseline score!

PURPOSE: The minimal important change (MIC) of a patient-reported outcome measure (PROM) is often suspected to be baseline dependent, typically in the sense that patients who are in a poorer baseline health condition need greater improvement to qualify as minimally important. Testing MIC baseline dependency is commonly performed by creating two or more subgroups, stratified on the baseline PROM score. This study's purpose was to show that this practice produces biased subgroup MIC estimates resulting in spurious MIC baseline dependency, and to develop alternative methods to evaluate MIC baseline dependency. METHODS: Datasets with PROM baseline and follow-up scores and transition ratings were simulated with and without MIC baseline dependency. Mean change MICs, ROC-based MICs, predictive MICs, and adjusted MICs were estimated before and after stratification on the baseline score. Three alternative methods were developed and evaluated. The methods were applied in a real data example for illustration. RESULTS: Baseline stratification resulted in biased subgroup MIC estimates and the false impression of MIC baseline dependency, due to redistribution of measurement error. Two of the alternative methods require a second baseline measurement with the same PROM or another correlated PROM. The third method involves the construction of two parallel tests based on splitting the PROM's item set. Two methods could be applied to the real data. CONCLUSION: MIC baseline dependency should not be tested in subgroups based on stratification on the baseline PROM score. Instead, one or more of the suggested alternative methods should be used.

Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures.

We define the minimal important change (MIC) as a threshold for a minimal within-person change over time above which patients perceive themselves importantly changed. There is a lot of confusion about the concept of MIC, particularly about the concepts of minimal important change and minimal detectable change, which questions the validity of published MIC values. The aims of this study were: (1) to clarify the concept of MIC and how to use it; (2) to provide practical guidance for estimating methodologically sound MIC values; and (3) to improve the applicability of PROMIS by summarizing the available evidence on plausible PROMIS MIC values. We discuss the concept of MIC and how to use it and provide practical guidance for estimating MIC values. In addition, we performed a systematic review in PubMed on MIC values of any PROMIS measure from studies using recommended approaches. A total of 50 studies estimated the MIC of a PROMIS measure, of which 19 studies used less appropriate methods. MIC values of the remaining 31 studies ranged from 0.1 to 12.7 T-score points. We recommend to use the predictive modeling method, possibly supplemented with the vignette-based method, in future MIC studies. We consider a MIC value of 2-6 T-score points for PROMIS measures reasonable to assume at this point. For surgical interventions a higher MIC value might be appropriate. We recommend more high-quality studies estimating MIC values for PROMIS.

Assessing measurement equivalence of the Danish and Dutch Four-Dimensional Symptom Questionnaire using differential item and test functioning analysis.

Aims: The Dutch Four-Dimensional Symptom Questionnaire (4DSQ) measures distress, depression, anxiety and somatisation, facilitating the distinction between stress-related problems and psychiatric disorder in primary and occupational health care. The aim of the study was to examine the measurement equivalence across the Danish and Dutch 4DSQ. Methods: Danish 4DSQ data were obtained from a cohort of Danish citizens on sick leave for mental-health problems. Dutch 4DSQ data were obtained from a cohort of Dutch employees on sick leave and a cohort of general practice attenders suspected of having mental-health problems. The study samples were matched on age and sex. The 4DSQ scales were assessed for essential unidimensionality using confirmatory factor analysis. Measurement equivalence of the 4DSQ across the groups was assessed using differential item and test functioning (DIF and DTF) analysis. Results: The study groups each consisted of 1363 people (63% female, Mage=42 years). The 4DSQ scales proved essentially unidimensional. DIF was detected in 20 items. In terms of Cohen's effect size, DIF was mostly small or moderate. In terms of effect size, the mean effect on the scale score (DTF) was negligible. Nevertheless, it is recommended to adjust some of the cut-off points for two Danish 4DSQ scales to retain the meaning of these cut-off points in Dutch respondents. Conclusions: The Danish version of the 4DSQ measures the same constructs as the original Dutch questionnaire. Twenty items functioned differently in Danish respondents than in Dutch respondents, but this had only a small impact on the scale scores.

Tests for central sensitization in general practice: a Delphi study.

INTRODUCTION: Central sensitization (CS) may explain the persistence of symptoms in patients with chronic pain and persistent physical symptoms (PPS). There is a need for assessing CS in the consultation room. In a recently published systematic review, we made an inventory of tests for CS. In this study we aimed to assess which tests might have added value, might be feasible and thus be suitable for use in general practice. METHODS: We conducted a Delphi study consisting of two e-mail rounds to reach consensus among experts in chronic pain and PPS. We invited 40 national and international experts on chronic pain and PPS, 27 agreed to participate. We selected 12 tests from our systematic review and additional searches; panellists added three more tests in the first round. We asked the panellists, both clinicians and researchers, to rate these 15 tests on technical feasibility for use in general practice, added value and to provide an overall judgement for suitability in general practice. RESULTS: In two rounds the panellists reached consensus on 14 of the 15 tests: three were included, eleven excluded. Included were the Central Sensitization Inventory (CSI), pressure pain thresholds (PPTs) and monofilaments. No consensus was reached on the Sensory Hypersensitivity Scale. CONCLUSION: In a Delphi study among an international panel of experts, three tests for measuring CS were considered to be suitable for use in general practice: the Central Sensitization Inventory (CSI), pressure pain thresholds (PPTs) and monofilaments.

Validation of the Four-Dimensional Symptom Questionnaire (4DSQ) in a mental health setting.

Mental health problems are highly prevalent in primary care. Validated tools to detect mental disorders in general practice are needed. The Four-Dimensional Symptom Questionnaire (4DSQ) was designed to help GPs differentiating between psychological distress and psychopathological conditions (depression, anxiety, somatization). The aim of the current study was to examine psychometric properties of the 4DSQ in a mental health setting. Reliability, factorial, construct, and criterion validity of the English translation of the 4DSQ were analyzed in an American sample of 159 patients attending a psychotherapy outpatient clinic. Measurement equivalence across languages was determined by analyzing differential item functioning (DIF) and differential test functioning (DTF) in the American sample and a Dutch mental health sample, matched by age and sex. A confirmatory factor analysis confirmed all 4DSQ subscales to be unidimensional. All 4DSQ subscales revealed excellent reliability (Cronbach's alpha and McDonald omega ≥.90) and high correlations with a symptom distress subscale of an instrument that is commonly used to monitor psychotherapy progress, the Outcome Questionnaire-45. Eight items were flagged with DIF. The Depression subscale was free of DIF. DTF analyses showed an impact of DIF on scale level for the lower cutoff score of the Distress scale. The 4DSQ Distress score was the best predictor of a mood disorder diagnosis and the Anxiety score outperformed other 4DSQ scales to predict an anxiety disorder. In conclusion, the 4DSQ demonstrates excellent reliability and validity in a mental health setting. Further research is needed to determine reliable cutoff values on 4DSQ subscales to predict psychiatric diagnoses.

Which Oxford Knee Score level represents a satisfactory symptom state after undergoing a total knee replacement?

Background and purpose - Meaningful interpretation of postoperative Oxford Knee Score (OKS) levels is challenging. We established Patient Acceptable Symptoms State (PASS) and Treatment Failure (TF) values for the OKS in patients undergoing primary total knee replacement (TKR) in Denmark.Patients and methods - Data from patients undergoing primary TKR between February 2015 and January 2019 was extracted from the arthroplasty registry at the Copenhagen University Hospital, Hvidovre in Denmark. Data included 3, 12, and 24 months postoperative responses to the OKS and 2 anchor questions asking whether they considered their symptom state to be satisfactory, and if not, whether they considered the treatment to have failed. PASS and TF threshold values were calculated using the adjusted predictive modeling method. Non-parametric bootstrapping was used to derive 95% confidence intervals (CI).Results - Complete 3, 12, and 24 months postoperative data was obtained for 187 of 209 (89%), 884 of 915 (97%), and 575 of 586 (98%) patients, with median ages from 68 to 70 years (59 to 64% female). 72%, 77%, and 79% considered as having satisfactory symptoms, while 6%, 11%, and 11% considered the treatment to have failed, at 3, 12, and 24 months postoperatively, respectively. OKS PASS values (CI) were 27 (26-28), 30 (29-31), and 30 (29-31) at 3, 12, and 24 months postoperatively. TF values were 27 (26-28) and 27 (26-29) at 12 and 24 months postoperatively.Interpretation - The OKS PASS values can be used to guide the interpretation of TKR outcome and support quality assessment in institutional and national registries.

Searching for the optimal number of response alternatives for the distress scale of the four-dimensional symptom questionnaire.

BACKGROUND: The Four-Dimensional Symptom Questionnaire (4DSQ) is a self-report questionnaire designed to measure distress, depression, anxiety, and somatization. Prior to computing scale scores from the item scores, the three highest response alternatives ('Regularly', 'Often', and 'Very often or constantly present') are usually collapsed into one category to reduce the influence of extreme responding on item- and scale scores. In this study, we evaluate the usefulness of this transformation for the distress scale based on a variety of criteria. METHODS: Specifically, by using the Graded Response Model, we investigated the effect of this transformation on model fit, local measurement precision, and various indicators of the scale's validity to get an indication on whether the current practice of recoding should be advocated or not. In particular, the effect on the convergent- (operationalized by the General Health Questionnaire and the Maastricht Questionnaire), divergent- (operationalized by the Neuroticism scale of the NEO-FFI), and predictive validity (operationalized as obtrusion with daily chores and activities, the Biographical Problem list and the Utrecht Burnout Scale) of the distress scale was investigated. RESULTS: Results indicate that recoding leads to (i) better model fit as indicated by lower mean probabilities of exact test statistics assessing item fit, (ii) small (<.02) losses in the sizes of various validity coefficients, and (iii) a decrease (DIFF (SE's) = .10-.25) in measurement precision for medium and high levels of distress. CONCLUSIONS: For clinical applications and applications in longitudinal research, the current practice of recoding should be avoided because recoding decreases measurement precision for medium and high levels of distress. It would be interesting to see whether this advice also holds for the three other domains of the 4DSQ.

Effectiveness of a tailored implementation strategy to improve adherence to a guideline on mental health problems in occupational health care.

BACKGROUND: As compliance to guidelines is generally low among health care providers, little is known about the impact of guidelines on the quality of delivery of care. To improve adherence to guideline recommendations on mental health problems, an implementation strategy was developed for Dutch occupational physicians (OPs). The aims were 1) to assess adherence to a mental health guideline in occupational health care and 2) to evaluate the effect of a tailored implementation strategy on guideline adherence compared to traditional guideline dissemination. METHODS: An audit of medical records was conducted as part of a larger RCT study. Participants were 66 OPs (32 intervention and 34 control) employed at one of six sites of an Occupational Health Service in southern Netherlands. OPs in the intervention group received multiple-session peer group training which focused on identifying and addressing barriers to using the guideline, using a Plan-Do-Check-Act approach. The control group did not receive training. Medical records of 114 workers sick-listed with mental health problems were assessed (56 intervention and 58 control). Guideline adherence was determined by auditing the records using 12 guideline-based performance indicators (PI), grouped into 5 PIs: process diagnosis, problem orientation, interventions/treatment, relapse prevention, and continuity of care. Differences in performance rates of the PIs between the intervention and control groups were analyzed, taking into account the cluster study design. RESULTS: OPs who received the training showed significantly greater adherence compared to the controls (p < .028) in 4 out of 5 grouped PIs, i.e. process diagnosis, problem orientation, interventions/treatment and relapse prevention. In one out of 12 PIs adherence was found adequate (53% of the medical records), in 6 PIs adherence was found minimal, and in 5 PIs the majority of the records showed no adherence. CONCLUSIONS: An implementation strategy which addressed key barriers for change and tailor-made interventions improves adherence to an occupational health guideline for mental health problems compared to traditional guideline dissemination. However, adherence to the guideline recommendations is still far from optimal. To optimize adherence, it is recommended that implementation strategies focus on the workers level, organizational level, and the professional level. TRIAL REGISTRATION: ISRCTN86605310 . Registered 30 June 2010.

Assessing the equivalence of Web-based and paper-and-pencil questionnaires using differential item and test functioning (DIF and DTF) analysis: a case of the Four-Dimensional Symptom Questionnaire (4DSQ).

PURPOSE: Many paper-and-pencil (P&P) questionnaires have been migrated to electronic platforms. Differential item and test functioning (DIF and DTF) analysis constitutes a superior research design to assess measurement equivalence across modes of administration. The purpose of this study was to demonstrate an item response theory (IRT)-based DIF and DTF analysis to assess the measurement equivalence of a Web-based version and the original P&P format of the Four-Dimensional Symptom Questionnaire (4DSQ), measuring distress, depression, anxiety, and somatization. METHODS: The P&P group (n = 2031) and the Web group (n = 958) consisted of primary care psychology clients. Unidimensionality and local independence of the 4DSQ scales were examined using IRT and Yen's Q3. Bifactor modeling was used to assess the scales' essential unidimensionality. Measurement equivalence was assessed using IRT-based DIF analysis using a 3-stage approach: linking on the latent mean and variance, selection of anchor items, and DIF testing using the Wald test. DTF was evaluated by comparing expected scale scores as a function of the latent trait. RESULTS: The 4DSQ scales proved to be essentially unidimensional in both modalities. Five items, belonging to the distress and somatization scales, displayed small amounts of DIF. DTF analysis revealed that the impact of DIF on the scale level was negligible. CONCLUSIONS: IRT-based DIF and DTF analysis is demonstrated as a way to assess the equivalence of Web-based and P&P questionnaire modalities. Data obtained with the Web-based 4DSQ are equivalent to data obtained with the P&P version.

Cross-cultural validation of the German version of the Four-Dimensional Symptom Questionnaire (4DSQ) in multimorbid elderly people.

PURPOSE: Psychosomatic symptoms and mental health problems are highly prevalent in multimorbid elderly people challenging general practitioners to differentiate between normal stress and psychopathological conditions. The 4DSQ is a Dutch questionnaire developed to detect anxiety, depression, somatization, and distress in primary care. This study aims to analyze measurement equivalence between a German version and the original Dutch instrument. METHODS: A Dutch and a German sample of multimorbid elderly people, matched by gender and age, were analyzed. Equivalence of scale structures was assessed by confirmatory factor analysis (CFA). To evaluate measurement equivalence across languages, differential item functioning (DIF) was analyzed using Mantel-Haenszel method and hybrid ordinal logistic regression analysis. Differential test functioning (DTF) was assessed using Rasch analysis. RESULTS: A total of 185 German and 185 Dutch participants completed the questionnaire. The CFA confirmed one-factor models for all scales of both 4DSQ versions. Nine items in three scales were flagged with DIF. The anxiety scale showed to be free of DIF. DTF analysis revealed negligible scale impact of DIF. CONCLUSIONS: The German 4DSQ demonstrated measurement equivalence to the original Dutch instrument. Hence, it can be considered a valid questionnaire for the screening for mental health problems in primary care.