Scoring Depression on a Common Metric: A Comparison of EAP Estimation, Plausible Value Imputation, and Full Bayesian IRT Modeling.

There are a growing number of item response theory (IRT) studies that calibrate different patient-reported outcome (PRO) measures, such as anxiety, depression, physical function, and pain, on common, instrument-independent metrics. In the case of depression, it has been reported that there are considerable mean score differences when scoring on a common metric from different, previously linked instruments. Ideally, those estimates should be the same. We investigated to what extent those differences are influenced by different scoring methods that take into account several levels of uncertainty, such as measurement error (through plausible value imputation) and item parameter uncertainty (through full Bayesian IRT modeling). Depression estimates from different instruments were more similar, and their corresponding confidence/credible intervals were larger when plausible value imputation or Bayesian modeling was used, compared to the direct use of expected a posteriori (EAP) estimates. Furthermore, we explored the use of Bayesian IRT models to update item parameters based on newly collected data.

An initial psychometric evaluation of the German PROMIS v1.2 Physical Function item bank in patients with a wide range of health conditions.

OBJECTIVES: To translate the PROMIS Physical Function (PF) item bank version 1.2 into German and to investigate psychometric properties of resulting full bank and seven derived short forms. DESIGN: Cross-sectional psychometric study. SETTING: Inpatient and outpatient clinics of the Department of Psychosomatic Medicine at Charité-Universitätsmedizin Berlin, Germany. SUBJECTS: A total of 10 adult patients with various chronic diseases participated in cognitive debriefing interviews. The final item bank was administered to n = 266 adult patients with a broad range of medical conditions. INTERVENTIONS: Patient-reported outcome assessment as part of routine care. MAIN MEASURES: PROMIS v1.2 PF bank; MOS SF-36 PF scale (PF-10). RESULTS: Cross-cultural adaptation of the item bank followed established guidelines. For the final German translation, the corrected item-total correlations ranged from 0.44 to 0.84. Cronbach's alpha was high for each PROMIS PF short form ( α = 0.88-0.96). The full PROMIS PF bank and most short forms correlated highly with the SF-36 PF-10 ( r = 0.85-0.90), with the exception of PROMIS Upper Extremity ( r = 0.64). PROMIS Upper Extremity showed ceiling effects and lower agreement with the full bank than other short forms. Unidimensionality was supported for all PROMIS PF measures using traditional factor analysis and nonparametric item response theory. CONCLUSION: The German PROMIS PF bank was found to be conceptually equivalent to the English version and fulfilled the psychometric requirements for use of short forms in clinical practice. Future studies should pay particular attention to samples with upper extremity functional limitations to further investigate the dimensional structure of PF as conceptualized according to PROMIS.

www.common-metrics.org: a web application to estimate scores from different patient-reported outcome measures on a common scale.

BACKGROUND: Recently, a growing number of Item-Response Theory (IRT) models has been published, which allow estimation of a common latent variable from data derived by different Patient Reported Outcomes (PROs). When using data from different PROs, direct estimation of the latent variable has some advantages over the use of sum score conversion tables. It requires substantial proficiency in the field of psychometrics to fit such models using contemporary IRT software. We developed a web application ( http://www.common-metrics.org ), which allows estimation of latent variable scores more easily using IRT models calibrating different measures on instrument independent scales. RESULTS: Currently, the application allows estimation using six different IRT models for Depression, Anxiety, and Physical Function. Based on published item parameters, users of the application can directly estimate latent trait estimates using expected a posteriori (EAP) for sum scores as well as for specific response patterns, Bayes modal (MAP), Weighted likelihood estimation (WLE) and Maximum likelihood (ML) methods and under three different prior distributions. The obtained estimates can be downloaded and analyzed using standard statistical software. CONCLUSIONS: This application enhances the usability of IRT modeling for researchers by allowing comparison of the latent trait estimates over different PROs, such as the Patient Health Questionnaire Depression (PHQ-9) and Anxiety (GAD-7) scales, the Center of Epidemiologic Studies Depression Scale (CES-D), the Beck Depression Inventory (BDI), PROMIS Anxiety and Depression Short Forms and others. Advantages of this approach include comparability of data derived with different measures and tolerance against missing values. The validity of the underlying models needs to be investigated in the future.

Screening for mental disorders in heart failure patients using computer-adaptive tests.

PURPOSE: Item response theory is increasingly used in the development of psychometric tests. This paper evaluates whether these modern psychometric methods can improve self-reported screening for depression and anxiety in patients with heart failure. METHODS: The mental health status of 194 patients with heart failure was assessed using six screening tools for depression (Patient Health Questionnaire -9 (9 items), Hospital Anxiety and Depression Scale (HADS) (7 items), PROMIS-Depression Short Form 8a (8 items)) and Anxiety (GAD-7 (7 items), Hospital Anxiety and Depression Scale (HADS) (7 items), PROMIS-Anxiety Short Form 8a (8 items)). An in-person structured clinical interview was used as the current gold standard to identify the presence of a mental disorder. The diagnostic accuracy of all static tools was compared when item response theory (IRT)-based person parameter were estimated instead of sum scores. Furthermore, we compared performance of static instruments with post hoc simulated individual-tailored computer-adaptive test (CATs) for both disorders and a common negative affect CAT. RESULTS: In general, screening for depression was highly efficient and showed a better performance than screening for anxiety with only minimal differences among the assessed instruments. IRT-based person parameters yielded the same diagnostic accuracy as sum scores. CATs showed similar screening performance compared to legacy instruments but required significantly fewer items to identify patients without mental conditions. Ideal cutoffs varied between male and female samples. CONCLUSIONS: Overall, the diagnostic performance of all investigated instruments was similar, regardless of the methods being used. However, CATs can individually tailor the test to each patient, thus significantly decreasing the respondent burden for patients with and without mental conditions. Such approach could efficiently increase the acceptability of mental health screening in clinical practice settings.

Using subdomain-specific item sets affected PROMIS physical function scores differently in cardiology and rheumatology patients.

OBJECTIVES: The Patient-Reported Outcomes Measurement Information System (PROMIS) physical function (PF) item bank has been developed to standardize patient-reported PF across medical fields. However, evidence of scoring equivalence across cardiology and rheumatology patients is still missing. Therefore, this study aims to investigate both (1) the extent of disease-related differential item functioning (DIF) and (2) the impact of the disease group on using subdomain-specific item sets for generating PROMIS PF scores in cardiology and rheumatology patients. STUDY DESIGN AND SETTING: Ordinal regression was used to evaluate DIF between cardiology (n = 201) and rheumatology (n = 200) inpatients. To explore the disease-specific impact of PF subdomains on scoring, we compared scores derived from the full item bank with scores derived from subdomain-specific item sets for each disease group. RESULTS: DIF was detected in 18 items, predominately from the upper extremity subdomain. When upper extremity items were used, cardiology patients reached systematically higher scores than using the full item bank. Rheumatology patients scored substantially higher when mobility items were used. CONCLUSION: Applying the PROMIS PF metric to disease-specific item sets including items from differing subdomains may lead to biased comparisons of PF levels across disease groups. Disease-specific item parameters should be provided for items showing DIF, and subdomain-related content balancing is recommended for scoring the generic PROMIS PF construct.

Language-related differential item functioning between English and German PROMIS Depression items is negligible.

To investigate differential item functioning (DIF) of PROMIS Depression items between US and German samples we compared data from the US PROMIS calibration sample (n = 780), a German general population survey (n = 2,500) and a German clinical sample (n = 621). DIF was assessed in an ordinal logistic regression framework, with 0.02 as criterion for R2 -change and 0.096 for Raju's non-compensatory DIF. Item parameters were initially fixed to the PROMIS Depression metric; we used plausible values to account for uncertainty in depression estimates. Only four items showed DIF. Accounting for DIF led to negligible effects for the full item bank as well as a post hoc simulated computer-adaptive test (< 0.1 point on the PROMIS metric [mean = 50, standard deviation =10]), while the effect on the short forms was small (< 1 point). The mean depression severity (43.6) in the German general population sample was considerably lower compared to the US reference value of 50. Overall, we found little evidence for language DIF between US and German samples, which could be addressed by either replacing the DIF items by items not showing DIF or by scoring the short form in German samples with the corrected item parameters reported.

Varying the item format improved the range of measurement in patient-reported outcome measures assessing physical function.

BACKGROUND: Physical function (PF) is a core patient-reported outcome domain in clinical trials in rheumatic diseases. Frequently used PF measures have ceiling effects, leading to large sample size requirements and low sensitivity to change. In most of these instruments, the response category that indicates the highest PF level is the statement that one is able to perform a given physical activity without any limitations or difficulty. This study investigates whether using an item format with an extended response scale, allowing respondents to state that the performance of an activity is easy or very easy, increases the range of precise measurement of self-reported PF. METHODS: Three five-item PF short forms were constructed from the Patient-Reported Outcomes Measurement Information System (PROMIS®) wave 1 data. All forms included the same physical activities but varied in item stem and response scale: format A ("Are you able to …"; "without any difficulty"/"unable to do"); format B ("Does your health now limit you …"; "not at all"/"cannot do"); format C ("How difficult is it for you to …"; "very easy"/"impossible"). Each short-form item was answered by 2217-2835 subjects. We evaluated unidimensionality and estimated a graded response model for the 15 short-form items and remaining 119 items of the PROMIS PF bank to compare item and test information for the short forms along the PF continuum. We then used simulated data for five groups with different PF levels to illustrate differences in scoring precision between the short forms using different item formats. RESULTS: Sufficient unidimensionality of all short-form items and the original PF item bank was supported. Compared to formats A and B, format C increased the range of reliable measurement by about 0.5 standard deviations on the positive side of the PF continuum of the sample, provided more item information, and was more useful in distinguishing known groups with above-average functioning. CONCLUSIONS: Using an item format with an extended response scale is an efficient option to increase the measurement range of self-reported physical function without changing the content of the measure or affecting the latent construct of the instrument.

The effect of attending steiner schools during childhood on health in adulthood: a multicentre cross-sectional study.

BACKGROUND: It is speculated that attending Steiner schools, whose pedagogical principles include an account for healthy psycho-physical development, may have long-term beneficial health effects. We examined whether the current health status differed between former attendees of German Steiner schools and adults from the general population. Furthermore, we examined factors that might explain those differences. METHODS: We included former Steiner school attendees from 4 schools in Berlin, Hanover, Nuremberg and Stuttgart and randomly selected population controls. Using a self-report questionnaire we assessed sociodemographics, current and childhood lifestyle and health status. Outcomes were self-reports on 16 diseases: atopic dermatitis, allergic rhinitis, bronchial asthma, chronic obstructive pulmonary disease (COPD), cardiac arrhythmia, cardiac insufficiency, angina pectoris, arteriosclerosis, hypertension, hypercholesterolemia, osteoarthritis, rheumatism, cancer, diabetes, depression and multiple sclerosis. Furthermore, participants rated the symptom burden resulting from back pain, cold symptoms, headache, insomnia, joint pain, gastrointestinal symptoms and imbalance. Unadjusted and adjusted odds ratios were calculated for each outcome. RESULTS: 1136 Steiner school attendees and 1746 controls were eligible for analysis. Both groups were comparable regarding sex, age and region, but differed in nationality and educational status. After adjusting for possible confounders, we found statistically significant effects of Steiner school attendance for osteoarthritis (OR 0.69 [0.49-0.97]) and allergic rhinitis (OR 0.77, [0.59-1.00]) as well as for symptom burden from back pain (OR 0.80, [0.64-1.00]), insomnia (OR 0.65, [0.50-0.84]), joint pain (OR 0.62, [0.48-0.82]), gastrointestinal symptoms (OR 0.76, [0.58-1.00]) and imbalance (OR 0.60, [0.38-0.93]). CONCLUSIONS: The risk of most examined diseases did not differ between former Steiner school attendees and the general population after adjustment for sociodemographics, current and childhood lifestyle features, but symptom burden from some current health complaints was reported less by former Steiner school attendees. Results must be interpreted with caution since the analysis was exploratory.

Key issues in clinical and epidemiological research in complementary and alternative medicine--a systematic literature review.

BACKGROUND: In the last 2 decades there has been a large increase in publications on complementary and alternative medicine (CAM). However, CAM research methodology was heterogeneous and often of low quality. The aim of this systematic review was to investigate scientific publications with regards to general issues, concepts and strategies. We also looked at research priorities and methods employed to evaluate the clinical and epidemiological research of CAM in the past to identify the basis for consensus-based research strategies. METHODS: We performed a systematic literature search for papers published between 1990 and 2010 in 7 electronic databases (Medline, Web of Science, PsychArticles, PsycInfo, CINAHL, EMBASE and Cochrane Library) on December 16 and 17, 2010. In addition, experts were asked to nominate relevant papers. Inclusion criteria were publications dealing with research methodology, priorities or complexities in the scientific evaluation of CAM. All references were assessed in a multistage process to identify relevant papers. RESULTS: From the 3,279 references derived from the search and 98 references contributed by CAM experts, 170 papers fulfilled the criteria and were included in the analysis. The following key issues were identified: difficulties in past CAM research (e.g., randomisation, blinding), utility of quantitative and qualitative research methods in CAM, priority setting in CAM research and specific issues regarding various CAM modalities. CONCLUSIONS: Most authors vote for the use of commonly accepted research methods to evaluate CAM. There was broad consensus that a mixed methods approach is the most suitable for gathering conclusive knowledge about CAM.

How to compare scores from different depression scales: equating the Patient Health Questionnaire (PHQ) and the ICD-10-Symptom Rating (ISR) using Item Response Theory.

A wide range of questionnaires for measuring depression are available. Item Response Theory models can help to evaluate the questionnaires exceeding the boundaries of Classical Test Theory and provide an opportunity to equate the questionnaires. In this study after checking for unidimensionality, a General Partial Credit Model was applied to data from two different depression scales [Patient Health Questionnaire (PHQ-9) and ICD-10-Symptom Rating (ISR)] obtained in clinical settings from a consecutive sample, including 4517 observations from a total of 2999 inpatients and outpatients of a psychosomatic clinic. The precision of each questionnaire was compared and the model was used to transform scores based on the assumed underlying latent trait. Both instruments were constructed to measure the same construct and their estimates of depression severity are highly correlated. Our analysis showed that the predicted scores provided by the conversion tables are similar to the observed scores in a validation sample. The PHQ-9 and ISR depression scales measure depression severity across a broad range with similar precision. While the PHQ-9 shows advantages in measuring low or high depression severity, the ISR is more parsimonious and also suitable for clinical purposes. Furthermore, the equation tables derived in this study enhance the comparability of studies using either one of the instruments, but due to substantial statistical spread the comparison of individual scores is imprecise.