An Item Bank for Abuse of Prescription Pain Medication from the Patient-Reported Outcomes Measurement Information System (PROMIS®).

OBJECTIVE: There is a need to monitor patients receiving prescription opioids to detect possible signs of abuse. To address this need, we developed and calibrated an item bank for severity of abuse of prescription pain medication as part of the Patient-Reported Outcomes Measurement Information System (PROMIS ® ). METHODS: Comprehensive literature searches yielded an initial bank of 5,310 items relevant to substance use and abuse, including abuse of prescription pain medication, from over 80 unique instruments. After qualitative item analysis (i.e., focus groups, cognitive interviewing, expert review, and item revision), 25 items for abuse of prescribed pain medication were included in field testing. Items were written in a first-person, past-tense format, with a three-month time frame and five response options reflecting frequency or severity. The calibration sample included 448 respondents, 367 from the general population (ascertained through an internet panel) and 81 from community treatment programs participating in the National Drug Abuse Treatment Clinical Trials Network. RESULTS: A final bank of 22 items was calibrated using the two-parameter graded response model from item response theory. A seven-item static short form was also developed. The test information curve showed that the PROMIS ® item bank for abuse of prescription pain medication provided substantial information in a broad range of severity. CONCLUSION: The initial psychometric characteristics of the item bank support its use as a computerized adaptive test or short form, with either version providing a brief, precise, and efficient measure relevant to both clinical and community samples.

Individuals with knee impairments identify items in need of clarification in the Patient Reported Outcomes Measurement Information System (PROMIS®) pain interference and physical function item banks - a qualitative study.

BACKGROUND: The content and wording of the Patient Reported Outcome Measurement Information System (PROMIS) Physical Function and Pain Interference item banks have not been qualitatively assessed by individuals with knee joint impairments. The purpose of this investigation was to identify items in the PROMIS Physical Function and Pain Interference Item Banks that are irrelevant, unclear, or otherwise difficult to respond to for individuals with impairment of the knee and to suggest modifications based on cognitive interviews. METHODS: Twenty-nine individuals with knee joint impairments qualitatively assessed items in the Pain Interference and Physical Function Item Banks in a mixed-methods cognitive interview. Field notes were analyzed to identify themes and frequency counts were calculated to identify items not relevant to individuals with knee joint impairments. RESULTS: Issues with clarity were identified in 23 items in the Physical Function Item Bank, resulting in the creation of 43 new or modified items, typically changing words within the item to be clearer. Interpretation issues included whether or not the knee joint played a significant role in overall health and age/gender differences in items. One quarter of the original items (31 of 124) in the Physical Function Item Bank were identified as irrelevant to the knee joint. All 41 items in the Pain Interference Item Bank were identified as clear, although individuals without significant pain substituted other symptoms which interfered with their life. CONCLUSIONS: The Physical Function Item Bank would benefit from additional items that are relevant to individuals with knee joint impairments and, by extension, to other lower extremity impairments. Several issues in clarity were identified that are likely to be present in other patient cohorts as well.

Evaluating PROMIS® instruments and methods for patient-centered outcomes research: Patient and provider voices in a substance use treatment setting.

PURPOSE: Our work as a primary research site of the Patient-Reported Outcomes Measurement Information System (PROMIS®), combined with support from the Patient-Centered Outcomes Research Institute, allowed us to evaluate the real-world applicability and acceptability of PROMIS measures in an addiction medicine setting. METHODS: As part of a 3-month prospective observational study, 225 outpatients at a substance abuse treatment clinic completed PROMIS item banks for alcohol use (as well as 15 additional item banks from 8 other PROMIS domains, including emotional distress, sleep, and pain), with assessments at intake, 1-month follow-up, and 3-month follow-up. A subsample of therapists and their patients completed health domain importance ratings and qualitative interviews to elicit feedback regarding the content and format of the patients' assessment results. RESULTS: The importance ratings revealed that depression, anxiety, and lack of emotional support were rated highest of the non-alcohol-related domains among both patients and clinicians. General alcohol use was considered most important by both patients and clinicians. Based on their suggestions, changes were made to item response feedback to facilitate comprehension and communication. CONCLUSIONS: Both therapists and patients agreed that their review of the graphical display of scores, as well as individual item responses, helped them to identify areas of greatest concern and was useful for treatment planning. The results of our pilot work demonstrated the value and practicality of incorporating a comprehensive health assessment within a substance abuse treatment setting.

Measuring nonspecific factors in treatment: item banks that assess the healthcare experience and attitudes from the patient's perspective.

PURPOSE: Nonspecific factors that accompany healthcare treatments, such as patients' attitudes and expectations, are important parts of the experience of care and can influence outcomes. However, no precise, concise, and generalizable instruments to measure these factors exist. We report on the development and calibration of new item banks, titled the Healing Encounters and Attitudes Lists (HEAL), that assess nonspecific factors across a broad range of treatments and conditions. METHODS: The instrument development methodology of the Patient-Reported Outcomes Measurement Information System (PROMIS(®)) was used. Patient focus groups and clinician interviews informed our HEAL conceptual model. Literature searches of eight databases yielded over 500 instruments and resulted in an initial item pool of several thousand items. After qualitative item analysis, including cognitive interviewing, 296 items were included in field testing. The calibration sample included 1657 respondents, 1400 obtained through an Internet panel and 257 from conventional and integrative medicine clinics. Following exploratory and confirmatory factor analyses, the HEAL item banks were calibrated using item response theory. RESULTS: The final HEAL item banks were Patient-Provider Connection (57 items), Healthcare Environment (25 items), Treatment Expectancy (27 items), Positive Outlook (27 items), and Spirituality (26 items). Short forms were also developed from each item bank. A six-item short form, Attitudes toward Complementary and Alternative Medicine (CAM), was also created. CONCLUSIONS: HEAL item banks provided substantial information across a broad range of each construct. HEAL item banks showed initial evidence of predictive and concurrent validity, suggesting that they are suitable for measuring nonspecific factors in treatment.

Using peer review to improve research and promote collaboration.

OBJECTIVE: The declining success rate of National Institutes of Health (NIH) grant applications highlights the need for interdisciplinary work within a large, diverse department to improve chances of federal funding success. The authors demonstrate how systematic peer review promotes two goals: enhancing the quality of research proposals and cultivating a collaborative departmental culture. METHODS: Changes to the Research Review Committee (RRC) in the Department of Psychiatry at the University of Pittsburgh were instituted to accommodate the increasingly interdisciplinary nature of grant applications, integrate revisions to NIH grant application processes, and incorporate advances in computer technology. RESULTS: The internal peer review process is associated with success in obtaining research support and with significant levels of collaborative scientific work reflected in both grant applications and peer-reviewed publications. CONCLUSIONS: A rich collaborative environment promoted through a rigorous internal peer review system has many benefits for both the quality of scholarly work and the collegiality of the research environment.

Development of short forms from the PROMIS™ sleep disturbance and Sleep-Related Impairment item banks.

This article reports on the development of short forms from the Patient-Reported Outcomes Measurement Information System (PROMIS™) Sleep Disturbance (SD) and Sleep-Related Impairment (SRI) item banks. Results from post-hoc computerized adaptive testing (CAT) simulations, item discrimination parameters, item means, and clinical judgments were used to select the best-performing 8 items for SD and SRI. The final 8-item short forms provided less test information than the corresponding full banks, but correlated strongly with the longer forms. The short forms had greater measurement precision than the Pittsburgh Sleep Quality Index (PSQI) and the Epworth Sleepiness Scale (ESS), as indicated by larger test information values across the continuum of severity, despite having fewer total items--a major advantage for both research and clinical settings.

Validation of the Three-Item Screener for Personality Disorders From the Inventory of Interpersonal Problems (IIP-3).

We previously developed a three-item screener for identifying respondents with any personality disorder (PD) using the Inventory of Interpersonal Problems (IIP). The current goal was to examine the convergent validity of the IIP-3 with other PD screeners and diagnostic tools and to investigate its relationship to measures of adult attachment and emotion regulation. The sample consisted of participants from five studies (total N = 852), with data from collateral informants available for a subsample (N = 353). Despite its brevity, the IIP-3 showed moderate to strong relationships with other longer PD screeners, with PD symptom scores from the Structured Interview for DSM-IV Personality (SIDP-IV), and with a global rating of PD severity. It was most sensitive to the stylistic aspects of PD typical of the traditional DSM cluster B (dramatic, expressive) PDs. These results emerged with data from both participants and informants, although correlations using informant data were generally smaller.

Development and validation of patient-reported outcome measures for sleep disturbance and sleep-related impairments.

STUDY OBJECTIVES: To develop an archive of self-report questions assessing sleep disturbance and sleep-related impairments (SRI), to develop item banks from this archive, and to validate and calibrate the item banks using classic validation techniques and item response theory analyses in a sample of clinical and community participants. DESIGN: Cross-sectional self-report study. SETTING: Academic medical center and participant homes. PARTICIPANTS: One thousand nine hundred ninety-three adults recruited from an Internet polling sample and 259 adults recruited from medical, psychiatric, and sleep clinics. INTERVENTIONS: None. MEASUREMENTS AND RESULTS: This study was part of PROMIS (Patient-Reported Outcomes Information System), a National Institutes of Health Roadmap initiative. Self-report item banks were developed through an iterative process of literature searches, collecting and sorting items, expert content review, qualitative patient research, and pilot testing. Internal consistency, convergent validity, and exploratory and confirmatory factor analysis were examined in the resulting item banks. Factor analyses identified 2 preliminary item banks, sleep disturbance and SRI. Item response theory analyses and expert content review narrowed the item banks to 27 and 16 items, respectively. Validity of the item banks was supported by moderate to high correlations with existing scales and by significant differences in sleep disturbance and SRI scores between participants with and without sleep disorders. CONCLUSIONS: The PROMIS sleep disturbance and SRI item banks have excellent measurement properties and may prove to be useful for assessing general aspects of sleep and SRI with various groups of patients and interventions.

Screening for Personality Disorders: A Three-Item Screener From the Inventory of Interpersonal Problems (IIP-3).

To encourage screening for personality disorders (PDs), we developed (in previous work) self-report scales for PDs using the Inventory of Interpersonal Problems (IIP). The combined score from three of the scales-inter-personal sensitivity, interpersonal ambivalence, and aggression-requiring 15 items (IIP-15) did the best job of distinguishing between respondents with any versus no PD. The goals for the present work were (a) to cross-validate the IIP-15 by examining its performance using receiver operating characteristics (ROC) analyses in a new sample (N = 410), and (b) to investigate the utility of a brief three-item variant (IIP-3). The present results again documented the good operating characteristics of the IIP-15. Sensitivity, specificity, and positive and negative predictive values were all above. 70. The operating characteristics of the IIP-3 were nearly as good despite its brevity and support its use as an initial screen for PDs.

Validation of the alcohol use item banks from the Patient-Reported Outcomes Measurement Information System (PROMIS).

BACKGROUND: The Patient-Reported Outcomes Measurement Information System (PROMIS) includes five item banks for alcohol use. There are limited data, however, regarding their validity (e.g., convergent validity, responsiveness to change). To provide such data, we conducted a prospective study with 225 outpatients being treated for substance abuse. METHODS: Assessments were completed shortly after intake and at 1-month and 3-month follow-ups. The alcohol item banks were administered as computerized adaptive tests (CATs). Fourteen CATs and one six-item short form were also administered from eight other PROMIS domains to generate a comprehensive health status profile. After modeling treatment outcome for the sample as a whole, correlates of outcome from the PROMIS health status profile were examined. RESULTS: For convergent validity, the largest correlation emerged between the PROMIS alcohol use score and the Alcohol Use Disorders Identification Test (r=.79 at intake). Regarding treatment outcome, there were modest changes across the target problem of alcohol use and other domains of the PROMIS health status profile. However, significant heterogeneity was found in initial severity of drinking and in rates of change for both abstinence and severity of drinking during follow-up. This heterogeneity was associated with demographic (e.g., gender) and health-profile (e.g., emotional support, social participation) variables. CONCLUSIONS: The results demonstrated the validity of PROMIS CATs, which require only 4-6 items in each domain. This efficiency makes it feasible to use a comprehensive health status profile within the substance use treatment setting, providing important prognostic information regarding abstinence and severity of drinking.

Item banks for substance use from the Patient-Reported Outcomes Measurement Information System (PROMIS(®)): Severity of use and positive appeal of use.

BACKGROUND: Two item banks for substance use were developed as part of the Patient-Reported Outcomes Measurement Information System (PROMIS(®)): severity of substance use and positive appeal of substance use. METHODS: Qualitative item analysis (including focus groups, cognitive interviewing, expert review, and item revision) reduced an initial pool of more than 5300 items for substance use to 119 items included in field testing. Items were written in a first-person, past-tense format, with 5 response options reflecting frequency or severity. Both 30-day and 3-month time frames were tested. The calibration sample of 1336 respondents included 875 individuals from the general population (ascertained through an internet panel) and 461 patients from addiction treatment centers participating in the National Drug Abuse Treatment Clinical Trials Network. RESULTS: Final banks of 37 and 18 items were calibrated for severity of substance use and positive appeal of substance use, respectively, using the two-parameter graded response model from item response theory (IRT). Initial calibrations were similar for the 30-day and 3-month time frames, and final calibrations used data combined across the time frames, making the items applicable with either interval. Seven-item static short forms were also developed from each item bank. CONCLUSIONS: Test information curves showed that the PROMIS item banks provided substantial information in a broad range of severity, making them suitable for treatment, observational, and epidemiological research in both clinical and community settings.

Validation of the depression item bank from the Patient-Reported Outcomes Measurement Information System (PROMIS) in a three-month observational study.

The Patient-Reported Outcomes Measurement Information System (PROMIS) is an NIH Roadmap initiative devoted to developing better measurement tools for assessing constructs relevant to the clinical investigation and treatment of all diseases-constructs such as pain, fatigue, emotional distress, sleep, physical functioning, and social participation. Following creation of item banks for these constructs, our priority has been to validate them, most often in short-term observational studies. We report here on a three-month prospective observational study with depressed outpatients in the early stages of a new treatment episode (with assessments at intake, one-month follow-up, and three-month follow-up). The protocol was designed to compare the psychometric properties of the PROMIS depression item bank (administered as a computerized adaptive test, CAT) with two legacy self-report instruments: the Center for Epidemiological Studies Depression scale (CESD; Radloff, 1977) and the Patient Health Questionnaire (PHQ-9; Spitzer et al., 1999). PROMIS depression demonstrated strong convergent validity with the CESD and the PHQ-9 (with correlations in a range from .72 to .84 across all time points), as well as responsiveness to change when characterizing symptom severity in a clinical outpatient sample. Identification of patients as "recovered" varied across the measures, with the PHQ-9 being the most conservative. The use of calibrations based on models from item response theory (IRT) provides advantages for PROMIS depression both psychometrically (creating the possibility of adaptive testing, providing a broader effective range of measurement, and generating greater precision) and practically (these psychometric advantages can be achieved with fewer items-a median of 4 items administered by CAT-resulting in less patient burden).