COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study.

BACKGROUND: Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designed and analyzed. The aim of this study was to develop standards to assess the quality of studies on reliability and measurement error of clinician-reported outcome measurement instruments, performance-based outcome measurement instrument, and laboratory values. METHODS: We conducted a 3-round Delphi study involving 52 panelists. RESULTS: Consensus was reached on how a comprehensive research question can be deduced from the design of a reliability study to determine how the results of a study inform us about the quality of the outcome measurement instrument at issue. Consensus was reached on components of outcome measurement instruments, i.e. the potential sources of variation. Next, we reached consensus on standards on design requirements (n = 5), standards on preferred statistical methods for reliability (n = 3) and measurement error (n = 2), and their ratings on a four-point scale. There was one term for a component and one rating of one standard on which no consensus was reached, and therefore required a decision by the steering committee. CONCLUSION: We developed a tool that enables researchers with and without thorough knowledge on measurement properties to assess the quality of a study on reliability and measurement error of outcome measurement instruments.

Circumstances Surrounding High-risk Sexual Experiences Among Primary Care Patients Living With and Without HIV.

INTRODUCTION: Rates of sexually transmitted infection (STI) are rising in the USA, yet STI risk remains under-addressed by providers, even in HIV care, and with high-risk patients. We interviewed primary care patients living with and without HIV regarding circumstances surrounding sexual risk behavior to identify opportunities for providers to address and reduce STI risk. METHODS: We conducted semi-structured 1:1 interviews with patients living with and without HIV reporting ≥ 1 sex partner and varying STI exposure risk in the past 12 months from four geographically diverse US HIV and primary care clinics. We audio-recorded, transcribed, and coded interviews by circumstance type, using double-coding to ensure inter-coder reliability. We used Fisher's exact and T tests to determine associations with demographic/risk factors. RESULTS: Patients (n = 91) identified a mean of 3 of 11 circumstances. These included substance use (54%), desire for physical/emotional intimacy (48%), lack of HIV/STI status disclosure (44%), psychological drivers (i.e., coping, depression; 38%), personal dislike of condoms (22%), partner condom dislike/refusal (19%), receiving payment for sex (13%), and condom unavailability (9%). Higher proportions of those who were high STI-exposure risk patients, defined as those with ≥ 2 sex partners in the past 3 months reporting never or sometimes using condoms, reported disliking condoms (p = .006); higher proportions of the high-risk and moderate-risk (≥ 2 partners and condom use "most of the time") groups reported substance use as a circumstance (p = .04). CONCLUSION: Circumstances surrounding perceived STI exposure risk were diverse, often overlapping, and dependent on internal, environmental, and partner-related factors and inadequate communication. Meaningful care-based interventions regarding HIV/STI transmission behavior must address the diversity and interplay of these factors.

COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures.

PURPOSE: The original COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to assess the methodological quality of single studies on measurement properties of Patient-Reported Outcome Measures (PROMs). Now it is our aim to adapt the COSMIN checklist and its four-point rating system into a version exclusively for use in systematic reviews of PROMs, aiming to assess risk of bias of studies on measurement properties. METHODS: For each standard (i.e., a design requirement or preferred statistical method), it was discussed within the COSMIN steering committee if and how it should be adapted. The adapted checklist was pilot-tested to strengthen content validity in a systematic review on the quality of PROMs for patients with hand osteoarthritis. RESULTS: Most important changes were the reordering of the measurement properties to be assessed in a systematic review of PROMs; the deletion of standards that concerned reporting issues and standards that not necessarily lead to biased results; the integration of standards on general requirements for studies on item response theory with standards for specific measurement properties; the recommendation to the review team to specify hypotheses for construct validity and responsiveness in advance, and subsequently the removal of the standards about formulating hypotheses; and the change in the labels of the four-point rating system. CONCLUSIONS: The COSMIN Risk of Bias checklist was developed exclusively for use in systematic reviews of PROMs to distinguish this application from other purposes of assessing the methodological quality of studies on measurement properties, such as guidance for designing or reporting a study on the measurement properties.

COSMIN guideline for systematic reviews of patient-reported outcome measures.

PURPOSE: Systematic reviews of patient-reported outcome measures (PROMs) differ from reviews of interventions and diagnostic test accuracy studies and are complex. In fact, conducting a review of one or more PROMs comprises of multiple reviews (i.e., one review for each measurement property of each PROM). In the absence of guidance specifically designed for reviews on measurement properties, our aim was to develop a guideline for conducting systematic reviews of PROMs. METHODS: Based on literature reviews and expert opinions, and in concordance with existing guidelines, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) steering committee developed a guideline for systematic reviews of PROMs. RESULTS: A consecutive ten-step procedure for conducting a systematic review of PROMs is proposed. Steps 1-4 concern preparing and performing the literature search, and selecting relevant studies. Steps 5-8 concern the evaluation of the quality of the eligible studies, the measurement properties, and the interpretability and feasibility aspects. Steps 9 and 10 concern formulating recommendations and reporting the systematic review. CONCLUSIONS: The COSMIN guideline for systematic reviews of PROMs includes methodology to combine the methodological quality of studies on measurement properties with the quality of the PROM itself (i.e., its measurement properties). This enables reviewers to draw transparent conclusions and making evidence-based recommendations on the quality of PROMs, and supports the evidence-based selection of PROMs for use in research and in clinical practice.

COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.

BACKGROUND: Content validity is the most important measurement property of a patient-reported outcome measure (PROM) and the most challenging to assess. Our aims were to: (1) develop standards for evaluating the quality of PROM development; (2) update the original COSMIN standards for assessing the quality of content validity studies of PROMs; (3) develop criteria for what constitutes good content validity of PROMs, and (4) develop a rating system for summarizing the evidence on a PROM's content validity and grading the quality of the evidence in systematic reviews of PROMs. METHODS: An online 4-round Delphi study was performed among 159 experts from 21 countries. Panelists rated the degree to which they (dis)agreed to proposed standards, criteria, and rating issues on 5-point rating scales ('strongly disagree' to 'strongly agree'), and provided arguments for their ratings. RESULTS: Discussion focused on sample size requirements, recording and field notes, transcribing cognitive interviews, and data coding. After four rounds, the required 67% consensus was reached on all standards, criteria, and rating issues. After pilot-testing, the steering committee made some final changes. Ten criteria for good content validity were defined regarding item relevance, appropriateness of response options and recall period, comprehensiveness, and comprehensibility of the PROM. DISCUSSION: The consensus-based COSMIN methodology for content validity is more detailed, standardized, and transparent than earlier published guidelines, including the previous COSMIN standards. This methodology can contribute to the selection and use of high-quality PROMs in research and clinical practice.

Validity assessment of the PROMIS fatigue domain among people living with HIV.

PURPOSE: To evaluate psychometric characteristics and cross-sectional and longitudinal validity of the 7-item PROMIS® Fatigue Short Form and additional fatigue items among people living with HIV (PLWH) in a nationally distributed network of clinics collecting patient reported data at the time of routine clinical care. METHODS: Cross-sectional and longitudinal fatigue data were collected from September 2012 through April 2013 across clinics participating in the Centers for AIDS Research Network of Integrated Clinical Systems (CNICS). We analyzed data regarding psychometric characteristics including simulated computerized adaptive testing and differential item functioning, and regarding associations with clinical characteristics. RESULTS: We analyzed data from 1597 PLWH. Fatigue was common in this cohort. Scores from the PROMIS® Fatigue Short Form and from the item bank had acceptable psychometric characteristics and strong evidence for validity, but neither performed better than shorter instruments already integrated in CNICS. CONCLUSIONS: The PROMIS® Fatigue Item Bank is a valid approach to measuring fatigue in clinical care settings among PLWH, but in our analyses did not perform better than instruments associated with less respondent burden.

Next Generation Multiresponsive Nanocarriers for Targeted Drug Delivery to Cancer Cells.

C-H bond activation of 2-methoxyethylamino-bis(phenolate)-yttrium catalysts allowed the synthesis of BAB block copolymers comprised of 2-vinylpyridine (2VP; monomer A) and diethylvinylphosphonate (DEVP; monomer B) as the A and B blocks, respectively, by rare-earth-metal-mediated group-transfer polymerization (REM-GTP). The inherent multi-stimuli-responsive character and drug-loading and -release capabilities were observed to be dependent on the chain length and monomer ratios. Cytotoxicity assays revealed the biocompatibility and nontoxic nature of the obtained micelles toward ovarian cancer (HeLa) cells. The BAB block copolymers effectively encapsulated, transported, and released doxorubicin (DOX) within HeLa cells. REM-GTP enables access to previously unattainable vinylphosphonate copolymer structures, and thereby unlocks their full potential as nanocarriers for stimuli-responsive drug delivery in HeLa cells. The self-evident consequence is the application of these new micelles as potent drug-delivery vehicles with reduced side effects in future cancer therapies.

Clinical benefit in patients with metastatic bone disease: results of a phase 3 study of denosumab versus zoledronic acid.

BACKGROUND: Patients with metastatic bone disease are living longer in the metastatic stage due to improvements in cancer therapy, making strategies to prevent the aggravation of bone disease and its complications, such as skeletal-related events (SREs) and pain, increasingly important. PATIENTS AND RESULTS: In this phase 3 trial in patients with advanced cancer (excluding breast and prostate cancer) or multiple myeloma, denosumab reduced the risk of radiation to bone by 22% relative to zoledronic acid (P = 0.026), prevented worsening of pain and pain interference (2-point increase in Brief Pain Inventory score; P < 0.05 versus zoledronic acid), and reduced the frequency of a shift from no/weak opioid analgesic use to strong opioids (P < 0.05 versus zoledronic acid at months 3-5). Denosumab delayed the time to moderate-to-severe pain compared with zoledronic acid in patients with mild or no pain at the baseline (P = 0.04), supporting early treatment. Health-related quality-of-life scores were similar in both groups. The number needed to treat to avoid one SRE for denosumab was 3 patient-years versus placebo and 10 patient-years versus zoledronic acid. CONCLUSION: The use of denosumab was associated with better prevention of the complications of metastatic bone disease secondary to solid tumors or multiple myeloma versus zoledronic acid.

Polymethoxy-1-alkenes from Aphanizomenon ovalisporum inhibit vertebrate development in the zebrafish (Danio rerio) embryo model.

Cyanobacteria are recognized producers of a wide array of toxic or otherwise bioactive secondary metabolites. The present study utilized the zebrafish (Danio rerio) embryo as an aquatic animal model of vertebrate development to identify, purify and characterize lipophilic inhibitors of development (i.e., developmental toxins) from an isolate of the freshwater cyanobacterial species, Aphanizomenon ovalisporum.Bioassay-guided fractionation led to the purification, and subsequent chemical characterization, of an apparent homologous series of isotactic polymethoxy-1-alkenes (1-6), including three congeners (4-6) previously identified from the strain, and two variants previously identified from other species (2 and 3), as well as one apparently novel member of the series (1). Five of the PMAs in the series (1-5) were purified in sufficient quantity for comparative toxicological characterization, and toxicity in the zebrafish embryo model was found to generally correlate with relative chain length and/or methoxylation. Moreover, exposure of embryos to a combination of variants indicates an apparent synergistic interaction between the congeners. Although PMAs have been identified previously in cyanobacteria, this is the first report of their apparent toxicity. These results, along with the previously reported presence of the PMAs from several cyanobacterial species, suggest a possibly widespread distribution of the PMAs as toxic secondary metabolites and warrants further chemical and toxicological investigation.

Comparison of 7-day and repeated 24-h recall of type 2 diabetes.

PURPOSE: Patient reporting of type 2 diabetes symptoms in a questionnaire with a 7-day recall period was expected to be different from symptom reports using a 7-day diary with repeated 24-h recall based on cognitive theory of memory processes and prior literature. This study compared these two types of recall in patients diagnosed with type 2 diabetes (T2D). METHODS: One hundred and forty adults with T2D completed a daily diary for 7 days containing 9 T2D-related symptom and impact items. On day 7, patients completed the same items with a 7-day recall period. We examined the concordance of 7-day recall with summary descriptors of the daily reports and compared the scores and the discriminant ability of 7-day recall and mean of daily reports. RESULTS: Seven-day recall was most concordant with the mean of daily reports. The average difference in scores was small (range 0.22-0.77 on 11-point scale) and less than 0.5 standard deviations. For some items, the difference was positively associated with the variation in daily reports. The discriminant ability was comparable. CONCLUSIONS: In this study population, a questionnaire with 7-day recall provided information consistent with a daily diary measure of the average week-long experience of T2D symptoms and impacts.

Weight loss and changes in generic and weight-specific quality of life in obese adolescents.

PURPOSE: To investigate how weight loss correlates with changes in generic and weight-specific quality of life (QoL). METHODS: Youth generic (YQOL-S) and weight-specific instruments (YQOL-W) from 133 youth age 11-19 were analyzed at the beginning and end of 4-week immersion camp sessions known to produce weight loss. Paired samples t tests were used to test mean difference between baseline and final Body Mass Index (BMI) and YQOL-S and YQOL-W scores. YQOL-S and YQOL-W scores were transformed to values between 0 and 100, with higher values indicating better QOL. Cohen's d effect sizes were calculated to assess magnitude of effects. Percent weight loss (as % of baseline weight), change in BMI (baseline kg/m²-follow-up kg/m²), and change in % overweight ((BMI-50th% BMI for age and sex)/50th% BMI for age and sex × 100) were calculated. Multiple regressions were used to model final YQOL scores in the 11-14 and 15-19 age groups as functions of each measure of weight change, sex, age, and baseline YQOL score. RESULTS: Youth experienced significant reductions in BMI (Mean change = 3.7, SD = 1.4, t = 34.1, P < 0.001) and in the other measures of weight change. YQOL-S and YQOL-W scores improved significantly (P < 0.001), and effect sizes were 0.61 and 0.66, respectively. CONCLUSION: Changes in generic and weight-specific quality of life scores are associated with weight loss. The weight-specific measure is slightly more sensitive to weight changes; however, when controlling for modifiers, the YQOL-W remained significantly associated with weight loss, while the generic QoL measure did not.

Mode of communication, perceived level of understanding, and perceived quality of life in youth who are deaf or hard of hearing.

Given the important role of parent-youth communication in adolescent well-being and quality of life, we sought to examine the relationship between specific communication variables and youth perceived quality of life in general and as a deaf or hard-of-hearing (DHH) individual. A convenience sample of 230 youth (mean age = 14.1, standard deviation = 2.2; 24% used sign only, 40% speech only, and 36% sign + speech) was surveyed on communication-related issues, generic and DHH-specific quality of life, and depression symptoms. Higher youth perception of their ability to understand parents' communication was significantly correlated with perceived quality of life as well as lower reported depressive symptoms and lower perceived stigma. Youth who use speech as their single mode of communication were more likely to report greater stigma associated with being DHH than youth who used both speech and sign. These findings demonstrate the importance of youths' perceptions of communication with their parents on generic and DHH-specific youth quality of life.

"Molecule corrals" for studies of monolayer organic films.

Scanning tunneling microscopy (STM) studies have demonstrated that monolayer-deep, flat-bottomed, circular etch pits can be grown on highly ordered pyrolytic graphite by high-temperature etching in the presence of oxygen. In this work, these graphite etch pits are used as "molecule corrals" to isolate ensembles of molecules for study by STM. The nucleation of self-assembled molecular films in the corrals took place by nucleation events separate from those leading to self-assembly on the surrounding terrace and allowed the measurement of the nucleation rate constant in the corrals. The dependence of the nucleation rate for self-assembly on pit size shows that nucleation occurs at open terrace sites and that step edges (that is, the corral's perimeter) and confinement inhibit film growth.

Toxicity of cylindrospermopsin, and other apparent metabolites from Cylindrospermopsis raciborskii and Aphanizomenon ovalisporum, to the zebrafish (Danio rerio) embryo.

Cyanobacteria produce a diverse array of toxic or otherwise bioactive compounds that pose growing threats to human and environmental health. We utilized the zebrafish (Danio rerio) embryo, as a model of vertebrate development, to investigate the inhibition of development pathways (i.e. developmental toxicity) by the cyanobacterial toxin, cylindrospermopsin (CYN), as well as extracts from various isolates of Cylindrospermopsis raciborskii and Aphanizomenon ovalisporum. CYN was toxic only when injected directly into embryos, but not by direct immersion at doses up to 50mug/ml. Despite the dose dependency of toxicity observed following injection of CYN, no consistent patterns of developmental defects were observed, suggesting that toxic effects of CYN may not target specific developmental pathways. In contrast, direct immersion of embryos in all of the extracts resulted in both increased mortality and reproducible, consistent, developmental dysfunctions. Interestingly, there was no correlation of developmental toxicity observed for these extracts with the presence of CYN or with previously reported toxicity for these strains. These results suggest that CYN is lethal to zebrafish embryos, but apparently inhibits no specific developmental pathways, whereas other apparent metabolites from C. raciborskii and A. ovalisporum seem to reproducibly inhibit development in the zebrafish model. Continued investigation of these apparent, unknown metabolites is needed.

Development and content validation of measures assessing adherence barriers and behaviors for use in clinical care.

BACKGROUND: Providers are often unaware of poor adherence to prescribed medications for their patients with chronic diseases. OBJECTIVE: To develop brief, computer-administered patient-reported measures in English and Spanish assessing adherence behaviors and barriers. Design, Participants, and Main Measures: Item pools were constructed from existing measures of medication adherence behaviors and barriers, which informed development of a patient concept elicitation interview guide to identify medication adherence behavior and barrier-related concepts. Two hundred six patients either living with HIV (PLWH) or without were interviewed. Interviews were coded, concepts matched to item pool content, and new items were developed for novel concepts. A provider/investigator team highlighted clinically relevant items. Cognitive interviews were conducted with patients on final candidate items (n = 37). The instruments were administered to 2081 PLWH. KEY RESULTS: Behavioral themes from concept elicitation interviews included routines incorporating time of day, placement, visual cues, and intentionality to miss or skip doses. Barrier themes included health-related (e.g. depressed mood, feeling ill), attitudes/beliefs (e.g., need for medication), access (e.g., cost/insurance problems), and circumstantial barriers (e.g., lack of privacy, disruption of daily routine). The final instruments included 6 behavior items, and 1 barrier item with up to 23 response options. PLWH endorsed a mean (SD) of 3.5 (1.1) behaviors. The 201 PLWH who missed ≥2 doses in the previous week endorsed a mean (SD) of 3.1 (2.5) barriers. The intraclass correlation coefficient (ICC) for the numbers of behaviors endorsed in 61 PLWH after 4-16 days was 0.54 and for the number of barriers for the 20 PLWH with ≥2 missed doses the ICC was 0.89, representing fair and excellent test-retest reliability. CONCLUSION: Measures of medication adherence behaviors and barriers were developed for use with patients living with chronic diseases focusing on clinical relevance, brevity, and content validity for use in clinical care.

Initial development and pilot testing of observer-reported outcomes (ObsROs) for children with cystic fibrosis ages 0-11years.

PURPOSE: Patient-reported outcomes are important clinical trial endpoints. Young children may not be able to reliably report on how they feel or function, so observer-reported outcomes (ObsROs) may be more appropriate for them. The purpose of this study was to develop and pilot field test electronic parent-reported observational instruments for children with cystic fibrosis (CF) 0-6 and 7-11years of age. METHODS: We performed concept elicitation interviews with parents of children with CF ≤11years of age to elicit the respiratory signs they could observe at baseline and during an acute respiratory illness. The resulting instruments were refined based on interviews with parents and clinicians. We conducted a pilot field test to evaluate test-retest reliability and the ability of items to distinguish well and sick periods. RESULTS: The instruments consist of 17 items assessing respiratory signs and observable CF-related impacts. Test-retest reliability was acceptable for both age groups but discrimination was low for ages 7-11, likely reflecting less direct observation of older children by their parents. CONCLUSIONS: An ObsRO for children with CF ages 0-6 appears promising, while self-report may be more appropriate for children >6years of age. Next steps for the 0-6year old instrument will be utilizing it as an exploratory endpoint in clinical trials to enable item reduction, scale development, and further reliability and validity testing. Ultimately, this ObsRO could be a promising endpoint for early intervention trials in young children with CF.

Medication understanding among patients living with multiple chronic conditions: Implications for patient-reported measures of adherence.

BACKGROUND: Low health literacy is associated with poor medication adherence and poor health outcomes. Limited understanding of prescribed medications may decrease validity of patient-reported adherence measures. OBJECTIVES: To assess knowledge of names and purposes of prescribed medications among patients with multiple chronic conditions. METHODS: Individual interviews were conducted with a convenience sample of patients from six U.S. primary care clinics. Participants (n = 57) were English and/or Spanish-speaking patients prescribed 3+ medications for chronic conditions, for which non-adherence may lead to disability or death. In individual interviews, patients were asked to name their medications, explain the purpose of each, and to explain how they distinguish them from one another. Interviews were audio recorded, transcribed, and coded; coded content was quantified by 1) whether or not the patient could name medications; 2) method of categorizing medications; 3) whether or not the purpose of the medication was understood. Descriptive statistics were compiled using Fisher's exact test to determine the relationship between patient knowledge and medication characteristics. RESULTS: Thirty percent of patients could not name at least one of their medications; 19% did not know their purpose; 30% held misconceptions about the purpose of one or more medications. There was no significant difference in ability to name medications or state their medication's purpose between patients using medi-sets, pre-packaged rolls, or blister packs, and patients who stored pills in their original containers (p = 0.56 and p = 0.73, respectively), or across demographic groups (p = 0.085 to 0.767). CONCLUSIONS: Many patients demonstrated difficulty identifying the name and purpose of prescribed medications; this did not differ by demographic group or medication storage type. Patients may benefit from routine review of medications with their provider in order to improve health literacy, outcomes, and patient-reported adherence measurement.

Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments.

BACKGROUND: Choosing an adequate measurement instrument depends on the proposed use of the instrument, the concept to be measured, the measurement properties (e.g. internal consistency, reproducibility, content and construct validity, responsiveness, and interpretability), the requirements, the burden for subjects, and costs of the available instruments. As far as measurement properties are concerned, there are no sufficiently specific standards for the evaluation of measurement properties of instruments to measure health status, and also no explicit criteria for what constitutes good measurement properties. In this paper we describe the protocol for the COSMIN study, the objective of which is to develop a checklist that contains COnsensus-based Standards for the selection of health Measurement INstruments, including explicit criteria for satisfying these standards. We will focus on evaluative health related patient-reported outcomes (HR-PROs), i.e. patient-reported health measurement instruments used in a longitudinal design as an outcome measure, excluding health care related PROs, such as satisfaction with care or adherence. The COSMIN standards will be made available in the form of an easily applicable checklist. METHOD: An international Delphi study will be performed to reach consensus on which and how measurement properties should be assessed, and on criteria for good measurement properties. Two sources of input will be used for the Delphi study: (1) a systematic review of properties, standards and criteria of measurement properties found in systematic reviews of measurement instruments, and (2) an additional literature search of methodological articles presenting a comprehensive checklist of standards and criteria. The Delphi study will consist of four (written) Delphi rounds, with approximately 30 expert panel members with different backgrounds in clinical medicine, biostatistics, psychology, and epidemiology. The final checklist will subsequently be field-tested by assessing the inter-rater reproducibility of the checklist. DISCUSSION: Since the study will mainly be anonymous, problems that are commonly encountered in face-to-face group meetings, such as the dominance of certain persons in the communication process, will be avoided. By performing a Delphi study and involving many experts, the likelihood that the checklist will have sufficient credibility to be accepted and implemented will increase.