The core epistemic responsibilities of universities: Results from a Delphi study.

Epistemic responsibilities (ERs) of universities concern equipping and empowering its researchers, educators and students to attain, produce, exchange and disseminate knowledge. ERs can potentially guide universities in improving education, research and in service to society. Building on earlier philosophical work, we applied empirical methods to identify core ERs of universities and their constituting elements. We used a three-round Delphi survey, alternating between closed questions to gain consensus, and open questions to let panelists motivate their answers. 46 panelists participated in our study. We reached consensus on six ERs: 1) to foster research integrity, 2) to stimulate the development of intellectual virtues, 3) to address the big questions of life, 4) to cultivate the diversity of the disciplinary fields, 5) to serve and engage with society at large, and 6) to cultivate and safeguard academic freedom. Together the six ERs contain 27 elements. Consensus rates ranged from 73%-100% for both the ERs and their elements. Participants' detailed responses led to substantial improvements in the accompanying descriptions of the ERs. Our findings can inform the debate about the roles and responsibilities of universities, and inform researchers and policy makers to emphasize epistemic tasks of universities.

From qualitative data to a measurement instrument: A clarification and elaboration of choices made in the development of the Patient Scale of the Patient and Observer Scar Assessment Scale (POSAS) 3.0.

PURPOSE: To clarify and elaborate on the choices that were made in the development of the Patient Scale of the Patient and Observer Scar Assessment Scale 3.0 (POSAS 3.0), based upon the rich information obtained from patients during focus groups and pilot tests. METHODS: The discussions described in this paper are a reflection of the focus group study and pilot tests that were conducted in order to develop the Patient Scale of the POSAS3.0. The focus groups took place in the Netherlands and Australia and included 45 participants. Pilot tests were performed with 15 participants in Australia, the Netherlands, and the United Kingdom. RESULTS: We discussed the selection, wording and merging of 17 included items. Additionally, the reason for exclusion of 23 characteristics are given. CONCLUSION: Based upon the unique and rich material of patient input obtained, two versions of the Patient Scale of the POSAS3.0 were developed: the Generic version, and the Linear scar version. The discussions and decisions taken during the development are informative for a good understanding of the POSAS 3.0 and are indispensable as a background for future translations and cross-cultural adaptations.

Development of the Patient Scale of the Patient and Observer Scar Assessment Scale (POSAS) 3.0: a qualitative study.

PURPOSE: The Patient and Observer Scar Assessment Scale (POSAS) is widely used for measurements of scar quality. This encompasses visual, tactile and sensory characteristics of the scar. The Patient Scale of previous POSAS versions was lacking input from patients. Therefore, the aim of this study was to develop the POSAS3.0, Patient Scale with involvement of adults patients with all scar types, complying with the highest clinimetric standards. METHODS: From February 2018 to April 2019, a series of six focus group interviews were performed in the Netherlands and Australia to identify scar quality characteristics that adults with scars consider to be important. All focus groups were transcribed, anonymized and analysed using a thematic analysis. Relevant characteristics were formulated into items, resulting in a Dutch and English version of the Patient Scale. These drafts were pilot tested in Australia, the Netherlands and the United Kingdom, and refined accordingly. RESULTS: A total of 21 relevant scar quality characteristics were identified during the focus groups. Two distinct versions of the POSAS3.0, Patient Scale were developed. The Generic version contains 16 items and can be used for all scar types, except linear scars. The Linear Scar version of the Patient Scale contains the same 16 items, with an extra item referring to the widening of scar margins. All included items are rated on a verbal rating scale with five response options. CONCLUSION: Two versions of the POSAS3.0 Patient Scale were developed. Further field tests are being performed to establish the measurement properties and scoring algorithm of the scales.

COSMIN Risk of Bias tool to assess the quality of studies on reliability or measurement error of outcome measurement instruments: a Delphi study.

BACKGROUND: Scores on an outcome measurement instrument depend on the type and settings of the instrument used, how instructions are given to patients, how professionals administer and score the instrument, etc. The impact of all these sources of variation on scores can be assessed in studies on reliability and measurement error, if properly designed and analyzed. The aim of this study was to develop standards to assess the quality of studies on reliability and measurement error of clinician-reported outcome measurement instruments, performance-based outcome measurement instrument, and laboratory values. METHODS: We conducted a 3-round Delphi study involving 52 panelists. RESULTS: Consensus was reached on how a comprehensive research question can be deduced from the design of a reliability study to determine how the results of a study inform us about the quality of the outcome measurement instrument at issue. Consensus was reached on components of outcome measurement instruments, i.e. the potential sources of variation. Next, we reached consensus on standards on design requirements (n = 5), standards on preferred statistical methods for reliability (n = 3) and measurement error (n = 2), and their ratings on a four-point scale. There was one term for a component and one rating of one standard on which no consensus was reached, and therefore required a decision by the steering committee. CONCLUSION: We developed a tool that enables researchers with and without thorough knowledge on measurement properties to assess the quality of a study on reliability and measurement error of outcome measurement instruments.

Measuring vincristine-induced peripheral neuropathy in children with cancer: validation of the Dutch pediatric-modified Total Neuropathy Score.

PURPOSE: The aims were to evaluate the construct validity and reliability of the Dutch version of the pediatric-modified Total Neuropathy Score (ped-mTNS) for assessing vincristine-induced peripheral neuropathy (VIPN) in Dutch pediatric oncology patients aged 5-18 years. METHODS: Construct validity (primary aim) of the ped-mTNS was determined by testing hypotheses about expected correlation between scores of the ped-mTNS (range: 0-32) and the Common Terminology Criteria for Adverse Events (CTCAE) (range: 0-18) for patients and healthy controls and by comparing patients and controls regarding their total ped-mTNS scores and the proportion of children identified with VIPN. Inter-rater and intra-rater reliability and measurement error (secondary aims) were assessed in a subgroup of study participants. RESULTS: Among the 112 children (56 patients and 56 age- and gender-matched healthy controls) evaluated, correlation between CTCAE and ped-mTNS scores was as expected (moderate (r = 0.60)). Moreover, as expected, patients had significantly higher ped-mTNS scores and more frequent symptoms of VIPN compared with controls (both p < .001). Reliability as measured within the intra-rater group (n = 10) (intra-class correlation coefficient (ICCagreement) = 0.64, standard error of measurement (SEMagreement) = 2.92, and smallest detectable change (SDCagreement) = 8.1) and within the inter-rater subgroup (n = 10) (ICCagreement = 0.63, SEMagreement = 3.7, and SDCagreement = 10.26) indicates insufficient reliability. CONCLUSION: The Dutch version of the ped-mTNS appears to have good construct validity for assessing VIPN in a Dutch pediatric oncology population, whereas reliability appears to be insufficient and measurement error high. To improve standardization of VIPN assessment in children, future research aimed at evaluating and further optimizing the psychometric characteristics of the ped-mTNS is needed.

Responsiveness of quality-of-life measures in patients with peripheral vascular malformations: the OVAMA project.

BACKGROUND: The OVAMA (Outcome Measures for Vascular Malformations) project determined quality of life (QoL) as a core outcome domain for patients with vascular malformations. In order to measure how current therapeutic strategies alter QoL in these patients, a patient-reported outcome measurement (PROM) responsive to changes in QoL is required. OBJECTIVES: To assess the responsiveness of two widely used generic QoL PROMs, the Medical Outcomes Study Short Form 36 (SF-36) and Skindex-29, in adult patients with vascular malformations. METHODS: In an international multicentre prospective study, treated and untreated patients completed the SF-36 and Skindex-29 at baseline and after a follow-up period of 6-8 weeks. Global rating of change (GRC) scales assessing various QoL-related outcome domains were additionally completed. Per subscale, responsiveness was assessed using two methods: by testing hypotheses on expected correlation strength between change scores of the questionnaires and the GRC scales, and by calculating the area under the receiver operating characteristics curve (AUC). The questionnaires were considered responsive if ≥ 75% of the hypotheses were confirmed or if the AUC was ≥ 0·7. RESULTS: Eighty-nine participants were recruited in three centres in the Netherlands and the U.S.A., of whom 67 completed all baseline and follow-up questionnaires. For all subscales of the SF-36 and Skindex-29, < 75% of the hypotheses were confirmed and the AUC was < 0·7. CONCLUSIONS: Our findings suggest that the SF-36 and Skindex-29 seemed unresponsive to change in QoL. This suggests that alternative PROMs are needed to measure - and ultimately improve - QoL in patients with vascular malformations. What's already known about this topic? Quality of life is often impaired in patients with vascular malformations. Quality of life is considered a core outcome domain for evaluating treatment of vascular malformations. To measure the effect of treatment on quality of life, a patient-reported outcome measure is required that is responsive to changes in quality of life. What does this study add? This is the first study assessing the responsiveness of quality-of-life measures in patients with vascular malformations. The results seem to indicate that the Medical Outcomes Study Short Form 36 (SF-36) and Skindex-29 are not responsive to changes in quality of life in patients with vascular malformations. What are the clinical implications of this work? Medical Outcomes Study Short Form 36 (SF-36) and Skindex-29 are not ideal to assess the effect on quality of life over time, of treatment strategies for peripheral vascular malformations.

Evaluation of Measurement Properties of Patient-Reported Outcome Measures After Rhinoplasty: A Systematic Review.

IMPORTANCE: The number of available rhinoplasty outcome measurement instruments has increased rapidly over the past years. A large heterogeneity of instruments of different quality now exists, causing difficulty in pooling and comparing outcome data. OBJECTIVE: To critically appraise, summarize, and compare the measurement properties of all patient-reported outcome measures (PROMs) that measure functional or aesthetic symptoms of patients undergoing rhinoplasty, using consensus-based methodology and guidelines. This facilitates an evidence-based recommendation on the most suitable instrument to measure rhinoplasty outcomes and identifies promising instruments worthy of further research. EVIDENCE REVIEW: A systematic literature search of Embase, Medline, and Web of Science was conducted from the databases' respective inception dates to May 18, 2018. Thirty-three articles evaluating 1 or more measurement properties of instruments measuring symptoms related to nasal breathing or satisfaction with nasal appearance in patients who had undergone septoplasty and/or rhinoplasty were included. Measurement properties were graded according to the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) guidelines for systematic reviews of PROMs. FINDINGS: The search strategy identified 33 studies that used 12 different measurement instruments. In general, high-quality studies on measurement properties of instruments measuring aesthetic and/or functional symptom-specific outcome of rhinoplasty are scarce. The Nasal Obstruction Symptom Evaluation (NOSE) scale demonstrated high-quality evidence for sufficient structural validity, internal consistency, reliability, construct validity, and responsiveness, along with favorable interpretability and feasibility aspects, and was therefore selected as the most suitable instrument to measure functional outcome. Among instruments measuring aesthetic outcome, the FACE-Q and Standardized Cosmesis and Health Nasal Outcomes Survey (SCHNOS) aesthetic subscale are recommended for further study. Future studies on the measurement properties of the identified PROMs, in particular content validity studies, are necessary. CONCLUSIONS AND RELEVANCE: Three instruments with high potential for further use were identified in a systematic review of rhinoplasty outcome instruments using a standardized, consensus-based methodology: the NOSE, FACE-Q, and SCHNOS. These findings may contribute to standardized collection of outcome data in rhinoplasty.

COSMIN methodology for evaluating the content validity of patient-reported outcome measures: a Delphi study.

BACKGROUND: Content validity is the most important measurement property of a patient-reported outcome measure (PROM) and the most challenging to assess. Our aims were to: (1) develop standards for evaluating the quality of PROM development; (2) update the original COSMIN standards for assessing the quality of content validity studies of PROMs; (3) develop criteria for what constitutes good content validity of PROMs, and (4) develop a rating system for summarizing the evidence on a PROM's content validity and grading the quality of the evidence in systematic reviews of PROMs. METHODS: An online 4-round Delphi study was performed among 159 experts from 21 countries. Panelists rated the degree to which they (dis)agreed to proposed standards, criteria, and rating issues on 5-point rating scales ('strongly disagree' to 'strongly agree'), and provided arguments for their ratings. RESULTS: Discussion focused on sample size requirements, recording and field notes, transcribing cognitive interviews, and data coding. After four rounds, the required 67% consensus was reached on all standards, criteria, and rating issues. After pilot-testing, the steering committee made some final changes. Ten criteria for good content validity were defined regarding item relevance, appropriateness of response options and recall period, comprehensiveness, and comprehensibility of the PROM. DISCUSSION: The consensus-based COSMIN methodology for content validity is more detailed, standardized, and transparent than earlier published guidelines, including the previous COSMIN standards. This methodology can contribute to the selection and use of high-quality PROMs in research and clinical practice.

COSMIN guideline for systematic reviews of patient-reported outcome measures.

PURPOSE: Systematic reviews of patient-reported outcome measures (PROMs) differ from reviews of interventions and diagnostic test accuracy studies and are complex. In fact, conducting a review of one or more PROMs comprises of multiple reviews (i.e., one review for each measurement property of each PROM). In the absence of guidance specifically designed for reviews on measurement properties, our aim was to develop a guideline for conducting systematic reviews of PROMs. METHODS: Based on literature reviews and expert opinions, and in concordance with existing guidelines, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) steering committee developed a guideline for systematic reviews of PROMs. RESULTS: A consecutive ten-step procedure for conducting a systematic review of PROMs is proposed. Steps 1-4 concern preparing and performing the literature search, and selecting relevant studies. Steps 5-8 concern the evaluation of the quality of the eligible studies, the measurement properties, and the interpretability and feasibility aspects. Steps 9 and 10 concern formulating recommendations and reporting the systematic review. CONCLUSIONS: The COSMIN guideline for systematic reviews of PROMs includes methodology to combine the methodological quality of studies on measurement properties with the quality of the PROM itself (i.e., its measurement properties). This enables reviewers to draw transparent conclusions and making evidence-based recommendations on the quality of PROMs, and supports the evidence-based selection of PROMs for use in research and in clinical practice.

COSMIN Risk of Bias checklist for systematic reviews of Patient-Reported Outcome Measures.

PURPOSE: The original COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was developed to assess the methodological quality of single studies on measurement properties of Patient-Reported Outcome Measures (PROMs). Now it is our aim to adapt the COSMIN checklist and its four-point rating system into a version exclusively for use in systematic reviews of PROMs, aiming to assess risk of bias of studies on measurement properties. METHODS: For each standard (i.e., a design requirement or preferred statistical method), it was discussed within the COSMIN steering committee if and how it should be adapted. The adapted checklist was pilot-tested to strengthen content validity in a systematic review on the quality of PROMs for patients with hand osteoarthritis. RESULTS: Most important changes were the reordering of the measurement properties to be assessed in a systematic review of PROMs; the deletion of standards that concerned reporting issues and standards that not necessarily lead to biased results; the integration of standards on general requirements for studies on item response theory with standards for specific measurement properties; the recommendation to the review team to specify hypotheses for construct validity and responsiveness in advance, and subsequently the removal of the standards about formulating hypotheses; and the change in the labels of the four-point rating system. CONCLUSIONS: The COSMIN Risk of Bias checklist was developed exclusively for use in systematic reviews of PROMs to distinguish this application from other purposes of assessing the methodological quality of studies on measurement properties, such as guidance for designing or reporting a study on the measurement properties.

Psychometric Properties of the Dutch Depression Stigma Scale (DSS) and Associations with Personal and Perceived Stigma in a Depressed and Community Sample.

BACKGROUND: Research on depression stigma is needed to gain more insight into the underlying construct and to reduce the level of stigma in the community. However, few validated measurements of depression stigma are available in the Netherlands. Therefore, this study first sought to examine the psychometric properties of the Dutch translation of the Depression Stigma Scale (DSS). Second, we examined which demographic (gender, age, education, partner status) and other variables (anxiety and knowledge of depression) are associated with personal and perceived stigma within these samples. METHODS: The study population consisted of an adult convenience sample (n = 253) (study 1) and a community adult sample with elevated depressive symptoms (n = 264) (study 2). Factor structure, internal consistency, and validity were assessed. The associations between stigma, demographic variables and anxiety level were examined with regression analyses. RESULTS: Confirmatory factor analysis supported the validity and internal consistency of the DSS personal stigma scale. Internal consistency was sufficient (Cronbach's alpha = .70 (study 1) and .77 (study 2)). The results regarding the perceived stigma scale revealed no clear factor structure. Regression analyses showed that personal stigma was higher in younger people, those with no experience with depression, and those with lower education. CONCLUSIONS: This study established the validity and internal consistency of the DSS personal scale in the Netherlands, in a community sample and in people with elevated depressive symptoms. However, additional research is needed to examine the factor structure of the DSS perceived scale and its use in other samples.

Comment on "Should we reframe how we think about physical activity and sedentary behavior measurement? Validity and reliability reconsidered".

With great interest we read the article by Kelly et al. on the measurement of physical activity (PA) and sedentary behavior (SB) (Kelly P et al. Int J Behav Nutr Phys Act 13:(1) 32, 2016). We appreciate the invitation of the authors to provide feedback on their ideas and we take this opportunity to contribute to the discussion. Our main proposition is that this field can learn much from the field of quality of life research and the methodology developed for validating quality of life questionnaires.

The quality of systematic reviews of health-related outcome measurement instruments.

BACKGROUND: Systematic reviews of outcome measurement instruments are important tools for the selection of instruments for research and clinical practice. Our aim was to assess the quality of systematic reviews of health-related outcome measurement instruments and to determine whether the quality has improved since our previous study in 2007. METHODS: A systematic literature search was performed in MEDLINE and EMBASE between July 1, 2013, and June 19, 2014. The quality of the reviews was rated using a study-specific checklist. RESULTS: A total of 102 reviews were included. In many reviews the search strategy was considered not comprehensive; in only 59 % of the reviews a search was performed in EMBASE and in about half of the reviews there was doubt about the comprehensiveness of the search terms used for type of measurement instruments and measurement properties. In 41 % of the reviews, compared to 30 % in our previous study, the methodological quality of the included studies was assessed. In 58 %, compared to 55 %, the quality of the included instruments was assessed. In 42 %, compared to 7 %, a data synthesis was performed in which the results from multiple studies on the same instrument were somehow combined. CONCLUSION: Despite a clear improvement in the quality of systematic reviews of outcome measurement instruments in comparison with our previous study in 2007, there is still room for improvement with regard to the search strategy, and especially the quality assessment of the included studies and the included instruments, and the data synthesis.

Measurement properties of the Client-centered Care Questionnaire (CCCQ): factor structure, reliability and validity of a questionnaire to assess self-reported client-centeredness of home care services in a population of frail, older people.

BACKGROUND: The increasing and specific use of home care services by frail, older people asks for the evaluation of the client-centeredness of these services. To our knowledge, no instrument that measures client-centeredness of home care from this group's unique perspective exists. We therefore tested the factor structure, reliability, content validity and acceptability of the Client-centered Care Questionnaire (CCCQ), an existing instrument developed for general home care users, in a population of frail, older people in the Netherlands. METHODS: We used data from a 2-year clinical trial. STUDY POPULATION: frail, older people who received home care. Data were collected at baseline (n = 600) and 24-month measurements (n = 389); retest data (n = 67) were collected 7-14 days after the 24-month measurements. ANALYSES: We performed confirmatory factor analysis, investigated reliability and validity parameters and assessed acceptability. RESULTS: The factor analysis yielded a bifactor model with essential unidimensionality. Internal consistency was high (omega total .88). We found a test-retest reliability of total test scores of .81; the standard error of measurement was 2.61 (total score range 15-75) and the limits of agreement were -7.03 and 7.86. We rejected three out of four hypotheses for construct validity. CONCLUSIONS: The CCCQ is sufficiently unidimensional to permit the use of total test scores. We found acceptable reliability values, but considered our results on construct validity inconclusive. Respondents found the CCCQ questions challenging to answer, which is indicative of a high degree of respondent burden. Future instruments that measure client-centeredness of home care from the frail, older client's perspective should therefore be tailored to the specific circumstances of this population.

Factor structure of Guy's Neurological Disability Scale in a sample of Dutch patients with multiple sclerosis.

BACKGROUND: Guy's Neurological Disability Scale (GNDS) is designed to assess disability (i.e. activity limitations) in patients with multiple sclerosis. It contains 12 functional domains, each indicating a level of disability. Four domain scores and a total score can be calculated. OBJECTIVE: The aim of this study was to evaluate the structural validity of the GNDS in a Dutch population who were definitely diagnosed as having multiple sclerosis. METHODS: Data of 974 patients were available. The structural validity of the GNDS was evaluated by confirmatory item factor analysis (CIFA). Two first-order models and two bifactor models were investigated. RESULTS: The best fitted model was a bifactor model with a general factor underlying all items, and 10 items loading on 3 group factors. Cronbach's alpha on the general factor (0.78) and on the group factor spinal-plus (0.74) were satisfying. Cronbach's alpha on the group factors mental (0.56) and bulbar (0.48) were low. Reliability based on CIFA was 0.85. CONCLUSION: Results showed a clear factor structure of the GNDS. It justifies the use of the total score of the GNDS. In addition, three sub-scale scores could be used.

Performance-based methods for measuring the physical function of patients with osteoarthritis of the hip or knee: a systematic review of measurement properties.

OBJECTIVE: To systematically review the measurement properties (i.e. internal consistency, reproducibility, validity, responsiveness and interpretability) of all performance-based methods which have been used to measure the physical function of patients with osteoarthritis of the hip or knee. METHODS: A systematic search was conducted in Medline, CINAHL, PsychINFO and Embase. Standardized criteria were applied to assess the quality of the clinimetric studies and the measurement properties. RESULTS: Twenty-six performance-based methods were included: 13 walking tests, two stair-climb tests, one chair test and ten multi-item tests. Three out of seven multi-activity tests were tested for internal consistency and two were rated positively. Fourteen tests were tested for reliability and five were rated positively. The absolute measurement error (agreement) was assessed for 10 tests. Only one test received a positive rating. Fourteen tests were tested for construct validity. Only two tests received positive ratings. Responsiveness was assessed for 12 tests, but none of them received a positive rating. A lot of indeterminate ratings were given, mostly for small studies or non-optimal analyses. CONCLUSION: Many more well-designed studies are needed to assess the measurement properties of performance-based methods. More importantly, however, before one can make a justified choice of a particular performance-based method, consensus is needed on what activities should be included in a performance-based test for patients with hip or knee osteoarthritis and which aspects of function should be measured.

Protocol of the COSMIN study: COnsensus-based Standards for the selection of health Measurement INstruments.

BACKGROUND: Choosing an adequate measurement instrument depends on the proposed use of the instrument, the concept to be measured, the measurement properties (e.g. internal consistency, reproducibility, content and construct validity, responsiveness, and interpretability), the requirements, the burden for subjects, and costs of the available instruments. As far as measurement properties are concerned, there are no sufficiently specific standards for the evaluation of measurement properties of instruments to measure health status, and also no explicit criteria for what constitutes good measurement properties. In this paper we describe the protocol for the COSMIN study, the objective of which is to develop a checklist that contains COnsensus-based Standards for the selection of health Measurement INstruments, including explicit criteria for satisfying these standards. We will focus on evaluative health related patient-reported outcomes (HR-PROs), i.e. patient-reported health measurement instruments used in a longitudinal design as an outcome measure, excluding health care related PROs, such as satisfaction with care or adherence. The COSMIN standards will be made available in the form of an easily applicable checklist. METHOD: An international Delphi study will be performed to reach consensus on which and how measurement properties should be assessed, and on criteria for good measurement properties. Two sources of input will be used for the Delphi study: (1) a systematic review of properties, standards and criteria of measurement properties found in systematic reviews of measurement instruments, and (2) an additional literature search of methodological articles presenting a comprehensive checklist of standards and criteria. The Delphi study will consist of four (written) Delphi rounds, with approximately 30 expert panel members with different backgrounds in clinical medicine, biostatistics, psychology, and epidemiology. The final checklist will subsequently be field-tested by assessing the inter-rater reproducibility of the checklist. DISCUSSION: Since the study will mainly be anonymous, problems that are commonly encountered in face-to-face group meetings, such as the dominance of certain persons in the communication process, will be avoided. By performing a Delphi study and involving many experts, the likelihood that the checklist will have sufficient credibility to be accepted and implemented will increase.