Quality indicators for appropriate in-hospital pharmacotherapeutic stewardship: An international modified Delphi study.

AIMS: In-hospital prescribing errors may result in patient harm, such as prolonged hospitalisation and hospital (re)admission, and may be an emotional burden for the prescribers and healthcare professionals involved. Despite efforts, in-hospital prescribing errors and related harm still occur, necessitating an innovative approach. We therefore propose a novel approach, in-hospital pharmacotherapeutic stewardship (IPS). The aim of this study was to reach consensus on a set of quality indicators (QIs) as a basis for IPS. METHODS: A three-round modified Delphi procedure was performed. Potential QIs were retrieved from two systematic searches of the literature, in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement. In two written questionnaires and a focus meeting (held between the written questionnaire rounds), potential QIs were appraised by an international, multidisciplinary expert panel composed of members of the European Association for Clinical Pharmacology and Therapeutics (EACPT). RESULTS: The expert panel rated 59 QIs and four general statements, of which 35 QIs were accepted with consensus rates ranging between 79% and 97%. These QIs describe the activities of an IPS programme, the team delivering IPS, the patients eligible for the programme and the outcome measures that should be used to evaluate the care delivered. CONCLUSIONS: A framework of 35 QIs for an IPS programme was systematically developed. These QIs can guide hospitals in setting up a pharmacotherapeutic stewardship programme to reduce in-hospital prescribing errors and improve in-hospital medication safety.

Methodological quality of 100 recent systematic reviews of health-related outcome measurement instruments: an overview of reviews.

PURPOSE: Systematic reviews evaluating and comparing the measurement properties of outcome measurement instruments (OMIs) play an important role in OMI selection. Earlier overviews of review quality (2007, 2014) evidenced substantial concerns with regards to alignment to scientific standards. This overview aimed to investigate whether the quality of recent systematic reviews of OMIs lives up to the current scientific standards. METHODS: One hundred systematic reviews of OMIs published from June 1, 2021 onwards were randomly selected through a systematic literature search performed on March 17, 2022 in MEDLINE and EMBASE. The quality of systematic reviews was appraised by two independent reviewers. An updated data extraction form was informed by the earlier studies, and results were compared to these earlier studies' findings. RESULTS: A quarter of the reviews had an unclear research question or aim, and in 22% of the reviews the search strategy did not match the aim. Half of the reviews had an incomprehensive search strategy, because relevant search terms were not included. In 63% of the reviews (compared to 41% in 2014 and 30% in 2007) a risk of bias assessment was conducted. In 73% of the reviews (some) measurement properties were evaluated (58% in 2014 and 55% in 2007). In 60% of the reviews the data were (partly) synthesized (42% in 2014 and 7% in 2007); evaluation of measurement properties and data syntheses was not conducted separately for subscales in the majority. Certainty assessments of the quality of the total body of evidence were conducted in only 33% of reviews (not assessed in 2014 and 2007). The majority (58%) did not make any recommendations on which OMI (not) to use. CONCLUSION: Despite clear improvements in risk of bias assessments, measurement property evaluation and data synthesis, specifying the research question, conducting the search strategy and performing a certainty assessment remain poor. To ensure that systematic reviews of OMIs meet current scientific standards, more consistent conduct and reporting of systematic reviews of OMIs is needed.

Patient-reported outcomes for people with diabetes: what and how to measure? A narrative review.

Patient-reported outcomes (PROs) are valuable for shared decision making and research. Patient-reported outcome measures (PROMs) are questionnaires used to measure PROs, such as health-related quality of life (HRQL). Although core outcome sets for trials and clinical practice have been developed separately, they, as well as other initiatives, recommend different PROs and PROMs. In research and clinical practice, different PROMs are used (some generic, some disease-specific), which measure many different things. This is a threat to the validity of research and clinical findings in the field of diabetes. In this narrative review, we aim to provide recommendations for the selection of relevant PROs and psychometrically sound PROMs for people with diabetes for use in clinical practice and research. Based on a general conceptual framework of PROs, we suggest that relevant PROs to measure in people with diabetes are: disease-specific symptoms (e.g. worries about hypoglycaemia and diabetes distress), general symptoms (e.g. fatigue and depression), functional status, general health perceptions and overall quality of life. Generic PROMs such as the 36-Item Short Form Health Survey (SF-36), WHO Disability Assessment Schedule (WHODAS 2.0), or Patient-Reported Outcomes Measurement Information System (PROMIS) measures could be considered to measure commonly relevant PROs, supplemented with disease-specific PROMs where needed. However, none of the existing diabetes-specific PROM scales has been sufficiently validated, although the Diabetes Symptom Self-Care Inventory (DSSCI) for measuring diabetes-specific symptoms and the Diabetes Distress Scale (DDS) and Problem Areas in Diabetes (PAID) for measuring distress showed sufficient content validity. Standardisation and use of relevant PROs and psychometrically sound PROMs can help inform people with diabetes about the expected course of disease and treatment, for shared decision making, to monitor outcomes and to improve healthcare. We recommend further validation studies of diabetes-specific PROMs that have sufficient content validity for measuring disease-specific symptoms and consider generic item banks developed based on item response theory for measuring commonly relevant PROs.

Reliability of brain atrophy measurements in multiple sclerosis using MRI: an assessment of six freely available software packages for cross-sectional analyses.

PURPOSE: Volume measurement using MRI is important to assess brain atrophy in multiple sclerosis (MS). However, differences between scanners, acquisition protocols, and analysis software introduce unwanted variability of volumes. To quantify theses effects, we compared within-scanner repeatability and between-scanner reproducibility of three different MR scanners for six brain segmentation methods. METHODS: Twenty-one people with MS underwent scanning and rescanning on three 3 T MR scanners (GE MR750, Philips Ingenuity, Toshiba Vantage Titan) to obtain 3D T1-weighted images. FreeSurfer, FSL, SAMSEG, FastSurfer, CAT-12, and SynthSeg were used to quantify brain, white matter and (deep) gray matter volumes both from lesion-filled and non-lesion-filled 3D T1-weighted images. We used intra-class correlation coefficient (ICC) to quantify agreement; repeated-measures ANOVA to analyze systematic differences; and variance component analysis to quantify the standard error of measurement (SEM) and smallest detectable change (SDC). RESULTS: For all six software, both between-scanner agreement (ICCs ranging 0.4-1) and within-scanner agreement (ICC range: 0.6-1) were typically good, and good to excellent (ICC > 0.7) for large structures. No clear differences were found between filled and non-filled images. However, gray and white matter volumes did differ systematically between scanners for all software (p < 0.05). Variance component analysis yielded within-scanner SDC ranging from 1.02% (SAMSEG, whole-brain) to 14.55% (FreeSurfer, CSF); and between-scanner SDC ranging from 4.83% (SynthSeg, thalamus) to 29.25% (CAT12, thalamus). CONCLUSION: Volume measurements of brain, GM and WM showed high repeatability, and high reproducibility despite substantial differences between scanners. Smallest detectable change was high, especially between different scanners, which hampers the clinical implementation of atrophy measurements.

Content Validity of Patient-Reported Outcome Measures Developed for Assessing Health-Related Quality of Life in People with Type 2 Diabetes Mellitus: a Systematic Review.

PURPOSE OF REVIEW: We aimed to systematically evaluate the content validity of patient-reported outcome measures (PROMs) specifically developed to measure (aspects of) health-related quality of life (HRQOL) in people with type 2 diabetes. A systematic review was performed in PubMed and Embase of PROMs measuring perceived symptoms, physical function, mental function, social function/participation, and general health perceptions, and that were validated to at least some extent. Content validity (relevance, comprehensiveness, and comprehensibility) was evaluated using COSMIN methodology. RECENT FINDINGS: We identified 54 (different versions of) PROMs, containing 150 subscales. We found evidence for sufficient content validity for only 41/150 (27%) (subscales of) PROMs. The quality of evidence was generally very low. We found 66 out of 150 (44%) (subscales of) PROMs with evidence for either insufficient relevance, insufficient comprehensiveness, or insufficient comprehensibility. For measuring diabetes-specific symptoms, physical function, mental function, social function/participation, and general health perceptions, we identified one to 11 (subscales of) PROMs with sufficient content validity, although quality of the evidence was generally low. For measuring depressive symptoms, no PROM with sufficient content validity was identified. For each aspect of HRQL, we found at least one PROM with sufficient content validity, except for depressive symptoms. The quality of the evidence was mostly very low.

EXpert consensus On Diaphragm UltraSonography in the critically ill (EXODUS): a Delphi consensus statement on the measurement of diaphragm ultrasound-derived parameters in a critical care setting.

BACKGROUND: Diaphragm ultrasonography is rapidly evolving in both critical care and research. Nevertheless, methodologically robust guidelines on its methodology and acquiring expertise do not, or only partially, exist. Therefore, we set out to provide consensus-based statements towards a universal measurement protocol for diaphragm ultrasonography and establish key areas for research. METHODS: To formulate a robust expert consensus statement, between November 2020 and May 2021, a two-round, anonymous and online survey-based Delphi study among experts in the field was performed. Based on the literature review, the following domains were chosen: "Anatomy and physiology", "Transducer Settings", "Ventilator Impact", "Learning and expertise", "Daily practice" and "Future directions". Agreement of ≥ 68% (≥ 10 panelists) was needed to reach consensus on a question. RESULTS: Of 18 panelists invited, 14 agreed to participate in the survey. After two rounds, the survey included 117 questions of which 42 questions were designed to collect arguments and opinions and 75 questions aimed at reaching consensus. Of these, 46 (61%) consensus was reached. In both rounds, the response rate was 100%. Among others, there was agreement on measuring thickness between the pleura and peritoneum, using > 10% decrease in thickness as cut-off for atrophy and using 40 examinations as minimum training to use diaphragm ultrasonography in clinical practice. In addition, key areas for research were established. CONCLUSION: This expert consensus statement presents the first set of consensus-based statements on diaphragm ultrasonography methodology. They serve to ensure high-quality and homogenous measurements in daily clinical practice and in research. In addition, important gaps in current knowledge and thereby key areas for research are established. Trial registration The study was pre-registered on the Open Science Framework with registration digital object identifier https://doi.org/10.17605/OSF.IO/HM8UG .

Differential item functioning of the Arm function in Multiple Sclerosis Questionnaire (AMSQ) by language, a study in six countries.

BACKGROUND: The Arm function in Multiple Sclerosis Questionnaire (AMSQ) has been developed as a self-reported measure of arm and hand functioning for patients with multiple sclerosis (MS). The AMSQ was originally developed in Dutch and to date translated into five languages (i.e. English, German, Spanish, French, and Italian). OBJECTIVE: The aim of this study was to evaluate differential item functioning (DIF) of the AMSQ in these languages. METHODS: We performed DIF analyses, using "language" as the polytomous group variable. To detect DIF, logistic regression and item response theory principles were applied. Multiple logistic regression models were evaluated. We used a pseudo R2 value of 0.02 or more as the DIF threshold. RESULTS: A total of 1733 male and female patients with all subtypes of MS were included. The DIF analysis for the whole dataset showed no uniform or non-uniform DIF on any of the 31 items. All R2 values were below 0.02. CONCLUSION: The AMSQ is validated in six languages. All items have the same meaning to MS patients in Dutch, English, German, Spanish, French, and Italian. This validation study enables use of the AMSQ in international studies, for monitoring treatment response and disease progression.

COSMIN reporting guideline for studies on measurement properties of patient-reported outcome measures.

PURPOSE: To develop a set of consensus and empirically based reporting recommendations for primary studies of the measurement properties of patient-reported outcome measures (PROMs). METHODS: This study included four phases: 1. Conducting an extensive literature review of recommendations for reporting of studies testing measurement properties of PROMs; 2. Preparing for the Delphi study by identifying experts; 3. Conducting three Delphi rounds aiming for consensus on the item list of recommendations found in phase 1; 4. Developing the COSMIN reporting guideline and user manual. RESULTS: The literature review resulted in 93 reporting items, included in the first Delphi round. A total of 84 individuals (from 12 countries) agreed to participate in the Delphi study, with 47, 30 and 25 responding in rounds one, two and three, respectively. After three rounds, we achieved consensus on a set of 71 items separated into a set of 35 "common" items (relevant to all studies on measurement properties) and 41 "specific" items (exclusively relevant to one of the nine measurement properties). CONCLUSION: Consensus was achieved on a set of 71 items for inclusion in a reporting guideline for studies on measurement properties of PROMs. These items will guide researchers on the necessary information to include in their reports of investigations of measurement properties of PROMs. This guideline will likely improve the completeness of reporting of these important studies.

Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures.

We define the minimal important change (MIC) as a threshold for a minimal within-person change over time above which patients perceive themselves importantly changed. There is a lot of confusion about the concept of MIC, particularly about the concepts of minimal important change and minimal detectable change, which questions the validity of published MIC values. The aims of this study were: (1) to clarify the concept of MIC and how to use it; (2) to provide practical guidance for estimating methodologically sound MIC values; and (3) to improve the applicability of PROMIS by summarizing the available evidence on plausible PROMIS MIC values. We discuss the concept of MIC and how to use it and provide practical guidance for estimating MIC values. In addition, we performed a systematic review in PubMed on MIC values of any PROMIS measure from studies using recommended approaches. A total of 50 studies estimated the MIC of a PROMIS measure, of which 19 studies used less appropriate methods. MIC values of the remaining 31 studies ranged from 0.1 to 12.7 T-score points. We recommend to use the predictive modeling method, possibly supplemented with the vignette-based method, in future MIC studies. We consider a MIC value of 2-6 T-score points for PROMIS measures reasonable to assume at this point. For surgical interventions a higher MIC value might be appropriate. We recommend more high-quality studies estimating MIC values for PROMIS.

Niche-related outcomes after caesarean section and quality of life: a focus group study and review of literature.

BACKGROUND: A niche in the uterus, present in 60% of women after caesarean section (CS), is associated with several gynaecological symptoms and possibly with subfertility. Studies that focus on quality of life (QoL) in relation to a niche are lacking. PURPOSE: To identify niche-related outcomes that influence QoL and to compare patient-reported outcomes with outcomes studied in the literature. METHODS: Two focus group discussions (FGDs, N = 8 and 5) were conducted in Amsterdam UMC-location VUmc. Participants were Dutch patients with a large niche, with (planned) surgical treatment for their symptoms. Niche-related symptoms and impact on functioning or participation were fixed topics. The transcripts of the FGDs were coded into outcomes, themes and domains of QoL according to the WHOQOL model. Additionally, participants created a top five important outcomes. Next, we performed a systematic review (SR) on niche-related outcomes and compared the FGDs with niche-related outcomes from the SR. RESULTS: In four domains (physical health, psychological domain, social relationships and environment), fifteen themes were reported in the FGDs. Abnormal uterine bleeding (AUB), subfertility, sexual activity, abdominal pain and self-esteem were themes prioritised by participants. In the literature, gynaecological symptoms and reproductive outcomes were predominantly studied. Sexuality and self-esteem were prioritised in the FGDs but hardly or never studied in the literature. CONCLUSION: We found a broad range of niche-related outcomes influencing QoL. Apart from symptoms evaluated in the literature such as AUB, abdominal pain and subfertility, clinicians and researchers should be more aware of sexual activity and self-esteem in this population.

Development of the Arm Function in Multiple Sclerosis Questionnaire-Short Form (AMSQ-SF): A static 10-item version.

BACKGROUND: Assessing arm and hand function of multiple sclerosis (MS) patients is important as impaired functioning may impact daily activities and reduce quality of life. OBJECTIVE: A short-form of the Arm Function in Multiple Sclerosis Questionnaire (AMSQ), a recently developed patient-reported outcome measure containing 31 items, is developed to allow non-adaptive application. METHODS: Complete data from 690 patients with MS, recruited via outpatient clinics, a residential center or via a Dutch website aimed at MS patients, were included in the analyses. A graded response model was fit to these data to estimate item response theory (IRT) parameters, which were used to perform post hoc computerized adaptive test (CAT) simulations with a cutoff standard error of measurement (SEM) of 0.32. The optimal test length was determined by the correlation between the static short-form and full-length theta, the mean SEM, and the amount of patients reaching a satisfactory SEM in CAT simulations. RESULTS AND CONCLUSION: Based on five selection criteria (i.e. discrimination parameters, total information, times selected in CAT simulations, raw item means, and item content), 10 items were selected for inclusion in the short-form. The score on the final 10-item short-form correlated strongly with the full-length AMSQ and provided reliable ability estimations, indicating its usefulness instrument in research and clinical settings.

Reliability and validity of the C-BiLLT: a new instrument to assess comprehension of spoken language in young children with cerebral palsy and complex communication needs.

In clinical practice, a variety of diagnostic tests are available to assess a child's comprehension of spoken language. However, none of these tests have been designed specifically for use with children who have severe motor impairments and who experience severe difficulty when using speech to communicate. This article describes the process of investigating the reliability and validity of the Computer-Based Instrument for Low Motor Language Testing (C-BiLLT), which was specifically developed to assess spoken Dutch language comprehension in children with cerebral palsy and complex communication needs. The study included 806 children with typical development, and 87 nonspeaking children with cerebral palsy and complex communication needs, and was designed to provide information on the psychometric qualities of the C-BiLLT. The potential utility of the C-BiLLT as a measure of spoken Dutch language comprehension abilities for children with cerebral palsy and complex communication needs is discussed.

Guideline for reporting systematic reviews of outcome measurement instruments (OMIs): PRISMA-COSMIN for OMIs 2024.

PURPOSE: Although comprehensive and widespread guidelines on how to conduct systematic reviews of outcome measurement instruments (OMIs) exist, for example from the COSMIN (COnsensus-based Standards for the selection of health Measure- ment INstruments) initiative, key information is often missing in published reports. This article describes the development of an extension of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 guideline: PRISMA-COSMIN for OMIs 2024. METHODS: The development process followed the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) guidelines and included a literature search, expert consultations, a Delphi study, a hybrid workgroup meeting, pilot testing, and an end-of-project meeting, with integrated patient/public involvement. RESULTS: From the literature and expert consultation, 49 potentially relevant reporting items were identified. Round 1 of the Delphi study was completed by 103 panelists, whereas round 2 and 3 were completed by 78 panelists. After 3 rounds, agreement (≥ 67%) on inclusion and wording was reached for 44 items. Eleven items without consensus for inclusion and/or wording were discussed at a workgroup meeting attended by 24 participants. Agreement was reached for the inclusion and wording of 10 items, and the deletion of 1 item. Pilot testing with 65 authors of OMI systematic reviews further improved the guideline through minor changes in wording and structure, finalized during the end-of-project meeting. The final check- list to facilitate the reporting of full systematic review reports contains 54 (sub)items addressing the review's title, abstract, plain language summary, open science, introduction, methods, results, and discussion. Thirteen items pertaining to the title and abstract are also included in a separate abstract checklist, guiding authors in reporting for example conference abstracts. CONCLUSION: PRISMA-COSMIN for OMIs 2024 consists of two checklists (full reports; abstracts), their corresponding expla- nation and elaboration documents detailing the rationale and examples for each item, and a data flow diagram. PRISMA- COSMIN for OMIs 2024 can improve the reporting of systematic reviews of OMIs, fostering their reproducibility and allowing end-users to appraise the quality of OMIs and select the most appropriate OMI for a specific application.

Validation of a Dutch version of the Neurological Fatigue Index (NFI-MS) for patients with multiple sclerosis in the Netherlands.

PURPOSE: Fatigue is a complex, multidimensional construct. Most questionnaires measuring fatigue do not meet the standards for good measurement properties. Therefore, the Neurological Fatigue Index for multiple sclerosis (NFI-MS) is recently developed using thorough methods. The aim of this study was to develop and validate the Dutch version of the NFI-MS. METHODS: The original English version of the NFI-MS was translated into Dutch using a forward-backward procedure. Reliability (intraclass correlation coefficient (ICC) and standard error of measurement (SEM)) were assessed for the summary score and the four subscale scores separately. Construct validity was assessed by testing hypotheses about expected correlations between the NFI-MS and other constructs (e.g. impact on functioning and depression) and expected differences on NFI scores between groups defined on the basis of disease type and work status. RESULTS: The reliability analyses (n = 118) showed that the ICC was 0.78 (NFI-MS summary) and ranged from 0.75 to 0.83 for the four subscales and the SEM was 2.5 (NFI-MS summary) and ranged from 1.1 to 2.2 for the four subscales. For construct validity (n = 136), 50 out of the 57 expected hypotheses were confirmed (88 %). CONCLUSION: The present study shows that the Dutch version of the NFI-MS is a reliable and valid self-report questionnaire for measuring fatigue in Dutch patients with MS.

Patient-Reported Outcome Measures in Key Sub-Saharan African Languages to Promote Diversity: A Scoping Review.

OBJECTIVES: Sub-Saharan Africa is a world region rich and diverse in cultures and languages; yet, it is also challenged with regard to resources that may facilitate the cultural adaptation or development of patient-reported outcome measures (PROMs). Systematic exclusion of patients' "voices," because of gaps in the availability of PROMs, may perpetuate health inequity. Hence, the objective is to describe the availability of PROMs in the non-English, sub-Saharan African languages. METHODS: A scoping review was conducted to identify PROMs that had been translated, validated, or developed for use in 32 selected, non-English, sub-Saharan African languages pertaining to health outcomes. Four databases were searched (May 7, 2021), and additional articles were identified through reference screening and via corresponding authors. Data were extracted in terms of country, language, population, construct, and PROM characteristics (eg, number of items). RESULTS: A total of 220 unique articles were included from 7451 records, leading to the identification of 126 unique PROMs. Most studies were conducted in either Ethiopia, Nigeria, or South Africa. As such, prevalent languages included Amharic, Yoruba, and non-English languages common to South Africa (eg, Setswana, Xhosa, and Zulu). No PROMs were identified in any of the languages for 27 sub-Saharan African countries or 10 of the 32 included languages. CONCLUSIONS: There are significant gaps in the availability of PROMs across the non-English African languages included. Nevertheless, the PROMs that were identified largely align with core outcome sets relevant to the prevalent disease burden in this world region. Consensus-based priority setting may inform the most pertinent gaps to be addressed.

Reliability of two lumbar motor control tests for people with low back pain that are feasible in clinical practice.

BACKGROUND: Clinically feasible and reliable methods to measure motor control in people with low back pain (LBP) are lacking. This reliability and measurement error study design (i.e. repeated measurements in stable patients) aimed to determine the intra- and interrater reliability, and measurement errors of several parameters for two clinical lumbar motor control tests. METHOD: Participants 18-65 years of age, with current or a history of LBP performed a spiral tracking task (n = 33; i.e., tracing a spiral on a computer monitor by making spinal movements) or a repositioning task (n = 34; i.e., returning the trunk to a predefined position). Accelerometers were used to measure trunk positions. To explore the potential of these tests, we evaluated a broad range of parameters. To assess intra- and interrater reliability, we calculated the intraclass correlation coefficient (ICC(2,1) for absolute agreement), standard error of measurement and smallest detectable change for each parameter. FINDINGS: Overall, the interrater reliability of the spiral tracking test was good (ICC>0.75). The reliability of the second and third trial revealed higher ICC values compared to the reliability of the first two trials. The intra- and interrater reliability of the repositioning test was overall poor (ICC <0.5, with the exception of trunk inclination: ICC: 0.5 to 0.75). CONCLUSION: The reliability and set-up of the spiral tracking test supports its feasibility for clinical use. Considering the poor reliability of the repositioning test, it is doubtful whether further development of this measurement protocol is indicated. Only for the direction trunk inclination further standardisation might be warranted.

Measurement properties of performance-based instruments to assess mental function during activity and participation in persons who have survived a stroke: A systematic review protocol.

BACKGROUND: A frequent consequence of stroke is impaired mental function, which often affects the ability to perform activities and participate in life situation. In occupational therapy practice, performance-based instruments during activity and participation are often used. However, it is important to assess if the instruments used are valid, reliable and responsive. OBJECTIVE: The objective of this systematic review is to investigate measurement properties of performance-based instruments to assess mental function during activity and participation in persons who have survived stroke. MATERIAL AND METHODS: Systematic database searches of PubMed, EMBASE, CINAHL, PsycINFO, and OTseeker will be conducted. COSMIN Risk of Bias checklist will be used to evaluate methodological quality of the included articles. Measurement properties of the included studies will be rated against criteria for good measurement properties. The overall evidence of each measurement property per instrument is graded using a modified GRADE approach. RESULTS: Results will be presented in text and tables. CONCLUSIONS: Conclusion will be drawn up-on the overall evidence to give recommendations on the most suitable instrument. SIGNIFICANCE: It is expected that findings of the review will provide evidence to guide professionals in the selection of a performance-based instruments to measure mental function in practice and research. TRIAL REGISTRATION NUMBER: PROSPERO CRD42018086744.

Interobserver variability studies in diagnostic imaging: a methodological systematic review.

OBJECTIVES: To review the methodology of interobserver variability studies; including current practice and quality of conducting and reporting studies. METHODS: Interobserver variability studies between January 2019 and January 2020 were included; extracted data comprised of study characteristics, populations, variability measures, key results, and conclusions. Risk of bias was assessed using the COSMIN tool for assessing reliability and measurement error. RESULTS: Seventy-nine full-text studies were included covering various imaging tests and clinical areas. The median number of patients was 47 (IQR:23-88), and observers were 4 (IQR:2-7), with sample size justified in 12 (15%) studies. Most studies used static images (n = 75, 95%), where all observers interpreted images for all patients (n = 67, 85%). Intraclass correlation coefficients (ICC) (n = 41, 52%), Kappa (κ) statistics (n = 31, 39%) and percentage agreement (n = 15, 19%) were most commonly used. Interpretation of variability estimates often did not correspond with study conclusions. The COSMIN risk of bias tool gave a very good/adequate rating for 52 studies (66%) including any studies that used variability measures listed in the tool. For studies using static images, some study design standards were not applicable and did not contribute to the overall rating. CONCLUSIONS: Interobserver variability studies have diverse study designs and methods, the impact of which requires further evaluation. Sample size for patients and observers was often small without justification. Most studies report ICC and κ values, which did not always coincide with the study conclusion. High ratings were assigned to many studies using the COSMIN risk of bias tool, with certain standards scored 'not applicable' when static images were used. ADVANCES IN KNOWLEDGE: The sample size for both patients and observers was often small without justification. For most studies, observers interpreted static images and did not evaluate the process of acquiring the imaging test, meaning it was not possible to assess many COSMIN risk of bias standards for studies with this design. Most studies reported intraclass correlation coefficient and κ statistics; study conclusions often did not correspond with results.

Studies on Reliability and Measurement Error of Measurements in Medicine - From Design to Statistics Explained for Medical Researchers.

Reliability and measurement error are measurement properties that quantify the influence of specific sources of variation, such as raters, type of machine, or time, on the score of the individual measurement. Several designs can be chosen to assess reliability and measurement error of a measurement. Differences in design are due to specific choices about which sources of variation are varied over the repeated measurements in stable patients, which potential sources of variation are kept stable (ie, restricted), and about whether or not the entire measurement instrument (or measurement protocol) was repeated or only part of it. We explain how these choices determine how intraclass correlation coefficients and standard errors of measurement formulas are built for different designs by using Venn diagrams. Strategies for improving the measurement are explained, and recommendations for reporting the essentials of these studies are described. We hope that this paper will facilitate the understanding and improve the design, analysis, and reporting of future studies on reliability and measurement error of measurements.

Measurement properties of performance-based instruments for assessing mental function during activity and participation in persons with stroke: A systematic review.

BACKGROUND: Various performance-based instruments exist to assess mental function after stroke and users have to select one for research or clinical practice. OBJECTIVES: To evaluate the measurement properties of performance-based instruments to assess (any aspect of) mental function during activity and participation in persons after stroke. MATERIAL AND METHODS: We searched in five electronic databases. COSMIN methodology was used to conduct the review. The strength of evidence was assessed using a modified GRADE approach. RESULTS: Fifty articles were included reporting on 20 instruments assessing (1) multiple mental functions including ≥ four subdomains (2) attention, memory and executive functions, or single subdomains (3) executive functions, (4) perception, and (5) mental function of language. Highest quality evidence for sufficient results was found for some measurement properties in seven instruments. These instruments included: FIM + FAM, MPAI-4 and EFPT, MET, CBS/KF-NAP, BIT and the Scenario Test. CONCLUSIONS: Further studies of high methodological quality are needed that evaluate the measurement properties of instruments to allow clinicians and researchers to select the most suitable performance-based measures for purpose. SIGNIFICANCE: Results may be used to select the most suitable performance-based instrument to measure mental function during activity and participation in persons with stroke. TRIAL REGISTRATION NUMBER: PROSPERO CRD42018086744.