Serious reporting deficiencies exist in minimal important difference studies: current state and suggestions for improvement.

BACKGROUND AND OBJECTIVES: To evaluate reporting of minimal important difference (MID) estimates using anchor-based methods for patient-reported outcome measures (PROMs), and the association with reporting deficiencies on their credibility. METHODS: Systematic survey of primary studies empirically estimating MIDs. We searched Medline, EMBASE, PsycINFO, and the Patient-Reported Outcome and Quality of Life Instruments Database until October 2018. We evaluated study reporting, focusing on participants' demographics, intervention(s), characteristics of PROMs and anchors, and MID estimation method(s). We assessed the impact of reporting issues on credibility of MID estimates. RESULTS: In 585 studies reporting on 5,324 MID estimates for 526 distinct PROMs, authors frequently failed to adequately report key characteristics of PROMs and MIDs, including minimum and maximum values of PROM scale, measure of variability accompanying the MID estimate and number of participants included in the MID calculation. Across MID estimates (n = 5,324), the most serious reporting issues impacting credibility included infrequent reporting of the correlation between the anchor and PROM (66%), inadequate details to judge precision of MID point estimate (13%), and insufficient information about the threshold used to ascertain MIDs (16%). CONCLUSION: Serious issues of incomplete reporting in the MID literature threaten the optimal use of MID estimates to inform the magnitude of effects of interventions on PROMs.

Minimal important difference estimates for patient-reported outcomes: A systematic survey.

OBJECTIVES: The objective of the study was to develop an inventory summarizing all anchor-based minimal important difference (MID) estimates for patient-reported outcome measures (PROMs) available in the medical literature. STUDY DESIGN AND SETTING: We searched MEDLINE, EMBASE, CINAHL, PsycINFO, and the Patient-Reported Outcome and Quality of Life Instruments Database internal library (January 1989-October 2018). We included primary studies empirically calculating an anchor-based MID estimate for any PROM in adults and adolescents. Pairs of reviewers independently screened and selected studies, extracted data, and evaluated the credibility of the MIDs. RESULTS: We identified 585 eligible studies, the majority conducted in Europe (n = 211) and North America (n = 179), reporting 5,324 MID estimates for 526 distinct PROMs. Investigators conducted their studies in the context of patients receiving surgical (n = 105, 18%), pharmacological (n = 85, 15%), rehabilitation (n = 65, 11%), or a combination of interventions (n = 194, 33%). Of all MID estimates, 59% (n = 3,131) used a global rating of change anchor. Major credibility limitations included weak correlation (n = 1,246, 23%) or no information regarding the correlation (n = 3,498, 66%) between the PROM and anchor and imprecision in the MID estimate (n = 2,513, 47%). CONCLUSION: A large number of MIDs for assisting in the interpretation of PROMs exist. The MID inventory will facilitate the use of MID estimates to inform the interpretation of the magnitude of treatment effects in clinical research and guideline development.

Evaluating the credibility of anchor based estimates of minimal important differences for patient reported outcomes: instrument development and reliability study.

OBJECTIVE: To develop an instrument to evaluate the credibility of anchor based minimal important differences (MIDs) for outcome measures reported by patients, and to assess the reliability of the instrument. DESIGN: Instrument development and reliability study. DATA SOURCES: Initial criteria were developed for evaluating the credibility of anchor based MIDs based on a literature review (Medline, Embase, CINAHL, and PsycInfo databases) and the experience of the authors in the methodology for estimation of MIDs. Iterative discussions by the team and pilot testing with experts and potential users facilitated the development of the final instrument. PARTICIPANTS: With the newly developed instrument, pairs of masters, doctoral, or postdoctoral students with a background in health research methodology independently evaluated the credibility of a sample of MID estimates. MAIN OUTCOME MEASURES: Core credibility criteria applicable to all anchor types, additional criteria for transition rating anchors, and inter-rater reliability coefficients were determined. RESULTS: The credibility instrument has five core criteria: the anchor is rated by the patient; the anchor is interpretable and relevant to the patient; the MID estimate is precise; the correlation between the anchor and the outcome measure reported by the patient is satisfactory; and the authors select a threshold on the anchor that reflects a small but important difference. The additional criteria for transition rating anchors are: the time elapsed between baseline and follow-up measurement for estimation of the MID is optimal; and the correlations of the transition rating with the baseline, follow-up, and change score in the patient reported outcome measures are satisfactory. Inter-rater reliability coefficients (ĸ) for the core criteria and for one item from the additional criteria ranged from 0.70 to 0.94. Reporting issues prevented the evaluation of the reliability of the three other additional criteria for the transition rating anchors. CONCLUSIONS: Researchers, clinicians, and healthcare policy decision makers can consider using this instrument to evaluate the design, conduct, and analysis of studies estimating anchor based minimal important differences.

What Do the OMERACT Shoulder Core Set Candidate Instruments Measure? An Analysis Using the Refined International Classification of Functioning, Disability, and Health Linking Rules.

OBJECTIVE: The objective of this paper is to assess the content and measurement constructs of the candidate instruments for the domains of "pain" and "physical function/activity" in the Outcome Measures in Rheumatology (OMERACT) shoulder core set. The results of this International Classification of Functioning, Disability, and Health (ICF)-based analysis may inform further decisions on which instruments should ultimately be included in the core set. METHODS: The materials for the analysis were the 13 candidate measurement instruments within pain and physical function/activity in the shoulder core domain set, which either passed or received amber ratings (meaning there were some issues with the instrument) in the OMERACT filtering process. The content of the candidate instruments was extracted and linked to the ICF using the refined linking rules. The linking rules enhance the comparability of instruments by providing a comprehensive overview of the content of the instruments, the context in which the measurements take place, the perspectives adopted, and the types of response options. RESULTS: The ICF content analysis showed a large variation in content and measurement constructs in the candidate instruments for the shoulder core outcome measurement set. CONCLUSION: Two of 6 pain instruments include constructs other than pain. Within the physical function/activity domain, 2 candidate instruments matched the domain, 3 included additional content, and 2 included meaningful concepts in the response options, suggesting that they should be omitted as candidate instruments. The analyses show that the content in most existing instruments of shoulder pain and functioning extends across core set domains.

Patients' experience of shoulder disorders: a systematic review of qualitative studies for the OMERACT Shoulder Core Domain Set.

OBJECTIVES: To describe the experiences (including symptoms and perceived impacts on daily living) of people with a shoulder disorder. METHODS: Systematic review of qualitative studies. We searched for eligible qualitative studies indexed in Ovid MEDLINE, Ovid Embase, CINAHL (EBSCO), SportDiscus (EBSCO) and Ovid PsycINFO up until November 2017. Two authors independently screened studies for inclusion, appraised their methodological quality using the Critical Appraisal Skills Programme checklist, used thematic synthesis methods to generate themes describing the experiences reported by participants and assessed the confidence in the findings using the Grading of Recommendations Assessment, Development and Evaluation Confidence in Evidence from Reviews of Qualitative research (GRADE-CERQual) approach. RESULTS: The inclusion criteria were met by eight studies, which included 133 participants (49 females and 84 males) with either rotator cuff disease, adhesive capsulitis, proximal humeral fracture, shoulder instability or unspecified shoulder pain. We generated seven themes to describe what people in the included studies reported experiencing: pain; physical function/activity limitations; participation restriction; sleep disruption; cognitive dysfunction; emotional distress; and other pathophysiological manifestations (other than pain). There were interactions between the themes, with particular experiences impacting on others (e.g. pain leading to reduced activities and sleep disruption). Following grading of the evidence, we considered it likely that most of the review findings were a reasonable representation of the experiences of people with shoulder disorders. CONCLUSION: Patients with shoulder disorders contend with considerable disruption to their life. The experiences described should be considered by researchers seeking to select the most appropriate outcomes to measure in clinical trials and other research studies in people with shoulder disorders.

The OMERACT Core Domain Set for Clinical Trials of Shoulder Disorders.

OBJECTIVE: To reach consensus on the core domains to be included in a core domain set for clinical trials of shoulder disorders using the Outcome Measures in Rheumatology (OMERACT) Filter 2.1 Core Domain Set process. METHODS: At OMERACT 2018, the OMERACT Shoulder Working Group conducted a workshop that presented the OMERACT 2016 preliminary core domain set and its rationale based upon a systematic review of domains measured in shoulder trials and international Delphi sessions involving patients, clinicians, and researchers, as well as a new systematic review of qualitative studies on the experiences of people with shoulder disorders. After discussions in breakout groups, the OMERACT core domain set for clinical trials of shoulder disorders was presented for endorsement by OMERACT 2018 participants. RESULTS: The qualitative review (n = 8) identified all domains included in the preliminary core set. An additional domain, cognitive dysfunction, was also identified, but confidence that this represents a core domain was very low. The core domain set that was endorsed by the OMERACT participants, with 71% agreement, includes 4 "mandatory" trial domains: pain, function, patient global - shoulder, and adverse events including death; and 4 "important but optional" domains: participation (recreation/work), sleep, emotional well-being, and condition-specific pathophysiological manifestations. Cognitive dysfunction was voted out of the core domain set. CONCLUSION: OMERACT 2018 delegates endorsed a core domain set for clinical trials of shoulder disorders. The next step includes identification of a core outcome measurement set that passes the OMERACT 2.1 Filter for measuring each domain.

Psychometric Properties of Patient-Reported Outcome Measures for Use in Patients with Anterior Cruciate Ligament Injuries: A Systematic Review.

BACKGROUND: Despite the high costs of anterior cruciate ligament (ACL) injuries to patients and society, we are not aware of any systematic reviews that have been performed on patient-reported outcome instruments designed for this patient population. The purpose of the present review is to identify currently available patient-reported outcome questionnaires for patients with ACL injuries, to appraise and assess the methodological quality and psychometric evidence of the evaluations of identified instruments, and to summarize the overall evidence for each included questionnaire. METHODS: A computerized literature search was performed to identify articles appropriate for inclusion in this systematic review. Search terms were related to outcome instruments, ACL injuries, and psychometric properties. Two independent reviewers assessed the suitability of articles for inclusion. The methodological quality of instrument evaluation was assessed with use of the Consensus-based Standards for the Selection of Health Status Measurement Instruments (COSMIN) checklist. Psychometric properties were assessed with use of the quality criteria established by Terwee et al. The 2 assessments were synthesized to obtain an overall rating of the psychometric evidence for each questionnaire. RESULTS: Forty-one studies investigating 24 instruments were included. The International Knee Documentation Committee (IKDC) Form and the Knee injury and Osteoarthritis Outcome Score (KOOS) were the most frequently assessed instruments. The ACL-Return to Sport after Injury (ACL-RSI) scale had 6 properties with positive evidence and was the highest-quality instrument overall. CONCLUSIONS: Many of the included studies had low COSMIN ratings. The ACL-RSI is a reliable and valid scale to measure outcomes for patients with ACL injuries. Further assessments with improved methodological quality are needed to determine the psychometric properties of other instruments.

Measurement Properties of Patient-Reported Outcome Measures Used in Patients Undergoing Total Hip Arthroplasty: A Systematic Review.

BACKGROUND: Many patient-reported outcome measures (PROMs) have been developed to evaluate patient conditions before and after total hip arthroplasty. Also, many studies have been conducted to evaluate and compare the qualities of these instruments. Previously published reports suggest that most of these studies have poor methodology. Recently, 2 sets of criteria were developed for guiding and assessing the methodological and psychometric quality of these PROMs. We reviewed PROMs for total hip arthroplasty patients and appraised the methodological quality and psychometric evidence of evaluations of each identified instrument. METHODS: Databases including PubMed, MEDLINE, Embase, CINAHL, the Cochrane Library, and others were searched for English-language articles published on or before April 14, 2017, using search terms related to outcome instrument, the condition or procedure of interest (hip arthroplasty), and psychometric properties. The methodological quality of the studies and the evidence of the psychometric properties were summarized and appraised using the COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) checklist and the psychometric evidence criteria. Overall psychometric ratings were derived by combining the 2 criteria. RESULTS: Seventy-three studies investigating 26 instruments were included. The Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC), Oxford Hip Score, Harris hip score, and the Hip disability and Osteoarthritis Outcome Score (HOOS) were the most frequently assessed instruments. The WOMAC had 5 properties with positive evidence and was the highest-quality instrument overall, followed by the HOOS and the European Health Interview Survey (EUROHIS)-Quality of Life 8-item index. CONCLUSIONS: Despite a large number of included studies, many had low COSMIN ratings. We recommend additional rigorous studies to explore the psychometric properties of these instruments. Furthermore, the development of a core outcome set for total hip arthroplasty clinical trials is needed.

Outcome Reporting in Randomized Trials for Shoulder Disorders: Literature Review to Inform the Development of a Core Outcome Set.

OBJECTIVE: To explore the outcome domains and measurement instruments reported across randomized trials of any interventions for various shoulder disorders. METHODS: We searched for shoulder trials included in Cochrane reviews published up to Issue 10, 2015, or indexed in PubMed between 2006 and 2015. Trials were eligible for inclusion if they focused on any intervention for rotator cuff disease, adhesive capsulitis, shoulder instability, glenohumeral or acromioclavicular osteoarthritis, shoulder dislocation, proximal humeral or humeral head fractures, or unspecified shoulder pain. Two authors independently selected trials for inclusion and extracted information on the domains and measurement instruments reported, with consensus discussion among all authors where required. RESULTS: We included 409 trials, published between 1954 and 2015. Across the trials, we identified 319 different instruments that were classified into 32 domains. Most trials reported a measure of pain (90%), range of motion (78%), and physical function (71%). The recording of adverse events was reported in only 31% of the trials. Muscle strength was reported in 44% of the trials and imaging outcomes in 21%. Other patient-reported outcome measures, such as global assessment of treatment success, health-related quality of life, work ability, and psychological functioning, were each reported in ≤15% of the trials. Most of the domains were reported at similar frequencies across different shoulder disorders. CONCLUSION: The domains and measurement instruments reported were widely diverse. Our results provide a foundation for the development of a core outcome set for use in future trials across all shoulder disorders.

Creation of a core outcome set for clinical trials of people with shoulder pain: a study protocol.

BACKGROUND: The selection of appropriate outcomes or domains is crucial when designing clinical trials, to appreciate the effects of different interventions, pool results, and make valid comparisons between trials. If the findings are to influence policy and practice, then the chosen outcomes need to be relevant and important to key stakeholders, including patients and the public, healthcare professionals and others making decisions about health care. There is a growing recognition that insufficient attention has been paid to the outcomes measured in clinical trials. Recent reviews of the measurement properties of patient-reported outcome measures for shoulder disorders revealed a large selection of diverse measures, many with questionable validity, reliability, and responsiveness. These issues could be addressed through the development and use of an agreed standardized collection of outcomes, known as a core outcome set (COS), which should be measured and reported in all trials of shoulder disorders. The purpose of the present project is to develop and disseminate a COS for clinical trials in shoulder disorders. METHODS/DESIGN: The methods for the COS development will include 3 phases: (1) a comprehensive review of the core domains used in shoulder disorder trials; (2) an international Delphi study involving relevant stakeholders (patients, clinicians, scientists) to define which domains should be core; and (3) an international focus group informed by the evidence identified in phases 1 and 2, to determine which measurement instruments best measure the core domains and identification of any evidence gaps that require further empiric evidence. DISCUSSION: The aim of the current proposal is to convene several meetings of international experts and patients to develop a COS for clinical trials of shoulder disorders and to develop an implementation strategy to ensure rapid uptake of the core set of outcomes in clinical trials. There would be an expectation that the core set of outcomes would always be collected and reported, but it would not preclude use of additional outcomes in a particular trial.

A systematic review of measurement properties of patient-reported outcome measures for use in patients with foot or ankle diseases.

PURPOSE: To identify currently available patient-reported outcome measures (PROMs) used in patients with foot or ankle diseases; and to critically appraise, compare and synthesize the psychometric evidence for the identified PROMs. METHODS: Literature searches were performed in Medline and EMBASE from their inception to January 25th, 2016. Methodological quality was evaluated using the COSMIN checklist. The final rating of the methodological quality of each study for each property was the lowest rating among the items within that property. The psychometric evidence of the properties investigated in the included articles was assessed using the quality criteria established by Terwee et al. The methodological quality ratings and psychometric evidence assessments were synthesized using the method first proposed by Schellingerhout et al. RESULTS: In total, 3077 articles were identified by the literature search, from which 115 studies investigating 50 PRO instruments were included in the review process. The Foot Function Index (FFI) was the most explored instrument, while the Manchester-Oxford Foot Questionnaire (MOXFQ) demonstrated the best properties. CONCLUSION: Most PROMs on foot and ankle diseases have limited evidence for their psychometric properties. The MOXFQ, with the highest overall ratings, could be a useful PROM for evaluating patients with foot or ankle diseases, based on current available evidence. More research is needed to improve the quality of the standards used to assess PROMs and the studies making these assessments.

A Systematic Review of Measurement Properties of Patient-Reported Outcome Measures Used in Patients Undergoing Total Knee Arthroplasty.

BACKGROUND: While clinical research on total knee arthroplasty (TKA) outcomes is prevalent in the literature, studies often have poor methodological and reporting quality. A high-quality patient-reported outcome instrument is reliable, valid, and responsive. Many studies evaluate these properties, but none have done so with a systematic and accepted method. The objectives of this study were to identify patient-reported outcome measures (PROMs) for TKA, and to critically appraise, compare, and summarize their psychometric properties using accepted methods. METHODS: MEDLINE, EMBASE, SCOPUS, Web of Science, PsycINFO, and SPORTDiscus were systematically searched for articles with the following inclusion criteria: publication before December 2014, English language, non-generic PRO, and evaluation in the TKA population. Methodological quality and evidence of psychometric properties were assessed with the COnsensus-based standards for the selection of health Status Measurement INstruments (COSMIN) checklist and criteria for psychometric evidence proposed by the COSMIN group and Terwee et al. RESULTS: One-hundred fifteen studies on 32 PROMs were included in this review. Only the Work, Osteoarthritis or joint-Replacement Questionnaire, the Oxford Knee Score, and the Western Ontario and McMaster Universities Arthritis Index had 4 or more properties with positive evidence. CONCLUSION: Most TKA PROMs have limited evidence for their psychometric properties. Although not all the properties were studied, the Work, Osteoarthritis or joint-Replacement Questionnaire, with the highest overall ratings, could be a useful PROM for evaluating patients undergoing TKA. The methods and reporting of this literature can improve by following accepted guidelines.

A Preliminary Core Domain Set for Clinical Trials of Shoulder Disorders: A Report from the OMERACT 2016 Shoulder Core Outcome Set Special Interest Group.

OBJECTIVE: The Outcome Measures in Rheumatology (OMERACT) Shoulder Core Outcome Set Special Interest Group (SIG) was established to develop a core outcome set (COS) for clinical trials of shoulder disorders. METHODS: In preparation for OMERACT 2016, we systematically examined all outcome domains and measurement instruments reported in 409 randomized trials of interventions for shoulder disorders published between 1954 and 2015. Informed by these data, we conducted an international Delphi consensus study including shoulder trial experts, clinicians, and patients to identify key domains that should be included in a shoulder disorder COS. Findings were discussed at a stakeholder premeeting of OMERACT. At OMERACT 2016, we sought consensus on a preliminary core domain set and input into next steps. RESULTS: There were 13 and 15 participants at the premeeting and the OMERACT 2016 SIG meeting, respectively (9 attended both meetings). Consensus was reached on a preliminary core domain set consisting of an inner core of 4 domains: pain, physical function/activity, global perceived effect, and adverse events including death. A middle core consisted of 3 domains: emotional well-being, sleep, and participation (recreation and work). An outer core of research required to inform the final COS was also formulated. CONCLUSION: Our next steps are to (1) analyze whether participation (recreation and work) should be in the inner core, (2) conduct a third Delphi round to finalize definitions and wording of domains and reach final endorsement for the domains, and (3) determine which instruments fulfill the OMERACT criteria for measuring each domain.

Identifying a core set of outcome domains to measure in clinical trials for shoulder disorders: a modified Delphi study.

OBJECTIVE: To achieve consensus on the most important outcome domains to measure across all clinical trials for shoulder disorders. METHODS: We performed an online modified Delphi study with an international, multidisciplinary and multistakeholder panel. A literature review and the OMERACT Filter 2.0 framework was used to generate a list of potential core domains, which were presented to patients, clinicians and researchers in two Delphi rounds. Participants were asked to judge the importance of each potential core domain and provide a rationale for their response. A core domain was defined a priori as a domain that at least 67% of participants considered core. RESULTS: In both rounds, 335 individuals were invited to participate (268 clinicians/researchers and 67 patients); response rates were 27% (n=91) and 29% (n=96), respectively. From a list of 41 potential core domains, four domains met our criteria for inclusion: 'pain', 'physical functioning', 'global assessment of treatment success' and 'health-related quality of life'. Two additional domains, 'sleep functioning' and 'psychological functioning', met the criteria for inclusion by some, but not all stakeholder groups. There was consensus that 'number of deaths' was not a core domain, but insufficient agreement on whether or not several other domains, including 'range of motion' and 'muscle strength', were core domains. CONCLUSIONS: Based on international consensus from patients, clinicians and researchers, 'pain', 'physical functioning', 'global assessment of treatment success' and 'health-related quality of life' were considered core outcome domains for shoulder disorder trials. The value of several other domains needs further consideration.

A Systematic Review of the Psychometric Properties of Patient-Reported Outcome Instruments for Use in Patients With Rotator Cuff Disease.

BACKGROUND: Many patient-reported outcome instruments (or questionnaires) have been developed for use in patients with rotator cuff disease. Before an instrument is implemented, its psychometric properties should be carefully assessed, and the methodological quality of papers that investigate a psychometric component of an instrument must be carefully evaluated. Together, the psychometric evidence and the methodological quality can then be used to arrive at an estimate of an instrument's quality. PURPOSE: To identify patient-reported outcome instruments used in patients with rotator cuff disease and to critically appraise and summarize their psychometric properties to guide researchers and clinicians in using high-quality patient-reported outcome instruments in this population. STUDY DESIGN: Systematic review. METHODS: Systematic literature searches were performed to find English-language articles concerning the development or evaluation of a psychometric property of a patient-reported outcome instrument for use in patients with rotator cuff disease. Methodological quality and psychometric evidence were critically appraised and summarized through 2 standardized sets of criteria. RESULTS: A total of 1881 articles evaluating 39 instruments were found per the search strategy, of which 73 articles evaluating 16 instruments were included in this study. The Constant-Murley score, the DASH (Disability of the Arm, Shoulder, and Hand), and the Shoulder Pain and Disability Index were the 3 most frequently evaluated instruments. In contrast, the psychometric properties of the Korean Shoulder Scoring System, Shoulder Activity Level, Subjective Shoulder Value, and Western Ontario Osteoarthritis Shoulder index were evaluated by only 1 study each. The Western Ontario Rotator Cuff Index was found to have the best overall quality of psychometric properties per the established criteria, with positive evidence found in internal consistency, reliability, content validity, hypothesis testing, and responsiveness. The DASH, Shoulder Pain and Disability Index, and Simple Shoulder Test had good evidence in support of internal consistency, reliability, structural validity, hypothesis testing, and responsiveness. Inadequate methodological quality was found across many studies, particularly in internal consistency, reliability, measurement error, hypothesis testing, and responsiveness. CONCLUSION: More high-quality methodological studies should be performed to assess the properties in all identified instruments.