Reviewer training for improving grant and journal peer review.

BACKGROUND: Funders and scientific journals use peer review to decide which projects to fund or articles to publish. Reviewer training is an intervention to improve the quality of peer review. However, studies on the effects of such training yield inconsistent results, and there are no up-to-date systematic reviews addressing this question. OBJECTIVES: To evaluate the effect of peer reviewer training on the quality of grant and journal peer review. SEARCH METHODS: We used standard, extensive Cochrane search methods. The latest search date was 27 April 2022. SELECTION CRITERIA: We included randomized controlled trials (RCTs; including cluster-RCTs) that evaluated peer review with training interventions versus usual processes, no training interventions, or other interventions to improve the quality of peer review. DATA COLLECTION AND ANALYSIS: We used standard Cochrane methods. Our primary outcomes were 1. completeness of reporting and 2. peer review detection of errors. Our secondary outcomes were 1. bibliometric scores, 2. stakeholders' assessment of peer review quality, 3. inter-reviewer agreement, 4. process-centred outcomes, 5. peer reviewer satisfaction, and 6. completion rate and speed of funded projects. We used the first version of the Cochrane risk of bias tool to assess the risk of bias, and we used GRADE to assess the certainty of evidence. MAIN RESULTS: We included 10 RCTs with a total of 1213 units of analysis. The unit of analysis was the individual reviewer in seven studies (722 reviewers in total), and the reviewed manuscript in three studies (491 manuscripts in total). In eight RCTs, participants were journal peer reviewers. In two studies, the participants were grant peer reviewers. The training interventions can be broadly divided into dialogue-based interventions (interactive workshop, face-to-face training, mentoring) and one-way communication (written information, video course, checklist, written feedback). Most studies were small. We found moderate-certainty evidence that emails reminding peer reviewers to check items of reporting checklists, compared with standard journal practice, have little or no effect on the completeness of reporting, measured as the proportion of items (from 0.00 to 1.00) that were adequately reported (mean difference (MD) 0.02, 95% confidence interval (CI) -0.02 to 0.06; 2 RCTs, 421 manuscripts). There was low-certainty evidence that reviewer training, compared with standard journal practice, slightly improves peer reviewer ability to detect errors (MD 0.55, 95% CI 0.20 to 0.90; 1 RCT, 418 reviewers). We found low-certainty evidence that reviewer training, compared with standard journal practice, has little or no effect on stakeholders' assessment of review quality in journal peer review (standardized mean difference (SMD) 0.13 standard deviations (SDs), 95% CI -0.07 to 0.33; 1 RCT, 418 reviewers), or change in stakeholders' assessment of review quality in journal peer review (SMD -0.15 SDs, 95% CI -0.39 to 0.10; 5 RCTs, 258 reviewers). We found very low-certainty evidence that a video course, compared with no video course, has little or no effect on inter-reviewer agreement in grant peer review (MD 0.14 points, 95% CI -0.07 to 0.35; 1 RCT, 75 reviewers). There was low-certainty evidence that structured individual feedback on scoring, compared with general information on scoring, has little or no effect on the change in inter-reviewer agreement in grant peer review (MD 0.18 points, 95% CI -0.14 to 0.50; 1 RCT, 41 reviewers, low-certainty evidence). AUTHORS' CONCLUSIONS: Evidence from 10 RCTs suggests that training peer reviewers may lead to little or no improvement in the quality of peer review. There is a need for studies with more participants and a broader spectrum of valid and reliable outcome measures. Studies evaluating stakeholders' assessments of the quality of peer review should ensure that these instruments have sufficient levels of validity and reliability.

Responsiveness and minimal important change of the Norwegian version of the Disabilities of the Arm, Shoulder and Hand questionnaire (DASH) in patients with subacromial pain syndrome.

BACKGROUND: The Disabilities of the Arm, Shoulder, and Hand questionnaire (DASH) is a valid and reliable patient-reported outcome measure (PROM). It was designed to measure physical disability and symptoms in patients with musculoskeletal disorders of the upper extremity, and is one of the most commonly used PROMs for patients with shoulder pain. The aim of this study was to examine responsiveness, the smallest detectable change (SDC) and the minimal important change (MIC) of the DASH, in line with international (COSMIN) recommendations. METHODS: The study sample consisted of 50 patients with subacromial pain syndrome, undergoing physical therapy for 3-4 months. Responsiveness to change was examined by calculating area under the receiver operating characteristic curves (AUC) and testing a priori-formulated hypothesis regarding correlations with changes in other instruments that measuring the same construct. The SDC was calculated using a test re-test protocol, and the MIC was calculated by the anchor-based MIC distribution. MIC values for patients with low and high baseline scores were also calculated. RESULTS: DASH appeared to be responsive, as it was able to distinguish patients who reported to be improved from those unchanged (AUC 0.77). All of the hypotheses were accepted. The SDC was 11.8, and the MIC was 4.4. CONCLUSION: This study shows that the Norwegian version of the DASH has good responsiveness to change and may thus be recommended to measure outcome in patients with shoulder pain in Norway.

Physical impairments in symptomatic femoroacetabular impingement: a systematic review of the evidence.

BACKGROUND: Femoroacetabular impingement (FAI) and accompanying pathologies are associated with pain and reduced quality of life. Physical impairments can be associated with worse symptoms and may be an important target of rehabilitation programmes in this patient group. Knowledge regarding physical impairments in individuals with symptomatic FAI is limited. HYPOTHESIS: In adults aged 18-50 years with symptomatic FAI, to: (1) identify physical impairments in range of motion (ROM), hip muscle function and functional tasks; (2) to compare physical impairments with healthy controls; and (3) to evaluate the effects of interventions targeting physical impairments. STUDY DESIGN: Systematic review. METHODS: A systematic review of the literature was conducted in accordance with the PRISMA statement. The modified Downs and Black checklist was used for quality appraisal. Studies of adults aged 18-50 years with symptomatic FAI that examined ROM, hip muscle function and functional tasks were included. Standardised mean differences (SMDs) were calculated where possible or best evidence synthesis and study conclusions were presented. RESULTS: 22 studies fulfilled all inclusion criteria. Methodological quality was varied (47-82% using Downs and Black Appraisal Criteria). Hip joint ROM did not differ in individuals with symptomatic FAI compared with control participants. Individuals with symptomatic FAI demonstrated deficits in hip muscle strength and reduced dynamic balance on 1 leg when compared with control participants. For hip joint ROM, there were no significant within-group differences between preintervention and postintervention time points. Hip muscle strength improved significantly from prehip to posthip arthroscopy in a single case series. No randomised controlled trails evaluated the effect of different types of interventions for symptomatic patients with symptomatic FAI. CONCLUSIONS: Individuals with symptomatic FAI demonstrate impairments in hip muscle strength and dynamic single leg balance. This information may assist therapists in providing targeted rehabilitation programmes for individuals with FAI and associated pathology. Further research is needed to determine whether symptomatic FAI affects other aspects of functional performance; and to evaluate whether targeted interventions are effective in symptomatic FAI. CLINICAL RELEVANCE: This information may assist therapists in providing targeted rehabilitation programmes for individuals with symptomatic FAI.

Exercise therapy may postpone total hip replacement surgery in patients with hip osteoarthritis: a long-term follow-up of a randomised trial.

BACKGROUND: Exercise treatment is recommended for all patients with hip osteoarthritis (OA), but its effect on the long-term need for total hip replacement (THR) is unknown. METHODS: We conducted a long-term follow-up of a randomised trial investigating the efficacy of exercise therapy and patient education versus patient education only on the 6-year cumulative survival of the native hip to THR in 109 patients with symptomatic and radiographic hip OA. Results regarding the primary outcome measure of the trial, self-reported pain at 16 months follow-up, have been reported previously. RESULTS: There were no group differences at baseline. The response rate at follow-up was 94%. 22 patients in the group receiving both exercise therapy and patient education and 31 patients in the group receiving patient education only underwent THR during the follow-up period, giving a 6-year cumulative survival of the native hip of 41% and 25%, respectively (p=0.034). The HR for survival of the native hip was 0.56 (CI 0.32 to 0.96) for the exercise therapy group compared with the control group. Median time to THR was 5.4 and 3.5 years, respectively. The exercise therapy group had better self-reported hip function prior to THR or end of study, but no significant differences were found for pain and stiffness. CONCLUSIONS: Our findings in this explanatory study suggest that exercise therapy in addition to patient education can reduce the need for THR by 44% in patients with hip OA. ClinicalTrials.gov number NCT00319423 (original project protocol) and NCT01338532 (additional protocol for long-term follow-up).

Reliability and validity of the Norwegian version of the Disabilities of the Arm, Shoulder and Hand questionnaire in patients with shoulder impingement syndrome.

BACKGROUND: Patient-rated outcome measures (PROMs) are an important part of clinical decision-making in rehabilitation of patients with shoulder pain. The Disabilities of Arm, Shoulder and Hand (DASH) questionnaire was designed to measure physical disability and symptoms in patients with musculoskeletal disorders of the upper extremity and is one the most commonly used outcome measures for patients with shoulder pain. The purpose of this study was to investigate the reliability and validity of the Norwegian version of the DASH in patients with shoulder impingement syndrome. METHODS: Sixty-three patients diagnosed with shoulder impingement syndrome at an orthopaedic outpatient clinic were included in the study. Internal consistency of the DASH was evaluated by the Cronbach's alpha and item-to-total correlations. Test-retest reliability was analyzed by the intraclass correlation coefficient (ICC) and limits of agreement (LoA) according to the Bland Altman method. Standard error of measurement (SEM) and minimally detectable change (MDC) were calculated for the total DASH score. Construct validity was evaluated by testing six a priori hypotheses for the Pearson's correlation coefficient between the DASH and the Shoulder Pain and Disability Index (SPADI), the 36-item Short Form Health Survey (SF-36) and a Numeric Pain Rating Scale (NPRS). RESULTS: Reliability: Cronbach's alpha of the DASH was 0.93 and item-to-total correlations ranged from 0.36 to 0.81. ICC was 0.89. The 95 percent LoA was calculated to be between -11.9 and 14.1. SEM was 4.7 and MDC 13.1. Construct validity: Eighty-three percent of the a priori hypotheses of correlation were confirmed. The DASH showed a high positive correlation of 0.75 with the SPADI, a negative moderate correlation of -0.48 to -0.62 with physical functioning, bodily pain and physical component summary of the SF-36 and a moderate positive correlation of 0.58 with the NPRS. DASH correlated higher with the physical component summary than with the mental component summary of the SF-36. CONCLUSIONS: The Norwegian version of the DASH is a reliable and valid outcome measure for patients with shoulder impingement syndrome.

Reliability and validity of the Physical Activity Scale for the Elderly (PASE) in patients with hip osteoarthritis.

BACKGROUND: Physical activity (PA) is beneficial in reducing pain and improving function in lower limb osteoarthritis (OA), and is recommended as a first line treatment. Self-administered questionnaires are used to assess PA, but knowledge about reliability and validity of these PA questionnaires are limited, in particular for patients with OA. The purpose of this study was to evaluate the reliability and validity of the Physical Activity Scale for the Elderly (PASE) in patients with hip OA. METHODS: Forty patients with hip OA (20 men and 20 women, mean age 61.3 ± 10 years) were included. For test-retest reliability PASE was administered twice with a mean time between tests of 9 ± 4 days. Intraclass correlation coefficient (ICC), standard error of measurement (SEM) and minimal detectable change (MDC) were calculated for the total score and for the particular items assessing different PA intensity levels. In addition a Bland-Altman analysis for the total PASE score was performed. Construct validity was evaluated by comparing the PASE results with the Actigraph GT1M accelerometer and the International Physical Activity Questionnaire (IPAQ), using the Spearman rank correlation coefficient. RESULTS: ICC for the total PASE score was 0.78, with relatively large error of measurement; SEM = 31 and MDC = 87. ICC for the intensity items was 0.20 for moderate PA intensity, 0.46 for light PA intensity and to 0.68 for vigorous PA intensity. The Spearman rank correlation coefficient between the Actigraph GT1M total counts per minute and the total PASE score was 0.30 (p = 0.089), and ranging from 0.20-0.38 for the different PA intensity categories. The Spearman rank correlation between IPAQ and PASE was 0.61 (p = 0.001) for the total scores. CONCLUSIONS: In patients with hip OA the test-retest reliability of the total PASE score was moderate, with acceptable ICC, but with large measurement errors. The construct validity of the PASE was poor when compared to the Actigraph GT1M accelerometer. Test-retest reliability and construct validity revealed that the PASE was unable to assess PA intensity levels. PASE is not recommended as a valid tool to examine PA level for patients with hip OA.

Time-dependent improvement in functional outcome following Oxford medial unicompartmental knee arthroplasty. A prospective longitudinal multicenter study involving 96 patients.

BACKGROUND AND PURPOSE: 10-year survival rates after unicompartmental knee replacement (UKR) have been up to 97% in single-center studies, but they have been as low as 80% in studies from arthroplasty registers. Few studies have evaluated short-term functional outcome and its improvement with time. We determined the time course of functional outcome as evaluated by the knee injury and osteoarthritis outcome score (KOOS) over the first 2 years after Oxford medial UKR. PATIENTS AND METHODS: In a prospective multicenter study, we included 99 unselected knees (96 patients, mean age 65 (51-80) years, 57 women) operated with Oxford medial UKR at 3 hospitals in the southeast of Norway between November 2003 and October 2006. Data were collected by independent investigators preoperatively and at 6 weeks, 3 months, 6 months, 1 year, and 2 years postoperatively. KOOS and range of motion (ROM) were determined at all follow-ups. RESULTS: Mean KOOS values for pain and activities of daily living were improved already after 6 weeks, and increased between each time point up to 2 years postoperatively. However, no statistically significant improvements were seen after 6 months. Mean active and passive ROM gradually improved up to 2 years after UKR, and were then better than before surgery. INTERPRETATION: Most of the expected improvements in pain and function after UKR are achieved within 6 months of surgery. Only minimal improvement can be expected beyond this time.

The Bridge Building Model: connecting evidence-based practice, evidence-based research, public involvement and needs led research.

This paper describes a model developed by an interdisciplinary team of research and public engagement specialists, with backgrounds in health and social care research, higher education, evidence-based practice, leadership, commissioning research and public involvement and engagement. The model we propose combines evidence-based practice, evidence-based research, public involvement and needs led research. Our aim is to capitalise on the joining of the rationale and methods for these approaches, which have all been highlighted as important, but for which there has been a lack of models for integration. Our ambition is to argue for and show an effective and evidence-based way of working that bridges health and social care needs identification, evidence-based practice and research.

This paper describes a model that combines different concepts in healthcare research that the authors suggest belong together. The concepts are; Evidence Based Practice, where healthcare is based on the best research evidence, Evidence Based Research, where new research is developed in the context of what research already exists, public involvement in research, where service users, carers and health and social care professionals engage with research, and Needs Led Research, where new research addresses the needs of people that use healthcare services.Combining these important approaches in our model is new and we argue for and show a way of working that 'builds bridges' between health and social care needs, healthcare practice and research.The Bridge Building Model has been developed by a team of research and public involvement specialists, with a variety of backgrounds in health and social care research. These include education, healthcare leadership using evidence in healthcare and research funding.

Individual versus general structured feedback to improve agreement in grant peer review: a randomized controlled trial.

BACKGROUND: Vast sums are distributed based on grant peer review, but studies show that interrater reliability is often low. In this study, we tested the effect of receiving two short individual feedback reports compared to one short general feedback report on the agreement between reviewers. METHODS: A total of 42 reviewers at the Norwegian Foundation Dam were randomly assigned to receive either a general feedback report or an individual feedback report. The general feedback group received one report before the start of the reviews that contained general information about the previous call in which the reviewers participated. In the individual feedback group, the reviewers received two reports, one before the review period (based on the previous call) and one during the period (based on the current call). In the individual feedback group, the reviewers were presented with detailed information on their scoring compared with the review committee as a whole, both before and during the review period. The main outcomes were the proportion of agreement in the eligibility assessment and the average difference in scores between pairs of reviewers assessing the same proposal. The outcomes were measured in 2017 and after the feedback was provided in 2018. RESULTS: A total of 2398 paired reviews were included in the analysis. There was a significant difference between the two groups in the proportion of absolute agreement on whether the proposal was eligible for the funding programme, with the general feedback group demonstrating a higher rate of agreement. There was no difference between the two groups in terms of the average score difference. However, the agreement regarding the proposal score remained critically low for both groups. CONCLUSIONS: We did not observe changes in proposal score agreement between 2017 and 2018 in reviewers receiving different feedback. The low levels of agreement remain a major concern in grant peer review, and research to identify contributing factors as well as the development and testing of interventions to increase agreement rates are still needed. TRIAL REGISTRATION: The study was preregistered at OSF.io/n4fq3 .

At a 10-Year Follow-up, Tendon Repair Is Superior to Physiotherapy in the Treatment of Small and Medium-Sized Rotator Cuff Tears.

BACKGROUND: Tendon repair and physiotherapy are frequently used treatment methods for small and medium-sized rotator cuff tears. In 2 previous publications of the 1 and 5-year results of this study, we reported significant but small between-group differences in favor of tendon repair. Long-term results are needed to assess whether the results in both groups remain stable over time. METHODS: In this study, 103 patients with a rotator cuff tear not exceeding 3 cm were randomly assigned to primary tendon repair or physiotherapy with optional secondary repair. Blinded follow-up was performed after 6 months and 1, 2, 5, and 10 years. Outcome measures included the Constant score; the self-report section of the American Shoulder and Elbow Surgeons score; the measurement of shoulder pain, motion, and strength; and patient satisfaction. Magnetic resonance imaging (MRI) was performed on surgically treated shoulders after 1 year, and ultrasound was performed on all shoulders after 5 and 10 years. The main analysis was by 1-way analysis of covariance and by intention to treat. RESULTS: Ninety-one of 103 patients attended the last follow-up. After 10 years, the results were better for primary tendon repair, by 9.6 points on the Constant score (p = 0.002), 15.7 points on the American Shoulder and Elbow Surgeons score (p < 0.001), 1.8 cm on a 10-cm visual analog scale for pain (p < 0.001), 19.6° for pain-free abduction (p = 0.007), and 14.3° for pain-free flexion (p = 0.01). Fourteen patients had crossed over from physiotherapy to secondary surgery and had an outcome on the Constant score that was 10.0 points inferior compared with that of the primary tendon repair group (p = 0.03). CONCLUSIONS: At 10 years, the differences in outcome between primary tendon repair and physiotherapy for small and medium-sized rotator cuff tears had increased, with better results for primary tendon repair. LEVEL OF EVIDENCE: Therapeutic Level I. See Instructions for Authors for a complete description of levels of evidence.

Long-Term Effect of Exercise Therapy and Patient Education on Impairments and Activity Limitations in People With Hip Osteoarthritis: Secondary Outcome Analysis of a Randomized Clinical Trial.

BACKGROUND: The effect of exercise on specific impairments and activity limitations in people with hip osteoarthritis (OA) is limited. OBJECTIVE: The study objective was to evaluate the long-term effect of exercise therapy and patient education on range of motion (ROM), muscle strength, physical fitness, walking capacity, and pain during walking in people with hip OA. DESIGN: This was a secondary outcome analysis of a randomized clinical trial. SETTING: The setting was a university hospital. PARTICIPANTS: One hundred nine people with clinically and radiographically evident hip OA were randomly allocated to receive both exercise therapy and patient education (exercise group) or patient education only (control group). INTERVENTION: All participants attended a patient education program consisting of 3 group meetings led by 2 physical therapists. Two other physical therapists were responsible for providing the exercise therapy program, consisting of 2 or 3 weekly sessions of strengthening, functional, and stretching exercises over 12 weeks. Both interventions were conducted at a sports medicine clinic. MEASUREMENTS: Outcome measures included ROM, isokinetic muscle strength, predicted maximal oxygen consumption determined with the Astrand bicycle ergometer test, and distance and pain during the Six-Minute Walk Test (6MWT). Follow-up assessments were conducted 4, 10, and 29 months after enrollment by 5 physical therapists who were unaware of group allocations. RESULTS: No significant group differences were found for ROM, muscle strength, predicted maximal oxygen consumption, or distance during the 6MWT over the follow-up period, but the exercise group had less pain during the 6MWT than the control group at 10 months (mean difference=-8.5 mm; 95% confidence interval=-16.1, -0.9) and 29 months (mean difference=-9.3 mm; 95% confidence interval=-18.1, -0.6). LIMITATIONS: Limitations of the study were reduced statistical power and 53% rate of adherence to the exercise therapy program. CONCLUSIONS: The previously described effect of exercise on self-reported function was not reflected by beneficial results for ROM, muscle strength, physical fitness, and walking capacity, but exercise in addition to patient education resulted in less pain during walking in the long term.

Tendon repair compared with physiotherapy in the treatment of rotator cuff tears: a randomized controlled study in 103 cases with a five-year follow-up.

BACKGROUND: There is limited Level-I evidence that compares operative and nonoperative treatment of rotator cuff tears. We compared outcomes of patients treated with primary tendon repair with outcomes of those treated with physiotherapy and optional secondary tendon repair if needed. METHODS: A single-center, pragmatic, randomized controlled study with follow-ups after six months and one, two, and five years was conducted in a secondary-care institution. One hundred and three patients with a rotator cuff tear not exceeding 3 cm were randomized to primary tendon repair (n = 52) or physiotherapy (n = 51). The primary outcome measure was the Constant score. Secondary outcome measures included the self-report section of the American Shoulder and Elbow Surgeons score; the physical component summary measure of the Short Form 36 Health Survey; the measurement of pain, strength, and shoulder motion; patient satisfaction; and findings from magnetic resonance imaging and sonography. Analysis was by intention to treat. RESULTS: The five-year follow-up rate was 98%. Twelve of the fifty-one patients in the physiotherapy group were treated with secondary tendon repair. The results from primary tendon repair were superior to those from physiotherapy plus secondary repair, with between-group mean differences of 5.3 points on the Constant score (p = 0.05), 9.0 points on the American Shoulder and Elbow Surgeons score (p < 0.001), 1.1 cm on a 10-cm visual analog scale for pain (p < 0.001), and 1.0 cm on a 10-cm visual analog scale for patient satisfaction (p = 0.03). In 37% of tears treated with physiotherapy only, there were increasing tear sizes on ultrasound of >5 mm, over five years, associated with an inferior outcome. CONCLUSIONS: Although primary repair of small and medium-sized rotator cuff tears was associated with better outcome than physiotherapy treatment, the differences were small and may be below clinical importance. In the physiotherapy treatment group, there were increasing tear sizes and inferior outcomes in one-third of patients who did not undergo repair.

Development and reliability of a clinician-rated instrument to evaluate function in individuals with shoulder pain: a preliminary study.

BACKGROUND AND PURPOSE: Subacromial impingement syndrome (SIS) is a common and disabling condition in the population. Interventions are often evaluated with patient-rated outcome measures. The purpose of this study was to develop a simple clinician-rated measure to detect difficulties in the execution of movement-related tasks among patients with subacromial impingement syndrome. METHOD: The steps in the scale development included a review of the clinical literature of shoulder pain to identify condition-specific questionnaires, pilot testing, clinical testing and scale construction. Twenty-one eligible items from thirteen questionnaires were extracted and included in a pilot test. All items were scored on a five-point ordinal scale ranging from 1 (no difficulty) to 5 (cannot perform). Fourteen items were excluded after pilot testing because of difficulties in standardization or other practical considerations. The remaining seven items were included in a clinical test-retest study with outpatients at a hospital. Of these, four were excluded because of psychometric reasons. From the remaining three items, a measure named Shoulder Activity Scale (summed score ranging from 3 to 15) was developed. RESULTS: A total of 33 men and 30 women were included in the clinical study; age range 27-80 years. The intraclass correlation coefficient results for inter-rater reliability and test-retest reliability were 0.80 (95% CI = 0.51-0.90) and 0.74 (95% CI = 0.58-0.84), respectively. The standard error of measurement and minimal detectable change were 1.19 and 3.32, respectively. The scale was linked to the International Classification of Functioning, Disability and Health second level categories lifting and carrying objects (d430), dressing (d540), hand and arm use (d445) and control of voluntary movement (b760). CONCLUSION: The Shoulder Activity Scale showed acceptable reliability in a sample of outpatients at a hospital, rated by clinicians experienced in shoulder rehabilitation. The validity of the scale should be investigated in future studies before application to common practice. © 2013 The Authors. Physiotherapy Research International published by John Wiley & Sons Ltd.