A survey of experts to identify methods to detect problematic studies: Stage 1 of the INSPECT-SR Project.

Background: Randomised controlled trials (RCTs) inform healthcare decisions. Unfortunately, some published RCTs contain false data, and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs which have been conducted on a given topic. This means that any of these 'problematic studies' are likely to be included, but there are no agreed methods for identifying them. The INSPECT-SR project is developing a tool to identify problematic RCTs in systematic reviews of healthcare-related interventions. The tool will guide the user through a series of 'checks' to determine a study's authenticity. The first objective in the development process is to assemble a comprehensive list of checks to consider for inclusion. Methods: We assembled an initial list of checks for assessing the authenticity of research studies, with no restriction to RCTs, and categorised these into five domains: Inspecting results in the paper; Inspecting the research team; Inspecting conduct, governance, and transparency; Inspecting text and publication details; Inspecting the individual participant data. We implemented this list as an online survey, and invited people with expertise and experience of assessing potentially problematic studies to participate through professional networks and online forums. Participants were invited to provide feedback on the checks on the list, and were asked to describe any additional checks they knew of, which were not featured in the list. Results: Extensive feedback on an initial list of 102 checks was provided by 71 participants based in 16 countries across five continents. Fourteen new checks were proposed across the five domains, and suggestions were made to reword checks on the initial list. An updated list of checks was constructed, comprising 116 checks. Many participants expressed a lack of familiarity with statistical checks, and emphasized the importance of feasibility of the tool. Conclusions: A comprehensive list of trustworthiness checks has been produced. The checks will be evaluated to determine which should be included in the INSPECT-SR tool.

Study found increasing use of core outcome sets in Cochrane systematic reviews and identified facilitators and barriers.

OBJECTIVES: In 2019, only 7% of Cochrane systematic reviews (SRs) cited a core outcome set (COS) in relation to choosing outcomes, even though a relevant COS existed but was not mentioned (or cited) for a further 29% of SRs. Our objectives for the current work were to (1) examine the extent to which authors are currently considering COS to inform outcome choice in Cochrane protocols and completed SRs, and (2) understand author facilitators and barriers to using COS. STUDY DESIGN AND SETTING: We examined all completed Cochrane SRs published in the last 3 months of 2022 and all Cochrane protocols published in 2022 for the extent to which they: (a) cited a COS, (b) searched for COS, (c) used outcomes from existing COS, and (d) reported outcome inconsistency among included studies and/or noted the need for COS. One investigator extracted information; a second extractor verified all information, discussing discrepancies to achieve consensus. We then conducted an online survey of authors of the included SRs to assess awareness of COS and identify facilitators and barriers to using COS to inform outcome choice. RESULTS: Objective 1: We included 294 SRs of interventions (84 completed SRs and 210 published SR protocols), of which 13% cited specific COS and 5% did not cite but mentioned searching for COS. A median of 83% of core outcomes from cited COS (interquartile range [IQR] 57%-100%) were included in the corresponding SR. We identified a relevant COS for 39% of SRs that did not cite a COS. A median of 50% of core outcomes from noncited COS (IQR 35%-72%) were included in the corresponding SR. Objective 2: Authors of 236 (80%) of the 294 eligible SRs completed our survey. Seventy-seven percent of authors noted being aware of COS before the survey. Fifty-five percent of authors who did not cite COS but were aware of them reported searching for a COS. The most reported facilitators of using COS were author awareness of the existence of COS (59%), author positive perceptions of COS (52%), and recommendation in the Cochrane Handbook regarding COS use (48%). The most reported barriers related to matching of the scope of the COS and the SR: the COS target population was too narrow/broad relative to the SR population (29%) or the COS target intervention was too narrow/broad relative to the SR intervention (21%). Most authors (87%) mentioned that they would consider incorporating missing core outcomes in the SR/update. CONCLUSION: Since 2019, there is increasing consideration and awareness of COS when choosing outcomes for Cochrane SRs of interventions, but uptake remains low and can be improved further. Use of COS in SRs is important to improve outcome standardization, reduce research waste, and improve evidence syntheses of the relevant effects of interventions across health research.

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions.

INTRODUCTION: Randomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare-related interventions. METHODS AND ANALYSIS: The INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: (1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, (2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, (3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in, (4) a consensus meeting to select checks to be included in a draft tool and to determine its format and (5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies and will help patients by protecting them from the influence of false data on their healthcare. ETHICS AND DISSEMINATION: The University of Manchester ethics decision tool was used, and this returned the result that ethical approval was not required for this project (30 September 2022), which incorporates secondary research and surveys of professionals about subjects relating to their expertise. Informed consent will be obtained from all survey participants. All results will be published as open-access articles. The final tool will be made freely available.

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions.

Introduction: Randomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INSPECT-SR project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare related interventions. Methods and analysis: The INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: 1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, 2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, 3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in 4) a consensus meeting to select checks to be included in a draft tool and to determine its format, 5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies, and will help patients by protecting them from the influence of false data on their healthcare.

Pressure modification or humidification for improving usage of continuous positive airway pressure machines in adults with obstructive sleep apnoea.

BACKGROUND: Obstructive sleep apnoea (OSA) is the repetitive closure of the upper airway during sleep. This results in disturbed sleep and excessive daytime sleepiness. It is a risk factor for long-term cardiovascular morbidity. Continuous positive airway pressure (CPAP) machines can be applied during sleep. They deliver air pressure by a nasal or oronasal mask to prevent the airway from closing, reducing sleep disturbance and improving sleep quality. Some people find them difficult to tolerate because of high pressure levels and other symptoms such as a dry mouth. Switching to machines that vary the level of air pressure required to reduce sleep disturbance could increase comfort and promote more regular use. Humidification devices humidify the air that is delivered to the upper airway through the CPAP circuit. Humidification may reduce dryness of the throat and mouth and thus improve CPAP tolerability. This updated Cochrane Review looks at modifying the delivery of positive pressure and humidification on machine usage and other clinical outcomes in OSA. OBJECTIVES: To determine the effects of positive pressure modification or humidification on increasing CPAP machine usage in adults with OSA. SEARCH METHODS: We searched Cochrane Airways Specialised Register and clinical trials registries on 15 October 2018. SELECTION CRITERIA: Randomised parallel group or cross-over trials in adults with OSA. We included studies that compared automatically adjusting CPAP (auto-CPAP), bilevel positive airway pressure (bi-PAP), CPAP with expiratory pressure relief (CPAPexp), heated humidification plus fixed CPAP, automatically adjusting CPAP with expiratory pressure relief, Bi-PAP with expiratory pressure relief, auto bi-PAP and CPAPexp with wakefulness detection with fixed pressure setting. DATA COLLECTION AND ANALYSIS: We used standard methods expected by Cochrane. We assessed the certainty of evidence using GRADE for the outcomes of machine usage, symptoms (measured by the Epworth Sleepiness Scale (ESS)), Apnoea Hypopnoea Index (AHI), quality of life measured by Functional Outcomes of Sleep Questionnaire (FOSQ), blood pressure, withdrawals and adverse events (e.g. nasal blockage or mask intolerance). The main comparison of interest in the review is auto-CPAP versus fixed CPAP. MAIN RESULTS: We included 64 studies (3922 participants, 75% male). The main comparison of auto-CPAP with fixed CPAP is based on 36 studies with 2135 participants from Europe, USA, Hong Kong and Australia. The majority of studies recruited participants who were recently diagnosed with OSA and had not used CPAP previously. They had excessive sleepiness (ESS: 13), severe sleep disturbance (AHI ranged from 22 to 59), and average body mass index (BMI) of 35 kg/m2. Interventions were delivered at home and the duration of most studies was 12 weeks or less. We judged that studies at high or unclear risk of bias likely influenced the effect of auto-CPAP on machine usage, symptoms, quality of life and tolerability, but not for other outcomes. Primary outcome Compared with average usage of about five hours per night with fixed CPAP, people probably use auto-CPAP for 13 minutes longer per night at about six weeks (mean difference (MD) 0.21 hours/night, 95% confidence interval (CI) 0.11 to 0.31; 31 studies, 1452 participants; moderate-certainty evidence). We do not have enough data to determine whether auto-CPAP increases the number of people who use machines for more than four hours per night compared with fixed CPAP (odds ratio (OR) 1.16, 95% CI 0.75 to 1.81; 2 studies, 346 participants; low-certainty evidence). Secondary outcomes Auto-CPAP probably reduces daytime sleepiness compared with fixed CPAP at about six weeks by a small amount (MD -0.44 ESS units, 95% CI -0.72 to -0.16; 25 studies, 1285 participants; moderate-certainty evidence). AHI is slightly higher with auto-CPAP than with fixed CPAP (MD 0.48 events per hour, 95% CI 0.16 to 0.80; 26 studies, 1256 participants; high-certainty evidence), although it fell with both machine types from baseline values in the studies. Ten per cent of people in auto-CPAP and 11% in the fixed CPAP arms withdrew from the studies (OR 0.90, 95% CI 0.64 to 1.27; moderate-certainty evidence). Auto-CPAP and fixed CPAP may have similar effects on quality of life, as measured by the FOSQ but more evidence is needed to be confident in this result (MD 0.12, 95% CI -0.21 to 0.46; 3 studies, 352 participants; low-certainty evidence). Two studies (353 participants) provided data on clinic-measured blood pressure. Auto-CPAP may be slightly less effective at reducing diastolic blood pressure compared to fixed CPAP (MD 2.92 mmHg, 95% CI 1.06 to 4.77 mmHg; low-certainty evidence). The two modalities of CPAP probably do not differ in their effects on systolic blood pressure (MD 1.87 mmHg, 95% CI -1.08 to 4.82; moderate-certainty evidence). Nine studies (574 participants) provided information on adverse events such as nasal blockage, dry mouth, tolerance of treatment pressure and mask leak. They used different scales to capture these outcomes and due to variation in the direction and size of effect between the studies, the comparative effects on tolerability outcomes are uncertain (very low-certainty evidence).  The evidence base for other interventions is smaller, and does not provide sufficient information to determine whether there are important differences between pressure modification strategies and fixed CPAP on machine usage outcomes, symptoms and quality of life. As with the evidence for the auto-CPAP, adverse events are measured disparately. AUTHORS' CONCLUSIONS: In adults with moderate to severe sleep apnoea starting positive airway pressure therapy, auto-CPAP probably increases machine usage by about 13 minutes per night. The effects on daytime sleepiness scores with auto-CPAP are not clinically meaningful. AHI values are slightly lower with fixed CPAP. Use of validated quality of life instruments in the studies to date has been limited, although where they have been used the effect sizes have not exceeded proposed clinically important differences. The adoption of a standardised approach to measuring tolerability would help decision-makers to balance benefits with harms from the different treatment options available. The evidence available for other pressure modification strategies does not provide a reliable basis on which to draw firm conclusions. Future studies should look at the effects of pressure modification devices and humidification in people who have already used CPAP but are unable to persist with treatment.

A cross-sectional audit showed that most Cochrane intervention reviews searched trial registers.

OBJECTIVE: The objective of this study was to assess current Cochrane Review practice in identifying and incorporating information from clinical trial registers. STUDY DESIGN AND SETTING: A cross-sectional study was conducted to assess a sample of new or updated intervention reviews from all Cochrane Review Groups up to February 1, 2017. Two assessors independently extracted data from each review using a pretested audit questionnaire. Data were analyzed relating to the frequency of reporting (1) the register source and search strategy; (2) the results of trial register searches; and (3) the use of trial register information in the review. RESULTS: Over 90% (236/260) of Cochrane Reviews reported searching a trial register (e.g., ClinicalTrials.gov or the WHO International Clinical Trials Registry Platform). In reviews that reported trial register searches, 39% (92/236) indicated the number of trial records retrieved and 56.8% (134/236) used information from the trial register records in the review. Trial record information was incorporated into the results (39.6%; 53/134), risk of bias assessments (53.7%; 72/134), and discussion (24.6%, 33/134) and conclusion sections (25.4%, 34/134). CONCLUSION: Most audited reviews used trial register information. Guidance may be needed to better incorporate information from these valuable resources in Cochrane Reviews to assist future research decisions made by funders and prospective study investigators.

School-based self-management interventions for asthma in children and adolescents: a mixed methods systematic review.

BACKGROUND: Asthma is a common respiratory condition in children that is characterised by symptoms including wheeze, shortness of breath, chest tightness, and cough. Children with asthma may be able to manage their condition more effectively by improving inhaler technique, and by recognising and responding to symptoms. Schools offer a potentially supportive environment for delivering interventions aimed at improving self-management skills among children. The educational ethos aligns with skill and knowledge acquisition and makes it easier to reach children with asthma who do not regularly engage with primary care. Given the multi-faceted nature of self-management interventions, there is a need to understand the combination of intervention features that are associated with successful delivery of asthma self-management programmes. OBJECTIVES: This review has two primary objectives.• To identify the intervention features that are aligned with successful intervention implementation.• To assess effectiveness of school-based interventions provided to improve asthma self-management among children.We addressed the first objective by performing qualitative comparative analysis (QCA), a synthesis method described in depth later, of process evaluation studies to identify the combination of intervention components and processes that are aligned with successful intervention implementation.We pursued the second objective by undertaking meta-analyses of outcomes reported by outcome evaluation studies. We explored the link between how well an intervention is implemented and its effectiveness by using separate models, as well as by undertaking additional subgroup analyses. SEARCH METHODS: We searched the Cochrane Airways Trials Register for randomised studies. To identify eligible process evaluation studies, we searched MEDLINE, Embase, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycINFO, the Cochrane Database of Systematic Reviews (CDSR), Web of Knowledge, the Database of Promoting Health Effectiveness Reviews (DoPHER), the Database of Abstracts of Reviews of Effects (DARE), the International Biography of Social Science (IBSS), Bibliomap, Health Technology Assessment (HTA), Applied Social Sciences Index and Abstracts (ASSIA), and Sociological Abstracts (SocAbs). We conducted the latest search on 28 August 2017. SELECTION CRITERIA: Participants were school-aged children with asthma who received the intervention in school. Interventions were eligible if their purpose was to help children improve management of their asthma by increasing knowledge, enhancing skills, or changing behaviour. Studies relevant to our first objective could be based on an experimental or quasi-experimental design and could use qualitative or quantitative methods of data collection. For the second objective we included randomised controlled trials (RCTs) where children were allocated individually or in clusters (e.g. classrooms or schools) to self-management interventions or no intervention control. DATA COLLECTION AND ANALYSIS: We used qualitative comparative analysis (QCA) to identify intervention features that lead to successful implementation of asthma self-management interventions. We measured implementation success by reviewing reports of attrition, intervention dosage, and treatment adherence, irrespective of effects of the interventions.To measure the effects of interventions, we combined data from eligible studies for our primary outcomes: admission to hospital, emergency department (ED) visits, absence from school, and days of restricted activity due to asthma symptoms. Secondary outcomes included unplanned visits to healthcare providers, daytime and night-time symptoms, use of reliever therapies, and health-related quality of life as measured by the Asthma Quality of Life Questionnaire (AQLQ). MAIN RESULTS: We included 55 studies in the review. Thirty-three studies in 14,174 children provided information for the QCA, and 33 RCTs in 12,623 children measured the effects of interventions. Eleven studies contributed to both the QCA and the analysis of effectiveness. Most studies were conducted in North America in socially disadvantaged populations. High school students were better represented among studies contributing to the QCA than in studies contributing to effectiveness evaluations, which more commonly included younger elementary and junior high school students. The interventions all attempted to improve knowledge of asthma, its triggers, and stressed the importance of regular practitioner review, although there was variation in how they were delivered.QCA results highlighted the importance of an intervention being theory driven, along with the importance of factors such as parent involvement, child satisfaction, and running the intervention outside the child's own time as drivers of successful implementation.Compared with no intervention, school-based self-management interventions probably reduce mean hospitalisations by an average of about 0.16 admissions per child over 12 months (SMD -0.19, 95% CI -0.35 to -0.04; 1873 participants; 6 studies, moderate certainty evidence). They may reduce the number of children who visit EDs from 7.5% to 5.4% over 12 months (OR 0.70, 95% CI 0.53 to 0.92; 3883 participants; 13 studies, low certainty evidence), and probably reduce unplanned visits to hospitals or primary care from 26% to 21% at 6 to 9 months (OR 0.74, 95% CI 0.60 to 0.90; 3490 participants; 5 studies, moderate certainty evidence). Self-management interventions probably reduce the number of days of restricted activity by just under half a day over a two-week period (MD 0.38 days 95% CI -0.41 to -0.18; 1852 participants; 3 studies, moderate certainty evidence). Effects of interventions on school absence are uncertain due to the variation between the results of the studies (MD 0.4 fewer school days missed per year with self-management (-1.25 to 0.45; 4609 participants; 10 studies, low certainty evidence). Evidence is insufficient to show whether the requirement for reliever medications is affected by these interventions (OR 0.52, 95% CI 0.15 to 1.81; 437 participants; 2 studies; very low-certainty evidence). Self-management interventions probably improve children's asthma-related quality of life by a small amount (MD 0.36 units higher on the Paediatric AQLQ(95% CI 0.06 to 0.64; 2587 participants; 7 studies, moderate certainty evidence). AUTHORS' CONCLUSIONS: School-based asthma self-management interventions probably reduce hospital admission and may slightly reduce ED attendance, although their impact on school attendance could not be measured reliably. They may also reduce the number of days where children experience asthma symptoms, and probably lead to small improvements in asthma-related quality of life. Many of the studies tested the intervention in younger children from socially disadvantaged populations. Interventions that had a theoretical framework, engaged parents and were run outside of children's free time were associated with successful implementation.

Influenza vaccination for healthcare workers who care for people aged 60 or older living in long-term care institutions.

BACKGROUND: A systematic review found that 3% of working adults who had received influenza vaccine and 5% of those who were unvaccinated had laboratory-proven influenza per season; in healthcare workers (HCWs) these percentages were 5% and 8% respectively. Healthcare workers may transmit influenza to patients. OBJECTIVES: To identify all randomised controlled trials (RCTs) and non-RCTs assessing the effects of vaccinating healthcare workers on the incidence of laboratory-proven influenza, pneumonia, death from pneumonia and admission to hospital for respiratory illness in those aged 60 years or older resident in long-term care institutions (LTCIs). SEARCH METHODS: We searched CENTRAL (2015, Issue 9), MEDLINE (1966 to October week 3, 2015), EMBASE (1974 to October 2015) and Web of Science (2006 to October 2015), but Biological Abstracts only from 1969 to March 2013 and Science Citation Index-Expanded from 1974 to March 2013 due to lack of institutional access in 2015. SELECTION CRITERIA: Randomised controlled trials (RCTs) and non-RCTs of influenza vaccination of healthcare workers caring for individuals aged 60 years or older in LTCIs and the incidence of laboratory-proven influenza and its complications (lower respiratory tract infection, or hospitalisation or death due to lower respiratory tract infection) in individuals aged 60 years or older in LTCIs. DATA COLLECTION AND ANALYSIS: Two authors independently extracted data and assessed risk of bias. Effects on dichotomous outcomes were measured as risk differences (RDs) with 95% confidence intervals (CIs). We assessed the quality of evidence with GRADE. MAIN RESULTS: We identified four cluster-RCTs and one cohort study (n = 12,742) of influenza vaccination for HCWs caring for individuals ≥ 60 years in LTCIs. Four cluster RCTs (5896 residents) provided outcome data that addressed the objectives of our review. The studies were comparable in their study populations, intervention and outcome measures. The studies did not report adverse events. The principal sources of bias in the studies related to attrition, lack of blinding, contamination in the control groups and low rates of vaccination coverage in the intervention arms, leading us to downgrade the quality of evidence for all outcomes due to serious risk of bias.Offering influenza vaccination to HCWs based in long term care homes may have little or no effect on the number of residents who develop laboratory-proven influenza compared with those living in care homes where no vaccination is offered (RD 0 (95% CI -0.03 to 0.03), two studies with samples taken from 752 participants; low quality evidence). HCW vaccination probably leads to a reduction in lower respiratory tract infection in residents from 6% to 4% (RD -0.02 (95% CI -0.04 to 0.01), one study of 3400 people; moderate quality evidence). HCW vaccination programmes may have little or no effect on the number of residents admitted to hospital for respiratory illness (RD 0 (95% CI -0.02 to 0.02, one study of 1059 people; low quality evidence). We decided not to combine data on deaths from lower respiratory tract infection (two studies of 4459 people) or all cause deaths (four studies of 8468 people). The direction and size of difference in risk varied between the studies. We are uncertain as to the effect of vaccination on these outcomes due to the very low quality of evidence. Adjusted analyses, which took into account the cluster design, did not differ substantively from the pooled analysis with unadjusted data. AUTHORS' CONCLUSIONS: Our review findings have not identified conclusive evidence of benefit of HCW vaccination programmes on specific outcomes of laboratory-proven influenza, its complications (lower respiratory tract infection, hospitalisation or death due to lower respiratory tract illness), or all cause mortality in people over the age of 60 who live in care institutions. This review did not find information on co-interventions with healthcare worker vaccination: hand-washing, face masks, early detection of laboratory-proven influenza, quarantine, avoiding admissions, antivirals and asking healthcare workers with influenza or influenza-like illness (ILI) not to work. This review does not provide reasonable evidence to support the vaccination of healthcare workers to prevent influenza in those aged 60 years or older resident in LTCIs. High quality RCTs are required to avoid the risks of bias in methodology and conduct identified by this review and to test further these interventions in combination.

Do Cochrane summaries help student midwives understand the findings of Cochrane systematic reviews: the BRIEF randomised trial.

BACKGROUND: Abstracts and plain language summaries (PLS) are often the first, and sometimes the only, point of contact between readers and systematic reviews. It is important to identify how these summaries are used and to know the impact of different elements, including the authors' conclusions. The trial aims to assess whether (a) the abstract or the PLS of a Cochrane Review is a better aid for midwifery students in assessing the evidence, (b) inclusion of authors' conclusions helps them and (c) there is an interaction between the type of summary and the presence or absence of the conclusions. METHODS: Eight hundred thirteen midwifery students from nine universities in the UK and Ireland were recruited to this 2 × 2 factorial trial (abstract versus PLS, conclusions versus no conclusions). They were randomly allocated to one of four groups and asked to recall knowledge after reading one of four summary formats of two Cochrane Reviews, one with clear findings and one with uncertain findings. The primary outcome was the proportion of students who identified the appropriate statement to describe the main findings of the two reviews as assessed by an expert panel. RESULTS: There was no statistically significant difference in correct response between the abstract and PLS groups in the clear finding example (abstract, 59.6 %; PLS, 64.2 %; risk difference 4.6 %; CI -0.2 to 11.3) or the uncertain finding example (42.7 %, 39.3 %, -3.4 %, -10.1 to 3.4). There was no significant difference between the conclusion and no conclusion groups in the example with clear findings (conclusions, 63.3 %; no conclusions, 60.5 %; 2.8 %; -3.9 to 9.5), but there was a significant difference in the example with uncertain findings (44.7 %; 37.3 %; 7.3 %; 0.6 to 14.1, p = 0.03). PLS without conclusions in the uncertain finding review had the lowest proportion of correct responses (32.5 %). Prior knowledge and belief predicted student response to the clear finding review, while years of midwifery education predicted response to the uncertain finding review. CONCLUSIONS: Abstracts with and without conclusions generated similar student responses. PLS with conclusions gave similar results to abstracts with and without conclusions. Removing the conclusions from a PLS with uncertain findings led to more problems with interpretation.

A new classification of spin in systematic reviews and meta-analyses was developed and ranked according to the severity.

OBJECTIVES: We aimed to (1) identify and classify spin (i.e., a description that overstates efficacy and/or understates harm) in systematic reviews and (2) rank spin in abstracts of systematic reviews according to their severity (i.e., the likelihood of distorting readers' interpretation of the results). STUDY DESIGN: First, we used a four-phase consensus process to develop a classification of different types of spin. Second, we ranked the types of spin in abstracts according to their severity using a Q-sort survey with members of the Cochrane Collaboration. RESULTS: We identified 39 types of spin, 28 from the main text and 21 from the abstract; 13 were specific to the systematic review design. Spin was classified into three categories: (1) misleading reporting, (2) misleading interpretation, and (3) inappropriate extrapolation. Spin ranked as the most severe by the 122 people who participated in the survey were (1) recommendations for clinical practice not supported by findings in the conclusion, (2) misleading title, and (3) selective reporting. CONCLUSION: This study allowed for identifying spin that is likely to distort interpretation. Our classification could help authors, editors, and reviewers avoid spin in reports of systematic reviews.

Improving GRADE evidence tables part 3: detailed guidance for explanatory footnotes supports creating and understanding GRADE certainty in the evidence judgments.

BACKGROUND: The Grading of Recommendations Assessment, Development and Evaluation (GRADE) is widely used and reliable and accurate for assessing the certainty in the body of health evidence. The GRADE working group has provided detailed guidance for assessing the certainty in the body of evidence in systematic reviews and health technology assessments (HTAs) and how to grade the strength of health recommendations. However, there is limited advice regarding how to maximize transparency of these judgments, in particular through explanatory footnotes or explanations in Summary of Findings tables and Evidence Profiles (GRADE evidence tables). METHODS: We conducted this study to define the essential attributes of useful explanations and to develop specific guidance for explanations associated with GRADE evidence tables. We used a sample of explanations according to their complexity, type of judgment involved, and appropriateness from a database of published GRADE evidence tables in Cochrane reviews and World Health Organization guidelines. We used an iterative process and group consensus to determine the attributes and develop guidance. RESULTS: Explanations in GRADE evidence tables should be concise, informative, relevant, easy to understand, and accurate. We provide general and domain-specific guidance to assist authors with achieving these desirable attributes in their explanations associated with GRADE evidence tables. CONCLUSIONS: Adhering to the general and GRADE domain-specific guidance should improve the quality of explanations associated with GRADE evidence tables, assist authors of systematic reviews, HTA reports, or guidelines with information that they can use in other parts of their evidence synthesis. This guidance will also support editorial evaluation of evidence syntheses using GRADE and provide a minimum quality standard of judgments across tables.

Improving GRADE evidence tables part 2: a systematic survey of explanatory notes shows more guidance is needed.

OBJECTIVES: The Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group has developed GRADE evidence profiles (EP) and summary of findings (SoF) tables to present evidence summaries in systematic reviews, clinical guidelines, and health technology assessments. Explanatory notes are used to explain choices and judgments in these summaries, for example, on rating of the quality of evidence. STUDY DESIGN AND SETTING: A systematic survey of the explanations in SoF tables in 132 randomly selected Cochrane Intervention reviews and in EPs of 10 guidelines. We analyzed the content of 1,291 explanations using a predefined list of criteria. RESULTS: Most explanations were used to describe or communicate results and to explain downgrading of the quality of evidence, in particular for risk of bias and imprecision. Addressing the source of baseline risk (observational data or control group risk) was often missing. For judgments about downgrading the quality of evidence, the percentage of informative explanations ranged between 41% (imprecision) and 79% (indirectness). CONCLUSION: We found that by and large explanations were informative but detected several areas for improvement (e.g., source of baseline risk and judgments on imprecision). Guidance about explanatory footnotes and comments will be provided in the last article in this series.

Implementing systematic review techniques in chemical risk assessment: Challenges, opportunities and recommendations.

Systematic review (SR) is a rigorous, protocol-driven approach designed to minimise error and bias when summarising the body of research evidence relevant to a specific scientific question. Taking as a comparator the use of SR in synthesising research in healthcare, we argue that SR methods could also pave the way for a "step change" in the transparency, objectivity and communication of chemical risk assessments (CRA) in Europe and elsewhere. We suggest that current controversies around the safety of certain chemicals are partly due to limitations in current CRA procedures which have contributed to ambiguity about the health risks posed by these substances. We present an overview of how SR methods can be applied to the assessment of risks from chemicals, and indicate how challenges in adapting SR methods from healthcare research to the CRA context might be overcome. Regarding the latter, we report the outcomes from a workshop exploring how to increase uptake of SR methods, attended by experts representing a wide range of fields related to chemical toxicology, risk analysis and SR. Priorities which were identified include: the conduct of CRA-focused prototype SRs; the development of a recognised standard of reporting and conduct for SRs in toxicology and CRA; and establishing a network to facilitate research, communication and training in SR methods. We see this paper as a milestone in the creation of a research climate that fosters communication between experts in CRA and SR and facilitates wider uptake of SR methods into CRA.

Educational, supportive and behavioural interventions to improve usage of continuous positive airway pressure machines in adults with obstructive sleep apnoea.

BACKGROUND: Although effective in the treatment of obstructive sleep apnoea (OSA), continuous positive airway pressure (CPAP) is not universally accepted by users. Educational, supportive and behavioural interventions may help people with OSA recognise the need for regular and continued use of CPAP. OBJECTIVES: To assess the effectiveness of strategies that are educational, supportive or behavioural in encouraging people who have been prescribed CPAP to use their machines. SEARCH METHODS: Searches were conducted on the Cochrane Airways Group Specialised Register of trials. Searches are current to 17 January 2013. SELECTION CRITERIA: We included randomised parallel controlled trials that assessed an intervention designed to inform participants about CPAP or OSA, to support them in using CPAP or to modify their behaviour in increasing their use of CPAP machines. Studies of any duration were considered. DATA COLLECTION AND ANALYSIS: Two review authors assessed studies to determine their suitability for inclusion in the review. Data were extracted independently and were entered into Review Manager software for analysis. MAIN RESULTS: Thirty studies (2047 participants) were included. We categorised studies by intervention type: supportive interventions during follow-up, educational interventions and behavioural therapy. Across all three intervention classes, most studies incorporated elements of more than one intervention. For the purposes of this systematic review, we categorised them by the prevailing type of intervention, which we expected would have the greatest impact on the study outcome.Baseline Epworth Sleepiness Scale (ESS) scores indicated that most participants experienced daytime sleepiness, and CPAP was indicated on the basis of sleep disturbance indices. A vast majority of recruited participants had not used CPAP previously. Most of the studies were at an unclear risk of bias overall, although because of the nature of the intervention, blinding of both study personnel and participants was not feasible, and this affected a number of key outcomes. Adverse events were not reported in these studies.Low- to moderate-quality evidence showed that all three types of interventions led to increased machine usage in CPAP-naive participants with moderate to severe OSA syndrome. Compared with usual care, supportive ongoing interventions increased machine usage by about 50 minutes per night (0.82 hours, 95% confidence interval (CI) 0.36 to 1.27, N = 803, 13 studies; low-quality evidence), increased the number of participants who used their machines for longer than four hours per night from 59 to 75 per 100 (odds ratio (OR) 2.06, 95% CI 1.22 to 3.47, N = 268, four studies; low-quality evidence) and reduced the likelihood of study withdrawal (OR 0.65, 95% CI 0.44 to 0.97, N = 903, 12 studies; moderate-quality evidence). With the exception of study withdrawal, considerable variation was evident between the results of individual studies across these outcomes. Evidence of an effect on symptoms and quality of life was statistically imprecise (ESS score -0.60 points, 95% CI -1.81 to 0.62, N = 501, eight studies; very low-quality evidence; Functional Outcomes of Sleep Questionnaire 0.98 units, 95% CI -0.84 to 2.79, N = 70, two studies; low-quality evidence, respectively).Educational interventions increased machine usage by about 35 minutes per night (0.60 hours, 95% CI 0.27 to 0.93, N = 508, seven studies; moderate-quality evidence), increased the number of participants who used their machines for longer than four hours per night from 57 to 70 per 100 (OR 1.80, 95% CI 1.09 to 2.95, N = 285, three studies; low-quality evidence) and reduced the likelihood of withdrawal from the study (OR 0.67, 95% CI 0.45 to 0.98, N = 683, eight studies; low-quality evidence). Participants experienced a small improvement in symptoms, the size of which may not be clinically significant (ESS score -1.17 points, 95% CI -2.07 to -0.26, N = 336, five studies).Behavioural therapy led to substantial improvement in average machine usage of 1.44 hours per night (95% CI 0.43 to 2.45, N = 584, six studies; low-quality evidence) and increased the number of participants who used their machines for longer than four hours per night from 28 to 47 per 100 (OR 2.23, 95% CI 1.45 to 3.45, N = 358, three studies; low-quality evidence) but with high levels of statistical heterogeneity. The estimated lower rate of withdrawal with behavioural interventions was imprecise and did not reach statistical significance (OR 0.85, 95% CI 0.57 to 1.25, N = 609, five studies, very low-quality evidence). AUTHORS' CONCLUSIONS: In CPAP-naive people with severe sleep apnoea, low-quality evidence indicates that supportive interventions that encourage people to continue to use their CPAP machines increase usage compared with usual care. Moderate-quality evidence shows that a short-term educational intervention results in a modest increase in CPAP usage. Low-quality evidence indicates that behavioural therapy leads to a large increase in CPAP machine usage. The impact of improved CPAP usage on daytime sleepiness, quality of life and long-term cardiovascular risks remains unclear. For outcomes reflecting machine usage, we downgraded for risk of bias and inconsistency. An additional limitation for daytime sleepiness and quality of life measures was imprecision. Trials in people who have struggled to persist with treatment are needed, as currently little evidence is available for this population. Optimal timing and duration and long-term effectiveness of interventions remain uncertain. The relationship between improved machine usage and effect on symptoms and quality of life requires further assessment. Studies addressing the choice of interventions that best match individual patient needs and therefore result in the most successful and cost-effective therapy are needed.