Confidence interval of risk difference by different statistical methods and its impact on the study conclusion in antibiotic non-inferiority trials.

BACKGROUND: Numerous statistical methods can be used to calculate the confidence interval (CI) of risk differences. There is consensus in previous literature that the Wald method should be discouraged. We compared five statistical methods for estimating the CI of risk difference in terms of CI width and study conclusion in antibiotic non-inferiority trials. METHODS: In a secondary analysis of a systematic review, we included non-inferiority trials that compared different antibiotic regimens, reported risk differences for the primary outcome, and described the number of successes and/or failures as well as patients in each arm. For each study, we re-calculated the risk difference CI using the Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and skewness-corrected asymptotic score (SCAS) methods. The CIs by different statistical methods were compared in terms of CI width and conclusion on non-inferiority. A wider CI was considered to be more conservative. RESULTS: The analysis included 224 comparisons from 213 studies. The statistical method used to calculate CI was not reported in 134 (59.8%) cases. The median (interquartile range IQR) for CI width by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods was 13.0% (10.8%, 17.4%), 13.3% (10.9%, 18.5%), 13.6% (11.1%, 18.9%), 13.6% (11.1% and 19.0%), and 13.4% (11.1%, 18.9%), respectively. In 216 comparisons that reported a non-inferiority margin, the conclusion on non-inferiority was the same across the five statistical methods in 211 (97.7%) cases. The differences in CI width were more in trials with a sample size of 100 or less in each group and treatment success rate above 90%. Of the 18 trials in this subgroup with a specified non-inferiority margin, non-inferiority was shown in 17 (94.4%), 16 (88.9%), 14 (77.8%), 14 (77.8%), and 15 (83.3%) cases based on CI by Wald, Agresti-Caffo, Newcombe, Miettinen-Nurminen, and SCAS methods, respectively. CONCLUSIONS: The statistical method used to calculate CI was not reported in the majority of antibiotic non-inferiority trials. Different statistical methods for CI resulted in different conclusions on non-inferiority in 2.3% cases. The differences in CI widths were highest in trials with a sample size of 100 or less in each group and a treatment success rate above 90%. TRIAL REGISTRATION: PROSPERO CRD42020165040 . April 28, 2020.

Intention-to-treat analysis may be more conservative than per protocol analysis in antibiotic non-inferiority trials: a systematic review.

BACKGROUND: In non-inferiority trials, there is a concern that intention-to-treat (ITT) analysis, by including participants who did not receive the planned interventions, may bias towards making the treatment and control arms look similar and lead to mistaken claims of non-inferiority. In contrast, per protocol (PP) analysis is viewed as less likely to make this mistake and therefore preferable in non-inferiority trials. In a systematic review of antibiotic non-inferiority trials, we compared ITT and PP analyses to determine which analysis was more conservative. METHODS: In a secondary analysis of a systematic review, we included non-inferiority trials that compared different antibiotic regimens, used absolute risk reduction (ARR) as the main outcome and reported both ITT and PP analyses. All estimates and confidence intervals (CIs) were oriented so that a negative ARR favored the control arm, and a positive ARR favored the treatment arm. We compared ITT to PP analyses results. The more conservative analysis between ITT and PP analyses was defined as the one having a more negative lower CI limit. RESULTS: The analysis included 164 comparisons from 154 studies. In terms of the ARR, ITT analysis yielded the more conservative point estimate and lower CI limit in 83 (50.6%) and 92 (56.1%) comparisons respectively. The lower CI limits in ITT analysis favored the control arm more than in PP analysis (median of - 7.5% vs. -6.9%, p = 0.0402). CIs were slightly wider in ITT analyses than in PP analyses (median of 13.3% vs. 12.4%, p < 0.0001). The median success rate was 89% (interquartile range IQR 82 to 93%) in the PP population and 44% (IQR 23 to 60%) in the patients who were included in the ITT population but excluded from the PP population (p < 0.0001). CONCLUSIONS: Contrary to common belief, ITT analysis was more conservative than PP analysis in the majority of antibiotic non-inferiority trials. The lower treatment success rate in the ITT analysis led to a larger variance and wider CI, resulting in a more conservative lower CI limit. ITT analysis should be mandatory and considered as either the primary or co-primary analysis for non-inferiority trials. TRIAL REGISTRATION: PROSPERO registration number CRD42020165040 .

Methodological and Reporting Quality of Noninferiority Randomized Controlled Trials Comparing Antibiotic Therapies: A Systematic Review.

BACKGROUND: Antibiotic noninferiority randomized controlled trials (RCTs) are used for approval of new antibiotics and making changes to antibiotic prescribing in clinical practice. We conducted a systematic review to assess the methodological and reporting quality of antibiotic noninferiority RCTs. METHODS: We searched MEDLINE, Embase, the Cochrane Database of Systematic Reviews, and the Food and Drug Administration drug database from inception until November 22, 2019, for noninferiority RCTs comparing different systemic antibiotic therapies. Comparisons between antibiotic types, doses, administration routes, or durations were included. Methodological and reporting quality indicators were based on the Consolidated Standards of Reporting Trials reporting guidelines. Two independent reviewers extracted the data. RESULTS: The systematic review included 227 studies. Of these, 135 (59.5%) studies were supported by pharmaceutical industry. Only 83 (36.6%) studies provided a justification for the noninferiority margin. Reporting of both intention-to-treat (ITT) and per-protocol (PP) analyses were done in 165 (72.7%) studies. The conclusion was misleading in 34 (15.0%) studies. The studies funded by pharmaceutical industry were less likely to be stopped early because of logistical reasons (3.0% vs 19.1%; odds ratio [OR] = 0.13; 95% confidence interval [CI], .04-.37) and to show inconclusive results (11.1% vs 42.9%; OR = 0.17; 95% CI, .08-.33). The quality of studies decreased over time with respect to blinding, early stopping, reporting of ITT with PP analysis, and having misleading conclusions. CONCLUSIONS: There is room for improvement in the methodology and reporting of antibiotic noninferiority trials. Quality can be improved across the entire spectrum from investigators, funding agencies, as well as during the peer-review process.There is room for improvement in the methodology and reporting of antibiotic noninferiority trials including justification of noninferiority margin, reporting of intention-to-treat analysis with per-protocol analysis, and having conclusions that are concordant with study results.Clinical Trials Registration PROSPERO registration number CRD42020165040.

Novel Antibiotics May Be Noninferior but Are They Becoming Less Effective?: a Systematic Review.

Novel antibiotics approved by noninferiority trials may become less effective over time in two scenarios: (i) the treatment effect in studies of novel antibiotics may be consistently worse than studies of older antibiotics; (ii) when a decreasingly effective control arm is used in a series of noninferiority trials. Our systematic review of 175 noninferiority antibiotic trials found these scenarios to be rare.

Patient engagement in hospital health service planning and improvement: a scoping review.

OBJECTIVES: Patient engagement (PE) improves patient, organisation and health system outcomes, but most research is based on primary care. The primary purpose of this study was to describe the characteristics of published empirical research that evaluated PE in hospital health service improvement. DESIGN: Scoping review. METHODS: Five databases were searched from 2006 to September 2016. English language studies that evaluated patient or provider beliefs, participation in PE, influencing factors or impact were eligible. Screening and data extraction were done in triplicate. PE characteristics, influencing factors and impact were extracted and summarised. RESULTS: From a total of 3939 search results, 227 studies emerged as potentially relevant; of these, 217 were not eligible, and 10 studies were included in the review. None evaluated behavioural interventions to promote or support PE. While most studies examined involvement in standing committees or projects, patient input and influence on decisions were minimal. Lack of skill and negative beliefs among providers were PE barriers. PE facilitators included careful selection and joint training of patients and providers, formalising patient roles, informal interaction to build trust, involving patients early in projects, small team size, frequent meetings, active solicitation of patient input in meetings and debriefing after meetings. Asking patients to provide insight into problems rather than solutions and deploying provider champions may enhance patient influence on hospital services. CONCLUSIONS: Given the important role of PE in improving hospital services and the paucity of research on this topic, future research should develop and evaluate behavioural interventions for PE directed at patients and providers informed by the PE barriers and facilitators identified here. Future studies should also assess the impact on various individual and organisational outcomes.

Interventions are needed to support patient-provider decision-making for DCIS: a scoping review.

PURPOSE: Prognostic and treatment uncertainty make ductal carcinoma in situ (DCIS) complex to manage. The purpose of this study was to describe research that evaluated DCIS communication experiences, needs and interventions among DCIS patients or physicians. METHODS: MEDLINE, EMBASE, CINAHL and The Cochrane Library were searched from inception to February 2017. English language studies that evaluated patient or physician DCIS needs, experiences or behavioural interventions were eligible. Screening and data extraction were done in duplicate. Summary statistics were used to describe study characteristics and findings. RESULTS: A total of 51 studies published from 1997 to 2016 were eligible for review, with a peak of 8 articles in year 2010. Women with DCIS lacked knowledge about the condition and its prognosis, although care partners were more informed, desired more information and experienced decisional conflict. Many chose mastectomy or prophylactic mastectomy, often based on physician's recommendation. Following treatment, women had anxiety and depression, often at levels similar to those with invasive breast cancer. Disparities were identified by education level, socioeconomic status, ethnicity and literacy. Physicians said that they had difficulty explaining DCIS and many referred to DCIS as cancer. Despite the challenges reported by patients and physicians, only two studies developed interventions designed to improve patient-physician discussion and decision-making. CONCLUSIONS: As most women with DCIS undergo extensive treatment, and many experience treatment-related complications, the paucity of research on PE to improve and support informed decision-making for DCIS is profound. Research is needed to improve patient and provider discussions and decision-making for DCIS management.

Number and type of guideline implementation tools varies by guideline, clinical condition, country of origin, and type of developer organization: content analysis of guidelines.

BACKGROUND: Guideline implementation tools (GI tools) can improve clinician behavior and patient outcomes. Analyses of guidelines published before 2010 found that many did not offer GI tools. Since 2010 standards, frameworks and instructions for GI tools have emerged. This study analyzed the number and types of GI tools offered by guidelines published in 2010 or later. METHODS: Content analysis and a published GI tool framework were used to categorize GI tools by condition, country, and type of organization. English-language guidelines on arthritis, asthma, colorectal cancer, depression, diabetes, heart failure, and stroke management were identified in the National Guideline Clearinghouse. Screening and data extraction were in triplicate. Findings were reported with summary statistics. RESULTS: Eighty-five (67.5%) of 126 eligible guidelines published between 2010 and 2017 offered one or more of a total of 464 GI tools. The mean number of GI tools per guideline was 5.5 (median 4.0, range 1 to 28) and increased over time. The majority of GI tools were for clinicians (239, 51.5%), few were for patients (113, 24.4%), and fewer still were to support implementation (66, 14.3%) or evaluation (46, 9.9%). Most clinician GI tools were guideline summaries (116, 48.5%), and most patient GI tools were condition-specific information (92, 81.4%). Government agencies (patient 23.5%, clinician 28.9%, implementation 24.1%, evaluation 23.5%) and developers in the UK (patient 18.5%, clinician 25.2%, implementation 27.2%, evaluation 29.1%) were more likely to generate guidelines that offered all four types of GI tools. Professional societies were more likely to generate guidelines that included clinician GI tools. CONCLUSIONS: Many guidelines do not include any GI tools, or a variety of GI tools for different stakeholders that may be more likely to prompt guideline uptake (point-of-care forms or checklists for clinicians, decision-making or self-management tools for patients, implementation and evaluation tools for managers and policy-makers). While this may vary by country and type of organization, and suggests that developers could improve the range of GI tools they develop, further research is needed to identify determinants and potential solutions. Research is also needed to examine the cost-effectiveness of various types of GI tools so that developers know where to direct their efforts and scarce resources.

Use of theory to plan or evaluate guideline implementation among physicians: a scoping review.

BACKGROUND: Guidelines support health care decision-making and high quality care and outcomes. However, their implementation is sub-optimal. Theory-informed, tailored implementation is associated with guideline use. Few guideline implementation studies published up to 1998 employed theory. This study aimed to describe if and how theory is now used to plan or evaluate guideline implementation among physicians. METHODS: A scoping review was conducted. MEDLINE, EMBASE, and The Cochrane Library were searched from 2006 to April 2016. English language studies that planned or evaluated guideline implementation targeted to physicians based on explicitly named theory were eligible. Screening and data extraction were done in duplicate. Study characteristics and details about theory use were analyzed. RESULTS: A total of 1244 published reports were identified, 891 were unique, and 716 were excluded based on title and abstract. Among 175 full-text articles, 89 planned or evaluated guideline implementation targeted to physicians; 42 (47.2%) were based on theory and included. The number of studies using theory increased yearly and represented a wide array of countries, guideline topics and types of physicians. The Theory of Planned Behavior (38.1%) and the Theoretical Domains Framework (23.8%) were used most frequently. Many studies rationalized choice of theory (83.3%), most often by stating that the theory described implementation or its determinants, but most failed to explicitly link barriers with theoretical constructs. The majority of studies used theory to inform surveys or interviews that identified barriers of guideline use as a preliminary step in implementation planning (76.2%). All studies that evaluated interventions reported positive impact on reported physician or patient outcomes. CONCLUSIONS: While the use of theory to design or evaluate interventions appears to be increasing over time, this review found that one half of guideline implementation studies were based on theory and many of those provided scant details about how theory was used. This limits interpretation and replication of those interventions, and seems to result in multifaceted interventions, which may not be feasible outside of scientific investigation. Further research is needed to better understand how to employ theory in guideline implementation planning or evaluation.

The quantity and quality of complementary and alternative medicine clinical practice guidelines on herbal medicines, acupuncture and spinal manipulation: systematic review and assessment using AGREE II.

BACKGROUND: Complementary and alternative medicine (CAM) use is often not disclosed by patients, and can be unfamiliar to health care professionals. This may lead to underuse of beneficial CAM therapies, and overuse of other CAM therapies with little proven benefit or known contraindications. No prior research has thoroughly evaluated the credibility of knowledge-based resources. The purpose of this research was to assess the quantity and quality of CAM guidelines. METHODS: A systematic review was conducted to identify CAM guidelines. MEDLINE, EMBASE and CINAHL were searched in January 2016 from 2003 to 2015. The National Guideline Clearinghouse, National Center for Complementary and Integrative Health web site, and two CAM journals were also searched. Eligible guidelines published in English language by non-profit agencies on herbal medicine, acupuncture, or spinal manipulation for adults with any condition were assessed with the Appraisal of Guidelines, Research and Evaluation II (AGREE II) instrument. RESULTS: From 3,126 unique search results, 17 guidelines (two herbal medicine, three acupuncture, four spinal manipulation, eight mixed CAM therapies) published in 2003 or later and relevant to several clinical conditions were eligible. Scaled domain percentages from highest to lowest were clarity of presentation (85.3 %), scope and purpose (83.3 %), rigour of development (61.2 %), editorial independence (60.1 %), stakeholder involvement (52.0 %) and applicability (20.7 %). Quality varied within and across guidelines. None of the 17 guidelines were recommended by both appraisers; 14 were recommended as Yes or Yes with modifications. CONCLUSIONS: Guidelines that scored well could be used by patients and health care professionals as the basis for discussion about the use of these CAM therapies. In future updates, guidelines that achieved variable or lower scores could be improved according to specifications in the AGREE II instrument, and with insight from a large number of resources that are available to support guideline development and implementation. Future research should identify CAM therapies other than those reviewed here for which guidelines are available. Research is also needed on the safety and effectiveness of CAM therapies.