Harms reporting in randomized controlled trials underpinning the American Academy of Orthopaedic Surgeons clinical practice guidelines for glenohumeral osteoarthritis.

BACKGROUND: Glenohumeral osteoarthritis is one of the most common causes of shoulder pain. As such, the American Academy of Orthopaedic Surgeons (AAOS) has developed clinical practice guidelines (CPGs) to address the management of glenohumeral osteoarthritis. These CPG recommendations stem from the findings of randomized controlled trials (RCTs), which have been shown to influence clinical decision making and health policy. Therefore, it is essential that trial outcomes, including harms data (ie, adverse events), are adequately reported. We intend to evaluate the reporting quality of harms-related data in orthopedic literature specifically relating to AAOS CPG recommendations on the management of glenohumeral osteoarthritis. METHODS: We adhered to the Preferred Reporting Items for Systematic Reviews (PRISMA) as well as guidance for reporting meta-research. The AAOS CPGs for glenohumeral osteoarthritis were obtained from orthoguidelines.org, and 2 authors independently screened the guidelines for the RCTs referenced. A total of 14 studies were identified. Data were extracted from the 14 included studies independently by the same 2 authors. Adherence to the Consolidated Standards of Reporting Trials (CONSORT) Extension for Harms Checklist was assessed using an 18-item scoring chart, with 1 point being awarded for meeting a checklist item and 0 points being awarded for not meeting a checklist item. Descriptive statistics, such as frequencies, percentages, and 95% confidence intervals were used to summarize RCT adherence to the CONSORT checklist. RESULTS: The average score among the studies included was 7.36/18 items (39% adherence). No study adhered to all criteria, with the highest-performing study meeting 11 of 18 items (58%) and the lowest meeting 3 of 18 items (16%). A positive correlation between checklist score and year of publication was observed, with studies published more recently receiving a higher score on the CONSORT checklist (P < .05). Studies that disclosed funding information received a higher score than those that did not (P < .05), but there was no significant difference when the different funding sources were compared. Finally, double-blinded studies scored higher on the checklist than those with lower levels of blinding (single or no blinding, P < .05). CONCLUSION: Adverse events are poorly reported amongst RCTs cited as supporting evidence for AAOS Management of Glenohumeral Osteoarthritis CPGs, evidenced by a CONSORT checklist compliance rate of only 41% in this study. We recommend the development of an updated checklist with information that makes it easier for authors to recognize, evaluate, and report on harms data. Additionally, we encourage authors to include information about adverse events or negative outcomes in the abstract.

Harms Reporting in Randomized Controlled Trials Underpinning the American Academy of Orthopaedic Surgeons Clinical Practice Guidelines for Distal Radius Fractures.

PURPOSE: The purpose of this study was to measure the harms-related reporting among randomized controlled trials (RCTs) cited as supporting evidence for the American Academy of Orthopaedic Surgeons clinical practice guidelines regarding the management of distal radius fractures. METHODS: We adhered to the guidance for reporting metaresearch and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines throughout the course of this investigation. We used the American Academy of Orthopaedic Surgeons clinical practice guidelines for distal radius fractures available on Orthoguidelines.org. A linear regression analysis was conducted to model the relationship between the year of publication and the total Consolidated Standards of Reporting Trials percentage adherence over time. RESULTS: Thirty-five RCTs were included in the final sample. The average number of Consolidated Standards of Reporting Trials Extension for Harms items adequately reported across all included RCTs was 9.2 (9.2/18, 50.9%). None of the included trials adequately reported all 18 items. Ten items had a compliance of more than 50% (10/18, 55.6%), 4 items had a compliance of 20%-50% (4/18, 22.2%), and 4 items had a compliance of less than 20% (4/18, 22.2%). The results of the linear regression model showed no significant improvement in Consolidated Standards of Reporting Trials Harms reporting over time. CONCLUSIONS: Adverse events are incompletely reported among RCTs cited as supporting evidence for American Academy of Orthopaedic Surgeons clinical practice guidelines for the management of distal radius fractures. CLINICAL RELEVANCE: Given our findings, specific attention should be paid to improving the standardization of the classification of adverse events to facilitate ease in the reporting process.

An analysis of harms reporting in systematic reviews regarding ketorolac for management of perioperative pain.

BACKGROUND: Owing to the frequent perioperative use of ketorolac tromethamine and its ability to minimise postoperative opioid requirements, it is important to continually reassess harms associated with its use. Our primary objective was to investigate the extent of harms reporting in systematic reviews (SRs) on ketorolac for perioperative pain. METHODS: In May 2022, we conducted a search of major databases, MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews to identify eligible SRs on ketorolac for perioperative pain. Screening and data extraction were performed in masked, duplicate fashion. A MeaSurement Tool to Assess systematic Reviews-2 (AMSTAR-2) was used to appraise the methodological quality of included SRs. Corrected covered area (CCA) was calculated to determine overlap of primary studies between SR dyads. RESULTS: A total of 28 SRs evaluating 630 primary studies met the inclusion criteria. Seven SRs (7/28, 25%) reported no harms and 17 SRs (17/28, 60.7%) reported ≤50% of harms items. A significant association was found between completeness of harms reporting and whether harms were specified as a primary outcome (P<0.001). No other associations were statistically significant. Regarding methodological quality, 22 SRs were appraised as 'critically low' (22/28, 78.6%), 5 as 'low' (5/28, 17.9%), and 1 as 'high' (1/28, 3.6%). One SR dyad had a CCA >50% but neither reported harms. CONCLUSIONS: The extent of harms reporting in systematic reviews was inadequate. Given the importance that systematic reviews have on guiding perioperative decision-making, it is essential to improve the completeness of harms reporting.

Harms-related data are poorly reported among randomized controlled trials underpinning the American Academy of Orthopaedic Surgeons clinical practice guideline recommendations for rotator cuff injuries.

BACKGROUND: Results produced from randomized controlled trials (RCTs) help guide clinical decision making and health policy. Therefore, it is essential that RCT outcomes- including harms (eg, adverse events)-are adequately reported such that clinicians, patients, and policy makers are equipped with all necessary information to complete risk-benefit assessment of the RCT's intervention. Here, we evaluated the quality of reporting of harms (eg, adverse events) in RCTs cited as supporting evidence for recommendations in the American Academy of Orthopaedic Surgeons (AAOS) Management of Rotator Cuff Injuries clinical practice guidelines (CPGs) using the Consolidated Standards of Reporting Trials (CONSORT) Extension for Harms Checklist. METHODS: To quantify adherence to CONSORT Extension for Harms items, each RCT was screened for pertinent information satisfying each checklist item. Screening of CPG reference sections for RCTs underpinning CPG recommendations, as well as data extraction from each of the included RCTs, was performed in a blind and duplicate manner. Descriptive statistics-including frequencies, percentages, and 95% confidence intervals-were used to summarize overall percent adherence to checklist items. A linear regression model assessed the relationship of CONSORT Harms reporting over time. RESULTS: Ninety-nine RCTs were included in our final sample. Fifty-seven RCTs (of 99; 57.6%) were conducted at a single center. Common funding sources included private (nonindustry) (17/99; 17.2%), private (industry) (8/99; 8.1%), and public (7/99; 7.1%) sources. Sample size for each trial most often consisted of <50 participants (29/99; 29.3%) or 51-100 participants (50/99; 50.5%). The average number of CONSORT Extension for Harms items adequately reported across all included RCTs was 5.7 (of 18; 31.7%). None of the included trials reported all 18 items. Twenty-six RCTs (of 99; 26.3%) adequately reported ≥50% of eligible checklist items. Fifty-nine RCTs (of 99; 59.6%) adequately reported ≤33% of eligible checklist items. Items with ≥50% adherence included item 2, item 7a, and item 8a. Items with ≤20% adherence included item 3b, item 4d, and item 5. Results from our linear regression demonstrated a slight, yet nonsignificant, improvement in adherence to the Harms Extension over time (R2 = 0.009; P = .407). CONCLUSIONS: Our results illustrate the poor state of harms reporting within RCTs cited as supporting evidence for the AAOS Management of Rotator Cuff Injuries CPG. Efforts to address these gaps in reporting are warranted, as complete knowledge of potential harms is critical to patients, clinicians, and health policy makers when determining best practice decisions in orthopedic surgery.

Inadequate Reporting of Complications in Randomized Controlled Trials Cited as Supporting Evidence Underpinning AAOS CPG Recommendations for Hip and Knee Osteoarthritis: Application of the CONSORT Harms Checklist.

BACKGROUND: Randomized controlled trials (RCTs) have been shown to influence clinical decision-making and health policy. Therefore, it is essential that trial outcomes-including harms-are completely reported. METHODS: We included all RCTs cited as supporting evidence for the American Academy of Orthopaedic Surgeons Surgical Management of Osteoarthritis of the Knee, Osteoarthritis of the Knee, and Osteoarthritis of theHip Clinical Practice Guideline recommendations. Manuscripts were analyzed for compliance with the Consolidated Standards of Reporting Trials (CONSORT) Extension for Harms items. We determined the Extension for Harms' influence on harms reporting by comparing RCTs published before and after the extension's release. RESULTS: One hundred and seventy-three RCTs were included, of which 81 (47%) adequately reported ≥50% of the checklist and 75 (43%) reported ≤33% of the checklist items. The mean number of checklist items reported was 8 items (of 18; 45%). Our interrupted time-series analysis suggests the implementation of the CONSORT Extension for Harms did not have a statistically significant effect on the completeness of harms reporting (P = .35; 95% Confidence interval = -0.0041 to 0.0014). CONCLUSION: Harms-related data are poorly reported within RCTs cited as supporting evidence for the American Academy of Orthopaedic Surgeons management for hip and knee OA Clinical Practice Guideline. Our time series analysis illustrates the failure of the CONSORT Extension for Harms on improving the reporting of harms-related data. Future efforts to improve the quality of harms reporting is crucial for patients, clinicians, and policy makers to perform thorough risk-benefit appraisals as RCT results directly influence clinical decision-making in orthopaedic surgery.

Transparency Matters: Assessing Harms Reporting Compliance in Randomized Controlled Trials Underpinning the AAOS Clinical Practice Guideline for Anterior Cruciate Ligament Injuries.

BACKGROUND: The American Academy of Orthopaedic Surgeons (AAOS) has developed a clinical practice guideline (CPG) for management of anterior cruciate ligament (ACL) injuries. Primary studies such as randomized controlled trials (RCTs) are cited as evidence for the guidelines. Given the influence that these trials have on patient care, adherence to standardized protocols for conducting and reporting RCTs is essential. PURPOSE: To evaluate the CONSORT (Consolidated Standards of Reporting Trials) Extension for Harms-related reporting of RCTs cited as supporting evidence for the AAOS CPG on the management of ACL tears. STUDY DESIGN: Cross-sectional study. METHODS: The reference section of the AAOS guideline for ACL tears was first screened for RCTs cited in the CPG. Next, each RCT was evaluated for adherence to the CONSORT Extension for Harms checklist. Both identification of RCTs and assessment of adherence were performed in a masked and duplicate process. Descriptive statistics were used to summarize adherence to CONSORT Extension for Harms items. A Pearson correlation test was conducted to assess the relationship between the year of publication and adherence to CONSORT harms reporting. RESULTS: The sample included 113 RCTs, of which 16 (14.2%) were published before the CONSORT Extension for Harms was implemented in 2004. Sample sizes ranged from 24 to 4564 participants, with a mean of 228. None of the included RCTs included all 18 items in the CONSORT Extension for Harms checklist. The mean number of checklist items reported was 4 (of 18; 22.2%). A moderate, positive, and statistically significant correlation was found between the RCT publication year and the adherence with reporting of the CONSORT Extension for Harms (t111 = 3.54; P < .001) (r = 0.32; 95% CI, 0.14-0.47). CONCLUSION: Harms were infrequently reported in RCTs cited as supporting evidence in the AAOS CPG for the management of ACL tears. One encouraging finding was the positive correlation between the year when RCTs were published and how well they adhered to reporting harms. Efforts to improve adverse event reporting are warranted, as RCTs are commonly used to make clinical decisions in orthopaedic surgery.

Inadequate harms reporting among randomized controlled trials cited as supporting evidence in the AAOS management of hip fractures in older adults clinical practice guideline recommendations.

BACKGROUND: Recommendations within clinical practice guidelines (CPGs) are heavily influenced by results from randomized controlled trials (RCTs). Therefore, it is imperative that all RCT outcomes are reported thoroughly to ensure CPGs are created using accurate information. Here, we evaluate the quality of harms reporting using the CONSORT Extension for Harms in RCTs underpinning recommendations in the American Academy of Orthopedic Surgeons (AAOS) Management of Hip Fractures in Older Adults CPG. METHODS: Each RCT cited as evidence for recommendations in the AAOS Management of Hip Fractures in Older Adults CPG was evaluated using the CONSORT Extension for Harms to determine the quality of harms reporting. Descriptive statistics (frequencies, percentages, 95 % confidence intervals) were used to summarize adherence to CONSORT Harms items. A linear regression model was used to evaluate the CONSORT Harms influence on the quality of reporting over time. RESULTS: Among the 156 RCTs identified, there were a total of 31,848 participants. Most RCTs were conducted at a single center (137; 87.8 %) and in a single-blind manner (130; 83.3 %). Fifty-four (34.6 %) RCTs did not provide funding statements. Trials adequately reported an average of 6.65 out of 18 CONSORT Extension for Harms items (37.0 %). One RCT adequately reported all items, while five reported zero items. Forty-seven RCTs (30.1 %) reported ≥ 50 % of items and 73 (46.8 %) reported ≤ 33.3 % of items. The linear regression model demonstrated no significant increase in mean adherence over time (adjusted R2 = -0.006; p = 0.563). CONCLUSION: Our results highlight inadequate harms reporting among RCTs in the AAOS Management of Hip Fractures in Older Patients CPG. While the CONSORT Harms Extension was intended to enhance reporting, the linear regression model did not demonstrate significant improvements over time.

A cross-sectional analysis of harms reporting in systematic reviews evaluating laminectomy.

Background Context: Laminectomy is a common vertebral decompression procedure that has multiple potential adverse events which are not always reported in SRs. Purpose: To evaluate the completeness of harms reporting in systematic reviews (SRs) on laminectomy. Study Design: Cross-sectional analysis. Methods: Eligible studies were SRs that evaluated laminectomy for any indication. MEDLINE (PubMed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews were searched in May 2022 to locate studies for inclusion. Screening and data extraction on harms reporting and study characteristics were performed in duplicate. AMSTAR-2 was used to evaluate the methodological quality of included SRs. Corrected covered area (CCA) was calculated for SR pairs. Results: We included 26 SRs comprising 426 primary studies. Most SRs studied laminectomy for spinal stenosis, declared harms as a secondary outcome, and lacked or did not mention funding. Two SRs completely omitted harms, 9 had between 0% and 50.0% completion of harms items, and 15 had more than 50.1% completion. AMSTAR-2 graded 25 SRs (25/26, 96.2%) as 'critically low' and 1 SR (1/26, 3.8%) as 'low'. We found a statistically significant association between completeness of harms reporting and outcome specification. No other associations were statistically significant. Three SR pairs had CCAs >50% and were compared for unique and shared harms. Conclusions: The completeness of harms reporting in SRs was inadequate. Because SRs often serve as tools for constructing clinical practice guidelines and clinical decision making, improvements must be made to enhance and refine harms reporting.

Harms Reporting in Systematic Reviews of the Microvascular Free Flap in Head and Neck Reconstruction.

OBJECTIVE: To evaluate harms reporting in systematic reviews (SRs) of microvascular free flap (MFF) in head and neck reconstruction. DATA SOURCES: This cross-sectional analysis included searches from the following major databases from 2012 to June 1, 2022: MEDLINE (Pubmed and Ovid), Embase, Epistemonikos, and the Cochrane Database of Systematic Reviews. REVIEW METHODS: In a masked duplicate manner, screening was performed using Rayyan, and data were extracted using a pilot-tested Google form. A MeaSurement Tool to Assess systematic Reviews-2 (AMSTAR-2) was used to appraise the methodological quality of reviews and the corrected covered area was calculated to detect primary study overlap across all reviews. Reviews were then grouped in pairs of 2, called dyads, and the corrected covered area was calculated again for each individual dyad. Dyads with high overlap (≥50%) were further investigated for the accuracy of harms reporting. RESULTS: Our initial search yielded 268 records, with 50 SRs meeting the inclusion criteria. A total of 46 (92%) of the included reviews demonstrated 50% or more adherence to the items assessed in our harms checklist. Our corrected covered area tool revealed 0.6% primary study overlap across all reviews, and 1 dyad with high overlap (≥50%). No statistically significant relationship was observed between the completeness of harms reporting and reviews listing harms as a primary outcome, reviews reporting adherence to Preferred Reporting Items of Systematic Reviews and Meta-Analyses, or a review's AMSTAR rating. CONCLUSION: This study identifies how harms reporting in SRs of MFF reconstruction of the head and neck can be improved and provides suggestions with the potential to mitigate the paucity in current literature.

Quality of adverse event reporting in phase III randomized controlled trials of breast and colorectal cancer: A systematic review.

BACKGROUND: Clinical trial reports often emphasize efficacy over harms, leading to misinterpretation of the risk-to-benefit ratio of new therapies. Clear and sufficiently detailed reporting of methods and results is especially important in the abstracts of trial reports, as readers often base their assessment of a trial on such information. In this study, we evaluated the quality of adverse event (AE) reporting and abstract quality in phase III randomized controlled trials (RCTs) of systemic therapies in breast and colorectal cancer. METHODS: Medline, EMBASE, Cochrane Database of RCTs, and Cochrane Database of Systematic Reviews were searched from November 2005 to September 2018. Phase III RCTs evaluating systemic therapies in breast or colorectal cancer were included. Each article was independently reviewed by two investigators using a standardized data extraction form based on guidelines developed by the Consolidated Standards of Reporting Trials (CONSORT) group. Descriptive statistics, bivariate analysis, and multivariable linear regression were used to analyze data. All statistical tests were two-sided. RESULTS: Of 166 RCTs identified, 99.4% reported harms in the manuscript body, and 59.6% reported harms in the abstract. Reporting was restricted to severe harms in 15.6% of RCTs. Statistical comparison of AE rates went unreported in 59.0% of studies. Information regarding AEs leading to dose reductions, treatment discontinuations, or study withdrawals went unreported in 59.3%, 18.7%, and 86.8% of studies, respectively. Recently published RCTs (P = .009) and those sponsored at least partially by for-profit companies (P = .003) had higher abstract quality scores. CONCLUSIONS: Breast and colorectal cancer phase III RCTs inadequately report CONSORT-compliant AE data. Improved guideline adherence and abstract reporting is required to properly weigh benefits and harms of new oncologic therapies. SYSTEMATIC REVIEW REGISTRATION NUMBER: CRD42019140673.

Reporting of adverse events and statistical details of efficacy estimates in randomized clinical trials of pain in temporomandibular disorders: Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks systematic review.

BACKGROUND: Statistical methods and adverse events (that is, harms) data affect the accuracy of conclusions about the risk-to-benefit ratio of treatments for temporomandibular disorders (TMDs). The authors reviewed the quality of reporting in TMD clinical trials to highlight practices that are in need of improvement. TYPES OF STUDIES REVIEWED: The authors included articles published between 1969 and May 31, 2013, in which the investigators reported randomized clinical trials of TMD treatments with pain as a principal outcome variable. Investigators in trials of nonpharmacologic and noninvasive treatments were required to at least mask the participants and assessors; all others were required to be double masked. RESULTS: Ninety articles qualified for this review: 39 published between 1971 and 2005 (older articles) and 51 published between 2006 and 2013 (newer articles). Specification of primary outcome analyses, methods to accommodate missing data, and adverse event collection methods and rates were generally poor. In some cases, there was apparent improvement from the older to the newer cohort; however, reporting of these methodological details remained inadequate even in the newer articles. PRACTICAL IMPLICATIONS: This review is designed to alert authors, reviewers, editors, and readers of TMD clinical trials to these issues and improve reporting quality in the future.

Adverse event reporting in nonpharmacologic, noninterventional pain clinical trials: ACTTION systematic review.

Assessment of treatment safety is 1 of the primary goals of clinical trials. Organizations and working groups have created reporting guidelines for adverse events (AEs). Previous research examining AE reporting for pharmacologic clinical trials of analgesics in major pain journals found many reporting inadequacies, suggesting that analgesic trials are not adhering to existing AE reporting guidelines. The present systematic review documented AE reporting in 3 main pain journals for nonpharmacologic, noninterventional (NP/NI) trials examining pain treatments. To broaden our pool of nonpharmacologic trials, we also included trials examining acupuncture, leech therapy, and noninvasive stimulation techniques (eg, transcutaneous electrical nerve stimulation). We documented AE reporting at 2 levels of specificity using coding manuals based on the Consolidated Standards of Reporting Trials (CONSORT) harms reporting standards and Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks (ACTTION) AE reporting checklist. We identified a number of inadequacies in AE reporting across the 3 journals. For example, using the ACTTION coding manual, we found that less than one-half of the trials reported specific AE assessment methods; approximately one-third of the trials reported withdrawals due to AEs for each study arm; and about one-fourth of the trials reported all specific AEs. We also examined differences in AE reporting across several trial characteristics, finding that AE reporting was generally more detailed in trials with patients versus those using healthy volunteers undergoing experimentally evoked pain. These results suggest that investigators conducting and reporting NP/NI clinical trials are not adequately describing the assessment and occurrence of AEs.