Reporting of Harms in Randomized Controlled Trials Published in Urology Journals: An Updated Analysis.

PURPOSE: Harms are often overlooked, but important, outcomes of randomized controlled trial reporting. Our goal was to determine if harms reporting has improved in high-impact urology journals. MATERIALS AND METHODS: Randomized controlled trials published in The Journal of Urology®, Urology, European Urology, and BJU International in 2012 and 2020 were analyzed. Each randomized controlled trial was evaluated by 2 authors in a masked-duplicate fashion to evaluate for adherence to harms reporting guidelines recommended by the Consolidated Standards of Reporting Trials (CONSORT) group. RESULTS: One hundred and thirty-two published studies met inclusion criteria. Between 2012 and 2020, there was a statistically significant increase in the median number of harms criteria reported between 2012 and 2020 (5.3 vs 7.2; P = .01). Methods criteria demonstrating the greatest improvements included item #3 "which harms were assessed," item #4a "when harm information was collected," and item #4b "methods to attribute harm to intervention." Results sections with the most improvement in reporting include item #6 "reasons for patient withdrawal," item #8a "effect size for harms," and item #8b "stratified serious + minor harms." CONCLUSIONS: Reporting of adverse events in randomized trials published in several top urology journals has demonstrated marked improvement. Studies published in 2020 reported approximately 70% of CONSORT-Harms criteria-an increase of nearly 40% since 2004. While these improvements mark significant change, deficits remain present and should be addressed to provide clinicians with the most complete perspective possible.

A review of reproducible and transparent research practices in urology publications from 2014 to2018.

BACKGROUND: Reproducibility is essential for the integrity of scientific research. Reproducibility is measured by the ability of different investigators to replicate the outcomes of an original publication using the same materials and procedures. Unfortunately, reproducibility is not currently a standard being met by most scientific research. METHODS: For this review, we sampled 300 publications in the field of urology to assess for 14 indicators of reproducibility including material availability, raw data availability, analysis script availability, pre-registration information, links to protocols, and if the publication was available free to the public. Publications were also assessed for statements about conflicts of interest and funding sources. RESULTS: Of the 300 sample publications, 171 contained empirical data available for analysis of reproducibility. Of the 171 articles with empirical data to analyze, 0.58% provided links to protocols, 4.09% provided access to raw data, 3.09% provided access to materials, and 4.68% were pre-registered. None of the studies provided analysis scripts. Our review is cross-sectional in nature, including only PubMed indexed journals-published in English-and within a finite time period. Thus, our results should be interpreted in light of these considerations. CONCLUSION: Current urology research does not consistently provide the components needed to reproduce original studies. Collaborative efforts from investigators and journal editors are needed to improve research quality while minimizing waste and patient risk.

Evaluation of Selective Outcome Reporting Bias in Efficacy Endpoints in Print and Television Advertisements for Oncology Drugs.

IMPORTANCE: Selective outcome reporting bias in oncology drug advertisements may encourage misconceptions about a drug's efficacy profile. OBJECTIVE: We sought to determine the rates of selective outcome reporting in published cancer clinical trials and in television and print advertisements for anticancer medications. We also quantified the number of advertisements that did not include or cite any studies with mature overall survival (OS) data (i.e., data with all required patient events for final analysis). DESIGN/SETTING/PARTICIPANTS: We conducted a cross-sectional investigation of advertisements uploaded to the AdPharm Database (repository of pharmaceutical advertisements); the clinical trials supporting the ads; and the trial registrations associated with the trials. Data were extracted by two investigators who were blinded to each other's data. MAIN OUTCOME MEASURES: The first co-primary objective was to investigate selective outcome reporting between trial registrations and published trials. The second co-primary objective was to investigate selective outcome reporting between the same published trials and drug advertisements. RESULTS: We included 74 advertisements and 48 clinical trials. Print ads were the most common (n = 66), and most print advertisements were targeted to health care providers (n = 55, 83.3%). Overall, 41/48 (85.4%) trials were registered prior to study enrollment, and 41/48 (85.4%) did not deviate from the registered primary endpoints. Across all advertisements (n = 74), statistically significant endpoints were more often reported (unadjusted risk ratio [uRR] 1.26; 95% confidence interval [CI] (1.14-1.40)) and 22/55 (40.0%) advertisements cited trials with immature overall survival data (i.e., data without the required number of events for final analysis). CONCLUSIONS: In our sample, statistically significant endpoints were more commonly reported than nonsignificant endpoints. Immature endpoints (those analyzed before the required number of accrued patient events) were often reported. By reporting only significant endpoints and those that are immature, advertisers may encourage misconceptions about a drug's efficacy profile.

What's in a P-Value? A Fragility Analysis of RCTs in the AUA Guidelines for Benign Prostatic Hyperplasia.

OBJECTIVE: To use the fragility index (FI) and fragility quotient (FQ) to assess the strength of statistically significant findings for randomized controlled trials (RCTs) cited in the American Urological Association (AUA) guidelines for benign prostatic hyperplasia. METHODS: Two investigators independently screened the AUA guidelines for management of benign prostatic hyperplasia for RCTs cited as evidence for recommendations. Investigators extracted data related to event rate per group and loss to follow-up which was compared against the FI. Stata 17.0 was used to calculate the FI and FQ which was then summarized and reported according to primary or secondary endpoints. RESULTS: Among the 373 citations in the AUA guidelines, 24 RCTs met inclusion criteria with 29 distinct outcomes analyzed. The median fragility index was 12 (IQR = 4-38), indicating that twelve alternative events to either study arm would nullify statistical significance. Six studies had a FI of ≤2, indicating that only 1-2 outcomes would need to be changed in order to render nonsignificance of results. In 10/24 RCTs, the number of patients lost to follow-up was greater than the FI. CONCLUSION: The AUA Clinical Practice Guidelines for management of benign prostatic hyperplasia cite RCTs with more robust findings when compared to previous studies assessing fragility in the field of Urology. While several included studies had high fragility, the median FI in our analysis was approximately 4-5 times higher than comparable studies of urologic RCTs. However, there are areas where improvement is necessary to support the highest quality of evidence-based medicine.

Trans-arterial embolization of renal cell carcinoma: a systematic review and meta-analysis.

PURPOSE: To evaluate if trans-arterial embolization (TAE) of the primary tumor in patients with renal cell carcinoma (RCC) improves symptomatology such as pain and hematuria or oncologic outcomes such as progression-free survival (PFS) and overall survival (OS). MATERIALS AND METHODS: The systematic review search included PubMed, Ovid/MEDLINE, and Embase for full-text English articles including randomized and non-randomized prospective trials as well as prospective and retrospective case series. To be included, prospective trials needed ≥ 25 patients in each arm while case series and retrospective chart reviews required at least two patients. Evaluated outcomes included PFS, OS, change in tumor size, improvements in pain, improvements in hematuria, and adverse events (AEs). RESULTS: 1327 articles were retrieved and screened. Nine studies met inclusion criteria (retrospective case series, n = 8; non-randomized prospective trial, n = 1) which included 237 patients (M = 156 (65.8%); F = 56 (23.6%); gender unreported = 25 (10.5%); mean age: 69.4 (range: 38-87)) with a mean tumor diameter of 9.3 cm (5.2-10.5). When reported, the TNM stages were stage I (n = 10), II (n = 18), III (n = 36), and IV (n = 121). 60 patients were treated for pain and hematuria. After TAE, pain improved in 59 patients (98.3%) and hematuria improved in 57 patients (95%). A meta-analysis for improvements in pain and hematuria demonstrated an event rate of pain improvement of 0.952 (0.788-0.990; p < 0.001) and an event rate for hematuria improvement of 0.923 (0.809-0.971; p < 0.001). Median OS ranged from 1 to 39 months but only one study reported PFS (10.5 months). Only one study demonstrated a statistically significant improvement in OS with TAE when compared with patients that did not undergo TAE (p = 0.02). A reduction in tumor size was only achieved in 17 patients (17/49; 34.7%) limiting evaluation. AEs included fever (n = 115/237; 48.5%), flank pain (n = 72/237; 30.4%), nausea (n = 58/237; 24.5%), hematuria (n = 12/237; 5.1%), hypertension (n = 12/237; 5.1%), reduced GFR (n = 6/237; 2.5%), hematoma (n = 6/237,2.5%), and ileus (n = 3/237; 1.3%). CONCLUSION: TAE monotherapy of the primary tumor in patients with RCC improves symptomatology such as pain and hematuria with an acceptable safety profile.

An Analysis of the Evidence Underpinning the American Urologic Association Clinical Practice Guidelines.

OBJECTIVE: To evaluate the reporting quality of systematic reviews (SRs) underpinning the American Urologic Association (AUA) clinical practice guidelines (CPGs). METHODS: We searched the AUA for CPGs from 2015-2021. We extracted all SRs from the reference sections and two independent investigators evaluated eligible SR/meta-analysis using the PRISMA (Preferred Reporting Instrument for Systematic Reviews and Meta-Analyses) and AMSTAR-2 (A Measurement Tool to Assess Systematic Reviews 2) instruments. We compared SRs conducted by the Cochrane group to non-Cochrane SRs using a Mann-Whitney test. A multivariate regression was used to compare study characteristics. RESULTS: Eighteen CPG's met inclusion criteria. We extracted 120 unique SRs, which accounted for 5.1% (n = 120/2346) of all citations. Mean percent adherence to PRISMA and AMSTAR-2 was 65.4% -d 55.2% respectively. SRs conducted by the Cochrane Collaboration scored higher on AMSTAR-2 compared to non-Cochrane (z = -4.41, P <.01) and a positive correlation between PRISMA and AMSTAR-2 scores (r = 0.56, P <.001) was determined. CONCLUSION: Our study indicated the quality of SRs used to develop AUA CPGs across both PRISMA and AMSTAR-2 was variable. Despite higher evaluations, Cochrane SRs accounted for less than 15% of SRs underpinning CPG recommendations. Given the importance placed on CPGs within clinical practice, we recommended a synergistic relationship between the AUA and the Cochrane Collaboration to increase the number of quality urologic SRs.

Evaluating Reproducibility and Transparency in Emergency Medicine Publications.

INTRODUCTION: We aimed to assess the reproducibility of empirical research by determining the availability of components required for replication of a study, including materials, raw data, analysis scripts, protocols, and preregistration. METHODS: We used the National Library of Medicine catalog to identify MEDLINE-indexed emergency medicine (EM) journals. Thirty journals met the inclusion criteria. From January 1, 2014-December 31, 2018, 300 publications were randomly sampled using a PubMed search. Additionally, we included four high-impact general medicine journals, which added 106 publications. Two investigators were blinded for independent extraction. Extracted data included statements regarding the availability of materials, data, analysis scripts, protocols, and registration. RESULTS: After the search, we found 25,473 articles, from which we randomly selected 300. Of the 300, only 287 articles met the inclusion criteria. Additionally, we added 106 publications from high-impact journals of which 77 met the inclusion criteria. Together, 364 publications were included, of which 212 articles contained empirical data to analyze. Of the eligible empirical articles, 2.49%, (95% confidence interval [CI], 0.33% to 4.64%] provided a material statement, 9.91% (95% CI, 5.88% to 13.93%) provided a data statement, 0 provided access to analysis scripts, 25.94% (95% CI, 20.04% to 31.84%) linked the protocol, and 39.15% (95% CI, 32.58% to 45.72%) were preregistered. CONCLUSION: Studies in EM lack indicators required for reproducibility. The majority of studies fail to report factors needed to reproduce research to ensure credibility. Thus, an intervention is required and can be achieved through the collaboration of researchers, peer reviewers, funding agencies, and journals.

Are randomized controlled trials in urology being conducted with justification?

CONTEXT: Considering the substantial increase in research funding in the field of urology, minimizing research waste should be a top priority. Systematic reviews (SRs) compile available evidence regarding a clinical question into a single critical resource. If properly utilized, SRs can help minimize redundant studies, focus attention to unsubstantiated treatments, and reduce research waste. OBJECTIVES: To appraise the use of SRs as justification for conducting randomized controlled trials (RCTs) published in high impact urology journals, and to report the ways SRs were incorporated into RCT manuscripts published in the top four urology journals by h5 index. METHODS: On December 13, 2019, a PubMed search was conducted for RCTs published in the top four urology journals according to the Google Scholar h5-index: European Urology, BJU International, The Journal of Urology, and Urology. For an article to be eligible for inclusion in this study, it must have been a full length RCT, published between November 30, 2014, and November 30, 2019 in one of the identified journals, reported only human subjects, and been accessible in English. The following data points were extracted independently by select investigators from each included RCT: manuscript title, year of publication, journal title, type of intervention (drug, medical device, procedure, other), funding source (government, hospital/university, industry, mixed) type of trial (parallel groups, crossover, cluster), and total number of participants reported in each RCT. The included RCTs were searched for reference to an SR, which was then recorded as "yes - verbatim," "yes - inferred," or "not used as justification" and the location in the manuscript where the SR was cited was recorded. RESULTS: Of the 566 articles retrieved, 276 were included. Overall, 150 (54.3%) RCTs cited an SR as either verbatim (108; 39.1%) or inferred (42; 15.2%) trial justification, while 126 (45.7%) did not use an SR for RCT justification. Of those 126, 107 (84.9%) RCTs did not cite an SR to any extent. A significant association was noted between verbatim justification and type of intervention (x2=20.23, p=0.017), with 18 of 31 (58.1%) "other" interventions (i.e. psychosocial intervention, exercise programs, and online therapy) having an SR cited as verbatim justification. Only 39 of 118 (33.1%) pharmaceutical trials referenced an SR as verbatim justification. Of 403 systematic review citations, 205 (50.8%) appeared in the Discussion section, while 15 (3.7%) were in the Methods section. CONCLUSIONS: We found that RCTs published in four high impact urology journals inconsistently referenced an SR as justification and 39.1% of our entire sample did not reference an SR at all. These findings indicate that a divide exists between the instruction and implementation of evidence based medicine in the field of urology concerning RCTs published in the top four journals. Educating clinicians and researchers on the use of SR as justification for RCTs in urology may reduce research waste and increase the quality of RCTs in the field.

Using Google Trends and Twitter for Prostate Cancer Awareness: A Comparative Analysis of Prostate Cancer Awareness Month and Breast Cancer Awareness Month.

Background We evaluated (1) whether the public interest in prostate cancer and prostate cancer screening increased following Prostate Cancer Awareness Month (PCAM) and (2) whether PCAM was as effective as Breast Cancer Awareness Month (BCAM) at generating public interest. Methods  Using Google Trends, we measured search volume in PCAM and BCAM. We used the search volume in Google Trends as a proxy for changes in public interest from January 01, 2009 to December 31, 2018 worldwide, including the specific keywords: "Prostate Cancer"; "Prostate-Specific Antigen"; "Prostate Cancer Screening"; "Prostate Cancer Management"; "Breast Cancer"; "Breast Cancer Screening"; "Mammography"; and "Breast Cancer Management". Also, we measured tweets containing "prostate cancer" and "breast cancer". We used an autoregressive integrated moving algorithm (ARIMA) to forecast expected weekly search volumes during PCAM and BCAM. We then compared the Google Trends data from during PCAM and BCAM to the forecasted values and determined a "greater than expected" range. Results The mean pooled percent increase in tweets associated with "prostate cancer" during PCAM from 2012 through 2018 was 15.9% (95% CI, -1% - 33%). The mean pooled percent increase in tweets associated with "breast cancer" during BCAM from 2012 through 2018 was 318.5% (95% CI, 268% - 369%). BCAM was associated with a 302.6% greater effect on increasing tweets referencing the disease of interest than PCAM from 2012-2018. "Breast cancer" Google searches were found to be 36.7% (95% CI, 34% - 39%) more frequent than "prostate cancer" per month from 2009-2019. Google Searches for "breast cancer screening" were 29.6% (95% CI, 28% - 31%) greater than "prostate cancer screening". Conclusions Our results indicate that PCAM is not generating substantial internet interest, especially when compared to BCAM. The search volume for Google Trends search terms related to PCAM was less than BCAM in every comparison, and Twitter indicated only a slight increase of Tweets during the month of PCAM. Suggestions are provided to improve the effect of PCAM and men's health.

Association between author conflicts of interest and industry-sponsorship with the favorability of outcomes of systematic reviews focusing on treatments of erectile dysfunction.

BACKGROUND: Authors' conflicts of interest and industry sponsorship have been shown to influence study outcomes. OBJECTIVE: We aimed to determine whether author conflicts of interest and industry sponsorship influenced the nature of results and conclusions of systematic reviews focusing on treatment interventions for erectile dysfunction. MATERIALS AND METHODS: We searched PubMed and Embase for systematic reviews and meta-analyses focusing on erectile dysfunction treatments published between September 1, 2016, and June 2, 2020. Authors' conflicts of interest were collected from the systematic reviews' disclosure statements. These disclosures were verified using the information provided by the Open Payments, Dollars for Profs, Google Patents, and US Patent and Trademark Office databases and from previously published disclosure statements. RESULTS: Our study included 24 systematic reviews authored by 138 authors. Nineteen authors (13.8%) were found to have conflicts of interest (disclosed, undisclosed, or both). No authors completely disclosed all conflicts. Nine reviews (37.5%) contained at least one author with conflicts of interest; of which eight reported narrative results favoring the treatment group, and seven reported conclusions favoring the treatment group. Of the 15 (62.5%) reviews without a conflicted author, 11 reported results favoring the treatment group, and 12 reported conclusions favoring the treatment group. DISCUSSION: The results and conclusions of systematic reviews for erectile dysfunction treatments did not appear to be influenced by authors who reported conflicts of interest. However, our search algorithm relied on the US-based Open Payments database and a large percentage of reviews in our study were produced by authors with international affiliations. Our study results underscore the difficulties in conducting such analyses. CONCLUSION: Although we found that undisclosed conflicts of interest (COI) were problematic among systematic reviews of erectile dysfunction treatment, only 14% of authors in our sample possessed them and these COI did not appear to influence the favorability of systematic review outcomes.

Scholarly Research Productivity among Otolaryngology Residency Graduates and its Relationship to Future Academic Achievement.

OBJECTIVE: Scholastic activity through research involvement is a fundamental aspect of a physician's training and may have a significant influence on future academic success. Here, we explore publication rates before, during, and after otolaryngology residency training and whether publication efforts correlate with future academic achievement. METHODS: This cross-sectional analysis included a random sample of 50 otolaryngology residency programs. From these programs, we assembled a list of residents graduating from the years in 2013, 2014, and 2015. Using SCOPUS, PubMed, and Google Scholar, we compiled the publications for each graduate, and data were extracted in an independent, double-blinded fashion. RESULTS: We included 32 otolaryngology residency programs representing 249 residents in this analysis. Graduates published a mean of 1.3 (SD = 2.7) articles before residency, 3.5 (SD = 4.3) during residency, and 5.3 (SD = 9.3) after residency. Residents who pursued a fellowship had more total publications (t247 = -6.1, P < .001) and more first author publications (t247 = -5.4, P < .001) than residents without fellowship training. Graduates who chose a career in academic medicine had a higher number of mean total publications (t247 = -8.2, P < .001) and first author publications (t247 = -7.9, P < .001) than those who were not in academic medicine. There was a high positive correlation between residency program size and publications during residency (r = 0.76). CONCLUSION: Research productivity correlated with a number of characteristics such as future fellowship training, the pursuit of an academic career, and overall h-index in this study.

The use of systematic reviews to justify phase III ophthalmology trials: an analysis.

BACKGROUND/OBJECTIVE: Given the drastic increase in publication output in recent years, minimizing research waste should be a top priority. There are established areas of concern regarding research waste within ophthalmology along with a lack of systematic review usage to inform trial design in other areas of medicine. Given these concerns, the aim of this study is to evaluate the use of systematic reviews as justification for conducting randomized controlled trials (RCT) in top ophthalmology and optometry journals. METHODS: We searched PubMed on December 5, 2018 for RCTs published in one of the top five Google Scholar h-5 index journals within Ophthalmology and Optometry. We used a pilot-tested Google Form and searched each RCT for systematic reviews. Each systematic review was then given the designation of "verbatim", "inferred", or "not used as justification for conducting the RCT" based on the context the systematic review was used. RESULTS: Our analysis yielded 152 included phase III RCTs. We found 22.4% (34 of 152) of phase III ophthalmology clinical trials cited a systematic review as justification for conducting the trial. A total of 102 systematic reviews were cited in the 152 RCTs. Fifty-seven of the one hundred fifty-two (37.5%) RCTs cited a systematic review somewhere in the manuscript. CONCLUSION: Less than one-quarter of phase III RCTs cited systematic reviews as justification for conducting the RCT. We believe placing a higher priority on justifying RCTs with systematic reviews would go a long way to minimizing research waste within ophthalmology.

Quality of Intervention Reporting in Randomized Clinical Trials Published in Urology Journals.

We used the Template for Intervention Description and Replication (TIDieR) to find evidence that intervention reporting in urology randomized controlled trials is suboptimal. Action to improve intervention reporting is warranted and we advise extending TIDieR into Consolidated Standards for Reporting Trials guidelines.