Shortcomings in the evaluation of biomarkers in ovarian cancer: a systematic review.

Background Shortcomings in study design have been hinted at as one of the possible causes of failures in the translation of discovered biomarkers into the care of ovarian cancer patients, but systematic assessments of biomarker studies are scarce. We aimed to document study design features of recently reported evaluations of biomarkers in ovarian cancer. Methods We performed a systematic search in PubMed (MEDLINE) for reports of studies evaluating the clinical performance of putative biomarkers in ovarian cancer. We extracted data on study designs and characteristics. Results Our search resulted in 1026 studies; 329 (32%) were found eligible after screening, of which we evaluated the first 200. Of these, 93 (47%) were single center studies. Few studies reported eligibility criteria (17%), sampling methods (10%) or a sample size justification or power calculation (3%). Studies often used disjoint groups of patients, sometimes with extreme phenotypic contrasts; 46 studies included healthy controls (23%), but only five (3%) had exclusively included advanced stage cases. Conclusions Our findings confirm the presence of suboptimal features in clinical evaluations of ovarian cancer biomarkers. This may lead to premature claims about the clinical value of these markers or, alternatively, the risk of discarding potential biomarkers that are urgently needed.

Potential benefits and harms of offering ultrasound surveillance to men aged 65 years and older with a subaneurysmal (2.5-2.9 cm) infrarenal aorta.

OBJECTIVE: The objective of this review was to perform a rapid evidence summary to determine the prevalence of subaneurysmal aortic aneurysms, growth rates, and risk factors that modulate growth in average-risk men aged 65 years and older. Secondary objectives were to evaluate benefits and harms of lifelong ultrasound (US) surveillance and treatment outcomes for any large aneurysms that develop in the screened population. METHODS: We searched multiple databases (eg, Ovid MEDLINE, Embase Classic and Embase, and the Cochrane Library) on February 16, 2016. Using a liberal accelerated method, two reviewers screened titles and abstracts for relevance and subsequently screened full-text studies. General study characteristics (eg, country, study design, number of participants) and data (eg, number of men with subaneurysmal aortas, quality of life [QoL], mortality) were extracted. One reviewer performed data extraction and risk of bias assessments, and a second reviewer verified 100% of studies. Any disagreements were resolved by consensus. RESULTS: The search identified 37 relevant studies ranging in size from 3 to 52,690 participants. Prevalence of subaneurysmal aortas ranged from 1.14% to 8.53%, and 55% to 88% of these men progressed to a 3.0-cm aneurysm by 5 years of follow-up. Risk factors for growth included the infrarenal aortic diameter at age 65 years, having a subaneurysmal aorta at age 65 years, and current smoking. The 36-Item Short Form Health Survey was the most commonly used tool to measure QoL, and QoL was typically lower in people with abdominal aortic aneurysm. Anxiety and depression levels did not differ significantly between comparison groups in any studies. Four studies reported on the number of men whose aorta was subaneurysmal on initial US who went on to surgery. Overall, 10% (57/547) of men initially measuring in the subaneurysmal range progressed to abdominal aortic aneurysm >5.4 cm and received elective surgery; 1% (6/547) received emergency surgery because of a ruptured aorta. Among those who did, mortality rates were much lower for elective (9.5%) vs emergency surgery (50%). Risk of bias was usually low for studies measuring prevalence and moderate and high for studies measuring psychological harms of screening and harms and benefits of surgery. Overall, using the Grading of Recommendations Assessment, Development, and Evaluation framework as guidance, the quality of the evidence was generally very low. CONCLUSIONS: Because of the limited evidence and the low quality of the existing evidence, it is not possible to determine confidently whether men with abdominal aortas measuring 2.5 to 2.9 cm should be observed in a lifelong US surveillance program.

SPIN-PM: a consensus framework to evaluate the presence of spin in studies on prediction models.

OBJECTIVES: To develop a framework to identify and evaluate spin practices and its facilitators in studies on clinical prediction model regardless of the modeling technique. STUDY DESIGN AND SETTING: We followed a three-phase consensus process: (1) premeeting literature review to generate items to be included; (2) a series of structured meetings to provide comments discussed and exchanged viewpoints on items to be included with a panel of experienced researchers; and (3) postmeeting review on final list of items and examples to be included. Through this iterative consensus process, a framework was derived after all panel's researchers agreed. RESULTS: This consensus process involved a panel of eight researchers and resulted in SPIN-Prediction Models which consists of two categories of spin (misleading interpretation and misleading transportability), and within these categories, two forms of spin (spin practices and facilitators of spin). We provide criteria and examples. CONCLUSION: We proposed this guidance aiming to facilitate not only the accurate reporting but also an accurate interpretation and extrapolation of clinical prediction models which will likely improve the reporting quality of subsequent research, as well as reduce research waste.

The score after 10 years of registration of systematic review protocols.

BACKGROUND: With the exponential growth of published systematic reviews (SR), there is a high potential for overlapping and redundant duplication of work. Prospective protocol registration gives the opportunity to assess the added value of a new study or review, thereby potentially reducing research waste and simultaneously increasing transparency and research quality. The PROSPERO database for SR protocol registration was launched 10 years ago. This study aims to assess the proportion SRs of intervention studies with a protocol registration (or publication) and explore associations of SR characteristics with protocol registration status. METHODS: PubMed was searched for SRs of human intervention studies published in January 2020 and January 2021. After random-stratified sampling and eligibility screening, data extraction on publication and journal characteristics, and protocol registration status, was performed. Both descriptive and multivariable comparative statistical analyses were performed. RESULTS: A total of 357 SRs (2020: n = 163; 2021: n = 194) were included from a random sample of 1267 publications. Of the published SRs, 38% had a protocol. SRs that reported using PRISMA as a reporting guideline had higher odds of having a protocol than publications that did not report PRISMA (OR 2.71; 95% CI: 1.21 to 6.09). SRs with a higher journal impact factor had higher odds of having a protocol (OR 1.12; 95% CI 1.04 to 1.25). Publications from Asia had a lower odds of having a protocol (OR 0.43; 95% CI 0.23 to 0.80, reference category = Europe). Of the 33 SRs published in journals that endorse PROSPERO, 45% did not have a protocol. Most SR protocols were registered in PROSPERO (n = 129; 96%). CONCLUSIONS: We found that 38% of recently published SRs of interventions reported a registered or published protocol. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent self-reported use of the PRISMA guidelines. In some parts of the world, SR protocols are more often registered or published than others. To guide strategies to increase the uptake of SR protocol registration, further research is needed to gain understanding of the benefits and informativeness of SRs protocols among different stakeholders. SYSTEMATIC REVIEW REGISTRATION: osf.io/9kj7r/.

A randomized trial of an editorial intervention to reduce spin in the abstract's conclusion of manuscripts showed no significant effect.

OBJECTIVE: To estimate the effect of an intervention compared to the usual peer-review process on reducing spin in the abstract's conclusion of biomedical study reports. STUDY DESIGN AND SETTING: We conducted a two-arm, parallel-group RCT in a sample of primary research manuscripts submitted to BMJ Open. The authors received short instructions alongside the peer reviewers' comments in the intervention group. We assessed the presence of spin (primary outcome), types of spin, and wording change in the revised abstract's conclusion. Outcome assessors were blinded to the intervention assignment. RESULTS: Of the 184 manuscripts randomized, 108 (54 intervention, 54 control) were selected for revision and could be evaluated for the presence of spin. The proportion of manuscripts with spin was 6% lower (95% CI: 24% lower to 13% higher) in the intervention group (57%, 31/54) than in the control group (63%, 34/54). The wording of the revised abstract's conclusion was changed in 34/54 (63%) manuscripts in the intervention group and 26/54 (48%) in the control group. The four prespecified types of spin involved (i) selective reporting (12 in the intervention group vs. 8 in the control group), (ii) including information not supported by evidence (9 vs. 9), and (iii) interpretation not consistent with the study results (14 vs. 18), and (iv) unjustified recommendations for practice (5 vs. 11). CONCLUSION: These short instructions to authors did not have a statistically significant effect on reducing spin in revised abstract conclusions, and based on the confidence interval, the existence of a large effect can be excluded. Other interventions to reduce spin in reports of original research should be evaluated. STUDY REGISTRATION: osf.io/xnuyt.

No evidence found for an association between trial characteristics and treatment effects in randomized trials of testosterone therapy in men: a meta-epidemiological study.

OBJECTIVE: The objective of this study was to identify potential trial characteristics associated with reported treatment effect estimates in randomized trials of testosterone therapy in adult men. STUDY DESIGN AND SETTING: This is a meta-epidemiological study. MEDLINE was searched for meta-analyses of randomized trials of testosterone therapy in men published between 2008 and 2018. Data on trial characteristics were extracted independently by two reviewers. The impact of trial characteristics on reported treatment effects was investigated using a two-step meta-analytic approach. RESULTS: We identified 132 randomized trials, included in 19 meta-analyses, comprising data from 10,725 participants. None of the investigated design characteristics, including year of publication, sample size, trial registration status, center status, regionality, funding source, and conflict of interest were statistically significantly associated with reported treatment effects of testosterone therapy in men. Although trials rated at high risk of bias overall reported treatment effects that were 21% larger compared with trials rated at low risk of bias overall, the 95% confidence interval included the null (ratio of odds ratio: 0.79, 95% confidence interval: 0.60 to 1.03). CONCLUSION: The present study found no clear evidence that trial characteristics are associated with treatment effects in randomized trials of testosterone therapy in men. To establish stronger evidence about the treatment effects of testosterone therapy in men, future randomized trials should not only be adequately designed but also transparently reported. STUDY REGISTRATION: osf.io/x9g6m.

A systematic review finds that spin or interpretation bias is abundant in evaluations of ovarian cancer biomarkers.

BACKGROUND: In the scientific literature, "spin" refers to reporting practices that make the study findings appear more favorable than results justify. The practice of "spin" or misrepresentation and overinterpretation may lead to an imbalanced and unjustified optimism in the interpretation of study results about performance of putative biomarkers. We aimed to classify spin (i.e., misrepresentation and overinterpretation of study findings) in recent clinical studies evaluating the performance of biomarkers in ovarian cancer. METHODS: We searched PubMed systematically for all evaluations of ovarian cancer biomarkers published in 2015. Studies eligible for inclusion reported the clinical performance of prognostic, predictive, or diagnostic biomarkers. RESULTS: Our search identified 1,026 studies; 326 studies met all eligibility criteria, of which we evaluated the first 200 studies. Of these, 140 (70%) contained one or more form of spin in the title, abstract, or main-text conclusion, exaggerating the performance of the biomarker. The most frequent forms of spin identified were (1) other purposes of biomarker claimed not investigated (65; 32.5%); (2) mismatch between intended aim and conclusion (57; 28.5%); and (3) incorrect presentation of results (40; 20%). CONCLUSION: Our study provides evidence of misrepresentation and overinterpretation of finding in recent clinical evaluations of ovarian cancer biomarkers.

The prevalence of patient engagement in published trials: a systematic review.

Plain English summary: With the growing movement to engage patients in research, questions are being asked about who is engaging patients and how they are being engaged. Internationally, research groups are supporting and funding patient-oriented research studies that engage patients in the identification of research priorities and the design, conduct and uptake of research. As we move forward, we need to know what meaningful patient engagement looks like, how it benefits research and clinical practice, and what are the barriers to patient engagement?We conducted a review of the published literature looking for trials that report engaging patients in the research. We included both randomized controlled trials and non-randomized comparative trials. We looked at these trials for important study characteristics, including how patients were engaged, to better understand the practices used in trials. Importantly, we also discuss the number of trials reporting patient engagement practices relative to all published trials. We found that very few trials report any patient engagement activities even though it is widely supported by many major funding organizations. The findings of our work will advance patient-oriented research by showing how patients can be engaged and by stressing that patient engagement practices need to be better reported. Background: Patient-Oriented Research (POR) is research informed by patients and is centred on what is of importance to them. A fundamental component of POR is that patients are included as an integral part of the research process from conception to dissemination and implementation, and by extension, across the research continuum from basic research to pragmatic trials [J Comp Eff Res 2012, 1:181-94, JAMA 2012, 307:1587-8]. Since POR's inception, questions have been raised as to how best to achieve this goal.We conducted a systematic review of randomized controlled trials and non-randomized comparative trials that report engaging patients in their research. Our main goal was to describe the characteristics of published trials engaging patients in research, and to identify the extent of patient engagement activities reported in these trials. Methods: The MEDLINE®, EMBASE®, Cinahl, PsycINFO, Cochrane Methodology Registry, and Pubmed were searched from May 2011 to June 16th, 2016. Title, abstract and full text screening of all reports were conducted independently by two reviewers. Data were extracted from included trials by one reviewer and verified by a second. All trials that report patient engagement for the purposes of research were included. Results: Of the 9490 citations retrieved, 2777 were reviewed at full text, of which 23 trials were included. Out of the 23 trials, 17 were randomized control trials, and six were non-randomized comparative trials. The majority of these trials (83%, 19/23) originated in the United States and United Kingdom. The trials engaged a range of 2-24 patients/ community representatives per study. Engagement of children and minorities occurred in 13% (3/23) and 26% (6/23) of trials; respectively. Engagement was identified in the development of the research question, the selection of study outcomes, and the dissemination and implementation of results. Conclusions: The prevalence of patient engagement in patient-oriented interventional research is very poor with 23 trials reporting activities engaging patients. Research dedicated to determining the best practice for meaningful engagement is still needed, but adequate reporting measures also need to be defined.

Mesenchymal Stromal Cell Therapy in Bronchopulmonary Dysplasia: Systematic Review and Meta-Analysis of Preclinical Studies.

Extreme prematurity is the leading cause of death among children under 5 years of age. Currently, there is no treatment for bronchopulmonary dysplasia (BPD), the most common complication of extreme prematurity. Experimental studies in animal models of BPD suggest that mesenchymal stromal cells (MSCs) are lung protective. To date, no systematic review and meta-analysis has evaluated the preclinical evidence of this promising therapy. Our protocol was registered with Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies prior to searching MEDLINE (1946 to June 1, 2015), Embase (1947 to 2015 Week 22), Pubmed, Web of Science, and conference proceedings (1990 to present) for controlled comparative studies of neonatal animal models that received MSCs or cell free MSC-derived conditioned media (MSC-CM). Lung alveolarization was the primary outcome. We used random effects models for data analysis and followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses reporting guidelines. We screened 990 citations; 25 met inclusion criteria. All used hyperoxia-exposed neonatal rodents to model BPD. MSCs significantly improved alveolarization (Standardized mean difference of -1.330, 95% confidence interval [CI -1.724, -0.94, I2 69%]), irrespective of timing of treatment, source, dose, or route of administration. MSCs also significantly ameliorated pulmonary hypertension, lung inflammation, fibrosis, angiogenesis, and apoptosis. Similarly, MSC-CM significantly improved alveolarization, angiogenesis, and pulmonary artery remodeling. MSCs, tested exclusively in hyperoxic rodent models of BPD, show significant therapeutic benefit. Unclear risk of bias and incomplete reporting in the primary studies highlights nonadherence to reporting standards. Overall, safety and efficacy in other species/large animal models may provide useful information for guiding the design of clinical trials. Stem Cells Translational Medicine 2017;6:2079-2093.