Evaluation of reproducible and transparent research practices in pulmonology.

BACKGROUND: Study reproducibility is valuable for validating or refuting results. Provision of reproducibility indicators, such as materials, protocols, and raw data in a study improve its potential for reproduction. Efforts to reproduce noteworthy studies in the biomedical sciences have resulted in an overwhelming majority of them being found to be unreplicable, causing concern for the integrity of research in other fields, including medical specialties. Here, we analyzed the reproducibility of studies in the field of pulmonology. METHODS: 500 pulmonology articles were randomly selected from an initial PubMed search for data extraction. Two authors scoured these articles for reproducibility indicators including materials, protocols, raw data, analysis scripts, inclusion in systematic reviews, and citations by replication studies as well as other factors of research transparency including open accessibility, funding source and competing interest disclosures, and study preregistration. FINDINGS: Few publications included statements regarding materials (10%), protocols (1%), data (15%), and analysis script (0%) availability. Less than 10% indicated preregistration. More than half of the publications analyzed failed to provide a funding statement. Conversely, 63% of the publications were open access and 73% included a conflict of interest statement. INTERPRETATION: Overall, our study indicates pulmonology research is currently lacking in efforts to increase replicability. Future studies should focus on providing sufficient information regarding materials, protocols, raw data, and analysis scripts, among other indicators, for the sake of clinical decisions that depend on replicable or refutable results from the primary literature.

Discrepancies between author- and industry-reported disclosures of financial relationships at an annual gynecologic oncology research meeting.

OBJECTIVE: Trillions of dollars pass to physicians from industry-related businesses annually, leading to many opportunities for financial conflicts of interest. The Open Payments Database (OPD) was created to ensure transparency. We describe the industry relationships as reported in the OPD for presenters at the 2019 Society of Gynecologic Oncology (SGO) Annual Meeting and evaluate concordance between author disclosures of their financial interests and information provided by the OPD. METHODS: This is an observational, cross-sectional study. Disclosure data were collected from authors with oral and featured abstract presentations in the 2019 SGO annual conference. These disclosures were compared to data available for each author in the 2018 OPD, which included the amount and nature of industry payments. RESULTS: We examined the disclosures of 301 authors who met inclusion criteria. Of 161 authors who had disclosure statements on their presentations,147 reported "no disclosures," and 14 disclosed industry relationships. The remaining 140 did not list any disclosure information. Sixty percent (184/301) of authors had industry relationships in the 2018 OPD, including 173 of 287 (60.3%) of authors who either reported no disclosures or did not have disclosure data available in their presentations. These transactions totaled over 43 million USD from 122 different companies, with most payments (46%) categorized as "Research or Associated Research." Accurate disclosure reporting was associated with receiving higher payments or research payments, and being a presenting author. CONCLUSIONS: Most authors at the SGO annual conference did not correctly disclose their industry relationships when compared with their entries in the OPD.

Publication rates of research projects of an internal funding program of a university medical center in Germany: A retrospective study (2004-2013).

OBJECTIVES: Non-publication and publication bias are topics of considerable importance to the scientific community. These issues may limit progress toward the 3R principle for animal research, promote waste of public resources, and generate biased interpretations of clinical outcomes. To investigate current publishing practices and to gain some understanding of the extent to which research results are reported, we examined publication rates of research projects that were approved within an internal funding program of the Faculty of Medicine at a university medical center in Germany, which is exemplary for comparable research funding programs for the promotion of young researchers in Germany and Europe. METHODS: We analyzed the complete set (n = 363) of research projects that were supported by an internal funding program between 2004 and 2013. We divided the projects into four different proposal types that included those that required an ethics vote, those that included an animal proposal, those that included both requirements, and those that included neither requirement. RESULTS: We found that 65% of the internally funded research projects resulted in at least one peer-reviewed publication; this increased to 73% if other research contributions were considered, including abstracts, book and congress contributions, scientific posters, and presentations. There were no significant differences with respect to publication rates based on (a) the clinic/institute of the applicant, (b) project duration, (c) scope of funding or (d) proposal type. CONCLUSION: To the best of our knowledge, this is the first study to explore publication rates associated with early-career medical research funding. As >70% of the projects ultimately generated some form of publication, the program was overall effective toward this goal; however, non-publication of research results is still prevalent. Further research will explore the reasons underlying non-publication. We hope to use these findings to develop strategies that encourage publication of research results.

Publication rates from biomedical and behavioral and social science R01s funded by the National Institutes of Health.

Prior research has shown a serious lack of research transparency resulting from the failure to publish study results in a timely manner. The National Institutes of Health (NIH) has increased its use of publication rate and time to publication as metrics for grant productivity. In this study, we analyze the publications associated with all R01 and U01 grants funded from 2008 through 2014, providing sufficient time for these grants to publish their findings, and identify predictors of time to publication based on a number of variables, including if a grant was coded as a behavioral and social sciences research (BSSR) grant or not. Overall, 2.4% of the 27,016 R01 and U01 grants did not have a publication associated with the grant within 60 months of the project start date, and this rate of zero publications was higher for BSSR grants (4.6%) than for non-BSSR grants (1.9%). Mean time in months to first publication was 15.2 months, longer for BSSR grants (22.4 months) than non-BSSR grants (13.6 months). Survival curves showed a more rapid reduction of risk to publish from non-BSSR vs BSSR grants. Cox regression models showed that human research (vs. animal, neither, or both) and clinical trials research (vs. not) are the strongest predictors of time to publication and failure to publish, but even after accounting for these and other predictors, BSSR grants continued to show longer times to first publication and greater risk of no publications than non-BSSR grants. These findings indicate that even with liberal criteria for publication (any publication associated with a grant), a small percentage of R01 and U01 grantees fail to publish in a timely manner, and that a number of factors, including human research, clinical trial research, child research, not being an early stage investigator, and conducting behavioral and social sciences research increase the risk of time to first publication.

Dementia Research Productivity and Associations with Socioeconomic Factors and Burden of Disease in Southeast Asia.

BACKGROUND: Scientific output in Southeast Asia (SEA) on the topic of dementia is postulated to be low in quality and quantity. It is also speculated that certain socioeconomic variables and measures of disease burden for dementia may play a significant role in driving the research output of a particular country. OBJECTIVE: This study aimed to determine the research impact of published journal articles on dementia in SEA and its association with country-level socioeconomic factors and measures of disease burden for dementia. METHODS: A systematic search was conducted using electronic healthcare databases. We included articles published on dementia until August 2019 with at least 1 author affiliated with any SEA institution. We obtained bibliometric indices, relevant socioeconomic factors, and measures of disease burden for dementia from published sources. RESULTS: One thousand six articles fulfilled the inclusion criteria. The majority of publications were related to Alzheimer's disease (n = 775, 77.0%). Singapore contributed the highest number of publications (n = 457, 45.4%). Gross domestic product (GDP) per capita, % GDP for research and development, and total neurologists significantly correlated with several bibliometric indices. On the other hand, the measures of disease burden for dementia in SEA countries were not significantly associated with research productivity. CONCLUSION: Research productivity in SEA on dementia has substantially increased in recent years. Augmenting GDP per capita and expanding the apportionment of resources to research and development (R&D) may have a significant role in the advancement of dementia research in SEA.

A network analysis of research productivity by country, discipline, and wealth.

INTRODUCTION: Research productivity has been linked to a country's intellectual and economic wealth. Further analysis is needed to assess the association between the distribution of research across disciplines and the economic status of countries. METHODS: By using 55 years of data, spanning 1962 to 2017, of Elsevier publications across a large set of research disciplines and countries globally, this manuscript explores the relationship and evolution of relative research productivity across different disciplines through a network analysis. It also explores the associations of those with economic productivity categories, as measured by the World Bank economic classification. Additional analysis of discipline similarities is possible by exploring the cross-country evolution of those disciplines. RESULTS: Results show similarities in the relative importance of research disciplines among most high-income countries, with larger idiosyncrasies appearing among the remaining countries. This group of high-income countries shows similarities in the dynamics of the relative distribution of research productivity over time, forming a stable research productivity cluster. Lower income countries form smaller, more independent and evolving clusters, and differ significantly from each other and from higher income countries in the relative importance of their research emphases. Country-based similarities in research productivity profiles also appear to be influenced by geographical proximity. CONCLUSIONS: This new form of analyses of research productivity, and its relation to economic status, reveals novel insights to the dynamics of the economic and research structure of countries. This allows for a deeper understanding of the role a country's research structure may play in shaping its economy, and also identification of benchmark resource allocations across disciplines for developing countries.

Scientists' political behaviors are not driven by individual-level government benefits.

Is it appropriate for scientists to engage in political advocacy? Some political critics of scientists argue that scientists have become partisan political actors with self-serving financial agendas. However, most scientists strongly reject this view. While social scientists have explored the effects of science politicization on public trust in science, little empirical work directly examines the drivers of scientists' interest in and willingness to engage in political advocacy. Using a natural experiment involving the U.S. National Science Foundation Graduate Research Fellowship (NSF-GRF), we causally estimate for the first time whether scientists who have received federal science funding are more likely to engage in both science-related and non-science-related political behaviors. Comparing otherwise similar individuals who received or did not receive NSF support, we find that scientists' preferences for political advocacy are not shaped by receiving government benefits. Government funding did not impact scientists' support of the 2017 March for Science nor did it shape the likelihood that scientists donated to either Republican or Democratic political groups. Our results offer empirical evidence that scientists' political behaviors are not motivated by self-serving financial agendas. They also highlight the limited capacity of even generous government support programs to increase civic participation by their beneficiaries.

The learning hospital: From theory to practice in a hospital infection prevention program.

The learning hospital is distinguished by ceaseless evolution of erudition, enhancement, and implementation of clinical best practices. We describe a model for the learning hospital within the framework of a hospital infection prevention program and argue that a critical assessment of safety practices is possible without significant grant funding. We reviewed 121 peer-reviewed manuscripts published by the VCU Hospital Infection Prevention Program over 16 years. Publications included quasi-experimental studies, observational studies, surveys, interrupted time series analyses, and editorials. We summarized the articles based on their infection prevention focus, and we provide a brief summary of the findings. We also summarized the involvement of nonfaculty learners in these manuscripts as well as the contributions of grant funding. Despite the absence of significant grant funding, infection prevention programs can critically assess safety strategies under the learning hospital framework by leveraging a diverse collaboration of motivated nonfaculty learners. This model is a valuable adjunct to traditional grant-funded efforts in infection prevention science and is part of a successful horizontal infection control program.

Identifying publications in questionable journals in the context of performance-based research funding.

In this article we discuss the five yearly screenings for publications in questionable journals which have been carried out in the context of the performance-based research funding model in Flanders, Belgium. The Flemish funding model expanded from 2010 onwards, with a comprehensive bibliographic database for research output in the social sciences and humanities. Along with an overview of the procedures followed during the screenings for articles in questionable journals submitted for inclusion in this database, we present a bibliographic analysis of the publications identified. First, we show how the yearly number of publications in questionable journals has evolved over the period 2003-2016. Second, we present a disciplinary classification of the identified journals. In the third part of the results section, three authorship characteristics are discussed: multi-authorship, the seniority-or experience level-of authors in general and of the first author in particular, and the relation of the disciplinary scope of the journal (cognitive classification) with the departmental affiliation of the authors (organizational classification). Our results regarding yearly rates of publications in questionable journals indicate that awareness of the risks of questionable journals does not lead to a turn away from open access in general. The number of publications in open access journals rises every year, while the number of publications in questionable journals decreases from 2012 onwards. We find further that both early career and more senior researchers publish in questionable journals. We show that the average proportion of senior authors contributing to publications in questionable journals is somewhat higher than that for publications in open access journals. In addition, this paper yields insight into the extent to which publications in questionable journals pose a threat to the public and political legitimacy of a performance-based research funding system of a western European region. We include concrete suggestions for those tasked with maintaining bibliographic databases and screening for publications in questionable journals.

Scientific output scales with resources. A comparison of US and European universities.

By using a comprehensive dataset of US and European universities, we demonstrate super-linear scaling between university revenues and their volume of publications and (field-normalized) citations. We show that this relationship holds both in the US and in Europe. In terms of resources, our data show that three characteristics differentiate the US system: (1) a significantly higher level of resources for the entire system, (2) a clearer distinction between education-oriented institutions and doctoral universities and (3) a higher concentration of resources among doctoral universities. Accordingly, a group of US universities receive a much larger amount of resources and have a far higher number of publications and citations when compared to their European counterparts. These results demonstrate empirically that international rankings are by and large richness measures and, therefore, can be interpreted only by introducing a measure of resources. Implications for public policies and institutional evaluation are finally discussed.

Scientific sinkhole: The pernicious price of formatting.

OBJECTIVE: To conduct a time-cost analysis of formatting in scientific publishing. DESIGN: International, cross-sectional study (one-time survey). SETTING: Internet-based self-report survey, live between September 2018 and January 2019. PARTICIPANTS: Anyone working in research, science, or academia and who submitted at least one peer-reviewed manuscript for consideration for publication in 2017. Completed surveys were available for 372 participants from 41 countries (60% of respondents were from Canada). MAIN OUTCOME MEASURE: Time (hours) and cost (wage per hour x time) associated with formatting a research paper for publication in a peer-reviewed academic journal. RESULTS: The median annual income category was US$61,000-80,999, and the median number of publications formatted per year was four. Manuscripts required a median of two attempts before they were accepted for publication. The median formatting time was 14 hours per manuscript, or 52 hours per person, per year. This resulted in a median calculated cost of US$477 per manuscript or US$1,908 per person, per year. CONCLUSIONS: To our knowledge, this is the first study to analyze the cost of manuscript formatting in scientific publishing. Our results suggest that scientific formatting represents a loss of 52 hours, costing the equivalent of US$1,908 per researcher per year. These results identify the hidden and pernicious price associated with scientific publishing and provide evidence to advocate for the elimination of strict formatting guidelines, at least prior to acceptance.

Financial Conflicts of Interest Among Authors of Clinical Practice Guidelines for Routine Screening Mammography.

PURPOSE: Financial conflicts of interest (FCOIs) may influence or undermine the credibility of clinical practice guidelines or society recommendations. Given the wide regard of such publications, understanding the prevalence and extent of FCOIs among their authors is essential. METHODS: The most current guidelines containing recommendations for breast cancer screening from the US Preventive Services Task Force, American Cancer Society, American College of Obstetricians and Gynecologists, International Agency for Research on Cancer, ACR, and American College of Physicians were retrieved from their respective organizational websites. Industry payments received by authors were then extracted using CMS Open Payments database (OPD), and the values and types of these payments were evaluated. Finally, financial disclosures were compared with open payments. RESULTS: Among a total of 43 authors and 7 guideline documents, 14 authors (33%) received at least one industry payment according to OPD payment records, whereas a majority of 29 authors (67%) had none. The median total payment from all sources across all breast imaging guidelines was $0 (interquartile range, $0-$84). Four authors (9%) declared at least one significant FCOI, five (12%) received more than $5,000 from a single company in a single year, and one author had a significant FCOI (2%) identified from OPD records but not disclosed within the guideline document. CONCLUSIONS: These findings suggest that FCOIs likely have little to no influence on the adoption of consensus recommendations regarding routine screening mammography for all cohorts of women.

Most UK scientists who publish extremely highly-cited papers do not secure funding from major public and charity funders: A descriptive analysis.

The UK is one of the largest funders of health research in the world, but little is known about how health funding is spent. Our study explores whether major UK public and charitable health research funders support the research of UK-based scientists producing the most highly-cited research. To address this question, we searched for UK-based authors of peer-reviewed papers that were published between January 2006 and February 2018 and received over 1000 citations in Scopus. We explored whether these authors have held a grant from the National Institute for Health Research (NIHR), the Medical Research Council (MRC) and the Wellcome Trust and compared the results with UK-based researchers who serve currently on the boards of these bodies. From the 1,370 papers relevant to medical, biomedical, life and health sciences with more than 1000 citations in the period examined, we identified 223 individuals from a UK institution at the time of publication who were either first/last or single authors. Of those, 164 are still in UK academic institutions, while 59 are not currently in UK academia (have left the country, are retired, or work in other sectors). Of the 164 individuals, only 59 (36%; 95% CI: 29-43%) currently hold an active grant from one of the three funders. Only 79 (48%; 95% CI: 41-56%) have held an active grant from any of the three funders between 2006-2017. Conversely, 457 of the 664 board members of MRC, Wellcome Trust, and NIHR (69%; 95% CI: 65-72%) have held an active grant in the same period by any of these funders. Only 7 out of 655 board members (1.1%) were first, last or single authors of an extremely highly-cited paper. There are many reasons why the majority of the most influential UK authors do not hold a grant from the country's major public and charitable funding bodies. Nevertheless, the results are worrisome and subscribe to similar patterns shown in the US. We discuss possible implications and suggest ways forward.

Science Without Publication Paywalls: cOAlition S for the Realisation of Full and Immediate Open Access.

In this Perspective, a group of national funders, joined by the European Commission and the European Research Council, announce plans to make Open Access publishing mandatory for recipients of their agencies' research funding.

Characteristics and publication fate of unregistered and retrospectively registered clinical trials submitted to The BMJ over 4 years.

OBJECTIVES: We sought to evaluate the characteristics and publication fate of improperly registered clinical trials submitted to a medical journal (The BMJ) over a 4-year period to identify common types of registration issues and their relation to publication outcomes. DESIGN: Research articles submitted to The BMJ and identified as unregistered or retrospectively registered by editors were included if they reported outcomes of a clinical trial. Relevant data regarding the trials were then extracted from each paper. Trials were categorised as prospectively registered, registered in an unapproved registry, unregistered or other, and explanations for registration deficiencies were grouped into six categories. We searched PubMed and Google to determine whether, where and when improperly registered studies were subsequently published and whether registration issues were disclosed. RESULTS: 123 research papers reporting apparently unregistered or retrospectively registered clinical trials were identified. 110 studies (89.4%) were retrospectively registered, nine (7.3%) were unregistered, three (2.4%) had been registered in an unapproved registry and one study originally lacking registration details was later discovered to have been prospectively registered. 82 studies (66.6%) were funded entirely or in part by government sources, and only seven studies (5.7%) received funding from industry. Of those papers submitted to The BMJ through the end of 2015, 67 of the 70 papers rejected for registration problems (95.7%) were subsequently published in another journal. The registration problem was disclosed in only 2 (2.9%). CONCLUSIONS: Improper registration remains a problem, particularly for clinical trials that are government or foundation-funded. Nonetheless, improperly registered trials are almost always published, suggesting that medical journal editors may not actively enforce registration requirements.

Favorable and publicly funded studies are more likely to be published: a systematic review and meta-analysis.

OBJECTIVES: The aim of this study was to identify and quantify the characteristics of studies associated with the likelihood of publication. STUDY DESIGN AND SETTING: We searched for manuscripts that tracked cohorts of clinical studies ("cohorts") that from launch to publication. We explored the association of study characteristics with the probability of publication via traditional meta-analyses and meta-regression using random effects models. RESULTS: The literature review identified 85 cohorts of studies that met our inclusion criteria. The probability of publication was significantly higher for studies whose characteristics were favorable (odds ratio [OR] = 2.04; 95% confidence interval [CI]: 1.62, 2.57) or statistically significant (OR = 2.07; 95% CI: 1.52, 2.81), had a multicenter design (OR = 1.32; 95% CI: 1.16, 1.45), and were of later regulatory phase (3/4 vs. 1/2, OR = 1.34; 95% CI: 1.14, 1.49). Industry funding was modestly associated with lower (OR = 0.81; 95% CI: 0.67, 0.99) probability of publication. An exploratory analysis of effect modification revealed that the effect of the study characteristic "favorable results" on likelihood for publication was stronger for industry-funded studies. CONCLUSION: The study characteristics of favorable and significant results were associated with greater probability of publication.

The cost of developing a computerized tailored interactive multimedia intervention vs. a print based Photonovella intervention for HPV vaccine education.

Mobile technology is opening new avenues for healthcare providers to create and implement tailored and personalized health education programs. We estimate and compare the cost of developing an i-Pad based tailored interactive multimedia intervention (TIMI) and a print based (Photonovella) intervention to increase human papillomavirus (HPV) immunization. The development costs of the interventions were calculated using a societal perspective. Direct cost included the cost of planning the study, conducting focus groups, and developing the intervention materials by the research staff. Costs also included the amount paid to the vendors who produced the TIMI and Photonovella. Micro cost data on the staff time and materials were recorded in logs for tracking personnel time, meeting time, supplies and software purchases. The costs were adjusted for inflation and reported in 2015 USD. The total cost of developing the Photonovella was $66,468 and the cost of developing the TIMI was $135,978. The amortized annual cost for the interventions calculated at a 3% discount rate and over a 7-year period was $10,669 per year for the Photonovella and $21,825 per year for the TIMI intervention. The results would inform decision makers when planning and investing in the development of interactive multimedia health interventions.