Individual, institutional, and scientific environment factors associated with questionable research practices in the reporting of messages and conclusions in scientific health services research publications.

BACKGROUND: Health Services Research findings (HSR) reported in scientific publications may become part of the decision-making process on healthcare. This study aimed to explore associations between researcher's individual, institutional, and scientific environment factors and the occurrence of questionable research practices (QRPs) in the reporting of messages and conclusions in scientific HSR publications. METHODS: We employed a mixed-methods study design. We identified factors possibly contributing to QRPs in the reporting of messages and conclusions through a literature review, 14 semi-structured interviews with HSR institutional leaders, and 13 focus-groups amongst researchers. A survey corresponding with these factors was developed and shared with 172 authors of 116 scientific HSR publications produced by Dutch research institutes in 2016. We assessed the included publications for the occurrence of QRPs. An exploratory factor analysis was conducted to identify factors within individual, institutional, and environmental domains. Next, we conducted bivariate analyses using simple Poisson regression to explore factors' association with the number of QRPs in the assessed HSR publications. Factors related to QRPs with a p-value < .30 were included in four multivariate models tested through a multiple Poisson regression. RESULTS: In total, 78 (45%) participants completed the survey (51.3% first authors and 48.7% last authors). Twelve factors were included in the multivariate analyses. In all four multivariate models, a higher score of "pressure to create societal impact" (Exp B = 1.28, 95% CI [1.11, 1.47]), was associated with higher number of QRPs. Higher scores on "specific training" (Exp B = 0.85, 95% CI [0.77-0.94]) and "co-author conflict of interest" (Exp B = 0.85, 95% CI [0.75-0.97]) factors were associated with a lower number of QRPs. Stratification between first and last authors indicated different factors were related to the occurrence of QRPs for these groups. CONCLUSION: Experienced pressure to create societal impact is associated with more QRPs in the reporting of messages and conclusions in HSR publications. Specific training in reporting messages and conclusions and awareness of co-author conflict of interests are related to fewer QRPs. Our results should stimulate awareness within the field of HSR internationally on opportunities to better support reporting in scientific HSR publications.

Reporting of Randomized Trials in Common Cancers in the Lay Media.

BACKGROUND: Limited data exist about the role of the lay media in the dissemination of results of randomized controlled trials (RCTs) in common cancers. METHODS: Completed phase III RCTs evaluating new drugs in common cancers between January 2005 and October 2016 were identified from ClinicalTrials.gov. Lay media reporting was identified by searching LexisNexis Academic. Scientific reporting was defined as presentation at an academic conference or publication in full. Associations between reporting in the lay media before scientific reporting and study design and sponsorship were evaluated using logistic regression. RESULTS: Of 180 RCTs identified, 52% were reported in the lay media and in 27%, lay media reporting occurred before scientific reporting with an increasing trend over time (p = 0.009). Reporting in the lay media before scientific reporting was associated with positive results (OR: 2.10, p = 0.04), targeted therapy compared to chemotherapy (OR: 4.75, p = 0.006), immunotherapy compared to chemotherapy (OR: 7.60, p = 0.02), and prostate cancer compared to breast cancer (OR: 3.25, p = 0.02). CONCLUSIONS: Over a quarter of all RCTs in common cancers are reported in the lay media before they are reported scientifically with an increasing proportion over time. Positive trials, studies in prostate cancer, and trials of immunotherapy are associated with early reporting in the lay media.

Improving interpretation of publically reported statistics on health and healthcare: the Figure Interpretation Assessment Tool (FIAT-Health).

BACKGROUND: Policy-makers, managers, scientists, patients and the general public are confronted daily with figures on health and healthcare through public reporting in newspapers, webpages and press releases. However, information on the key characteristics of these figures necessary for their correct interpretation is often not adequately communicated, which can lead to misinterpretation and misinformed decision-making. The objective of this research was to map the key characteristics relevant to the interpretation of figures on health and healthcare, and to develop a Figure Interpretation Assessment Tool-Health (FIAT-Health) through which figures on health and healthcare can be systematically assessed, allowing for a better interpretation of these figures. METHODS: The abovementioned key characteristics of figures on health and healthcare were identified through systematic expert consultations in the Netherlands on four topic categories of figures, namely morbidity, healthcare expenditure, healthcare outcomes and lifestyle. The identified characteristics were used as a frame for the development of the FIAT-Health. Development of the tool and its content was supported and validated through regular review by a sounding board of potential users. RESULTS: Identified characteristics relevant for the interpretation of figures in the four categories relate to the figures' origin, credibility, expression, subject matter, population and geographical focus, time period, and underlying data collection methods. The characteristics were translated into a set of 13 dichotomous and 4-point Likert scale questions constituting the FIAT-Health, and two final assessment statements. Users of the FIAT-Health were provided with a summary overview of their answers to support a final assessment of the correctness of a figure and the appropriateness of its reporting. CONCLUSIONS: FIAT-Health can support policy-makers, managers, scientists, patients and the general public to systematically assess the quality of publicly reported figures on health and healthcare. It also has the potential to support the producers of health and healthcare data in clearly communicating their data to different audiences. Future research should focus on the further validation of the tool in practice.

Health Literacy in Orthopedic Surgery: A Systematic Review.

Background: Limited health literacy has been associated with adverse health outcomes. Undergoing orthopedic surgery often requires patients to make complex decisions and adhere to complicated instructions, suggesting that health literacy skills might have a profound impact on orthopedic surgery outcomes. Purpose: We sought to review the literature for studies investigating the level of health literacy in patients undergoing orthopedic surgery and also to assess how those studies report factors affecting health equity. Methods: We conducted a systematic search of PubMed, Embase, and Cochrane Library for all health literacy studies published in the orthopedic surgery literature up to February 8, 2022. Search terms included synonyms for health literacy and for all orthopedic surgery subspecialties. Two reviewers independently extracted study data in addition to indicators of equity reporting using the PROGRESS+ checklist (Place of Residence, Race/Ethnicity, Occupation, Gender/sex, Religion, Education, Social capital, Socioeconomic status, plus age, disability, and sexual orientation). Results: The search resulted in 616 studies; 9 studies remained after exclusion criteria were applied. Most studies were of arthroplasty (4/9; 44%) or trauma (3/9; 33%) patients. Validated health literacy assessments were used in 4 of the included studies, and only 3 studies reported the rate of limited health literacy in the patients studied, which ranged between 34% and 38.5%. At least one PROGRESS+ item was reported in 88% (8/9) of the studies. Conclusions: We found a paucity of appropriately designed studies that used validated measures of health literacy in the field of orthopedic surgery. The potential impact of health literacy on orthopedic patients and their outcomes has yet to be elucidated. Thoughtful, high-quality trials across diverse demographics and geographies are warranted.

Poor statistical reporting, inadequate data presentation and spin persist despite Journal awareness and updated Information for Authors.

Sound reporting of research results is fundamental to good science. Unfortunately, poor reporting is common and does not improve with editorial educational strategies. We investigated whether publicly highlighting poor reporting at a journal can lead to improved reporting practices. We also investigated whether reporting practices that are required or strongly encouraged in journal Information for Authors are enforced by journal editors and staff. A 2016 audit highlighted poor reporting practices in the Journal of Neurophysiology. In August 2016 and 2018, the American Physiological Society updated the Information for Authors, which included the introduction of several required or strongly encouraged reporting practices. We audited Journal of Neurophysiology papers published in 2019 and 2020 (downloaded through the library of the University of New South Wales) on reporting items selected from the 2016 audit, the newly introduced reporting practices, and items from previous audits. Summary statistics (means, counts) were used to summarize audit results. In total, 580 papers were audited. Compared to results from the 2016 audit, several reporting practices remained unchanged or worsened. For example, 60% of papers erroneously reported standard errors of the mean, 23% of papers included undefined measures of variability, 40% of papers failed to define a statistical threshold for their tests, and when present, 64% of papers with p-values between 0.05 and 0.1 misinterpreted them as statistical trends. As for the newly introduced reporting practices, required practices were consistently adhered to by 34 to 37% of papers, while strongly encouraged practices were consistently adhered to by 9 to 26% of papers. Adherence to the other audited reporting practices was comparable to our previous audits. Publicly highlighting poor reporting practices did little to improve research reporting. Similarly, requiring or strongly encouraging reporting practices was only partly effective. Although the present audit focused on a single journal, this is likely not an isolated case. Stronger, more strategic measures are required to improve poor research reporting.

A Systematic Review of Propensity Score Matching in the Orthopedic Literature.

Background: Propensity score matching (PSM) is a statistical technique used to reduce bias in observational studies by controlling for measured confounders. Given its complexity and popularity, it is imperative that researchers comprehensively report their methodologies to ensure accurate interpretation and reproducibility. Purpose: This systematic review sought to define how often PSM has been used in recent orthopedic research and to describe how such studies reported their methods. Secondary aims included analyzing study reproducibility, bibliometric factors associated with reproducibility, and associations between methodology and the reporting of statistically significant results. Methods: PubMed and Embase databases were queried for studies containing "propensity score" and "match*" published in 20 orthopedic journals prior to 2020. All studies meeting inclusion criteria were used for trend analysis. Articles published between 2017 and 2019 were used for analysis of reporting quality and reproducibility. Results: In all, 261 studies were included for trend analysis, and 162 studies underwent full-text review. The proportion of orthopedic studies using PSM significantly increased over time. Seventy-one (41%) articles did not provide justification for covariate selection. The majority of studies illustrated covariate balance through P values. We found that 19% of the studies were fully reproducible. Most studies failed to report the use of replacement (67.3%) or independent or paired statistical methods (34.0%). Studies reporting standardized mean differences to illustrate covariate balance were less likely to report statistically significant results. Conclusion: Despite the increased use of PSM in orthopedic research, observational studies employing PSM have largely failed to adequately report their methodology.

Transparent scientific reporting is imperative during the pandemic.

The coronavirus pandemic has exposed not only the lack of preparation to combat the deadly disease, but also the nature of response by governments worldwide. This article analyses how some governments suppress science reporting in the Asia Pacific region during the pandemic. It also highlights how the political interference in science undermines liability and openness leading to the lack of freedom to express facts honestly.

Preliminary Minimum Reporting Requirements for In-Vivo Neural Interface Research: I. Implantable Neural Interfaces.

The pace of research and development in neuroscience, neurotechnology, and neurorehabilitation is rapidly accelerating, with the number of publications doubling every 4.2 years. Maintaining this progress requires technological standards and scientific reporting guidelines to provide frameworks for communication and interoperability. The present lack of such neurotechnology standards limits the transparency, repro-ducibility, and meta-analysis of this growing body of literature, posing an ongoing barrier to research, clinical, and commercial objectives. Continued neurotechnological innovation requires the development of some minimal standards to promote integration between this broad spectrum of technologies and therapies. To preserve design freedom and accelerate the translation of research into safe and effective technologies with maximal user benefit, such standards must be collaboratively co-developed by the full range of neuroscience and neurotechnology stakeholders. This paper summarizes the preliminary recommendations of IEEE P2794 Standards Working Group, developing a Reporting Standard for in-vivo Neural Interface Research (RSNIR).

The JASP guidelines for conducting and reporting a Bayesian analysis.

Despite the increasing popularity of Bayesian inference in empirical research, few practical guidelines provide detailed recommendations for how to apply Bayesian procedures and interpret the results. Here we offer specific guidelines for four different stages of Bayesian statistical reasoning in a research setting: planning the analysis, executing the analysis, interpreting the results, and reporting the results. The guidelines for each stage are illustrated with a running example. Although the guidelines are geared towards analyses performed with the open-source statistical software JASP, most guidelines extend to Bayesian inference in general.

Toward Good In Vitro Reporting Standards.

A good experiment reported badly is worthless. Meaningful contributions to the body of science are made by sharing the full methodology and results so that they can be evaluated and reproduced by peers. Erroneous and incomplete reporting does not do justice to the resources spent on conducting the experiment and the time peers spend reading the article. In theory peer-review should ensure adequate reporting - in practice it does not. Many areas have developed reporting standards and checklists to support the adequate reporting of scientific efforts, but in vitro research still has no generally accepted criteria. It is characterized by a "Wild West" or "anything goes" attitude. Such a culture may undermine trust in the reproducibility of animal-free methods, and thus parallel the "reproducibility crisis" discussed for other life science fields. The increasing data retrieval needs of computational approaches (in extreme as "big data" and artificial intelligence) makes reporting quality even more important so that the scientific community can take full advantage of the results. The first priority of reporting standards is to ensure the completeness and transparency of information provided (data focus). The second tier is a quality of data display that makes information digestible and easy to grasp, compare and further analyze (information focus). This article summarizes a series of initiatives geared towards improving the quality of in vitro work and its reporting. This shall ultimately lead to Good In Vitro Reporting Standards (GIVReSt).

Occurrence and nature of questionable research practices in the reporting of messages and conclusions in international scientific Health Services Research publications: a structured assessment of publications authored by researchers in the Netherlands.

OBJECTIVES: Explore the occurrence and nature of questionable research practices (QRPs) in the reporting of messages and conclusions in international scientific Health Services Research (HSR) publications authored by researchers from HSR institutions in the Netherlands. DESIGN: In a joint effort to assure the overall quality of HSR publications in the Netherlands, 13 HSR institutions in the Netherlands participated in this study. Together with these institutions, we constructed and validated an assessment instrument covering 35 possible QRPs in the reporting of messages and conclusions. Two reviewers independently assessed a random sample of 116 HSR articles authored by researchers from these institutions published in international peer-reviewed scientific journals in 2016. SETTING: Netherlands, 2016. SAMPLE: 116 international peer-reviewed HSR publications. MAIN OUTCOME MEASURES: Median number of QRPs per publication, the percentage of publications with observed QRP frequencies, occurrence of specific QRPs and difference in total number of QRPs by methodological approach, type of research and study design. RESULTS: We identified a median of six QRPs per publication out of 35 possible QRPs. QRPs occurred most frequently in the reporting of implications for practice, recommendations for practice, contradictory evidence, study limitations and conclusions based on the results and in the context of the literature. We identified no differences in total number of QRPs in papers based on different methodological approach, type of research or study design. CONCLUSIONS: Given the applied nature of HSR, both the severity of the identified QRPs, and the recommendations for policy and practice in HSR publications warrant discussion. We recommend that the HSR field further define and establish its own scientific norms in publication practices to improve scientific reporting and strengthen the impact of HSR. The results of our study can serve as an empirical basis for continuous critical reflection on the reporting of messages and conclusions.

Impact of peer review on discussion of study limitations and strength of claims in randomized trial reports: a before and after study.

BACKGROUND: In their research reports, scientists are expected to discuss limitations that their studies have. Previous research showed that often, such discussion is absent. Also, many journals emphasize the importance of avoiding overstatement of claims. We wanted to see to what extent editorial handling and peer review affects self-acknowledgment of limitations and hedging of claims. METHODS: Using software that automatically detects limitation-acknowledging sentences and calculates the level of hedging in sentences, we compared the submitted manuscripts and their ultimate publications of all randomized trials published in 2015 in 27 BioMed Central (BMC) journals and BMJ Open. We used mixed linear and logistic regression models, accounting for clustering of manuscript-publication pairs within journals, to quantify before-after changes in the mean numbers of limitation-acknowledging sentences, in the probability that a manuscript with zero self-acknowledged limitations ended up as a publication with at least one and in hedging scores. RESULTS: Four hundred forty-six manuscript-publication pairs were analyzed. The median number of manuscripts per journal was 10.5 (interquartile range 6-18). The average number of distinct limitation sentences increased by 1.39 (95% CI 1.09-1.76), from 2.48 in manuscripts to 3.87 in publications. Two hundred two manuscripts (45.3%) did not mention any limitations. Sixty-three (31%, 95% CI 25-38) of these mentioned at least one after peer review. Changes in mean hedging scores were negligible. CONCLUSIONS: Our findings support the idea that editorial handling and peer review lead to more self-acknowledgment of study limitations, but not to changes in linguistic nuance.

How are risk ratios reported in orthopaedic surgery journals? A descriptive study of formats used to report absolute risks.

PURPOSE: The numerical format in which risks are communicated can affect risk comprehension and perceptions of medical professionals. We investigated what numerical formats are used to report absolute risks in empirical articles, estimated the frequency of biasing formats and rated the quality of figures used to display the risks. DESIGN: Descriptive study of reporting practices. METHOD: We randomly sampled articles published in seven leading orthopaedic surgery journals during a period of 13 years. From these, we selected articles that reported group comparisons on a binary outcome (eg, revision rates in two groups) and recorded the numerical format used to communicate the absolute risks in the results section. The quality of figures was assessed according to published guidelines for transparent visual aids design. OUTCOME MEASURES: Prevalence of information formats and quality of figures. RESULTS: The final sample consisted of 507 articles, of which 14% reported level 1 evidence, 13% level 2 and 73% level 3 or lower. The majority of articles compared groups of different sizes (90%), reported both raw numbers and percentages (64%) and did not report the group sizes alongside (50%). Fifteen per cent of articles used two formats identified as biasing: only raw numbers (8%, '90 patients vs 100 patients') or raw numbers reported alongside different group sizes (7%, '90 out of 340 patients vs 100 out of 490 patients'). The prevalence of these formats decreased in more recent publications. Figures (n=79) had on average two faults that could distort comprehension, and the majority were rated as biasing. CONCLUSION: Authors use a variety of formats to report absolute risks in scientific articles and are likely not aware of how some formats and graph design features can distort comprehension. Biases can be reduced if journals adopt guidelines for transparent risk communication but more research is needed into the effects of different formats.

Minimum Reporting Standards for Adult Cochlear Implantation.

This article outlines new minimum standards for reporting adult cochlear implant outcomes. These standards have been endorsed by the Implantable Hearing Devices Committee and the Hearing Committee of the American Academy of Otolaryngology-Head and Neck Surgery. The lack of a standardized method for reporting outcomes following cochlear implantation in clinical trials has hampered the ability of investigators to draw comparisons across studies. Variability in data reported in articles and presentation formats inhibits meta-analyses, making it impossible to accumulate the large patient cohorts needed for statistically significant inference. While investigators remain unrestricted in publishing their adult cochlear implant outcome data in additional formats that they believe to be valuable, they should include the presently proposed minimal data set to facilitate interstudy comparability and consistency of reporting.

Beyond sex and gender difference in funding and reporting of health research.

BACKGROUND: Understanding sex and gender in health research can improve the quality of scholarship and enhance health outcomes. Funding agencies and academic journals are two key gatekeepers of knowledge production and dissemination, including whether and how sex/gender is incorporated into health research. Though attention has been paid to key issues and practices in accounting for sex/gender in health funding agencies and academic journals, to date, there has been no systematic analysis documenting whether and how agencies and journals require attention to sex/gender, what conceptual explanations and practical guidance are given for such inclusion, and whether existing practices reflect the reality that sex/gender cannot be separated from other axes of inequality. METHODS: Our research systematically examines official statements about sex/gender inclusion from 45 national-level funding agencies that fund health research across 36 countries (covering the regions of the EU and associated countries, North America, and Australia) and from ten top-ranking general health (the top five in "science" and the top five in "social science") and ten sex- and/or gender-related health journals. We explore the extent to which agencies and journals require inclusion of sex/gender considerations and to what extent existing strategies reflect state of the art understandings of sex/gender, including intersectional perspectives. RESULTS: The research highlights the following: (a) there is no consistency in whether sex/gender are mentioned in funding and publishing guidelines; (b) there is wide variation in how sex/gender are conceptualized and how researchers are asked to address the inclusion/exclusion of sex/gender in research; (c) funding agencies tend to prioritize male/female equality in research teams and funding outcomes over considerations of sex/gender in research content and knowledge production; and (d) with very few exceptions, agency and journal criteria fail to recognize the complexity of sex/gender, including the intersection of sex/gender with other key factors that shape health. CONCLUSIONS: The conceptualization and integration of sex/gender needs to better capture the interacting and complex factors that shape health-an imperative that can be informed by an intersectional approach. This can strengthen current efforts to advance scientific excellence in the production and reporting of research. We provide recommendations and supporting questions to strengthen consideration of sex/gender in policies and practices of health journals and funding agencies.

Perspective: An Extension of the STROBE Statement for Observational Studies in Nutritional Epidemiology (STROBE-nut): Explanation and Elaboration.

Nutritional epidemiology is an inherently complex and multifaceted research area. Dietary intake is a complex exposure and is challenging to describe and assess, and links between diet, health, and disease are difficult to ascertain. Consequently, adequate reporting is necessary to facilitate comprehension, interpretation, and generalizability of results and conclusions. The STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) statement is an international and collaborative initiative aiming to enhance the quality of reporting of observational studies. We previously presented a checklist of 24 reporting recommendations for the field of nutritional epidemiology, called "the STROBE-nut." The STROBE-nut is an extension of the general STROBE statement, intended to complement the STROBE recommendations to improve and standardize the reporting in nutritional epidemiology. The aim of the present article is to explain the rationale for, and elaborate on, the STROBE-nut recommendations to enhance the clarity and to facilitate the understanding of the guidelines. Examples from the published literature are used as illustrations, and references are provided for further reading.

Design, analysis, and conclusions: Telling a consistent causal story.

Pharmacy educators are dedicated to providing the best education to student pharmacists and post-graduate trainees. This involves developing and evaluating new educational approaches or activities, as well as working to understand better other factors that may affect student and program outcomes. Although we may not realize it, the idea of causality, or the presence of a cause-and-effect relationship, is embedded in much of what we do when engaging in pharmacy educational scholarship. Saying that an educational intervention is effective at improving test scores implies that the intervention caused the increase. Perhaps more subtly, a cause-and-effect relationship is implied when identifying poor interviewing skills as a factor that reduces the likelihood that a student will obtain a pharmacy practice residency. The extent to which causal claims can be made depends on appropriate study design and analysis. Similarly, the language used to present these claims is related to study design and analysis. Unfortunately, study designs used when evaluating educational approaches do not always align with the language used to report the results. This review provides a brief overview of current thinking related to causal inference. The role of study design and analysis in causal inference is also discussed along with recommendations for study design and analysis to facilitate making appropriate causal claims. After reading this article, readers should be able to select a study design that best aligns with their particular research question and report their findings in ways that are faithful to the strengths of the study design.