Analysis of 567,758 randomized controlled trials published over 30 years reveals trends in phrases used to discuss results that do not reach statistical significance.

The power of language to modify the reader's perception of interpreting biomedical results cannot be underestimated. Misreporting and misinterpretation are pressing problems in randomized controlled trials (RCT) output. This may be partially related to the statistical significance paradigm used in clinical trials centered around a P value below 0.05 cutoff. Strict use of this P value may lead to strategies of clinical researchers to describe their clinical results with P values approaching but not reaching the threshold to be "almost significant." The question is how phrases expressing nonsignificant results have been reported in RCTs over the past 30 years. To this end, we conducted a quantitative analysis of English full texts containing 567,758 RCTs recorded in PubMed between 1990 and 2020 (81.5% of all published RCTs in PubMed). We determined the exact presence of 505 predefined phrases denoting results that approach but do not cross the line of formal statistical significance (P < 0.05). We modeled temporal trends in phrase data with Bayesian linear regression. Evidence for temporal change was obtained through Bayes factor (BF) analysis. In a randomly sampled subset, the associated P values were manually extracted. We identified 61,741 phrases in 49,134 RCTs indicating almost significant results (8.65%; 95% confidence interval (CI): 8.58% to 8.73%). The overall prevalence of these phrases remained stable over time, with the most prevalent phrases being "marginally significant" (in 7,735 RCTs), "all but significant" (7,015), "a nonsignificant trend" (3,442), "failed to reach statistical significance" (2,578), and "a strong trend" (1,700). The strongest evidence for an increased temporal prevalence was found for "a numerical trend," "a positive trend," "an increasing trend," and "nominally significant." In contrast, the phrases "all but significant," "approaches statistical significance," "did not quite reach statistical significance," "difference was apparent," "failed to reach statistical significance," and "not quite significant" decreased over time. In a random sampled subset of 29,000 phrases, the manually identified and corresponding 11,926 P values, 68,1% ranged between 0.05 and 0.15 (CI: 67. to 69.0; median 0.06). Our results show that RCT reports regularly contain specific phrases describing marginally nonsignificant results to report P values close to but above the dominant 0.05 cutoff. The fact that the prevalence of the phrases remained stable over time indicates that this practice of broadly interpreting P values close to a predefined threshold remains prevalent. To enhance responsible and transparent interpretation of RCT results, researchers, clinicians, reviewers, and editors may reduce the focus on formal statistical significance thresholds and stimulate reporting of P values with corresponding effect sizes and CIs and focus on the clinical relevance of the statistical difference found in RCTs.

Designing and implementing a research integrity promotion plan: Recommendations for research funders.

Various stakeholders in science have put research integrity high on their agenda. Among them, research funders are prominently placed to foster research integrity by requiring that the organizations and individual researchers they support make an explicit commitment to research integrity. Moreover, funders need to adopt appropriate research integrity practices themselves. To facilitate this, we recommend that funders develop and implement a Research Integrity Promotion Plan (RIPP). This Consensus View offers a range of examples of how funders are already promoting research integrity, distills 6 core topics that funders should cover in a RIPP, and provides guidelines on how to develop and implement a RIPP. We believe that the 6 core topics we put forward will guide funders towards strengthening research integrity policy in their organization and guide the researchers and research organizations they fund.

The methodological quality of 176,620 randomized controlled trials published between 1966 and 2018 reveals a positive trend but also an urgent need for improvement.

Many randomized controlled trials (RCTs) are biased and difficult to reproduce due to methodological flaws and poor reporting. There is increasing attention for responsible research practices and implementation of reporting guidelines, but whether these efforts have improved the methodological quality of RCTs (e.g., lower risk of bias) is unknown. We, therefore, mapped risk-of-bias trends over time in RCT publications in relation to journal and author characteristics. Meta-information of 176,620 RCTs published between 1966 and 2018 was extracted. The risk-of-bias probability (random sequence generation, allocation concealment, blinding of patients/personnel, and blinding of outcome assessment) was assessed using a risk-of-bias machine learning tool. This tool was simultaneously validated using 63,327 human risk-of-bias assessments obtained from 17,394 RCTs evaluated in the Cochrane Database of Systematic Reviews (CDSR). Moreover, RCT registration and CONSORT Statement reporting were assessed using automated searches. Publication characteristics included the number of authors, journal impact factor (JIF), and medical discipline. The annual number of published RCTs substantially increased over 4 decades, accompanied by increases in authors (5.2 to 7.8) and institutions (2.9 to 4.8). The risk of bias remained present in most RCTs but decreased over time for allocation concealment (63% to 51%), random sequence generation (57% to 36%), and blinding of outcome assessment (58% to 52%). Trial registration (37% to 47%) and the use of the CONSORT Statement (1% to 20%) also rapidly increased. In journals with a higher impact factor (>10), the risk of bias was consistently lower with higher levels of RCT registration and the use of the CONSORT Statement. Automated risk-of-bias predictions had accuracies above 70% for allocation concealment (70.7%), random sequence generation (72.1%), and blinding of patients/personnel (79.8%), but not for blinding of outcome assessment (62.7%). In conclusion, the likelihood of bias in RCTs has generally decreased over the last decades. This optimistic trend may be driven by increased knowledge augmented by mandatory trial registration and more stringent reporting guidelines and journal requirements. Nevertheless, relatively high probabilities of bias remain, particularly in journals with lower impact factors. This emphasizes that further improvement of RCT registration, conduct, and reporting is still urgently needed.

Web Search Query Volume Correlates With Prescription Volumes of Antidepressants and Antipsychotics in the Netherlands and United Kingdom: An Explorative Study.

BACKGROUND: The significant increase in Internet availability has resulted in a rise in search queries on health-related topics. Previous research has demonstrated the potential for analyzing web search query volume for nonpsychotropic prescription drugs, while studies on psychotropic drugs remain scarce. The aims of this study were to expand upon this scarce knowledge by investigating the relationship between web search query volumes and prescription volumes of antidepressants and antipsychotics in the United Kingdom and the Netherlands and to gain insight in topics of concern, such as withdrawal symptoms and discontinuation. METHODS: Data were obtained for the United Kingdom and the Netherlands from January 2010 until January 2021. Prescription volume data for 5 antidepressants (paroxetine, fluoxetine, sertraline, citalopram, venlafaxine) and 5 antipsychotics (quetiapine, olanzapine, clozapine, aripiprazole, and risperidone) were obtained. Web search query volumes and data on related search queries of these substances were acquired from Google Trends. Descriptive statistics and Pearson correlation analyses were performed. RESULTS: A strong, positive, and statistically significant correlation between web search query volume and prescription volume was observed for most included substances in both the Netherlands and the United Kingdom. The search queries related to the included antidepressants and antipsychotics indicate important topics of concern for specific substances, such as withdrawal symptoms and discontinuation. CONCLUSIONS: Web search data from Google Trends could potentially be used as a proxy for prescribing trends of antidepressants and antipsychotics and to gain insight in topics of concern of users of these substances. These findings highlight the importance of providing reliable patient information, particularly regarding adverse effects, withdrawal, and discontinuation.

Risk of suicidality in mental and neurological disorders in low and middle-income countries: A systematic review and meta-analysis.

BACKGROUND: Both fatal and nonfatal suicidal behaviours are important complications of mental, neurological, and substance use disorders (MNSDs) worldwide. We aimed at quantifying the association of suicidal behaviour with MNSDs in Low and Middle Income Countries (LMICs) where varying environmental and socio-cultural factors may impact outcome. METHODS: We conducted a systematic review and meta-analysis to report the associations between MNSDs and suicidality in LMICs and the study-level factors of these associations. We searched the following electronic databases: PUBMED, PsycINFO, MEDLINE, CINAHL, World Cat, and Cochrane library for studies on suicide risk in MNSDs, with a comparison/control group of persons without MNSDs, published from January 1, 1995 to September 3, 2020. Median estimates were calculated for relative risks for suicide behaviour and MNSDs, and when appropriate, these were pooled using random effects metanalytic model. This study was registered with PROSPERO, CRD42020178772. RESULTS: The search identified 73 eligible studies: 28 were used for quantitative synthesis of estimates and 45 for description of risk factors. Studies included came from low and upper middle-income countries with a majority of these from Asia and South America and none from a low-income country. The sample size was 13,759 for MNSD cases and 11,792 hospital or community controls without MNSD. The most common MNSD exposure for suicidal behaviour was depressive disorders (47 studies (64%)), followed by schizophrenia spectrum, and other psychotic disorders (28 studies (38%)). Pooled estimates from the meta-analysis were statistically significant for suicidal behaviour with any MNSDs (odds ratios (OR) = 1∙98 (95%CI = 1∙80-2∙16))) and depressive disorder (OR = 3∙26 (95%CI = 2∙88-3∙63))), with both remaining significant after inclusion of high-quality studies only. Meta-regression identified only hospital-based studies (ratio of OR = 2∙85, CI:1∙24-6∙55) and sample size (OR = 1∙00, CI:0∙99-1∙00) as possible sources of variability in estimates. Risk for suicidal behaviour in MNSDs was increased by demographic factors (e.g., male sex, and unemployment), family history, psychosocial context and physical illness. INTERPRETATION: There is an association between suicidal behaviour and MNSDs in LMICs, the association is greater for depressive disorder in LMICs than what has been reported in High Income Countries (HICs). There is urgent need to improve access for MNSDs care in LMICs. FUNDING: None.

Co-creating Research Integrity Education Guidelines for Research Institutions.

To foster research integrity (RI), research institutions should develop a continuous RI education approach, addressing various target groups. To support institutions to achieve this, we developed RI education guidelines together with RI experts and research administrators, exploring similarities and differences in recommendations across target groups, as well as recommendations about RI education using approaches other than formal RI training. We used an iterative co-creative process. We conducted four half-day online co-creation workshops with 16 participants in total, which were informed by the RI education evidence-base. In the first two workshops, participants generated ideas for guidelines' content, focusing on different target groups and various approaches to RI education. Based on this content we developed first drafts of the guidelines. Participants in the third and fourth workshop refined those drafts. We next organized a working group which further prioritized, reorganized, and optimized the content of the guidelines. We developed four guidelines on RI education focusing on (a) bachelor, master and PhD students; (b) post-doctorate and senior researchers; (c) other RI stakeholders; as well as (d) continuous RI education. Across guidelines, we recommend mandatory RI training; follow-up refresher training; informal discussions about RI; appropriate rewards and incentives for active participation in RI education; and evaluation of RI educational events. Our work provides experience-based co-created guidance to research institutions on what to consider when developing a successful RI education strategy. Each guideline is offered as a distinct, publicly available tool in our toolbox ( www.sops4ri.eu/toolbox ) which institutions can access, adapt and implement to meet their institution-specific RI education needs.Trial registration https://osf.io/zej5b .

Practices for Research Integrity Promotion in Research Performing Organisations and Research Funding Organisations: A Scoping Review.

Research integrity (RI) is a continuously developing concept, and increasing emphasis is put on creating RI promotion practices. This study aimed to map the existing RI guidance documents at research performing organisations (RPOs) and research funding organisations (RFOs). A search of bibliographic databases and grey literature sources was performed, and retrieved documents were screened for eligibility. The search of bibliographical databases and reference lists of selected articles identified a total of 92 documents while the search of grey literature sources identified 118 documents for analysis. The retrieved documents were analysed based on their geographical origin, research field and organisational origin (RPO or RFO) of RI practices, types of guidance presented in them, and target groups to which RI practices are directed. Most of the identified practices were developed for research in general, and are applicable to all research fields (n = 117) and medical sciences (n = 78). They were mostly written in the form of guidelines (n = 136) and targeted researchers (n = 167). A comprehensive search of the existing RI promotion practices showed that initiatives mostly come from RPOs while only a few RI practices originate from RFOs. This study showed that more RI guidance documents are needed for natural sciences, social sciences, and humanities since only a small number of documents was developed specifically for these research fields. The explored documents and the gaps in knowledge identified in this study can be used for further development of RI promotion practices in RPOs and RFOs.

Expanding Research Integrity: A Cultural-Practice Perspective.

Research integrity (RI) is usually discussed in terms of responsibilities that individual researchers bear towards the scientific work they conduct, as well as responsibilities that institutions have to enable those individual researchers to do so. In addition to these two bearers of responsibility, a third category often surfaces, which is variably referred to as culture and practice. These notions merit further development beyond a residual category that is to contain everything that is not covered by attributions to individuals and institutions. This paper discusses how thinking in RI can take benefit from more specific ideas on practice and culture. We start by articulating elements of practice and culture, and explore how values central to RI are related to these elements. These insights help identify additional points of intervention for fostering responsible conduct. This helps to build "cultures and practices of research integrity", as it makes clear that specific times and places are connected to specific practices and cultures and should have a place in the debate on Research Integrity. With this conceptual framework, practitioners as well as theorists can avoid using the notions as residual categories that de facto amount to vague, additional burdens of responsibility for the individual.

Important Topics for Fostering Research Integrity by Research Performing and Research Funding Organizations: A Delphi Consensus Study.

To foster research integrity (RI), it is necessary to address the institutional and system-of-science factors that influence researchers' behavior. Consequently, research performing and research funding organizations (RPOs and RFOs) could develop comprehensive RI policies outlining the concrete steps they will take to foster RI. So far, there is no consensus on which topics are important to address in RI policies. Therefore, we conducted a three round Delphi survey study to explore which RI topics to address in institutional RI policies by seeking consensus from research policy experts and institutional leaders. A total of 68 RPO and 52 RFO experts, representing different disciplines, countries and genders, completed one, two or all rounds of the study. There was consensus among the experts on the importance of 12 RI topics for RPOs and 11 for RFOs. The topics that ranked highest for RPOs concerned education and training, supervision and mentoring, dealing with RI breaches, and supporting a responsible research process (e.g. through quality assurance). The highest ranked RFO topics concerned dealing with breaches of RI, conflicts of interest, and setting expectations on RPOs (e.g. about educating researchers about RI). Together with the research policy experts and institutional leaders, we developed a comprehensive overview of topics important for inclusion in the RI policies of RPOs and RFOs. The topics reflect preference for a preventative approach to RI, coupled with procedures for dealing with RI breaches. RPOs and RFOs should address each of these topics in order to support researchers in conducting responsible research.

Safe and informed prescribing of psychotropic medication during the COVID-19 pandemic.

Treatment with psychotropic medication may sometimes be jeopardised because of the COVID-19 pandemic. One underlying reason is the lack of COVID-19-specific psychopharmacology guidelines. Here, we discuss five considerations arising from our clinical experience and pharmacological background knowledge to enable safe and well-informed psychopharmacotherapy during the COVID-19 pandemic.

Making researchers responsible: attributions of responsibility and ambiguous notions of culture in research codes of conduct.

BACKGROUND: Research codes of conduct offer guidance to researchers with respect to which values should be realized in research practices, how these values are to be realized, and what the respective responsibilities of the individual and the institution are in this. However, the question of how the responsibilities are to be divided between the individual and the institution has hitherto received little attention. We therefore performed an analysis of research codes of conduct to investigate how responsibilities are positioned as individual or institutional, and how the boundary between the two is drawn. METHOD: We selected 12 institutional, national and international codes of conduct that apply to medical research in the Netherlands and subjected them to a close-reading content analysis. We first identified the dominant themes and then investigated how responsibility is attributed to individuals and institutions. RESULTS: We observed that the attribution of responsibility to either the individual or the institution is often not entirely clear, and that the notion of culture emerges as a residual category for such attributions. We see this notion of responsible research cultures as important; it is something that mediates between the individual level and the institutional level. However, at the same time it largely lacks substantiation. CONCLUSIONS: While many attributions of individual and institutional responsibility are clear, the exact boundary between the two is often problematic. We suggest two possible avenues for improving codes of conduct: either to clearly attribute responsibilities to individuals or institutions and depend less on the notion of culture, or to make culture a more explicit concern and articulate what it is and how a good culture might be fostered.

Researchers' Perceptions of a Responsible Research Climate: A Multi Focus Group Study.

The research climate plays a key role in fostering integrity in research. However, little is known about what constitutes a responsible research climate. We investigated academic researchers' perceptions on this through focus group interviews. We recruited researchers from the Vrije Universiteit Amsterdam and the Amsterdam University Medical Center to participate in focus group discussions that consisted of researchers from similar academic ranks and disciplinary fields. We asked participants to reflect on the characteristics of a responsible research climate, the barriers they perceived and which interventions they thought fruitful to improve the research climate. Discussions were recorded and transcribed at verbatim. We used inductive content analysis to analyse the focus group transcripts. We conducted 12 focus groups with 61 researchers in total. We identified fair evaluation, openness, sufficient time, integrity, trust and freedom to be mentioned as important characteristics of a responsible research climate. Main perceived barriers were lack of support, unfair evaluation policies, normalization of overwork and insufficient supervision of early career researchers. Possible interventions suggested by the participants centered around improving support, discussing expectations and improving the quality of supervision. Some of the elements of a responsible research climate identified by participants are reflected in national and international codes of conduct, such as trust and openness. Although it may seem hard to change the research climate, we believe that the realisation that the research climate is suboptimal should provide the impetus for change informed by researchers' experiences and opinions.

The effects of industry funding and positive outcomes in the interpretation of clinical trial results: a randomized trial among Dutch psychiatrists.

BACKGROUND: Most studies are inclined to report positive rather than negative or inconclusive results. It is currently unknown how clinicians appraise the results of a randomized clinical trial. For example, how does the study funding source influence the appraisal of an RCT, and do positive findings influence perceived credibility and clinical relevance? This study investigates whether psychiatrists' appraisal of a scientific abstract is influenced by industry funding disclosures and a positive outcome. METHODS: Dutch psychiatrists were randomized to evaluate a scientific abstract describing a fictitious RCT for a novel antipsychotic drug. Four different abstracts were created reporting either absence or presence of industry funding disclosure as well as a positive or a negative outcome. Primary outcomes were the perceived credibility and clinical relevance of the study results (10-point Likert scale). Secondary outcomes were the assessment of methodological quality and interest in reading the full article. RESULTS: Three hundred ninety-five psychiatrists completed the survey (completion rate 45%). Industry funding disclosure was found not to influence perceived credibility (Mean Difference MD 0.12; 95% CI - 0.28 to 0.47, p?) nor interpretation of its clinical relevance (MD 0.14; 95% CI - 0.54 to 0.27, p?). A negative outcome was perceived as more credible than a positive outcome (MD 0.81 points; 95% Confidence Interval (CI) 0.43 to 1.18, p?), but did not affect clinical relevance scores (MD -0.14; 95% CI - 0.54 to 0.27). CONCLUSIONS: In this study, industry funding disclosure was not associated with the perceived credibility nor judgement of clinical relevance of a fictional RCT by psychiatrists. Positive study outcomes were found to be less credible compared to negative outcomes, but industry funding had no significant effects. Psychiatrists may underestimate the influence of funding sources on research results. The fact that physicians indicated negative outcomes to be more credible may point to more awareness of existing publication bias in the scientific literature.

Emotional exhaustion and burnout among medical professors; a nationwide survey.

BACKGROUND: Although job-related burnout and its core feature emotional exhaustion are common among medical professionals and compromise job satisfaction and professional performance, they have never been systematically studied in medical professors, who have central positions in academic medicine. METHODS: We performed an online nationwide survey inviting all 1206 medical professors in The Netherlands to participate. They were asked to fill out the Maslach Burnout Inventory, a 'professional engagement' inventory, and to provide demographic and job-specific data. RESULTS: A total of 437 Professors completed the questionnaire. Nearly one quarter (23.8%) scored above the cut-off used for the definition of emotional exhaustion. Factors related to being in an early career stage (i.e. lower age, fewer years since appointment, having homeliving children, having a relatively low Hirsch index) were significantly associated with higher emotional exhaustion scores. There was a significant inverse correlation between emotional exhaustion and the level of professional engagement. CONCLUSIONS: Early career medical professors have higher scores on emotional exhaustion and may be prone for developing burnout. Based upon this finding, preventive strategies to prevent burnout could be targeted to young professors.

Superb supervision: A pilot study on training supervisors to convey responsible research practices onto their PhD candidates.

One way to strengthen research integrity, is through supervision. According to previous research, a supervisor should be well-versed in responsible research practices (RRPs) and possess the necessary interpersonal skills to convey RRPs. We developed a 3-day pilot training for PhD supervisors that combined RRPs and interpersonal skills. Our aim was to assess: perceptions regarding supervision skills (before and after the pilot) and participants' views on combining RRPs and interpersonal skills. Before and after the pilot, we sent the Research Supervision Quality Evaluation survey to the participating PhD supervisors and their PhD candidates. The pilot was concluded with a focus group where participants deliberated over the combination of training in interpersonal skills and RRPs and whether such training should become compulsory. Both supervisors and PhD candidates were more positive about the supervisor's interpersonal skills and the ability to foster RRPs after the training. Participants were enthusiastic about the training's dual focus but believed that making the training compulsory would be undesirable. The results highlight the potential of RRPs training for supervisors. However, caution is warranted, as the results regard a small sample of volunteering supervisors, underscoring the need for larger programs to foster responsible supervision that are rigorously evaluated.

[Mental health and climate change: reconceptualizing eco-anxiety]. / Bestaat er zoiets als eco-angst?

Climate change may bring about anxiety, which may be referred to as eco-anxiety. Commonly accepted conceptual or diagnostic criteria for eco-anxiety are currently lacking. Here, we briefly summarize the current literature on climate change and mental illness. We suggest dividing the concept of eco-anxiety into adaptive eco-anxiety and an anxiety disorder where climate change plays a major role. This distinction may be helpful in clinical practice to discern relatively common and potentially healthy eco-anxiety from a disorder causing impairment in daily functioning. Benefits of adaptive eco-anxiety include the development of active coping strategies (increasing resilience) as well as behavioural changes to mitigate climate change. When debilitating anxiety comes with avoidance and centers around climate change, a specific phobia called eco-anxiety disorder may be considered. Importantly, as validated diagnostic criteria for this disorder are currently lacking, further conceptualization is highly needed. Future clinical research may help fill these current knowledge gaps.

How to combine rules and commitment in fostering research integrity?

Research integrity (RI) is crucial for trustworthy research. Rules are important in setting RI standards and improving research practice, but they can lead to increased bureaucracy; without commensurate commitment amongst researchers toward RI, they are unlikely to improve research practices. In this paper, we explore how to combine rules and commitment in fostering RI. Research institutions can govern RI using markets (using incentives), bureaucracies (using rules), and network processes (through commitment and agreements). Based on Habermas' Theory of Communicative Action, we argue that network processes, as part of the lifeworld, can legitimize systems - that is, market or bureaucratic governance modes. This can regulate and support RI practices in an efficient way. Systems can also become dominant and repress consensus processes. Fostering RI requires a balance between network, market and bureaucratic governance modes. We analyze the institutional response to a serious RI case to illustrate how network processes can be combined with bureaucratic rules. Specifically, we analyze how the Science Committee established at Tilburg University in 2012 has navigated different governance modes, resulting in a normatively grounded and efficient approach to fostering RI. Based on this case, we formulate recommendations to research institutions on how to combine rules and commitment.

In Defense of the Netherlands Code of Conduct for Research Integrity: Response to Radder.

We assess Radder's criticisms of the Netherlands Code of Conduct for Research Integrity and show that they either miss their mark or depend on controversial background assumptions about the purpose of the Code. Although Radder raises important questions about the broader roles and purposes of research in society, his conclusion that the Code should be revised in the ways he proposes is unjustified.

Using co-creation methods for research integrity guideline development - how, what, why and when?

Existing research integrity (RI) guideline development methods are limited in including various perspectives. While co-creation methods could help to address this, there is little information available to researchers and practitioners on how, why and when to use co-creation for developing RI guidelines, nor what the outcomes of co-creation methods are. In this paper, we aim to address this gap. First, we discuss how co-creation methods can be used for RI guideline development, based on our experience of developing RI guidelines. We elaborate on steps including preparation of the aims and design; participant sensitization; organizing and facilitating workshops; and analyzing data and translating them into guidelines. Secondly, we present the resulting RI guidelines, to show what the outcome of co-creation methods are. Thirdly, we reflect on why and when researchers might want to use co-creation methods for developing RI guidelines. We discuss that stakeholder engagement and inclusion of diverse perspectives are key strengths of co-creation methods. We also reflect that co-creation methods have the potential to make guidelines implementable if followed by additional steps such as revision working groups. We conclude that co-creation methods are a valuable approach to creating new RI guidelines when used together with additional methods.