How can meta-research be used to evaluate and improve the quality of research in the field of traditional, complementary, and integrative medicine?

The field of traditional, complementary, and integrative medicine (TCIM) has garnered increasing attention due to its holistic approach to health and well-being. While the quantity of published research about TCIM has increased exponentially, critics have argued that the field faces challenges related to methodological rigour, reproducibility, and overall quality. This article proposes meta-research as one approach to evaluating and improving the quality of TCIM research. Meta-research, also known as research about research, can be defined as "the study of research itself: its methods, reporting, reproducibility, evaluation, and incentives". By systematically evaluating methodological rigour, identifying biases, and promoting transparency, meta-research can enhance the reliability and credibility of TCIM research. Specific topics of interest that are discussed in this article include the following: 1) study design and research methodology, 2) reporting of research, 3) research ethics, integrity, and misconduct, 4) replicability and reproducibility, 5) peer review and journal editorial practices, 6) research funding: grants and awards, and 7) hiring, promotion, and tenure. For each topic, we provide case examples to illustrate meta-research applications in TCIM. We argue that meta-research initiatives can contribute to maintaining public trust, safeguarding research integrity, and advancing evidence based TCIM practice, while challenges include navigating methodological complexities, biases, and disparities in funding and academic recognition. Future directions involve tailored research methodologies, interdisciplinary collaboration, policy implications, and capacity building in meta-research.

Methodological and reporting quality of systematic reviews on aesthetics and reconstructive breast surgery: A meta-research.

OBJECTIVE: To evaluate the methodological and reporting quality of systematic reviews (SR) of randomized controlled trials on esthetics and reconstructive breast surgery. METHODS: Meta-research study with a broad search strategy was developed to retrieve all relevant systematic reviews. We evaluated the methodological and reporting guidance adopted by these reviews and assessed their adequacy to items from AMSTAR-2 (methodological quality) and PRISMA 2020 (reporting quality). The protocol of this study was prospectively published in: https://osf.io/preprints/osf/ucpgd. RESULTS: After the selection process, 15 SR were included; eight (60%) referred the use of a methodological guide and five (33.3%) invertedly referred PRISMA as the methodological guide. Reporting guidelines were referred by none of the included systematic review. The median adequacy to PRISMA-2020 items was 42.9% (Q1 - 38.1%/Q3 - 95.2%) and to AMSTAR-2 items was 33.3% (Q1 - 23.3%/Q3 - 93.3%) which reflects overall low reporting and methodological quality of included SR. The overall confidence in the results using AMSTAR-2 framework was critically low in 73.3% of included SR. Although a small number of SR were included, a high correlation between the methodological and reporting quality was observed (Spearmean rho = 0.96, 95% bias-corrected confidence interval = 0.84 to 0.99). CONCLUSION: Methodological and reposting quality of SR of randomized clinical trials on esthetic or reconstructive breast surgery is poor. Half of the authors referred to the use of valid guidance to plan and conduct their reviews and none of them referred the use of a guidance for reporting their results.

Can large language models help predict results from a complex behavioural science study?

We tested whether large language models (LLMs) can help predict results from a complex behavioural science experiment. In study 1, we investigated the performance of the widely used LLMs GPT-3.5 and GPT-4 in forecasting the empirical findings of a large-scale experimental study of emotions, gender, and social perceptions. We found that GPT-4, but not GPT-3.5, matched the performance of a cohort of 119 human experts, with correlations of 0.89 (GPT-4), 0.07 (GPT-3.5) and 0.87 (human experts) between aggregated forecasts and realized effect sizes. In study 2, providing participants from a university subject pool the opportunity to query a GPT-4 powered chatbot significantly increased the accuracy of their forecasts. Results indicate promise for artificial intelligence (AI) to help anticipate-at scale and minimal cost-which claims about human behaviour will find empirical support and which ones will not. Our discussion focuses on avenues for human-AI collaboration in science.

A Critical Appraisal of Reporting in Randomized Controlled Trials Investigating Osteopathic Manipulative Treatment: A Meta-Research Study.

Background/Objectives: In osteopathy, it becomes necessary to produce high-quality evidence to demonstrate its effectiveness. The aim of this meta-research study is to assess the reporting quality of RCTs published in the osteopathic field. Methods: The protocol was preliminarily registered on the "Open Science Framework (OSF)" website. For reporting, we considered the PRISMA 2020 checklist. We included all the RCTs, published between 2011 and 2023, investigating the effectiveness of Osteopathic Manipulative Treatment (OMT) in any possible condition. The search process was conducted on four major biomedical databases including PubMed, Central, Scopus and Embase. A data extraction form was implemented to collect all relevant information. The completeness of reporting was calculated as the percentage of adherence to the CONSORT checklist; the Cochrane ROB 2 tool was considered to assess the risk of bias (RoB) in the following five major domains: randomization (D1), interventions (D2), missing data (D3), outcome measurement (D4), selective reporting (D5). Results: A total of 131 studies were included and the overall adherence was 57%, with the worst section being "other information" (42%). Studies with a lower RoB showed higher adherence to the CONSORT. The "results" section presented the highest differences as follows: D1 (-36.7%), D2 (-27.2%), D3 (-21.5%) and D5 (-25.5%). Significant correlations were also found between the preliminary protocol registration, higher journal quartile, publication in hybrid journals and the completeness of reporting (ß: 19.22, CI: 14.45-24.00, p < 0.001; ß: 5.41; CI: 2.80-8.02, p ≤ 0.001; ß: 5.64, CI: 1.06-10.23, p = 0.016, respectively). Conclusions: The adherence to the CONSORT checklist in osteopathic RCTs is lacking. An association was found between a lower completeness of reporting and a higher RoB, a good journal ranking, publication in hybrid journals and a prospective protocol registration. Journals and authors should adopt all the strategies to adhere to reporting guidelines to guarantee generalization of the results arising from RCTs.

Assessing computational reproducibility in Behavior Research Methods.

Psychological science has thrived thanks to new methods and innovative practices. Journals, including Behavior Research Methods (BRM), continue to support the dissemination and evaluation of research assets including data, software/hardware, statistical code, and databases of stimuli. However, such research assets rarely allow for computational reproducibility, meaning they are difficult to reuse. Therefore, in this preregistered report, we explore how BRM's authors and BRM structures shape the landscape of functional research assets. Our broad research questions concern: (1) How quickly methods and analytical techniques reported in BRM can be used and developed further by other scientists; (2) Whether functionality has improved following changes to BRM journal policy in support of computational reproducibility; (3) Whether we can disentangle such policy changes from changes in reproducibility over time. We randomly sampled equal numbers of papers (N = 204) published in BRM before and after the implementation of policy changes. Pairs of researchers recorded how long it took to ensure assets (data, software/hardware, statistical code, and materials) were fully operational. They also coded the completeness and reusability of the assets. While improvements were observed in all measures, only changes to completeness were altered significantly following the policy changes (d = .37). The effects varied between different types of research assets, with data sets from surveys/experiments showing the largest improvements in completeness and reusability. Perhaps more importantly, changes to policy do appear to have improved the life span of research products by reducing natural decline. We conclude with a discussion of how, in the future, research and policy might better support computational reproducibility within and beyond psychological science.

Mapping Systematic Reviews on the Management of Dental Caries in Primary Teeth: A Meta-Research.

BACKGROUND: Systematic reviews (SRs) represent the most robust source of evidence for informing decision-making. While there are rigorous protocols for properly conducting SRs, sometimes the methodological biases in the primary studies are accounted for in the conclusions of the SRs. OBJECTIVE: This study aimed to map the evidence regarding the management of caries lesions in primary teeth. METHODS: Two reviewers conducted a systematic search up to March 2024 in electronic data-bases. Any SR concerning the management of caries lesions in primary teeth was considered eli-gible. RESULTS: About 162 SRs were included. Among these, 80 focused on restorative treatments, 64 on endodontic treatments, and 18 on non-invasive treatments. Only 42.6% presented a study registra-tion protocol. The majority (67.9%) performed a meta-analysis, while a minority exclusively car-ried out qualitative data analysis. Despite 92.6% of the SRs evaluating the methodological quality or risk of bias of the primary studies using some tool, only 24% assessed the certainty of evidence using the GRADE approach, resulting in classifications ranging from very low to moderate. CONCLUSION: There is a limited adherence to study registration protocols, indicating a need for improvements in this practice. Additionally, among the few SRs that used the GRADE approach, the majority demonstrated levels of very low to moderate certainty.

Reporting quality of scoping reviews in endodontics: A meta-research study.

OBJECTIVES: To evaluate the reporting quality of Scoping Reviews (ScRs) in endodontics according to the PRISMA Extension Checklist for Scoping Reviews (PRISMA-ScR) and to analyse their association with a range of publication and methodological/reporting characteristics. METHODS: Pubmed, Scopus, and Web of Science databases were searched up to 31 January 2024 to identify scoping reviews in the field of endodontics. An additional search was performed in three leading endodontic journals. Study selection and appraising the quality of the studies was carried out independently by two reviewers. Each of the 20 PRISMA-ScR items were allocated a score of either 0, 0.5 or 1 to reflect the completeness of the reporting. An item-specific and overall percentage reporting quality score was calculated and reported through descriptive statistics across a range of publication, as well as methodological/reporting characteristics. A univariable and multivariable quantile regression was performed to identify the effect of publication and methodological/reporting characteristics (year of publication, journal, inclusion of an appropriate reporting guideline, and study registration) on the overall percentage reporting quality score. Association of reporting quality score with publication characteristics was then investigated. RESULTS: A total of 40 ScRs were identified and included for appraisal. Most of the studies were published from 2021 onwards. The overall median reporting quality score was 86%. The most frequent items not included in the studies were: a priori protocol registration (22/40 compliant; 55%), and reporting of funding (16/40 compliant; 40%). Other key elements that were inadequately reported were the abstract (7/40 compliant; 18%), the rationale and justification of the ScR (21/40 compliant; 52%) and the objectives of the study (18/40 compliant; 45%). Studies that adhered to appropriate reporting guidelines were associated with greater reporting quality scores (ß-coefficient: 10; 95%CI: 1.1, 18.9; p = .03). ScRs with protocols registered a priori had significantly greater reporting quality scores (ß-coefficient: 12.5; 95%CI: 6.1, 18.9; p < .001), compared with non-registered reviews. CONCLUSIONS: The reporting quality of the ScRs in endodontics varied and was greater when the ScR protocols were registered a priori and when the authors adhered to reporting guidelines.

Effect of bispectral index on intra-operative awareness: A meta-analysis of randomized controlled studies.

Background: Randomized controlled trials (RCTs) investigating the efficacy of bispectral index (BIS) to reduce intra-operative awareness (IOA) have reported conflicting results. The purpose of this meta-analysis is to consolidate results from RCTs to assess the efficacy of BIS in reducing IOA when compared to controls. Secondary outcomes included time to extubation, time to spontaneous and/or verbal eye opening, PACU discharge time, and utilization of inhaled anesthetics. Methods: RCTs which reported on one of the primary and/or secondary outcomes were included. Literature search utilized keywords "randomized control trial" and "intraoperative awareness." Meta-analysis was performed using RevMan 5. Results: Twenty-seven RCTs were included in the study with a total of 35,585 patients, with 18,146 patients in the BIS and 17,439 in the control group. Eighteen of 14,062 patients (0.12%) and 42 of 16,765 (0.25%) reported definite IOA in the BIS and control group, respectively, with no statistically significant difference. BIS was effective in reducing the time to spontaneous eye opening by an average of 1.3 minutes and the time to extubation by an average of 1.97 minutes. There was no difference in PACU discharge times among the groups. There was a significant decrease in consumption of sevoflurane but no difference in desflurane and propofol compared to the control group. Conclusion: While BIS monitoring results in decreased incidence of intra-operative awareness by half, it was not statistically significant. BIS provides modest benefits with regard to reducing the time to extubation, the time to spontaneous eye opening, and consumption of sevoflurane.Level of evidence: I.

Assessing transparency practices in dental randomized controlled trials.

BACKGROUND: To evaluate transparency practices in randomized controlled trials (RCTs) in dentistry. METHODS: This meta-research study included RCTs in dentistry regardless of topic, methods, or level of detail reported. Only studies in English were considered. We searched PubMed for RCTs in dentistry published in English from December 31, 2016, to December 31, 2021. The screening was performed in duplicate, and data extracted included journal and author details, dental specialty, protocol registration, data and code sharing, conflict of interest declaration, and funding information. A descriptive analysis of the data was performed. We generated maps illustrating the reporting of transparency items by country of the corresponding author and a heat table reflecting reporting levels by dental specialty. RESULTS: A total of 844 RCTs were included. Only 12.86% of studies reported any information about data and code sharing. Protocol registration was reported for 50.36% of RCTs. Conflict of interest (83.41%) and funding (71.68%) declarations were present in most studies. Conflicts of interest and funding were consistently reported regardless of country or specialty, while data and code sharing had a low level of reporting across specialties, as well as low dissemination across the world. Protocol registration exhibited considerable variability. CONCLUSIONS: Considering the importance of RCTs for evidence-based dentistry, it is crucial that everyone who participates in the scientific production and dissemination process actively and consistently promotes adherence to transparent scientific standards, particularly registration of protocols, and sharing of data and code.

The policies on the use of large language models in radiological journals are lacking: a meta-research study.

OBJECTIVE: To evaluate whether and how the radiological journals present their policies on the use of large language models (LLMs), and identify the journal characteristic variables that are associated with the presence. METHODS: In this meta-research study, we screened Journals from the Radiology, Nuclear Medicine and Medical Imaging Category, 2022 Journal Citation Reports, excluding journals in non-English languages and relevant documents unavailable. We assessed their LLM use policies: (1) whether the policy is present; (2) whether the policy for the authors, the reviewers, and the editors is present; and (3) whether the policy asks the author to report the usage of LLMs, the name of LLMs, the section that used LLMs, the role of LLMs, the verification of LLMs, and the potential influence of LLMs. The association between the presence of policies and journal characteristic variables was evaluated. RESULTS: The LLM use policies were presented in 43.9% (83/189) of journals, and those for the authors, the reviewers, and the editor were presented in 43.4% (82/189), 29.6% (56/189) and 25.9% (49/189) of journals, respectively. Many journals mentioned the aspects of the usage (43.4%, 82/189), the name (34.9%, 66/189), the verification (33.3%, 63/189), and the role (31.7%, 60/189) of LLMs, while the potential influence of LLMs (4.2%, 8/189), and the section that used LLMs (1.6%, 3/189) were seldomly touched. The publisher is related to the presence of LLM use policies (p < 0.001). CONCLUSION: The presence of LLM use policies is suboptimal in radiological journals. A reporting guideline is encouraged to facilitate reporting quality and transparency. CRITICAL RELEVANCE STATEMENT: It may facilitate the quality and transparency of the use of LLMs in scientific writing if a shared complete reporting guideline is developed by stakeholders and then endorsed by journals. KEY POINTS: The policies on LLM use in radiological journals are unexplored. Some of the radiological journals presented policies on LLM use. A shared complete reporting guideline for LLM use is desired.

Analysis of risk of bias assessments in a sample of intervention systematic reviews, Part II: focus on risk of bias tools reveals few meet current appraisal standards.

OBJECTIVES: Risk of bias (RoB) assessment is a critical part of any systematic review (SR). There are multiple tools available for assessing RoB of the studies included in a SR. The conduct of these assessments in intervention SRs are addressed by three items in AMSTAR-2, considered the preferred tool for critically appraising an intervention SR. This study focuses attention on item 9, which assesses the ability of a RoB tool to adequately address sources of bias, particularly in randomized trials (RCTs) and nonrandomized studies of interventions (NRSI). Our main objective is to report the detailed results of our examination of both Cochrane and non-Cochrane RoB tools and distinguish those that meet AMSTAR-2 item 9 appraisal standards. STUDY DESIGN AND SETTING: We identified critical appraisal tools reported in a sample of 126 SRs reporting on interventions for persons with cerebral palsy published from 2014 to 2021. Eligible tools were those that had been used to assess the primary studies included in these SRs and for which assessment results were reported in enough detail to allow appraisal of the tool. We identified the version of the tool applied as original, modified, or novel and established the applicable study designs as intended by the tools' developers. We then evaluated the potential ability of these tools to assess the four sources of bias specified in AMSTAR-2 item 9 for RCTs and NRSI. We adapted item 9 to appraise tools applied to single-case experimental designs, which we also encountered in this sample of SRs. RESULTS: Most of the eligible tools are recognized by name in the published literature and were applied in the original or modified form. Modifications were applied with considerable variability across the sample. Of the 37 tools we examined, those judged to fully meet the appraisal standards for RCTs included all the Cochrane tools, the original and modified Downs and Black Checklist, and the quality assessment standard for a cross-over study by Ding et al; for NRSI, these included all the Cochrane tools, the original and modified Downs and Black Checklist, and the Research Triangle Institute item bank on Risk of Bias and Precision of Observational Studies for NRSI. In general, tools developed for a specific study design were judged to meet the appraisal standards fully or partially for that design. These results suggest it is unlikely that a single tool will be adequate by AMSTAR-2 item 9 appraisal standards for an intervention SR that includes studies of various designs. CONCLUSION: To our knowledge, this is the first resource providing SR authors with practical information about the appropriateness and adequacy of RoB tools by the appraisal standards specified in AMSTAR-2 item 9 for RCTs and NRSI. We propose similar methods for appraisal of tools applied to single-case experimental design. We encourage authors to seek contemporary RoB tools developed for use in healthcare-related intervention SRs and designed to evaluate relevant study design features. The tools should address attributes unique to the review topic and research question but not be subjected to unjustified and excessive modifications. We promote recognition of the potential shortcomings of both Cochrane and non-Cochrane RoB tools, even those that perform well by AMSTAR-2 item 9 appraisal standards.

Primary outcomes and characteristics of clinical trial registries (up to October 2021) on COVID-19 vaccines.

OBJECTIVES: To analyse the general and primary outcome-related characteristics of clinical trials protocols on COVID-19 vaccines. STUDY DESIGN AND SETTING: A meta-research study. A search for clinical trial protocols on COVID-19 vaccines was conducted on the ClinicalTrials.gov platform. We considered all protocols of comparative trials registered up to October 26, 2021. RESULTS: Two hundred and eighty-two trials were analysed. The median expected trial duration was 445 days (interquartile range [IQR] = 225), and the median target sample size was 420 participants (IQR = 1638). A retrospective registry (after the start date) was observed for 42.55% of the trials. Randomization procedures were planned by 84.75% and full-blinding procedures by 34.75% of the 282 trials. Most trials were labelled as active or still recruiting, and 14 trials (5%) were completed. None of the 14 trials labelled as completed on our search date had results available. Industry funding was reported by 198 trials (70.2%). Most studies declared more than one primary outcome, usually a safety or immunogenicity outcome, and 59 studies (20.9%) had at least one primary efficacy outcome. The description of the primary efficacy outcomes was limited in most cases, referred to as a non-specified 'efficacy' outcome (18.6%) or described as 'COVID-19 cases' (32.2%). CONCLUSION: the primary outcomes of clinical trials on COVID-19 vaccines are poorly described, and the registers provide insufficient information about them. The registry was retrospectively fulfilled for many trials, which may lead to bias and research waste. Outcomes were generically described and did not provide transparent information for replication in practice, further trials or meta-analyses.

Evaluating strategies to recruit health researchers to participate in online survey research.

BACKGROUND: Engaging researchers as research subjects is key to informing the development of effective and relevant research practices. It is important to understand how best to engage researchers as research subjects. METHODS: A 24 factorial experiment, as part of a Multiphase Optimization Strategy, was performed to evaluate effects of four recruitment strategy components on participant opening of an emailed survey link and survey completion. Participants were members of three US-based national health research consortia. A stratified simple random sample was used to assign potential survey participants to one of 16 recruitment scenarios. Recruitment strategy components were intended to address both intrinsic and extrinsic sources of motivation, including: $50 gift, $1,000 raffle, altruistic messaging, and egoistic messaging. Multivariable generalized linear regression analyses adjusting for consortium estimated component effects on outcomes. Potential interactions among components were tested. Results are reported as adjusted odds ratios (aOR) with 95% confidence intervals (95% CI). RESULTS: Surveys were collected from June to December 2023. A total of 418 participants were included from the consortia, with final analytical sample of 400 eligible participants. Out of the final sample, 82% (341) opened the survey link and 35% (147) completed the survey. Altruistic messaging increased the odds of opening the survey (aOR 2.02, 95% CI: 1.35-2.69, p = 0.033), while egoistic messaging significantly reduced the odds of opening the survey (aOR 0.56, 95%CI 0.38-0.75, p = 0.08). The receipt of egoistic messaging increased the odds of completing the survey once opened (aOR 1.81, 95%CI: 1.39-2.23, p < 0.05). There was a significant negative interaction effect between the altruistic appeal and egoistic messaging strategies for survey completion outcome. Monetary incentives did not a have a significant impact on survey completion. CONCLUSION: Intrinsic motivation is likely to be a greater driver of health researcher participation in survey research than extrinsic motivation. Altruistic and egoistic messaging may differentially impact initial interest and survey completion and when combined may lead to improved rates of recruitment, but not survey completion. Further research is needed to determine how to best optimize message content and whether the effects observed are modified by survey burden.

Underserved groups could be better considered within population-based eye health surveys: a methodological study.

OBJECTIVES: In pursuit of health equity, the World Health Organization has recently called for more extensive monitoring of inequalities in eye health. Population-based eye health surveys can provide this information, but whether underserved groups are considered in the design, implementation, and reporting of surveys is unknown. We conducted a systematic methodological review of surveys published since 2000 to examine how many population-based eye health surveys have considered underserved groups in their design, implementation, or reporting. STUDY DESIGN AND SETTING: We identified all population-based cross-sectional surveys reporting the prevalence of objectively measured vision impairment or blindness. Using the PROGRESS + framework to identify underserved groups, we assessed whether each study considered underserved groups within 15 items across the rationale, sampling or recruitment methods, or the reporting of participation and prevalence rates. RESULTS: 388 eye health surveys were included in this review. Few studies prospectively considered underserved groups during study planning or implementation, for example within their sample size calculations (n = 5, ∼1%) or recruitment strategies (n = 70, 18%). The most common way that studies considered underserved groups was in the reporting of prevalence estimates (n = 374, 96%). We observed a modest increase in the number of distinct PROGRESS + factors considered by a publication over the study period. Gender/sex was considered within at least one item by 95% (n = 367) of studies. Forty-three percent (n = 166) of included studies were conducted primarily on underserved population groups, particularly for subnational studies of people living in rural areas, and we identified examples of robust population-based studies in socially excluded groups. CONCLUSION: More effort is needed to improve the design, implementation, and reporting of surveys to monitor inequality and promote equity in eye health. Ideally, national-level monitoring of vision impairment and service coverage would be supplemented with smaller-scale studies to understand the disparities experienced by the most underserved groups.

On the value of meta-research for early career researchers: A commentary.

Meta-research, also known as "research on research" is a field of study that investigates the methods, reporting, reproducibility, evaluation, and incentives along the research continuum. Meta-research literacy is imperative to ensure high quality, transparent and reproducible primary data or meta-research products. In this commentary, we propose that early career researchers should be trained in meta-research as a foundation to develop a deeper understanding of the research process and ability to appraise the research literature and design high-quality original studies, irrespective of their chosen field of study. We discuss the importance of meta-research and open science from the perspective of an early career trainee, highlighting essential areas for growth and obstacles one may encounter.

Trends in research approaches and gender in plant ecology dissertations over four decades.

Dissertations are a foundational scientific product; they are the formative product that early-career scientists create and share original knowledge. The methodological approaches used in dissertations vary with the research field. In plant ecology, these approaches include observations, experiments (field or controlled environment), literature reviews, theoretical approaches, or analyses of existing data (including "big data"). Recently, concerns have been raised about the rise of "big data" studies and the loss of observational and field-based studies in ecology, but such trends have not been formally quantified. Therefore, we examined how the emphasis on each of these categories has changed over time and whether male and female authors differ in the methods employed. We found remarkable temporal consistency, with observational studies being dominant over the entire time span examined. There was an increase in the number of approaches employed per dissertation, with increases in analyses of databases and theoretical studies adding to rather than replacing traditional methodologies (like observations and field experiments). The representation of women increased over time. There were some differences in the approaches taken by men and women, which requires further investigation.

Replication of null results: Absence of evidence or evidence of absence?

In several large-scale replication projects, statistically non-significant results in both the original and the replication study have been interpreted as a 'replication success.' Here, we discuss the logical problems with this approach: Non-significance in both studies does not ensure that the studies provide evidence for the absence of an effect and 'replication success' can virtually always be achieved if the sample sizes are small enough. In addition, the relevant error rates are not controlled. We show how methods, such as equivalence testing and Bayes factors, can be used to adequately quantify the evidence for the absence of an effect and how they can be applied in the replication setting. Using data from the Reproducibility Project: Cancer Biology, the Experimental Philosophy Replicability Project, and the Reproducibility Project: Psychology we illustrate that many original and replication studies with 'null results' are in fact inconclusive. We conclude that it is important to also replicate studies with statistically non-significant results, but that they should be designed, analyzed, and interpreted appropriately.

Enhancing reporting through structure: a before and after study on the effectiveness of SPIRIT-based templates to improve the completeness of reporting of randomized controlled trial protocols.

BACKGROUND: Despite the improvements in the completeness of reporting of randomized trial protocols after the publication of the Standard Protocol Items: Recommendations for Interventional Trial (SPIRIT) guidelines, many items remain poorly reported. This study aimed to assess the effectiveness of using SPIRIT-tailored templates for trial protocols to improve the completeness of reporting of the protocols that master's students write as part of their master's theses. METHODS: Before and after experimental study performed at the University Master's Degree in Orthopaedic Manual Physiotherapy of the Universitat Internacional de Catalunya (Barcelona, Spain). While students in the post-intervention period were instructed to use a trial protocol template that was tailored to SPIRIT, students in the pre-intervention period did not use the template. PRIMARY OUTCOME: Difference between the pre- and post-intervention periods in the mean number of adequately reported items (0-10 scale). The outcomes were evaluated independently and in duplicate by two blinded assessors. Students and their supervisors were not aware that they were part of a research project. For the statistical analysis, we used a generalized linear regression model (dependent variable: number of adequately reported items in the protocol; independent variables: intervention period, call, language). RESULTS: Thirty-four trial protocols were included (17, pre-intervention; 17, post-intervention). Protocols produced during the post-intervention period (mean: 8.24; SD: 1.52) were more completely reported than those produced during the pre-intervention period (mean: 6.35; SD: 1.80); adjusted difference: 1.79 (95% CI: 0.58 to 3.00). CONCLUSIONS: SPIRIT-based templates could be used to improve the completeness of reporting of randomized trial protocols.

Scientometrics Evaluation of Published Scientific Papers on the Use of Proteomics Technologies in Mastitis Research in Ruminants.

The objective of this study was the presentation of quantitative characteristics regarding the scientific content and bibliometric details of the relevant publications. In total, 156 papers were considered. Most papers presented original studies (n = 135), and fewer were reviews (n = 21). Most original articles (n = 101) referred to work involving cattle. Most original articles described work related to the diagnosis (n = 72) or pathogenesis (n = 62) of mastitis. Most original articles included field work (n = 75), whilst fewer included experimental (n = 31) or laboratory (n = 30) work. The tissue assessed most frequently in the studies was milk (n = 59). Milk was assessed more frequently in studies on the diagnosis (61.1% of relevant studies) or pathogenesis (30.6%) of the infection, but mammary tissue was assessed more frequently in studies on the treatment (31.0%). In total, 47 pathogens were included in the studies described; most were Gram-positive bacteria (n = 34). The three bacteria most frequently included in the studies were Staphylococcus aureus (n = 55 articles), Escherichia coli (n = 31) and Streptococcus uberis (n = 19). The proteomics technology employed more often in the respective studies was liquid chromatography-tandem mass spectrometry (LC-MS/MS), either on its own (n = 56) or in combination with other technologies (n = 40). The median year of publication of articles involving bioinformatics or LC-MS/MS and bioinformatics was the most recent: 2022. The 156 papers were published in 78 different journals, most frequently in the Journal of Proteomics (n = 16 papers) and the Journal of Dairy Science (n = 12). The median number of cited references in the papers was 48. In the papers, there were 1143 co-authors (mean: 7.3 ± 0.3 co-authors per paper, median: 7, min.-max.: 1-19) and 742 individual authors. Among them, 15 authors had published at least seven papers (max.: 10). Further, there were 218 individual authors who were the first or last authors in the papers. Most papers were submitted for open access (n = 79). The median number of citations received by the 156 papers was 12 (min.-max.: 0-339), and the median yearly number of citations was 2.0 (min.-max.: 0.0-29.5). The h-index of the papers was 33, and the m-index was 2. The increased number of cited references in papers and international collaboration in the respective study were the variables associated with most citations to published papers. This is the first ever scientometrics evaluation of proteomics studies, the results of which highlighted the characteristics of published papers on mastitis and proteomics. The use of proteomics in mastitis research has focused on the elucidation of pathogenesis and diagnosis of the infection; LC-MS/MS has been established as the most frequently used proteomics technology, although the use of bioinformatics has also emerged recently as a useful tool.