The quality of reporting in case reports of permanent neonatal diabetes mellitus: a cross-sectional study.

BACKGROUND: Although randomized trials and systematic reviews provide the best evidence to guide medical practice, many permanent neonatal diabetes mellitus (PNDM) studies have been published as case reports. However, the quality of these studies has not been assessed. The purpose of this study was to assess the extent to which the current case reports for PNDM comply with the Case Report (CARE) guidelines and to explore variables associated with the reporting. METHOD: Six English and four Chinese databases were searched from their inception to December 2022 for PNDM case reports. The 23 items CARE checklist was used to measure reporting quality. Primary outcome was the adherence rate of each CARE item and second outcome was total reporting score for each included PNDM case report. Linear and logistic regression analyses were used to examine the connection between five pre-specified predictor variables and the reporting quality. The predictor variables were impact factor of the published journal (<3.4 vs. ≥3.4, categorized according to the median), funding (yes vs. no), language (English vs. other language), published journal type (general vs. special) and year of publication (>2013 vs. ≤ 2013). RESULT: In total, 105 PNDM case reports were included in this study. None of the 105 PNDM case reports fulfilled all 23 items of the CARE checklist. The response rate of 11 items were under 50%, including prognostic characteristics presentation (0%), patient perspective interpretation (0%), diagnostic challenges statement (2.9%), clinical course summary (21.0%), diagnostic reasoning statement (22.9%), title identification (24.8%), case presentation (33.3%), disease history description (34.3%), strengths and limitations explanation (41.0%), informed consent statement (45.7%), and lesson elucidation (47.6%). This study identified that the PNDM case reports published in higher impact factor journals were statistically associated with a higher reporting quality. CONCLUSION: The reporting of case reports for PNDM is generally poor. As a result, this information may be misleading to providers, and the clinical applications may be detrimental to patient care. To improve reporting quality, journals should encourage strict adherence to the CARE guidelines.

Adaptive designs were primarily used but inadequately reported in early phase drug trials.

BACKGROUND: Faced with the high cost and limited efficiency of classical randomized controlled trials, researchers are increasingly applying adaptive designs to speed up the development of new drugs. However, the application of adaptive design to drug randomized controlled trials (RCTs) and whether the reporting is adequate are unclear. Thus, this study aimed to summarize the epidemiological characteristics of the relevant trials and assess their reporting quality by the Adaptive designs CONSORT Extension (ACE) checklist. METHODS: We searched MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials (CENTRAL) and ClinicalTrials.gov from inception to January 2020. We included drug RCTs that explicitly claimed to be adaptive trials or used any type of adaptative design. We extracted the epidemiological characteristics of included studies to summarize their adaptive design application. We assessed the reporting quality of the trials by Adaptive designs CONSORT Extension (ACE) checklist. Univariable and multivariable linear regression models were used to the association of four prespecified factors with the quality of reporting. RESULTS: Our survey included 108 adaptive trials. We found that adaptive design has been increasingly applied over the years, and was commonly used in phase II trials (n = 45, 41.7%). The primary reasons for using adaptive design were to speed the trial and facilitate decision-making (n = 24, 22.2%), maximize the benefit of participants (n = 21, 19.4%), and reduce the total sample size (n = 15, 13.9%). Group sequential design (n = 63, 58.3%) was the most frequently applied method, followed by adaptive randomization design (n = 26, 24.1%), and adaptive dose-finding design (n = 24, 22.2%). The proportion of adherence to the ACE checklist of 26 topics ranged from 7.4 to 99.1%, with eight topics being adequately reported (i.e., level of adherence ≥ 80%), and eight others being poorly reported (i.e., level of adherence ≤ 30%). In addition, among the seven items specific for adaptive trials, three were poorly reported: accessibility to statistical analysis plan (n = 8, 7.4%), measures for confidentiality (n = 14, 13.0%), and assessments of similarity between interim stages (n = 25, 23.1%). The mean score of the ACE checklist was 13.9 (standard deviation [SD], 3.5) out of 26. According to our multivariable regression analysis, later published trials (estimated ß = 0.14, p < 0.01) and the multicenter trials (estimated ß = 2.22, p < 0.01) were associated with better reporting. CONCLUSION: Adaptive design has shown an increasing use over the years, and was primarily applied to early phase drug trials. However, the reporting quality of adaptive trials is suboptimal, and substantial efforts are needed to improve the reporting.

Automatic categorization of self-acknowledged limitations in randomized controlled trial publications.

OBJECTIVE: Acknowledging study limitations in a scientific publication is a crucial element in scientific transparency and progress. However, limitation reporting is often inadequate. Natural language processing (NLP) methods could support automated reporting checks, improving research transparency. In this study, our objective was to develop a dataset and NLP methods to detect and categorize self-acknowledged limitations (e.g., sample size, blinding) reported in randomized controlled trial (RCT) publications. METHODS: We created a data model of limitation types in RCT studies and annotated a corpus of 200 full-text RCT publications using this data model. We fine-tuned BERT-based sentence classification models to recognize the limitation sentences and their types. To address the small size of the annotated corpus, we experimented with data augmentation approaches, including Easy Data Augmentation (EDA) and Prompt-Based Data Augmentation (PromDA). We applied the best-performing model to a set of about 12K RCT publications to characterize self-acknowledged limitations at larger scale. RESULTS: Our data model consists of 15 categories and 24 sub-categories (e.g., Population and its sub-category DiagnosticCriteria). We annotated 1090 instances of limitation types in 952 sentences (4.8 limitation sentences and 5.5 limitation types per article). A fine-tuned PubMedBERT model for limitation sentence classification improved upon our earlier model by about 1.5 absolute percentage points in F1 score (0.821 vs. 0.8) with statistical significance (p<.001). Our best-performing limitation type classification model, PubMedBERT fine-tuning with PromDA (Output View), achieved an F1 score of 0.7, improving upon the vanilla PubMedBERT model by 2.7 percentage points, with statistical significance (p<.001). CONCLUSION: The model could support automated screening tools which can be used by journals to draw the authors' attention to reporting issues. Automatic extraction of limitations from RCT publications could benefit peer review and evidence synthesis, and support advanced methods to search and aggregate the evidence from the clinical trial literature.

The Reporting Quality of Machine Learning Studies on Pediatric Diabetes Mellitus: Systematic Review.

BACKGROUND: Diabetes mellitus (DM) is a major health concern among children with the widespread adoption of advanced technologies. However, concerns are growing about the transparency, replicability, biasedness, and overall validity of artificial intelligence studies in medicine. OBJECTIVE: We aimed to systematically review the reporting quality of machine learning (ML) studies of pediatric DM using the Minimum Information About Clinical Artificial Intelligence Modelling (MI-CLAIM) checklist, a general reporting guideline for medical artificial intelligence studies. METHODS: We searched the PubMed and Web of Science databases from 2016 to 2020. Studies were included if the use of ML was reported in children with DM aged 2 to 18 years, including studies on complications, screening studies, and in silico samples. In studies following the ML workflow of training, validation, and testing of results, reporting quality was assessed via MI-CLAIM by consensus judgments of independent reviewer pairs. Positive answers to the 17 binary items regarding sufficient reporting were qualitatively summarized and counted as a proxy measure of reporting quality. The synthesis of results included testing the association of reporting quality with publication and data type, participants (human or in silico), research goals, level of code sharing, and the scientific field of publication (medical or engineering), as well as with expert judgments of clinical impact and reproducibility. RESULTS: After screening 1043 records, 28 studies were included. The sample size of the training cohort ranged from 5 to 561. Six studies featured only in silico patients. The reporting quality was low, with great variation among the 21 studies assessed using MI-CLAIM. The number of items with sufficient reporting ranged from 4 to 12 (mean 7.43, SD 2.62). The items on research questions and data characterization were reported adequately most often, whereas items on patient characteristics and model examination were reported adequately least often. The representativeness of the training and test cohorts to real-world settings and the adequacy of model performance evaluation were the most difficult to judge. Reporting quality improved over time (r=0.50; P=.02); it was higher than average in prognostic biomarker and risk factor studies (P=.04) and lower in noninvasive hypoglycemia detection studies (P=.006), higher in studies published in medical versus engineering journals (P=.004), and higher in studies sharing any code of the ML pipeline versus not sharing (P=.003). The association between expert judgments and MI-CLAIM ratings was not significant. CONCLUSIONS: The reporting quality of ML studies in the pediatric population with DM was generally low. Important details for clinicians, such as patient characteristics; comparison with the state-of-the-art solution; and model examination for valid, unbiased, and robust results, were often the weak points of reporting. To assess their clinical utility, the reporting standards of ML studies must evolve, and algorithms for this challenging population must become more transparent and replicable.

Radiomics Studies on Ischemic Stroke and Carotid Atherosclerotic Disease: A Reporting Quality Assessment.

Objective: To assess the reporting quality of radiomics studies on ischemic stroke, intracranial and carotid atherosclerotic disease using the Image Biomarker Standardization Initiative (IBSI) reporting guidelines with the aim of finding avenues of improvement for future publications. Method: PubMed database was searched to identify relevant radiomics studies. Of 560 articles, 41 original research articles were included in this analysis. Based on IBSI radiomics reporting guidelines, checklists for CT-based and MRI-based studies were created to allow a structured and comprehensive evaluation of each study's adherence to these guidelines. Results: The main topics covered by the included radiomics studies were ischemic stroke, intracranial artery disease, and carotid atherosclerotic disease. The reporting checklist median score was 17/40 for the 20 CT-based radiomics studies and 22.5/50 for the 20 MRI-based studies. Basic items like imaging modality, region of interest, and image biomarker set utilized were included in all studies. However, details regarding image acquisition and reconstruction, post-acquisition image processing, and image biomarkers computation were inconsistently detailed across studies. Conclusion: The overall reporting quality of the included radiomics studies was suboptimal. These findings underscore a pressing need for improved reporting practices in radiomics research, to ensure validation and reproducibility of results. Our study provides insights into current reporting standards and highlights specific areas where adherence to IBSI guidelines could be significantly improved.

Statistical significance and publication reporting bias in abstracts of reproductive medicine studies.

STUDY QUESTION: What were the frequency and temporal trends of reporting P-values and effect measures in the abstracts of reproductive medicine studies in 1990-2022, how were reported P-values distributed, and what proportion of articles that present with statistical inference reported statistically significant results, i.e. 'positive' results? SUMMARY ANSWER: Around one in six abstracts reported P-values alone without effect measures, while the prevalence of effect measures, whether reported alone or accompanied by P-values, has been increasing, especially in meta-analyses and randomized controlled trials (RCTs); the reported P-values were frequently observed around certain cut-off values, notably at 0.001, 0.01, or 0.05, and among abstracts present with statistical inference (i.e. P-value, CIs, or significant terms), a large majority (77%) reported at least one statistically significant finding. WHAT IS KNOWN ALREADY: Publishing or reporting only results that show a 'positive' finding causes bias in evaluating interventions and risk factors and may incur adverse health outcomes for patients.Despite efforts to minimize publication reporting bias in medical research, it remains unclear whether the magnitude and patterns of the bias have changed over time. STUDY DESIGN, SIZE, DURATION: We studied abstracts of reproductive medicine studies from 1990 to 2022. The reproductive medicine studies were published in 23 first-quartile journals under the category of Obstetrics and Gynaecology and Reproductive Biology in Journal Citation Reports and 5 high-impact general medical journals (The Journal of the American Medical Association, The Lancet, The BMJ, The New England Journal of Medicine, and PLoS Medicine). Articles without abstracts, animal studies, and non-research articles, such as case reports or guidelines, were excluded. PARTICIPANTS/MATERIALS, SETTING, METHODS: Automated text-mining was used to extract three types of statistical significance reporting, including P-values, CIs, and text description. Meanwhile, abstracts were text-mined for the presence of effect size metrics and Bayes factors. Five hundred abstracts were randomly selected and manually checked for the accuracy of automatic text extraction. The extracted statistical significance information was then analysed for temporal trends and distribution in general as well as in subgroups of study designs and journals. MAIN RESULTS AND THE ROLE OF CHANCE: A total of 24 907 eligible reproductive medicine articles were identified from 170 739 screened articles published in 28 journals. The proportion of abstracts not reporting any statistical significance inference halved from 81% (95% CI, 76-84%) in 1990 to 40% (95% CI, 38-44%) in 2021, while reporting P-values alone remained relatively stable, at 15% (95% CI, 12-18%) in 1990 and 19% (95% CI, 16-22%) in 2021. By contrast, the proportion of abstracts reporting effect measures alone increased considerably from 4.1% (95% CI, 2.6-6.3%) in 1990 to 26% (95% CI, 23-29%) in 2021. Similarly, the proportion of abstracts reporting effect measures together with P-values showed substantial growth from 0.8% (95% CI, 0.3-2.2%) to 14% (95% CI, 12-17%) during the same timeframe. Of 30 182 statistical significance inferences, 56% (n = 17 077) conveyed statistical inferences via P-values alone, 30% (n = 8945) via text description alone such as significant or non-significant, 9.3% (n = 2820) via CIs alone, and 4.7% (n = 1340) via both CI and P-values. The reported P-values (n = 18 417), including both a continuum of P-values and dichotomized P-values, were frequently observed around common cut-off values such as 0.001 (20%), 0.05 (16%), and 0.01 (10%). Of the 13 200 reproductive medicine abstracts containing at least one statistical inference, 77% of abstracts made at least one statistically significant statement. Among articles that reported statistical inference, a decline in the proportion of making at least one statistically significant inference was only seen in RCTs, dropping from 71% (95% CI, 48-88%) in 1990 to 59% (95% CI, 42-73%) in 2021, whereas the proportion in the rest of study types remained almost constant over the years. Of abstracts that reported P-value, 87% (95% CI, 86-88%) reported at least one statistically significant P-value; it was 92% (95% CI, 82-97%) in 1990 and reached its peak at 97% (95% CI, 93-99%) in 2001 before declining to 81% (95% CI, 76-85%) in 2021. LIMITATIONS, REASONS FOR CAUTION: First, our analysis focused solely on reporting patterns in abstracts but not full-text papers; however, in principle, abstracts should include condensed impartial information and avoid selective reporting. Second, while we attempted to identify all types of statistical significance reporting, our text mining was not flawless. However, the manual assessment showed that inaccuracies were not frequent. WIDER IMPLICATIONS OF THE FINDINGS: There is a welcome trend that effect measures are increasingly reported in the abstracts of reproductive medicine studies, specifically in RCTs and meta-analyses. Publication reporting bias remains a major concern. Inflated estimates of interventions and risk factors could harm decisions built upon biased evidence, including clinical recommendations and planning of future research. STUDY FUNDING/COMPETING INTEREST(S): No funding was received for this study. B.W.M. is supported by an NHMRC Investigator grant (GNT1176437); B.W.M. reports research grants and travel support from Merck and consultancy from Merch and ObsEva. W.L. is supported by an NHMRC Investigator Grant (GNT2016729). Q.F. reports receiving a PhD scholarship from Merck. The other author has no conflict of interest to declare. TRIAL REGISTRATION NUMBER: N/A.

How about the RCTs' quality of acupuncture treatment for female urinary incontinence in recent 20 years? A report quality assessment.

PURPOSE: To evaluate the quality of randomized controlled trials (RCTs) of acupuncture on female urinary incontinence (UI). METHODS: We searched for related RCTs of acupuncture on female UI from seven databases (PubMed, Embase, Cochrane library, Web of Science, Medline, ClinicalKey, and Clinical trials). We applied CONSORT (2010 year) and STRICTA criteria to evaluate the relevant factors of included RCTs. Two trained researchers scored independently, and concordance was assessed by Cohen's к-statistic. The median and interquartile range summarized the CONSORT and STRICTA scores of the included studies. In addition, two independent sample t tests were used to assess the differences in the study quality between the 2000-2010 and the 2011-2022 years. RESULTS: A total of 25 RCTs were finally included, and the scores were consistent between different assessors. The average CONSORT score was 10.50 (IQR 9.0-15.0) (total score was 25.0). Overall, the studies generally included scientific background (24/25, 96%), inclusion and exclusion criteria (24/25, 96%), outcome indicators (24/25, 96%), randomization methods (21/25, 84%), generalizability (19/25, 76%), and financial support (15/25, 60%). Most lacked sample size calculation (5/25, 20%), type of randomization (5/25, 20%), blinding (6/25, 24%), case screening period and follow-up (4/25, 16%), and study registration (6/25, 24%), etc. The average STRICTA score was 3.380 (IQR 3.02-3.95) (total score was 6.0). There had a lack of education on treatment methods for patients (3/25, 12%) and the qualification of acupuncturists (6/25, 24%). CONCLUSION: The overall quality of RCTs on acupuncture treatment of female UI was suboptimal. It is still necessary to improve the research methods, especially the application of random methods, blinding, the interpretation of treatment, and the identification qualification of acupuncturists.

The reporting of prognostic prediction models for obstetric care was poor: a cross-sectional survey of 10-year publications.

BACKGROUND: To investigate the reporting of prognostic prediction model studies in obstetric care through a cross-sectional survey design. METHODS: PubMed was searched to identify prognostic prediction model studies in obstetric care published from January 2011 to December 2020. The quality of reporting was assessed by the TRIPOD checklist. The overall adherence by study and the adherence by item were calculated separately, and linear regression analysis was conducted to explore the association between overall adherence and prespecified study characteristics. RESULTS: A total of 121 studies were included, while no study completely adhered to the TRIPOD. The results showed that the overall adherence was poor (median 46.4%), and no significant improvement was observed after the release of the TRIPOD (43.9 to 46.7%). Studies including both model development and external validation had higher reporting quality versus those including model development only (68.1% vs. 44.8%). Among the 37 items required by the TRIPOD, 10 items were reported adequately with an adherence rate over of 80%, and the remaining 27 items had an adherence rate ranging from 2.5 to 79.3%. In addition, 11 items had a report rate lower than 25.0% and even covered key methodological aspects, including blinding assessment of predictors (2.5%), methods for model-building procedures (4.5%) and predictor handling (13.5%), how to use the model (13.5%), and presentation of model performance (14.4%). CONCLUSIONS: In a 10-year span, prognostic prediction studies in obstetric care continued to be poorly reported and did not improve even after the release of the TRIPOD checklist. Substantial efforts are warranted to improve the reporting of obstetric prognostic prediction models, particularly those that adhere to the TRIPOD checklist are highly desirable.

A cross-sectional study of the endorsement proportion of reporting guidelines in 1039 Chinese medical journals.

BACKGROUND: Reporting quality is a critical issue in health sciences. Adopting the reporting guidelines has been approved to be an effective way of enhancing the reporting quality and transparency of clinical research. In 2012, we found that only 7 (7/1221, 0.6%) journals adopted the Consolidated Standards of Reporting Trials (CONSORT) statement in China. The aim of the study was to know the implementation status of CONSORT and other reporting guidelines about clinical studies in China. METHODS: A cross-sectional bibliometric study was conducted. Eight medical databases were systematically searched, and 1039 medical journals published in mainland China, Hong Kong, Macau, and Taiwan were included. The basic characteristics, including subject, language, publication place, journal-indexed databases, and journal impact factors were extracted. The endorsement of reporting guidelines was assessed by a modified 5-level evaluation tool, namely i) positive active, ii) positive weak, iii) passive moderate, iv) passive weak and v) none. RESULTS: Among included journals, 24.1% endorsed CONSORT, and 0.8% endorsed CONSORT extensions. For STROBE (STrengthening the Reporting of Observational Studies in Epidemiology), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), STARD (An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies), CARE (CAse REport guidelines), the endorsement proportion were 17.2, 16.6, 16.4, and 14.8% respectively. The endorsement proportion for SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials), TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis), AGREE (Appraisal of Guidelines, Research, and Evaluation), and RIGHT (Reporting Items for Practice Guidelines in Healthcare) were below 0.7%. CONCLUSIONS: Our results showed that the implementation of reporting guidelines was low. We suggest the following initiatives including i) enhancing the level of journal endorsement for reporting guidelines; ii) strengthening the collaboration among authors, reviewers, editors, and other stakeholders; iii) providing training courses for stakeholders; iv) establishing bases for reporting guidelines network in China; v) adopting the endorsement of reporting guidelines in the policies of the China Periodicals Association (CPA); vi) promoting Chinese medical journals into the international evaluation system and publish in English.

Methodological quality and reporting quality of COVID-19 living systematic review: a cross-sectional study.

OBJECTIVES: The main objective of this study is to evaluate the methodological quality and reporting quality of living systematic reviews (LSRs) on Coronavirus disease 2019 (COVID-19), while the secondary objective is to investigate potential factors that may influence the overall quality of COVID-19 LSRs. METHODS: Six representative databases, including Medline, Excerpta Medica Database (Embase), Cochrane Library, China national knowledge infrastructure (CNKI), Wanfang Database, and China Science, Technology Journal Database (VIP) were systematically searched for COVID-19 LSRs. Two authors independently screened articles, extracted data, and then assessed the methodological and reporting quality of COVID-19 LSRs using the "A Measurement Tool to Assess systematic Reviews-2" (AMSTAR-2) tool and "Preferred Reporting Items for Systematic reviews and Meta-Analyses" (PRISMA) 2020 statement, respectively. Univariate linear regression and multivariate linear regression were used to explore eight potential factors that might affect the methodological quality and reporting quality of COVID-19 LSRs. RESULTS: A total of 64 COVID-19 LSRs were included. The AMSTAR-2 evaluation results revealed that the number of "yes" responses for each COVID-19 LSR was 13 ± 2.68 (mean ± standard deviation). Among them, 21.9% COVID-19 LSRs were rated as "high", 4.7% as "moderate", 23.4% as "low", and 50% as "critically low". The evaluation results of the PRISMA 2020 statement showed that the sections with poor adherence were methods, results and other information. The number of "yes" responses for each COVID-19 LSR was 21 ± 4.18 (mean ± standard deviation). The number of included studies and registration are associated with better methodological quality; the number of included studies and funding are associated with better reporting quality. CONCLUSIONS: Improvement is needed in the methodological and reporting quality of COVID-19 LSRs. Researchers conducting COVID-19 LSRs should take note of the quality-related factors identified in this study to generate evidence-based evidence of higher quality.

Completeness and consistency of primary outcome reporting in COVID-19 publications in the early pandemic phase: a descriptive study.

BACKGROUND: The COVID-19 pandemic saw a steep increase in the number of rapidly published scientific studies, especially early in the pandemic. Some have suggested COVID-19 trial reporting is of lower quality than typical reports, but there is limited evidence for this in terms of primary outcome reporting. The objective of this study was to assess the prevalence of completely defined primary outcomes reported in registry entries, preprints, and journal articles, and to assess consistent primary outcome reporting between these sources. METHODS: This is a descriptive study of a cohort of registered interventional clinical trials for the treatment and prevention of COVID-19, drawn from the DIssemination of REgistered COVID-19 Clinical Trials (DIRECCT) study dataset. The main outcomes are: 1) Prevalence of complete primary outcome reporting; 2) Prevalence of consistent primary outcome reporting between registry entry and preprint as well as registry entry and journal article pairs. RESULTS: We analyzed 87 trials with 116 corresponding publications (87 registry entries, 53 preprints and 63 journal articles). All primary outcomes were completely defined in 47/87 (54%) registry entries, 31/53 (58%) preprints and 44/63 (70%) journal articles. All primary outcomes were consistently reported in 13/53 (25%) registry-preprint pairs and 27/63 (43%) registry-journal article pairs. No primary outcome was specified in 13/53 (25%) preprints and 8/63 (13%) journal articles. In this sample, complete primary outcome reporting occurred more frequently in trials with vs. without involvement of pharmaceutical companies (76% vs. 45%), and in RCTs vs. other study designs (68% vs. 49%). The same pattern was observed for consistent primary outcome reporting (with vs. without pharma: 56% vs. 12%, RCT vs. other: 43% vs. 22%). CONCLUSIONS: In COVID-19 trials in the early phase of the pandemic, all primary outcomes were completely defined in 54%, 58%, and 70% of registry entries, preprints and journal articles, respectively. Only 25% of preprints and 43% of journal articles reported primary outcomes consistent with registry entries.

Evaluation of reporting quality of cohort studies using real-world data based on RECORD: systematic review.

OBJECTIVE: Real-world data (RWD) and real-world evidence (RWE) have been paid more and more attention in recent years. We aimed to evaluate the reporting quality of cohort studies using real-world data (RWD) published between 2013 and 2021 and analyze the possible factors. METHODS: We conducted a comprehensive search in Medline and Embase through the OVID interface for cohort studies published from 2013 to 2021 on April 29, 2022. Studies aimed at comparing the effectiveness or safety of exposure factors in the real-world setting were included. The evaluation was based on the REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) statement. Agreement for inclusion and evaluation was calculated using Cohen's kappa. Pearson chi-square test or Fisher's exact test and Mann-Whitney U test were used to analyze the possible factors, including the release of RECORD, journal IFs, and article citations. Bonferroni's correction was conducted for multiple comparisons. Interrupted time series analysis was performed to display the changes in report quality over time. RESULTS: 187 articles were finally included. The mean ± SD of the percentage of adequately reported items in the 187 articles was 44.7 ± 14.3 with a range of 11.1-87%. Of 23 items, the adequate reporting rate of 10 items reached 50%, and the reporting rate of some vital items was inadequate. After Bonferroni's correction, the reporting of only one item significantly improved after the release of RECORD and there was no significant improvement in the overall report quality. For interrupted time series analysis, there were no significant changes in the slope (p = 0.42) and level (p = 0.12) of adequate reporting rate. The journal IFs and citations were respectively related to 2 areas and the former significantly higher in high-reporting quality articles. CONCLUSION: The endorsement of the RECORD cheklist was generally inadequate in cohort studies using RWD and has not improved in recent years. We encourage researchers to endorse relevant guidelines when utilizing RWD for research.

Abstracts of randomized controlled trials in pediatric dentistry: reporting quality and spin.

BACKGROUND: Abstracts provide readers a concise and readily accessible information of the trials. However, poor reporting quality and spin (misrepresentation of research findings) can lead to an overestimation in trial validity. This methodological study aimed to assess the reporting quality and spin among randomized controlled trial (RCT) abstracts in pediatric dentistry. METHODS: We hand-searched RCTs in five leading pediatric dental journals between 2015 and 2021. Reporting quality in each abstract was assessed using the original 16-item CONSORT for abstracts checklist. Linear regression analyses were performed to identify factors associated with reporting quality. We evaluated the presence and characteristics of spin only in abstracts of parallel-group RCTs with nonsignificant primary outcomes according to pre-determined spin strategies. RESULTS: One hundred eighty-two abstracts were included in reporting quality evaluation. The mean overall quality score was 4.57 (SD, 0.103; 95% CI, 4.36-4.77; score range, 1-10). Only interventions, objective, and conclusions were adequately reported. Use of flow diagram (P < 0.001) was the only significant factor of higher reporting quality. Of the 51 RCT abstracts included for spin analysis, spin was identified in 40 abstracts (78.4%), among which 23 abstracts (45.1%) had spin in the Results section and 39 in the Conclusions Sect. (76.5%). CONCLUSIONS: The reporting quality of RCT abstracts in pediatric dentistry is suboptimal and the prevalence of spin is high. Joint efforts are needed to improve reporting quality and minimize spin.

Reporting quality of randomized controlled trials of angina pectoris with integrated traditional Chinese and western medicine interventions: a cross-sectional study.

BACKGROUND AND OBJECTIVE: Integrated traditional Chinese and western medicine (ITCWM), as a representative type of complex intervention, is commonly used for the treatment of angina pectoris (AP) in clinical practice. However, it is unclear whether the details of ITCWM interventions, such as rationale for selection and design, implementation and potential interactions for different therapies, were adequately reported. Therefore, this study aimed to describe the reporting characteristics and quality in randomized controlled trials (RCTs) of AP with ITCWM interventions. METHODS: Through a search of 7 electronic databases, we identified RCTs of AP with ITCWM interventions published in both English and Chinese from 1st Jan 2017 to 6th Aug 2022. The general characteristics of included studies were summarized, further, the quality of reporting was assessed based on three Checklists, including the CONSORT with 36 items (except for one item 1b about abstract), the CONSORT for abstracts (17 items), and a self-designed ITCWM-related checklist (21 items covering rationale and details of interventions, outcome assessment and analysis). The quality of RCTs published in English and Chinese, as well as journals and dissertations were also compared. RESULTS: A total of 451 eligible RCTs were included. For the reporting compliance, the mean score (95% Confidence Interval) of the CONSORT (72 scores in total), CONSORT for abstract (34 scores in total), and ITCWM-related (42 scores in total) checklists was 27.82 (27.44-28.19), 14.17 (13.98-14.37) and 21.06 (20.69-21.43), respectively. More than half items were evaluated as poor quality (reporting rate < 50%) among each Checklist. Moreover, the reporting quality of publications in English journals was higher than that in Chinese journals in terms of the CONSORT items. The reporting of published dissertations was better than that in journal publications regarding both the CONSORT and ITCWM-specific items. CONCLUSION: Although the CONSORT appears to have enhanced the reporting of RCTs in AP, the quality of ITCWM specifics is variable and in need of improvement. Reporting guideline of the ITCWM recommendations should be developed thus to improve their quality.

Measuring quality of reporting in systematic reviews of diagnostic test accuracy studies in medical imaging: comparison of PRISMA-DTA and PRISMA.

OBJECTIVES: To compare the reporting quality measured by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnostic Test Accuracy studies (PRISMA-DTA) vs the original PRISMA checklist for systematic reviews of diagnostic test accuracy studies in imaging and survey the use of PRISMA-DTA by researchers and endorsement by journals. METHODS: Systematic reviews of DTA studies published in 2020 and 2021 in Quartile 1 and Quartile 3 medical imaging journals (defined by Journal Citation Reports) were identified through PubMed. The reporting of each systematic review was assessed using PRISMA-DTA, PRISMA-2009 and PRISMA-2020. The item scores and overall score were compared among the three checklists. We also examined checklist adoption by the included systematic reviews and surveyed checklist endorsement from author instructions of included journals. RESULTS: A total of 173 systematic reviews from 66 journals were included. The use of PRISMA-DTA, compared with PRISMA-2009 and PRISMA-2020, identified more issues in the reporting of title (proportion of systematic reviews with proper reporting, 27.2% vs 98.8% vs 98.8%), abstract (39.3% vs 97.1% vs 64.7%), eligibility criteria (67.6% vs 94.2% vs 94.2%), search (28.9% vs 72.3% vs 28.9%), definitions for data extraction (14.5% vs 91.9% vs 91.9%), diagnostic accuracy measures (38.2% vs 93.6% vs 93.6%), synthesis of results (28.9% vs 89.6% vs 73.4%) and results of individual studies (40.5% vs 80.3% vs 80.3%). The overall median reporting score measured by PRISMA-DTA (72.0% (interquartile range (IQR), 66.7-77.8%)) was lower than that measured by PRISMA-2009 (88.9% (IQR, 84.0-92.6%)) and similar to that measured by PRISMA-2020 (74.1% (IQR, 66.7-77.8%)). Additionally, PRISMA-DTA was used by only 43 (24.9%) systematic reviews and endorsed by two (3.0%) journals. These trends remained consistent for reviews published in journals with diverse scientific impact. CONCLUSIONS: The use of PRISMA-DTA may identify more reporting inadequacies compared with the original PRISMA checklists when assessing diagnostic test accuracy systematic reviews, especially in critical sections such as title, abstract and methods. However, this tool is not commonly used by researchers and is inadequately endorsed by imaging journals. Our findings suggest a strong need to use PRISMA-DTA for reporting of diagnostic test accuracy systematic reviews by authors and its endorsement by journals. © 2022 International Society of Ultrasound in Obstetrics and Gynecology.

Reporting quality of randomized controlled trials evaluating non-vitamin K oral anticoagulants in atrial fibrillation: a systematic review.

BACKGROUND: Randomized controlled trials (RCTs) are subject to bias if they lack methodological quality. Furthermore, optimal and transparent reporting of RCT findings aids their critical appraisal and interpretation. This study aimed to comprehensively evaluate the report quality of RCTs of non-vitamin K oral anticoagulants (NOACs) for the treatment of atrial fibrillation (AF) and to analyze the factors influencing the quality. METHODS: By searching PubMed, Embase, Web of Science, and Cochrane Library databases RCTs published from inception to 2022 evaluating the efficacy of NOACs on AF were collected. By using the 2010 Consolidated Standards for Reporting Tests (CONSORT) statement, the overall quality of each report was assessed. RESULTS: Sixty-two RCTs were retrieved in this study. The median of overall quality score in 2010 was 14 (range: 8.5-20). The extent of compliance with the Consolidated Standards of Reporting Trials reporting guideline differed substantially across items: 9 items were reported adequately (more than 90%), and 3 were reported adequately in less than 10% of trials. Multivariate linear regression analysis showed that the higher reporting scores were associated with higher journal impact factor (P = 0.01), international collaboration (P < 0.01), and Sources of trial funding (P = 0.02). CONCLUSIONS: Although a large number of randomized controlled trials of NOACs for the treatment of AF were published after the CONSORT statement in 2010, the overall quality is still not satisfactory, thus weakening their potential utility and may mislead clinical decisions. This survey provides the first hint for researchers conducting trials of NOACs for AF to improve the quality of reports and to actively apply the CONSORT statement.

Reporting quality of interventions using a wearable activity tracker to improve physical activity in patients with inflammatory arthritis or osteoarthritis: a systematic review.

To assess the reporting quality of interventions aiming at promoting physical activity (PA) using a wearable activity tracker (WAT) in patients with inflammatory arthritis (IA) or hip/knee osteoarthritis (OA). A systematic search was performed in eight databases (including PubMed, Embase and Cochrane Library) for studies published between 2000 and 2022. Two reviewers independently selected studies and extracted data on study characteristics and the reporting of the PA intervention using a WAT using the Consensus on Exercise Reporting Template (CERT) (12 items) and Consolidated Standards of Reporting Trials (CONSORT) E-Health checklist (16 items). The reporting quality of each study was expressed as a percentage of reported items of the total CERT and CONSORT E-Health (50% or less = poor; 51-79% = moderate; and 80-100% = good reporting quality). Sixteen studies were included; three involved patients with IA and 13 with OA. Reporting quality was poor in 6/16 studies and moderate in 10/16 studies, according to the CERT and poor in 8/16 and moderate in 8/16 studies following the CONSORT E-Health checklist. Poorly reported checklist items included: the description of decision rule(s) for determining progression and the starting level, the number of adverse events and how adherence or fidelity was assessed. In clinical trials on PA interventions using a WAT in patients with IA or OA, the reporting quality of delivery process is moderate to poor. The poor reporting quality of the progression and tailoring of the PA programs makes replication difficult. Improvements in reporting quality are necessary.

Methodological and reporting quality of systematic reviews on health effects of air pollutants were higher than extreme temperatures: a comparative study.

BACKGROUND: An increasing number of systematic reviews (SRs) in the environmental field have been published in recent years as a result of the global concern about the health impacts of air pollution and temperature. However, no study has assessed and compared the methodological and reporting quality of SRs on the health effects of air pollutants and extreme temperatures. This study aims to assess and compare the methodological and reporting quality of SRs on the health effects of ambient air pollutants and extreme temperatures. METHODS: PubMed, Embase, the Cumulative Index to Nursing and Allied Health Literature (CINAHL), Cochrane Library, Web of Science, and Epistemonikos databases were searched. Two researchers screened the literature and extracted information independently. The methodological quality of the SRs was assessed through A Measurement Tool to Assess Systematic Reviews 2 (AMSTAR 2). The reporting quality was assessed through Preferred Reporting Items of Systematic reviews and Meta-Analyses (PRISMA). RESULTS: We identified 405 SRs (286 for air pollution, 108 for temperature, and 11 for the synergistic effects). The methodological and reporting quality of the included SRs were suboptimal, with major deficiencies in protocol registration. The methodological quality of SRs of air pollutants was better than that of temperature, especially in terms of satisfactory explanations for any heterogeneity (69.6% v. 45.4%). The reporting quality of SRs of air pollution was better than temperature, however, adherence to the reporting of the assessment results of risk of bias in all SRs (53.5% v. 34.3%) was inadequate. CONCLUSIONS: Methodological and reporting quality of SRs on the health effect of air pollutants were higher than those of temperatures. However, deficiencies in protocol registration and the assessment of risk of bias remain an issue for both pollutants and temperatures. In addition, developing a risk-of-bias assessment tool applicable to the temperature field may improve the quality of SRs.

How does research activity align with research need in chronic subdural haematoma: a gap analysis of systematic reviews with end-user selected knowledge gaps.

BACKGROUND: Chronic subdural haematoma (CSDH) is increasingly common. Although treatment is triaged and provided by neurosurgery, the role of non-operative care, alongside observed peri-operative morbidity and patient complexity, suggests that optimum care requires a multi-disciplinary approach. A UK consortium (Improving Care in Elderly Neurosurgery Initiative [ICENI]) has been formed to develop the first comprehensive clinical practice guideline. This starts by identifying critical questions to ask of the literature. The aim of this review was to consider whether existing systematic reviews had suitably addressed these questions. METHODS: Critical research questions to inform CSDH care were identified using multi-stakeholder workshops, including patient and public representation. A CSDH umbrella review of full-text systematic reviews and meta-analysis was conducted in accordance with the PRISMA statement (CRD42022328562). Four databases were searched from inception up to 30 April 2022. Review quality was assessed using AMSTAR-2 criteria, mapped to critical research questions. RESULTS: Forty-four critical research questions were identified, across 12 themes. Seventy-three articles were included in the umbrella review, comprising 206,369 patients. Most reviews (86.3%, n=63) assessed complications and recurrence after surgery. ICENI themes were not addressed in current literature, and duplication of reviews was common (54.8%, n=40). AMSTAR-2 confidence rating was high in 7 (9.6%) reviews, moderate in 8 (11.0%), low in 10 (13.7%) and critically low in 48 (65.8%). CONCLUSIONS: The ICENI themes have yet to be examined in existing secondary CSDH literature, and a series of new reviews is now required to address these questions for a clinical practice guideline. There is a need to broaden and redirect research efforts to meet the organisation of services and clinical needs of individual patients.

Reporting quality of systematic reviews with network meta-analyses in Endodontics.

OBJECTIVES: To evaluate the reporting quality of systematic reviews with network meta-analyses (NMAs) in Endodontics using the the Preferred Reporting Items for Systematic reviews and Meta-analyses (PRISMA) for NMA checklist. METHODS: The current investigation extends a recently published study in the International Endodontic Journal (Nagendrababu V, Faggion Jr CM, Pulikkotil SJ, Alatta A, Dummer PM Methodological assessment and overall confidence in the results of systematic reviews with network meta-analyses in Endodontics. International Endodontic Journal 2022;55:393-404) that assessed the methodological quality of systematic reviews with NMAs in Endodontics using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR 2) tool. In the present study, the PRISMA for NMA checklist with 32 items was used to assess the reporting quality of the systematic reviews with NMAs (n = 12). Two independent assessors assigned '1' when an item was completely addressed, '0.5' when it was partially addressed, and '0' when it was not addressed. Disagreements were resolved through reviewer discussion until consensus was reached. If conflicts persisted, a third reviewer made the final decision. The PRISMA for NMA scores were shared with the relevant authors of the individual reviews to reduce the likelihood of misinterpretation and verify the scores assigned. The results for each individual item of the PRISMA-NMA items were calculated by summing the individual scores awarded; the maximum score for each item was 12. RESULTS: All the systematic reviews with NMAs adequately reported the following items: Title, Introduction section (Objectives), Methods section (Eligibility criteria and Information sources), Results section (Study selection, Study characteristics and Risk of bias within studies), and Discussion section (Summary of evidence). The items that were reported least often were the "geometry of the network" and "the summary of network geometry" with only 2 manuscripts (17%) including these items. CONCLUSION: A number of the items in the PRISMA-NMA checklist were adequately addressed in the NMAs; however, none adequately reported all the PRISMA-NMA items. The inadequacies of published NMAs that have been identified should be taken into consideration by authors of NMAs in Endodontics and by editors when managing the peer review process. In future, researchers who are writing systematic reviews with NMAs should comply with the PRISMA-NMA checklist. CLINICAL RELEVANCE: None of the included systematic reviews with NMA adequately reported all the PRISMA-NMA items. Inadequate reporting of a systematic review with NMA increases the possibility that it will provide invalid results. Therefore, authors should follow the PRISMA-NMA guidelines when reporting systematic reviews with NMA in Endodontics.