Assessment of transparency indicators in space medicine.

Space medicine is a vital discipline with often time-intensive and costly projects and constrained opportunities for studying various elements such as space missions, astronauts, and simulated environments. Moreover, private interests gain increasing influence in this discipline. In scientific disciplines with these features, transparent and rigorous methods are essential. Here, we undertook an evaluation of transparency indicators in publications within the field of space medicine. A meta-epidemiological assessment of PubMed Central Open Access (PMC OA) eligible articles within the field of space medicine was performed for prevalence of code sharing, data sharing, pre-registration, conflicts of interest, and funding. Text mining was performed with the rtransparent text mining algorithms with manual validation of 200 random articles to obtain corrected estimates. Across 1215 included articles, 39 (3%) shared code, 258 (21%) shared data, 10 (1%) were registered, 110 (90%) contained a conflict-of-interest statement, and 1141 (93%) included a funding statement. After manual validation, the corrected estimates for code sharing, data sharing, and registration were 5%, 27%, and 1%, respectively. Data sharing was 32% when limited to original articles and highest in space/parabolic flights (46%). Overall, across space medicine we observed modest rates of data sharing, rare sharing of code and almost non-existent protocol registration. Enhancing transparency in space medicine research is imperative for safeguarding its scientific rigor and reproducibility.

Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics.

Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

Estimating the extent of selective reporting: An application to economics.

Using a sample of 70,399 published p-values from 192 meta-analyses, we empirically estimate the counterfactual distribution of p-values in the absence of any biases. Comparing observed p-values with counterfactually expected p-values allows us to estimate how many p-values are published as being statistically significant when they should have been published as non-significant. We estimate the extent of selectively reported p-values to range between 57.7% and 71.9% of the significant p-values. The counterfactual p-value distribution also allows us to assess shifts of p-values along the entire distribution of published p-values, revealing that particularly very small p-values (p < 0.001) are unexpectedly abundant in the published literature. Subsample analysis suggests that the extent of selective reporting is reduced in research fields that use experimental designs, analyze microeconomics research questions, and have at least some adequately powered studies.

Principles for good scholarship in systematic reviews.

Many sources document problems that jeopardize the trustworthiness of systematic reviews. This is a major concern given their potential to influence patient care and impact people's lives. Responsibility for producing trustworthy conclusions on the evidence in systematic reviews is borne primarily by authors who need the necessary training and resources to correctly report on the current knowledge base. Peer reviewers and editors are also accountable; they must ensure that systematic reviews are accurate by demonstrating proper methods. To support all these stakeholders, we attempt to distill the sprawling guidance that is currently available in our recent co-publication about best tools and practices for systematic reviews. We specifically address how to meet methodological conduct standards applicable to key components of systematic reviews. In this complementary invited review, we place these standards in the context of good scholarship principles for systematic review development. Our intention is to reach a broad audience and potentially improve the trustworthiness of evidence syntheses published in the developmental medicine literature and beyond.

Quantitative research assessment: using metrics against gamed metrics.

Quantitative bibliometric indicators are widely used and widely misused for research assessments. Some metrics have acquired major importance in shaping and rewarding the careers of millions of scientists. Given their perceived prestige, they may be widely gamed in the current "publish or perish" or "get cited or perish" environment. This review examines several gaming practices, including authorship-based, citation-based, editorial-based, and journal-based gaming as well as gaming with outright fabrication. Different patterns are discussed, including massive authorship of papers without meriting credit (gift authorship), team work with over-attribution of authorship to too many people (salami slicing of credit), massive self-citations, citation farms, H-index gaming, journalistic (editorial) nepotism, journal impact factor gaming, paper mills and spurious content papers, and spurious massive publications for studies with demanding designs. For all of those gaming practices, quantitative metrics and analyses may be able to help in their detection and in placing them into perspective. A portfolio of quantitative metrics may also include indicators of best research practices (e.g., data sharing, code sharing, protocol registration, and replications) and poor research practices (e.g., signs of image manipulation). Rigorous, reproducible, transparent quantitative metrics that also inform about gaming may strengthen the legacy and practices of quantitative appraisals of scientific work.

An umbrella review of systematic reviews on the impact of the COVID-19 pandemic on cancer prevention and management, and patient needs.

The relocation and reconstruction of health care resources and systems during the coronavirus disease 2019 (COVID-19) pandemic may have affected cancer care. An umbrella review was undertaken to summarize the findings from systematic reviews on impact of the COVID-19 pandemic on cancer treatment modification, delays, and cancellations; delays or cancellations in screening and diagnosis; psychosocial well-being, financial distress, and use of telemedicine as well as on other aspects of cancer care. Bibliographic databases were searched for relevant systematic reviews with or without meta-analysis published before November 29th, 2022. Abstract, full- text screening, and data extraction were performed by two independent reviewers. AMSTAR-2 was used for critical appraisal of included systematic reviews. Fifty-one systematic reviews were included in our analysis. Most reviews were based on observational studies judged to be at medium and high risk of bias. Only two reviews had high or moderate scores based on AMSTAR-2. Findings suggest treatment modifications in cancer care during the pandemic versus the pre-pandemic period were based on low level of evidence. Different degrees of delays and cancellations in cancer treatment, screening, and diagnosis were observed, with low- and- middle- income countries and countries that implemented lockdowns being disproportionally affected. A shift from in-person appointments to telemedicine use was observed, but utility of telemedicine, challenges in implementation and cost-effectiveness in cancer care were little explored. Evidence was consistent in suggesting psychosocial well-being of patients with cancer deteriorated, and cancer patients experienced financial distress, albeit results were in general not compared to pre-pandemic levels. Impact of cancer care disruption during the pandemic on cancer prognosis was little explored. In conclusion, substantial but heterogenous impact of COVID-19 pandemic on cancer care has been observed.

The onset of the COVID-19 pandemic disrupted many aspects of human life, not least healthcare. As resources were redistributed towards the crisis, social isolation rules also limited access to medical professionals. In particular, these measures may have affected many aspects of cancer care, such as early detection or treatment. Many studies have aimed to capture the impact of these changes, but most have been observational, with researchers recording events without trying to impose a controlled design. These investigations also often faced limitations such as small sample sizes, or only focusing on one aspect of cancer care. Systemic reviews, which synthetize and assess existing research on a topic, have helped to bypass these constraints. However, they are themselves not devoid of biases. Overall, a clear, unified picture of the impact of COVID-19 on cancer care is yet to emerge. In response, Muka et al. carried an umbrella analysis of 51 systematic reviews on this topic. They used a well-known critical appraisal tool to assess the methodological rigor of each of these studies, while also summarising their findings. This work aimed to capture many aspects of the patients' experience, from diagnosis to treatment and the financial, psychological, physical and social impact of the disease. The results confirmed that the pandemic had a substantial impact on cancer care, including delays in screening, diagnosis and treatment. Throughout this period cancer patients experienced increased rates of depression, post-traumatic stress and fear of their cancer progressing. The long-term consequences of these disruptions remain to be uncovered. However, Muka et al. also showed that, overall, these conclusions rely on low-quality studies which may have introduced unaccountable biases. In addition, their review highlights that most of the data currently available has been collected in high- and middle-income countries, with evidence lacking from regions of the world with more limited resources. In the short-term, these results indicate that interventions may be needed to mitigate the negative impact of the pandemic on cancer care; in the long-term, they also demonstrate the importance of rigorous systematic reviews in guiding decision making. By shining a light on the ripple effects of certain decisions about healthcare resources, this work could also help to shape the response to future pandemics.

A meta-epidemiological assessment of transparency indicators of infectious disease models.

Mathematical models have become very influential, especially during the COVID-19 pandemic. Data and code sharing are indispensable for reproducing them, protocol registration may be useful sometimes, and declarations of conflicts of interest (COIs) and of funding are quintessential for transparency. Here, we evaluated these features in publications of infectious disease-related models and assessed whether there were differences before and during the COVID-19 pandemic and for COVID-19 models versus models for other diseases. We analysed all PubMed Central open access publications of infectious disease models published in 2019 and 2021 using previously validated text mining algorithms of transparency indicators. We evaluated 1338 articles: 216 from 2019 and 1122 from 2021 (of which 818 were on COVID-19); almost a six-fold increase in publications within the field. 511 (39.2%) were compartmental models, 337 (25.2%) were time series, 279 (20.9%) were spatiotemporal, 186 (13.9%) were agent-based and 25 (1.9%) contained multiple model types. 288 (21.5%) articles shared code, 332 (24.8%) shared data, 6 (0.4%) were registered, and 1197 (89.5%) and 1109 (82.9%) contained COI and funding statements, respectively. There was no major changes in transparency indicators between 2019 and 2021. COVID-19 articles were less likely to have funding statements and more likely to share code. Further validation was performed by manual assessment of 10% of the articles identified by text mining as fulfilling transparency indicators and of 10% of the articles lacking them. Correcting estimates for validation performance, 26.0% of papers shared code and 41.1% shared data. On manual assessment, 5/6 articles identified as registered had indeed been registered. Of articles containing COI and funding statements, 95.8% disclosed no conflict and 11.7% reported no funding. Transparency in infectious disease modelling is relatively low, especially for data and code sharing. This is concerning, considering the nature of this research and the heightened influence it has acquired.

Aggressive measures, rising inequalities, and mass formation during the COVID-19 crisis: An overview and proposed way forward.

A series of aggressive restrictive measures were adopted around the world in 2020-2022 to attempt to prevent SARS-CoV-2 from spreading. However, it has become increasingly clear the most aggressive (lockdown) response strategies may involve negative side-effects such as a steep increase in poverty, hunger, and inequalities. Several economic, educational, and health repercussions have fallen disproportionately on children, students, young workers, and especially on groups with pre-existing inequalities such as low-income families, ethnic minorities, and women. This has led to a vicious cycle of rising inequalities and health issues. For example, educational and financial security decreased along with rising unemployment and loss of life purpose. Domestic violence surged due to dysfunctional families being forced to spend more time with each other. In the current narrative and scoping review, we describe macro-dynamics that are taking place because of aggressive public health policies and psychological tactics to influence public behavior, such as mass formation and crowd behavior. Coupled with the effect of inequalities, we describe how these factors can interact toward aggravating ripple effects. In light of evidence regarding the health, economic and social costs, that likely far outweigh potential benefits, the authors suggest that, first, where applicable, aggressive lockdown policies should be reversed and their re-adoption in the future should be avoided. If measures are needed, these should be non-disruptive. Second, it is important to assess dispassionately the damage done by aggressive measures and offer ways to alleviate the burden and long-term effects. Third, the structures in place that have led to counterproductive policies should be assessed and ways should be sought to optimize decision-making, such as counteracting groupthink and increasing the level of reflexivity. Finally, a package of scalable positive psychology interventions is suggested to counteract the damage done and improve humanity's prospects.

Post-publication critique at top-ranked journals across scientific disciplines: a cross-sectional assessment of policies and practice.

Journals exert considerable control over letters, commentaries and online comments that criticize prior research (post-publication critique). We assessed policies (Study One) and practice (Study Two) related to post-publication critique at 15 top-ranked journals in each of 22 scientific disciplines (N = 330 journals). Two-hundred and seven (63%) journals accepted post-publication critique and often imposed limits on length (median 1000, interquartile range (IQR) 500-1200 words) and time-to-submit (median 12, IQR 4-26 weeks). The most restrictive limits were 175 words and two weeks; some policies imposed no limits. Of 2066 randomly sampled research articles published in 2018 by journals accepting post-publication critique, 39 (1.9%, 95% confidence interval [1.4, 2.6]) were linked to at least one post-publication critique (there were 58 post-publication critiques in total). Of the 58 post-publication critiques, 44 received an author reply, of which 41 asserted that original conclusions were unchanged. Clinical Medicine had the most active culture of post-publication critique: all journals accepted post-publication critique and published the most post-publication critique overall, but also imposed the strictest limits on length (median 400, IQR 400-550 words) and time-to-submit (median 4, IQR 4-6 weeks). Our findings suggest that top-ranked academic journals often pose serious barriers to the cultivation, documentation and dissemination of post-publication critique.

Preventive psychiatry: a blueprint for improving the mental health of young people.

Preventive approaches have latterly gained traction for improving mental health in young people. In this paper, we first appraise the conceptual foundations of preventive psychiatry, encompassing the public health, Gordon's, US Institute of Medicine, World Health Organization, and good mental health frameworks, and neurodevelopmentally-sensitive clinical staging models. We then review the evidence supporting primary prevention of psychotic, bipolar and common mental disorders and promotion of good mental health as potential transformative strategies to reduce the incidence of these disorders in young people. Within indicated approaches, the clinical high-risk for psychosis paradigm has received the most empirical validation, while clinical high-risk states for bipolar and common mental disorders are increasingly becoming a focus of attention. Selective approaches have mostly targeted familial vulnerability and non-genetic risk exposures. Selective screening and psychological/psychoeducational interventions in vulnerable subgroups may improve anxiety/depressive symptoms, but their efficacy in reducing the incidence of psychotic/bipolar/common mental disorders is unproven. Selective physical exercise may reduce the incidence of anxiety disorders. Universal psychological/psychoeducational interventions may improve anxiety symptoms but not prevent depressive/anxiety disorders, while universal physical exercise may reduce the incidence of anxiety disorders. Universal public health approaches targeting school climate or social determinants (demographic, economic, neighbourhood, environmental, social/cultural) of mental disorders hold the greatest potential for reducing the risk profile of the population as a whole. The approach to promotion of good mental health is currently fragmented. We leverage the knowledge gained from the review to develop a blueprint for future research and practice of preventive psychiatry in young people: integrating universal and targeted frameworks; advancing multivariable, transdiagnostic, multi-endpoint epidemiological knowledge; synergically preventing common and infrequent mental disorders; preventing physical and mental health burden together; implementing stratified/personalized prognosis; establishing evidence-based preventive interventions; developing an ethical framework, improving prevention through education/training; consolidating the cost-effectiveness of preventive psychiatry; and decreasing inequalities. These goals can only be achieved through an urgent individual, societal, and global level response, which promotes a vigorous collaboration across scientific, health care, societal and governmental sectors for implementing preventive psychiatry, as much is at stake for young people with or at risk for emerging mental disorders.

EMA and FDA psychiatric drug trial guidelines: assessment of guideline development and trial design recommendations.

AIMS: The European Medicines Agency (EMA) and the US Food and Drug Administration (FDA) produce guidelines for the design of pivotal psychiatric drug trials used in new drug applications. It is unknown who are involved in the guideline development and what specific trial design recommendations they give. METHODS: Cross-sectional study of EMA Clinical Efficacy and Safety Guidelines and FDA Guidance Documents. Study outcomes: (1) guideline committee members and declared conflicts of interest; (2) guideline development and organisation of commenting phases; (3) categorisation of stakeholders who comment on draft and final guidelines according to conflicts of interest ('industry', 'not-industry but with industry-related conflicts', 'independent', 'unclear'); and (4) trial design recommendations (trial duration, psychiatric comorbidity, 'enriched design', efficacy outcomes, comparator choice). Protocol registration https://doi.org/10.1101/2020.01.22.20018499 (27 January 2020). RESULTS: We included 13 EMA and five FDA guidelines covering 15 psychiatric indications. Eleven months after submission, the EMA had not processed our request regarding committee member disclosures. FDA offices draft the Guidance Documents, but the Agency is not in possession of employee conflicts of interest declarations because FDA employees generally may not hold financial interests (although some employees may hold interests up to $15,000). The EMA and FDA guideline development phases are similar; drafts and final versions are publicly announced and everybody can submit comments. Seventy stakeholders commented on ten guidelines: 38 (54%) 'industry', 18 (26%) 'not-industry but with industry-related conflicts', six (9%) 'independent' and eight (11%) 'unclear'. They submitted 1014 comments: 640 (68%) 'industry', 243 (26%) 'not-industry but with industry-related conflicts', 44 (5%) 'independent' and 20 (2%) 'unclear' (67 could not be assigned to a specific stakeholder). The recommended designs were generally for trials of short duration; with restricted trial populations; allowing previous exposure to the drug; and often recommending rating scale efficacy outcomes. EMA mainly recommended three arm designs (both placebo and active comparators), whereas FDA mainly recommended placebo-controlled designs. There were also other important differences and FDA's recommendations regarding the exclusion of psychiatric comorbidity seemed less restrictive. CONCLUSIONS: The EMA and FDA clinical research guidelines for psychiatric pivotal trials recommend designs that tend to have limited generalisability. Independent and non-conflicted stakeholders are underrepresented in the guideline development. It seems warranted with more active involvement of scientists and independent organisations without conflicts of interest in the guideline development process.

Treatment effects in randomised trials using routinely collected data for outcome assessment versus traditional trials: meta-research study.

OBJECTIVE: To compare effect estimates of randomised clinical trials that use routinely collected data (RCD-RCT) for outcome ascertainment with traditional trials not using routinely collected data. DESIGN: Meta-research study. DATA SOURCE: Studies included in the same meta-analysis in a Cochrane review. ELIGIBILITY CRITERIA FOR STUDY SELECTION: Randomised clinical trials using any type of routinely collected data for outcome ascertainment, including from registries, electronic health records, and administrative databases, that were included in a meta-analysis of a Cochrane review on any clinical question and any health outcome together with traditional trials not using routinely collected data for outcome measurement. REVIEW METHODS: Effect estimates from trials using or not using routinely collected data were summarised in random effects meta-analyses. Agreement of (summary) treatment effect estimates from trials using routinely collected data and those not using such data was expressed as the ratio of odds ratios. Subgroup analyses explored effects in trials based on different types of routinely collected data. Two investigators independently assessed the quality of each data source. RESULTS: 84 RCD-RCTs and 463 traditional trials on 22 clinical questions were included. Trials using routinely collected data for outcome ascertainment showed 20% less favourable treatment effect estimates than traditional trials (ratio of odds ratios 0.80, 95% confidence interval 0.70 to 0.91, I2=14%). Results were similar across various types of outcomes (mortality outcomes: 0.92, 0.74 to 1.15, I2=12%; non-mortality outcomes: 0.71, 0.60 to 0.84, I2=8%), data sources (electronic health records: 0.81, 0.59 to 1.11, I2=28%; registries: 0.86, 0.75 to 0.99, I2=20%; administrative data: 0.84, 0.72 to 0.99, I2=0%), and data quality (high data quality: 0.82, 0.72 to 0.93, I2=0%). CONCLUSIONS: Randomised clinical trials using routinely collected data for outcome ascertainment show smaller treatment benefits than traditional trials not using routinely collected data. These differences could have implications for healthcare decision making and the application of real world evidence.

Correction: Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature.

[This corrects the article DOI: 10.1371/journal.pbio.2000797.].

Assessment of transparency indicators across the biomedical literature: How open is open?

Recent concerns about the reproducibility of science have led to several calls for more open and transparent research practices and for the monitoring of potential improvements over time. However, with tens of thousands of new biomedical articles published per week, manually mapping and monitoring changes in transparency is unrealistic. We present an open-source, automated approach to identify 5 indicators of transparency (data sharing, code sharing, conflicts of interest disclosures, funding disclosures, and protocol registration) and apply it across the entire open access biomedical literature of 2.75 million articles on PubMed Central (PMC). Our results indicate remarkable improvements in some (e.g., conflict of interest [COI] disclosures and funding disclosures), but not other (e.g., protocol registration and code sharing) areas of transparency over time, and map transparency across fields of science, countries, journals, and publishers. This work has enabled the creation of a large, integrated, and openly available database to expedite further efforts to monitor, understand, and promote transparency and reproducibility in science.

Precision shielding for COVID-19: metrics of assessment and feasibility of deployment.

The ability to preferentially protect high-risk groups in COVID-19 is hotly debated. Here, the aim is to present simple metrics of such precision shielding of people at high risk of death after infection by SARS-CoV-2; demonstrate how they can estimated; and examine whether precision shielding was successfully achieved in the first COVID-19 wave. The shielding ratio, S, is defined as the ratio of prevalence of infection among people in a high-risk group versus among people in a low-risk group. The contrasted risk groups examined here are according to age (≥70 vs <70 years), and institutionalised (nursing home) setting. For age-related precision shielding, data were used from large seroprevalence studies with separate prevalence data for elderly versus non-elderly and with at least 1000 assessed people≥70 years old. For setting-related precision shielding, data were analysed from 10 countries where information was available on numbers of nursing home residents, proportion of nursing home residents among COVID-19 deaths and overall population infection fatality rate (IFR). Across 17 seroprevalence studies, the shielding ratio S for elderly versus non-elderly varied between 0.4 (substantial shielding) and 1.6 (substantial inverse protection, that is, low-risk people being protected more than high-risk people). Five studies in the USA all yielded S=0.4-0.8, consistent with some shielding being achieved, while two studies in China yielded S=1.5-1.6, consistent with inverse protection. Assuming 25% IFR among nursing home residents, S values for nursing home residents ranged from 0.07 to 3.1. The best shielding was seen in South Korea (S=0.07) and modest shielding was achieved in Israel, Slovenia, Germany and Denmark. No shielding was achieved in Hungary and Sweden. In Belgium (S=1.9), the UK (S=2.2) and Spain (S=3.1), nursing home residents were far more frequently infected than the rest of the population. In conclusion, the experience from the first wave of COVID-19 suggests that different locations and settings varied markedly in the extent to which they protected high-risk groups. Both effective precision shielding and detrimental inverse protection can happen in real-life circumstances. COVID-19 interventions should seek to achieve maximal precision shielding.

Reporting of demographic data and representativeness in machine learning models using electronic health records.

OBJECTIVE: The development of machine learning (ML) algorithms to address a variety of issues faced in clinical practice has increased rapidly. However, questions have arisen regarding biases in their development that can affect their applicability in specific populations. We sought to evaluate whether studies developing ML models from electronic health record (EHR) data report sufficient demographic data on the study populations to demonstrate representativeness and reproducibility. MATERIALS AND METHODS: We searched PubMed for articles applying ML models to improve clinical decision-making using EHR data. We limited our search to papers published between 2015 and 2019. RESULTS: Across the 164 studies reviewed, demographic variables were inconsistently reported and/or included as model inputs. Race/ethnicity was not reported in 64%; gender and age were not reported in 24% and 21% of studies, respectively. Socioeconomic status of the population was not reported in 92% of studies. Studies that mentioned these variables often did not report if they were included as model inputs. Few models (12%) were validated using external populations. Few studies (17%) open-sourced their code. Populations in the ML studies include higher proportions of White and Black yet fewer Hispanic subjects compared to the general US population. DISCUSSION: The demographic characteristics of study populations are poorly reported in the ML literature based on EHR data. Demographic representativeness in training data and model transparency is necessary to ensure that ML models are deployed in an equitable and reproducible manner. Wider adoption of reporting guidelines is warranted to improve representativeness and reproducibility.

Empirical assessment of bias in machine learning diagnostic test accuracy studies.

OBJECTIVE: Machine learning (ML) diagnostic tools have significant potential to improve health care. However, methodological pitfalls may affect diagnostic test accuracy studies used to appraise such tools. We aimed to evaluate the prevalence and reporting of design characteristics within the literature. Further, we sought to empirically assess whether design features may be associated with different estimates of diagnostic accuracy. MATERIALS AND METHODS: We systematically retrieved 2 × 2 tables (n = 281) describing the performance of ML diagnostic tools, derived from 114 publications in 38 meta-analyses, from PubMed. Data extracted included test performance, sample sizes, and design features. A mixed-effects metaregression was run to quantify the association between design features and diagnostic accuracy. RESULTS: Participant ethnicity and blinding in test interpretation was unreported in 90% and 60% of studies, respectively. Reporting was occasionally lacking for rudimentary characteristics such as study design (28% unreported). Internal validation without appropriate safeguards was used in 44% of studies. Several design features were associated with larger estimates of accuracy, including having unreported (relative diagnostic odds ratio [RDOR], 2.11; 95% confidence interval [CI], 1.43-3.1) or case-control study designs (RDOR, 1.27; 95% CI, 0.97-1.66), and recruiting participants for the index test (RDOR, 1.67; 95% CI, 1.08-2.59). DISCUSSION: Significant underreporting of experimental details was present. Study design features may affect estimates of diagnostic performance in the ML diagnostic test accuracy literature. CONCLUSIONS: The present study identifies pitfalls that threaten the validity, generalizability, and clinical value of ML diagnostic tools and provides recommendations for improvement.