Safeguarding scientific integrity: A case study in examining manipulation in the peer review process.

This case study analyzes the expertise, potential conflicts of interest, and objectivity of editors, authors, and peer reviewers involved in a 2022 special journal issue on fertility, pregnancy, and mental health. Data were collected on qualifications, organizational affiliations, and relationships among six papers' authors, three guest editors, and twelve peer reviewers. Two articles were found to have undisclosed conflicts of interest between authors, an editor, and multiple peer reviewers affiliated with anti-abortion advocacy and lobbying groups, indicating compromised objectivity. This lack of transparency undermines the peer review process and enables biased research and disinformation proliferation.Our study is limited by a few factors including: difficulty collecting peer reviewer data, potentially missing affiliations, and a small sample without comparisons. While this is a case study of one special issue, we do have suggestions for increasing integrity.

Coordinating culture change across the research landscape.

Scientific integrity necessitates applying scientific methods properly, collecting and analyzing data appropriately, protecting human subjects rightly, performing studies rigorously, and communicating findings transparently. But who is responsible for upholding research integrity, mitigating misinformation, and increasing trust in science beyond individual researchers? We posit that supporting the scientific reputation requires a coordinated approach across all stakeholders: funding agencies, publishers, scholarly societies, research institutions, and journalists and media, and policy-makers.

Guiding principles for technical infrastructure to support computable biomedical knowledge.

Over the past 4 years, the authors have participated as members of the Mobilizing Computable Biomedical Knowledge Technical Infrastructure working group and focused on conceptualizing the infrastructure required to use computable biomedical knowledge. Here, we summarize our thoughts and lay the foundation for future work in the development of CBK infrastructure, including: explaining the difference between computable knowledge and data, and contextualizing the conversation with the Learning Health Systems and the FAIR principles. Specifically, we provide three guiding principles to advance the development of CBK infrastructure: (a) Promote interoperable systems for data and knowledge to be findable, accessible, interoperable, and reusable. (b) Enable stable, trustworthy knowledge representations that are human and machine readable. (c) Computable knowledge resources should, when possible, be open. Standards supporting computable knowledge infrastructures must be open.

Summary of third annual MCBK public meeting: Mobilizing computable biomedical knowledge-Accelerating the second knowledge revolution.

The volume of biomedical knowledge is growing exponentially and much of this knowledge is represented in computer executable formats, such as models, algorithms, and programmatic code. There is a growing need to apply this knowledge to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations do not yet have the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are not sufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community was formed in 2016 to address these needs. This report summarizes the main outputs of the third annual MCBK public meeting, which was held virtually from June 30 to July 1, 2020 and brought together over 200 participants from various domains to frame and address important dimensions for mobilizing CBK.

RipetaScore: Measuring the Quality, Transparency, and Trustworthiness of a Scientific Work.

A wide array of existing metrics quantifies a scientific paper's prominence or the author's prestige. Many who use these metrics make assumptions that higher citation counts or more public attention must indicate more reliable, better quality science. While current metrics offer valuable insight into scientific publications, they are an inadequate proxy for measuring the quality, transparency, and trustworthiness of published research. Three essential elements to establishing trust in a work include: trust in the paper, trust in the author, and trust in the data. To address these elements in a systematic and automated way, we propose the ripetaScore as a direct measurement of a paper's research practices, professionalism, and reproducibility. Using a sample of our current corpus of academic papers, we demonstrate the ripetaScore's efficacy in determining the quality, transparency, and trustworthiness of an academic work. In this paper, we aim to provide a metric to evaluate scientific reporting quality in terms of transparency and trustworthiness of the research, professionalism, and reproducibility.

Summary of second annual MCBK public meeting: Mobilizing Computable Biomedical Knowledge-A movement to accelerate translation of knowledge into action.

The volume of biomedical knowledge is growing exponentially and much of this knowledge is represented in computer executable formats, such as models, algorithms and programmatic code. There is a growing need to apply this knowledge to improve health in Learning Health Systems, health delivery organizations, and other settings. However, most organizations do not yet have the infrastructure required to consume and apply computable knowledge, and national policies and standards adoption are not sufficient to ensure that it is discoverable and used safely and fairly, nor is there widespread experience in the process of knowledge implementation as clinical decision support. The Mobilizing Computable Biomedical Knowledge (MCBK) community formed in 2016 to address these needs. This report summarizes the main outputs of the Second Annual MCBK public meeting, which was held at the National Institutes of Health on July 18-19, 2019 and brought together over 150 participants from various domains to frame and address important dimensions for mobilizing CBK.

Tobacco Use Prevalence and Smoking Cessation Pharmacotherapy Prescription Patterns Among Hospitalized Patients by Medical Specialty.

INTRODUCTION: Effective smoking cessation medications are readily available but may be underutilized in hospital settings. In our large, tertiary care hospital, we aimed to (1) characterize patient tobacco use prevalence across medical specialties, (2) determine smoking cessation pharmacotherapy prescription variation across specialties, and (3) identify opportunities for improvement in practice. METHODS: Using electronic health records at Barnes Jewish Hospital, we gathered demographic data, admitting service, admission route, length of stay, self-reported tobacco use, and smoking cessation prescriptions over a 6-year period, from 2010 to 2016. We then compared tobacco use prevalence and smoking cessation prescriptions across medical specialties using a cross-sectional, retrospective design. RESULTS: Past 12-month tobacco use was reported by patients in 27.9% of inpatient admissions; prescriptions for smoking cessation pharmacotherapy were provided during 21.5% of these hospitalizations. The proportion of patients reporting tobacco use was highest in psychiatry (55.3%) and lowest in orthopedic surgery (17.1%). Psychiatric patients who reported tobacco use were most likely to receive pharmacotherapy (71.8% of admissions), and plastic surgery patients were least likely (4.7% of admissions). Compared with Caucasian tobacco users, African American patients who used tobacco products were less likely to receive smoking cessation medications (adjusted odds ratio [aOR] = 0.65; 95% confidence interval [CI] = 0.62 to 0.68). CONCLUSIONS: Among hospitalized tobacco users, safe and cost-effective pharmacotherapies are under-prescribed. We identified substantial variation in prescribing practices across different medical specialties and demographic groups, suggesting the need for an electronic medical record protocol that facilitates consistent tobacco use cessation pharmacotherapy treatment. IMPLICATIONS: Tobacco use cessation pharmacotherapy is underutilized during hospitalization, and prescription rates vary greatly across medical specialties and patient characteristics. Hospitals may benefit from implementing policies and practices that standardize and automate the offer of smoking pharmacotherapy for all hospitalized patients who use tobacco.

Sustainability considerations for clinical and translational research informatics infrastructure.

A robust biomedical informatics infrastructure is essential for academic health centers engaged in translational research. There are no templates for what such an infrastructure encompasses or how it is funded. An informatics workgroup within the Clinical and Translational Science Awards network conducted an analysis to identify the scope, governance, and funding of this infrastructure. After we identified the essential components of an informatics infrastructure, we surveyed informatics leaders at network institutions about the governance and sustainability of the different components. Results from 42 survey respondents showed significant variations in governance and sustainability; however, some trends also emerged. Core informatics components such as electronic data capture systems, electronic health records data repositories, and related tools had mixed models of funding including, fee-for-service, extramural grants, and institutional support. Several key components such as regulatory systems (e.g., electronic Institutional Review Board [IRB] systems, grants, and contracts), security systems, data warehouses, and clinical trials management systems were overwhelmingly supported as institutional infrastructure. The findings highlighted in this report are worth noting for academic health centers and funding agencies involved in planning current and future informatics infrastructure, which provides the foundation for a robust, data-driven clinical and translational research program.

Repeat: a framework to assess empirical reproducibility in biomedical research.

BACKGROUND: The reproducibility of research is essential to rigorous science, yet significant concerns of the reliability and verifiability of biomedical research have been recently highlighted. Ongoing efforts across several domains of science and policy are working to clarify the fundamental characteristics of reproducibility and to enhance the transparency and accessibility of research. METHODS: The aim of the proceeding work is to develop an assessment tool operationalizing key concepts of research transparency in the biomedical domain, specifically for secondary biomedical data research using electronic health record data. The tool (RepeAT) was developed through a multi-phase process that involved coding and extracting recommendations and practices for improving reproducibility from publications and reports across the biomedical and statistical sciences, field testing the instrument, and refining variables. RESULTS: RepeAT includes 119 unique variables grouped into five categories (research design and aim, database and data collection methods, data mining and data cleaning, data analysis, data sharing and documentation). Preliminary results in manually processing 40 scientific manuscripts indicate components of the proposed framework with strong inter-rater reliability, as well as directions for further research and refinement of RepeAT. CONCLUSIONS: The use of RepeAT may allow the biomedical community to have a better understanding of the current practices of research transparency and accessibility among principal investigators. Common adoption of RepeAT may improve reporting of research practices and the availability of research outputs. Additionally, use of RepeAT will facilitate comparisons of research transparency and accessibility across domains and institutions.

Incorporating Data Citation in a Biomedical Repository: An Implementation Use Case.

Research data is a dynamic and evolving entity and the ability to cite such data depends on recreating the same datasets utilized in the original research. Despite the availability of several existing technologies, most data repositories lack the necessary setup to recreate a point-in-time snapshot of data, let alone long-term sustainability of dynamic data without restoring an entire database. Through this project, we adopted a subset of the Research Data Alliance data citation working group recommendations to establish a robust informatics system supporting dynamic data and its use for reproducible research within our evolving clinical data repository. We implemented key recommendations: data versioning, times-stamping, query storing, query time-stamping, query PID, and data citation in one data repository, implemented entirely at the database level, and were able to successfully reproduce a previous dataset as it existed at a specific point-in-time using only the PID as provided in a citation.

Electronic nutritional intake assessment in patients with urolithiasis: A decision impact analysis.

PURPOSE: To evaluate a physician's impression of a urinary stone patient's dietary intake and whether it was dependent on the medium through which the nutritional data were obtained. Furthermore, we sought to determine if using an electronic food frequency questionnaire (FFQ) impacted dietary recommendations for these patients. MATERIALS AND METHODS: Seventy-six patients attended the Stone Clinic over a period of 6 weeks. Seventy-five gave consent for enrollment in our study. Patients completed an office-based interview with a fellowship-trained endourologist, and a FFQ administered on an iPad. The FFQ assessed intake of various dietary components related to stone development, such as oxalate and calcium. The urologists were blinded to the identity of patients' FFQ results. Based on the office-based interview and the FFQ results, the urologists provided separate assessments of the impact of nutrition and hydration on the patient's stone disease (nutrition impact score and hydration impact score, respectively) and treatment recommendations. Multivariate logistic regressions were used to compare pre-FFQ data to post-FFQ data. RESULTS: Higher FFQ scores for sodium (odds ratio [OR], 1.02; p=0.02) and fluids (OR, 1.03, p=0.04) were associated with a higher nutritional impact score. None of the FFQ parameters impacted hydration impact score. A higher FFQ score for oxalate (OR, 1.07; p=0.02) was associated with the addition of at least one treatment recommendation. CONCLUSIONS: Information derived from a FFQ can yield a significant impact on a physician's assessment of stone risks and decision for management of stone disease.

caTissue Suite to OpenSpecimen: Developing an extensible, open source, web-based biobanking management system.

The National Cancer Institute (NCI) Cancer Biomedical Informatics Grid® (caBIG®) program established standards and best practices for biorepository data management by creating an infrastructure to propagate biospecimen resource sharing while maintaining data integrity and security. caTissue Suite, a biospecimen data management software tool, has evolved from this effort. More recently, the caTissue Suite continues to evolve as an open source initiative known as OpenSpecimen. The essential functionality of OpenSpecimen includes the capture and representation of highly granular, hierarchically-structured data for biospecimen processing, quality assurance, tracking, and annotation. Ideal for multi-user and multi-site biorepository environments, OpenSpecimen permits role-based access to specific sets of data operations through a user-interface designed to accommodate varying workflows and unique user needs. The software is interoperable, both syntactically and semantically, with an array of other bioinformatics tools given its integration of standard vocabularies thus enabling research involving biospecimens. End-users are encouraged to share their day-to-day experiences in working with the application, thus providing to the community board insight into the needs and limitations which need be addressed. Users are also requested to review and validate new features through group testing environments and mock screens. Through this user interaction, application flexibility and interoperability have been recognized as necessary developmental focuses essential for accommodating diverse adoption scenarios and biobanking workflows to catalyze advances in biomedical research and operations. Given the diversity of biobanking practices and workforce roles, efforts have been made consistently to maintain robust data granularity while aiding user accessibility, data discoverability, and security within and across applications by providing a lower learning curve in using OpenSpecimen. Iterative development and testing cycles provide continuous maintenance and up-to-date capabilities for this freely available, open-access, web-based software application that is globally-adopted at over 25 institutions.

It's Just (Academic) Business: A Use Case in Improving Informatics Operations with Business Intelligence.

Academic biomedical informatics cores are beholden to funding agencies, institutional administration, collaborating researchers, and external agencies for ongoing funding and support. Services provided and translational research outcomes are increasingly important to monitor, report and analyze, to demonstrate value provided to the organization and the greater scientific community. Thus, informatics operations are also business operations. As such, adopting business intelligence practices offers an opportunity to improve the efficiency of evaluation efforts while fulfilling reporting requirements. Organizing informatics development documentation, service requests, and work performed with adaptable tools have greatly facilitated these and related business activities within our informatics center. Through the identification and measurement of key performance indicators, informatics objectives and results are now quickly and nimbly assessed using dashboards. Acceptance of the informatics operation as a business venture and the adoption of business intelligence strategies has allowed for data-driven decision making, faster corrective action, and greater transparency for interested stakeholders.

Validation of a next-generation sequencing assay for clinical molecular oncology.

Currently, oncology testing includes molecular studies and cytogenetic analysis to detect genetic aberrations of clinical significance. Next-generation sequencing (NGS) allows rapid analysis of multiple genes for clinically actionable somatic variants. The WUCaMP assay uses targeted capture for NGS analysis of 25 cancer-associated genes to detect mutations at actionable loci. We present clinical validation of the assay and a detailed framework for design and validation of similar clinical assays. Deep sequencing of 78 tumor specimens (≥ 1000× average unique coverage across the capture region) achieved high sensitivity for detecting somatic variants at low allele fraction (AF). Validation revealed sensitivities and specificities of 100% for detection of single-nucleotide variants (SNVs) within coding regions, compared with SNP array sequence data (95% CI = 83.4-100.0 for sensitivity and 94.2-100.0 for specificity) or whole-genome sequencing (95% CI = 89.1-100.0 for sensitivity and 99.9-100.0 for specificity) of HapMap samples. Sensitivity for detecting variants at an observed 10% AF was 100% (95% CI = 93.2-100.0) in HapMap mixes. Analysis of 15 masked specimens harboring clinically reported variants yielded concordant calls for 13/13 variants at AF of ≥ 15%. The WUCaMP assay is a robust and sensitive method to detect somatic variants of clinical significance in molecular oncology laboratories, with reduced time and cost of genetic analysis allowing for strategic patient management.

Examining the role of training in evidence-based public health: a qualitative study.

The use of evidence-based public health (EBPH) approaches is generally recognized as essential to changing public health outcomes. However, using an EBPH decision-making process requires public health practitioners to have the skills to review the evidence and pick the most workable strategy to address the problem at hand for their population of interest and the local context. Although there has been a growing body of academic literature examining the skills needed to translate evidence-based programs in local settings, many public health practitioners have not had the opportunity to learn or develop these skills. This article reports on qualitative interviews conducted to evaluate the process and impact of an EBPH course. The course has been found to assist participants in integrating new and existing skills to make evidence-based decisions. However, participants emphasize that factors external to the course influence their ability to engage in the EBPH process they learned.