The phenotype-genotype reference map: Improving biobank data science through replication.

Population-scale biobanks linked to electronic health record data provide vast opportunities to extend our knowledge of human genetics and discover new phenotype-genotype associations. Given their dense phenotype data, biobanks can also facilitate replication studies on a phenome-wide scale. Here, we introduce the phenotype-genotype reference map (PGRM), a set of 5,879 genetic associations from 523 GWAS publications that can be used for high-throughput replication experiments. PGRM phenotypes are standardized as phecodes, ensuring interoperability between biobanks. We applied the PGRM to five ancestry-specific cohorts from four independent biobanks and found evidence of robust replications across a wide array of phenotypes. We show how the PGRM can be used to detect data corruption and to empirically assess parameters for phenome-wide studies. Finally, we use the PGRM to explore factors associated with replicability of GWAS results.

Optimizing Spectronaut Search Parameters to Improve Data Quality with Minimal Proteome Coverage Reductions in DIA Analyses of Heterogeneous Samples.

Data-independent acquisition has seen breakthroughs that enable comprehensive proteome profiling using short gradients. As the proteome coverage continues to increase, the quality of the data generated becomes much more relevant. Using Spectronaut, we show that the default search parameters can be easily optimized to minimize the occurrence of false positives across different samples. Using an immunological infection model system to demonstrate the impact of adjusting search settings, we analyzed Mus musculus macrophages and compared their proteome to macrophages spiked withCandida albicans. This experimental system enabled the identification of "false positives" as Candida albicans peptides and proteins should not be present in the Mus musculus-only samples. We show that adjusting the search parameters reduced "false positive" identifications by 89% at the peptide and protein level, thereby considerably increasing the quality of the data. We also show that these optimized parameters incurred a moderate cost, only reducing the overall number of "true positive" identifications across each biological replicate by <6.7% at both the peptide and protein level. We believe the value of our updated search parameters extends beyond a two-organism analysis and would be of great value to any DIA experiment analyzing heterogeneous populations of cell types or tissues.

More Than a Decade Since the Latest CONSORT Non-inferiority Trials Extension: Do Infectious Diseases Trials Do Enough?

More than a decade after the Consolidated Standards of Reporting Trials group released a reporting items checklist for non-inferiority randomized controlled trials, the infectious diseases literature continues to underreport these items. Trialists, journals, and peer reviewers should redouble their efforts to ensure infectious diseases studies meet these minimum reporting standards.

Validation of algorithms in studies based on routinely collected health data: general principles.

Clinicians, researchers, regulators, and other decision-makers increasingly rely on evidence from real-world data (RWD), including data routinely accumulating in health and administrative databases. RWD studies often rely on algorithms to operationalize variable definitions. An algorithm is a combination of codes or concepts used to identify persons with a specific health condition or characteristic. Establishing the validity of algorithms is a prerequisite for generating valid study findings that can ultimately inform evidence-based health care. This paper aims to systematize terminology, methods, and practical considerations relevant to the conduct of validation studies of RWD-based algorithms. We discuss measures of algorithm accuracy; gold/reference standard; study size; prioritizing accuracy measures; algorithm portability; and implication for interpretation. Information bias is common in epidemiologic studies, underscoring the importance of transparency in decisions regarding choice and prioritizing measures of algorithm validity. The validity of an algorithm should be judged in the context of a data source, and one size does not fit all. Prioritizing validity measures within a given data source depends on the role of a given variable in the analysis (eligibility criterion, exposure, outcome or covariate). Validation work should be part of routine maintenance of RWD sources.

A Bayesian nonparametric approach to correct for underreporting in count data.

We propose a nonparametric compound Poisson model for underreported count data that introduces a latent clustering structure for the reporting probabilities. The latter are estimated with the model's parameters based on experts' opinion and exploiting a proxy for the reporting process. The proposed model is used to estimate the prevalence of chronic kidney disease in Apulia, Italy, based on a unique statistical database covering information on m = 258 municipalities obtained by integrating multisource register information. Accurate prevalence estimates are needed for monitoring, surveillance, and management purposes; yet, counts are deemed to be considerably underreported, especially in some areas of Apulia, one of the most deprived and heterogeneous regions in Italy. Our results agree with previous findings and highlight interesting geographical patterns of the disease. We compare our model to existing approaches in the literature using simulated as well as real data on early neonatal mortality risk in Brazil, described in previous research: the proposed approach proves to be accurate and particularly suitable when partial information about data quality is available.

Metabolomics 2023 workshop report: moving toward consensus on best QA/QC practices in LC-MS-based untargeted metabolomics.

INTRODUCTION: During the Metabolomics 2023 conference, the Metabolomics Quality Assurance and Quality Control Consortium (mQACC) presented a QA/QC workshop for LC-MS-based untargeted metabolomics. OBJECTIVES: The Best Practices Working Group disseminated recent findings from community forums and discussed aspects to include in a living guidance document. METHODS: Presentations focused on reference materials, data quality review, metabolite identification/annotation and quality assurance. RESULTS: Live polling results and follow-up discussions offered a broad international perspective on QA/QC practices. CONCLUSIONS: Community input gathered from this workshop series is being used to shape the living guidance document, a continually evolving QA/QC best practices resource for metabolomics researchers.

Data quality in Brazilian population-based cancer registries for gastrointestinal cancers.

BACKGROUND: Population-based cancer registries (PBCRs) are the primary source of information for cancer surveillance and monitoring. Currently, there are 30 active PBCRs in Brazil. The objective of this study was to analyze the data quality of five gastrointestinal cancers (esophagus, stomach, colorectal, liver, and pancreas) according to the criteria of comparability, validity, completeness, and timeliness in Brazilian cancer registries. METHODS: This study included data from Brazilian PBCRs with more than ten years of historical data starting in the year 2000, regardless of the type of defined geographical coverage (state, metropolitan region, or capital), totaling 16 registries. Brazilian PBCRs were evaluated based on four international data quality criteria: comparability, validity (accuracy), completeness, and timeliness. All cancer cases were analyzed, except for nonmelanoma skin cancer cases (C44) and five gastrointestinal tumors (esophageal cancer, stomach cancer, colorectal cancer, liver cancer, and pancreatic cancer) per cancer registry and sex, according to the available period. RESULTS: The 16 Brazilian PBCRs represent 17% of the population (36 million inhabitants in 2021) according to data from 2000 to 2018. There was a variation in the incidence in the historical series ranging from 12 to 19 years. The proportion of morphologically verified (MV%) cases varied from 74.3% (Manaus) to 94.8% (Aracaju), and the proportion of incidentally reported death certificate only (DCO%) cases varied from 3.0% (São Paulo) to 23.9% (Espírito Santo). High-lethality malignant neoplasms, such as liver and pancreas, had DCO percentages greater than 30% in most cancer registries. The sixteen registries have more than a 48-month delay in data release compared to the 2022 calendar year. CONCLUSION: The studied Brazilian cancer registries met international comparability criteria; however, half of the registries showed indices below the expected levels for validity and completeness criteria for high-lethality tumors such as liver and pancreas tumors, in addition to a long delay in data availability and disclosure. Significant efforts are necessary to ensure the operational and stability of the PBCR in Brazil, which continues to be a tool for monitoring cancer incidence and assessing national cancer control policies.

Automated Quality Evaluation Index for Arterial Spin Labeling Derived Cerebral Blood Flow Maps.

BACKGROUND: Arterial spin labeling (ASL) derived cerebral blood flow (CBF) maps are prone to artifacts and noise that can degrade image quality. PURPOSE: To develop an automated and objective quality evaluation index (QEI) for ASL CBF maps. STUDY TYPE: Retrospective. POPULATION: Data from N = 221 adults, including patients with Alzheimer's disease (AD), Parkinson's disease, and traumatic brain injury. FIELD STRENGTH/SEQUENCE: Pulsed or pseudocontinuous ASL acquired at 3 T using non-background suppressed 2D gradient-echo echoplanar imaging or background suppressed 3D spiral spin-echo readouts. ASSESSMENT: The QEI was developed using N = 101 2D CBF maps rated as unacceptable, poor, average, or excellent by two neuroradiologists and validated by 1) leave-one-out cross validation, 2) assessing if CBF reproducibility in N = 53 cognitively normal adults correlates inversely with QEI, 3) if iterative discarding of low QEI data improves the Cohen's d effect size for CBF differences between preclinical AD (N = 27) and controls (N = 53), 4) comparing the QEI with manual ratings for N = 50 3D CBF maps, and 5) comparing the QEI with another automated quality metric. STATISTICAL TESTS: Inter-rater reliability and manual vs. automated QEI were quantified using Pearson's correlation. P < 0.05 was considered significant. RESULTS: The correlation between QEI and manual ratings (R = 0.83, CI: 0.76-0.88) was similar (P = 0.56) to inter-rater correlation (R = 0.81, CI: 0.73-0.87) for the 2D data. CBF reproducibility correlated negatively (R = -0.74, CI: -0.84 to -0.59) with QEI. The effect size comparing patients and controls improved (R = 0.72, CI: 0.59-0.82) as low QEI data was discarded iteratively. The correlation between QEI and manual ratings (R = 0.86, CI: 0.77-0.92) of 3D ASL was similar (P = 0.09) to inter-rater correlation (R = 0.78, CI: 0.64-0.87). The QEI correlated (R = 0.87, CI: 0.77-0.92) significantly better with manual ratings than did an existing approach (R = 0.54, CI: 0.30-0.72). DATA CONCLUSION: Automated QEI performed similarly to manual ratings and can provide scalable ASL quality control. EVIDENCE LEVEL: 2 TECHNICAL EFFICACY: Stage 1.

Effects of epileptic seizures on the quality of biosignals recorded from wearables.

OBJECTIVE: Wearable nonelectroencephalographic biosignal recordings captured from the wrist offer enormous potential for seizure monitoring. However, signal quality remains a challenging factor affecting data reliability. Models trained for seizure detection depend on the quality of recordings in peri-ictal periods in performing a feature-based separation of ictal periods from interictal periods. Thus, this study aims to investigate the effect of epileptic seizures on signal quality, ensuring accurate and reliable monitoring. METHODS: This study assesses the signal quality of wearable data during peri-ictal phases of generalized tonic-clonic and focal to bilateral tonic-clonic seizures (TCS), focal motor seizures (FMS), and focal nonmotor seizures (FNMS). We evaluated accelerometer (ACC) activity and the signal quality of electrodermal activity (EDA) and blood volume pulse (BVP) data. Additionally, we analyzed the influence of peri-ictal movements as assessed by ACC (ACC activity) on signal quality and examined intraictal subphases of focal to bilateral TCS. RESULTS: We analyzed 386 seizures from 111 individuals in three international epilepsy monitoring units. BVP signal quality and ACC activity levels differed between all seizure types. We found the largest decrease in BVP signal quality and increase in ACC activity when comparing the ictal phase to the pre- and postictal phases for TCS. Additionally, ACC activity was strongly negatively correlated with BVP signal quality for TCS and FMS, and weakly for FNMS. Intraictal analysis revealed that tonic and clonic subphases have the lowest BVP signal quality and the highest ACC activity. SIGNIFICANCE: Motor elements of seizures significantly impair BVP signal quality, but do not have significant effect on EDA signal quality, as assessed by wrist-worn wearables. The results underscore the importance of signal quality assessment methods and careful selection of robust modalities to ensure reliable seizure detection. Future research is needed to explain whether seizure detection models' decisions are based on signal responses induced by physiological processes as opposed to artifacts.

Effect of Data Quality and Data Quantity on the Estimation of Intrinsic Solubility: Analysis Based on a Single-Source Data Set.

Aqueous solubility is one of the most important physicochemical properties of drug molecules and a major driving force for oral drug absorption. To date, the performance of in silico models for the estimation of solubility for novel chemical space is limited. To investigate possible reasons and remedies for this, the Johnson and Johnson in-house aqueous solubility data with over 40,000 compounds was leveraged. All data were generated through the same high-throughput assay, providing a unique opportunity to explore the relationship between data quality, quantity, and model estimations. Six intrinsic solubility data sets with different sizes and noise levels were generated by making use of three different approaches: (i) inclusion or exclusion of amorphous solid residue, (ii) measured or experimental log D to identify the intrinsic solubility, and (iii) adopting or omitting a quality check process in the data processing workflow. A random forest regressor was trained on the data sets with three different sets of descriptors calculated from RDKit, ADMET predictor, or Mordred, and the performances were evaluated with nested cross-validation as well as ten refined test sets. The models confirm, as expected, that with the same data set size, high-quality data leads to better model performance; however, also, models trained with larger data sets containing analytical variability can give equally accurate estimations compared to models trained with small, clean, and diverse data sets. However, noise introduced by including the presence of amorphous solid postsolubility measurement in the training data set cannot be overcome by increasing data size, as they are introducing a biased systematic positive error in the data set, confirming the importance of critical data review. Finally, two top-performing models were tested on the first test set from the second solubility challenge, achieving RMSE values of 0.74 and 0.72 and log S ± 0.5 of 46 and 48%, respectively. These results demonstrated improved performance compared to those reported in the findings of the competition, highlighting that a single-source curated data set can enhance the prediction of intrinsic solubility.

A registered report of a two-site study of variations of the flanker task: ERN experimental effects and data quality.

Error-related negativity is a widely used measure of error monitoring, and many projects are independently moving ERN recorded during a flanker task toward standardization, optimization, and eventual clinical application. However, each project uses a different version of the flanker task and tacitly assumes ERN is functionally equivalent across each version. The routine neglect of a rigorous test of this assumption undermines efforts to integrate ERN findings across tasks, optimize and standardize ERN assessment, and widely apply ERN in clinical trials. The purpose of this registered report was to determine whether ERN shows similar experimental effects (correct vs. error trials) and data quality (intraindividual variability) during three commonly used versions of a flanker task. ERN was recorded from 172 participants during three versions of a flanker task across two study sites. ERN scores showed numerical differences between tasks, raising questions about the comparability of ERN findings across studies and tasks. Although ERN scores from all three versions of the flanker task yielded high data quality and internal consistency, one version did outperform the other two in terms of the size of experimental effects and the data quality. Exploratory analyses of the error positivity (Pe) provided tentative support for the other two versions of the task over the paradigm that appeared optimal for ERN. The present study provides a roadmap for how to statistically compare psychometric characteristics of ERP scores across paradigms and gives preliminary recommendations for flanker tasks to use for ERN- and Pe-focused studies.

Assessing Real-World Data From Electronic Health Records for Health Technology Assessment: The SUITABILITY Checklist: A Good Practices Report of an ISPOR Task Force.

This ISPOR Good Practices report provides a framework for assessing the suitability of electronic health records data for use in health technology assessments (HTAs). Although electronic health record (EHR) data can fill evidence gaps and improve decisions, several important limitations can affect its validity and relevance. The ISPOR framework includes 2 components: data delineation and data fitness for purpose. Data delineation provides a complete understanding of the data and an assessment of its trustworthiness by describing (1) data characteristics; (2) data provenance; and (3) data governance. Fitness for purpose comprises (1) data reliability items, ie, how accurate and complete the estimates are for answering the question at hand and (2) data relevance items, which assess how well the data are suited to answer the particular question from a decision-making perspective. The report includes a checklist specific to EHR data reporting: the ISPOR SUITABILITY Checklist. It also provides recommendations for HTA agencies and policy makers to improve the use of EHR-derived data over time. The report concludes with a discussion of limitations and future directions in the field, including the potential impact from the substantial and rapid advances in the diffusion and capabilities of large language models and generative artificial intelligence. The report's immediate audiences are HTA evidence developers and users. We anticipate that it will also be useful to other stakeholders, particularly regulators and manufacturers, in the future.

Looking at the future of the medical certification of cause of death (MCCD) in England and Wales.

Internationally, the quality of death certification is poor although there are multiple efforts underway to improve the process. In England, a new medical certification system has been proposed to improve the quality of data. We surveyed general practitioners (n = 95) across the West Yorkshire area of England to appraise their views regarding whether further possible changes to the death certification system could promote their quality.

The quality of routine data for measuring facility-based maternal mortality in public and private health facilities in Kampala City, Uganda.

BACKGROUND: Routine health facility data are an important source of health information in resource-limited settings. Regular quality assessments are necessary to improve the reliability of routine data for different purposes, including estimating facility-based maternal mortality. This study aimed to assess the quality of routine data on deliveries, livebirths and maternal deaths in Kampala City, Uganda. METHODS: We reviewed routine health facility data from the district health information system (DHIS2) for 2016 to 2021. This time period included an upgrade of DHIS2, resulting in two datasets (2016-2019 and 2020-2021) that were managed separately. We analysed data for all facilities that reported at least one delivery in any of the six years, and for a subset of facilities designated to provide emergency obstetric care (EmOC). We adapted the World Health Organization data quality review framework to assess completeness and internal consistency of the three data elements, using 2019 and 2021 as reference years. Primary data were collected to verify reporting accuracy in four purposively selected EmOC facilities. Data were disaggregated by facility level and ownership. RESULTS: We included 255 facilities from 2016 to 2019 and 247 from 2020 to 2021; of which 30% were EmOC facilities. The overall completeness of data for deliveries and livebirths ranged between 53% and 55%, while it was < 2% for maternal deaths (98% of monthly values were zero). Among EmOC facilities, completeness was higher for deliveries and livebirths at 80%; and was < 6% for maternal deaths. For the whole sample, the prevalence of outliers for all three data elements was < 2%. Inconsistencies over time were mostly observed for maternal deaths, with the highest difference of 96% occurring in 2021. CONCLUSIONS: Routine data from childbirth facilities in Kampala were generally suboptimal, but the quality was better in EmOC facilities. Given likely underreporting of maternal deaths, further efforts to verify and count all facility-related maternal deaths are essential to accurately estimate facility-based maternal mortality. Data reliability could be enhanced by improving reporting practices in EmOC facilities and streamlining reporting processes in private-for-profit facilities. Further qualitative studies should identify critical points where data are compromised, and data quality assessments should consider service delivery standards.

Critical Evaluation and Meta-Analysis of Ecotoxicological Data on Per- and Polyfluoroalkyl Substances (PFAS) in Freshwater Species.

Despite the increasing concern regarding the ecological risks posed by per- and polyfluoroalkyl substances (PFAS), a lack of comprehensive understanding of their actual ecotoxicity remains. Through a meticulous examination of 91 peer-reviewed studies investigating effects at a population level and constructing probabilistic species sensitivity distributions (PSSDs), we present a state-of-the-science hazard assessment of PFAS in freshwater species. Using data subsets containing suboptimal data led to an overestimation of the predicted no-effect concentrations (PNECs) of PFAS. We report PNECs of perfluoroalkyl carboxylic acids (PFCAs) and perfluoroalkyl sulfonates (PFSAs) in freshwater to be 4.8-2000 µg/L and 0.4-8.9 µg/L, respectively, derived from high-quality data. Statistical analyses revealed that both functional groups and carbon chain length significantly influenced (p < 0.05) the variations in toxicity observed among different PFAS. This study underscores the importance of obtaining high-quality PFAS ecotoxicity data to comprehend associated hazards. The PNECs of PFAS derived in this study are higher compared to those of micro/nanoplastics and persistent organic pollutants. Our research offers valuable insights into prioritizing the regulation of more toxic PFAS.

The predictive power of data: machine learning analysis for Covid-19 mortality based on personal, clinical, preclinical, and laboratory variables in a case-control study.

BACKGROUND AND PURPOSE: The COVID-19 pandemic has presented unprecedented public health challenges worldwide. Understanding the factors contributing to COVID-19 mortality is critical for effective management and intervention strategies. This study aims to unlock the predictive power of data collected from personal, clinical, preclinical, and laboratory variables through machine learning (ML) analyses. METHODS: A retrospective study was conducted in 2022 in a large hospital in Abadan, Iran. Data were collected and categorized into demographic, clinical, comorbid, treatment, initial vital signs, symptoms, and laboratory test groups. The collected data were subjected to ML analysis to identify predictive factors associated with COVID-19 mortality. Five algorithms were used to analyze the data set and derive the latent predictive power of the variables by the shapely additive explanation values. RESULTS: Results highlight key factors associated with COVID-19 mortality, including age, comorbidities (hypertension, diabetes), specific treatments (antibiotics, remdesivir, favipiravir, vitamin zinc), and clinical indicators (heart rate, respiratory rate, temperature). Notably, specific symptoms (productive cough, dyspnea, delirium) and laboratory values (D-dimer, ESR) also play a critical role in predicting outcomes. This study highlights the importance of feature selection and the impact of data quantity and quality on model performance. CONCLUSION: This study highlights the potential of ML analysis to improve the accuracy of COVID-19 mortality prediction and emphasizes the need for a comprehensive approach that considers multiple feature categories. It highlights the critical role of data quality and quantity in improving model performance and contributes to our understanding of the multifaceted factors that influence COVID-19 outcomes.

A standard to report biological variation data studies - based on an expert opinion.

There is a need for standards for generation and reporting of Biological Variation (BV) reference data. The absence of standards affects the quality and transportability of BV data, compromising important clinical applications. To address this issue, international expert groups under the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) have developed an online resource (https://tinyurl.com/bvmindmap) in the form of an interactive mind map that serves as a guideline for researchers planning, performing and reporting BV studies. The mind map addresses study design, data analysis, and reporting criteria, providing embedded links to relevant references and resources. It also incorporates a checklist approach, identifying a Minimum Data Set (MDS) to enable the transportability of BV data and incorporates the Biological Variation Data Critical Appraisal Checklist (BIVAC) to assess study quality. The mind map is open to access and is disseminated through the EFLM BV Database website, promoting accessibility and compliance to a reporting standard, thereby providing a tool to be used to ensure data quality, consistency, and comparability of BV data. Thus, comparable to the STARD initiative for diagnostic accuracy studies, the mind map introduces a Standard for Reporting Biological Variation Data Studies (STARBIV), which can enhance the reporting quality of BV studies, foster user confidence, provide better decision support, and be used as a tool for critical appraisal. Ongoing refinement is expected to adapt to emerging methodologies, ensuring a positive trajectory toward improving the validity and applicability of BV data in clinical practice.

Identifying erroneous height and weight values from adult electronic health records in the All of Us research program.

INTRODUCTION: Electronic Health Records (EHR) are a useful data source for research, but their usability is hindered by measurement errors. This study investigated an automatic error detection algorithm for adult height and weight measurements in EHR for the All of Us Research Program (All of Us). METHODS: We developed reference charts for adult heights and weights that were stratified on participant sex. Our analysis included 4,076,534 height and 5,207,328 wt measurements from âˆ¼ 150,000 participants. Errors were identified using modified standard deviation scores, differences from their expected values, and significant changes between consecutive measurements. We evaluated our method with chart-reviewed heights (8,092) and weights (9,039) from 250 randomly selected participants and compared it with the current cleaning algorithm in All of Us. RESULTS: The proposed algorithm classified 1.4 % of height and 1.5 % of weight errors in the full cohort. Sensitivity was 90.4 % (95 % CI: 79.0-96.8 %) for heights and 65.9 % (95 % CI: 56.9-74.1 %) for weights. Precision was 73.4 % (95 % CI: 60.9-83.7 %) for heights and 62.9 (95 % CI: 54.0-71.1 %) for weights. In comparison, the current cleaning algorithm has inferior performance in sensitivity (55.8 %) and precision (16.5 %) for height errors while having higher precision (94.0 %) and lower sensitivity (61.9 %) for weight errors. DISCUSSION: Our proposed algorithm outperformed in detecting height errors compared to weights. It can serve as a valuable addition to the current All of Us cleaning algorithm for identifying erroneous height values.

The Oncology QCARD Initiative: Fostering efficient evaluation of initial real-world data proposals.

PURPOSE: The oncology quality, characterization, and assessment of real-world data (Oncology QCARD) Initiative was formed to develop a set of minimum study design and data elements needed to evaluate the fitness of the real-world data (RWD) source(s) proposed in an initial study concept as part of early interaction with scientific reviewers. METHODS: A multidisciplinary executive committee (EC) was established to guide the Oncology QCARD Initiative. The EC conducted a landscape review of published literature, guidances, and guidelines to evaluate relevant dimensions of data quality measurement. Guided by the review and informed by expert feedback, the Oncology QCARD Initial Protocol Characterization (IPC) provides a summary of minimum elements needed to adequately describe an initial clinical study concept that involves RWD and is intended to support decision-making. RESULTS: Fit-for-use data and fit-for-purpose design emerged as themes from the landscape analysis. Data that are fit-for-use are both relevant (sufficiently capturing exposure, outcomes, and covariates) and reliable (understanding data accrual and quality control and whether the data represent the underlying concepts they are intended to represent) to answer a specific research question. A fit-for-purpose design takes appropriate steps to ensure internal and external validity and allows for transparency in reporting. The QCARD-IPC focuses on high-level characteristics of RWD sources and study design domains including data temporality, population, medical product exposure, comparators, and covariates, endpoints, statistical analysis, and data quality assurance plans. CONCLUSIONS: Evaluation of studies including RWD requires understanding the data source, study design, and potential biases to preliminarily evaluate whether selected RWD are fit-for-use for the research question. The Oncology QCARD-IPC provides a structured, transparent approach to facilitate early review and enhanced communication between study sponsors and scientific reviewers of initial study proposals including RWD.

How effective is score-based data quality assessment? An illustration with fish BCF data.

Increasingly rigorous data quality (DQ) evaluations and/or screening practices are being applied to environmental and ecotoxicological datasets. DQ is predominantly evaluated by scoring given data against preselected criteria. This study provides the first examination on the effectiveness of score-based DQ evaluation in providing statistically meaningful differentiation of measurements using fish bioconcentration factor (BCF) dataset as an illustration. This is achieved by inspecting how log BCF differs with the built-in overall-DQ and specific-DQ evaluations, and how it is influenced by interactive effects and hierarchy of DQ criteria. Approximately 80-90% of analyzable chemicals show no statistical difference in log BCF between low-quality (LQ) and high-quality (HQ) measurements in overall evaluation (n = 183) or in individual evaluation of 6 DQ criteria (n = 53 to 101). Further examination shows that log BCF may/may not change with different combinations or total number of criteria violations. Tree analysis and nodal structures of deviation in log BCF also reveal the absence of common structural dependence on the criteria violated. Finally, simple averaging of all measurements without DQ differentiation yields comparable log BCFs as those derived using strictly HQ data with ≤0.5 log unit difference in over 93% of the chemicals (n = 158) and no dependence on number of measurements, fraction of LQ measurements, or bioaccumulation potential of the chemicals. For accurate log BCF, DQ appears no more important than having more independent measurements irrespective of their individual DQ statuses. This work concludes by calling for: (i) re-documentation of experimental details in legacy environmental and ecotoxicological datasets, (ii) examination of other DQ-categorized datasets using the tests and tools applied here, and (ii) a thorough and systematic reflection on how DQ should be assessed for modeling, benchmarking, and other data-based analyses or applications.