Effect-Size Discrepancies in Literature Versus Raw Datasets from Experimental Spinal Cord Injury Studies: A CLIMBER Meta-Analysis.

Translation of spinal cord injury (SCI) therapeutics from pre-clinical animal studies into human studies is challenged by effect size variability, irreproducibility, and misalignment of evidence used by pre-clinical versus clinical literature. Clinical literature values reproducibility, with the highest grade evidence (class 1) consisting of meta-analysis demonstrating large therapeutic efficacy replicating across multiple studies. Conversely, pre-clinical literature values novelty over replication and lacks rigorous meta-analyses to assess reproducibility of effect sizes across multiple articles. Here, we applied modified clinical meta-analysis methods to pre-clinical studies, comparing effect sizes extracted from published literature to raw data on individual animals from these same studies. Literature-extracted data (LED) from numerical and graphical outcomes reported in publications were compared with individual animal data (IAD) deposited in a federally supported repository of SCI data. The animal groups from the IAD were matched with the same cohorts in the LED for a direct comparison. We applied random-effects meta-analysis to evaluate predictors of neuroconversion in LED versus IAD. We included publications with common injury models (contusive injuries) and standardized end-points (open field assessments). The extraction of data from 25 published articles yielded n = 1841 subjects, whereas IAD from these same articles included n = 2441 subjects. We observed differences in the number of experimental groups and animals per group, insufficient reporting of dropout animals, and missing information on experimental details. Meta-analysis revealed differences in effect sizes across LED versus IAD stratifications, for instance, severe injuries had the largest effect size in LED (standardized mean difference [SMD = 4.92]), but mild injuries had the largest effect size in IAD (SMD = 6.06). Publications with smaller sample sizes yielded larger effect sizes, while studies with larger sample sizes had smaller effects. The results demonstrate the feasibility of combining IAD analysis with traditional LED meta-analysis to assess effect size reproducibility in SCI.

Improving rigor and reproducibility in western blot experiments with the blotRig analysis.

Western blot is a popular biomolecular analysis method for measuring the relative quantities of independent proteins in complex biological samples. However, variability in quantitative western blot data analysis poses a challenge in designing reproducible experiments. The lack of rigorous quantitative approaches in current western blot statistical methodology may result in irreproducible inferences. Here we describe best practices for the design and analysis of western blot experiments, with examples and demonstrations of how different analytical approaches can lead to widely varying outcomes. To facilitate best practices, we have developed the blotRig tool for designing and analyzing western blot experiments to improve their rigor and reproducibility. The blotRig application includes functions for counterbalancing experimental design by lane position, batch management across gels, and analytics with covariates and random effects.

Improving rigor and reproducibility in western blot experiments with the blotRig analysis software.

Western blot is a popular biomolecular analysis method for measuring the relative quantities of independent proteins in complex biological samples. However, variability in quantitative western blot data analysis poses a challenge in designing reproducible experiments. The lack of rigorous quantitative approaches in current western blot statistical methodology may result in irreproducible inferences. Here we describe best practices for the design and analysis of western blot experiments, with examples and demonstrations of how different analytical approaches can lead to widely varying outcomes. To facilitate best practices, we have developed the blotRig tool for designing and analyzing western blot experiments to improve their rigor and reproducibility. The blotRig application includes functions for counterbalancing experimental design by lane position, batch management across gels, and analytics with covariates and random effects.