Systematic assessment of the replicability and generalizability of preclinical findings: Impact of protocol harmonization across laboratory sites.

The influence of protocol standardization between laboratories on their replicability of preclinical results has not been addressed in a systematic way. While standardization is considered good research practice as a means to control for undesired external noise (i.e., highly variable results), some reports suggest that standardized protocols may lead to idiosyncratic results, thus undermining replicability. Through the EQIPD consortium, a multi-lab collaboration between academic and industry partners, we aimed to elucidate parameters that impact the replicability of preclinical animal studies. To this end, 3 experimental protocols were implemented across 7 laboratories. The replicability of results was determined using the distance travelled in an open field after administration of pharmacological compounds known to modulate locomotor activity (MK-801, diazepam, and clozapine) in C57BL/6 mice as a worked example. The goal was to determine whether harmonization of study protocols across laboratories improves the replicability of the results and whether replicability can be further improved by systematic variation (heterogenization) of 2 environmental factors (time of testing and light intensity during testing) within laboratories. Protocols were tested in 3 consecutive stages and differed in the extent of harmonization across laboratories and standardization within laboratories: stage 1, minimally aligned across sites (local protocol); stage 2, fully aligned across sites (harmonized protocol) with and without systematic variation (standardized and heterogenized cohort); and stage 3, fully aligned across sites (standardized protocol) with a different compound. All protocols resulted in consistent treatment effects across laboratories, which were also replicated within laboratories across the different stages. Harmonization of protocols across laboratories reduced between-lab variability substantially compared to each lab using their local protocol. In contrast, the environmental factors chosen to introduce systematic variation within laboratories did not affect the behavioral outcome. Therefore, heterogenization did not reduce between-lab variability further compared to the harmonization of the standardized protocol. Altogether, these findings demonstrate that subtle variations between lab-specific study protocols may introduce variation across independent replicate studies even after protocol harmonization and that systematic heterogenization of environmental factors may not be sufficient to account for such between-lab variation. Differences in replicability of results within and between laboratories highlight the ubiquity of study-specific variation due to between-lab variability, the importance of transparent and fine-grained reporting of methodologies and research protocols, and the importance of independent study replication.

Assessment of Behavioral Characteristics With Procedures of Minimal Human Interference in the mdx Mouse Model for Duchenne Muscular Dystrophy.

Duchenne muscular dystrophy (DMD) is a severe, progressive neuromuscular disorder caused by mutations in the DMD gene resulting in loss of functional dystrophin protein. The muscle dystrophin isoform is essential to protect muscles from contraction-induced damage. However, most dystrophin isoforms are expressed in the brain. In addition to progressive muscle weakness, many DMD patients therefore also exhibit intellectual and behavioral abnormalities. The most commonly used mouse model for DMD, the mdx mouse, lacks only the full-length dystrophin isoforms and has been extensively characterized for muscle pathology. In this study, we assessed behavioral effects of a lack of full-length dystrophins on spontaneous behavior, discrimination and reversal learning, anxiety, and short-term spatial memory and compared performance between male and female mdx mice. In contrast to our previous study using only female mdx mice, we could not reproduce the earlier observed reversal learning deficit. However, we did notice small differences in the number of visits made during the Y-maze and dark-light box. Results indicate that it is advisable to establish standard operating procedures specific to behavioral testing in mdx mice to allow the detection of the subtle phenotypic differences and to eliminate inter and intra laboratory variance.

Simultaneous assessment of cognitive function, circadian rhythm, and spontaneous activity in aging mice.

Cognitive function declines substantially with age in both humans and animal models. In humans, this decline is associated with decreases in independence and quality of life. Although the methodology for analysis of cognitive function in human models is relatively well established, similar analyses in animal models have many technical issues (e.g., unintended experimenter bias, motivational issues, stress, and testing during the light phase of the light dark cycle) that limit interpretation of the results. These caveats, and others, potentially bias the interpretation of studies in rodents and prevent the application of current tests of learning and memory as part of an overall healthspan assessment in rodent models of aging. The goal of this study was to establish the methodology to assess cognitive function in aging animals that addresses many of these concerns. Here, we use a food reward-based discrimination procedure with minimal stress in C57Bl/6J male mice at 6, 21, and 27 months of age, followed by a reversal task to assess behavioral flexibility. Importantly, the procedures minimize issues related to between-experimenter confounds and are conducted during both the dark and light phases of the light dark cycle in a home-cage setting. During cognitive testing, we were able to assess multiple measures of spontaneous movement and diurnal activity in young and aged mice including, distance moved, velocity, and acceleration over a 90-h period. Both initial discrimination and reversal learning significantly decreased with age and, similar to rats and humans, not all old mice demonstrated impairments in learning with age. These results permitted classification of animals based on their cognitive status. Analysis of movement parameters indicated decreases in distance moved as well as velocity and acceleration with increasing age. Based on these data, we developed preliminary models indicating, as in humans, a close relationship exists between age-related movement parameters and cognitive ability. Our results provide a reliable method for assessing cognitive performance with minimal stress and simultaneously provide key information on movement and diurnal activity. These methods represent a novel approach to developing non-invasive healthspan measures in rodent models that allow standardization across laboratories.