Investigating the replicability of preclinical cancer biology.

Replicability is an important feature of scientific research, but aspects of contemporary research culture, such as an emphasis on novelty, can make replicability seem less important than it should be. The Reproducibility Project: Cancer Biology was set up to provide evidence about the replicability of preclinical research in cancer biology by repeating selected experiments from high-impact papers. A total of 50 experiments from 23 papers were repeated, generating data about the replicability of a total of 158 effects. Most of the original effects were positive effects (136), with the rest being null effects (22). A majority of the original effect sizes were reported as numerical values (117), with the rest being reported as representative images (41). We employed seven methods to assess replicability, and some of these methods were not suitable for all the effects in our sample. One method compared effect sizes: for positive effects, the median effect size in the replications was 85% smaller than the median effect size in the original experiments, and 92% of replication effect sizes were smaller than the original. The other methods were binary - the replication was either a success or a failure - and five of these methods could be used to assess both positive and null effects when effect sizes were reported as numerical values. For positive effects, 40% of replications (39/97) succeeded according to three or more of these five methods, and for null effects 80% of replications (12/15) were successful on this basis; combining positive and null effects, the success rate was 46% (51/112). A successful replication does not definitively confirm an original finding or its theoretical interpretation. Equally, a failure to replicate does not disconfirm a finding, but it does suggest that additional investigation is needed to establish its reliability.

Challenges for assessing replicability in preclinical cancer biology.

We conducted the Reproducibility Project: Cancer Biology to investigate the replicability of preclinical research in cancer biology. The initial aim of the project was to repeat 193 experiments from 53 high-impact papers, using an approach in which the experimental protocols and plans for data analysis had to be peer reviewed and accepted for publication before experimental work could begin. However, the various barriers and challenges we encountered while designing and conducting the experiments meant that we were only able to repeat 50 experiments from 23 papers. Here we report these barriers and challenges. First, many original papers failed to report key descriptive and inferential statistics: the data needed to compute effect sizes and conduct power analyses was publicly accessible for just 4 of 193 experiments. Moreover, despite contacting the authors of the original papers, we were unable to obtain these data for 68% of the experiments. Second, none of the 193 experiments were described in sufficient detail in the original paper to enable us to design protocols to repeat the experiments, so we had to seek clarifications from the original authors. While authors were extremely or very helpful for 41% of experiments, they were minimally helpful for 9% of experiments, and not at all helpful (or did not respond to us) for 32% of experiments. Third, once experimental work started, 67% of the peer-reviewed protocols required modifications to complete the research and just 41% of those modifications could be implemented. Cumulatively, these three factors limited the number of experiments that could be repeated. This experience draws attention to a basic and fundamental concern about replication - it is hard to assess whether reported findings are credible.

Experiments from unfinished Registered Reports in the Reproducibility Project: Cancer Biology.

As part of the Reproducibility Project: Cancer Biology, we published Registered Reports that described how we intended to replicate selected experiments from 29 high-impact preclinical cancer biology papers published between 2010 and 2012. Replication experiments were completed and Replication Studies reporting the results were submitted for 18 papers, of which 17 were accepted and published by eLife with the rejected paper posted as a preprint. Here, we report the status and outcomes obtained for the remaining 11 papers. Four papers initiated experimental work but were stopped without any experimental outcomes. Two papers resulted in incomplete outcomes due to unanticipated challenges when conducting the experiments. For the remaining five papers only some of the experiments were completed with the other experiments incomplete due to mundane technical or unanticipated methodological challenges. The experiments from these papers, along with the other experiments attempted as part of the Reproducibility Project: Cancer Biology, provides evidence about the challenges of repeating preclinical cancer biology experiments and the replicability of the completed experiments.

Replication study: androgen receptor splice variants determine taxane sensitivity in prostate cancer.

In 2015, as part of the Prostate Cancer Foundation-Movember Foundation Reproducibility Initiative, we published a Registered Report (Shan et al., 2015) that described how we intended to replicate selected experiments from the paper "Androgen Receptor Splice Variants Determine Taxane Sensitivity in Prostate Cancer" (Thadani-Mulero et al., 2014). Here we report the results of those experiments. Growth of tumor xenografts from two prostate cancer xenograft lines, LuCaP 86.2, which expresses wild-type androgen receptor (AR) and AR variant 567, and LuCaP 23.1, which expresses wild-type AR and AR variant 7, were not affected by docetaxel treatment. The LuCaP 23.1 tumor xenografts grew slower than in the original study. This result is different from the original study, which reported significant reduction of tumor growth in the LuCaP 86.2. Furthermore, we were unable to detect ARv7 in the LuCaP 23.1, although we used the antibody as stated in the original study and ensured that it was detecting ARv7 via a known positive control (22rv1, Hörnberg et al., 2011). Finally, we report a meta-analysis of the result.

Registered report: the androgen receptor induces a distinct transcriptional program in castration-resistant prostate cancer in man.

The Prostate Cancer Foundation-Movember Foundation Reproducibility Initiative (PCFMFRI) seeks to address growing concerns about reproducibility in scientific research by conducting replications of recent papers in the field of prostate cancer. This Registered Report describes the proposed replication plan of key experiments from "The Androgen Receptor Induces a Distinct Transcriptional Program in Castration-Resistant Prostate Cancer in Man" by Sharma and colleagues (2013), published in Cancer Cell in 2013. Of thousands of targets for the androgen receptor (AR), the authors elucidated a subset of 16 core genes that were consistently downregulated with castration and re-emerged with castration resistance. These 16 AR binding sites were distinct from those observed in cells in culture. The authors suggested that cellular context can have dramatic effects on downstream transcriptional regulation of AR binding sites. The present study will attempt to replicate Fig. 7C by comparing gene expression of the 16 core genes identified by Sharma and colleagues in xenograft tumor tissue compared to androgen treated LNCaP cells in vitro. The Prostate Cancer Foundation-Movember Foundation Reproducibility Initiative is a collaboration between the Prostate Cancer Foundation, the Movember Initiative, and Science Exchange, and the results of the replications will be published by PeerJ.

Registered report: androgen receptor splice variants determine taxane sensitivity in prostate cancer.

The Prostate Cancer Foundation-Movember Foundation Reproducibility Initiative seeks to address growing concerns about reproducibility in scientific research by conducting replications of recent papers in the field of prostate cancer. This Registered Report describes the proposed replication plan of key experiments from "Androgen Receptor Splice Variants Determine Taxane Sensitivity in Prostate Cancer" by Thadani-Mulero and colleagues (2014) published in Cancer Research in 2014. The experiment that will be replicated is reported in Fig. 6A. Thadani-Mulero and colleagues generated xenografts from two prostate cancer cell lines; LuCaP 86.2, which expresses predominantly the ARv567 splice variant of the androgen receptor (AR), and LuCaP 23.1, which expresses the full length AR as well as the ARv7 variant. Treatment of the tumors with the taxane docetaxel showed that the drug inhibited tumor growth of the LuCaP 86.2 cells but not of the LuCaP 23.1 cells, indicating that expression of splice variants of the AR can affect sensitivity to docetaxel. The Prostate Cancer Foundation-Movember Foundation Reproducibility Initiative is a collaboration between the Prostate Cancer Foundation, the Movember Foundation and Science Exchange, and the results of the replications will be published by PeerJ.

Food, stress, and circulating testosterone: Cue integration by the testes, not the brain, in male zebra finches (Taeniopygia guttata).

Food abundance is closely associated with reproductive readiness in vertebrates. Food scarcity can activate the hypothalamo-pituitary-adrenal axis, decrease sex steroid secretion, and dampen reproductive behavior. However, the mechanisms underlying these transient effects are unclear. Gonadotropin inhibitory hormone (GnIH), a neuropeptide present in the brain and gonads, is also influenced by glucocorticoids and fasting in some species. We investigated whether fasting stress activated the GnIH system in zebra finches (Taeniopygia guttata), with the potential for downstream effects on reproductive physiology and behavior. We fasted or fed males ad libitum for 10h. Fasting increased corticosterone and decreased testosterone in circulation. To assess whether the decrease in testosterone was mediated by changes in the hypothalamus and/or the gonads, we (1) quantified GnRH- and GnIH-positive neurons in the hypothalamus, (2) assessed hypothalamic gene expression for GnRH and GnIH, and (3) examined gene expression for proteins involved in testosterone synthesis in fasted and control birds. No measure of hypothalamic neuropeptides was related to treatment or circulating steroids. However, birds with higher corticosterone had higher testicular GnIH expression and lower testosterone. StAR and LHR expression were lower in the testes of fasted birds than controls. Thus, the decrease in testosterone was not likely mediated by hypothalamic GnIH, but rather by direct actions of fasting and/or corticosterone on the testes, indicating that the testes can integrate and respond to cues of stress directly. Such local inhibition of testosterone synthesis may allow for rapid and reversible changes in physiology and behavior when conditions are inappropriate for breeding.

Social cues regulate reciprocal switching of hypothalamic Dio2/Dio3 and the transition into final follicle maturation in European starlings (Sturnus vulgaris).

With final maturation of ovarian follicles, birds are committed to a major energetic investment: egg laying. Follicles develop in a 2-step process: 1) initial development of regressed follicles stimulated by long days and 2) yolk incorporation into hierarchical follicles, ovulation, and oviposition. We know little about how females transduce environmental cues into neuroendocrine signals regulating the second step. The present study measures gene expression in tissues within the hypothalamo-pituitary-gonadal axis. Females were housed in seminatural enclosures experiencing natural changes in photoperiod and environmental cues (eg, temperature, rainfall, etc), without males or with constant access to males (January to April). By April, females with males had begun to lay eggs, whereas those without males had not. In a second study, females without males for 3.5 months were then given access to males for 7 days. Restricting male access completely inhibited final follicle maturation, whereas 7-day male access stimulated full vitellogenesis and follicle maturation. Few gene expression changes were attributable to constant male access (January to March), but naïve females given 7-day male access had increased type 2 deiodinase (DIO2) and decreased DIO3 synthesis in the hypothalamus, potentially influencing local thyroid hormone metabolism, increased expression of LH receptor and aromatase in follicles and vitellogenin in liver. Our data suggest that initial follicle development may be more heavily influenced by photoperiod, but the second step (final maturation) is sensitive to other cues such as social interactions. This is the first demonstration of a social effect on the Dio2/Dio3 system, previously thought only responsive to photoperiod cues.

Testicular growth and regression are not correlated with Dio2 expression in a wild male songbird, sturnus vulgaris, exposed to natural changes in photoperiod.

Timing of seasonal breeding in birds and mammals is regulated by changing the day length and is dependent on the presence of thyroid hormones. A mechanism for thyroid-dependent control of seasonality has been proposed, in which exposure to long day lengths induces rapid local conversion of T4 to its bioactive form, T3, via the up-regulation of the enzyme type 2 iodothyronine deiodinase (Dio2) in the brain, and the down-regulation of Dio3 (which inactivates T3). Such changes were correlated with gonadotropin release and gonadal growth in quail. This mechanism was elucidated in a domesticated species (quail) exposed to unnatural acute changes in day length. Here we investigated the Dio2/Dio3 mechanism in a wild species, the European starling, under naturally changing day length. Although Dio2 expression varied seasonally, Dio3 did not. We found no correlation of Dio2 with photoperiod, seasonal regulation of GnRH, or testicular volume. The observed differences in data from starlings and quail could be a result of phylogeny, genetic drift from founder populations, or differences in reproductive seasonality in addition to or instead of arising from domestication or use of artificially changing photoperiods. Overall, the data indicate that in a wild species exposed to natural changes in day length, the current proposed mechanism for photoperiodic timing is less straightforward than is generally accepted and might not be as universally applicable as previously thought.

Anticipating spring: wild populations of great tits (Parus major) differ in expression of key genes for photoperiodic time measurement.

Measuring day length is critical for timing annual changes in physiology and behavior in many species. Recently, rapid changes in several photoperiodically-controlled genes following exposure to a single long day have been described. Components of this 'first day release' model have so far only been tested in highly domesticated species: quail, sheep, goats and rodents. Because artificial selection accompanying domestication acts on genes related to photoperiodicity, we must also study this phenomenon in wild organisms for it to be accepted as universal. In a songbird, the great tit (Parus major), we tested whether a) these genes are involved in photoperiodic time measurement (PTM) in a wild species, and b) whether predictable species and population differences in expression patterns exist. Using quantitative RT-PCR, we compared gene expression after a single long day in male great tits from Sweden (57°42'N) with that from a German (47°43'N) population. Hypothalamic gene expression key for PTM changed only in the northern population, and occurred earlier after dawn during the single long day than demonstrated in quail; however, gonadotropins (secretion and synthesis) were stimulated in both populations, albeit with different timing. Our data are the first to show acute changes in gene expression in response to photostimulation in any wild species not selected for study of photoperiodism. The pronounced differences in gene expression in response to a single long day between two populations raise exciting new questions about potential environmental selection on photoperiodic cue sensitivity.

Potential roles for GNIH and GNRH-II in reproductive axis regulation of an opportunistically breeding songbird.

The ability to breed at any time of year enables opportunistically breeding species to respond to good conditions whenever they occur. We investigate the neuroendocrine basis for this relatively unusual reproductive pattern in the avian world. One proposed mechanism for year-round breeding ability is tonic activation of gonadotropin-releasing hormone-I (GnRH-I) production that is flexibly modified by gonadotropin-inhibitory hormone (GnIH) production during unfavorable conditions. GnIH could inhibit GnRH secretion from the hypothalamus and/or inhibit GnRH action on the anterior pituitary gland. We studied neuroendocrine patterns in wild Australian zebra finches (Taeniopygia guttata) sampled during a breeding period in Southern Australia, a non-breeding period in central Australia, and in juvenile males in the latter location. We asked whether patterns in immunoreactivity of three neuropeptides important for reproductive axis regulation, GnRH-I, GnRH-II and GnIH, during periods of breeding and non-breeding reflect this flexibility. We found that the numbers and sizes of immunoreactive (-ir) GnRH-I cells did not vary between breeding stages and ages. Contrary to our predictions, irGnIH cell number and size, as well as the synthesis of GnIH mRNA were similar in breeding and non-breeding conditions. However, breeding males had more and larger irGnRH-II cells in the midbrain compared to non-breeding males. Hence, while changes in irGnIH cells are not associated with fluctuations in gonadotropin secretion or gonad volume, the regulation of irGnRH-II cells might represent a previously-unidentified mechanism by which reproductive flexibility can be achieved; namely via behavioral neurotransmitter actions of GnRH-II rather than through the typical sensory-CNS integration-GnRH-I route.

The control of reproductive physiology and behavior by gonadotropin-inhibitory hormone.

Gonadotropin-releasing hormone (GnRH) controls the reproductive physiology and behavior of vertebrates by stimulating synthesis and release of gonadotropin from the pituitary gland. In 2000, another hypothalamic neuropeptide, gonadotropin-inhibitory hormone (GnIH), was discovered in quail and found to be an inhibiting factor for gonadotropin release. GnIH homologs are present in the brains of vertebrates, including birds, mammals, amphibians, and fish. These peptides, categorized as RF amide-related peptides (RFRPs), possess a characteristic LPXRF-amide (X = L or Q) motif at their C-termini. GnIH/RFRP precursor mRNA encodes a polypeptide that is possibly cleaved into three mature peptides in birds and two in mammals. The names of these peptides are GnIH, GnIH-related peptide-1 (GnIH-RP-1) and GnIH-RP-2 in birds, and RFRP-1 and RFRP-3 in mammals. GnIH/RFRP is synthesized in neurons of the paraventricular nucleus of the hypothalamus in birds and the dorsomedial hypothalamic area in mammals. GnIH neurons project to the median eminence, thus providing a functional neuroanatomical infrastructure to regulate anterior pituitary function. In quail, GnIH inhibits gonadal activity by decreasing synthesis and release of gonadotropin. The widespread distribution of GnIH/RFRP immunoreactive fibers in all animals tested suggests various actions within the brain. In accordance, GnIH/RFRP receptor mRNA is also expressed widely in the brain and the pituitary. GnIH/RFRP immunoreactive axon terminals are in probable contact with GnRH neurons in birds and mammals, and we recently demonstrated expression of GnIH receptor mRNA in GnRH-I and GnRH-II neurons in European starlings. Thus, GnIH/RFRP may also inhibit gonadotropin synthesis and release by inhibiting GnRH neurons in addition to having direct actions on the pituitary gland. Intracerebroventricular administration of GnIH/RFRP further inhibits reproductive behaviors in songbirds and rodents, possibly via direct actions on the GnRH system. The expression of GnIH/RFRP is regulated by melatonin which is an internal indicator of day length in vertebrates. Stress stimuli also regulate the expression of GnIH/RFRP in songbirds and rodents. Accordingly, GnIH/RFRP may serve as a transducer of environmental information and social interactions into endogenous physiology and behavior of the animal. Recently, it was shown that GnIH/RFRP and its receptor are also expressed in the gonads of birds, rodents and primates. In sum, the existing data suggest that GnIH/RFRP is an important mediator of reproductive function acting at the level of the brain, pituitary, and the gonad in birds and mammals.

Interactions of gonadotropin-releasing hormone (GnRH) and gonadotropin-inhibitory hormone (GnIH) in birds and mammals.

Gonadotropin-releasing hormone (GnRH) regulates secretion of both of the gonadotropins, luteinizing hormone (LH) and follicle-stimulating hormone. Thus, it is a key hormone for vertebrate reproduction. GnRH was considered to be unusual among hypothalamic neuropeptides in that it appeared to have no direct antagonist, although some neurochemicals and peripheral hormones (opiates, GABA, gonadal steroids, inhibin) can modulate gonadotropin release to a degree. Five years ago, a vertebrate hypothalamic neuropeptide that inhibited pituitary gonadotropin release in a dose-dependent manner was discovered in quail by Tsutsui et al. (2000. Biochem Biophys Res Commun 275:661-667). We now know that this inhibitory peptide, named gonadotropin-inhibitory hormone, or GnIH, is a regulator of gonadotropin release in vitro and in vivo. Its discovery has opened the door to an entirely new line of research within the realm of reproductive biology. In our collaborative studies, we have begun to elucidate the manner in which GnIH interacts with GnRH to time release of gonadotropins and thus time reproductive activity in birds and mammals. This paper reviews the distribution of GnIH in songbirds relative to GnRHs, and our findings on its modes of action in vitro and in vivo, based on laboratory and field studies. These data are simultaneously compared with our findings in mammals, highlighting how the use of different model species within different vertebrate classes can be a useful approach to identify the conserved actions of this novel neuropeptide, along with its potential importance to vertebrate reproduction.

Tonic activation of brain GnRH immunoreactivity despite reduction of peripheral reproductive parameters in opportunistically breeding zebra finches.

Opportunistically breeding species offer the unique opportunity to understand mechanisms in reproductive physiology that allow for extreme flexibility in the regulation of reproduction. We studied a well-known opportunistic breeder, the zebra finch (Taeniopygia guttata) to test the hypothesis that the reproductive axis of opportunists is in a constant state of 'near-readiness'. In wild zebra finches, reproduction is highly correlated with rainfall, and in the laboratory, water availability and humidity are the strongest cues to affect reproductive activation. We therefore subjected individuals to water restriction for eleven weeks followed by a two week period of ad libitum access to water. The control group had water freely available for the entire experiment. We measured the state of activation of the hypothalamo-pituitary gonad (HPG) axis at three levels: in the hypothalamus by measuring immunoreactive (ir) cGnRH-I and cGnRH-II; in the anterior pituitary gland by measuring plasma luteinizing hormone (LH); and in the gonads by measuring gonadal volume and function. We found that water restriction caused a reduction in circulating LH concentrations and that testis volume was more likely to decrease in water restricted than in control birds. Subsequent short-term return to ad libitum water availability caused LH to return to baseline in water restricted birds. These changes occurred without significant changes in ir-cGnRH-I, ir-cGnRH-II, or in testis function. These data suggest that in these opportunistic breeders, an inhibition of parts of the reproductive axis is not necessarily correlated with full inactivation of reproductive potential. GnRH-ir cells in the hypothalamus appear to remain active and able to respond to subsequent stimulation.

Latitudinal variation in plasma testosterone levels in birds of the genus Zonotrichia.

Birds breeding in northern latitudes generally have elevated plasma testosterone levels throughout the breeding season with a peak at the onset of the breeding season. In contrast, tropical birds tend to have extremely low plasma testosterone levels year round with only a slight increase during breeding. While these patterns have been consistent in the species investigated, closely related species have not been investigated across a range of latitudes. Birds of the genus Zonotrichia present an ideal opportunity to investigate latitudinal variation in plasma testosterone levels as breeding populations occur from northern Alaska to southern Argentina. We studied three taxa of Zonotrichia: (1) Gambel's white-crowned sparrows, Zonotrichia leucophrys gambelii, breeding at high latitude in northern Alaska, (2) Puget Sound white-crowned sparrows, Z. l. pugetensis, breeding at mid-latitude in Washington state, and (3) an equatorial population of the rufous-collared sparrow, Z. capensis, in Ecuador. To compare both baseline breeding and maximal testosterone levels, males from the three taxa were either bled immediately upon capture during the breeding season or first challenged with gonadotropin-releasing hormone (GnRH) and then bled. As a control, another group of birds experienced a saline challenge. Two-way ANCOVA, with treatment and taxa as factors and testis volume as a covariate, showed a significant effect of treatment on plasma testosterone levels. There was also a significant interaction between taxa and treatment. Contrary to expectations, the equatorial species exhibited greater plasma testosterone levels in response to the GnRH challenge than the high latitude species. There were no differences between the mid- and high-latitude species. The equatorial species had the smallest average testis size and within each taxa there were no relationships between plasma testosterone and testis size. These data suggest that latitudinal clines in plasma testosterone levels in Zonotrichia do not follow previously described patterns and that tropical birds can have levels of testosterone similar to northern latitude species.