Realism and robustness require increased sample size when studying both sexes.

A recent article claimed that researchers need not increase the overall sample size for a study that includes both sexes. This Formal Comment points out that that study assumed two sexes to have the same variance, and explains why this is a unrealistic assumption.

Finding the right power balance: Better study design and collaboration can reduce dependence on statistical power.

Power analysis currently dominates sample size determination for experiments, particularly in grant and ethics applications. Yet, this focus could paradoxically result in suboptimal study design because publication biases towards studies with the largest effects can lead to the overestimation of effect sizes. In this Essay, we propose a paradigm shift towards better study designs that focus less on statistical power. We also advocate for (pre)registration and obligatory reporting of all results (regardless of statistical significance), better facilitation of team science and multi-institutional collaboration that incorporates heterogenization, and the use of prospective and living meta-analyses to generate generalizable results. Such changes could make science more effective and, potentially, more equitable, helping to cultivate better collaborations.

Species sensitivities to artificial light at night: A phylogenetically controlled multilevel meta-analysis on melatonin suppression.

The rapid urbanization of our world has led to a surge in artificial lighting at night (ALAN), with profound effects on wildlife. Previous research on wildlife's melatonin, a crucial mechanistic indicator and mediator, has yielded inconclusive evidence due to a lack of comparative analysis. We compiled and analysed an evidence base including 127 experiments with 437 observations across 31 wild vertebrates using phylogenetically controlled multilevel meta-analytic models. The evidence comes mainly from the effects of white light on melatonin suppression in birds and mammals. We show a 36% average decrease in melatonin secretion in response to ALAN across a diverse range of species. This effect was observed for central and peripheral melatonin, diurnal and nocturnal species, and captive and free-living populations. We also reveal intensity-, wavelength-, and timing-dependent patterns of ALAN effects. Exposure to ALAN led to a 23% rise in inter-individual variability in melatonin suppression, with important implications for natural selection in wild vertebrates, as some individuals may display higher tolerance to ALAN. The cross-species evidence has strong implications for conservation of wild populations that are subject to natural selection of ALAN. We recommend measures to mitigate harmful impacts of ALAN, such as using 'smart' lighting systems to tune the spectra to less harmful compositions.

Academic publishing requires linguistically inclusive policies.

Scientific knowledge is produced in multiple languages but is predominantly published in English. This practice creates a language barrier to generate and transfer scientific knowledge between communities with diverse linguistic backgrounds, hindering the ability of scholars and communities to address global challenges and achieve diversity and equity in science, technology, engineering and mathematics (STEM). To overcome those barriers, publishers and journals should provide a fair system that supports non-native English speakers and disseminates knowledge across the globe. We surveyed policies of 736 journals in biological sciences to assess their linguistic inclusivity, identify predictors of inclusivity, and propose actions to overcome language barriers in academic publishing. Our assessment revealed a grim landscape where most journals were making minimal efforts to overcome language barriers. The impact factor of journals was negatively associated with adopting a number of inclusive policies whereas ownership by a scientific society tended to have a positive association. Contrary to our expectations, the proportion of both open access articles and editors based in non-English speaking countries did not have a major positive association with the adoption of linguistically inclusive policies. We proposed a set of actions to overcome language barriers in academic publishing, including the renegotiation of power dynamics between publishers and editorial boards.

Systematic approaches to assessing high-temperature limits to fertility in animals.

Critical thermal limits (CTLs) gauge the physiological impact of temperature on survival or critical biological function, aiding predictions of species range shifts and climatic resilience. Two recent Drosophila species studies, using similar approaches to determine temperatures that induce sterility (thermal fertility limits [TFLs]), reveal that TFLs are often lower than CTLs and that TFLs better predict both current species distributions and extinction probability. Moreover, many studies show fertility is more sensitive at less extreme temperatures than survival (thermal sensitivity of fertility [TSF]). These results present a more pessimistic outlook on the consequences of climate change. However, unlike CTLs, TFL data are limited to Drosophila, and variability in TSF methods poses challenges in predicting species responses to increasing temperature. To address these data and methodological gaps, we propose 3 standardized approaches for assessing thermal impacts on fertility. We focus on adult obligate sexual terrestrial invertebrates but also provide modifications for other animal groups and life-history stages. We first outline a "gold-standard" protocol for determining TFLs, focussing on the effects of short-term heat shocks and simulating more frequent extreme heat events predicted by climate models. As this approach may be difficult to apply to some organisms, we then provide a standardized TSF protocol. Finally, we provide a framework to quantify fertility loss in response to extreme heat events in nature, given the limitations in laboratory approaches. Applying these standardized approaches across many taxa, similar to CTLs, will allow robust tests of the impact of fertility loss on species responses to increasing temperatures.

Next steps after airing disagreement on a scientific issue with policy implications: a meta-analysis, multi-lab replication and adversarial collaboration.

Canadian policymakers are interested in determining whether farmed Atlantic salmon, frequently infected with Piscine orthoreovirus (PRV), may threaten wild salmon populations in the Pacific Northwest. A relevant work has been published in BMC Biology by Polinksi and colleagues, but their conclusion that PRV has a negligible impact on the energy expenditure and respiratory performance of sockeye salmon is disputed by Mordecai and colleagues, whose re-analysis is presented in a correspondence article. So, what is the true effect and what should follow this unresolved dispute? We suggest a 'registered multi-lab replication with adversaries'.

The influence of immune challenges on the mean and variance in reproductive investment: a meta-analysis of the terminal investment hypothesis.

Finding the optimal balance between survival and reproduction is a central puzzle in life-history theory. The terminal investment hypothesis predicts that when individuals encounter a survival threat that compromises future reproductive potential, they will increase immediate reproductive investment to maximise fitness. Despite decades of research on the terminal investment hypothesis, findings remain mixed. We examined the terminal investment hypothesis with a meta-analysis of studies that measured reproductive investment of multicellular iteroparous animals after a non-lethal immune challenge. We had two main aims. The first was to investigate whether individuals, on average, increase reproductive investment in response to an immune threat, as predicted by the terminal investment hypothesis. We also examined whether such responses vary adaptively on factors associated with the amount of reproductive opportunities left (residual reproductive value) in the individuals, as predicted by the terminal investment hypothesis. The second was to provide a quantitative test of a novel prediction based on the dynamic threshold model: that an immune threat increases between-individual variance in reproductive investment. Our results provided some support for our hypotheses. Older individuals, who are expected to have lower residual reproductive values, showed stronger mean terminal investment response than younger individuals. In terms of variance, individuals showed a divergence in responses, leading to an increase in variance. This increase in variance was especially amplified in longer-living species, which was consistent with our prediction that individuals in longer-living species should respond with greater individual variation due to increased phenotypic plasticity. We find little statistical evidence of publication bias. Together, our results highlight the need for a more nuanced view on the terminal investment hypothesis and a greater focus on the factors that drive individual responses.

Publication bias impacts on effect size, statistical power, and magnitude (Type M) and sign (Type S) errors in ecology and evolutionary biology.

Collaborative efforts to directly replicate empirical studies in the medical and social sciences have revealed alarmingly low rates of replicability, a phenomenon dubbed the 'replication crisis'. Poor replicability has spurred cultural changes targeted at improving reliability in these disciplines. Given the absence of equivalent replication projects in ecology and evolutionary biology, two inter-related indicators offer the opportunity to retrospectively assess replicability: publication bias and statistical power. This registered report assesses the prevalence and severity of small-study (i.e., smaller studies reporting larger effect sizes) and decline effects (i.e., effect sizes decreasing over time) across ecology and evolutionary biology using 87 meta-analyses comprising 4,250 primary studies and 17,638 effect sizes. Further, we estimate how publication bias might distort the estimation of effect sizes, statistical power, and errors in magnitude (Type M or exaggeration ratio) and sign (Type S). We show strong evidence for the pervasiveness of both small-study and decline effects in ecology and evolution. There was widespread prevalence of publication bias that resulted in meta-analytic means being over-estimated by (at least) 0.12 standard deviations. The prevalence of publication bias distorted confidence in meta-analytic results, with 66% of initially statistically significant meta-analytic means becoming non-significant after correcting for publication bias. Ecological and evolutionary studies consistently had low statistical power (15%) with a 4-fold exaggeration of effects on average (Type M error rates = 4.4). Notably, publication bias reduced power from 23% to 15% and increased type M error rates from 2.7 to 4.4 because it creates a non-random sample of effect size evidence. The sign errors of effect sizes (Type S error) increased from 5% to 8% because of publication bias. Our research provides clear evidence that many published ecological and evolutionary findings are inflated. Our results highlight the importance of designing high-power empirical studies (e.g., via collaborative team science), promoting and encouraging replication studies, testing and correcting for publication bias in meta-analyses, and adopting open and transparent research practices, such as (pre)registration, data- and code-sharing, and transparent reporting.

A robust and readily implementable method for the meta-analysis of response ratios with and without missing standard deviations.

The log response ratio, lnRR, is the most frequently used effect size statistic for meta-analysis in ecology. However, often missing standard deviations (SDs) prevent estimation of the sampling variance of lnRR. We propose new methods to deal with missing SDs via a weighted average coefficient of variation (CV) estimated from studies in the dataset that do report SDs. Across a suite of simulated conditions, we find that using the average CV to estimate sampling variances for all observations, regardless of missingness, performs with minimal bias. Surprisingly, even with missing SDs, this simple method outperforms the conventional approach (basing each effect size on its individual study-specific CV) with complete data. This is because the conventional method ultimately yields less precise estimates of the sampling variances than using the pooled CV from multiple studies. Our approach is broadly applicable and can be implemented in all meta-analyses of lnRR, regardless of 'missingness'.

The impact of rising temperatures on the prevalence of coral diseases and its predictability: A global meta-analysis.

Coral reefs are under threat from disease as climate change alters environmental conditions. Rising temperatures exacerbate coral disease, but this relationship is likely complex as other factors also influence coral disease prevalence. To better understand this relationship, we meta-analytically examined 108 studies for changes in global coral disease over time alongside temperature, expressed using average summer sea surface temperature (SST) and cumulative heat stress as weekly sea surface temperature anomalies (WSSTAs). We found that both rising average summer SST and WSSTA were associated with global increases in the mean and variability in coral disease prevalence. Global coral disease prevalence tripled, reaching 9.92% in the 25 years examined, and the effect of 'year' became more stable (i.e. prevalence has lower variance over time), contrasting the effects of the two temperature stressors. Regional patterns diverged over time and differed in response to average summer SST. Our model predicted that, under the same trajectory, 76.8% of corals would be diseased globally by 2100, even assuming moderate average summer SST and WSSTA. These results highlight the need for urgent action to mitigate coral disease. Mitigating the impact of rising ocean temperatures on coral disease is a complex challenge requiring global discussion and further study.

Low statistical power and overestimated anthropogenic impacts, exacerbated by publication bias, dominate field studies in global change biology.

Field studies are essential to reliably quantify ecological responses to global change because they are exposed to realistic climate manipulations. Yet such studies are limited in replicates, resulting in less power and, therefore, potentially unreliable effect estimates. Furthermore, while manipulative field experiments are assumed to be more powerful than non-manipulative observations, it has rarely been scrutinized using extensive data. Here, using 3847 field experiments that were designed to estimate the effect of environmental stressors on ecosystems, we systematically quantified their statistical power and magnitude (Type M) and sign (Type S) errors. Our investigations focused upon the reliability of field experiments to assess the effect of stressors on both ecosystem's response magnitude and variability. When controlling for publication bias, single experiments were underpowered to detect response magnitude (median power: 18%-38% depending on effect sizes). Single experiments also had much lower power to detect response variability (6%-12% depending on effect sizes) than response magnitude. Such underpowered studies could exaggerate estimates of response magnitude by 2-3 times (Type M errors) and variability by 4-10 times. Type S errors were comparatively rare. These observations indicate that low power, coupled with publication bias, inflates the estimates of anthropogenic impacts. Importantly, we found that meta-analyses largely mitigated the issues of low power and exaggerated effect size estimates. Rather surprisingly, manipulative experiments and non-manipulative observations had very similar results in terms of their power, Type M and S errors. Therefore, the previous assumption about the superiority of manipulative experiments in terms of power is overstated. These results call for highly powered field studies to reliably inform theory building and policymaking, via more collaboration and team science, and large-scale ecosystem facilities. Future studies also require transparent reporting and open science practices to approach reproducible and reliable empirical work and evidence synthesis.

Meta-analytic approaches and effect sizes to account for 'nuisance heterogeneity' in comparative physiology.

Meta-analysis is a powerful tool used to generate quantitatively informed answers to pressing global challenges. By distilling data from broad sets of research designs and study systems into standardised effect sizes, meta-analyses provide physiologists with opportunities to estimate overall effect sizes and understand the drivers of effect variability. Despite this ambition, research designs in the field of comparative physiology can appear, at the outset, as being vastly different to each other because of 'nuisance heterogeneity' (e.g. different temperatures or treatment dosages used across studies). Methodological differences across studies have led many to believe that meta-analysis is an exercise in comparing 'apples with oranges'. Here, we dispel this myth by showing how standardised effect sizes can be used in conjunction with multilevel meta-regression models to both account for the factors driving differences across studies and make them more comparable. We assess the prevalence of nuisance heterogeneity in the comparative physiology literature - showing it is common and often not accounted for in analyses. We then formalise effect size measures (e.g. the temperature coefficient, Q10) that provide comparative physiologists with a means to remove nuisance heterogeneity without the need to resort to more complex statistical models that may be harder to interpret. We also describe more general approaches that can be applied to a variety of different contexts to derive new effect sizes and sampling variances, opening up new possibilities for quantitative synthesis. By using effect sizes that account for components of effect heterogeneity, in combination with existing meta-analytic models, comparative physiologists can explore exciting new questions while making results from large-scale data sets more accessible, comparable and widely interpretable.

Towards open, reliable, and transparent ecology and evolutionary biology.

Unreliable research programmes waste funds, time, and even the lives of the organisms we seek to help and understand. Reducing this waste and increasing the value of scientific evidence require changing the actions of both individual researchers and the institutions they depend on for employment and promotion. While ecologists and evolutionary biologists have somewhat improved research transparency over the past decade (e.g. more data sharing), major obstacles remain. In this commentary, we lift our gaze to the horizon to imagine how researchers and institutions can clear the path towards more credible and effective research programmes.

Mapping the past, present and future research landscape of paternal effects.

BACKGROUND: Although in all sexually reproducing organisms an individual has a mother and a father, non-genetic inheritance has been predominantly studied in mothers. Paternal effects have been far less frequently studied, until recently. In the last 5 years, research on environmentally induced paternal effects has grown rapidly in the number of publications and diversity of topics. Here, we provide an overview of this field using synthesis of evidence (systematic map) and influence (bibliometric analyses). RESULTS: We find that motivations for studies into paternal effects are diverse. For example, from the ecological and evolutionary perspective, paternal effects are of interest as facilitators of response to environmental change and mediators of extended heredity. Medical researchers track how paternal pre-fertilization exposures to factors, such as diet or trauma, influence offspring health. Toxicologists look at the effects of toxins. We compare how these three research guilds design experiments in relation to objects of their studies: fathers, mothers and offspring. We highlight examples of research gaps, which, in turn, lead to future avenues of research. CONCLUSIONS: The literature on paternal effects is large and disparate. Our study helps in fostering connections between areas of knowledge that develop in parallel, but which could benefit from the lateral transfer of concepts and methods.

The jury is still out regarding the generality of adaptive 'transgenerational' effects.

A recent meta-analysis concluded, 'transgenerational effects are widespread, strong and persistent'. We identify biases in the literature search, data and analyses, questioning that conclusion. Re-analyses indicate few studies actually tested transgenerational effects - making it challenging to disentangle condition-transfer from anticipatory parental effects, and providing little insight into the underlying mechanisms.

Intervention for children with developmental coordination disorder: How robust is our recent evidence?

BACKGROUND: The evaluation of methods of intervention for children with developmental coordination disorder (DCD) is an ongoing process. Systematic and meta-analytical reviews play an important role in this process. Before 2015, only four reviews of this type were published on this topic. By 2019, the number had tripled, raising the question of whether the increase in quantity is accompanied by a similar increase in quality. The aim of this meta-review was to evaluate eight new review studies published during this time period with particular focus on the consistency of the inclusion criteria, the coverage of primary studies and the quality of evidence on which the conclusions were based. METHOD: Methodological quality was evaluated using the Assessment of Multiple Systematic Reviews, Second Edition (AMSTAR 2). RESULTS: The eight reviews differed in the inclusion criteria employed, the sources of primary evidence cited and in the quality of evidence provided. Using AMSTAR 2 terminology, the overall methodology of the reviews was judged to be acceptably high for only one. All others contained serious flaws. The conclusions drawn were not consistent. CONCLUSIONS: In spite of the increase in the number of reviews on intervention for children with DCD, quality remains an issue. Persistent methodological problems mean that consumers still cannot be confident in any of the claims made for intervention effects.

Meta-evaluation of meta-analysis: ten appraisal questions for biologists.

Meta-analysis is a statistical procedure for analyzing the combined data from different studies, and can be a major source of concise up-to-date information. The overall conclusions of a meta-analysis, however, depend heavily on the quality of the meta-analytic process, and an appropriate evaluation of the quality of meta-analysis (meta-evaluation) can be challenging. We outline ten questions biologists can ask to critically appraise a meta-analysis. These questions could also act as simple and accessible guidelines for the authors of meta-analyses. We focus on meta-analyses using non-human species, which we term 'biological' meta-analysis. Our ten questions are aimed at enabling a biologist to evaluate whether a biological meta-analysis embodies 'mega-enlightenment', a 'mega-mistake', or something in between.

Divide and conquer? Size adjustment with allometry and intermediate outcomes.

Many trait measurements are size-dependent, and while we often divide these traits by size before fitting statistical models to control for the effect of size, this approach does not account for allometry and the intermediate outcome problem. We describe these problems and outline potential solutions.

Nonindependence and sensitivity analyses in ecological and evolutionary meta-analyses.

Meta-analysis is an important tool for synthesizing research on a variety of topics in ecology and evolution, including molecular ecology, but can be susceptible to nonindependence. Nonindependence can affect two major interrelated components of a meta-analysis: (i) the calculation of effect size statistics and (ii) the estimation of overall meta-analytic estimates and their uncertainty. While some solutions to nonindependence exist at the statistical analysis stages, there is little advice on what to do when complex analyses are not possible, or when studies with nonindependent experimental designs exist in the data. Here we argue that exploring the effects of procedural decisions in a meta-analysis (e.g. inclusion of different quality data, choice of effect size) and statistical assumptions (e.g. assuming no phylogenetic covariance) using sensitivity analyses are extremely important in assessing the impact of nonindependence. Sensitivity analyses can provide greater confidence in results and highlight important limitations of empirical work (e.g. impact of study design on overall effects). Despite their importance, sensitivity analyses are seldom applied to problems of nonindependence. To encourage better practice for dealing with nonindependence in meta-analytic studies, we present accessible examples demonstrating the impact that ignoring nonindependence can have on meta-analytic estimates. We also provide pragmatic solutions for dealing with nonindependent study designs, and for analysing dependent effect sizes. Additionally, we offer reporting guidelines that will facilitate disclosure of the sources of nonindependence in meta-analyses, leading to greater transparency and more robust conclusions.