Reporting and interpreting non-significant results in animal cognition research.

How statistically non-significant results are reported and interpreted following null hypothesis significance testing is often criticized. This issue is important for animal cognition research because studies in the field are often underpowered to detect theoretically meaningful effect sizes, i.e., often produce non-significant p-values even when the null hypothesis is incorrect. Thus, we manually extracted and classified how researchers report and interpret non-significant p-values and examined the p-value distribution of these non-significant results across published articles in animal cognition and related fields. We found a large amount of heterogeneity in how researchers report statistically non-significant p-values in the result sections of articles, and how they interpret them in the titles and abstracts. Reporting of the non-significant results as "No Effect" was common in the titles (84%), abstracts (64%), and results sections (41%) of papers, whereas reporting of the results as "Non-Significant" was less common in the titles (0%) and abstracts (26%), but was present in the results (52%). Discussions of effect sizes were rare (<5% of articles). A p-value distribution analysis was consistent with research being performed with low power of statistical tests to detect effect sizes of interest. These findings suggest that researchers in animal cognition should pay close attention to the evidence used to support claims of absence of effects in the literature, and-in their own work-report statistically non-significant results clearly and formally correct, as well as use more formal methods of assessing evidence against theoretical predictions.

Medial temporal lobe structure, mnemonic and perceptual discrimination in healthy older adults and those at risk for mild cognitive impairment.

Cognitive tests sensitive to the integrity of the medial temporal lobe (MTL), such as mnemonic discrimination of perceptually similar stimuli, may be useful early markers of risk for cognitive decline in older populations. Perceptual discrimination of stimuli with overlapping features also relies on MTL but remains relatively unexplored in this context. We assessed mnemonic discrimination in two test formats (Forced Choice, Yes/No) and perceptual discrimination of objects and scenes in 111 community-dwelling older adults at different risk status for cognitive impairment based on neuropsychological screening. We also investigated associations between performance and MTL sub-region volume and thickness. The at-risk group exhibited reduced entorhinal thickness and impaired perceptual and mnemonic discrimination. Perceptual discrimination impairment partially explained group differences in mnemonic discrimination and correlated with entorhinal thickness. Executive dysfunction accounted for Yes/No deficits in at-risk adults, demonstrating the importance of test format for the interpretation of memory decline. These results suggest that perceptual discrimination tasks may be useful tools for detecting incipient cognitive impairment related to reduced MTL integrity in nonclinical populations.

Investigation of mirror-self recognition in ravens (Corvus corax).

Large-brained birds, such as corvids and parrots, tend to fail tests for self-recognition (mirror self-recognition [MSR]), but the limited positive evidence for MSR in these species has been questioned due to methodological limitations. In the present study, we aimed to investigate MSR in ravens by performing three mirror tests: a mirror exposure test, a mirror preference test, and a mark test. Across all three tests, the ravens' behavior was not consistent with MSR. Three out of six ravens infrequently interacted with the mirror and the nonmirror surfaces. Two birds explored the mirror and occasionally displayed contingent behaviors. Finally, the ravens made very few social displays toward the mirror, suggesting that at this stage they did not treat their reflection as a conspecific. These findings, along with the current evidence available, raise further questions on the validity of relying on one test to establish self-recognition and call for the development of methods beyond mirror tests to explore self-recognition in nonhuman animals. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Little evidence that Eurasian jays protect their caches by responding to cues about a conspecific's desire and visual perspective.

Eurasian jays have been reported to protect their caches by responding to cues about either the visual perspective or current desire of an observing conspecific, similarly to other corvids. Here, we used established paradigms to test whether these birds can - like humans - integrate multiple cues about different mental states and perform an optimal response accordingly. Across five experiments, which also include replications of previous work, we found little evidence that our jays adjusted their caching behaviour in line with the visual perspective and current desire of another agent, neither by integrating these social cues nor by responding to only one type of cue independently. These results raise questions about the reliability of the previously reported effects and highlight several key issues affecting reliability in comparative cognition research.

Eurasian jays, Garrulus glandarius, are members of the crow family. These large-brained birds hide food when it is abundant, and eat it later, when it is scarce. Previous studies have found that jays avoid theft by other jays by carefully deciding what food to hide, and where. In one study, they preferred to hide their food behind an opaque barrier, rather than a transparent one, when another jay was watching. In a second study, they preferred to hide food that the watching jay had already eaten enough of, and thus did not want. These studies suggest that jays have flexible cognitive skills when it comes to protecting their food. They respond to whether a potential thief can see their hiding place and to how much a thief might want the food they are stashing. The next question is, can Eurasian jays combine these two pieces of information? For example, if a jay has two types of food they could hide when another jay is present, but only has one place to hide them (either in view or out-of-view of the other jay), does the first jay prefer to stash the food that the second jay has already eaten, and therefore does not want anymore, only when the hiding place is visible to second jay? To find out, Amodio et al. watched Eurasian jays hiding macadamia nuts or peanuts in the presence of another jay. In the first setup, jays were given one food to hide and two possible hiding places, one opaque and one transparent, while being watched by a jay that had either had its fill of the food, or not tried it. In the second setup, jays were given both foods to hide, but only had one place to hide them (either transparent or opaque); while being watched by a jay that had eaten enough of one of the foods. Contrary to expectations, the jays did not seem to be able to combine the information about what the other jay could see and what it had been eating. In fact, they seemed unable to respond to either piece of information. When Amodio et al. repeated the original experiments, the jays did not seem to prefer to hide food out of sight, or to hide food that the watcher had already eaten. These results raise questions about the repeatability of experiments on food hiding strategies in birds of the crow family. It suggests that previous findings should be further investigated, potentially to identify important factors that might affect the repeatability of food-hiding tactics. Repeating the experiments may show how best to investigate behavioural patterns in jays in the future.

The hidden side of animal cognition research: Scientists' attitudes toward bias, replicability and scientific practice.

Animal cognition research aims to understand animal minds by using a diverse range of methods across an equally diverse range of species. Throughout its history, the field has sought to mitigate various biases that occur when studying animal minds, from experimenter effects to anthropomorphism. Recently, there has also been a focus on how common scientific practices might affect the reliability and validity of published research. Usually, these issues are discussed in the literature by a small group of scholars with a specific interest in the topics. This study aimed to survey a wider range of animal cognition researchers to ask about their attitudes towards classic and contemporary issues facing the field. Two-hundred and ten active animal cognition researchers completed our survey, and provided answers on questions relating to bias, replicability, statistics, publication, and belief in animal cognition. Collectively, researchers were wary of bias in the research field, but less so in their own work. Over 70% of researchers endorsed Morgan's canon as a useful principle but many caveated this in their free-text responses. Researchers self-reported that most of their studies had been published, however they often reported that studies went unpublished because they had negative or inconclusive results, or results that questioned "preferred" theories. Researchers rarely reported having performed questionable research practices themselves-however they thought that other researchers sometimes (52.7% of responses) or often (27.9% of responses) perform them. Researchers near unanimously agreed that replication studies are important but too infrequently performed in animal cognition research, 73.0% of respondents suggested areas of animal cognition research could experience a 'replication crisis' if replication studies were performed. Consistently, participants' free-text responses provided a nuanced picture of the challenges animal cognition research faces, which are available as part of an open dataset. However, many researchers appeared concerned with how to interpret negative results, publication bias, theoretical bias and reliability in areas of animal cognition research. Collectively, these data provide a candid overview of barriers to progress in animal cognition and can inform debates on how individual researchers, as well as organizations and journals, can facilitate robust scientific research in animal cognition.

Replications, Comparisons, Sampling and the Problem of Representativeness in Animal Cognition Research.

Animal cognition research often involves small and idiosyncratic samples. This can constrain the generalizability and replicability of a study's results and prevent meaningful comparisons between samples. However, there is little consensus about what makes a strong replication or comparison in animal research. We apply a resampling definition of replication to answer these questions in Part 1 of this article, and, in Part 2, we focus on the problem of representativeness in animal research. Through a case study and a simulation study, we highlight how and when representativeness may be an issue in animal behavior and cognition research and show how the representativeness problems can be viewed through the lenses of, i) replicability, ii) generalizability and external validity, iii) pseudoreplication and, iv) theory testing. Next, we discuss when and how researchers can improve their ability to learn from small sample research through, i) increasing heterogeneity in experimental design, ii) increasing homogeneity in experimental design, and, iii) statistically modeling variation. Finally, we describe how the strongest solutions will vary depending on the goals and resources of individual research programs and discuss some barriers towards implementing them.

Testing two competing hypotheses for Eurasian jays' caching for the future.

Previous research reported that corvids preferentially cache food in a location where no food will be available or cache more of a specific food in a location where this food will not be available. Here, we consider possible explanations for these prospective caching behaviours and directly compare two competing hypotheses. The Compensatory Caching Hypothesis suggests that birds learn to cache more of a particular food in places where that food was less frequently available in the past. In contrast, the Future Planning Hypothesis suggests that birds recall the 'what-when-where' features of specific past events to predict the future availability of food. We designed a protocol in which the two hypotheses predict different caching patterns across different caching locations such that the two explanations can be disambiguated. We formalised the hypotheses in a Bayesian model comparison and tested this protocol in two experiments with one of the previously tested species, namely Eurasian jays. Consistently across the two experiments, the observed caching pattern did not support either hypothesis; rather it was best explained by a uniform distribution of caches over the different caching locations. Future research is needed to gain more insight into the cognitive mechanism underpinning corvids' caching for the future.

It's not just the animals that are STRANGE.

A "STRANGE" framework has been proposed to mitigate the effects of sampling biases in animal experiments (Webster & Rutz, Nature, 582, 337-340, 2020). While sampling biases are likely a major cause of poor replicability and generalizability in animal behavior research, a more comprehensive and cautious approach is needed to effectively guide decision-making in animal research.

Trialling Meta-Research in Comparative Cognition: Claims and Statistical Inference in Animal Physical Cognition.

Scientific disciplines face concerns about replicability and statistical inference, and these concerns are also relevant in animal cognition research. This paper presents a first attempt to assess how researchers make and publish claims about animal physical cognition, and the statistical inferences they use to support them. We surveyed 116 published experiments from 63 papers on physical cognition, covering 43 different species. The most common tasks in our sample were trap-tube tasks (14 papers), other tool use tasks (13 papers), means-end understanding and string-pulling tasks (11 papers), object choice and object permanence tasks (9 papers) and access tasks (5 papers). This sample is not representative of the full scope of physical cognition research; however, it does provide data on the types of statistical design and publication decisions researchers have adopted. Across the 116 experiments, the median sample size was 7. Depending on the definitions we used, we estimated that between 44% and 59% of our sample of papers made positive claims about animals' physical cognitive abilities, between 24% and 46% made inconclusive claims, and between 10% and 17% made negative claims. Several failures of animals to pass physical cognition tasks were reported. Although our measures had low inter-observer reliability, these findings show that negative results can and have been published in the field. However, publication bias is still present, and consistent with this, we observed a drop in the frequency of p-values above .05. This suggests that some non-significant results have not been published. More promisingly, we found that researchers are likely making many correct statistical inferences at the individual-level. The strength of evidence of statistical effects at the group-level was weaker, and its p-value distribution was consistent with some effect sizes being overestimated. Studies such as ours can form part of a wider investigation into statistical reliability in comparative cognition. However, future work should focus on developing the validity and reliability of the measurements they use, and we offer some starting points.

Replications in Comparative Cognition: What Should We Expect and How Can We Improve?

Direct replication studies follow an original experiment's methods as closely as possible. They provide information about the reliability and validity of an original study's findings. The present paper asks what comparative cognition should expect if its studies were directly replicated, and how researchers can use this information to improve the reliability of future research. Because published effect sizes are likely overestimated, comparative cognition researchers should not expect findings with p-values just below the significance level to replicate consistently. Nevertheless, there are several statistical and design features that can help researchers identify reliable research. However, researchers should not simply aim for maximum replicability when planning studies; comparative cognition faces strong replicability-validity and replicability-resource trade-offs. Next, the paper argues that it may not even be possible to perform truly direct replication studies in comparative cognition because of: 1) a lack of access to the species of interest; 2) real differences in animal behavior across sites; and 3) sample size constraints producing very uncertain statistical estimates, meaning that it will often not be possible to detect statistical differences between original and replication studies. These three reasons suggest that many claims in the comparative cognition literature are practically unfalsifiable, and this presents a challenge for cumulative science in comparative cognition. To address this challenge, comparative cognition can begin to formally assess the replicability of its findings, improve its statistical thinking and explore new infrastructures that can help to form a field that can create and combine the data necessary to understand how cognition evolves.

Does social distance modulate adults' egocentric biases when reasoning about false beliefs?

When given privileged information of an object's true location, adults often overestimate the likelihood that a protagonist holding a false belief will search in the correct location for that object. This type of egocentric bias is often labelled the 'curse of knowledge'. Interestingly, the magnitude of this bias may be modulated by the social distance between the perspective taker and target. However, this social distance effect has yet to be fully demonstrated when adults reason about false beliefs. Using a continuous false belief task, we investigated i) whether adults were biased by their own knowledge when reasoning about another's false belief, ii) whether the magnitude of this egocentric bias was modulated by social distance, and iii) whether this social distance effect extended to a heterospecific out-group, namely a dog. To test these hypotheses we conducted three experiments. In Experiment 1 (N = 283), we used an established continuous false belief task, in Experiment 2 (N = 281) we modified this task, and Experiment 3 (N = 744) was a direct replication of Experiment 2. Across these experiments, the curse of knowledge effect was reliably replicated when adults mentalised about an in-group protagonist, and replicated in two of the three studies (Experiments 1 and 3) when adults mentalised about out-group protagonists. In an internal-meta analysis, the curse of knowledge effect was present across all conditions, and there was no effect of social distance. Hence, overall these data are not consistent with the hypothesis that social distance modulates adults' egocentric biases when reasoning about false beliefs. The finding that egocentric biases of a similar magnitude were observed when adults mentalised about an in-group protagonist and a dog suggests that interpersonal dissimilarity is not in itself sufficient to reduce egocentric bias when reasoning about false beliefs.