Replication of published results is crucial for ensuring the robustness and self-correction of research, yet replications are scarce in many fields. Replicating researchers will therefore often have to decide which of several relevant candidates to target for replication. Formal strategies for efficient study selection have been proposed, but none have been explored for practical feasibility - a prerequisite for validation. Here we move one step closer to efficient replication study selection by exploring the feasibility of a particular selection strategy that estimates replication value as a function of citation impact and sample size (Isager, van 't Veer, & Lakens, 2021). We tested our strategy on a sample of fMRI studies in social neuroscience. We first report our efforts to generate a representative candidate set of replication targets. We then explore the feasibility and reliability of estimating replication value for the targets in our set, resulting in a dataset of 1358 studies ranked on their value of prioritising them for replication. In addition, we carefully examine possible measures, test auxiliary assumptions, and identify boundary conditions of measuring value and uncertainty. We end our report by discussing how future validation studies might be designed. Our study demonstrates the importance of investigating how to implement study selection strategies in practice. Our sample and study design can be extended to explore the feasibility of other formal study selection strategies that have been proposed.
Cognitive Neuroscience , Humans , Feasibility Studies , Reproducibility of Results , Uncertainty , Research Design
Increased execution of replication studies contributes to the effort to restore credibility of empirical research. However, a second generation of problems arises: the number of potential replication targets is at a serious mismatch with available resources. Given limited resources, replication target selection should be well-justified, systematic and transparently communicated. At present the discussion on what to consider when selecting a replication target is limited to theoretical discussion, self-reported justifications and a few formalized suggestions. In this Registered Report, we proposed a study involving the scientific community to create a list of considerations for consultation when selecting a replication target in psychology. We employed a modified Delphi approach. First, we constructed a preliminary list of considerations. Second, we surveyed psychologists who previously selected a replication target with regards to their considerations. Third, we incorporated the results into the preliminary list of considerations and sent the updated list to a group of individuals knowledgeable about concerns regarding replication target selection. Over the course of several rounds, we established consensus regarding what to consider when selecting a replication target. The resulting checklist can be used for transparently communicating the rationale for selecting studies for replication.
Robust scientific knowledge is contingent upon replication of original findings. However, replicating researchers are constrained by resources, and will almost always have to choose one replication effort to focus on from a set of potential candidates. To select a candidate efficiently in these cases, we need methods for deciding which out of all candidates considered would be the most useful to replicate, given some overall goal researchers wish to achieve. In this article we assume that the overall goal researchers wish to achieve is to maximize the utility gained by conducting the replication study. We then propose a general rule for study selection in replication research based on the replication value of the set of claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by conducting a replication of the claim, and is a function of (a) the value of being certain about the claim, and (b) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value. Our goal in this article is not to define concrete guidelines for study selection, but to provide the necessary theoretical foundations on which such concrete guidelines could be built. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
Knowledge , Models, Theoretical , Humans , Uncertainty
Preregistration clarifies the distinction between planned and unplanned research by reducing unnoticed flexibility. This improves credibility of findings and calibration of uncertainty. However, making decisions before conducting analyses requires practice. During report writing, respecting both what was planned and what actually happened requires good judgment and humility in making claims.
Registries , Research , Humans , Reproducibility of Results , Research Design
The underlying mechanisms of paternal responses to infant signals are poorly understood. Vasopressin has previously been proposed to affect these responses. Using a double-blind, placebo-controlled, within-subject design (N = 25 expectant fathers), we examined the effect of vasopressin administration on the use of excessive handgrip force during exposure to infant crying versus matched control sounds, while participants saw morphed images representing their own infant versus an unknown infant. We found that, compared to placebo, AVP administration elicited more excessive force while viewing an unknown infant image compared to viewing the image representing one's own infant, while the reverse was true under placebo. The results are discussed in light of vasopressin's role in parenting and parental protection among human fathers.
Perceiving potential threat to an infant and responding to it is crucial for offspring survival and parent-child bonding. Using a combination of functional magnetic resonance imaging and multi-informant reports, this longitudinal study explores the neural basis for paternal responses to threat to infants pre-natally (N = 21) and early post-natally (n = 17). Participants viewed videos showing an infant in danger and matched control videos, while instructed to imagine that the infant was their own or someone else's. Effects were found for infant-threatening vs neutral situations in the amygdala (region-of-interest analyses) and in clusters spanning cortical and subcortical areas (whole-brain analyses). An interaction effect revealed increased activation for own (vs unknown) infants in threatening (vs neutral) situations in bilateral motor areas, possibly indicating preparation for action. Post-natal activation patterns were similar; however, in part of the superior frontal gyrus the distinction between threat to own and unknown infant faded. Fathers showing more protective behavior in daily life recruited part of the frontal pole more when confronted with threat to their own vs an unknown infant. This exploratory study is the first to describe neural mechanisms involved in paternal protection and provides a basis for future work on fathers' protective parenting.
Brain/physiology , Fathers , Object Attachment , Parenting , Paternal Behavior/physiology , Adult , Brain/diagnostic imaging , Humans , Infant , Longitudinal Studies , Magnetic Resonance Imaging/methods , Male , Young Adult
In a randomized, double blind, placebo-controlled, within-subject magnetic resonance imaging study, we examined the effect of 20â¯IU intranasal vasopressin on the neural processing of infant crying in 25 fathers-to-be. We explored whether familial background modulates vasopressin effects, and whether vasopressin differentially affects cry processing coupled with neutral or emotional contextual information. Participants listened to cries accompanied by neutral ('this is an infant') or emotional ('this infant is sick/bored') contextual information, and neutral control sounds ('this is a saw'). Additionally, participants reported on their childhood experiences of parental love-withdrawal and abuse. Infant crying (vs control sounds) was associated with increased activation in the bilateral auditory cortex and posterior medial cortex. No effects of vasopressin were found in this 'cry network'. Exploratory whole-brain analyses suggested that effects of vasopressin in the anterior cingulate cortex, paracingulate gyrus and supplemental motor area were stronger in fathers who experienced lower (vs higher) levels of love-withdrawal. No interaction was observed for abuse. Vasopressin increased activation in response to cries accompanied by emotional vs neutral contextual information in several brain regions, e.g. the cerebellum, brainstem (midbrain), posterior medial cortex, hippocampus, putamen, and insula. Our results suggest that the experience of love-withdrawal may modulate the vasopressin system, influencing effects of vasopressin administration on cry processing. Results further suggest a role for vasopressin in the processing of cry sounds with emotional contextual information.
Brain/drug effects , Crying/psychology , Emotions/drug effects , Fathers/psychology , Vasopressins/pharmacology , Administration, Intranasal , Adult , Auditory Perception/drug effects , Auditory Perception/physiology , Brain/physiology , Brain Mapping/methods , Double-Blind Method , Female , Humans , Infant , Magnetic Resonance Imaging/methods , Male , Nasal Sprays , Placebos , Pregnancy/psychology , Vasopressins/administration & dosage , Young Adult
A pre-registered experiment was conducted to examine psychophysiological responses to being lied to. Bridging research on social cognition and deception detection, we hypothesized that observing a liar compared to a truth-teller would decrease finger skin temperature of observers. Participants first watched two targets while not forewarned that they would later be asked to judge (direct and indirect) veracity, and then watched another two targets while forewarned about this. During both these phases finger skin temperature was measured. Findings pertaining to temperature partly confirmed our main hypothesis. When participants were observing a liar, irrespective of being forewarned, on average finger skin temperature declined over time. In the forewarned phase, temperature trajectories of truth-tellers were higher than those of liars, however, in the not forewarned phase, this pattern was reversed. Results confirmed our further hypotheses that participants judge liars as less likeable and less trustworthy than truth-tellers-an indication of indirect deception detection. Our hypothesis that the effect size for trustworthiness would be bigger than that of liking was not supported by the data. Additionally, and also confirming our hypothesis, participants performed around chance level when directly judging whether the target person was lying. Exploratory analyses are reported with regard to truth bias and dependency between direct and indirect veracity judgments. Limitations and directions for future work related to the existence of psychophysiological indicators of deception detection are discussed.
