What's in a Badge? A Computational Reproducibility Investigation of the Open Data Badge Policy in One Issue of Psychological Science.

In April 2019, Psychological Science published its first issue in which all Research Articles received the Open Data badge. We used that issue to investigate the effectiveness of this badge, focusing on the adherence to its aim at Psychological Science: sharing both data and code to ensure reproducibility of results. Twelve researchers of varying experience levels attempted to reproduce the results of the empirical articles in the target issue (at least three researchers per article). We found that all 14 articles provided at least some data and six provided analysis code, but only one article was rated to be exactly reproducible, and three were rated as essentially reproducible with minor deviations. We suggest that researchers should be encouraged to adhere to the higher standard in force at Psychological Science. Moreover, a check of reproducibility during peer review may be preferable to the disclosure method of awarding badges.

Gun violence and media effects: challenges for science and public policy.

In response to the Sandy Hook shooting in December 2012, the White House published an action plan to reduce gun violence that, among other things, calls for research into the relationship with violence in digital games or other media images. We acknowledge the administration's efforts to reduce violent crime in society and their obligation to dedicate resources to matters of public interest, such as media effects. However, research projects launched in the midst of a moral panic bear the risk of introducing bias and distracting from more important issues. Ideological rigidity has repeatedly shaped past research on media violence. Current initiatives could be an opportunity to restore credibility to the field and to engage in a responsible dialogue on media effects. In order to inform public policy, we need to close gaps, both in empirical research and the academic debate, while being alert for potential political and social influences.

A weak scientific basis for gaming disorder: Let us err on the side of caution.

We greatly appreciate the care and thought that is evident in the 10 commentaries that discuss our debate paper, the majority of which argued in favor of a formalized ICD-11 gaming disorder. We agree that there are some people whose play of video games is related to life problems. We believe that understanding this population and the nature and severity of the problems they experience should be a focus area for future research. However, moving from research construct to formal disorder requires a much stronger evidence base than we currently have. The burden of evidence and the clinical utility should be extremely high, because there is a genuine risk of abuse of diagnoses. We provide suggestions about the level of evidence that might be required: transparent and preregistered studies, a better demarcation of the subject area that includes a rationale for focusing on gaming particularly versus a more general behavioral addictions concept, the exploration of non-addiction approaches, and the unbiased exploration of clinical approaches that treat potentially underlying issues, such as depressive mood or social anxiety first. We acknowledge there could be benefits to formalizing gaming disorder, many of which were highlighted by colleagues in their commentaries, but we think they do not yet outweigh the wider societal and public health risks involved. Given the gravity of diagnostic classification and its wider societal impact, we urge our colleagues at the WHO to err on the side of caution for now and postpone the formalization.

A Multilab Preregistered Replication of the Ego-Depletion Effect.

Good self-control has been linked to adaptive outcomes such as better health, cohesive personal relationships, success in the workplace and at school, and less susceptibility to crime and addictions. In contrast, self-control failure is linked to maladaptive outcomes. Understanding the mechanisms by which self-control predicts behavior may assist in promoting better regulation and outcomes. A popular approach to understanding self-control is the strength or resource depletion model. Self-control is conceptualized as a limited resource that becomes depleted after a period of exertion resulting in self-control failure. The model has typically been tested using a sequential-task experimental paradigm, in which people completing an initial self-control task have reduced self-control capacity and poorer performance on a subsequent task, a state known as ego depletion Although a meta-analysis of ego-depletion experiments found a medium-sized effect, subsequent meta-analyses have questioned the size and existence of the effect and identified instances of possible bias. The analyses served as a catalyst for the current Registered Replication Report of the ego-depletion effect. Multiple laboratories (k = 23, total N = 2,141) conducted replications of a standardized ego-depletion protocol based on a sequential-task paradigm by Sripada et al. Meta-analysis of the studies revealed that the size of the ego-depletion effect was small with 95% confidence intervals (CIs) that encompassed zero (d = 0.04, 95% CI [-0.07, 0.15]. We discuss implications of the findings for the ego-depletion effect and the resource depletion model of self-control.

Press CRTT to measure aggressive behavior: the unstandardized use of the competitive reaction time task in aggression research.

The competitive reaction time task (CRTT) is the measure of aggressive behavior most commonly used in laboratory research. However, the test has been criticized for issues in standardization because there are many different test procedures and at least 13 variants to calculate a score for aggressive behavior. We compared the different published analyses of the CRTT using data from 3 different studies to scrutinize whether it would yield the same results. The comparisons revealed large differences in significance levels and effect sizes between analysis procedures, suggesting that the unstandardized use and analysis of the CRTT have substantial impacts on the results obtained, as well as their interpretations. Based on the outcome of our comparisons, we provide suggestions on how to address some of the issues associated with the CRTT, as well as a guideline for researchers studying aggressive behavior in the laboratory.