What's in a Badge? A Computational Reproducibility Investigation of the Open Data Badge Policy in One Issue of Psychological Science.

In April 2019, Psychological Science published its first issue in which all Research Articles received the Open Data badge. We used that issue to investigate the effectiveness of this badge, focusing on the adherence to its aim at Psychological Science: sharing both data and code to ensure reproducibility of results. Twelve researchers of varying experience levels attempted to reproduce the results of the empirical articles in the target issue (at least three researchers per article). We found that all 14 articles provided at least some data and six provided analysis code, but only one article was rated to be exactly reproducible, and three were rated as essentially reproducible with minor deviations. We suggest that researchers should be encouraged to adhere to the higher standard in force at Psychological Science. Moreover, a check of reproducibility during peer review may be preferable to the disclosure method of awarding badges.

A reassessment of the Resistance to Framing scale.

Risky-choice and attribute framing effects are well-known cognitive biases, where choices are influenced by the way information is presented. To assess susceptibility to these framing types, the Resistance to Framing scale is often used, although its performance has rarely been extensively tested. In an online survey among university students from Bulgaria (N = 245) and North America (N = 261), we planned to examine the scale's psychometric properties, structural validity, and measurement invariance. However, some of these examinations were not possible because the scale displayed low and mostly non-significant inter-item correlations as well as low item-total correlations. Followingly, exploratory item response theory analyses indicated that the scale's reliability was low, especially for high levels of resistance to framing. This suggests problems with the scale at a basic level of conceptualization, namely that the items may not represent the same content domain. Overall, the scale in its current version is of limited use, at least in university student samples, due to the identified problems. We discuss potential remedies to these problems, as well as provide open code and data ( https://osf.io/j5n6f ) which facilitates testing the scale in other samples (e.g., general population, different languages and countries) to obtain a comprehensive picture of its performance.

Smart Distance Lab's art fair, experimental data on social distancing during the COVID-19 pandemic.

In the absence of a vaccine, social distancing behaviour is pivotal to mitigate COVID-19 virus spread. In this large-scale behavioural experiment, we gathered data during Smart Distance Lab: The Art Fair (n = 839) between August 28 and 30, 2020 in Amsterdam, the Netherlands. We varied walking directions (bidirectional, unidirectional, and no directions) and supplementary interventions (face mask and buzzer to alert visitors of 1.5 metres distance). We captured visitors' movements using cameras, registered their contacts (defined as within 1.5 metres) using wearable sensors, and assessed their attitudes toward COVID-19 as well as their experience during the event using questionnaires. We also registered environmental measures (e.g., humidity). In this paper, we describe this unprecedented, multi-modal experimental data set on social distancing, including psychological, behavioural, and environmental measures. The data set is available on figshare and in a MySQL database. It can be used to gain insight into (attitudes toward) behavioural interventions promoting social distancing, to calibrate pedestrian models, and to inform new studies on behavioural interventions.

The general fault in our fault lines.

Pervading global narratives suggest that political polarization is increasing, yet the accuracy of such group meta-perceptions has been drawn into question. A recent US study suggests that these beliefs are inaccurate and drive polarized beliefs about out-groups. However, it also found that informing people of inaccuracies reduces those negative beliefs. In this work, we explore whether these results generalize to other countries. To achieve this, we replicate two of the original experiments with 10,207 participants across 26 countries. We focus on local group divisions, which we refer to as fault lines. We find broad generalizability for both inaccurate meta-perceptions and reduced negative motive attribution through a simple disclosure intervention. We conclude that inaccurate and negative group meta-perceptions are exhibited in myriad contexts and that informing individuals of their misperceptions can yield positive benefits for intergroup relations. Such generalizability highlights a robust phenomenon with implications for political discourse worldwide.