Variability in the analysis of a single neuroimaging dataset by many teams.

Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2-5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.

Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs.

Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity-variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity-estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs-indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis.

Market shocks and professionals' investment behavior - Evidence from the COVID-19 crash.

We investigate how the experience of extreme events, such as the COVID-19 market crash, influence risk-taking behavior. To isolate changes in risk-taking from other factors, we ran controlled experiments with finance professionals in December 2019 and March 2020. We observe that their investments in the experiment were 12 percent lower in March 2020 than in December 2019, although their price expectations had not changed, and although they considered the experimental asset less risky during the crash than before. This lower perceived risk is likely due to adaptive normalization, as volatility during the shock is compared to volatility experienced in real markets (which was low in December 2019, but very high in March 2020). Lower investments during the crash can be supported by higher risk aversion, not by changes in beliefs.

On the role of monetary incentives in risk preference elicitation experiments.

Incentivized experiments in which individuals receive monetary rewards according to the outcomes of their decisions are regarded as the gold standard for preference elicitation in experimental economics. These task-related real payments are considered necessary to reveal subjects' "true preferences." Using a systematic, large-sample approach with three subject pools of private investors, professional investors, and students, we test the effect of task-related monetary incentives on risk preferences in four standard experimental tasks. We find no significant differences in behavior between and within subjects in the incentivized and non-incentivized regimes. We discuss implications for academic research and forions in the field. Supplementary Information: The online version contains supplementary material available at 10.1007/s11166-022-09377-w.

The impact of a financial transaction tax on stylized facts of price returns-Evidence from the lab.

As the introduction of financial transaction taxes is increasingly discussed by political leaders we explore possible consequences such taxes could have on markets. Here we examine how "stylized facts", namely fat tails and volatility clustering, are affected by different tax regimes in laboratory experiments. We find that leptokurtosis of price returns is highest and clustered volatility is weakest in unilaterally taxed markets (where tax havens exist). Instead, tails are slimmest and volatility clustering is strongest in tax havens. When an encompassing financial transaction tax is levied, stylized facts hardly change compared to a scenario with no tax on all markets.

Crowdsourcing interventions to promote uptake of COVID-19 booster vaccines.

Background: COVID-19 booster vaccine uptake rates are behind the rate of primary vaccination in many countries. Governments and non-governmental institutions rely on a range of interventions aiming to increase booster uptake. Yet, little is known how experts and the general public evaluate these interventions. Methods: We applied a novel crowdsourcing approach to provide rapid insights on the most promising interventions to promote uptake of COVID-19 booster vaccines. In the first phase (December 2021), international experts (n = 78 from 17 countries) proposed 46 unique interventions. To reduce noise and potential bias, in the second phase (January 2022), experts (n = 307 from 34 countries) and representative general population samples from the UK (n = 299) and the US (n = 300) rated the proposed interventions on several evaluation criteria, including effectiveness and acceptability, on a 5-point Likert-type scale. Findings: Sanctions were evaluated as potentially most effective but least accepted. Evaluations by expert and general population samples were considerably aligned. Interventions that received the most positive evaluations regarding both effectiveness and acceptability across evaluation groups were: a day off work after getting vaccinated, financial incentives, tax benefits, promotional campaigns, and mobile vaccination teams. Interpretation: The results provide useful insights to help governmental and non-governmental institutions in their decisions about which interventions to implement. Additionally, the applied crowdsourcing method may be used in future studies to retrieve rapid insights on the comparative evaluation of (health) policies. Funding: This study received funding from the Austrian Science Fund (SFB F63) and the University of Vienna.

Market microstructure matters when imposing a Tobin tax-Evidence from the lab.

TRADING IN FX MARKETS IS DOMINATED BY TWO MICROSTRUCTURES: exchanges with market makers and OTC-markets without market makers. Using laboratory experiments we test whether the impact of a Tobin tax is different in these two market microstructures. We find that (i) in markets without market makers an unilaterally imposed Tobin tax (i.e. a tax haven exists) increases volatility. (ii) In contrast, in markets with market makers we observe a decrease in volatility in unilaterally taxed markets. (iii) An encompassing Tobin tax has no impact on volatility in either setting. Efficiency does not vary significantly across tax regimes.

Consensus-based guidance for conducting and reporting multi-analyst studies.

Any large dataset can be analyzed in a number of ways, and it is possible that the use of different analysis strategies will lead to different results and conclusions. One way to assess whether the results obtained depend on the analysis strategy chosen is to employ multiple analysts and leave each of them free to follow their own approach. Here, we present consensus-based guidance for conducting and reporting such multi-analyst studies, and we discuss how broader adoption of the multi-analyst approach has the potential to strengthen the robustness of results and conclusions obtained from analyses of datasets in basic and applied research.

Where to look for the morals in markets?

There is a heated debate on whether markets erode social responsibility and moral behavior. However, it is a challenging task to identify and measure moral behavior in markets. Based on a theoretical model, we examine in an experiment the relation between trading volume, prices and moral behavior by setting up markets that either impose a negative externality on third parties or not. We find that moral behavior reveals itself in lower trading volume in markets with a negative externality, while prices mostly depend on the market structure. We further investigate individual characteristics that explain trading behavior in markets with negative externalities.

Predicting the replicability of social science lab experiments.

We measure how accurately replication of experimental results can be predicted by black-box statistical models. With data from four large-scale replication projects in experimental psychology and economics, and techniques from machine learning, we train predictive models and study which variables drive predictable replication. The models predicts binary replication with a cross-validated accuracy rate of 70% (AUC of 0.77) and estimates of relative effect sizes with a Spearman ρ of 0.38. The accuracy level is similar to market-aggregated beliefs of peer scientists [1, 2]. The predictive power is validated in a pre-registered out of sample test of the outcome of [3], where 71% (AUC of 0.73) of replications are predicted correctly and effect size correlations amount to ρ = 0.25. Basic features such as the sample and effect sizes in original papers, and whether reported effects are single-variable main effects or two-variable interactions, are predictive of successful replication. The models presented in this paper are simple tools to produce cheap, prognostic replicability metrics. These models could be useful in institutionalizing the process of evaluation of new findings and guiding resources to those direct replications that are likely to be most informative.

fMRI data of mixed gambles from the Neuroimaging Analysis Replication and Prediction Study.

There is an ongoing debate about the replicability of neuroimaging research. It was suggested that one of the main reasons for the high rate of false positive results is the many degrees of freedom researchers have during data analysis. In the Neuroimaging Analysis Replication and Prediction Study (NARPS), we aim to provide the first scientific evidence on the variability of results across analysis teams in neuroscience. We collected fMRI data from 108 participants during two versions of the mixed gambles task, which is often used to study decision-making under risk. For each participant, the dataset includes an anatomical (T1 weighted) scan and fMRI as well as behavioral data from four runs of the task. The dataset is shared through OpenNeuro and is formatted according to the Brain Imaging Data Structure (BIDS) standard. Data pre-processed with fMRIprep and quality control reports are also publicly shared. This dataset can be used to study decision-making under risk and to test replicability and interpretability of previous results in the field.

Immaterial and monetary gifts in economic transactions: evidence from the field.

Reciprocation of monetary gifts is well-understood in economics. In contrast, there is little research on reciprocal behavior following immaterial gifts like compliments. We narrow this gap and investigate how employees reciprocate after receiving immaterial gifts and material gifts over time. We purchase (1) ice cream from fast food restaurants, and (2) durum doner, a common lunch snack, from independent vendors. Prior to the food's preparation, we either compliment or tip the salesperson. We find that salespersons reciprocate compliments with higher product weight than in a control treatment. Importantly, this reciprocal behavior following immaterial gifts grows over repeated transactions. Tips, in contrast, have a stronger level effect which does not change over time.

Ethnical discrimination in Europe: Field evidence from the finance industry.

The integration of ethnical minorities has been a hotly discussed topic in the political, societal, and economic debate. Persistent discrimination of ethnical minorities can hinder successful integration. Given that unequal access to investment and financing opportunities can cause social and economic disparities due to inferior economic prospects, we conducted a field experiment on ethnical discrimination in the finance sector with 1,218 banks in seven European countries. We contacted banks via e-mail, either with domestic or Arabic sounding names, asking for contact details only. We find pronounced discrimination in terms of a substantially lower response rate to e-mails from Arabic senders. Remarkably, the observed discrimination effect is robust for loan- and investment-related requests, across rural and urban locations of banks, and across countries.

Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015.

Being able to replicate scientific findings is crucial for scientific progress1-15. We replicate 21 systematically selected experimental studies in the social sciences published in Nature and Science between 2010 and 201516-36. The replications follow analysis plans reviewed by the original authors and pre-registered prior to the replications. The replications are high powered, with sample sizes on average about five times higher than in the original studies. We find a significant effect in the same direction as the original study for 13 (62%) studies, and the effect size of the replications is on average about 50% of the original effect size. Replicability varies between 12 (57%) and 14 (67%) studies for complementary replicability indicators. Consistent with these results, the estimated true-positive rate is 67% in a Bayesian analysis. The relative effect size of true positives is estimated to be 71%, suggesting that both false positives and inflated effect sizes of true positives contribute to imperfect reproducibility. Furthermore, we find that peer beliefs of replicability are strongly related to replicability, suggesting that the research community could predict which results would replicate and that failures to replicate were not the result of chance alone.

Market versus Residence Principle: Experimental Evidence on the Effects of a Financial Transaction Tax.

The effects of a financial transaction tax (FTT) are scientifically disputed, as seemingly small details of its implementation may matter a lot. In this article, we provide experimental evidence on the different effects of an FTT, depending on whether it is implemented as a tax on markets, on residents, or a combination of both. We find that a tax on markets has negative effects on volatility and trading volume, whereas a tax on residents shows none of these undesired effects. Additionally, we observe that individual risk attitude is not related to traders' reaction to the different forms of an FTT.

The effect of fast and slow decisions on risk taking.

We experimentally compare fast and slow decisions in a series of experiments on financial risk taking in three countries involving over 1700 subjects. To manipulate fast and slow decisions, subjects were randomly allocated to responding within 7 seconds (time pressure) or waiting for at least 7 or 20 seconds (time delay) before responding. To control for different effects of time pressure and time delay on measurement noise, we estimate separate parameters for noise and risk preferences within a random utility framework. We find that time pressure increases risk aversion for gains and risk taking for losses compared to time delay, implying that time pressure increases the reflection effect of Prospect Theory. The results for gains are weaker and less robust than the results for losses. We find no significant difference between time pressure and time delay for loss aversion (tested in only one of the experiments). Time delay also leads to less measurement noise than time pressure and unconstrained decisions, and appears to be an effective way of decreasing noise in experiments.

Intuition and Moral Decision-Making - The Effect of Time Pressure and Cognitive Load on Moral Judgment and Altruistic Behavior.

Do individuals intuitively favor certain moral actions over others? This study explores the role of intuitive thinking-induced by time pressure and cognitive load-in moral judgment and behavior. We conduct experiments in three different countries (Sweden, Austria, and the United States) involving over 1,400 subjects. All subjects responded to four trolley type dilemmas and four dictator games involving different charitable causes. Decisions were made under time pressure/time delay or while experiencing cognitive load or control. Overall we find converging evidence that intuitive states do not influence moral decisions. Neither time-pressure nor cognitive load had any effect on moral judgments or altruistic behavior. Thus we find no supporting evidence for the claim that intuitive moral judgments and dictator game giving differ from more reflectively taken decisions. Across all samples and decision tasks men were more likely to make utilitarian moral judgments and act selfishly compared to women, providing further evidence that there are robust gender differences in moral decision-making. However, there were no significant interactions between gender and the treatment manipulations of intuitive versus reflective decision-making.

Evaluating replicability of laboratory experiments in economics.

The replicability of some scientific findings has recently been called into question. To contribute data about replicability in economics, we replicated 18 studies published in the American Economic Review and the Quarterly Journal of Economics between 2011 and 2014. All of these replications followed predefined analysis plans that were made publicly available beforehand, and they all have a statistical power of at least 90% to detect the original effect size at the 5% significance level. We found a significant effect in the same direction as in the original study for 11 replications (61%); on average, the replicated effect size is 66% of the original. The replicability rate varies between 67% and 78% for four additional replicability indicators, including a prediction market measure of peer beliefs.