Heterogeneity in effect size estimates.

A typical empirical study involves choosing a sample, a research design, and an analysis path. Variation in such choices across studies leads to heterogeneity in results that introduce an additional layer of uncertainty, limiting the generalizability of published scientific findings. We provide a framework for studying heterogeneity in the social sciences and divide heterogeneity into population, design, and analytical heterogeneity. Our framework suggests that after accounting for heterogeneity, the probability that the tested hypothesis is true for the average population, design, and analysis path can be much lower than implied by nominal error rates of statistically significant individual studies. We estimate each type's heterogeneity from 70 multilab replication studies, 11 prospective meta-analyses of studies employing different experimental designs, and 5 multianalyst studies. In our data, population heterogeneity tends to be relatively small, whereas design and analytical heterogeneity are large. Our results should, however, be interpreted cautiously due to the limited number of studies and the large uncertainty in the heterogeneity estimates. We discuss several ways to parse and account for heterogeneity in the context of different methodologies.

Competition and moral behavior: A meta-analysis of forty-five crowd-sourced experimental designs.

Does competition affect moral behavior? This fundamental question has been debated among leading scholars for centuries, and more recently, it has been tested in experimental studies yielding a body of rather inconclusive empirical evidence. A potential source of ambivalent empirical results on the same hypothesis is design heterogeneity-variation in true effect sizes across various reasonable experimental research protocols. To provide further evidence on whether competition affects moral behavior and to examine whether the generalizability of a single experimental study is jeopardized by design heterogeneity, we invited independent research teams to contribute experimental designs to a crowd-sourced project. In a large-scale online data collection, 18,123 experimental participants were randomly allocated to 45 randomly selected experimental designs out of 95 submitted designs. We find a small adverse effect of competition on moral behavior in a meta-analysis of the pooled data. The crowd-sourced design of our study allows for a clean identification and estimation of the variation in effect sizes above and beyond what could be expected due to sampling variance. We find substantial design heterogeneity-estimated to be about 1.6 times as large as the average standard error of effect size estimates of the 45 research designs-indicating that the informativeness and generalizability of results based on a single experimental design are limited. Drawing strong conclusions about the underlying hypotheses in the presence of substantive design heterogeneity requires moving toward much larger data collections on various experimental designs testing the same hypothesis.

Nobel and novice: Author prominence affects peer review.

Peer review is a well-established cornerstone of the scientific process, yet it is not immune to biases like status bias, which we explore in this paper. Merton described this bias as prominent researchers getting disproportionately great credit for their contribution, while relatively unknown researchers get disproportionately little credit [R. K. Merton, Science 159, 56-63 (1968)]. We measured the extent of this bias in the peer-review process through a preregistered field experiment. We invited more than 3,300 researchers to review a finance research paper jointly written by a prominent author (a Nobel laureate) and by a relatively unknown author (an early career research associate), varying whether reviewers saw the prominent author's name, an anonymized version of the paper, or the less-well-known author's name. We found strong evidence for the status bias: More of the invited researchers accepted to review the paper when the prominent name was shown, and while only 23% recommended "reject" when the prominent researcher was the only author shown, 48% did so when the paper was anonymized, and 65% did when the little-known author was the only author shown. Our findings complement and extend earlier results on double-anonymized vs. single-anonymized review [R. Blank, Am. Econ. Rev. 81, 1041-1067 (1991); M. A. Ucci, F. D'Antonio, V. Berghella, Am. J. Obstet. Gynecol. MFM 4, 100645 (2022)].

Market shocks and professionals' investment behavior - Evidence from the COVID-19 crash.

We investigate how the experience of extreme events, such as the COVID-19 market crash, influence risk-taking behavior. To isolate changes in risk-taking from other factors, we ran controlled experiments with finance professionals in December 2019 and March 2020. We observe that their investments in the experiment were 12 percent lower in March 2020 than in December 2019, although their price expectations had not changed, and although they considered the experimental asset less risky during the crash than before. This lower perceived risk is likely due to adaptive normalization, as volatility during the shock is compared to volatility experienced in real markets (which was low in December 2019, but very high in March 2020). Lower investments during the crash can be supported by higher risk aversion, not by changes in beliefs.

Quadriceps tendon vs. patellar tendon autograft for ACL reconstruction using a hardware-free press-fit fixation technique: comparable stability, function and return-to-sport level but less donor site morbidity in athletes after 10 years.

INTRODUCTION: The use of quadriceps tendon-patellar bone (QTB) autograft for anterior cruciate ligament (ACL) reconstruction is gaining momentum. Yet, long-term results that compare this procedure with established methods are lacking. The aim of this study was to report and compare long-term results of ACL reconstruction using QTB autografts versus bone-patellar tendon-bone (BPTB) autografts, both anchored using a hardware-free press-fit fixation technique. MATERIALS AND METHODS: 60 athletes (Tegner score ≥6) with primary ACL rupture were prospectively randomized into two groups. 56 patients were evaluated after a mean duration of 12.2 ± 1.9 months (range 10-14) and 43 patients after 10.3 ± 0.2 years (range 10-11). RESULTS: On final follow-up, 90% of patients scored very good and good results in the functional Lysholm score (mean 99 ± 7.1, range 74-100 points). Normal or almost normal IKDC score was reported by 84% of the patients (mean 97 ± 9.5, range 60-100 points). The activity level decreased in the Tegner score from median of 7 before injury to 6 after 10 years. The KT-1000 arthrometer showed a difference in the anterior translation of less than 3 mm (mean 1.0 ± 1.2, range - 1 to 5 mm) in 91% of the patients. Significant degeneration was radiologically detected in one patient per group. No tunnel widening was seen in any patient. Up to 97% of all patients were satisfied with the operative procedure. No significant differences were found in the mentioned parameters between the two groups and also in comparison with the 1-year results. The only significant difference was in the donor site morbidity. Significantly more patients in the BPTB group had complaints during kneeling both at 1 (p < 0.001) and 10 years (p = 0.019). Squatting was also subjectively more problematic in the BPTB group than in the QTB group both after 1 (p = 0.003) and 10 years (p = 0.046). CONCLUSIONS: This study shows equally good functional, clinical and radiological long-term results for both hardware-free methods of ACL reconstruction. These results clinically confirm the safety of press-fit anchoring after 10 years. The failure rate in this study was very low, with only one re-rupture in 10 years. The increased donor site morbidity when using the BPTB autograft compared to the QTB autograft supports already reported data. It was also seen in this study for the implant-free press-fit techniques. STUDY DESIGN: Prospective and randomized, level of evidence 2.

Revision ACL reconstruction using quadriceps or hamstring autografts leads to similar results after 4 years: good objective stability but low rate of return to pre-injury sport level.

PURPOSE: Due to the increased importance of revision ACL reconstruction, this study aims to evaluate the outcome 4 years after the surgery, compare two revision strategies and identify factors that influence the results. METHODS: Seventy-nine patients who received a revision ACL reconstruction were retrospectively evaluated. All patients were assessed with an average follow-up of 4.4 years (range 3.3-5.5 years). The results of patients treated with a quadriceps autograft were compared with those treated with a hamstring autograft. RESULTS: Ninety-seven percent of patients had a KT-1000 side-to-side difference of ≤ 5 mm (mean 1.7 ± 2.0 mm). Pivot-shift test was absent or minor in 95%. In the SLTH-test, 70% of patients reached 90% of the contralateral side. The mean Lysholm score on follow-up was 83 ± 12 (56% excellent/good). The mean IKDC 2000 subjective evaluation score was 81 ± 14 (58% normal/almost normal). The median Tegner activity score was 6 (range 3-10), a median of 2 levels worse than before the first injury. Return to sport rate was 89% but only 34% of patients reached their pre-injury sport level. Most common cause for this reduction was fear of another injury. Three patients suffered a re-rupture. Patients with a hamstring autograft performed pivoting sports more often, but had worse pivot-shift results compared to those with a quadriceps autograft. No significant influence was seen for other parameters. Young, male patients with a high activity level and no chondral damage had the best results. CONCLUSION: Through revision ACL reconstruction, the goal of stabilizing the knee can be achieved in the majority of patients. However, a good function and a high activity level are significantly less common in these patients. The main reason for this is fear of a renewed ACL-injury. Both quadriceps and hamstring autografts were able to achieve a good outcome. Young, male, patients with a normal BMI, a high activity level and without cartilage damage seem to benefit the most from revision ACL surgery. The discrepancy between the good laxity restoration and the lower activity rate should therefore be a main point in clinical counseling when deciding for or against revision ACL-Reconstruction. LEVEL OF EVIDENCE: III.

Long-term follow-up after anterior cruciate ligament reconstruction using a press-fit quadriceps tendon-patellar bone autograft.

BACKGROUND: The use of a quadriceps tendon-patellar bone (QTPB) autograft provides an alternative procedure in primary reconstruction of the anterior cruciate ligament (ACL). Using the press-fit technique for femoral fixation and knotting over a bone bridge as well as additional spongiosa filling for tibial fixation can completely eliminate the need for fixation implants. The objective of this study was to evaluate the long-term clinical, functional and radiological results of this operating method. METHODS: Sixty-nine patients (27 female-42 male) were included in this study. Fifty-seven patients (83%) received a comprehensive follow-up review after an average period of 7.5 years (range: 7-8.7). All other patients were surveyed by telephone. Six patients (9%) suffered a re-rupture of the ACL graft caused by a new related trauma and were therefore excluded from the statistical analysis. RESULTS: Of all patients, 98% were satisfied with the operation. Normal or almost normal results were recorded in the subjective IKDC scores form by 88% of the patients. The Lysholm score demonstrated very good and good results in 83% of the patients. Only 1 patient reported minor complaints in the donor area. Seven (12%) patients developed Cyclops syndrome with limited knee extension. This complication was treated arthroscopically within the first year postoperatively. Their results on follow-up were not worse than the results of the patients without Cyclops syndrome. Regarding the 57 patients who received a comprehensive evaluation, the stability test with the KT-1000 Arthrometer yielded a difference of less than 3 mm in the contralateral comparison for 89% of the operated knees. The pivot-shift test was normal in 79% and almost normal in 21%. In the Single-leg Triple Hop Test, patients achieved an average of 98% of the hopping distance attained with the contralateral leg. The radiological examination revealed a slight deterioration in the Kellgren-Lawrence Score in 2 patients. CONCLUSION: The ACL reconstruction using the QTPB autograft performed with the press-fit technique leads to good results in comparison with published results of established procedures for primary ACL surgery using other autografts. Further investigations should involve comparative studies with the objective of providing evidence-based, individually adapted therapy for ACL rupture.

Subjective evidence evaluation survey for many-analysts studies.

Many-analysts studies explore how well an empirical claim withstands plausible alternative analyses of the same dataset by multiple, independent analysis teams. Conclusions from these studies typically rely on a single outcome metric (e.g. effect size) provided by each analysis team. Although informative about the range of plausible effects in a dataset, a single effect size from each team does not provide a complete, nuanced understanding of how analysis choices are related to the outcome. We used the Delphi consensus technique with input from 37 experts to develop an 18-item subjective evidence evaluation survey (SEES) to evaluate how each analysis team views the methodological appropriateness of the research design and the strength of evidence for the hypothesis. We illustrate the usefulness of the SEES in providing richer evidence assessment with pilot data from a previous many-analysts study.

Short-selling constraints as cause for price distortions: An experimental study.

In this paper we explore the influence of the possibility to short stocks and/or borrow money in laboratory markets. A key innovation of our study is that subjects can simultaneously trade two risky assets on two double-auction markets, allowing us to differentiate between assets with relatively high versus low capitalization. Divergence of opinions is created by providing each trader with noisy information on the intrinsic values of both assets. We find that when borrowing money or shorting stocks is restricted prices are systematically distorted. Specifically, stocks with high (low) capitalization are traded at lower (higher) prices than their fundamental value. Lifting the restrictions leads to more efficient prices and more liquidity, thereby also lowering volatility and bid-ask spreads.

The impact of a financial transaction tax on stylized facts of price returns-Evidence from the lab.

As the introduction of financial transaction taxes is increasingly discussed by political leaders we explore possible consequences such taxes could have on markets. Here we examine how "stylized facts", namely fat tails and volatility clustering, are affected by different tax regimes in laboratory experiments. We find that leptokurtosis of price returns is highest and clustered volatility is weakest in unilaterally taxed markets (where tax havens exist). Instead, tails are slimmest and volatility clustering is strongest in tax havens. When an encompassing financial transaction tax is levied, stylized facts hardly change compared to a scenario with no tax on all markets.

Acceptance or rejection of welfare migration-an experimental investigation.

We study decisions on welcoming or opposing welfare migration in a laboratory setting with two societies in which one subject can migrate from the poorer to the richer society, provided a majority in the richer society votes to allow that. In each society, subjects indicate their preference for a percentage contribution to a public pool. The median of these rates sets the contributions paid by everybody; a feature that results in high contribution rates with an average of 90%. Varying the multiplier with which contributions are magnified before redistribution to society members, and thus the expected gain/loss associated with migration, we find that subjects overwhelmingly welcome migrants if they expect an economic benefit, while most participants oppose migration if they would be negatively affected by it. Regarding participants' attitudes, we find that more altruistic people are more in favor of migration than more selfish people and that center right-wing oriented subjects propose lower contribution levels than center left-wing oriented subjects. We conclude that economic motives are a crucial factor for accepting or rejecting welfare migration. Therefore, a key to promoting acceptance of new migrants is to ensure and then communicate that their net effect on growth, society, and the public purse is positive. Supplementary Information: The online version contains supplementary material available at 10.1007/s43546-022-00356-6.

Unplanned reoperation and reintervention after pancreatic resections: an analysis of risk factors.

BACKGROUND: The purpose of the study was to determine the incidence of any unplanned reoperation or reintervention procedure after pancreatic resection and to identify the underlying risk factors. METHODS: A total of 189 consecutive pancreatic resections performed from 2001-2008 were searched for any unplanned reoperation, percutaneous drainage, or angiographic reintervention. A retrospective analysis of a prospectively maintained database, including patient characteristics, comorbidities, details of surgery, specific complications, incidence of reoperation/reintervention, and mortality was performed. RESULTS: Overall rates of reoperation, reintervention, and mortality were 6.3% (12/189), 7.9% (15/189), and 1.6% (3/189), respectively. Four patients underwent reintervention and reoperation, so the combined reoperation/reintervention rate was 12.2% (23/189). Reoperation (P < 0.001) and reintervention (P = 0.002) correlated with mortality. Hemorrhage (relative risk [RR], 58; P = 0.0017) and the combination of hemorrhage and pancreatic fistula (RR, 117; P < 0.0001) were identified as risk factors for unplanned reoperation, hemorrhage (RR, 82; P = 0.005), pancreatic fistula (RR, 42; P < 0.001), and the combination of both complications (RR, 246; P < 0.001) for reoperation and/or reintervention. Other patient- or procedure-related factors did not influence the reoperation and/or reintervention rates significantly. CONCLUSIONS: Pancreatic fistula and hemorrhage are the predominant factors that afford unplanned reoperation/reintervention. Although reporting the incidence of unplanned reoperation will include the most severe postoperative complications, a considerable number of reinterventions are missed. Therefore, in outcome analyses of pancreatic surgery, not only reoperations but also any interventional therapies should be included.

Market microstructure matters when imposing a Tobin tax-Evidence from the lab.

TRADING IN FX MARKETS IS DOMINATED BY TWO MICROSTRUCTURES: exchanges with market makers and OTC-markets without market makers. Using laboratory experiments we test whether the impact of a Tobin tax is different in these two market microstructures. We find that (i) in markets without market makers an unilaterally imposed Tobin tax (i.e. a tax haven exists) increases volatility. (ii) In contrast, in markets with market makers we observe a decrease in volatility in unilaterally taxed markets. (iii) An encompassing Tobin tax has no impact on volatility in either setting. Efficiency does not vary significantly across tax regimes.

Myc suppression of Nfkb2 accelerates lymphomagenesis.

BACKGROUND: Deregulated c-Myc expression is a hallmark of several human cancers where it promotes proliferation and an aggressive tumour phenotype. Myc overexpression is associated with reduced activity of Rel/NF-kappaB, transcription factors that control the immune response, cell survival, and transformation, and that are frequently altered in cancer. The Rel/NF-kappaB family member NFKB2 is altered by chromosomal translocations or deletions in lymphoid malignancies and deletion of the C-terminal ankyrin domain of NF-kappaB2 augments lymphocyte proliferation. METHODS: Precancerous Emicro-Myc-transgenic B cells, Emicro-Myc lymphomas and human Burkitt lymphoma samples were assessed for Nfkb2 expression. The contribution of Nfkb2 to Myc-driven apoptosis, proliferation, and lymphomagenesis was tested genetically in vivo. RESULTS: Here we report that the Myc oncoprotein suppresses Nfkb2 expression in vitro in primary mouse fibroblasts and B cells, and in vivo in the Emicro-Myc transgenic mouse model of human Burkitt lymphoma (BL). NFKB2 suppression by Myc was also confirmed in primary human BL. Promoter-reporter assays indicate that Myc-mediated suppression of Nfkb2 occurs at the level of transcription. The contribution of Nfkb2 to Myc-driven lymphomagenesis was tested in vivo, where Nfkb2 loss was shown to accelerate lymphoma development in Emicro-Myc transgenic mice, by impairing Myc's apoptotic response. CONCLUSIONS: Nfkb2 is suppressed by c-Myc and harnesses Myc-driven lymphomagenesis. These data thus link Myc-driven lymphomagenesis to the non-canonical NF-kappaB pathway.

Aggregation mechanisms for crowd predictions.

When the information of many individuals is pooled, the resulting aggregate often is a good predictor of unknown quantities or facts. This aggregate predictor frequently outperforms the forecasts of experts or even the best individual forecast included in the aggregation process ("wisdom of crowds"). However, an appropriate aggregation mechanism is considered crucial to reaping the benefits of a "wise crowd". Of the many possible ways to aggregate individual forecasts, we compare (uncensored and censored) arithmetic and geometric mean and median, continuous double auction market prices and sealed bid-offer call market prices in a controlled experiment. We use an asymmetric information structure, where participants know different sub-sets of the total information needed to exactly calculate the asset value to be estimated. We find that prices from continuous double auction markets clearly outperform all alternative approaches for aggregating dispersed information and that information lets only the best-informed participants generate excess returns.

Where to look for the morals in markets?

There is a heated debate on whether markets erode social responsibility and moral behavior. However, it is a challenging task to identify and measure moral behavior in markets. Based on a theoretical model, we examine in an experiment the relation between trading volume, prices and moral behavior by setting up markets that either impose a negative externality on third parties or not. We find that moral behavior reveals itself in lower trading volume in markets with a negative externality, while prices mostly depend on the market structure. We further investigate individual characteristics that explain trading behavior in markets with negative externalities.

Scale matters: risk perception, return expectations, and investment propensity under different scalings.

With a novel experimental design we investigate whether risk perception, return expectations, and investment propensity are influenced by the scale of the vertical axis in charts. We explore this for two presentation formats, namely return charts and price charts, where we depict low- and high-volatility assets with distinct trends. We find that varying the scale strongly affects people's risk perception, as a narrower scale of the vertical axis leads to significantly higher perceived riskiness of an asset even if the underlying volatility is the same. Furthermore, past returns predict future return expectations almost perfectly. In our setting perceived profitability was considered more important than perceived riskiness when making investment choices. Overall we show that adapting the scale of a chart makes it easier to recognize yearly return variations within a single security, but at the same time makes it harder to identify differences between dissimilar securities. This is something regulators should be aware of and take into account in the rules they set.

fMRI data of mixed gambles from the Neuroimaging Analysis Replication and Prediction Study.

There is an ongoing debate about the replicability of neuroimaging research. It was suggested that one of the main reasons for the high rate of false positive results is the many degrees of freedom researchers have during data analysis. In the Neuroimaging Analysis Replication and Prediction Study (NARPS), we aim to provide the first scientific evidence on the variability of results across analysis teams in neuroscience. We collected fMRI data from 108 participants during two versions of the mixed gambles task, which is often used to study decision-making under risk. For each participant, the dataset includes an anatomical (T1 weighted) scan and fMRI as well as behavioral data from four runs of the task. The dataset is shared through OpenNeuro and is formatted according to the Brain Imaging Data Structure (BIDS) standard. Data pre-processed with fMRIprep and quality control reports are also publicly shared. This dataset can be used to study decision-making under risk and to test replicability and interpretability of previous results in the field.

Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015.

Being able to replicate scientific findings is crucial for scientific progress1-15. We replicate 21 systematically selected experimental studies in the social sciences published in Nature and Science between 2010 and 201516-36. The replications follow analysis plans reviewed by the original authors and pre-registered prior to the replications. The replications are high powered, with sample sizes on average about five times higher than in the original studies. We find a significant effect in the same direction as the original study for 13 (62%) studies, and the effect size of the replications is on average about 50% of the original effect size. Replicability varies between 12 (57%) and 14 (67%) studies for complementary replicability indicators. Consistent with these results, the estimated true-positive rate is 67% in a Bayesian analysis. The relative effect size of true positives is estimated to be 71%, suggesting that both false positives and inflated effect sizes of true positives contribute to imperfect reproducibility. Furthermore, we find that peer beliefs of replicability are strongly related to replicability, suggesting that the research community could predict which results would replicate and that failures to replicate were not the result of chance alone.

Market versus Residence Principle: Experimental Evidence on the Effects of a Financial Transaction Tax.

The effects of a financial transaction tax (FTT) are scientifically disputed, as seemingly small details of its implementation may matter a lot. In this article, we provide experimental evidence on the different effects of an FTT, depending on whether it is implemented as a tax on markets, on residents, or a combination of both. We find that a tax on markets has negative effects on volatility and trading volume, whereas a tax on residents shows none of these undesired effects. Additionally, we observe that individual risk attitude is not related to traders' reaction to the different forms of an FTT.