A discipline-wide investigation of the replicability of Psychology papers over the past two decades.

Conjecture about the weak replicability in social sciences has made scholars eager to quantify the scale and scope of replication failure for a discipline. Yet small-scale manual replication methods alone are ill-suited to deal with this big data problem. Here, we conduct a discipline-wide replication census in science. Our sample (N = 14,126 papers) covers nearly all papers published in the six top-tier Psychology journals over the past 20 y. Using a validated machine learning model that estimates a paper's likelihood of replication, we found evidence that both supports and refutes speculations drawn from a relatively small sample of manual replications. First, we find that a single overall replication rate of Psychology poorly captures the varying degree of replicability among subfields. Second, we find that replication rates are strongly correlated with research methods in all subfields. Experiments replicate at a significantly lower rate than do non-experimental studies. Third, we find that authors' cumulative publication number and citation impact are positively related to the likelihood of replication, while other proxies of research quality and rigor, such as an author's university prestige and a paper's citations, are unrelated to replicability. Finally, contrary to the ideal that media attention should cover replicable research, we find that media attention is positively related to the likelihood of replication failure. Our assessments of the scale and scope of replicability are important next steps toward broadly resolving issues of replicability.

Authentic self-expression on social media is associated with greater subjective well-being.

Social media users face a tension between presenting themselves in an idealized or authentic way. Here, we explore how prioritizing one over the other impacts users' well-being. We estimate the degree of self-idealized vs. authentic self-expression as the proximity between a user's self-reported personality and the automated personality judgements made on the basis Facebook Likes and status updates. Analyzing data of 10,560 Facebook users, we find that individuals who are more authentic in their self-expression also report greater Life Satisfaction. This effect appears consistent across different personality profiles, countering the proposition that individuals with socially desirable personalities benefit from authentic self-expression more than others. We extend this finding in a pre-registered, longitudinal experiment, demonstrating the causal relationship between authentic posting and positive affect and mood on a within-person level. Our findings suggest that the extent to which social media use is related to well-being depends on how individuals use it.

Estimating the deep replicability of scientific findings using human and artificial intelligence.

Replicability tests of scientific papers show that the majority of papers fail replication. Moreover, failed papers circulate through the literature as quickly as replicating papers. This dynamic weakens the literature, raises research costs, and demonstrates the need for new approaches for estimating a study's replicability. Here, we trained an artificial intelligence model to estimate a paper's replicability using ground truth data on studies that had passed or failed manual replication tests, and then tested the model's generalizability on an extensive set of out-of-sample studies. The model predicts replicability better than the base rate of reviewers and comparably as well as prediction markets, the best present-day method for predicting replicability. In out-of-sample tests on manually replicated papers from diverse disciplines and methods, the model had strong accuracy levels of 0.65 to 0.78. Exploring the reasons behind the model's predictions, we found no evidence for bias based on topics, journals, disciplines, base rates of failure, persuasion words, or novelty words like "remarkable" or "unexpected." We did find that the model's accuracy is higher when trained on a paper's text rather than its reported statistics and that n-grams, higher order word combinations that humans have difficulty processing, correlate with replication. We discuss how combining human and machine intelligence can raise confidence in research, provide research self-assessment techniques, and create methods that are scalable and efficient enough to review the ever-growing numbers of publications-a task that entails extensive human resources to accomplish with prediction markets and manual replication alone.

Birds of a Feather Do Flock Together.

Friends and spouses tend to be similar in a broad range of characteristics, such as age, educational level, race, religion, attitudes, and general intelligence. Surprisingly, little evidence has been found for similarity in personality-one of the most fundamental psychological constructs. We argue that the lack of evidence for personality similarity stems from the tendency of individuals to make personality judgments relative to a salient comparison group, rather than in absolute terms (i.e., the reference-group effect), when responding to the self-report and peer-report questionnaires commonly used in personality research. We employed two behavior-based personality measures to circumvent the reference-group effect. The results based on large samples provide evidence for personality similarity between romantic partners ( n = 1,101; rs = .20-.47) and between friends ( n = 46,483; rs = .12-.31). We discuss the practical and methodological implications of the findings.

Computer-based personality judgments are more accurate than those made by humans.

Judging others' personalities is an essential skill in successful social living, as personality is a key driver behind people's interactions, behaviors, and emotions. Although accurate personality judgments stem from social-cognitive skills, developments in machine learning show that computer models can also make valid judgments. This study compares the accuracy of human and computer-based personality judgments, using a sample of 86,220 volunteers who completed a 100-item personality questionnaire. We show that (i) computer predictions based on a generic digital footprint (Facebook Likes) are more accurate (r = 0.56) than those made by the participants' Facebook friends using a personality questionnaire (r = 0.49); (ii) computer models show higher interjudge agreement; and (iii) computer personality judgments have higher external validity when predicting life outcomes such as substance use, political attitudes, and physical health; for some outcomes, they even outperform the self-rated personality scores. Computers outpacing humans in personality judgment presents significant opportunities and challenges in the areas of psychological assessment, marketing, and privacy.