Promotional language and the adoption of innovative ideas in science.

How are the merits of innovative ideas communicated in science? Here, we conduct semantic analyses of grant application success with a focus on scientific promotional language, which may help to convey an innovative idea's originality and significance. Our analysis attempts to surmount the limitations of prior grant studies by examining the full text of tens of thousands of both funded and unfunded grants from three leading public and private funding agencies: the NIH, the NSF, and the Novo Nordisk Foundation, one of the world's largest private science funding foundations. We find a robust association between promotional language and the support and adoption of innovative ideas by funders and other scientists. First, a grant proposal's percentage of promotional language is associated with up to a doubling of the grant's probability of being funded. Second, a grant's promotional language reflects its intrinsic innovativeness. Third, the percentage of promotional language is predictive of the expected citation and productivity impact of publications that are supported by funded grants. Finally, a computer-assisted experiment that manipulates the promotional language in our data demonstrates how promotional language can communicate the merit of ideas through cognitive activation. With the incidence of promotional language in science steeply rising, and the pivotal role of grants in converting promising and aspirational ideas into solutions, our analysis provides empirical evidence that promotional language is associated with effectively communicating the merits of innovative scientific ideas.

Data, measurement and empirical methods in the science of science.

The advent of large-scale datasets that trace the workings of science has encouraged researchers from many different disciplinary backgrounds to turn scientific methods into science itself, cultivating a rapidly expanding 'science of science'. This Review considers this growing, multidisciplinary literature through the lens of data, measurement and empirical methods. We discuss the purposes, strengths and limitations of major empirical approaches, seeking to increase understanding of the field's diverse methodologies and expand researchers' toolkits. Overall, new empirical developments provide enormous capacity to test traditional beliefs and conceptual frameworks about science, discover factors associated with scientific productivity, predict scientific outcomes and design policies that facilitate scientific progress.

A discipline-wide investigation of the replicability of Psychology papers over the past two decades.

Conjecture about the weak replicability in social sciences has made scholars eager to quantify the scale and scope of replication failure for a discipline. Yet small-scale manual replication methods alone are ill-suited to deal with this big data problem. Here, we conduct a discipline-wide replication census in science. Our sample (N = 14,126 papers) covers nearly all papers published in the six top-tier Psychology journals over the past 20 y. Using a validated machine learning model that estimates a paper's likelihood of replication, we found evidence that both supports and refutes speculations drawn from a relatively small sample of manual replications. First, we find that a single overall replication rate of Psychology poorly captures the varying degree of replicability among subfields. Second, we find that replication rates are strongly correlated with research methods in all subfields. Experiments replicate at a significantly lower rate than do non-experimental studies. Third, we find that authors' cumulative publication number and citation impact are positively related to the likelihood of replication, while other proxies of research quality and rigor, such as an author's university prestige and a paper's citations, are unrelated to replicability. Finally, contrary to the ideal that media attention should cover replicable research, we find that media attention is positively related to the likelihood of replication failure. Our assessments of the scale and scope of replicability are important next steps toward broadly resolving issues of replicability.

Weak ties, failed tries, and success.

A large-scale study provides a causal test for a cornerstone of social science.

Gender-diverse teams produce more novel and higher-impact scientific ideas.

Science's changing demographics raise new questions about research team diversity and research outcomes. We study mixed-gender research teams, examining 6.6 million papers published across the medical sciences since 2000 and establishing several core findings. First, the fraction of publications by mixed-gender teams has grown rapidly, yet mixed-gender teams continue to be underrepresented compared to the expectations of a null model. Second, despite their underrepresentation, the publications of mixed-gender teams are substantially more novel and impactful than the publications of same-gender teams of equivalent size. Third, the greater the gender balance on a team, the better the team scores on these performance measures. Fourth, these patterns generalize across medical subfields. Finally, the novelty and impact advantages seen with mixed-gender teams persist when considering numerous controls and potential related features, including fixed effects for the individual researchers, team structures, and network positioning, suggesting that a team's gender balance is an underrecognized yet powerful correlate of novel and impactful scientific discoveries.

Quantifying the selective forgetting and integration of ideas in science and technology.

How long will this article be remembered? How long will people reference it in their conversations, and for how many years will other authors cite its findings in their own works? A community's attention to a cultural object decays as time passes, a process known as collective forgetting. Recent work models this decay as the result of two different processes. One linked to communicative memory-memories sustained by human communication-and the other linked to cultural memory-memories sustained by the physical recording of content. Collective forgetting has significant impacts on communities, yet little is known about how the collective forgetting dynamic changes over time. Here, we study the temporal changes of collective memory and attention by focusing on two knowledge communities: inventors and physicists. We use data on patents from the United States Patent and Trademark Office (USPTO) and physics papers published by the American Physical Society (APS) to quantify those changes over time. The model enables us to distinguish between two branches of forgetting. One branch is short-lived, going directly from communicative memory to oblivion. The other branch is long-lived, going from communicative memory to cultural memory before going on to oblivion. The data analysis shows an increase in the forgetting rate for both communities as the amount of information in each of them grows. That growth of information forces knowledge communities to increase their selectivity regarding what is stored in their cultural memory. These findings confirm the forgetting as annulment hypothesis and show that knowledge communities can slow down collective forgetting and improve selectivity processes. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Scientific prizes and the extraordinary growth of scientific topics.

Fast growing scientific topics have famously been key harbingers of the new frontiers of science, yet, large-scale analyses of their genesis and impact are rare. We investigated one possible factor connected with a topic's extraordinary growth: scientific prizes. Our longitudinal analysis of nearly all recognized prizes worldwide and over 11,000 scientific topics from 19 disciplines indicates that topics associated with a scientific prize experience extraordinary growth in productivity, impact, and new entrants. Relative to matched non-prizewinning topics, prizewinning topics produce 40% more papers and 33% more citations, retain 55% more scientists, and gain 37 and 47% more new entrants and star scientists, respectively, in the first five-to-ten years after the prize. Funding do not account for a prizewinning topic's growth. Rather, growth is positively related to the degree to which the prize is discipline-specific, conferred for recent research, or has prize money. These findings reveal new dynamics behind scientific innovation and investment.

Importance of scientific collaboration in contemporary drug discovery and development: a detailed network analysis.

BACKGROUND: Growing evidence shows that scientific collaboration plays a crucial role in transformative innovation in the life sciences. For example, contemporary drug discovery and development reflects the work of teams of individuals from academic centers, the pharmaceutical industry, the regulatory science community, health care providers, and patients. However, public understanding of how collaborations between academia and industry catalyze novel target identification and first-in-class drug discovery is limited. RESULTS: We perform a comprehensive network analysis on a large scientific corpus of collaboration and citations (97,688 papers with 1,862,500 citations from 170 million scientific records) to quantify the success trajectory of innovative drug development. By focusing on four types of cardiovascular drugs, we demonstrate how knowledge flows between institutions to highlight the underlying contributions of many different institutions in the development of a new drug. We highlight how such network analysis could help to increase industrial and governmental support, and improve the efficiency or accelerate decision-making in drug discovery and development. CONCLUSION: We demonstrate that network analysis of large public databases can identify and quantify investigator and institutional relationships in drug discovery and development. If broadly applied, this type of network analysis may help to enhance public understanding of and support for biomedical research, and could identify factors that facilitate decision-making in first-in-class drug discovery among academia, the pharmaceutical industry, and healthcare systems.

Mentorship and protégé success in STEM fields.

Einstein believed that mentors are especially influential in a protégé's intellectual development, yet the link between mentorship and protégé success remains a mystery. We marshaled genealogical data on nearly 40,000 scientists who published 1,167,518 papers in biomedicine, chemistry, math, or physics between 1960 and 2017 to investigate the relationship between mentorship and protégé achievement. In our data, we find groupings of mentors with similar records and reputations who attracted protégés of similar talents and expected levels of professional success. However, each grouping has an exception: One mentor has an additional hidden capability that can be mentored to their protégés. They display skill in creating and communicating prizewinning research. Because the mentor's ability for creating and communicating celebrated research existed before the prize's conferment, protégés of future prizewinning mentors can be uniquely exposed to mentorship for conducting celebrated research. Our models explain 34-44% of the variance in protégé success and reveals three main findings. First, mentorship strongly predicts protégé success across diverse disciplines. Mentorship is associated with a 2×-to-4× rise in a protégé's likelihood of prizewinning, National Academy of Science (NAS) induction, or superstardom relative to matched protégés. Second, mentorship is significantly associated with an increase in the probability of protégés pioneering their own research topics and being midcareer late bloomers. Third, contrary to conventional thought, protégés do not succeed most by following their mentors' research topics but by studying original topics and coauthoring no more than a small fraction of papers with their mentors.

Estimating the deep replicability of scientific findings using human and artificial intelligence.

Replicability tests of scientific papers show that the majority of papers fail replication. Moreover, failed papers circulate through the literature as quickly as replicating papers. This dynamic weakens the literature, raises research costs, and demonstrates the need for new approaches for estimating a study's replicability. Here, we trained an artificial intelligence model to estimate a paper's replicability using ground truth data on studies that had passed or failed manual replication tests, and then tested the model's generalizability on an extensive set of out-of-sample studies. The model predicts replicability better than the base rate of reviewers and comparably as well as prediction markets, the best present-day method for predicting replicability. In out-of-sample tests on manually replicated papers from diverse disciplines and methods, the model had strong accuracy levels of 0.65 to 0.78. Exploring the reasons behind the model's predictions, we found no evidence for bias based on topics, journals, disciplines, base rates of failure, persuasion words, or novelty words like "remarkable" or "unexpected." We did find that the model's accuracy is higher when trained on a paper's text rather than its reported statistics and that n-grams, higher order word combinations that humans have difficulty processing, correlate with replication. We discuss how combining human and machine intelligence can raise confidence in research, provide research self-assessment techniques, and create methods that are scalable and efficient enough to review the ever-growing numbers of publications-a task that entails extensive human resources to accomplish with prediction markets and manual replication alone.

Quantifying the future lethality of terror organizations.

As terror groups proliferate and grow in sophistication, a major international concern is the development of scientific methods that explain and predict insurgent violence. Approaches to estimating a group's future lethality often require data on the group's capabilities and resources, but by the nature of the phenomenon, these data are intentionally concealed by the organizations themselves via encryption, the dark web, back-channel financing, and misinformation. Here, we present a statistical model for estimating a terror group's future lethality using latent-variable modeling techniques to infer a group's intrinsic capabilities and resources for inflicting harm. The analysis introduces 2 explanatory variables that are strong predictors of lethality and raise the overall explained variance when added to existing models. The explanatory variables generate a unique early-warning signal of an individual group's future lethality based on just a few of its first attacks. Relying on the first 10 to 20 attacks or the first 10 to 20% of a group's lifetime behavior, our model explains about 60% of the variance in a group's future lethality as would be explained by a group's complete lifetime data. The model's robustness is evaluated with out-of-sample testing and simulations. The findings' theoretical and pragmatic implications for the science of human conflict are discussed.

Structural balance emerges and explains performance in risky decision-making.

Polarization affects many forms of social organization. A key issue focuses on which affective relationships are prone to change and how their change relates to performance. In this study, we analyze a financial institutional over a two-year period that employed 66 day traders, focusing on links between changes in affective relations and trading performance. Traders' affective relations were inferred from their IMs (>2 million messages) and trading performance was measured from profit and loss statements (>1 million trades). Here, we find that triads of relationships, the building blocks of larger social structures, have a propensity towards affective balance, but one unbalanced configuration resists change. Further, balance is positively related to performance. Traders with balanced networks have the "hot hand", showing streaks of high performance. Research implications focus on how changes in polarization relate to performance and polarized states can depolarize.

Author Correction: Prior shared success predicts victory in team competitions.

In the version of this article initially published, errors occurred in the Acknowledgments.

Prior shared success predicts victory in team competitions.

Debate over the impact of team composition on the outcome of a contest has attracted sports enthusiasts and sports scientists for years. A commonly held belief regarding team success is the superstar effect; that is, including more talent improves the performance of a team1. However, studies of team sports have suggested that previous relations and shared experiences among team members improve the mutual understanding of individual habits, techniques and abilities and therefore enhance team coordination and strategy2-9. We explored the impact of within-team relationships on the outcome of competition between sports teams. Relations among teammates consist of two aspects: qualitative and quantitative. While quantitative aspects measure the number of times two teammates collaborated, qualitative aspects focus on 'prior shared success'; that is, whether teamwork succeeded or failed. We examined the association between qualitative team interactions and the probability of winning using historical records from professional sports-basketball in the National Basketball Association, football in the English Premier League, cricket in the Indian Premier League and baseball in Major League Baseball-and the multiplayer online battle game Defense of the Ancients 2. Our results show that prior shared success between team members significantly improves the odds of the team winning in all sports beyond the talents of individuals.

A network's gender composition and communication pattern predict women's leadership success.

Many leaders today do not rise through the ranks but are recruited directly out of graduate programs into leadership positions. We use a quasi-experiment and instrumental-variable regression to understand the link between students' graduate school social networks and placement into leadership positions of varying levels of authority. Our data measure students' personal characteristics and academic performance, as well as their social network information drawn from 4.5 million email correspondences among hundreds of students who were placed directly into leadership positions. After controlling for students' personal characteristics, work experience, and academic performance, we find that students' social networks strongly predict placement into leadership positions. For males, the higher a male student's centrality in the school-wide network, the higher his leadership-job placement will be. Men with network centrality in the top quartile have an expected job placement level that is 1.5 times greater than men in the bottom quartile of centrality. While centrality also predicts women's placement, high-placing women students have one thing more: an inner circle of predominantly female contacts who are connected to many nonoverlapping third-party contacts. Women with a network centrality in the top quartile and a female-dominated inner circle have an expected job placement level that is 2.5 times greater than women with low centrality and a male-dominated inner circle. Women who have networks that resemble those of high-placing men are low-placing, despite having leadership qualifications comparable to high-placing women.

Scholar Plot: Design and Evaluation of an Information Interface for Faculty Research Performance.

The ability to objectively assess academic performance is critical to rewarding academic merit, charting academic policy, and promoting science. Quintessential to performing these functions is first the ability to collect valid and current data through increasingly automated online interfaces. Moreover, it is crucial to remove disciplinary and other biases from these data, presenting them in ways that support insightful analysis at various levels. Existing systems are lacking in some of these respects. Here we present Scholar Plot (SP), an interface that harvests bibliographic and research funding data from online sources. SP addresses systematic biases in the collected data through nominal and normalized metrics. Eventually, SP combines synergistically these metrics in a plot form for expert appraisal, and an iconic form for broader consumption. SP's plot and iconic forms are scalable, representing equally well individual scholars and their academic units, thus contributing to consistent ranking practices across the university organizational structure. In order to appreciate the design principles underlying SP, in particular the informativeness of nominal vs. normalized metrics, we also present the results of an evaluation survey taken by senior faculty (n = 28) with significant promotion and tenure assessment experience.