Optimally generate policy-based evidence before scaling.

Social scientists have increasingly turned to the experimental method to understand human behaviour. One critical issue that makes solving social problems difficult is scaling up the idea from a small group to a larger group in more diverse situations. The urgency of scaling policies impacts us every day, whether it is protecting the health and safety of a community or enhancing the opportunities of future generations. Yet, a common result is that, when we scale up ideas, most experience a 'voltage drop'-that is, on scaling, the cost-benefit profile depreciates considerably. Here I argue that, to reduce voltage drops, we must optimally generate policy-based evidence. Optimality requires answering two crucial questions: what information should be generated and in what sequence. The economics underlying the science of scaling provides insights into these questions, which are in some cases at odds with conventional approaches. For example, there are important situations in which I advocate flipping the traditional social science research model to an approach that, from the beginning, produces the type of policy-based evidence that the science of scaling demands. To do so, I propose augmenting efficacy trials by including relevant tests of scale in the original discovery process, which forces the scientist to naturally start with a recognition of the big picture: what information do I need to have scaling confidence?

Is the Political Slant of Psychology Research Related to Scientific Replicability?

Social science researchers are predominantly liberal, and critics have argued this representation may reduce the robustness of research by embedding liberal values into the research process. In an adversarial collaboration, we examined whether the political slant of research findings in psychology is associated with lower rates of scientific replicability. We analyzed 194 original psychology articles reporting studies that had been subject to a later replication attempt (N = 1,331,413 participants across replications) by having psychology doctoral students (Study 1) and an online sample of U.S. residents (Study 2) from across the political spectrum code the political slant (liberal vs. conservative) of the original research abstracts. The methods and analyses were preregistered. In both studies, the liberal or conservative slant of the original research was not associated with whether the results were successfully replicated. The results remained consistent regardless of the ideology of the coder. Political slant was unrelated to both subsequent citation patterns and the original study's effect size and not consistently related to the original study's sample size. However, we found modest evidence that research with greater political slant-whether liberal or conservative-was less replicable, whereas statistical robustness consistently predicted replication success. We discuss the implications for social science, politics, and replicability.

The new Scopus CiteScore formula and the Journal Impact Factor: a look at top ranking journals and middle ranking journals in the Scopus categories of General Physics and Astronomy, Materials Science, General Medicine and Social Sciences.

In June 2020 Elsevier announced that the CiteScore metric of journals underwent a change. This work examines the effect of these changes for 40 journals, chosen from the top five and middle five (ranked by CiteScore) journals in the subject areas of General Physics and Astronomy, Materials Science, Medicine, Social Sciences) and compares to the Journal Impact Factor. It is shown that in the data studied here, the new methodology is less susceptible to influence of the proportion of editorial material in a journal, but tends to favour journals in research fields that publish articles which get cited more quickly.

Switching cluster membership in cluster randomized control trials: Implications for design and analysis.

Randomized control trials (RCTs) often use clustered designs, where intact clusters (such as classroom, schools, or treatment centers) are randomly assigned to treatment and control conditions. Hierarchical linear models (HLMs) are used almost universally to estimate the effects in such experiments. While study designs that utilize intact clusters have many potential advantages, there is little guidance in the literature on how to respond when cluster switching induces noncompliance with the randomization protocol. In the presence of noncompliance the intent-to-treat (ITT) effect becomes the estimand of interest. When fitting the HLM, these individuals who switch clusters can be assigned to either their as-assigned cluster (the cluster they belonged to at the time of randomization) or their as-treated cluster (the cluster they belonged to at the time the outcome was collected). We show analytically and via simulation, that using the as-treated cluster in HLM will bias the estimate of the ITT effect and using the as-assigned cluster will bias the standard error estimates when heterogeneity among clusters is because of heterogeneity in treatment effects. We show that using linear regression with two-way cluster adjusted standard errors can yield unbiased ITT estimates and consistent standard errors regardless of the source of the random effects. We recommend this method replace HLM as the method of choice for testing intervention effects with cluster-randomized trials with noncompliance and cluster switching. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Measuring the predictability of life outcomes with a scientific mass collaboration.

How predictable are life trajectories? We investigated this question with a scientific mass collaboration using the common task method; 160 teams built predictive models for six life outcomes using data from the Fragile Families and Child Wellbeing Study, a high-quality birth cohort study. Despite using a rich dataset and applying machine-learning methods optimized for prediction, the best predictions were not very accurate and were only slightly better than those from a simple benchmark model. Within each outcome, prediction error was strongly associated with the family being predicted and weakly associated with the technique used to generate the prediction. Overall, these results suggest practical limits to the predictability of life outcomes in some settings and illustrate the value of mass collaborations in the social sciences.

The impact of penalties for wrong answers on the gender gap in test scores.

Multiple-choice examinations play a critical role in university admissions across the world. A key question is whether imposing penalties for wrong answers on these examinations deters guessing from women more than men, disadvantaging female test-takers. We consider data from a large-scale, high-stakes policy change that removed penalties for wrong answers on the national college entry examination in Chile. The policy change reduced a large gender gap in questions skipped. It also narrowed gender gaps in performance, primarily among high-performing test-takers, and in the fields of math, social science, and chemistry.

Perceived publication pressure in Amsterdam: Survey of all disciplinary fields and academic ranks.

Publications determine to a large extent the possibility to stay in academia ("publish or perish"). While some pressure to publish may incentivise high quality research, too much publication pressure is likely to have detrimental effects on both the scientific enterprise and on individual researchers. Our research question was: What is the level of perceived publication pressure in the four academic institutions in Amsterdam and does the pressure to publish differ between academic ranks and disciplinary fields? Investigating researchers in Amsterdam with the revised Publication Pressure Questionnaire, we find that a negative attitude towards the current publication climate is present across academic ranks and disciplinary fields. Postdocs and assistant professors (M = 3.42) perceive the greatest publication stress and PhD-students (M = 2.44) perceive a significant lack of resources to relieve publication stress. Results indicate the need for a healthier publication climate where the quality and integrity of research is rewarded.

Health Sciences cultural safety education in Australia, Canada, New Zealand, and the United States: a literature review.

OBJECTIVES: To review the research literature on cultural safety education within post-secondary health science programs. METHODS: We conducted health and social science database searches from 1996-2016, using combined keywords: cultural competence or safety; teaching or curriculum; universities, polytechnics or professional programs; and Aboriginal or Indigenous. In dyads, authors selected, and reviewed studies independently followed by discussion and consensus to identify thematic linkages of major findings. RESULTS: A total of 1583 abstracts and 122 full-text articles were reviewed with 40 selected for final inclusion. Publications from Australia, Canada, New Zealand and the United States described curriculum development and delivery. A variety of evaluation approaches were used including anecdotal reports, focus groups, interviews, course evaluations, reflective journals, pre-post surveys, critical reflective papers, and exam questions. Duration and depth of curricular exposure ranged from one day to integration across a six-year program.  Changes in student knowledge, attitude, self-confidence, and behaviour when working with Indigenous populations were reported. Cultural safety education and application to practice were shown to be linked to improved relationships, healthier outcomes, and increased number of Indigenous people entering health education programs and graduates interested in working in diverse communities. CONCLUSIONS: This review provides a summary of multidisciplinary didactic and experiential instructional approaches to cultural safety education and the impact on students, educators and Indigenous people.  Institutional support, strategic planning and cultural safety curriculum policy within post-secondary settings and community engagement are imperative for positive student experiences, advocacy, and actions toward health equity and improved health for Indigenous people and communities.

Setting the bar of social enterprise research high. Learning from medical science.

Both dysfunctional blood cells and physical exercise, despite the former being pathological and the latter completely normal, make one's blood more acidic, carrying less oxygen, which presents as fatigue. It is up to scientists to develop proper accounts of such mechanisms in the form of theories that are able to distinguish false from true positives. This piece's purpose is to make a case for preventing false positives in social enterprise research. This is achieved by showing how recent research portrays social enterprises as shallow nested systems defined by epiphenomenal enterprise activities as perceived by stakeholders, instead of the internal causal mechanisms that gear such activities. In achieving this piece's purpose, the analysis turns to an analogy with psychiatry, whose status as a medical profession was forged by telling disorders as harmful dysfunctions apart from the false positives of problems of living in general. Similarly, it is concluded that the relevance of social enterprises does not result from stakeholder rationales to use social enterprise-led activities per se, but from the designed functions of social enterprise mechanisms in conjunction with a value judgment that such mechanisms have specific benefits compared to alternatives.

Improving public health by respecting autonomy: using social science research to enfranchise vulnerable prison populations.

It is widely recognised that prisoners constitute a vulnerable population that is subject to numerous health inequalities and merits special protection. Improving prisoners' access to healthcare by ensuring adherence to the principle of equivalence has been the main focus of efforts to ensure that their health is not jeopardised. However, another means of respecting prisoners' autonomy and improving their health is to involve them (and prison staff) in social science research within prisons. Such research not only produces valuable data which can be used to assess whether the principle of equivalence is being respected; it also enfranchises prisoners by allowing them to air concerns about perceived ill-treatment and influence their environment. If prison authorities enable such research and adjust policy accordingly, both they and prisoners will benefit from the increased level of respect for prisoners' autonomy, and the improvements in individual and public health that flow from this. Conducting social science research in prisons enables the creation of a virtuous cycle of respect that makes prisons safer and healthier places.

The promises of qualitative inquiry.

We address the significance and implications of the formal entry of qualitative inquiry into the American Psychological Association. In our view, the discipline is enriched in new and important ways. Most prominently, the qualitative movement brings with it a pluralist orientation to knowledge and to practices of inquiry. Adding to the traditional view of knowledge as empirically supported theory are research practices congenial with varying accounts of knowledge, including, for example, knowledge as hermeneutic understanding, social construction, and practice-based experience. Added to the goal of prediction are investments in increasing cultural understanding, challenging cultural conventions, and directly fostering social change. The qualitative movement also enriches the discipline as a whole through the special ways in which it inspires new ranges of theory, fosters minority inclusion, and invites interdisciplinary collaboration. Finally, the movement holds promise in terms of the discipline's contribution to society at large. Here we focus on the advantages of knowing with others in addition to about them, and on ways in which qualitative work enhances communication with the society and the world. Realizing these potentials will depend on developments in responsible research and reporting, academic and journal policies, along with the discipline's capacities for appreciating a more comprehensive orientation to inquiry.