Lawmakers' use of scientific evidence can be improved.

Core to the goal of scientific exploration is the opportunity to guide future decision-making. Yet, elected officials often miss opportunities to use science in their policymaking. This work reports on an experiment with the US Congress-evaluating the effects of a randomized, dual-population (i.e., researchers and congressional offices) outreach model for supporting legislative use of research evidence regarding child and family policy issues. In this experiment, we found that congressional offices randomized to the intervention reported greater value of research for understanding issues than the control group following implementation. More research use was also observed in legislation introduced by the intervention group. Further, we found that researchers randomized to the intervention advanced their own policy knowledge and engagement as well as reported benefits for their research following implementation.

Clearinghouse Standards of Evidence on the Transparency, Openness, and Reproducibility of Intervention Evaluations.

Clearinghouses are influential repositories of information on the effectiveness of social interventions. To identify which interventions are "evidence-based," clearinghouses review intervention evaluations using published standards of evidence that focus primarily on internal validity and causal inferences. Open science practices can improve trust in evidence from evaluations on the effectiveness of social interventions. Including open science practices in clearinghouse standards of evidence is one of many efforts that could increase confidence in designations of interventions as "evidence-based." In this study, we examined the policies, procedures, and practices of 10 federal evidence clearinghouses that review preventive interventions-an important and influential subset of all evidence clearinghouses. We found that seven consider at least one open science practice when evaluating interventions: replication (6 of 10 clearinghouses), public availability of results (6), investigator conflicts of interest (3), design and analysis transparency (3), study registration (2), and protocol sharing (1). We did not identify any policies, procedures, or practices related to analysis plan registration, data sharing, code sharing, material sharing, and citation standards. We provide a framework with specific recommendations to help federal and other evidence clearinghouses implement the Transparency and Openness Promotion (TOP) Guidelines. Our proposed "TOP Guidelines for Clearinghouses" includes reporting whether evaluations used open science practices, incorporating open science practices in their standards for receiving "evidence-based" designations, and verifying that evaluations used open science practices. Doing so could increase the trustworthiness of evidence used for policy making and support improvements throughout the evidence ecosystem.

The Role of Open Science Practices in Scaling Evidence-Based Prevention Programs.

The goal of creating evidence-based programs is to scale them at sufficient breadth to support population-level improvements in critical outcomes. However, this promise is challenging to fulfill. One of the biggest issues for the field is the reduction in effect sizes seen when a program is taken to scale. This paper discusses an economic perspective that identifies the underlying incentives in the research process that lead to scale up problems and to deliver potential solutions to strengthen outcomes at scale. The principles of open science are well aligned with this goal. One prevention program that has begun to scale across the USA is early childhood home visiting. While there is substantial impact research on home visiting, overall average effect size is .10 and a recent national randomized trial found attenuated effect sizes in programs implemented under real-world conditions. The paper concludes with a case study of the relevance of the economic model and open science in developing and scaling evidence-based home visiting. The case study considers how the traditional approach for testing interventions has influenced home visiting's evolution to date and how open science practices could have supported efforts to maintain impacts while scaling home visiting. It concludes by considering how open science can accelerate the refinement and scaling of home visiting interventions going forward, through accelerated translation of research into policy and practice.

Transparent, Open, and Reproducible Prevention Science.

The field of prevention science aims to understand societal problems, identify effective interventions, and translate scientific evidence into policy and practice. There is growing interest among prevention scientists in the potential for transparency, openness, and reproducibility to facilitate this mission by providing opportunities to align scientific practice with scientific ideals, accelerate scientific discovery, and broaden access to scientific knowledge. The overarching goal of this manuscript is to serve as a primer introducing and providing an overview of open science for prevention researchers. In this paper, we discuss factors motivating interest in transparency and reproducibility, research practices associated with open science, and stakeholders engaged in and impacted by open science reform efforts. In addition, we discuss how and why different types of prevention research could incorporate open science practices, as well as ways that prevention science tools and methods could be leveraged to advance the wider open science movement. To promote further discussion, we conclude with potential reservations and challenges for the field of prevention science to address as it transitions to greater transparency, openness, and reproducibility. Throughout, we identify activities that aim to strengthen the reliability and efficiency of prevention science, facilitate access to its products and outputs, and promote collaborative and inclusive participation in research activities. By embracing principles of transparency, openness, and reproducibility, prevention science can better achieve its mission to advance evidence-based solutions to promote individual and collective well-being.

Scaling up Evidence-Based Interventions in US Public Systems to Prevent Behavioral Health Problems: Challenges and Opportunities.

A number of programs, policies, and practices have been tested using rigorous scientific methods and shown to prevent behavioral health problems (Catalano et al., Lancet 379:1653-1664, 2012; National Research Council and Institute of Medicine, 2009). Yet these evidence-based interventions (EBIs) are not widely used in public systems, and they have limited reach (Glasgow et al., American Journal of Public Health 102:1274-1281, 2012; National Research Council and Institute of Medicine 2009; Prinz and Sanders, Clinical Psychology Review 27:739-749, 2007). To address this challenge and improve public health and well-being at a population level, the Society for Prevention Research (SPR) formed the Mapping Advances in Prevention Science (MAPS) IV Translation Research Task Force, which considered ways to scale up EBIs in five public systems: behavioral health, child welfare, education, juvenile justice, and public health. After reviewing other efforts to scale up EBIs in public systems, a common set of factors were identified as affecting scale-up in all five systems. The most important factor was the degree to which these systems enacted public policies (i.e., statutes, regulations, and guidance) requiring or recommending EBIs and provided public funds for EBIs. Across systems, other facilitators of scale-up were creating EBIs that are ready for scale-up, public awareness of and support for EBIs, community engagement and capacity to implement EBIs, leadership support for EBIs, a skilled workforce capable of delivering EBIs, and data monitoring and evaluation capacity. It was concluded that the following actions are needed to significantly increase EBI scale-up in public systems: (1) provide more public policies and funding to support the creation, testing, and scaling up of EBIs; (2) develop and evaluate specific frameworks that address systems level barriers impeding EBI scale-up; and (3) promote public support for EBIs, community capacity to implement EBIs at scale, and partnerships between community stakeholders, policy makers, practitioners, and scientists within and across systems.

Principles of Precision Prevention Science for Improving Recruitment and Retention of Participants.

Precision medicine and precision public health focus on identifying and providing the right intervention to the right population at the right time. Expanding on the concept, precision prevention science could allow the field to examine prevention programs to identify ways to make them more efficient and effective at scale, including addressing issues related to engagement and retention of participants. Research to date on engagement and retention has often focused on demographics and risk factors. The current paper proposes using McCurdy and Daro (Family Relations, 50, 113-121, 2001) model that posits a complex mixture of individual, provider, program, and community-level factors synergistically affect enrollment, engagement, and retention. The paper concludes recommending the use of research-practice partnerships and innovative, rapid cycle methods to design and improve prevention programs related to participant engagement and retention at scale.

Behavior Problems in Elementary School among Low-Income Males: The Role of Teacher-Child Relationships.

The present study identified trajectories of teacher-child relationship conflict and closeness from first through sixth grades, and associations between these trajectories and externalizing and internalizing behaviors at age 11 among low-income, urban males (N = 262). There were three main findings. Nagin cluster analyses indicated five trajectories for conflict with all children evidencing increases in conflict, and four trajectories for closeness with all children demonstrating decreases in closeness. Trajectories with higher levels of conflict and lower levels of closeness were associated with higher levels of externalizing and internalizing behavior problems at age 11. Moreover, conflictual teacher-child relationships exacerbated the effects of externalizing and internalizing behavior problems in early childhood; children with conflictual teacher-child relationships had higher levels of behavior problems in middle childhood relative to children with low conflictual teacher-child relationships. Implications of targeting teacher-child relationships as interventions to help prevent behavior problems are discussed.

The Intersection Between Prevention Science and Evidence-Based Policy: How the SPR Evidence Standards Support Human Services Prevention Programs.

The revised Society for Prevention Research (SPR) standards of evidence are an exciting advance in the field of prevention science. We appreciate the committee's vision that the standards represent goals to aspire to rather than a set of benchmarks for where prevention science is currently. The discussion about the standards highlights how much has changed in the field over the last 10 years and as knowledge, theory, and methods continue to advance, the new standards push the field toward increasing rigor and relevance. This commentary discusses how the revised standards support work of translating high-quality evaluations to support evidence-based policy and work supporting evidence-based programs' ability to implement at scale. The commentary ends by raising two areas, generating evidence at scale and transparency of research, as additional areas for consideration in future standards.

Supporting replication and scale-up of evidence-based home visiting programs: assessing the implementation knowledge base.

In recent years, researchers, policymakers, and practitioners have expressed a growing interest in the use of interventions with scientific evidence of effectiveness. Reproducing positive effects shown in research, however, requires more than simply adopting an evidence-based program. There is growing recognition across disciplines of the importance of implementation research to guide adoption, replication, and scale-up of evidence-based interventions. We evaluate the state of the knowledge base supporting replication and scale-up of evidence-based programs by reviewing information on implementation included in the research literature on 22 home visiting programs that have or are building an evidence base. We used the Interactive Systems Framework for Dissemination and Implementation to assess programs.

Exploring enablers and barriers to implementing the Transparency and Openness Promotion Guidelines: a theory-based survey of journal editors.

The Transparency and Openness Promotion (TOP) Guidelines provide a framework to help journals develop open science policies. Theories of behaviour change can guide understanding of why journals do (not) implement open science policies and the development of interventions to improve these policies. In this study, we used the Theoretical Domains Framework to survey 88 journal editors on their capability, opportunity and motivation to implement TOP. Likert-scale questions assessed editor support for TOP, and enablers and barriers to implementing TOP. A qualitative question asked editors to provide reflections on their ratings. Most participating editors supported adopting TOP at their journal (71%) and perceived other editors in their discipline to support adopting TOP (57%). Most editors (93%) agreed their roles include maintaining policies that reflect current best practices. However, most editors (74%) did not see implementing TOP as a high priority compared with other editorial responsibilities. Qualitative responses expressed structural barriers to implementing TOP (e.g. lack of time, resources and authority to implement changes) and varying support for TOP depending on study type, open science standard, and level of implementation. We discuss how these findings could inform the development of theoretically guided interventions to increase open science policies, procedures and practices.

Behavior problems in late childhood: the roles of early maternal attachment and teacher-child relationship trajectories.

The purposes of the current study were: (1) to examine the roles of early maternal attachment relationships and teacher-child relationships during childhood for externalizing and internalizing behaviors in late childhood, and (2) to investigate teacher-child relationships, as well as externalizing and internalizing behaviors in early childhood as possible mechanisms linking early maternal attachment relationships to behavior problems in late childhood. Longitudinal data from the National Institute of Child Health and Human Development Early Child Care Research Network Study of Early Child Care and Youth Development (N = 1140 mothers and children) were used in this investigation. There were three main findings. First, insecure/other maternal attachment relationships in early childhood (i.e., 36 months) were associated with externalizing and internalizing behaviors in late childhood (Grade 5). Second, elevated levels of teacher-child conflict during childhood were associated with externalizing behaviors in late childhood whereas low levels of teacher-child closeness were associated with internalizing behaviors. Third, the effects of insecure/other attachment on externalizing and internalizing behaviors in late childhood were mediated through teacher-child relationships during childhood and early externalizing and internalizing behaviors. Implications for attachment theory are discussed.

Expert-generated standard practice elements for evidence-based home visiting programs using a Delphi process.

BACKGROUND: States, territories, non-profits, and tribes are eligible to obtain federal funding to implement federally endorsed evidence-based home visiting programs. This represents a massive success in translational science, with $400 million a year allocated to these implementation efforts. This legislation also requires that 3% of this annual funding be allocated to tribal entities implementing home visiting in their communities. However, implementing stakeholders face challenges with selecting which program is best for their desired outcomes and context. Moreover, recent reviews have indicated that when implemented in practice and delivered at scale, many evidence-based home visiting programs fail to replicate the retention rates and effects achieved during clinical trials. To inform program implementers and better identify the active ingredients in home visiting programs that drive significant impacts, we aimed to develop an expert derived consensus taxonomy on the elements used in home visiting practice that are essential to priority outcome domains. METHODS: We convened a panel of 16 experts representing researchers, model representatives, and program implementers using a Delphi approach. We first elicited standard practice elements (SPEs) using open-ended inquiry, then compared these elements to behavior change techniques (BCTs) given their general importance in the field of home visiting; and finally rated their importance to 10 outcome domains. RESULTS: Our process identified 48 SPEs derived from the panel, with 83 additional BCTs added based on the literature. Six SPEs, mostly related to home visitor characteristics and skills, were rated essential across all outcome domains. Fifty-three of the 83 BCTs were rated unnecessary across all outcome domains. CONCLUSIONS: This work represents the first step in a consensus-grounded taxonomy of techniques and strategies necessary for home visiting programs and provides a framework for future hypothesis testing and replication studies.

Preschool boys' development of emotional self-regulation strategies in a sample at risk for behavior problems.

Little longitudinal research has been conducted on changes in children's emotional self-regulation strategy (SRS) use after infancy, particularly for children at risk. In this study, the authors examined changes in boys' emotional SRS from toddlerhood through preschool. Repeated observational assessments using delay of gratification tasks at ages 2, 3, and 4 years were examined with both variable- and person-oriented analyses in a low-income sample of boys (N = 117) at risk for early problem behavior. Results were consistent with theory on emotional SRS development in young children. Children initially used more emotion-focused SRS (e.g., comfort seeking) and transitioned to greater use of planful SRS (e.g., distraction) by 4 years of age. Person-oriented analysis using trajectory analysis found similar patterns from 2 to 4 years, with small groups of boys showing delayed movement away from emotion-focused strategies or delay in the onset of regular use of distraction. The results provide a foundation for future researchers to examine the development of SRS in low-income young children.

Evaluating implementation of the Transparency and Openness Promotion (TOP) guidelines: the TRUST process for rating journal policies, procedures, and practices.

BACKGROUND: The Transparency and Openness Promotion (TOP) Guidelines describe modular standards that journals can adopt to promote open science. The TOP Factor is a metric to describe the extent to which journals have adopted the TOP Guidelines in their policies. Systematic methods and rating instruments are needed to calculate the TOP Factor. Moreover, implementation of these open science policies depends on journal procedures and practices, for which TOP provides no standards or rating instruments. METHODS: We describe a process for assessing journal policies, procedures, and practices according to the TOP Guidelines. We developed this process as part of the Transparency of Research Underpinning Social Intervention Tiers (TRUST) Initiative to advance open science in the social intervention research ecosystem. We also provide new instruments for rating journal instructions to authors (policies), manuscript submission systems (procedures), and published articles (practices) according to standards in the TOP Guidelines. In addition, we describe how to determine the TOP Factor score for a journal, calculate reliability of journal ratings, and assess coherence among a journal's policies, procedures, and practices. As a demonstration of this process, we describe a protocol for studying approximately 345 influential journals that have published research used to inform evidence-based policy. DISCUSSION: The TRUST Process includes systematic methods and rating instruments for assessing and facilitating implementation of the TOP Guidelines by journals across disciplines. Our study of journals publishing influential social intervention research will provide a comprehensive account of whether these journals have policies, procedures, and practices that are consistent with standards for open science and thereby facilitate the publication of trustworthy findings to inform evidence-based policy. Through this demonstration, we expect to identify ways to refine the TOP Guidelines and the TOP Factor. Refinements could include: improving templates for adoption in journal instructions to authors, manuscript submission systems, and published articles; revising explanatory guidance intended to enhance the use, understanding, and dissemination of the TOP Guidelines; and clarifying the distinctions among different levels of implementation. Research materials are available on the Open Science Framework: https://osf.io/txyr3/ .

Emotion regulation strategies and later externalizing behavior among European American and African American children.

Children's early emotion regulation strategies (ERS) have been related to externalizing problems; however, most studies have included predominantly European American, middle-class children. The current study explores whether ERS use may have differential outcomes as a function of the mother's ethnic culture. The study utilizes two diverse samples of low-income male toddlers to examine observed ERS during a delay of gratification task in relation to maternal and teacher reports of children's externalizing behavior 2 to 6 years later. Although the frequencies of ERS were comparable between ethnic groups in both samples, the use of physical comfort seeking and self-soothing was positively related to African American children's later externalizing behavior but negatively related to externalizing behavior for European American children in Sample 1. Data from Sample 2 appear to support this pattern for self-soothing in maternal, but not teacher, report of externalizing behavior. Within group differences by income were examined as a possible explanatory factor accounting for the ethnic differences, but it was not supported. Alternative explanations are discussed to explain the pattern of findings.

Innovative Research Methods to Advance Precision in Home Visiting for More Efficient and Effective Programs.

Home visiting during early childhood can improve a range of outcomes for children and families. As evidence-based models are implemented across the nation, two questions have emerged. First, can home visiting improve outcomes more efficiently? Second, can overall effects be strengthened for specific subgroups of families? For the past several decades, research focused on testing the average effects of home visiting models on short- to long-term outcomes has found small impacts. These effects are not the same for all families. The field needs new evidence produced in new ways to overcome these challenges. In this article, we provide an overview of the evidence in this field, including what works and for whom. Next, we explain precision approaches to various fields and how this approach could be used in home visiting programs. Research on precision home visiting focuses on the ingredients of home visiting models, collaborating with practitioners to identify the ingredients and testing them on near-term outcomes, and using innovative study designs to learn more quickly what works best for which families. We conclude by proposing four pillars of research that will help achieve precision home visiting services.

The role of psychology in evidence-based policymaking: Mapping opportunities for strategic investment in poverty reduction.

Increasing efforts are being undertaken to understand how to improve the use of research evidence in policy settings. In particular, growing efforts to understand the use of research in legislative contexts. Although high-profile examples of psychology's contributions to public policy exist-particularly around antipoverty legislation-little systematic review has quantified how the field has informed federal policy across time. Recognizing the importance of exploring psychology's use in policymaking, we provide an overview of psychology's presence in federal antipoverty legislation over the last 2 decades by reviewing the over 6,000 antipoverty bills introduced to the U.S. Congress since 1993 for mentions of psychology. Further, to explore how psychology's contributions are related to policymakers' attributions about the causes of poverty, their public statements and voting behavior is considered. Key gaps in our scientific knowledge for informing poverty-related policy are identified. Opportunities to enhance the relevance of psychology in poverty reduction efforts, including the evidence-based policy movement, are described. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Randomized prevention trial for early conduct problems: effects on proactive parenting and links to toddler disruptive behavior.

Despite knowledge of early pathways to conduct problems, few preventive interventions are specifically designed to modify disruptive behavior in toddlerhood. One potential prevention target is proactive and positive parenting, which is associated with reduced risk of conduct problems in preschoolers. This randomized trial with 120 low-income 2-year-old boys examined whether a brief family-centered intervention that reduces disruptive behavior (D. Shaw, T. Dishion, L. Supplee, F. Gardner, & K. Arnds, 2006) also leads to increases in proactive and positive parenting. It also explored whether change in parenting predicts change in disruptive behavior. In the intervention group, proactive and positive parenting skills increased among parents of 3-year-olds. Change in proactive and positive parenting of 2- to 3-year-old toddlers correlated with change in child disruptive behavior, although the mediation effect of positive parenting was of only borderline significance. Findings suggest that even within a brief and multifaceted preventive intervention, change in proactive parenting skills contributes modestly but significantly to change in child problem behavior.