Repeated measures of implementation variables.

It is commonly acknowledged that implementation work is long-term and contextual in nature and often takes years to accomplish. Repeated measures are needed to study the trajectory of implementation variables over time. To be useful in typical practice settings, measures that are relevant, sensitive, consequential, and practical are needed to inform planning and action. If implementation independent variables and implementation dependent variables are to contribute to a science of implementation, then measures that meet these criteria must be established. This exploratory review was undertaken to "see what is being done" to evaluate implementation variables and processes repeatedly in situations where achieving outcomes was the goal (i.e., more likely to be consequential). No judgement was made about the adequacy of the measure (e.g., psychometric properties) in the review. The search process resulted in 32 articles that met the criteria for a repeated measure of an implementation variable. 23 different implementation variables were the subject of repeated measures. The broad spectrum of implementation variables identified in the review included innovation fidelity, sustainability, organization change, and scaling along with training, implementation teams, and implementation fidelity. Given the long-term complexities involved in providing implementation supports to achieve the full and effective use of innovations, repeated measurements of relevant variables are needed to promote a more complete understanding of implementation processes and outcomes. Longitudinal studies employing repeated measures that are relevant, sensitive, consequential, and practical should become common if the complexities involved in implementation are to be understood.

The Teaching-Family Model: The First 50 Years.

The Teaching-Family Model was perhaps the first "evidence-based program" in human services. This article describes the development of the treatment model, the failure of the first attempts to replicate the treatment model, the discovery of larger units for replication, the modest success of first attempts to replicate larger units, and the eventual success of replications. The Teaching-Family Model is a testament to the sustainability (and continual improvement) of innovation and implementation methods and the value of the Teaching-Family Association for sustaining a community of practice and for managing the practitioner fidelity and organization fidelity data systems nationally. The benefits of applied behavior analysis and the implications for a new science of implementation for having research purposefully used in practice are explored.

Selecting an EBP to reduce long-term foster care: lessons from a university-child welfare agency partnership.

A growing implementation literature outlines broad evidence-based practice implementation principles and pitfalls. Less robust is knowledge about the real-world process by which a state or agency chooses an evidence-based practice to implement and evaluate. Using a major U.S. initiative to reduce long-term foster care as the case, this article describes three major aspects of the evidence-based practice selection process: defining a target population, selecting an evidence-based practice model and purveyor, and tailoring the model to the practice context. Use of implementation science guidelines and lessons learned from a unique private-public-university partnership are discussed.

Usability testing, initial implementation, and formative evaluation of an evidence-based intervention: lessons from a demonstration project to reduce long-term foster care.

The field of child welfare faces an undersupply of evidence-based interventions to address long-term foster care. The Permanency Innovations Initiative is a five-year federal demonstration project intended to generate evidence to reduce long stays in foster care for those youth who encounter the most substantial barriers to permanency. This article describes a systematic and staged approach to implementation and evaluation of a PII project that included usability testing as one of its key activities. Usability testing is an industry-derived practice which analyzes early implementation processes and evaluation procedures before they are finalized. This article describes the iterative selection, testing, and analysis of nine usability metrics that were designed to assess three important constructs of the project's initial implementation and evaluation: intervening early, obtaining consent, and engaging parents. Results showed that seven of nine metrics met a predetermined target. This study demonstrates how findings from usability testing influenced the initial implementation and formative evaluation of an evidence-supported intervention. Implications are discussed for usability testing as a quality improvement cycle that may contribute to better operationalized interventions and more reliable, valid, and replicable evidence.

Mobilizing communities for implementing evidence-based youth violence prevention programming: a commentary.

Evidence-based programs have struggled for acceptance in human service settings. Information gleaned from these experiences indicates that implementation is the missing link in the science to service chain. The science and practice of implementation is progressing and can inform approaches to full and effective uses of youth violence prevention programs nationally. Implementation Teams that know (a) innovations, (b) implementation best practices, and (c) improvement cycles are essential to mobilizing support for successful uses of evidence-based programs on a socially significant scale. The next wave of development in implementation science and practice is underway: establishing infrastructures for implementation to make implementation expertise available to communities nationally. Good science, implemented well in practice, can benefit all human services, including youth violence prevention.