Five-year fidelity assessment of an evidence-based parenting program (GenerationPMTO): inter-rater reliability following international implementation.

BACKGROUND: Implementing evidence-based programs in community service settings introduces the challenge of ensuring sustained fidelity to the original program. We employ a fidelity measure based on direct observation of practitioners' competence and adherence to the evidence-based parenting program (EBPP) GenerationPMTO following installation in national and international sites. Fidelity monitoring is crucial, especially when the program purveyor transfers administration of the program to the community as was done in this case. In previous studies, the Fidelity of Implementation rating system (FIMP) was used to evaluate practitioners' fidelity to the GenerationPMTO intervention in six countries following implementation showing high levels of adherence up to 17 years post certification. Other studies showed FIMP to have predictive validity. The present study provides inter-rater reliability data for this fidelity tool across teams of the purveyor, Implementation Sciences International, Inc./ISII, and national and international sites over a five-year period. METHODS: Data assess inter-rater reliability in terms of percent agreement and intraclass correlation (ICC) for the purveyor's two fidelity teams and the fidelity teams in seven implementation sites. RESULTS: Results report stable good to excellent levels of inter-rater reliability and ICCs as well as good attendance at fidelity meetings for all fidelity teams. CONCLUSIONS: This observational method of assessing fidelity post implementation is a promising approach to enable EBPPs to be transferred safely from purveyors to communities while maintaining reliable fidelity to the intervention.

Development and Evaluation of a Component Level Implementation Fidelity Rating System for the GenerationPMTO Intervention.

Empirically determining the components of evidence-based interventions contributing to positive change is a crucial, yet understudied area of research. In support of this aim, we describe the development and evaluation of an observational rating system for measuring fidelity to specific components of the evidence-based GenerationPMTO parenting intervention. A five-step process was employed to systematically develop the rating system, which included consultation with the intervention developer and input from additional GenerationPMTO experts. The rating system was then tested using 247 h of video data from 184 parenting group intervention sessions. Study findings support the psychometric properties of the new measure with regard to item performance, reliability (i.e., inter-rater reliability of items, dimensionality of components, internal consistency of component scales), and validity (i.e., content validity, convergent validity, discriminant validity, and predictive validity of the component scales) for seven of the eight scales evaluated. The seven components include clear directions, skill encouragement, emotion regulation, limit setting, effective communication, problem solving, and monitoring. Data did not support the psychometric properties of the positive involvement scale. Overall, the ability to assess component-specific fidelity allows for a more nuanced examination of change processes, with meaningful implications for research and practice.

Implementing an Evidence-Based Intervention for Children in Europe: Evaluating the Full-Transfer Approach.

This study evaluated the implementation outcomes of GenerationPMTO, an evidence-based parenting intervention for child and adolescent behavior problems, in three European countries. The implementation approach was full transfer, in which purveyors train a first generation (G1) of practitioners; adopting sites assume oversight, training, certification, and fidelity assessment for subsequent generations (Forgatch & DeGarmo, 2011; Forgatch & Gewirtz, 2017). Three hundred therapists participated in trainings in GenerationPMTO in Iceland, Denmark, and the Netherlands. Data are from the implementation's initiation in each country through 2016, resulting in 6 generations in Iceland, 8 in Denmark, and 4 in the Netherlands. Therapist fidelity was measured at certification with an observation-based tool, the Fidelity of Implementation Rating System (Knutson, Forgatch, Rains, & Sigmarsdóttir, 2009). Candidates in all generations achieved fidelity scores at or above the required standard. Certification fidelity scores were evaluated for G1 candidates, who were trained by the purveyor, and subsequent generations trained by the adopting implementation site. In each country, certification fidelity scores declined for G2 candidates compared with G1 and recovered to G1 levels for subsequent generations, partially replicating findings from a previous Norwegian study (Forgatch & DeGarmo, 2011). Recovery to G1 levels of fidelity scores was obtained in Iceland and the Netherlands by G3; in Denmark, the recovery was obtained by G5. The mean percentage of certification in each country was more than 80%; approximately 70% of certified therapists remained active in 2017. Findings support full transfer as an effective implementation approach with long-term sustainability and fidelity.

Scaling up an Empirically Supported Intervention with Long-Term Outcomes: the Nationwide Implementation of GenerationPMTO in Norway.

Effective implementation outcomes are necessary preconditions for effective service and positive treatment outcomes for children with behavioral problems. The aim of this study is to assess outcomes of the transfer of the empirically supported intervention GenerationPMTO from the developer in the USA to a nationwide implementation in Norway. Adoption, sustainability, reach, and fidelity are tested across seven generations of therapists in Norway. Participants in the study were 521 therapists who began training in the program. The developer's team trained the first generation (G1) and the Norwegian team trained the next six generations (G2-G7). The mean rate of certification was 94.2% (n = 491). Intervention fidelity was assessed from 1964 video recordings of intervention sessions submitted for certification evaluation using the observation-based measure Fidelity of Implementation Rating System (FIMP). A small but significant drop in fidelity scores was previously observed from G1 to G2; however, fidelity scores recovered at or above G1 scores for G3 Forgatch and DeGarmo (Prevention Science 12, 235-246, 2011). Twenty years since the inception of implementation, 314 certified therapists practice the model today, a retention rate of 64%. The outcomes show sustained fidelity scores across seven generations, increasing heterogeneity among therapists trained, and a shift of focus in the target population from clinical to primary services. The present study contributes to the field with the systematic evaluation of outcomes for the full transfer implementation approach with continuing adoption and sustainability, increasing reach and sustained intervention fidelity across several generations of practitioners.

Resurrecting the chimera: Progressions in parenting and peer processes.

This report uses 6-year outcomes of the Oregon Divorce Study to examine the processes by which parenting practices affect deviant peer association during two developmental stages: early to middle childhood and late childhood to early adolescence. The participants were 238 newly divorced mothers and their 5- to 8-year-old sons who were randomly assigned to Parent Management Training-Oregon Model (PMTO®) or to a no-treatment control group. Parenting practices, child delinquent behavior, and deviant peer association were repeatedly assessed from baseline to 6 years after baseline using multiple methods and informants. PMTO had a beneficial effect on parenting practices relative to the control group. Two stage models linking changes in parenting generated by PMTO to children's growth in deviant peer association were supported. During the early to middle childhood stage, the relationship of improved parenting practices on deviant peer association was moderated by family socioeconomic status (SES); effective parenting was particularly important in mitigating deviant peer association for lower SES families whose children experience higher densities of deviant peers in schools and neighborhoods. During late childhood and early adolescence, the relationship of improved parenting to youths' growth in deviant peer association was mediated by reductions in the growth of delinquency during childhood; higher levels of early delinquency are likely to promote deviant peer association through processes of selective affiliation and reciprocal deviancy training. The results are discussed in terms of multilevel developmental progressions of diminished parenting, child involvement in deviancy producing processes in peer groups, and increased variety and severity of antisocial behavior, all exacerbated by ecological risks associated with low family SES.

Parent Management Training-Oregon Model: Adapting Intervention with Rigorous Research.

Parent Management Training-Oregon Model (PMTO(®) ) is a set of theory-based parenting programs with status as evidence-based treatments. PMTO has been rigorously tested in efficacy and effectiveness trials in different contexts, cultures, and formats. Parents, the presumed agents of change, learn core parenting practices, specifically skill encouragement, limit setting, monitoring/supervision, interpersonal problem solving, and positive involvement. The intervention effectively prevents and ameliorates children's behavior problems by replacing coercive interactions with positive parenting practices. Delivery format includes sessions with individual families in agencies or families' homes, parent groups, and web-based and telehealth communication. Mediational models have tested parenting practices as mechanisms of change for children's behavior and found support for the theory underlying PMTO programs. Moderating effects include children's age, maternal depression, and social disadvantage. The Norwegian PMTO implementation is presented as an example of how PMTO has been tailored to reach diverse populations as delivered by multiple systems of care throughout the nation. An implementation and research center in Oslo provides infrastructure and promotes collaboration between practitioners and researchers to conduct rigorous intervention research. Although evidence-based and tested within a wide array of contexts and populations, PMTO must continue to adapt to an ever-changing world.

Examining the process of change in an evidence-based parent training intervention: a qualitative study grounded in the experiences of participants.

While strong research evidence demonstrates that parent training interventions are capable of preventing child behavioral problems, much less is known about how the participants in these programs experience the change process. The purpose of this study was to provide a better understanding of how parents' experiences in an evidence-based parent training intervention led to change in their parenting practices, based on the first-person accounts of program participants. Qualitative data were collected through in-depth, individual interviews with parents who had completed the intervention known as Parent Management Training-the Oregon Model (PMTO™). Data were analyzed according to principles of the grounded theory approach, using the constant comparative method and a sequential process of open, axial, and selective coding. Study findings suggest that parents make active and intentional efforts to attempt, appraise, and apply the intervention material within their various life contexts, contributing to change in their parenting practices. Aspects of intervention content, method of delivery, and the role of the interventionist were also found to be important. This study can guide further prevention research into the mechanisms of change operating in parent training interventions and has the potential to inform continued efforts to adapt and implement evidence-based parent training interventions.

Treatment effectiveness of PMTO for children's behavior problems in Iceland: assessing parenting practices in a randomized controlled trial.

Findings are presented from an Icelandic randomized control trial (RCT) evaluating parent management training - Oregon model (PMTO™), a parent training intervention designed to improve parenting practices and reduce child behavior problems. In a prior report from this effectiveness study that focused on child outcomes, children in the PMTO condition showed greater reductions in reported child adjustment problems relative to the comparison group. The present report focuses on observed parenting practices as the targeted outcome, with risk by treatment moderators also tested. It was hypothesized that mothers assigned to the PMTO condition would show greater gains in pre-post parenting practices relative to controls. The sample was recruited from five municipalities throughout Iceland and included 102 participating families of children with behavior problems. Cases were referred by community professionals and randomly assigned to either PMTO (n = 51) or community services usually offered (n = 51). Child age ranged from 5 to 12 years; 73% were boys. Contrary to expectations, findings showed no main effects for changes in maternal parenting. However, evaluation of risk by treatment moderators showed greater gains in parenting practices for mothers who increased in depressed mood within the PMTO group relative to their counterparts in the comparison group. This finding suggests that PMTO prevented the expected damaging effects of depression on maternal parenting. Failure to find hypothesized main effects may indicate that there were some unobserved factors regarding the measurement and a need to further adapt the global observational procedures to Icelandic culture.

A confidant support and problem solving model of divorced fathers' parenting.

This study tested a hypothesized social interaction learning (SIL) model of confidant support and paternal parenting. The latent growth curve analysis employed 230 recently divorced fathers, of which 177 enrolled support confidants, to test confidant support as a predictor of problem solving outcomes and problem solving outcomes as predictors of change in fathers' parenting. Fathers' parenting was hypothesized to predict growth in child behavior. Observational measures of support behaviors and problem solving outcomes were obtained from structured discussions of personal and parenting issues faced by the fathers. Findings replicated and extended prior cross-sectional studies with divorced mothers and their confidants. Confidant support predicted better problem solving outcomes, problem solving predicted more effective parenting, and parenting in turn predicted growth in children's reduced total problem behavior T scores over 18 months. Supporting a homophily perspective, fathers' antisociality was associated with confidant antisociality but only fathers' antisociality influenced the support process model. Intervention implications are discussed regarding SIL parent training and social support.

Investigating active ingredients of the GenerationPMTO intervention: Predictors of postintervention change trajectories in parenting practices.

Empirically determining the active ingredients of evidence-based parenting interventions is a promising means for strengthening interventions and enhancing their public health impact. This study aimed to determine which distinct ingredients of the GenerationPMTO (GenPMTO) intervention were associated with subsequent changes in parenting practices. Using a sample of 153 participants randomly assigned to the GenPMTO condition, we employed multilevel modeling to identify intervention ingredients empirically linked with change trajectories in parenting practices observed across the 2 years following intervention exposure. Coercive parenting and positive parenting outcomes were examined. Study results indicated that emotion regulation, effective communication, problem solving, and monitoring each demonstrated a significant pattern of findings for coercive parenting. Differential exposure to each of these ingredients significantly predicted the level of coercive parenting immediately postintervention and/or trajectories of change in coercive parenting across the subsequent 2-year period, controlling for coercive parenting at baseline. No significant predictors were found for positive parenting trajectories. Our findings suggest four components as active ingredients of the GenPMTO intervention for coercive parenting. Identification of these active ingredients may lead to strengthening future iterations of GenPMTO by expanding the set of core components specified in the model, which may further improve public health benefits. Implications for further understanding change stemming from evidence-based parenting interventions are also discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Sustaining fidelity following the nationwide PMTO™ implementation in Norway.

This report describes three studies from the nationwide Norwegian implementation of Parent Management Training-Oregon Model (PMTO™), an empirically supported treatment for families of children with behavior problems (Forgatch and Patterson 2010). Separate stages of the implementation were evaluated using a fidelity measure based on direct observation of intervention sessions. Study 1 assessed growth in fidelity observed early, mid, and late in the training of a group of practitioners. We hypothesized increased fidelity and decreased variability in practice. Study 2 evaluated method fidelity over the course of three generations of practitioners trained in PMTO. Generation 1 (G1) was trained by the PMTO developer/purveyors; Generation 2 (G2) was trained by selected G1 Norwegian trainers; and Generation 3 (G3) was trained by G1 and G2 trainers. We hypothesized decrease in fidelity with each generation. Study 3 tested the predictive validity of fidelity in a cross-cultural replication, hypothesizing that higher fidelity scores would correlate with improved parenting practices observed in parent-child interactions before and after treatment. In Study 1, trainees' performance improved and became more homogeneous as predicted. In Study 2, a small decline in fidelity followed the transfer from the purveyor trainers to Norwegian trainers in G2, but G3 scores were equivalent to those attained by G1. Thus, the hypothesis was not fully supported. Finally, the FIMP validity model replicated; PMTO fidelity significantly contributed to improvements in parenting practices from pre- to post-treatment. The data indicate that PMTO was transferred successfully to Norwegian implementation with sustained fidelity and cross-cultural generalization.

Helping military families through the deployment process: Strategies to support parenting.

Recent studies have highlighted the impact of deployment on military families and children and the corresponding need for interventions to support them. Historically, however, little emphasis has been placed on family-based interventions in general, and parenting interventions in particular, with returning service members. This paper provides an overview of research on the associations between combat deployment, parental adjustment of service members and spouses, parenting impairments, and children's adjustment problems, and provides a social interaction learning framework for research and practice to support parenting among military families affected by a parent's deployment. We then describe the Parent Management Training-Oregon model (PMTO(™)), a family of interventions that improves parenting practices and child adjustment in highly stressed families, and briefly present work on an adaptation of PMTO for use in military families (After Deployment: Adaptive Parenting Tools, or ADAPT). The article concludes with PMTO-based recommendations for clinicians providing parenting support to military families.

Cascading effects following intervention.

Four different sources for cascade effects were examined using 9-year process and outcome data from a randomized controlled trial of a preventive intervention using the Parent Management Training-Oregon Model (PMTO™). The social interaction learning model of child antisocial behavior serves as one basis for predicting change. A second source addresses the issue of comorbid relationships among clinical diagnoses. The third source, collateral changes, describes events in which changes in one family member correlate with changes in another. The fourth component is based on the long-term effects of reducing coercion and increasing positive interpersonal processes within the family. New findings from the 9-year follow-up show that mothers experienced benefits as measured by standard of living (i.e., income, occupation, education, and financial stress) and frequency of police arrests. It is assumed that PMTO reduces the level of coercion, which sets the stage for a massive increase in positive social interaction. In effect, PMTO alters the family environment and thereby opens doors to healthy new social environments.

Testing the Oregon delinquency model with 9-year follow-up of the Oregon Divorce Study.

This paper presents experimental tests of the Oregon delinquency model applied within a randomized design of an at-risk sample of single mothers and their elementary school-aged sons. In the theoretical model, ineffective parenting practices and deviant peer association serve as the primary mechanisms for growth in adolescent delinquent behavior and early arrests. Multiple-method assessments of 238 mothers and sons include delinquency as measured by teacher reports and official arrest records, parenting skills measured by observations of parent-child interactions, and deviant peer association as reported by focal boys. Analyses of the 9-year follow-up data indicate that the Oregon model of parent management training significantly reduced teacher-reported delinquency and police arrests for focal boys. As hypothesized, the experiments demonstrated that improving parenting practices and reducing contacts with deviant peers served as mediating mechanisms for reducing rates of adolescent delinquency. As predicted, there was also a significant delay in the timing of police arrests for youth in the experimental as compared to the control group.

Teaching GenerationPMTO, an evidence-based parent intervention, in a university setting using a blended learning strategy.

BACKGROUND: Despite the large number of evidence-based practices (EBPs) ready for implementation, they are the exception in usual care, especially for ethnic minority patients, who may not have access to trained health professionals. Providing EBP training as part of a graduate curriculum could help build the pipeline of professionals to provide quality care. METHODS: We conducted a before-after study to determine whether we could implement a blended learning strategy (BL; i.e., in vivo and online training) to teach an EBP in university settings. Feasibility in this pilot was operationalized as knowledge acquisition, satisfaction, fidelity, acceptability, and usability. Using GenerationPMTO as the EBP, our aim was to train graduate students enrolled in Psychology, Social Work, and Family Therapy programs in the EBP in one academic year. Two therapists from a community agency were also students in this pilot. A total of 13 students from five universities were trained in the intervention. Adaptations were made to the intervention and training strategy to optimize training fidelity. Focus groups were conducted with the students to capture their perspective about the training. RESULTS: Students demonstrated significant knowledge acquisition from baseline (Mean = 61.79, SD = 11.18) to training completion (Mean = 85.27, SD = 5.08, mean difference = - 23.48, 95% CI = - 29.62, - 17.34). They also reported satisfaction with the BL format, as measured by teaching evaluations at the end of the course. Instructors received acceptable fidelity scores (range of 7-9 in a 9-point scale). Qualitative findings from focus groups showed support for acceptability and usability of BL training. CONCLUSIONS: BL training in university settings can be conducted with fidelity when provided by appropriately trained instructors. BL that integrates EBP and adaptations may be uniquely applicable for training providers in low-resource and ethnically diverse settings. The BL enhanced knowledge of GenerationPMTO was acceptable and usable to students, and was delivered with high instructor fidelity to the training model.

Evaluating fidelity: predictive validity for a measure of competent adherence to the Oregon model of parent management training.

When efficacious interventions are implemented in real-world conditions, it is important to evaluate whether or not the programs are practiced as intended. This article presents the Fidelity of Implementation Rating System (FIMP), an observation-based measure assessing competent adherence to the Oregon model of Parent Management Training (PMTO). FIMP evaluates 5 dimensions of competent adherence to PMTO (i.e., knowledge, structure, teaching skill, clinical skill, and overall effectiveness) specified in the intervention model. Predictive validity for FIMP was evaluated with a subsample of stepfamilies participating in a preventive PMTO intervention. As hypothesized, high FIMP ratings predicted change in observed parenting practices from baseline to 12 months. The rigor and scope of adherence measures are discussed.

An efficacious theory-based intervention for stepfamilies.

This article evaluates the efficacy of the Oregon model of Parent Management Training (PMTO) in the stepfamily context. Sixty-seven of 110 participants in the Marriage and Parenting in Stepfamilies (MAPS) program received a PMTO-based intervention. Participants in the randomly assigned experimental group displayed a large effect in benefits to effective parenting practices with resultant decreases in child noncompliance and in home and school problem behaviors. MAPS findings replicate and extend those of the Oregon Divorce Study with single mothers while employing a similar multiple-method, multiple-measure randomized design, with Intent-to-Treat and structural equation modeling analyses. We discuss direct and indirect effects on distal outcomes and the generalizability of PMTO to differing family contexts.

Systematic changes in families following prevention trials.

A selective prevention design was applied to 238 recently separated families. Of these, 153 mothers randomly assigned to the experimental (E) group participated in 14 group sessions focused on Parent Management Treatment (PMT). Prior analyses showed that, over time, the group of families in the untreated group deteriorated in both parenting practices and in child outcomes. In keeping with the classic prevention pattern, families in the E group showed modest improvements in parenting and in child outcomes. Improvements in parenting were associated with significant reductions in problem behavior. The data showed that those mothers who improved their parenting skills during the first 12 months also showed significant reductions in maternal depression during that same interval. A cross-lagged panel analysis showed that a reduction in maternal depression during the first year of the study was a significant predictor of maintenance or improvements over the next 18 months. The findings are consistent with the concept of the family as a system.

Adjusting to change: linking family structure transitions with parenting and boys' adjustment.

This study examined links between family structure transitions and children's academic, behavioral, and emotional outcomes in a sample of 238 divorcing mothers and their sons in Grades 1-3. Multiple methods and agents were used in assessing family process variables and child outcomes. Findings suggest that greater accumulations of family transitions were associated with poorer academic functioning, greater acting-out behavior, and worse emotional adjustment for boys. However, in all three cases, these relationships were mediated by parenting practices: Parental academic skill encouragement mediated the relationship between transitions and academic functioning, and a factor of more general effective parenting practices mediated the relationships between transitions and acting out and emotional adjustment.