The Content of Usual Clinical Care for Youth with Primary Anxiety Problems.

The current study was designed to describe usual clinical care for youth with primary anxiety problems in community mental health centers. The observer-rated Therapy Process Observational Coding System for Child Psychotherapy - Revised Strategies scale (TPOCS-RS), designed to assess therapeutic techniques from five theory-based domains, was used to code sessions (N = 403) from the usual clinical care group of two randomized effectiveness trials: (a) Youth Anxiety Study (YAS) with 21 youth (M age = 10.44 years, SD = 1.91; 49.2% Latinx; 46.6%, 53.4% male) and 16 clinicians (77.5% female; 43.8% White), and (b) Child STEPS Multisite Trial with 17 youth (M age = 10.00 years, SD = 1.87; 58.8% male; 41.2% White) and 13 clinicians (M age = 40.00 years; SD = 9.18; 76.9% female; 61.5% White). The average number of TPOCS-RS items observed per treatment session was more than 10, and multiple techniques were used together in each session. All TPOCS-RS items were observed at least once throughout a clinical case, and most items reoccurred (i.e., observed in two or more sessions). The dosage of TPOCS-RS in all items was below 5 on a 7-point scale. In conclusion, clinicians in both usual care samples used a wide range of techniques from several theory-based domains at a low to medium dose. However, the type and dosage of the techniques used did vary across the two samples.

Core Elements of Family Therapy for Adolescent Behavioral Health Problems: Validity Generalization in Community Settings.

OBJECTIVE: The core elements of family therapy for adolescent mental health and substance use problems, originally distilled from high-fidelity sessions conducted by expert clinicians, were tested for validity generalization when delivered by community therapists in routine settings. METHOD: The study sampled recorded sessions from 161 cases participating in one of three treatment pools: implementation trial of Functional Family Therapy (98 sessions/50 cases/22 therapists), adaptation trial of Multisystemic Therapy (115 sessions/59 cases/2 therapists), and naturalistic trial of non-manualized family therapy in usual care (107 sessions/52 cases/21 therapists). Adolescents were identified as 60% male and 40% female with an average age of 15.4 years; 49% were Latinx, 27% White Non-Latinx, 15% African American, 3% another race/ethnicity, 6% race/ethnicity unknown. Session recordings (n = 320) were randomly selected for each case and coded for 21 discrete family therapy techniques. Archived data of one-year clinical outcomes were gathered. RESULTS: Confirmatory factor analyses replicated the factor structure from the original distillation study, retaining all four clinically coherent treatment modules comprised of all 21 techniques: Interactional Change (ICC = .77, Cronbach's α = .81); Relational Reframe (ICC = .75, α = .81); Adolescent Engagement (ICC = .72, α = .78); Relational Emphasis (ICC = .76, α = .80). Exploratory analyses found that greater use of core techniques predicted symptom improvements in one treatment pool. CONCLUSIONS: Core techniques of family therapy distilled from manualized treatments for adolescent behavioral health problems showed strong evidence of validity generalization, and initial evidence of links to client outcomes, in community settings.

Development and Initial Psychometrics of a Generic Treatment Integrity Measure Designed to Assess Practice Elements Targeting Social, Emotional, and Behavioral Outcomes in Early Childhood Settings.

Though treatment integrity measurement is important for research intended to promote social and behavioral outcomes of children at risk for emotional and behavioral disorders (EBDs) in early childhood settings, measurement gaps exist in the field. This paper reports on the development and preliminary psychometric assessment of the treatment integrity measure for early childhood settings (TIMECS), an observational measure designed to address existing measurement gaps related to treatment integrity with tier 2 interventions in the early childhood field. To assess the preliminary score reliability (interrater) and validity (construct, discriminant) of the TIMECS, live observations (N = 650) in early childhood classrooms from 54 teachers (92.6% female, 7.4% male; 61.1% White) and 91 children (M age = 4.53 years, SD = .44; 45.1% female, 54.9% male; 45.1% Black) at risk for EBDs were scored by 12 coders using the TIMECS and an observational measure designed to assess teacher-child interactions. Teachers also self-reported on the quality of the teacher-child relationship. Interrater reliability (intraclass correlation coefficients, ICC [2,2]) for the quantity (i.e., adherence) item scores had a mean of .81 (SD = .07; range from .68 to .95), and the quality (i.e., competence) item scores had a mean of .69 (SD = .08; range from .52 to .80). Scores on the TIMECS Quantity and Quality items and scales showed evidence of construct validity, with the magnitude of the correlations suggesting that the quantity and quality items assess distinct components of treatment integrity. A TIMECS quantity scale also showed promise for intervention evaluation research by discriminating between teachers who had and had not been trained in a specific evidence-based intervention targeting social and behavioral skills in early childhood. The findings support the potential of the TIMECS to assess treatment integrity of teacher-delivered practices designed to address child social and behavioral outcomes of children at risk for EBDs in early childhood settings.

Assessing Stakeholder Perceptions of the Utility of Genetic Information for the Clinical Care of Mental Health Disorders: We Have a Will but Need to See the Way.

Academic stakeholders' (primarily mental health researchers and clinicians) practices and attitudes related to the translation of genetic information into mental health care were assessed. A three-part survey was administered at two large, urban universities. Response frequencies were calculated. Participants (N = 64) reported moderate levels of translational practice, adequate levels of genetic knowledge, and variable levels of genetic competence. They held positive attitudes toward translating genetic information about mental health broadly but negative attitudes about the impact that such information would have on specific aspects of care. The current study lays the groundwork for further inquiry into translating genetic information to mental health care.

Testing Robustness of Child STEPs Effects with Children and Adolescents: A Randomized Controlled Effectiveness Trial.

A critical task in psychotherapy research is identifying the conditions within which treatment benefits can be replicated and outside of which those benefits are reduced. We tested the robustness of beneficial effects found in two previous trials of the modular Child STEPs treatment program for youth anxiety, depression, trauma, and conduct problems. We conducted a randomized trial, with two significant methodological changes from previous trials: (a) shifting from cluster- to person-level randomization, and (b) shifting from individual to more clinically feasible group-based consultation with STEPs therapists. Fifty community clinicians from multiple outpatient clinics were randomly assigned to receive training and consultation in STEPs (n= 25) or to provide usual care (UC; n= 25). There were 156 referred youths-ages 6-16 (M= 10.52, SD = 2.53); 48.1% male; 79.5% Caucasian, 12.8% multiracial, 4.5% Black, 1.9% Latino, 1.3% Other-who were randomized to STEPs (n= 77) or UC (n= 79). Following previous STEPs trials, outcome measures included parent- and youth-reported internalizing, externalizing, total, and idiographic top problems, with repeated measures collected weekly during treatment and longer term over 2 years. Participants in both groups showed statistically significant improvement on all measures, leading to clinically meaningful problem reductions. However, in contrast to previous trials, STEPs was not superior to UC on any measure. As with virtually all treatments, the benefits of STEPs may depend on the conditions-for example, of study design and implementation support-in which it is tested. Identifying those conditions may help guide appropriate use of STEPs, and other treatments, in the future.

Benchmarking Treatment Adherence and Therapist Competence in Individual Cognitive-Behavioral Treatment for Youth Anxiety Disorders.

Evidence-based treatments (EBTs) for youth are typically developed and established through studies in research settings designed to ensure treatment integrity, that is, protocol adherence and competence by therapists. An important question for implementation science is how well integrity is maintained when these EBTs are delivered in community settings. The present study investigated whether the integrity achieved by therapists in community settings achieved a benchmark set by therapists in a research setting when they delivered the same EBT-an individual cognitive-behavioral treatment (ICBT) for youth anxiety. Therapists (N = 29; 68.97% White; 13.79% male) provided ICBT to 68 youths (M age = 10.60 years, SD = 2.03; 82.35% White; 52.94% male) diagnosed with a principal anxiety disorder in research or community settings. Training and supervision protocols were the same across settings. Two independent teams of trained coders rated 744 sessions using observational instruments designed to assess ICBT adherence and competence. Both adherence and competence were higher in the research setting. Group differences in competence were consistent across treatment, but differences in adherence were most pronounced when treatment shifted to exposure, widely viewed as the most critical component of ICBT. When using the benchmarks from the research setting, therapists from the community settings fell short for indices of adherence and competence. However, given differences between therapists and clients, as well as the fact that treatment outcomes were similar across settings, our findings raise questions about whether it is appropriate to use treatment integrity benchmarks from research settings for community.

Core Elements of Family Therapy for Adolescent Behavior Problems: Empirical Distillation of Three Manualized Treatments.

Family therapy has the strongest evidence base for treating adolescent conduct and substance use problems, yet there remain substantial barriers to widespread delivery of this approach in community settings. This study aimed to promote the feasibility of implementing family-based interventions in usual care by empirically distilling the core practice elements of three manualized treatments. The study sampled 302 high-fidelity treatment sessions from 196 cases enrolled in 1 of 3 manualized family therapy models: multidimensional family therapy (102 sessions/56 cases), brief strategic family therapy (100 sessions/94 cases), or functional family therapy (100 sessions/46 cases). Adolescents were 57% male; 41% were African American, 31% White non-Hispanic, 9% Hispanic American, 6% another race/ethnicity, and 13% unknown. The observational fidelity measures of all three models were used to code all 302 sessions. Fidelity ratings were analyzed to derive model-shared treatment techniques via exploratory factor analyses on half the sample; the derived factors were then validated via confirmatory factor analyses supplemented by Bayesian structural equation modeling on the remaining half. Factor analyses distilled 4 clinically coherent practice elements with strong internal consistency: Interactional Change (6 treatment techniques; Cronbach's α = .93), Relational Reframe (7 techniques; α = .79), Adolescent Engagement (4 techniques; α = .68), and Relational Emphasis (4 techniques; α = .67). The 4 empirically derived factors represent the core elements of 3 manualized family therapy models for adolescent behavior problems, setting the foundation of a more sustainable option for delivering evidence-based family interventions in routine practice settings. Public Health Significance: Increasing implementation of high-fidelity family-based interventions would improve the quality of treatment services for adolescent conduct and substance use problems.

Development and Initial Psychometrics for a Therapist Competence Instrument for CBT for Youth Anxiety.

Therapist competence is an important component of treatment integrity. This article reports on the development and initial psychometric assessment of the Cognitive-Behavioral Treatment for Anxiety in Youth Competence Scale (CBAY-C), an observational instrument designed to capture therapist limited-domain competence (i.e., competence in the delivery of core interventions and delivery methods found in a specific psychosocial treatment program) in the delivery of the core practice elements in individual cognitive-behavioral treatment (ICBT) for youth anxiety. Treatment sessions (N = 744) from 68 youth participants (M age = 10.60 years, SD = 2.03; 82.3% Caucasian; 52.9% male) of the same ICBT program for youth anxiety from (a) an efficacy study and (b) an effectiveness study were independently scored by 4 coders using observational instruments designed to assess therapist competence, treatment adherence, treatment differentiation, alliance, and client involvement. Interrater reliability-intraclass correlation coefficients (2,2)-for the item scores averaged 0.69 (SD = 0.11). The CBAY-C item, scale, and subscale (Skills, Exposure) scores showed evidence of validity via associations with observational instruments of treatment adherence to ICBT for youth anxiety, theory-based domains (cognitive-behavioral treatment, psychodynamic, family, client centered), alliance, and client involvement. Important to note, although the CBAY-C scale, subscale, and item scores did overlap with a corresponding observational treatment adherence instrument independently rated by coders, the degree of overlap was moderate, indicating that the CBAY-C assesses a distinct component of treatment integrity. Applications of the instrument and future research directions discussed include the measurement of treatment integrity and testing integrity-outcome relations.

The global therapist competence scale for youth psychosocial treatment: Development and initial validation.

OBJECTIVE: We describe the development and initial psychometric properties of the observer-rated Global Therapist Competence Scale for Youth Psychosocial Treatment (G-COMP) in the context of cognitive-behavioral treatment (CBT) for youth anxiety disorders. METHOD: Independent coders rated 744 sessions from a sample of 68 youth (mean age = 10.56 years) using the G-COMP and the instruments of alliance, involvement, CBT adherence, CBT competence. RESULTS: Inter-rater reliability coefficients, ICC(2,2), were greater than .60 for the 5 G-COMP domain scores. G-COMP scores yielded small to medium correlations with instruments of alliance (rs = .17-.44) and youth involvement in treatment (rs = .08-.53), and medium to large correlations with instruments of CBT competence and adherence (rs = .26-.63). Therapists in the research setting were rated higher compared to newly trained therapists in community clinics. CONCLUSION: Preliminary reliability and validity of the G-COMP are promising, but future research is needed with non-CBT samples.

Identifying Common Practice Elements to Improve Social, Emotional, and Behavioral Outcomes of Young Children in Early Childhood Classrooms.

Educators are increasingly being encouraged to implement evidence-based interventions and practices to address the social, emotional, and behavioral needs of young children who exhibit problem behavior in early childhood settings. Given the nature of social-emotional learning during the early childhood years and the lack of a common set of core evidence-based practices within the early childhood literature, selection of instructional practices that foster positive social, emotional, and behavioral outcomes for children in early childhood settings can be difficult. The purpose of this paper is to report findings from a study designed to identify common practice elements found in comprehensive intervention models (i.e., manualized interventions that include a number of components) or discrete practices (i.e., a specific behavior or action) designed to target social, emotional, and behavioral learning of young children who exhibit problem behavior. We conducted a systematic review of early childhood classroom interventions that had been evaluated in randomized group designs, quasi-experimental designs, and single-case experimental designs. A total of 49 published articles were identified, and an iterative process was used to identify common practice elements. The practice elements were subsequently reviewed by experts in social-emotional and behavioral interventions for young children. Twenty-four practice elements were identified and classified into content (the goal or general principle that guides a practice element) and delivery (the way in which a teacher provides instruction to the child) categories. We discuss implications that the identification of these practice elements found in the early childhood literature has for efforts to implement models and practices.

Distilling the Core Elements of Family Therapy for Adolescent Substance Use: Conceptual and Empirical Solutions.

This article describes several barriers to widespread dissemination of manualized family-based treatments for adolescent substance use (ASU). We then offer a highly promising solution for adopting and sustaining family therapy in usual care: distilling the core practice elements of empirically validated family therapy models for ASU. We present a conceptual distillation of family therapy for ASU grounded in existing observational fidelity measures for three manualized models, a process that yielded four core elements: Family Engagement, Relational Reframing, Family Behavior Change, and Family Restructuring. We then introduce an innovative empirical method for distilling core elements that can serve as a template for rigorous distillation of other treatment approaches. Finally, we discuss how core elements can enhance family therapy services within the diverse workforce of usual care for ASU.

Emotional and behavior problems in an urban pediatric primary care setting.

We sought to identify needs for behavioral health care in a large, urban pediatric primary care (PPC) clinic serving a population covered by Medicaid. Specifically, children (N = 197; 120 girls; 91 % African American) ages 8-17 years and their caregivers completed measures of internalizing and externalizing symptoms (RCADS, RCADS-P, MASC, and SDQ). Clinical elevations on all but one domain of the SDQ were significantly higher than expected. However, self-reported anxiety and depression symptoms were consistent with expectations. These findings suggest urban, low-income, primarily African American youth presenting at a PPC clinic demonstrate significant levels of behavioral and emotional symptoms. Implications of the findings include the need to ask both parents and children about child behavioral health problems and the possible influence of screening tool selection on detection.

Sustained Implementation of Cognitive-Behavioral Therapy for Youth Anxiety and Depression: Long-term Effects of Structured Training and Consultation on Therapist Practice in the Field.

Identifying factors that promote sustained implementation of evidence-based treatments (EBTs) after therapists receive training is critical for professional psychology. To address the field's minimal knowledge in this area, we interviewed community-based therapists (N = 23) who had completed intensive training in cognitive behavioral therapy (CBT) for either anxiety or depression as part of a randomized effectiveness trial (Southam-Gerow et al., 2010; Weisz et al., 2009). Therapists were interviewed three to five years after completion of the initial trial, representing one of the longest-term follow-ups of therapist practices after training. Therapists viewed each protocol and their individual CBT strategies as effective and appropriate for the majority of their current anxiety and depression caseloads. However, therapists used parts of each protocol much more frequently than the protocol as a whole (i.e., 78.5% used parts of the Coping Cat, and 7.5% used the whole protocol; 58.6% used parts of the PASCET, and 20% used the whole protocol). Therapists reported using problem-solving the most and exposure exercises the least for current anxious cases; they used cognitive restructuring the most and homework the least for current depression cases. Interventions that were more difficult to implement in usual care settings were less likely to be sustained. Future efforts should evaluate the characteristics and structure of EBTs that are most acceptable to therapists and should investigate which kinds of ongoing learning supports will maintain therapist skills in and continued use of EBTs.

Qualitative and mixed methods research in dissemination and implementation science: introduction to the special issue.

This special issue provides examples of how qualitative and mixed methods research approaches can be used in dissemination and implementation science. In this introductory article, we provide a brief rationale for why and how qualitative and mixed methods approaches can be useful in moving the field forward. Specifically, we provide a brief primer on common qualitative methods, including a review of guidelines provided by the National Institutes of Health. Next, we introduce the six articles in the issue. The first of the articles by Palinkas represents a more thorough and authoritative discussion related to qualitative methods, using the other five articles in the issue (and other published works) as examples. The remaining five articles are empirical and/or descriptive articles of recently completed or ongoing qualitative or mixed methods studies related to dissemination and implementation of evidence-based practices for children and adolescents.

Evidence base updates: the evolution of the evaluation of psychological treatments for children and adolescents.

This article introduces a new feature in the journal: Evidence Base Updates. Starting with this issue of the journal, there will be one such update in each issue. The updates will focus on reviewing the treatments studies focused on a specific child/adolescent problem area and identifying those treatments with the strongest evidence base. Updates will use a revised set of criteria for evaluating the evidence, based on past work, but modified to emphasize methodological rigor of studies and designed to identify those treatments with questionable efficacy. The article also places the evaluation of psychological treatments in historical context. As well, the changes made in the criteria are discussed.

An analysis of stakeholder views on children's mental health services.

The purpose was to examine the perspectives of mental health stakeholders as a means to guide the adaptation of evidence-based treatments. The Mental Health System Ecological (MHSE) model was used to organize therapist, administrator, and parent perspectives gathered using qualitative methods. The MHSE model posits the influences of client-level, provider-level, intervention-specific, service delivery, organizational, and service system characteristics on implementation. Focus groups and interviews were conducted with community mental health stakeholders and included parents, therapists, and administrators (N = 21). Participants included 11 primarily Caucasian (90.48%) female participants, ranging in ages between 31 and 57 years. Data were analyzed according to the MHSE model. Frequency counts were tabulated for each theme and stakeholder group differences were determined using the Mann-Whitney test. Stakeholder groups mentioned needs at each level of the MHSE model. Stakeholder group differences most notably emerged with child and family themes, which included complexity of mental health issues, parenting differences, and family stressors. Stakeholders identified challenges for optimal mental health services for children across multiple levels of an ecological model. Implications of the findings are discussed, including the continued relevance of adapting mental health interventions by increasing their flexible application across multiple target problems and the promise of partnership approaches.

MAPping Los Angeles County: taking an evidence-informed model of mental health care to scale.

We describe the scaling up of an evidence-informed model of care, Managing and Adapting Practice (MAP) in Los Angeles County, California. MAP complemented an array of evidence-based programs selected by the county as part of a large system reform effort designed to improve care for children and adolescents. In addition, we discuss the MAP model for training therapists and present data both on how the training model performed and on the outcomes of youths treated by therapists trained in MAP. We examined the success of two different training pathways for MAP therapists: (a) national training model and (b) MAP agency supervisor model (i.e., train the trainer). We also examined utilization of MAP and outcomes of clients served by MAP. Both the national training and MAP agency supervisor model were successful in producing MAP therapists in a timely fashion and with acceptable competency scores. Furthermore, a large number of clients were receiving MAP services. Finally, outcomes for youth treated with MAP were strong, with effect sizes ranging from .59 to .80 on the Youth Outcome Questionnaire. These data support the notion that scaling up a mental health services approach in a system can be achieved through a strong and broad partnership among relevant stakeholders, can involve a train-the-trainer model, and can result in strong outcomes for clients.

Acting Locally and Globally: Dissemination and Implementation Around the World and Next Door.

Murray et al. (this issue) present a fascinating account of their international dissemination and implementation (D&I) research focused on training therapists in Thailand and Iraq to provide a modular treatment approach called Common Elements Treatment Approach to youth. In this commentary, we use Murray et al. as a springboard to discuss a few general conclusions about the current direction of D&I research. Specifically, we reflect on current D&I models, highlighting their ecological focus and their emphasis on stakeholder involvement. Next, we discuss the central importance of implementation supports such as treatment programs, training approaches, assessment and outcome monitoring tools, and organizational interventions. We conclude with a consideration of how D&I work that aims to adapt implementation supports for local needs represent a key path to our goal of sustainability.

Using Adherence and Competence Measures Based on Practice Elements to Evaluate Treatment Fidelity for Two CBT Programs for Youth Anxiety.

Measures designed to assess the quantity and quality of practices found across treatment programs for specific youth emotional or behavioral problems may be a good fit for evaluating treatment fidelity in effectiveness and implementation research. Treatment fidelity measures must demonstrate certain reliability and validity characteristics to realize this potential. This study examines the extent to which two observational measures, the Cognitive-Behavioral Treatment for Anxiety in Youth Adherence Scale (CBAY-A) and the CBAY Competence Scale (CBAY-C), can assess the quantity (the degree to which prescribed therapeutic techniques are delivered as intended) or quality (the competence with which prescribed techniques are delivered) of practices found in two distinct treatment programs for youth anxiety. Treatment sessions (N = 796) from 55 youth participants (M age = 9.89 years, SD = 1.71; 46% female; 55% White) with primary anxiety problems who participated in an effectiveness study were independently coded by raters who coded quantity, quality, and the youth-clinician alliance. Youth received one of three treatments: (a) standard (i.e., cognitive-behavioral therapy program), (b) modular (i.e., a cognitive-behavioral and parent-training program), and (c) usual clinical care. Interrater reliability for the CBAY-A items was good across the standard and modular conditions but mixed for the CBAY-C items. Across the standard and modular conditions, the CBAY-A Model subscale scores demonstrated evidence of construct validity, but the CBAY-C Model subscale scores showed mixed evidence. The results provide preliminary evidence that the CBAY-A can be used across different treatment programs but raise concerns about the generalizability of the CBAY-C.

Science for behavioral health systems change: evolving research-policy-public partnerships.

Potent partnerships among researchers, policymakers, and community members have potential to produce positive changes in communities on a range of topics, including behavioral health. The paper provides a brief illustrative review of such partnerships and then describes the development and evolution of one partnership in particular in Virginia. The origin of the partnership is traced, along with its founding vision, mission, and values. Some of its several projects are described, including (a) needs assessment for implementation of evidence-based programs (EBPs) pursuant to the Family First Prevention Services Act; (b) statewide fidelity monitoring of key EBPs; and (c) projects to synergize state investments in specific EBPs, like multisystemic therapy, functional family therapy, and high fidelity wraparound. The paper concludes with some themes around which the center has evolved to serve the state and its citizens more effectively.