Pragmatic fidelity measurement in youth service settings.

Background: Fidelity measurement is critical for developing, evaluating, and implementing evidence-based treatments (EBTs). However, traditional fidelity measurement tools are often not feasible for community-based settings. We developed a short fidelity rating form for the Collaborative Problem Solving (CPS) approach from an existing manualized coding system that requires extensive training. We examined the reliability and accuracy of this short form when completed by trained observers, untrained observers, and self-reporting providers to evaluate multiple options for reducing barriers to fidelity measurement in community-based settings. Methods: Community-based treatment providers submitted recordings of youth service sessions in which they did, or did not, use CPS. For 60 recordings, we compared short-form fidelity ratings assigned by trained observers and untrained observers to those provided by trained observers on the manualized coding system. For 141 recordings, we compared providers' self-reported fidelity on the short form to ratings provided by trained observers on the manualized coding system and examined providers' accuracy as a function of their global fidelity. Results & Conclusions: The short form was reliable and accurate for trained observers. An assigned global integrity score and a calculated average of component scores on the short form, but not component scores themselves, were reliable and accurate for observers who had CPS expertise but no specific training on rating CPS fidelity. When providers self-reported fidelity on the short form, their global integrity score was a reliable estimate of their CPS integrity; however, providers with better CPS fidelity were most accurate in their self-reports. We discuss the costs and benefits of these more pragmatic fidelity measurement options in community-based settings.

Developing brief, easy-to-use, and reliable tools to measure how well providers deliver evidence-based treatments (EBTs) in community clinical settings is critical to ensure the benefits of EBTs. However, reliable tools are often too time-consuming and not feasible to use in community settings because they require independent observers to receive intensive training on a coding system and to observe live or recorded treatment sessions for reliable and accurate evaluation. This paper describes steps we took to develop a more practical measure of how well providers deliver one EBT, Collaborative Problem Solving (CPS), based on a previously validated measure, to explore whether the quality of the measure can be maintained while reducing the need for training independent observers and the need for recording treatment sessions. This work contributes to the growing efforts of developing more pragmatic fidelity measures and introduces a new tool, the CPS Practice Integrity Form (CPS-PIF), as a promising measure for community-based clinical settings using CPS.

Efficient Monitoring of Treatment Response during Youth Psychotherapy: The Behavior and Feelings Survey.

An emerging trend in youth psychotherapy is measurement-based care (MBC): treatment guided by frequent measurement of client response, with ongoing feedback to the treating clinician. MBC is especially needed for treatment that addresses internalizing and externalizing problems, which are common among treatment-seeking youths. A very brief measure is needed, for frequent administration, generating both youth- and caregiver-reports, meeting psychometric standards, and available at no cost. We developed such a measure to monitor youth response during psychotherapy for internalizing and externalizing problems. Across 4 studies, we used ethnically diverse, clinically relevant samples of caregivers and youths ages 7-15 to develop and test the Behavior and Feelings Survey (BFS). In Study 1, candidate items identified by outpatient youths and their caregivers were examined via an MTurk survey, with item response theory methods used to eliminate misfitting items. Studies 2-4 used separate clinical samples of youths and their caregivers to finalize the 12-item BFS (6 internalizing and 6 externalizing items), examine its psychometric properties, and assess its performance in monitoring progress during psychotherapy. The BFS showed robust factor structure, internal consistency, test-retest reliability, convergent and discriminant validity in relation to three well-established symptom measures, and slopes of change indicating efficacy in monitoring treatment progress during therapy. The BFS is a brief, free youth- and caregiver-report measure of internalizing and externalizing problems, with psychometric evidence supporting its use for MBC in clinical and research contexts.

Parental Autonomy Support in Two Cultures: The Moderating Effects of Adolescents' Self-Construals.

Parental autonomy support has been related to positive adolescent outcomes, however, its relation to outcomes in collectivist cultural groups is unclear. This study examined relations of specific autonomy supportive behaviors and outcomes among 401 adolescents (Mage  = 12.87) from the United States (N = 245) and collectivist-oriented Ghana (N = 156). It also examined whether adolescents' self-construals moderated the relations of specific types of autonomy support with outcomes. Factor analyses indicated two types of autonomy support: perspective taking/open exchange and allowance of decision making/choice. In both countries, perspective taking/open exchange predicted positive outcomes, but decision making/choice only did so in the United States. With regard to moderation, the more independent adolescents' self-construals, the stronger the relations of decision making/choice to parental controllingness and school engagement.

When the torch is passed, does the flame still burn? Testing a "train the supervisor" model for the Child STEPs treatment program.

OBJECTIVE: We assessed sustainability of an empirically supported, transdiagnostic youth psychotherapy program when therapist supervision was shifted from external experts to internal clinic staff. METHOD: One hundred sixty-eight youths, aged 6-15 years, 59.5% male, 85.1% Caucasian, were treated for anxiety, depression, traumatic stress, or conduct problems by clinicians employed in community mental health clinics. In Phase 1 (2.7 years), 1 group of clinicians, the Sustain group, received training in Child STEPs (a modular transdiagnostic treatment + weekly feedback on youth response) and treated clinic-referred youths, guided by weekly supervision from external STEPs experts. In Phase 2 (2.9 years), Sustain clinicians treated additional youths but with supervision by clinic staff who had been trained to supervise STEPs. Also in Phase 2, a new group, External Supervision clinicians, received training and supervision from external STEPs experts and treated referred youths. Phase 2 youths were randomized to Sustain or External Supervision clinicians. Groups were compared on 3 therapist fidelity measures and 14 clinical outcome measures. RESULTS: Sustain clinicians maintained their previous levels of fidelity and youth outcomes after switching from external to internal supervision; and in Phase 2, the Sustain and External Supervision groups also did not differ on fidelity or youth outcomes. Whereas all 34 group comparisons were nonsignificant, trends with the largest effect sizes showed better clinical outcomes for internal than external supervision. CONCLUSIONS: Implementation of empirically supported transdiagnostic treatment may be sustained when supervision is transferred from external experts to trained clinic staff, potentially enhancing cost-effectiveness and staying power in clinical practice. (PsycINFO Database Record