Concordance between clinician, supervisor and observer ratings of therapeutic competence in CBT and treatment as usual: does clinician competence or supervisor session observation improve agreement?

BACKGROUND: Lowering the cost of assessing clinicians' competence could promote the scalability of evidence-based treatments such as cognitive behavioral therapy (CBT). AIMS: This study examined the concordance between clinicians', supervisors' and independent observers' session-specific ratings of clinician competence in school-based CBT and treatment as usual (TAU). It also investigated the association between clinician competence and supervisory session observation and rater agreement. METHOD: Fifty-nine school-based clinicians (90% female, 73% Caucasian) were randomly assigned to implement TAU or modular CBT for youth anxiety. Clinicians rated their confidence after each therapy session (n = 1898), and supervisors rated clinicians' competence after each supervision session (n = 613). Independent observers rated clinicians' competence from audio recordings (n = 395). RESULTS: Patterns of rater discrepancies differed between the TAU and CBT groups. Correlations with independent raters were low across groups. Clinician competence and session observation were associated with higher agreement among TAU, but not CBT, supervisors and clinicians. CONCLUSIONS: These results support the gold standard practice of obtaining independent ratings of adherence and competence in implementation contexts. Further development of measures and/or rater training methods for clinicians and supervisors is needed.

Community Study of Outcome Monitoring for Emotional Disorders in Teens (COMET): A comparative effectiveness trial of a transdiagnostic treatment and a measurement feedback system.

Emotional disorders, encompassing a range of anxiety and depressive disorders, are the most prevalent and comorbid psychiatric disorders in adolescence. Unfortunately, evidence-based psychosocial therapies typically focus on single disorders, are rarely adopted by community mental health center clinicians, and effect sizes are modest. This article describes the protocol for a comparative effectiveness study of two novel interventions designed to address these challenges. The first intervention is a transdiagnostic treatment (the Unified Protocol for Transdiagnostic Treatment of Emotional Disorders in Adolescents, UP-A), a promising new approach that uses a small number of common strategies to treat a broad range of emotional disorders, and their underlying shared emotional vulnerabilities. The second intervention is a standardized measurement feedback system, the Youth Outcomes Questionnaire (YOQ), designed to improve clinical decision making using weekly symptom and relational data. The three study arms are treatment as usual (TAU), TAU plus the YOQ (TAU+), and UP-A (used in combination with the YOQ). The primary aims of the study are to [1] compare the effects of the UP-A and TAU+ to TAU in community mental health clinics, [2] to isolate the effects of measurement and feedback by comparing the UP-A and TAU+ condition, and [3] to examine the mechanisms of action of both interventions. Design considerations and study methods are provided to inform future effectiveness research.