Therapist-Level Moderators of Patient-Therapist Match Effectiveness in Community Psychotherapy.

Based on patient-reported outcomes data analyzed at the provider level, there is evidence that psychotherapists can possess effectiveness strengths and weaknesses when treating patients with different presenting concerns. These within-therapist differences hold promise for personalizing care by prospectively matching patients to therapists' historical effectiveness strengths. In a double-masked randomized controlled trial (RCT; NCT02990000), such matching outperformed pragmatically determined usual case assignment-which leaves personalized, measurement-based matching to chance-in naturalistic outpatient psychotherapy (Constantino et al., JAMA Psychiatry 78:960-969, 2021). Demonstrating that personalization can be even more precise, some research has demonstrated that the strength of this positive match effect was moderated by certain patient characteristics. Notably, though, it could also be that matching is especially important for some therapists to achieve more effective outcomes. Examining this novel question, the present study drew on the Constantino et al. (JAMA Psychiatry 78:960-969, 2021) trial data to explore three therapist-level moderators of matching: (a) effectiveness "spread" (i.e., greater performance variability across patients' presenting problem domains), (b) overestimation of their measurement-based and problem-specific effectiveness, and (c) the frequency with which they use patient-reported routine outcomes monitoring in their practice. Patients were 206 adults, randomized to the match or control condition, treated by 40 therapists who were crossed over conditions. The therapist variables were assessed at the trial's baseline and patients' symptomatic/functional impairment and global distress were assessed regularly up to 16 weeks of treatment. Hierarchical linear models revealed that only therapist effectiveness spread significantly moderated the match effect for the global distress outcome; for therapists with more spread, the match effect was more pronounced, whereas the match effect was minimal for therapists with less effectiveness spread. Notably, two therapist-level covariates unexpectedly emerged as significant moderators for the symptomatic/functional impairment outcome; for clinicians who consistently treated patients with higher versus lower average severity levels and who relatedly treated a higher proportion of patients with primary presenting problems of substance misuse or violence, the beneficial match effect was even stronger. Thus, measurement-based matching may be especially potent for therapists with more variable effectiveness across problem domains, and who consistently treat patients with more severe presenting concerns or with particular primary problems, which provides further precision in conceptualizing personalized care.

Predicting Successful Placements for Youth in Child Welfare with Machine Learning.

Out-of-home placement decisions have extremely high stakes for the present and future well-being of children in care because some placement types, and multiple placements, are associated with poor outcomes. We propose that a clinical decision support system (CDSS) using existing data about children and their previous placement success could inform future placement decision-making for their peers. The objective of this study was to test the feasibility of developing machine learning models to predict the best level of care placement (i.e., the placement with the highest likelihood of doing well in treatment) based on each youth's behavioral health needs and characteristics. We developed machine learning models to predict the probability of each youth's treatment success in psychiatric residential care (i.e., Psychiatric Residential Treatment Facility [PRTF]) versus any other placement (AUROCs > 0.70) using data collected in standard care at a behavioral health organization. Placement recommendations based on these machine learning models distinguished between youth who did well in residential care versus non-residential care (e.g., 80% of those who received care in the recommended setting with the highest predicted likelihood of success had above average risk-adjusted outcomes). Then we developed and validated machine learning models to predict the probability of each youth's treatment success across specific placement types in a state-wide system, achieving an average AUROC score of greater than 0.75. Machine learning models based on risk-adjusted behavioral health and functional data show promise in predicting positive placement outcomes and informing future placement decisions for youth in care. Related ethical considerations are discussed.

A Qualitative Analysis of Stakeholder Attitudes Regarding Personalized Provider Selection and Patient-Therapist Matching.

This study explored mental health care patients and therapists' perspectives on using therapists' measurement-based and problem-specific effectiveness data to inform case assignments - a type of treatment personalization that has been shown to outperform non-measurement-based case assignment as usual (Constantino et al., 2021). We conducted semi-structured qualitative interviews with 8 patients (75% women; M age = 33.75 years) and 8 therapists (75% women; M age = 47.50 years). The interview protocols were unique to stakeholder group. Recorded responses were transcribed and qualitatively analyzed by four judges using a blend of consensual qualitative research and grounded theory methods. Derived patient domains included preferred characteristics of a provider, and experiences and suggestions regarding provider selection. Within the domains, most patients expressed an interest in accessing more specific provider information online. Additionally, most patients indicated that both provider outcome track records and personal preference information (e.g., therapist characteristics) should be considered in the therapist selection process. All patients endorsed being comfortable with having the ability to select a provider based on a list of empirically well-matched recommendations. Derived therapist domains included using routine outcomes monitoring for patient-provider matching, referral source and direct patient use of preferred provider lists, and improvements to the provider selection process. Within the domains, all therapists remarked that outcome data would be useful for matching patients to providers; however, most also indicated that outcome data should not be the only factor used in provider selection. All therapists expressed a willingness to be included in preferred provider lists that incorporate track record data. Overall, both patients and therapists held generally positive views toward using therapist effectiveness data to help personalize mental health care. Yet, both stakeholder groups acknowledged that other personalization factors should be considered alongside these data. Based on these results, our team is in the process of implementing patient-therapist match strategies in larger and more diverse mental health care contexts.

Revisiting our approach to behavioral health referrals.

This system of matching referrals to behavioral health practitioners' historical effectiveness seeks to strengthen patient outcomes.

Therapist perceptions of their own measurement-based, problem-specific effectiveness.

OBJECTIVE: Patient-reported outcomes data reveal differences both in therapists' global effectiveness across their average patient (between-therapist effect) and in treating different problems within their caseload (within-therapist effects). Yet, it is unclear how accurately therapists perceive their own measurement-based, problem-specific effectiveness and whether such self-perceptions predict global between-therapist performance differences. We explored these questions in naturalistic psychotherapy. METHOD: For 50 therapists, we drew on data from a mean of 27 past patients (total N = 1,363) who completed a multidimensional outcome measure-Treatment Outcome Package (TOP)-at pre- and posttreatment. For each of 12 outcome domains (e.g., depression, anxiety), TOP data classified therapists as historically "effective," "neutral," or "ineffective." Unaware of their data-driven classifications, therapists rated their perceived effectiveness for each domain. We conducted chi-square analyses to determine whether therapists predicted their own measurement-based effectiveness classifications to a level greater than chance. We then used multilevel modeling to test whether therapists' problem-specific perceptions predicted global between-therapist performance differences. RESULTS: For all but one outcome domain, therapists were no better than chance at predicting their measurement-based effectiveness classification. Additionally, controlling for patient baseline impairment, therapists who consistently overestimated their problem-specific effectiveness had patients who reported worse global outcomes than patients whose therapist more accurately estimated their effectiveness. Conversely, therapists who underestimated their problem-specific effectiveness had patients who reported better outcomes than patients whose therapist over- or accurately estimated their effectiveness. CONCLUSIONS: Therapist humility may differentiate the most from least globally effective therapists, and this virtue should be cultivated in clinical trainings. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Patient-therapist expectancy convergence and outcome in naturalistic psychotherapy.

Research on close relationships demonstrates that dyadic convergence, or two people becoming more similar in their experiences and/or beliefs over time, is commonplace and adaptive. As psychotherapy involves a close relationship, patient-therapist convergence processes may influence treatment-specific outcomes. Although prior research supports that patients and therapists tend to converge on their alliance perspectives over time, which associates with subsequent patient improvement, no research has similarly examined belief convergence during therapy. Accordingly, this study focused on patient-therapist convergence in their outcome expectations (OE), a belief variable associated with patient improvement when measured from individual participant perspectives. We predicted both that significant OE convergence would occur and relate to better posttreatment outcome. Data derived from a trial of naturalistic psychotherapy. Patients and therapists repeatedly rated their respective OE through treatment, and patients rated their symptom/functional outcomes at posttreatment. For dyads with the requisite OE data (N = 154), we tested our questions using multilevel structural equation modeling. Counter to our hypotheses, there was no discernable OE convergence pattern over treatment (γ100 = 0.01, SE = 0.03, p = .690) and OE convergence was unrelated to outcome at the between-dyad level (γ020 = 2.37, SE = 10.28, p = .818). However, on its own, higher early patient OE was significantly associated with better outcome at the between-dyad level (γ050 = -0.04, SE = 0.01, p = .007). Results suggest that OE may be more of a facilitative patient versus relational process factor. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Therapist-level moderation of within- and between-therapist process-outcome associations.

OBJECTIVE: Although higher quality patient-therapist alliance and more positive patient outcome expectation (OE) consistently predict symptomatic/functional improvement in psychotherapy, most research has failed to capture the nuance in these process-outcome relations by parsing them into within-therapist (i.e., differences between patients treated by the same therapist) and between-therapist (i.e., differences between therapists' average process/outcome ratings across all patients in their caseloads) components. Moreover, the few studies that have done so have produced mixed results, suggesting the possibility of systematic variability in these associations (i.e., moderators). One potential source of such variability could be providers themselves; that is, different therapists could use these processes to differing therapeutic benefit. This study tested the alliance- and OE-outcome associations at both the within- and between-therapist levels and explored therapist-level moderators of them. METHOD: Data derived from 212 adult outpatients treated naturalistically by 42 psychotherapists as part of a randomized trial that compared different case-assignment methods. Patients completed measures of alliance, OE, and outcome repeatedly throughout treatment. Therapist characteristics were assessed at baseline. RESULTS: Multilevel structural equation models revealed that, at the between-therapist level, only higher alliance quality, but not more optimistic OE, was associated with greater caseload-level improvement. At the within-therapist level, only more optimistic OE, but not higher alliance quality, was associated with patient improvement. Finally, therapists' self-perceived alliance-fostering effectiveness and cognitive-behavioral orientation moderated the within- and between-therapist alliance-outcome associations, respectively. CONCLUSION: Results indicate that different therapists use common treatment processes to differing therapeutic benefit, which can inform more personalized clinical practices and trainings. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

For whom does a match matter most? Patient-level moderators of evidence-based patient-therapist matching.

OBJECTIVE: A double-blind, randomized controlled trial tested the effectiveness of a personalized Match System in which patients are assigned to therapists with a "track record" of effectively treating a given patient's primary concern(s) (e.g., anxiety). Matched patients demonstrated significantly better outcomes than those assigned through usual pragmatic means. The present study examined patient-level moderators of this match effect. We hypothesized that the match benefits would be especially pronounced for patients who presented with (a) greater overall problem severity, and (b) greater problem complexity (i.e., number of elevated problem domains). We also explored if patient racial/ethnic minority status moderated the condition effect. METHOD: Patients were 218 adults randomized to the Match or as-usual assignment condition, and then treated naturalistically by 48 therapists. The primary outcome was the Treatment Outcome Package (TOP), a multidimensional assessment tool that also primed the Match algorithm (based on historical, therapist-level effectiveness data), and assessed trial patients' symptoms/functioning and demographic information at baseline. Moderator effects were tested as patient-level interactions in three-level hierarchical linear models. RESULTS: The beneficial match effect was significantly more pronounced for patients with higher initial severity (-0.03, 95% CI -0.05, -0.01) and problem complexity (-0.01, 95% CI -0.02, -0.004), yet the high correlation between severity and complexity called into question the uniqueness of the complexity moderator effect. Moreover, the match effect was more pronounced for racial/ethnic minority patients (i.e., nonwhite; -0.05, 95% CI -0.09, -0.01). CONCLUSIONS: Measurement-based matching is especially effective for patients with certain characteristics, which further informs mental health treatment personalization. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Effect of Matching Therapists to Patients vs Assignment as Usual on Adult Psychotherapy Outcomes: A Randomized Clinical Trial.

Importance: Psychotherapists possess strengths and weaknesses in treating different mental health problems, yet performance information is rarely harnessed in mental health care (MHC). To our knowledge, no prior studies have tested the causal efficacy of prospectively matching patients to therapists with empirically derived strengths in treating patients' specific concerns. Objective: To test the effect of measurement-based matching vs case assignment as usual (CAU) on psychotherapy outcomes. Design, Setting, and Participants: In this randomized clinical trial, adult outpatients were recruited between November 2017 and April 2019. Assessments occurred at baseline and repeatedly during treatment at 6 community MHC clinics in Cleveland, Ohio. To be eligible, patients had to make their own MHC decisions. Of 1329 individuals screened, 288 were randomized. Excluding those who withdrew or provided no assessments beyond baseline, 218 patients treated by 48 therapists were included in the primary modified intent-to-treat analyses. Interventions: Therapist performance was assessed pretrial across 15 or more historical cases based on patients' pre-post reporting across 12 problem domains of the routinely administered Treatment Outcome Package (TOP). Therapists were classified in each domain as effective (on average, patients' symptoms reliably improved), neutral (on average, patients' symptoms neither reliably improved nor deteriorated), or ineffective (on average, patients' symptoms reliably deteriorated). Trial patients were randomly assigned to good-fitting therapists (matched group) or were assigned to therapists pragmatically (CAU group). There were multiple match levels, ranging from therapists being effective on the 3 most elevated domains reported by patients and not ineffective on any others (highest) to not effective on the most elevated domains reported by patients but also not ineffective on any domain (lowest). Therapists treated patients in the matched and CAU groups, and treatment was unmanipulated. Main Outcomes and Measures: General symptomatic and functional impairment across all TOP domains (average z scores relative to the general population mean; higher scores indicate greater impairment), global distress (Symptom Checklist-10; higher scores indicate greater distress), and domain-specific impairment on each individual's most elevated TOP-assessed problem. Results: Of 218 patients, 147 (67.4%) were female, and 193 (88.5%) were White. The mean (SD) age was 33.9 (11.2) years. Multilevel modeling indicated a match effect on reductions in weekly general symptomatic and functional impairment (γ110 = -0.03; 95% CI, -0.05 to -0.01; d = 0.75), global distress (γ110 = -0.16; 95% CI, -0.30 to -0.02; d = 0.50), and domain-specific impairment (γ110 = -0.01; 95% CI -0.01 to -0.006; d = 0.60), with no adverse events. Conclusions and Relevance: Matching patients with therapists based on therapists' performance strengths can improve MHC outcomes. Trial Registration: ClinicalTrials.gov Identifier: NCT02990000.

Harnessing the Therapist Effect in Patient-Centered Mental Health Care Decision Making.

In this commentary, we address the critical importance of the research literature documenting therapist differences in skill and outcomes. In particular, we focus on the implications of therapist differences for patient-centered decision making, including the matching of patients to specific psychotherapists who have empirically-based track records of positive performance. In addition, we present preliminary results from a grant supported study of patients' values and preferences regarding the use of provider performance track records in routine mental health care decision making.

Predicting therapist effectiveness from their own practice-based evidence.

OBJECTIVE: Differences between therapists (therapist effect) are often larger than differences between treatments (treatment effect) in explaining client outcomes, and thus should be considered relevant to providing optimal treatment to clients. However, research on therapist effectiveness has focused largely on global measures of distress as opposed to a multidimensional assessment, and has failed to risk-adjust for client characteristics. The purpose of this study was to examine the stability and predictive validity of therapist effectiveness across multiple outcome domains using risk-adjusted outcomes. METHOD: Initial and follow-up outcome data on the Treatment Outcome Package (Kraus, Seligman, & Jordan, 2005) were collected on 3,540 clients who were treated in naturalistic settings by a sample of 59 therapists. After risk-adjusting outcomes based on case-mix variables using random forest models, outcome data from the first 30 clients of each therapist were used to classify each therapist's effectiveness on 12 outcome domains. These results were then compared with outcome data from the therapist's next 30 clients. RESULTS: Results demonstrated that therapist effectiveness was relatively stable, although somewhat domain specific. Therapists classified as "exceptional" were significantly more likely to remain above average with future cases, suggesting that a therapist's past performance is an important predictor of their future performance. CONCLUSIONS: Clients are likely to experience differential benefit depending on the particular therapist and his or her strengths. Clinical outcomes may be improved by developing the best possible prediction model for each new client and then providing that client with referrals to therapists with well-matched strengths. (PsycINFO Database Record

The Expanding Relevance of Routinely Collected Outcome Data for Mental Health Care Decision Making.

Evidence shows that routine outcome monitoring (ROM) and feedback using standardized measurement tools enhances the outcomes of individual patients. When outcome data from a large number of patients and clinicians are collected, patterns can be tracked and comparisons can be made at multiple levels. Variability in skills and outcomes among clinicians and service settings has been documented, and the relevance of ROM for decision making is rapidly expanding alongside the transforming health care landscape. In this article, we highlight several developing core implications of ROM for mental health care, and frame points of future work and discussion.

Treatment Outcome Package: Measuring and facilitating multidimensional change.

The Treatment Outcome Package (TOP; D. R. Kraus, Seligman, & Jordan, 2005) is a multidimensional routine progress and outcome measure developed for use in diverse naturalistic practice settings. In this article, we (a) provide a brief review and summary of the extant psychometric and research support for the TOP, (b) provide examples of the TOP's use in clinical training and practice, and (c) discuss the implications of the TOP for future psychotherapy training, research, and practice. In particular, we focus on the implications of risk-adjusted progress monitoring for systems of care and mental health care decision making.

Attachment history as a moderator of the alliance outcome relationship in adolescents.

The role of the alliance in predicting treatment outcome is robust and long established. However, much less attention has been paid to mechanisms of change, including moderators, particularly for youth. This study examined the moderating role of pretreatment adolescent-caregiver attachment and its impact on the working alliance-treatment outcome relationship. One hundred adolescents and young adults with primary substance dependence disorders were treated at a residential facility, with a cognitive-behavioral emphasis. The working alliance and clinical symptoms were measured at regular intervals throughout treatment. A moderator hypothesis was tested using a path analytic approach. Findings suggested that attachment to the primary caregiver moderated the impact of the working alliance on treatment outcome, such that for youth with the poorest attachment history, working alliance had a stronger relationship with outcome. Conversely, for those with the strongest attachment histories, alliance was not a significant predictor of symptom reduction. This finding may help elucidate alliance-related mechanisms of change, lending support for theories of corrective emotional experience as one function of the working alliance in youth psychotherapy.

Conducting research and collaborating with researchers: the experience of clinicians in a residential treatment center.

This paper describes the experience of clinicians in conducting research and collaborating with academic researchers. As part of clinical routine of a residential program for adolescent substance abusers, empirical data have been collected to assess client's needs before and after treatment, improve clinical practice, and identify barriers to change. Some of the challenges faced and the benefits learned in conducting these studies are presented. In addition to highlighting the convergence of research interests between clinicians and academicians, the conclusion offers general recommendations to foster these partnerships and solidify the scientific-practitioner model.

Implementing routine outcome monitoring in clinical practice: benefits, challenges, and solutions.

This article reviews the benefits, obstacles, and challenges that can hinder (and have hindered) implementation of routine outcome monitoring in clinical practice. Recommendations for future routine outcome assessment efforts are also provided. Spanning three generations, as well as multiple developed tools and approaches, the four authors of this article have spent much of their careers working to address these issues and attempt to consolidate this learning and experience briefly here. Potential "elephants in the room" are brought into the discussion wherever relevant, rather than leaving them to obstruct silently the field's efforts. Some of these topics have been largely ignored, yet must be addressed if we are to fulfill our promise of integrating science and practice. This article is an attempt to identify these important issues and start an honest and open dialogue.

The treatment outcome package: facilitating practice and clinically relevant research.

As an effort to foster the use of standardized data in clinical practice, this article discusses an outcome measure developed by clinicians for use in naturalistic settings: the treatment outcome package (TOP). In addition to describing how the TOP can be used in day-to-day practice, the article illustrates how its multidimensional structure can inform several aspects of clinical work, such as the development of case formulations, facilitation of client-therapist communication, tracking (positive and negative) change during treatment, and the documentation of specific area of therapist expertise. This article recognizes the challenges and drawbacks that are associated with the use of a standardized outcome measure and also describes three ways by which the TOP can contribute to the strengthening of the relationship between research and practice.

Therapist effectiveness: implications for accountability and patient care.

Significant therapist variability has been demonstrated in both psychotherapy outcomes and process (e.g., the working alliance). In an attempt to provide prevalence estimates of "effective" and "harmful" therapists, the outcomes of 6960 patients seen by 696 therapists in the context of naturalistic treatment were analyzed across multiple symptom and functioning domains. Therapists were defined based on whether their average client reliably improved, worsened, or neither improved nor worsened. Results varied by domain with the widespread pervasiveness of unclassifiable/ineffective and harmful therapists ranging from 33 to 65%. Harmful therapists demonstrated large, negative treatment effect sizes (d= -0.91 to -1.49) while effective therapists demonstrated large, positive treatment effect sizes (d=1.00 to 1.52). Therapist domain-specific effectiveness correlated poorly across domains, suggesting that therapist competencies may be domain or disorder specific, rather than reflecting a core attribute or underlying therapeutic skill construct. Public policy and clinical implications of these findings are discussed, including the importance of integrating benchmarked outcome measurement into both routine care and training.

Factor structure of the treatment outcome package for children.

The Treatment Outcome Package (TOP) is a behavioral health assessment and outcome battery with modules that assess a wide array of behavioral health symptoms and functioning, demographics, case-mix variables, and treatment satisfaction. The TOP was developed to follow the design specifications set forth by the Core Battery Conference (Horowitz, Lambert, & Strupp, 1997), but also to ensure the battery's applicability to naturalistic treatment. The TOP also includes a child version that addresses recommendations recently voiced by several experts in youth assessment and treatment. The purpose of this manuscript is to document the process involved in developing the shortened version of the Child TOP. With large samples of diverse patients, 103 items were reduced to 48, with 13 stable and clinically useful subscales. The final confirmatory factor analysis of 7,267 patients replicated the model with excellent results. Although some of these subscales were similar to factors derived from the adult version of the TOP, others were specific to childhood disorders. In addition, the analyses demonstrated that the TOP is not restricted to manifestations of distress and impairment, but also captures a unique factor of childhood strengths (i.e., resiliency).

Validation of a behavioral health treatment outcome and assessment tool designed for naturalistic settings: The Treatment Outcome Package.

In 1994, the American Psychological Association and the Society for Psychotherapy Research convened a Core Battery Conference to develop a set of criteria for the selection of a universal core battery that could be used as a common outcome tool across all outcome studies. The Treatment Outcome Package (TOP) is a behavioral health assessment and outcome battery with modules for assessing a wide array of behavioral health symptoms and functioning, demographics, case-mix, and treatment satisfaction. It was developed to follow the design specifications set forth by the Core Battery Conference, but also to ensure the battery's applicability to naturalistic treatment settings in which randomization may be impossible. In this article we discuss a number of studies that evaluate the initial psychometrics of the items that comprise the mental health symptom and functional modules of the TOP. We conclude that the TOP has an excellent factor structure, good test-retest reliability, promising initial convergent and discriminant validity, measures the full range of pathology on each scale, and has some ability to distinguish between behavioral health clients and members of the general population.