Why are some cases not on track? An item analysis of the Assessment for Signal Cases during inpatient psychotherapy.

Within the Routine Outcome Monitoring system "OQ-Analyst," the questionnaire "Assessment for Signal Cases" (ASC) supports therapists in detecting potential reasons for not-on-track trajectories. Factor analysis and a machine learning algorithm (LASSO with 10-fold cross-validation) were applied, and potential predictors of not-on-track classifications were tested using logistic multilevel modeling methods. The factor analysis revealed a shortened (30 items) version of the ASC with good internal consistency (α = 0.72-0.89) and excellent predictive value (area under the curve = 0.98; positive predictive value = 0.95; negative predictive value = 0.94). Item-level analyses showed that interpersonal problems captured by specific ASC items (not feeling able to speak about problems with family members; feeling rejected or betrayed) are the most important predictors of not-on-track trajectories. It should be considered that our results are based on analyses of ASC items only. Our findings need to be replicated in future studies including other potential predictors of not-on-track trajectories (e.g., changes in medication, specific therapeutic techniques, or treatment adherence), which were not measured this study.

Using the Assessment for Signal Clients as a feedback tool for reducing treatment failure.

OBJECTIVE: The Clinical Support Tools (CSTs) were developed to help therapists organize and target potential problems that might account for negative outcomes in psychotherapy. The core of CST feedback is the Assessment for Signal Clients. The purpose of this study was to describe and identify patterns of problems that typically characterize off-track cases. METHOD: The responses and scores from 107 off-track clients from a hospital-based outpatient clinic were analyzed. RESULTS: A cluster analysis of the 107 off-track clients revealed three client types: those whose problems were characterized by alliance and motivational difficulties; those characterized by social support and life event difficulties; and those whose problems had an indistinguishable pattern. Log-linear modeling showed that if patients had less therapeutic alliance problems they were also less likely to have motivational problems. Findings were also consistent with the cluster analysis, which showed that a relatively higher percentage of not-on-track participants received signal alerts for the social support items and scale. CONCLUSIONS: Individuals whose progress goes off-track while in psychotherapy appear to have their greatest difficulty with social support, followed closely by motivation for therapy and therapeutic alliance.

An investigation of self-assessment bias in mental health providers.

Previous research has consistently found self-assessment bias (an overly positive assessment of personal performance) to be present in a wide variety of work situations. The present investigation extended this area of research with a multi-disciplinary sample of mental health professionals. Respondents were asked to: (a) compare their own overall clinical skills and performance to others in their profession, and (b) indicate the percentage of their clients who improved, remained the same, or deteriorated as a result of treatment with them. Results indicated that 25% of mental health professionals viewed their skill to be at the 90th percentile when compared to their peers, and none viewed themselves as below average. Further, when compared to the published literature, clinicians tended to overestimate their rates of client improvement and underestimate their rates of client deterioration. The implications of this self-assessment bias for improvement of psychotherapy outcomes are discussed.

Identifying highly effective psychotherapists in a managed care environment.

OBJECTIVE: To investigate the variability and stability of psychotherapists' effectiveness and the implications of this differential effectiveness for quality improvement in a managed care environment. STUDY DESIGN: Subset archival outcome data for patients receiving behavioral health treatment were divided into 2 time periods to cross-validate the treating therapists' effectiveness. After categorizing the therapists as "highly effective" and "others" during the baseline period, the stability of their individual effectiveness was cross-validated in the remaining time period. METHODS: Outcomes for 10 812 patients (76.0% adults, 24.0% children and adolescents) treated by 281 therapists were included. Patients initiated treatment between January 1999 and June 2004. Mean residual change scores obtained by multiple regression were used to adjust for differences in case mix among therapists. Raw change scores as well as mean residualized change scores were compared between the 71 psychotherapists identified as highly effective (25%) and those identified as other (remaining 75%). RESULTS: During the cross-validation period, mean differences in residualized change score between highly effective therapists and others were statistically significant (difference = 2.8; P < .001), which corresponded to an average of 53.3% more change in raw change scores with the highly effective therapists. Results could not be explained by case mix differences in diagnosis, age, sex, intake scores, prior outpatient treatment history, length of treatment, or therapist training/experience. CONCLUSION: Behavioral health outcomes for a large system of care could be significantly improved by measuring clinical outcomes and referring patients to therapists with superior outcomes.

An evaluation of the dose-response relationship in naturalistic treatment settings using survival analysis.

To date, few studies have been published on the dose-response relationship in psychotherapy. The current study addresses limitations of previous research by using (1) clinical significance methodology to address the meaningfulness of patient change, (2) survival analysis to assess change across time, (3) assessment of patient change on a session-by-session basis, and (4) a large data set representing a variety of treatment settings. A total of 4,761 patients representing standard treatment settings in the United States were tracked at each session of therapy. A survival analysis of this data reveals that between 15 and 19 sessions of therapy are required for a 50% recovery rate using clinical significance methodology. The results of this study provide a useful overview of time-to-change in naturalistic settings that can be used to estimate reliable treatment expectations and as a baseline for comparison when modifications are made within treatment delivery systems.