How patients and clinicians experience the utility of a personalized clinical feedback system in routine practice.

OBJECTIVE: The objective was to explore how a person-adaptive clinical feedback system (CFS) effects its users, and how meaning and relevance are negotiated. METHODS: We conducted a 10-month case-study of the implementation and practice of Norse Feedback, a personalized CFS. The data material consisted of 12 patient interviews, 22 clinician interviews, 23 field notes, and 16 archival documents. RESULTS: We identified four main categories or themes: (i) patients' use of clinical feedback for enhanced awareness and insight; (ii) patients work to make clinical feedback a communication mode; (iii) patients and clinicians negotiate clinical feedback as a way to influence treatment; and (iv) clinical feedback requires an interactive sense-making effort. CONCLUSION: Patients and therapists produced the meaning and relevance of the CFS by interpreting the CFS measures to reflect the unique patient experience of the patient-therapist relationship. Patients regarded CFS as a tool to inform therapy with important issues. Patients became more self-aware and prepared for therapy.

Assessing Patient Progress in Psychological Therapy Through Feedback in Supervision: the MeMOS* Randomized Controlled Trial (*Measuring and Monitoring clinical Outcomes in Supervision: MeMOS).

BACKGROUND: Psychological therapy services are often required to demonstrate their effectiveness and are implementing systematic monitoring of patient progress. A system for measuring patient progress might usefully 'inform supervision' and help patients who are not progressing in therapy. AIMS: To examine if continuous monitoring of patient progress through the supervision process was more effective in improving patient outcomes compared with giving feedback to therapists alone in routine NHS psychological therapy. METHOD: Using a stepped wedge randomized controlled design, continuous feedback on patient progress during therapy was given either to the therapist and supervisor to be discussed in clinical supervison (MeMOS condition) or only given to the therapist (S-Sup condition). If a patient failed to progress in the MeMOS condition, an alert was triggered and sent to both the therapist and supervisor. Outcome measures were completed at beginning of therapy, end of therapy and at 6-month follow-up and session-by-session ratings. RESULTS: No differences in clinical outcomes of patients were found between MeMOS and S-Sup conditions. Patients in the MeMOS condition were rated as improving less, and more ill. They received fewer therapy sessions. CONCLUSIONS: Most patients failed to improve in therapy at some point. Patients' recovery was not affected by feeding back outcomes into the supervision process. Therapists rated patients in the S-Sup condition as improving more and being less ill than patients in MeMOS. Those patients in MeMOS had more complex problems.

Client and therapist agreement about the client's problems-Associations with treatment alliance and outcome.

OBJECTIVE: A shared understanding of the patient's symptoms and problems is seen by most theories as a crucial aspect of the collaboration in therapy, presumably influencing alliance and outcome. The empirical ground for this argument is not solid, however. Several studies have found weak associations between a common view of the patient's problems and outcome. The purpose of the present study was to analyze whether agreement in the understanding of the patient's depression and anxiety problems was important for alliance and outcome. METHOD: The study used data from a practice-based study using the CORE system with 846 patients who received psychological treatment in primary care. RESULTS: The analyses indicated that although patients who were assessed by their therapists as having depression and anxiety problems scored higher on these subscales than other patients, about half of the patients reported such problems when the therapists did not, and vice versa. Agreement was not associated with better alliance or outcome. CONCLUSIONS: Productive collaboration in psychotherapy may be based on other factors than agreement about symptoms.

Leading Horses to Water: Lessons from a Decade of Helping Psychological Therapy Services Use Routine Outcome Measurement to Improve Practice.

We summarise the recent reflections of five thought leaders in the field of routine outcome measurement (ROM) for psychological therapy, and then add our own experience of introducing a national ROM system in the UK. We highlight, in particular, the post-implementation challenge of securing data of sufficient reliability to help inform service quality improvements. We ground our conclusions and recommendations in the rapidly evolving discipline of implementation science, and offer a best practice model for applying research recommendations in practice settings. In this context we portray ROM implementation as significant organizational change that benefits from rigorous process and clearly defined, well-communicated targets.

Acceptability, reliability, referential distributions and sensitivity to change in the Young Person's Clinical Outcomes in Routine Evaluation (YP-CORE) outcome measure: replication and refinement.

BACKGROUND: Many outcome measures for young people exist, but the choices for services are limited when seeking measures that (a) are free to use in both paper and electronic format, and (b) have evidence of good psychometric properties. METHOD: Data on the Young Person's Clinical Outcomes in Routine Evaluation (YP-CORE), completed by young people aged 11-16, are reported for a clinical sample (N = 1269) drawn from seven services and a nonclinical sample (N = 380). Analyses report item omission, reliability, referential distributions and sensitivity to change. RESULTS: The YP-CORE had a very low rate of missing items, with 95.6% of forms at preintervention fully completed. The overall alpha was .80, with the values for all four subsamples (11-13 and 14-16 by gender) exceeding .70. There were significant differences in mean YP-CORE scores by gender and age band, as well as distinct reliable change indices and clinically significant change cut-off points. CONCLUSIONS: These findings suggest that the YP-CORE satisfies standard psychometric requirements for use as a routine outcome measure for young people. Its status as a free to use measure and the availability of an increasing number of translations makes the YP-CORE a candidate outcome measure to be considered for routine services.

Correcting Psychotherapists' Blindsidedness: Formal Feedback as a Means of Overcoming the Natural Limitations of Therapists.

PURPOSE: Monitoring of client progress in psychological therapy using formal outcome measures at each session has been shown to increase the effectiveness of treatment. It seems likely that this 'feedback' effect is achieved by enabling therapists to identify clients at risk of treatment failure so that therapists can pay greater attention to client difficulties, which may be hindering therapeutic work. To date, little attention has been given to understanding relevant mechanisms of formal feedback in psychological therapy. In order to understand and maximize the benefits of feedback, it is essential to explore potential mechanisms contributing to this effect. Research in social psychology may help to explain how feedback works. METHODS: Findings on cognitive biases in the field of social psychology are explored and linked to preliminary findings in the field of psychotherapy research. RESULTS: Research on cognitive biases and expertise is congruent with indications that clinical prediction in psychotherapy is unreliable and that it may be difficult for clinicians to detect errors in their judgement as a result of a lack of clear corrective feedback. This problem is linked to the fact that clinical outcomes occur in a complex 'noisy' environment where prediction is inherently difficult. CONCLUSION: Formal feedback may derive its benefits from its ability to help correct naturally occurring biases in therapists' assessment of their work. If these biases are seen as normal, but often avoidable if feedback is used, this may pave the way to greater acceptance of formal feedback by clinicians and enhanced outcomes for clients. KEY PRACTITIONER MESSAGE: The use of formal feedback tools can help therapists overcome inevitable limitations in their ability to predict poor response to treatment, enhancing the likelihood of detecting and resolving client difficulties in therapy.

Tracking Responses to Items in Measures as a Means of Increasing Therapeutic Engagement in Clients: A Complementary Clinical Approach to Tracking Outcomes.

UNLABELLED: This article presents a novel clinical application of questionnaire feedback, which focuses on change at the individual question level rather than the total mean or clinical score level. We term the approach 'Tracking Responses to Items in Measures' (TRIM) and promote the key aims to be (1) providing both client and practitioner with feedback on areas of positive change that may be masked by numerical feedback, (2) reinforcing client strengths and self-efficacy, (3) exploring potential extra-therapeutic factors that may contribute to the lack of change or deterioration on individual questions and (4) establishing a collaborative dialogue relating clients' problems to their goals and the consequent aims of treatment. This paper profiles the clinical origins and technical development of TRIM as a clear, user-friendly display of item change across sessions using colour codes and illustrates the clinical utility through two clinical vignettes. Although the profile of the TRIM method herein uses the Clinical Outcomes in Routine Evaluation Outcome Measure, we believe the method could easily be used with other measures. These could include Generalized Anxiety Disorder 7 and Patient Health Questionnaire 9 used in English National Health Service primary care Improving Access to Psychological Therapies services, or disorder specific measures for particular problems commonly used in National Health Service specialist services. We suggest TRIM is a practical complement to existing feedback systems, especially in work with clients who may be less likely to show empirically meaningful change on mean item or clinical score levels. KEY PRACTITIONER MESSAGE: Using outcome questionnaires as conversational tools helps practitioners focus on change at the individual item level rather than the numeric level. Tracking Responses to Items in Measures helps provide clients and practitioners with feedback on areas of positive change that may be masked by summary score analysis. Exploring the lack of change or deterioration on particular questions helps practitioners to assess extra-therapeutic factors that may be compromising change. Using individual item change profiles as feedback for clients helps validate their progress and reinforce their strengths and self-efficacy.

Exhausted but not cynical: burnout in therapists working within Improving Access to Psychological Therapy Services.

BACKGROUND: Burnout is common in mental health professionals and has serious personal and professional consequences. Levels and predictors of burnout for therapists within "Improving Access to Psychological Therapies" (IAPT) services are thus far unknown. AIMS: This study investigated levels and predictors of three burnout dimensions--Emotional Exhaustion (EE), Depersonalisation (DP) and Personal Accomplishment (PA)--in IAPT therapists. METHODS: Therapists from eight services completed a web-based survey measuring burnout levels and hypothesised burnout predictors. Reponses were matched to demographic information on clients with whom they had completed treatment in the last 2 months. RESULTS: 116 (n) therapists showed comparatively high levels of EE, and relatively low levels of DP and PA. These were predicted by some factors from the General Burnout Model and by Work Involvement styles. Stressful Involvement, in particular in-sessions feelings of anxiety predicted EE and DP, and Healing Involvement predicted Personal Accomplishment. The most important predictors of therapist burnout were service-related, particularly work demands and autonomy, and in-session feelings. CONCLUSIONS: Addressing these factors in IAPT services may prevent therapist burnout.

Responsive regulation of treatment duration in routine practice in United Kingdom primary care settings: replication in a larger sample.

Replicating a previous study (M. Barkham et al., 2006), the authors examined rates of improvement in psychotherapy in United Kingdom primary care settings as a function of the number of sessions attended. Included in the study were adult clients who returned valid assessments at the beginning and the end of their treatment, had planned endings, began treatment above the clinical cutoff score, and were seen for 20 or fewer sessions (N = 9,703; 72.4% female; 87.7% Caucasian; average age = 40.9 years). Clients' average assessment scores improved substantially across treatment, with a pretreatment-posttreatment effect size of 1.96; 62.0% achieved reliable and clinically significant improvement (RCSI). Clients' mean pretreatment-posttreatment change was approximately constant regardless of treatment duration (in the range of 0 to 20 sessions); the RCSI rate decreased slightly with treatment duration, as fewer clients fell below the cutoff at longer durations. Results were interpreted as suggesting that therapists and clients tend to make appropriately responsive decisions about treatment duration.

Effects of psychological therapies in randomized trials and practice-based studies.

BACKGROUND: Randomized trials of the effects of psychological therapies seek internal validity via homogeneous samples and standardized treatment protocols. In contrast, practice-based studies aim for clinical realism and external validity via heterogeneous samples of clients treated under routine practice conditions. We compared indices of treatment effects in these two types of studies. METHOD: Using published transformation formulas, the Beck Depression Inventory (BDI) scores from five randomized trials of depression (N = 477 clients) were transformed into Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) scores and compared with CORE-OM data collected in four practice-based studies (N = 4,196 clients). Conversely, the practice-based studies' CORE-OM scores were transformed into BDI scores and compared with randomized trial data. RESULTS: Randomized trials showed a modest advantage over practice-based studies in amount of pre-post improvement. This difference was compressed or exaggerated depending on the direction of the transformation but averaged about 12%. There was a similarly sized advantage to randomized trials in rates of reliable and clinically significant improvement (RCSI). The largest difference was yielded by comparisons of effect sizes which suggested an advantage more than twice as large, reflecting narrower pre-treatment distributions in the randomized trials. CONCLUSIONS: Outcomes of completed treatments for depression in randomized trials appeared to be modestly greater than those in routine care settings. The size of the difference may be distorted depending on the method for calculating degree of change. Transforming BDI scores into CORE-OM scores and vice versa may be a preferable alternative to effect sizes for comparisons of studies using these measures.

Dose-effect relations and responsive regulation of treatment duration: the good enough level.

This study examined rates of improvement in psychotherapy as a function of the number of sessions attended. The clients (N=1,868; 73.1% female; 92.4% White; average age=40), who were seen for a variety of problems in routine primary care mental health practices, attended 1 to 12 sessions, had planned endings, and completed the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) at the beginning and end of their treatment. The percentage of clients achieving reliable and clinically significant improvement (RCSI) on the CORE-OM did not increase with number of sessions attended. Among clients who began treatment above the CORE-OM clinical cutoff (n=1,472), the RCSI rate ranged from 88% for clients who attended 1 session down to 62% for clients who attended 12 sessions (r=-.91). Previously reported negatively accelerating aggregate curves may reflect progressive ending of treatment by clients who had achieved a good enough level of improvement.

A CORE approach to progress monitoring and feedback: Enhancing evidence and improving practice.

This article describes the Clinical Outcomes in Routine Evaluation (CORE) System and reports on its scientific yield and practice impact. First, we describe the suite of CORE measures, including the centerpiece CORE-Outcome Measure (CORE-OM), its short forms, special purpose forms, translations, and psychometric properties, along with the pretreatment CORE Therapy Assessment Form and the CORE End of Therapy Form. Second, we provide an overview of the scientific yield arising from analyses of large CORE data sets collected in routine practice. Third, we describe the use of CORE measures for feedback in practice settings. Finally, we consider future directions for monitoring and feedback in research and practice.

Psychological treatment outcomes in routine NHS services: what do we mean by treatment effectiveness?

OBJECTIVES: The question of how effective therapies are in routine practice is crucial. The answer depends on how we define effectiveness. Both the definition of who was treated and the index chosen to represent outcome can affect estimates dramatically. DESIGN: We used data from the Clinical Outcomes in Routine Evaluation (CORE) Primary Care National Practice-Based Evidence database-2005 on the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) from 33,587 patients and examined rates of improvement in psychological therapies in UK National Health Service (NHS) primary care services using various definitions of effectiveness. METHODS: We constructed successively more restrictive sub-samples of patients, including practice-based analogues of several types of intent-to-treat (ITT) groups and completer groups. We focussed on patients scoring above clinical cut-off at intake, but we also considered samples that included sub-clinical patients. We assessed two types of recovery rates, improvement rates, mean pre-post change, and pre-post effect sizes for each sub-sample. RESULTS: There was wide variation in the overall effectiveness of treatments as a function of which subset of data was considered and which specific criterion of recovery rate was adopted. Recovery rates and pre-post effect sizes ranged from 19% to 65% and 0.60 to 1.95, respectively. CONCLUSIONS: Because estimates of effectiveness could have significant policy implications, clarity on the meanings of the differing constructions is essential.

Effectiveness of cognitive-behavioural, person-centred, and psychodynamic therapies in UK primary-care routine practice: replication in a larger sample.

BACKGROUND: Psychotherapy's equivalence paradox is that treatments tend to have equivalently positive outcomes despite non-equivalent theories and techniques. We replicated an earlier comparison of treatment approaches in a sample four times larger and restricted to primary-care mental health. METHOD: Patients (n=5613) who received cognitive-behavioural therapy (CBT), person-centred therapy (PCT) or psychodynamic therapy (PDT) at one of 32 NHS primary-care services during a 3-year period (2002-2005) completed the Clinical Outcomes in Routine Evaluation - Outcome Measure (CORE-OM) at the beginning and end of treatment. Therapists indicated which approaches were used on an End of Therapy form. We compared outcomes of groups treated with CBT (n=1045), PCT (n=1709), or PDT (n=261) only or with one of these plus one additional approach (e.g. integrative, supportive, art), designated CBT+1 (n=1035), PCT+1 (n=1033), or PDT+1 (n=530), respectively. RESULTS: All six groups began treatment with equivalent CORE-OM scores, and all averaged marked improvement (overall pre/post effect size=1.39). Neither treatment approach nor degree of purity ('only' v. '+1') had a statistically significant effect. Distributions of change scores were all similar. CONCLUSIONS: Replicating the earlier results, the theoretically different approaches tended to have equivalent outcomes. Caution is warranted because of limited treatment specification, non-random assignment, incomplete data, and other issues. Insofar as these routine treatments appear effective for patients who complete them, those who fail to complete (or to begin) treatment deserve attention by researchers and policymakers.

Effectiveness of cognitive-behavioural, person-centred and psychodynamic therapies as practised in UK National Health Service settings.

BACKGROUND: Psychotherapy's equivalence paradox is that treatments have equivalently positive outcomes despite non-equivalent theories and techniques. We compared the outcomes of contrasting approaches practised in routine care. METHOD: Patients (n = 1309) who received cognitive-behavioural therapy (CBT), person-centred therapy (PCT) and psychodynamic therapy (PDT) at one of 58 National Health Service (NHS) primary and secondary care sites in the UK during a 3-year period completed the Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) at the beginning and end of their treatment. Therapists indicated which treatment approaches were used on an End of Therapy form. We compared outcomes of six groups: three treated with CBT, PCT or PDT only, and three treated with one of these plus one additional approach (e.g. integrative, supportive, art), designated CBT+1, PCT+1 or PDT+1 respectively. RESULTS: All six groups averaged marked improvement (pre-post effect size = 1.36). Treatment approach and degree of purity ('only' vs. '+1') each accounted for statistically significant but comparatively tiny proportions of the variance in CORE-OM scores (respectively 1% and 0.5% as much as pre-post change). Distributions of change scores were largely overlapping. CONCLUSIONS: Results for these three treatment approaches as practised routinely across a range of NHS settings were generally consistent with previous findings that theoretically different approaches tend to have equivalent outcomes. Caution is warranted because of limited treatment specification, non-random assignment, lack of a control group, missing data and other issues.

Towards a standardised brief outcome measure: psychometric properties and utility of the CORE-OM.

BACKGROUND: An acceptable, standardised outcome measure to assess efficacy and effectiveness is needed across multiple disciplines offering psychological therapies. AIMS: To present psychometric data on reliability, validity and sensitivity to change for the CORE-OM (Clinical Outcomes in Routine Evaluation--Outcome Measure). METHOD: A 34-item self-report instrument was-developed, with domains of subjective well-being, symptoms, function and risk. Analysis includes internal reliability, test-retest reliability, socio-demographic differences, exploratory principal-component analysis, correlations with other instruments, differences between clinical and non-clinical samples and assessment of change within a clinical group. RESULTS: Internal and test-retest reliability were good (0.75-0.95), as was convergent validity with seven other instruments, with large differences between clinical and non-clinical samples and good sensitivity to change. CONCLUSIONS: The CORE-OM is a reliable and valid instrument with good sensitivity to change. It is acceptable in a wide range of practice settings.