A mixed-method investigation of patient monitoring and enhanced feedback in routine practice: Barriers and facilitators.

OBJECTIVE: To investigate the barriers and facilitators of an effective implementation of an outcome monitoring and feedback system in a UK National Health Service psychological therapy service. METHOD: An outcome monitoring system was introduced in two services. Enhanced feedback was given to therapists after session 4. Qualitative and quantitative methods were used, including questionnaires for therapists and patients. Thematic analysis was carried out on written and verbal feedback from therapists. Analysis of patient outcomes for 202 episodes of therapy was compared with benchmark data of 136 episodes of therapy for which feedback was not given to therapists. RESULTS: Themes influencing the feasibility and acceptability of the feedback system were the extent to which therapists integrated the measures and feedback into the therapy, availability of administrative support, information technology, and complexity of the service. There were low levels of therapist actions resulting from the feedback, including discussing the feedback in supervision and with patients. CONCLUSIONS: The findings support the feasibility and acceptability of setting up a routine system in a complex service, but a number of challenges and barriers have to be overcome and therapist differences are apparent. More research on implementation and effectiveness is needed in diverse clinical settings.

Early changes, attrition, and dose-response in low intensity psychological interventions.

OBJECTIVES: To investigate if early symptom changes in brief low intensity psychological interventions (guided self-help and psycho-education using cognitive behavioural therapy principles) are predictive of final treatment outcome. DESIGN: Retrospective cohort data analysis. METHOD: Clinical records for 1,850 patients who screened positive for depression and/or an anxiety disorder were analysed. Reliable and clinically significant improvement (RCSI) on depression (Patient Health Questionnaire-9: PHQ-9) or anxiety (generalized anxiety disorder-7: GAD-7) outcome measures after treatment was the primary outcome. Change scores ≥6 on PHQ-9 and ≥5 on GAD-7 were taken as indicative of reliable improvement (RI). The model assumed that RI in the earliest treatment sessions would be predictive of RCSI post-treatment. Predictive accuracy was assessed by calculating the area under the curve (AUC), as well as positive and negative predictive values. Diagnostic odds ratios were also estimated, adjusting for confounders such as baseline severity, use of medication, and pre-treatment symptom change. RESULTS: The AUC estimates for session-to-session change scores ranged between .62 and .88, indicative of modest to high predictive reliability. Predictive accuracy was higher for patients who had four or more treatment sessions, with more than 70% of patients with RCSI being accurately identified as early as sessions 1-3. Attrition rates were significantly associated with poor outcomes. Results suggest that at least four therapy sessions are necessary to achieve more than 50% RCSI rates, and the dose-response effect appears to decline in treatments longer than six sessions. CONCLUSIONS: Patients showing RI early in treatment were at least twice as likely to fully recover compared to those without early RI.

Benchmarking routine psychological services: a discussion of challenges and methods.

BACKGROUND: Policy developments in recent years have led to important changes in the level of access to evidence-based psychological treatments. Several methods have been used to investigate the effectiveness of these treatments in routine care, with different approaches to outcome definition and data analysis. AIMS: To present a review of challenges and methods for the evaluation of evidence-based treatments delivered in routine mental healthcare. This is followed by a case example of a benchmarking method applied in primary care. METHOD: High, average and poor performance benchmarks were calculated through a meta-analysis of published data from services working under the Improving Access to Psychological Therapies (IAPT) Programme in England. Pre-post treatment effect sizes (ES) and confidence intervals were estimated to illustrate a benchmarking method enabling services to evaluate routine clinical outcomes. RESULTS: High, average and poor performance ES for routine IAPT services were estimated to be 0.91, 0.73 and 0.46 for depression (using PHQ-9) and 1.02, 0.78 and 0.52 for anxiety (using GAD-7). Data from one specific IAPT service exemplify how to evaluate and contextualize routine clinical performance against these benchmarks. CONCLUSIONS: The main contribution of this report is to summarize key recommendations for the selection of an adequate set of psychometric measures, the operational definition of outcomes, and the statistical evaluation of clinical performance. A benchmarking method is also presented, which may enable a robust evaluation of clinical performance against national benchmarks. Some limitations concerned significant heterogeneity among data sources, and wide variations in ES and data completeness.

Factors Predicting Surgical Effort Using Explainable Artificial Intelligence in Advanced Stage Epithelial Ovarian Cancer.

(1) Background: Surgical cytoreduction for epithelial ovarian cancer (EOC) is a complex procedure. Encompassed within the performance skills to achieve surgical precision, intra-operative surgical decision-making remains a core feature. The use of eXplainable Artificial Intelligence (XAI) could potentially interpret the influence of human factors on the surgical effort for the cytoreductive outcome in question; (2) Methods: The retrospective cohort study evaluated 560 consecutive EOC patients who underwent cytoreductive surgery between January 2014 and December 2019 in a single public institution. The eXtreme Gradient Boosting (XGBoost) and Deep Neural Network (DNN) algorithms were employed to develop the predictive model, including patient- and operation-specific features, and novel features reflecting human factors in surgical heuristics. The precision, recall, F1 score, and area under curve (AUC) were compared between both training algorithms. The SHapley Additive exPlanations (SHAP) framework was used to provide global and local explainability for the predictive model; (3) Results: A surgical complexity score (SCS) cut-off value of five was calculated using a Receiver Operator Characteristic (ROC) curve, above which the probability of incomplete cytoreduction was more likely (area under the curve [AUC] = 0.644; 95% confidence interval [CI] = 0.598−0.69; sensitivity and specificity 34.1%, 86.5%, respectively; p = 0.000). The XGBoost outperformed the DNN assessment for the prediction of the above threshold surgical effort outcome (AUC = 0.77; 95% [CI] 0.69−0.85; p < 0.05 vs. AUC 0.739; 95% [CI] 0.655−0.823; p < 0.95). We identified "turning points" that demonstrated a clear preference towards above the given cut-off level of surgical effort; in consultant surgeons with <12 years of experience, age <53 years old, who, when attempting primary cytoreductive surgery, recorded the presence of ascites, an Intraoperative Mapping of Ovarian Cancer score >4, and a Peritoneal Carcinomatosis Index >7, in a surgical environment with the optimization of infrastructural support. (4) Conclusions: Using XAI, we explain how intra-operative decisions may consider human factors during EOC cytoreduction alongside factual knowledge, to maximize the magnitude of the selected trade-off in effort. XAI techniques are critical for a better understanding of Artificial Intelligence frameworks, and to enhance their incorporation in medical applications.

Predicting complete cytoreduction for advanced ovarian cancer patients using nearest-neighbor models.

BACKGROUND: The foundation of modern ovarian cancer care is cytoreductive surgery to remove all macroscopic disease (R0). Identification of R0 resection patients may help individualise treatment. Machine learning and AI have been shown to be effective systems for classification and prediction. For a disease as heterogenous as ovarian cancer, they could potentially outperform conventional predictive algorithms for routine clinical use. We investigated the performance of an AI system, the k-nearest neighbor (k-NN) classifier, to predict R0, comparing it with logistic regression. Patients diagnosed with advanced stage, high grade serous ovarian, tubal and primary peritoneal cancer, undergoing surgical cytoreduction from 2015 to 2019, was selected from the ovarian database. Performance variables included age, BMI, Charlson Comorbidity Index, timing of surgery, surgical complexity and disease score. The k-NN algorithm classified R0 vs non-R0 patients using 3-20 nearest neighbors. Prediction accuracy was estimated as percentage of observations in the training set correctly classified. RESULTS: 154 patients were identified, with mean age of 64.4 + 10.5 yrs., BMI of 27.2 + 5.8 and mean SCS of 3 + 1 (1-8). Complete and optimal cytoreduction was achieved in 62 and 88% patients. The mean predictive accuracy was 66%. R0 resection prediction of true negatives was as high as 90% using k = 20 neighbors. CONCLUSIONS: The k-NN algorithm is a promising and versatile tool for R0 resection prediction. It slightly outperforms logistic regression and is expected to improve accuracy with data expansion.

Effects of psychological therapies in randomized trials and practice-based studies.

BACKGROUND: Randomized trials of the effects of psychological therapies seek internal validity via homogeneous samples and standardized treatment protocols. In contrast, practice-based studies aim for clinical realism and external validity via heterogeneous samples of clients treated under routine practice conditions. We compared indices of treatment effects in these two types of studies. METHOD: Using published transformation formulas, the Beck Depression Inventory (BDI) scores from five randomized trials of depression (N = 477 clients) were transformed into Clinical Outcomes in Routine Evaluation-Outcome Measure (CORE-OM) scores and compared with CORE-OM data collected in four practice-based studies (N = 4,196 clients). Conversely, the practice-based studies' CORE-OM scores were transformed into BDI scores and compared with randomized trial data. RESULTS: Randomized trials showed a modest advantage over practice-based studies in amount of pre-post improvement. This difference was compressed or exaggerated depending on the direction of the transformation but averaged about 12%. There was a similarly sized advantage to randomized trials in rates of reliable and clinically significant improvement (RCSI). The largest difference was yielded by comparisons of effect sizes which suggested an advantage more than twice as large, reflecting narrower pre-treatment distributions in the randomized trials. CONCLUSIONS: Outcomes of completed treatments for depression in randomized trials appeared to be modestly greater than those in routine care settings. The size of the difference may be distorted depending on the method for calculating degree of change. Transforming BDI scores into CORE-OM scores and vice versa may be a preferable alternative to effect sizes for comparisons of studies using these measures.

Shapes of early change in psychotherapy under routine outpatient conditions.

Although improvement of clients' state is a central concern for psychotherapy, relatively little is known about how change in outcome variables unfolds during psychotherapy. Client progress may follow highly variable temporal courses, and this variation in treatment courses may have important clinical implications. By analyzing treatment progress using growth mixture modeling up to the 6th session in a sample of 192 outpatients treated under routine clinic conditions, the authors identified 5 client groups based on similar progress on the short form versions of the Clinical Outcomes in Routine Evaluation-Outcome Measure. The shapes of early change typical for these client groups were characterized by (a) high initial impairment, (b) low initial impairment, (c) early improvement, (d) medium impairment with continuous treatment progress, or (e) medium impairment with discontinuous treatment progress. Moreover, the shapes of early change were associated with different treatment outcomes and durations, and several intake variables (depression, anxiety, and age) enabled prediction of the shape of early change and/or prediction of individual treatment progress within client groups with similar shapes of change.

Stability of the CORE-OM and the BDI-I prior to therapy: evidence from routine practice.

BACKGROUND: It is important to know the stability of standard outcome measures prior to therapy over differing periods of time that map onto the realities of waiting times in routine service settings. METHOD: We studied 1,684 clients who completed one or both the targeted measures Clinical Outcomes in Routine Evaluation-Outcome Measures (CORE-OM) and Beck Depression Inventory-I (BDI-I) two times, at intervals of up to 12 months, prior to beginning psychotherapy. We also selected an additional 1,623 clients who completed the CORE-OM (N=1,623), BDI-I (N=980) or both at referral, but had no records of further contact with the service. RESULTS: There was little change in the mean CORE-OM or BDI-I scores between referral and clinical assessment. The test-retest correlations showed substantial stability on both measures, declining only moderately at the longer intervals studied. CONCLUSION: The high test-retest correlations for periods of up to 6 months suggest that psychological disturbance was both reliably measured by the CORE-OM and the BDI-I, and reasonably stable among clients waiting to be assessed. Implications for routine practice are discussed.

How durable is the effect of low intensity CBT for depression and anxiety? Remission and relapse in a longitudinal cohort study.

BACKGROUND: Depression and anxiety disorders are relapse-prone conditions, even after successful treatment with pharmacotherapy or psychotherapy. Cognitive behavioural therapy (CBT) is known to prevent relapse, but there is little evidence of the durability of remission after low intensity forms of CBT (LiCBT). METHOD: This study aimed to examine relapse rates 12 months after completing routinely-delivered LiCBT. A cohort of 439 LiCBT completers with remission of symptoms provided monthly depression (PHQ-9) and anxiety (GAD-7) measures during 12 months after treatment. Survival analysis was conducted to model time-to-relapse while controlling for patient characteristics. RESULTS: Overall, 53% of cases relapsed within 1 year. Of these relapse events, the majority (79%) occurred within the first 6 months post-treatment. Cases reporting residual depression symptoms (PHQ-9 = 5 to 9) at the end of treatment had significantly higher risk of relapse (hazard ratio = 1.90, p < 0.001). CONCLUSIONS: The high rate of relapse after LiCBT highlights the need for relapse prevention, particularly for those with residual depression symptoms.

Transforming between Beck Depression Inventory and CORE-OM scores in routine clinical practice.

OBJECTIVES: The Clinical Outcomes in Routine Evaluation - Outcome Measure (CORE-OM) and the Beck Depression Inventory (BDI) are routinely used to assess emotional problems. It would be helpful to be able to compare scores when only one of the measures is available. We investigated the relationship between the measures and produced translation tables. METHODS: Level of agreement between CORE-OM and BDI-I was assessed for 2,234 clients who had completed both measures at referral for routine secondary care. Tables for predicting between the measures were constructed using several methods, including non-linear regression and non-parametric smoothing. Results were cross-validated on a separate sample of 326 clients. RESULTS: High correlations between the measures were obtained (r=.862 for female clients; r=.855 for male clients). Accuracy in predicting caseness is higher than predicting depression levels. CONCLUSIONS: CORE-OM and BDI can be compared in routine clinical settings with acceptable accuracy.

Therapists' recall of early sudden gains in routine clinical practice.

This study followed up one by Stiles et al. (2003), which identified sudden gains -- large reductions from one session to the next on a short form of clinical outcomes in routine evaluation outcome measure (CORE-SF) -- by some clients in routine clinical practice. We interviewed the therapists who had treated sudden gain and non-sudden gain clients. Results showed that therapists could retrospectively identify which clients had or had not experienced sudden CORE-SF gains at substantially better than chance rates, although they identified only about half of the clients who had experienced such gains.

Predicting change for individual psychotherapy clients on the basis of their nearest neighbors.

This study extended client-focused research by using the nearest neighbor (NN) approach, a client-specific sampling and prediction strategy derived from research on alpine avalanches. Psychotherapy clients (N=203) seen in routine practice settings in the United Kingdom completed a battery of intake measures and then completed symptom intensity ratings before each session. Forecasts of each client's rate of change and session-by-session variability were computed on the basis of that client's NNs (n=10-50 in different comparisons). Alternative forecasts used linear or log-linear slopes and were compared with an alternative prediction strategy. Results showed that the NN approach was superior to the alternative model in predicting rate of change, though the advantage was less clear for predicting variability.

The relationship between social deprivation and unilateral termination (attrition) from psychotherapy at various stages of the health care pathway.

The relationship between social deprivation and attrition from psychotherapy was examined at various stages of the health care pathway. Data providing information on service users' discharge status gave a measure of attrition at different stages along this pathway. On the basis of their postcode, service users were allocated a Townsend deprivation score, which is a measure of social deprivation. Of the sample, 60% terminated therapy prior to agreed discharge at various pathway stages. Service users who failed to attend their first appointment had significantly higher social deprivation scores than those who completed therapy. Early terminators, who stopped attending before their fifth session, had a significantly higher level of social deprivation than the late terminators, and those who completed therapy. The late terminators did not significantly differ from completers on social deprivation. These results support earlier findings showing that socio-economic status influences attrition from therapy. However, socioeconomic status only affects two stages of the pathway - attendance at the first appointment, and the early stage of attending therapy. It does not affect earlier or later stages of the pathway. Applying the health action process approach (HAPA) model to the results, the action/maintenance stage can be represented by attendance for therapy. Socio-economic status influences this stage of the model, because social support and resources are important determinants for compliance. Earlier stages, characterized by HAPA as a decisional/motivation stage thought to be influenced by beliefs, was not affected by social deprivation. It is concluded that attrition from therapy should be studied separately for each pathway stage. Earlier conflicting findings on causes of attrition may have resulted from studying different combinations of pathway stages.

Needs assessment in forensic learning disability.

This paper gives the results of a needs assessment of a group of learning-disabled forensic patients from two Strategic Health Authority areas in the north of England. The patients were found to be a heterogeneous group with wide-ranging psychiatric needs. The majority were cared for outside their geographical area of origin, either in specialist NHS facilities or the independent sector. Those with an additional diagnosis of mental illness were most likely to be detained in NHS facilities within the region: a diagnosis of personality disorder was associated with placement in either a high secure setting or the independent sector. Individuals with a clinical diagnosis of mental retardation were most likely to be detained in services provided by specialist learning disability/mental health trusts out of area. There was a small group of females who were all placed outside the region. Offending behaviour was most likely to consist of violence against the person, sexual offences and arson. The majority assessed were felt to have long term needs. The study raised important implications for future provision of forensic services in the area, particularly the need to offer services with treatment programmes tailored to the needs of the population under review.

Early sudden gains in psychotherapy under routine clinic conditions: practice-based evidence.

Sudden gains--large, enduring reductions in symptom intensity from one session to the next--were identified by T. Z. Tang and R. J. DeRubeis (1999b) on the basis of data from 2 manualized clinical trials of cognitive therapy for depression. The authors found similar sudden gains among clients with a variety of disorders treated with a variety of approaches in routine clinic settings. Clients (N = 135 who met inclusion criteria) completed short forms of the Clinical Outcomes in Routine Evaluation (CORE-SF) preceding 7 to 74 individual sessions. Those who experienced sudden gains within their first 16 sessions (n = 23) had significantly lower CORE-SF scores in their final 3 sessions than did the other clients.

Personal constructs, childhood sexual abuse and revictimization.

Within the theoretical framework of Ryle's Procedural Sequence Object Relations Model and Kelly's Personal Construct Theory, this study investigates sex-role polarization of incest survivors and the centrality of abuse within survivors' constructs of men that may contribute to revictimization. Repertory grid methodology was used with 40 female survivors of childhood sexual abuse and 28 non-abused women. Grid measures and psychometric measures were compared between groups of women who had and had not experienced childhood sexual abuse, revictimized and non-revictimized survivors, and survivors who had and had not experienced incestuous abuse. Results showed significant differences between survivors and non-abused women, with survivors having higher levels of depression and perceived distress, lower self-esteem and higher self/ideal self discrepancy. Hypothesized differences in sex-role polarization were not found. There were few differences between revictimized and non-revictimized survivors, although revictimized survivors rated 'self now' as more powerful than non-revictimized survivors. No differences were found between survivors who had and had not experienced incestuous abuse. In addition to the value of exploring personal constructs, a range of models need to be considered in understanding revictimization and women's construal of men. The implications of using repertory grid methodology for research and clinical work are discussed.