Impact of racial microaggressions in psychotherapy vignettes with african american clients: An experimental analogue design.

Mental health researchers have focused on promoting culturally sensitive clinical care (Herman et al., 2007; Whaley & Davis, 2007), emphasizing the need to understand how biases may impact client well-being. Clients report that their therapists commit racial microaggressions-subtle, sometimes unintentional, racial slights-during treatment (Owen et al., 2014). Yet, existing studies often rely on retrospective evaluations of clients and cannot establish the causal impact of varying ambiguity of microaggressions on clients. This study uses an experimental analogue design to examine offensiveness, emotional reactions, and evaluations of the interaction across three distinct levels of microaggression statements: subtle, moderate, and overt. We recruited 158 adult African American participants and randomly assigned them to watch a brief counseling vignette. We found significant differences between the control and three microaggression statements on all outcome variables. We did not find significant differences between the microaggression conditions. This study, in conjunction with previous correlational research, highlights the detrimental impact of microaggressions within psychotherapy, regardless of racially explicit content. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Threat alert: The effect of outliers on the alliance-outcome correlation.

Meta-analyses have established the alliance as the most robust predictor of outcome in psychotherapy. A growing number of studies have evaluated potential threats to the conclusion that alliance is a causal factor in psychotherapy. One potential threat that has not been systematically examined is the possibility that the alliance-outcome association is driven by low alliance outliers. We examined the influence of removing low alliance outliers on the alliance-outcome association using data drawn from two large-scale, naturalistic psychotherapy data sets (Ns = 1,052; 11,029). These data sets differed in setting (university counseling center, community mental health center), country (United States and Canada), alliance measure (four-item Working Alliance Inventory Short Form Revised, 10-item Session Rating Scale), and outcome measure (Counseling Center Assessment of Psychological Symptoms-34, Outcome Questionnaire-45). We examined the impact of treating outliers in five different ways: retaining them, removing values three or two standard deviations from the mean, and winsorizing values three or two standard deviations from the mean. We also examined the effect of outliers after disaggregating alliance ratings into within-therapist and between-therapist components. The alliance-outcome correlation and the proportion of variance in posttest outcomes explained by alliance when controlling for pretest outcomes were similar regardless of how low alliance outliers were treated (change in r ≤ .04, change in R² ≤ 1%). Results from the disaggregation were similar. Thus, it appears that the alliance-outcome association is not an artifact of the influence of low alliance outliers. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Facilitating dyadic synchrony in psychotherapy sessions: Systematic review and meta-analysis.

Objective: This paper highlights the facilitation of dyadic synchrony as a core psychotherapist skill that occurs at the non-verbal level and underlies many other therapeutic methods. We define dyadic synchrony, differentiate it from similar constructs, and provide an excerpt illustrating dyadic synchrony in a psychotherapy session. Method: We then present a systematic review of 17 studies that have examined the associations between dyadic synchrony and psychotherapy outcomes. We also conduct a meta-analysis of 8 studies that examined whether there is more synchrony between clients and therapists than would be expected by chance. Results: Weighted box score analysis revealed that the overall association of synchrony and proximal as well as distal outcomes was neutral to mildly positive. The results of the meta-analysis indicated that real client-therapist dyad pairs exhibited synchronized behavioral patterns to a much greater extent than a sample of randomly paired people who did not actually speak. Conclusion: Our discussion revolves around how synchrony can be facilitated in a beneficial way, as well as situations in which it may not be beneficial. We conclude with training implications and therapeutic practices.

Enhancing the quality of cognitive behavioral therapy in community mental health through artificial intelligence generated fidelity feedback (Project AFFECT): a study protocol.

BACKGROUND: Each year, millions of Americans receive evidence-based psychotherapies (EBPs) like cognitive behavioral therapy (CBT) for the treatment of mental and behavioral health problems. Yet, at present, there is no scalable method for evaluating the quality of psychotherapy services, leaving EBP quality and effectiveness largely unmeasured and unknown. Project AFFECT will develop and evaluate an AI-based software system to automatically estimate CBT fidelity from a recording of a CBT session. Project AFFECT is an NIMH-funded research partnership between the Penn Collaborative for CBT and Implementation Science and Lyssn.io, Inc. ("Lyssn") a start-up developing AI-based technologies that are objective, scalable, and cost efficient, to support training, supervision, and quality assurance of EBPs. Lyssn provides HIPAA-compliant, cloud-based software for secure recording, sharing, and reviewing of therapy sessions, which includes AI-generated metrics for CBT. The proposed tool will build from and be integrated into this core platform. METHODS: Phase I will work from an existing software prototype to develop a LyssnCBT user interface geared to the needs of community mental health (CMH) agencies. Core activities include a user-centered design focus group and interviews with community mental health therapists, supervisors, and administrators to inform the design and development of LyssnCBT. LyssnCBT will be evaluated for usability and implementation readiness in a final stage of Phase I. Phase II will conduct a stepped-wedge, hybrid implementation-effectiveness randomized trial (N = 1,875 clients) to evaluate the effectiveness of LyssnCBT to improve therapist CBT skills and client outcomes and reduce client drop-out. Analyses will also examine the hypothesized mechanism of action underlying LyssnCBT. DISCUSSION: Successful execution will provide automated, scalable CBT fidelity feedback for the first time ever, supporting high-quality training, supervision, and quality assurance, and providing a core technology foundation that could support the quality delivery of a range of EBPs in the future. TRIAL REGISTRATION: ClinicalTrials.gov; NCT05340738 ; approved 4/21/2022.

Automated evaluation of psychotherapy skills using speech and language technologies.

With the growing prevalence of psychological interventions, it is vital to have measures which rate the effectiveness of psychological care to assist in training, supervision, and quality assurance of services. Traditionally, quality assessment is addressed by human raters who evaluate recorded sessions along specific dimensions, often codified through constructs relevant to the approach and domain. This is, however, a cost-prohibitive and time-consuming method that leads to poor feasibility and limited use in real-world settings. To facilitate this process, we have developed an automated competency rating tool able to process the raw recorded audio of a session, analyzing who spoke when, what they said, and how the health professional used language to provide therapy. Focusing on a use case of a specific type of psychotherapy called "motivational interviewing", our system gives comprehensive feedback to the therapist, including information about the dynamics of the session (e.g., therapist's vs. client's talking time), low-level psychological language descriptors (e.g., type of questions asked), as well as other high-level behavioral constructs (e.g., the extent to which the therapist understands the clients' perspective). We describe our platform and its performance using a dataset of more than 5000 recordings drawn from its deployment in a real-world clinical setting used to assist training of new therapists. Widespread use of automated psychotherapy rating tools may augment experts' capabilities by providing an avenue for more effective training and skill improvement, eventually leading to more positive clinical outcomes.

Knowledge and Attitudes Toward an Artificial Intelligence-Based Fidelity Measurement in Community Cognitive Behavioral Therapy Supervision.

To capitalize on investments in evidence-based practices, technology is needed to scale up fidelity assessment and supervision. Stakeholder feedback may facilitate adoption of such tools. This evaluation gathered stakeholder feedback and preferences to explore whether it would be fundamentally feasible or possible to implement an automated fidelity-scoring supervision tool in community mental health settings. A partially mixed, sequential research method design was used including focus group discussions with community mental health therapists (n = 18) and clinical leadership (n = 12) to explore typical supervision practices, followed by discussion of an automated fidelity feedback tool embedded in a cloud-based supervision platform. Interpretation of qualitative findings was enhanced through quantitative measures of participants' use of technology and perceptions of acceptability, appropriateness, and feasibility of the tool. Initial perceptions of acceptability, appropriateness, and feasibility of automated fidelity tools were positive and increased after introduction of an automated tool. Standard supervision was described as collaboratively guided and focused on clinical content, self-care, and documentation. Participants highlighted the tool's utility for supervision, training, and professional growth, but questioned its ability to evaluate rapport, cultural responsiveness, and non-verbal communication. Concerns were raised about privacy and the impact of low scores on therapist confidence. Desired features included intervention labeling and transparency about how scores related to session content. Opportunities for asynchronous, remote, and targeted supervision were particularly valued. Stakeholder feedback suggests that automated fidelity measurement could augment supervision practices. Future research should examine the relations among use of such supervision tools, clinician skill, and client outcomes.

Psychotherapy racial/ethnic disparities in treatment outcomes: The role of university racial/ethnic composition.

OBJECTIVE: Mental health disparities between racial/ethnic minorities (REM) and White individuals are well documented. These disparities extend into psychotherapy and have been observed among clients receiving care at university/college counseling centers. However, less is known about if campus RE composition affects outcomes from psychotherapy for REM and White clients. METHOD: This study examined psychotherapy outcomes from 16,011 clients who engaged in services at 33 university/college counseling centers. Each of these clients completed the Behavioral Health Measure as a of part routine practice. Campus RE composition was coded from publicly available data. RESULTS: The results demonstrated that White clients had better therapy outcomes than REM clients when they were at campuses where there were more White students. For universities 1 SD below the mean percentage of White students, the average difference in therapy outcomes for White and REM clients was Cohen's d = .21 (with White students experiencing more improvement); however, for universities 1 SD above the mean, the between group outcome disparity was greater (Cohen's d = .38). CONCLUSION: Therapists and higher education professionals should consider environmental impacts on counseling services. Implications for higher education, counseling centers, and mental health disparities are provided. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Determining the therapist's contribution in therapist multicultural competence process and outcome.

Efforts to help therapists improve their multicultural competence (MCC) rely on measures that can distinguish between different levels of competence. MCC is often assessed by asking clients to rate their experiences with their therapists. However, differences in client ratings of therapist MCC do not necessarily provide information about the relative performance of therapists and can be influenced by other factors including the client's own characteristics. In this study, we used a repeated measures design of 8,497 observations from 1,458 clients across 35 therapists to clarify the proportion of variability in MCC ratings attributed to the therapist versus the client and better understand the extent that an MCC measure detects therapist differences. Overall, we found that a small amount of variability in MCC ratings was attributed to the therapist (2%) and substantial amount attributed to the client (70%). These findings suggest that our measure of MCC primarily detected differences at the client level versus therapist level, indicating that therapist MCC scores were largely dependent on the client. Clinical implications and recommendations for future MCC research and measurement are discussed. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

How do you feel? Using natural language processing to automatically rate emotion in psychotherapy.

Emotional distress is a common reason for seeking psychotherapy, and sharing emotional material is central to the process of psychotherapy. However, systematic research examining patterns of emotional exchange that occur during psychotherapy sessions is often limited in scale. Traditional methods for identifying emotion in psychotherapy rely on labor-intensive observer ratings, client or therapist ratings obtained before or after sessions, or involve manually extracting ratings of emotion from session transcripts using dictionaries of positive and negative words that do not take the context of a sentence into account. However, recent advances in technology in the area of machine learning algorithms, in particular natural language processing, have made it possible for mental health researchers to identify sentiment, or emotion, in therapist-client interactions on a large scale that would be unattainable with more traditional methods. As an attempt to extend prior findings from Tanana et al. (2016), we compared their previous sentiment model with a common dictionary-based psychotherapy model, LIWC, and a new NLP model, BERT. We used the human ratings from a database of 97,497 utterances from psychotherapy to train the BERT model. Our findings revealed that the unigram sentiment model (kappa = 0.31) outperformed LIWC (kappa = 0.25), and ultimately BERT outperformed both models (kappa = 0.48).

Can a computer detect interpersonal skills? Using machine learning to scale up the Facilitative Interpersonal Skills task.

Objective: Therapist interpersonal skills are foundational to psychotherapy. However, assessment is labor intensive and infrequent. This study evaluated if machine learning (ML) tools can automatically assess therapist interpersonal skills. Method: Data were drawn from a previous study in which 164 undergraduate students (i.e., not clinical trainees) completed the Facilitative Interpersonal Skills (FIS) task. This task involves responding to video vignettes depicting interpersonally challenging moments in psychotherapy. Trained raters scored the responses. We used an elastic net model on top of a term frequency-inverse document frequency representation to predict FIS scores. Results: Models predicted FIS total and item-level scores above chance (rhos = .27-.53, ps < .001), achieving 31-60% of human reliability. Models explained 13-24% of the variance in FIS total and item-level scores on a held out set of data (R2), with the exception of the two items most reliant on vocal cues (verbal fluency, emotional expression), for which models explained ≤1% of variance. Conclusion: ML may be a promising approach for automating assessment of constructs like interpersonal skill previously coded by humans. ML may perform best when the standardized stimuli limit the "space" of potential responses (vs. naturalistic psychotherapy) and when models have access to the same data available to raters (i.e., transcripts).

Machine learning and natural language processing in psychotherapy research: Alliance as example use case.

Artificial intelligence generally and machine learning specifically have become deeply woven into the lives and technologies of modern life. Machine learning is dramatically changing scientific research and industry and may also hold promise for addressing limitations encountered in mental health care and psychotherapy. The current paper introduces machine learning and natural language processing as related methodologies that may prove valuable for automating the assessment of meaningful aspects of treatment. Prediction of therapeutic alliance from session recordings is used as a case in point. Recordings from 1,235 sessions of 386 clients seen by 40 therapists at a university counseling center were processed using automatic speech recognition software. Machine learning algorithms learned associations between client ratings of therapeutic alliance exclusively from session linguistic content. Using a portion of the data to train the model, machine learning algorithms modestly predicted alliance ratings from session content in an independent test set (Spearman's ρ = .15, p < .001). These results highlight the potential to harness natural language processing and machine learning to predict a key psychotherapy process variable that is relatively distal from linguistic content. Six practical suggestions for conducting psychotherapy research using machine learning are presented along with several directions for future research. Questions of dissemination and implementation may be particularly important to explore as machine learning improves in its ability to automate assessment of psychotherapy process and outcome. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Coregulation of therapist and client emotion during psychotherapy.

OBJECTIVE: Close interpersonal relationships are fundamental to emotion regulation. Clinical theory suggests that one role of therapists in psychotherapy is to help clients regulate emotions, however, if and how clients and therapists serve to regulate each other's emotions has not been empirically tested. Emotion coregulation - the bidirectional emotional linkage of two people that promotes emotional stability - is a specific, temporal process that provides a framework for testing the way in which therapists' and clients' emotions may be related on a moment to moment basis in clinically relevant ways. METHOD: Utilizing 227 audio recordings from a relationally oriented treatment (Motivational Interviewing), we estimated continuous values of vocally encoded emotional arousal via mean fundamental frequency. We used dynamic systems models to examine emotional coregulation, and tested the hypothesis that each individual's emotional arousal would be significantly associated with fluctuations in the other's emotional state over the course of a psychotherapy session. RESULTS: Results indicated that when clients became more emotionally labile over the course of the session, therapists became less so. When changes in therapist arousal increased, the client's tendency to become more aroused during session slowed. Alternatively, when changes in client arousal increased, the therapist's tendency to become less aroused slowed.

Development and Evaluation of ClientBot: Patient-Like Conversational Agent to Train Basic Counseling Skills.

BACKGROUND: Training therapists is both expensive and time-consuming. Degree-based training can require tens of thousands of dollars and hundreds of hours of expert instruction. Counseling skills practice often involves role-plays, standardized patients, or practice with real clients. Performance-based feedback is critical for skill development and expertise, but trainee therapists often receive minimal and subjective feedback, which is distal to their skill practice. OBJECTIVE: In this study, we developed and evaluated a patient-like neural conversational agent, which provides real-time feedback to trainees via chat-based interaction. METHODS: The text-based conversational agent was trained on an archive of 2354 psychotherapy transcripts and provided specific feedback on the use of basic interviewing and counseling skills (ie, open questions and reflections-summary statements of what a client has said). A total of 151 nontherapists were randomized to either (1) immediate feedback on their use of open questions and reflections during practice session with ClientBot or (2) initial education and encouragement on the skills. RESULTS: Participants in the ClientBot condition used 91% (21.4/11.2) more reflections during practice with feedback (P<.001) and 76% (14.1/8) more reflections after feedback was removed (P<.001) relative to the control group. The treatment group used more open questions during training but not after feedback was removed, suggesting that certain skills may not improve with performance-based feedback. Finally, after feedback was removed, the ClientBot group used 31% (32.5/24.7) more listening skills overall (P<.001). CONCLUSIONS: This proof-of-concept study demonstrates that practice and feedback can improve trainee use of basic counseling skills.

The impact of race and ethnicity on rates of return to psychotherapy for depression.

BACKGROUND: There are many limitations with the evidence base for the role of race and ethnicity in continuation of psychotherapy for depression. METHODS: The study sample consisted of 242,765 patients ≥ 18 years old from six healthcare systems in the Mental Health Research Network (MHRN) who had a new episode of psychotherapy treatment for depression between 1/1/2010 and 12/31/2013. Data were from electronic medical records and organized in a Virtual Data Warehouse (VDW). The odds of racial and ethnic minority patients returning for a second psychotherapy visit within 45 days of the initial session were examined using multilevel regression. RESULTS: The sample was primarily middle aged (68%, 30-64 years old), female (68.5%), and non-Hispanic white (50.7%), had commercial insurance (81.4%), and a low comorbidity burden (68.8% had no major comorbidities). Return rates within 45 days of the first psychotherapy visit were 47.6%. Compared to their non-Hispanic white counterparts, racial and ethnic minority patients were somewhat less likely to return to psychotherapy for a second visit (adjusted odds ratios [aORs] ranged from 0.80 to 0.90). Healthcare system was a much stronger predictor of return rates (aORs ranged from 0.89 to 5.53), while providers accounted for 21.1% of the variance in return rates. CONCLUSIONS: Provider and healthcare system variation were stronger predictors of patient return to psychotherapy than race and ethnicity. More research is needed to understand why providers and healthcare systems determine psychotherapy return rates for patients of all racial and ethnic groups.

Introduction to special section on advanced methodology: Counseling the dog to wag its methodological tail.

In this article, we introduce a special section focused on the application of advanced methodologies to specific research questions in counseling psychology. The articles include applications of natural language processing, dynamic systems, mediation analyses in single studies and meta-analysis, and synthesis of qualitative research. We provide a brief overview of each article. (PsycINFO Database Record

Technology-enhanced human interaction in psychotherapy.

Psychotherapy is on the verge of a technology-inspired revolution. The concurrent maturation of communication, signal processing, and machine learning technologies begs an earnest look at how these technologies may be used to improve the quality of psychotherapy. Here, we discuss 3 research domains where technology is likely to have a significant impact: (1) mechanism and process, (2) training and feedback, and (3) technology-mediated treatment modalities. For each domain, we describe current and forthcoming examples of how new technologies may change established applications. Moreover, for each domain we present research questions that touch on theoretical, systemic, and implementation issues. Ultimately, psychotherapy is a decidedly human endeavor, and thus the application of modern technology to therapy must capitalize on-and enhance-our human capacities as counselors, students, and supervisors. (PsycINFO Database Record

Computational Analysis and Simulation of Empathic Behaviors: a Survey of Empathy Modeling with Behavioral Signal Processing Framework.

Empathy is an important psychological process that facilitates human communication and interaction. Enhancement of empathy has profound significance in a range of applications. In this paper, we review emerging directions of research on computational analysis of empathy expression and perception as well as empathic interactions, including their simulation. We summarize the work on empathic expression analysis by the targeted signal modalities (e.g., text, audio, and facial expressions). We categorize empathy simulation studies into theory-based emotion space modeling or application-driven user and context modeling. We summarize challenges in computational study of empathy including conceptual framing and understanding of empathy, data availability, appropriate use and validation of machine learning techniques, and behavior signal processing. Finally, we propose a unified view of empathy computation and offer a series of open problems for future research.

Introduction to the special section "Big'er' Data": Scaling up psychotherapy research in counseling psychology.

This article introduces the special section on utilizing large data sets to explore psychotherapy processes and outcomes. The increased use of technology has provided new opportunities for psychotherapy researchers. In particular, there is a rise in large databases of tens of thousands clients. Additionally, there are new ways to pool valuable resources for meta-analytic processes. At the same time, these tools also come with limitations. These issues are introduced as well as brief overview of the articles.

"It sounds like...": A natural language processing approach to detecting counselor reflections in motivational interviewing.

The dissemination and evaluation of evidence-based behavioral treatments for substance abuse problems rely on the evaluation of counselor interventions. In Motivational Interviewing (MI), a treatment that directs the therapist to utilize a particular linguistic style, proficiency is assessed via behavioral coding-a time consuming, nontechnological approach. Natural language processing techniques have the potential to scale up the evaluation of behavioral treatments such as MI. We present a novel computational approach to assessing components of MI, focusing on 1 specific counselor behavior-reflections, which are believed to be a critical MI ingredient. Using 57 sessions from 3 MI clinical trials, we automatically detected counselor reflections in a maximum entropy Markov modeling framework using the raw linguistic data derived from session transcripts. We achieved 93% recall, 90% specificity, and 73% precision. Results provide insight into the linguistic information used by coders to make ratings and demonstrate the feasibility of new computational approaches to scaling up the evaluation of behavioral treatments.

A meta-analysis of multicultural competencies and psychotherapy process and outcome.

For decades, psychologists have emphasized the provision of multiculturally competent psychotherapy to reduce racial and ethnic disparities in mental health treatment. However, the relationship between multicultural competencies (MC) and other measures of clinical process and treatment outcome has shown heterogeneity in effect sizes. This meta-analysis tested the association of client ratings of therapist MC with measures of therapeutic processes and outcome, including: (a) working alliance, (b) client satisfaction, (c) general counseling competence, (d) session impact, and (e) symptom improvement. Among 18 studies (20 independent samples) included in the analysis, the correlation between therapist MC and outcome (r = .29) was much smaller than the association with process measures (r = .75), but there were no significant differences in correlations across different types of MC or clinical process measures. Providing some evidence of publication bias, effect sizes from published studies (r = .67) were larger than those from unpublished dissertations (r = .28). Moderator analyses indicated that client age, gender, the representation of racial-ethnic minority (R-EM) clients, and clinical setting were not associated with effect size variability. Based on these findings, we discuss implications and recommendations for future research that might lead to a better understanding of the effects of therapist MC on treatment process and outcome. Primary needs in future research include the development and evaluation of observer ratings of therapist MC and the implementation of longitudinal research designs.