Taking a Magnifying Glass to Measurement-Based Care Consultation Sessions: with What Issues Do Mental Health Clinicians Struggle?

PURPOSE: Ongoing consultation following initial training is one of the most commonly deployed implementation strategies to facilitate uptake of evidence-based practices, such as measurement-based care (MBC). Group consultation provides an interactive experience with an expert and colleagues to get feedback on actual issues faced, yet there is little research that unpacks the questions raised in consultation and what types of issues are important to address. METHODS: The current study characterized the questions and concerns raised by community mental health clinicians (N = 38 across six clinics) during group consultation sessions completed as part of an MBC implementation trial. We conducted a qualitative content analysis of consultation forms completed by clinicians before each MBC consultation session. RESULTS: Clinicians sought MBC consultation for clients across a range of ages and levels of depression severity. Qualitative results revealed five main questions and concerns in consultation sessions: (1) how to administer the PHQ-9, (2) how to review PHQ-9 scores, (3) how to respond to PHQ-9 score, (4) the types of clients for whom MBC would be appropriate, and (5) how MBC could impact a clinician's usual care. CONCLUSION: Findings highlight the need for ongoing consultation and limitations of workshop training alone. Practical recommendations for addressing the common questions and concerns identified are presented to support MBC use.

Social Risk Factors and Desire for Assistance Among Patients Receiving Subsidized Health Care Insurance in a US-Based Integrated Delivery System.

PURPOSE: Because social conditions such as food insecurity and housing instability shape health outcomes, health systems are increasingly screening for and addressing patients' social risks. This study documented the prevalence of social risks and examined the desire for assistance in addressing those risks in a US-based integrated delivery system. METHODS: A survey was administered to Kaiser Permanente members on subsidized exchange health insurance plans (2018-2019). The survey included questions about 4 domains of social risks, desire for help, and attitudes. We conducted a descriptive analysis and estimated multivariate modified Poisson regression models. RESULTS: Of 438 participants, 212 (48%) reported at least 1 social risk factor. Housing instability was the most common (70%) factor reported. Members with social risks reported more discomfort being screened for social risks (14.2% vs 5.4%; P = .002) than those without risks, although 90% of participants believed that health systems should assist in addressing social risks. Among those with 1-2 social risks, however, only 27% desired assistance. Non-Hispanic Black participants who reported a social risk were more than twice as likely to desire assistance compared with non-Hispanic White participants (adjusted relative risk [RR] 2.2; 95% CI, 1.3-3.8). CONCLUSIONS: Athough most survey participants believed health systems have a role in addressing social risks, a minority of those reporting a risk wanted assistance and reported more discomfort being screened for risk factors than those without risks. Health systems should work to increase the comfort of patients in reporting risks, explore how to successfully assist them when desired, and offer resources to address these risks outside the health care sector.VISUAL ABSTRACT.

Social risks and social needs in a health insurance exchange sample: a longitudinal evaluation of utilization.

BACKGROUND: Health systems are increasingly attempting to intervene on social adversity as a strategy to improve health care outcomes. To inform health system efforts to screen for social adversity, we sought to explore the stability of social risk and interest in assistance over time and to evaluate whether the social risk was associated with subsequent healthcare utilization. METHODS: We surveyed Kaiser Permanente members receiving subsidies from the healthcare exchange in Southern California to assess their social risk and desire for assistance using the Accountable Health Communities instrument. A subset of initial respondents was randomized to be re-surveyed at either three or six months later. RESULTS: A total of 228 participants completed the survey at both time points. Social risks were moderate to strongly stable across three and six months (Kappa range = .59-.89); however, social adversity profiles that included participants' desire for assistance were more labile (3-month Kappa = .52; 95% CI = .41-.64 & 6-month Kappa = .48; 95% CI = .36-.6). Only housing-related social risks were associated with an increase in acute care (emergency, urgent care) six months after initial screening; no other associations between social risk and utilization were observed. CONCLUSIONS: This study suggests that screening for social risk may be appropriate at intervals of six months, or perhaps longer, but that assessing desire for assistance may need to occur more frequently. Housing risks were associated with increases in acute care. Health systems may need to engage in screening and referral to resources to improve overall care and ultimately patient total health.

Implementation of the Wolverine Mental Health Program, Part 2: Implementation Phase.

To enhance mental health care for youth in a midwestern residential treatment facility, Wolverine Human Services partnered with the Beck Institute (an intermediary) and an implementation research team to implement cognitive-behavioral therapy (CBT). CBT has strong evidence supporting effectiveness for treating youth internalizing and externalizing problems, but it is a complex psychosocial intervention that demands a thoughtful implementation approach. This study outlines the implementation phase (2.5 years) of a 5-year collaborative effort. The implementation phase focused on (a) adapting CBT to fit the complex youth needs and the roles of the multidisciplinary team members resulting in a new comprehensive and coordinated care model, and (b) the strategies utilized to support its competent integration by all team members. Six blended implementation strategies were deployed in this phase: forging implementation teams, installing progress monitoring, adapting CBT, training, providing supervision and consultation, and training the trainers. A components-based approach to CBT yielded six core skills: active listening, problem solving, mood monitoring and intervention mapping, activity scheduling, distress tolerance, and cognitive restructuring. By the end of this phase, all staff had robust exposure to and experience with the adapted form of CBT. The work of our academic-community partnership has both research and clinical implications, with respect to integrating an adapted version of CBT for residential environments (CBT-RE).

Implementation of the Wolverine Mental Health Program, Part 3: Sustainment Phase.

Sustaining the implementation of an evidence-based practice (EBP) is the ultimate goal of often years of significant personnel and financial investment. Some conceptualize sustainment as a distinct phase following an active implementation period where the contextual factors, processes, and supports are bolstered to ensure continued EBP delivery. This study provides an overview of the sustainment strategies deployed to embed cognitive-behavioral therapy (CBT) in a Midwestern residential treatment facility serving youth with complex mental health needs. Seven key strategies and their outcomes are described: use of CBT teams, new hire orientation plans, monthly campaigns, change in job descriptions and performance evaluations, development of a behavioral reinforcement system for youth, and a pathway to CBT certification. This study provides a window into how one might sustain an EBP by addressing barriers unique to this phase of work.

Implementation of the Wolverine Mental Health Program, Part 1: Adoption Phase.

Residential treatment facilities (RTFs) are a first-line treatment option for juvenile justice-involved youth. However, RTFs rarely offer evidence-based interventions for youth with internalizing or externalizing mental health problems. Wolverine Human Services (WHS) is one of the first RTFs in the nation to implement cognitive-behavioral therapy (CBT) to enhance mental health care for their youth. This study outlines the preimplementation phase of a 5-year collaborative CBT implementation effort among WHS, the Beck Institute, and an implementation science research team. The preimplementation phase included a needs assessment across two sites of WHS to identify and prioritize barriers to CBT implementation. Of the 76 unique barriers, 23 were prioritized as important and feasible to address. Implementation teams, consisting of clinician and staff champions and opinion leaders, worked across 8 months to deploy 10 strategies from a collaboratively designed blueprint. Upon reevaluation of the needs assessment domains, all prioritized barriers to CBT implementation were removed and WHS's readiness for CBT implementation was enhanced. This study serves as a model of a preimplementation process that can be employed to enhance the potential for successful evidence-based practice implementation in youth RTFs.

MFA and ASA-MF: A Psychometric Analysis of Attitudes Towards Measurement-Based Care.

Measurement based care (MBC) improves client outcomes by providing clinicians with routine mental health outcome data that can be used to inform treatment planning but is rarely used in practice. The Monitoring and Feedback Attitudes Scale (MFA) and Attitudes Towards Standardized Assessment Scales-Monitoring and Feedback (ASA-MF) (Jensen-Doss et al., 2016) may identify attitudinal barriers to MBC, which could help trainings and implementation strategies. This study examines the psychometric properties of the MFA and ASA-MF, including the factor structure, longitudinal invariance, and indicators of validity, in a sample of community mental health clinicians (N = 164). The measures demonstrate adequate fit to their factor structures across time and predict MBC use as captured in a client's electronic health record. Given that clinician attitudes are associated with MBC use, using instruments with psychometric support to assess attitudes fills a research to practice gap.

A Guideline for Reporting Mediation Analyses of Randomized Trials and Observational Studies: The AGReMA Statement.

Importance: Mediation analyses of randomized trials and observational studies can generate evidence about the mechanisms by which interventions and exposures may influence health outcomes. Publications of mediation analyses are increasing, but the quality of their reporting is suboptimal. Objective: To develop international, consensus-based guidance for the reporting of mediation analyses of randomized trials and observational studies (A Guideline for Reporting Mediation Analyses; AGReMA). Design, Setting, and Participants: The AGReMA statement was developed using the Enhancing Quality and Transparency of Health Research (EQUATOR) methodological framework for developing reporting guidelines. The guideline development process included (1) an overview of systematic reviews to assess the need for a reporting guideline; (2) review of systematic reviews of relevant evidence on reporting mediation analyses; (3) conducting a Delphi survey with panel members that included methodologists, statisticians, clinical trialists, epidemiologists, psychologists, applied clinical researchers, clinicians, implementation scientists, evidence synthesis experts, representatives from the EQUATOR Network, and journal editors (n = 19; June-November 2019); (4) having a consensus meeting (n = 15; April 28-29, 2020); and (5) conducting a 4-week external review and pilot test that included methodologists and potential users of AGReMA (n = 21; November 2020). Results: A previously reported overview of 54 systematic reviews of mediation studies demonstrated the need for a reporting guideline. Thirty-three potential reporting items were identified from 3 systematic reviews of mediation studies. Over 3 rounds, the Delphi panelists ranked the importance of these items, provided 60 qualitative comments for item refinement and prioritization, and suggested new items for consideration. All items were reviewed during a 2-day consensus meeting and participants agreed on a 25-item AGReMA statement for studies in which mediation analyses are the primary focus and a 9-item short-form AGReMA statement for studies in which mediation analyses are a secondary focus. These checklists were externally reviewed and pilot tested by 21 expert methodologists and potential users, which led to minor adjustments and consolidation of the checklists. Conclusions and Relevance: The AGReMA statement provides recommendations for reporting primary and secondary mediation analyses of randomized trials and observational studies. Improved reporting of studies that use mediation analyses could facilitate peer review and help produce publications that are complete, accurate, transparent, and reproducible.

How Do Peers Shape Mental Health Clinicians' Attitudes Toward New Treatments?

Through everyday interactions, mental health clinicians are exposed to their colleagues' views toward new treatments, which can influence clinicians' own attitudes and implementation especially in high-stress environments. This study examines how exposure to peers' attitudes in the workplace through three common workplace interactions (advice sharing, discussion, and friendship) shapes clinicians' (n = 163) own attitudes toward measurement-based care (MBC). Clinicians tended to have more positive attitudes toward MBC with greater exposure to peers with positive attitudes through advice-sharing and informal workplace discussions, but not through friendships. Results highlight the importance of strong workplace relationships for implementation.

What Gets Measured Gets Done: How Mental Health Agencies can Leverage Measurement-Based Care for Better Patient Care, Clinician Supports, and Organizational Goals.

Mental health clinicians and administrators are increasingly asked to collect and report treatment outcome data despite numerous challenges to select and use instruments in routine practice. Measurement-based care (MBC) is an evidence-based practice for improving patient care. We propose that data collected from MBC processes with patients can be strategically leveraged by agencies to also support clinicians and respond to accountability requirements. MBC data elements are outlined using the Precision Mental Health Framework (Bickman et al. in Adm Policy Mental Health Mental Health Serv Res 43:271-276, 2016), practical guidance is provided for agency administrators, and conceptual examples illustrate strategic applications of one or more instruments to meet various needs throughout the organization.

Ready or Not? Transitions of Depressed Adolescents During Acute Phase of Treatment.

Readiness to change has been identified as a predictor, moderator, and mediator of treatment. Individuals may start treatment in one stage and either stay, regress, or progress across stages, but there is little research exploring these transitions within mental health treatment. The present study addressed two aims: characterize the prevalence of stage membership and transitions, and explore predictors of stage membership and transitions. A Treatment for Adolescents with Depression Study sub-sample was used and participants (n = 383) ranged in age from 12 to 17, with a primary diagnosis of Major Depressive Disorder. The 18-item self-report Stages of Change Questionnaire was administered at baseline and week 6 of treatment. A latent transition analysis determined stage membership and transitions. Most adolescents initiated treatment in precontemplation or contemplation, and hopelessness predicted stage membership and stage transitions. This study revealed that readiness to change and hopelessness are related within the first few weeks of treatment, which may have implications for depressed adolescent's ability to benefit from care.

A meta-analysis of the effect of therapist experience on outcomes for clients with internalizing disorders.

Objective: This meta-analysis synthesized the literature regarding the effect of therapist experience on internalizing client outcomes to evaluate the utility of lay providers in delivering treatment and to inform therapist training. Method: The analysis included 22 studies, contributing 208 effect sizes. Study and client characteristics were coded to examine moderators. We conducted subgroup meta-analyses examining the relationship of therapist experience across a diverse set of internalizing client outcomes. Results: Results demonstrated a small, but significant relationship between therapist experience and internalizing client outcomes. There was no relationship between therapist experience and outcomes in clients with primary anxiety disorders. In samples of clients with primary depressive disorders and in samples of clients with mixed internalizing disorders, there was a significant relationship between experience and outcomes. The relationship between therapist experience and outcomes was stronger when clients were randomized to therapists, treatment was not manualized, and for measures of client satisfaction and "other" outcomes (e.g., dropout). Conclusions: It appears that therapist experience may matter for internalizing clients under certain circumstances, but this relationship is modest. Continuing methodological concerns in the literature are noted, as well as recommendations to address these concerns.

Patterns of practice in community mental health treatment of adult depression.

OBJECTIVE: Community mental health therapists often endorse an eclectic orientation, but few studies reveal how therapists utilize elements of evidence-based psychotherapies. This study aimed to characterize treatment as usual patterns of practice among therapists treating depressed adults in community mental health settings. METHOD: Therapists (N = 165) from the USA's largest not-for-profit provider of community-based mental health services completed surveys assessing their demographics and practice element use with depressed adult clients. Specifically, therapists indicated whether they utilized each of 45 unique practice elements from the following evidence-based psychotherapies: Acceptance and Commitment Therapy, Behavioral Activation, Brief Psychodynamic Therapy, Cognitive Behavioral Therapy (CBT), Interpersonal Therapy, Mindfulness-Based CBT, Problem-Solving Therapy, and Self-Control Therapy. Principal component analysis was employed to identify practice patterns. RESULTS: The principal component analysis included 31 practice elements and revealed a three-factor model with distinct patterns of practice that did not align with traditional evidence-based practice approaches, including: (i) Planning, Practice, and Monitoring; (ii) Cognitive, Didactic, and Interpersonal; and (iii) Between Session Activities. CONCLUSIONS: Therapist-reported practice patterns confirmed an eclectic approach that brought together elements from theoretically distinct evidence-based psychotherapies. Future research is needed to explore how these patterns of practice relate to client outcomes to inform focused training and/or de-implementation efforts.

Testing and Exploring the Limits of Traditional Notions of Fidelity and Adaptation in Implementation of Preventive Interventions.

This commentary reflects on key challenges raised across the articles of this special issue, notably the tension between fidelity and adaptation, the importance of articulating core components and principles of evidence-based programs, the need for pragmatic measures, and the challenges associated with articulating and testing mechanisms of implementation strategies. These challenges are amplified in the context of prevention research where task shifting, or revising professional roles, is especially common. Synergies with work emerging from the Society for Implementation Research Collaboration are highlighted throughout.

Operationalizing the 'pragmatic' measures construct using a stakeholder feedback and a multi-method approach.

CONTEXT: Implementation science measures are rarely used by stakeholders to inform and enhance clinical program change. Little is known about what makes implementation measures pragmatic (i.e., practical) for use in community settings; thus, the present study's objective was to generate a clinical stakeholder-driven operationalization of a pragmatic measures construct. EVIDENCE ACQUISITION: The pragmatic measures construct was defined using: 1) a systematic literature review to identify dimensions of the construct using PsycINFO and PubMed databases, and 2) interviews with an international stakeholder panel (N = 7) who were asked about their perspectives of pragmatic measures. EVIDENCE SYNTHESIS: Combined results from the systematic literature review and stakeholder interviews revealed a final list of 47 short statements (e.g., feasible, low cost, brief) describing pragmatic measures, which will allow for the development of a rigorous, stakeholder-driven conceptualization of the pragmatic measures construct. CONCLUSIONS: Results revealed significant overlap between terms related to the pragmatic construct in the existing literature and stakeholder interviews. However, a number of terms were unique to each methodology. This underscores the importance of understanding stakeholder perspectives of criteria measuring the pragmatic construct. These results will be used to inform future phases of the project where stakeholders will determine the relative importance and clarity of each dimension of the pragmatic construct, as well as their priorities for the pragmatic dimensions. Taken together, these results will be incorporated into a pragmatic rating system for existing implementation science measures to support implementation science and practice.

Cognitive Behavioural Therapy Competency: Pilot Data from a Comparison of Multiple Perspectives.

BACKGROUND: Measurement of cognitive behavioural therapy (CBT) competency is often resource intensive. A popular emerging alternative to independent observers' ratings is using other perspectives for rating competency. AIMS: This pilot study compared ratings of CBT competency from four perspectives - patient, therapist, supervisor and independent observer using the Cognitive Therapy Scale (CTS). METHOD: Patients (n = 12, 75% female, mean age 30.5 years) and therapists (n = 5, female, mean age 26.6 years) completed the CTS after therapy sessions, and clinical supervisor and independent observers rated recordings of the same session. RESULTS: Analyses of variance revealed that therapist average CTS competency ratings were not different from supervisor ratings, and supervisor ratings were not different from independent observer ratings; however, therapist ratings were higher than independent observer ratings and patient ratings were higher than all other raters. CONCLUSIONS: Raters differed in competency ratings. Implications for potential use and adaptation of CBT competency measurement methods to enhance training and implementation are discussed.

A conceptual model for generating and validating in-session clinical judgments.

OBJECTIVE: Little attention has been paid to the nuanced and complex decisions made in the clinical session context and how these decisions influence therapy effectiveness. Despite decades of research on the dual-processing systems, it remains unclear when and how intuitive and analytical reasoning influence the direction of the clinical session. METHOD: This paper puts forth a testable conceptual model, guided by an interdisciplinary integration of the literature, that posits that the clinical session context moderates the use of intuitive versus analytical reasoning. RESULTS: A synthesis of studies examining professional best practices in clinical decision-making, empirical evidence from clinical judgment research, and the application of decision science theories indicate that intuitive and analytical reasoning may have profoundly different impacts on clinical practice and outcomes. CONCLUSIONS: The proposed model is discussed with respect to its implications for clinical practice and future research.

Monitoring Treatment Progress and Providing Feedback is Viewed Favorably but Rarely Used in Practice.

Numerous trials demonstrate that monitoring client progress and using feedback for clinical decision-making enhances treatment outcomes, but available data suggest these practices are rare in clinical settings and no psychometrically validated measures exist for assessing attitudinal barriers to these practices. This national survey of 504 clinicians collected data on attitudes toward and use of monitoring and feedback. Two new measures were developed and subjected to factor analysis: The monitoring and feedback attitudes scale (MFA), measuring general attitudes toward monitoring and feedback, and the attitudes toward standardized assessment scales-monitoring and feedback (ASA-MF), measuring attitudes toward standardized progress tools. Both measures showed good fit to their final factor solutions, with excellent internal consistency for all subscales. Scores on the MFA subscales (Benefit, Harm) indicated that clinicians hold generally positive attitudes toward monitoring and feedback, but scores on the ASA-MF subscales (Clinical Utility, Treatment Planning, Practicality) were relatively neutral. Providers with cognitive-behavioral theoretical orientations held more positive attitudes. Only 13.9 % of clinicians reported using standardized progress measures at least monthly and 61.5 % never used them. Providers with more positive attitudes reported higher use, providing initial support for the predictive validity of the ASA-MF and MFA. Thus, while clinicians report generally positive attitudes toward monitoring and feedback, routine collection of standardized progress measures remains uncommon. Implications for the dissemination and implementation of monitoring and feedback systems are discussed.

What Did They Learn? Effects of a Brief Cognitive Behavioral Therapy Workshop on Community Therapists' Knowledge.

Knowledge gain has been identified as necessary but not sufficient for therapist behavior change. Declarative knowledge, or factual knowledge, is thought to serve as a prerequisite for procedural knowledge, the how to knowledge system, and reflective knowledge, the skill refinement system. The study aimed to examine how a 1-day workshop affected therapist cognitive behavioral therapy declarative knowledge. Participating community therapists completed a test before and after training that assessed cognitive behavioral therapy knowledge. Results suggest that the workshop significantly increased declarative knowledge. However, post-training total scores remained moderately low, with several questions answered incorrectly despite content coverage in the workshop. These findings may have important implications for structuring effective cognitive behavioral therapy training efforts and for the successful implementation of cognitive behavioral therapy in community settings.

Capabilities and Characteristics of Digital Measurement Feedback Systems: Results from a Comprehensive Review.

Measurement feedback systems (MFS) are a class of health information technology (HIT) that function as an implementation support strategy for integrating measurement based care or routine outcome monitoring into clinical practice. Although many MFS have been developed, little is known about their functions. This paper reports findings from an application of health information technology-academic and commercial evaluation (HIT-ACE), a systematic and consolidated evaluation method, to MFS designed for use in behavioral healthcare settings. Forty-nine MFS were identified and subjected to systematic characteristic and capability coding. Results are presented with respect to the representation of characteristics and capabilities across MFS.