Dose-Response Effects of MittEcho, a Measurement Feedback System, in an Indicated Mental Health Intervention for Children in Municipal and School Services in Norway.

Including routine client feedback can increase the effectiveness of mental health interventions for children, especially when implemented as intended. Rate of implementation, or dose, of such feedback interventions has been shown to moderate results in some studies. Variation in implementation and use of client feedback may also contribute to the mixed results observed within the feedback literature. This study evaluates dose-response associations of client feedback using a novel Measurement Feedback System (MFS) within an indicated group intervention. The primary aim was to determine whether the rate of MFS implementation predicts symptom reduction in anxiety and depression among school-aged children. The secondary aim was to assess whether the rate of MFS implementation influences children's satisfaction with the group intervention or their dropout rates. Data were collected via a randomized factorial study (clinicaltrials.gov NCT04263558) across 58 primary schools in Norway. Children aged 8 to 12 years (N = 701) participated in a group-based, transdiagnostic intervention targeting elevated symptoms of anxiety or depression. Half of the child groups also received the feedback intervention using the MittEcho MFS. Group leaders (N = 83), recruited locally, facilitated the interventions. The MFS dose was measured using the Implementation Index, which combines the use of MFS by both children and providers (group leaders) into a single dose variable. Results showed no significant additional effect of dose of MFS on change in depression or anxiety scores, on user satisfaction with the intervention or on intervention dropout. The discussion addresses potential reasons for these non-significant findings and implications for MFS implementation in preventive, group-based interventions in school settings.

Measurement Feedback System for Intensive Neurorehabilitation after Severe Acquired Brain Injury.

Outcome of acquired brain injury (ABI) and the potential for neurorehabilitation are subject to distinct heterogeneity between patients. Limited knowledge of the complex constellation of determinants at play interferes with the possibility to deploy precision medicine in neurorehabilitation. Measurement Feedback Systems (MFS) structure clinical data collection and deliver the measurement results as feedback to clinicians, thereby facilitating progress monitoring, promoting balanced patient-centered discussion and shared decision making. Accumulation of clinical data in the MFS also enables data-driven precision rehabilitation medicine. This article describes the development and implementation of a MFS for neurorehabilitation after ABI. The MFS consists of specialized measurement tracks which are developed together with representatives of each discipline in the multidisciplinary team. The MFS is built into a digital platform that automatically distributes measurements among clinicians, at predetermined time points during the inpatient treatment, outpatient treatment and follow-up. The results of all measurements are visualized in individual patient dashboards that are accessible for all clinicians involved in treatment. Since step-wise implementation, 124 patients have been registered on the MFS platform so far, providing an average of more than 200 new measurements per week. Currently, more than 15,000 clinical measurements are captured in the MFS. The current overall completion rate of measurements is 86,4%. This study shows that structured clinical assessment and feedback is feasible in the context of neurorehabilitation after severe ABI. The future directions are discussed for MFS data in our Health Intelligence Program, which aims at periodic care evaluation and the transition of neurorehabilitation care towards precision medicine.

Clinician perceptions of nomothetic and individualized patient-reported outcome measures in measurement-based care.

Objective: Measurement-based care (MBC), which encompasses routine outcome monitoring (ROM) and measurement feedback systems (MFSs), is an evidence-based practice (EBP) supporting treatment personalization and clinical responsiveness. Despite MBC's effectiveness, clinicians report reservations regarding its utility, which may be a function of overreliance on nomothetic (i.e., standardized) measures. Although research suggests that individualized (i.e., idiographic) patient-reported outcome measures (I-PROMs) may have the potential to overcome these obstacles, little is known regarding clinicians' perceptions of different measurement approaches to MBC. Methods: This study examined clinicians' perceptions of the clinical utility, relevance to treatment planning, and practicality of nomothetic, individualized, and combined clinical feedback provided by a simulated MFS. Three hundred and twenty-nine clinicians were randomized to one of three conditions that presented a clinical vignette comprising: (a) nomothetic, (b) individualized, or (c) combined clinical feedback. Results: Participants' perceptions of the clinical feedback were not affected by the measurement approach. However, cognitive behavioral participants reported more positive perceptions of all aspects associated with the clinical feedback. Conclusion: These results were consistent with previous findings, suggesting that clinicians' theoretical orientation may have a significant impact on their perceptions of MBC, and should be considered when designing and implementing these systems.

Improving measurement feedback systems for measurement-based care.

AbstractObjectives: Measurement feedback systems (MFS) are a class of health information technologies developed to facilitate measurement-based care. The individual clinical decision support features within MFS are diverse and their influence on clinicians is largely unknown. This study tested the impact of MFS features on clinicians' progress assessments and treatment decisions in different scenarios. Method: Clinicians (n = 299) were randomly assigned to view one of six combinations of the following MFS features: graph, expected change trajectory line, alert, and treatment suggestions. The assigned feature combination was paired with three vignettes and clinical data representing three clinical scenarios: patient deterioration, no progress, and approaching remission. Clinicians answered questions after each vignette, and at the conclusion. Results: MFS features differentially impacted clinicians' progress assessment accuracy, their likelihood of making a treatment change, and their treatment choices. Which feature was most impactful varied depending on the clinical scenario. Clinicians reported graphs influenced their assessments and choices significantly more than the other features, and the majority stated they would prefer to use all of the features. Conclusions: Specific MFS features impact clinicians' assessments and choices to greater degrees, and the impact of those features can be influenced by the clinical state of the patient.

Introducing a Measurement Feedback System for Youth Mental Health: Predictors and Impact of Implementation in a Community Agency.

Measurement feedback systems (MFSs) that routinely collect and report client progress to mental health therapists have demonstrated beneficial impact on outcomes in numerous studies, with evidence that there is a dose-response relationship related to the implementation of the MFS. The current study examined the impact of MFS implementation (Implementation Index) on youth symptom outcomes separately by caregiver and youth self-report. Additionally, we tested the extent to which Implementation Index rates varied by individual therapists and clients, and whether therapist and client characteristics predicted MFS implementation. Methods: Administrative data (client charts, youth- and caregiver-reported Youth Outcome Questionnaires) for 229 youth (52.83% Latinx, 42.79% girls, M age = 10.33) treated during a 1-year period at a community mental health organization in Central Texas were analyzed using multi-level modeling. Caregiver-reported symptoms decreased faster for those with a higher MFS Implementation Index. Between-group differences among therapists accounted for a significant proportion of variance in the Implementation Index for caregiver report, whereas client differences accounted for most of the variance in the Implementation Index for youth self-report. Therapist trainee status predicted a significant increase in the Implementation Index for caregiver-report data. Youth symptom improvement as reported by caregivers varied with the extent of MFS implementation fidelity, and MFS implementation fidelity was higher for clients treated by trainees relative to staff therapists for caregiver report of symptoms.

Middle and High School Student Perspectives on Digitally-Delivered Mental Health Assessments and Measurement Feedback Systems.

Implementation of measurement-based care (MBC) by child-serving community mental health providers, particularly school-based providers, is low. To inform user-centered design of measurement feedback systems (MFSs) and MBC implementation more broadly, semi-structured interviews were conducted with 61 middle and high school students. Interviews explored student preferences for and perceived helpfulness of different assessment methods and use of MFS in counseling. Results indicate that student preference for digitally-administered assessment is equivocal, with preferences being influenced by student perceptions of the ease of use, impersonalization, and confidentiality. Students with exposure to the MFS found it helpful when used by their provider to share assessment feedback.

Multilevel Predictors of Case Managers' Assessment Administration Behavior in a Precursor to a Measurement Feedback System.

This study examines factors associated with administration of a treatment monitoring assessment measure (the Ohio Scales) in the context of a precursor to a measurement feedback system in a youth public mental health setting. 82% of all state case managers (N = 46) completed interviews and administered at least one Ohio Scale over a 12-month period. A multi-level model accounting for variance between both case managers and their administrative offices indicated that case manager characteristics (lower self-reported burnout, more experience) and monthly caseload characteristics (fewer active cases, younger average age of youth, and increased time since initial administration) predicted increased monthly administration proportions.

Implementing for Sustainability: Promoting Use of a Measurement Feedback System for Innovation and Quality Improvement.

Measurement feedback systems (MFSs) are increasingly recognized as evidence-based treatments for improving mental health outcomes, in addition to being a useful administrative tool for service planning and reporting. Promising research findings have driven practice administrators and policymakers to emphasize the incorporation of outcomes monitoring into electronic health systems. To promote MFS integrity and protect against potentially negative outcomes, it is vital that adoption and implementation be guided by scientifically rigorous yet practical principles. In this point of view, the authors discuss and provide examples of three user-centered and theory-based principles: emphasizing integration with clinical values and workflow, promoting administrative leadership with the 'golden thread' of data-informed decision-making, and facilitating sustainability by encouraging innovation. In our experience, enacting these principles serves to promote sustainable implementation of MFSs in the community while also allowing innovation to occur, which can inform improvements to guide future MFS research.

Development and Piloting of a Classroom-Focused Measurement Feedback System.

The present study used a community partnered research method to develop and pilot a classroom-focused measurement feedback system (MFS) for school mental health providers to support teachers' use of effective universal and target classroom practices related to student emotional and behavioral issues. School personnel from seven urban elementary and middle school classrooms participated. Phase I involved development and refinement of the system through a baseline needs assessment and rapid-cycle feedback. Phase II involved detailed case study analysis of pre-to-post quantitative and implementation process data. Results suggest that teachers who used the dashboard along with consultation showed improvement in observed classroom organization and emotional support. Results also suggest that MFS use was tied closely to consultation dose, and that broader support at the school level was critical. Classroom-focused MFSs are a promising tool to support classroom improvement, and warrant future research focused on their effectiveness and broad applicability.

Facilitators and Barriers of Implementing a Measurement Feedback System in Public Youth Mental Health.

This study examines implementation facilitators and barriers of a statewide roll-out of a measurement feedback system (MFS) in a youth public mental health system. 76 % of all state care coordinators (N = 47) completed interviews, which were coded via content analysis until saturation. Facilitators (e.g., recognition of the MFS's clinical utility) and barriers (e.g., MFS's reliability and validity) emerged paralleling the Exploration, Adoption/Preparation, Implementation, and Sustainment framework outlined by Aarons et al. (Adm Policy Mental Health Mental Health Serv Res, 38:4-23, 2011). Sustainment efforts may leverage innovation fit, individual adopter, and system related facilitators.

Capabilities and Characteristics of Digital Measurement Feedback Systems: Results from a Comprehensive Review.

Measurement feedback systems (MFS) are a class of health information technology (HIT) that function as an implementation support strategy for integrating measurement based care or routine outcome monitoring into clinical practice. Although many MFS have been developed, little is known about their functions. This paper reports findings from an application of health information technology-academic and commercial evaluation (HIT-ACE), a systematic and consolidated evaluation method, to MFS designed for use in behavioral healthcare settings. Forty-nine MFS were identified and subjected to systematic characteristic and capability coding. Results are presented with respect to the representation of characteristics and capabilities across MFS.

Applying User Input to the Design and Testing of an Electronic Behavioral Health Information System for Wraparound Care Coordination.

Health information technology (HIT) and care coordination for individuals with complex needs are high priorities for quality improvement in health care. However, there is little empirical guidance about how best to design electronic health record systems and related technologies to facilitate implementation of care coordination models in behavioral health, or how best to apply user input to the design and testing process. In this paper, we describe an iterative development process that incorporated user/stakeholder perspectives at multiple points and resulted in an electronic behavioral health information system (EBHIS) specific to the wraparound care coordination model for youth with serious emotional and behavioral disorders. First, we review foundational HIT research on how EBHIS can enhance efficiency and outcomes of wraparound that was used to inform development. After describing the rationale for and functions of a prototype EBHIS for wraparound, we describe methods and results for a series of six small studies that informed system development across four phases of effort-predevelopment, development, initial user testing, and commercialization-and discuss how these results informed system design and refinement. Finally, we present next steps, challenges to dissemination, and guidance for others aiming to develop specialized behavioral health HIT. The research team's experiences reinforce the opportunity presented by EBHIS to improve care coordination for populations with complex needs, while also pointing to a litany of barriers and challenges to be overcome to implement such technologies.

Implementing a Measurement Feedback System: A Tale of Two Sites.

A randomized experiment was conducted in two outpatient clinics evaluating a measurement feedback system called contextualized feedback systems. The clinicians of 257 Youth 11-18 received feedback on progress in mental health symptoms and functioning either every 6 months or as soon as the youth's, clinician's or caregiver's data were entered into the system. The ITT analysis showed that only one of the two participating clinics (Clinic R) had an enhanced outcome because of feedback, and only for the clinicians' ratings of youth symptom severity on the SFSS. A dose-response effect was found only for Clinic R for both the client and clinician ratings. Implementation analyses showed that Clinic R had better implementation of the feedback intervention. Clinicians' questionnaire completion rate and feedback viewing at Clinic R were 50 % higher than clinicians at Clinic U. The discussion focused on the differences in implementation at each site and how these differences may have contributed to the different outcomes of the experiment.

Feedback mechanisms of change: How problem alerts reported by youth clients and their caregivers impact clinician-reported session content.

OBJECTIVE: This study explored how clinician-reported content addressed in treatment sessions was predicted by clinician feedback group and multi-informant cumulative problem alerts that appeared in computerized feedback reports for 299 clients aged 11-18 years receiving home-based community mental health treatment. METHOD: Measures included a clinician report of content addressed in sessions and additional measures of treatment progress and process (e.g., therapeutic alliance) completed by clinicians, clients, and their caregivers. Item responses in the top 25th percentile in severity from these measures appeared as "problem alerts" on corresponding computerized feedback reports. Clinicians randomized to the feedback group received feedback weekly while the control group did not. Analyses were conducted using the Cox proportional hazards regression for recurrent events. RESULTS: For all content domains, the results of the survival analyses indicated a robust effect of the feedback group on addressing specific content in sessions, with feedback associated with shorter duration to first occurrence and increased likelihood of addressing or focusing on a topic compared to the non-feedback group. CONCLUSION: There appears to be an important relationship between feedback and cumulative problem alerts reported by multiple informants as they influence session content.

Measurement feedback system implementation in public youth mental health treatment services: a mixed methods analysis.

BACKGROUND: Prior studies indicate the effectiveness of measurement-based care (MBC), an evidence-based practice, in improving and accelerating positive outcomes for youth receiving behavioral health services. MBC is the routine collection and use of client-reported progress measures to inform shared decision-making and collaborative treatment adjustments and is a relatively feasible and scalable clinical practice, particularly well-suited for under-resourced community mental health settings. However, uptake of MBC remains low, so information on determinants related to MBC practice patterns is needed. METHODS: Quantitative and qualitative data from N = 80 clinicians who implemented MBC using a measurement feedback system (MFS) were merged to understand and describe determinants of practice over three study phases. Quantitative, latent class analysis identified clinician groups based on participants' ratings of MFS acceptability, appropriateness, and feasibility and describes similarities/differences between classes in clinician-level characteristics (e.g., age; perceptions of implementation climate; reported MFS use; phase I). Qualitative analyses of clinicians' responses to open-ended questions about their MFS use and feedback about the MFS and implementation supports were conducted separately to understand multi-level barriers and facilitators to MFS implementation (phase II). Mixing occurred during interpretation, examining clinician experiences and opinions across groups to understand the needs of different classes of clinicians, describe class differences, and inform selection of implementation strategies in future research (phase III). RESULTS: We identified two classes of clinicians: "Higher MFS" and "Lower MFS," and found similarities and differences in MFS use across groups. Compared to Lower MFS participants, clinicians in the Higher MFS group reported facilitators at a higher rate. Four determinants of practice were associated with the uptake of MBC and MFS in youth-serving community mental health settings for all clinicians: clarity, appropriateness, and feasibility of the MFS and its measures; clinician knowledge and skills; client preferences and behaviors; and incentives and resources (e.g., time; continuing educational support). Findings also highlighted the need for individual-level implementation strategies to target clinician needs, skills, and perceptions for future MBC and MFS implementation efforts. CONCLUSION: This study has implications for the adoption of evidence-based practices, such as MBC, in the context of community-based mental health services for youth.

Implementing Measurement-Based Care in Various Practice Settings.

Measurement-based care (MBC) is recognized as a valuable component to maximize quality in psychiatric care; however, actual use of MBC by practitioners is poor. A host of implementation barriers have been noted, and are likely significant contributors to this poor adoption. Many of these barriers are related to work-flow issues that can be managed or mitigated by appropriate infrastructure considerations. This article offers an overview of the continuum of infrastructures to support MBC in clinical practice, delineating the tradeoffs between these infrastructures, and then identifying specific experience-based strategies for addressing several major patient-, provider-, and organization-level barriers to MBC implementation.

Measurement training and feedback system for implementation of evidence-based treatment for adolescent externalizing problems: protocol for a randomized trial of pragmatic clinician training.

BACKGROUND: Innovations in clinical training and support that enhance fidelity to evidence-based treatment (EBT) for adolescent behavior problems are sorely needed. This study will develop an online training system to address this gap: Measurement Training and Feedback System for Implementation (MTFS-I). Using procedures intended to be practical and sustainable, MTFS-I is designed to increase two aspects of therapist behavior that are fundamental to boosting EBT fidelity: therapist self-monitoring of EBT delivery, and therapist utilization of core techniques of EBTs in treatment sessions. This version of MTFS-I focuses on two empirically supported treatment approaches for adolescent conduct and substance use problems: family therapy and cognitive behavioral therapy (CBT). METHODS/DESIGN: MTFS-I expands on conventional measurement feedback systems for client outcomes by adding training in observational coding to promote EBT self-monitoring and focusing on implementation of EBT treatment techniques. It has two primary components. (1) The training component, delivered weekly in two connected parts, involves self-monitored learning modules containing brief clinical descriptions of core EBT techniques and mock session coding exercises based on 5-8 min video segments that illustrate delivery of core techniques. (2) The feedback component summarizes aggregated therapist-reported data on EBT techniques used with their active caseloads. MTFS-I is hosted online and requires approximately 20 min per week to complete for each treatment approach. This randomized trial will first collect data on existing delivery of family therapy and CBT techniques for youth in outpatient behavioral health sites (Baseline phase). It will then randomize site clinicians to two study conditions (Implementation phase): Training Only versus Training + Feedback + Consultation. Therapists will choose whether to train in family therapy, CBT, or both. Study aims will compare clinician performance across study phase and between study conditions on MTFS-I uptake, reliability and accuracy in EBT self-monitoring, and utilization of EBT techniques in treatment sessions (based on observer coding of audiotapes). DISCUSSION: Study contributions to implementation science and considerations of MTFS-I sustainability are discussed. TRIAL REGISTRATION: ClinicalTrials.gov, NCT03722654. Registered on 29 October 2018.

Measurement Training and Feedback System for Implementation of family-based services for adolescent substance use: protocol for a cluster randomized trial of two implementation strategies.

BACKGROUND: This article describes a study protocol for testing the Measurement Training and Feedback System for Implementation (MTFS-I) and comparing two implementation strategies for MTFS-I delivery. MTFS-I is a web-based treatment quality improvement system designed to increase the delivery of evidence-based interventions for behavioral health problems in routine care settings. This version of MTFS-I focuses on family-based services (FBS) for adolescent substance use. FBS, comprising both family participation in treatment and family therapy technique use, have achieved the strongest evidence base for adolescent substance use and are a prime candidate for upgrading treatment quality in outpatient care. For FBS to fulfill their potential for widespread dissemination, FBS implementation must be bolstered by effective quality procedures that support sustainable delivery in usual care. METHODS/DESIGN: Adapted from measurement feedback systems for client outcomes, MTFS-I contains three synergistic components: (a) weekly reporter training modules to instruct therapists in reliable post-session self-reporting on FBS utilization; (b) weekly mock session videos of FBS interventions (5-8 min) for supportive training in, and practice coding of, high-quality FBS; and (c) monthly feedback reports to therapists and supervisors displaying aggregated data on therapist-reported FBS use. MTFS-I is hosted online and requires approximately 20 min per week to complete. The study will experimentally compare two well-established implementation strategies designed to foster ongoing MTFS-I usage: Core Training, consisting of two 3-h training sessions focused on FBS site mapping, selecting FBS improvement goals, and sustaining MTFS-I, followed by routine remote technical assistance; and Core + Facilitation, which boosts Core Training sessions with collaborative phone-based clinical consultation and on-site facilitation meetings for 1 year to promote FBS goal achievement. The study design is a cluster randomized trial testing Core Training versus Core + Facilitation in ten substance use treatment clinics. Study aims will compare conditions on MTFS-I uptake, FBS delivery (based on therapist-report and observational data), and 1-year client outcomes. DISCUSSION: Study contributions to implementation science and considerations of MTFS-I sustainability are discussed. TRIAL REGISTRATION: ClinicalTrials.gov NCT03342872 . Registered 10 November 2017.

Evaluating Modular Approach to Therapy for Children with Anxiety, Depression, Trauma and Conduct Problems (MATCH-ADCT) in Norwegian child and adolescent outpatient clinics: Study protocol for a randomized controlled trial.

BACKGROUND: Norwegian health, care, and welfare services are experiencing increased demands to deliver services that are safe, effective, of high quality, and that ensure user involvement. Yet, evidence-based treatment for common disorders such as depression, anxiety, trauma, and behavioral problems in children are not regularly used in clinical practice in Norway. Possible explanations for this are that many standard, evidence-based treatments may have difficulty addressing the complexity and comorbidity of referred children and the fact that children's treatment needs often shift during treatment. The Modular Approach to Therapy for children with Anxiety, Depression, Trauma and Conduct problems (MATCH-ADTC) was designed to address these challenges and reduce some of the barriers to therapists' use of evidence-based treatment in their practice. METHODS/DESIGN: Participants will include 280 children (aged 6-14.5 years at intake) who receive treatment in child and adolescent mental health outpatient clinics in Norway, and their families. Families are randomly assigned to either the experimental group receiving treatment from therapists trained in MATCH, or to the comparison group receiving treatment from therapists delivering treatment as usual (TAU). Data on children's symptomology, child and family functioning, demographics, background information, and mental health outcomes are collected as well as frequent feedback on treatment response, plus video-recordings of treatment sessions and implementation quality scores from each participating clinic. Questionnaires are administered in six waves. DISCUSSION: MATCH has been tested in the US with promising results, but we do not know whether this treatment approach will produce similar results in Norway. The implications of this study are 1. Possibly better treatment outcomes and/or more efficient improvements for children and families treated in mental health outpatient clinics in Norway 2. Clinicians learning to use more evidence-based practices in their treatment 3. Implementation of standard procedures for obtaining feedback from children and families and sharing the feedback with clinicians 4. Increased understanding, at the end of the trial, of whether introducing MATCH improves outcomes for children and families treated in mental health outpatient clinics TRIAL REGISTRATION: ISRCTN, registration number: ISRCTN24029895 . Registered on 8 August 2016.