Getting More Out of Clinical Documentation: Can Clinical Dashboards Yield Clinically Useful Information?

This study investigated coded data retrieved from clinical dashboards, which are decision-support tools that include a graphical display of clinical progress and clinical activities. Data were extracted from clinical dashboards representing 256 youth (M age = 11.9) from 128 practitioners who were trained in the Managing and Adapting Practice (MAP) system (Chorpita & Daleiden in BF Chorpita EL Daleiden 2014 Structuring the collaboration of science and service in pursuit of a shared vision. 43(2):323 338. 2014, Chorpita & Daleiden in BF Chorpita EL Daleiden 2018 Coordinated strategic action: Aspiring to wisdom in mental health service systems. 25(4):e12264. 2018) in 55 agencies across 5 regional mental health systems. Practitioners labeled up to 35 fields (i.e., descriptions of clinical activities), with the options of drawing from a controlled vocabulary or writing in a client-specific activity. Practitioners then noted when certain activities occurred during the episode of care. Fields from the extracted data were coded and reliability was assessed for Field Type, Practice Element Type, Target Area, and Audience (e.g., Caregiver Psychoeducation: Anxiety would be coded as Field Type = Practice Element; Practice Element Type = Psychoeducation; Target Area = Anxiety; Audience = Caregiver). Coders demonstrated moderate to almost perfect interrater reliability. On average, practitioners recorded two activities per session, and clients had 10 unique activities across all their sessions. Results from multilevel models showed that clinical activity characteristics and sessions accounted for the most variance in the occurrence, recurrence, and co-occurrence of clinical activities, with relatively less variance accounted for by practitioners, clients, and regional systems. Findings are consistent with patterns of practice reported in other studies and suggest that clinical dashboards may be a useful source of clinical information. More generally, the use of a controlled vocabulary for clinical activities appears to increase the retrievability and actionability of healthcare information and thus sets the stage for advancing the utility of clinical documentation.

The role of implementation organizations in scaling evidence-based psychosocial interventions.

BACKGROUND: To bring evidence-based interventions (EBIs) to individuals with behavioral health needs, psychosocial interventions must be delivered at scale. Despite an increasing effort to implement effective treatments in communities, most individuals with mental health and behavioral problems do not receive EBIs. We posit that organizations that commercialize EBIs play an important role in disseminating EBIs, particularly in the USA. The behavioral health and implementation industry is growing, bringing the implementation field to an important inflection point: how to scale interventions to improve access while maintaining EBI effectiveness and minimizing inequities in access to psychosocial intervention. MAIN BODY: We offer a first-hand examination of five illustrative organizations specializing in EBI implementation: Beck Institute for Cognitive Behavioral Therapy; Incredible Years, Inc.; the PAXIS Institute; PracticeWise, LLC; and Triple P International. We use the Five Stages of Small Business Growth framework to organize themes. We discuss practical structures (e.g., corporate structures, intellectual property agreements, and business models) and considerations that arise when trying to scale EBIs including balancing fidelity and reach of the intervention. Business models consider who will pay for EBI implementation and allow organizations to scale EBIs. CONCLUSION: We propose research questions to guide scaling: understanding the level of fidelity needed to maintain efficacy, optimizing training outcomes, and researching business models to enable organizations to scale EBIs.

Assessing practical implementation of modular psychotherapy for youth in community-based settings using benchmarking.

Background: Achieving high quality outcomes in a community context requires the strategic coordination of many activities in a service system, involving families, clinicians, supervisors, and administrators. In modern implementation trials, the therapy itself is guided by a treatment manual; however, structured supports for other parts of the service system may remain less well-articulated (e.g., supervision, administrative policies for planning and review, information/feedback flow, resource availability). This implementation trial investigated how a psychosocial intervention performed when those non-therapy supports were not structured by a research team, but were instead provided as part of a scalable industrial implementation, testing whether outcomes achieved would meet benchmarks from published research trials. Method: In this single-arm observational benchmarking study, a total of 59 community clinicians were trained in the Modular Approach to Therapy for Children (MATCH) treatment program. These clinicians delivered MATCH treatment to 166 youth ages 6 to 17 naturally presenting for psychotherapy services. Clinicians received substantially fewer supports from the treatment developers or research team than in the original MATCH trials and instead relied on explicit process management tools to facilitate implementation. Prior RCTs of MATCH were used to benchmark the results of the current initiative. Client improvement was assessed using the Top Problems Assessment and Brief Problem Monitor. Results: Analysis of client symptom change indicated that youth experienced improvement equal to or better than the experimental condition in published research trials. Similarly, caregiver-reported outcomes were generally comparable to those in published trials. Conclusions: Although results must be interpreted cautiously, they support the feasibility of using process management tools to facilitate the successful implementation of MATCH outside the context of a formal research or funded implementation trial. Further, these results illustrate the value of benchmarking as a method to evaluation industrial implementation efforts.Plain Language Summary: Randomized effectiveness trials are inclusive of clinicians and cases that are routinely encountered in community-based settings, while continuing to rely on the research team for both clinical and administrative guidance. As a result, the field still struggles to understand what might be needed to support sustainable implementation and how interventions will perform when brought to scale in community settings without those clinical trial supports. Alternative approaches are needed to delineate and provide the clinical and operational support needed for implementation and to efficiently evaluate how evidence-based treatments perform. Benchmarking findings in the community against findings of more rigorous clinical trials is one such approach. This paper offers two main contributions to the literature. First, it provides an example of how benchmarking is used to evaluate how the Modular Approach to Therapy for Children (MATCH) treatment program performed outside the context of a research trial. Second, this study demonstrates that MATCH produced comparable symptom improvements to those seen in the original research trials and describes the implementation strategies associated with this success. In particular, although clinicians in this study had less rigorous expert clinical supervision as compared with the original trials, clinicians were provided with process management tools to support implementation. This study highlights the importance of evaluating the performance of intervention programs when brought to scale in community-based settings. This study also provides support for the use of process management tools to assist providers in effective implementation.

Pilot Study of the MAP Curriculum for Psychotherapy Competencies in Child and Adolescent Psychiatry.

OBJECTIVE: This pilot study aimed to evaluate the relevance, feasibility, acceptability, and instructional efficacy of the Managing and Adapting Practice (MAP) curriculum for enhancing the teaching of psychotherapy to child and adolescent psychiatry (CAP) fellows. MAP is a system of resources and decision models that supports practitioners in selecting and implementing psychotherapeutic interventions for children and adolescents. The MAP curriculum includes modules to guide education about psychotherapeutic procedures (e.g., behavioral activation) common in evidence-based treatments for an array of childhood problems and to support development of competencies in assessment, treatment planning, and reflective practice. METHODS: Curriculum coding was used to examine the relevance of MAP's core components to the skills articulated in the Accreditation Council for Graduate Medical Education (ACGME) CAP milestones. Feasibility, acceptability, and learning outcomes were examined after delivery of the MAP curriculum to 12 CAP fellows at two sites, with instructional features tailored according to faculty preferences and training program structure. RESULTS: Coding suggested that the MAP curriculum was relevant to 95% of the 21 ACGME CAP training subcompetencies. Feasibility was indicated by the successful delivery of 100% of the planned MAP curriculum across the two sites. Acceptability was supported by positive feedback from the CAP fellows, and psychotherapy knowledge increased significantly. Finally, case review scores (mean±SD=2.21±0.15) showed positive posttraining application of MAP to two patients and exceeded scores achieved by other samples of mental health professionals. CONCLUSIONS: This pilot study demonstrated the potential for the MAP curriculum to support CAP education. MAP's versatility as a curriculum supports broader adoption, with continuing rigorous empirical evaluation.

Creating a prepared mental health workforce: comparative illustrations of implementation strategies.

BACKGROUND: Psychotherapy implementation must contend with the task of preparing a mental health workforce to provide the highest quality services to as much of a service population as possible, in high-income as well as low-to-middle income countries. OBJECTIVE: We outline general challenges and solutions and investigate how well various implementation strategies would fit a clinical population. METHODS: Using a data set from a prior cluster randomised trial with a clinically diverse population and 33 intervention practices, we presented multiple illustrations comparing the ability of different implementation strategies to serve youth and families with procedures in which service providers were trained. FINDINGS: A series of survival functions demonstrated that many common implementation strategies are unlikely to create a prepared workforce, given the large and diverse number of practices needed to be mastered by providers. CLINICAL IMPLICATIONS: 'Benchmark' solutions that afforded superior coverage of the service population could be supported through paced learning approaches (ie, training interventions a little at a time) using extensible, modular intervention designs.

The value of a modular, multi-focal, therapeutic approach to addressing child maltreatment: Hope for Children and Families Intervention Resources - a discussion article.

This discussion article begins by highlighting two trends apparent in the field of child maltreatment. The first, an awareness that multiple forms of maltreatment - polyvictimization - is the rule in populations of abused and neglected children rather than the exception. The second is that current types of child maltreatment are being extended to include Adverse Childhood Experiences (ACEs). These include intra-familial violence, mental health, substance misuse, and inter-generational abuse. The paper introduces an innovative strategy to help the field better organise and prevent the extensive sequelae of polyvictimization and ACEs. This strategy involves the development of a modular approach, which identifies common treatment elements and common factors across the field of effective interventions and organizes them, providing a co-ordinated framework for practitioners to use to address the diverse needs of children and families when vulnerability or maltreatment are identified. The development of this approach, the Hope for Children and Families (HfCF) Intervention Resources, is described using a case example to illustrate its logic and structure. Findings from the HFCF pilot and subsequent training programs suggest that this new approach could be an important milestone in the protection of children from violence, abuse and neglect on the 30th Anniversary of the United Nation's Convention on the Rights of the Child (1989).

Pediatric Consultation-Liaison: Patient Characteristics and Considerations for Training in Evidence-Based Practices.

Consultation-liaison services are an integral part of many pediatric hospital settings, yet characteristics of this patient population have not been extensively documented. The current study is a retrospective one-year chart review of the consultation-liaison service at a large pediatric hospital in the Southwestern United States. The purpose of this study is twofold: (1) to characterize this hospital's CL population and (2) to use these characteristics to identify preliminary evidence-based practices that should be considered for CL provider training. Identifying evidence-based practice elements that align with the characteristics of consultation-liaison patient populations may inform trainings for consultation-liaison staff. This would help to ensure that youth seen in hospital consultation-liaison services are getting the best available services, which is critical given the shortened time frame available to work with this patient population.

Design process and protocol description for a multi-problem mental health intervention within a stepped care approach for adolescents in India.

This paper documents the collaborative design of a mental health intervention for adolescents in India with anxiety, depression, or anger-related concerns. The process was characterized by three phases of formative activities: (1) an intensive review of the service context, (2) selection of an overall design strategy (e.g., whether to choose existing evidence-based treatments or build new treatments in context), and (3) a period of prototyping, testing, and refining. Each phase resulted in specific outputs, which were, respectively, (1) a detailed articulation of values and preferences (setting expectations for what the ideal protocol should be), (2) a set of build parameters representing a blueprint that managed strategic compromises for this context, and (3) a working protocol. We outline the steps of this design process, summarize data from an open-trial clinical case series, and illustrate the resulting working protocol, which will be tested in a future larger trial. We conclude with insights and observations likely to be relevant to protocol design activity in a variety of contexts, most particularly those in low-and-middle-income countries such as India.

Using relevance mapping methodology to design an adolescent mental health intervention in India.

BACKGROUND: Adolescents in low and middle-income countries experience pronounced mental health needs in contexts where infrastructure and resources are scarce. While evidence-based treatment are readily available, they may not fit the unique needs of certain contexts. OBJECTIVE: This manuscript illustrates the systematic process of applying 'relevance mapping' methodology to leverage the youth mental health evidence base to identify candidate practices for inclusion in the development of a contextually appropriate psychological treatment protocol for common adolescent mental health problems in India. METHODS: The practice identification was informed by two datasets obtained from adolescent samples in India. The first was an epidemiological dataset from a large community sample in Goa (N = 2,048); the second incorporated 'youth top problems' reported by service-seeking students presenting to school counsellors in Goa and Delhi (N = 78). Problems identified in each dataset were categorized using structured codes. Problem codes and youth demographics were then indexed against a database of hundreds of evidence-based psychological treatments and their associated clinical trials. This methodology revealed the most common practice elements (discrete therapeutic strategies) and their most efficient combinations with evidence of effectiveness matching the demographics and diagnostic category (anxiety, disruptive behaviors and depression) prevalent in the planned treatment population. RESULTS: For anxiety, the most common practice elements for this age group were exposure, cognitive coping, and psychoeducation. For disruptive behaviors, the most common practices were problem-solving, goal-setting, and rapport-building. For depression, cognitive coping, behavioral activation, and psychoeducation were the most common practice elements. CONCLUSION: These practice elements provided the treatment development team with a preliminary list of candidate content for the development of an intensive psychological treatment within a stepped care service model to address common adolescent mental health problems in schools in India.

Managing and Adapting Practice: Provider Perceptions of an Evidence-Informed Framework for Delivering Mental Health Services.

This study examined providers' reflections on delivering managing and adapting practice (MAP), an evidence-informed framework that guides decision-making from scientific and client data. Consensual qualitative research methods were used to analyze the reflections of 201 youth mental health providers. Results indicated that providers approached MAP according to their own preferences and particular cases. While most appeared to approach MAP from a practice management standpoint, when faced with challenging cases, providers used coordination and outcomes management resources. Regardless of approach, most providers came to appreciate the full framework through reflective practice. Their diverse approaches offer lessons for evidence-based practice implementation and sustainment.

Implementing Evidence Based Practices for Children's Mental Health: A Case Study in Implementing Modular Treatments in Community Mental Health.

There is strong enthusiasm for utilizing implementation science in the implementation of evidence-based programs in children's community mental health, but there remains work to be done to improve the process. Despite the proliferation of implementation frameworks, there is limited literature providing case examples of overcoming implementation barriers. This article examines whether the use of three implementations strategies, a structured training and coaching program, the use of professional development portfolios for coaching, and a progress monitoring data system, help to overcome barriers to implementation by facilitating four implementation drivers at a community mental health agency. Results suggest that implementation is a process of recognizing and adapting to both predictable and unpredictable barriers. Furthermore, the use of these implementation strategies is important in improving implementation outcomes.

Do Treatment Plans Matter? Moving From Recommendations to Action.

We investigated whether a service-planning document outlining recommendations for what providers should address in treatment (i.e., targets) and the associated clinical techniques they should employ (i.e., practices) influenced the targets and practices that providers reported actually implementing during the subsequent treatment episode. Participants included 94 youths ages 4 to 17 (M = 13.57, SD = 3.59) who received community-based mental health services from the Hawai'i Child and Adolescent Mental Health Division. Data on targets and practices were compared across initial Mental Health Treatment Plans and Monthly Treatment and Progress Summaries. Data were analyzed using two-level, generalized mixed effects models with two-way cross-classification or linear mixed effects models. Providers were more likely to report the use of targets and practices in treatment if they were included within the treatment plan. In addition, the more closely targets addressed during treatment followed the recommended targets from the treatment plan, the more closely implemented practices followed the recommended practices listed in the treatment plan. Furthermore, as providers shifted their focus to different targets, a shift in their use of practices was also evident over time. Last, practices for which there is demonstrated efficacy for particular targets were more likely to be used. Service planning documents appear to help organize care; however, results also suggest possible limitations to the current system. These findings highlight potential areas for improvement in planning and care delivery.

Replicating Evidence-Based Practices with Flexibility for Perinatal Home Visiting by Paraprofessionals.

Introduction Strategies are needed to improve the efficacy of paraprofessional home visitors for pregnant women in the United States. This study evaluates the maternal and child outcomes when evidence-based practices (EBP) are replicated with flexibility, rather than fidelity to a manualized intervention. Methods Pregnant mothers (N = 203) in five clinics were recruited in the waiting rooms and randomized to standard clinic care as the control condition (n = 104) or standard care plus home visiting (n = 99). Home visitors (n = 9) were selected, trained in foundational skills common to EBP and four problem domains (weight control, breastfeeding, daily habits, and depression). Independent interviewers assessed targeted outcomes at birth (82%) and 6 months later (83%). RESULTS: Home visitors, called Mentor Mothers [MM], made an average of 14.9 home visits or telephone contacts (SD = 9; total contacts = 1491) addressing maternal daily habits, breastfeeding, and depression. Intervention and control mothers were similar in weight, Body Mass Index (BMI), depression and social support at baseline and 6 months later. The percentage of low birth weight babies was similar; intervention infants' growth (weight/height Z score) tended to be significantly better compared to the control condition. DISCUSSION: There are many explanations for the failure to find significant benefits: insufficient statistical power; the benefits of repeated assessments by warm, supportive peers to improve outcomes; or the failure of EBP and the need to maintain replication with fidelity. All study mothers had better outcomes than documented among comparable published samples of low-income, Latina and Korean-American mothers in Los Angeles, CA. ClinicalTrials.gov registration NCT01687634.

Adapting Current Strategies to Implement Evidence-Based Prevention Programs for Paraprofessional Home Visiting.

This paper describes a strategy for using evidence-based interventions (EBI) that does not require replication and fidelity. Eight parents, identified as positive role models, conducted home visits for 101 low-income Latina and Korean pregnant women. The home visitors, called mentor mothers (MM), were trained in 10 of the practice elements common to 80% of child-focused EBI and how to apply these skills to support mothers in obesity prevention, to increase the duration of breastfeeding, and to reduce depression. MM reported the content and skills utilized on each home visit on mobile phones. Each MM made an average of 153 home visits (SD = 173.3), with 28 of these visits being phone contacts. Body mass index (BMI) at 6 months was significantly associated with the frequency of MM focused on coping with depression (r = .24), but was not related to practice elements used by MM. The duration of breastfeeding was significantly related to the frequency with which MM focused on the topic of breastfeeding (r = .28) and parenting (r = .3), and MM use of attending (r = .24) and relaxation (r = .27). Depression was significantly correlated with the frequency of addressing depression (r = .27), but not to practice elements. MM did use different strategies in the first 150 visits compared to their last 150 visits, reflecting data-informed supervision. Evidence synthesized from EBI was used as a novel training method, with real-time monitoring and data-informed supervision providing evidence of iterative quality improvements in MM behaviors over time, as well as a way for linking implementation processes to outcomes.

Overlapping and Non-overlapping Practices in Usual and Evidence-Based Care for Youth Anxiety.

This study compared consistencies and discrepancies in usual care with practices derived from the evidence-base (PDEB) for youth anxiety in a public mental health system. Youth-level factors (diagnosis, functional impairment) as predictors of the discrepancies were also examined. Psychosocial and service data from 2485 youth with an anxiety disorder and/or receiving services for an anxiety treatment target were extracted. Therapists (N = 616) identified the treatment targets and practices youth received. Although many PDEB for youth anxiety were used by therapists in this sample, Exposure was only used in 15% of cases. Practices not consistent with youth anxiety treatment were also reported and included: PDEB for other conditions, practices common to all therapies, and practices that are not consistent with evidence-based care. Age and diagnosis predicted the delivery of PDEB for youth anxiety. Usual care incorporated many components of evidence-based care but was more diffuse and less focused on well-supported practices.

Child STEPs in California: A cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress.

OBJECTIVE: This study reports outcomes from a randomized effectiveness trial testing modular treatment versus multiple community-implemented evidence-based treatments for youth. METHOD: An ethnoracially diverse sample of 138 youth ages 5 to 15 (62 girls, 76 boys) whose primary clinical concerns involved diagnoses or clinical elevations related to anxiety, depression, disruptive behavior, and/or traumatic stress were treated by community therapists randomly assigned to 1 of 2 conditions: (a) modular treatment, which involved a single modular protocol (i.e., modular approach to therapy for children; MATCH) that allowed flexible selection and sequencing of procedures to fit the chosen treatment focus in the context of measurement feedback, and (b) community-implemented treatment (CIT), which was a county-supported implementation of multiple evidence-based practices for youth. RESULTS: Youth treated with MATCH showed significantly faster rates of improvement over time on clinical and functional outcomes relative to youth in the CIT condition and required significantly fewer sessions delivered over significantly fewer days. Caregiver-reported clinical improvement rates were significantly greater for MATCH (60%) versus CIT (36.7%). Further, youth in the CIT condition were significantly more likely to receive additional psychosocial treatment services and were significantly more likely to use a variety of psychotropic medications during the active treatment phase. CONCLUSIONS: These results extend prior findings, supporting the effectiveness and efficiency of a modular, multifocus approach that incorporates monitoring and feedback relative to community implementation of evidence-based treatments. (PsycINFO Database Record

Evidence-Based Decision Making in Youth Mental Health Prevention.

Given the impressive amount of knowledge relevant to prevention efforts, this paper articulates strategies to capitalize on such knowledge through evidence-based decision making. Knowledge, or "evidence," is understood here as coming from multiple sources, including research, individual people, group history, and relevant theory. The presented strategies to facilitate evidence-based decision making are: (1) intervention knowledge management; (2) collaborative design; (3) knowledge resources for intervention; and (4) developmentally sensitive training and supervision. Examples and benefits are outlined for each strategy. It is ultimately argued that evidence-based decision making embodies the scientific approach, and is worth consideration within primary prevention, given its early success in secondary intervention of youth mental health.

At the Intersection of Health Information Technology and Decision Support: Measurement Feedback Systems... and Beyond.

We select and comment on concepts and examples from the target articles in this special issue on measurement feedback systems, placing them in the context of some of our own insights and ideas about measurement feedback systems, and where those systems lie at the intersection of technology and decision making. We contend that, connected to the many implementation challenges relevant to many new technologies, there are fundamental design challenges that await a more elaborate specification of the clinical information and decision models that underlie these systems. Candidate features of such models are discussed, which include referencing multiple evidence bases, facilitating observed and expected value comparisons, fostering collaboration, and allowing translation across multiple ontological systems. We call for a new metaphor for these technologies that goes beyond measurement feedback and encourages a deeper consideration of the increasingly complex clinical decision models needed to manage the uncertainty of delivering clinical care.

Building an evidence-informed service array: Considering evidence-based programs as well as their practice elements.

OBJECTIVE: This study empirically examined options for building an evidence-informed service array, comparing strategies to maximize the application of evidence-based treatment literature in a clinical service system. The overall goal was to determine the smallest set of treatments that could serve the largest percentage of clients. Solutions to this problem differ depending on how one defines "treatment." METHOD: Treatments were conceptualized as (a) programs (integrated treatments produced by specific research laboratories or investigators), and (b) collections of their constituent common procedures, referred to as practice elements. Programs listed by 2 separate government-sanctioned registries were selected to illustrate the effects of "program" conceptualizations, and all available clinical trials testing the programs were analyzed. Practice elements were identified from these same studies and from studies of other treatments that met a standard of evidence but had not been organized into programs on these lists. Relevance mapping methodology was used to identify optimal sets of programs and practice elements. RESULTS: Among a large, diverse clinical population, results identified 11%-22% of youths for whom practice elements provide an evidence-informed treatment option whereas no programs meeting the standard of evidence were available on the registries. Results for the practice elements were able to be matched by a hybrid combination: "best" programs, which were then extended by practice elements. CONCLUSIONS: These results demonstrated that there are multiple ways to conceptualize treatments when planning a service array, and these options have significant implications regarding who can be served by treatments supported by evidence.

The common elements of engagement in children's mental health services: which elements for which outcomes?

Using the distillation component of the Distillation and Matching Model framework (Chorpita, Daleiden, & Weisz, 2005 ), we examined which engagement practices were associated with three domains of treatment engagement: attendance, adherence, and cognitive preparation (e.g., understanding of, readiness for treatment). Eighty-nine engagement interventions from 40 randomized controlled trials in children' s mental health services were coded according to their engagement practices and outcomes. Analyses examined whether the practices used in successful interventions differed according to engagement domain. Practice patterns differed somewhat depending on whether attendance, adherence, or cognitive preparation was the outcome of interest. For example, assessment of barriers to treatment frequently occurred in successful interventions targeting attendance, whereas homework assignment frequently occurred in successful interventions when adherence was the target outcome. Modeling and expectation setting were frequently used in successful interventions targeting cognitive preparation for treatment. Distillation provides a method for examining the practice patterns associated with different engagement outcomes. An example of the application of these findings to clinical practice includes using certain practices (e.g., assessment, psychoeducation about services, and accessibility promotion) with all youth and families to promote attendance, adherence, and cognitive preparation. Then, other practices (e.g., modeling, homework assignment) can be added on an as-needed basis to boost engagement or to address interference in a particular engagement domain. The use of a distillation framework promotes a common language around engagement and highlights practices that lend themselves well to training, thereby promoting the dissemination of engagement interventions.