Getting More Out of Clinical Documentation: Can Clinical Dashboards Yield Clinically Useful Information?

This study investigated coded data retrieved from clinical dashboards, which are decision-support tools that include a graphical display of clinical progress and clinical activities. Data were extracted from clinical dashboards representing 256 youth (M age = 11.9) from 128 practitioners who were trained in the Managing and Adapting Practice (MAP) system (Chorpita & Daleiden in BF Chorpita EL Daleiden 2014 Structuring the collaboration of science and service in pursuit of a shared vision. 43(2):323 338. 2014, Chorpita & Daleiden in BF Chorpita EL Daleiden 2018 Coordinated strategic action: Aspiring to wisdom in mental health service systems. 25(4):e12264. 2018) in 55 agencies across 5 regional mental health systems. Practitioners labeled up to 35 fields (i.e., descriptions of clinical activities), with the options of drawing from a controlled vocabulary or writing in a client-specific activity. Practitioners then noted when certain activities occurred during the episode of care. Fields from the extracted data were coded and reliability was assessed for Field Type, Practice Element Type, Target Area, and Audience (e.g., Caregiver Psychoeducation: Anxiety would be coded as Field Type = Practice Element; Practice Element Type = Psychoeducation; Target Area = Anxiety; Audience = Caregiver). Coders demonstrated moderate to almost perfect interrater reliability. On average, practitioners recorded two activities per session, and clients had 10 unique activities across all their sessions. Results from multilevel models showed that clinical activity characteristics and sessions accounted for the most variance in the occurrence, recurrence, and co-occurrence of clinical activities, with relatively less variance accounted for by practitioners, clients, and regional systems. Findings are consistent with patterns of practice reported in other studies and suggest that clinical dashboards may be a useful source of clinical information. More generally, the use of a controlled vocabulary for clinical activities appears to increase the retrievability and actionability of healthcare information and thus sets the stage for advancing the utility of clinical documentation.

The role of implementation organizations in scaling evidence-based psychosocial interventions.

BACKGROUND: To bring evidence-based interventions (EBIs) to individuals with behavioral health needs, psychosocial interventions must be delivered at scale. Despite an increasing effort to implement effective treatments in communities, most individuals with mental health and behavioral problems do not receive EBIs. We posit that organizations that commercialize EBIs play an important role in disseminating EBIs, particularly in the USA. The behavioral health and implementation industry is growing, bringing the implementation field to an important inflection point: how to scale interventions to improve access while maintaining EBI effectiveness and minimizing inequities in access to psychosocial intervention. MAIN BODY: We offer a first-hand examination of five illustrative organizations specializing in EBI implementation: Beck Institute for Cognitive Behavioral Therapy; Incredible Years, Inc.; the PAXIS Institute; PracticeWise, LLC; and Triple P International. We use the Five Stages of Small Business Growth framework to organize themes. We discuss practical structures (e.g., corporate structures, intellectual property agreements, and business models) and considerations that arise when trying to scale EBIs including balancing fidelity and reach of the intervention. Business models consider who will pay for EBI implementation and allow organizations to scale EBIs. CONCLUSION: We propose research questions to guide scaling: understanding the level of fidelity needed to maintain efficacy, optimizing training outcomes, and researching business models to enable organizations to scale EBIs.

Assessing practical implementation of modular psychotherapy for youth in community-based settings using benchmarking.

Background: Achieving high quality outcomes in a community context requires the strategic coordination of many activities in a service system, involving families, clinicians, supervisors, and administrators. In modern implementation trials, the therapy itself is guided by a treatment manual; however, structured supports for other parts of the service system may remain less well-articulated (e.g., supervision, administrative policies for planning and review, information/feedback flow, resource availability). This implementation trial investigated how a psychosocial intervention performed when those non-therapy supports were not structured by a research team, but were instead provided as part of a scalable industrial implementation, testing whether outcomes achieved would meet benchmarks from published research trials. Method: In this single-arm observational benchmarking study, a total of 59 community clinicians were trained in the Modular Approach to Therapy for Children (MATCH) treatment program. These clinicians delivered MATCH treatment to 166 youth ages 6 to 17 naturally presenting for psychotherapy services. Clinicians received substantially fewer supports from the treatment developers or research team than in the original MATCH trials and instead relied on explicit process management tools to facilitate implementation. Prior RCTs of MATCH were used to benchmark the results of the current initiative. Client improvement was assessed using the Top Problems Assessment and Brief Problem Monitor. Results: Analysis of client symptom change indicated that youth experienced improvement equal to or better than the experimental condition in published research trials. Similarly, caregiver-reported outcomes were generally comparable to those in published trials. Conclusions: Although results must be interpreted cautiously, they support the feasibility of using process management tools to facilitate the successful implementation of MATCH outside the context of a formal research or funded implementation trial. Further, these results illustrate the value of benchmarking as a method to evaluation industrial implementation efforts.Plain Language Summary: Randomized effectiveness trials are inclusive of clinicians and cases that are routinely encountered in community-based settings, while continuing to rely on the research team for both clinical and administrative guidance. As a result, the field still struggles to understand what might be needed to support sustainable implementation and how interventions will perform when brought to scale in community settings without those clinical trial supports. Alternative approaches are needed to delineate and provide the clinical and operational support needed for implementation and to efficiently evaluate how evidence-based treatments perform. Benchmarking findings in the community against findings of more rigorous clinical trials is one such approach. This paper offers two main contributions to the literature. First, it provides an example of how benchmarking is used to evaluate how the Modular Approach to Therapy for Children (MATCH) treatment program performed outside the context of a research trial. Second, this study demonstrates that MATCH produced comparable symptom improvements to those seen in the original research trials and describes the implementation strategies associated with this success. In particular, although clinicians in this study had less rigorous expert clinical supervision as compared with the original trials, clinicians were provided with process management tools to support implementation. This study highlights the importance of evaluating the performance of intervention programs when brought to scale in community-based settings. This study also provides support for the use of process management tools to assist providers in effective implementation.

Pilot Study of the MAP Curriculum for Psychotherapy Competencies in Child and Adolescent Psychiatry.

OBJECTIVE: This pilot study aimed to evaluate the relevance, feasibility, acceptability, and instructional efficacy of the Managing and Adapting Practice (MAP) curriculum for enhancing the teaching of psychotherapy to child and adolescent psychiatry (CAP) fellows. MAP is a system of resources and decision models that supports practitioners in selecting and implementing psychotherapeutic interventions for children and adolescents. The MAP curriculum includes modules to guide education about psychotherapeutic procedures (e.g., behavioral activation) common in evidence-based treatments for an array of childhood problems and to support development of competencies in assessment, treatment planning, and reflective practice. METHODS: Curriculum coding was used to examine the relevance of MAP's core components to the skills articulated in the Accreditation Council for Graduate Medical Education (ACGME) CAP milestones. Feasibility, acceptability, and learning outcomes were examined after delivery of the MAP curriculum to 12 CAP fellows at two sites, with instructional features tailored according to faculty preferences and training program structure. RESULTS: Coding suggested that the MAP curriculum was relevant to 95% of the 21 ACGME CAP training subcompetencies. Feasibility was indicated by the successful delivery of 100% of the planned MAP curriculum across the two sites. Acceptability was supported by positive feedback from the CAP fellows, and psychotherapy knowledge increased significantly. Finally, case review scores (mean±SD=2.21±0.15) showed positive posttraining application of MAP to two patients and exceeded scores achieved by other samples of mental health professionals. CONCLUSIONS: This pilot study demonstrated the potential for the MAP curriculum to support CAP education. MAP's versatility as a curriculum supports broader adoption, with continuing rigorous empirical evaluation.

Creating a prepared mental health workforce: comparative illustrations of implementation strategies.

BACKGROUND: Psychotherapy implementation must contend with the task of preparing a mental health workforce to provide the highest quality services to as much of a service population as possible, in high-income as well as low-to-middle income countries. OBJECTIVE: We outline general challenges and solutions and investigate how well various implementation strategies would fit a clinical population. METHODS: Using a data set from a prior cluster randomised trial with a clinically diverse population and 33 intervention practices, we presented multiple illustrations comparing the ability of different implementation strategies to serve youth and families with procedures in which service providers were trained. FINDINGS: A series of survival functions demonstrated that many common implementation strategies are unlikely to create a prepared workforce, given the large and diverse number of practices needed to be mastered by providers. CLINICAL IMPLICATIONS: 'Benchmark' solutions that afforded superior coverage of the service population could be supported through paced learning approaches (ie, training interventions a little at a time) using extensible, modular intervention designs.

The value of a modular, multi-focal, therapeutic approach to addressing child maltreatment: Hope for Children and Families Intervention Resources - a discussion article.

This discussion article begins by highlighting two trends apparent in the field of child maltreatment. The first, an awareness that multiple forms of maltreatment - polyvictimization - is the rule in populations of abused and neglected children rather than the exception. The second is that current types of child maltreatment are being extended to include Adverse Childhood Experiences (ACEs). These include intra-familial violence, mental health, substance misuse, and inter-generational abuse. The paper introduces an innovative strategy to help the field better organise and prevent the extensive sequelae of polyvictimization and ACEs. This strategy involves the development of a modular approach, which identifies common treatment elements and common factors across the field of effective interventions and organizes them, providing a co-ordinated framework for practitioners to use to address the diverse needs of children and families when vulnerability or maltreatment are identified. The development of this approach, the Hope for Children and Families (HfCF) Intervention Resources, is described using a case example to illustrate its logic and structure. Findings from the HFCF pilot and subsequent training programs suggest that this new approach could be an important milestone in the protection of children from violence, abuse and neglect on the 30th Anniversary of the United Nation's Convention on the Rights of the Child (1989).

Design process and protocol description for a multi-problem mental health intervention within a stepped care approach for adolescents in India.

This paper documents the collaborative design of a mental health intervention for adolescents in India with anxiety, depression, or anger-related concerns. The process was characterized by three phases of formative activities: (1) an intensive review of the service context, (2) selection of an overall design strategy (e.g., whether to choose existing evidence-based treatments or build new treatments in context), and (3) a period of prototyping, testing, and refining. Each phase resulted in specific outputs, which were, respectively, (1) a detailed articulation of values and preferences (setting expectations for what the ideal protocol should be), (2) a set of build parameters representing a blueprint that managed strategic compromises for this context, and (3) a working protocol. We outline the steps of this design process, summarize data from an open-trial clinical case series, and illustrate the resulting working protocol, which will be tested in a future larger trial. We conclude with insights and observations likely to be relevant to protocol design activity in a variety of contexts, most particularly those in low-and-middle-income countries such as India.

Managing and Adapting Practice: Provider Perceptions of an Evidence-Informed Framework for Delivering Mental Health Services.

This study examined providers' reflections on delivering managing and adapting practice (MAP), an evidence-informed framework that guides decision-making from scientific and client data. Consensual qualitative research methods were used to analyze the reflections of 201 youth mental health providers. Results indicated that providers approached MAP according to their own preferences and particular cases. While most appeared to approach MAP from a practice management standpoint, when faced with challenging cases, providers used coordination and outcomes management resources. Regardless of approach, most providers came to appreciate the full framework through reflective practice. Their diverse approaches offer lessons for evidence-based practice implementation and sustainment.

Implementing Evidence Based Practices for Children's Mental Health: A Case Study in Implementing Modular Treatments in Community Mental Health.

There is strong enthusiasm for utilizing implementation science in the implementation of evidence-based programs in children's community mental health, but there remains work to be done to improve the process. Despite the proliferation of implementation frameworks, there is limited literature providing case examples of overcoming implementation barriers. This article examines whether the use of three implementations strategies, a structured training and coaching program, the use of professional development portfolios for coaching, and a progress monitoring data system, help to overcome barriers to implementation by facilitating four implementation drivers at a community mental health agency. Results suggest that implementation is a process of recognizing and adapting to both predictable and unpredictable barriers. Furthermore, the use of these implementation strategies is important in improving implementation outcomes.

Overlapping and Non-overlapping Practices in Usual and Evidence-Based Care for Youth Anxiety.

This study compared consistencies and discrepancies in usual care with practices derived from the evidence-base (PDEB) for youth anxiety in a public mental health system. Youth-level factors (diagnosis, functional impairment) as predictors of the discrepancies were also examined. Psychosocial and service data from 2485 youth with an anxiety disorder and/or receiving services for an anxiety treatment target were extracted. Therapists (N = 616) identified the treatment targets and practices youth received. Although many PDEB for youth anxiety were used by therapists in this sample, Exposure was only used in 15% of cases. Practices not consistent with youth anxiety treatment were also reported and included: PDEB for other conditions, practices common to all therapies, and practices that are not consistent with evidence-based care. Age and diagnosis predicted the delivery of PDEB for youth anxiety. Usual care incorporated many components of evidence-based care but was more diffuse and less focused on well-supported practices.

Child STEPs in California: A cluster randomized effectiveness trial comparing modular treatment with community implemented treatment for youth with anxiety, depression, conduct problems, or traumatic stress.

OBJECTIVE: This study reports outcomes from a randomized effectiveness trial testing modular treatment versus multiple community-implemented evidence-based treatments for youth. METHOD: An ethnoracially diverse sample of 138 youth ages 5 to 15 (62 girls, 76 boys) whose primary clinical concerns involved diagnoses or clinical elevations related to anxiety, depression, disruptive behavior, and/or traumatic stress were treated by community therapists randomly assigned to 1 of 2 conditions: (a) modular treatment, which involved a single modular protocol (i.e., modular approach to therapy for children; MATCH) that allowed flexible selection and sequencing of procedures to fit the chosen treatment focus in the context of measurement feedback, and (b) community-implemented treatment (CIT), which was a county-supported implementation of multiple evidence-based practices for youth. RESULTS: Youth treated with MATCH showed significantly faster rates of improvement over time on clinical and functional outcomes relative to youth in the CIT condition and required significantly fewer sessions delivered over significantly fewer days. Caregiver-reported clinical improvement rates were significantly greater for MATCH (60%) versus CIT (36.7%). Further, youth in the CIT condition were significantly more likely to receive additional psychosocial treatment services and were significantly more likely to use a variety of psychotropic medications during the active treatment phase. CONCLUSIONS: These results extend prior findings, supporting the effectiveness and efficiency of a modular, multifocus approach that incorporates monitoring and feedback relative to community implementation of evidence-based treatments. (PsycINFO Database Record

Evidence-Based Decision Making in Youth Mental Health Prevention.

Given the impressive amount of knowledge relevant to prevention efforts, this paper articulates strategies to capitalize on such knowledge through evidence-based decision making. Knowledge, or "evidence," is understood here as coming from multiple sources, including research, individual people, group history, and relevant theory. The presented strategies to facilitate evidence-based decision making are: (1) intervention knowledge management; (2) collaborative design; (3) knowledge resources for intervention; and (4) developmentally sensitive training and supervision. Examples and benefits are outlined for each strategy. It is ultimately argued that evidence-based decision making embodies the scientific approach, and is worth consideration within primary prevention, given its early success in secondary intervention of youth mental health.

At the Intersection of Health Information Technology and Decision Support: Measurement Feedback Systems... and Beyond.

We select and comment on concepts and examples from the target articles in this special issue on measurement feedback systems, placing them in the context of some of our own insights and ideas about measurement feedback systems, and where those systems lie at the intersection of technology and decision making. We contend that, connected to the many implementation challenges relevant to many new technologies, there are fundamental design challenges that await a more elaborate specification of the clinical information and decision models that underlie these systems. Candidate features of such models are discussed, which include referencing multiple evidence bases, facilitating observed and expected value comparisons, fostering collaboration, and allowing translation across multiple ontological systems. We call for a new metaphor for these technologies that goes beyond measurement feedback and encourages a deeper consideration of the increasingly complex clinical decision models needed to manage the uncertainty of delivering clinical care.

Building an evidence-informed service array: Considering evidence-based programs as well as their practice elements.

OBJECTIVE: This study empirically examined options for building an evidence-informed service array, comparing strategies to maximize the application of evidence-based treatment literature in a clinical service system. The overall goal was to determine the smallest set of treatments that could serve the largest percentage of clients. Solutions to this problem differ depending on how one defines "treatment." METHOD: Treatments were conceptualized as (a) programs (integrated treatments produced by specific research laboratories or investigators), and (b) collections of their constituent common procedures, referred to as practice elements. Programs listed by 2 separate government-sanctioned registries were selected to illustrate the effects of "program" conceptualizations, and all available clinical trials testing the programs were analyzed. Practice elements were identified from these same studies and from studies of other treatments that met a standard of evidence but had not been organized into programs on these lists. Relevance mapping methodology was used to identify optimal sets of programs and practice elements. RESULTS: Among a large, diverse clinical population, results identified 11%-22% of youths for whom practice elements provide an evidence-informed treatment option whereas no programs meeting the standard of evidence were available on the registries. Results for the practice elements were able to be matched by a hybrid combination: "best" programs, which were then extended by practice elements. CONCLUSIONS: These results demonstrated that there are multiple ways to conceptualize treatments when planning a service array, and these options have significant implications regarding who can be served by treatments supported by evidence.

The common elements of engagement in children's mental health services: which elements for which outcomes?

Using the distillation component of the Distillation and Matching Model framework (Chorpita, Daleiden, & Weisz, 2005 ), we examined which engagement practices were associated with three domains of treatment engagement: attendance, adherence, and cognitive preparation (e.g., understanding of, readiness for treatment). Eighty-nine engagement interventions from 40 randomized controlled trials in children' s mental health services were coded according to their engagement practices and outcomes. Analyses examined whether the practices used in successful interventions differed according to engagement domain. Practice patterns differed somewhat depending on whether attendance, adherence, or cognitive preparation was the outcome of interest. For example, assessment of barriers to treatment frequently occurred in successful interventions targeting attendance, whereas homework assignment frequently occurred in successful interventions when adherence was the target outcome. Modeling and expectation setting were frequently used in successful interventions targeting cognitive preparation for treatment. Distillation provides a method for examining the practice patterns associated with different engagement outcomes. An example of the application of these findings to clinical practice includes using certain practices (e.g., assessment, psychoeducation about services, and accessibility promotion) with all youth and families to promote attendance, adherence, and cognitive preparation. Then, other practices (e.g., modeling, homework assignment) can be added on an as-needed basis to boost engagement or to address interference in a particular engagement domain. The use of a distillation framework promotes a common language around engagement and highlights practices that lend themselves well to training, thereby promoting the dissemination of engagement interventions.

Fit of evidence-based treatment components to youths served by wraparound process: a relevance mapping analysis.

This study investigated whether and which evidence-based treatment (EBT) components might generalize to youths served by the wraparound process. To examine these questions, the study used relevance mapping, an empirical methodology that compares youths in a given clinical population with participants in published randomized trials to determine who may be "coverable" by EBTs and which treatments may collectively be most applicable. In a large diverse clinical sample, youths receiving wraparound services (n = 828) were compared with youths receiving other services (n = 3,104) regarding (a) demographic and clinical profiles, (b) "coverability" by any EBTs, and (c) specific practices from those EBTs that most efficiently applied to each group. Participants in studies of EBTs matched the demographic and clinical characteristics of nearly as many youths receiving wraparound (58-59%) as those receiving non-wraparound services (61-64%). Moreover, the best-fitting solutions of relevant sets of practices were highly similar across groups. These results provide the first large-scale empirical characterization of fit between EBTs and youths receiving wraparound and suggest that these youths are well suited to benefit from clinical strategies commonly used in EBTs.

Doing more with what we know: introduction to the special issue.

This article introduces a special issue that provides an overarching conceptual model for advancing the impact of science on service delivery and includes some illustrations of these concepts in action from a variety of research teams in a diversity of contexts. Themes of the special issue include balancing considerations of efficacy, effectiveness, and extensiveness--for example, ensuring that treatments are not only reliable but also robust and widely relevant. It is argued that such a balance might be achieved through an increased focus on improving coordination of system resources (e.g., clients, providers, treatments, training programs) and through dynamic and developmental designs. The special issue is intended to raise the broader question of whether our current paradigms are satisfactory in moving us toward our shared vision, or whether we could do more with what we already know.

Identifying the common elements of treatment engagement interventions in children's mental health services.

Difficulty engaging families in mental health treatment is seen as an underlying reason for the disparity between child mental health need and service use. Interpretation of the literature on how best to engage families is complicated by a diversity of operational definitions of engagement outcomes and related interventions. Thus, we sought to review studies of engagement interventions using a structured methodology allowing for an aggregate summary of the most common practices associated with effective engagement interventions. We identified 344 articles through a combination of database search methods and recommendations from engagement research experts; 38 articles describing 40 studies met our inclusion criteria. Following coding methods described by Chorpita and Daleiden (J Consul Clin Psychol 77(3):566-579, 2009, doi: 10.1037/a0014565 ), we identified 22 engagement practice elements from 89 study groups that examined or implemented family engagement strategies. Most frequently identified engagement practice elements included assessment, accessibility promotion, psychoeducation about services, homework assignment, and appointment reminders. Assessment and accessibility promotion were two practice elements present in at least 50 % of treatment groups that outperformed a control group in a randomized controlled trial. With the exception of appointment reminders, these frequently identified engagement practice elements had a high likelihood of being associated with winning treatments when they were used. This approach offers a novel way of summarizing the engagement literature and provides the foundation for enhancing clinical decision-making around treatment engagement.

MAPping Los Angeles County: taking an evidence-informed model of mental health care to scale.

We describe the scaling up of an evidence-informed model of care, Managing and Adapting Practice (MAP) in Los Angeles County, California. MAP complemented an array of evidence-based programs selected by the county as part of a large system reform effort designed to improve care for children and adolescents. In addition, we discuss the MAP model for training therapists and present data both on how the training model performed and on the outcomes of youths treated by therapists trained in MAP. We examined the success of two different training pathways for MAP therapists: (a) national training model and (b) MAP agency supervisor model (i.e., train the trainer). We also examined utilization of MAP and outcomes of clients served by MAP. Both the national training and MAP agency supervisor model were successful in producing MAP therapists in a timely fashion and with acceptable competency scores. Furthermore, a large number of clients were receiving MAP services. Finally, outcomes for youth treated with MAP were strong, with effect sizes ranging from .59 to .80 on the Youth Outcome Questionnaire. These data support the notion that scaling up a mental health services approach in a system can be achieved through a strong and broad partnership among relevant stakeholders, can involve a train-the-trainer model, and can result in strong outcomes for clients.

Structuring the collaboration of science and service in pursuit of a shared vision.

The enduring needs of our society highlight the importance of a shared vision to improve human functioning and yield better lives for families and communities. Science offers a powerful strategy for managing the inevitable uncertainty in pursuit of these goals. This article presents ideas and examples of methods that could preserve the strengths of the two major paradigms in children's mental health, evidence-based treatments and individualized care models, but that also have the potential to extend their applicability and impact. As exemplified in some of the articles throughout this issue, new models to connect science and service will likely emerge from novel consideration of better ways to structure and inform collaboration within mental health systems. We contend that the future models for effective systems will involve increased attention to (a) client and provider developmental pathways, (b) explicit frameworks for coordinating people and the knowledge and other resources they use, and (c) a balance of evidence-based planning and informed adaptation. We encourage the diverse community of scientists, providers, and administrators in our field to come together to enhance our collective wisdom through consideration of and reflection on these concepts and their illustrations.