The latent structure of the Delis-Kaplan system for autism.

A core feature of autism is deficits in executive functioning (EF), including difficulty with planning, cognitive flexibility, and working memory. Despite a growing need for evidence-based assessments of EF for autism populations, statistical models of many commonly used measures of EF, including the Delis-Kaplan Executive Function System (D-KEFS), have not been investigated for a sample of autistic participants. The purpose of this study was to address a gap in the literature regarding the latent structure of the D-KEFS in a sample of autistic individuals. The D-KEFS is one of the most widely used clinical assessments of executive function, but its factor structure has not been examined in a sample of autistic participants. Reliability analyses were performed for sample subgroups based on participants' clinical and demographic characteristics, including IQ, autism severity, age, and race/ethnicity. Verbal Fluency (VF) was found to consistently decrease or not affect the overall reliability score. Additionally, one- and two-factor structure models were tested for the D-KEFS with a sample of autistic participants. The one-factor model was not found to be a good fit for the data. However, the two-factor model, with Cognitive Flexibility and Abstraction latent factors, was found to fit the data relatively well. This two-factor model was reexamined excluding the VF observed variable, resulting in a better overall model fit. Communication deficits are a common feature of autism, which explains why the VF task, that requires participants to produce novel words, may not be an adequate measure of executive function for autism populations.

Randomized Feasibility Pilot of an Executive Functioning Intervention Adapted for Children's Mental Health Settings.

The critical role of executive functioning in autism as well as the co-occurring mental health challenges common among autistic youth support to the immense value of interventions targeting executive functioning for enhancing mental health services for autistic children. The goal of the present study was to conduct a randomized feasibility trial of Unstuck and On Target, an executive functioning intervention, adapted for delivery in children's community mental health setting. Mental health therapists (n = 26) enrolled with participating autistic clients (n = 32) were randomized to receive training in and deliver the adapted Unstuck intervention or to deliver care as usual. We completed masked observational measures of Unstuck strategy use (fidelity) during recorded sessions of participating therapist-client dyads and collected measures of acceptability from participating clients and their caregivers. We also collected measures of pre-post changes in executive functioning and mental health symptoms. Therapists trained in Unstuck demonstrated significantly higher use of Unstuck strategies compared to usual care therapists. Caregivers and autistic clients perceive adapted Unstuck as highly acceptability and helpful. Autistic clients whose therapists were trained in adapted Unstuck demonstrated larger pre-post changes in executive functioning compared to usual care. Across all participating clients, changes in executive functioning were significantly related to changes in mental health symptoms. Finally, clients of therapists trained in adapted Unstuck demonstrated moderate improvements in overall mental health symptoms. The current study provides preliminary evidence of the feasibility and impact of Unstuck and On Target for children's community mental health settings.

Enhancing Behavioral Health Implementation in a Care Coordination Program at a Federally Qualified Health Center: A Case Study Applying Implementation Frameworks.

Federally Qualified Health Centers are charged with providing comprehensive health care in traditionally underserved areas, underscoring their importance in caring for and promoting health equity for the large portion of historically marginalized communities in this setting. There is a significant need to ensure Federally Qualified Health Centers are equipped to appropriately address the immense behavioral health needs common among patients served. Care coordination is an evidence-based model that is increasingly utilized in Federally Qualified Health Centers to improve care equity and outcomes. Addressing and supporting behavioral health needs is a key aspect of such care coordination models. Context-specific considerations and programmatic supports, particularly those that address the needs of care coordinators and the complex patients they serve, are needed to ensure such models can appropriately meet and address the behavioral health concerns of the diverse populations served. The goal of this study was to present a mixed-methods case study that systematically applies implementation frameworks to conduct a needs and context assessment to inform the development and testing of evidence-based practice strategies and implementation support as part of a care coordination program within a partnered Federally Qualified Health Center.

Mental Health Therapist Perspectives on the Role of Executive Functioning in Children's Mental Health Services.

Executive functioning is considered a key transdiagnostic factor underlying multiple mental health conditions. Evidence-based interventions targeting executive functioning skills exist and there are ongoing efforts to implement these interventions in routine community-based care. However, there is limited research characterizing therapist perspectives regarding addressing executive functioning within community-based mental health services. The current mixed-methods study aims to characterize mental health therapist perspectives regarding the role of executive functioning in youth clinical presentation and the psychotherapy process and outcomes as well as their experience and training related to executive functioning. Forty-three therapists completed a survey about their perspectives about executive functioning and 14 participated in subsequent focus groups to expand survey results. Results indicated that therapists perceive executive functioning challenges as impacting a large portion of children on their caseload, present across multiple mental health conditions, and that executive functioning is frequently a focus of psychotherapy. Therapists also identified executive functioning challenges as a barrier to therapy engagement and effectiveness. However, therapists reported limited knowledge and training as well as significant motivation for executive functioning training, including executive functioning interventions. Findings confirm the significant role of executive functioning in youth presenting for mental health services and the psychotherapy process and outcomes. Findings also highlight the need for further translation of evidence-based interventions and tools. Challenges as well as potential solutions to inform ongoing and future work seeking to translate and implement evidence-based executive functioning interventions in community mental health services are discussed.

Mental health symptoms of youth initiating psychiatric care at different phases of the COVID-19 pandemic.

OBJECTIVE: To examine differences in caregiver and youth reported mental health symptoms for youth initiating mental health treatment through phases of the Coronavirus Disease (COVID-19) pandemic, compared with symptomology reported the prior year. STUDY DESIGN: This retrospective study analyzes group differences in mental health symptoms (Pediatric Symptom Checklist; PSC-35) based on 7874 youth seeking treatment in publicly funded mental health treatment programs during California's Stay-At-Home order (March-May, 2020) and the prolonged pandemic (May-December, 2020) phases of the COVID-19 pandemic as compared with matching groups in 2019. RESULTS: Youth entering mental health treatment services, and their caregivers, reported significantly increased internalizing, externalizing, and attention-related symptoms during the prolonged pandemic phase, but not during the acute stay-at-home phase of the COVID-19 pandemic, and with small effect sizes. Group comparison analyses did not detect a significantly larger effect for Sexual and Gender Diverse (SGD) youth who identify as lesbian, gay, bisexual, asexual, transgender, Two-Spirit, queer, and/or intersex, and Black, Indigenous, People of Color (BIPOC). CONCLUSIONS: A large-scale comparison of youth mental health symptoms before and during the COVID-19 pandemic suggests that mental health was disrupted for youth seeking treatment as the pandemic prolonged throughout 2020.

Characterization of multilevel influences of mental health care transitions: a comparative case study analysis.

PURPOSE: Mental health care transitions are increasingly prioritized given their potential to optimize care delivery and patient outcomes, especially those focused on the transition from inpatient to outpatient mental health care. However, limited efforts to date characterize such mental health transition practices, especially those spanning multiple service setting contexts. Examination of key influences of inpatient to outpatient mental health care transitions across care contexts is needed to inform ongoing and future efforts to improve mental health care transitions. The current work aims to characterize multilevel influences of mental health care transitions across three United States-based mental health system contexts. METHODS: A comparative multiple case study design was used to characterize transition practices within the literature examining children's, non-VA adult, and VA adult service contexts. Andersen's (1995) Behavioral Health Service Use Model was applied to identify and characterize relevant distinct and common domains of focus in care transitions across systems. RESULTS: Several key influences to mental health care transitions were identified spanning the environmental, individual, and health behavior domains, including: community capacity or availability, cross-system or agency collaboration, provider training and experience related to mental health care transitions, client care experience and expectations, and client clinical characteristics or complexity. CONCLUSIONS: Synthesis illustrated several common factors across system contexts as well as unique factors for further consideration. Our findings inform key considerations and recommendations for ongoing and future efforts aiming to plan, expand, and better support mental health care transitions. These include timely information sharing, enhanced care coordination and cross setting and provider communication, continued provider/client education, and appropriate tailoring of services to improve mental health care transitions.

Applying Implementation Mapping to Expand a Care Coordination Program at a Federally Qualified Health Center.

Background: A large and growing percentage of medically underserved groups receive care at federally qualified health centers (FQHCs). Care coordination is an evidence-based approach to address disparities in healthcare services. A partnered FQHC established a care coordination model to improve receipt and quality of healthcare for patients most at risk for poor health outcomes. This care coordination model emphasizes identification and support of behavioral health needs (e.g., depression, anxiety) and two evidence-based behavioral health programs needs were selected for implementation within the context of this care coordination model. Implementation Mapping is a systematic process for specifying the implementation strategies and outcomes. The current case study describes the application of Implementation Mapping to inform the selection and testing of implementation strategies to improve implementation of two behavioral health programs in a Care Coordination Program at a partnered FQHC. Methods: We applied Implementation Mapping to inform the development, selection and testing of implementation strategies to improve the implementation of two evidence-based behavioral health programs within a care coordination program at a partnered FQHC. Results: Results are presented by Implementation Mapping task, from Task 1 through Task 5. We also describe the integration of additional implementation frameworks (The Consolidated Framework for Implementation Research, Health Equity Implementation Framework) within the Implementation Mapping process to inform determinant identification, performance and change objectives development, design and tailoring of implementation strategies and protocols, and resulting evaluation of implementation outcomes. Conclusions: The current project is an example of real-world application of Implementation Mapping methodology to improve care outcomes for a high priority population that is generalizable to other settings utilizing similar care models and health equity endeavors. Such case studies are critical to advance our understanding and application of innovative implementation science methods such as Implementation Mapping.

Provider perspectives and reach of an evidence-based intervention in community services for toddlers.

LAY ABSTRACT: Expert recommendations for toddlers who are likely to develop autism include caregivers being actively involved in the services children receive. However, many services available in the community may not follow these recommendations. Evidence suggests that an intervention named Project ImPACT for Toddlers demonstrates positive parent and child outcomes for families in the community. Project ImPACT for Toddlers was designed specifically for toddlers by a group of parents, clinicians, researchers, and funders. It teaches parents of young children strategies to support their child's development in daily routines. This study reports the perspectives of early intervention providers who learned to use Project ImPACT for Toddlers on whether the intervention was a good fit for their practice and easy to use. The study also examines how many agencies are using Project ImPACT for Toddlers and how many families have received the intervention in the community. The goal of the study is to inform the continued use of Project ImPACT for Toddlers in the community and support offering the intervention in other regions. Participants include 38 community providers who participated in a training study of Project ImPACT for Toddlers and completed a survey and semi-structured interview after approximately 3 months of using Project ImPACT for Toddlers with families. Participants perceived the training model as acceptable and appropriate, and identified the group-based model of training, comprehensive materials, and agency support as strengths of the approach. Survey findings complemented the results from the interviews. Data indicate an increasing number of agencies and families accessing Project ImPACT for Toddlers. Efforts to expand evidence-based intervention in early intervention should continue to build upon the model used for Project ImPACT for Toddlers.

Impact of Therapist Training on Parent Attendance in Mental Health Services for Children with ASD.

OBJECTIVE: The current study explored the impact of training therapists in a mental health intervention for children with autism spectrum disorder (ASD) on parent attendance in their children's therapy sessions. We also examined family, therapist, and program factors as potential moderators. METHOD: Data were drawn from a cluster-randomized community effectiveness trial of "An Individualized Mental Health Intervention for ASD (AIM HI)". Participants included 168 therapists yoked with 189 children recruited from publicly-funded mental health services. Data included family (caregiver strain, parent sense of competence, race/ethnicity), therapist (background, experience), and program (service setting) characteristics, and parent session attendance. Multilevel models were used to evaluate the effectiveness of AIM HI therapist training on caregiver attendance and identify moderators of training effects on parent attendance. RESULTS: Parents attended a higher percentage of sessions in the AIM HI training condition compared to the Usual Care condition. Program service setting moderated the effect of AIM HI training, with higher parent attendance in non-school (mostly outpatient) settings compared to school settings and a significantly smaller difference between the settings in the AIM HI condition. CONCLUSIONS: Effective strategies to promote parent engagement, especially in service settings such as schools, are warranted. Findings support the effectiveness of AIM HI training in promoting parent attendance across multiple publicly-funded mental health service settings. The larger effect in school-based programs supports the utility of training in evidence-based interventions such as AIM HI to increase the feasibility of parent attendance in such services.

Short report: A quantitative methodological review of participant characteristics in the literature testing mental health interventions for youth with autism spectrum disorder.

LAY ABSTRACT: Previous research has highlighted the importance of mental health treatment for autistic youth. In that research base, most studies focus on demonstrating the efficacy of a particular intervention with a sample of autistic youth. However, understanding the characteristics of samples used within these studies (i.e. demographics) is an important avenue for expanding this research to a more diverse, representative sample of autistic youth in community settings. As such, the current review examined and characterized participants included within mental health treatment research. We coded studies for various demographics among the youth sample, caregivers, and providers participants. Results indicated that while efforts have been made to increase diversity in research, very few studies including transition-aged youth, those identifying as female, and/or those identifying as non-Caucasian. Clinically, a few studies included youth with lower cognitive abilities and/or those with specific mental health problems (e.g. trauma and depression) or more than one co-occurring mental health conditions. Overall, our results highlight several critical gaps in our current evidence base regarding mental health treatment for autistic youth, including the limited clinical representativeness of both provider and child participants.

Caregiver Participation Engagement in Child Mental Health Prevention Programs: a Systematic Review.

Prevention programs are a key method to reduce the prevalence and impact of mental health disorders in childhood and adolescence. Caregiver participation engagement (CPE), which includes caregiver participation in sessions as well as follow-through with homework plans, is theorized to be an important component in the effectiveness of these programs. This systematic review aims to (1) describe the terms used to operationalize CPE and the measurement of CPE in prevention programs, (2) identify factors associated with CPE, (3) examine associations between CPE and outcomes, and (4) explore the effects of strategies used to enhance CPE. Thirty-nine articles representing 27 unique projects were reviewed. Articles were included if they examined CPE in a program that focused to some extent on preventing child mental health disorders. There was heterogeneity in both the terms used to describe CPE and the measurement of CPE. The majority of projects focused on assessment of caregiver home practice. There were no clear findings regarding determinants of CPE. With regard to the impact of CPE on program outcomes, higher levels of CPE predicted greater improvements in child and caregiver outcomes, as well as caregiver-child relationship quality. Finally, a small number of studies found that motivational and behavioral strategies (e.g., reinforcement, appointment reminders) were successful in promoting CPE. This review highlights the importance of considering CPE when developing, testing, and implementing prevention programs for child mental health disorders. Increased uniformity is needed in the measurement of CPE to facilitate a better understanding of determinants of CPE. In addition, the field would benefit from further evaluating strategies to increase CPE as a method of increasing the potency of prevention programs.

Mental Health Services for Autistic Individuals Across the Lifespan: Recent Advances and Current Gaps.

PURPOSE OF REVIEW: This synthesis of recent mental health services research with autistic individuals presents significant advances, current gaps, and recommendations for improving mental healthcare for this population. RECENT FINDINGS: Recent advances include improved understanding of co-occurring mental health conditions among autistic individuals, a growing evidence base for interventions to address them, the development and implementation of new service models to support mental health for this population, and a substantial increase in mental health services and implementation research focused on autism. Ongoing challenges include a lack of mental health interventions designed for community implementation with autistic individuals, limited workforce capacity, complex and disconnected service systems, and racial, ethnic, and socioeconomic disparities in accessibility and quality of mental health services. Despite the advances in our understanding of mental health needs and mental health services for autistic individuals, several critical gaps remain. We encourage future efforts to develop and test interventions that can be used in community settings, train and incentivize the workforce to provide them, realign policies and funding with best practice, and embrace an equity-focused approach to autism research and care.

Correction to: A Systematic Review of Mental Health Interventions for ASD: Characterizing Interventions, Intervention Adaptations, and Implementation Outcomes.

Due to the errors occurred in the originally published version, this article is being reprinted in its entirety as Correction. All errors have been corrected. It is the correct version.

Value of peer mentoring for early career professional, research, and personal development: a case study of implementation scientists.

Effective mentoring is a key mechanism propelling successful research and academic careers, particularly for early career scholars. Most mentoring programs focus on models pairing senior and early career researchers, with limited focus on peer mentoring. Peer mentoring may be especially advantageous within emerging areas such as implementation science (IS) where challenges to traditional mentoring may be more prevalent. This special communication highlights the value of peer mentoring by describing a case study of an early career IS peer mentoring group. We delineate our curriculum and structure; support and processes; and products and outcomes. We highlight important group member characteristics to consider during group formation and continuation. The group's long-term (6 years) success was attributed to the balance of similarities and differences among group members. Members were in a similar career phase and used similar methodologies but studied different health topics at different institutions. Group members gave and received instrumental and psychosocial support and shared resources and knowledge. Peer mentoring can serve an important function to provide emotional, logistical, and professional development support for early career scholars. Our case study highlights strategies to foster peer mentoring groups that provide a generalizable blueprint and opportunity for improved outcomes for early career professionals.

Concordance between Community Supervisor and Provider Ratings of Fidelity: Examination of Multi-Level Predictors and Outcomes.

Fidelity is conceptualized as a key mechanism impacting treatment outcomes, yet community examination of fidelity is limited, partly due to limited feasible training methods and tools. Provider-report tools represent a feasible method for improving routine fidelity monitoring. Additional work examining the effectiveness of these methods and concordance with other fidelity sources is needed. Further, provider and client factors impact both intervention implementation and concordance, but their impact on fidelity measurement is poorly understood. Using a train-the-trainer methodology, we examine concordance between three methods of assessing fidelity (trained independent coders, supervisor evaluation and provider self-report) using a fidelity assessment tool adapted for community use. Multilevel models examined predictors (provider- and child- level factors) and outcomes (provider average fidelity) of concordance. Results suggest supervisors and providers are able to use the fidelity tool, but demonstrated variable concordance, with higher concordance with trained coders for supervisors than providers. Provider credentials and child language were marginally predictive of rater concordance. Higher supervisor accuracy was associated with higher provider fidelity. Findings inform broader efforts to improve the development and implementation of feasible fidelity tools for routine use in community settings and highlight multi-level factors impacting the adoption and implementation of such tools.

A more practical guide to incorporating health equity domains in implementation determinant frameworks.

BACKGROUND: Due to striking disparities in the implementation of healthcare innovations, it is imperative that researchers and practitioners can meaningfully use implementation determinant frameworks to understand why disparities exist in access, receipt, use, quality, or outcomes of healthcare. Our prior work documented and piloted the first published adaptation of an existing implementation determinant framework with health equity domains to create the Health Equity Implementation Framework. We recommended integrating these three health equity domains to existing implementation determinant frameworks: (1) culturally relevant factors of recipients, (2) clinical encounter or patient-provider interaction, and (3) societal context (including but not limited to social determinants of health). This framework was developed for healthcare and clinical practice settings. Some implementation teams have begun using the Health Equity Implementation Framework in their evaluations and asked for more guidance. METHODS: We completed a consensus process with our authorship team to clarify steps to incorporate a health equity lens into an implementation determinant framework. RESULTS: We describe steps to integrate health equity domains into implementation determinant frameworks for implementation research and practice. For each step, we compiled examples or practical tools to assist implementation researchers and practitioners in applying those steps. For each domain, we compiled definitions with supporting literature, showcased an illustrative example, and suggested sample quantitative and qualitative measures. CONCLUSION: Incorporating health equity domains within implementation determinant frameworks may optimize the scientific yield and equity of implementation efforts by assessing and ideally addressing implementation and equity barriers simultaneously. These practical guidance and tools provided can assist implementation researchers and practitioners to concretely capture and understand barriers and facilitators to implementation disparities.

Forms and functions of bridging factors: specifying the dynamic links between outer and inner contexts during implementation and sustainment.

BACKGROUND: Bridging factors are relational ties, formal arrangements, and processes that connect outer system and inner organizational contexts. They may be critical drivers of evidence-based practice (EBP) implementation and sustainment. Yet, the complex interplay between outer and inner contexts is often not considered. Bridging factors were recently defined in the updated Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Further identification and specification of this construct will advance implementation models, measures, and methods. Our goal is to advance bridging factor research by identifying relevant dimensions and exemplifying these dimensions through illustrative case studies. METHODS: We used a multiple case study design. Each case (n = 10) represented different contexts, EBPs, and bridging factor types. Inclusion criteria were the presence of clearly distinguishable outer and inner contexts, identifiable bridging factor, sufficient information to describe how the bridging factor affected implementation, and variation from other cases. We used an iterative qualitative inquiry process to develop and refine a list of dimensions. Case data were entered into a matrix. Dimensions comprised the rows and case details comprised the columns. After a review of all cases, we collectively considered and independently coded each dimension as function or form. RESULTS: We drew upon the concepts of functions and forms, a distinction originally proposed in the complex health intervention literature. Function dimensions help define the bridging factor and illustrate its purpose as it relates to EBP implementation. Form dimensions describe the specific structures and activities that illustrate why and how the bridging factor has been customized to a local implementation experience. Function dimensions can help researchers and practitioners identify the presence and purpose of bridging factors, whereas form dimensions can help us understand how the bridging factor may be designed or modified to support EBP implementation in a specific context. We propose five function and three form bridging factor dimensions. CONCLUSIONS: Bridging factors are described in many implementation models and studies, but without explicit reference or investigation. Bridging factors are an understudied and critical construct that requires further attention to facilitate implementation research and practice. We present specific recommendations for a bridging factors research agenda.

A Systematic Review of Mental Health Interventions for ASD: Characterizing Interventions, Intervention Adaptations, and Implementation Outcomes.

Youth with autism spectrum disorder (ASD) have high rates of co-occurring mental health needs that necessitate mental health interventions. Given the unique clinical characteristics of youth with ASD, there have been significant efforts to adapt and test mental health interventions for this population. Yet, characterization of the nature and types of interventions adaptations is limited, especially across the wide range of interventions tested for youth with ASD with a focus on implementation factors. Additionally, understanding how these interventions may be implemented in community services is limited. The aims of this systematic review are to characterize the (1) types of interventions tested for co-occurring mental health conditions for youth with ASD; (2) adaptations to mental health interventions for use with youth with ASD; and (3) implementation strategies, outcomes, and determinants of mental health interventions to inform their translation to community service settings. Eighty-three articles testing interventions targeting mental health symptoms in youth with ASD that included implementation factors in analyses were reviewed. The Stirman et al. (2013; 2019) FRAME adaptation, Powell et al. (2012;2015) implementation strategies, and Proctor et al. (2011) implementation outcomes taxonomies were applied to characterize the nature and types of adaptations for use with youth with ASD and types of implementation strategies, outcomes, and determinants used, when available, respectively. Of the interventions examined, the majority (64.1%) were originally designed to target youth mental health concerns and were then adapted to be used with ASD. The most common adaptations included those to the intervention content, particularly adding elements with tailoring or refining aspects of the intervention while maintaining core functions. Half of the articles described at least one implementation strategy used during intervention testing. Fidelity and acceptability were the most frequently examined implementation outcomes, with some examination of appropriateness and feasibility. Nineteen percent of articles described implementation determinants (i.e. barriers/facilitators) of these implementation outcomes. The common adaptations for ASD provide direction for future intervention development and for training community therapists. Further examination, specification, and reporting of implementation strategies and outcomes within ongoing efforts to adapt and interventions to meet the co-occurring mental health needs of youth ASD are needed to facilitate their translation to community settings. Areas for future research as well as clinical implications are discussed.

Advancing the pragmatic measurement of sustainment: a narrative review of measures.

BACKGROUND: Sustainment, an outcome indicating an intervention continues to be implemented over time, has been comparatively less studied than other phases of the implementation process. This may be because of methodological difficulties, funding cycles, and minimal attention to theories and measurement of sustainment. This review synthesizes the literature on sustainment measures, evaluates the qualities of each measure, and highlights the strengths and gaps in existing sustainment measures. Results of the review will inform recommendations for the development of a pragmatic, valid, and reliable measure of sustainment. METHODS: A narrative review of published sustainment outcome and sustainability measures (i.e., factors that influence sustainment) was conducted, including appraising measures in the Society of Implementation Research Collaboration (SIRC) instrument review project (IRP) and the Dissemination and Implementation Grid-Enabled Measures database initiative (GEM-D&I). The narrative review used a snowballing strategy by searching the reference sections of literature reviews and definitions of sustainability and sustainment. Measures used frequently and judged to be comprehensive and/or validated by a team of implementation scientists were extracted for analysis. RESULTS: Eleven measures were evaluated. Three of the included measures were found in the SIRC-IRP, three in the GEM-D&I database, (one measure was in both databases) and six were identified in our additional searches. Thirteen constructs relating to sustainment were coded from selected measures. Measures covered a range of determinants for sustainment (i.e., construct of sustainability) as well as constructs of sustainment as an outcome. Strengths of the measures included, development by expert panels knowledgeable about particular interventions, fields or contexts, and utility in specific scenarios. A number of limitations were found in the measures analyzed including inadequate assessment of psychometric characteristics, being overly intervention or context specific, being lengthy and/or complex, and focusing on outer context factors. CONCLUSION: There is a lack of pragmatic and psychometrically sound measures of sustainment that can be completed by implementation stakeholders within inner context settings (e.g., frontline providers, supervisors).

Ten recommendations for using implementation frameworks in research and practice.

BACKGROUND: Recent reviews of the use and application of implementation frameworks in implementation efforts highlight the limited use of frameworks, despite the value in doing so. As such, this article aims to provide recommendations to enhance the application of implementation frameworks, for implementation researchers, intermediaries, and practitioners. DISCUSSION: Ideally, an implementation framework, or multiple frameworks should be used prior to and throughout an implementation effort. This includes both in implementation science research studies and in real-world implementation projects. To guide this application, outlined are ten recommendations for using implementation frameworks across the implementation process. The recommendations have been written in the rough chronological order of an implementation effort; however, we understand these may vary depending on the project or context: (1) select a suitable framework(s), (2) establish and maintain community stakeholder engagement and partnerships, (3) define issue and develop research or evaluation questions and hypotheses, (4) develop an implementation mechanistic process model or logic model, (5) select research and evaluation methods (6) determine implementation factors/determinants, (7) select and tailor, or develop, implementation strategy(s), (8) specify implementation outcomes and evaluate implementation, (9) use a framework(s) at micro level to conduct and tailor implementation, and (10) write the proposal and report. Ideally, a framework(s) would be applied to each of the recommendations. For this article, we begin by discussing each recommendation within the context of frameworks broadly, followed by specific examples using the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. SUMMARY: The use of conceptual and theoretical frameworks provides a foundation from which generalizable implementation knowledge can be advanced. On the contrary, superficial use of frameworks hinders being able to use, learn from, and work sequentially to progress the field. Following the provided ten recommendations, we hope to assist researchers, intermediaries, and practitioners to improve the use of implementation science frameworks.