Assessment of the quality of interdisciplinary communication (CritCom): evaluation and refinement of a center summary report.

Communication failures among clinicians in the ICU (intensive care unit) often lead to worse patient outcomes. CritCom is a bilingual (English and Spanish) tool to evaluate the quality of interdisciplinary communication around patient deterioration for pediatric oncology patients. The use of reports, such as the CritCom report, as dissemination methods lead to quicker knowledge translation and implementation of research findings into policy. Nurses and physicians at participating centers who care for patients at risk of deterioration completed the CritCom survey and center-specific reports were generated to communicate CritCom results. Focus groups were conducted with clinicians receiving CritCom reports in both English and Spanish to evaluate report clarity and usability. Participants found the reports to be useful and described the writing and design as clear and specific. Participants provided feedback to improve report design and requested actionable steps to improve communication at their center. Feedback illustrated that the report was easy to interpret and a useful way to disseminate information. Participants noted the utility of the report, illustrating that the use of reports can be a useful method to disseminate research findings back to participants in a way that is applicable to the local context. Communicating research findings through reports can minimize the significant time lag in knowledge translation and provide participants with actionable steps to implement in their setting.

Study protocol for transforming health equity research in integrated primary care: Antiracism as a disruptive innovation.

Among the consequences of systemic racism in health care are significant health disparities among Black/African American individuals with comorbid physical and mental health conditions. Despite decades of studies acknowledging health disparities based on race, significant change has not occurred. There are shockingly few evidence-based antiracism interventions. New paradigms are needed to intervene on, and not just document, racism in health care systems. We are developing a transformative paradigm for new antiracism interventions for primary care settings that integrate mental and physical health care. The paradigm is the first of its kind to integrate community-based participatory research and systems science, within an established model of early phase translation to rigorously define new antiracism interventions. This protocol will use a novel application of systems sciences by combining the qualitative systems sciences methods (group model building; GMB) with quantitative methods (simulation modeling) to develop a comprehensive and community-engaged view of both the drivers of racism and the potential impact of antiracism interventions. Community participants from two integrated primary health care systems will engage in group GMB workshops with researchers to 1) Describe and map the complex dynamic systems driving racism in health care practices, 2) Identify leverage points for disruptive antiracism interventions, policies and practices, and 3) Review and prioritize a list of possible intervention strategies. Advisory committees will provide feedback on the design of GMB procedures, screen potential intervention components for impact, feasibility, and acceptability, and identify gaps for further exploration. Simulation models will be generated based on contextual factors and provider/patient characteristics. Using Item Response Theory, we will initiate the process of developing core measures for assessing the effectiveness of interventions at the organizational-systems and provider levels to be tested under a variety of conditions. While we focus on Black/African Americans, we hope that the resulting transformative paradigm can be applied to improve health equity among other marginalized groups.

Supporting Teams with Designing for Dissemination and Sustainability: the Design, Development, and Usability of a Digital Interactive Platform.

Background: The use of Designing for Dissemination and Sustainability (D4DS) principles and methods can support the development of research products (interventions, tools, findings) to match well with the needs and context of the intended audience and setting. D4DS principles and methods are not well-known or used during clinical and public health research; research teams would benefit from applying D4DS. This paper presents the development of a new digital platform for teams to learn and apply a D4DS process to their work. Methods: A user-centered design (UCD) approach engaged users (n=14) and an expert panel (n=6) in an iterative design process from discovery to prototyping and testing. We led five design sessions using Zoom and Figma software over a 5-month period. Users (71% academics; 29% practitioners) participated in at least 2 sessions. Following design sessions, feedback from users were summarized and discussed to generate design decisions. A prototype was then built and heuristically tested with 11 users who were asked to complete multiple tasks within the platform while verbalizing their decision-making using the 'think aloud' procedure. The System Usability Scale (SUS) was administered at the end of each testing session. After refinements to the platform were made, usability was reassessed with 7 of 11 same users to examine changes. Results: The interactive digital platform (the D4DS Planner) has two main components: 1) the Education Hub (e.g., searchable platform with literature, videos, websites) and 2) the Action Planner. The Action Planner includes 7 interactive steps that walk users through a set of activities to generate a downloadable D4DS action plan for their project. Participants reported that the prototype tool was moderately usable (SUS=66) but improved following refinements (SUS=71). Conclusions: This is a first of its kind tool that supports research teams in learning about and explicitly applying D4DS to their work. The use of this publicly available tool may increase the adoption, impact, and sustainment of a wide range of research products. The use of UCD yielded a tool that is easy to use. The future use and impact of this tool will be evaluated, and the tool will continue to be refined and improved.

Connecting Clinical Capacity and Intervention Sustainability in Resource-Variable Pediatric Oncology Centers in Latin America.

Clinical capacity for sustainability, or the clinical resources needed to sustain an evidence-based practice, represent proximal determinants that contribute to intervention sustainment. We examine the relationship between clinical capacity for sustainability and sustainment of PEWS, an evidence-based intervention to improve outcomes for pediatric oncology patients in resource-variable hospitals. We conducted a cross-sectional survey among Latin American pediatric oncology centers participating in Proyecto Escala de Valoración de Alerta Temprana (EVAT), an improvement collaborative to implement Pediatric Early Warning Systems (PEWS). Hospitals were eligible if they had completed PEWS implementation. Clinicians were eligible to participate if they were involved in PEWS implementation or used PEWS in clinical work. The Spanish language survey consisted of 56 close and open-ended questions about the respondent, hospital, participants' assessment of clinical capacity to sustain PEWS using the clinical sustainability assessment tool (CSAT), and perceptions about PEWS and its use as an intervention. Results were analyzed using a multi-level modeling approach to examine the relationship between individual, hospital, intervention, and clinical capacity determinants to PEWS sustainment. A total of 797 responses from 37 centers in 13 countries were included in the analysis. Eighty-seven percent of participants reported PEWS sustainment. After controlling for individual, hospital, and intervention factors, clinical capacity was significantly associated with PEWS sustainment (OR 3.27, p < .01). Marginal effects from the final model indicate that an increasing capacity score has a positive influence (11% for every additional CSAT point) of predicting PEWS sustainment. PEWS is a sustainable intervention and clinical capacity to sustain PEWS contributes meaningfully to PEWS sustainment.

Action planning for building public health program sustainability: results from a group-randomized trial.

BACKGROUND: Public health programs are charged with implementing evidence-based interventions to support public health improvement; however, to achieve long-term population-based benefits, these interventions must be sustained. Empirical evidence suggests that program sustainability can be improved through training and technical assistance, but few resources are available to support public health programs in building capacity for sustainability. METHODS: This study sought to build capacity for sustainability among state tobacco control programs through a multiyear, group-randomized trial that developed, tested, and evaluated a novel Program Sustainability Action Planning Model and Training Curricula. Using Kolb's experiential learning theory, we developed this action-oriented training model to address the program-related domains proven to impact capacity for sustainability as outlined in the Program Sustainability Framework. We evaluated the intervention using a longitudinal mixed-effects model using Program Sustainability Assessment (PSAT) scores from three time points. The main predictors in our model included group (control vs intervention) and type of dosage (active and passive). Covariates included state-level American Lung Association Score (proxy for tobacco control policy environment) and percent of CDC-recommended funding (proxy for program resources). RESULTS: Twenty-three of the 24 state tobacco control programs were included in the analyses: 11 received the training intervention and 12 were control. Results of the longitudinal mixed-effects linear regression model, where the annual PSAT score was the outcome, showed that states in the intervention condition reported significantly higher PSAT scores. The effects of CDC-recommended funding and American Lung Association smoke-free scores (proxy for policy environment) were small but statistically significant. CONCLUSION: This study found that the Program Sustainability Action Planning Model and Training Curricula was effective in building capacity for sustainability. The training was most beneficial for programs that had made less policy progress than others, implying that tailored training may be most appropriate for programs possibly struggling to make progress. Finally, while funding had a small, statistically significant effect on our model, it virtually made no difference for the average program in our study. This suggests that other factors may be more or equally important as the level of funding a program receives. CLINICALTRIALS: gov, NCT03598114. Registered on July 26, 2018.

Bridges and Mechanisms: Integrating Systems Science Thinking into Implementation Research.

We present a detailed argument for how to integrate, or bridge, systems science thinking and methods with implementation science. We start by showing how fundamental systems science principles of structure, dynamics, information, and utility are relevant for implementation science. Then we examine the need for implementation science to develop and apply richer theories of complex systems. This can be accomplished by emphasizing a causal mechanisms approach. Identifying causal mechanisms focuses on the "cogs and gears" of public health, clinical, and organizational interventions. A mechanisms approach focuses on how a specific strategy will produce the implementation outcome. We show how connecting systems science to implementation science opens new opportunities for examining and addressing social determinants of health and conducting equitable and ethical implementation research. Finally, we present case studies illustrating successful applications of systems science within implementation science in community health policy, tobacco control, health care access, and breast cancer screening.

Introducing SoNHR-Reporting guidelines for Social Networks In Health Research.

OBJECTIVE: The overall goal of this work is to produce a set of recommendations (SoNHR-Social Networks in Health Research) that will improve the reporting and dissemination of social network concepts, methods, data, and analytic results within health sciences research. METHODS: This study used a modified-Delphi approach for recommendation development consistent with best practices suggested by the EQUATOR health sciences reporting guidelines network. An initial set of 28 reporting recommendations was developed by the author team. A group of 67 (of 147 surveyed) experienced network and health scientists participated in an online feedback survey. They rated the clarity and importance of the individual recommendations, and provided qualitative feedback on the coverage, usability, and dissemination opportunities of the full set of recommendations. After examining the feedback, a final set of 18 recommendations was produced. RESULTS: The final SoNHR reporting guidelines are comprised of 18 recommendations organized within five domains: conceptualization (how study research questions are linked to network conceptions or theories), operationalization (how network science portions of the study are defined and operationalized), data collection & management (how network data are collected and managed), analyses & results (how network results are analyzed, visualized, and reported), and ethics & equity (how network-specific human subjects, equity, and social justice concerns are reported). We also present a set of exemplar published network studies which can be helpful for seeing how to apply the SoNHR recommendations in research papers. Finally, we discuss how different audiences can use these reporting guidelines. CONCLUSIONS: These are the first set of formal reporting recommendations of network methods in the health sciences. Consistent with EQUATOR goals, these network reporting recommendations may in time improve the quality, consistency, and replicability of network science across a wide variety of important health research areas.

Sustainability determinants of an intervention to identify clinical deterioration and improve childhood cancer survival in Latin American hospitals: the INSPIRE study protocol.

BACKGROUND: More than 90% of children with cancer live in low-resourced settings, where survival is only 20%. Sustainable evidence-based (EB) interventions yielding ongoing beneficial patient outcomes are critical to improve childhood cancer survival. A better understanding of factors promoting intervention sustainability in these settings is urgently needed. The aim of this study is to provide an empirical understanding of how clinical capacity for sustainability, or the resources needed to sustain an intervention, impacts the sustainment of Pediatric Early Warning System (PEWS), an EB intervention that improves pediatric oncology outcomes in low-resource hospitals by detecting clinical deterioration and preventing the need for more intense treatment. METHODS: We will conduct a prospective, longitudinal study of approximately 100 resource-variable hospitals implementing and sustaining PEWS participating in Proyecto EVAT, a quality improvement collaborative of Latin American pediatric oncology centers. Aim 1: We will evaluate how clinical capacity for sustainability changes over time through 5 to 9 prospective measurements of capacity via survey of clinical staff using PEWS (approximately n = 13 per center) during the phases of PEWS adoption, implementation, and sustainability using the Clinical Sustainability Assessment Tool (CSAT). Aim 2: We will determine the relationship between capacity and a) PEWS sustainment and b) clinical deterioration mortality among pediatric oncology patients at centers sustaining PEWS for 2 to 10 years using chart review and an existing patient outcomes registry. Aim 3: We will develop novel strategies to promote sustainability by gaining a deeper understanding of perceived challenges to building capacity and PEWS sustainment. In combination with quantitative outcomes, we will conduct 24 focus groups with staff (doctors, nurses, and administrators) from hospitals with both high (n = 4) and low capacity (n = 4). We will then use implementation mapping to generate theoretically driven, empirically-supported sustainability strategies. DISCUSSION: This study will advance implementation science by providing a theoretically driven, foundational understanding of factors that predict sustainability among a large, diverse cohort of hospitals. We will then use this knowledge to develop sustainability evidence-informed strategies that optimize capacity and promote long-term sustainment of PEWS and improvements in patient outcomes, thus promoting equity in childhood cancer care globally.

CritCom: assessment of quality of interdisciplinary communication around deterioration in pediatric oncologic patients.

Background: High-quality clinical care requires excellent interdisciplinary communication, especially during emergencies, and no tools exist to evaluate communication in critical care. We describe the development of a pragmatic tool focusing on interdisciplinary communication during patient deterioration (CritCom). Methods: The preliminary CritCom tool was developed after a literature review and consultation with a multidisciplinary panel of global experts in communication, pediatric oncology, and critical care to review the domains and establish content validity iteratively. Face and linguistic validity were established through cognitive interviews, translation, and linguistic synthesis. We conducted a pilot study among an international group of clinicians to establish reliability and usability. Results: After reviewing 105 potential survey items, we identified 52 items across seven domains. These were refined through cognitive interviews with 36 clinicians from 15 countries. CritCom was piloted with 433 clinicians (58% nurses, 36% physicians, and 6% other) from 42 hospitals in 22 countries. Psychometric testing guided the refinement of the items for the final tool. CritCom comprised six domains with five items each (30 total). The final tool has excellent reliability (Cronbach's alpha 0.81-0.86), usability (93% agree or strongly agree that the tool is easy to use), and similar performance between English and Spanish tools. Confirmatory factor analysis was used to establish the final 6-domain structure. Conclusions: CritCom is a reliable and pragmatic bilingual tool to assess the quality of interdisciplinary communication around patient deterioration for children in diverse resource levels globally. Critcom results can be used to design and evaluate interventions to improve team communication.

A new measure for multi-professional medical team communication: design and methodology for multilingual measurement development.

Background: As implementation science in global health continues to evolve, there is a need for valid and reliable measures that consider diverse linguistic and cultural contexts. A standardized, reproducible process for multilingual measure development may improve accessibility and validity by participants in global health settings. To address this need, we propose a rigorous methodology for multilingual measurement development. We use the example of a novel measure of multi-professional team communication quality, a determinant of implementation efforts. Methods: The development and translation of this novel bilingual measure is comprised of seven steps. In this paper, we describe a measure developed in English and Spanish, however, this approach is not language specific. Participants are engaged throughout the process: first, an interprofessional panel of experts and second, through cognitive interviewing for measure refinement. The steps of measure development included: (1) literature review to identify previous measures of team communication; (2) development of an initial measure by the expert panel; (3) cognitive interviewing in a phased approach with the first language (English); (4): formal, forward-backward translation process with attention to colloquialisms and regional differences in languages; (5) cognitive interviewing repeated in the second language (Spanish); (6) language synthesis to refine both instruments and unify feedback; and (7) final review of the refined measure by the expert panel. Results: A draft measure to assess quality of multi-professional team communication was developed in Spanish and English, consisting of 52 questions in 7 domains. This measure is now ready for psychometric testing. Conclusions: This seven-step, rigorous process of multilingual measure development can be used in a variety of linguistic and resource settings. This method ensures development of valid and reliable tools to collect data from a wide range of participants, including those who have historically been excluded due to language barriers. Use of this method will increase both rigor and accessibility of measurement in implementation science and advance equity in research and practice.

Action Planning for Building Program Sustainability: Results from a Group-Randomized Trial.

Background: Public health programs are charged with implementing evidence-based interventions to support public health improvement, however, to achieve long term population based benefit these interventions must be sustained. Empirical evidence suggests that program sustainability can be improved through training and technical assistance, but few resources are available to support public health programs in building capacity for sustainability. Methods: This study sought to build capacity for sustainability among state tobacco control programs through a multiyear, group-randomized trial that developed, tested, and evaluated a novel Program Sustainability Action Planning Model and Training Curricula. Using Kolb's experiential learning theory, we developed this action-oriented training model to address the program-related domains proven to impact capacity for sustainability as outlined in the Program Sustainability Framework. We evaluated the intervention using a longitudinal mixed-effects model using Program Sustainability Assessment (PSAT) scores from three time points. The main predictors in our model included group (control vs intervention) and type of dosage (active and passive). Covariates included state level American Lung Association score (proxy for tobacco control policy environment) and percent of CDC-recommended funding (proxy for program resources). Results: Twenty-three of the 24 state tobacco control programs were included in the analyses: 11 received the training intervention and 12 were control. Results of the longitudinal mixed-effects linear regression model, where the annual PSAT score was the outcome, showed that states in the intervention condition reported significantly higher PSAT scores. The effects for CDC-recommended funding and American Lung Association smoke-free scores (proxy for policy environment) were small but statistically significant. Conclusion: This study found that the Program Sustainability Action Planning Model and Training Curricula was effective in building capacity for sustainability. The training was most beneficial for programs that had made less policy progress than others, implying that tailored training may be most appropriate for programs possibly struggling to make progress. Finally, while funding had a small, statistically significant effect in our model, it virtually made no difference for the average program in our study. This suggests that other factors may be more or equally important as the level of funding a program receives. Trial registration: NCT03598114, Registered on July 26, 2018 (clnicaltrials.gov/NCT03598114).

Understanding Misimplementation in U.S. State Health Departments: An Agent-Based Model.

INTRODUCTION: The research goal of this study is to explore why misimplementation occurs in public health agencies and how it can be reduced. Misimplementation is ending effective activities prematurely or continuing ineffective ones, which contributes to wasted resources and suboptimal health outcomes. METHODS: The study team created an agent-based model that represents how information flow, filtered through organizational structure, capacity, culture, and leadership priorities, shapes continuation decisions. This agent-based model used survey data and interviews with state health department personnel across the U.S. between 2014 and 2020; model design and analyses were conducted with substantial input from stakeholders between 2019 and 2021. The model was used experimentally to identify potential approaches for reducing misimplementation. RESULTS: Simulations showed that increasing either organizational evidence-based decision-making capacity or information sharing could reduce misimplementation. Shifting leadership priorities to emphasize effectiveness resulted in the largest reduction, whereas organizational restructuring did not reduce misimplementation. CONCLUSIONS: The model identifies for the first time a specific set of factors and dynamic pathways most likely driving misimplementation and suggests a number of actionable strategies for reducing it. Priorities for training the public health workforce include evidence-based decision making and effective communication. Organizations will also benefit from an intentional shift in leadership decision-making processes. On the basis of this initial, successful application of agent-based model to misimplementation, this work provides a framework for further analyses.

Draining the tobacco swamps: Shaping the built environment to reduce tobacco retailer proximity to residents in 30 big US cities.

Combining geospatial data on residential and tobacco retailer density in 30 big US cities, we find that a large majority of urban residents live in tobacco swamps - neighborhoods where there is a glut of tobacco retailers. In this study, we simulate the effects of tobacco retail reduction policies and compare probable changes in resident-to-retailer proximity and retailer density for each city. While measures of proximity and density at baseline are highly correlated, the results differ both between effects on proximity and density and across the 30 cities. Context, particularly baseline proximity of residents to retailers, is important to consider when designing policies to reduce retailer concentration.

Tailoring dissemination of evidence to preferences of tobacco control partners: results from an academic-community partnership.

BACKGROUND: Tobacco control program leaders and their partners, who often present evidence to policymakers, can increase the use of evidence in program and policy development. However, up-to-date evidence from the scientific community about what works is slow to reach leaders. We describe efforts to understand and utilize tobacco control leaders' preferences for receiving evidence and report on resulting dissemination strategies, translational products, and outcomes. METHODS: This work is part of the Advancing Science and Practice in the Retail Environment (ASPiRE) Center, an interdisciplinary research center focused on understanding and evaluating tobacco retail policy. Participants were members of the ASPiRE Community Advisory Board (CAB), comprised of tobacco control leaders from 30 metropolitan areas representing all regions of the US plus nine representatives from leading national tobacco control organizations (N = 39). During meetings in February 2019 and October 2020, all CAB members were invited to participate in live polls consisting of six survey questions each. Questions addressed preferences for receiving scientific evidence and their anticipated use of ASPiRE translational products. Responses were analyzed descriptively and informed translational product development and communications with ASPiRE contact list members (N = 125). ASPiRE email and website interactions were tracked from March 2019 to May 2021 as a complementary indication of content use. RESULTS: Response rates for 2019 and 2020 CAB meetings were 66% (n = 26) and 59% (n = 23), respectively. CAB members indicated preferences for email communication (33%) and webinars (31%), communications once per month (46%), and short-format documents (28%). In response, the team developed translational short-format products including case studies, fact sheets, and research briefs. On average, 52% (SD = 14%) of recipients opened the newsletter and 17% (SD = 9%) clicked a link within the newsletter. Overall, 95% of responding CAB members found the products useful and all responding CAB members reported using them to communicate evidence to policymakers, staff, and coalition members. CONCLUSIONS: Our successful dissemination approach to making evidence more accessible and useable for tobacco control leaders could be adapted by researchers working with community partners to assess and respond to stakeholders' preferences for receiving evidence in other areas of health policy.

Simulating the role of knowledge brokers in policy making in state agencies: An agent-based model.

OBJECTIVE: To model children's mental health policy making dynamics and simulate the impacts of knowledge broker interventions. DATA SOURCES: Primary data from surveys (n = 221) and interviews (n = 64) conducted in 2019-2021 with mental health agency (MHA) officials in state agencies. STUDY DESIGN: A prototype agent-based model (ABM) was developed using the PARTE (Properties, Actions, Rules, Time, Environment) framework and informed through primary data collection. In each simulation, a policy is randomly generated (salience weights: cost, contextual alignment, and strength of evidence) and discussed among agents. Agents are MHA officials and heterogenous in their properties (policy making power and network influence) and policy preferences (based on salience weights). Knowledge broker interventions add agents to the MHA social network who primarily focus on the policy's research evidence. DATA COLLECTION/EXTRACTION METHODS: A sequential explanatory mixed method approach was used. Descriptive and regression analyses were used for the survey data and directed content analysis was used to code interview data. Triangulated results informed ABM development. In the ABM, policy makers with various degrees of decision influence interact in a scale-free network before and after knowledge broker interventions. Over time, each decides to support or oppose a policy proposal based on policy salience weights and their own properties and interactions. The main outcome is an agency-level decision based on policy maker support. Each intervention and baseline simulation runs 250 times across 50 timesteps. PRINCIPAL FINDINGS: Surveys and interviews revealed that barriers to research use could be addressed by knowledge brokers. Simulations indicated that policy decision outcomes varied by policy making context within agencies. CONCLUSIONS: This is the first application of ABM to evidence-informed mental health policy making. Results suggest that the presence of knowledge brokers can: (1) influence consensus formation in MHAs, (2) accelerate policy decisions, and (3) increase the likelihood of evidence-informed policy adoption.

Designing for Dissemination and Sustainability to Promote Equitable Impacts on Health.

Designing for dissemination and sustainability (D4DS) refers to principles and methods for enhancing the fit between a health program, policy, or practice and the context in which it is intended to be adopted. In this article we first summarize the historical context of D4DS and justify the need to shift traditional health research and dissemination practices. We present a diverse literature according to a D4DS organizing schema and describe a variety of dissemination products, design processes and outcomes, and approaches to messaging, packaging, and distribution. D4DS design processes include stakeholder engagement, participatory codesign, and context and situation analysis, and leverage methods and frameworks from dissemination and implementation science, marketing and business, communications and visualarts, and systems science. Finally, we present eight recommendations to adopt a D4DS paradigm, reflecting shifts in ways of thinking, skills and approaches, and infrastructure and systems for training and evaluation.

Coordination of sustainable financing for evidence-based youth mental health treatments: protocol for development and evaluation of the fiscal mapping process.

BACKGROUND: Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health and economic impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies (hereafter, "service agencies"). Strategic planning tools are needed that can guide these service agencies in their coordination of sustainable funding for EBTs. This protocol describes a mixed-methods research project designed to (1) develop and (2) evaluate our novel fiscal mapping process that guides strategic planning efforts to finance the sustainment of EBTs in youth mental health services. METHOD: Participants will be 48 expert stakeholder participants, including representatives from ten service agencies and their partners from funding agencies (various public and private sources) and intermediary organizations (which provide guidance and support on the delivery of specific EBTs). Aim 1 is to develop the fiscal mapping process: a multi-step, structured tool that guides service agencies in selecting the optimal combination of strategies for financing their EBT sustainment efforts. We will adapt the fiscal mapping process from an established intervention mapping process and will incorporate an existing compilation of 23 financing strategies. We will then engage participants in a modified Delphi exercise to achieve consensus on the fiscal mapping process steps and gather information that can inform the selection of strategies. Aim 2 is to evaluate preliminary impacts of the fiscal mapping process on service agencies' EBT sustainment capacities (i.e., structures and processes that support sustainment) and outcomes (e.g., intentions to sustain). The ten agencies will pilot test the fiscal mapping process. We will evaluate how the fiscal mapping process impacts EBT sustainment capacities and outcomes using a comparative case study approach, incorporating data from focus groups and document review. After pilot testing, the stakeholder participants will conceptualize the process and outcomes of fiscal mapping in a participatory modeling exercise to help inform future use and evaluation of the tool. DISCUSSION: This project will generate the fiscal mapping process, which will facilitate the coordination of an array of financing strategies to sustain EBTs in community youth mental health services. This tool will promote the sustainment of youth-focused EBTs.

Assessing the sustainability capacity of evidence-based programs in community and health settings.

Background: Within many public health settings, there remain large challenges to sustaining evidence-based practices. The Program Sustainability Assessment Tool has been developed and validated to measure sustainability capacity of public health, social service, and educational programs. This paper describes how this tool was utilized between January 2014 and January 2019. We describe characteristics of programs that are associated with increased capacity for sustainability and ultimately describe the utility of the PSAT in sustainability research and practice. Methods: The PSAT is comprised of 8 subscales, measuring sustainability capacity in eight distinct conceptual domains. Each subscale is made up of five items, all assessed on a 7-point Likert scale. Data were obtained from persons who used the PSAT on the online website (https://sustaintool.org/), from 2014 to 2019. In addition to the PSAT scale, participants were asked about four program-level characteristics. The resulting dataset includes 5,706 individual assessments reporting on 2,892 programs. Results: The mean overall PSAT score was 4.73, with the lowest and highest scoring subscales being funding stability and program adaptation, respectively. Internal consistency for each subscale was excellent (average Cronbach's alpha = 0.90, ranging from 0.85 to 0.94). Confirmatory factor analysis highlighted good to excellent fit of the PSAT measurement model (eight distinct conceptual domains) to the observed data, with a comparative fit index of 0.902, root mean square error of approximation equal to 0.054, and standardized root mean square residual of 0.054. Overall sustainability capacity was significantly related to program size (F = 25.6; p < 0.001). Specifically, smaller programs (with staff sizes of ten or below) consistently reported lower program sustainability capacity. Capacity was not associated with program age and did not vary significantly by program level. Discussion: The PSAT maintained its excellent reliability when tested with a large and diverse sample over time. Initial criterion validity was explored through the assessment of program characteristics, including program type and program size. The data collected reinforces the ability of the PSAT to assess sustainability capacity for a wide variety of public health and social programs.

Sustainability in pediatric hospitals: An exploration at the intersection of quality improvement and implementation science.

Background: Although new evidence-based practices are frequently implemented in clinical settings, many are not sustained, limiting the intended impact. Within implementation science, there is a gap in understanding sustainability. Pediatric healthcare settings have a robust history of quality improvement (QI), which includes a focus on continuation of change efforts. QI capability and sustainability capacity, therefore, serve as a useful concept for connecting the broader fields of QI and implementation science to provide insights on improving care. This study addresses these gaps in understanding of sustainability in pediatric settings and its relationship to QI. Methods: This is a cross-sectional observational study conducted within pediatric academic medical centers in the United States. Clinicians surveyed worked with one of three evidence-based clinical programs: perioperative antimicrobial stewardship prescribing, early mobility in the intensive care unit, and massive blood transfusion administration. Participants completed two assessments: (1) the Clinical Sustainability Assessment Tool (CSAT) and (2) a 19-question assessment that included demographics and validation questions, specifically a subset of questions from the Change Process Capability Questionnaire, a QI scale. Initial descriptive and bivariate analyses were conducted prior to building mixed-effects models relating perceived QI to clinical sustainability capacity. Results: A total of 181 individuals from three different programs and 30 sites were included in the final analyses. QI capability scores were assessed as a single construct (5-point Likert scale), with an average response of 4.16 (higher scores indicate greater QI capability). The overall CSAT score (7-point Likert scale) was the highest for massive transfusion programs (5.51, SD = 0.91), followed by early mobility (5.25, SD = 0.92) and perioperative antibiotic prescribing (4.91, SD = 1.07). Mixed-effects modeling illustrated that after controlling for person and setting level variables, higher perceptions of QI capabilities were significantly related to overall clinical sustainability. Conclusion: Organizations and programs with higher QI capabilities had a higher sustainability capacity, even when controlling for differences at the individual and intervention levels. Organizational factors that enable evidence-based interventions should be further studied, especially as they relate to sustainability. Issues to be considered by practitioners when planning for sustainability include bedside provider perceptions, intervention achievability, frequency of delivery, and organizational influences.

Point of care tobacco treatment sustains during COVID-19, a global pandemic.

BACKGROUND: Tobacco cessation treatment for cancer patients is essential to providing comprehensive oncologic care. We have implemented a point of care tobacco treatment care model enabled by electronic health record (EHR) modifications in a comprehensive cancer center. Data are needed on the sustainability of both reach of treatment and effectiveness over time, including the COVID-19 pandemic. METHODS: Using EHR data from the pre-implementation (P: 5 months) and post-implementation periods (6 month-blocks, T1-T5 for a total of 30 months), we compared two primary outcomes: 1) reach of treatment among those smoking and 2) effectiveness assessed by smoking cessation among those smoking in the subsequent 6 month period. We analyzed the data using generalized estimation equation regression models. RESULTS: With the point of care tobacco treatment care model, reach of treatment increased from pre to post T5 (3.2 % vs. 48.4 %, RR 15.50, 95 % CI 10.56-22.74, p < 0.0001). Reach of treatment in all post periods (T1-T5 including the COVID-19 pandemic time) remained significantly higher than the pre period. Effectiveness, defined by smoking cessation among those smoking, increased from pre to post T2 before the pandemic (12.4 % vs. 21.4 %, RR 1.57, 95 % CI 1.31-1.87, p < 0.0001). However, effectiveness, while higher in later post periods (T3, T4), was no longer significantly increased compared with the pre period. CONCLUSION: A point of care EHR-enabled tobacco treatment care model demonstrates sustained reach up to 30 months following implementation, even during the COVID-19 pandemic and changes in healthcare prioritization. Effectiveness was sustained for 12 months, but did not sustain through the subsequent 12 months.