A structured approach to modifying an implementation package while scaling up a complex evidence-based practice.

OBJECTIVE: To describe a structured, iterative, data-driven approach for modifying implementation strategies for a complex evidence-based practice during a nationwide scale-up initiative. DATA SOURCES AND STUDY SETTING: We scaled-up implementation of Critical Time Intervention (CTI)-an evidence-based case management model-across 32 diverse community-based Veterans Affairs (VA) "Grant and Per Diem" case management (GPD-CM) agencies that serve homeless-experienced Veterans transitioning to independent living. Primary data were collected using qualitative methods. STUDY DESIGN: We embarked on a scale-up initiative while conducting a pragmatic randomized evaluation using a roll-out design, comparing two versions of a CTI implementation package tailored to VA's GPD-CM program. We iteratively assessed contextual factors and implementation outcomes (e.g., acceptability); findings informed package modifications that were characterized using the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies. DATA COLLECTION METHODS: We conducted semi-structured interviews with Veterans, GPD-CM staff, and liaising VA clinicians; periodic reflections with liaising VA clinicians and implementation team members; and drew upon detailed meeting notes. We used rapid qualitative methods and content analysis to integrate data and characterize modifications. PRINCIPAL FINDINGS: After each scale-up wave-in response to variations in agency-level characteristics- we made iterative modifications to the implementation package to increase CTI adoption and fidelity across the diverse contexts of our scale-up sites. Modifications included adding, deleting, integrating, and altering the package; core package components were preserved. CONCLUSIONS: Implementation packages for complex evidence-based practices undergoing scale-up in diverse contexts may benefit from iterative modifications to optimize practice adoption with fidelity. We offer a structured, pragmatic approach for iteratively identifying data-driven, midstream implementation package adjustments, for use in both VA and non-VA scale-up initiatives. Our project demonstrates the importance of assessing for and making modifications in a scale-up initiative, as well as the trade-offs of projects having simultaneous formative and summative evaluation aims.

Care coordination across healthcare systems: development of a research agenda, implications for practice, and recommendations for policy based on a modified Delphi panel.

OBJECTIVE: For large, integrated healthcare delivery systems, coordinating patient care across delivery systems with providers external to the system presents challenges. We explored the domains and requirements for care coordination by professionals across healthcare systems and developed an agenda for research, practice and policy. DESIGN: The modified Delphi approach convened a 2-day stakeholder panel with moderated virtual discussions, preceded and followed by online surveys. SETTING: The work addresses care coordination across healthcare systems. We introduced common care scenarios and differentiated recommendations for a large (main) healthcare organisation and external healthcare professionals that contribute additional care. PARTICIPANTS: The panel composition included health service providers, decision makers, patients and care community, and researchers. Discussions were informed by a rapid review of tested approaches to fostering collaboration, facilitating care coordination and improving communication across healthcare systems. OUTCOME MEASURES: The study planned to formulate a research agenda, implications for practice and recommendations for policy. RESULTS: For research recommendations, we found consensus for developing measures of shared care, exploring healthcare professionals' needs in different care scenarios and evaluating patient experiences. Agreed practice recommendations included educating external professionals about issues specific to the patients in the main healthcare system, educating professionals within the main healthcare system about the roles and responsibilities of all involved parties, and helping patients better understand the pros and cons of within-system and out-of-system care. Policy recommendations included supporting time for professionals with high overlap in patients to engage regularly and sustaining support for care coordination for high-need patients. CONCLUSIONS: Recommendations from the stakeholder panel created an agenda to foster further research, practice and policy innovations in cross-system care coordination.

Comparing two implementation strategies for implementing and sustaining a case management practice serving homeless-experienced veterans: a protocol for a type 3 hybrid cluster-randomized trial.

BACKGROUND: The Veterans Health Administration (VA) Grant and Per Diem case management "aftercare" program provides 6 months of case management for homeless-experienced veterans (HEVs) undergoing housing transitions. To standardize and improve aftercare services, we will implement critical time intervention (CTI), an evidence-based, structured, and time-limited case management practice. We will use two strategies to support the implementation and sustainment of CTI at 32 aftercare sites, conduct a mixed-methods evaluation of this implementation initiative, and generate a business case analysis and implementation playbook to support the continued spread and sustainment of CTI in aftercare. METHODS: We will use the Replicating Effective Programs (REP) implementation strategy to support CTI implementation at 32 sites selected by our partners. Half (n=16) of these sites will also receive 9 months of external facilitation (EF, enhanced REP). We will conduct a type 3 hybrid cluster-randomized trial to compare the impacts of REP versus enhanced REP. We will cluster potential sites into three implementation cohorts staggered in 9-month intervals. Within each cohort, we will use permuted block randomization to balance key site characteristics among sites receiving REP versus enhanced REP; sites will not be blinded to their assigned strategy. We will use mixed methods to assess the impacts of the implementation strategies. As fidelity to CTI influences its effectiveness, fidelity to CTI is our primary outcome, followed by sustainment, quality metrics, and costs. We hypothesize that enhanced REP will have higher costs than REP alone, but will result in stronger CTI fidelity, sustainment, and quality metrics, leading to a business case for enhanced REP. This work will lead to products that will support our partners in spreading and sustaining CTI in aftercare. DISCUSSION: Implementing CTI within aftercare holds the potential to enhance HEVs' housing and health outcomes. Understanding effective strategies to support CTI implementation could assist with a larger CTI roll-out within aftercare and support the implementation of other case management practices within and outside VA. TRIAL REGISTRATION: This project was registered with ClinicalTrials.gov as "Implementing and sustaining Critical Time Intervention in case management programs for homeless-experienced Veterans." Trial registration NCT05312229 , registered April 4, 2022.

Engaging stakeholders to inform national implementation of critical time intervention in a program serving homeless-experienced Veterans.

The Veterans Affairs (VA) Grant and Per Diem Case Management "Aftercare" program provides 6 months of case management for homeless-experienced Veterans (HEVs) transitioning to permanent housing, with the aim of decreasing returns to homelessness. Implementing Critical Time Intervention (CTI)-an evidence-based case management practice-would standardize care across the 128 community-based agencies that provide Aftercare services. To prepare for national CTI implementation in Aftercare, guided by Replicating Effective Programs (REP), we conducted a four-site pilot in which we adapted a CTI implementation package (training, technical assistance, and external facilitation); characterized stakeholder perspectives regarding the acceptability and appropriateness of this package; and identified contextual factors that affected CTI implementation. We engaged a stakeholder workgroup to tailor existing CTI training and technical assistance materials for Aftercare. To provide tailored support for providers and leaders to adopt and incorporate evidence-based practices (EBPs) into routine care, we also developed external facilitation materials and processes. Over 9 months, we implemented this package at four sites. We conducted semi-structured interviews at pre-implementation, mid-implementation, and 6 months post-implementation, with HEVs (n = 37), case managers (n = 16), supervisors (n = 10), and VA leaders (n = 4); these data were integrated with templated reflection notes from the project facilitator. We used rapid qualitative analysis and targeted coding to assess the acceptability and appropriateness of CTI and our implementation package and identify factors influencing CTI implementation. Stakeholders generally found CTI acceptable and appropriate; there was consensus that components of CTI were useful and compatible for this setting. To adapt our implementation package for scale-up, this pilot highlighted the value of robust and tangible CTI training and technical assistance-grounded in real-world cases-that highlights the congruence of CTI with relevant performance metrics. Variations in agency-level contextual factors may necessitate more intense and tailored supports to implement and sustain complex EBPs like CTI. Processes used in this pilot are relevant for implementing other EBPs in organizations that serve vulnerable populations. EBP scale-up and sustainment can be enhanced by engaging stakeholders to tailor EBPs for specific contexts; pilot testing and refining implementation packages for scale-up; and using qualitative methods to characterize contextual factors that affect EBP implementation.

The Coordination Toolkit and Coaching Project: Cluster-Randomized Quality Improvement Initiative to Improve Patient Experience of Care Coordination.

BACKGROUND: Given persistent gaps in coordination of care for medically complex primary care patients, efficient strategies are needed to promote better care coordination. OBJECTIVE: The Coordination Toolkit and Coaching project compared two toolkit-based strategies of differing intensity to improve care coordination at VA primary care clinics. DESIGN: Multi-site, cluster-randomized QI initiative. PARTICIPANTS: Twelve VA primary care clinics matched in 6 pairs. INTERVENTIONS: We used a computer-generated allocation sequence to randomize clinics within each pair to two implementation strategies. Active control clinics received an online toolkit with evidence-based tools and QI coaching manual. Intervention clinics received the online toolkit plus weekly assistance from a distance coach for 12 months. MAIN MEASURES: We quantified patient experience of general care coordination using the Health Care System Hassles Scale (primary outcome) mailed at baseline and 12-month follow-up to serial cross-sectional patient samples. We measured the difference-in-difference (DiD) in clinic-level-predicted mean counts of hassles between coached and non-coached clinics, adjusting for clustering and patient characteristics using zero-inflated negative binomial regression and bootstrapping to obtain 95% confidence intervals. Other measures included care coordination QI projects attempted, tools adopted, and patient-reported exposure to projects. KEY RESULTS: N = 2,484 (49%) patients completed baseline surveys and 2,481 (48%) completed follow-ups. Six coached clinics versus five non-coached clinics attempted QI projects. All coached clinics versus two non-coached clinics attempted more than one project or projects that were multifaceted (i.e., involving multiple components addressing a common goal). Five coached versus three non-coached clinics used 1-2 toolkit tools. Both the coached and non-coached clinics experienced pre-post reductions in hassle counts over the study period (- 0.42 (- 0.76, - 0.08) non-coached; - 0.40 (- 0.75, - 0.06) coached). However, the DiD (0.02 (- 0.47, 0.50)) was not statistically significant; coaching did not improve patient experience of care coordination relative to the toolkit alone. CONCLUSION: Although coached clinics attempted more or more complex QI projects and used more tools than non-coached clinics, coaching provided no additional benefit versus the online toolkit alone in patient-reported outcomes. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT03063294.

Reflective writing: a tool to support continuous learning and improved effectiveness in implementation facilitators.

BACKGROUND: Implementation facilitators support the adoption of evidence-based practices and other improvement efforts in complex healthcare settings. Facilitators are trained to develop essential facilitation skills and facilitator effectiveness is typically evaluated post-implementation, but little is known about how facilitators apply and adapt didactic knowledge after training, or how learning and refining experiential knowledge occurs during the facilitation process. We propose the use of reflective writing as a tool to document and support facilitator learning and facilitator effectiveness. METHODS: Using an instrumental case study of the Coordination Toolkit and Coaching (CTAC) project, we explore the use of reflective writing by facilitators to support their learning and effectiveness. Six primary care clinics participated in weekly hour-long facilitation calls over a 12-month period to implement quality improvement projects related to care coordination. Two facilitators completed templated reflections after each facilitation call for their assigned sites, totaling 269 reflections. We used the declarative-procedural-reflective model, which defines the process of skill development in clinical practice, to qualitatively analyze the reflections. Two independent coders used content analysis principles to code text that captured facilitators' observations, evaluations, interpretations, and communication. Descriptive statistics were used to analyze reflections by facilitator and by code within and across reflections. RESULTS: CTAC facilitators primarily used the reflections to summarize the calls (observation), assess the facilitation process and the tasks and activities they used (evaluation), document their thoughts about how to improve their own effectiveness (interpretation), and describe their communication with implementing teams. Ninety-one percent of reflections included observations, 42% interpretation, 41% evaluation, and 44% facilitator communication. In total, we coded 677 segments of text within reflections: 39% represented observation, 20% interpretation, 18% evaluation, and 23% facilitator communication. CONCLUSIONS: The process of reflective writing allowed the CTAC facilitators the time and structure to evaluate their facilitation and to think critically about how to adjust their facilitation in response to their observations and interpretations. Reflective writing is a feasible and acceptable tool to support and document facilitator learning and effectiveness. TRIAL REGISTRATION: The project was registered with ClinicalTrials.gov ( NCT03063294 ) on February 24, 2017.

Toolkit and distance coaching strategies: a mixed methods evaluation of a trial to implement care coordination quality improvement projects in primary care.

BACKGROUND: Care coordination tools and toolkits can be challenging to implement. Practice facilitation, an active but expensive strategy, may facilitate toolkit implementation. We evaluated the comparative effectiveness of distance coaching, a form of practice facilitation, for improving the implementation of care coordination quality improvement (QI) projects. METHODS: We conducted a mixed methods evaluation of the Coordination Toolkit and Coaching (CTAC) initiative. Twelve matched US Veterans Health Administration primary care clinics were randomized to receive coaching and an online care coordination toolkit ("coached"; n = 6) or access to the toolkit only ("non-coached"; n = 6). We did interviews at six, 12, and 18 months. For coached sites, we'ly collected site visit fieldnotes, prospective coach logs, retrospective coach team debriefs, and project reports. We employed matrix analysis using constructs from the Consolidated Framework for Implementation Research and a taxonomy of outcomes. We assessed each site's project(s) using an adapted Complexity Assessment Tool for Systematic Reviews. RESULTS: Eleven sites implemented a local CTAC project. Eight sites (5 coached, 3 non-coached) used at least one tool from the toolkit. Coached sites implemented significantly more complex projects than non-coached sites (11.5 vs 7.5, 95% confidence interval 1.75-6.25, p < 0.001); engaged in more formal implementation processes (planning, engaging, reflecting and evaluating); and generally had larger, more multidisciplinary QI teams. Regardless of coaching status, sites focused on internal organizational improvement and low-intensity educational projects rather than the full suite of care coordination tools. At 12 months, half the coached and non-coached sites had clinic-wide project implementation; the remaining coached sites had implemented most of their project(s), while the remaining non-coached sites had either not implemented anything or conducted limited pilots. At 18 months, coached sites reported ongoing effort to monitor, adapt, and spread their CTAC projects, while non-coached sites did not report much continuing work. Coached sites accrued benefits like improved clinic relationships and team QI skill building that non-coached sites did not describe. CONCLUSIONS: Coaching had a positive influence on QI skills of (and relationships among) coached sites' team members, and the scope and rigor of projects. However, a 12-month project period was potentially too short to ensure full project implementation or to address cross-setting or patient-partnered initiatives. TRIAL REGISTRATION: NCT03063294 .

Sustaining implementation facilitation: a model for facilitator resilience.

BACKGROUND: Implementation facilitators enable healthcare staff to effectively implement change, yet little is known about their affective (e.g., emotional, mental, physical) experiences of facilitation. We propose an expansion to the Integrated Promoting Action on Research in Health Services (i-PARIHS) framework that introduces facilitation intensity and facilitator resilience to better assess facilitators' affective experiences. METHODS: We used an instrumental case study and facilitator data (logged reflections and debrief session notes) from the Coordination Toolkit and Coaching initiative to conceptualize facilitation intensity and facilitator resilience and to better understand the psychological impact of the facilitation process on facilitator effectiveness and implementation success. RESULTS: We define facilitation intensity as both the quantitative and/or qualitative measure of the volume of tasks and activities needed to engage and motivate recipients in implementation, and the psychological impact on the facilitator of conducting facilitation tasks and activities. We define facilitator resilience as the ability to cope with and adapt to the complexities of facilitation in order to effectively engage and motivate staff, while nurturing and sustaining hope, self-efficacy, and adaptive coping behaviors in oneself. CONCLUSIONS: Facilitators' affective experience may help to identify potential relationships between the facilitation factors we propose (facilitation intensity and facilitator resilience). Future studies should test ways of reliably measuring facilitation intensity and facilitator resilience and specify their relationships in greater detail. By supporting facilitator resilience, healthcare delivery systems may help sustain the skilled facilitator workforce necessary for continued practice improvement. TRIAL REGISTRATION: The project was registered with ClinicalTrials.gov ( NCT03063294 ) on February 24, 2017.

Patient experience of health care system hassles: Dual-system vs single-system users.

OBJECTIVE: To compare health care system problems or "hassles" experienced by Veterans receiving VA health care only versus those receiving dual care from both VA and non-VA community providers. DATA SOURCES: We collected survey data in 2017-2018 from 2444 randomly selected Veterans with four or more primary care visits in the prior year at one of 12 VA primary care clinics located in four geographically diverse regions of the United States. STUDY DESIGN: We used baseline surveys from the Coordination Toolkit and Coaching quality improvement project to explore Veterans' experience of hassles (dependent variable), source of health care, self-rated physical and mental health, and sociodemographics. DATA COLLECTION: Participants responded to mailed surveys by mail, telephone, or online. PRINCIPAL FINDINGS: The number of reported hassles ranged from 0 to 16; 79 percent of Veterans reported experiencing one or more hassles. Controlling for sociodemographic characteristics and self-rated physical and mental health, zero-inflated negative binominal regression indicated that dual care users experienced more hassles than VA-only users (adjusted predicted average 5.5 [CI: 5.2, 5.8] vs 4.3 [CI: 4.1, 4.6] hassles [P < .0001]). CONCLUSIONS: Anticipated increases in Veterans accessing community-based care may require new strategies to help VA primary care teams optimize care coordination for dual care users.

Staff Perspectives on Primary Care Teams as De Facto "Hubs" for Care Coordination in VA: a Qualitative Study.

BACKGROUND: Improving care coordination is a key priority for many healthcare systems. However, initiatives to improve care coordination are complex to implement and have produced mixed results. A better understanding of how to craft and support implementation of effective care coordination strategies is needed. OBJECTIVE: To identify and understand the challenges and factors encountered by Patient-Aligned Care Team (PACT) staff in performing care coordination tasks in outpatient clinics in the Veterans Health Administration (VA). DESIGN: Qualitative study using semi-structured formative evaluation interviews. PARTICIPANTS: Fourteen interviews with 18 clinical frontline managers and staff from 12 clinic sites across five VA health systems. INTERVENTIONS: This paper reports on baseline data collected for the Coordination Toolkit and Coaching (CTAC) project. CTAC aims to improve patients' experience of care coordination within VA primary care and between PACT and other outpatient and community settings. APPROACH: We conducted pre-implementation telephone interviews with frontline managers and staff, primarily nurse managers. KEY RESULTS: PACT staff described challenges in aligning care coordination priorities across different levels of the VA system, including staff, patients, and leadership. Additionally, PACT staff noted challenges coordinating care both within and outside the VA, and identified resource barriers impeding their care coordination efforts. To address these challenges, staff made several recommendations for improvement, including (1) contingency staffing to address staff burnout; (2) additional PACT training for new staff; (3) clarification of care coordination roles and responsibilities; and (4) and care coordination initiatives that align both with centrally initiated care coordination programs and frontline needs. CONCLUSION: In the VA and similarly complex healthcare systems, our findings suggest the need for care coordination strategies that are buttressed by a system-level vision for care coordination, backed up by clear roles and responsibilities for information exchange between primary care staff and other settings, and multidimensional accountability metrics that encompass patient-, staff-, and system-level goals.

Development of a web-based toolkit to support improvement of care coordination in primary care.

Promising practices for the coordination of chronic care exist, but how to select and share these practices to support quality improvement within a healthcare system is uncertain. This study describes an approach for selecting high-quality tools for an online care coordination toolkit to be used in Veterans Health Administration (VA) primary care practices. We evaluated tools in three steps: (1) an initial screening to identify tools relevant to care coordination in VA primary care, (2) a two-clinician expert review process assessing tool characteristics (e.g. frequency of problem addressed, linkage to patients' experience of care, effect on practice workflow, and sustainability with existing resources) and assigning each tool a summary rating, and (3) semi-structured interviews with VA patients and frontline clinicians and staff. Of 300 potentially relevant tools identified by searching online resources, 65, 38, and 18 remained after steps one, two and three, respectively. The 18 tools cover five topics: managing referrals to specialty care, medication management, patient after-visit summary, patient activation materials, agenda setting, patient pre-visit packet, and provider contact information for patients. The final toolkit provides access to the 18 tools, as well as detailed information about tools' expected benefits, and resources required for tool implementation. Future care coordination efforts can benefit from systematically reviewing available tools to identify those that are high quality and relevant.