Implementation mapping to plan for a hybrid trial testing the effectiveness and implementation of a behavioral intervention for HIV medication adherence and care retention.

Background: Implementation mapping is a systematic, collaborative, and contextually-attentive method for developing implementation strategies. As an exemplar, we applied this method to strategy development for Managed Problem Solving Plus (MAPS+), an adapted evidence-based intervention for HIV medication adherence and care retention that will be delivered by community health workers and tested in an upcoming trial. Methods: In Step 1: Conduct Needs Assessment, we interviewed 31 stakeholders to identify determinants of MAPS+ implementation in 13 clinics serving people with HIV in Philadelphia County. In Step 2: Develop Logic Model, we used these determinants as inputs for a working logic model guided by the Consolidated Framework for Implementation Research. In Step 3: Operationalize Implementation Strategies, our team held a virtual stakeholder meeting to confirm determinants. We synthesized stakeholder feedback, then identified implementation strategies that conceptually matched to determinants using the Expert Recommendations for Implementing Change taxonomy. Next, we operationalized implementation strategies with specific examples for clinic settings. We linked strategies to behavior change theories to allow for a mechanistic understanding. We then held a second virtual stakeholder meeting to present the implementation menu for feedback and glean generalizable insights for how these strategies could be operationalized in each stakeholder's clinic. In Step 4: Protocolize Strategies, we incorporated stakeholder feedback and finalized the implementation strategy menu. Findings: Implementation mapping produced a menu of 39 strategies including revise professional roles, identify and prepare champions, use warm handoffs, and change record systems. The process of implementation mapping generated key challenges for implementation strategy development: lack of implementation strategies targeting the outer setting (i.e., sociopolitical context); tension between a one-size-fits-all and individualized approach for all clinics; conceptual confusion between facilitators and strategies; and challenges in translating the implementation science lexicon for partners. Implications: This case exemplar advances both MAPS+ implementation and implementation science methods by furthering our understanding of the use of implementation mapping to develop strategies that enhance uptake of evidence-based interventions. The implementation menu will inform MAPS+ deployment across Philadelphia in an upcoming hybrid trial. We will carry out Step 5: Test Strategies to test the effectiveness and implementation of MAPS+.

Comparing output from two methods of participatory design for developing implementation strategies: traditional contextual inquiry vs. rapid crowd sourcing.

BACKGROUND: Participatory design methods are a key component of designing tailored implementation strategies. These methods vary in the resources required to execute and analyze their outputs. No work to date has examined the extent to which the output obtained from different approaches to participatory design varies. METHODS: We concurrently used two separate participatory design methods: (1) field observations and qualitative interviews (i.e., traditional contextual inquiry) and (2) rapid crowd sourcing (an innovation tournament). Our goal was to generate and compare information to tailor implementation strategies to increase the use of evidence-based data collection practices among one-to-one aides working with children with autism. Each method was executed and analyzed by study team members blinded to the output of the other method. We estimated the personnel time and monetary costs associated with each method to further facilitate comparison. RESULTS: Observations and interviews generated nearly double the number of implementation strategies (n = 26) than did the innovation tournament (n = 14). When strategies were classified into implementation strategies from the Expert Recommendations for Implementing Change (ERIC) taxonomy, there was considerable overlap in the content of identified strategies. However, strategies derived from observations and interviews were more specific than those from the innovation tournament. Five strategies (13%) reflected content unique to observations and interviews and 3 (8%) strategies were unique to the innovation tournament. Only observations and interviews identified implementation strategies related to adapting and tailoring to context; only the innovation tournament identified implementation strategies that used incentives. Observations and interviews required more than three times the personnel hours than the innovation tournament, but the innovation tournament was more costly overall due to the technological platform used. CONCLUSIONS: There was substantial overlap in content derived from observations and interviews and the innovation tournament, although there was greater specificity in the findings from observations and interviews. However, the innovation tournament yielded unique information. To select the best participatory design approach to inform implementation strategy design for a particular context, researchers should carefully consider unique advantages of each method and weigh the resources available to invest in the process.

Stakeholder Perspectives on MAPS.

BACKGROUND: Managed problem solving (MAPS) is an evidence-based intervention that can boost HIV medication adherence and increase viral suppression, but it is not widely used in community clinics. Deploying community health workers to deliver MAPS could facilitate broader implementation, in support of the Ending the HIV Epidemic (EHE) initiative's goal of reducing new HIV infections in the US by 90% by 2030. SETTING: Ryan White-funded clinics in Philadelphia, 1 of 48 US counties prioritized in the EHE. METHODS: Semistructured stakeholder interviews were conducted with 13 clinics and 4 stakeholder groups: prescribing clinicians, nonprescribing clinical team members (eg, medical case managers), clinic administrators, and policymakers. Interviews were based on the Consolidated Framework for Implementation Research and investigated perceived barriers to and facilitators of MAPS delivery by community health workers. Rapid qualitative analytic techniques were used to synthesize interview data and identify key categories along an implementation pathway. Core determinants (ie, barriers and facilitators) of MAPS implementation were grouped within each category. RESULTS: Stakeholders were receptive to CHW-delivered MAPS and offered critical information on potential implementation determinants including preferences for identification and referral of patients, and the importance of integration and communication within the care team. CONCLUSIONS: This study elucidates insights regarding barriers and facilitators to delivering an evidence-based behavioral intervention in clinics serving people with HIV (PWH) and extends a rapid qualitative approach to HIV care that rigorously incorporates stakeholder data into the development of implementation strategies. It also offers insights for national implementation efforts associated with EHE.

Making Progress Monitoring Easier and More Motivating: Developing a Client Data Collection App Incorporating User-Centered Design and Behavioral Economics Insights.

Data collection is an important component of evidence-based behavioral interventions for children with autism, but many one-to-one aides (i.e., behavioral support staff) do not systemically collect quantitative data that are necessary for best-practice client progress monitoring. Data collection of clients' behaviors often involves labor-intensive pen-and-paper practices. In addition, the solitary nature of one-to-one work limits opportunities for timely supervisor feedback, potentially reducing motivation to collect data. We incorporated principles from behavioral economics and user-centered design to develop a phone-based application, Footsteps, to address these challenges. We interviewed nine one-to-one aides working with children with autism and seven supervisors to ask for their app development ideas. We then developed the Footsteps app prototype and tested the prototype with 10 one-to-one aides and supervisors through three testing cycles. At each cycle, one-to-one aides rated app usability. Participants provided 76 discrete suggestions for improvement, including 29 new app features (e.g., behavior timer), 20 feature modifications (e.g., numeric type-in option for behavior frequency), four flow modifications (e.g., deleting a redundant form), and 23 out-of-scope suggestions. Of the participants that tested the app, 90% rated usability as good or excellent. Results support continuing to develop Footsteps and testing its impact in a clinical trial.

Variation in Stakeholder Preferences for Implementing Evidence-Based Practices in Behavioral Health Care.

Objective: The authors examined whether stakeholders in behavioral health care differ in their preferences for strategies that support the implementation of evidence-based practices (EBPs). Methods: Using data collected in March and April 2019 in a survey of stakeholders in Philadelphia Medicaid's behavioral health care system, the authors compared empirical Bayes preference weights for implementation strategies across clinicians, supervisors, agency executives, and payers. Results: Preferences for implementation strategies overlapped among the stakeholders (N=357 survey respondents). Financial incentives were consistently ranked as most useful and performance feedback as the least useful for implementing EBPs. However, areas of divergence were identified. For example, payers preferred compensation for EBP delivery, whereas clinicians considered compensation for time spent on preparing for EBPs as equally useful. Conclusions: The observed variation in stakeholder preferences for strategies to implement EBPs may shed light on why the ongoing shift from volume to value in behavioral health care has had mixed results.

Behavioral economic implementation strategies to improve serious illness communication between clinicians and high-risk patients with cancer: protocol for a cluster randomized pragmatic trial.

BACKGROUND: Serious illness conversations (SICs) are an evidence-based approach to eliciting patients' values, goals, and care preferences that improve patient outcomes. However, most patients with cancer die without a documented SIC. Clinician-directed implementation strategies informed by behavioral economics ("nudges") that identify high-risk patients have shown promise in increasing SIC documentation among clinicians. It is unknown whether patient-directed nudges that normalize and prime patients towards SIC completion-either alone or in combination with clinician nudges that additionally compare performance relative to peers-may improve on this approach. Our objective is to test the effect of clinician- and patient-directed nudges as implementation strategies for increasing SIC completion among patients with cancer. METHODS: We will conduct a 2 × 2 factorial, cluster randomized pragmatic trial to test the effect of nudges to clinicians, patients, or both, compared to usual care, on SIC completion. Participants will include 166 medical and gynecologic oncology clinicians practicing at ten sites within a large academic health system and their approximately 5500 patients at high risk of predicted 6-month mortality based on a validated machine-learning prognostic algorithm. Data will be obtained via the electronic medical record, clinician survey, and semi-structured interviews with clinicians and patients. The primary outcome will be time to SIC documentation among high-risk patients. Secondary outcomes will include time to SIC documentation among all patients (assessing spillover effects), palliative care referral among high-risk patients, and aggressive end-of-life care utilization (composite of chemotherapy within 14 days before death, hospitalization within 30 days before death, or admission to hospice within 3 days before death) among high-risk decedents. We will assess moderators of the effect of implementation strategies and conduct semi-structured interviews with a subset of clinicians and patients to assess contextual factors that shape the effectiveness of nudges with an eye towards health equity. DISCUSSION: This will be the first pragmatic trial to evaluate clinician- and patient-directed nudges to promote SIC completion for patients with cancer. We expect the study to yield insights into the effectiveness of clinician and patient nudges as implementation strategies to improve SIC rates, and to uncover multilevel contextual factors that drive response to these strategies. TRIAL REGISTRATION: ClinicalTrials.gov , NCT04867850 . Registered on April 30, 2021. FUNDING: National Cancer Institute P50CA244690.

Rationale and protocol for a cluster randomized pragmatic clinical trial testing behavioral economic implementation strategies to improve tobacco treatment rates for cancer patients who smoke.

BACKGROUND: Routine evidence-based tobacco use treatment minimizes cancer-specific and all-cause mortality, reduces treatment-related toxicity, and improves quality of life among patients receiving cancer care. Few cancer centers employ mechanisms to systematically refer patients to evidence-based tobacco cessation services. Implementation strategies informed by behavioral economics can increase tobacco use treatment engagement within oncology care. METHODS: A four-arm cluster-randomized pragmatic trial will be conducted across nine clinical sites within the Implementation Science Center in Cancer Control Implementation Lab to compare the effect of behavioral economic implementation strategies delivered through embedded messages (or "nudges") promoting patient engagement with the Tobacco Use Treatment Service (TUTS). Nudges are electronic medical record (EMR)-based messages delivered to patients, clinicians, or both, designed to counteract known patient and clinician biases that reduce treatment engagement. We used rapid cycle approaches (RCA) informed by relevant stakeholder experiences to refine and optimize our implementation strategies and methods prior to trial initiation. Data will be obtained via the EMR, clinician survey, and semi-structured interviews with a subset of clinicians and patients. The primary measure of implementation is penetration, defined as the TUTS referral rate. Secondary outcome measures of implementation include patient treatment engagement (defined as the number of patients who receive FDA-approved medication or behavioral counseling), quit attempts, and abstinence rates. The semi-structured interviews, guided by the Consolidated Framework for Implementation Research, will assess contextual factors and patient and clinician experiences with the nudges. DISCUSSION: This study will be the first in the oncology setting to compare the effectiveness of nudges to clinicians and patients, both head-to-head and in combination, as implementation strategies to improve TUTS referral and engagement. We expect the study to (1) yield insights into the effectiveness of nudges as an implementation strategy to improve uptake of evidence-based tobacco use treatment within cancer care, and (2) advance our understanding of the multilevel contextual factors that drive response to these strategies. These results will lay the foundation for how patients with cancer who smoke are best engaged in tobacco use treatment and may lead to future research focused on scaling this approach across diverse centers. TRIAL REGISTRATION: Clinicaltrials.gov, NCT04737031 . Registered 3 February 2021.

Community stakeholder preferences for evidence-based practice implementation strategies in behavioral health: a best-worst scaling choice experiment.

BACKGROUND: Community behavioral health clinicians, supervisors, and administrators play an essential role in implementing new psychosocial evidence-based practices (EBP) for patients receiving psychiatric care; however, little is known about these stakeholders' values and preferences for implementation strategies that support EBP use, nor how best to elicit, quantify, or segment their preferences. This study sought to quantify these stakeholders' preferences for implementation strategies and to identify segments of stakeholders with distinct preferences using a rigorous choice experiment method called best-worst scaling. METHODS: A total of 240 clinicians, 74 clinical supervisors, and 29 administrators employed within clinics delivering publicly-funded behavioral health services in a large metropolitan behavioral health system participated in a best-worst scaling choice experiment. Participants evaluated 14 implementation strategies developed through extensive elicitation and pilot work within the target system. Preference weights were generated for each strategy using hierarchical Bayesian estimation. Latent class analysis identified segments of stakeholders with unique preference profiles. RESULTS: On average, stakeholders preferred two strategies significantly more than all others-compensation for use of EBP per session and compensation for preparation time to use the EBP (P < .05); two strategies were preferred significantly less than all others-performance feedback via email and performance feedback via leaderboard (P < .05). However, latent class analysis identified four distinct segments of stakeholders with unique preferences: Segment 1 (n = 121, 35%) strongly preferred financial incentives over all other approaches and included more administrators; Segment 2 (n = 80, 23%) preferred technology-based strategies and was younger, on average; Segment 3 (n = 52, 15%) preferred an improved waiting room to enhance client readiness, strongly disliked any type of clinical consultation, and had the lowest participation in local EBP training initiatives; Segment 4 (n = 90, 26%) strongly preferred clinical consultation strategies and included more clinicians in substance use clinics. CONCLUSIONS: The presence of four heterogeneous subpopulations within this large group of clinicians, supervisors, and administrators suggests optimal implementation may be achieved through targeted strategies derived via elicitation of stakeholder preferences. Best-worst scaling is a feasible and rigorous method for eliciting stakeholders' implementation preferences and identifying subpopulations with unique preferences in behavioral health settings.

Correction to: A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia.

.

A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia.

BACKGROUND: Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static. METHOD: Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia's public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised. RESULTS: Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048). CONCLUSIONS: System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.

The clinician crowdsourcing challenge: using participatory design to seed implementation strategies.

BACKGROUND: In healthcare settings, system and organization leaders often control the selection and design of implementation strategies even though frontline workers may have the most intimate understanding of the care delivery process, and factors that optimize and constrain evidence-based practice implementation within the local system. Innovation tournaments, a structured participatory design strategy to crowdsource ideas, are a promising approach to participatory design that may increase the effectiveness of implementation strategies by involving end users (i.e., clinicians). We utilized a system-wide innovation tournament to garner ideas from clinicians about how to enhance the use of evidence-based practices (EBPs) within a large public behavioral health system. METHODS: Our innovation tournament occurred in three phases. First, we invited over 500 clinicians to share, through a web-based platform, their ideas regarding how their organizations could best support use of EBPs. Clinicians could rate and comment on ideas submitted by others. Second, submissions were judged by an expert panel (including behavioral scientists, system leaders, and payers) based on their rated enthusiasm for the idea. Third, we held a community-facing event during which the six clinicians who submitted winning ideas presented their strategies to 85 attendees representing a cross-section of clinicians and system and organizational leaders. RESULTS: We had a high rate of participation (12.3%), more than double the average rate of previous tournaments conducted in other settings (5%). A total of 65 ideas were submitted by 55 participants representing 38 organizations. The most common categories of ideas pertained to training (42%), financing and compensation (26%), clinician support and preparation tools (22%), and EBP-focused supervision (17%). The expert panel and clinicians differed on their ratings of the ideas, highlighting value of seeking input from multiple stakeholder groups when developing implementation strategies. CONCLUSIONS: Innovation tournaments are a useful and feasible methodology for engaging end users, system leaders, and behavioral scientists through a structured approach to developing implementation strategies. The process and resultant strategies engendered significant enthusiasm and engagement from participants at all levels of a healthcare system. Research is needed to compare the effectiveness of strategies developed through innovation tournaments to strategies developed through design approaches.

Transforming Mental Health Delivery Through Behavioral Economics and Implementation Science: Protocol for Three Exploratory Projects.

BACKGROUND: Efficacious psychiatric treatments are not consistently deployed in community practice, and clinical outcomes are attenuated compared with those achieved in clinical trials. A major focus for mental health services research is to develop effective and cost-effective strategies that increase the use of evidence-based assessment, prevention, and treatment approaches in community settings. OBJECTIVE: The goal of this program of research is to apply insights from behavioral economics and participatory design to advance the science and practice of implementing evidence-based practice (EBP) for individuals with psychiatric disorders across the life span. METHODS: Project 1 (Assisting Depressed Adults in Primary care Treatment [ADAPT]) is patient-focused and leverages decision-making heuristics to compare ways to incentivize adherence to antidepressant medications in the first 6 weeks of treatment among adults newly diagnosed with depression. Project 2 (App for Strengthening Services In Specialized Therapeutic Support [ASSISTS]) is provider-focused and utilizes normative pressure and social status to increase data collection among community mental health workers treating children with autism. Project 3 (Motivating Outpatient Therapists to Implement: Valuing a Team Effort [MOTIVATE]) explores how participatory design can be used to design organizational-level implementation strategies to increase clinician use of EBPs. The projects are supported by a Methods Core that provides expertise in implementation science, behavioral economics, participatory design, measurement, and associated statistical approaches. RESULTS: Enrollment for project ADAPT started in 2018; results are expected in 2020. Enrollment for project ASSISTS will begin in 2019; results are expected in 2021. Enrollment for project MOTIVATE started in 2018; results are expected in 2019. Data collection had begun for ADAPT and MOTIVATE when this protocol was submitted. CONCLUSIONS: This research will advance the science of implementation through efforts to improve implementation strategy design, measurement, and statistical methods. First, we will test and refine approaches to collaboratively design implementation strategies with stakeholders (eg, discrete choice experiments and innovation tournaments). Second, we will refine the measurement of mechanisms related to heuristics used in decision making. Third, we will develop new ways to test mechanisms in multilevel implementation trials. This trifecta, coupled with findings from our 3 exploratory projects, will lead to improvements in our knowledge of what causes successful implementation, what variables moderate and mediate the effects of those causal factors, and how best to leverage this knowledge to increase the quality of care for people with psychiatric disorders. TRIAL REGISTRATION: ClinicalTrials.gov NCT03441399; https://www.clinicaltrials.gov/ct2/show/NCT03441399 (Archived by WebCite at http://www.webcitation.org/74dRbonBD). INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/12121.