The use of external change agents to promote quality improvement and organizational change in healthcare organizations: a systematic review.

BACKGROUND: External change agents can play an essential role in healthcare organizational change efforts. This systematic review examines the role that external change agents have played within the context of multifaceted interventions designed to promote organizational change in healthcare-specifically, in primary care settings. METHODS: We searched PubMed, CINAHL, Cochrane, Web of Science, and Academic Search Premier Databases in July 2016 for randomized trials published (in English) between January 1, 2005 and June 30, 2016 in which external agents were part of multifaceted organizational change strategies. The review was conducted according to PRISMA guidelines. A total of 477 abstracts were identified and screened by 2 authors. Full text articles of 113 studies were reviewed. Twenty-one of these studies were selected for inclusion. RESULTS: Academic detailing (AD) is the most prevalently used organizational change strategy employed as part of multi-component implementation strategies. Out of 21 studies, nearly all studies integrate some form of audit and feedback into their interventions. Eleven studies that included practice facilitation into their intervention reported significant effects in one or more primary outcomes. CONCLUSIONS: Our results demonstrate that practice facilitation with regular, tailored follow up is a powerful component of a successful organizational change strategy. Academic detailing alone or combined with audit and feedback alone is ineffective without intensive follow up. Provision of educational materials and use of audit and feedback are often integral components of multifaceted implementation strategies. However, we didn't find examples where those relatively limited strategies were effective as standalone interventions. System-level support through technology (such as automated reminders or alerts) is potentially helpful, but must be carefully tailored to clinic needs.

Implementing a Mobile Health System to Integrate the Treatment of Addiction Into Primary Care: A Hybrid Implementation-Effectiveness Study.

BACKGROUND: Despite the near ubiquity of mobile phones, little research has been conducted on the implementation of mobile health (mHealth) apps to treat patients in primary care. Although primary care clinicians routinely treat chronic conditions such as asthma and diabetes, they rarely treat addiction, a common chronic condition. Instead, addiction is most often treated in the US health care system, if it is treated at all, in a separate behavioral health system. mHealth could help integrate addiction treatment in primary care. OBJECTIVE: The objective of this paper was to report the effects of implementing an mHealth system for addiction in primary care on both patients and clinicians. METHODS: In this implementation research trial, an evidence-based mHealth system named Seva was introduced sequentially over 36 months to a maximum of 100 patients with substance use disorders (SUDs) in each of three federally qualified health centers (FQHCs; primary care clinics that serve patients regardless of their ability to pay). This paper reports on patient and clinician outcomes organized according to the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. RESULTS: The outcomes according to the RE-AIM framework are as follows: Reach-Seva reached 8.31% (268/3226) of appropriate patients. Reach was limited by our ability to pay for phones and data plans for a maximum of 100 patients per clinic. Effectiveness-Patients who were given Seva had significant improvements in their risky drinking days (44% reduction, (0.7-1.25)/1.25, P=.04), illicit drug-use days (34% reduction, (2.14-3.22)/3.22, P=.01), quality of life, human immunodeficiency virus screening rates, and number of hospitalizations. Through Seva, patients also provided peer support to one another in ways that are novel in primary care settings. Adoption-Patients sustained high levels of Seva use-between 53% and 60% of the patients at the 3 sites accessed Seva during the last week of the 12-month implementation period. Among clinicians, use of the technology was less robust than use by patients, with only a handful of clinicians using Seva in each clinic and behavioral health providers making most referrals to Seva in 2 of the 3 clinics. Implementation-At 2 sites, implementation plans were realized successfully; they were delayed in the third. Maintenance-Use of Seva dropped when grant funding stopped paying for the mobile phones and data plans. Two of the 3 clinics wanted to maintain the use of Seva, but they struggled to find funding to support this. CONCLUSIONS: Implementing an mHealth system can improve care among primary care patients with SUDs, and patients using the system can support one another in their recovery. Among clinicians, however, implementation requires figuring out how information from the mHealth system will be used and making mHealth data available in the electronic health (eHealth) record. In addition, paying for an mHealth system remains a challenge.

Reducing Symptom Distress in Patients With Advanced Cancer Using an e-Alert System for Caregivers: Pooled Analysis of Two Randomized Clinical Trials.

BACKGROUND: Symptom distress in patients toward the end of life can change rapidly. Family caregivers have the potential to help patients manage those symptoms, as well as their own stress, if they are equipped with the proper resources. Electronic health (eHealth) systems may be able to provide those resources. Very sick patients may not be able to use such systems themselves to report their symptoms but family caregivers could. OBJECTIVE: The aim of this paper was to assess the effects on cancer patient symptom distress of an eHealth system that alerts clinicians to significant changes in the patient's symptoms, as reported by a family caregiver. METHODS: A pooled analysis from two randomized clinical trials (NCT00214162 and NCT00365963) compared outcomes at 12 months for two unblinded groups: a control group (Comprehensive Health Enhancement Support System [CHESS]-Only) that gave caregivers access to CHESS, an online support system, and an experimental group (CHESS+CR [Clinician Report]), which also had CHESS but with a CR that automatically alerted clinicians if symptoms exceeded a predetermined threshold of severity. Participants were dyads (n=235) of patients with advanced lung, breast, or prostate cancer and their respective family caregivers from 5 oncology clinics in the United States of America. The proportion of improved patient threshold symptoms was compared between groups using area-under-the-curve analysis and binomial proportion tests. The proportion of threshold symptoms out of all reported symptoms was also examined. RESULTS: When severe caregiver-reported symptoms were shared with clinicians, the symptoms were more likely to be subsequently reported as improved than when the symptoms were not shared with clinicians (P<.001). Fewer symptom reports were completed in the group of caregivers whose reports went to clinicians than in the CHESS-Only group (P<.001), perhaps because caregivers, knowing their reports might be sent to a doctor, feared they might be bothering the clinician. CONCLUSIONS: This study suggests that an eHealth system designed for caregivers that alerts clinicians to worrisome changes in patient health status may lead to reduced patient distress. TRIAL REGISTRATION: Clinicaltrials.gov NCT00214162; https://clinicaltrials.gov/ct2/show/NCT00214162 (Archived by WebCite at http://www.webcitation.org/6nmgdGfuD) and Clinicaltrials.gov NCT00365963; https://clinicaltrials.gov/ct2/show/NCT00365963 (Archived by WebCite at http://www.webcitation.org/6nmh0U8VP).

Implementing an mHealth system for substance use disorders in primary care: a mixed methods study of clinicians' initial expectations and first year experiences.

BACKGROUND: Millions of Americans need but don't receive treatment for substance use, and evidence suggests that addiction-focused interventions on smart phones could support their recovery. There is little research on implementation of addiction-related interventions in primary care, particularly in Federally Qualified Health Centers (FQHCs) that provide primary care to underserved populations. We used mixed methods to examine three FQHCs' implementation of Seva, a smart-phone app that offers patients online support/discussion, health-tracking, and tools for coping with cravings, and offers clinicians information about patients' health tracking and relapses. We examined (a) clinicians' initial perspectives about implementing Seva, and (b) the first year of implementation at Site 1. METHODS: Prior to staggered implementation at three FQHCs (Midwest city in WI vs. rural town in MT vs. metropolitan NY), interviews, meetings, and focus groups were conducted with 53 clinicians to identify core themes of initial expectations about implementation. One year into implementation at Site 1, clinicians there were re-interviewed. Their reports were supplemented by quantitative data on clinician and patient use of Seva. RESULTS: Clinicians anticipated that Seva could help patients and make behavioral health appointments more efficient, but they were skeptical that physicians would engage with Seva (given high caseloads), and they were uncertain whether patients would use Seva. They were concerned about legal obligations for monitoring patients' interactions online, including possible "cries for help" or inappropriate interactions. One year later at Site 1, behavioral health care providers, rather than physicians, had incorporated Seva into patient care, primarily by discussing it during appointments. Given workflow/load concerns, only a few key clinicians monitored health tracking/relapses and prompted outreach when needed; two researchers monitored the discussion board and alerted the clinic as needed. Clinician turnover/leave complicated this approach. Contrary to clinicians' initial concerns, patients showed sustained, mutually supportive use of Seva, with few instances of misuse. CONCLUSIONS: Results suggest the value of (a) focusing implementation on behavioral health care providers rather than physicians, (b) assigning a few individuals (not necessarily clinicians) to monitor health tracking, relapses, and the discussion board, (c) anticipating turnover/leave and having designated replacements. Patients showed sustained, positive use of Seva. TRIAL REGISTRATION: ClinicalTrials.gov ( NCT01963234 ).

Systems consultation: protocol for a novel implementation strategy designed to promote evidence-based practice in primary care.

BACKGROUND: Adoption of evidence-based practices takes place at a glacial place in healthcare. This research will pilot test an innovative implementation strategy - systems consultation -intended to speed the adoption of evidence-based practice in primary care. The strategy is based on tenets of systems engineering and has been extensively tested in addiction treatment. Three innovations have been included in the strategy - translation of a clinical practice guideline into a checklist-based implementation guide, the use of physician peer coaches ('systems consultants') to help clinics implement the guide, and a focus on reducing variation in practices across prescribers and clinics. The implementation strategy will be applied to improving opioid prescribing practices in primary care, which may help ultimately mitigate the increasing prevalence of opioid abuse and addiction. METHODS/DESIGN: The pilot test will compare four intervention clinics to four control clinics in a matched-pairs design. A leading clinical guideline for opioid prescribing has been translated into a checklist-based implementation guide in a systematic process that involved experts who wrote the guideline in consultation with implementation experts and primary care physicians. Two physicians with expertise in family and addiction medicine are serving as the systems consultants. Each systems consultant will guide two intervention clinics, using two site visits and follow-up communication by phone and email, to implement the translated guideline. Mixed methods will be used to test the feasibility, acceptability, and preliminary effectiveness of the implementation strategy in an evaluation that meets standards for 'fully developed use' of the RE-AIM framework (Reach, Effectiveness, Adoption, Implementation, Maintenance). The clinic will be the primary unit of analysis. DISCUSSION: The systems consultation implementation strategy is intended to generalize to the adoption of other clinical guidelines. This pilot test is intended to prepare for a large randomized clinical trial that will test the strategy against other implementation strategies, such as audit/feedback and academic detailing, used to close the gap between knowledge and practice. The systems consultation approach has the potential to shorten the famously long time it takes to implement evidence-based practices and clinical guidelines in healthcare.

Feasibility of implementing a screening tool for risk of opioid misuse in a trauma surgical population.

Background: As the opioid crisis continues to affect communities across the United States, new interventions for screening and prevention are needed to mitigate its impact. Mental health diagnoses have been identified as a risk factor for opioid misuse, and surgical populations and injury survivors are at high risk for prolonged opioid use and misuse. This study investigated the implementation of a novel opioid risk screening tool that incorporated putative risk factors from a recent study in four trauma units across Wisconsin. Method: The screening tool was implemented across a 6-month period at four sites. Data was collected via monthly meeting notes and "Plan, Do, Study, Act" (PDSA) forms. Following implementation, focus groups reflected on the facilitators and barriers to implementation. Meeting notes, PDSA forms, and focus group data were analyzed using the consolidated framework for implementation research, followed by thematic analyses, to generate themes surrounding the facilitators and barriers to implementing an opioid misuse screener. Results: Implementation facilitators included ensuring patient understanding of the screener, minimizing staff burden from screening, and educating staff to encourage engagement. Barriers included infrastructure limitations that prevented seamless administration of the screener within current workflows, overlap of the screener with existing measures, and lack of guidance surrounding treatment options corresponding to risk. Recommended solutions to address barriers include careful timing of screener administration, accommodating workflows, integration of the screening tool within the electronic health record, and evidence-based interventions guided by screener results. Conclusion: Four trauma centers across Wisconsin successfully implemented a pilot opioid misuse screening tool. Trauma providers and unit staff members believe that this tool would be a beneficial addition to their repertoire if their recommendations were adopted. Future research should refine opioid misuse risk factors and ensure screening items are well-validated with psychometric research supporting treatment responses to screener-indicated risk categories.

As the opioid crisis continues to affect communities across the United States, new interventions for early screening and prevention are needed to minimize the related harms. Prior research has identified risk factors associated with opioid misuse among a trauma surgical patient population, with the highest risk associated with distress-related posttraumatic stress disorder symptoms. A pilot screening tool was created based on this prior research, which was then administered at four trauma surgical units across the state of Wisconsin. Each of the four trauma units successfully implemented the pilot screening tool, and each identified a number of facilitators and barriers to the implementation process. Recommendations for improvement of the implementation process were also gathered. If their recommended changes were to be adopted, trauma providers and trauma unit staff members believed that such a screener for opioid misuse would be a beneficial addition to their current workflow among traumatic injury patients. Future research should refine opioid misuse risk factors and develop a psychometrically sound, validated screener to detect varying levels of risk and tailor treatment approaches based on a patient's risk score. Additionally, future research in the field of opioid misuse prevention should prioritize the recruitment of a more diverse population to support the translation of study findings across populations.

Understanding the Implementation of Informal Meditation Practice in a Smartphone-Based Intervention: A Qualitative Analysis.

Objectives: Informal practice (i.e., brief meditation practices incorporated spontaneously into daily activities) may be important for increasing the efficacy and accessibility of meditation-based interventions (MedBIs). However, the facilitators and barriers to engaging in informal practice are largely unknown. The current study aimed to investigate factors associated with the implementation of informal practice. Method: Participants were drawn from a randomized trial testing the effects of 5- versus 15-min daily meditation practice in a 4-week smartphone-delivered meditation training. Qualitative interviews on informal practice were conducted with 17 participants (mean age: 37.12 years; 82.35% female; 52.94% non-Latinx White) following the intervention. Given that prior knowledge on this topic is limited, inductive content analysis was utilized to characterize participants' experiences in relation to implementing informal practice. Results: Four overarching categories emerged from the data, namely (a) reported benefits of informal practice, (b) integration of informal practice, (c) perceived barriers to informal practice, and (d) recommended facilitators of informal practice. Conclusion: This study underscores the importance of addressing barriers and facilitators (e.g., providing personalized app features, reminders, social support, and repeating intervention content) to encourage individuals' informal practice. Findings provide suggestions for methods to increase engagement in informal practice, which may, in turn, increase the accessibility and effectiveness of MedBIs. Preregistration: The larger trial from which the qualitative interview participants were drawn was preregistered through clinicaltrials.gov (NCT05229406) and the Open Science Framework (https://osf.io/fszvj/?view_only=039b14ccbf8848bd99808c983070b635). The qualitative analyses reported here were not preregistered.

Testing support models for implementing an evidence-based digital intervention for alcohol use disorder: results of a pragmatic hybrid implementation-effectiveness trial.

This paper reports results of a hybrid effectiveness-implementation randomized trial that systematically varied levels of human oversight required to support implementation of a digital medicine intervention for persons with mild to moderate alcohol use disorder (AUD). Participants were randomly assigned to three groups representing possible digital health support models within a health system: self-monitored use (n = 185), peer-supported use (n = 186), or a clinically integrated model (n = 187). Across all three groups, percentage of risky drinking days dropped from 38.4% at baseline (95%CI [35.8%, 41%]) to 22.5% (19.5%, 25.5%) at 12 months. The clinically integrated group showed significant improvements in mental health quality of life compared to the self-monitoring group (p = 0.011). However, higher rates of attrition in the clinically integrated group warrants consideration in interpreting this result. Results suggest that making a self-guided digital intervention available to patients may be a viable option for health systems looking to promote alcohol risk reduction.

Barriers to Type 2 Diabetes Mellitus Management for Older Hmong Patients with Minimal English Language Skills: Accounts from Caregivers, Case Managers, and Clinicians.

Type 2 diabetes mellitus prevalence rates for Hmong Americans in Wisconsin are more than double that of non-Hispanic Whites. The Hmong's history, lifestyle (dietary and behavioral patterns), and reliance on traditional medicine contribute to their increased risk of diabetes. This qualitative study aimed to better understand the barriers challenging older Hmong patients' ability to manage diabetes. Asian Americans have long been overlooked in health-related research, but recent disaggregated data of specific ethnic groups reveal significant health inequities. Among the different ethnic groups, there is a significant lack of research on the Hmong Americans. Three participant groups (Hmong American family caregivers, Hmong American case managers, and clinicians from different racial backgrounds who provide care for Hmong patients) were recruited from the community and interviewed to understand the barriers experienced by older Hmong patients with minimal English language skills in managing their diabetes. Directed content analysis of the data resulted in three major themes: adherence to culture, health inequity, and managing diabetes. Subthemes included Hmong herbs and shamans, lack of trust in Western medicine, the significance of rice, language barriers, lack of cultural sensitivity, health literacy, monitoring glucose, medicine compliance, and nutrition. Minimal English language skills and low literacy rates (health and education) contribute to their strong adherence to cultural practices which challenges Western medicine, creating difficulty for older Hmong patients to manage their diabetes. Recognizing cultural differences and barriers will enable healthcare providers to improve and cater the treatment options, bridging the gap between older Hmong patients and Western medicine.

Testing a Digital Health App for Patients With Alcohol-Associated Liver Disease: Mixed Methods Usability Study.

BACKGROUND: Alcohol-associated liver disease (ALD) is increasingly common and associated with serious and costly health consequences. Cessation of drinking can improve ALD morbidity and mortality; however, support for cessation is not routinely offered to those diagnosed with ALD, and continued drinking or resumption of drinking after diagnosis is common. Mobile health (mHealth) has the potential to offer convenient and scalable support for alcohol cessation to those diagnosed with ALD, but mHealth interventions for alcohol cessation have not been designed for or evaluated in a population with ALD. OBJECTIVE: This study aims to understand how individuals with ALD would perceive and use an mHealth tool for alcohol cessation and to gather their perspectives on potential refinements to the tool that would allow it to better meet their needs. METHODS: We interviewed 11 individuals who attended clinic visits related to their ALD to elicit their needs related to support for alcohol cessation and views on how mHealth could be applied. After completing initial interviews (pre), participants were provided with access to an mHealth app designed for alcohol cessation, which they used for 1 month. Afterward, they were interviewed again (post) to give feedback on their experiences, including aspects of the app that met their needs and potential refinements. We applied a mixed methods approach, including a qualitative analysis to identify major themes from the interview transcripts and descriptive analyses of use of the app over 1 month. RESULTS: First, we found that a diagnosis of ALD is perceived as a motivator to quit drinking but that patients had difficulty processing the overwhelming amount of information about ALD they received and finding resources for cessation of alcohol use. Second, we found that the app was perceived as usable and useful for supporting drinking recovery, with patients responding favorably to the self-tracking and motivational components of the app. Finally, patients identified areas in which the app could be adapted to meet the needs of patients with ALD, such as providing information on the medical implications of an ALD diagnosis and how to care for their liver as well as connecting individuals with ALD to one another via a peer-to-peer support forum. Rates of app use were high and sustained across the entire study, with participants using the app a little more than half the days during the study on average and with 100% (11/11) of participants logging in each week. CONCLUSIONS: Our results highlight the need for convenient access to resources for alcohol cessation after ALD diagnosis and support the potential of an mHealth approach to integrate recovery support into care for ALD. Our findings also highlight the ways the alcohol cessation app should be modified to address ALD-specific concerns.

A structured approach to applying systems analysis methods for examining implementation mechanisms.

BACKGROUND: It is challenging to identify and understand the specific mechanisms through which an implementation strategy affects implementation outcomes, as implementation happens in the context of complex, multi-level systems. These systems and the mechanisms within each level have their own dynamic environments that change frequently. For instance, sequencing may matter in that a mechanism may only be activated indirectly by a strategy through another mechanism. The dosage or strength of a mechanism may vary over time or across different health care system levels. To elucidate the mechanisms relevant to successful implementation amidst this complexity, systems analysis methods are needed to model and manage complexity. METHODS: The fields of systems engineering and systems science offer methods-which we refer to as systems analysis methods-to help explain the interdependent relationships between and within systems, as well as dynamic changes to systems over time. When applied to studying implementation mechanisms, systems analysis methods can help (i) better identify and manage unknown conditions that may or may not activate mechanisms (both expected mechanisms targeted by a strategy and unexpected mechanisms that the methods help detect) and (ii) flexibly guide strategy adaptations to address contextual influences that emerge after the strategy is selected and used. RESULTS: In this paper, we delineate a structured approach to applying systems analysis methods for examining implementation mechanisms. The approach includes explicit steps for selecting, tailoring, and evaluating an implementation strategy regarding the mechanisms that the strategy is initially hypothesized to activate, as well as additional mechanisms that are identified through the steps. We illustrate the approach using a case example. We then discuss the strengths and limitations of this approach, as well as when these steps might be most appropriate, and suggest work to further the contributions of systems analysis methods to implementation mechanisms research. CONCLUSIONS: Our approach to applying systems analysis methods can encourage more mechanisms research efforts to consider these methods and in turn fuel both (i) rigorous comparisons of these methods to alternative mechanisms research approaches and (ii) an active discourse across the field to better delineate when these methods are appropriate for advancing mechanisms-related knowledge.

Applying concepts from "rapid" and "agile" implementation to advance implementation research.

BACKGROUND: The translation of research findings into practice can be improved to maximize benefits more quickly and with greater flexibility. To expedite translation, researchers have developed innovative approaches to implementation branded as "rapid" and "agile" implementation. Rapid implementation has roots in precision medicine and agile implementation has roots in systems engineering and software design. Research has shown that innovation often derives from learning and applying ideas that have impacted other fields. IMPLICATIONS FOR IMPLEMENTATION RESEARCHERS: This commentary examines "rapid" and "agile" approaches to implementation and provides recommendations to implementation researchers stemming from these approaches. Four key ideas are synthesized that may be broadly applicable to implementation research, including (1) adopting a problem orientation, (2) applying lessons from behavioral economics, (3) using adaptive study designs and adaptive interventions, and (4) using multi-level models to guide implementation. Examples are highlighted from the field where researchers are applying these key ideas to illustrate their potential impact. CONCLUSIONS: "Rapid" and "agile" implementation approaches to implementation stem from diverse fields. Elements of these approaches show potential for advancing implementation research, although adopting them may entail shifting scientific norms in the field.

Screening in Trauma for Opioid Misuse Prevention (STOMP): Results from a prospective cohort of victims of traumatic injury.

BACKGROUND: Traumatic injury frequently requires opioid analgesia to manage pain and avoid catastrophic complications. Risk screening for opioid misuse and the development of use disorder remains uninvestigated. METHODS: Participants were Trauma/Orthopedic Surgical Services patients at a Level I Trauma Center who were English speaking, aged 18-75, received an opioids prescription at discharge, and were under control of their own medications at discharge. Baseline measures included validated self-report instruments for psychosocial factors, such as anxiety, depression, pain coping, and social support. Health record data included diagnosis codes, procedures, Injury Severity Score, and pain severity (0-10 scale). Opioid use disorder (by Clinical International Diagnostic Interview-Substance Abuse Module) or opioid misuse (Current Opioid Misuse Measure (COMM) and survey items) were assessed at 24 weeks post-discharge. RESULTS: 295 patients enrolled with 237 completing the 24 week assessments. Stepwise regression modeling demonstrated pre-injury PTSD symptoms, Opioid Risk score, medication use behaviors, social support, and length of stay predicted opioid misuse. Pre-injury PTSD symptoms, pain coping, and length of stay predicted use disorder. The final regression models for opioid misuse by COMM, opioid misuse via survey items, and for opioid use disorder had highly favorable areas under the receiver operating curve (0.880, 0.790, and 0.943 respectively). CONCLUSIONS: Pre-injury presence of PTSD-related symptoms, impaired pain coping, social support, and hospitalization > 6 days predicted opioid misuse and opioid addiction at 6 months after hospital discharge. Behavioral screening and management strategies appear warranted in the population of traumatic injury victims to reduce opioid-related risks.

The economics of adaptations to evidence-based practices.

BACKGROUND: Evidence-based practices (EBPs) are frequently adapted in response to the dynamic contexts in which they are implemented. Adaptation is defined as the degree to which an EBP is altered to fit the setting or to improve fit to local context and can be planned or unplanned. Although adaptations are common and necessary to maximizing the marginal impact of EBPs, little attention has been given to the economic consequences and how adaptations affect marginal costs. DISCUSSION: In assessing the economic consequences of adaptation, one should consider its impact on core components, the planned adaptive periphery, and the unplanned adaptive periphery. Guided by implementation science frameworks, we examine how various economic evaluation approaches accommodate the influence of adaptations and discuss the pros and cons of these approaches. Using the Framework for Reporting Adaptations and Modifications to Evidence-based interventions (FRAME), mixed methods can elucidate the economic reasons driving the adaptations. Micro-costing approaches are applied in research that integrates the adaptation of EBPs at the planning stage using innovative, adaptive study designs. In contrast, evaluation of unplanned adaptation is subject to confounding and requires sensitivity analysis to address unobservable measures and other uncertainties. A case study is presented using the RE-AIM framework to illustrate the costing of adaptations. In addition to empirical approaches to evaluating adaptation, simulation modeling approaches can be used to overcome limited follow-up in implementation studies. CONCLUSIONS: As implementation science evolves to improve our understanding of the mechanisms and implications of adaptations, it is increasingly important to understand the economic implications of such adaptations, in addition to their impact on clinical effectiveness. Therefore, explicit consideration is warranted of how costs can be evaluated as outcomes of adaptations to the delivery of EBPs.

Testing an mHealth System for Individuals With Mild to Moderate Alcohol Use Disorders: Protocol for a Type 1 Hybrid Effectiveness-Implementation Trial.

BACKGROUND: The extent of human interaction needed to achieve effective and cost-effective use of mobile health (mHealth) apps for individuals with mild to moderate alcohol use disorder (AUD) remains largely unexamined. This study seeks to understand how varying levels of human interaction affect the ways in which an mHealth intervention for the prevention and treatment of AUDs works or does not work, for whom, and under what circumstances. OBJECTIVE: The primary aim is to detect the effectiveness of an mHealth intervention by assessing differences in self-reported risky drinking patterns and quality of life between participants in three study groups (self-monitored, peer-supported, and clinically integrated). The cost-effectiveness of each approach will also be assessed. METHODS: This hybrid type 1 study is an unblinded patient-level randomized clinical trial testing the effects of using an evidence-based mHealth system on participants' drinking patterns and quality of life. There are two groups of participants for this study: individuals receiving the intervention and health care professionals practicing in the broader health care environment. The intervention is a smartphone app that encourages users to reduce their alcohol consumption within the context of integrative medicine using techniques to build healthy habits. The primary outcomes for quantitative analysis will be participant data on their risky drinking days and quality of life as well as app use from weekly and quarterly surveys. Cost measures include intervention and implementation costs. The cost per participant will be determined for each study arm, with intervention and implementation costs separated within each group. There will also be a qualitative assessment of health care professionals' engagement with the app as well as their thoughts on participant experience with the app. RESULTS: This protocol was approved by the Health Sciences Minimal Risk Institutional Review Board on November 18, 2019, with subsequent annual reviews. Recruitment began on March 6, 2020, but was suspended on March 13, 2020, due to the COVID-19 pandemic restrictions. Limited recruitment resumed on July 6, 2020. Trial status as of November 17, 2021, is as follows: 357 participants were enrolled in the study for a planned enrollment of 546 participants. CONCLUSIONS: The new knowledge gained from this study could have wide and lasting benefits related to the integration of mHealth systems for individuals with mild to moderate AUDs. The results of this study will guide policy makers and providers toward cost-effective ways to incorporate technology in health care and community settings. TRIAL REGISTRATION: ClinicalTrials.gov NCT04011644; https://clinicaltrials.gov/ct2/show/NCT04011644. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/31109.

Efficacy and comparative effectiveness of telephone and smartphone remote continuing care interventions for alcohol use disorder: a randomized controlled trial.

BACKGROUND AND AIMS: Management of alcohol use disorder (AUD) could be enhanced by effective remote treatments. This study tested whether supplementing intensive outpatient programs (IOPs) with continuing care delivered via (1) telephone, (2) smartphone or (3) their combination improves outcomes relative to (4) IOP only. Continuing care conditions were also compared. DESIGN: Randomized controlled trial of four groups with 3-, 6-, 9-, 12- and 18-month follow-ups. SETTING: University research center in Philadelphia, PA, USA. PARTICIPANTS: Participants (n = 262) met DSM-V criteria for AUD, were largely male (71%) and African American (82%). INTERVENTIONS AND COMPARATOR: Telephone monitoring and counseling (TMC; n = 59), addiction comprehensive health enhancement support system (ACHESS; n = 68) and TMC + ACHESS (n = 70) provided for 12 months. The control condition received IOP only (TAU; n = 65). MEASUREMENT: The primary outcome was percentage of days heavy drinking (PDHD) in months 1-12. Secondary outcomes were any drinking, any drug use, drinking consequences and quality of life. FINDINGS: Mean PDHD in months 1-12 was 10.29 in TAU, 5.41 in TMC, 6.80 in ACHESS and 5.99 in TMC + ACHESS. PDHD was lower in TMC [Cohen's d = 0.35, P = 0.018, 95% confidence interval (CI) = (-1.42, -0.20)], ACHESS [d = 0.31, P = 0.031, 95% CI = (-1.27, -0.06)] and TMC + ACHESS [d = 0.36, P = 0.009, 95% CI = (-1.40, -0.20)] than in TAU. Differences between TMC + ACHESS, TMC and ACHESS were small (d ≤ 0.06) and non-significant. Findings were inconclusive as to whether or not the treatment conditions differed on PDHD at 18 months. A significant effect was obtained on any drinking, which was higher in months 1-12 in TAU than in TMC [odds ratio (OR) = 3.02, standard error (SE) = 0.43, 95% CI = (1.30, 6.99), P = 0.01] and TMC + ACHESS [OR = 2.43, SE = 0.39, 95% CI = (1.12, 5.27), P = 0.025). No other significant effects were obtained on other secondary outcomes during or after treatment. CONCLUSIONS: A telephone-delivered intervention and a smartphone-delivered intervention, alone and in combination, provided effective remote continuing care for alcohol use disorder. The combination of both interventions was not superior to either alone and effects did not persist post-treatment.

Implementation science issues in understanding, collecting, and using cost estimates: a multi-stakeholder perspective.

Understanding the resources needed to achieve desired implementation and effectiveness outcomes is essential to implementing and sustaining evidence-based practices (EBPs). Despite this frequent observation, cost and economic measurement and reporting are rare, but becoming more frequent in implementation science, and when present is seldom reported from the perspective of multiple stakeholders (e.g., the organization, supervisory team), including those who will ultimately implement and sustain EBPs.Incorporating a multi-level framework is useful for understanding and integrating the perspectives and priorities of the diverse set of stakeholders involved in implementation. Stakeholders across levels, from patients to delivery staff to health systems, experience different economic impacts (costs, benefit, and value) related to EBP implementation and have different perspectives on these issues. Economic theory can aid in understanding multi-level perspectives and approaches to addressing potential conflict across perspectives.This paper provides examples of key cost components especially important to different types of stakeholders. It provides specific guidance and recommendations for cost assessment activities that address the concerns of various stakeholder groups, identifies areas of agreement and conflict in priorities, and outlines theoretically informed approaches to understanding conflicts among stakeholder groups and processes to address them. Involving stakeholders throughout the implementation process and presenting economic information in ways that are clear and meaningful to different stakeholder groups can aid in maximizing benefits within the context of limited resources. We posit that such approaches are vital to advancing economic evaluation in implementation science. Finally, we identify directions for future research and application.Considering a range of stakeholders is critical to informing economic evaluation that will support appropriate decisions about resource allocation across contexts to inform decisions about successful adoption, implementation, and sustainment. Not all perspectives need to be addressed in a given project but identifying and understanding perspectives of multiple groups of key stakeholders including patients and direct implementation staff not often explicitly considered in traditional economic evaluation are needed in implementation research.

Encouraging the scale-up of proven interventions: Infrastructure development for the "Evidence-to-Implementation" award.

BACKGROUND/OBJECTIVE: Although most research universities offer investigators help in obtaining patents for inventions, investigators generally have few resources for scaling up non-patentable innovations, such as health behavior change interventions. In 2017, the dissemination and implementation (D & I) team at the University of Wisconsin's Clinical and Translational Science Award (CTSA) created the Evidence-to-Implementation (E2I) award to encourage the scale-up of proven, non-patentable health interventions. The award was intended to give investigators financial support and business expertise to prepare evidence-based interventions for scale-up. METHODS: The D & I team adapted a set of criteria named Critical Factors Assessment, which has proven effective in predicting the success of entrepreneurial ventures outside the health care environment, to use as review criteria for the program. In March 2018 and February 2020, multidisciplinary panels assessed proposals using a review process loosely based on the one used by the NIH for grant proposals, replacing the traditional NIH scoring criteria with the eight predictive factors included in Critical Factors Assessment. RESULTS: two applications in 2018 and three applications in 2020 earned awards. Funding has ended for the first two awardees, and both innovations have advanced successfully. CONCLUSION: Late-stage translation, though often overlooked by the academic community, is essential to maximizing the overall impact of the science generated by CTSAs. The Evidence-to-implementation award provides a working model for supporting late-stage translation within a CTSA environment.

A cost-benefit analysis of Wisconsin's screening, brief intervention, and referral to treatment program: adding the employer's perspective.

OBJECTIVE: A previous cost-benefit analysis found Screening, Brief Intervention, and Referral to Treatment (SBIRT) to be cost-beneficial from a societal perspective. This paper develops a cost-benefit model that includes the employer's perspective by considering the costs of absenteeism and impaired presenteeism due to problem drinking. METHODS: We developed a Monte Carlo simulation model to estimate the costs and benefits of SBIRT implementation to an employer. We first presented the likely costs of problem drinking to a theoretical Wisconsin firm that does not currently provide SBIRT services. We then constructed a cost-benefit model in which the firm funds SBIRT for its employees. The net present value of SBIRT adoption was computed by comparing costs due to problem drinking both with and without the program. RESULTS: When absenteeism and impaired presenteeism costs were considered from the employer's perspective, the net present value of SBIRT adoption was $771 per employee. CONCLUSIONS: We concluded that implementing SBIRT is cost-beneficial from the employer's perspective and recommend that Wisconsin employers consider covering SBIRT services for their employees.

Building capacity for dissemination and implementation to maximize research impact in a CTSA: The University of Wisconsin story.

We report results of an 8-year process of stakeholder engagement aimed at building capacity in Dissemination and Implementation (D&I) research at the University of Wisconsin as part of the National Institutes of Health's Clinical and Translational Science Award (CTSA). Starting in 2008, annual individual interviews were held with leaders of the Wisconsin CTSA's community engagement core for strategic planning purposes. Interviews were followed by annual planning meetings that employed a facilitated group decision-making process aimed at identifying and prioritizing gaps in the translational research spectrum. In 2011, the stakeholder engagement process identified D&I as a primary gap limiting overall impact of the institution's research across the translational spectrum. Since that time, our CTSA has created an array of D&I resources falling into four broad categories: (1) relationship building with D&I partners, (2) D&I skill building, (3) translational research resources, and (4) resources to support D&I activities. Our systematic process of stakeholder engagement has increased the impact of research by providing D&I resources to meet investigator and community needs. CTSAs could engage with leaders of their community engagement cores, which are common to all CTSAs, to adapt or adopt these resources to build D&I capacity.