Implementing a resilience coach program to support first year housestaff during the COVID-19 pandemic: early pilot results and comparison with non-housestaff sessions.

BACKGROUND: In response to the COVID-19 pandemic, we launched the Penn Medicine Coping First Aid program to provide psychosocial supports to our health system community. Our approach leveraged lay health worker volunteers trained in principles of Psychological First Aid to deliver coaching services through a centralized virtual platform. METHODS: We emailed all (n = 408) first year housestaff (i.e., residents and fellows) with an invitation to schedule a session with a resilience coach. We compared the mental health concerns, symptoms, and Psychological First Aid techniques recorded in (n = 67) first year housestaff sessions with (n = 91) sessions of other employees in the health system. RESULTS: Between June and November 2020, forty-six first year housestaff attended at least one resilience coaching session. First year housestaff most commonly presented with feelings of anxiety and sadness and shared concerns related to the availability of social support. Resilience coaches most frequently provided practical assistance and ensured safety and comfort to first year housestaff. First year housestaff reported fewer physical or mental health symptoms and held shorter sessions with resilience coaches than non-housestaff. CONCLUSIONS: This work offers insights on how to address psychosocial functioning through low-intensity interventions delivered by lay personnel. More research is needed to understand the efficacy of this program and how best to engage housestaff in wellness and resilience programs throughout training, both during and beyond COVID-19.

"There's absolutely no downside to this, I mean, except community opposition:" A qualitative study of the acceptability of vending machines for harm reduction.

BACKGROUND: Vending machines for harm reduction (VMHR) are an innovative approach to deliver life-saving materials, information, and treatment for hard-to-reach populations, particularly for persons who inject drugs. The current study explores stakeholders' perspectives on the feasibility and acceptability of VMHR in Philadelphia. METHODS: From October 2021 to February 2022, we conducted 31 semi-structured interviews with potential end users, staff, and leadership at a local federally qualified health center, and community members. Trained coders extracted themes from interview transcripts across four key domains: materials and logistics, location, access, and community introduction. RESULTS: Interviewees from all stakeholder groups endorsed using VMHR to provide supplies for wound care, fentanyl test strips, naloxone, and materials to connect individuals to treatment and other services. Dispensing syringes and medications for opioid use disorder were commonly endorsed by health center staff but were more controversial among potential end users. Even within stakeholder groups, views varied with respect to where to locate the machines, but most agreed that the machine should be placed in the highest drug use areas. Across stakeholder groups, interviewees suggested several strategies to introduce and gain community acceptance of VMHR, including community education, one-on-one conversations with community members, and coupling the machine with safe disposal of syringes and information to link individuals to treatment. CONCLUSIONS: Stakeholders were generally receptive to VMHR. The current study findings are consistent with qualitative analyses from outside of the USA and contribute new ideas regarding the anticipated community response and best methods for introducing these machines to a community. With thoughtful planning and design, VMHR could be a feasible and acceptable modality to reduce death and disease transmission associated with the opioid and HIV epidemics in cities like Philadelphia.

It's all in the name: why exposure therapy could benefit from a new one.

Exposure therapy for anxiety and related disorders is the psychological intervention with the strongest support for its efficacy and effectiveness to date. Yet, it is the least used evidence-based intervention in routine clinical practice, with a long-acknowledged public relations problem. Despite a wealth of research aimed at improving uptake of exposure, exposure's marketing and branding remains an untapped target. We first introduce principles from the marketing literature to propose that the field take steps toward a rebranding and repackaging of exposure therapy to support efforts to implement it widely. Second, we present preliminary data on clinician preferences for the use of alternative terminology developed to be more palatable and marketable - "Supported Approach of Feared Experiences - Cognitive Behavioral Therapy (SAFE-CBT)" - compared to traditional terminology. This initial survey indicated that most clinicians preferred use of the SAFE-CBT term when talking to patients, whereas only a minority preferred it for use among training clinicians. We conclude by discussing implications of these results for future efforts to implement exposure therapy more widely and set an agenda for future research in this space.

Community stakeholder preferences for evidence-based practice implementation strategies in behavioral health: a best-worst scaling choice experiment.

BACKGROUND: Community behavioral health clinicians, supervisors, and administrators play an essential role in implementing new psychosocial evidence-based practices (EBP) for patients receiving psychiatric care; however, little is known about these stakeholders' values and preferences for implementation strategies that support EBP use, nor how best to elicit, quantify, or segment their preferences. This study sought to quantify these stakeholders' preferences for implementation strategies and to identify segments of stakeholders with distinct preferences using a rigorous choice experiment method called best-worst scaling. METHODS: A total of 240 clinicians, 74 clinical supervisors, and 29 administrators employed within clinics delivering publicly-funded behavioral health services in a large metropolitan behavioral health system participated in a best-worst scaling choice experiment. Participants evaluated 14 implementation strategies developed through extensive elicitation and pilot work within the target system. Preference weights were generated for each strategy using hierarchical Bayesian estimation. Latent class analysis identified segments of stakeholders with unique preference profiles. RESULTS: On average, stakeholders preferred two strategies significantly more than all others-compensation for use of EBP per session and compensation for preparation time to use the EBP (P < .05); two strategies were preferred significantly less than all others-performance feedback via email and performance feedback via leaderboard (P < .05). However, latent class analysis identified four distinct segments of stakeholders with unique preferences: Segment 1 (n = 121, 35%) strongly preferred financial incentives over all other approaches and included more administrators; Segment 2 (n = 80, 23%) preferred technology-based strategies and was younger, on average; Segment 3 (n = 52, 15%) preferred an improved waiting room to enhance client readiness, strongly disliked any type of clinical consultation, and had the lowest participation in local EBP training initiatives; Segment 4 (n = 90, 26%) strongly preferred clinical consultation strategies and included more clinicians in substance use clinics. CONCLUSIONS: The presence of four heterogeneous subpopulations within this large group of clinicians, supervisors, and administrators suggests optimal implementation may be achieved through targeted strategies derived via elicitation of stakeholder preferences. Best-worst scaling is a feasible and rigorous method for eliciting stakeholders' implementation preferences and identifying subpopulations with unique preferences in behavioral health settings.

A pilot study of participatory and rapid implementation approaches to increase depression screening in primary care.

BACKGROUND: Most individuals with depression go unidentified and untreated. In 2016 the US Preventive Services Task Force released guidelines recommending universal screening in primary care to identify patients with depression and to link them to treatment. Feasible, acceptable, and effective strategies to implement these guidelines are needed. METHODS: This three-phased study employed rapid participatory methods to design and test strategies to increase depression screening at Penn Medicine, a large health system with 90 primary care practices. First, researchers solicited ideas and barriers from stakeholders to increase screening using an innovation tournament-a crowdsourcing method that invites stakeholders to submit ideas to address a workplace challenge. Second, a panel of stakeholders and scientists deliberated over and ranked the tournament ideas. An instant runoff election was held to select the winning idea. Third, the research team piloted the winning idea in a primary care practice using rapid prototyping, an approach that quickly refines and iterates strategy designs. RESULTS: The innovation tournament yielded 31 ideas and 32 barriers from diverse stakeholders (12 primary care physicians, 10 medical assistants, 4 nurse practitioners, 2 practice managers, and 4 patient support assistants). A panel of 6 stakeholders and scientists deliberated on the ideas and voted for patient self-report (i.e., through tablet computers, text message, or an online patient portal) as the winning idea. The research team rapid prototyped tablets in one primary care practice with one physician over 5 five-hour shifts to examine the feasibility, acceptability, and effectiveness of the strategy. Most patients, the physician, and medical assistants found the tablets acceptable and feasible. However, patient support assistants struggled to incorporate them in their workflow and expressed concerns about scaling up the process. Depression screening rates were higher using tablets compared to usual care; follow-up was comparable between tablets and usual care. CONCLUSIONS: Rapid participatory methods engaged and amplified the voices of diverse stakeholders in primary care. These methods helped design an acceptable and feasible implementation strategy that showed promise for increasing depression screening in a primary care setting. The next step is to evaluate the strategy in a randomized controlled trial across primary care practices.

Applying NUDGE to Inform Design of EBP Implementation Strategies in Community Mental Health Settings.

We demonstrate the application of NUDGE (Narrow, Understand, Discover, Generate, Evaluate), a behavioral economics approach to systematically identifying behavioral barriers that impede behavior enactment, to the challenge of evidence-based practice (EBP) use in community behavioral health. Drawing on 65 clinician responses to a system-wide crowdsourcing challenge about EBP underutilization, we applied NUDGE to discover, synthesize and validate specific behavioral barriers to EBP utilization that directly inform the design of tailored implementation strategies. To our knowledge, this is the first study to apply behavioral economic insights to clinician-proposed solutions to implementation challenges in order to design implementation strategies. The study demonstrates the successful application of NUDGE to implementation strategy design and provides novel targets for intervention.

Evidence Base Update for Brief, Free, and Accessible Youth Mental Health Measures.

Evidence-based assessment (EBA) is foundational to high-quality mental health care for youth and is a critical component of evidence-based practice delivery, yet is underused in the community. Administration time and measure cost are barriers to use; thus, identifying and disseminating brief, free, and accessible measures are critical. This Evidence Base Update evaluates the empirical literature for brief, free, and accessible measures with psychometric support to inform research and practice with youth. A systematic review using PubMed and PsycINFO identified measures in the following domains: overall mental health, anxiety, depression, disruptive behavior, traumatic stress, disordered eating, suicidality, bipolar/mania, psychosis, and substance use. To be eligible for inclusion, measures needed to be brief (50 items or less), free, accessible, and have psychometric support for their use with youth. Eligible measures were evaluated using adapted criteria established by De Los Reyes and Langer (2018) and were classified as having excellent, good, or adequate psychometric properties. A total of 672 measures were identified; 95 (14%) met inclusion criteria. Of those, 21 (22%) were "excellent," 34 (36%) were "good," and 40 (42%) were "adequate." Few measures had support for their use to routinely monitor progress in therapy. Few measures with excellent psychometric support were identified for disordered eating, suicidality, psychosis, and substance use. Future research should evaluate existing measures for use with routine progress monitoring and ease of implementation in community settings. Measure development is needed for disordered eating, suicidality, psychosis, and substance use to increase availability of brief, free, accessible, and validated measures.

Applying the Policy Ecology Framework to Philadelphia's Behavioral Health Transformation Efforts.

Raghavan et al. (Implement Sci 3(26):1-9, 2008) proposed that effective implementation of evidence-based practices requires implementation strategies deployed at multiple levels of the "policy ecology," including the organizational, regulatory or purchaser agency, political, and social levels. However, much of implementation research and practice targets providers without accounting for contextual factors that may influence provider behavior. This paper examines Philadelphia's efforts to work toward an evidence-based and recovery-oriented behavioral health system, and uses the policy ecology framework to illustrate how multifaceted, multilevel implementation strategies can facilitate the widespread implementation of evidence-based practices. Ongoing challenges and implications for research and practice are discussed.

A Multi-Level Examination of Stakeholder Perspectives of Implementation of Evidence-Based Practices in a Large Urban Publicly-Funded Mental Health System.

Our goal was to identify barriers and facilitators to the implementation of evidence-based practices from the perspectives of multiple stakeholders in a large publicly funded mental health system. We completed 56 interviews with three stakeholder groups: treatment developers (n = 7), agency administrators (n = 33), and system leadership (n = 16). The three stakeholder groups converged on the importance of inner (e.g., agency competing resources and demands, therapist educational background) and outer context (e.g., funding) factors as barriers to implementation. Potential threats to implementation and sustainability included the fiscal landscape of community mental health clinics and an evolving workforce. Intervention characteristics were rarely endorsed as barriers. Inner context, outer context, and intervention characteristics were all seen as important facilitators. All stakeholders endorsed the importance of coordinated collaboration across stakeholder groups within the system to successfully implement evidence-based practices.

Free, brief, and validated: Standardized instruments for low-resource mental health settings.

Evidence-based assessment has received little attention despite its critical importance to the evidence-based practice movement. Given the limited resources in the public sector, it is necessary for evidence-based assessment to utilize tools with established reliability and validity metrics that are free, easily accessible, and brief. We review tools that meet these criteria for youth and adult mental health for the most prevalent mental health disorders to provide a clinical guide and reference for the selection of assessment tools for public sector settings. We also discuss recommendations for how to move forward the evidence-based assessment agenda.

Perspectives of substance use disorder counselors on the benefits and drawbacks of medications for opioid use disorder.

Background: Medications for opioid use disorder (MOUD) are among the best tools we have to combat the opioid epidemic. Yet, use of MOUD among people with opioid use disorder (OUD) remains low. Interventions to increase MOUD access in the United States have largely focused on improving organizational capacity and addressing funding barriers, yet stigma toward MOUD may inhibit uptake even where MOUD is readily available. Substance use disorder (SUD) treatment counselors likely have considerable influence on a client's choice to initiate and adhere to MOUD, but beliefs that counselors convey about MOUD in interaction with clients are understudied. The current study explores what advantages and disadvantages that SUD treatment counselors communicate about buprenorphine, methadone, and naltrexone. Methods: From June to December 2021, we surveyed counselors from publicly-funded SUD treatment agencies under a municipality-wide mandate to offer MOUD to all clients with OUD. Counselors were asked to describe, in a free-response format, the most important advantages and disadvantages to communicate to their clients about taking buprenorphine, methadone, and naltrexone. Counselor responses were coded for one or more advantage and disadvantage. Results: A total of 271 SUD counselors from 29 agencies completed the survey, generating 1,995 advantages and disadvantages across three types of MOUD. The most frequently reported advantage across all three types of MOUD was their ability to reduce cravings and illicit drug use. The most frequently reported disadvantage related to the potential for some types of MOUD to develop long-term medication dependence. Conclusions: As the availability and variety of MOUD treatment options continue to expand, it is important that SUD counselors are equipped with evidence-based recommendations for OUD care. We identified misalignments with the MOUD-prescribing evidence base and stigmatizing language toward MOUD within counselors' responses, highlighting the potential to refine training materials for MOUD and mitigate stigmatizing beliefs.

A systematic review of brief, freely accessible, and valid self-report measures for substance use disorders and treatment.

BACKGROUND: Self-report measures can improve evidence-based assessment practices in substance use disorder treatment, but many measures are burdensome and costly, limiting their utility in community practice and non-specialty healthcare settings. This systematic review identified and evaluated the psychometric properties of brief, free, and readily accessible self-report measures of substance use and related factors. METHODS: We searched two electronic databases (PsycINFO and PubMed) in May 2021 for published literature on scales, measures, or instruments related to substance use, substance use treatment, and recovery, and extracted the names of all measures. Measures were included if they were: (1) brief (25 items or fewer), (2) freely accessible in a ready-to-use format, and (3) had published psychometric data. RESULTS: An initial search returned 411 measures, of which 73 (18%) met criteria for inclusion. Included measures assessed a variety of substances (e.g., alcohol, nicotine, opioids, cannabinoids, cocaine) and measurement domains (e.g., use, severity, expectancies, withdrawal). Among these measures, 14 (19%) were classified as psychometrically "excellent," 27 (37%) were rated as "good," 32 (44%) were "adequate." CONCLUSIONS: Despite the shift toward evidence-based assessment in substance use disorder treatment in the last twenty years, key areas of public health concern are lacking pragmatic, psychometrically valid measures. Among the brief measures we reviewed, less than a fifth met criteria for psychometric "excellence" and most of these instruments fell into one measurement domain: screening for problematic substance use. Future research should focus both on improving the evidence base for existing brief self-report measures and creating new low-burden measures for specific substances and treatment constructs.

Not in my treatment center: Leadership's perception of barriers to MOUD adoption.

INTRODUCTION: Despite their well-established effectiveness, medications for opioid use disorder (MOUD) are widely underutilized across the United States. In the context of a large publicly funded behavioral health system, we examined the relationship between a range of implementation barriers and a substance use disorder treatment agency's level of adoption of MOUD. METHODS: We surveyed leadership of publicly funded substance use disorder treatment centers in Philadelphia about the significance of barriers to implementing MOUD related to their workforce, organization, funding, regulations, and beliefs about MOUD's efficacy and safety. We queried leaders on the percentage of their patients with opioid use disorder who receive MOUD and examined associations between implementation barriers and MOUD adoption. RESULTS: Ratings of regulatory, organizational, or funding barriers of respondents who led high MOUD adopting agencies (N = 20) were indistinguishable from those who led agencies that were low adopting of MOUD (N = 23). In contrast, agency leaders who denied MOUD-belief or workforce barriers were significantly more likely to lead high-MOUD-adopting organizations. CONCLUSIONS: These findings suggest that leadership beliefs about MOUD may be a key factor of the organizational decision to adopt and should be a target of implementation efforts to increase direct provision of these medications.

Where do psychologists turn to inform clinical decisions? Audience segmentation to guide dissemination strategies.

Background: Audience segmentation is an analysis technique that can identify meaningful subgroups within a population to inform the tailoring of dissemination strategies. We have conducted an empirical clustering audience segmentation study of licensed psychologists using survey data about the sources of knowledge they report most often consulting to guide their clinical decision-making. We identify meaningful subgroups within the population and inform the tailoring of dissemination strategies for evidence-based practice (EBP) materials. Method: Data come from a 2018-2019 web-based survey of licensed psychologists who were members of the American Psychological Association (APA; N = 518, response rate = 29.8%). Ten dichotomous variables assessed sources that psychologists regularly consult to inform clinical decision-making (e.g., colleagues, academic literature, and practice guidelines). We used latent class analysis to identify segments of psychologists who turn to similar sources and named each segment based on the segment's most salient characteristics. Results: Four audience segments were identified: the No-guidelines (45% of psychologists), Research-driven (16%), Thirsty-for-knowledge (9%), and No-reviews (30%). The four segments differed not only in their preferred sources of knowledge, but also in the types of evidence-based posttraumatic stress disorder (PTSD) treatments they provide, their awareness and usage intention of the APA PTSD clinical practice guideline, and attitudes toward clinical practice guidelines. Conclusion: The results demonstrate that licensed psychologists are heterogeneous in terms of their knowledge-seeking behaviors and preferences for knowledge sources. The distinctive characteristics of these segments could guide the tailoring of dissemination materials and strategies to subsequently enhance the implementation of EBP among psychologists.

Audience segmentation is a dissemination strategy that categorizes a group of intended users or audience into meaningful subgroups based on their beliefs, behaviors, and/or other characteristics. Like many other scientific or medical fields, clinical psychology also struggles to use clinically tested psychological treatments (or EBPs) in everyday practice due to practical challenges. To help address such barriers, professional organizations like the American Psychological Association (APA) publish clinical practice guidelines that practitioners can use to learn more about EBPs. However, even these clinical practice guidelines are not often used, so this study employed the audience segmentation analysis to better understand psychologists' diverse attitudes, behaviors, and preferences regarding clinical practice guidelines and other clinical information sources. Our study found four distinct subgroups within approximately 600 APA-registered psychologists based on their preferred source of knowledge: the no-guidelines (45% of psychologists), research-driven (16%), thirsty-for-knowledge (9%), and no-reviews (30%). Each subgroup also varied in the types of evidence-based treatments they provide, as well as their awareness, willingness to use, and attitudes toward clinical practice guidelines. This result shows that licensed psychologists are not a uniform group and that dissemination strategies should be adjusted to each subgroup's characteristics to maximize the effort to increase the use of EBPs among psychologists.

Mobile service delivery in response to the opioid epidemic in Philadelphia.

BACKGROUND: The harms of opioid use disorder (OUD) and HIV infection disproportionately impact marginalized populations, especially people experiencing homelessness and people who inject drugs (PWID). Mobile OUD service delivery models are emerging to increase access and reduce barriers to OUD care. While there is growing interest in these models, there is limited research about the services they provide, how they operate, and what barriers they face. We characterize the capacity, barriers, and sustainment of mobile OUD care services in a large city with a high incidence of OUD and HIV. METHODS: From May to August 2022, we conducted semi-structured interviews with leadership from all seven mobile OUD care units (MOCU) providing a medication for OUD or other substance use disorder services in Philadelphia. We surveyed leaders about their unit's services, staffing, operating location, funding sources, and linkages to care. Leaders were asked to describe their clinical approach, treatment process, and the barriers and facilitators to their operations. Interview recordings were coded using rapid qualitative analysis. RESULTS: MOCUs are run by small, multidisciplinary teams, typically composed of a clinician, one or two case managers, and a peer recovery specialist or outreach worker. MOCUs provide a range of services, including medications for OUD, wound care, medical services, case management, and screening for infectious diseases. No units provide methadone, but all units provide naloxone, six write prescriptions for buprenorphine, and one unit dispenses buprenorphine. The most frequently reported barriers include practical challenges of working on a MOCU (e.g. lack of space, safety), lack of community support, and patients with substantial medical and psychosocial needs. Interviewees reported concerns about funding and specifically as it relates to providing their staff with adequate pay. The most frequently reported facilitators include positive relationships with the community, collaboration with other entities (e.g. local nonprofits, the police department, universities), and having non-clinical staff (e.g. outreach workers, peer recovery specialists) on the unit. CONCLUSIONS: MOCUs provide life-saving services and engage marginalized individuals with OUD. These findings highlight the challenges and complexities of caring for PWID and demonstrate a need to strengthen collaborations between MOCU providers and the treatment system. Policymakers should consider programmatic funding for permanent mobile OUD care services.

Theoretical and practical barriers to practitioners' willingness to seek training in empirically supported treatments.

To identify barriers to the dissemination of empirically supported treatments (ESTs), a random sample of psychologists in independent practice (N=1291) was asked to complete measures of attitudes towards ESTs and willingness to attend a 3-hour, 1-day, or 3-day workshop in an EST of their choice. The strongest predictor of unwillingness to obtain EST training was the amount of time and cost required for the workshop, followed by objections to the need for EST training. Psychodynamic (compared to cognitive-behavioral) and more experienced practitioners agreed more strongly with the objections to ESTs overall, as did those whose graduate schools had not emphasized psychotherapy research. Results suggest that both practical and theoretical barriers are significant obstacles to EST dissemination.

A Qualitative Investigation of Practicing Psychologists' Attitudes Toward Research-Informed Practice: Implications for Dissemination Strategies.

This article presents the results of a qualitative analysis of interviews with 25 psychologists in independent practice, investigating everyday treatment decisions and attitudes about treatment outcome research and empirically supported treatments (ESTs). Clinicians noted positive aspects about treatment outcome research, such as being interested in what works. However, they had misgivings about the application of controlled research findings to their practices, were skeptical about using manualized protocols, and expressed concern that nonpsychologists would use EST lists to dictate practice. Clinicians reported practicing in an eclectic framework, and many reported including cognitive-behavioral elements in their practice. To improve their practice, they reported valuing clinical experience, peer networks, practitioner-oriented books, and continuing education when it was not too basic. Time and financial barriers concerned nearly all participants. Clinicians suggested they might be interested in ESTs if they could integrate them into their current frameworks, and if resources for learning ESTs were improved.

Variation in Stakeholder Preferences for Implementing Evidence-Based Practices in Behavioral Health Care.

Objective: The authors examined whether stakeholders in behavioral health care differ in their preferences for strategies that support the implementation of evidence-based practices (EBPs). Methods: Using data collected in March and April 2019 in a survey of stakeholders in Philadelphia Medicaid's behavioral health care system, the authors compared empirical Bayes preference weights for implementation strategies across clinicians, supervisors, agency executives, and payers. Results: Preferences for implementation strategies overlapped among the stakeholders (N=357 survey respondents). Financial incentives were consistently ranked as most useful and performance feedback as the least useful for implementing EBPs. However, areas of divergence were identified. For example, payers preferred compensation for EBP delivery, whereas clinicians considered compensation for time spent on preparing for EBPs as equally useful. Conclusions: The observed variation in stakeholder preferences for strategies to implement EBPs may shed light on why the ongoing shift from volume to value in behavioral health care has had mixed results.

Who gets coached? A qualitative inquiry into community clinicians' decisions to use caregiver coaching.

LAY ABSTRACT: Providers' beliefs about an intervention's fit with a family can affect whether or not they use that intervention with a family. The factors that affect providers' decisions to use evidence-based practices for young autistic children have not been studied. These factors may play a role in the major differences we see in the quality of and access to early intervention services in the community. We looked at differences in providers' use of caregiver coaching, an evidence-based practice, with families from minority or vulnerable backgrounds, and the possible reasons for those differences. We did this to figure out what factors affect providers' use of caregiver coaching. We interviewed 36 early intervention providers from early intervention agencies in two different parts of the United States. Providers pointed out things like what they thought about a family's circumstances that affected their beliefs about how well coaching fits with minority and vulnerable families. Our findings bring attention to these beliefs that likely make accessing evidence-based practices for minority and vulnerable families harder and lessen the quality of care for these families of young autistic children. These findings highlight the need to come up with and use strategies to improve both access to and the quality of evidence-based practices for young autistic children from minority and vulnerable groups.

Comparing output from two methods of participatory design for developing implementation strategies: traditional contextual inquiry vs. rapid crowd sourcing.

BACKGROUND: Participatory design methods are a key component of designing tailored implementation strategies. These methods vary in the resources required to execute and analyze their outputs. No work to date has examined the extent to which the output obtained from different approaches to participatory design varies. METHODS: We concurrently used two separate participatory design methods: (1) field observations and qualitative interviews (i.e., traditional contextual inquiry) and (2) rapid crowd sourcing (an innovation tournament). Our goal was to generate and compare information to tailor implementation strategies to increase the use of evidence-based data collection practices among one-to-one aides working with children with autism. Each method was executed and analyzed by study team members blinded to the output of the other method. We estimated the personnel time and monetary costs associated with each method to further facilitate comparison. RESULTS: Observations and interviews generated nearly double the number of implementation strategies (n = 26) than did the innovation tournament (n = 14). When strategies were classified into implementation strategies from the Expert Recommendations for Implementing Change (ERIC) taxonomy, there was considerable overlap in the content of identified strategies. However, strategies derived from observations and interviews were more specific than those from the innovation tournament. Five strategies (13%) reflected content unique to observations and interviews and 3 (8%) strategies were unique to the innovation tournament. Only observations and interviews identified implementation strategies related to adapting and tailoring to context; only the innovation tournament identified implementation strategies that used incentives. Observations and interviews required more than three times the personnel hours than the innovation tournament, but the innovation tournament was more costly overall due to the technological platform used. CONCLUSIONS: There was substantial overlap in content derived from observations and interviews and the innovation tournament, although there was greater specificity in the findings from observations and interviews. However, the innovation tournament yielded unique information. To select the best participatory design approach to inform implementation strategy design for a particular context, researchers should carefully consider unique advantages of each method and weigh the resources available to invest in the process.