Essential content for teaching implementation practice in healthcare: a mixed-methods study of teams offering capacity-building initiatives.

BACKGROUND: Applying the knowledge gained through implementation science can support the uptake of research evidence into practice; however, those doing and supporting implementation (implementation practitioners) may face barriers to applying implementation science in their work. One strategy to enhance individuals' and teams' ability to apply implementation science in practice is through training and professional development opportunities (capacity-building initiatives). Although there is an increasing demand for and offerings of implementation practice capacity-building initiatives, there is no universal agreement on what content should be included. In this study we aimed to explore what capacity-building developers and deliverers identify as essential training content for teaching implementation practice. METHODS: We conducted a convergent mixed-methods study with participants who had developed and/or delivered a capacity-building initiative focused on teaching implementation practice. Participants completed an online questionnaire to provide details on their capacity-building initiatives; took part in an interview or focus group to explore their questionnaire responses in depth; and offered course materials for review. We analyzed a subset of data that focused on the capacity-building initiatives' content and curriculum. We used descriptive statistics for quantitative data and conventional content analysis for qualitative data, with the data sets merged during the analytic phase. We presented frequency counts for each category to highlight commonalities and differences across capacity-building initiatives. RESULTS: Thirty-three individuals representing 20 capacity-building initiatives participated. Study participants identified several core content areas included in their capacity-building initiatives: (1) taking a process approach to implementation; (2) identifying and applying implementation theories, models, frameworks, and approaches; (3) learning implementation steps and skills; (4) developing relational skills. In addition, study participants described offering applied and pragmatic content (e.g., tools and resources), and tailoring and evolving the capacity-building initiative content to address emerging trends in implementation science. Study participants highlighted some challenges learners face when acquiring and applying implementation practice knowledge and skills. CONCLUSIONS: This study synthesized what experienced capacity-building initiative developers and deliverers identify as essential content for teaching implementation practice. These findings can inform the development, refinement, and delivery of capacity-building initiatives, as well as future research directions, to enhance the translation of implementation science into practice.

Planning and implementing practice changes in Ontario maternal-newborn hospital units: a secondary qualitative analysis.

BACKGROUND: Moving evidence into practice is complex, and pregnant and birthing people and their infants do not always receive care that aligns with the best available evidence. Implementation science can inform how to effectively move evidence into practice. While there are a growing number of examples of implementation science being studied in maternal-newborn care settings, it remains unknown how real-world teams of healthcare providers and leaders approach the overall implementation process when making practice changes. The purpose of this study was to describe maternal-newborn hospital teams' approaches to implementing practice changes. We aimed to identify what implementation steps teams take (or not) and identify strengths and potential areas for improvement based on best practices in implementation science. METHODS: We conducted a supplementary qualitative secondary analysis of 22 interviews completed in 2014-2015 with maternal-newborn nursing leaders in Ontario, Canada. We used directed content analysis to code the data to seven steps in an implementation framework (Implementation Roadmap): identify the problem and potential best practice; assemble local evidence; select and customize best practice; discover barriers and drivers; tailor implementation strategies; field-test, plan evaluation, prepare to launch; launch, evaluate, and sustain. Frequency counts are presented for each step. RESULTS: Participants reported completing a median of 4.5 of 7 Implementation Roadmap steps (range = 3-7), with the most common being identifying a practice problem. Other steps were described less frequently (e.g., selecting and adapting evidence, field-testing, outcome evaluation) or discussed frequently but not optimally (e.g., barriers assessment). Participants provided examples of how they engaged point-of-care staff throughout the implementation process, but provided fewer examples of engaging pregnant and birthing people and their families. Some participants stated they used a formal framework or process to guide their implementation process, with the most common being quality improvement approaches and tools. CONCLUSIONS: We identified variability across the 22 hospitals in the implementation steps taken. While we observed many strengths, we also identified areas where further support may be needed. Future work is needed to create opportunities and resources to support maternal-newborn healthcare providers and leaders to apply principles and tools from implementation science to their practice change initiatives.

Evaluating the quality of research co-production: Research Quality Plus for Co-Production (RQ + 4 Co-Pro).

BACKGROUND: Co-production is an umbrella term used to describe the process of generating knowledge through partnerships between researchers and those who will use or benefit from research. Multiple advantages of research co-production have been hypothesized, and in some cases documented, in both the academic and practice record. However, there are significant gaps in understanding how to evaluate the quality of co-production. This gap in rigorous evaluation undermines the potential of both co-production and co-producers. METHODS: This research tests the relevance and utility of a novel evaluation framework: Research Quality Plus for Co-Production (RQ + 4 Co-Pro). Following a co-production approach ourselves, our team collaborated to develop study objectives, questions, analysis, and results sharing strategies. We used a dyadic field-test design to execute RQ + 4 Co-Pro evaluations amongst 18 independently recruited subject matter experts. We used standardized reporting templates and qualitative interviews to collect data from field-test participants, and thematic assessment and deliberative dialogue for analysis. Main limitations include that field-test participation included only health research projects and health researchers and this will limit perspective included in the study, and, that our own co-production team does not include all potential perspectives that may add value to this work. RESULTS: The field test surfaced strong support for the relevance and utility of RQ + 4 Co-Pro as an evaluation approach and framework. Research participants shared opportunities for fine-tuning language and criteria within the prototype version, but also, for alternative uses and users of RQ + 4 Co-Pro. All research participants suggested RQ + 4 Co-Pro offered an opportunity for improving how co-production is evaluated and advanced. This facilitated our revision and publication herein of a field-tested RQ + 4 Co-Pro Framework and Assessment Instrument. CONCLUSION: Evaluation is necessary for understanding and improving co-production, and, for ensuring co-production delivers on its promise of better health.. RQ + 4 Co-Pro provides a practical evaluation approach and framework that we invite co-producers and stewards of co-production-including the funders, publishers, and universities who increasingly encourage socially relevant research-to study, adapt, and apply.

What do Speech-Language Pathologists want to know when assessing early vocal development in children who are deaf/hard-of-hearing?

PURPOSE: Delays in vocal development are an early predictor of ongoing language difficulty for children who are deaf/hard-of-hearing (CDHH). Despite the importance of monitoring early vocal development in clinical practice, there are few suitable tools. This study aimed to identify the clinical decisions that speech-language pathologists (SLPs) most want to make when assessing vocal development and their current barriers to doing so. METHOD: 58 SLPs who provide services to CDHH younger than 22 months completed a survey. The first section measured potential barriers to vocal development assessment. The second section asked SLPs to rate the importance of 15 clinical decisions they could make about vocal development. RESULTS: SLPs believed assessing vocal development was important for other stakeholders, and reported they had the necessary skills and knowledge to assess vocal development. Barriers primarily related to a lack of commercially available tests. SLPs rated all 15 clinical decisions as somewhat or very important. Their top 5 decisions included a variety of assessment purposes that tests are not typically designed to support, including measuring change, differential diagnosis, and goal setting. CONCLUSIONS: SLPs wish to make a number of clinical decisions when assessing vocal development in CDHH but lack access to appropriate tools to do so. Future work is needed to develop tools that are statistically equipped to fulfill these purposes. Understanding SLPs' assessment purposes will allow future tests to better map onto the clinical decisions that SLPs need to make to support CDHH and their families and facilitate implementation into clinical practice.

Evaluating research co-production: protocol for the Research Quality Plus for Co-Production (RQ+ 4 Co-Pro) framework.

BACKGROUND: Research co-production is an umbrella term used to describe research users and researchers working together to generate knowledge. Research co-production is used to create knowledge that is relevant to current challenges and to increase uptake of that knowledge into practice, programs, products, and/or policy. Yet, rigorous theories and methods to assess the quality of co-production are limited. Here we describe a framework for assessing the quality of research co-production-Research Quality Plus for Co-Production (RQ+ 4 Co-Pro)-and outline our field test of this approach. METHODS: Using a co-production approach, we aim to field test the relevance and utility of the RQ+ 4 Co-Pro framework. To do so, we will recruit participants who have led research co-production projects from the international Integrated Knowledge Translation Research Network. We aim to sample 16 to 20 co-production project leads, assign these participants to dyadic groups (8 to 10 dyads), train each participant in the RQ+ 4 Co-Pro framework using deliberative workshops and oversee a simulation assessment exercise using RQ+ 4 Co-Pro within dyadic groups. To study this experience, we use a qualitative design to collect participant demographic information and project demographic information and will use in-depth semi-structured interviews to collect data related to the experience each participant has using the RQ+ 4 Co-Pro framework. DISCUSSION: This study will yield knowledge about a new way to assess research co-production. Specifically, it will address the relevance and utility of using RQ+ 4 Co-Pro, a framework that includes context as an inseparable component of research, identifies dimensions of quality matched to the aims of co-production, and applies a systematic and transferable evaluative method for reaching conclusions. This is a needed area of innovation for research co-production to reach its full potential. The findings may benefit co-producers interested in understanding the quality of their work, but also other stewards of research co-production. Accordingly, we undertake this study as a co-production team representing multiple perspectives from across the research enterprise, such as funders, journal editors, university administrators, and government and health organization leaders.

When it's not appropriate to adapt: Toddlers' learning of novel speech patterns is affected by visual information.

In adults, perceptual learning for speech is constrained, such that learning of novel pronunciations is less likely to occur if the (e.g., visual) context indicates that they are transient. However, adults have had a lifetime of experience with the types of cues that signal stable vs. transient speech variation. We ask whether visual context affects toddlers' learning of a novel speech pattern. Across conditions, 19-month-olds (N = 117) were exposed to familiar words either pronounced typically or in a novel, consonant-shifting accent. During exposure, some toddlers heard the accented pronunciations without a face present; others saw a video of the speaker producing the words with a lollipop against her cheek or in her mouth. Toddlers showed the weakest learning of the accent when the speaker had the lollipop in her mouth, suggesting that they treated the lollipop as the cause of the atypical pronunciations. These results demonstrate that toddlers' adaptation to a novel speech pattern is influenced by extra-linguistic context.

Adopting a Conceptual Validity Framework for Testing in Speech-Language Pathology.

Purpose Limited evidence-based guidelines for test selection continue to result in inconsistency in test use and interpretation in speech-language pathology. A major barrier is the lack of explicit and consistent adoption of a validity framework by our field. In this viewpoint, we argue that adopting the conceptual validity framework in the Standards for Educational and Psychological Testing (American Educational Research Association et al., 2014) would support both the development of more meaningful and feasible clinical tests and more appropriate use and interpretation of tests in speech-language pathology. Method We describe and evaluate the Standards for Educational and Psychological Testing (American Educational Research Association et al., 2014) validity framework and consider its relevance to speech-language pathology. We describe how the validity framework could be integrated into clinical practice and include examples of how it could be applied to support common clinical decisions. We evaluate the costs and benefits of adopting this framework, from the perspectives of speech-language pathologists, clients, and test developers. Results The Standards' validity framework clarifies complex validity issues by shifting the focus of validity from tests to the decisions speech-language pathologists make based on test results. By focusing on decisions, the framework requires critical evaluation of test use, rather than evaluating tests against sets of criteria. Adopting this framework has the potential for appreciable improvement in the way tests are used and valued across our profession. Conclusions Speech-language pathologists, test developers, and clients will benefit from improved evidence-based assessment practices. It is recommended that regulators, test developers, professional associations, universities, and researchers adopt the framework and endorse it as best practice moving forward. This viewpoint proposes a series of first steps toward supporting uptake of the framework into research and practice.

Validity Evidence for the LittlEARS Early Speech Production Questionnaire: An English-Speaking, Canadian Sample.

Purpose This study reports validity evidence for an English translation of the LittlEARS Early Speech Production Questionnaire (LEESPQ). The LEESPQ was designed to support early spoken language outcome monitoring in young children who are deaf/hard of hearing. Methods Data from 90 children with normal hearing, ages 0-18 months, are reported. Parents completed the LEESPQ in addition to a concurrent measure of spoken language development, the Receptive-Expressive Emergent Language Test-Third Edition. Normal hearing status and development were confirmed. Results Traditional scale analyses, in addition to item parameters, are reported. The LEESPQ was highly correlated with the Receptive-Expressive Emergent Language Test-Third Edition (r = .92) and age (r = .90) and had high internal consistency (Ω = 0.92). Common factor analysis revealed 2 underlying factors conceptually mapping onto items measuring vocal and symbolic development. A latent traits model was the best fit to the data, and item difficulty broadly conformed to theoretical expectations. Conclusions The present work demonstrates that the LEESPQ accurately captures early spoken language development in a typically developing group of young children. The LEESPQ holds promise as a clinically feasible, spoken language outcome monitoring tool. Future work to identify differences in performance characteristics between typically developing children and clinical populations is warranted.

A Comment on Test Validation: The Importance of the Clinical Perspective.

Purpose The misuse of standardized assessments has been a long-standing concern in speech-language pathology and traditionally viewed as an issue of clinician competency and training. The purpose of this article is to consider the contribution of communication breakdowns between test developers and the end users to this issue. Method We considered the misuse of standardized assessments through the lens of the 2-communities theory, in which standardized tests are viewed as a product developed in 1 community (researchers/test developers) to be used by another community (frontline clinicians). Under this view, optimal test development involves a conversation to which both parties bring unique expertise and perspectives. Results Consideration of the interpretations that standardized tests are typically validated to support revealed a mismatch between these and the interpretations and decisions that speech-language pathologists typically need to make. Test development using classical test theory, which underpins many of the tests in our field, contributes to this mismatch. Application of item response theory could better equip clinicians with the psychometric evidence to support the interpretations they desire but is not commonly found in the standardized tests used by speech-language pathologists. Conclusions Advocacy and insistence on the consideration of clinical perspectives and decision making in the test validation process is a necessary part of our role. In improving the nature of the statistical evidence reported in standardized assessments, we can ensure these tools are appropriate to fulfill our professional obligations in a clinically feasible way.

Barriers to implementing evidence-based assessment procedures: Perspectives from the front lines in pediatric speech-language pathology.

PURPOSE: This practice-based research study was a collaborative effort between researchers and speech-language pathologists (SLPs). The purpose of the study was to assess SLPs' perceptions of barriers for two new evidence-based assessment procedures to be implemented in practice. Procedures were for (1) program-level outcome monitoring and (2) individual vulnerability testing for children who are deaf or hard of hearing. These procedures were summarized for clinicians in an online learning module. METHOD: After finishing the online learning module, SLPs completed electronic surveys to identify perceived barriers to implementation. Fifty-four SLPs completed an online survey specific to the program-level outcome monitoring procedures. Twenty-five also completed an online survey specific to individual vulnerability testing. Surveys were structured using the Ottawa Model of Research Use, which assesses barriers within three components: (1) the practice environment; (2) clinicians' knowledge, skills, and beliefs; and (3) the evidence-based innovation (development and content of the online learning module). RESULTS: Most items specific to program-level outcome monitoring were rated positively. Some barriers were identified within the practice environment and evidence-based innovation, but clinicians' skills, knowledge, and beliefs were not barriers to implementation. For individual vulnerability testing, barriers were noted across all components of the Ottawa Model of Research Use. CONCLUSIONS: This work demonstrates the value of including front-line clinicians in the development of evidence-based assessment procedures. Through practice-based research, we identified SLPs' perceived barriers to implementation, allowing for modifications to be made to the online learning module prior to wider-scale implementation. The barriers identified and methods used in our work may be useful to other researchers and programs involved in developing materials and methods for the implementation of evidence-based assessment procedures.

Language Outcomes in Children Who Are Deaf and Hard of Hearing: The Role of Language Ability Before Hearing Aid Intervention.

Purpose: Early auditory experiences are fundamental in infant language acquisition. Research consistently demonstrates the benefits of early intervention (i.e., hearing aids) to language outcomes in children who are deaf and hard of hearing. The nature of these benefits and their relation with prefitting development are, however, not well understood. Method: This study examined Ontario Infant Hearing Program birth cohorts to explore predictors of performance on the Preschool Language Scale-Fourth Edition at the time of (N = 47) and after (N = 19) initial hearing aid intervention. Results: Regression analyses revealed that, before the hearing aid fitting, severity of hearing loss negatively predicted 19% and 10% of the variance in auditory comprehension and expressive communication, respectively. After hearing aid fitting, children's standard scores on language measures remained stable, but they made significant improvement in their progress values, which represent individual skills acquired on the test, rather than standing relative to same-age peers. Magnitude of change in progress values was predicted by a negative interaction of prefitting language ability and severity of hearing loss for the Auditory Comprehension scale. Conclusions: These findings highlight the importance of considering a child's prefitting language ability in interpreting eventual language outcomes. Possible mechanisms of hearing aid benefit are discussed. Supplemental Materials: https://doi.org/10.23641/asha.5538868.