Does spaced education improve clinical knowledge among Family Medicine residents? A cluster randomized controlled trial.

Spaced education is a learning strategy to improve knowledge acquisition and retention. To date, no robust evidence exists to support the utility of spaced education in the Family Medicine residency. We aimed to test whether alerts to encourage spaced education can improve clinical knowledge as measured by scores on the Canadian Family Medicine certification examination. METHOD: We conducted a cluster randomized controlled trial to empirically and pragmatically test spaced education using two versions of the Family Medicine Study Guide mobile app. 12 residency training programs in Canada agreed to participate. At six intervention sites, we consented 335 of the 654 (51%) eligible residents. Residents in the intervention group were sent alerts through the app to encourage the answering of questions linked to clinical cases. At six control sites, 299 of 586 (51%) residents consented. Residents in the control group received the same app but with no alerts. Incidence rates of case completion between trial arms were compared using repeated measures analysis. We linked residents in both trial arms to their knowledge scores on the certification examination of the College of Family Physicians of Canada. RESULTS: Over 67 weeks, there was no statistically significant difference in the completion of clinical cases by participants. The difference in mean exam scores and the associated confidence interval did not exceed the pre-defined limit of 4 percentage points. CONCLUSION: Further research is recommended before deploying spaced educational interventions in the Family Medicine residency to improve knowledge.

Defining sustainability in practice: views from implementing real-world innovations in health care.

BACKGROUND: One of the key conceptual challenges in advancing our understanding of how to more effectively sustain innovations in health care is the lack of clarity and agreement on what sustainability actually means. Several reviews have helped synthesize and clarify how researchers conceptualize and operationalize sustainability. In this study, we sought to identify how individuals who implement and/or sustain evidence-informed innovations in health care define sustainability. METHODS: We conducted in-depth, semi-structured interviews with implementation leaders and relevant staff involved in the implementation of evidence-based innovations relevant to cancer survivorship care (n = 27). An inductive approach, using constant comparative analysis, was used for analysis of interview transcripts and field notes. RESULTS: Participants described sustainability as an ongoing and dynamic process that incorporates three key concepts and four important conditions. The key concepts were: (1) continued capacity to deliver the innovation, (2) continued delivery of the innovation, and (3) continued receipt of benefits. The key conditions related to (2) and (3), and included: (2a) innovations must continue in the absence of the champion or person/team who introduced it and (3a) adaptation is critical to ensuring relevancy and fit, and thus to delivering the intended benefits. CONCLUSIONS: Participants provided a nuanced view of sustainability, with both continued delivery and continued benefits only relevant under certain conditions. The findings reveal the interconnected elements of what sustainability means in practice, providing a unique and important perspective to the academic literature.

A mixed methods examination of knowledge brokers and their use of theoretical frameworks and evaluative practices.

BACKGROUND: Knowledge brokering is a knowledge translation approach that includes making connections between researchers and decision-makers to facilitate the latter's use of evidence in health promotion and the provision of healthcare. Despite knowledge brokering being well-established in Canada, many knowledge gaps exist, including understanding what theoretical frameworks have been developed and which evaluative practices knowledge brokers (KBs) use. METHODS: This study used a mixed methods design to examine how KBs in Canada (1) use frameworks, models and theories in their practice and (2) how they evaluate knowledge brokering interventions. We gathered interview and survey data from KB practitioners to better understand their perspectives on effective practices. Our analysis focused on understanding the theoretical frameworks used by KBs. RESULTS: This study demonstrates that KBs in Canada tend not to rely on theories or models that are specific to knowledge brokering. Rather, study participants/respondents draw on (sometimes multiple) theories and models that are fundamental to the broader field of knowledge translation - in particular, the Knowledge to Action model and the Promoting Action Research in Health Sciences framework. In evaluating the impact of their own knowledge brokering practice, participants/respondents use a wide variety of mechanisms. Evaluation was often seen as less important than supporting knowledge users and/or paying clients in accessing and utilising evidence. CONCLUSIONS: Knowledge brokering as a form of knowledge translation continues to expand, but the impact on its targeted knowledge users has yet to be clearly established. The quality of engagement between KBs and their clients might increase - the knowledge brokering can be more impactful - if KBs made efforts to describe, understand and evaluate their activities using theories or models specific to KB.

Trainees' Self-Reported Challenges in Knowledge Translation, Research and Practice.

BACKGROUND: Knowledge translation (KT) refers to the process of moving evidence into healthcare policy and practice. Understanding the experiences and perspectives of individuals who develop careers in KT is important for designing training programs and opportunities to enhance capacity in KT research and practice. To date, however, limited research has explored the challenges that trainees encounter as they develop their careers in KT. AIMS: The purpose of this study is to identify the challenges that KT trainees face in their KT research or practice. METHODS: An online survey was conducted with a sample of trainees associated with the Knowledge Translation Trainee Collaborative or the KT Canada Summer Institutes, with written responses thematically analyzed. FINDINGS: A total of 35 individual responses were analyzed, resulting in the identification of six interrelated themes, listed in descending order of prevalence: limited availability of KT-specific resources (54%), difficulty inherent in investigating KT (34%), KT not recognized as a distinct field (23%), colleagues' limited knowledge and understanding of KT (20%), competing priorities and limited time (20%), and difficulties in relation to collaboration (14%). DISCUSSION: KT trainees experience specific challenges in their work: limited understanding of KT in other stakeholder groups; limited structures or infrastructure to support those who do KT; the inherently interdisciplinary and applied nature of KT; and the resultant complexities of scientific inquiry in this field, such as designing and testing multifaceted, multilevel implementation strategies and accounting for contextual factors. LINKING EVIDENCE TO ACTION: KT training and capacity-building efforts are needed to better position health systems to routinely adopt knowledge into healthcare policy and practice.

Using evaluation theory in priority setting and resource allocation.

PURPOSE: Public sector interest in methods for priority setting and program or policy evaluation has grown considerably over the last several decades, given increased expectations for accountable and efficient use of resources and emphasis on evidence-based decision making as a component of good management practice. While there has been some occasional effort to conduct evaluation of priority setting projects, the literatures around priority setting and evaluation have largely evolved separately. In this paper, the aim is to bring them together. DESIGN/METHODOLOGY/APPROACH: The contention is that evaluation theory is a means by which evaluators reflect upon what it is they are doing when they do evaluation work. Theories help to organize thinking, sort out relevant from irrelevant information, provide transparent grounds for particular implementation choices, and can help resolve problematic issues which may arise in the conduct of an evaluation project. FINDINGS: A detailed review of three major branches of evaluation theory--methods, utilization, and valuing--identifies how such theories can guide the development of efforts to evaluate priority setting and resource allocation initiatives. Evaluation theories differ in terms of their guiding question, anticipated setting or context, evaluation foci, perspective from which benefits are calculated, and typical methods endorsed. ORIGINALITY/VALUE: Choosing a particular theoretical approach will structure the way in which any priority setting process is evaluated. The paper suggests that explicitly considering evaluation theory makes key aspects of the evaluation process more visible to all stakeholders, and can assist in the design of effective evaluation of priority setting processes; this should iteratively serve to improve the understanding of priority setting practices themselves.

Moral distress among health system managers: exploratory research in two British Columbia health authorities.

Moral distress is a concept used to date in clinical literature to describe the experience of staff in circumstances in which they are prevented from delivering the kind of bedside care they believe is expected of them, professionally and ethically. Our research objective was to determine if this concept has relevance in terms of key health care managerial functions, such as priority setting and resource allocation. We conducted interviews and focus groups with mid- and senior-level managers in two British Columbia (Canada) health authorities. Transcripts were analyzed qualitatively using constant comparison to identify key themes related to moral distress. Both mid- and senior-level managers appear to experience moral distress, with both similarities and differences in how their experiences manifest. Several examples of this concept were identified including the obligation to communicate or 'sell' organizational decisions or policies with which a manager personally may disagree and situations where scarce resources compel managers to place staff in situations where they meet with predictable and potentially avoidable risks. Given that moral distress appears to be a relevant issue for at least some health care managers, further research is warranted into its exact nature, prevalence, and possible organizational and personal responses.

Identifying factors influencing sustainability of innovations in cancer survivorship care: a qualitative study.

OBJECTIVES: Moving innovations into healthcare organisations to increase positive health outcomes remains a significant challenge. Even when knowledge and tools are adopted, they often fail to become integrated into the long-term routines of organisations. The objective of this study was to identify factors and processes influencing the sustainability of innovations in cancer survivorship care. DESIGN: Qualitative study using semistructured, in-depth interviews, informed by grounded theory. Data were collected and analysed concurrently using constant comparative analysis. SETTING: 25 cancer survivorship innovations based in six Canadian provinces. PARTICIPANTS: Twenty-seven implementation leaders and relevant staff from across Canada involved in the implementation of innovations in cancer survivorship. RESULTS: The findings were categorised according to determinants, processes and implementation outcomes, and whether a factor was necessary to sustainability, or important but not necessary. Seven determinants, six processes and three implementation outcomes were perceived to influence sustainability. The necessary determinants were (1) management support; (2) organisational and system-level priorities; and (3) key people and expertise. Necessary processes were (4) innovation adaptation; (5) stakeholder engagement; and (6) ongoing education and training. The only necessary implementation outcome was (7) widespread staff and organisational buy-in for the innovation. CONCLUSIONS: Factors influencing the sustainability of cancer survivorship innovations exist across multiple levels of the health system and are often interdependent. Study findings may be used by implementation teams to plan for sustainability from the beginning of innovation adoption initiatives.

Identifying research priorities for health care priority setting: a collaborative effort between managers and researchers.

BACKGROUND: To date there has been relatively little published about how research priorities are set, and even less about methods by which decision-makers can be engaged in defining a relevant and appropriate research agenda. We report on a recent effort in British Columbia to have researchers and decision-makers jointly establish an agenda for future research into questions of resource allocation. METHODS: The researchers enlisted decision-maker partners from each of British Columbia's six health authorities. Three forums were held, at which researchers and decision-makers from various levels in the health authorities considered possible research areas related to three key focus areas: (1) generation and use of decision criteria and measurement of 'benefit' against such criteria; (2) identification of so-called 'disinvestment' opportunities; and (3) evaluation of the effectiveness of priority setting procedures. Detailed notes were taken from each forum and synthesized into a set of qualitative themes. RESULTS: Forum participants suggested that future research into healthcare priority setting would benefit from studies that were longitudinal, comparative, and/or interdisciplinary. As well, participants identified two broad theme areas in which specific research projects were deemed desirable. First, future research might usefully consider how formal priority setting and resource allocation projects are situated within a larger organizational and political context. Second, additional research efforts should be devoted to better understanding and improving the actual implementation of priority setting frameworks, particularly with respect to issues of change management and the resolution of impediments to action on recommendations for resource allocation. CONCLUSION: We were able to validate the importance of initial areas posed to the group and observed emergence of additional concerns and directions of critical importance to these decision-makers at this time. It is likely that the results are broadly applicable to other healthcare contexts. The implementation of this research agenda in British Columbia will depend upon the ability of the researchers and decision-makers to develop particular projects that fit within the constraints of existing funding opportunities. The process of engagement itself had benefits in terms of connecting decision-makers with their peers and sparking increased interest in the use and refinement of priority setting frameworks.

Knowledge brokering for healthy aging: a scoping review of potential approaches.

BACKGROUND: Developing a healthcare delivery system that is more responsive to the future challenges of an aging population is a priority in Canada. The World Health Organization acknowledges the need for knowledge translation frameworks in aging and health. Knowledge brokering (KB) is a specific knowledge translation approach that includes making connections between people to facilitate the use of evidence. Knowledge gaps exist about KB roles, approaches, and guiding frameworks. The objective of the scoping review is to identify and describe KB approaches and the underlying conceptual frameworks (models, theories) used to guide the approaches that could support healthy aging. METHODS: Literature searches were done in PubMed, EMBASE, PsycINFO, EBM reviews (Cochrane Database of systematic reviews), CINAHL, and SCOPUS, as well as Google and Google Scholar using terms related to knowledge brokering. Titles, abstracts, and full reports were reviewed independently by two reviewers who came to consensus on all screening criteria. Documents were included if they described a KB approach and details about the underlying conceptual basis. Data about KB approach, target stakeholders, KB outcomes, and context were extracted independently by two reviewers. RESULTS: Searches identified 248 unique references. Screening for inclusion revealed 19 documents that described 15 accounts of knowledge brokering and details about conceptual guidance and could be applied in healthy aging contexts. Eight KB elements were detected in the approaches though not all approaches incorporated all elements. The underlying conceptual guidance for KB approaches varied. Specific KB frameworks were referenced or developed for nine KB approaches while the remaining six cited more general KT frameworks (or multiple frameworks) as guidance. CONCLUSIONS: The KB approaches that we found varied greatly depending on the context and stakeholders involved. Three of the approaches were explicitly employed in the context of health aging. Common elements of KB approaches that could be conducted in healthy aging contexts focussed on acquiring, adapting, and disseminating knowledge and networking (linkage). The descriptions of the guiding conceptual frameworks (theories, models) focussed on linkage and exchange but varied across approaches. Future research should gather KB practitioner and stakeholder perspectives on effective practices to develop KB approaches for healthy aging.

Fit for purpose? Introducing a rational priority setting approach into a community care setting.

Purpose - Program budgeting and marginal analysis (PBMA) is a priority setting approach that assists decision makers with allocating resources. Previous PBMA work establishes its efficacy and indicates that contextual factors complicate priority setting, which can hamper PBMA effectiveness. The purpose of this paper is to gain qualitative insight into PBMA effectiveness. Design/methodology/approach - A Canadian case study of PBMA implementation. Data consist of decision-maker interviews pre (n=20), post year-1 (n=12) and post year-2 (n=9) of PBMA to examine perceptions of baseline priority setting practice vis-à-vis desired practice, and perceptions of PBMA usability and acceptability. Findings - Fit emerged as a key theme in determining PBMA effectiveness. Fit herein refers to being of suitable quality and form to meet the intended purposes and needs of the end-users, and includes desirability, acceptability, and usability dimensions. Results confirm decision-maker desire for rational approaches like PBMA. However, most participants indicated that the timing of the exercise and the form in which PBMA was applied were not well-suited for this case study. Participant acceptance of and buy-in to PBMA changed during the study: a leadership change, limited organizational commitment, and concerns with organizational capacity were key barriers to PBMA adoption and thereby effectiveness. Practical implications - These findings suggest that a potential way-forward includes adding a contextual readiness/capacity assessment stage to PBMA, recognizing organizational complexity, and considering incremental adoption of PBMA's approach. Originality/value - These insights help us to better understand and work with priority setting conditions to advance evidence-informed decision making.

Sustaining organizational culture change in health systems.

PURPOSE: The questions addressed by this review are: first, what are the guiding principles underlying efforts to stimulate sustained cultural change; second, what are the mechanisms by which these principles operate; and, finally, what are the contextual factors that influence the likelihood of these principles being effective? The paper aims to discuss these issues. DESIGN/METHODOLOGY/APPROACH: The authors conducted a literature review informed by rapid realist review methodology that examined how interventions interact with contexts and mechanisms to influence the sustainability of cultural change. Reference and expert panelists assisted in refining the research questions, systematically searching published and grey literature, and helping to identify interactions between interventions, mechanisms and contexts. FINDINGS: Six guiding principles were identified: align vision and action; make incremental changes within a comprehensive transformation strategy; foster distributed leadership; promote staff engagement; create collaborative relationships; and continuously assess and learn from change. These principles interact with contextual elements such as local power distributions, pre-existing values and beliefs and readiness to engage. Mechanisms influencing how these principles sustain cultural change include activation of a shared sense of urgency and fostering flexible levels of engagement. PRACTICAL IMPLICATIONS: The principles identified in this review, along with the contexts and mechanisms that influence their effectiveness, are useful domains for policy and practice leaders to explore when grappling with cultural change. These principles are sufficiently broad to allow local flexibilities in adoption and application. ORIGINALITY/VALUE: This is the first study to adopt a realist approach for understanding how changes in organizational culture may be sustained. Through doing so, this review highlights the broad principles by which organizational action may be organized within enabling contextual settings.

Reflecting on backward design for knowledge translation Comment on "A call for a backward design to knowledge translation".

In a recent Editorial for this journal, El-Jardali and Fadlallah proposed a new framework for Knowledge Translation (KT) in healthcare. Many such frameworks already exist; thus, new entrants to the field must be scrutinized in regard to their unique contributions to advancing understanding and practice. The El-Jardali and Fadlallah framework focuses on policy-level discussions, a relatively under-studied issue to date. Their framework usefully incorporates both priority setting questions at the front-end (which KT efforts get undertaken and which do not) as well as evaluation questions at the back-end (how do we show that more evidence-informed decisions are actually better ones?). Their framework also emphasizes capacity building among both decision-makers and researchers. This is an area worthy of additional attention, particularly because it is likely to be far more challenging than El-Jardali and Fadlallah allow.

Identifying priorities in knowledge translation from the perspective of trainees: results from an online survey.

BACKGROUND: The need to identify priorities to help shape future directions for research and practice increases as the knowledge translation (KT) field advances. Since many KT trainees are developing their research programs, understanding their concerns and KT research and practice priorities is important to supporting the development and advancement of KT as a field. Our purpose was to identify research and practice priorities in the KT field from the perspectives of KT researcher/practitioner trainees. FINDINGS: Survey response rate was 62 % (44/71). Participants were mostly Canadian graduate students, post-doctoral fellows, residents, and learners from various disciplines; the majority was from Ontario (44 %) and Quebec (20 %). Seven percent (5/71) were from other countries including USA, UK, and Switzerland. Seven main KT priority themes were identified: determining the effectiveness of KT strategies, technology use, increased key stakeholder involvement, context, theory, expand ways of inquiry, and sustainability. CONCLUSIONS: Overall, the priorities identified by the trainees correspond with KT literature and with KT experts' views. The trainees appeared to push the boundaries of current KT literature with respect to creative use of communication technologies research.

Determining and broadening the definition of impact from implementing a rational priority setting approach in a healthcare organization.

Techniques to manage scarce healthcare resources continue to evolve in response to changing, growing and competing demands. Yet there is no standard definition in the priority setting literature of what might constitute the desired impact or success of resource management activities. In this 2006-09 study, using action research methodology, we determined the impact of implementing a formal priority setting model, Program Budgeting and Marginal Analysis (PBMA), in a Canadian health authority. Qualitative data were collected through post year-1 (n = 12) and year-2 (n = 9) participant interviews, meeting observation and document review. Interviews were analyzed using a constant comparison technique to identify major themes. Impact can be defined as effects at three levels: system, group, and individual. System-level impact can be seen in the actual selection of priorities and resource re-allocation. In this case, participants prioritized a list of $760,000 worth of investment proposals and $38,000 of disinvestment proposals; however, there was no clear evidence as to whether financial resources were reallocated as a result. Group and individual impacts, less frequently reported in the literature, included changes in priority setting knowledge, attitudes and practice. PBMA impacts at these three levels were found to be interrelated. This work argues in favor of attempts to expand the definition of priority setting success by including both desired system-level outcomes like resource re-allocation and individual or group level impacts like changes to priority setting knowledge, attitudes and practice. These latter impacts are worth pursuing as they appear to be intrinsic to successful system-wide priority setting. A broader definition of PBMA impact may also suggest conceptualizing PBMA as both a priority setting approach and as a tool to develop individual and group priority setting knowledge and practice. These results should be of interest to researchers and decision makers using or considering a formal priority setting approach to manage scarce healthcare resources.

Changing priority setting practice: the role of implementation in practice change.

BACKGROUND: Programme budgeting and marginal analysis (PBMA) is a priority setting approach that assists decision makers in choosing among resource demands. This paper describes and evaluates the process of implementing PBMA in a Canadian regional health authority, and draws out key lessons learned from this experience. METHODS: Qualitative data were collected through semi-structured participant interviews (twelve post year-1; nine post year-2), meeting attendance, and document review. Interview transcripts were analyzed using a constant comparison technique. Other data were analyzed to evaluate PBMA implementation. RESULTS: Desire for more clarity and for PBMA adaptations emerged as overarching themes. Participants desired greater clarity of their roles and how PBMA should be used to achieve PBMA's potential benefits. They argued that each PBMA stage should be useful independent of the others so that implementation could be adapted. To help improve clarity and ensure that resources were available to support PBMA, participants requested an organizational readiness and capacity assessment. CONCLUSION: We suggest tactics by which PBMA may be more closely aligned with real-world priority setting practice. Our results also contribute to the literature on PBMA use in various healthcare settings. Highlighting implementation issues and potential responses to these should be of interest to decision makers implementing PBMA and other evidence-informed practices.

A community of practice for knowledge translation trainees: an innovative approach for learning and collaboration.

A growing number of researchers and trainees identify knowledge translation (KT) as their field of study or practice. Yet, KT educational and professional development opportunities and established KT networks remain relatively uncommon, making it challenging for trainees to develop the necessary skills, networks, and collaborations to optimally work in this area. The Knowledge Translation Trainee Collaborative is a trainee-initiated and trainee-led community of practice established by junior knowledge translation researchers and practitioners to: examine the diversity of knowledge translation research and practice, build networks with other knowledge translation trainees, and advance the field through knowledge generation activities. In this article, we describe how the collaborative serves as an innovative community of practice for continuing education and professional development in knowledge translation and present a logic model that provides a framework for designing an evaluation of its impact as a community of practice. The expectation is that formal and informal networking will lead to knowledge sharing and knowledge generation opportunities that improve individual members' competencies (eg, combination of skills, abilities, and knowledge) in knowledge translation research and practice and contribute to the development and advancement of the knowledge translation field.

Knowledge translation in the discourse of professional practice.

Clients and practitioners desire up-to-date, safe and effective healthcare. Knowledge translation, a term used to describe the interchange of knowledge between its producers and users, aims to support this desire. Knowledge, and by extension knowledge translation, is subject to varying perspectives ranging from the objective truth-tested knowledge of empiricism, associated by some with academia, to knowledge in the practitioner realm. This latter knowledge is often based on multiple users' experiences and contexts, thus constructed to meet their needs. The goal of this paper is to compare and contrast knowledge and knowledge translation from empirical and constructivist perspectives. It then relates knowledge translation to professional practice discourse and concludes with thoughts on constructivist knowledge translation strategies, including practitioner-driven strategies derived from practice. For example, knowledge translation can be woven into processes to train/integrate new graduates into the healthcare system, it can be captured in practitioner-driven provision of continuing education, and/or it can be facilitated through practitioner collaboration in research via action research approaches. Regardless of the perspective taken, delivery of up-to-date, safe and effective care requires useful, relevant knowledge available when necessary and applicable to real-life issues as perceived, critically, by the knowledge end-user.

Creating a knowledge translation trainee collaborative: from conceptualization to lessons learned in the first year.

Trainees (e.g., graduate students, residents, fellows) are increasingly identifying knowledge translation as their research discipline. In Canada, a group of trainees have created a trainee-initiated and trainee-led national collaborative to provide a vehicle for trainees to examine the diversity of knowledge translation research and practice, and to link trainees from diverse geographical areas and disciplines. The aim of this paper is to describe our experience and lessons learned in creating the Knowledge Translation Trainee Collaborative. In this meeting report, we outline the process, challenges, and opportunities in planning and experiencing the collaborative's inaugural meeting as participant organizers, and present outcomes and learnings to date.

Moral Distress among Healthcare Managers: Conditions, Consequences and Potential Responses.

Moral distress - the physical and emotional response to feeling prevented from carrying out ethically proper action - can have serious consequences for health professionals and healthcare organizations. We investigated perceived moral distress qualitatively with managers in two BC health authorities.RESPONDENTS DESCRIBED CONDITIONS UNDER WHICH THEY EXPERIENCED DISTRESS: when they set priorities within highly resource-constrained environments, when they observed inequities between budget allocations and management responsibilities, and when organizational priorities did not align with their personal values. When coping proved insufficient, managers would respond by leaving positions, organizations or the healthcare field altogether.Respondents asked for leadership development and the creation of spaces in which moral distress could be openly discussed. However, formal training in priority setting did not appear to be helpful on its own. Rather, it increased managers' awareness of the ethical dimensions of resource allocation without (in this instance) entrenching supports that would help them resolve these concerns.