Development of the Engage with Impact Toolkit: A comprehensive resource to support the evaluation of patient, family and caregiver engagement in health systems.

INTRODUCTION: Recent shifts in the patient, family and caregiver engagement field have focused greater attention on measurement and evaluation, including the impacts of engagement efforts. Current evaluation tools offer limited support to organizations seeking to reorient their efforts in this way. We addressed this gap through the development of an impact measurement framework and accompanying evaluation toolkit-the Engage with Impact Toolkit. METHODS: The measurement framework and toolkit were co-designed with the Evaluating Patient Engagement Working Group, a multidisciplinary group of patient, family and caregiver partners, engagement specialists, researchers and government personnel. Project activities occurred over four phases: (1) project scoping and literature review; (2) modified concept mapping; (3) working group deliberations and (4) toolkit web design. RESULTS: The project scope was to develop a measurement framework and an evaluation toolkit for patient engagement in health systems that were practical, accessible, menu-driven and aligned with current system priorities. Concept mapping yielded 237 impact statements that were sorted, discussed and combined into 81 unique items. A shorter list of 50 items (rated 8.0 or higher out of 10) was further consolidated to generate a final list of 35 items mapped across 8 conceptual domains of impact: (1) knowledge and skills; (2) confidence and trust; (3) equity and inclusivity; (4) priorities and decisions; (5) effectiveness and efficiency; (6) patient-centredness; (7) culture change and (8) patient outcomes and experience. Working Group members rated the final list for importance (1-5) and identified a core set of 33 items (one for each of the 8 domains and 25 supplementary items). Two domains (priorities and decisions; and culture change) yielded the highest overall importance ratings (4.8). A web-based toolkit (www.evaluateengagement.ca) hosts the measurement framework and related evaluation supports. CONCLUSION: The Engage with Impact Toolkit builds on existing engagement evaluation tools but brings a more explicit focus to supporting organizations to assess the impacts of their engagement work. PATIENT CONTRIBUTION: Patient, family and caregiver partners led the early conceptualization of this work and were involved at all stages and in all aspects of the work. As end-users of the toolkit, their perspectives, knowledge and opinions were critical.

Supporting quality public and patient engagement in health system organizations: development and usability testing of the Public and Patient Engagement Evaluation Tool.

OBJECTIVES: Only rudimentary tools exist to support health system organizations to evaluate their public and patient engagement (PPE) activities. This study responds to this gap by developing a generic evaluation tool for use in a wide range of organizations. METHODS: The evaluation tool was developed through an iterative, collaborative process informed by a review of published and grey literature and with the input of Canadian PPE researchers and practitioners. Over a 3-year period, structured e-mail, telephone and face-to-face exchanges, including a modified Delphi process, were used to produce an evaluation tool that includes core principles of high-quality engagement, expected outcomes for each principle and three unique evaluation questionnaires that were tested and revised with input from 65 end users. RESULTS: The tool is structured around four core principles of 'quality engagement': (i) integrity of design and process; (ii) influence and impact; (iii) participatory culture; and (iv) collaboration and common purpose. Three unique questionnaires were developed to assess each of these four evaluation domains from the following perspectives: (i) those who participate in PPE activities; (ii) those who plan, execute or sponsor PPE activities within organizations; and (iii) those who provide the leadership and capacity for PPE within their organizations. CONCLUSIONS: This is the first known collaboration of researchers and practitioners in the co-design of a comprehensive PPE evaluation tool aimed at three distinct respondent groups and for use in a wide range of health system organization settings.

Ethical considerations in public engagement: developing tools for assessing the boundaries of research and involvement.

Public engagement with research (PEwR) has become increasingly integral to research practices. This paper explores the process and outcomes of a collaborative effort to address the ethical implications of PEwR activities and develop tools to navigate them within the context of a University Medical School. The activities this paper reflects on aimed to establish boundaries between research data collection and PEwR activities, support colleagues in identifying the ethical considerations relevant to their planned activities, and build confidence and capacity among staff to conduct PEwR projects. The development process involved the creation of a taxonomy outlining key terms used in PEwR work, a self-assessment tool to evaluate the need for formal ethical review, and a code of conduct for ethical PEwR. These tools were refined through iterative discussions and feedback from stakeholders, resulting in practical guidance for researchers navigating the ethical complexities of PEwR. Additionally, reflective prompts were developed to guide researchers in planning and conducting engagement activities, addressing a crucial aspect often overlooked in formal ethical review processes. The paper reflects on the broader regulatory landscape and the limitations of existing approval and governance processes, and prompts critical reflection on the compatibility of formal approval processes with the ethos of PEwR. Overall, the paper offers insights and practical guidance for researchers and institutions grappling with ethical considerations in PEwR, contributing to the ongoing conversation surrounding responsible research practices.

Shared decision-making in Canada: Update on integration of evidence in health decisions and patient-centred care government mandates.

In Canada, government mandates for patient-centred care (PCC) vary across the 10 provinces and three territories. Although basic medical and hospital services are provided for all, health care options for patients also depend on having private insurance. Thus, the current design of the Canadian healthcare system has several implications for PCC and shared decision-making (SDM). Since 2007, this is our fourth update on SDM in Canada. The aim of this paper is to provide an update on the current state of SDM and patient and public involvement in Canada. Overall, we still observed the difficulty of implementing any sort of national strategy partly because of the decentralized nature of the healthcare system. Second, national professional education programs are complicated by licensure and scope of practice variations across jurisdictions. Third, there are variations in the availability of different options covered by universal healthcare. Canada has experienced some favorable development as PCC is now explicitly articulated in the policies of most provinces and territories and there are increased efforts to give patients more access to their electronic health records. However, patient and public engagement (PPE) reform in health programs and governance remains an exception, and continuing centralization of governance structures may reduce their responsiveness to patient priorities. In a 2018 survey, 47.2% of respondents reported that they were not told by their health professional that they had a choice about treatment. Nonetheless, decision aids and decision coaching are increasingly available for health-related decisions and the Ottawa Hospital Research Institute's decision aid inventory has ensured continued leadership in this area. Diverse jurisdictions are starting to embed decision aids into care pathways, with some decision aids being included in clinical practice guidelines. The COVID-19 pandemic may have had a negative impact on SDM by removing decision choices due to emergency public health mandates, but stimulated new research and decision aids. Canada continues to assign health research funding to SDM and PCC, and a program dedicated to patient-oriented research is central to this effort. Guides and frameworks are increasingly available for planning and evaluating PPE. Finally, various initiatives are attempting to involve and empower Indigenous peoples through PPE and SDM.

Linking Podcasts With Social Media to Promote Community Health and Medical Research: Feasibility Study.

BACKGROUND: Linking podcasts with social media is a strategy to promote and disseminate health and health research information to the community without constraints of time, weather, and geography. OBJECTIVE: To describe the process of creating a podcast library and promoting it on social media as a strategy for disseminating health and biomedical research topics to the community. METHODS: We used a community and patient engagement in research approach for developing a process to use podcasts for dissemination of health and health research information. We have reported the aspects of audience reach, impressions, and engagement on social media through the number of downloads, shares, and reactions posted on SoundCloud, Twitter, and Facebook, among others. RESULTS: In collaboration with our local community partner, we produced 45 podcasts focused on topics selected from a community health needs assessment with input from health researchers. Episodes lasted about 22 minutes and presented health-related projects, community events, and community resources, with most featured guests from Olmsted County (24/45, 53%). Health research was the most frequently discussed topic. Between February 2016 and June 2017, episodes were played 1843 times on SoundCloud and reached 1702 users on our Facebook page. CONCLUSIONS: This study demonstrated the process and feasibility of creating a content library of podcasts for disseminating health- and research-related information. Further examination is needed to determine the best methods to develop a sustainable social media plan that will further enhance dissemination (audience reach), knowledge acquisition, and communication of health topics.

Impact of electronic personal health record use on engagement and intermediate health outcomes among cardiac patients: a quasi-experimental study.

OBJECTIVES: To determine the impact of tethered personal health record (PHR) use on patient engagement and intermediate health outcomes among patients with coronary artery disease (CAD). METHODS: Adult CAD patients (N = 200) were enrolled in this prospective, quasi-experimental observational study. Each patient received a PHR account and training on its use. PHRs were populated with information from patient electronic medical records, hosted by a Health Information Exchange. Intermediate health outcomes including blood pressure, body mass index, and hemoglobin A1c (HbA1c) were evaluated through electronic medical record review or laboratory tests. Trends in patient activation measure® (PAM) were determined through three surveys conducted at baseline, 6 and 12 months. Frequency of PHR use data was collected and used to classify participants into groups for analysis: Low, Active, and Super users. RESULTS: There was no statistically significant improvement in patient engagement as measured by PAM scores during the study period. HbA1c levels improved significantly in the Active and Super user groups at 6 months; however, no other health outcome measures improved significantly. Higher PAM scores were associated with lower body mass index and lower HbA1c, but there was no association between changes in PAM scores and changes in health outcomes. Use of the PHR health diary increased significantly following PHR education offered at the 6-month study visit and an elective group refresher course. CONCLUSIONS: The study findings show that PHR use had minimal impact on intermediate health outcomes and no significant impact on patient engagement among CAD patients.