Evaluating the impacts of patient engagement on a national health research network: results of a case study of the Chronic Pain Network.

BACKGROUND: The Chronic Pain Network (CPN) is a pan-Canadian research network focused on innovating and improving the quality and delivery of pain prevention, assessment, management and research for all Canadians. An important focus of the CPN is to work in collaboration with patient partners. Patient partners, researchers and clinicians work together in all aspects of the research network including on funded research projects and in the governance of the Network. Given this focus, the CPN identified the importance of evaluating their patient engagement work to understand its functioning and impact. METHODS: The objective of this exploratory evaluation case study was to understand the impacts of patient engagement on the CPN. The CPN worked with an external evaluation team which established an arms-length approach to the evaluation. Interviews were conducted with CPN members, including patient partners, leadership, funded researchers and committee co-chairs, at three discrete time points to trace the evolution of the patient engagement program within the Network. Key Network documents were also collected and reviewed. Data were analyzed following each set of interviews using content analysis guided by the principles of constant comparison and qualitative description. A final round of analysis was conducted using the Engage with Impact Toolkit, an impact measurement framework, to identify impacts of engagement. RESULTS: Impacts of patient engagement were identified at the individual, network, funded research project and research community levels. These impacts were observed in the following areas: (1) building community; (2) developing knowledge, skills and resources; (3) increasing confidence; (4) influencing priorities and decisions; (5) enabling additional opportunities; (6) promoting culture change; and, (7) coping with experiences of living with chronic pain. CONCLUSIONS: While not without challenges, the patient engagement efforts of the CPN demonstrates the impact engaging patient partners can have on a national research network and related policy activities. Understanding the approaches to, and impacts of, patient engagement on health research networks can illuminate the value of having patient partners engaged in all aspects of a research network and should serve as encouragement to others who look to take on similar work.

The Chronic Pain Network (CPN) is one of a group of research networks that was funded by the Canadian Institutes of Health Research (CIHR) to support patient-oriented research in chronic diseases. From the beginning of its work, the CPN has included patients as partners. Patient partners are co-chairs of all Network governance committees, funded projects are required to include patient partners in their work and there is a committee dedicated to engagement, the Patient Engagement (PE) committee. The PE Committee determined that it was important to evaluate how the CPN was engaging with patient partners and collaborated with the Public and Patient Engagement Collaborative (PPEC) to evaluate this work. The PPEC, along with members of the PE Committee, identified understanding the impact of patient engagement as an important part of the evaluation. This paper provides a description of the impacts of patient engagement on the people who were involved in the CPN, on the CPN's work and way of being, and on the broader pain research community. Based on the results from three sets of interviews and review of Network documents, we share impacts identified in seven areas: (1) building community; (2) developing knowledge, skills and resources; (3) increasing confidence; (4) influencing priorities and decisions; (5) enabling additional opportunities; (6) promoting culture change; and, (7) coping with experiences of living with chronic pain. This research shows us the impact that engaging patient partners can have on a national research network, and the areas where greater focus could, perhaps, lead to even greater impacts in future networks.

Development of the Engage with Impact Toolkit: A comprehensive resource to support the evaluation of patient, family and caregiver engagement in health systems.

INTRODUCTION: Recent shifts in the patient, family and caregiver engagement field have focused greater attention on measurement and evaluation, including the impacts of engagement efforts. Current evaluation tools offer limited support to organizations seeking to reorient their efforts in this way. We addressed this gap through the development of an impact measurement framework and accompanying evaluation toolkit-the Engage with Impact Toolkit. METHODS: The measurement framework and toolkit were co-designed with the Evaluating Patient Engagement Working Group, a multidisciplinary group of patient, family and caregiver partners, engagement specialists, researchers and government personnel. Project activities occurred over four phases: (1) project scoping and literature review; (2) modified concept mapping; (3) working group deliberations and (4) toolkit web design. RESULTS: The project scope was to develop a measurement framework and an evaluation toolkit for patient engagement in health systems that were practical, accessible, menu-driven and aligned with current system priorities. Concept mapping yielded 237 impact statements that were sorted, discussed and combined into 81 unique items. A shorter list of 50 items (rated 8.0 or higher out of 10) was further consolidated to generate a final list of 35 items mapped across 8 conceptual domains of impact: (1) knowledge and skills; (2) confidence and trust; (3) equity and inclusivity; (4) priorities and decisions; (5) effectiveness and efficiency; (6) patient-centredness; (7) culture change and (8) patient outcomes and experience. Working Group members rated the final list for importance (1-5) and identified a core set of 33 items (one for each of the 8 domains and 25 supplementary items). Two domains (priorities and decisions; and culture change) yielded the highest overall importance ratings (4.8). A web-based toolkit (www.evaluateengagement.ca) hosts the measurement framework and related evaluation supports. CONCLUSION: The Engage with Impact Toolkit builds on existing engagement evaluation tools but brings a more explicit focus to supporting organizations to assess the impacts of their engagement work. PATIENT CONTRIBUTION: Patient, family and caregiver partners led the early conceptualization of this work and were involved at all stages and in all aspects of the work. As end-users of the toolkit, their perspectives, knowledge and opinions were critical.

Towards conceptualizing patients as partners in health systems: a systematic review and descriptive synthesis.

BACKGROUND: With the sharp increase in the involvement of patients (including family and informal caregivers) as active participants, collaborators, advisors and decision-makers in health systems, a new role has emerged: the patient partner. The role of patient partner differs from other forms of patient engagement in its longitudinal and bidirectional nature. This systematic review describes extant work on how patient partners are conceptualized and engaged in health systems. In doing so, it furthers the understanding of the role and activities of patient partners, and best practices for future patient partnership activities. METHODS: A systematic review was conducted of peer-reviewed literature published in English or French that describes patient partner roles between 2000 and 2021 in any country or sector of the health system. We used a broad search strategy to capture descriptions of longitudinal patient engagement that may not have used words such as "partner" or "advisor". RESULTS: A total of 506 eligible papers were identified, representing patient partnership activities in mostly high-income countries. These studies overwhelmingly described patient partnership in health research. We identified clusters of literature about patient partnership in cancer and mental health. The literature is saturated with single-site descriptive studies of patient partnership on individual projects or initiatives. There is a lack of work synthesizing impacts, facilitating factors and outcomes of patient partnership in healthcare. CONCLUSIONS: There is not yet a consolidated understanding of the role, activities or impacts of patient partners. Advancement of the literature has been stymied by a lack of consistently used terminology. The literature is ready to move beyond single-site descriptions, and synthesis of existing pockets of high-quality theoretical work will be essential to this evolution.

The impact of COVID-19 on patient engagement in the health system: Results from a Pan-Canadian survey of patient, family and caregiver partners.

INTRODUCTION: The COVID-19 pandemic has had an impact on all aspects of the health system. Little is known about how the activities and experiences of patient, family and caregiver partners, as a large group across a variety of settings within the health system, changed due to the substantial health system shifts catalysed by the pandemic. This paper reports on the results of a survey that included questions about this topic. METHODS: Canadian patient, family and caregiver partners were invited to participate in an online anonymous survey in the Fall of 2020. A virtual snowballing approach to recruitment was used. Survey invitations were shared on social media and emailed to health system and governmental organizations with the request that they share the survey with patient partners. This paper focuses on responses to two questions related to patient partner experiences during the COVID-19 pandemic. RESULTS: The COVID-19 questions were completed by 533 respondents. Over three quarters of respondents (77.9%, n = 415) indicated their patient engagement activities had been impacted by COVID-19. The majority (62.5%, n = 230) experienced at least a temporary or partial reduction in their patient engagement activities. Some respondents did see increases in their patient engagement activities (11.4%, n = 42). Many respondents provided insights into their experience with virtual platforms for engagement (n = 194), most expressed negative or mixed experiences with this shift. CONCLUSIONS: This study provides a snapshot of Canadian patient, family and caregiver partners' perspectives on the impact of COVID-19 on their engagement activities. Understanding how engagement unfolded during a crisis is critical for our future planning if patient engagement is to be fully integrated into the health system. Identifying how patient partners were engaged and not engaged during this time period, as well as the benefits and challenges of virtual engagement opportunities, offers instructive lessons for sustaining patient engagement, including the supports needed to engage with a more diverse set of patient, family and caregiver partners. PATIENT CONTRIBUTION: Patient partners were important members of the Canadian Patient Partner Study research team. They were engaged from the outset, participating in all stages of the research project. Additional patient partners were engaged to develop and pilot test the survey, and all survey respondents were patient, family or caregiver partners. The manuscript is coauthored by two patient partners.

Understanding patient partnership in health systems: lessons from the Canadian patient partner survey.

OBJECTIVES: To examine the sociodemographic characteristics, activities, motivations, experiences, skills and challenges of patient partners working across multiple health system settings in Canada. DESIGN: Online cross-sectional survey of self-identified patient partners. SETTING: Patient partners in multiple jurisdictions and health system organisations. PARTICIPANTS: 603 patient partners who had drawn on their experiences with the health system as a patient, family member or informal caregiver to try to improve it in some way, through their involvement in the activities of a group, organisation or government. RESULTS: Survey respondents predominantly identified as female (76.6%), white (84%) and university educated (70.2%) but were a heterogeneous group in the scope (activities and organisations), intensity (number of hours) and longevity (number of years) of their role. Primary motivations for becoming a patient partner were the desire to improve the health system based on either a negative (36.2%) or positive (23.3%) experience. Respondents reported feeling enthusiastic (83.6%), valued (76.9%) and needed (63.3%) always or most of the time; just under half felt they had always or often been adequately compensated in their role. Knowledge of the health system and the organisation they partner with are key skills needed. Two-thirds faced barriers in their role with over half identifying power imbalances. Less than half were able to see how their input was reflected in decisions or changes always or most of the time, and 40.3% had thought about quitting. CONCLUSIONS: This survey is the first of its kind to examine at a population level, the characteristics, experiences and dynamics of a large sample of self-identified patient partners. Patient partners in this sample are a sociodemographically homogenous group, yet heterogeneous in the scope, intensity and longevity of roles. Our findings provide key insights at a critical time, to inform the future of patient partnership in health systems.

Supporting women at average risk to make informed decisions about mammography when there is no "right" answer: a qualitative citizen deliberation study.

BACKGROUND: Women are encouraged to make informed choices about mammography screening that align with their values and preferences, yet information materials developed by screening programs rarely provide complete, balanced information about screening. Through a series of deliberations with Ontario citizens, we elicited perspectives on materials developed by screening programs to support informed decision-making. METHODS: We held 4 deliberative engagement events with citizens to discuss the current evidence about mammography and informed decision-making for the general population (i.e., women not at high risk) in the context of organized screening programs. Participants reviewed and provided feedback on the educational materials currently produced by screening programs in 8 provinces (British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Quebec, Nova Scotia and Newfoundland and Labrador) and 2 territories (Yukon Territory and Northwest Territory) and identified the key features that should guide the design of these materials to optimally support informed decision-making. RESULTS: In general, participants viewed the educational materials as insufficient to support informed decision-making. They identified the following key features of optimal educational materials: they should be accessible, complete and accurate, and provide information on both benefits and risks of screening in a comprehensive, easy-to-understand manner. Information materials should evoke the trust of the reader, and they should be consistent across Canada. INTERPRETATION: Canadian women have insufficient access to reliable information sources and complete evidence about mammography screening, and, without this information, they are unable to make fully informed decisions. Canadian breast screening programs must take steps to improve the information shared with women to support informed decision-making that aligns with women's values and preferences.

Supporting the evaluation of public and patient engagement in health system organizations: Results from an implementation research study.

BACKGROUND: As citizens, patients and family members are participating in numerous and expanding roles in health system organizations, attention has turned to evaluating these efforts. The context-specific nature of engagement requires evaluation tools to be carefully designed for optimal use. We sought to address this need by assessing the appropriateness and feasibility of a generic tool across a range of health system organizations, engagement activities and patient groups. METHODS: We used a mixed-methods implementation research design to study the implementation of an engagement evaluation tool in seven health system organizations in Ontario, Canada focusing on two key implementation outcome variables: appropriateness and feasibility. Data were collected through respondent feedback questions (binary and open-ended) at the end of the tool's three questionnaires as well as interviews and debriefing discussions with engagement professionals and patient partners from collaborating organizations. RESULTS: The three questionnaires comprising the evaluation tool were collectively administered 29 times to 405 respondents yielding a 52% response rate (90% and 53% of respondents respectively assessed the survey's appropriateness and feasibility [quantitatively or qualitatively]). The questionnaires' basic properties were rated highly by all respondents. Concrete suggestions were provided for improving the appropriateness and feasibility of the questionnaires (or components within) for different engagement activity and organization types, and for enhancing the timing of implementation. DISCUSSION AND CONCLUSIONS: Our study findings offer guidance for health system organizations and evaluators to support the optimal use of engagement evaluation tools across a variety of health system settings, engagement activities and respondent groups.

Ethical Challenges Related to Patient Involvement in Health Technology Assessment.

Including information and values from patients in HTA has the potential to improve both the process and outcomes of health technology policy decisions. Accordingly, funding and structural incentives to include patients in HTA activities have increased over the past several years. Unfortunately, these incentives have not yet been accompanied by a corresponding increase in resources, time, or commitment to responsiveness. In this Perspectives piece, we reflect on our collective experiences participating in, conducting, and overseeing patient engagement activities within HTA to highlight the ethical challenges associated with this area of activity. While we remain committed to the idea that patient engagement activities strengthen the findings, relevance, and legitimacy of health technology policy, we are deeply concerned about the potential for these activities to do ethical harm. We use this analysis to call for action to introduce strong protections against ethical violations that may harm patients participating in HTA engagement activities.

Addressing the affordability of cancer drugs: using deliberative public engagement to inform health policy.

BACKGROUND: Health system expenditure on cancer drugs is rising rapidly in many OECD countries given the costly new treatments and increased rates of use due to a growing and ageing population. These factors put considerable strain on the sustainability of health systems worldwide, sparking public debate among clinicians, pharmaceutical companies, policy-makers and citizens on issues of affordability and equity. We engaged Canadians through a series of deliberative public engagement events to determine their priorities for making cancer drug funding decisions fair and sustainable in Canada's publicly financed health system. METHODS: An approach to deliberation was developed based on the McMaster Health Forum's citizen panels and the established Burgess and O'Doherty model of deliberative public engagement. Six deliberations were held across Canada in 2016. Transcripts were coded in NVivo and analysed to determine where participants' views converged and diverged. Recommendations were grouped thematically. RESULTS: A total of 115 Canadians participated in the deliberative events and developed 86 recommendations. Recommendations included the review and regular re-review of approved drugs using 'real-world' evidence on effectiveness and cost-effectiveness; prioritisation of treatments that restore patients' independence, mental health and general well-being; ensuring that decision processes, results and their rationales are transparent; and commitment to people with similar needs receiving the same care regardless of where in Canada they live. CONCLUSIONS: The next steps for policy-makers should be to develop mechanisms for (1) re-reviewing effectiveness and cost-effectiveness data for all cancer drugs; (2) making disinvestments in cancer drugs that satisfy requirements relating to grandfathering and compassionate access; (3) ensuring fair and equitable access to cancer drugs for all Canadians; and (4) fostering a pan-Canadian approach to cancer drug funding decisions.

'I just want to be able to make a choice': Results from citizen deliberations about mammography screening in Ontario, Canada.

Despite Canada's long history with mammography screening, little is known about citizens' perspectives about mammography and how best to support women to make informed choices about screening. To address this gap, a series of four citizen deliberation events were held in 2015-16 in Ontario, a Canadian province with an organized population-based breast screening program in place since 1990. Forty-nine individuals participated in four citizen panels, each comprising an information session highlighting the evidence about mammography, and large- and small-group deliberations about approaches to support informed decision making for screening. Following their engagement with the research evidence about mammography, participants expressed concern about their lack of full awareness of the risks and benefits and a strong desire for choice when it comes to screening. To support informed choice, mammography programs need to reflect the values of information sharing, trust and transparency, financial accountability, and allow for personal interactions and shared decision-making. Citizens are looking for balanced information about the risks and benefits of screening presented in an easy to understand, comprehensive, and transparent manner. Primary health care providers and organized screening programs are important sources of information about mammography and must be vigilant in their efforts to support informed decision-making in this area by ensuring that the information materials they are using are balanced and reflect current evidence.

Uncertain times: A survey of Canadian women's perspectives toward mammography screening.

Evolving scientific evidence about mammography has raised new questions about the net benefits of organized screening, yet gaps remain about women's current screening practices, knowledge, attitudes and values toward screening to support informed decision making in this area. We addressed this gap through an online survey of 2000 screen-eligible women from Ontario, Canada in January 2016. Likert-scaled and categorical questions were used to collect information about screening practices, knowledge of benefits and risks of screening and underlying attitudes and values toward screening. Results for all responses were summarized using descriptive statistics. Comparison of results between ever screened versus never screened respondents was performed using chi-squared tests. Most women felt informed about screening yet had doubts about how informed their decisions were. They were more confident in their knowledge of the benefits than the risks which aligned with the emphasis given to benefits in discussions with health care providers. The benefits of screening were linked with lowered anxiety about breast cancer. The never screened were less likely to overstate the benefits of screening, more likely to give weight to the risks, and less likely to report anxiety or worry about breast cancer. Findings highlight the need for improved communication strategies and decision supports that emphasize the provision of current, balanced information about the benefits and risks of screening, both at the population-level (through mass media) and within patient-provider interactions. Sensitivity to the psychosocial factors that shape women's attitudes toward mammography screening should be central to any strategy.