Access to novel drugs and therapeutics for children and youth: Eliciting citizens' values to inform public funding decisions.

INTRODUCTION: The unique evidentiary, economic and ethical challenges associated with health technology assessment (HTA) of precision therapies limit access to novel drugs and therapeutics for children and youth, for whom such challenges are amplified. We elicited citizens' perspectives about values-based criteria relevant to the assessment of paediatric precision therapies to inform the development of a child-tailored HTA framework. METHODS: We held four citizen panels virtually in May-June 2021, informed by a plain-language citizen brief summarizing global and local evidence about the challenges, policy and programmatic options and implementation strategies related to enhancing access to precision therapies for Canadian children and youth. Panellists were recruited through a nationally representative database, medical/patient networks and social media. We inductively coded and thematically analysed panel transcripts to generate themes and identify priority values. RESULTS: The perspectives of panellists (n = 45) coalesced into four overlapping themes, with attendant subthemes, relevant to a child-tailored HTA framework: (1) Childhood Distinctions: vulnerability, 'fair innings', future potential, family impacts; (2) Voice: agency of children and youth; lived versus no lived experience; (3) One versus Many: disease severity, rarity, equity, unmet need and (4) Health System Governance: funding, implementation inequities, effectiveness and safety. Participants broadly agreed that childhood distinctions, particularly family impacts, justify child-tailored HTA. Dissent arose over whose voice should inform HTA and how such perspectives are best incorporated. CONCLUSIONS: Citizens can offer unique insights into criteria relevant to the development or revision of HTA frameworks to capture holistic, societally responsive dimensions of value attached to unique contexts or populations, including children. Balancing the hopes and expectations of patients and caregivers for access to expensive but potential life-altering therapies against the opportunity costs borne by encompassing health systems is a fundamental challenge that will require rigorous methods to elicit, weigh and reconcile varied views. PATIENT OR PUBLIC CONTRIBUTION: A patient advocate served on the steering committee of this study and co-authored this article. Key informants for the Citizen Brief included patient advocates and caregivers; a separate patient advocate reviewed the Brief before dissemination. Qualitative and quantitative data were collected from the general public and caregivers of children, with written consent.

Development of the Engage with Impact Toolkit: A comprehensive resource to support the evaluation of patient, family and caregiver engagement in health systems.

INTRODUCTION: Recent shifts in the patient, family and caregiver engagement field have focused greater attention on measurement and evaluation, including the impacts of engagement efforts. Current evaluation tools offer limited support to organizations seeking to reorient their efforts in this way. We addressed this gap through the development of an impact measurement framework and accompanying evaluation toolkit-the Engage with Impact Toolkit. METHODS: The measurement framework and toolkit were co-designed with the Evaluating Patient Engagement Working Group, a multidisciplinary group of patient, family and caregiver partners, engagement specialists, researchers and government personnel. Project activities occurred over four phases: (1) project scoping and literature review; (2) modified concept mapping; (3) working group deliberations and (4) toolkit web design. RESULTS: The project scope was to develop a measurement framework and an evaluation toolkit for patient engagement in health systems that were practical, accessible, menu-driven and aligned with current system priorities. Concept mapping yielded 237 impact statements that were sorted, discussed and combined into 81 unique items. A shorter list of 50 items (rated 8.0 or higher out of 10) was further consolidated to generate a final list of 35 items mapped across 8 conceptual domains of impact: (1) knowledge and skills; (2) confidence and trust; (3) equity and inclusivity; (4) priorities and decisions; (5) effectiveness and efficiency; (6) patient-centredness; (7) culture change and (8) patient outcomes and experience. Working Group members rated the final list for importance (1-5) and identified a core set of 33 items (one for each of the 8 domains and 25 supplementary items). Two domains (priorities and decisions; and culture change) yielded the highest overall importance ratings (4.8). A web-based toolkit (www.evaluateengagement.ca) hosts the measurement framework and related evaluation supports. CONCLUSION: The Engage with Impact Toolkit builds on existing engagement evaluation tools but brings a more explicit focus to supporting organizations to assess the impacts of their engagement work. PATIENT CONTRIBUTION: Patient, family and caregiver partners led the early conceptualization of this work and were involved at all stages and in all aspects of the work. As end-users of the toolkit, their perspectives, knowledge and opinions were critical.

Towards conceptualizing patients as partners in health systems: a systematic review and descriptive synthesis.

BACKGROUND: With the sharp increase in the involvement of patients (including family and informal caregivers) as active participants, collaborators, advisors and decision-makers in health systems, a new role has emerged: the patient partner. The role of patient partner differs from other forms of patient engagement in its longitudinal and bidirectional nature. This systematic review describes extant work on how patient partners are conceptualized and engaged in health systems. In doing so, it furthers the understanding of the role and activities of patient partners, and best practices for future patient partnership activities. METHODS: A systematic review was conducted of peer-reviewed literature published in English or French that describes patient partner roles between 2000 and 2021 in any country or sector of the health system. We used a broad search strategy to capture descriptions of longitudinal patient engagement that may not have used words such as "partner" or "advisor". RESULTS: A total of 506 eligible papers were identified, representing patient partnership activities in mostly high-income countries. These studies overwhelmingly described patient partnership in health research. We identified clusters of literature about patient partnership in cancer and mental health. The literature is saturated with single-site descriptive studies of patient partnership on individual projects or initiatives. There is a lack of work synthesizing impacts, facilitating factors and outcomes of patient partnership in healthcare. CONCLUSIONS: There is not yet a consolidated understanding of the role, activities or impacts of patient partners. Advancement of the literature has been stymied by a lack of consistently used terminology. The literature is ready to move beyond single-site descriptions, and synthesis of existing pockets of high-quality theoretical work will be essential to this evolution.

Understanding the Conceptualization and Operationalization of Trauma-Informed Care Within and Across Systems: A Critical Interpretive Synthesis.

Policy Points In order to achieve successful operationalization of trauma-informed care (TIC), TIC policies must include conceptual clarity regarding the definition of both trauma and TIC. Furthermore, TIC requires clear and cohesive policies that address operational factors such as clearly delineated roles of service providers, protocol for positive trauma screens, necessary financial infrastructure, and mechanisms of intersectoral collaboration. Additionally, policy procedures need to be considered for how TIC is provided at the program and service level as well as what TIC means at the organizational, system, and intersectoral level. CONTEXT: Increased recognition of the epidemiology of trauma and its impact on individuals within and across human service delivery systems has contributed to the development of trauma-informed care (TIC). How TIC can be conceptualized and implemented, however, remains unclear. This study seeks to review and analyze the TIC literature from within and across systems of care and to generate a conceptual framework regarding TIC. METHODS: Our study followed a critical interpretive synthesis methodology. We searched multiple databases (Campbell Collaboration, Econlit, Health Systems Evidence, Embase, ERIC, HealthSTAR, IPSA, JSTOR, Medline, PsychINFO, Social Sciences Abstracts, Sociological Abstracts and Web of Science),as well as relevant gray literature and information-rich websites. We used a coding tool, adapted to the TIC literature, for data extraction. FINDINGS: Electronic database searches yielded 2,439 results and after inclusion/exclusion criteria were applied, a purposive sample of 98 information-rich articles was generated. Conceptual clarity and definitional understanding of TIC is lacking in the literature, which has led to poor operationalization of TIC. Additionally, infrastructural and ideological barriers, such as insufficient funding and service provider "buy-in," have hindered TIC implementation. The resulting conceptual framework defines trauma and depicts critical elements of vertical TIC, including the bidirectional relationship between the trauma-affected individual and the system, and horizontal TIC, which requires intersectoral collaboration, an established referral network, and standardized TIC language. CONCLUSIONS: Successful operationalization of TIC requires policies that address current gaps in systems arrangements, such as the lack of funding structures for TIC, and political factors, such as the role of policy legacies. The emergent conceptual framework acknowledges critical factors affecting operationalization.

Designing and Implementing Deliberative Processes for Health Technology Assessment: A Good Practices Report of a Joint HTAi/ISPOR Task Force.

OBJECTIVES: Deliberative processes for health technology assessment (HTA) are intended to facilitate participatory decision making, using discussion and open dialogue between stakeholders. Increasing attention is being given to deliberative processes, but guidance is lacking for those who wish to design or use them. Health Technology Assessment International (HTAi) and ISPOR-The Professional Society for Health Economics and Outcomes Research initiated a joint Task Force to address this gap. METHODS: The joint Task Force consisted of 15 members with different backgrounds, perspectives, and expertise relevant to the field. It developed guidance and a checklist for deliberative processes for HTA. The guidance builds upon the few, existing initiatives in the field, as well as input from the HTA community following an established consultation plan. In addition, the guidance was subject to 2 rounds of peer review. RESULTS: A deliberative process for HTA consists of procedures, activities, and events that support the informed and critical examination of an issue and the weighing of arguments and evidence to guide a subsequent decision. Guidance and an accompanying checklist are provided for (i) developing the governance and structure of an HTA program and (ii) informing how the various stages of an HTA process might be managed using deliberation. CONCLUSIONS: The guidance and the checklist contain a series of questions, grouped by 6 phases of a model deliberative process. They are offered as practical tools for those wishing to establish or improve deliberative processes for HTA that are fit for local contexts. The tools can also be used for independent scrutiny of deliberative processes.

Rates of health services use among residents of retirement homes in Ontario: a population-based cohort study.

BACKGROUND: Because there are no standardized reporting systems specific to residents of retirement homes in North America, little is known about the health of this distinct population of older adults. We evaluated rates of health services use by residents of retirement homes relative to those of residents of long-term care homes and other populations of older adults. METHODS: We conducted a retrospective cohort study using population health administrative data from 2018 on adults 65 years or older in Ontario. We matched the postal codes of individuals to those of licensed retirement homes to identify residents of retirement homes. Outcomes included rates of hospital-based care and physician visits. RESULTS: We identified 54 733 residents of 757 retirement homes (mean age 86.7 years, 69.0% female) and 2 354 385 residents of other settings. Compared to residents of long-term care homes, residents of retirement homes had significantly higher rates per 1000 person months of emergency department visits (10.62 v. 4.48, adjusted relative rate [RR] 2.61, 95% confidence interval [CI] 2.55 to 2.67), hospital admissions (5.42 v. 2.08, adjusted RR 2.77, 95% CI 2.71 to 2.82), alternate level of care (ALC) days (6.01 v. 2.96, adjusted RR 1.51, 95% CI 1.48 to 1.54), and specialist physician visits (6.27 v. 3.21, adjusted RR 1.64, 95% CI 1.61 to 1.68), but a significantly lower rate of primary care visits (16.71 v. 108.47, adjusted RR 0.13, 95% CI 0.13 to 0.14). INTERPRETATION: Residents of retirement homes are a distinct population with higher rates of hospital-based care. Our findings can help to inform policy debates about the need for more coordinated primary and supportive health care in privately operated congregate care homes.

The impact of COVID-19 on patient engagement in the health system: Results from a Pan-Canadian survey of patient, family and caregiver partners.

INTRODUCTION: The COVID-19 pandemic has had an impact on all aspects of the health system. Little is known about how the activities and experiences of patient, family and caregiver partners, as a large group across a variety of settings within the health system, changed due to the substantial health system shifts catalysed by the pandemic. This paper reports on the results of a survey that included questions about this topic. METHODS: Canadian patient, family and caregiver partners were invited to participate in an online anonymous survey in the Fall of 2020. A virtual snowballing approach to recruitment was used. Survey invitations were shared on social media and emailed to health system and governmental organizations with the request that they share the survey with patient partners. This paper focuses on responses to two questions related to patient partner experiences during the COVID-19 pandemic. RESULTS: The COVID-19 questions were completed by 533 respondents. Over three quarters of respondents (77.9%, n = 415) indicated their patient engagement activities had been impacted by COVID-19. The majority (62.5%, n = 230) experienced at least a temporary or partial reduction in their patient engagement activities. Some respondents did see increases in their patient engagement activities (11.4%, n = 42). Many respondents provided insights into their experience with virtual platforms for engagement (n = 194), most expressed negative or mixed experiences with this shift. CONCLUSIONS: This study provides a snapshot of Canadian patient, family and caregiver partners' perspectives on the impact of COVID-19 on their engagement activities. Understanding how engagement unfolded during a crisis is critical for our future planning if patient engagement is to be fully integrated into the health system. Identifying how patient partners were engaged and not engaged during this time period, as well as the benefits and challenges of virtual engagement opportunities, offers instructive lessons for sustaining patient engagement, including the supports needed to engage with a more diverse set of patient, family and caregiver partners. PATIENT CONTRIBUTION: Patient partners were important members of the Canadian Patient Partner Study research team. They were engaged from the outset, participating in all stages of the research project. Additional patient partners were engaged to develop and pilot test the survey, and all survey respondents were patient, family or caregiver partners. The manuscript is coauthored by two patient partners.

Designing and Implementing Deliberative Processes for Health Technology Assessment: A Good Practices Report of a Joint HTAi/ISPOR Task Force.

OBJECTIVES: Deliberative processes for health technology assessment (HTA) are intended to facilitate participatory decision making, using discussion and open dialogue between stakeholders. Increasing attention is being given to deliberative processes, but guidance is lacking for those who wish to design or use them. Health Technology Assessment International (HTAi) and ISPOR-The Professional Society for Health Economics and Outcomes Research initiated a joint Task Force to address this gap. METHODS: The joint Task Force consisted of fifteen members with different backgrounds, perspectives, and expertise relevant to the field. It developed guidance and a checklist for deliberative processes for HTA. The guidance builds upon the few, existing initiatives in the field, as well as input from the HTA community following an established consultation plan. In addition, the guidance was subject to two rounds of peer review. RESULTS: A deliberative process for HTA consists of procedures, activities, and events that support the informed and critical examination of an issue and the weighing of arguments and evidence to guide a subsequent decision. Guidance and an accompanying checklist are provided for (i) developing the governance and structure of an HTA program and (ii) informing how the various stages of an HTA process might be managed using deliberation. CONCLUSIONS: The guidance and the checklist contain a series of questions, grouped by six phases of a model deliberative process. They are offered as practical tools for those wishing to establish or improve deliberative processes for HTA that are fit for local contexts. The tools can also be used for independent scrutiny of deliberative processes.

Do assisted living facilities that offer a dementia care program differ from those that do not? A population-level cross-sectional study in Ontario, Canada.

BACKGROUND: Many residents of assisted living facilities live with dementia, but little is known about the characteristics of assisted living facilities that provide specialized care for older adults who live with dementia. In this study, we identify the characteristics of assisted living facilities that offer a dementia care program, compared to those that do not offer such a program. METHODS: We conducted a population-level cross-sectional study on all licensed assisted living facilities in Ontario, Canada in 2018 (n = 738). Facility-level characteristics (e.g., resident and suite capacities, etc.) and the provision of the other 12 provincially regulated care services (e.g., pharmacist and medical services, skin and wound care, etc.) attributed to assisted living facilities were examined. Multivariable Poisson regression with robust standard errors was used to model the characteristics of assisted living facilities associated with the provision of a dementia care program. RESULTS: There were 123 assisted living facilities that offered a dementia care program (16.7% versus 83.3% no dementia care). Nearly half of these facilities had a resident capacity exceeding 140 older adults (44.7% versus 21.6% no dementia care) and more than 115 suites (46.3% versus 20.8% no dementia care). All assisted living facilities that offered a dementia care program also offered nursing services, meals, assistance with bathing and hygiene, and administered medications. After adjustment for facility characteristics and other provincially regulated care services, the prevalence of a dementia care program was nearly three times greater in assisted living facilities that offered assistance with feeding (Prevalence Ratio [PR] 2.91, 95% Confidence Interval [CI] 1.98 to 4.29), and almost twice as great among assisted living facilities that offered medical services (PR 1.78, 95% CI 1.00 to 3.17), compared to those that did not. CONCLUSIONS: A dementia care program was more prevalent in assisted living facilities that housed many older adults, had many suites, and offered at least five of the other 12 regulated care services. Our findings deepen the understanding of specialized care for dementia in assisted living facilities.

An institutional ethnographic analysis of public and patient engagement activities at a national health technology assessment agency.

OBJECTIVE: The practice of public and patient engagement (PPE) in health technology assessment (HTA) has spread worldwide, yet gaps in knowledge remain. We carried out an institutional ethnography of the Canadian Agency for Drugs and Technologies in Health (CADTH) public and patient involvement in HTA. METHODS: The research took place over 15 months and included observational work in the institutional settings, text review, and interviews with individuals working for or involved with the agency. RESULTS: We found that despite demonstrated commitment to PPE, organizational history, governance structure, and practices were impediments to a unified approach to PPE. Unclear role descriptions for committee members and differences in philosophy and priority given to PPE across the organization presented challenges to effective participation. The high degree of value given to evidence-based principles at times conflicted with meaningful integration of patient input. A lack of clear goals and processes, roles, and differential treatment of evidence in PPE served to minimize the importance of patient experiences and to displace their validity. An acknowledgment of conflicts between multiple epistemic traditions at work within HTA activities may strengthen organizational approaches to PPE. CONCLUSION: HTA organizations can learn from this study by reflecting on the challenges described and the recommendations offered to address them. We suggest solidifying CADTH's commitment to PPE with clear agency-wide roles and direction, values, and outcomes, a comprehensive framework, and policy and procedures. An acknowledgment of diverse epistemic traditions, as well as leadership and expertise in PPE, will strengthen CADTH's PPE activities and sustain its leadership position in the HTA field.

'The problem is small enough, the problem is big enough': a qualitative study of health technology assessment and public policy on drug funding decisions for children.

BACKGROUND: Public policy approaches to funding paediatric medicines in developed public health systems remain understudied. Current approaches to HTA present a variety of conceptual, methodological and practical problems in the context of child health. This study explores the technical and sociopolitical determinants of public funding decisions on paediatric drugs, through the analysis of interviews with stakeholders involved in or impacted by HTA for child health technologies at the provincial and national levels in Canada. METHODS: We undertook in-depth interviews with a purposive sample (n = 22) of stakeholders involved with or affected by drug funding decisions for children at the provincial (Ontario) and national levels in Canada. Grounded theory methods were employed to guide data collection and analysis. Theory on 'technology-as-policy' and the sociopolitics of health technologies served as sensitizing concepts for inductive data coding and analysis. Emergent themes informed the development of conceptual and practical insights on social values and system dynamics related to child HTA, of relevance to public policymaking on the coverage of health technologies for children in Canada. RESULTS: Participant reflection on the normative and systems dimensions of drug funding for children formed two broad categories: HTA paradigms and sociopolitical context. Our analysis revealed notable differences of context and substance related to child health technology production, evaluation and use. These differences spanned the major phases of HTA (from assembly to assessment to integration) and the surrounding sociopolitical milieu (from markets to governance to politics). Careful analysis of these differences sets in relief a number of substantive and procedural shortcomings of current HTA paradigms in respect of child health. Our findings suggest a need to rethink how HTA is structured and operationalized for child health technologies. CONCLUSIONS: Current approaches to health technology assessment are not well calibrated to the realities of child health and illness. Our study presents a nuanced and contextually grounded analysis of concepts instrumental to drug funding decisions for children. The insights generated are directly applicable to the Canadian and Ontario contexts, but also yield fundamental knowledge about HTA for children that are germane to drug policy in other health systems.

Priority measures for publicly reporting primary care performance: Results of public engagement through deliberative dialogues in 3 Canadian provinces.

OBJECTIVE: While public reporting of hospital-based performance measurement is commonplace, it has lagged in the primary care sector, especially in Canada. Despite the increasing recognition of patients as active partners in the health-care system, little is known about what information about primary care performance is relevant to the Canadian public. We explored patient perspectives and priorities for the public reporting of primary care performance measures. METHODS: We conducted six deliberative dialogue sessions across three Canadian provinces (British Columbia, Ontario, Nova Scotia). Participants were asked to rank and discuss the importance of collecting and reporting on specific dimensions and indicators of primary care performance. We conducted a thematic analysis of the data. RESULTS: Fifty-six patients participated in the dialogue sessions. Measures of access to primary care providers, communication with providers and continuity of information across all providers involved in a patient's care were identified as the highest priority indicators of primary care performance from a patient perspective. Several common measures of quality of care, such as rates of cancer screening, were viewed as too patient dependent to be used to evaluate the health system or primary care provider's performance. CONCLUSIONS: Our findings suggest that public reporting aimed at patient audiences should focus on a nuanced measure of access, incorporation of context reported alongside measurement that is for public audiences, clear reporting on provider communication and a measure of information continuity. Participants highlighted the importance the public places on their providers staying up to date with advances in care.

Understanding health professional role integration in complex adaptive systems: a multiple-case study of physician assistants in Ontario, Canada.

BACKGROUND: To meet the complex needs of healthcare delivery, the Ministry of Health and Long Term Care (MOHLTC) introduced Physician Assistants (PAs) into the Ontario health care system in 2006 with the goal of helping to increase access to care, decrease wait times, improve continuity of care and provide a flexible addition to the healthcare workforce. The characterization of healthcare organizations as complex adaptive systems (CAS) may offer insight into the relationships and interactions that optimize and restrict successful PA integration. The aim of this study is to explore the integration of PAs across multiple case settings and to understand the role of PAs within complex adaptive systems. METHODS: An exploratory, multiple-case study was used to examine PA role integration in four settings: family medicine, emergency medicine, general surgery, and inpatient medicine. Interviews were conducted with 46 healthcare providers and administrators across 13 hospitals and 6 family medicine clinics in Ontario, Canada. Analysis was conducted in three phases including an inductive thematic analysis within each of the four cases, a cross-case thematic analysis, and a broader, deductive exploration of cross-case patterns pertaining to specific complexity theory principles of interest. RESULTS: Forty-six health care providers were interviewed across 19 different healthcare sites. Support for PA contributions across various health care settings, the importance of role awareness, supervisory relationship attributes, and role vulnerability are interconnected and dynamic. Findings represent the experiences of PAs and other healthcare providers, and demonstrate how the PAs willingness to work and ability to build relationships allows for the establishment of interprofessional, collaborative, and person-centered care. As a self-organizing agent in complex adaptive systems (i.e., health organizations), PA role exploration revealed patterns of team behavior, non-linear interconnections, open relationships, dynamic systems, and the legacy of role implementation as defined by complexity theory. CONCLUSIONS: By exploring the role of PAs across multiple sites, the complexity theory lens concurrently fosters an awareness of emerging patterns, relationships and non-linear interactions within the defined context of the Ontario healthcare system. By establishing collaborative, interprofessional care models in hospital and community settings, PAs are making a significant contribution to Ontario healthcare settings.

A framework for explaining the role of values in health policy decision-making in Latin America: a critical interpretive synthesis.

BACKGROUND: Although values underpin the goals pursued in health systems, including how health systems benefit the population, it is often not clear how values are incorporated into policy decision-making about health systems. The challenge is to encompass social/citizen values, health system goals, and financial realities and to incorporate them into the policy-making process. This is a challenge for all health systems and of particular importance for Latin American (LA) countries. Our objective was to understand how and under what conditions societal values inform decisions about health system financing in LA countries. METHODS: A critical interpretive synthesis approach was utilised for this work. We searched 17 databases in December 2016 to identify articles written in English, Spanish or Portuguese that focus on values that inform the policy process for health system financing in LA countries at the macro and meso levels. Two reviewers independently screened records and assessed them for inclusion. One researcher conceptually mapped the included articles, created structured summaries of key findings from each, and selected a purposive sample of articles to thematically synthesise the results across the domains of agenda-setting/prioritisation, policy development and implementation. RESULTS: We identified 5925 references, included 199 papers, and synthesised 68 papers. We identified 116 values and developed a framework to explain how values have been used to inform policy decisions about financing in LA countries. This framework has four categories - (1) goal-related values (i.e. guiding principles of the health system); (2) technical values (those incorporated into the instruments adopted by policy-makers to ensure a sustainable and efficient health system); (3) governance values (those applied in the policy process to ensure a transparent and accountable process of decision-making); and (4) situational values (a broad category of values that represent competing strategies to make decisions in the health systems, their influence varying according to the four factors). CONCLUSIONS: It is an effort to consolidate and explain how different social values are considered and how they support policy decision-making about health system financing. This can help policy-makers to explicitly incorporate values into the policy process and understand how values are supporting the achievement of policy goals in health system financing. TRIAL REGISTRATION: The protocol was registered with PROSPERO, ID=CRD42017057049 .

Supporting the evaluation of public and patient engagement in health system organizations: Results from an implementation research study.

BACKGROUND: As citizens, patients and family members are participating in numerous and expanding roles in health system organizations, attention has turned to evaluating these efforts. The context-specific nature of engagement requires evaluation tools to be carefully designed for optimal use. We sought to address this need by assessing the appropriateness and feasibility of a generic tool across a range of health system organizations, engagement activities and patient groups. METHODS: We used a mixed-methods implementation research design to study the implementation of an engagement evaluation tool in seven health system organizations in Ontario, Canada focusing on two key implementation outcome variables: appropriateness and feasibility. Data were collected through respondent feedback questions (binary and open-ended) at the end of the tool's three questionnaires as well as interviews and debriefing discussions with engagement professionals and patient partners from collaborating organizations. RESULTS: The three questionnaires comprising the evaluation tool were collectively administered 29 times to 405 respondents yielding a 52% response rate (90% and 53% of respondents respectively assessed the survey's appropriateness and feasibility [quantitatively or qualitatively]). The questionnaires' basic properties were rated highly by all respondents. Concrete suggestions were provided for improving the appropriateness and feasibility of the questionnaires (or components within) for different engagement activity and organization types, and for enhancing the timing of implementation. DISCUSSION AND CONCLUSIONS: Our study findings offer guidance for health system organizations and evaluators to support the optimal use of engagement evaluation tools across a variety of health system settings, engagement activities and respondent groups.

Citizen perspectives on the use of publicly reported primary care performance information: Results from citizen-patient dialogues in three Canadian provinces.

OBJECTIVE: Performance measurement and reporting is proliferating in all sectors of the healthcare system, including primary care, despite a dearth of evidence on how the public uses reports on primary care performance. We explored how the public might use this information, to guide the development of effective reporting systems for primary care. METHODS: We conducted six full-day deliberative dialogue sessions with a purposive sample of 56 citizen-patients across three Canadian provinces (British Columbia, Ontario and Nova Scotia). Participants identified how they would use publicly reported performance data. We conducted a thematic analysis of the data by region. RESULTS: Common uses for primary care performance information emerged across all sessions. Participants most often discussed the utility of this information for community advocacy and participation in health system decision making. Similar barriers for using performance information to choose a primary care provider were identified in each region including the perceived lack of choice of providers and the high value placed on relationships with current providers. Finally, the value of public performance reporting in enhancing trust that people would receive good care was also a common theme. CONCLUSIONS: Citizen-patient perspectives highlight that public reporting on primary care performance could promote the health system's responsiveness by enabling public engagement in decision making at the community level. The role of public reporting in promoting trust rather than empowering patient choice may reflect unique elements of the Canadian health system's context.

Evaluating community deliberations about health research priorities.

CONTEXT: Engaging underrepresented communities in health research priority setting could make the scientific agenda more equitable and more responsive to their needs. OBJECTIVE: Evaluate democratic deliberations engaging minority and underserved communities in setting health research priorities. METHODS: Participants from underrepresented communities throughout Michigan (47 groups, n = 519) engaged in structured deliberations about health research priorities in professionally facilitated groups. We evaluated some aspects of the structure, process, and outcomes of deliberations, including representation, equality of participation, participants' views of deliberations, and the impact of group deliberations on individual participants' knowledge, attitudes, and points of view. Follow-up interviews elicited richer descriptions of these and also explored later effects on deliberators. RESULTS: Deliberators (age 18-88 years) overrepresented minority groups. Participation in discussions was well distributed. Deliberators improved their knowledge about disparities, but not about health research. Participants, on average, supported using their group's decision to inform decision makers and would trust a process like this to inform funding decisions. Views of deliberations were the strongest predictor of these outcomes. Follow-up interviews revealed deliberators were particularly struck by their experience hearing and understanding other points of view, sometimes surprised at the group's ability to reach agreement, and occasionally activated to volunteer or advocate. CONCLUSIONS: Deliberations using a structured group exercise to engage minority and underserved community members in setting health research priorities met some important criteria for a fair, credible process that could inform policy. Deliberations appeared to change some opinions, improved some knowledge, and were judged by participants worth using to inform policymakers.

Ethical Challenges Related to Patient Involvement in Health Technology Assessment.

Including information and values from patients in HTA has the potential to improve both the process and outcomes of health technology policy decisions. Accordingly, funding and structural incentives to include patients in HTA activities have increased over the past several years. Unfortunately, these incentives have not yet been accompanied by a corresponding increase in resources, time, or commitment to responsiveness. In this Perspectives piece, we reflect on our collective experiences participating in, conducting, and overseeing patient engagement activities within HTA to highlight the ethical challenges associated with this area of activity. While we remain committed to the idea that patient engagement activities strengthen the findings, relevance, and legitimacy of health technology policy, we are deeply concerned about the potential for these activities to do ethical harm. We use this analysis to call for action to introduce strong protections against ethical violations that may harm patients participating in HTA engagement activities.

Addressing the affordability of cancer drugs: using deliberative public engagement to inform health policy.

BACKGROUND: Health system expenditure on cancer drugs is rising rapidly in many OECD countries given the costly new treatments and increased rates of use due to a growing and ageing population. These factors put considerable strain on the sustainability of health systems worldwide, sparking public debate among clinicians, pharmaceutical companies, policy-makers and citizens on issues of affordability and equity. We engaged Canadians through a series of deliberative public engagement events to determine their priorities for making cancer drug funding decisions fair and sustainable in Canada's publicly financed health system. METHODS: An approach to deliberation was developed based on the McMaster Health Forum's citizen panels and the established Burgess and O'Doherty model of deliberative public engagement. Six deliberations were held across Canada in 2016. Transcripts were coded in NVivo and analysed to determine where participants' views converged and diverged. Recommendations were grouped thematically. RESULTS: A total of 115 Canadians participated in the deliberative events and developed 86 recommendations. Recommendations included the review and regular re-review of approved drugs using 'real-world' evidence on effectiveness and cost-effectiveness; prioritisation of treatments that restore patients' independence, mental health and general well-being; ensuring that decision processes, results and their rationales are transparent; and commitment to people with similar needs receiving the same care regardless of where in Canada they live. CONCLUSIONS: The next steps for policy-makers should be to develop mechanisms for (1) re-reviewing effectiveness and cost-effectiveness data for all cancer drugs; (2) making disinvestments in cancer drugs that satisfy requirements relating to grandfathering and compassionate access; (3) ensuring fair and equitable access to cancer drugs for all Canadians; and (4) fostering a pan-Canadian approach to cancer drug funding decisions.

Members of Minority and Underserved Communities Set Priorities for Health Research.

Policy Points Engaging and involving underrepresented communities when setting research priorities could make the scientific research agenda more equitable, more just, and more responsive to their needs and values. Groups and individuals from minority and underserved communities strongly prioritized child health and mental health research, often choosing to invest at the highest possible level. Groups consisting of predominantly Native American or Arab American participants invested in culture and beliefs research at the highest level, while many groups did not select it at all. The priority given to culture and beliefs research by these groups illustrates the importance of paying special attention to unique preferences, and not just commonly held views, when getting public input on spending priorities for research. CONTEXT: A major contributor to health disparities is the relative lack of resources-including resources for science-allocated to address the health problems of those with disproportionately greater needs. Engaging and involving underrepresented communities in setting research priorities could make the scientific research agenda more equitable, more just, and more responsive to their needs and values. We engaged minority and underserved communities in informed deliberations and report here their priorities for health research. METHODS: Academic-community partnerships adapted the simulation exercise CHAT for setting health research priorities. We had participants from minority and medically underserved communities (47 groups, n = 519) throughout Michigan deliberate about health research priorities, and we used surveys and CHAT software to collect the demographic characteristics and priorities selected by individuals and groups. FINDINGS: The participants ranged in age (18 to 88), included more women than men, and were overrepresented by minority groups. Nearly all the deliberating groups selected child health and mental health research (93.6% and 95.7%), and most invested at the highest level. Aging, access, promote health, healthy environment, and what causes disease were also prioritized by groups. Research on mental health and child health were high priorities for individuals both before and after group deliberations. Access was the only category more likely to be selected by individuals after group deliberation (77.0 vs 84.0%, OR = 1.63, p = .005), while improve research, health policy, and culture and beliefs were less likely to be selected after group deliberations (all, p < .001). However, the level of investment in many categories changed after the group deliberations. Participants identifying as Black/African American were less likely to prioritize mental health research, and those of Other race were more likely to prioritize culture and beliefs research. CONCLUSIONS: Minority and medically underserved communities overwhelmingly prioritized mental health and child health research in informed deliberations about spending priorities.