What do end-users want to know about managing the performance of healthcare delivery systems? Co-designing a context-specific and practice-relevant research agenda.

BACKGROUND: Despite increasing interest in joint research priority-setting, few studies engage end-user groups in setting research priorities at the intersection of the healthcare and management disciplines. With health systems increasingly establishing performance management programmes to account for and incentivize performance, it is important to conduct research that is actionable by the end-users involved with or impacted by these programmes. The aim of this study was to co-design a research agenda on healthcare performance management with and for end-users in a specific jurisdictional and policy context. METHODS: We undertook a rapid review of the literature on healthcare performance management (n = 115) and conducted end-user interviews (n = 156) that included a quantitative ranking exercise to prioritize five directions for future research. The quantitative rankings were analysed using four methods: mean, median, frequency ranked first or second, and frequency ranked fifth. The interview transcripts were coded inductively and analysed thematically to identify common patterns across participant responses. RESULTS: Seventy-three individual and group interviews were conducted with 156 end-users representing diverse end-user groups, including administrators, clinicians and patients, among others. End-user groups prioritized different research directions based on their experiences and information needs. Despite this variation, the research direction on motivating performance improvement had the highest overall mean ranking and was most often ranked first or second and least often ranked fifth. The research direction was modified based on end-user feedback to include an explicit behaviour change lens and stronger consideration for the influence of context. CONCLUSIONS: Joint research priority-setting resulted in a practice-driven research agenda capable of generating results to inform policy and management practice in healthcare as well as contribute to the literature. The results suggest that end-users are keen to open the "black box" of performance management to explore more nuanced questions beyond "does performance management work?" End-users want to know how, when and why performance management contributes to behaviour change (or fails to) among front-line care providers.

From Measurement to Improvement in Ontario's Cancer System: Analyzing the Performance of 28 Provincial Indicators over 15 Years.

Over the past 15 years, Cancer Care Ontario has used a robust performance management approach to drive improvements in care. Each year, priority indicators and targets are selected or retained, and performance is reviewed quarterly with each of Ontario's Regional Cancer Programs. Improvement support and encouragement are provided, such as data analysis, program ranking, communities of practice, consultations, action plan requests and certificates. This article analyzes data on 28 indicators prioritized over these years and demonstrates that 25 have shown sustained improvement over time. The performance management approach, lessons learned and gaps in knowledge are described to inform future research and practice.

Consolidated principles for screening based on a systematic review and consensus process.

BACKGROUND: In 1968, Wilson and Jungner published 10 principles of screening that often represent the de facto starting point for screening decisions today; 50 years on, are these principles still the right ones? Our objectives were to review published work that presents principles for population-based screening decisions since Wilson and Jungner's seminal publication, and to conduct a Delphi consensus process to assess the review results. METHODS: We conducted a systematic review and modified Delphi consensus process. We searched multiple databases for articles published in English in 1968 or later that were intended to guide population-based screening decisions, described development and modification of principles, and presented principles as a set or list. Identified sets were compared for basic characteristics (e.g., number, categorization), a citation analysis was conducted, and principles were iteratively synthesized and consolidated into categories to assess evolution. Participants in the consensus process assessed the level of agreement with the importance and interpretability of the consolidated screening principles. RESULTS: We identified 41 sets and 367 unique principles. Each unique principle was coded to 12 consolidated decision principles that were further categorized as disease/condition, test/intervention or program/system principles. Program or system issues were the focus of 3 of Wilson and Jungner's 10 principles, but comprised almost half of all unique principles identified in the review. The 12 consolidated principles were assessed through 2 rounds of the consensus process, leading to specific refinements to improve their relevance and interpretability. No gaps or missing principles were identified. INTERPRETATION: Wilson and Jungner's principles are remarkably enduring, but increasingly reflect a truncated version of contemporary thinking on screening that does not fully capture subsequent focus on program or system principles. Ultimately, this review and consensus process provides a comprehensive and iterative modernization of guidance to inform population-based screening decisions.

Interviewee Transcript Review: assessing the impact on qualitative research.

BACKGROUND: This paper assesses interviewee transcript review (ITR) as a technique for improving the rigour of interview-based, qualitative research. ITR is a process whereby interviewees are provided with verbatim transcripts of their interviews for the purposes of verifying accuracy, correcting errors or inaccuracies and providing clarifications. ITR, in various forms, is widely used among qualitative researchers, however there is limited methodological guidance on how it should be employed and little is known about its actual impact on the transcript, the data, the interviewee or the researcher. METHODS: ITR was incorporated into a qualitative research study in which 51 key informant interviews were conducted with a range of senior stakeholders within the Canadian health care system. The changes made by interviewees to their transcripts were systematically tracked and categorized using a set of mutually exclusive categories. RESULTS: The study found that ITR added little to the accuracy of the transcript and may create complications if the goal of the researcher is to produce a transcript which reflects precisely what was said at the time of the interview. The advantages of ITR are that it allows interviewees the opportunity to edit or clarify information provided in the original interview, with many interviewees providing corrections, clarifications, and in some cases, adding new material to their transcripts. There are also potential disadvantages, such as a bias created by inconsistent data sources or the loss of data when an interviewee chooses to remove valuable material. The impact of ITR on the interviewee may be both positive and negative, depending on the specific circumstances and the nature of the study. The impact of ITR on the researcher was minimal in this study, but is again subject to specific circumstances of the research context. CONCLUSION: While ITR is employed by many researchers across numerous fields, the advantages of its use may be relatively small in terms of verifying the accuracy of qualitative interview transcripts. Researchers are advised to carefully consider both the potential advantages and disadvantages of ITR outlined in this paper before deciding to incorporate the practice within their qualitative study designs.