Evaluating comprehensive cancer networks; a review of standards and evaluation methods for care networks to inform a comparison with the OECI comprehensive cancer network standards.

INTRODUCTION: To ensure that all citizens have equal access to high-quality cancer diagnosis and care, the EU4Health Programme, Europe's Beating Cancer Plan, and Horizon Europe's Cancer Mission propose Comprehensive Cancer Infrastructures in every European Union Member State. It is therefore important to establish the basic principles for high-performing cancer networks and a methodology for evaluating their quality and effectiveness. This article describes methods and standards/indicators for network evaluation found in literature, gives a comparative overview of the new OECI European Cancer Network Quality standards, and proposes principles for evaluating the performance of Comprehensive Cancer Networks as a basis for continuous improvement. MATERIALS AND METHODS: We performed a scoping literature review on methods and standards/indicators for care-network evaluation. We then compared the OECI set with literature findings, categorised standards that were similar, reflected on standards that were different, and deduced principles for quality standards for cancer networks. RESULTS: Of 1002 articles identified, 17 reported on evaluation methods and/or (mostly) qualitative indicators. Sixteen studies described indicators/standards for evaluating care networks, critical success factors or desirable outcomes. Of the 54 present OECI standards, 32 had a literature equivalent. No literature equivalent was found for 22 standards, especially on those related to the combination of care and research. The proposed OECI evaluation methods (survey, document review, and interviews) were all reported in the literature. From the conformity of these results, we deduced 8 principles for standards evaluating the effectiveness of Comprehensive Cancer Networks. CONCLUSIONS: Research on the evaluation of the effectiveness of care networks is scarce. Evaluation methods vary and are often single time-point assessments. The OECI set contributes to establishing clear principles and standards to evaluate the effectiveness of Comprehensive Cancer Networks.

Benchmarking Cancer Centers: From Care Pathways to Integrated Practice Units.

Background: Structuring cancer care into pathways can reduce variability in clinical practice and improve patient outcomes. International benchmarking can help centers with regard to development, implementation, and evaluation. A further step in the development of multidisciplinary care is to organize care in integrated practice units (IPUs), encompassing the whole pathway and relevant organizational aspects. However, research on this topic is limited. This article describes the development and results of a benchmark tool for cancer care pathways and explores IPU development in cancer centers. Methods: The benchmark tool was developed according to a 13-step benchmarking method and piloted in 7 European cancer centers. Centers provided data and site visits were performed to understand the context in which the cancer center operates and to clarify additional questions. Benchmark data were structured into pathway development and evaluation and assessed against key IPU features. Results: Benchmark results showed that most centers have formalized multidisciplinary pathways and that care teams differed in composition, and found almost 2-fold differences in mammography use efficiency. Suggestions for improvement included positioning pathways formally and structurally evaluating outcomes at a sufficiently high frequency. Based on the benchmark, 3 centers indicating that they had a breast cancer IPU were scored differently on implementation. Overall, we found that centers in Europe are in various stages of development of pathways and IPUs, ranging from an informal pathway structure to a full IPU-type of organization. Conclusions: A benchmark tool for care pathways was successfully developed and tested, and is available in an open format. Our tool allows for the assessment of pathway organization and can be used to assess the status of IPU development. Opportunities for improvement were identified regarding the organization of care pathways and the development toward IPUs. Three centers are in varying degrees of implementation and can be characterized as breast cancer IPUs. Organizing cancer care in an IPU could yield multiple performance improvements.

Development of a benchmark tool for cancer centers; results from a pilot exercise.

BACKGROUND: Differences in cancer survival exist between countries in Europe. Benchmarking of good practices can assist cancer centers to improve their services aiming for reduced inequalities. The aim of the BENCH-CAN project was to develop a cancer care benchmark tool, identify performance differences and yield good practice examples, contributing to improving the quality of interdisciplinary care. This paper describes the development of this benchmark tool and its validation in cancer centers throughout Europe. METHODS: A benchmark tool was developed and executed according to a 13 step benchmarking process. Indicator selection was based on literature, existing accreditation systems, and expert opinions. A final format was tested in eight cancer centers. Center visits by a team of minimally 3 persons, including a patient representative, were performed to verify information, grasp context and check on additional questions (through semi-structured interviews). Based on the visits, the benchmark methodology identified opportunities for improvement. RESULTS: The final tool existed of 61 qualitative and 141 quantitative indicators, which were structured in an evaluative framework. Data from all eight participating centers showed inter-organization variability on many indicators, such as bed utilization and provision of survivorship care. Subsequently, improvement suggestions for centers were made; 85% of which were agreed upon. CONCLUSION: A benchmarking tool for cancer centers was successfully developed and tested and is available in an open format. The tool allows comparison of inter-organizational performance. Improvement opportunities were successfully identified for every center involved and the tool was positively evaluated.

Assessing and comparing the quality of wound centres: a literature review and benchmarking pilot.

Multidisciplinary wound centres are currently facing an increase in both the incidence of wounds and the complexity of care. This has resulted in rising costs and increased interest in the effectiveness of treatments. Little evidence is available regarding optimal wound centre organisation and effectiveness; therefore, measuring the quality of wound centres has become more important. This study aims to assess the evidence concerning quality by describing the state of the art of wound centres and organisational effectiveness by developing indicators of quality and by assessing their suitability in a pilot study. A multi-method approach was used: a literature review performed resulted in the development of an indicator list that was consequently subjected to expert review, and a benchmark study was completed comparing eight wound centres in the Netherlands. We thus provide a description of the relevant state-of-the-art aspects of wound centre organisation, which were multidisciplinary collaborations and standardisation of the organisation of care. In literature, significant patient-related effects were observed in improved healing rates and decreased costs. A total of 48 indicators were selected. The indicator list was tested by a benchmark study pilot. In practice, the outcome indicators were especially difficult to generate. Six indicators regarding structure, three regarding process and five regarding outcome proved feasible to measure and improve quality of wound centres.

Piloting a generic cancer consumer quality index in six European countries.

BACKGROUND: Accounting for patients' perspective has become increasingly important. Based on the Consumer Quality Index method (founded on Consumer Assessment of Healthcare Providers and Systems) a questionnaire was recently developed for Dutch cancer patients. As a next step, this study aimed to adapt and pilot this questionnaire for international comparison of cancer patients experience and satisfaction with care in six European countries. METHOD: The Consumer Quality Index was translated into the local language at the participating pilot sites using cross-translation. A minimum of 100 patients per site were surveyed through convenience sampling. Data from seven pilot sites in six countries was collected through an online and paper-based survey. Internal consistency was tested by calculating Cronbach's alpha and validity by means of cognitive interviews. Demographic factors were compared as possible influencing factors. RESULTS: A total of 698 patients from six European countries filled the questionnaire. Cronbach's alpha was good or satisfactory in 8 out of 10 categories. Patient satisfaction significantly differed between the countries. We observed no difference in patient satisfaction for age, gender, education, and tumor type, but satisfaction was significantly higher in patients with a higher level of activation. CONCLUSION: This European Cancer Consumer Quality Index(ECCQI) showed promising scores on internal consistency (reliability) and a good internal validity. The ECCQI is to our knowledge the first to measure and compare experiences and satisfaction of cancer patients on an international level, it may enable healthcare providers to improve the quality of cancer care.

Quality assessments for cancer centers in the European Union.

BACKGROUND: Cancer centers are pressured to deliver high-quality services that can be measured and improved, which has led to an increase of assessments in many countries. A critical area of quality improvement is to improve patient outcome. An overview of existing assessments can help stakeholders (e.g., healthcare professionals, managers and policy makers) improve the quality of cancer research and care and lead to patient benefits. This paper presents key aspects of assessments undertaken by European cancer centers, such as: are assessments mandatory or voluntary? Do they focus on evaluating research, care or both? And are they international or national? METHODS: A survey was sent to 33 cancer centers in 28 European Union member states. Participants were asked to score the specifics for each assessment that they listed. RESULTS: Based on the responses from 19 cancer centers from 18 member states, we found 109 assessments. The numbers have steadily increased from 1990's till 2015. Although, a majority of assessments are on patient-care aspects (n = 45), it is unclear how many of those include assessing patient benefits. Only few assessments cover basic research. There is an increasing trend towards mixed assessments (i.e., combining research and patient-care aspects) CONCLUSIONS: The need for assessments in cancer centers is increasing. To improve efforts in the quality of research and patient care and to prevent new assessments that "reinvent the wheel", it is advised to start comparative research into the assessments that are likely to bring patient benefits and improve patient outcome. Do assessments provide consistent and reliable information that create added value for all key stakeholders?

Improving shared decision making for lung cancer treatment by developing and validating an open-source web based patient decision aid for stage I-II non-small cell lung cancer.

The aim of this study was to develop and evaluate a proof-of-concept open-source individualized Patient Decision Aid (iPDA) with a group of patients, physicians, and computer scientists. The iPDA was developed based on the International Patient Decision Aid Standards (IPDAS). A previously published questionnaire was adapted and used to test the user-friendliness and content of the iPDA. The questionnaire contained 40 multiple-choice questions, and answers were given on a 5-point Likert Scale (1-5) ranging from "strongly disagree" to "strongly agree." In addition to the questionnaire, semi-structured interviews were conducted with patients. We performed a descriptive analysis of the responses. The iPDA was evaluated by 28 computer scientists, 21 physicians, and 13 patients. The results demonstrate that the iPDA was found valuable by 92% (patients), 96% (computer scientists), and 86% (physicians), while the treatment information was judged useful by 92%, 96%, and 95%, respectively. Additionally, the tool was thought to be motivating for patients to actively engage in their treatment by 92%, 93%, and 91% of the above respondents groups. More multimedia components and less text were suggested by the respondents as ways to improve the tool and user interface. In conclusion, we successfully developed and tested an iPDA for patients with stage I-II Non-Small Cell Lung Cancer (NSCLC).

Patient involvement in clinical pathway development, implementation and evaluation - A scoping review of international literature.

OBJECTIVE: Although various pathway design methods recognize patients as stakeholders, an overview of current practice is lacking. This article describes the results of a literature review assessing patient involvement in clinical cancer pathway development, implementation and evaluation. METHODS: A scoping review was conducted following PRISMA-ScR. Two databases were searched to identify studies published in English between 2014 and 2021. RESULTS: Of 12841articles identified 22 articles met the inclusion criteria and reported on one or more of the three phases: development phase (N = 2), implementation (N = 4), evaluation (N = 11), development/evaluation (N = 3), and implementation/evaluation (N = 2) of clinical pathways. The numbers of involved patients ranged from 10 to 793, and the reported methods varied considerably. CONCLUSION: This review presents a synthesis of methods for involving patients in the clinical pathway lifecycle. No relationship was found between methods and the number of involved patients or between pathway complexity and methods. Although patients are seen as valuable stakeholders in the pathway design, to involve them in practice using the best practice can be improved. PRACTICE IMPLICATIONS: The lack of a clear justification for the choice of methods and number of involved patients calls for further research and framework development to inform pathway developers.

Evaluating Cancer Care Networks; A Case Study of a Lung Cancer Care Network.

BACKGROUND: Networks are promoted as an organizational form that enables integrated care as well as enhanced patient outcomes. However, implementing networks is complex. It is therefore important to evaluate the quality and effectiveness of networks to ensure it is worth developing and maintaining them. This article describes the development of an evaluation tool for cancer care networks and the results of a pilot study with a regional lung cancer care network. METHODS: This study used a combination of qualitative and quantitative evaluation methods. The qualitative evaluation was based on a framework with 10 standards for the organization of an oncological (tumor-specific) care network. Data for the quantitative evaluation were obtained from the Dutch Cancer Registry. The evaluation was performed at a network of three hospitals collaborating in the field of lung oncology. RESULTS: The qualitative evaluation framework consisted of 10 standards/questions which were divided into 38 sub-questions. The evaluation showed that in general patients are satisfied with the collaboration in the network. However, some improvement points were found such as the need for more attention for the implementation and periodic evaluation of a regional care pathway. The start of a regional multidisciplinary meeting has been a major step for improving the collaboration. CONCLUSION: An evaluation tool for (lung) cancer care networks was successfully developed and piloted within a cancer care network. The tool has proven to be a useful method for evaluating collaboration within an oncological network. It helped network partners to understand what they see as important and allowed them to learn about their program's dynamics. Improvement opportunities were successfully identified. To keep the tool up to date continuous improvement is needed, following the Plan Do Check Act (PDCA) cycle.

Open Source Repository and Online Calculator of Prediction Models for Diagnosis and Prognosis in Oncology.

(1) Background: The main aim was to develop a prototype application that would serve as an open-source repository for a curated subset of predictive and prognostic models regarding oncology, and provide a user-friendly interface for the included models to allow online calculation. The focus of the application is on providing physicians and health professionals with patient-specific information regarding treatment plans, survival rates, and side effects for different expected treatments. (2) Methods: The primarily used models were the ones developed by our research group in the past. This selection was completed by a number of models, addressing the same cancer types but focusing on other outcomes that were selected based on a literature search in PubMed and Medline databases. All selected models were publicly available and had been validated TRIPOD (Transparent Reporting of studies on prediction models for Individual Prognosis Or Diagnosis) type 3 or 2b. (3) Results: The open source repository currently incorporates 18 models from different research groups, evaluated on datasets from different countries. Model types included logistic regression, Cox regression, and recursive partition analysis (decision trees). (4) Conclusions: An application was developed to enable physicians to complement their clinical judgment with user-friendly patient-specific predictions using models that have received internal/external validation. Additionally, this platform enables researchers to display their work, enhancing the use and exposure of their models.

Analysing the attributes of Comprehensive Cancer Centres and Cancer Centres across Europe to identify key hallmarks.

There is a persistent variation in cancer outcomes among and within European countries suggesting (among other causes) inequalities in access to or delivery of high-quality cancer care. European policy (EU Cancer Mission and Europe's Beating Cancer Plan) is currently moving towards a mission-oriented approach addressing these inequalities. In this study, we used the quantitative and qualitative data of the Organisation of European Cancer Institutes' Accreditation and Designation Programme, relating to 40 large European cancer centres, to describe their current compliance with quality standards, to identify the hallmarks common to all centres and to show the distinctive features of Comprehensive Cancer Centres. All Comprehensive Cancer Centres and Cancer Centres accredited by the Organisation of European Cancer Institutes show good compliance with quality standards related to care, multidisciplinarity and patient centredness. However, Comprehensive Cancer Centres on average showed significantly better scores on indicators related to the volume, quality and integration of translational research, such as high-impact publications, clinical trial activity (especially in phase I and phase IIa trials) and filing more patents as early indicators of innovation. However, irrespective of their size, centres show significant variability regarding effective governance when functioning as entities within larger hospitals.

Staff perceptions of change resulting from participation in a European cancer accreditation programme: a snapshot from eight cancer centres.

BACKGROUND: Healthcare accreditation is considered to be an essential quality improvement tool. However, its effectiveness has been critiqued. METHODS: Twenty-four interviews were conducted with clinicians (five), nurses (six), managers (eight), and basic/translational researchers (five) from eight European cancer centres on changes observed from participating in a European cancer accreditation programme. Data were thematically analysed and verified with participants and checked against auditor's feedback. RESULTS: Four change categories emerged: (i) the growing importance of the nursing and supportive care field (role change). Nurses gained more autonomy/clarity on their daily duties. Importance was given to the hiring and training of supportive care personnel (ii) critical thinking on data integration (strategic change). Managers gained insight on how to integrate institutional level data (iii) improved processes within multidisciplinary team (MDT) meetings (procedural change). Clinical staff experienced improved communication between MDTs (iv) building trust (organisational change). Accreditation improved the centre's credibility with its own staff and externally with funders and patients. No motivational changes were perceived. Researchers perceived no changes. The auditor's feedback included changes in 13 areas: translational research, biobanks, clinical trials, patient privacy and satisfaction, cancer registries, clinical practice guidelines, patient education, screening, primary prevention, role of nurses, MDT, supportive care, and data integration. However, our study revealed that staff perceived changes only in the last four areas. CONCLUSION: Staff perceived changes in data integration, nursing and supportive care, and in certain clinical aspects. Accreditation programmes must pay attention to the needs of different stakeholder groups, track changes, and observe how/why change happens.