Outcome selection, measurement and reporting for new surgical procedures and devices: a systematic review of IDEAL/IDEAL-D studies to inform development of a core outcome set.

BACKGROUND: Outcome selection, measurement and reporting for the evaluation of new surgical procedures and devices is inconsistent and lacks standardization. A core outcome set may promote the safe and transparent evaluation of surgical innovations. This systematic review examined outcome selection, measurement and reporting in studies conducted within the IDEAL (Idea, Development, Exploration, Assessment and Long-term monitoring) framework to examine current practice and inform the development of a core outcome set for early-phase studies of surgical procedures/devices. METHODS: Web of Science and Scopus citation searches were performed to identify author-reported IDEAL/IDEAL-D studies for any surgical procedure/device. Outcomes were extracted verbatim, including contextual information regarding outcome selection and measurement. Outcomes were categorized to inform a conceptual framework of outcome domains relevant to evaluating innovation. RESULTS: Some 48 studies were identified. Outcome selection, measurement and reporting varied widely across studies in different IDEAL stages. From 1737 outcomes extracted, 22 domains specific to evaluating innovation were conceptualized under seven broad categories: procedure completion success/failure; modifications; unanticipated events; surgeons' experiences; patients' experiences; resource use specific to the innovative procedure/device; and other innovation-specific outcomes. Most innovation-specific outcomes were measured and reported in only a small number of studies. CONCLUSION: This review highlighted the need for guidance and standardization in outcome selection and reporting in the evaluation of new surgical procedures/devices. Novel outcome domains specific to innovation have been identified to establish a core outcome set for future evaluations of surgical innovations.

ANTECEDENTES: La selección de resultados, mediciones y redacción de los informes para la evaluación de nuevos procedimientos y dispositivos quirúrgicos es inconsistente y carece de estandarización. Determinar un conjunto de resultados básicos (core outcome set, COS) podría contribuir a la transparencia y seguridad de las evaluaciones de las innovaciones quirúrgicas. Esta revisión sistemática analizó la selección de resultados, medición de los mismos e informes de estudios efectuados en el marco metodológico IDEAL, a fin de valorar la práctica actual y presentar el desarrollo de un COS para estudios en fase inicial de procedimientos/dispositivos quirúrgicos. MÉTODOS: Se realizaron búsquedas en las bases de datos Web of Science y Scopus para identificar estudios efectuados por los autores en el marco IDEAL/IDEAL-D referentes a cualquier procedimiento/dispositivo quirúrgico. Los resultados se extrajeron literalmente, incluida la información contextualizada referente a la selección y medición de los resultados. Los resultados se categorizaron para presentar información en un marco conceptual de dominios de resultados que fueran relevantes para poder evaluar la innovación. RESULTADOS: Se identificaron 48 estudios. Los estudios mostraban una amplia variedad en la selección de resultados, mediciones e información correspondientes a diferentes estadios IDEAL. A partir de 1.737 resultados extraídos, se conceptualizaron 22 dominios específicos para evaluar la innovación agrupados en 7 amplias categorías: éxito/fracaso para completar el procedimiento; modificaciones; eventos imprevistos; experiencias de los cirujanos; experiencias de los pacientes; uso de recursos específicos del procedimiento/dispositivo innovador y otros resultados específicos de la innovación. La mayoría de los resultados específicos de la innovación se midieron y notificaron solo en un pequeño número de estudios. CONCLUSIÓN: Esta revisión ha puesto de manifiesto la necesidad de orientación y estandarización en la selección de resultados y la notificación en la evaluación de nuevos procedimientos/dispositivos quirúrgicos. Se han identificado nuevos dominios de resultados específicos de innovación para establecer un COS para futuras evaluaciones de innovaciones quirúrgicas.

A systematic review and in-depth analysis of outcome reporting in early phase studies of colorectal cancer surgical innovation.

AIM: Early phase studies are essential to evaluate new technologies prior to randomized evaluation. Evaluation is limited, however, by inconsistent measurement and reporting of outcomes. This study examines outcome reporting in studies of innovative colorectal cancer surgery. METHODS: Systematic searches identified studies of invasive procedures treating primary colorectal adenocarcinoma. Included were a random sample of studies which authors reported as 'new' or 'modified'. Outcomes were extracted verbatim and categorized using an existing framework of 32 domains relevant to early phase studies. Outcomes were classified as 'measured' (where there was an explicit statement to that effect or evidence that data collection had occurred) or 'mentioned but not measured' (where outcomes were discussed but data collection was not evident). Patterns of identified outcomes are described. RESULTS: Of 8373 records, 816 were potentially eligible. Full-text review of a random sample of 218 studies identified 51 for inclusion of which 34 (66%) were 'new' and 17 (33%) were 'modified'. Some 2073 outcomes were identified, and all mapped to domains. 'Anticipated disadvantages' were most frequently identified [660 (32%) outcomes identified across 50 (98%) studies]. No domain was represented in all studies. Under half (944, 46%) of outcomes were 'measured'. 'Surgeon's/operator's experience of the innovation' was more frequently 'mentioned but not measured' [207 (18%) outcomes across 46 (90%) studies] than 'measured' [17 (2%) outcomes, 11 (22%) studies]. CONCLUSION: There is outcome reporting heterogeneity in studies of early phase colorectal cancer surgery. The adoption of core outcome sets may help to resolve these inconsistencies and enable efficient evaluation of surgical innovations.

Core information set for informed consent to surgery for oral or oropharyngeal cancer: A mixed-methods study.

OBJECTIVES: To develop a core information set for informed consent to surgery for oral/oropharyngeal surgery. A core information set is baseline information rated important by patients and surgeons and is intended to improve patients' understanding of the intended procedure. DESIGN: A mixed-methods study. Systematic reviews of scientific and written healthcare literature, qualitative interviews and observations, Delphi surveys, and group consensus meetings identified information domains of importance for consent. SETTING: A regional head and neck clinic in the United Kingdom. Questionnaire participants were recruited from around the UK. PARTICIPANTS: Patients about to undergo, or who had previously undergone, surgery for oral/oropharyngeal cancer. Healthcare professionals involved in the management of head and neck cancer. MAIN OUTCOME MEASURES: The main outcome was a core information set. RESULTS: Systematic reviews, interviews and consultation observations yielded 887 pieces of information that were categorised into 87 information domains. Survey response rates were 67% (n = 50) and 71% (n = 52) for patient and healthcare professional groups in round one. More than 90% responded in each group in the second round. Healthcare professionals were more likely to rate information about short-term or peri-operative events as important while patients rated longer term issues about survival and quality of life. The consensus-building process resulted in an agreed core information set of 13 domains plus two procedure-specific domains about tracheostomy and free-flap surgery. CONCLUSION: This study produced a core information set for surgeons and patients to discuss before surgery for oral/oropharyngeal cancer. Future work will optimise ways to integrate core information into routine consultations.

A national patient and public colorectal research agenda: integration of consumer perspectives in bowel disease through early consultation.

AIM: There is a recognized need to include the views of patients and the public in prioritizing health research. This study aimed: (i) to explore patients' views on colorectal research; and (ii) to prioritize research topics with patients and the public. METHOD: In phase 1, 12 charitable organizations and patient groups with an interest in bowel disease were invited to attend a consultation exercise. Participants were briefed on 25 colorectal research topics prioritized by members of the Association of Coloproctology of Great Britain and Ireland. Focus groups were conducted and discussions were recorded with field notes. Analysis was conducted using principles of thematic analysis. In phase 2, a free public consultation was undertaken. Participants were recruited from newspaper advertisements, were briefed on the same research topics and were asked to rate the importance of each on a five-point Likert scale. Descriptive statistics were used to rank the topics. Univariable linear regression compared recorded demographic details with mean topic scores. RESULTS: Focus groups were attended by 12 patients who highlighted the importance of patient-centred information for trial recruitment and when selecting outcome measures. Some 360 people attended the public consultation, of whom 277 (77%) were recruited. Participants rated 'What is the best way to treat early cancer in the back passage?' highest, with 227 (85%) scoring it 4 or 5. There was no correlation between participant demographics and mean topic scores. CONCLUSION: The present study prioritized a colorectal research agenda with the input of patients and the public. Further research is required to translate this agenda into real improvements in patient care.

Assessing the quality of written information provision for surgical procedures: a case study in oesophagectomy.

OBJECTIVE: To examine the content and quality of written information provided by surgical centres for patients undergoing oesophagectomy for cancer. DESIGN: Cross-sectional study of the content of National Health Service (NHS) patient information leaflets (PILs) about oesophageal cancer surgery, using a modified framework approach. DATA SOURCES: Written information leaflets from 41 of 43 cancer centres undertaking surgery for oesophageal cancer in England and Wales (response rate 95.3%). ELIGIBILITY CRITERIA: All English language versions of PILs about oesophagectomy. RESULTS: 32 different PILs were identified, of which 2 were generic tools (Macmillan 'understanding cancer of the gullet' and EIDO 'oesophagectomy'). Although most PILs focused on describing in-hospital adverse events, information varied widely and was often misleading. Just 1 leaflet described survival benefits of surgery and 2 mentioned the possibility of disease recurrence. CONCLUSIONS: Written information provided for patients by NHS cancer centres undertaking oesophagectomy is inconsistent and incomplete. It is recommended that surgeons work together with patients to agree on standards of information provision of relevance to all stakeholders' needs.

Synthesis and summary of patient-reported outcome measures to inform the development of a core outcome set in colorectal cancer surgery.

AIM: Patient-reported outcome (PRO) measures (PROMs) are standard measures in the assessment of colorectal cancer (CRC) treatment, but the range and complexity of available PROMs may be hindering the synthesis of evidence. This systematic review aimed to: (i) summarize PROMs in studies of CRC surgery and (ii) categorize PRO content to inform the future development of an agreed minimum 'core' outcome set to be measured in all trials. METHOD: All PROMs were identified from a systematic review of prospective CRC surgical studies. The type and frequency of PROMs in each study were summarized, and the number of items documented. All items were extracted and independently categorized by content by two researchers into 'health domains', and discrepancies were discussed with a patient and expert. Domain popularity and the distribution of items were summarized. RESULTS: Fifty-eight different PROMs were identified from the 104 included studies. There were 23 generic, four cancer-specific, 11 disease-specific and 16 symptom-specific questionnaires, and three ad hoc measures. The most frequently used PROM was the EORTC QLQ-C30 (50 studies), and most PROMs (n = 40, 69%) were used in only one study. Detailed examination of the 50 available measures identified 917 items, which were categorized into 51 domains. The domains comprising the most items were 'anxiety' (n = 85, 9.2%), 'fatigue' (n = 67, 7.3%) and 'physical function' (n = 63, 6.9%). No domains were included in all PROMs. CONCLUSION: There is major heterogeneity of PRO measurement and a wide variation in content assessed in the PROMs available for CRC. A core outcome set will improve PRO outcome measurement and reporting in CRC trials.

Core information set for oesophageal cancer surgery.

BACKGROUND: Surgeons provide patients with information before surgery, although standards of information are lacking and practice varies. The development and use of a 'core information set' as baseline information before surgery may improve understanding. A core set is a minimum set of information to use in all consultations before a specific procedure. This study developed a core information set for oesophageal cancer surgery. METHODS: Information was identified from the literature, observations of clinical consultations and patient interviews. This was integrated to create a questionnaire survey. Stakeholders (patients and professionals) were surveyed twice to assess views on importance of information from 'not essential' to 'absolutely essential' using Delphi methods. Items not meeting predefined criteria were discarded after each survey and the final retained items were voted on, in separate patient and professional stakeholder meetings, to agree the core set. RESULTS: Some 67 information items were identified initially from multiple sources. Survey response rates were 76·5 per cent (185 of 242) and 54·8 per cent (126 of 230) for patients and professionals respectively (first round), and over 83 per cent in both groups thereafter. Health professionals rated short-term clinical outcomes most highly (technical complications), whereas patients prioritized information related to long-term benefits. The consensus meetings agreed the final set, which consisted of: in-hospital milestones to recovery, rates of open-and-close surgery, in-hospital mortality, major complications (reoperation), milestones in recovery after discharge, longer-term eating and drinking and overall quality of life, and chances of survival. CONCLUSION: This study has established a core information set for surgery for oesophageal cancer.

Reporting outcomes of definitive radiation-based treatment for esophageal cancer: a review of the literature.

Accurate evaluation of radical radiotherapy requires well designed research with valid and appropriate outcomes. This study reviewed standards of outcome reporting and study design in randomized controlled trials (RCTs) of radiation-based therapy for esophageal cancer and made recommendations for future work. Randomized controlled trials reporting outcomes of definitive radiation-based treatment alone or in combination with chemotherapy were systematically identified and summarized. The types, frequency, and definitions of all clinical and patient-reported outcomes (PROs) reported in the methods and results sections of papers were examined. Studies providing a definition for at least one outcome and presenting all outcomes reported in the methods were classified as high quality. From 1425 abstracts, 16 RCTs including 1803 patients were identified. The primary outcome was overall survival in 13 studies, but five different definitions were reported. Outcomes for treatment failure included local, regional, and distant failures, and inconsistent definitions were applied. An observer assessment of dysphagia was reported in seven RCTs but PROs were reported in only one. Only three RCTs were at low risk of bias, with all lacking reports of sequence generation and only a minority reporting allocation concealment. The quality of outcome reporting in RCTs was inconsistent and risked bias. A core outcome set including clinical and PROs is needed to improve reporting of trials of definitive radiation-based treatment for esophageal cancer.

What surgeons should tell patients with oesophago-gastric cancer: a cross sectional study of information needs.

BACKGROUND: Evidence shows that patients with cancer have many information needs, but specific requirements of patients undergoing surgery for oesophago-gastric (OG) cancer have not been well explored. This study surveyed information needs of patients with OG cancer and explored associations between patient characteristics and information needs. PATIENTS: A validated questionnaire assessing the importance of information about i) the nature, process and prognosis of the cancer, ii) tests, iii) treatments, and iv) physical and psychosocial outcomes (quality of life information) was completed by patients before and after OG cancer treatment. Items responses were transformed to a 0-100 score and linear regression explored associations between clinical and socio-demographic patient characteristics and patient information needs. RESULTS: Questionnaires from 136 respondents were received (response rate 60%, 25% pre-operative, 77% men, mean age 66). Most types of information were rated as important, with information about prognosis and quality of life issues being scored as highly important by over 112 (82%) patients. Linear regression showed that women rated information relating to treatment and psychosocial effects as more important than men (p < 0.038), but no other associations were identified. CONCLUSIONS: Most patients with OG cancer want detailed information, especially information about prognosis and quality of life. It is recommended that surgeons provide this for patients alongside information that the surgeon considers important.

A systematic review of outcome reporting in colorectal cancer surgery.

AIM: Evaluation of surgery for colorectal cancer (CRC) is necessary to inform clinical decision-making and healthcare policy. The standards of outcome reporting after CRC surgery have not previously been considered. METHOD: Systematic literature searches identified randomized and nonrandomized prospective studies reporting clinical outcomes of CRC surgery. Outcomes were listed verbatim, categorized into broad groups (outcome domains) and examined for a definition (an appropriate textual explanation or a supporting citation). Outcome reporting was considered inconsistent if results of the outcome specified in the methods were not reported. Outcome reporting was compared between randomized and nonrandomized studies. RESULTS: Of 5644 abstracts, 194 articles (34 randomized and 160 nonrandomized studies) were included reporting 766 different clinical outcomes, categorized into seven domains. A mean of 14 ± 8 individual outcomes were reported per study. 'Anastomotic leak', 'overall survival' and 'wound infection' were the three most frequently reported outcomes in 72, 60 and 44 (37.1%, 30.9% and 22.7%) studies, respectively, and no single outcome was reported in every publication. Outcome definitions were significantly more often provided in randomized studies than in nonrandomized studies (19.0% vs 14.9%, P = 0.015). One-hundred and twenty-seven (65.5%) papers reported results of all outcomes specified in the methods (randomized studies, n = 21, 61.5%; nonrandomized studies, n = 106, 66.2%; P = 0.617). CONCLUSION: Outcome reporting in CRC surgery lacks consistency and method. Improved standards of outcome measurement are recommended to permit data synthesis and transparent cross-study comparisons.

Optimising methods for communicating survival data to patients undergoing cancer surgery.

BACKGROUND: Patients undergoing cancer surgery require outcome data to inform decisions, but communication of numerical risk is difficult. This study assessed patient understanding of survival data presented in different formats. METHODS: Semi-structured interviews in which patients interpreted four presentation formats of survival data (three graphical and one narrative) were audio-recorded. The interviewer and a blinded observer (listening to the audio-recordings) scored patients' understanding of each format. Logistic regression examined associations between understanding and clinical and socio-demographic details. RESULTS: Seventy participants with colorectal cancer were interviewed and 67 [95.7%, 95% confidence intervals (CIs) 90.9-100%] correctly interpreted a simplified Kaplan-Meier survival curve. A high proportion accurately understood data presented as a bar chart or pictograph (94.3%, 95% CIs 88.7-99.9% and 92.9%, 95% CIs 86.7-99.0% respectively). Standard narrative alone was least well understood (n = 53, 75.7%, 95% CIs 65.4-86.0%). Multivariable analyses demonstrated that older and female patients had poorer overall understanding (OR 0.93 per year, 95% CIs 0.87-0.98, p = 0.01 and OR 0.24, 95% CIs 0.07-0.86, p = 0.03). CONCLUSION: Patient understanding of survival data was higher when presented with graphs compared to narrative alone. Further work examining understanding in the clinical context and before surgery is recommended before this can be used routinely.

Maximising recruitment into randomised controlled trials: the role of multidisciplinary cancer teams.

Multidisciplinary cancer teams offer many theoretical benefits, although few have been formally examined. This study evaluated the role of multidisciplinary team (MDT) meetings in recruitment into randomised controlled trials (RCTs). Consecutive MDT patient records were categorised into those with or without a recommendation for a national multicentre RCT. Clinical trial office records identified whether patients were subsequently screened and randomised. In 125 MDT meetings, 350 new patients were discussed, of whom 103 were potentially suitable for a RCT. The MDT recommended 68 patients for the trial, of whom 58 (85%) were screened for trial eligibility. Of the 35 without an MDT trial recommendation, only 23 (66%) were screened (p=0.022). This difference persisted and resulted in a greater proportion of MDT recommended patients being recruited (65% versus 49%; p=0.12). This study demonstrates that trial recommendation by an MDT significantly increases trial screening rates and may improve recruitment.

Practical aspects to undertaking research in the primary care setting: experience from two studies.

AIMS: To discuss the practical aspects of conducting research in a primary care setting, from the perspectives of the practitioner and the research team. METHODS: Various issues are discussed, including the relevance of research questions being generated in this setting, the advantages to both parties and the processes involved in conducting a study in specialist practice. This paper describes two recent studies (a randomized clinical trial and a qualitative study) conducted within specialist practice, to illustrate some of the potential difficulties. CONCLUSIONS: The success of conducting a study in primary care is determined by a variety of factors, including an interested specialist practitioner, motivated staff in a well-organized practice and the close support of an academic-based research team.