Using Machine Learning to Predict Unplanned Hospital Utilization and Chemotherapy Management From Patient-Reported Outcome Measures.

PURPOSE: Adverse effects of chemotherapy often require hospital admissions or treatment management. Identifying factors contributing to unplanned hospital utilization may improve health care quality and patients' well-being. This study aimed to assess if patient-reported outcome measures (PROMs) improve performance of machine learning (ML) models predicting hospital admissions, triage events (contacting helpline or attending hospital), and changes to chemotherapy. MATERIALS AND METHODS: Clinical trial data were used and contained responses to three PROMs (European Organisation for Research and Treatment of Cancer Core Quality of Life Questionnaire [QLQ-C30], EuroQol Five-Dimensional Visual Analogue Scale [EQ-5D], and Functional Assessment of Cancer Therapy-General [FACT-G]) and clinical information on 508 participants undergoing chemotherapy. Six feature sets (with following variables: [1] all available; [2] clinical; [3] PROMs; [4] clinical and QLQ-C30; [5] clinical and EQ-5D; [6] clinical and FACT-G) were applied in six ML models (logistic regression [LR], decision tree, adaptive boosting, random forest [RF], support vector machines [SVMs], and neural network) to predict admissions, triage events, and chemotherapy changes. RESULTS: The comprehensive analysis of predictive performances of the six ML models for each feature set in three different methods for handling class imbalance indicated that PROMs improved predictions of all outcomes. RF and SVMs had the highest performance for predicting admissions and changes to chemotherapy in balanced data sets, and LR in imbalanced data set. Balancing data led to the best performance compared with imbalanced data set or data set with balanced train set only. CONCLUSION: These results endorsed the view that ML can be applied on PROM data to predict hospital utilization and chemotherapy management. If further explored, this study may contribute to health care planning and treatment personalization. Rigorous comparison of model performance affected by different imbalanced data handling methods shows best practice in ML research.

Cost-Effectiveness of eRAPID eHealth Intervention for Symptom Management During Chemotherapy.

PURPOSE: A randomized controlled trial of online symptom monitoring during chemotherapy with electronic patient self-Reporting of Adverse-events: Patient Information and aDvice (eRAPID) system found improved symptom control and patient self-efficacy, without increasing hospital admissions and visits. The aim of this study was to evaluate the cost-effectiveness of the eRAPID eHealth intervention compared with usual care for patients receiving systemic treatment for colorectal, breast, or gynecologic cancers in the United Kingdom. METHODS: An embedded economic evaluation was conducted alongside the trial evaluating the effectiveness of eRAPID from health care provider and societal perspectives. Costs and quality-adjusted life-years (QALYs) of patients were compared over 18 weeks of the trial. Incremental cost-effectiveness ratios (ICERs) were estimated and compared with the National Institute for Health and Care Excellence cost-effectiveness threshold. Uncertainty around the ICER was explored using nonparametric bootstrapping and sensitivity analyses. Follow-up data were collected 12-months after random assignment for a subset of the study sample to conduct exploratory analysis of potential longer-term effects. RESULTS: Patients in the eRAPID group had the highest QALY gain and lowest costs over 18 weeks. Although differences were small and not statistically significant, eRAPID had a 55%-58% probability of being more cost-effective than usual care. Patient out-of-pocket costs were lower in the eRAPID group, indicating eRAPID may help patients access support needed within the National Health Service. Exploratory 12-months analysis showed small differences in costs and QALYs, with higher QALY gains in the eRAPID group but also higher costs. Exploratory subgroup analysis by disease status indicated that the eRAPID intervention was cost-effective for patients with early-stage cancers but not for patients with metastatic disease. CONCLUSION: Despite small differences in QALYs and costs, the analyses show potential cost-effectiveness of online symptom monitoring, when added to usual care, particularly during adjuvant systemic treatment for early-stage cancers.

Collection of cancer Patient Reported Outcome Measures (PROMS) to link with primary and secondary electronic care records to understand and improve long term cancer outcomes: A protocol paper.

INTRODUCTION: More people are living with and beyond a cancer diagnosis. There is limited understanding of the long-term effects of cancer and cancer treatment on quality of life and personal and household finances when compared to people without cancer. In a separate protocol we have proposed to link de-identified data from electronic primary care and hospital records for a large population of cancer survivors and matched controls. In this current protocol, we propose the linkage of Patient Reported Outcomes Measures data to the above data for a subset of this population. The aim of this study is to investigate the full impact of living with and beyond a cancer diagnosis compared to age and gender matched controls. A secondary aim is to test the feasibility of the collection of Patient Reported Outcomes Measures (PROMS) data and the linkage procedures of the PROMs data to electronic health records data. MATERIALS AND METHODS: This is a cross-sectional study, aiming to recruit participants treated at the Leeds Teaching Hospitals National Health Service Trust. Eligible patients will be cancer survivors at around 5 years post-diagnosis (breast, colorectal and ovarian cancer) and non-cancer patient matched controls attending dermatology out-patient clinics. They will be identified by running a query on the Leeds Teaching Hospitals Trust patient records system. Approximately 6000 patients (2000 cases and 4000 controls) will be invited to participate via post. Participants will be invited to complete PROMs assessing factors such as quality of life and finances, which can be completed on paper or online (surveys includes established instruments, and bespoke instruments (demographics, financial costs). This PROMs data will then be linked to routinely collected de-identified data from patient's electronic primary care and hospital records. DISCUSSION: This innovative work aims to create a truly 'comprehensive patient record' to provide a broad picture of what happens to cancer patients across their cancer pathway, and the long-term impact of cancer treatment. Comparisons can be made between the cases and controls, to identify the aspects of life that has had the greatest impact following a cancer diagnosis. The feasibility of linking PROMs data to electronic health records can also be assessed. This work can inform future support offered to people living with and beyond a cancer diagnosis, clinical practice, and future research methodologies.

Phase III Randomized Controlled Trial of eRAPID: eHealth Intervention During Chemotherapy.

PURPOSE: Electronic patient self-Reporting of Adverse-events: Patient Information and aDvice (eRAPID) is an online eHealth system for patients to self-report symptoms during cancer treatment. It provides automated severity-dependent patient advice guiding self-management or medical contact and displays the reports in electronic patient records. This trial evaluated the impact of eRAPID on symptom control, healthcare use, patient self-efficacy, and quality of life (QOL) in a patient population treated predominantly with curative intent. METHODS: Patients with colorectal, breast, or gynecological cancers commencing chemotherapy were randomly assigned to usual care (UC) or the addition of eRAPID (weekly online symptom reporting for 18 weeks). Primary outcome was symptom control (Functional Assessment of Cancer Therapy-General, Physical Well-Being subscale [FACT-PWB]) assessed at 6, 12, and 18 weeks. Secondary outcomes were processes of care (admissions or chemotherapy delivery), patient self-efficacy, and global quality of life (Functional Assessment of Cancer Therapy-General, EQ5D-VAS, and EORTC QLQ-C30 summary score). Multivariable mixed-effects repeated-measures models were used for analyses. Trial registration: ISRCTN88520246. RESULTS: Participants were 508 consenting patients (73.6% of 690 eligible) and 55 health professionals. eRAPID compared to UC showed improved physical well-being at 6 (P = .028) and 12 (P = .039) weeks and no difference at 18 weeks (primary end point) (P = .69). Fewer eRAPID patients (47%) had clinically meaningful physical well-being deterioration than UC (56%) at 12 weeks. Subgroup analysis found benefit in the nonmetastatic group at 6 weeks (P = .0426), but not in metastatic disease. There were no differences for admissions or chemotherapy delivery. At 18 weeks, patients using eRAPID reported better self-efficacy (P = .007) and better health on EQ5D-VAS (P = .009). Average patient compliance with weekly symptom reporting was 64.7%. Patient adherence was associated with clinician's data use and improved FACT-PWB at 12 weeks. CONCLUSION: Real-time monitoring with electronic patient-reported outcomes improved physical well-being (6 and 12 weeks) and self-efficacy (18 weeks) in a patient population predominantly treated with curative intent, without increasing hospital workload.

Online tool for monitoring adverse events in patients with cancer during treatment (eRAPID): field testing in a clinical setting.

OBJECTIVES: Electronic patient self-Reporting of Adverse-events: Patient Information and aDvice (eRAPID) is an online system developed to support patient care during cancer treatment by improving the detection and management of treatment-related symptoms. Patients can complete symptom reports from home and receive severity-based self-management advice, including notifications to contact the hospital for severe symptoms. Patient data are available in electronic records for staff to review. Prior to the commencement of a randomised controlled trial (RCT), field testing of the intervention was undertaken to troubleshoot practical issues with intervention integration in clinical practice. DESIGN: Observational clinical field testing. SETTING: Medical oncology breast service in a UK cancer centre. PARTICIPANTS: 12 patients receiving chemotherapy for early breast cancer and 10 health professionals (oncologists and specialist nurses). INTERVENTION: Patients were asked to use the eRAPID intervention and complete weekly online symptom reports during four cycles of chemotherapy. Clinical staff were invited to access and use patient data in clinical assessments. ANALYSIS: Descriptive data on the frequency of online symptom report completion and severe symptom notifications were collated. Verbal and written feedback was collected from patients and staff and semistructured interviews were conducted to explore patient experiences. Interviews were transcribed and analysed thematically. RESULTS: The testing ran from January 2014 to March 2014. Feedback from patients and staff was largely positive. Patients described eRAPID as 'reassuring' and 'comforting' and valued the tailored management advice. Several changes were made to refine eRAPID. In particular, improvement of the clinical notification, patient reminder systems and changes to patient and staff training. CONCLUSIONS: The field testing generated valuable results used to guide refinement of eRAPID prior to formal intervention evaluation. Feedback indicated that eRAPID has the potential to improve patients' self-efficacy, knowledge and confidence with managing symptoms during treatment. A large-scale RCT is underway with data collection due to finish in October 2018.

Electronic Systems for Patients to Report and Manage Side Effects of Cancer Treatment: Systematic Review.

BACKGROUND: There has been a dramatic increase in the development of electronic systems to support cancer patients to report and manage side effects of treatment from home. Systems vary in the features they offer to patients, which may affect how patients engage with them and how they improve patient-centered outcomes. OBJECTIVE: This review aimed to (1) describe the features and functions of existing electronic symptom reporting systems (eg, symptom monitoring, tailored self-management advice), and (2) explore which features may be associated with patient engagement and patient-centered outcomes. METHODS: The review was registered with the International Prospective Register of Systematic Reviews (PROSPERO) and followed guidelines from the Centre for Reviews and Dissemination (University of York, United Kingdom). Primary searches were undertaken of MEDLINE, Embase, PsycInfo, Web of Science, Cochrane Central Register of Controlled Trials, and the Health Technology Assessment databases. Secondary searches were undertaken by screening reference lists and citations. Two researchers applied broad inclusion criteria to identify and select relevant records. Data were extracted and summarized using Microsoft Excel. In order to meet the aims, the study selection, data extraction, and data synthesis comprised two stages: (1) identifying and characterizing available systems and (2) summarizing data on patient engagement and patient-centered outcomes. RESULTS: We identified 77 publications relating to 41 distinct systems. In Stage 1, all publications were included (N=77). The features identified that supported clinicians and care were facility for health professionals to remotely access and monitor patient-reported data (24/41, 58%) and function to send alerts to health professionals for severe symptoms (17/41, 41%). Features that supported patients were facility for patients to monitor/review their symptom reports over time (eg, graphs) (19/41, 46%), general patient information about cancer treatment and side effects (17/41, 41%), tailored automated patient advice on symptom management (12/41, 29%), feature for patients to communicate with the health care team (6/41, 15%), and a forum for patients to communicate with one another (4/41, 10%). In Stage 2, only publications that included some data on patient engagement or patient-centered outcomes were included (N=29). A lack of consistency between studies in how engagement was defined, measured, or reported, and a wide range of methods chosen to evaluate systems meant that it was not possible to compare across studies or make conclusions on relationships with system features. CONCLUSIONS: Electronic systems have the potential to help patients manage side effects of cancer treatment, with some evidence to suggest a positive effect on patient-centered outcomes. However, comparison across studies is difficult due to the wide range of assessment tools used. There is a need to develop guidelines for assessing and reporting engagement with systems, and a set of core outcomes for evaluation. We hope that this review will contribute to the field by introducing a taxonomy for characterizing system features. TRIAL REGISTRATION: PROSPERO CRD42016035915; www.crd.york.ac.uk/PROSPERO/display_record.asp?ID=CRD42016035915.

Electronic patient self-Reporting of Adverse-events: Patient Information and aDvice (eRAPID): a randomised controlled trial in systemic cancer treatment.

BACKGROUND: eRAPID (electronic patient self-Reporting of Adverse-events: Patient Information and aDvice) is an internet based system for patients to self-report symptoms and side effects (adverse events or AE) of cancer treatments. eRAPID allows AE reporting from home and patient reported data is accessible via Electronic Patient Records (EPR) for use in routine care. The system can generate alerts to clinical teams for severe AE and provides patient advice on managing mild AEs. The overall aims of eRAPID are to improve the safe delivery of cancer treatments, enhance patient care and standardise AE documentation. METHODS: The trial is a prospective randomised two-arm parallel group design study with repeated measures and mixed methods. Participants (adult patients with breast cancer on neo-adjuvant or adjuvant chemotherapy, colorectal and gynaecological cancer receiving chemotherapy) are randomised to receive the eRAPID intervention or usual care over 18 weeks of treatment. Participants in the intervention arm receive training in using the eRAPID system to provide routine weekly adverse event reports from home. Hospital staff can access eRAPID reports via the EPR and use the information during consultations or phone calls with patients. Prior to commencing the full trial an internal pilot phase was conducted (N = 87 participants) to assess recruitment procedures, consent and attrition rates, the integrity of the intervention information technology and establish procedures for collecting outcome data. The overall target sample for the trial is N = 504. The primary outcome of the trial is quality of life (FACT-G) with secondary outcomes including health economics (costs to patients and the NHS), process of care (e.g. contacts with the hospital, number of admissions, clinic appointments and changes to treatment/medications) and patient self-efficacy. Outcome data is collected at baseline, 6, 12, 18 weeks and 12 months. The intervention is also being evaluated via end of study interviews with patient participants and clinical staff. DISCUSSION: The pilot phase was completed in February 2016 and recruitment and attrition rates met criteria for continuing to the full trial. Recruitment recommenced in May 2016 and is planned to continue until December 2017. Overall findings will determine the value of the eRAPID intervention for supporting the care of patients receiving systemic cancer treatment. TRIAL REGISTRATION: Current Controlled Trials ISRCTN88520246 . Registered 11 September 2014.

Asking the right questions to get the right answers: using cognitive interviews to review the acceptability, comprehension and clinical meaningfulness of patient self-report adverse event items in oncology patients.

BACKGROUND: Standardized reporting of treatment-related adverse events (AE) is essential in clinical trials, usually achieved by using the National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events (CTCAE) reported by clinicians. Patient-reported adverse events (PRAE) may add value to clinician assessments, providing patient perspective on subjective toxicity. We developed an online patient symptom report and self-management system for real-time reporting and managing AE during cancer treatment integrated with electronic patient records (eRAPID). As part of this program we developed a patient version of the CTCAE (version 4.0), rephrasing terminology into a self-report format. We explored patient understanding of these items via cognitive interviews. MATERIAL AND METHOD: Sixty patients (33 female, 27 male) undergoing treatment were purposively sampled by age, gender and tumor group (median age 61.5, range 35-84, 12 breast, 12 gynecological, 13 colorectal, 12 lung and 11 renal). Twenty-one PRAE items were completed on a touch-screen computer. Subsequent audio-recorded cognitive interviews and thematic analysis explored patients' comprehension of items via verbal probing techniques during three interview rounds (n = 20 patients/round). RESULTS: In total 33 item amendments were made; 29% related to question comprehension, 68% response option and 3% order effects. These amendments to phrasing and language improved patient understanding but maintained CTCAE grading and key medical information. Changes were endorsed by members of a patient advisory group (N = 11). CONCLUSION: Item adaptations resulted in a bank of consistently interpreted self-report AE items for use in future research program. In-depth analysis of items through cognitive interviews is an important step towards developing an internationally valid system for PRAE, thus improving patient safety and experiences during cancer treatment.

An audit of acute oncology services: patient experiences of admission procedures and staff utilisation of a new telephone triage system.

OBJECTIVES: In 2010, St. James Institute of Oncology (Leeds, UK) created a new acute oncology service (AOS) consisting of a new admissions unit with a nurse-led telephone triage (TT) system. This audit cycle (March 2011 and June 2013) evaluated patient experiences of the reconfigured AOS and staff use of the TT system. METHODS: Patient views were elicited via a questionnaire and semi-structured interviews. The TT forms were analysed descriptively evaluating completion and data quality, reported symptoms and their severity and advice given (including admission rates). RESULTS: Patients (n = 40) reported high satisfaction with the new AOS. However, 56 % of patients delayed 2 days or more before contacting the unit. In 2011, 26 % of all the admitted patients were triaged via the TT system; 133 TT forms were completed. In June 2013, 49 % of the admitted patients were triaged; 264 forms were completed. The most commonly reported symptoms on the TT forms were pain, pyrexia/rigors/infection, diarrhoea, vomiting and dyspnoea. Half of the patients using the TT system were admitted (52 % in 2011, 49 % in 2013). CONCLUSIONS: Our audit provided evidence of successful implementation of the TT system with the number of TT forms doubling from 2011 to 2013. The new AOS was endorsed by patients, with the majority satisfied with the care they received.

Integrated care pathways for cancer survivors - a role for patient-reported outcome measures and health informatics.

UNLABELLED: Modern cancer treatments have improved survival rates and changed the nature of cancer care. The acute and long-term physical and psychosocial comorbidities associated with treatment place increasing demands on healthcare services to provide suitable models of follow-up care for the survivor population. AIM: We discuss the value and challenges of incorporating patient-reported outcome measures (PROMs) and eHealth interventions into routine follow-up care. We draw on our 15 years' experience of developing electronic systems for capturing patient-reported data in oncology settings, with particular reference to eRAPID a new online symptom reporting system for cancer patients. THE REDESIGN OF HEALTHCARE PATHWAYS: New stratified care pathways have been proposed for cancer survivors with an emphasis on supported self-management and shared care. THE POTENTIAL ROLE OF PROMS IN SURVIVORSHIP CARE PATHWAYS: PROMs can be used to evaluate rehabilitation services, provide epidemiological 'Big Data' and screen patients for physical and psychological morbidities to determine the need for further support. In addition, electronic PROMs systems linked to electronic patient records (EPRs) have the capability to provide tailored self-management advice to individual patients. INTEGRATION OF PROMS INTO CLINICAL PRACTICE: The successful clinical utilisation of PROMs is dependent on a number of components including; choosing appropriate questionnaires, developing evidence-based scoring algorithms, the creation of robust electronic platforms for recording and transferring data into EPRs, and training staff and patients to engage effectively with PROMs. DISCUSSION: There is increasingly positive evidence for using PROMs and eHealth approaches to support cancer patients' care during treatment. Much of what has been learnt can be applied to cancer survivorship. PROMs integrated into eHealth platforms and with EPR have the potential to play a valuable role in the development of appropriate and sustainable long-term follow-up models for cancer survivors.