Characterising user engagement with mHealth for chronic disease self-management and impact on machine learning performance.

Mobile Health (mHealth) has the potential to be transformative in the management of chronic conditions. Machine learning can leverage self-reported data collected with apps to predict periods of increased health risk, alert users, and signpost interventions. Despite this, mHealth must balance the treatment burden of frequent self-reporting and predictive performance and safety. Here we report how user engagement with a widely used and clinically validated mHealth app, myCOPD (designed for the self-management of Chronic Obstructive Pulmonary Disease), directly impacts the performance of a machine learning model predicting an acute worsening of condition (i.e., exacerbations). We classify how users typically engage with myCOPD, finding that 60.3% of users engage frequently, however, less frequent users can show transitional engagement (18.4%), becoming more engaged immediately ( < 21 days) before exacerbating. Machine learning performed better for users who engaged the most, however, this performance decrease can be mostly offset for less frequent users who engage more near exacerbation. We conduct interviews and focus groups with myCOPD users, highlighting digital diaries and disease acuity as key factors for engagement. Users of mHealth can feel overburdened when self-reporting data necessary for predictive modelling and confidence of recognising exacerbations is a significant barrier to accurate self-reported data. We demonstrate that users of mHealth should be encouraged to engage when they notice changes to their condition (rather than clinically defined symptoms) to achieve data that is still predictive for machine learning, while reducing the likelihood of disengagement through desensitisation.

Exploring the Validity of GOLD 2023 Guidelines: Should GOLD C and D Be Combined?

Introduction: The GOLD (Global Initiative for Chronic Obstructive Lung Disease) 2023 guidelines proposed important changes to the stratification of disease severity using the "ABCD" assessment tool. The highest risk groups "C" and "D" were combined into a single category "E" based on exacerbation history, no longer considering symptomology. Purpose: We quantify the differential disease progression of individuals initially stratified by the GOLD 2022 "ABCD" scheme to evaluate these proposed changes. Patients and Methods: We utilise data collected from 1529 users of the myCOPD mobile app, a widely used and clinically validated app supporting people living with COPD in the UK. For patients in each GOLD group, we quantify symptoms using COPD Assessment Tests (CAT) and rate of exacerbation over a 12-month period post classification. Results: CAT scores for users initially classified into GOLD C and GOLD D remain significantly different after 12 months (Kolmogorov-Smirnov statistic = 0.59, P = 8.2 × 10-23). Users initially classified into GOLD C demonstrate a significantly lower exacerbation rate over the 12 months post classification than those initially in GOLD D (Kolmogorov-Smirnov statistic = 0.26; P = 3.1 × 10-2; all exacerbations). Further, those initially classified as GOLD B have higher CAT scores and exacerbation rates than GOLD C in the following 12 months. Conclusion: CAT scores remain important for stratifying disease progression both in-terms of symptomology and future exacerbation risk. Based on this evidence, the merger of GOLD C and GOLD D should be reconsidered.

Prediction of Chronic Obstructive Pulmonary Disease Exacerbation Events by Using Patient Self-reported Data in a Digital Health App: Statistical Evaluation and Machine Learning Approach.

BACKGROUND: Self-reporting digital apps provide a way of remotely monitoring and managing patients with chronic conditions in the community. Leveraging the data collected by these apps in prognostic models could provide increased personalization of care and reduce the burden of care for people who live with chronic conditions. This study evaluated the predictive ability of prognostic models for the prediction of acute exacerbation events in people with chronic obstructive pulmonary disease by using data self-reported to a digital health app. OBJECTIVE: The aim of this study was to evaluate if data self-reported to a digital health app can be used to predict acute exacerbation events in the near future. METHODS: This is a retrospective study evaluating the use of symptom and chronic obstructive pulmonary disease assessment test data self-reported to a digital health app (myCOPD) in predicting acute exacerbation events. We include data from 2374 patients who made 68,139 self-reports. We evaluated the degree to which the different variables self-reported to the app are predictive of exacerbation events and developed both heuristic and machine learning models to predict whether the patient will report an exacerbation event within 3 days of self-reporting to the app. The model's predictive ability was evaluated based on self-reports from an independent set of patients. RESULTS: Users self-reported symptoms, and standard chronic obstructive pulmonary disease assessment tests displayed correlation with future exacerbation events. Both a baseline model (area under the receiver operating characteristic curve [AUROC] 0.655, 95% CI 0.689-0.676) and a machine learning model (AUROC 0.727, 95% CI 0.720-0.735) showed moderate ability in predicting exacerbation events, occurring within 3 days of a given self-report. Although the baseline model obtained a fixed sensitivity and specificity of 0.551 (95% CI 0.508-0.596) and 0.759 (95% CI 0.752-0.767) respectively, the sensitivity and specificity of the machine learning model can be tuned by dichotomizing the continuous predictions it provides with different thresholds. CONCLUSIONS: Data self-reported to health care apps designed to remotely monitor patients with chronic obstructive pulmonary disease can be used to predict acute exacerbation events with moderate performance. This could increase personalization of care by allowing preemptive action to be taken to mitigate the risk of future exacerbation events.

Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19.

A key task of emergency departments is to promptly identify patients who require hospital admission. Early identification ensures patient safety and aids organisational planning. Supervised machine learning algorithms can use data describing historical episodes to make ahead-of-time predictions of clinical outcomes. Despite this, clinical settings are dynamic environments and the underlying data distributions characterising episodes can change with time (data drift), and so can the relationship between episode characteristics and associated clinical outcomes (concept drift). Practically this means deployed algorithms must be monitored to ensure their safety. We demonstrate how explainable machine learning can be used to monitor data drift, using the COVID-19 pandemic as a severe example. We present a machine learning classifier trained using (pre-COVID-19) data, to identify patients at high risk of admission during an emergency department attendance. We then evaluate our model's performance on attendances occurring pre-pandemic (AUROC of 0.856 with 95%CI [0.852, 0.859]) and during the COVID-19 pandemic (AUROC of 0.826 with 95%CI [0.814, 0.837]). We demonstrate two benefits of explainable machine learning (SHAP) for models deployed in healthcare settings: (1) By tracking the variation in a feature's SHAP value relative to its global importance, a complimentary measure of data drift is found which highlights the need to retrain a predictive model. (2) By observing the relative changes in feature importance emergent health risks can be identified.