Characterising user engagement with mHealth for chronic disease self-management and impact on machine learning performance.

Mobile Health (mHealth) has the potential to be transformative in the management of chronic conditions. Machine learning can leverage self-reported data collected with apps to predict periods of increased health risk, alert users, and signpost interventions. Despite this, mHealth must balance the treatment burden of frequent self-reporting and predictive performance and safety. Here we report how user engagement with a widely used and clinically validated mHealth app, myCOPD (designed for the self-management of Chronic Obstructive Pulmonary Disease), directly impacts the performance of a machine learning model predicting an acute worsening of condition (i.e., exacerbations). We classify how users typically engage with myCOPD, finding that 60.3% of users engage frequently, however, less frequent users can show transitional engagement (18.4%), becoming more engaged immediately ( < 21 days) before exacerbating. Machine learning performed better for users who engaged the most, however, this performance decrease can be mostly offset for less frequent users who engage more near exacerbation. We conduct interviews and focus groups with myCOPD users, highlighting digital diaries and disease acuity as key factors for engagement. Users of mHealth can feel overburdened when self-reporting data necessary for predictive modelling and confidence of recognising exacerbations is a significant barrier to accurate self-reported data. We demonstrate that users of mHealth should be encouraged to engage when they notice changes to their condition (rather than clinically defined symptoms) to achieve data that is still predictive for machine learning, while reducing the likelihood of disengagement through desensitisation.

Exploring the Validity of GOLD 2023 Guidelines: Should GOLD C and D Be Combined?

Introduction: The GOLD (Global Initiative for Chronic Obstructive Lung Disease) 2023 guidelines proposed important changes to the stratification of disease severity using the "ABCD" assessment tool. The highest risk groups "C" and "D" were combined into a single category "E" based on exacerbation history, no longer considering symptomology. Purpose: We quantify the differential disease progression of individuals initially stratified by the GOLD 2022 "ABCD" scheme to evaluate these proposed changes. Patients and Methods: We utilise data collected from 1529 users of the myCOPD mobile app, a widely used and clinically validated app supporting people living with COPD in the UK. For patients in each GOLD group, we quantify symptoms using COPD Assessment Tests (CAT) and rate of exacerbation over a 12-month period post classification. Results: CAT scores for users initially classified into GOLD C and GOLD D remain significantly different after 12 months (Kolmogorov-Smirnov statistic = 0.59, P = 8.2 × 10-23). Users initially classified into GOLD C demonstrate a significantly lower exacerbation rate over the 12 months post classification than those initially in GOLD D (Kolmogorov-Smirnov statistic = 0.26; P = 3.1 × 10-2; all exacerbations). Further, those initially classified as GOLD B have higher CAT scores and exacerbation rates than GOLD C in the following 12 months. Conclusion: CAT scores remain important for stratifying disease progression both in-terms of symptomology and future exacerbation risk. Based on this evidence, the merger of GOLD C and GOLD D should be reconsidered.

Explainable Machine Learning for Real-Time Hypoglycemia and Hyperglycemia Prediction and Personalized Control Recommendations.

BACKGROUND: The occurrences of acute complications arising from hypoglycemia and hyperglycemia peak as young adults with type 1 diabetes (T1D) take control of their own care. Continuous glucose monitoring (CGM) devices provide real-time glucose readings enabling users to manage their control proactively. Machine learning algorithms can use CGM data to make ahead-of-time risk predictions and provide insight into an individual's longer term control. METHODS: We introduce explainable machine learning to make predictions of hypoglycemia (<70 mg/dL) and hyperglycemia (>270 mg/dL) up to 60 minutes ahead of time. We train our models using CGM data from 153 people living with T1D in the CITY (CGM Intervention in Teens and Young Adults With Type 1 Diabetes)survey totaling more than 28 000 days of usage, which we summarize into (short-term, medium-term, and long-term) glucose control features along with demographic information. We use machine learning explanations (SHAP [SHapley Additive exPlanations]) to identify which features have been most important in predicting risk per user. RESULTS: Machine learning models (XGBoost) show excellent performance at predicting hypoglycemia (area under the receiver operating curve [AUROC]: 0.998, average precision: 0.953) and hyperglycemia (AUROC: 0.989, average precision: 0.931) in comparison with a baseline heuristic and logistic regression model. CONCLUSIONS: Maximizing model performance for glucose risk prediction and management is crucial to reduce the burden of alarm fatigue on CGM users. Machine learning enables more precise and timely predictions in comparison with baseline models. SHAP helps identify what about a CGM user's glucose control has led to predictions of risk which can be used to reduce their long-term risk of complications.

COVID-19 Oximetry @home: evaluation of patient outcomes.

BACKGROUND: COVID-19 has placed unprecedented demands on hospitals. A clinical service, COVID-19 Oximetry @home (CO@h) was launched in November 2020 to support remote monitoring of COVID-19 patients in the community. Remote monitoring through CO@h aims to identify early patient deterioration and provide timely escalation for cases of silent hypoxia, while reducing the burden on secondary care. METHODS: We conducted a retrospective service evaluation of COVID-19 patients onboarded to CO@h from November 2020 to March 2021 in the North Hampshire (UK) community led service (a collaboration of 15 General Practitioner (GP) practices covering 230 000 people). We have compared outcomes for patients admitted to Basingstoke and North Hampshire Hospital who were CO@h patients (COVID-19 patients with home monitoring of oxygen saturation (SpO2; n=115), with non-CO@h patients (those directly admitted without being monitored by CO@h (n=633)). Crude and adjusted OR analysis was performed to evaluate the effects of CO@h on patient outcomes of 30-day mortality, Intensive care unit (ICU) admission and hospital length of stay greater than 3, 7, 14 and 28 days. RESULTS: Adjusted ORs for CO@h show an association with a reduction for several adverse patient outcome: 30-day hospital mortality (p<0.001, OR 0.21, 95% CI 0.08 to 0.47), hospital length of stay larger than 3 days (p<0.05, OR 0.62, 95% CI 0.39 to 1.00), 7 days (p<0.001, OR 0.35, 95% CI 0.22 to 0.54), 14 days (p<0.001, OR 0.22 95% CI, 0.11 to 0.41), and 28 days (p<0.05, OR 0.21, 95% CI 0.05 to 0.59). No significant reduction ICU admission was observed (p>0.05, OR 0.43, 95% CI 0.15 to 1.04). Within 30 days of hospital admission, there were no hospital readmissions for those on the CO@h service as opposed to 8.7% readmissions for those not on the service. CONCLUSIONS: We have demonstrated a significant association between CO@h and better patient outcomes; most notably a reduction in the odds of hospital lengths of stays longer than 7, 14 and 28 days and 30-day hospital mortality.

Using explainable machine learning to characterise data drift and detect emergent health risks for emergency department admissions during COVID-19.

A key task of emergency departments is to promptly identify patients who require hospital admission. Early identification ensures patient safety and aids organisational planning. Supervised machine learning algorithms can use data describing historical episodes to make ahead-of-time predictions of clinical outcomes. Despite this, clinical settings are dynamic environments and the underlying data distributions characterising episodes can change with time (data drift), and so can the relationship between episode characteristics and associated clinical outcomes (concept drift). Practically this means deployed algorithms must be monitored to ensure their safety. We demonstrate how explainable machine learning can be used to monitor data drift, using the COVID-19 pandemic as a severe example. We present a machine learning classifier trained using (pre-COVID-19) data, to identify patients at high risk of admission during an emergency department attendance. We then evaluate our model's performance on attendances occurring pre-pandemic (AUROC of 0.856 with 95%CI [0.852, 0.859]) and during the COVID-19 pandemic (AUROC of 0.826 with 95%CI [0.814, 0.837]). We demonstrate two benefits of explainable machine learning (SHAP) for models deployed in healthcare settings: (1) By tracking the variation in a feature's SHAP value relative to its global importance, a complimentary measure of data drift is found which highlights the need to retrain a predictive model. (2) By observing the relative changes in feature importance emergent health risks can be identified.