Development and evaluation of a text analytics algorithm for automated application of national COVID-19 shielding criteria in rheumatology patients.

INTRODUCTION: At the beginning of the COVID-19 pandemic, the UK's Scientific Committee issued extreme social distancing measures, termed 'shielding', aimed at a subpopulation deemed extremely clinically vulnerable to infection. National guidance for risk stratification was based on patients' age, comorbidities and immunosuppressive therapies, including biologics that are not captured in primary care records. This process required considerable clinician time to manually review outpatient letters. Our aim was to develop and evaluate an automated shielding algorithm by text-mining outpatient letter diagnoses and medications, reducing the need for future manual review. METHODS: Rheumatology outpatient letters from a large UK foundation trust were retrieved. Free-text diagnoses were processed using Intelligent Medical Objects software (Concept Tagger), which used interface terminology for each condition mapped to Systematized Medical Nomenclature for Medicine-Clinical Terminology (SNOMED-CT) codes. We developed the Medication Concept Recognition tool (Named Entity Recognition) to retrieve medications' type, dose, duration and status (active/past) at the time of the letter. Age, diagnosis and medication variables were then combined to calculate a shielding score based on the most recent letter. The algorithm's performance was evaluated using clinical review as the gold standard. The time taken to deploy the developed algorithm on a larger patient subset was measured. RESULTS: In total, 5942 free-text diagnoses were extracted and mapped to SNOMED-CT, with 13 665 free-text medications (n=803 patients). The automated algorithm demonstrated a sensitivity of 80% (95% CI: 75%, 85%) and specificity of 92% (95% CI: 90%, 94%). Positive likelihood ratio was 10 (95% CI: 8, 14), negative likelihood ratio was 0.21 (95% CI: 0.16, 0.28) and F1 score was 0.81. Evaluation of mismatches revealed that the algorithm performed correctly against the gold standard in most cases. The developed algorithm was then deployed on records from an additional 15 865 patients, which took 18 hours for data extraction and 1 hour to deploy. DISCUSSION: An automated algorithm for risk stratification has several advantages including reducing clinician time for manual review to allow more time for direct care, improving efficiency and increasing transparency in individual patient communication. It has the potential to be adapted for future public health initiatives that require prompt automated review of hospital outpatient letters.

Charting a course for smartphones and wearables to transform population health research.

The use of data from smartphones and wearable devices has huge potential for population health research given high device ownership, the range of novel health-relevant data types available from consumer devices, and the frequency and duration over which data are, or could be, collected. Yet the uptake and success of large-scale mobile health research in the last decade has not matched the hyped opportunity. We make the argument that digital person-generated health data is required and necessary to answer many top priority research questions through illustrative examples taken from the James Lind Alliance Priority Setting Partnership. We then summarise the findings from two UK initiatives that considered the challenges and possible solutions for what needs to be done, and in what way, to realise the future opportunities of digital person-generated health data for clinically important population health research. Examples of important areas to be addressed to advance the field include digital inequality and addressing possible selection bias, easy access for researchers to the appropriate data collection tools including how best to harmonise data items, analysis methodology for time series data, methods for patient and public involvement and engagement to optimise recruitment, retention and public trust, and providing greater control of their data to research participants. There is also a major opportunity through the linkage of digital persongenerated health data to routinely-collected data to support novel population health research, bringing together clinician-reported and patient-reported measures. We recognise that well conducted studies need a wide range of diverse challenges to be skilfully addressed in unison: for example, epidemiology, data science and biostatistics, psychometrics, behavioural and social science, software engineering, user interface design, information governance, data management and patient and public involvement and engagement. Consequently, progress would be accelerated by the establishment of a new interdisciplinary community where all relevant and necessary skills are brought together to allow excellence throughout the lifecycle of a research study. This will require a partnership of diverse people, of methods and of technology. Get this right and the synergy has the potential to transform many millions of people's lives for the better.

Classifying Self-Reported Rheumatoid Arthritis Flares Using Daily Patient-Generated Data From a Smartphone App: Exploratory Analysis Applying Machine Learning Approaches.

BACKGROUND: The ability to predict rheumatoid arthritis (RA) flares between clinic visits based on real-time, longitudinal patient-generated data could potentially allow for timely interventions to avoid disease worsening. OBJECTIVE: This exploratory study aims to investigate the feasibility of using machine learning methods to classify self-reported RA flares based on a small data set of daily symptom data collected on a smartphone app. METHODS: Daily symptoms and weekly flares reported on the Remote Monitoring of Rheumatoid Arthritis (REMORA) smartphone app from 20 patients with RA over 3 months were used. Predictors were several summary features of the daily symptom scores (eg, pain and fatigue) collected in the week leading up to the flare question. We fitted 3 binary classifiers: logistic regression with and without elastic net regularization, a random forest, and naive Bayes. Performance was evaluated according to the area under the curve (AUC) of the receiver operating characteristic curve. For the best-performing model, we considered sensitivity and specificity for different thresholds in order to illustrate different ways in which the predictive model could behave in a clinical setting. RESULTS: The data comprised an average of 60.6 daily reports and 10.5 weekly reports per participant. Participants reported a median of 2 (IQR 0.75-4.25) flares each over a median follow-up time of 81 (IQR 79-82) days. AUCs were broadly similar between models, but logistic regression with elastic net regularization had the highest AUC of 0.82. At a cutoff requiring specificity to be 0.80, the corresponding sensitivity to detect flares was 0.60 for this model. The positive predictive value (PPV) in this population was 53%, and the negative predictive value (NPV) was 85%. Given the prevalence of flares, the best PPV achieved meant only around 2 of every 3 positive predictions were correct (PPV 0.65). By prioritizing a higher NPV, the model correctly predicted over 9 in every 10 non-flare weeks, but the accuracy of predicted flares fell to only 1 in 2 being correct (NPV and PPV of 0.92 and 0.51, respectively). CONCLUSIONS: Predicting self-reported flares based on daily symptom scorings in the preceding week using machine learning methods was feasible. The observed predictive accuracy might improve as we obtain more data, and these exploratory results need to be validated in an external cohort. In the future, analysis of frequently collected patient-generated data may allow us to predict flares before they unfold, opening opportunities for just-in-time adaptative interventions. Depending on the nature and implication of an intervention, different cutoff values for an intervention decision need to be considered, as well as the level of predictive certainty required.

Postoperative opioids administered to inpatients with major or orthopaedic surgery: A retrospective cohort study using data from hospital electronic prescribing systems.

BACKGROUND: Opioids administered in hospital during the immediate postoperative period are likely to influence post-surgical outcomes, but inpatient prescribing during the admission is challenging to access. Modified-release(MR) preparations have been especially associated with harm, whilst certain populations such as the elderly or those with renal impairment may be vulnerable to complications. This study aimed to assess postoperative opioid utilisation patterns during hospital stay for people admitted for major/orthopaedic surgery. METHODS: Patients admitted to a teaching hospital in the North-West of England between 2010-2021 for major/orthopaedic surgery with an admission for ≥1 day were included. We examined opioid administrations in the first seven days post-surgery in hospital, and "first 48 hours" were defined as the initial period. Proportions of MR opioids, initial immediate-release(IR) oxycodone and initial morphine milligram equivalents (MME)/day were calculated and summarised by calendar year. We also assessed the proportion of patients prescribed an opioid at discharge. RESULTS: Among patients admitted for major/orthopaedic surgery, 71.1% of patients administered opioids during their hospitalisation. In total 50,496 patients with 60,167 hospital admissions were evaluated. Between 2010-2017 MR opioids increased from 8.7% to 16.1% and dropped to 11.6% in 2021. Initial use of oxycodone IR among younger patients (≤70 years) rose from 8.3% to 25.5% (2010-2017) and dropped to 17.2% in 2021. The proportion of patients on ≥50MME/day ranged from 13% (2021) to 22.9% (2010). Of the patients administered an opioid in hospital, 26,920 (53.3%) patients were discharged on an opioid. CONCLUSIONS: In patients hospitalised with major/orthopaedic surgery, 4 in 6 patients were administered an opioid. We observed a high frequency of administered MR opioids in adult patients and initial oxycodone IR in the ≤70 age group. Patients prescribed with ≥50MME/day ranged between 13-22.9%. This is the first published study evaluating UK inpatient opioid use, which highlights opportunities for improving safer prescribing in line with latest recommendations.

Identifying Weekly Trajectories of Pain Severity Using Daily Data From an mHealth Study: Cluster Analysis.

BACKGROUND: People with chronic pain experience variability in their trajectories of pain severity. Previous studies have explored pain trajectories by clustering sparse data; however, to understand daily pain variability, there is a need to identify clusters of weekly trajectories using daily pain data. Between-week variability can be explored by quantifying the week-to-week movement between these clusters. We propose that future work can use clusters of pain severity in a forecasting model for short-term (eg, daily fluctuations) and longer-term (eg, weekly patterns) variability. Specifically, future work can use clusters of weekly trajectories to predict between-cluster movement and within-cluster variability in pain severity. OBJECTIVE: This study aims to understand clusters of common weekly patterns as a first stage in developing a pain-forecasting model. METHODS: Data from a population-based mobile health study were used to compile weekly pain trajectories (n=21,919) that were then clustered using a k-medoids algorithm. Sensitivity analyses tested the impact of assumptions related to the ordinal and longitudinal structure of the data. The characteristics of people within clusters were examined, and a transition analysis was conducted to understand the movement of people between consecutive weekly clusters. RESULTS: Four clusters were identified representing trajectories of no or low pain (1714/21,919, 7.82%), mild pain (8246/21,919, 37.62%), moderate pain (8376/21,919, 38.21%), and severe pain (3583/21,919, 16.35%). Sensitivity analyses confirmed the 4-cluster solution, and the resulting clusters were similar to those in the main analysis, with at least 85% of the trajectories belonging to the same cluster as in the main analysis. Male participants spent longer (participant mean 7.9, 95% bootstrap CI 6%-9.9%) in the no or low pain cluster than female participants (participant mean 6.5, 95% bootstrap CI 5.7%-7.3%). Younger people (aged 17-24 y) spent longer (participant mean 28.3, 95% bootstrap CI 19.3%-38.5%) in the severe pain cluster than older people (aged 65-86 y; participant mean 9.8, 95% bootstrap CI 7.7%-12.3%). People with fibromyalgia (participant mean 31.5, 95% bootstrap CI 28.5%-34.4%) and neuropathic pain (participant mean 31.1, 95% bootstrap CI 27.3%-34.9%) spent longer in the severe pain cluster than those with other conditions, and people with rheumatoid arthritis spent longer (participant mean 7.8, 95% bootstrap CI 6.1%-9.6%) in the no or low pain cluster than those with other conditions. There were 12,267 pairs of consecutive weeks that contributed to the transition analysis. The empirical percentage remaining in the same cluster across consecutive weeks was 65.96% (8091/12,267). When movement between clusters occurred, the highest percentage of movement was to an adjacent cluster. CONCLUSIONS: The clusters of pain severity identified in this study provide a parsimonious description of the weekly experiences of people with chronic pain. These clusters could be used for future study of between-cluster movement and within-cluster variability to develop accurate and stakeholder-informed pain-forecasting tools.

Reliability, validity, and responsiveness of a smartphone-based manikin to support pain self-reporting.

Introduction: Many people worldwide suffer from chronic pain. Improving our knowledge on chronic pain prevalence and management requires methods to collect pain self-reports in large populations. Smartphone-based tools could aid data collection by allowing people to use their own device, but the measurement properties of such tools are largely unknown. Objectives: To assess the reliability, validity, and responsiveness of a smartphone-based manikin to support pain self-reporting. Methods: We recruited people with fibromyalgia, rheumatoid arthritis, and/or osteoarthritis and access to a smartphone and the internet. Data collection included the Global Pain Scale at baseline and follow-up, and 30 daily pain drawings completed on a 2-dimensional, gender-neutral manikin. After deriving participants' pain extent from their manikin drawings, we evaluated convergent and discriminative validity, test-retest reliability, and responsiveness and assessed findings against internationally agreed criteria for good measurement properties. Results: We recruited 131 people; 104 were included in the full sample, submitting 2185 unique pain drawings. Manikin-derived pain extent had excellent test-retest reliability (intraclass correlation coefficient, 0.94), moderate convergent validity (ρ, 0.46), and an ability to distinguish fibromyalgia and osteoarthritis from rheumatoid arthritis (F statistics, 30.41 and 14.36, respectively; P < 0.001). Responsiveness was poor (ρ, 0.2; P, 0.06) and did not meet the respective criterion for good measurement properties. Conclusion: Our findings suggest that smartphone-based manikins can be a reliable and valid method for pain self-reporting, but that further research is warranted to explore, enhance, and confirm the ability of such manikins to detect a change in pain over time.

Digital health technologies to strengthen patient-centred outcome assessment in clinical trials in inflammatory arthritis.

Common to all inflammatory arthritides, namely rheumatoid arthritis, psoriatic arthritis, axial spondyloarthritis, and juvenile idiopathic arthritis, is a potential for reduced mobility that manifests through joint pain, swelling, stiffness, and ultimately joint damage. Across these conditions, consensus has been reached on the need to capture outcomes related to mobility, such as functional capacity and physical activity, as core domains in randomised controlled trials. Existing endpoints within these core domains rely wholly on self-reported questionnaires that capture patients' perceptions of their symptoms and activities. These questionnaires are subjective, inherently vulnerable to recall bias, and do not capture the granularity of fluctuations over time. Several early adopters have integrated sensor-based digital health technology (DHT)-derived endpoints to measure physical function and activity in randomised controlled trials for conditions including Parkinson's disease, Duchenne's muscular dystrophy, chronic obstructive pulmonary disease, and heart failure. Despite these applications, there have been no sensor-based DHT-derived endpoints in clinical trials recruiting patients with inflammatory arthritis. Borrowing from case studies across medicine, we outline the opportunities and challenges in developing novel sensor-based DHT-derived endpoints that capture the symptoms and disease manifestations most relevant to patients with inflammatory arthritis.

Users' views on the use of a smartwatch app to collect daily symptom data in individuals with multiple long-term conditions (Multimorbidity): A qualitative study.

Introduction: Long-term conditions are a major burden on health systems. One way to facilitate more research and better clinical care among patients with long-term conditions is to collect accurate data on their daily symptoms (patient-generated health data) using wearable technologies. Whilst evidence is growing for the use of wearable technologies in single conditions, there is less evidence of the utility of frequent symptom tracking in those who have more than one condition. Aims: To explore patient views of the acceptability of collecting daily patient-generated health data for three months using a smartwatch app. Methods: Watch Your Steps was a longitudinal study which recruited 53 patients to track over 20 symptoms per day for a 90-day period using a study app on smartwatches. Semi-structured interviews were conducted with a sub-sample of 20 participants to explore their experience of engaging with the app. Results: In a population of older people with multimorbidity, patients were willing and able to engage with a patient-generated health data app on a smartwatch. It was suggested that to maintain engagement over a longer period, more 'real-time' feedback from the app should be available. Participants did not seem to consider the management of more than one condition to be a factor in either engagement or use of the app, but the presence of severe or chronic pain was at times a barrier. Conclusion: This study has provided preliminary evidence that multimorbidity was not a major barrier to engagement with patient-generated health data via a smartwatch symptom tracking app.

Digital endpoints in clinical trials: emerging themes from a multi-stakeholder Knowledge Exchange event.

BACKGROUND: Digital technologies, such as wearable devices and smartphone applications (apps), can enable the decentralisation of clinical trials by measuring endpoints in people's chosen locations rather than in traditional clinical settings. Digital endpoints can allow high-frequency and sensitive measurements of health outcomes compared to visit-based endpoints which provide an episodic snapshot of a person's health. However, there are underexplored challenges in this emerging space that require interdisciplinary and cross-sector collaboration. A multi-stakeholder Knowledge Exchange event was organised to facilitate conversations across silos within this research ecosystem. METHODS: A survey was sent to an initial list of stakeholders to identify potential discussion topics. Additional stakeholders were identified through iterative discussions on perspectives that needed representation. Co-design meetings with attendees were held to discuss the scope, format and ethos of the event. The event itself featured a cross-disciplinary selection of talks, a panel discussion, small-group discussions facilitated via a rolling seating plan and audience participation via Slido. A transcript was generated from the day, which, together with the output from Slido, provided a record of the day's discussions. Finally, meetings were held following the event to identify the key challenges for digital endpoints which emerged and reflections and recommendations for dissemination. RESULTS: Several challenges for digital endpoints were identified in the following areas: patient adherence and acceptability; algorithms and software for devices; design, analysis and conduct of clinical trials with digital endpoints; the environmental impact of digital endpoints; and the need for ongoing ethical support. Learnings taken for next generation events include the need to include additional stakeholder perspectives, such as those of funders and regulators, and the need for additional resources and facilitation to allow patient and public contributors to engage meaningfully during the event. CONCLUSIONS: The event emphasised the importance of consortium building and highlighted the critical role that collaborative, multi-disciplinary, and cross-sector efforts play in driving innovation in research design and strategic partnership building moving forward. This necessitates enhanced recognition by funders to support multi-stakeholder projects with patient involvement, standardised terminology, and the utilisation of open-source software.