Assessing the Performance of Machine Learning Methods Trained on Public Health Observational Data: A Case Study From COVID-19.

From early in the coronavirus disease 2019 (COVID-19) pandemic, there was interest in using machine learning methods to predict COVID-19 infection status based on vocal audio signals, for example, cough recordings. However, early studies had limitations in terms of data collection and of how the performances of the proposed predictive models were assessed. This article describes how these limitations have been overcome in a study carried out by the Turing-RSS Health Data Laboratory and the UK Health Security Agency. As part of the study, the UK Health Security Agency collected a dataset of acoustic recordings, SARS-CoV-2 infection status and extensive study participant meta-data. This allowed us to rigorously assess state-of-the-art machine learning techniques to predict SARS-CoV-2 infection status based on vocal audio signals. The lessons learned from this project should inform future studies on statistical evaluation methods to assess the performance of machine learning techniques for public health tasks.

A large-scale and PCR-referenced vocal audio dataset for COVID-19.

The UK COVID-19 Vocal Audio Dataset is designed for the training and evaluation of machine learning models that classify SARS-CoV-2 infection status or associated respiratory symptoms using vocal audio. The UK Health Security Agency recruited voluntary participants through the national Test and Trace programme and the REACT-1 survey in England from March 2021 to March 2022, during dominant transmission of the Alpha and Delta SARS-CoV-2 variants and some Omicron variant sublineages. Audio recordings of volitional coughs, exhalations, and speech were collected in the 'Speak up and help beat coronavirus' digital survey alongside demographic, symptom and self-reported respiratory condition data. Digital survey submissions were linked to SARS-CoV-2 test results. The UK COVID-19 Vocal Audio Dataset represents the largest collection of SARS-CoV-2 PCR-referenced audio recordings to date. PCR results were linked to 70,565 of 72,999 participants and 24,105 of 25,706 positive cases. Respiratory symptoms were reported by 45.6% of participants. This dataset has additional potential uses for bioacoustics research, with 11.3% participants self-reporting asthma, and 27.2% with linked influenza PCR test results.

SYMptoms in chronic heart failure imPACT on burden of treatment (SYMPACT): a cross-sectional survey.

AIMS: This study aimed to describe patient-reported symptoms and burden of treatment (BoT) experienced by patients with chronic heart failure (CHF). BoT describes the illness workload, individual capacity to perform that work, and resultant impact on the individual. Overwhelming BoT is related to poor quality of life and worse clinical outcomes. This research is the first to explore symptoms and BoT in people with CHF, in the UK. METHODS AND RESULTS: This is a cross-sectional questionnaire survey of CHF patients. Participants completed the Heart Failure Symptom Survey (HFSS; max score 10) and the Minnesota Living with Heart Failure Questionnaire (MLHFQ; max scores: physical 40, emotional 25, and total 105), which measured symptoms. BoT was measured with the Patient Experience with Treatment and Self-management (PETS; max score 100) questionnaires. Participant characteristics and questionnaire results were summarized using descriptive statistics. Relationships between symptoms and BoT, summarized by the workload and impact indices, were explored using Spearman's and Pearson's correlation coefficients together with scatter plots. The survey was completed by 333 participants, mean age of 71 (±13) years old. The majority (89%) were recruited from secondary care NHS trusts, and 25% were female. All types of heart failure were represented. Mean symptom scores were as follows: HFSS burden score: 2.4 (±2.1), and MLHFQ scores: physical score 20 (±12.4), emotional score 9.9 (±8.1), and total score 41.3 (±26.3). The highest mean PETS domain scores were exercise [51.3 (±24.7)], diet [40.3 (±22.7)], difficulty with healthcare services [39.9 (±21.3)], and physical and mental fatigue [36.0 (±25.7)]. Pairwise correlations were observed between HFSS scores and MLHFQ physical and emotional sub-scores with PETS workload and impact indices. Positive correlations were weak to moderate (0.326-0.487) between workload index and symptoms, and moderate to strong between impact index and symptoms (0.553-0.725). The P value was 0.006, adjusted by Bonferroni's correction. CONCLUSIONS: Symptoms are associated with BoT in CHF patients. Although symptom burden was low, CHF patients reported higher levels of burden around self-care activities of exercise, diet, healthcare interaction, as well as physical and mental fatigue due to engagement with self-care regimens. Observed higher levels of burden were in key self-care areas for CHF and suggest areas where service delivery and support of CHF patients may be improved to reduce BoT. Clinicians could individualize their consultations by focusing on troublesome symptoms, as well as alleviating illness workload, which may better enable patients to live well with CHF.

Non-Invasive Biomarkers of Musculoskeletal Health with High Discriminant Ability for Age and Gender.

A novel approach to ageing studies assessed the discriminatory ability of a combination of routine physical function tests and novel measures, notably muscle mechanical properties and thigh composition (ultrasound imaging) to classify healthy individuals according to age and gender. The cross-sectional study included 138 community-dwelling, self-reported healthy males and females (65 young, mean age ± SD = 25.7 ± 4.8 years; 73 older, 74.9 ± 5.9 years). Handgrip strength; quadriceps strength; respiratory peak flow; timed up and go; stair climbing time; anterior thigh tissue thickness; muscle stiffness, tone, elasticity (Myoton technology), and self-reported health related quality of life (SF36) were assessed. Stepwise feature selection using cross-validation with linear discriminant analysis was used to classify cases based on criterion variable derived from known effects of age on physical function. A model was trained and features selected using 126 cases with 0.92 accuracy (95% CI = 0.86-0.96; Kappa = 0.89). The final model included five features (peak flow, timed up and go, biceps brachii elasticity, anterior thigh muscle thickness, and percentage thigh muscle) with high sensitivity (0.82-0.96) and specificity (0.94-0.99). The most sensitive novel biomarkers require no volition, highlighting potentially useful tests for screening and monitoring effects of interventions on musculoskeletal health for vulnerable older people with pain or cognitive impairment.