Age, sex and race bias in automated arrhythmia detectors.

Despite the recent explosion of machine learning applied to medical data, very few studies have examined algorithmic bias in any meaningful manner, comparing across algorithms, databases, and assessment metrics. In this study, we compared the biases in sex, age, and race of 56 algorithms on over 130,000 electrocardiograms (ECGs) using several metrics and propose a machine learning model design to reduce bias. Participants of the 2021 PhysioNet Challenge designed and implemented working, open-source algorithms to identify clinical diagnosis from 2- lead ECG recordings. We grouped the data from the training, validation, and test datasets by sex (male vs female), age (binned by decade), and race (Asian, Black, White, and Other) whenever possible. We computed recording-wise accuracy, area under the receiver operating characteristic curve (AUROC), area under the precision recall curve (AUPRC), F-measure, and the Challenge Score for each of the 56 algorithms. The Mann-Whitney U and the Kruskal-Wallis tests assessed the performance differences of algorithms across these demographic groups. Group trends revealed similar values for the AUROC, AUPRC, and F-measure for both male and female groups across the training, validation, and test sets. However, recording-wise accuracies were 20% higher (p < 0.01) and the Challenge Score 12% lower (p = 0.02) for female subjects on the test set. AUPRC, F-measure, and the Challenge Score increased with age, while recording-wise accuracy and AUROC decreased with age. The results were similar for the training and test sets, but only recording-wise accuracy (12% decrease per decade, p < 0.01), Challenge Score (1% increase per decade, p < 0.01), and AUROC (1% decrease per decade, p < 0.01) were statistically different on the test set. We observed similar AUROC, AUPRC, Challenge Score, and F-measure values across the different race categories. But, recording-wise accuracies were significantly lower for Black subjects and higher for Asian subjects on the training (31% difference, p < 0.01) and test (39% difference, p < 0.01) sets. A top performing model was then retrained using an additional constraint which simultaneously minimized differences in performance across sex, race and age. This resulted in a modest reduction in performance, with a significant reduction in bias. This work provides a demonstration that biases manifest as a function of model architecture, population, cost function and optimization metric, all of which should be closely examined in any model.

Heart murmur detection from phonocardiogram recordings: The George B. Moody PhysioNet Challenge 2022.

Cardiac auscultation is an accessible diagnostic screening tool that can help to identify patients with heart murmurs, who may need follow-up diagnostic screening and treatment for abnormal cardiac function. However, experts are needed to interpret the heart sounds, limiting the accessibility of cardiac auscultation in resource-constrained environments. Therefore, the George B. Moody PhysioNet Challenge 2022 invited teams to develop algorithmic approaches for detecting heart murmurs and abnormal cardiac function from phonocardiogram (PCG) recordings of heart sounds. For the Challenge, we sourced 5272 PCG recordings from 1452 primarily pediatric patients in rural Brazil, and we invited teams to implement diagnostic screening algorithms for detecting heart murmurs and abnormal cardiac function from the recordings. We required the participants to submit the complete training and inference code for their algorithms, improving the transparency, reproducibility, and utility of their work. We also devised an evaluation metric that considered the costs of screening, diagnosis, misdiagnosis, and treatment, allowing us to investigate the benefits of algorithmic diagnostic screening and facilitate the development of more clinically relevant algorithms. We received 779 algorithms from 87 teams during the Challenge, resulting in 53 working codebases for detecting heart murmurs and abnormal cardiac function from PCG recordings. These algorithms represent a diversity of approaches from both academia and industry, including methods that use more traditional machine learning techniques with engineered clinical and statistical features as well as methods that rely primarily on deep learning models to discover informative features. The use of heart sound recordings for identifying heart murmurs and abnormal cardiac function allowed us to explore the potential of algorithmic approaches for providing more accessible diagnostic screening in resource-constrained environments. The submission of working, open-source algorithms and the use of novel evaluation metrics supported the reproducibility, generalizability, and clinical relevance of the research from the Challenge.

Comparative associations of oximetry patterns in Obstructive Sleep Apnea with incident cardiovascular disease.

STUDY OBJECTIVES: Intermittent hypoxia is a key mechanism linking Obstructive Sleep Apnea (OSA) to cardiovascular disease (CVD). Oximetry analysis could enhance understanding of which OSA phenotypes are associated with CVD risk. The aim of this study was to compare associations of different oximetry patterns with incident CVD in men and women with OSA. METHODS: Sleep Heart Health Study data were used for analysis. n = 2878 Participants (51.8% female; mean age 63.5 ±â€…10.5 years) with OSA (Apnea Hypopnea Index [AHI] ≥ 5 events/h) and no pre-existing CVD at baseline or within the first 2 years of follow-up were included. Four oximetry analysis approaches were applied: desaturation characteristics, time series analysis, power spectral density, and non-linear analysis. Thirty-one resulting oximetry patterns were compared to incident CVD using proportional hazards regression models adjusted for age, race, smoking, BMI, and sex. RESULTS: There were no associations between OSA oximetry patterns and incident CVD in the total sample or in men. In women, there were some associations between incident CVD and time series analysis (e.g. SpO2 distribution standard deviation, HR 0.81, 95% CI 0.68-0.96, p = 0.014) and power spectral density oximetry patterns (e.g. Full frequency band mean HR 0.75; 95% CI 0.59-0.95; p = 0.015). CONCLUSIONS: Comprehensive comparison of baseline oximetry patterns in OSA found none were related to development of CVD. There were no standout individual oximetry patterns that appear to be candidates for CVD risk phenotyping in OSA, but some showed marginal relationships with CVD risk in women. Further work is required to understand whether OSA phenotypes can be used to predict susceptibility to cardiovascular disease.

Issues in the automated classification of multilead ecgs using heterogeneous labels and populations.

Objective.The standard twelve-lead electrocardiogram (ECG) is a widely used tool for monitoring cardiac function and diagnosing cardiac disorders. The development of smaller, lower-cost, and easier-to-use ECG devices may improve access to cardiac care in lower-resource environments, but the diagnostic potential of these devices is unclear. This work explores these issues through a public competition: the 2021 PhysioNet Challenge. In addition, we explore the potential for performance boosting through a meta-learning approach.Approach.We sourced 131,149 twelve-lead ECG recordings from ten international sources. We posted 88,253 annotated recordings as public training data and withheld the remaining recordings as hidden validation and test data. We challenged teams to submit containerized, open-source algorithms for diagnosing cardiac abnormalities using various ECG lead combinations, including the code for training their algorithms. We designed and scored the algorithms using an evaluation metric that captures the risks of different misdiagnoses for 30 conditions. After the Challenge, we implemented a semi-consensus voting model on all working algorithms.Main results.A total of 68 teams submitted 1,056 algorithms during the Challenge, providing a variety of automated approaches from both academia and industry. The performance differences across the different lead combinations were smaller than the performance differences across the different test databases, showing that generalizability posed a larger challenge to the algorithms than the choice of ECG leads. A voting model improved performance by 3.5%.Significance.The use of different ECG lead combinations allowed us to assess the diagnostic potential of reduced-lead ECG recordings, and the use of different data sources allowed us to assess the generalizability of the algorithms to diverse institutions and populations. The submission of working, open-source code for both training and testing and the use of a novel evaluation metric improved the reproducibility, generalizability, and applicability of the research conducted during the Challenge.

Predicting Cardiovascular Outcomes Using the Respiratory Event Desaturation Transient Area Derived from Overnight Sleep Studies.

A new method for calculation of an overnight oximetry signal metric which is predictive of cardiovascular disease (CVD) outcomes in individuals undergoing an overnight sleep test is presented. The metric - the respiratory event desaturation transient area (REDTA) - quantifies the desaturation associated with respiratory events. Data from the Sleep Heart Health Study, which includes overnight oximetry signals and long-term CVD outcomes, was used to develop and test the parameter. Performance of the REDTA parameter was assessed using Cox proportional hazard ratios and compared to established metrics of hypoxia. Results show that hazard ratios in adjusted Cox analysis for predicting cardiovascular death using REDTA are up to 1.90 (95%CI: 1.22-2.96) which compares with the best of the established metrics. A big advantage of our metric compared to other high performing metrics is its ease of computation.

Automated Annotation of Polysomnogram Epochs for Apnoea and Non-apnoea Arousals.

A system for automated annotation of selected signals from the polysomnogram (PSG) for the presence of apnoea and non-apnoea arousals is presented. Fifty nine time- and frequency-domain features were derived from the PSG for each 15 second epoch and after combining features from adjacent epochs, the feature information was processed with a bank of feed-forward neural networks that provided a probability estimate that each epoch was associated with an apnoea or non-apnoea arousal, or no-arousal. Data from the Physionet Computing in Cardiology Challenge 2018 was used to develop and test the system. Performance of the system was assessed using volume under the receiver operator characteristic surface (VUROS) as well as no-arousal specificity and arousal sensitivities. Using a bank of ten feed-forward neural networks with each network processing ±4 epochs of features and each used a single hidden layer of 20 units, the system achieved a VUROS of 0.73 with a specificity of 70%, a sensitivity of 75% for the apnoea arousals, and a sensitivity of 70% for the non-apnoea arousals.

Is Cumulative Time of Oxygen Desaturation a Better Predictor of Cardiovascular Mortality than Apnoea Hypopnoea Index?

In this paper, we explored the link between sleep apnoea and cardiovascular disease (CVD) using a time-series statistical measure of sleep apnoea-related oxygen desaturation. We compared the performance of a hypoxic measure derived from the polysomnogram with the Apnoea Hypopnoea Index (AHI) in predicting CVD mortality in patients of the Sleep Heart Health Study.We estimated the relative cumulative time of SpO2 below 90% (Tr90) using pulse oximetry signals from polysomnogram recordings as the hypoxic measure of desaturation patterns. Then, the survival curves for hypoxia quintiles were evaluated for the prediction of CVD mortality and were compared with the results using AHI for prediction. We also calculated the Cox hazard ratios for Tr90 and AHI. Our results show that the Tr90 was a better predictor of CVD mortality outcomes than AHI.

Non-invasive Diagnosis of Sleep Apnoea Using ECG and Respiratory Bands.

In this paper, we used ECG signals and repiratory inductance plethysmography (RIP) or respiratory bands. We evaluated the performance of the signals individually as well as different combinations of features and signals for sleep apnoea detection. We implemented two methods (QRS area, and fast principal component analysis (PCA) methods) for estimating the ECG derived respiratory (EDR) signal and the cardiopulmonary coupling (CPC) spectrum. We then extracted features from the time and frequency representations of the ECG and RIP signals. Finally, we applied different features sets to a linear discriminant analysis (LDA) for classification. The results were examined on the MIT PhysioNet Apnea-ECG database. Apnoea classification was carried out using leave-one-record-out crossvalidation approach. The highest performance of our algorithm was achieved using the RIP and RR-interval features as well as using the RIP and PCA CPC features with an accuracy of 90% and AUC of 0.97. The highest performance results of using only RIP or ECG features achieved an accuracy of 87% and AUC of 0.95. We conclude that although ECG sensors are more convenient for patients in sleep studies, using both RIP and ECG sensors enhances the performance results for automated diagnosis of sleep apnoea.

Comparing Different Methods of Hand-crafted HRV, EDR and CPC Features for Sleep Apnoea Detection.

In this paper, we extracted hand crafted features from the ECG signals and evaluated the performance of different combination of features for sleep apnoea detection. We calculated the ECG derived respiratory (EDR) signal using three methods (QRS area, amplitude demodulation and fast PCA methods) and then calculated the cardiopulmonary coupling (CPC) spectrum using each EDR method. We then extracted features from the CPC spectrums and the time and frequency representations of the heart rate variability (HRV) and EDR signals Then, we compared the performance results of different combinations of the features used for automated sleep apnoea detection. We also applied a temporal optimisation method by averaging the features of every three adjacent epochs. Two classifiers were used to detect sleep apnoea: the extreme learning machine (ELM), and linear discriminant analysis. The features were evaluated on the MIT PhysioNet Apnea-ECG database. Apnoea detection was evaluated with leave-one-record-out cross-validation. The PCA CPC features obtained the highest accuracy of 86.5% and AUC of 0.94 using LDA classifier. The performance results of the combined features (of PCA method) obtained the same results. We conclude that for this study, the CPC features using fast PCA method are our best feature set for sleep apnoea detection.

An Investigation of the Relationship between the Apnoea Hypopnoea Index and Apnoea Hypopnoea Duration per Hour.

A number of automated apnoea hypopnoea index (AHI) prediction algorithms first divide the signal(s) of interest into epochs, make a prediction for each epoch, determine the number of epoch predictions per hour and map this to an AHI value. An underlying assumption of this approach is a smooth relationship between the apnoea plus hypopnoea duration and the AHI value. In this study we investigate this relationship to establish if this assumption impacts the final system. We compare two models: one which divides the duration by recording time, and the second which divides the duration by sleep time. Data for study was obtained from 200 scored overnight polysomnogram recordings. Our results show that the relationship is a power-law distribution with an exponent of 0.9 and a multiplicative noise term. Analysis of the variance of the noise term revealed that algorithms that scale the apnoea duration by the recording time will have an inherent 37% error in the AHI estimate, while algorithms that scale by sleep time will have an inherent 25% error in the AHI estimate. Receiver operator curve (ROC) analysis of the duration-based models at the clinically important values of AHI 5 and 15 revealed an area under the ROC of greater than 0.96. We conclude that epoch-based models for AHI estimation do have an inherent error in their estimates, with models that divide the duration by sleep time providing a better estimate. Both models can correctly identify normal and apnoeic patients at the clinically important values with high sensitivity and specificity.

Automatic scoring of non-apnoea arousals using hand-crafted features from the polysomnogram.

OBJECTIVE: We present a system for automated annotation of non-apnoea arousals using twelve signals from the polysomnogram (PSG) including airflow, six signals of electroencephalogram, the electrooculogram, chin electromyogram, oximetry signal, and chest and abdominal respiratory effort signals. APPROACH: Fifty-nine time- and frequency-domain features were extracted from the twelve signals using 15 s epochs. Features from an epoch were combined with features from adjacent epochs and then processed with a bank of feed-forward networks that provided a probability estimate of the occurrence of a non-apnoea arousal event in every epoch. Data from the 2018 PhysioNet/Computing in Cardiology Challenge was used to develop and test the system. Ten-fold cross validation on the 994 PSGs of training data was used to compare the performance of different network configurations. MAIN RESULTS: Our highest performing configuration utilised a bank of 30 feed-forward neural networks. Each network processed ±4 epochs of features and each used a single hidden layer of 20 units. The performance of this configuration was evaluated on the independent test set of 989 PSGs and achieved an area under the receiver operator curve of 0.848 and an area under the precision-recall curve of 0.325 for correctly discriminating non-apnoea arousals from non-arousals samples. SIGNIFICANCE: The classification performance results of our system demonstrate that automated annotation of non-apnoea arousals can be achieved with a high degree of reliability.

A Fast Principal Component Analysis Method For Calculating The ECG Derived Respiration.

In this paper, we present a principal component analysis (PCA) method for estimating the respiration from overnight ECG recording. In comparison to other published methods, our method is very fast to compute and has low memory requirements, which makes it suitable for processing long duration ECG recordings. We used our method to derive respiratory features for the ECG which were then used to identify epochs of sleep apnoea from the ECG. Three classifiers including the extreme learning machine (ELM), linear discriminant analysis, and support vector machine were used to detect sleep apnoea. The method was evaluated on the MIT PhysioNet Apnea-ECG database. Apnoea detection was evaluated with leave-one-record-out cross-validation. Our PCA method obtained the highest accuracy of 74% by ELM classifier. We conclude that the fast PCA method is useful to apply PCA to long ECG recordings.

Optimising the Apnoea Classification Performance of a Neural Network Classifier Processing ECG-Oximetry Signals.

In this paper we investigate using principal components analysis to optimize the performance of a neural network system processing simultaneously acquired electrocardiogram (ECG) and oximetry signals. The algorithm identifies epochs of normal breathing, central apnoea (CA), and obstructive apnoea (OA) by processing a pooled feature set containing information capturing the desaturations from the oximeter sensor as well as time and spectral features from the ECG. Training and testing of the system was facilitated with a dataset of 125 scored polysomnogram recordings with accompanying respiratory event annotations. When classifying the three epoch types, our system achieved a specificity of 91%, a sensitivity to CA of 28% and sensitivity to OA of 63%. A sensitivity of 81% was achieved when the CA and OA epochs were combined into one class.

A low-complexity algorithm for detection of atrial fibrillation using an ECG.

OBJECTIVES: We present a method for automatic processing of single-lead electrocardiogram (ECG) with duration of up to 60 s for the detection of atrial fibrillation (AF). The method categorises an ECG recording into one of four categories: normal, AF, other and noisy rhythm. For training the classification model, 8528 scored ECG signals were used; for independent performance assessment, 3658 scored ECG signals. APPROACH: Our method was based on features derived from RR interbeat intervals. The features included time domain, frequency domain and distribution features. We assessed the performance of three different classifiers (linear and quadratic discriminant analysis, and quadratic neural network (QNN)) on the training set using 100-fold cross-validation. The QNN was selected as the highest performing classifier, and a further performance assessment on the test data made. MAIN RESULTS: On the test set, our method achieved an F1 score for the normal, AF, other and noisy classes of 0.90, 0.75, 0.68 and 0.32, respectively. The overall F1 score was 0.78. SIGNIFICANCE: The computational cost of our algorithm is low as all features are derived from RR intervals and are processed by a single hidden layer neural network. This makes it potentially suitable for low-power devices.

Sleep apnoea diagnosis using respiratory effort-based signals - a comparative study.

A measure of the respiratory effort during a sleep study is an important contributor to the diagnosis of sleep apnoea. A common way of measuring respiratory effort is with bands with stretch sensors placed around the chest and/or abdomen. An alternative, and more convenient method from the patient's perspective, is via the ECG derived respiration (EDR) signal which provides an estimate of the respiratory effort at each heartbeat. In this study we performed a side-by-side comparison of the discrimination information for identifying epochs of sleep apnoea contained in the chest respiratory effort signal and three methods of calculating the EDR signal. Using simultaneously recorded chest band and ECG signals extracted from overnight polysomnogram (PSG) data from 8 subjects (4 controls, 4 apnoeas. MIT PhysioNet Apnea-ECG database), we extracted identical features from the two sensors and used the features to train a linear discriminant classifier to classify one-minute epochs as being apneic or normal. Ground truth labelling of each epoch was achieved with an expert using the full PSG as a reference. Our cross validation results revealed that the full respiratory effort signal resulted in an accuracy of 87% in correctly identifying the epoch label. When the respiratory signal was resampled at each heartbeat (as occurs with the EDR signal) the accuracy was 86%, suggesting that the sampling process inherent to the EDR signal does not materially affect its discrimination ability. The best EDR method was based on the calculating the QRS area for every heart and achieved an accuracy of 81%. Our results suggest that, while there is some information loss in the EDR estimation process, the EDR signal is a convenient and useful signal for sleep apnoea diagnosis.

Sleep apnoea classification using heart rate variability, ECG derived respiration and cardiopulmonary coupling parameters.

We investigated using heart rate variability (HRV), ECG derived respiration and cardiopulmonary coupling features (CPC) calculated from night-time single lead ECG signals to classify one-minute epochs for the presence or absence of sleep apnoea. We used the 35 training recordings of the M.I.T. Physionet Apnea-ECG database. Performance was assessed with leave-one-record-out cross-validation. The best classification performance was achieved using the CPC features in conjunction with the time-domain based HRV parameters. The cross-validated results on the 17,045 epochs of the dataset were an accuracy of 89.8%, a specificity of 92.9%, a sensitivity of 84.7%, and a kappa value of 0.78. These results are comparable with best results reported on this database.

A fast approximation method for principal component analysis applied to ECG derived respiration for OSA detection.

In this paper, we present an approximation method for principal component analysis (PCA) and apply it to estimating the respiration from the overnight ECG signal. The approximation method is computationally fast with low memory requirements. We compare it to a full PCA method which is applied to segments of the ECG. Features were calculated from the two ECG derived respiration signals (EDR) and classifiers trained to detect obstructive sleep apnoea (OSA). The Extreme Learning Machine and Linear Discriminant classifier were used to classify the recordings. The data from 35 overnight ECG recordings from MIT PhysioNet Apnea-ECG training database was utilized in the paper. Apnoea detection was evaluated with leave-one-record-out cross validation. The approximated PCA method obtained the highest accuracy of 76.4% by ELM classifier at fan-out 10 and accuracy of 78.4% by LDA. While, the segmented PCA achieved lower accuracies for both classifiers, 75.9% by ELM classifier and 76.6% by LDA. We conclude that the approximation method for PCA is well suited to deriving the respiration signal from overnight ECGs.

Reducing false arrhythmia alarms in the ICU using multimodal signals and robust QRS detection.

This study developed algorithms to decrease the arrhythmia false alarms in the ICU by processing multimodal signals of photoplethysmography (PPG), arterial blood pressure (ABP), and two ECG signals. The goal was to detect the five critical arrhythmias comprising asystole (ASY), extreme bradycardia (EBR), extreme tachycardia (ETC), ventricular tachycardia (VTA), and ventricular flutter or fibrillation (VFB). The different characteristics of the arrhythmias suggested the application of individual signal processing for each alarm and the combination of the algorithms to enhance false alarm detection. Thus, different features and signal processing techniques were used for each arrhythmia type. The ECG signals were first processed to reduce the signal interference. Then, a Hilbert-transform based QRS detector algorithm was utilized to identify the QRS complexes, which were then processed to determine the instantaneous heart rate. The pulsatile signals (PPG and ABP) were processed to discover the pulse onset of beats which were then employed to measure the heart rate. The signal quality index (SQI) of the signals was implemented to verify the integrity of the heart rate information. The overall score obtained by our algorithms in the 2015 Computing in Cardiology Challenge was a score of 74.03% for retrospective and 69.92% for real-time analysis.

An ECG oximetry system for identifying obstructive and central apnoea events.

An automatic algorithm for processing simultaneously acquired electrocardiogram (ECG) and oximetry signals that identifies epochs of pure central apnoea, epochs containing obstructive apnoea and epochs of normal breathing is presented. The algorithm uses time and spectral features from the ECG derived heart-rate and respiration information, as well as features capturing desaturations from the oximeter sensor. Evaluation of performance of the system was achieved by using leave-one-record-out cross validation on the St. Vincent's University Hospital / University College Dublin Sleep Apnea Database from the Physionet collections of recorded physiologic signals. When classifying the three epoch types, our system achieved a specificity of 80%, a sensitivity to central apnoea of 44% and sensitivity to obstructive apnoea of 35%. A sensitivity of 81% was achieved when the central and obstructive epochs were combined into one class.

Sleep apnoea episodes recognition by a committee of ELM classifiers from ECG signal.

This paper describes a system for the recognition of sleep apnoea episodes from ECG signals using a committee of extreme learning machine (ELM) classifiers. RR-interval parameters (heart rate variability) have been used as the identifying features as they are directly affected by sleep apnoea. The MIT PhysioNet Apnea-ECG database was used. A committee of five ELM classifiers has been employed to classify one-minute epochs of ECG into normal or apnoeic epochs. Our results show that the classification performance from the committee of networks was superior to the results of a single ELM classifier for fan-outs from 1 to 100. Classification performance reached a plateau at a fan-out of 10. The maximum accuracy was 82.5% with a sensitivity of 81.9% and a specificity of 82.8%. The results were comparable to other published research with the same input data.