Bridging AI and Clinical Practice: Integrating Automated Sleep Scoring Algorithm with Uncertainty-Guided Physician Review.

Purpose: This study aims to enhance the clinical use of automated sleep-scoring algorithms by incorporating an uncertainty estimation approach to efficiently assist clinicians in the manual review of predicted hypnograms, a necessity due to the notable inter-scorer variability inherent in polysomnography (PSG) databases. Our efforts target the extent of review required to achieve predefined agreement levels, examining both in-domain (ID) and out-of-domain (OOD) data, and considering subjects' diagnoses. Patients and Methods: A total of 19,578 PSGs from 13 open-access databases were used to train U-Sleep, a state-of-the-art sleep-scoring algorithm. We leveraged a comprehensive clinical database of an additional 8832 PSGs, covering a full spectrum of ages (0-91 years) and sleep-disorders, to refine the U-Sleep, and to evaluate different uncertainty-quantification approaches, including our novel confidence network. The ID data consisted of PSGs scored by over 50 physicians, and the two OOD sets comprised recordings each scored by a unique senior physician. Results: U-Sleep demonstrated robust performance, with Cohen's kappa (K) at 76.2% on ID and 73.8-78.8% on OOD data. The confidence network excelled at identifying uncertain predictions, achieving AUROC scores of 85.7% on ID and 82.5-85.6% on OOD data. Independently of sleep-disorder status, statistical evaluations revealed significant differences in confidence scores between aligning vs discording predictions, and significant correlations of confidence scores with classification performance metrics. To achieve κ ≥ 90% with physician intervention, examining less than 29.0% of uncertain epochs was required, substantially reducing physicians' workload, and facilitating near-perfect agreement. Conclusion: Inter-scorer variability limits the accuracy of the scoring algorithms to ~80%. By integrating an uncertainty estimation with U-Sleep, we enhance the review of predicted hypnograms, to align with the scoring taste of a responsible physician. Validated across ID and OOD data and various sleep-disorders, our approach offers a strategy to boost automated scoring tools' usability in clinical settings.

U-Sleep's resilience to AASM guidelines.

AASM guidelines are the result of decades of efforts aiming at standardizing sleep scoring procedure, with the final goal of sharing a worldwide common methodology. The guidelines cover several aspects from the technical/digital specifications, e.g., recommended EEG derivations, to detailed sleep scoring rules accordingly to age. Automated sleep scoring systems have always largely exploited the standards as fundamental guidelines. In this context, deep learning has demonstrated better performance compared to classical machine learning. Our present work shows that a deep learning-based sleep scoring algorithm may not need to fully exploit the clinical knowledge or to strictly adhere to the AASM guidelines. Specifically, we demonstrate that U-Sleep, a state-of-the-art sleep scoring algorithm, can be strong enough to solve the scoring task even using clinically non-recommended or non-conventional derivations, and with no need to exploit information about the chronological age of the subjects. We finally strengthen a well-known finding that using data from multiple data centers always results in a better performing model compared with training on a single cohort. Indeed, we show that this latter statement is still valid even by increasing the size and the heterogeneity of the single data cohort. In all our experiments we used 28528 polysomnography studies from 13 different clinical studies.

Wearables in the home-based assessment of abnormal movements in Parkinson's disease: a systematic review of the literature.

At present, the standard practices for home-based assessments of abnormal movements in Parkinson's disease (PD) are based either on subjective tools or on objective measures that often fail to capture day-to-day fluctuations and long-term information in real-life conditions in a way that patient's compliance and privacy are secured. The employment of wearable technologies in PD represents a great paradigm shift in healthcare remote diagnostics and therapeutics monitoring. However, their applicability in everyday clinical practice seems to be still limited. We carried out a systematic search across the Medline Database. In total, 246 publications, published until 1 June 2020, were identified. Among them, 26 reports met the inclusion criteria and were included in the present review. We focused more on clinically relevant aspects of wearables' application including feasibility and efficacy of the assessment, the number, type and body position of the wearable devices, type of PD motor symptom, environment and duration of assessments and validation methodology. The aim of this review is to provide a systematic overview of the current knowledge and state-of-the-art of the home-based assessment of motor symptoms and fluctuations in PD patients using wearable technology, highlighting current problems and laying foundations for future works.

Temporal dependency in automatic sleep scoring via deep learning based architectures: An empirical study.

The present study evaluates how effectively a deep learning based sleep scoring system does encode the temporal dependency from raw polysomnography signals. An exhaustive range of neural networks, including state of the art architecture, have been used in the evaluation. The architectures have been assessed using a single-channel EEG Fpz-Cz from the open source Sleep-EDF expanded database. The best performing model reached an overall accuracy of 85.2% and a Cohen's kappa of 0.8, with an F1-score of stage N1 equal to 50.2%. We have introduced a new metric, δnorm, to better evaluate temporal dependencies. A simple feed forward architecture not only achieves comparable performance to most up-to-date complex architectures, but also does better encode the continuous temporal characteristics of sleep.Clinical relevance - A better understanding of the capability of the network in encoding sleep temporal patterns could lead to improve the automatic sleep scoring.

Personalized Sleep Spindle Detection in Whole Night Polysomnography.

The present study proposes a new personalized sleep spindle detection algorithm, suggesting the importance of an individualized approach. We identify an optimal set of features that characterize the spindle and exploit a support vector machine to distinguish between spindle and nonspindle patterns. The algorithm is assessed on the open source DREAMS database, that contains only selected part of the polysomnography, and on whole night polysomnography recordings from the SPASH database. We show that on the former database the personalization can boost sensitivity, from 84.2% to 89.8%, with a slight increase in specificity, from 97.6% to 98.1%. On a whole night polysomnography instead, the algorithm reaches a sensitivity of 98.6% and a specificity of 98.1%, thanks to the personalization approach. Future work will address the integration of the spindle detection algorithm within a sleep scoring automated procedure.

Automated sleep scoring: A review of the latest approaches.

Clinical sleep scoring involves a tedious visual review of overnight polysomnograms by a human expert, according to official standards. It could appear then a suitable task for modern artificial intelligence algorithms. Indeed, machine learning algorithms have been applied to sleep scoring for many years. As a result, several software products offer nowadays automated or semi-automated scoring services. However, the vast majority of the sleep physicians do not use them. Very recently, thanks to the increased computational power, deep learning has also been employed with promising results. Machine learning algorithms can undoubtedly reach a high accuracy in specific situations, but there are many difficulties in their introduction in the daily routine. In this review, the latest approaches that are applying deep learning for facilitating and accelerating sleep scoring are thoroughly analyzed and compared with the state of the art methods. Then the obstacles in introducing automated sleep scoring in the clinical practice are examined. Deep learning algorithm capabilities of learning from a highly heterogeneous dataset, in terms both of human data and of scorers, are very promising and should be further investigated.