Exploring scoring methods for research studies: Accuracy and variability of visual and automated sleep scoring.

Sleep studies face new challenges in terms of data, objectives and metrics. This requires reappraising the adequacy of existing analysis methods, including scoring methods. Visual and automatic sleep scoring of healthy individuals were compared in terms of reliability (i.e., accuracy and stability) to find a scoring method capable of giving access to the actual data variability without adding exogenous variability. A first dataset (DS1, four recordings) scored by six experts plus an autoscoring algorithm was used to characterize inter-scoring variability. A second dataset (DS2, 88 recordings) scored a few weeks later was used to explore intra-expert variability. Percentage agreements and Conger's kappa were derived from epoch-by-epoch comparisons on pairwise and consensus scorings. On DS1 the number of epochs of agreement decreased when the number of experts increased, ranging from 86% (pairwise) to 69% (all experts). Adding autoscoring to visual scorings changed the kappa value from 0.81 to 0.79. Agreement between expert consensus and autoscoring was 93%. On DS2 the hypothesis of intra-expert variability was supported by a systematic decrease in kappa scores between autoscoring used as reference and each single expert between datasets (.75-.70). Although visual scoring induces inter- and intra-expert variability, autoscoring methods can cope with intra-scorer variability, making them a sensible option to reduce exogenous variability and give access to the endogenous variability in the data.

Automatic analysis of single-channel sleep EEG: validation in healthy individuals.

STUDY OBJECTIVE: To assess the performance of automatic sleep scoring software (ASEEGA) based on a single EEG channel comparatively with manual scoring (2 experts) of conventional full polysomnograms. DESIGN: Polysomnograms from 15 healthy individuals were scored by 2 independent experts using conventional R&K rules. The results were compared to those of ASEEGA scoring on an epoch-by-epoch basis. SETTING: Sleep laboratory in the physiology department of a teaching hospital. PARTICIPANTS: Fifteen healthy volunteers. MEASUREMENTS AND RESULTS: The epoch-by-epoch comparison was based on classifying into 2 states (wake/sleep), 3 states (wake/REM/ NREM), 4 states (wake/REM/stages 1-2/SWS), or 5 states (wake/REM/ stage 1/stage 2/SWS). The obtained overall agreements, as quantified by the kappa coefficient, were 0.82, 0.81, 0.75, and 0.72, respectively. Furthermore, obtained agreements between ASEEGA and the expert consensual scoring were 96.0%, 92.1%, 84.9%, and 82.9%, respectively. Finally, when classifying into 5 states, the sensitivity and positive predictive value of ASEEGA regarding wakefulness were 82.5% and 89.7%, respectively. Similarly, sensitivity and positive predictive value regarding REM state were 83.0% and 89.1%. CONCLUSIONS: Our results establish the face validity and convergent validity of ASEEGA for single-channel sleep analysis in healthy individuals. ASEEGA appears as a good candidate for diagnostic aid and automatic ambulant scoring.

Overnight duty impairs behaviour, awake activity and sleep in medical doctors.

BACKGROUND: The impact of prolonged work cycles among senior doctors remains disputed. We evaluated the effects of overnight duty on awake activity and sleep quality in senior doctors in emergency medical specialties. METHODS: Thirty-six healthy doctors were monitored during a 2-week period including three separate 84 h on-call cycles. An on-call cycle consisted of the night and the day before night duty; the night duty itself and the subsequent 2 days and nights after night duty. The first day after night duty could either be worked or not. Actigraphy was used to measure physical activity and to evaluate sleep duration and quality. A standardized questionnaire was used to assess daytime performance and night sleep quality. RESULTS: Night actigraphy demonstrated that on-call work induced a significant reduction in sleep duration that was not recovered during the subsequent two nights. Sleep during the night duty itself was fragmented and of poor quality. Awake activity was significantly impaired on the day after night duty. Although subjectively night sleep quality did not differ between the nights before and after night duty, all subjective daytime parameters were impaired the day after night duty, and mood, fatigue and concentration remained altered on the second day. Working the day after night duty impaired objective measurements of daytime activity and sleep quality during the subsequent two nights. CONCLUSIONS: On-call night work in acute specialties induces sleep debt associated with prolonged impairment of awake activity, sleep quality and performance. Not working the following day after an on-call night allows partial recovery of sleep quality to begin.