DistillSleep: Leverage Self-distillation to Improve Performance After Representation Learning for Sleep Staging
摘要
Supervised learning for automatic sleep staging often requires large amounts of expert-labeled data. However, the rapid growth of data has not been matched by a corresponding increase in its effective utilization. While representation learning has enabled leveraging large volumes of unlabeled data, there is still much room for improvement in effectively applying learned features to downstream tasks. To tackle this challenge, we introduce a representation enhancement framework with self-distillation for sleep staging (DistillSleep). By performing self-distillation learning on the sleep staging task, DistillSleep is able to further optimize the alignment of features of the self-supervised learning phase with downstream sleep staging tasks, compared to the fine-tuning method alone. Our approach provides new insights into improving the performance of sleep staging tasks with limited labeled data. Experiments are conducted on a widely used public Polysomnography (PSG) dataset, SleepEDF, which demonstrates that our approach can successfully achieve a competitive performance compared with current methods on a single-channel electroencephalogram (EEG) using only 10% labeled data. In addition, to validate the general applicability of our method, we also successfully evaluate its effectiveness on the self-collected Ballistocardiogram (BCG) sleep staging dataset. (we will release our code and BCG dataset used in this paper after it is accepted)