Sleep is essential for human wellbeing and good health. One of the diseases impacting sleep quality is the so-called REM (Rapid Eye Movements) sleep behavior disorder (RBD), which can be due to serious neuro-degenerative pathologies. Typically, sleep is investigated by monitoring the patient during sleep adopting the polysomnography (PSG), which is the standard examination protocol for diagnosing and monitoring RBD events. Unfortunately, PSG is a costly and invasive procedure, requiring the patient to be physically connected to multiple sensors recording physiological signals while being monitored by a video camera. In our study, we demonstrate how multimodal distillation can be leveraged to perform RBD event classification based on video data only. By casting the problem as a binary classification task, a teacher model distills multimodal PSG physiological data to guide a student model that relies solely on the input video stream. Our method improves F1-score by up to 7.5% over the baseline, demonstrating the feasibility of non-invasive, cost-effective video-based RBD classification systems.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Distillation for Video-Based Sleep Behavior Analysis

  • Jacopo Donà,
  • Marco Carletti,
  • Davide Tonon,
  • Gianluca Rossato,
  • Vittorio Murino

摘要

Sleep is essential for human wellbeing and good health. One of the diseases impacting sleep quality is the so-called REM (Rapid Eye Movements) sleep behavior disorder (RBD), which can be due to serious neuro-degenerative pathologies. Typically, sleep is investigated by monitoring the patient during sleep adopting the polysomnography (PSG), which is the standard examination protocol for diagnosing and monitoring RBD events. Unfortunately, PSG is a costly and invasive procedure, requiring the patient to be physically connected to multiple sensors recording physiological signals while being monitored by a video camera. In our study, we demonstrate how multimodal distillation can be leveraged to perform RBD event classification based on video data only. By casting the problem as a binary classification task, a teacher model distills multimodal PSG physiological data to guide a student model that relies solely on the input video stream. Our method improves F1-score by up to 7.5% over the baseline, demonstrating the feasibility of non-invasive, cost-effective video-based RBD classification systems.