<p>Sleep staging is used to assist diagnosis of sleep disorders. Polysomnography (PSG) provides multiple channels of signal recording that reflects different working patterns of each sleep stage and can be utilized for automatic sleep staging. However, previous deep learning based sleep staging researches ignored the dependencies between local and global temporal sequences of multi-modal PSG signals. To solve this problem, this work proposed a multi-modal fusion network via temporal sequence for sleep staging (MMTSleepNet), consisting of local and global feature extraction (LGFE) block, adaptive modal feature recalibration (AMFR) block, multi-modal feature fusion (MFF) block and sleep staging block. LGFE block extracts both local and global contextual features to learn differentiated characteristics across sleep stages. AMFR block adaptively adjusts channel-wise weights on the concatenated feature maps to highlight more contributing PSG modalities, where each channel corresponds to a specific PSG signal or sub-signal. MFF block models feature dependencies to quantify the correlations between multi-modal temporal sequences. The proposed MMTSleepNet outperforms the state-of-the-art methods on Sleep-EDF-20, Sleep-EDF-78 and Sleep Heart Health Study (SHHS) datasets, with the accuracy rates of 87.5%, 82.9% and 87.8%, respectively.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MMTSleepNet: a multi-modal fusion network via temporal sequence for sleep staging

  • Jie Pan,
  • Wenlong Lv,
  • Xiaoyu Zou,
  • Ying Gao,
  • Ying Liu,
  • Ying Liu,
  • Xuelin Peng

摘要

Sleep staging is used to assist diagnosis of sleep disorders. Polysomnography (PSG) provides multiple channels of signal recording that reflects different working patterns of each sleep stage and can be utilized for automatic sleep staging. However, previous deep learning based sleep staging researches ignored the dependencies between local and global temporal sequences of multi-modal PSG signals. To solve this problem, this work proposed a multi-modal fusion network via temporal sequence for sleep staging (MMTSleepNet), consisting of local and global feature extraction (LGFE) block, adaptive modal feature recalibration (AMFR) block, multi-modal feature fusion (MFF) block and sleep staging block. LGFE block extracts both local and global contextual features to learn differentiated characteristics across sleep stages. AMFR block adaptively adjusts channel-wise weights on the concatenated feature maps to highlight more contributing PSG modalities, where each channel corresponds to a specific PSG signal or sub-signal. MFF block models feature dependencies to quantify the correlations between multi-modal temporal sequences. The proposed MMTSleepNet outperforms the state-of-the-art methods on Sleep-EDF-20, Sleep-EDF-78 and Sleep Heart Health Study (SHHS) datasets, with the accuracy rates of 87.5%, 82.9% and 87.8%, respectively.