Recent advances in circuit design and sensing technology have made it increasingly feasible to collect data from multiple sensors simultaneously, enabling more accurate and comprehensive inferences in biomedical applications. This chapter presents a novel convolutional neural network (CNN) architecture designed specifically for the fusion of multimodal and multiresolution sensor data—signals acquired at differing sampling frequencies. Unlike conventional approaches that rely on signal padding or resampling to align resolutions, this model directly integrates disparate inputs, supporting high resolution temporal inference without the need for frequency normalization. The model’s performance is evaluated for sleep apnea event detection, by fusing three different sensor signals, namely, electrocardiogram (ECG), peripheral oxygen saturation signal (SpO2), and abdominal movement signal, obtained from UCD St. Vincent University Hospital’s sleep apnea database. Results show that fusion using all three signals yields an accuracy of 99.72% and a sensitivity of 98.98%. The chapter also introduces a selective dropout mechanism to prevent overfitting to any single high-resolution input and enhance model robustness in the presence of corrupted or missing data. Furthermore, the model exhibits incremental performance gains as more sensor inputs are added, demonstrating its scalability and generalizability. Model pruning techniques for complexity reduction are also explored, highlighting the potential for real-time deployment in resource-constrained environments.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Multimodal and Multiresolution Fusion Using Convolutional Neural Networks

  • Arlene John,
  • Barry Cardiff,
  • Deepu John

摘要

Recent advances in circuit design and sensing technology have made it increasingly feasible to collect data from multiple sensors simultaneously, enabling more accurate and comprehensive inferences in biomedical applications. This chapter presents a novel convolutional neural network (CNN) architecture designed specifically for the fusion of multimodal and multiresolution sensor data—signals acquired at differing sampling frequencies. Unlike conventional approaches that rely on signal padding or resampling to align resolutions, this model directly integrates disparate inputs, supporting high resolution temporal inference without the need for frequency normalization. The model’s performance is evaluated for sleep apnea event detection, by fusing three different sensor signals, namely, electrocardiogram (ECG), peripheral oxygen saturation signal (SpO2), and abdominal movement signal, obtained from UCD St. Vincent University Hospital’s sleep apnea database. Results show that fusion using all three signals yields an accuracy of 99.72% and a sensitivity of 98.98%. The chapter also introduces a selective dropout mechanism to prevent overfitting to any single high-resolution input and enhance model robustness in the presence of corrupted or missing data. Furthermore, the model exhibits incremental performance gains as more sensor inputs are added, demonstrating its scalability and generalizability. Model pruning techniques for complexity reduction are also explored, highlighting the potential for real-time deployment in resource-constrained environments.