MSPFM: Multi-Scale Pyramid Fusion Mamba for Medical Image Classification
摘要
Medical image classification is an essential computer vision task that helps clinical care and treatment. However, the rich semantic information is hard to extract due to the complexity of medical images. In addition, medical images contain a multitude of irrelevant information, making it challenging for models to identify effective features. Inspired by Mamba, which is a kind of state space model (SSM) that is excel in capturing effective long-range dependencies and filtering out the irrelevant, we propose a Multi-Scale Pyramid Fusion Mamba (MSPFM) for medical image classification. We construct the global feature extracting SSM (GFE-SSM) and the Input-Dependent Local Feature Extractor (IDLF-Extractor) by introducing 2D-Selective-Scan and Input-Dependent Dynamic Convolution, which distill key information while ignoring the irrelevant, enhancing the learning of representations of medical images and mitigating the degradation of accuracy. To further extract the rich semantic information, we designed the Multi-Scale Pyramid Fusion (MSPF) module, which combines the GFE-SSM and IDLF-Extractor to form our proposed MSPFM for effective distillation of key information by fusing multi-scale features with rich representations of medical images. Comprehensive comparative experiments prove that our proposed MSPFM performs the best compared to other advanced models for medical image classification.