Significant progress has been made in the segmentation of brain microstructure from 3D electron microscopic (EM) images, owing to the development of deep neural networks, such as CNN and Transformers. However, existing methods face challenges in balancing global relationship modeling and efficient computation. Recently, state-space model (SSM) from classical control theory, particularly the improved Mamba with a selection mechanism, has excelled in modeling long-range dependencies. Yet, their application to 3D EM image segmentation is hindered by challenges such as anisotropic voxel structures. To address this issue, we propose a network with an adaptive module based on state-space model. Our approach incorporates a novel scanning strategy tailored to anisotropic voxels, enabling more effective feature ex-traction. Based on this strategy, we introduce two key components: SSM-based anisotropic adaptation module and SSM-based isotropic adaptation module. In addition, we integrate convolutional layers and an inter-leaved combination of these components to facilitate multi-scale feature learning. Extensive experiments on two public datasets demonstrate that our method outperforms existing approaches in segmentation accuracy while achieving lower memory consumption.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Anisotropic 3D EM Image Segmentation with State-Space Mode

  • Jiachi Chen,
  • Hongyu Yang,
  • Meng Xing

摘要

Significant progress has been made in the segmentation of brain microstructure from 3D electron microscopic (EM) images, owing to the development of deep neural networks, such as CNN and Transformers. However, existing methods face challenges in balancing global relationship modeling and efficient computation. Recently, state-space model (SSM) from classical control theory, particularly the improved Mamba with a selection mechanism, has excelled in modeling long-range dependencies. Yet, their application to 3D EM image segmentation is hindered by challenges such as anisotropic voxel structures. To address this issue, we propose a network with an adaptive module based on state-space model. Our approach incorporates a novel scanning strategy tailored to anisotropic voxels, enabling more effective feature ex-traction. Based on this strategy, we introduce two key components: SSM-based anisotropic adaptation module and SSM-based isotropic adaptation module. In addition, we integrate convolutional layers and an inter-leaved combination of these components to facilitate multi-scale feature learning. Extensive experiments on two public datasets demonstrate that our method outperforms existing approaches in segmentation accuracy while achieving lower memory consumption.