Human parsing, which classifies each pixel in human images into predefined classes of human parts or clothing, faces challenges in efficient modeling complex scenarios involving pose variations and occlusion. While existing methods have shown that pose estimation benefits human parsing via multi-task learning, most of them use simplistic feature fusion strategies, making it difficult to achieve efficient parsing-pose interaction. In this paper, we propose Mamba-Driven Feature Fusion Network (MFNet), leveraging Mamba’s global modeling capabilities to establish deep bidirectional parsing-pose interaction, enhancing fused features for joint human parsing and pose estimation. MFNet employs a shared backbone for initial feature extraction. Central to the framework, we introduce a cross fusion mamba module, enabling efficient parsing-pose interaction while capturing comprehensive global information. Simultaneously, an attention atrous spatial pyramid pooling module refines parsing features using dual channel-spatial attention to adaptively enhance multi-scale context. Through the combination of the two modules, MFNet effectively integrates parsing and pose features while capturing comprehensive global information. Results on two datasets demonstrate MFNet’s superiority in various metrics.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MFNet: Mamba-Driven Feature Fusion for Human Parsing

  • Xinyu Liu,
  • Xiaomei Zhang,
  • Jing Gu,
  • Ajian Liu,
  • Fengmei Liang,
  • Hongsen Bi,
  • Zhen Lei

摘要

Human parsing, which classifies each pixel in human images into predefined classes of human parts or clothing, faces challenges in efficient modeling complex scenarios involving pose variations and occlusion. While existing methods have shown that pose estimation benefits human parsing via multi-task learning, most of them use simplistic feature fusion strategies, making it difficult to achieve efficient parsing-pose interaction. In this paper, we propose Mamba-Driven Feature Fusion Network (MFNet), leveraging Mamba’s global modeling capabilities to establish deep bidirectional parsing-pose interaction, enhancing fused features for joint human parsing and pose estimation. MFNet employs a shared backbone for initial feature extraction. Central to the framework, we introduce a cross fusion mamba module, enabling efficient parsing-pose interaction while capturing comprehensive global information. Simultaneously, an attention atrous spatial pyramid pooling module refines parsing features using dual channel-spatial attention to adaptively enhance multi-scale context. Through the combination of the two modules, MFNet effectively integrates parsing and pose features while capturing comprehensive global information. Results on two datasets demonstrate MFNet’s superiority in various metrics.