Pedestrian Open-Attribute Recognition (POAR) involves identifying appearance attributes within an open-world setting. In this work, we focus on improving the performance of the model in identifying both seen and unseen attributes. To achieve this, we propose a novel POAR framework based on Dynamic Semantic Masking (DSM). The proposed framework first employs an Adaptive Coupled Localization (ACL) module, synergizing OpenPose and SAM outputs for precise localization of fine-grained attribute parts. Subsequently, a Keypoint-driven Dynamic Attention Masking (KDAM) mechanism produces attribute-specific image masks. The model is then optimized via a vision-language multimodal contrastive loss. Finally, to enhance generalization towards unseen attributes, the knowledge distillation method is employed to transfer the generalization capabilities of CLIP to our model. The experimental results demonstrate the effectiveness of the proposed method on three public PAR datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Pedestrian Open-Attribute Recognition via Dynamic Semantic Masking

  • Yue Zhang,
  • Zhehao Zhang,
  • Sen Feng,
  • Fanghui Zhang,
  • Guoqi Liu,
  • Yigang Cen

摘要

Pedestrian Open-Attribute Recognition (POAR) involves identifying appearance attributes within an open-world setting. In this work, we focus on improving the performance of the model in identifying both seen and unseen attributes. To achieve this, we propose a novel POAR framework based on Dynamic Semantic Masking (DSM). The proposed framework first employs an Adaptive Coupled Localization (ACL) module, synergizing OpenPose and SAM outputs for precise localization of fine-grained attribute parts. Subsequently, a Keypoint-driven Dynamic Attention Masking (KDAM) mechanism produces attribute-specific image masks. The model is then optimized via a vision-language multimodal contrastive loss. Finally, to enhance generalization towards unseen attributes, the knowledge distillation method is employed to transfer the generalization capabilities of CLIP to our model. The experimental results demonstrate the effectiveness of the proposed method on three public PAR datasets.