<p>Multi-label image classification has gained widespread application across various domains. Capturing label correlations and extracting image spatial features from images have emerged as focal points in the study of multi-label classification tasks. However, existing methods often cannot fully extract the spatial combinatorial features of the objects and only capture dynamic or static label correlations, unable to establish high-quality mapping relationships between labels and object features. This paper aims to fill this gap by proposing a multi-label image classification method via Graph Attention Network (GAT) with dynamic and static label correlations. Initially, we use an image feature extraction module based on the transformer attention mechanism to extract the spatial combinatorial features of the objects and generate the learned label feature sequence. Subsequently, we construct a dynamic-static label correlations fusion module based on GAT to optimize the label features and establish high-quality mapping relationships between labels and object features. Ultimately, based on the multi-label image classification method via GAT with dynamic and static label correlations, we construct a classification head named Encoder Decoder GAT Encoder (EDGE) and a multi-label classification model named TResNet-L-EDGE. Experimental results on the VOC 2007, VOC 2012, and COCO 2014 datasets demonstrate the effectiveness of this method in enhancing the accuracy of multi-label image classification. Public code is available at: <a href="https://github.com/lizm-109/TesNet-L-EDGE">https://github.com/lizm-109/TesNet-L-EDGE</a></p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A multi-label image classification method via graph attention network with dynamic and static label correlations

  • Zhiming Li,
  • Kai Zhou,
  • Bingnan Chen,
  • Yuchen Lv,
  • Yihao He,
  • Dianlong You

摘要

Multi-label image classification has gained widespread application across various domains. Capturing label correlations and extracting image spatial features from images have emerged as focal points in the study of multi-label classification tasks. However, existing methods often cannot fully extract the spatial combinatorial features of the objects and only capture dynamic or static label correlations, unable to establish high-quality mapping relationships between labels and object features. This paper aims to fill this gap by proposing a multi-label image classification method via Graph Attention Network (GAT) with dynamic and static label correlations. Initially, we use an image feature extraction module based on the transformer attention mechanism to extract the spatial combinatorial features of the objects and generate the learned label feature sequence. Subsequently, we construct a dynamic-static label correlations fusion module based on GAT to optimize the label features and establish high-quality mapping relationships between labels and object features. Ultimately, based on the multi-label image classification method via GAT with dynamic and static label correlations, we construct a classification head named Encoder Decoder GAT Encoder (EDGE) and a multi-label classification model named TResNet-L-EDGE. Experimental results on the VOC 2007, VOC 2012, and COCO 2014 datasets demonstrate the effectiveness of this method in enhancing the accuracy of multi-label image classification. Public code is available at: https://github.com/lizm-109/TesNet-L-EDGE