A multi-label image classification method via graph attention network with dynamic and static label correlations
摘要
Multi-label image classification has gained widespread application across various domains. Capturing label correlations and extracting image spatial features from images have emerged as focal points in the study of multi-label classification tasks. However, existing methods often cannot fully extract the spatial combinatorial features of the objects and only capture dynamic or static label correlations, unable to establish high-quality mapping relationships between labels and object features. This paper aims to fill this gap by proposing a multi-label image classification method via Graph Attention Network (GAT) with dynamic and static label correlations. Initially, we use an image feature extraction module based on the transformer attention mechanism to extract the spatial combinatorial features of the objects and generate the learned label feature sequence. Subsequently, we construct a dynamic-static label correlations fusion module based on GAT to optimize the label features and establish high-quality mapping relationships between labels and object features. Ultimately, based on the multi-label image classification method via GAT with dynamic and static label correlations, we construct a classification head named Encoder Decoder GAT Encoder (EDGE) and a multi-label classification model named TResNet-L-EDGE. Experimental results on the VOC 2007, VOC 2012, and COCO 2014 datasets demonstrate the effectiveness of this method in enhancing the accuracy of multi-label image classification. Public code is available at: https://github.com/lizm-109/TesNet-L-EDGE