<p>Existing methods for compound facial expression recognition (FER) rely heavily on large-scale labeled data for training. Unfortunately, in many practical applications, only limited annotated data are available. In this paper, we address the compound FER task under the cross-domain few-shot learning setting, which involves a large number of easily accessible basic expression samples in the source domain and only a few labeled compound expression samples in the target domain. To this end, we propose a novel visual-textual cascaded decomposition network (VCD-Net) for compound FER. VCD-Net consists of a textual-aware visual module (TVM) and a relation-aware textual module (RTM). The two modules are engaged in a bidirectional cross-modal interaction framework. TVM uses textual information from RTM to guide visual feature extraction, while RTM uses visual information from TVM to refine textual representations. In particular, TVM includes a visual-textual fusion net (VFNet) and a cascaded decomposition net (CDNet). VFNet leverages textual priors to guide the learning of visual features, while the CDNet effectively cascades parameter-shared learn-to-decompose modules based on a sequential decomposition mechanism. Meanwhile, RTM leverages a graph convolutional network to model the dependency between expression category names and refines graph relations using visual expression features. By training across similar tasks, VCD-Net successfully learns the ability to decompose features, and thus it can extract highly transferable expression features. Extensive experiments demonstrate the superiority of VCD-Net over several state-of-the-art few-shot learning methods. Code is released at <a href="https://github.com/cxtjl/VCD-Net">https://github.com/cxtjl/VCD-Net</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

VCD-Net: Visual-Textual Cascaded Decomposition Network for Cross-Domain Few-Shot Facial Expression Recognition

  • Xueting Chen,
  • Xinyi Zou,
  • Yan Yan,
  • Xinbo Gao,
  • Hanzi Wang

摘要

Existing methods for compound facial expression recognition (FER) rely heavily on large-scale labeled data for training. Unfortunately, in many practical applications, only limited annotated data are available. In this paper, we address the compound FER task under the cross-domain few-shot learning setting, which involves a large number of easily accessible basic expression samples in the source domain and only a few labeled compound expression samples in the target domain. To this end, we propose a novel visual-textual cascaded decomposition network (VCD-Net) for compound FER. VCD-Net consists of a textual-aware visual module (TVM) and a relation-aware textual module (RTM). The two modules are engaged in a bidirectional cross-modal interaction framework. TVM uses textual information from RTM to guide visual feature extraction, while RTM uses visual information from TVM to refine textual representations. In particular, TVM includes a visual-textual fusion net (VFNet) and a cascaded decomposition net (CDNet). VFNet leverages textual priors to guide the learning of visual features, while the CDNet effectively cascades parameter-shared learn-to-decompose modules based on a sequential decomposition mechanism. Meanwhile, RTM leverages a graph convolutional network to model the dependency between expression category names and refines graph relations using visual expression features. By training across similar tasks, VCD-Net successfully learns the ability to decompose features, and thus it can extract highly transferable expression features. Extensive experiments demonstrate the superiority of VCD-Net over several state-of-the-art few-shot learning methods. Code is released at https://github.com/cxtjl/VCD-Net.