<p>Existing scene graph generation methods primarily focus on addressing the long-tail problem in the labeling. However, most debiasing approaches struggle with a trade-off between head and tail class performance compared to biased-trained models. In this paper, we propose a balanced fusion strategy to leverage the strengths of both model types. From the perspective of causality, we proposed a mutual supervision intervention (MSI) method, which consists of a batch-level intervention and a model-level alignment part. This method is integrated into the existing Introd framework to address the bias in the fusion process of scene graph generation models. We propose a new metric called DP@100 to compare the output of the student model and the teacher model, providing an evaluation of the fusion performance under this distillation framework. The results of experiments performed on the Visual Genome dataset show the effectiveness of the proposed MSI method.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Mutual introspective distillation for unbiased scene graph generation

  • Bo Sun,
  • Zhuo Hao,
  • Lejun Yu,
  • Jun He

摘要

Existing scene graph generation methods primarily focus on addressing the long-tail problem in the labeling. However, most debiasing approaches struggle with a trade-off between head and tail class performance compared to biased-trained models. In this paper, we propose a balanced fusion strategy to leverage the strengths of both model types. From the perspective of causality, we proposed a mutual supervision intervention (MSI) method, which consists of a batch-level intervention and a model-level alignment part. This method is integrated into the existing Introd framework to address the bias in the fusion process of scene graph generation models. We propose a new metric called DP@100 to compare the output of the student model and the teacher model, providing an evaluation of the fusion performance under this distillation framework. The results of experiments performed on the Visual Genome dataset show the effectiveness of the proposed MSI method.